2024-08-18T23:29:43.130 INFO:root:teuthology version: 1.2.0 2024-08-18T23:29:43.155 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-18T23:29:43.247 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860013 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7860013' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: de09dd018de4baddfa6ca4d43bc8fb784cf75833 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: de09dd018de4baddfa6ca4d43bc8fb784cf75833 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 78215f5a1cf5eaf321d506a66539942e12851f34 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1664 sha1: de09dd018de4baddfa6ca4d43bc8fb784cf75833 sleep_before_teardown: 0 subset: 36/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 78215f5a1cf5eaf321d506a66539942e12851f34 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi078.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNp37dBS0rsORKbJUtHu4khMk4SL5ROXUg1hW3y+BevEKE/UaszAZhEuRi3d/sn1L71p4prTwECrx6ceq5Yd12w= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGGRKz0rnhbRM3ZkPD21c4YjQmk2XKQRdIfzhzNxN6nwv2U5gIPF3+ELDQRpLsi2nT9mqM0TVJY1XH5uGR+/ruw= smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFjs0J5fIZEbQbP9OHfDrOiDZLgz23AtmcauznZkY1SsXjnbFrSyxOJV9H+8Luqr1fNM664GcU3kzBtIG4tQyhU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - quiescer: null - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 67db276914d29505c9707d04f3a3b8fb9d8aee4b timestamp: 2024-08-18_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-08-18T23:29:43.247 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa; will attempt to use it 2024-08-18T23:29:43.248 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa/tasks 2024-08-18T23:29:43.248 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-18T23:29:43.250 INFO:teuthology.task.internal:Checking packages... 2024-08-18T23:29:43.271 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'de09dd018de4baddfa6ca4d43bc8fb784cf75833' 2024-08-18T23:29:43.271 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-18T23:29:43.271 INFO:teuthology.packaging:ref: None 2024-08-18T23:29:43.272 INFO:teuthology.packaging:tag: None 2024-08-18T23:29:43.272 INFO:teuthology.packaging:branch: squid 2024-08-18T23:29:43.272 INFO:teuthology.packaging:sha1: de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-18T23:29:43.272 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-08-18T23:29:43.470 INFO:teuthology.task.internal:Found packages for ceph version 19.1.0-1386.g78215f5a 2024-08-18T23:29:43.472 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-18T23:29:43.479 INFO:teuthology.task.internal:no buildpackages task found 2024-08-18T23:29:43.479 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-18T23:29:43.488 INFO:teuthology.task.internal:Saving configuration 2024-08-18T23:29:43.506 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-18T23:29:43.514 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-18T23:29:43.540 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860013', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-18 23:24:43.021281', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNp37dBS0rsORKbJUtHu4khMk4SL5ROXUg1hW3y+BevEKE/UaszAZhEuRi3d/sn1L71p4prTwECrx6ceq5Yd12w='} 2024-08-18T23:29:43.563 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860013', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-18 23:24:43.018596', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGGRKz0rnhbRM3ZkPD21c4YjQmk2XKQRdIfzhzNxN6nwv2U5gIPF3+ELDQRpLsi2nT9mqM0TVJY1XH5uGR+/ruw='} 2024-08-18T23:29:43.583 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860013', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-18 23:24:43.020284', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFjs0J5fIZEbQbP9OHfDrOiDZLgz23AtmcauznZkY1SsXjnbFrSyxOJV9H+8Luqr1fNM664GcU3kzBtIG4tQyhU='} 2024-08-18T23:29:43.584 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-18T23:29:43.589 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-08-18T23:29:43.590 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-08-18T23:29:43.590 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-08-18T23:29:43.590 INFO:teuthology.run_tasks:Running task console_log... 2024-08-18T23:29:43.674 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbcfb173be0>, signals=[15]) 2024-08-18T23:29:43.675 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-18T23:29:43.681 INFO:teuthology.task.internal:Opening connections... 2024-08-18T23:29:43.681 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi078.front.sepia.ceph.com 2024-08-18T23:29:43.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-18T23:29:43.759 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-08-18T23:29:43.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-18T23:29:43.833 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2024-08-18T23:29:43.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-18T23:29:43.909 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-18T23:29:43.918 DEBUG:teuthology.orchestra.run.smithi078:> uname -m 2024-08-18T23:29:43.937 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64 2024-08-18T23:29:43.937 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2024-08-18T23:29:43.995 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="CentOS Stream" 2024-08-18T23:29:43.995 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="9" 2024-08-18T23:29:43.995 INFO:teuthology.orchestra.run.smithi078.stdout:ID="centos" 2024-08-18T23:29:43.996 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="rhel fedora" 2024-08-18T23:29:43.996 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="9" 2024-08-18T23:29:43.996 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el9" 2024-08-18T23:29:43.996 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-18T23:29:43.996 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2024-08-18T23:29:43.996 INFO:teuthology.orchestra.run.smithi078.stdout:LOGO="fedora-logo-icon" 2024-08-18T23:29:43.996 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-18T23:29:43.996 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://centos.org/" 2024-08-18T23:29:43.996 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-18T23:29:43.996 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-18T23:29:43.997 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-18T23:29:43.997 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2024-08-18T23:29:44.016 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-08-18T23:29:44.035 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-08-18T23:29:44.035 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-08-18T23:29:44.093 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-08-18T23:29:44.093 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2024-08-18T23:29:44.093 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-08-18T23:29:44.093 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-08-18T23:29:44.094 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2024-08-18T23:29:44.094 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2024-08-18T23:29:44.094 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-18T23:29:44.094 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-08-18T23:29:44.094 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2024-08-18T23:29:44.094 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-18T23:29:44.094 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-08-18T23:29:44.094 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-18T23:29:44.094 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-18T23:29:44.094 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-18T23:29:44.095 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-08-18T23:29:44.116 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2024-08-18T23:29:44.134 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2024-08-18T23:29:44.135 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-08-18T23:29:44.193 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Stream" 2024-08-18T23:29:44.193 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="9" 2024-08-18T23:29:44.193 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos" 2024-08-18T23:29:44.193 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora" 2024-08-18T23:29:44.193 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="9" 2024-08-18T23:29:44.193 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el9" 2024-08-18T23:29:44.193 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-18T23:29:44.193 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2024-08-18T23:29:44.193 INFO:teuthology.orchestra.run.smithi161.stdout:LOGO="fedora-logo-icon" 2024-08-18T23:29:44.193 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-18T23:29:44.193 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/" 2024-08-18T23:29:44.193 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-18T23:29:44.194 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-18T23:29:44.194 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-18T23:29:44.194 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-08-18T23:29:44.216 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-18T23:29:44.247 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-18T23:29:44.254 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-18T23:29:44.254 DEBUG:teuthology.orchestra.run.smithi078:> test '!' -e /home/ubuntu/cephtest 2024-08-18T23:29:44.257 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-08-18T23:29:44.259 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2024-08-18T23:29:44.275 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-18T23:29:44.283 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-18T23:29:44.283 DEBUG:teuthology.orchestra.run.smithi078:> test -z $(ls -A /var/lib/ceph) 2024-08-18T23:29:44.315 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-08-18T23:29:44.317 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2024-08-18T23:29:44.340 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-18T23:29:44.421 INFO:teuthology.run_tasks:Running task kernel... 2024-08-18T23:29:44.436 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-08-18T23:29:44.436 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-08-18T23:29:44.436 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-08-18T23:29:44.437 DEBUG:teuthology.orchestra.run.smithi078:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-18T23:29:44.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-18T23:29:44.454 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2024-08-18T23:29:44.513 INFO:teuthology.orchestra.run.smithi078.stdout:5.14.0-480.el9.x86_64 2024-08-18T23:29:44.513 INFO:teuthology.task.kernel:Running kernel on smithi078: 5.14.0-480.el9.x86_64 2024-08-18T23:29:44.513 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum install -y kernel 2024-08-18T23:29:47.045 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-08-18T23:29:49.062 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-08-18T23:29:53.355 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.6 MB 00:00 2024-08-18T23:29:54.947 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 18 kB 00:00 2024-08-18T23:29:56.068 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 22 MB/s | 22 MB 00:01 2024-08-18T23:30:01.440 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-08-18T23:30:03.397 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-08-18T23:30:03.397 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-5.14.0-480.el9.x86_64 is already installed. 2024-08-18T23:30:03.471 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-08-18T23:30:03.472 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-08-18T23:30:03.472 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-08-18T23:30:03.596 DEBUG:teuthology.orchestra.run.smithi078:> echo no | sudo yum reinstall kernel || true 2024-08-18T23:30:04.115 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:00:03 ago on Sun 18 Aug 2024 11:30:01 PM UTC. 2024-08-18T23:30:04.266 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-08-18T23:30:04.274 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-08-18T23:30:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: Package Architecture Version Repository Size 2024-08-18T23:30:04.274 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-08-18T23:30:04.274 INFO:teuthology.orchestra.run.smithi078.stdout:Reinstalling: 2024-08-18T23:30:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: kernel x86_64 5.14.0-480.el9 baseos 1.5 M 2024-08-18T23:30:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:30:04.274 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-08-18T23:30:04.275 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-08-18T23:30:04.275 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:30:04.275 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 1.5 M 2024-08-18T23:30:04.275 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 0 2024-08-18T23:30:04.275 INFO:teuthology.orchestra.run.smithi078.stderr:Operation aborted. 2024-08-18T23:30:04.336 INFO:teuthology.orchestra.run.smithi078.stdout:Is this ok [y/N]: 2024-08-18T23:30:04.337 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum reinstall -y kernel || true 2024-08-18T23:30:04.852 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:00:03 ago on Sun 18 Aug 2024 11:30:01 PM UTC. 2024-08-18T23:30:04.975 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-08-18T23:30:04.976 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-08-18T23:30:04.976 INFO:teuthology.orchestra.run.smithi078.stdout: Package Architecture Version Repository Size 2024-08-18T23:30:04.976 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-08-18T23:30:04.976 INFO:teuthology.orchestra.run.smithi078.stdout:Reinstalling: 2024-08-18T23:30:04.976 INFO:teuthology.orchestra.run.smithi078.stdout: kernel x86_64 5.14.0-480.el9 baseos 1.5 M 2024-08-18T23:30:04.976 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:30:04.976 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-08-18T23:30:04.977 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-08-18T23:30:04.977 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:30:04.977 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 1.5 M 2024-08-18T23:30:04.977 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 0 2024-08-18T23:30:04.977 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-08-18T23:30:06.026 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-5.14.0-480.el9.x86_64.rpm 1.5 MB/s | 1.5 MB 00:01 2024-08-18T23:30:06.026 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-08-18T23:30:06.026 INFO:teuthology.orchestra.run.smithi078.stdout:Total 1.4 MB/s | 1.5 MB 00:01 2024-08-18T23:30:06.061 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-08-18T23:30:06.302 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-08-18T23:30:06.302 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-08-18T23:30:06.326 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-08-18T23:30:06.327 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-08-18T23:30:06.457 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-08-18T23:30:06.569 INFO:teuthology.orchestra.run.smithi078.stdout: Reinstalling : kernel-5.14.0-480.el9.x86_64 1/2 2024-08-18T23:30:06.569 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : kernel-5.14.0-480.el9.x86_64 2/2 2024-08-18T23:30:07.357 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-5.14.0-480.el9.x86_64 2/2 2024-08-18T23:30:07.357 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-5.14.0-480.el9.x86_64 1/2 2024-08-18T23:30:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-5.14.0-480.el9.x86_64 2/2 2024-08-18T23:30:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:30:08.143 INFO:teuthology.orchestra.run.smithi078.stdout:Reinstalled: 2024-08-18T23:30:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-5.14.0-480.el9.x86_64 2024-08-18T23:30:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:30:08.144 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-08-18T23:30:08.322 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q kernel | sort -rV | head -n 1 2024-08-18T23:30:08.379 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-5.14.0-480.el9.x86_64 2024-08-18T23:30:08.379 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-480.el9.x86_64 2024-08-18T23:30:08.380 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-08-18T23:30:08.380 DEBUG:teuthology.parallel:result is None 2024-08-18T23:30:08.380 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-18T23:30:08.388 INFO:teuthology.task.internal:Creating test directory... 2024-08-18T23:30:08.389 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-18T23:30:08.391 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-18T23:30:08.394 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-18T23:30:08.412 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-18T23:30:08.423 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-18T23:30:08.437 INFO:teuthology.task.internal:Creating archive directory... 2024-08-18T23:30:08.437 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-18T23:30:08.448 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-18T23:30:08.452 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-18T23:30:08.560 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-18T23:30:08.571 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-18T23:30:08.572 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-18T23:30:08.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-18T23:30:08.589 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-18T23:30:08.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-18T23:30:08.608 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-18T23:30:08.610 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-18T23:30:08.632 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-18T23:30:08.645 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-18T23:30:08.662 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-18T23:30:08.688 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-18T23:30:08.690 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-18T23:30:08.690 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-18T23:30:08.721 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-18T23:30:08.723 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-18T23:30:08.734 INFO:teuthology.task.internal:Configuring sudo... 2024-08-18T23:30:08.734 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-18T23:30:08.737 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-18T23:30:08.739 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-18T23:30:08.791 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-18T23:30:08.804 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-18T23:30:08.804 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-18T23:30:08.806 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-18T23:30:08.809 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-18T23:30:08.847 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-18T23:30:08.901 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-18T23:30:08.958 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-18T23:30:09.039 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-18T23:30:09.080 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:30:09.080 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-18T23:30:09.144 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-18T23:30:09.194 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-18T23:30:09.249 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-18T23:30:09.334 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-18T23:30:09.379 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-18T23:30:09.379 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-18T23:30:09.447 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-18T23:30:09.493 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-18T23:30:09.545 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-18T23:30:09.631 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-18T23:30:09.674 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-18T23:30:09.674 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-18T23:30:09.740 DEBUG:teuthology.orchestra.run.smithi078:> sudo service rsyslog restart 2024-08-18T23:30:09.743 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-08-18T23:30:09.745 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2024-08-18T23:30:09.790 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-18T23:30:09.793 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-18T23:30:09.815 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-18T23:30:10.262 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-18T23:30:10.270 INFO:teuthology.task.internal:Starting timer... 2024-08-18T23:30:10.270 INFO:teuthology.run_tasks:Running task pcp... 2024-08-18T23:30:10.287 INFO:teuthology.run_tasks:Running task selinux... 2024-08-18T23:30:10.299 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-08-18T23:30:10.300 DEBUG:teuthology.orchestra.run.smithi078:> sudo service auditd rotate 2024-08-18T23:30:10.354 INFO:teuthology.orchestra.run.smithi078.stdout:Rotating logs: 2024-08-18T23:30:10.356 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-08-18T23:30:10.452 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-08-18T23:30:10.454 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2024-08-18T23:30:10.523 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: 2024-08-18T23:30:10.525 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-18T23:30:10.525 DEBUG:teuthology.orchestra.run.smithi078:> /usr/sbin/getenforce 2024-08-18T23:30:10.553 INFO:teuthology.orchestra.run.smithi078.stdout:Permissive 2024-08-18T23:30:10.553 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-08-18T23:30:10.581 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-08-18T23:30:10.582 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2024-08-18T23:30:10.608 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2024-08-18T23:30:10.609 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi078.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi123.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi161.front.sepia.ceph.com': 'permissive'} 2024-08-18T23:30:10.609 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-18T23:30:10.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-18T23:30:10.639 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-18T23:30:10.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-18T23:30:10.670 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-18T23:30:10.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-18T23:30:10.699 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-18T23:30:10.699 DEBUG:teuthology.orchestra.run.smithi078:> sudo /usr/sbin/setenforce permissive 2024-08-18T23:30:10.727 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-08-18T23:30:10.767 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2024-08-18T23:30:10.820 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-18T23:30:10.829 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-18T23:30:10.840 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-18T23:30:10.841 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi078.front.sepia.ceph.com,smithi123.front.sepia.ceph.com,smithi161.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-18T23:32:11.547 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi078.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com')] 2024-08-18T23:32:11.549 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-08-18T23:32:11.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-18T23:32:11.670 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-08-18T23:32:11.713 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-08-18T23:32:11.713 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-08-18T23:32:11.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-18T23:32:11.789 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-08-18T23:32:11.870 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-08-18T23:32:11.870 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-08-18T23:32:11.871 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-18T23:32:11.946 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-08-18T23:32:12.026 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-08-18T23:32:12.026 INFO:teuthology.run_tasks:Running task clock... 2024-08-18T23:32:12.036 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-18T23:32:12.036 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-18T23:32:12.037 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-18T23:32:12.039 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-18T23:32:12.039 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-18T23:32:12.042 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-18T23:32:12.042 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-18T23:32:12.066 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-18T23:32:12.072 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-18T23:32:12.080 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-18T23:32:12.087 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-18T23:32:12.099 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-18T23:32:12.104 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: ntpd: command not found 2024-08-18T23:32:12.112 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-08-18T23:32:12.115 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-18T23:32:12.115 INFO:teuthology.orchestra.run.smithi078.stdout:506 Cannot talk to daemon 2024-08-18T23:32:12.124 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-08-18T23:32:12.129 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-18T23:32:12.138 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-18T23:32:12.140 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2024-08-18T23:32:12.144 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-18T23:32:12.151 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2024-08-18T23:32:12.153 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-18T23:32:12.166 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-18T23:32:12.181 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-18T23:32:12.198 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-08-18T23:32:12.201 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-18T23:32:12.202 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-08-18T23:32:12.202 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-18T23:32:12.202 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-18T23:32:12.202 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-18T23:32:12.202 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-18T23:32:12.205 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-08-18T23:32:12.209 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-18T23:32:12.209 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-08-18T23:32:12.209 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-18T23:32:12.209 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-18T23:32:12.209 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-18T23:32:12.209 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-18T23:32:12.236 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-08-18T23:32:12.239 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-18T23:32:12.239 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-08-18T23:32:12.240 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-18T23:32:12.240 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-18T23:32:12.240 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-18T23:32:12.240 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-18T23:32:12.241 INFO:teuthology.run_tasks:Running task install... 2024-08-18T23:32:12.252 DEBUG:teuthology.task.install:project ceph 2024-08-18T23:32:12.252 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833'}} 2024-08-18T23:32:12.253 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833'} 2024-08-18T23:32:12.253 INFO:teuthology.task.install:Using flavor: default 2024-08-18T23:32:12.262 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-18T23:32:12.262 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-08-18T23:32:12.262 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833', 'tag': None, 'wait_for_package': False} 2024-08-18T23:32:12.263 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-18T23:32:12.266 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833', 'tag': None, 'wait_for_package': False} 2024-08-18T23:32:12.266 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-18T23:32:12.267 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833', 'tag': None, 'wait_for_package': False} 2024-08-18T23:32:12.267 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-18T23:32:12.408 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/ 2024-08-18T23:32:12.409 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1382.gde09dd01 2024-08-18T23:32:12.412 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/ 2024-08-18T23:32:12.412 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1382.gde09dd01 2024-08-18T23:32:12.422 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/ 2024-08-18T23:32:12.423 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1382.gde09dd01 2024-08-18T23:32:12.519 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-18T23:32:12.520 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:32:12.520 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-18T23:32:12.524 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-18T23:32:12.524 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-18T23:32:12.524 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-18T23:32:12.531 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-18T23:32:12.531 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-18T23:32:12.531 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-18T23:32:12.546 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-18T23:32:12.546 DEBUG:teuthology.orchestra.run.smithi078:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/de09dd018de4baddfa6ca4d43bc8fb784cf75833/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-18T23:32:12.551 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-18T23:32:12.551 DEBUG:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/de09dd018de4baddfa6ca4d43bc8fb784cf75833/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-18T23:32:12.558 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-18T23:32:12.558 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/de09dd018de4baddfa6ca4d43bc8fb784cf75833/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-18T23:32:12.612 DEBUG:teuthology.orchestra.run.smithi078:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-18T23:32:12.617 DEBUG:teuthology.orchestra.run.smithi161:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-18T23:32:12.624 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-18T23:32:12.687 DEBUG:teuthology.orchestra.run.smithi078:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-18T23:32:12.692 DEBUG:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-18T23:32:12.699 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-18T23:32:12.751 INFO:teuthology.orchestra.run.smithi078.stdout:check_obsoletes = 1 2024-08-18T23:32:12.753 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum clean all 2024-08-18T23:32:12.756 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2024-08-18T23:32:12.757 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum clean all 2024-08-18T23:32:12.764 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-08-18T23:32:12.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-08-18T23:32:12.977 INFO:teuthology.orchestra.run.smithi078.stdout:45 files removed 2024-08-18T23:32:12.981 INFO:teuthology.orchestra.run.smithi161.stdout:45 files removed 2024-08-18T23:32:12.988 INFO:teuthology.orchestra.run.smithi123.stdout:45 files removed 2024-08-18T23:32:13.003 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-18T23:32:13.008 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-18T23:32:13.015 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-18T23:32:13.542 INFO:teuthology.orchestra.run.smithi078.stdout:ceph packages for x86_64 313 kB/s | 84 kB 00:00 2024-08-18T23:32:13.548 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 321 kB/s | 84 kB 00:00 2024-08-18T23:32:13.566 INFO:teuthology.orchestra.run.smithi161.stdout:ceph packages for x86_64 295 kB/s | 84 kB 00:00 2024-08-18T23:32:13.825 INFO:teuthology.orchestra.run.smithi078.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-08-18T23:32:13.840 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-08-18T23:32:13.859 INFO:teuthology.orchestra.run.smithi161.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-08-18T23:32:14.092 INFO:teuthology.orchestra.run.smithi078.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-08-18T23:32:14.107 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-08-18T23:32:14.126 INFO:teuthology.orchestra.run.smithi161.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-08-18T23:32:14.584 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.2 MB 00:00 2024-08-18T23:32:14.818 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-08-18T23:32:14.882 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-08-18T23:32:16.742 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-08-18T23:32:17.277 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - AppStream 13 MB/s | 20 MB 00:01 2024-08-18T23:32:17.353 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - AppStream 14 MB/s | 20 MB 00:01 2024-08-18T23:32:21.001 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.6 MB 00:00 2024-08-18T23:32:21.645 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - CRB 18 MB/s | 6.6 MB 00:00 2024-08-18T23:32:21.744 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.6 MB 00:00 2024-08-18T23:32:23.253 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 18 kB 00:00 2024-08-18T23:32:23.276 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 18 kB 00:01 2024-08-18T23:32:23.320 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 18 kB 00:00 2024-08-18T23:32:24.399 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 22 MB/s | 22 MB 00:01 2024-08-18T23:32:24.516 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 21 MB/s | 22 MB 00:01 2024-08-18T23:32:24.689 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 17 MB/s | 22 MB 00:01 2024-08-18T23:32:29.788 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-08-18T23:32:29.938 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-08-18T23:32:30.278 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-08-18T23:32:32.050 INFO:teuthology.orchestra.run.smithi161.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-18T23:32:32.051 INFO:teuthology.orchestra.run.smithi161.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-18T23:32:32.064 INFO:teuthology.orchestra.run.smithi161.stdout:Package NetworkManager-1:1.48.2-2.el9.x86_64 is already installed. 2024-08-18T23:32:32.065 INFO:teuthology.orchestra.run.smithi161.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-18T23:32:32.065 INFO:teuthology.orchestra.run.smithi161.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-18T23:32:32.066 INFO:teuthology.orchestra.run.smithi161.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-18T23:32:32.068 INFO:teuthology.orchestra.run.smithi161.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-18T23:32:32.071 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-18T23:32:32.072 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsprogs-6.4.0-3.el9.x86_64 is already installed. 2024-08-18T23:32:32.125 INFO:teuthology.orchestra.run.smithi078.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-18T23:32:32.126 INFO:teuthology.orchestra.run.smithi078.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-18T23:32:32.132 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-08-18T23:32:32.139 INFO:teuthology.orchestra.run.smithi078.stdout:Package NetworkManager-1:1.48.2-2.el9.x86_64 is already installed. 2024-08-18T23:32:32.140 INFO:teuthology.orchestra.run.smithi078.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-18T23:32:32.141 INFO:teuthology.orchestra.run.smithi078.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-18T23:32:32.142 INFO:teuthology.orchestra.run.smithi078.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-18T23:32:32.143 INFO:teuthology.orchestra.run.smithi078.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-18T23:32:32.146 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-18T23:32:32.147 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsprogs-6.4.0-3.el9.x86_64 is already installed. 2024-08-18T23:32:32.149 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2024-08-18T23:32:32.149 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-08-18T23:32:32.149 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2024-08-18T23:32:32.149 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-08-18T23:32:32.150 INFO:teuthology.orchestra.run.smithi161.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-18T23:32:32.150 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 6.6 k 2024-08-18T23:32:32.150 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-18T23:32:32.150 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 896 k 2024-08-18T23:32:32.150 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 145 k 2024-08-18T23:32:32.150 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 1.5 M 2024-08-18T23:32:32.150 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 146 k 2024-08-18T23:32:32.150 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 3.6 M 2024-08-18T23:32:32.150 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 7.4 M 2024-08-18T23:32:32.150 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 49 k 2024-08-18T23:32:32.150 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 11 M 2024-08-18T23:32:32.151 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 49 M 2024-08-18T23:32:32.151 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 285 k 2024-08-18T23:32:32.151 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 766 k 2024-08-18T23:32:32.151 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 225 k 2024-08-18T23:32:32.151 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 27 k 2024-08-18T23:32:32.151 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 18 k 2024-08-18T23:32:32.151 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-18T23:32:32.151 INFO:teuthology.orchestra.run.smithi161.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-18T23:32:32.151 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-18T23:32:32.151 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 32 k 2024-08-18T23:32:32.151 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 753 k 2024-08-18T23:32:32.151 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 127 k 2024-08-18T23:32:32.152 INFO:teuthology.orchestra.run.smithi161.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-18T23:32:32.152 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-18T23:32:32.152 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-18T23:32:32.152 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-08-18T23:32:32.152 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-08-18T23:32:32.152 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-08-18T23:32:32.152 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 188 k 2024-08-18T23:32:32.152 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 369 k 2024-08-18T23:32:32.152 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 350 k 2024-08-18T23:32:32.152 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 117 k 2024-08-18T23:32:32.152 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 85 k 2024-08-18T23:32:32.152 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.1 M 2024-08-18T23:32:32.153 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 170 k 2024-08-18T23:32:32.153 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-18T23:32:32.153 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2024-08-18T23:32:32.153 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.4 M 2024-08-18T23:32:32.153 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.2 M 2024-08-18T23:32:32.153 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-08-18T23:32:32.153 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-18T23:32:32.153 INFO:teuthology.orchestra.run.smithi161.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-18T23:32:32.153 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-18T23:32:32.153 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 21 M 2024-08-18T23:32:32.153 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 32 k 2024-08-18T23:32:32.153 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 2.4 M 2024-08-18T23:32:32.154 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 249 k 2024-08-18T23:32:32.154 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 4.7 M 2024-08-18T23:32:32.154 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 17 M 2024-08-18T23:32:32.154 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 16 k 2024-08-18T23:32:32.154 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 25 k 2024-08-18T23:32:32.154 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-18T23:32:32.154 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-18T23:32:32.154 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-18T23:32:32.154 INFO:teuthology.orchestra.run.smithi161.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-18T23:32:32.154 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-18T23:32:32.154 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-18T23:32:32.154 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-18T23:32:32.155 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-18T23:32:32.155 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 163 k 2024-08-18T23:32:32.155 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-18T23:32:32.155 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-18T23:32:32.155 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-18T23:32:32.155 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-18T23:32:32.155 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-18T23:32:32.155 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 507 k 2024-08-18T23:32:32.155 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-18T23:32:32.155 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-18T23:32:32.155 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-18T23:32:32.155 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-18T23:32:32.156 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-18T23:32:32.156 INFO:teuthology.orchestra.run.smithi161.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-18T23:32:32.156 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-18T23:32:32.156 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-18T23:32:32.156 INFO:teuthology.orchestra.run.smithi161.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-18T23:32:32.156 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-18T23:32:32.156 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-18T23:32:32.156 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-08-18T23:32:32.156 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-18T23:32:32.156 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-18T23:32:32.156 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-18T23:32:32.156 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-18T23:32:32.157 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-18T23:32:32.157 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-18T23:32:32.157 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-18T23:32:32.157 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 45 k 2024-08-18T23:32:32.157 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 136 k 2024-08-18T23:32:32.157 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-18T23:32:32.157 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-18T23:32:32.157 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-18T23:32:32.157 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-18T23:32:32.157 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-18T23:32:32.157 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-18T23:32:32.157 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-18T23:32:32.157 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel x86_64 3.9.19-4.el9 appstream 244 k 2024-08-18T23:32:32.158 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-18T23:32:32.158 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-18T23:32:32.158 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-18T23:32:32.158 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-18T23:32:32.158 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-18T23:32:32.158 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-18T23:32:32.158 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-18T23:32:32.158 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-18T23:32:32.158 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-18T23:32:32.158 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-18T23:32:32.158 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-18T23:32:32.158 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-18T23:32:32.159 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-18T23:32:32.159 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-18T23:32:32.159 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-18T23:32:32.159 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-18T23:32:32.159 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-18T23:32:32.159 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-18T23:32:32.159 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-18T23:32:32.159 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-18T23:32:32.159 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-18T23:32:32.159 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-18T23:32:32.159 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-18T23:32:32.159 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-18T23:32:32.159 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-18T23:32:32.159 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-18T23:32:32.160 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-18T23:32:32.160 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-18T23:32:32.160 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-18T23:32:32.160 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-18T23:32:32.160 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-18T23:32:32.160 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-18T23:32:32.160 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-18T23:32:32.160 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-18T23:32:32.160 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-18T23:32:32.160 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-18T23:32:32.160 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-18T23:32:32.160 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-18T23:32:32.161 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-18T23:32:32.161 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-18T23:32:32.161 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-18T23:32:32.161 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-18T23:32:32.161 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-18T23:32:32.161 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-18T23:32:32.161 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-18T23:32:32.161 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-18T23:32:32.161 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-18T23:32:32.161 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-18T23:32:32.161 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-18T23:32:32.161 INFO:teuthology.orchestra.run.smithi161.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-18T23:32:32.161 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-18T23:32:32.162 INFO:teuthology.orchestra.run.smithi161.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-18T23:32:32.162 INFO:teuthology.orchestra.run.smithi161.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-18T23:32:32.162 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-18T23:32:32.162 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-18T23:32:32.162 INFO:teuthology.orchestra.run.smithi161.stdout:Installing weak dependencies: 2024-08-18T23:32:32.162 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-18T23:32:32.162 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:32:32.162 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-08-18T23:32:32.162 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2024-08-18T23:32:32.162 INFO:teuthology.orchestra.run.smithi161.stdout:Install 148 Packages 2024-08-18T23:32:32.162 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 2 Packages 2024-08-18T23:32:32.162 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:32:32.163 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 218 M 2024-08-18T23:32:32.163 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-08-18T23:32:32.208 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-08-18T23:32:32.216 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-08-18T23:32:32.216 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-08-18T23:32:32.216 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-08-18T23:32:32.216 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-08-18T23:32:32.216 INFO:teuthology.orchestra.run.smithi078.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-18T23:32:32.216 INFO:teuthology.orchestra.run.smithi078.stdout: ceph x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 6.6 k 2024-08-18T23:32:32.217 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-18T23:32:32.217 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 896 k 2024-08-18T23:32:32.217 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 145 k 2024-08-18T23:32:32.217 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 1.5 M 2024-08-18T23:32:32.217 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 146 k 2024-08-18T23:32:32.217 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 3.6 M 2024-08-18T23:32:32.217 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 7.4 M 2024-08-18T23:32:32.217 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 49 k 2024-08-18T23:32:32.217 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 11 M 2024-08-18T23:32:32.217 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 49 M 2024-08-18T23:32:32.217 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 285 k 2024-08-18T23:32:32.217 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 766 k 2024-08-18T23:32:32.217 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 225 k 2024-08-18T23:32:32.218 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-shell x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 27 k 2024-08-18T23:32:32.218 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 18 k 2024-08-18T23:32:32.218 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-18T23:32:32.218 INFO:teuthology.orchestra.run.smithi078.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-18T23:32:32.218 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-18T23:32:32.218 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 32 k 2024-08-18T23:32:32.218 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 753 k 2024-08-18T23:32:32.218 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 127 k 2024-08-18T23:32:32.218 INFO:teuthology.orchestra.run.smithi078.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-18T23:32:32.218 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-18T23:32:32.218 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-18T23:32:32.218 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-08-18T23:32:32.219 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-08-18T23:32:32.219 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-08-18T23:32:32.219 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 188 k 2024-08-18T23:32:32.219 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 369 k 2024-08-18T23:32:32.219 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 350 k 2024-08-18T23:32:32.219 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 117 k 2024-08-18T23:32:32.219 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 85 k 2024-08-18T23:32:32.219 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.1 M 2024-08-18T23:32:32.219 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 170 k 2024-08-18T23:32:32.219 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-18T23:32:32.219 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrading: 2024-08-18T23:32:32.219 INFO:teuthology.orchestra.run.smithi078.stdout: librados2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.4 M 2024-08-18T23:32:32.219 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.2 M 2024-08-18T23:32:32.220 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-08-18T23:32:32.220 INFO:teuthology.orchestra.run.smithi078.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-18T23:32:32.220 INFO:teuthology.orchestra.run.smithi078.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-18T23:32:32.220 INFO:teuthology.orchestra.run.smithi078.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-18T23:32:32.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 21 M 2024-08-18T23:32:32.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 32 k 2024-08-18T23:32:32.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 2.4 M 2024-08-18T23:32:32.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 249 k 2024-08-18T23:32:32.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 4.7 M 2024-08-18T23:32:32.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 17 M 2024-08-18T23:32:32.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 16 k 2024-08-18T23:32:32.220 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 25 k 2024-08-18T23:32:32.220 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-18T23:32:32.221 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-18T23:32:32.221 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-18T23:32:32.221 INFO:teuthology.orchestra.run.smithi078.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-18T23:32:32.221 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-18T23:32:32.221 INFO:teuthology.orchestra.run.smithi078.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-18T23:32:32.221 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-18T23:32:32.221 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-18T23:32:32.221 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 163 k 2024-08-18T23:32:32.221 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-18T23:32:32.221 INFO:teuthology.orchestra.run.smithi078.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-18T23:32:32.221 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-18T23:32:32.221 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-18T23:32:32.221 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-18T23:32:32.221 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 507 k 2024-08-18T23:32:32.222 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-18T23:32:32.222 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-18T23:32:32.222 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-18T23:32:32.222 INFO:teuthology.orchestra.run.smithi078.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-18T23:32:32.222 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-18T23:32:32.222 INFO:teuthology.orchestra.run.smithi078.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-18T23:32:32.222 INFO:teuthology.orchestra.run.smithi078.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-18T23:32:32.222 INFO:teuthology.orchestra.run.smithi078.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-18T23:32:32.222 INFO:teuthology.orchestra.run.smithi078.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-18T23:32:32.222 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-18T23:32:32.222 INFO:teuthology.orchestra.run.smithi078.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-18T23:32:32.222 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-08-18T23:32:32.223 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-18T23:32:32.223 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-18T23:32:32.223 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-18T23:32:32.223 INFO:teuthology.orchestra.run.smithi078.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-18T23:32:32.223 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-18T23:32:32.223 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-18T23:32:32.223 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-18T23:32:32.223 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 45 k 2024-08-18T23:32:32.223 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 136 k 2024-08-18T23:32:32.223 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-18T23:32:32.223 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-18T23:32:32.223 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-18T23:32:32.223 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-18T23:32:32.224 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-18T23:32:32.224 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-18T23:32:32.224 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-18T23:32:32.224 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel x86_64 3.9.19-4.el9 appstream 244 k 2024-08-18T23:32:32.224 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-18T23:32:32.224 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-18T23:32:32.224 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-18T23:32:32.224 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-18T23:32:32.224 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-18T23:32:32.224 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-18T23:32:32.224 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-18T23:32:32.224 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-18T23:32:32.224 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-18T23:32:32.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-18T23:32:32.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-18T23:32:32.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-18T23:32:32.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-18T23:32:32.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-18T23:32:32.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-18T23:32:32.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-18T23:32:32.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-18T23:32:32.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-18T23:32:32.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-18T23:32:32.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-18T23:32:32.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-18T23:32:32.225 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-18T23:32:32.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-18T23:32:32.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-18T23:32:32.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-18T23:32:32.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-18T23:32:32.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-18T23:32:32.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-18T23:32:32.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-18T23:32:32.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-18T23:32:32.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-18T23:32:32.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-18T23:32:32.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-18T23:32:32.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-18T23:32:32.226 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-18T23:32:32.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-18T23:32:32.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-18T23:32:32.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-18T23:32:32.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-18T23:32:32.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-18T23:32:32.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-18T23:32:32.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-18T23:32:32.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-18T23:32:32.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-18T23:32:32.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-18T23:32:32.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-18T23:32:32.227 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-18T23:32:32.227 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-18T23:32:32.228 INFO:teuthology.orchestra.run.smithi078.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-18T23:32:32.228 INFO:teuthology.orchestra.run.smithi078.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-18T23:32:32.228 INFO:teuthology.orchestra.run.smithi078.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-18T23:32:32.228 INFO:teuthology.orchestra.run.smithi078.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-18T23:32:32.228 INFO:teuthology.orchestra.run.smithi078.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-18T23:32:32.228 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-18T23:32:32.228 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-18T23:32:32.228 INFO:teuthology.orchestra.run.smithi078.stdout:Installing weak dependencies: 2024-08-18T23:32:32.228 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-18T23:32:32.228 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:32:32.228 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-08-18T23:32:32.228 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-08-18T23:32:32.228 INFO:teuthology.orchestra.run.smithi078.stdout:Install 148 Packages 2024-08-18T23:32:32.228 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrade 2 Packages 2024-08-18T23:32:32.229 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:32:32.229 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 218 M 2024-08-18T23:32:32.229 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-08-18T23:32:32.507 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-18T23:32:32.507 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-18T23:32:32.521 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.48.2-2.el9.x86_64 is already installed. 2024-08-18T23:32:32.522 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-18T23:32:32.523 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-18T23:32:32.524 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-18T23:32:32.526 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-18T23:32:32.529 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-18T23:32:32.530 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-6.4.0-3.el9.x86_64 is already installed. 2024-08-18T23:32:32.590 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-08-18T23:32:32.610 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-08-18T23:32:32.610 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-08-18T23:32:32.610 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-08-18T23:32:32.610 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-08-18T23:32:32.610 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-18T23:32:32.610 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 6.6 k 2024-08-18T23:32:32.610 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-18T23:32:32.610 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 896 k 2024-08-18T23:32:32.610 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 145 k 2024-08-18T23:32:32.610 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 1.5 M 2024-08-18T23:32:32.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 146 k 2024-08-18T23:32:32.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 3.6 M 2024-08-18T23:32:32.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 7.4 M 2024-08-18T23:32:32.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 49 k 2024-08-18T23:32:32.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 11 M 2024-08-18T23:32:32.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 49 M 2024-08-18T23:32:32.611 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 285 k 2024-08-18T23:32:32.611 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 766 k 2024-08-18T23:32:32.611 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 225 k 2024-08-18T23:32:32.611 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 27 k 2024-08-18T23:32:32.612 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 18 k 2024-08-18T23:32:32.612 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-18T23:32:32.612 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-18T23:32:32.612 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-18T23:32:32.612 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 32 k 2024-08-18T23:32:32.612 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 753 k 2024-08-18T23:32:32.612 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 127 k 2024-08-18T23:32:32.612 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-18T23:32:32.612 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-18T23:32:32.612 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-18T23:32:32.612 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-08-18T23:32:32.612 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-08-18T23:32:32.613 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-08-18T23:32:32.613 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 188 k 2024-08-18T23:32:32.613 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 369 k 2024-08-18T23:32:32.613 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 350 k 2024-08-18T23:32:32.613 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 117 k 2024-08-18T23:32:32.613 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 85 k 2024-08-18T23:32:32.613 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.1 M 2024-08-18T23:32:32.613 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 170 k 2024-08-18T23:32:32.613 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-18T23:32:32.613 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-08-18T23:32:32.613 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.4 M 2024-08-18T23:32:32.613 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 3.2 M 2024-08-18T23:32:32.614 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-08-18T23:32:32.614 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-18T23:32:32.614 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-18T23:32:32.614 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-18T23:32:32.614 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 21 M 2024-08-18T23:32:32.614 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 32 k 2024-08-18T23:32:32.614 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 2.4 M 2024-08-18T23:32:32.614 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 249 k 2024-08-18T23:32:32.614 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 4.7 M 2024-08-18T23:32:32.614 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 17 M 2024-08-18T23:32:32.615 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1382.gde09dd01.el9 ceph-noarch 16 k 2024-08-18T23:32:32.615 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 25 k 2024-08-18T23:32:32.615 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-18T23:32:32.615 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-18T23:32:32.615 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-18T23:32:32.615 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-18T23:32:32.615 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-18T23:32:32.615 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-18T23:32:32.615 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-18T23:32:32.615 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-18T23:32:32.615 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 163 k 2024-08-18T23:32:32.616 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-18T23:32:32.616 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-18T23:32:32.616 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-18T23:32:32.616 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-18T23:32:32.616 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-18T23:32:32.616 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 507 k 2024-08-18T23:32:32.616 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-18T23:32:32.616 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 5.4 M 2024-08-18T23:32:32.616 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-18T23:32:32.616 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-18T23:32:32.616 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-18T23:32:32.617 INFO:teuthology.orchestra.run.smithi123.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-18T23:32:32.617 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-18T23:32:32.617 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-18T23:32:32.617 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-18T23:32:32.617 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-18T23:32:32.617 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-18T23:32:32.617 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-08-18T23:32:32.617 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-18T23:32:32.617 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-18T23:32:32.617 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-18T23:32:32.617 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-18T23:32:32.617 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-18T23:32:32.617 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-18T23:32:32.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-18T23:32:32.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 45 k 2024-08-18T23:32:32.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:19.1.0-1382.gde09dd01.el9 ceph 136 k 2024-08-18T23:32:32.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-18T23:32:32.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-18T23:32:32.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-18T23:32:32.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-18T23:32:32.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-18T23:32:32.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-18T23:32:32.618 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-18T23:32:32.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel x86_64 3.9.19-4.el9 appstream 244 k 2024-08-18T23:32:32.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-18T23:32:32.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-18T23:32:32.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-18T23:32:32.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-18T23:32:32.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-18T23:32:32.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-18T23:32:32.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-18T23:32:32.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-18T23:32:32.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-18T23:32:32.619 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-18T23:32:32.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-18T23:32:32.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-18T23:32:32.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-18T23:32:32.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-18T23:32:32.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-18T23:32:32.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-18T23:32:32.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-18T23:32:32.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-18T23:32:32.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-18T23:32:32.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-18T23:32:32.621 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-18T23:32:32.621 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-18T23:32:32.621 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-18T23:32:32.621 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-18T23:32:32.621 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-18T23:32:32.621 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-18T23:32:32.621 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-18T23:32:32.621 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-18T23:32:32.621 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-18T23:32:32.621 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-18T23:32:32.621 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-18T23:32:32.622 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-18T23:32:32.622 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-18T23:32:32.622 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-18T23:32:32.622 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-18T23:32:32.622 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-18T23:32:32.622 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-18T23:32:32.622 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-18T23:32:32.622 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-18T23:32:32.622 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-18T23:32:32.622 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-18T23:32:32.623 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-18T23:32:32.623 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-18T23:32:32.623 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-18T23:32:32.623 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-18T23:32:32.623 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-18T23:32:32.623 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-18T23:32:32.623 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-18T23:32:32.623 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-18T23:32:32.623 INFO:teuthology.orchestra.run.smithi123.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-18T23:32:32.623 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-18T23:32:32.624 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-18T23:32:32.624 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-18T23:32:32.624 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-18T23:32:32.624 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-18T23:32:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-08-18T23:32:32.624 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-18T23:32:32.624 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-18T23:32:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-08-18T23:32:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-08-18T23:32:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:Install 148 Packages 2024-08-18T23:32:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 2 Packages 2024-08-18T23:32:32.624 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-18T23:32:32.625 INFO:teuthology.orchestra.run.smithi078.stdout:(1/150): ceph-19.1.0-1382.gde09dd01.el9.x86_64. 54 kB/s | 6.6 kB 00:00 2024-08-18T23:32:32.625 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 218 M 2024-08-18T23:32:32.626 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-08-18T23:32:32.882 INFO:teuthology.orchestra.run.smithi078.stdout:(2/150): ceph-fuse-19.1.0-1382.gde09dd01.el9.x8 3.4 MB/s | 896 kB 00:00 2024-08-18T23:32:32.982 INFO:teuthology.orchestra.run.smithi078.stdout:(3/150): ceph-base-19.1.0-1382.gde09dd01.el9.x8 11 MB/s | 5.4 MB 00:00 2024-08-18T23:32:33.007 INFO:teuthology.orchestra.run.smithi078.stdout:(4/150): ceph-immutable-object-cache-19.1.0-138 1.1 MB/s | 145 kB 00:00 2024-08-18T23:32:33.014 INFO:teuthology.orchestra.run.smithi161.stdout:(1/150): ceph-19.1.0-1382.gde09dd01.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-08-18T23:32:33.090 INFO:teuthology.orchestra.run.smithi078.stdout:(5/150): ceph-mds-19.1.0-1382.gde09dd01.el9.x86 22 MB/s | 2.4 MB 00:00 2024-08-18T23:32:33.299 INFO:teuthology.orchestra.run.smithi078.stdout:(6/150): ceph-mgr-19.1.0-1382.gde09dd01.el9.x86 5.2 MB/s | 1.5 MB 00:00 2024-08-18T23:32:33.375 INFO:teuthology.orchestra.run.smithi078.stdout:(7/150): ceph-mon-19.1.0-1382.gde09dd01.el9.x86 17 MB/s | 4.7 MB 00:00 2024-08-18T23:32:33.429 INFO:teuthology.orchestra.run.smithi123.stdout:(1/150): ceph-19.1.0-1382.gde09dd01.el9.x86_64. 34 kB/s | 6.6 kB 00:00 2024-08-18T23:32:33.606 INFO:teuthology.orchestra.run.smithi161.stdout:(2/150): ceph-fuse-19.1.0-1382.gde09dd01.el9.x8 1.5 MB/s | 896 kB 00:00 2024-08-18T23:32:33.616 INFO:teuthology.orchestra.run.smithi078.stdout:(8/150): ceph-common-19.1.0-1382.gde09dd01.el9. 19 MB/s | 21 MB 00:01 2024-08-18T23:32:33.667 INFO:teuthology.orchestra.run.smithi078.stdout:(9/150): ceph-selinux-19.1.0-1382.gde09dd01.el9 505 kB/s | 25 kB 00:00 2024-08-18T23:32:33.671 INFO:teuthology.orchestra.run.smithi123.stdout:(2/150): ceph-fuse-19.1.0-1382.gde09dd01.el9.x8 3.6 MB/s | 896 kB 00:00 2024-08-18T23:32:33.698 INFO:teuthology.orchestra.run.smithi161.stdout:(3/150): ceph-immutable-object-cache-19.1.0-138 1.5 MB/s | 145 kB 00:00 2024-08-18T23:32:33.779 INFO:teuthology.orchestra.run.smithi123.stdout:(3/150): ceph-immutable-object-cache-19.1.0-138 1.3 MB/s | 145 kB 00:00 2024-08-18T23:32:34.051 INFO:teuthology.orchestra.run.smithi078.stdout:(10/150): ceph-radosgw-19.1.0-1382.gde09dd01.el 16 MB/s | 11 MB 00:00 2024-08-18T23:32:34.142 INFO:teuthology.orchestra.run.smithi078.stdout:(11/150): cephfs-mirror-19.1.0-1382.gde09dd01.e 2.4 MB/s | 225 kB 00:00 2024-08-18T23:32:34.238 INFO:teuthology.orchestra.run.smithi123.stdout:(4/150): ceph-base-19.1.0-1382.gde09dd01.el9.x8 5.4 MB/s | 5.4 MB 00:01 2024-08-18T23:32:34.242 INFO:teuthology.orchestra.run.smithi078.stdout:(12/150): cephfs-shell-19.1.0-1382.gde09dd01.el 271 kB/s | 27 kB 00:00 2024-08-18T23:32:34.343 INFO:teuthology.orchestra.run.smithi078.stdout:(13/150): libcephfs-devel-19.1.0-1382.gde09dd01 317 kB/s | 32 kB 00:00 2024-08-18T23:32:34.448 INFO:teuthology.orchestra.run.smithi161.stdout:(4/150): ceph-base-19.1.0-1382.gde09dd01.el9.x8 3.3 MB/s | 5.4 MB 00:01 2024-08-18T23:32:34.472 INFO:teuthology.orchestra.run.smithi123.stdout:(5/150): ceph-mgr-19.1.0-1382.gde09dd01.el9.x86 6.5 MB/s | 1.5 MB 00:00 2024-08-18T23:32:34.538 INFO:teuthology.orchestra.run.smithi123.stdout:(6/150): ceph-mds-19.1.0-1382.gde09dd01.el9.x86 3.1 MB/s | 2.4 MB 00:00 2024-08-18T23:32:34.732 INFO:teuthology.orchestra.run.smithi161.stdout:(5/150): ceph-mgr-19.1.0-1382.gde09dd01.el9.x86 5.4 MB/s | 1.5 MB 00:00 2024-08-18T23:32:34.743 INFO:teuthology.orchestra.run.smithi078.stdout:(14/150): ceph-osd-19.1.0-1382.gde09dd01.el9.x8 12 MB/s | 17 MB 00:01 2024-08-18T23:32:34.823 INFO:teuthology.orchestra.run.smithi161.stdout:(6/150): ceph-mds-19.1.0-1382.gde09dd01.el9.x86 2.1 MB/s | 2.4 MB 00:01 2024-08-18T23:32:34.862 INFO:teuthology.orchestra.run.smithi078.stdout:(15/150): libcephsqlite-19.1.0-1382.gde09dd01.e 1.3 MB/s | 163 kB 00:00 2024-08-18T23:32:34.939 INFO:teuthology.orchestra.run.smithi123.stdout:(7/150): ceph-mon-19.1.0-1382.gde09dd01.el9.x86 10 MB/s | 4.7 MB 00:00 2024-08-18T23:32:34.943 INFO:teuthology.orchestra.run.smithi078.stdout:(16/150): librados-devel-19.1.0-1382.gde09dd01. 1.5 MB/s | 127 kB 00:00 2024-08-18T23:32:35.002 INFO:teuthology.orchestra.run.smithi078.stdout:(17/150): libcephfs2-19.1.0-1382.gde09dd01.el9. 1.1 MB/s | 753 kB 00:00 2024-08-18T23:32:35.044 INFO:teuthology.orchestra.run.smithi078.stdout:(18/150): libradosstriper1-19.1.0-1382.gde09dd0 4.9 MB/s | 507 kB 00:00 2024-08-18T23:32:35.144 INFO:teuthology.orchestra.run.smithi078.stdout:(19/150): python3-ceph-argparse-19.1.0-1382.gde 452 kB/s | 45 kB 00:00 2024-08-18T23:32:35.236 INFO:teuthology.orchestra.run.smithi078.stdout:(20/150): python3-ceph-common-19.1.0-1382.gde09 1.4 MB/s | 136 kB 00:00 2024-08-18T23:32:35.345 INFO:teuthology.orchestra.run.smithi078.stdout:(21/150): python3-cephfs-19.1.0-1382.gde09dd01. 1.7 MB/s | 188 kB 00:00 2024-08-18T23:32:35.524 INFO:teuthology.orchestra.run.smithi161.stdout:(7/150): ceph-mon-19.1.0-1382.gde09dd01.el9.x86 5.9 MB/s | 4.7 MB 00:00 2024-08-18T23:32:35.778 INFO:teuthology.orchestra.run.smithi078.stdout:(22/150): python3-rados-19.1.0-1382.gde09dd01.e 852 kB/s | 369 kB 00:00 2024-08-18T23:32:35.805 INFO:teuthology.orchestra.run.smithi123.stdout:(8/150): ceph-radosgw-19.1.0-1382.gde09dd01.el9 12 MB/s | 11 MB 00:00 2024-08-18T23:32:35.862 INFO:teuthology.orchestra.run.smithi078.stdout:(23/150): librgw2-19.1.0-1382.gde09dd01.el9.x86 6.2 MB/s | 5.4 MB 00:00 2024-08-18T23:32:35.880 INFO:teuthology.orchestra.run.smithi123.stdout:(9/150): ceph-selinux-19.1.0-1382.gde09dd01.el9 336 kB/s | 25 kB 00:00 2024-08-18T23:32:35.887 INFO:teuthology.orchestra.run.smithi078.stdout:(24/150): python3-rbd-19.1.0-1382.gde09dd01.el9 3.1 MB/s | 350 kB 00:00 2024-08-18T23:32:35.937 INFO:teuthology.orchestra.run.smithi078.stdout:(25/150): python3-rgw-19.1.0-1382.gde09dd01.el9 1.5 MB/s | 117 kB 00:00 2024-08-18T23:32:35.971 INFO:teuthology.orchestra.run.smithi078.stdout:(26/150): rbd-fuse-19.1.0-1382.gde09dd01.el9.x8 1.0 MB/s | 85 kB 00:00 2024-08-18T23:32:36.080 INFO:teuthology.orchestra.run.smithi078.stdout:(27/150): rbd-nbd-19.1.0-1382.gde09dd01.el9.x86 1.5 MB/s | 170 kB 00:00 2024-08-18T23:32:36.180 INFO:teuthology.orchestra.run.smithi078.stdout:(28/150): ceph-grafana-dashboards-19.1.0-1382.g 318 kB/s | 32 kB 00:00 2024-08-18T23:32:36.288 INFO:teuthology.orchestra.run.smithi078.stdout:(29/150): ceph-mgr-cephadm-19.1.0-1382.gde09dd0 1.3 MB/s | 146 kB 00:00 2024-08-18T23:32:36.472 INFO:teuthology.orchestra.run.smithi078.stdout:(30/150): rbd-mirror-19.1.0-1382.gde09dd01.el9. 5.8 MB/s | 3.1 MB 00:00 2024-08-18T23:32:37.002 INFO:teuthology.orchestra.run.smithi078.stdout:(31/150): ceph-mgr-dashboard-19.1.0-1382.gde09d 5.0 MB/s | 3.6 MB 00:00 2024-08-18T23:32:37.098 INFO:teuthology.orchestra.run.smithi078.stdout:(32/150): ceph-mgr-modules-core-19.1.0-1382.gde 2.6 MB/s | 249 kB 00:00 2024-08-18T23:32:37.132 INFO:teuthology.orchestra.run.smithi161.stdout:(8/150): ceph-common-19.1.0-1382.gde09dd01.el9. 4.9 MB/s | 21 MB 00:04 2024-08-18T23:32:37.173 INFO:teuthology.orchestra.run.smithi078.stdout:(33/150): ceph-mgr-rook-19.1.0-1382.gde09dd01.e 658 kB/s | 49 kB 00:00 2024-08-18T23:32:37.224 INFO:teuthology.orchestra.run.smithi161.stdout:(9/150): ceph-selinux-19.1.0-1382.gde09dd01.el9 276 kB/s | 25 kB 00:00 2024-08-18T23:32:37.256 INFO:teuthology.orchestra.run.smithi078.stdout:(34/150): ceph-prometheus-alerts-19.1.0-1382.gd 198 kB/s | 16 kB 00:00 2024-08-18T23:32:37.516 INFO:teuthology.orchestra.run.smithi161.stdout:(10/150): ceph-radosgw-19.1.0-1382.gde09dd01.el 5.3 MB/s | 11 MB 00:01 2024-08-18T23:32:37.674 INFO:teuthology.orchestra.run.smithi161.stdout:(11/150): ceph-osd-19.1.0-1382.gde09dd01.el9.x8 5.9 MB/s | 17 MB 00:02 2024-08-18T23:32:37.699 INFO:teuthology.orchestra.run.smithi161.stdout:(12/150): cephfs-mirror-19.1.0-1382.gde09dd01.e 1.2 MB/s | 225 kB 00:00 2024-08-18T23:32:37.725 INFO:teuthology.orchestra.run.smithi161.stdout:(13/150): cephfs-shell-19.1.0-1382.gde09dd01.el 542 kB/s | 27 kB 00:00 2024-08-18T23:32:37.750 INFO:teuthology.orchestra.run.smithi161.stdout:(14/150): libcephfs-devel-19.1.0-1382.gde09dd01 632 kB/s | 32 kB 00:00 2024-08-18T23:32:37.765 INFO:teuthology.orchestra.run.smithi078.stdout:(35/150): ceph-test-19.1.0-1382.gde09dd01.el9.x 12 MB/s | 49 MB 00:04 2024-08-18T23:32:37.806 INFO:teuthology.orchestra.run.smithi123.stdout:(10/150): ceph-osd-19.1.0-1382.gde09dd01.el9.x8 5.1 MB/s | 17 MB 00:03 2024-08-18T23:32:37.811 INFO:teuthology.orchestra.run.smithi161.stdout:(15/150): libcephfs2-19.1.0-1382.gde09dd01.el9. 8.6 MB/s | 753 kB 00:00 2024-08-18T23:32:37.837 INFO:teuthology.orchestra.run.smithi161.stdout:(16/150): libcephsqlite-19.1.0-1382.gde09dd01.e 1.9 MB/s | 163 kB 00:00 2024-08-18T23:32:37.857 INFO:teuthology.orchestra.run.smithi078.stdout:(36/150): ceph-mgr-diskprediction-local-19.1.0- 5.3 MB/s | 7.4 MB 00:01 2024-08-18T23:32:37.861 INFO:teuthology.orchestra.run.smithi161.stdout:(17/150): librados-devel-19.1.0-1382.gde09dd01. 2.5 MB/s | 127 kB 00:00 2024-08-18T23:32:37.883 INFO:teuthology.orchestra.run.smithi078.stdout:(37/150): ceph-volume-19.1.0-1382.gde09dd01.el9 456 kB/s | 285 kB 00:00 2024-08-18T23:32:37.908 INFO:teuthology.orchestra.run.smithi078.stdout:(38/150): cephfs-top-19.1.0-1382.gde09dd01.el9. 365 kB/s | 18 kB 00:00 2024-08-18T23:32:37.912 INFO:teuthology.orchestra.run.smithi161.stdout:(18/150): libradosstriper1-19.1.0-1382.gde09dd0 6.6 MB/s | 507 kB 00:00 2024-08-18T23:32:37.950 INFO:teuthology.orchestra.run.smithi078.stdout:(39/150): cephadm-19.1.0-1382.gde09dd01.el9.noa 4.1 MB/s | 766 kB 00:00 2024-08-18T23:32:37.962 INFO:teuthology.orchestra.run.smithi161.stdout:(19/150): python3-ceph-argparse-19.1.0-1382.gde 904 kB/s | 45 kB 00:00 2024-08-18T23:32:38.013 INFO:teuthology.orchestra.run.smithi161.stdout:(20/150): python3-ceph-common-19.1.0-1382.gde09 2.6 MB/s | 136 kB 00:00 2024-08-18T23:32:38.014 INFO:teuthology.orchestra.run.smithi123.stdout:(11/150): ceph-common-19.1.0-1382.gde09dd01.el9 4.4 MB/s | 21 MB 00:04 2024-08-18T23:32:38.040 INFO:teuthology.orchestra.run.smithi123.stdout:(12/150): cephfs-mirror-19.1.0-1382.gde09dd01.e 963 kB/s | 225 kB 00:00 2024-08-18T23:32:38.065 INFO:teuthology.orchestra.run.smithi123.stdout:(13/150): cephfs-shell-19.1.0-1382.gde09dd01.el 541 kB/s | 27 kB 00:00 2024-08-18T23:32:38.067 INFO:teuthology.orchestra.run.smithi078.stdout:(40/150): fuse-2.9.9-16.el9.x86_64.rpm 434 kB/s | 80 kB 00:00 2024-08-18T23:32:38.071 INFO:teuthology.orchestra.run.smithi161.stdout:(21/150): python3-cephfs-19.1.0-1382.gde09dd01. 3.2 MB/s | 188 kB 00:00 2024-08-18T23:32:38.090 INFO:teuthology.orchestra.run.smithi123.stdout:(14/150): libcephfs-devel-19.1.0-1382.gde09dd01 632 kB/s | 32 kB 00:00 2024-08-18T23:32:38.157 INFO:teuthology.orchestra.run.smithi123.stdout:(15/150): libcephsqlite-19.1.0-1382.gde09dd01.e 2.4 MB/s | 163 kB 00:00 2024-08-18T23:32:38.158 INFO:teuthology.orchestra.run.smithi078.stdout:(41/150): libquadmath-11.4.1-3.el9.x86_64.rpm 936 kB/s | 195 kB 00:00 2024-08-18T23:32:38.171 INFO:teuthology.orchestra.run.smithi161.stdout:(22/150): python3-rados-19.1.0-1382.gde09dd01.e 3.6 MB/s | 369 kB 00:00 2024-08-18T23:32:38.184 INFO:teuthology.orchestra.run.smithi078.stdout:(42/150): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-08-18T23:32:38.191 INFO:teuthology.orchestra.run.smithi123.stdout:(16/150): libcephfs2-19.1.0-1382.gde09dd01.el9. 5.9 MB/s | 753 kB 00:00 2024-08-18T23:32:38.208 INFO:teuthology.orchestra.run.smithi123.stdout:(17/150): librados-devel-19.1.0-1382.gde09dd01. 2.5 MB/s | 127 kB 00:00 2024-08-18T23:32:38.209 INFO:teuthology.orchestra.run.smithi078.stdout:(43/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-08-18T23:32:38.226 INFO:teuthology.orchestra.run.smithi078.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-08-18T23:32:38.260 INFO:teuthology.orchestra.run.smithi078.stdout:(45/150): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-08-18T23:32:38.271 INFO:teuthology.orchestra.run.smithi161.stdout:(23/150): librgw2-19.1.0-1382.gde09dd01.el9.x86 13 MB/s | 5.4 MB 00:00 2024-08-18T23:32:38.275 INFO:teuthology.orchestra.run.smithi123.stdout:(18/150): libradosstriper1-19.1.0-1382.gde09dd0 5.9 MB/s | 507 kB 00:00 2024-08-18T23:32:38.285 INFO:teuthology.orchestra.run.smithi078.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-08-18T23:32:38.297 INFO:teuthology.orchestra.run.smithi161.stdout:(24/150): python3-rbd-19.1.0-1382.gde09dd01.el9 2.7 MB/s | 350 kB 00:00 2024-08-18T23:32:38.302 INFO:teuthology.orchestra.run.smithi078.stdout:(47/150): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-08-18T23:32:38.322 INFO:teuthology.orchestra.run.smithi161.stdout:(25/150): python3-rgw-19.1.0-1382.gde09dd01.el9 2.3 MB/s | 117 kB 00:00 2024-08-18T23:32:38.325 INFO:teuthology.orchestra.run.smithi123.stdout:(19/150): python3-ceph-argparse-19.1.0-1382.gde 906 kB/s | 45 kB 00:00 2024-08-18T23:32:38.327 INFO:teuthology.orchestra.run.smithi078.stdout:(48/150): python3-urllib3-1.26.5-6.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-08-18T23:32:38.347 INFO:teuthology.orchestra.run.smithi161.stdout:(26/150): rbd-fuse-19.1.0-1382.gde09dd01.el9.x8 1.7 MB/s | 85 kB 00:00 2024-08-18T23:32:38.392 INFO:teuthology.orchestra.run.smithi123.stdout:(20/150): python3-ceph-common-19.1.0-1382.gde09 2.0 MB/s | 136 kB 00:00 2024-08-18T23:32:38.414 INFO:teuthology.orchestra.run.smithi161.stdout:(27/150): rbd-nbd-19.1.0-1382.gde09dd01.el9.x86 2.5 MB/s | 170 kB 00:00 2024-08-18T23:32:38.435 INFO:teuthology.orchestra.run.smithi078.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 990 kB/s | 107 kB 00:00 2024-08-18T23:32:38.442 INFO:teuthology.orchestra.run.smithi123.stdout:(21/150): python3-cephfs-19.1.0-1382.gde09dd01. 3.7 MB/s | 188 kB 00:00 2024-08-18T23:32:38.473 INFO:teuthology.orchestra.run.smithi161.stdout:(28/150): ceph-grafana-dashboards-19.1.0-1382.g 546 kB/s | 32 kB 00:00 2024-08-18T23:32:38.509 INFO:teuthology.orchestra.run.smithi123.stdout:(22/150): python3-rados-19.1.0-1382.gde09dd01.e 5.4 MB/s | 369 kB 00:00 2024-08-18T23:32:38.511 INFO:teuthology.orchestra.run.smithi078.stdout:(50/150): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-08-18T23:32:38.536 INFO:teuthology.orchestra.run.smithi078.stdout:(51/150): elfutils-libelf-devel-0.191-4.el9.x86 216 kB/s | 22 kB 00:00 2024-08-18T23:32:38.540 INFO:teuthology.orchestra.run.smithi161.stdout:(29/150): ceph-mgr-cephadm-19.1.0-1382.gde09dd0 2.1 MB/s | 146 kB 00:00 2024-08-18T23:32:38.561 INFO:teuthology.orchestra.run.smithi078.stdout:(52/150): bison-3.7.4-5.el9.x86_64.rpm 3.6 MB/s | 943 kB 00:00 2024-08-18T23:32:38.576 INFO:teuthology.orchestra.run.smithi123.stdout:(23/150): python3-rbd-19.1.0-1382.gde09dd01.el9 5.1 MB/s | 350 kB 00:00 2024-08-18T23:32:38.587 INFO:teuthology.orchestra.run.smithi078.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-08-18T23:32:38.590 INFO:teuthology.orchestra.run.smithi161.stdout:(30/150): rbd-mirror-19.1.0-1382.gde09dd01.el9. 12 MB/s | 3.1 MB 00:00 2024-08-18T23:32:38.612 INFO:teuthology.orchestra.run.smithi078.stdout:(54/150): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-08-18T23:32:38.635 INFO:teuthology.orchestra.run.smithi123.stdout:(24/150): python3-rgw-19.1.0-1382.gde09dd01.el9 2.0 MB/s | 117 kB 00:00 2024-08-18T23:32:38.637 INFO:teuthology.orchestra.run.smithi078.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-08-18T23:32:38.662 INFO:teuthology.orchestra.run.smithi078.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-08-18T23:32:38.679 INFO:teuthology.orchestra.run.smithi078.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-08-18T23:32:38.701 INFO:teuthology.orchestra.run.smithi123.stdout:(25/150): rbd-fuse-19.1.0-1382.gde09dd01.el9.x8 1.2 MB/s | 85 kB 00:00 2024-08-18T23:32:38.704 INFO:teuthology.orchestra.run.smithi078.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-08-18T23:32:38.772 INFO:teuthology.orchestra.run.smithi078.stdout:(59/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-08-18T23:32:38.797 INFO:teuthology.orchestra.run.smithi078.stdout:(60/150): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-08-18T23:32:38.830 INFO:teuthology.orchestra.run.smithi078.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-08-18T23:32:38.856 INFO:teuthology.orchestra.run.smithi078.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-08-18T23:32:38.881 INFO:teuthology.orchestra.run.smithi078.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-08-18T23:32:38.906 INFO:teuthology.orchestra.run.smithi078.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-08-18T23:32:38.931 INFO:teuthology.orchestra.run.smithi078.stdout:(65/150): libtool-2.4.6-46.el9.x86_64.rpm 2.5 MB/s | 581 kB 00:00 2024-08-18T23:32:38.957 INFO:teuthology.orchestra.run.smithi078.stdout:(66/150): openblas-0.3.21-2.el9.x86_64.rpm 467 kB/s | 35 kB 00:00 2024-08-18T23:32:39.002 INFO:teuthology.orchestra.run.smithi123.stdout:(26/150): librgw2-19.1.0-1382.gde09dd01.el9.x86 6.8 MB/s | 5.4 MB 00:00 2024-08-18T23:32:39.032 INFO:teuthology.orchestra.run.smithi161.stdout:(31/150): ceph-mgr-dashboard-19.1.0-1382.gde09d 7.2 MB/s | 3.6 MB 00:00 2024-08-18T23:32:39.040 INFO:teuthology.orchestra.run.smithi078.stdout:(67/150): openblas-openmp-0.3.21-2.el9.x86_64.r 35 MB/s | 4.7 MB 00:00 2024-08-18T23:32:39.083 INFO:teuthology.orchestra.run.smithi161.stdout:(32/150): ceph-mgr-modules-core-19.1.0-1382.gde 4.9 MB/s | 249 kB 00:00 2024-08-18T23:32:39.083 INFO:teuthology.orchestra.run.smithi078.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-08-18T23:32:39.085 INFO:teuthology.orchestra.run.smithi123.stdout:(27/150): rbd-mirror-19.1.0-1382.gde09dd01.el9. 8.1 MB/s | 3.1 MB 00:00 2024-08-18T23:32:39.111 INFO:teuthology.orchestra.run.smithi123.stdout:(28/150): rbd-nbd-19.1.0-1382.gde09dd01.el9.x86 1.5 MB/s | 170 kB 00:00 2024-08-18T23:32:39.116 INFO:teuthology.orchestra.run.smithi078.stdout:(69/150): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-08-18T23:32:39.141 INFO:teuthology.orchestra.run.smithi078.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-08-18T23:32:39.144 INFO:teuthology.orchestra.run.smithi123.stdout:(29/150): ceph-grafana-dashboards-19.1.0-1382.g 542 kB/s | 32 kB 00:00 2024-08-18T23:32:39.170 INFO:teuthology.orchestra.run.smithi123.stdout:(30/150): ceph-mgr-cephadm-19.1.0-1382.gde09dd0 2.4 MB/s | 146 kB 00:00 2024-08-18T23:32:39.191 INFO:teuthology.orchestra.run.smithi161.stdout:(33/150): ceph-mgr-diskprediction-local-19.1.0- 12 MB/s | 7.4 MB 00:00 2024-08-18T23:32:39.208 INFO:teuthology.orchestra.run.smithi078.stdout:(71/150): openssl-devel-3.2.2-2.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-08-18T23:32:39.216 INFO:teuthology.orchestra.run.smithi161.stdout:(34/150): ceph-mgr-rook-19.1.0-1382.gde09dd01.e 369 kB/s | 49 kB 00:00 2024-08-18T23:32:39.242 INFO:teuthology.orchestra.run.smithi161.stdout:(35/150): ceph-prometheus-alerts-19.1.0-1382.gd 327 kB/s | 16 kB 00:00 2024-08-18T23:32:39.242 INFO:teuthology.orchestra.run.smithi078.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-08-18T23:32:39.267 INFO:teuthology.orchestra.run.smithi078.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-08-18T23:32:39.267 INFO:teuthology.orchestra.run.smithi161.stdout:(36/150): ceph-volume-19.1.0-1382.gde09dd01.el9 5.6 MB/s | 285 kB 00:00 2024-08-18T23:32:39.325 INFO:teuthology.orchestra.run.smithi161.stdout:(37/150): cephadm-19.1.0-1382.gde09dd01.el9.noa 9.0 MB/s | 766 kB 00:00 2024-08-18T23:32:39.351 INFO:teuthology.orchestra.run.smithi161.stdout:(38/150): cephfs-top-19.1.0-1382.gde09dd01.el9. 218 kB/s | 18 kB 00:00 2024-08-18T23:32:39.359 INFO:teuthology.orchestra.run.smithi078.stdout:(74/150): postgresql-server-13.14-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-08-18T23:32:39.384 INFO:teuthology.orchestra.run.smithi078.stdout:(75/150): python3-colorama-0.4.6-3.el9.noarch.r 524 kB/s | 61 kB 00:00 2024-08-18T23:32:39.409 INFO:teuthology.orchestra.run.smithi078.stdout:(76/150): python3-devel-3.9.19-4.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-08-18T23:32:39.434 INFO:teuthology.orchestra.run.smithi078.stdout:(77/150): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-08-18T23:32:39.460 INFO:teuthology.orchestra.run.smithi078.stdout:(78/150): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-08-18T23:32:39.485 INFO:teuthology.orchestra.run.smithi078.stdout:(79/150): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-08-18T23:32:39.511 INFO:teuthology.orchestra.run.smithi078.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-08-18T23:32:39.536 INFO:teuthology.orchestra.run.smithi078.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-08-18T23:32:39.586 INFO:teuthology.orchestra.run.smithi123.stdout:(31/150): ceph-test-19.1.0-1382.gde09dd01.el9.x 13 MB/s | 49 MB 00:03 2024-08-18T23:32:39.628 INFO:teuthology.orchestra.run.smithi078.stdout:(82/150): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-08-18T23:32:39.653 INFO:teuthology.orchestra.run.smithi078.stdout:(83/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-08-18T23:32:39.662 INFO:teuthology.orchestra.run.smithi123.stdout:(32/150): ceph-mgr-dashboard-19.1.0-1382.gde09d 6.9 MB/s | 3.6 MB 00:00 2024-08-18T23:32:39.678 INFO:teuthology.orchestra.run.smithi078.stdout:(84/150): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-08-18T23:32:39.687 INFO:teuthology.orchestra.run.smithi123.stdout:(33/150): ceph-mgr-modules-core-19.1.0-1382.gde 2.4 MB/s | 249 kB 00:00 2024-08-18T23:32:39.703 INFO:teuthology.orchestra.run.smithi078.stdout:(85/150): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-08-18T23:32:39.712 INFO:teuthology.orchestra.run.smithi123.stdout:(34/150): ceph-mgr-rook-19.1.0-1382.gde09dd01.e 986 kB/s | 49 kB 00:00 2024-08-18T23:32:39.729 INFO:teuthology.orchestra.run.smithi078.stdout:(86/150): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-08-18T23:32:39.737 INFO:teuthology.orchestra.run.smithi123.stdout:(35/150): ceph-prometheus-alerts-19.1.0-1382.gd 328 kB/s | 16 kB 00:00 2024-08-18T23:32:39.754 INFO:teuthology.orchestra.run.smithi078.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-08-18T23:32:39.763 INFO:teuthology.orchestra.run.smithi123.stdout:(36/150): ceph-volume-19.1.0-1382.gde09dd01.el9 5.6 MB/s | 285 kB 00:00 2024-08-18T23:32:39.776 INFO:teuthology.orchestra.run.smithi161.stdout:(39/150): fuse-2.9.9-16.el9.x86_64.rpm 177 kB/s | 80 kB 00:00 2024-08-18T23:32:39.779 INFO:teuthology.orchestra.run.smithi078.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-08-18T23:32:39.804 INFO:teuthology.orchestra.run.smithi078.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-08-18T23:32:39.813 INFO:teuthology.orchestra.run.smithi123.stdout:(37/150): cephfs-top-19.1.0-1382.gde09dd01.el9. 366 kB/s | 18 kB 00:00 2024-08-18T23:32:39.830 INFO:teuthology.orchestra.run.smithi078.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-08-18T23:32:39.838 INFO:teuthology.orchestra.run.smithi123.stdout:(38/150): cephadm-19.1.0-1382.gde09dd01.el9.noa 7.4 MB/s | 766 kB 00:00 2024-08-18T23:32:39.855 INFO:teuthology.orchestra.run.smithi078.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-08-18T23:32:39.922 INFO:teuthology.orchestra.run.smithi078.stdout:(92/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-08-18T23:32:39.947 INFO:teuthology.orchestra.run.smithi078.stdout:(93/150): python3-toml-0.10.2-6.el9.noarch.rpm 453 kB/s | 42 kB 00:00 2024-08-18T23:32:39.955 INFO:teuthology.orchestra.run.smithi123.stdout:(39/150): fuse-2.9.9-16.el9.x86_64.rpm 563 kB/s | 80 kB 00:00 2024-08-18T23:32:39.972 INFO:teuthology.orchestra.run.smithi078.stdout:(94/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-08-18T23:32:39.997 INFO:teuthology.orchestra.run.smithi078.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-08-18T23:32:40.023 INFO:teuthology.orchestra.run.smithi078.stdout:(96/150): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-08-18T23:32:40.043 INFO:teuthology.orchestra.run.smithi161.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 733 kB/s | 195 kB 00:00 2024-08-18T23:32:40.048 INFO:teuthology.orchestra.run.smithi078.stdout:(97/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-08-18T23:32:40.081 INFO:teuthology.orchestra.run.smithi123.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-08-18T23:32:40.081 INFO:teuthology.orchestra.run.smithi078.stdout:(98/150): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-08-18T23:32:40.093 INFO:teuthology.orchestra.run.smithi161.stdout:(41/150): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-08-18T23:32:40.106 INFO:teuthology.orchestra.run.smithi123.stdout:(41/150): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-08-18T23:32:40.107 INFO:teuthology.orchestra.run.smithi078.stdout:(99/150): uuid-1.6.2-55.el9.x86_64.rpm 991 kB/s | 58 kB 00:00 2024-08-18T23:32:40.132 INFO:teuthology.orchestra.run.smithi078.stdout:(100/150): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-08-18T23:32:40.157 INFO:teuthology.orchestra.run.smithi078.stdout:(101/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-08-18T23:32:40.183 INFO:teuthology.orchestra.run.smithi078.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-08-18T23:32:40.198 INFO:teuthology.orchestra.run.smithi123.stdout:(42/150): ceph-mgr-diskprediction-local-19.1.0- 7.2 MB/s | 7.4 MB 00:01 2024-08-18T23:32:40.223 INFO:teuthology.orchestra.run.smithi123.stdout:(43/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-08-18T23:32:40.235 INFO:teuthology.orchestra.run.smithi161.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-08-18T23:32:40.244 INFO:teuthology.orchestra.run.smithi078.stdout:(103/150): lua-devel-5.4.4-4.el9.x86_64.rpm 260 kB/s | 22 kB 00:00 2024-08-18T23:32:40.257 INFO:teuthology.orchestra.run.smithi123.stdout:(44/150): python3-cryptography-36.0.1-4.el9.x86 8.1 MB/s | 1.2 MB 00:00 2024-08-18T23:32:40.274 INFO:teuthology.orchestra.run.smithi123.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-08-18T23:32:40.302 INFO:teuthology.orchestra.run.smithi078.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-08-18T23:32:40.307 INFO:teuthology.orchestra.run.smithi123.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-08-18T23:32:40.332 INFO:teuthology.orchestra.run.smithi123.stdout:(47/150): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-08-18T23:32:40.335 INFO:teuthology.orchestra.run.smithi161.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-08-18T23:32:40.358 INFO:teuthology.orchestra.run.smithi123.stdout:(48/150): python3-ply-3.11-14.el9.noarch.rpm 669 kB/s | 106 kB 00:00 2024-08-18T23:32:40.369 INFO:teuthology.orchestra.run.smithi161.stdout:(44/150): python3-cryptography-36.0.1-4.el9.x86 4.4 MB/s | 1.2 MB 00:00 2024-08-18T23:32:40.445 INFO:teuthology.orchestra.run.smithi161.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-08-18T23:32:40.458 INFO:teuthology.orchestra.run.smithi123.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-08-18T23:32:40.461 INFO:teuthology.orchestra.run.smithi078.stdout:(105/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-08-18T23:32:40.478 INFO:teuthology.orchestra.run.smithi161.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-08-18T23:32:40.486 INFO:teuthology.orchestra.run.smithi078.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 1.6 MB/s | 309 kB 00:00 2024-08-18T23:32:40.493 INFO:teuthology.orchestra.run.smithi123.stdout:(50/150): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-08-18T23:32:40.508 INFO:teuthology.orchestra.run.smithi123.stdout:(51/150): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-08-18T23:32:40.520 INFO:teuthology.orchestra.run.smithi078.stdout:(107/150): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-08-18T23:32:40.534 INFO:teuthology.orchestra.run.smithi123.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-08-18T23:32:40.545 INFO:teuthology.orchestra.run.smithi161.stdout:(47/150): python3-urllib3-1.26.5-6.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-08-18T23:32:40.550 INFO:teuthology.orchestra.run.smithi123.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 781 kB/s | 33 kB 00:00 2024-08-18T23:32:40.553 INFO:teuthology.orchestra.run.smithi078.stdout:(108/150): grpc-data-1.46.7-10.el9.noarch.rpm 211 kB/s | 19 kB 00:00 2024-08-18T23:32:40.576 INFO:teuthology.orchestra.run.smithi123.stdout:(54/150): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-08-18T23:32:40.578 INFO:teuthology.orchestra.run.smithi078.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-08-18T23:32:40.593 INFO:teuthology.orchestra.run.smithi123.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 435 kB/s | 18 kB 00:00 2024-08-18T23:32:40.604 INFO:teuthology.orchestra.run.smithi078.stdout:(110/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 317 kB/s | 26 kB 00:00 2024-08-18T23:32:40.610 INFO:teuthology.orchestra.run.smithi123.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 334 kB/s | 11 kB 00:00 2024-08-18T23:32:40.629 INFO:teuthology.orchestra.run.smithi078.stdout:(111/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-08-18T23:32:40.646 INFO:teuthology.orchestra.run.smithi078.stdout:(112/150): luarocks-3.9.2-5.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-08-18T23:32:40.652 INFO:teuthology.orchestra.run.smithi123.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-08-18T23:32:40.668 INFO:teuthology.orchestra.run.smithi123.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-08-18T23:32:40.670 INFO:teuthology.orchestra.run.smithi161.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-08-18T23:32:40.696 INFO:teuthology.orchestra.run.smithi078.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-08-18T23:32:40.712 INFO:teuthology.orchestra.run.smithi161.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-08-18T23:32:40.719 INFO:teuthology.orchestra.run.smithi123.stdout:(59/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-08-18T23:32:40.730 INFO:teuthology.orchestra.run.smithi078.stdout:(114/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-08-18T23:32:40.753 INFO:teuthology.orchestra.run.smithi123.stdout:(60/150): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-08-18T23:32:40.754 INFO:teuthology.orchestra.run.smithi161.stdout:(50/150): elfutils-libelf-devel-0.191-4.el9.x86 526 kB/s | 22 kB 00:00 2024-08-18T23:32:40.755 INFO:teuthology.orchestra.run.smithi078.stdout:(115/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 742 kB/s | 43 kB 00:00 2024-08-18T23:32:40.769 INFO:teuthology.orchestra.run.smithi123.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 508 kB/s | 25 kB 00:00 2024-08-18T23:32:40.779 INFO:teuthology.orchestra.run.smithi161.stdout:(51/150): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 943 kB 00:00 2024-08-18T23:32:40.781 INFO:teuthology.orchestra.run.smithi078.stdout:(116/150): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-08-18T23:32:40.794 INFO:teuthology.orchestra.run.smithi123.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-08-18T23:32:40.805 INFO:teuthology.orchestra.run.smithi161.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-08-18T23:32:40.806 INFO:teuthology.orchestra.run.smithi078.stdout:(117/150): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-08-18T23:32:40.830 INFO:teuthology.orchestra.run.smithi161.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-08-18T23:32:40.831 INFO:teuthology.orchestra.run.smithi078.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-08-18T23:32:40.841 INFO:teuthology.orchestra.run.smithi123.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.0 MB/s | 292 kB 00:00 2024-08-18T23:32:40.856 INFO:teuthology.orchestra.run.smithi078.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-08-18T23:32:40.870 INFO:teuthology.orchestra.run.smithi123.stdout:(64/150): libtool-2.4.6-46.el9.x86_64.rpm 2.8 MB/s | 581 kB 00:00 2024-08-18T23:32:40.872 INFO:teuthology.orchestra.run.smithi161.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-08-18T23:32:40.882 INFO:teuthology.orchestra.run.smithi078.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-08-18T23:32:40.895 INFO:teuthology.orchestra.run.smithi123.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-08-18T23:32:40.907 INFO:teuthology.orchestra.run.smithi078.stdout:(121/150): python3-google-auth-2.33.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-08-18T23:32:40.914 INFO:teuthology.orchestra.run.smithi161.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 269 kB/s | 11 kB 00:00 2024-08-18T23:32:40.921 INFO:teuthology.orchestra.run.smithi123.stdout:(66/150): openblas-0.3.21-2.el9.x86_64.rpm 447 kB/s | 35 kB 00:00 2024-08-18T23:32:40.932 INFO:teuthology.orchestra.run.smithi078.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-08-18T23:32:40.958 INFO:teuthology.orchestra.run.smithi078.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-08-18T23:32:40.964 INFO:teuthology.orchestra.run.smithi161.stdout:(56/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-08-18T23:32:40.983 INFO:teuthology.orchestra.run.smithi078.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-08-18T23:32:40.989 INFO:teuthology.orchestra.run.smithi161.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-08-18T23:32:41.005 INFO:teuthology.orchestra.run.smithi123.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-08-18T23:32:41.008 INFO:teuthology.orchestra.run.smithi078.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 927 kB/s | 23 kB 00:00 2024-08-18T23:32:41.015 INFO:teuthology.orchestra.run.smithi161.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-08-18T23:32:41.057 INFO:teuthology.orchestra.run.smithi161.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-08-18T23:32:41.071 INFO:teuthology.orchestra.run.smithi123.stdout:(68/150): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-08-18T23:32:41.075 INFO:teuthology.orchestra.run.smithi078.stdout:(126/150): libarrow-9.0.0-11.el9.x86_64.rpm 7.6 MB/s | 4.4 MB 00:00 2024-08-18T23:32:41.090 INFO:teuthology.orchestra.run.smithi161.stdout:(60/150): libtool-2.4.6-46.el9.x86_64.rpm 7.6 MB/s | 581 kB 00:00 2024-08-18T23:32:41.100 INFO:teuthology.orchestra.run.smithi078.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-08-18T23:32:41.116 INFO:teuthology.orchestra.run.smithi161.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 434 kB/s | 25 kB 00:00 2024-08-18T23:32:41.126 INFO:teuthology.orchestra.run.smithi078.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-08-18T23:32:41.130 INFO:teuthology.orchestra.run.smithi123.stdout:(69/150): openssl-devel-3.2.2-2.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-08-18T23:32:41.151 INFO:teuthology.orchestra.run.smithi078.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-18T23:32:41.155 INFO:teuthology.orchestra.run.smithi123.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-08-18T23:32:41.166 INFO:teuthology.orchestra.run.smithi161.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-08-18T23:32:41.176 INFO:teuthology.orchestra.run.smithi078.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-18T23:32:41.191 INFO:teuthology.orchestra.run.smithi161.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-08-18T23:32:41.197 INFO:teuthology.orchestra.run.smithi123.stdout:(71/150): postgresql-contrib-13.14-1.el9.x86_64 4.6 MB/s | 911 kB 00:00 2024-08-18T23:32:41.210 INFO:teuthology.orchestra.run.smithi078.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-08-18T23:32:41.216 INFO:teuthology.orchestra.run.smithi161.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-08-18T23:32:41.241 INFO:teuthology.orchestra.run.smithi123.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-08-18T23:32:41.242 INFO:teuthology.orchestra.run.smithi161.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 704 kB/s | 35 kB 00:00 2024-08-18T23:32:41.244 INFO:teuthology.orchestra.run.smithi078.stdout:(132/150): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-08-18T23:32:41.266 INFO:teuthology.orchestra.run.smithi123.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 87 kB 00:00 2024-08-18T23:32:41.278 INFO:teuthology.orchestra.run.smithi078.stdout:(133/150): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-08-18T23:32:41.303 INFO:teuthology.orchestra.run.smithi078.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-08-18T23:32:41.316 INFO:teuthology.orchestra.run.smithi123.stdout:(74/150): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-08-18T23:32:41.353 INFO:teuthology.orchestra.run.smithi078.stdout:(135/150): python3-grpcio-1.46.7-10.el9.x86_64. 4.3 MB/s | 2.0 MB 00:00 2024-08-18T23:32:41.378 INFO:teuthology.orchestra.run.smithi078.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-08-18T23:32:41.392 INFO:teuthology.orchestra.run.smithi123.stdout:(75/150): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-08-18T23:32:41.404 INFO:teuthology.orchestra.run.smithi078.stdout:(137/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 715 kB/s | 90 kB 00:00 2024-08-18T23:32:41.417 INFO:teuthology.orchestra.run.smithi123.stdout:(76/150): python3-devel-3.9.19-4.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-08-18T23:32:41.429 INFO:teuthology.orchestra.run.smithi078.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-08-18T23:32:41.442 INFO:teuthology.orchestra.run.smithi123.stdout:(77/150): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-08-18T23:32:41.454 INFO:teuthology.orchestra.run.smithi078.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-08-18T23:32:41.467 INFO:teuthology.orchestra.run.smithi123.stdout:(78/150): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-08-18T23:32:41.479 INFO:teuthology.orchestra.run.smithi078.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-08-18T23:32:41.493 INFO:teuthology.orchestra.run.smithi123.stdout:(79/150): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-08-18T23:32:41.505 INFO:teuthology.orchestra.run.smithi078.stdout:(141/150): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-08-18T23:32:41.518 INFO:teuthology.orchestra.run.smithi123.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-08-18T23:32:41.538 INFO:teuthology.orchestra.run.smithi078.stdout:(142/150): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-08-18T23:32:41.543 INFO:teuthology.orchestra.run.smithi123.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-08-18T23:32:41.564 INFO:teuthology.orchestra.run.smithi078.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-08-18T23:32:41.589 INFO:teuthology.orchestra.run.smithi078.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-08-18T23:32:41.614 INFO:teuthology.orchestra.run.smithi078.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-08-18T23:32:41.627 INFO:teuthology.orchestra.run.smithi123.stdout:(82/150): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-08-18T23:32:41.639 INFO:teuthology.orchestra.run.smithi078.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-08-18T23:32:41.652 INFO:teuthology.orchestra.run.smithi123.stdout:(83/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-08-18T23:32:41.665 INFO:teuthology.orchestra.run.smithi078.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-08-18T23:32:41.667 INFO:teuthology.orchestra.run.smithi161.stdout:(66/150): ceph-test-19.1.0-1382.gde09dd01.el9.x 11 MB/s | 49 MB 00:04 2024-08-18T23:32:41.677 INFO:teuthology.orchestra.run.smithi123.stdout:(84/150): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-08-18T23:32:41.703 INFO:teuthology.orchestra.run.smithi123.stdout:(85/150): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-08-18T23:32:41.707 INFO:teuthology.orchestra.run.smithi078.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-08-18T23:32:41.728 INFO:teuthology.orchestra.run.smithi123.stdout:(86/150): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-08-18T23:32:41.753 INFO:teuthology.orchestra.run.smithi123.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-08-18T23:32:41.759 INFO:teuthology.orchestra.run.smithi161.stdout:(67/150): openblas-openmp-0.3.21-2.el9.x86_64.r 8.7 MB/s | 4.7 MB 00:00 2024-08-18T23:32:41.778 INFO:teuthology.orchestra.run.smithi123.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-08-18T23:32:41.804 INFO:teuthology.orchestra.run.smithi123.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-08-18T23:32:41.826 INFO:teuthology.orchestra.run.smithi161.stdout:(68/150): openssl-devel-3.2.2-2.el9.x86_64.rpm 7.6 MB/s | 4.4 MB 00:00 2024-08-18T23:32:41.829 INFO:teuthology.orchestra.run.smithi123.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-08-18T23:32:41.854 INFO:teuthology.orchestra.run.smithi123.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-08-18T23:32:41.859 INFO:teuthology.orchestra.run.smithi161.stdout:(69/150): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-08-18T23:32:41.885 INFO:teuthology.orchestra.run.smithi161.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-08-18T23:32:41.899 INFO:teuthology.orchestra.run.smithi078.stdout:(149/150): librados2-19.1.0-1382.gde09dd01.el9. 13 MB/s | 3.4 MB 00:00 2024-08-18T23:32:41.930 INFO:teuthology.orchestra.run.smithi123.stdout:(92/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-08-18T23:32:41.955 INFO:teuthology.orchestra.run.smithi123.stdout:(93/150): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-08-18T23:32:41.980 INFO:teuthology.orchestra.run.smithi123.stdout:(94/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-08-18T23:32:41.993 INFO:teuthology.orchestra.run.smithi161.stdout:(71/150): postgresql-server-13.14-1.el9.x86_64. 43 MB/s | 5.8 MB 00:00 2024-08-18T23:32:42.005 INFO:teuthology.orchestra.run.smithi123.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-08-18T23:32:42.027 INFO:teuthology.orchestra.run.smithi161.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-08-18T23:32:42.031 INFO:teuthology.orchestra.run.smithi123.stdout:(96/150): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-08-18T23:32:42.040 INFO:teuthology.orchestra.run.smithi078.stdout:(150/150): librbd1-19.1.0-1382.gde09dd01.el9.x8 8.4 MB/s | 3.2 MB 00:00 2024-08-18T23:32:42.047 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-08-18T23:32:42.047 INFO:teuthology.orchestra.run.smithi078.stdout:Total 22 MB/s | 218 MB 00:09 2024-08-18T23:32:42.052 INFO:teuthology.orchestra.run.smithi161.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-08-18T23:32:42.056 INFO:teuthology.orchestra.run.smithi123.stdout:(97/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-08-18T23:32:42.081 INFO:teuthology.orchestra.run.smithi123.stdout:(98/150): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-08-18T23:32:42.094 INFO:teuthology.orchestra.run.smithi161.stdout:(74/150): postgresql-13.14-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-08-18T23:32:42.106 INFO:teuthology.orchestra.run.smithi123.stdout:(99/150): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-08-18T23:32:42.120 INFO:teuthology.orchestra.run.smithi161.stdout:(75/150): python3-colorama-0.4.6-3.el9.noarch.r 917 kB/s | 61 kB 00:00 2024-08-18T23:32:42.132 INFO:teuthology.orchestra.run.smithi123.stdout:(100/150): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-08-18T23:32:42.145 INFO:teuthology.orchestra.run.smithi161.stdout:(76/150): python3-devel-3.9.19-4.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-08-18T23:32:42.157 INFO:teuthology.orchestra.run.smithi123.stdout:(101/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-08-18T23:32:42.170 INFO:teuthology.orchestra.run.smithi161.stdout:(77/150): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-08-18T23:32:42.182 INFO:teuthology.orchestra.run.smithi123.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-08-18T23:32:42.195 INFO:teuthology.orchestra.run.smithi161.stdout:(78/150): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-08-18T23:32:42.221 INFO:teuthology.orchestra.run.smithi161.stdout:(79/150): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-08-18T23:32:42.246 INFO:teuthology.orchestra.run.smithi161.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-08-18T23:32:42.263 INFO:teuthology.orchestra.run.smithi161.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 827 kB/s | 35 kB 00:00 2024-08-18T23:32:42.313 INFO:teuthology.orchestra.run.smithi161.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-08-18T23:32:42.397 INFO:teuthology.orchestra.run.smithi161.stdout:(83/150): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-08-18T23:32:42.422 INFO:teuthology.orchestra.run.smithi161.stdout:(84/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-08-18T23:32:42.441 INFO:teuthology.orchestra.run.smithi123.stdout:(103/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-08-18T23:32:42.447 INFO:teuthology.orchestra.run.smithi161.stdout:(85/150): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-08-18T23:32:42.466 INFO:teuthology.orchestra.run.smithi123.stdout:(104/150): lua-devel-5.4.4-4.el9.x86_64.rpm 72 kB/s | 22 kB 00:00 2024-08-18T23:32:42.472 INFO:teuthology.orchestra.run.smithi161.stdout:(86/150): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-08-18T23:32:42.498 INFO:teuthology.orchestra.run.smithi161.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-08-18T23:32:42.516 INFO:teuthology.orchestra.run.smithi123.stdout:(105/150): abseil-cpp-20211102.0-3.el9.x86_64.r 7.2 MB/s | 548 kB 00:00 2024-08-18T23:32:42.523 INFO:teuthology.orchestra.run.smithi161.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-08-18T23:32:42.533 INFO:teuthology.orchestra.run.smithi123.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-08-18T23:32:42.548 INFO:teuthology.orchestra.run.smithi161.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-08-18T23:32:42.559 INFO:teuthology.orchestra.run.smithi123.stdout:(107/150): gperftools-libs-2.9.1-2.el9.x86_64.r 3.3 MB/s | 309 kB 00:00 2024-08-18T23:32:42.574 INFO:teuthology.orchestra.run.smithi161.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-08-18T23:32:42.576 INFO:teuthology.orchestra.run.smithi123.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-08-18T23:32:42.599 INFO:teuthology.orchestra.run.smithi161.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-08-18T23:32:42.601 INFO:teuthology.orchestra.run.smithi123.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-18T23:32:42.626 INFO:teuthology.orchestra.run.smithi123.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-08-18T23:32:42.660 INFO:teuthology.orchestra.run.smithi123.stdout:(111/150): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-08-18T23:32:42.683 INFO:teuthology.orchestra.run.smithi161.stdout:(92/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-08-18T23:32:42.708 INFO:teuthology.orchestra.run.smithi161.stdout:(93/150): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 42 kB 00:00 2024-08-18T23:32:42.710 INFO:teuthology.orchestra.run.smithi123.stdout:(112/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-08-18T23:32:42.733 INFO:teuthology.orchestra.run.smithi161.stdout:(94/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-08-18T23:32:42.752 INFO:teuthology.orchestra.run.smithi123.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-08-18T23:32:42.758 INFO:teuthology.orchestra.run.smithi161.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-08-18T23:32:42.784 INFO:teuthology.orchestra.run.smithi161.stdout:(96/150): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-08-18T23:32:42.809 INFO:teuthology.orchestra.run.smithi161.stdout:(97/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-08-18T23:32:42.810 INFO:teuthology.orchestra.run.smithi123.stdout:(114/150): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-08-18T23:32:42.834 INFO:teuthology.orchestra.run.smithi161.stdout:(98/150): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-08-18T23:32:42.836 INFO:teuthology.orchestra.run.smithi123.stdout:(115/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 519 kB/s | 43 kB 00:00 2024-08-18T23:32:42.860 INFO:teuthology.orchestra.run.smithi161.stdout:(99/150): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-08-18T23:32:42.861 INFO:teuthology.orchestra.run.smithi123.stdout:(116/150): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-08-18T23:32:42.885 INFO:teuthology.orchestra.run.smithi161.stdout:(100/150): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-08-18T23:32:42.886 INFO:teuthology.orchestra.run.smithi123.stdout:(117/150): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-08-18T23:32:42.910 INFO:teuthology.orchestra.run.smithi161.stdout:(101/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-08-18T23:32:42.912 INFO:teuthology.orchestra.run.smithi123.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-08-18T23:32:42.935 INFO:teuthology.orchestra.run.smithi161.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-08-18T23:32:42.937 INFO:teuthology.orchestra.run.smithi123.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-08-18T23:32:42.962 INFO:teuthology.orchestra.run.smithi123.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-08-18T23:32:42.987 INFO:teuthology.orchestra.run.smithi123.stdout:(121/150): python3-google-auth-2.33.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-08-18T23:32:43.021 INFO:teuthology.orchestra.run.smithi123.stdout:(122/150): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-08-18T23:32:43.046 INFO:teuthology.orchestra.run.smithi123.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-08-18T23:32:43.080 INFO:teuthology.orchestra.run.smithi123.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 673 kB/s | 18 kB 00:00 2024-08-18T23:32:43.105 INFO:teuthology.orchestra.run.smithi123.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 1.0 MB/s | 23 kB 00:00 2024-08-18T23:32:43.131 INFO:teuthology.orchestra.run.smithi123.stdout:(126/150): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-08-18T23:32:43.156 INFO:teuthology.orchestra.run.smithi123.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-08-18T23:32:43.166 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-08-18T23:32:43.190 INFO:teuthology.orchestra.run.smithi123.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 332 kB/s | 20 kB 00:00 2024-08-18T23:32:43.223 INFO:teuthology.orchestra.run.smithi123.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-08-18T23:32:43.236 INFO:teuthology.orchestra.run.smithi161.stdout:(103/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-08-18T23:32:43.248 INFO:teuthology.orchestra.run.smithi123.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-18T23:32:43.274 INFO:teuthology.orchestra.run.smithi123.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-18T23:32:43.286 INFO:teuthology.orchestra.run.smithi161.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-08-18T23:32:43.300 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-08-18T23:32:43.300 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-08-18T23:32:43.307 INFO:teuthology.orchestra.run.smithi123.stdout:(132/150): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-08-18T23:32:43.320 INFO:teuthology.orchestra.run.smithi161.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-08-18T23:32:43.350 INFO:teuthology.orchestra.run.smithi123.stdout:(133/150): python3-grpcio-1.46.7-10.el9.x86_64. 5.3 MB/s | 2.0 MB 00:00 2024-08-18T23:32:43.361 INFO:teuthology.orchestra.run.smithi161.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 735 kB/s | 19 kB 00:00 2024-08-18T23:32:43.375 INFO:teuthology.orchestra.run.smithi123.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-08-18T23:32:43.400 INFO:teuthology.orchestra.run.smithi123.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-08-18T23:32:43.425 INFO:teuthology.orchestra.run.smithi123.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-08-18T23:32:43.459 INFO:teuthology.orchestra.run.smithi123.stdout:(137/150): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-08-18T23:32:43.484 INFO:teuthology.orchestra.run.smithi123.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 368 kB/s | 31 kB 00:00 2024-08-18T23:32:43.487 INFO:teuthology.orchestra.run.smithi161.stdout:(107/150): libarrow-9.0.0-11.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-08-18T23:32:43.509 INFO:teuthology.orchestra.run.smithi123.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-18T23:32:43.512 INFO:teuthology.orchestra.run.smithi161.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-08-18T23:32:43.534 INFO:teuthology.orchestra.run.smithi123.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-08-18T23:32:43.537 INFO:teuthology.orchestra.run.smithi161.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-18T23:32:43.560 INFO:teuthology.orchestra.run.smithi123.stdout:(141/150): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-08-18T23:32:43.563 INFO:teuthology.orchestra.run.smithi161.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-08-18T23:32:43.576 INFO:teuthology.orchestra.run.smithi123.stdout:(142/150): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-08-18T23:32:43.588 INFO:teuthology.orchestra.run.smithi161.stdout:(111/150): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-08-18T23:32:43.602 INFO:teuthology.orchestra.run.smithi123.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-08-18T23:32:43.627 INFO:teuthology.orchestra.run.smithi123.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 797 kB/s | 20 kB 00:00 2024-08-18T23:32:43.630 INFO:teuthology.orchestra.run.smithi161.stdout:(112/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-08-18T23:32:43.652 INFO:teuthology.orchestra.run.smithi123.stdout:(145/150): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-08-18T23:32:43.655 INFO:teuthology.orchestra.run.smithi161.stdout:(113/150): lua-devel-5.4.4-4.el9.x86_64.rpm 30 kB/s | 22 kB 00:00 2024-08-18T23:32:43.678 INFO:teuthology.orchestra.run.smithi123.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-08-18T23:32:43.703 INFO:teuthology.orchestra.run.smithi123.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-08-18T23:32:43.705 INFO:teuthology.orchestra.run.smithi161.stdout:(114/150): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-08-18T23:32:43.731 INFO:teuthology.orchestra.run.smithi161.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 1.4 MB/s | 32 kB 00:00 2024-08-18T23:32:43.753 INFO:teuthology.orchestra.run.smithi123.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-08-18T23:32:43.756 INFO:teuthology.orchestra.run.smithi161.stdout:(116/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 433 kB/s | 43 kB 00:00 2024-08-18T23:32:43.781 INFO:teuthology.orchestra.run.smithi161.stdout:(117/150): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-08-18T23:32:43.815 INFO:teuthology.orchestra.run.smithi161.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-08-18T23:32:43.849 INFO:teuthology.orchestra.run.smithi161.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-08-18T23:32:43.882 INFO:teuthology.orchestra.run.smithi161.stdout:(120/150): python3-google-auth-2.33.0-1.el9.noa 6.5 MB/s | 220 kB 00:00 2024-08-18T23:32:43.907 INFO:teuthology.orchestra.run.smithi161.stdout:(121/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-08-18T23:32:43.941 INFO:teuthology.orchestra.run.smithi161.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-08-18T23:32:43.966 INFO:teuthology.orchestra.run.smithi161.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-08-18T23:32:43.992 INFO:teuthology.orchestra.run.smithi161.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-08-18T23:32:44.009 INFO:teuthology.orchestra.run.smithi161.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-08-18T23:32:44.025 INFO:teuthology.orchestra.run.smithi161.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-08-18T23:32:44.045 INFO:teuthology.orchestra.run.smithi123.stdout:(149/150): librados2-19.1.0-1382.gde09dd01.el9. 9.3 MB/s | 3.4 MB 00:00 2024-08-18T23:32:44.059 INFO:teuthology.orchestra.run.smithi161.stdout:(127/150): protobuf-compiler-3.14.0-13.el9.x86_ 768 kB/s | 863 kB 00:01 2024-08-18T23:32:44.076 INFO:teuthology.orchestra.run.smithi161.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-08-18T23:32:44.102 INFO:teuthology.orchestra.run.smithi161.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 2.6 MB/s | 46 kB 00:00 2024-08-18T23:32:44.103 INFO:teuthology.orchestra.run.smithi123.stdout:(150/150): librbd1-19.1.0-1382.gde09dd01.el9.x8 7.9 MB/s | 3.2 MB 00:00 2024-08-18T23:32:44.109 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-08-18T23:32:44.109 INFO:teuthology.orchestra.run.smithi123.stdout:Total 19 MB/s | 218 MB 00:11 2024-08-18T23:32:44.135 INFO:teuthology.orchestra.run.smithi161.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 2.5 MB/s | 79 kB 00:00 2024-08-18T23:32:44.161 INFO:teuthology.orchestra.run.smithi161.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-18T23:32:44.202 INFO:teuthology.orchestra.run.smithi161.stdout:(132/150): python3-grpcio-1.46.7-10.el9.x86_64. 6.4 MB/s | 2.0 MB 00:00 2024-08-18T23:32:44.236 INFO:teuthology.orchestra.run.smithi161.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 490 kB/s | 16 kB 00:00 2024-08-18T23:32:44.261 INFO:teuthology.orchestra.run.smithi161.stdout:(134/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-08-18T23:32:44.286 INFO:teuthology.orchestra.run.smithi161.stdout:(135/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-08-18T23:32:44.312 INFO:teuthology.orchestra.run.smithi161.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-08-18T23:32:44.337 INFO:teuthology.orchestra.run.smithi161.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-08-18T23:32:44.362 INFO:teuthology.orchestra.run.smithi161.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-18T23:32:44.388 INFO:teuthology.orchestra.run.smithi161.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-08-18T23:32:44.413 INFO:teuthology.orchestra.run.smithi161.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-08-18T23:32:44.438 INFO:teuthology.orchestra.run.smithi161.stdout:(141/150): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-08-18T23:32:44.463 INFO:teuthology.orchestra.run.smithi161.stdout:(142/150): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-08-18T23:32:44.489 INFO:teuthology.orchestra.run.smithi161.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-08-18T23:32:44.522 INFO:teuthology.orchestra.run.smithi161.stdout:(144/150): python3-kubernetes-26.1.0-2.el9.noar 2.2 MB/s | 1.0 MB 00:00 2024-08-18T23:32:44.547 INFO:teuthology.orchestra.run.smithi161.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-08-18T23:32:44.581 INFO:teuthology.orchestra.run.smithi161.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 5.7 MB/s | 192 kB 00:00 2024-08-18T23:32:44.607 INFO:teuthology.orchestra.run.smithi161.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-08-18T23:32:44.656 INFO:teuthology.orchestra.run.smithi161.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-08-18T23:32:44.890 INFO:teuthology.orchestra.run.smithi161.stdout:(149/150): librados2-19.1.0-1382.gde09dd01.el9. 11 MB/s | 3.4 MB 00:00 2024-08-18T23:32:44.966 INFO:teuthology.orchestra.run.smithi161.stdout:(150/150): librbd1-19.1.0-1382.gde09dd01.el9.x8 8.8 MB/s | 3.2 MB 00:00 2024-08-18T23:32:44.973 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-08-18T23:32:44.973 INFO:teuthology.orchestra.run.smithi161.stdout:Total 17 MB/s | 218 MB 00:12 2024-08-18T23:32:45.227 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-08-18T23:32:45.377 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-08-18T23:32:45.377 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-08-18T23:32:45.382 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-08-18T23:32:45.383 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-08-18T23:32:46.088 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-08-18T23:32:46.227 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-08-18T23:32:46.227 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-08-18T23:32:47.492 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-08-18T23:32:47.492 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-08-18T23:32:48.360 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-08-18T23:32:48.360 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-08-18T23:32:51.222 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-08-18T23:32:51.302 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-18T23:32:51.327 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-18T23:32:51.401 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-18T23:32:51.737 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-18T23:32:51.878 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/152 2024-08-18T23:32:52.001 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/152 2024-08-18T23:32:52.038 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-18T23:32:52.134 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-18T23:32:52.180 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 7/152 2024-08-18T23:32:52.209 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-18T23:32:52.234 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-18T23:32:52.439 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-08-18T23:32:52.491 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-08-18T23:32:52.518 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-18T23:32:52.607 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-18T23:32:52.636 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 13/152 2024-08-18T23:32:52.708 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 13/152 2024-08-18T23:32:52.737 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-18T23:32:52.760 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-18T23:32:53.098 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-18T23:32:53.129 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-18T23:32:53.239 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-08-18T23:32:53.334 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-18T23:32:53.378 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-08-18T23:32:53.413 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-08-18T23:32:53.488 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-08-18T23:32:53.496 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-08-18T23:32:53.540 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-08-18T23:32:53.564 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-18T23:32:53.580 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-packaging-20.9-5.el9.noarch 22/152 2024-08-18T23:32:53.598 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-18T23:32:53.624 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/152 2024-08-18T23:32:53.665 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/152 2024-08-18T23:32:53.670 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-18T23:32:53.692 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/152 2024-08-18T23:32:53.762 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/152 2024-08-18T23:32:53.824 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 27/152 2024-08-18T23:32:53.854 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/152 2024-08-18T23:32:53.897 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/152 2024-08-18T23:32:53.974 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/152 2024-08-18T23:32:54.005 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-18T23:32:54.007 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 31/152 2024-08-18T23:32:54.054 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 32/152 2024-08-18T23:32:54.080 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 33/152 2024-08-18T23:32:54.115 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/152 2024-08-18T23:32:54.152 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/152 2024-08-18T23:32:54.204 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : luarocks-3.9.2-5.el9.noarch 35/152 2024-08-18T23:32:54.237 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/152 2024-08-18T23:32:54.266 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-18T23:32:54.269 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-08-18T23:32:54.279 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/152 2024-08-18T23:32:54.343 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-18T23:32:54.361 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-18T23:32:54.393 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-18T23:32:54.410 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 7/152 2024-08-18T23:32:54.415 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/152 2024-08-18T23:32:54.438 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-18T23:32:54.455 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-18T23:32:54.464 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/152 2024-08-18T23:32:54.609 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-18T23:32:54.643 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-08-18T23:32:54.702 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-08-18T23:32:54.730 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-18T23:32:54.834 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-18T23:32:54.864 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 13/152 2024-08-18T23:32:54.948 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 13/152 2024-08-18T23:32:54.949 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-18T23:32:54.982 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-18T23:32:54.993 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/152 2024-08-18T23:32:55.005 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-18T23:32:55.052 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/152 2024-08-18T23:32:55.085 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/152 2024-08-18T23:32:55.113 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 41/152 2024-08-18T23:32:55.157 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/152 2024-08-18T23:32:55.158 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 5/152 2024-08-18T23:32:55.194 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 43/152 2024-08-18T23:32:55.195 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-18T23:32:55.227 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/152 2024-08-18T23:32:55.275 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-18T23:32:55.283 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 45/152 2024-08-18T23:32:55.321 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 7/152 2024-08-18T23:32:55.322 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-18T23:32:55.350 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-18T23:32:55.356 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-18T23:32:55.357 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-18T23:32:55.375 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-18T23:32:55.395 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 47/152 2024-08-18T23:32:55.431 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-18T23:32:55.469 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-08-18T23:32:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-18T23:32:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-18T23:32:55.560 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-18T23:32:55.566 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-18T23:32:55.589 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-08-18T23:32:55.604 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-08-18T23:32:55.608 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-18T23:32:55.615 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-08-18T23:32:55.664 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-08-18T23:32:55.689 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-08-18T23:32:55.692 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-18T23:32:55.734 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-08-18T23:32:55.748 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-18T23:32:55.774 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-packaging-20.9-5.el9.noarch 22/152 2024-08-18T23:32:55.785 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-18T23:32:55.813 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-18T23:32:55.817 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/152 2024-08-18T23:32:55.826 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-18T23:32:55.842 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 13/152 2024-08-18T23:32:55.849 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/152 2024-08-18T23:32:55.886 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/152 2024-08-18T23:32:55.908 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 13/152 2024-08-18T23:32:55.929 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-18T23:32:55.955 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/152 2024-08-18T23:32:55.960 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-18T23:32:55.968 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 57/152 2024-08-18T23:32:55.982 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-18T23:32:56.009 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 27/152 2024-08-18T23:32:56.009 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-18T23:32:56.039 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/152 2024-08-18T23:32:56.041 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-18T23:32:56.066 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-18T23:32:56.082 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/152 2024-08-18T23:32:56.157 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/152 2024-08-18T23:32:56.161 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-18T23:32:56.192 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 31/152 2024-08-18T23:32:56.238 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 32/152 2024-08-18T23:32:56.264 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 33/152 2024-08-18T23:32:56.287 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-18T23:32:56.324 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-18T23:32:56.330 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-18T23:32:56.334 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/152 2024-08-18T23:32:56.360 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-18T23:32:56.370 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-18T23:32:56.379 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : luarocks-3.9.2-5.el9.noarch 35/152 2024-08-18T23:32:56.395 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-18T23:32:56.443 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-18T23:32:56.453 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/152 2024-08-18T23:32:56.490 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-08-18T23:32:56.585 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-18T23:32:56.597 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/152 2024-08-18T23:32:56.602 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-18T23:32:56.618 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-08-18T23:32:56.648 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/152 2024-08-18T23:32:56.654 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-08-18T23:32:56.659 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-18T23:32:56.688 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-18T23:32:56.723 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-18T23:32:56.728 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-08-18T23:32:56.756 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-18T23:32:56.789 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-08-18T23:32:56.798 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-18T23:32:56.828 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-packaging-20.9-5.el9.noarch 22/152 2024-08-18T23:32:56.832 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-18T23:32:56.871 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/152 2024-08-18T23:32:56.904 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/152 2024-08-18T23:32:56.921 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-18T23:32:56.931 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/152 2024-08-18T23:32:56.972 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-18T23:32:57.003 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/152 2024-08-18T23:32:57.019 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-18T23:32:57.080 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 27/152 2024-08-18T23:32:57.110 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/152 2024-08-18T23:32:57.125 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-18T23:32:57.154 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/152 2024-08-18T23:32:57.168 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/152 2024-08-18T23:32:57.179 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-18T23:32:57.240 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/152 2024-08-18T23:32:57.252 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/152 2024-08-18T23:32:57.272 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 31/152 2024-08-18T23:32:57.281 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 41/152 2024-08-18T23:32:57.319 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 32/152 2024-08-18T23:32:57.324 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/152 2024-08-18T23:32:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 33/152 2024-08-18T23:32:57.362 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 43/152 2024-08-18T23:32:57.394 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/152 2024-08-18T23:32:57.416 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/152 2024-08-18T23:32:57.449 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 45/152 2024-08-18T23:32:57.459 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : luarocks-3.9.2-5.el9.noarch 35/152 2024-08-18T23:32:57.523 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-18T23:32:57.535 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/152 2024-08-18T23:32:57.554 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 47/152 2024-08-18T23:32:57.590 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-18T23:32:57.628 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-18T23:32:57.664 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-18T23:32:57.670 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/152 2024-08-18T23:32:57.707 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-18T23:32:57.721 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/152 2024-08-18T23:32:57.749 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-18T23:32:57.885 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-18T23:32:57.911 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-18T23:32:57.926 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-18T23:32:57.967 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-18T23:32:57.968 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-18T23:32:58.010 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-18T23:32:58.069 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-18T23:32:58.112 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-18T23:32:58.117 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 57/152 2024-08-18T23:32:58.158 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-18T23:32:58.166 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-devel-3.9.19-4.el9.x86_64 83/152 2024-08-18T23:32:58.191 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-18T23:32:58.211 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-18T23:32:58.216 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-18T23:32:58.255 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/152 2024-08-18T23:32:58.300 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-18T23:32:58.311 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-18T23:32:58.333 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/152 2024-08-18T23:32:58.346 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-18T23:32:58.360 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 41/152 2024-08-18T23:32:58.413 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/152 2024-08-18T23:32:58.434 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-18T23:32:58.450 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 43/152 2024-08-18T23:32:58.473 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-18T23:32:58.483 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/152 2024-08-18T23:32:58.502 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-18T23:32:58.522 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-08-18T23:32:58.540 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 45/152 2024-08-18T23:32:58.540 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-18T23:32:58.560 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-08-18T23:32:58.588 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-18T23:32:58.613 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-18T23:32:58.625 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-18T23:32:58.639 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-18T23:32:58.651 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 47/152 2024-08-18T23:32:58.657 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-18T23:32:58.686 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-18T23:32:58.687 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-18T23:32:58.721 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-18T23:32:58.734 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-18T23:32:58.754 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-18T23:32:58.770 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-18T23:32:58.796 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-18T23:32:58.813 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-18T23:32:58.830 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-18T23:32:58.855 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-18T23:32:58.918 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-18T23:32:58.961 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-18T23:32:58.992 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-18T23:32:58.995 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-18T23:32:59.049 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-18T23:32:59.090 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-18T23:32:59.096 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-18T23:32:59.195 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-18T23:32:59.240 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 57/152 2024-08-18T23:32:59.281 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-18T23:32:59.311 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-18T23:32:59.311 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-18T23:32:59.314 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-18T23:32:59.338 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-18T23:32:59.427 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-18T23:32:59.427 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-18T23:32:59.559 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-18T23:32:59.597 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-18T23:32:59.640 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-18T23:32:59.676 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-18T23:32:59.721 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-18T23:32:59.757 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-18T23:32:59.981 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-18T23:33:00.037 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-18T23:33:00.060 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-18T23:33:00.090 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-18T23:33:00.095 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-18T23:33:00.123 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-18T23:33:00.128 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-18T23:33:00.170 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-18T23:33:00.204 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-18T23:33:00.224 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-18T23:33:00.279 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-devel-3.9.19-4.el9.x86_64 83/152 2024-08-18T23:33:00.294 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-18T23:33:00.323 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-18T23:33:00.343 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-18T23:33:00.373 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-18T23:33:00.412 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-18T23:33:00.459 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-18T23:33:00.479 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-18T23:33:00.542 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-18T23:33:00.635 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-08-18T23:33:00.666 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-08-18T23:33:00.700 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-18T23:33:00.744 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-18T23:33:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-18T23:33:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-18T23:33:00.851 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-08-18T23:33:01.166 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-18T23:33:01.206 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 97/152 2024-08-18T23:33:01.283 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-18T23:33:01.302 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 97/152 2024-08-18T23:33:01.328 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 98/152 2024-08-18T23:33:01.329 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-18T23:33:01.363 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-18T23:33:01.364 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-18T23:33:01.412 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-18T23:33:01.464 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-18T23:33:01.519 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-devel-3.9.19-4.el9.x86_64 83/152 2024-08-18T23:33:01.523 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-18T23:33:01.564 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-18T23:33:01.653 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-18T23:33:01.692 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-18T23:33:01.838 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-18T23:33:01.867 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 101/152 2024-08-18T23:33:01.868 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-08-18T23:33:01.898 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-08-18T23:33:01.941 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 101/152 2024-08-18T23:33:01.969 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 102/152 2024-08-18T23:33:01.976 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-18T23:33:02.647 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-18T23:33:02.765 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-18T23:33:02.809 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-18T23:33:02.887 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-18T23:33:02.917 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-18T23:33:02.948 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-08-18T23:33:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-18T23:33:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 97/152 2024-08-18T23:33:03.396 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 97/152 2024-08-18T23:33:03.423 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 98/152 2024-08-18T23:33:03.460 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-18T23:33:03.933 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-18T23:33:03.963 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 101/152 2024-08-18T23:33:04.035 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 101/152 2024-08-18T23:33:04.059 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-18T23:33:04.065 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 102/152 2024-08-18T23:33:04.144 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-18T23:33:04.176 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-18T23:33:04.205 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-08-18T23:33:04.288 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-18T23:33:04.324 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-18T23:33:04.513 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-18T23:33:04.551 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 97/152 2024-08-18T23:33:04.645 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 97/152 2024-08-18T23:33:04.673 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 98/152 2024-08-18T23:33:04.709 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-18T23:33:04.918 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-18T23:33:04.953 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 104/152 2024-08-18T23:33:05.030 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 104/152 2024-08-18T23:33:05.031 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-18T23:33:05.031 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-18T23:33:05.031 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:33:05.071 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 105/152 2024-08-18T23:33:05.182 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-18T23:33:05.212 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 101/152 2024-08-18T23:33:05.285 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 101/152 2024-08-18T23:33:05.314 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 102/152 2024-08-18T23:33:06.393 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-18T23:33:06.436 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-18T23:33:06.995 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-18T23:33:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 104/152 2024-08-18T23:33:07.112 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 104/152 2024-08-18T23:33:07.112 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-18T23:33:07.112 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-18T23:33:07.112 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-18T23:33:07.158 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 105/152 2024-08-18T23:33:07.669 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-18T23:33:07.719 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-18T23:33:08.283 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 103/152 2024-08-18T23:33:08.322 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 104/152 2024-08-18T23:33:08.413 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 104/152 2024-08-18T23:33:08.414 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-18T23:33:08.414 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-18T23:33:08.414 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:33:08.449 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 105/152 2024-08-18T23:33:16.586 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 105/152 2024-08-18T23:33:16.587 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /sys 2024-08-18T23:33:16.587 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /proc 2024-08-18T23:33:16.587 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /mnt 2024-08-18T23:33:16.587 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /var/tmp 2024-08-18T23:33:16.587 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /home 2024-08-18T23:33:16.587 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /root 2024-08-18T23:33:16.587 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /tmp 2024-08-18T23:33:16.587 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:33:16.760 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 106/152 2024-08-18T23:33:16.777 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 106/152 2024-08-18T23:33:16.777 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:16.777 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-18T23:33:16.777 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:33:18.255 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 107/152 2024-08-18T23:33:18.283 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 107/152 2024-08-18T23:33:18.283 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:18.283 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-18T23:33:18.283 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-18T23:33:18.283 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-18T23:33:18.283 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:33:18.506 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 108/152 2024-08-18T23:33:18.530 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 108/152 2024-08-18T23:33:18.530 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:18.530 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-18T23:33:18.530 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-18T23:33:18.530 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-18T23:33:18.530 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:33:18.712 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 105/152 2024-08-18T23:33:18.713 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-08-18T23:33:18.713 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-08-18T23:33:18.713 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-08-18T23:33:18.713 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-08-18T23:33:18.713 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-08-18T23:33:18.713 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /root 2024-08-18T23:33:18.713 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-08-18T23:33:18.713 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-18T23:33:18.893 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 106/152 2024-08-18T23:33:18.910 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 106/152 2024-08-18T23:33:18.910 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:18.911 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-18T23:33:18.911 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-18T23:33:18.973 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 109/152 2024-08-18T23:33:18.997 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 109/152 2024-08-18T23:33:18.997 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:18.997 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-18T23:33:18.997 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-18T23:33:18.997 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-18T23:33:18.997 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:33:19.104 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-18T23:33:19.152 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-18T23:33:19.208 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-18T23:33:19.395 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-18T23:33:19.449 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-18T23:33:19.510 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-18T23:33:19.601 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-18T23:33:19.749 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-18T23:33:19.782 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 105/152 2024-08-18T23:33:19.782 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /sys 2024-08-18T23:33:19.782 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /proc 2024-08-18T23:33:19.782 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /mnt 2024-08-18T23:33:19.782 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /var/tmp 2024-08-18T23:33:19.782 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /home 2024-08-18T23:33:19.782 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /root 2024-08-18T23:33:19.782 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /tmp 2024-08-18T23:33:19.782 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:33:19.792 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-18T23:33:19.823 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-18T23:33:19.843 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-18T23:33:19.961 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 106/152 2024-08-18T23:33:19.979 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 106/152 2024-08-18T23:33:19.979 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:19.979 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-18T23:33:19.979 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:33:20.339 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 121/152 2024-08-18T23:33:20.363 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 107/152 2024-08-18T23:33:20.366 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 121/152 2024-08-18T23:33:20.390 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 107/152 2024-08-18T23:33:20.390 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:20.390 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-18T23:33:20.390 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-18T23:33:20.390 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-18T23:33:20.390 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-18T23:33:20.398 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 122/152 2024-08-18T23:33:20.461 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 123/152 2024-08-18T23:33:20.498 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 124/152 2024-08-18T23:33:20.622 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 108/152 2024-08-18T23:33:20.646 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 108/152 2024-08-18T23:33:20.646 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:20.646 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-18T23:33:20.646 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-18T23:33:20.646 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-18T23:33:20.646 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-18T23:33:21.081 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 109/152 2024-08-18T23:33:21.104 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 109/152 2024-08-18T23:33:21.105 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:21.105 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-18T23:33:21.105 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-18T23:33:21.105 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-18T23:33:21.105 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-18T23:33:21.211 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-18T23:33:21.267 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-18T23:33:21.323 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-18T23:33:21.447 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 107/152 2024-08-18T23:33:21.474 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 107/152 2024-08-18T23:33:21.474 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:21.475 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-18T23:33:21.475 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-18T23:33:21.475 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-18T23:33:21.475 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:33:21.504 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-18T23:33:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 124/152 2024-08-18T23:33:21.565 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-18T23:33:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 125/152 2024-08-18T23:33:21.633 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-18T23:33:21.706 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 108/152 2024-08-18T23:33:21.721 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-18T23:33:21.730 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 108/152 2024-08-18T23:33:21.730 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:21.730 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-18T23:33:21.730 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-18T23:33:21.731 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-18T23:33:21.731 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:33:21.860 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-18T23:33:21.907 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-18T23:33:21.939 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-18T23:33:21.967 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-18T23:33:22.174 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 109/152 2024-08-18T23:33:22.197 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 109/152 2024-08-18T23:33:22.198 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:22.198 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-18T23:33:22.198 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-18T23:33:22.198 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-18T23:33:22.198 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:33:22.312 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-18T23:33:22.361 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-18T23:33:22.412 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 121/152 2024-08-18T23:33:22.416 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-18T23:33:22.440 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 121/152 2024-08-18T23:33:22.471 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 122/152 2024-08-18T23:33:22.534 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 123/152 2024-08-18T23:33:22.543 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 125/152 2024-08-18T23:33:22.571 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 124/152 2024-08-18T23:33:22.575 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 126/152 2024-08-18T23:33:22.601 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-18T23:33:22.658 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-18T23:33:22.677 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 126/152 2024-08-18T23:33:22.746 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-18T23:33:22.832 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-18T23:33:22.845 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 127/152 2024-08-18T23:33:22.877 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 128/152 2024-08-18T23:33:22.903 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 128/152 2024-08-18T23:33:22.903 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:22.903 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-18T23:33:22.903 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-18T23:33:22.903 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-18T23:33:22.903 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:33:22.953 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 129/152 2024-08-18T23:33:22.966 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 129/152 2024-08-18T23:33:22.971 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-18T23:33:23.021 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-18T23:33:23.057 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-18T23:33:23.063 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 130/152 2024-08-18T23:33:23.084 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-18T23:33:23.120 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 131/152 2024-08-18T23:33:23.529 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 121/152 2024-08-18T23:33:23.567 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 121/152 2024-08-18T23:33:23.589 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 122/152 2024-08-18T23:33:23.609 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 124/152 2024-08-18T23:33:23.652 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 125/152 2024-08-18T23:33:23.652 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 123/152 2024-08-18T23:33:23.689 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 124/152 2024-08-18T23:33:24.036 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-18T23:33:24.064 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 133/152 2024-08-18T23:33:24.088 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 133/152 2024-08-18T23:33:24.088 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:24.089 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-18T23:33:24.089 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-18T23:33:24.089 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-18T23:33:24.089 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:33:24.131 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 134/152 2024-08-18T23:33:24.155 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1382.gde09d 134/152 2024-08-18T23:33:24.155 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:24.155 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-18T23:33:24.155 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:33:24.207 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 135/152 2024-08-18T23:33:24.232 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 135/152 2024-08-18T23:33:24.232 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:24.232 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-18T23:33:24.232 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-18T23:33:24.232 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-18T23:33:24.232 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:33:24.533 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 136/152 2024-08-18T23:33:24.557 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 136/152 2024-08-18T23:33:24.557 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:24.557 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-18T23:33:24.557 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-18T23:33:24.558 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-18T23:33:24.558 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:33:24.608 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 125/152 2024-08-18T23:33:24.640 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 126/152 2024-08-18T23:33:24.736 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 124/152 2024-08-18T23:33:24.743 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 126/152 2024-08-18T23:33:24.766 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 125/152 2024-08-18T23:33:24.910 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 127/152 2024-08-18T23:33:24.959 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 128/152 2024-08-18T23:33:24.983 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 128/152 2024-08-18T23:33:24.983 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:24.983 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-18T23:33:24.983 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-18T23:33:24.983 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-18T23:33:24.983 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-18T23:33:25.034 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 129/152 2024-08-18T23:33:25.047 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 129/152 2024-08-18T23:33:25.143 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 130/152 2024-08-18T23:33:25.195 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 131/152 2024-08-18T23:33:25.724 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 125/152 2024-08-18T23:33:25.750 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 126/152 2024-08-18T23:33:25.852 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 126/152 2024-08-18T23:33:26.019 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 127/152 2024-08-18T23:33:26.044 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 128/152 2024-08-18T23:33:26.068 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 128/152 2024-08-18T23:33:26.068 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:26.069 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-18T23:33:26.069 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-18T23:33:26.069 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-18T23:33:26.069 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:33:26.102 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-18T23:33:26.119 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 129/152 2024-08-18T23:33:26.128 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 133/152 2024-08-18T23:33:26.132 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 129/152 2024-08-18T23:33:26.153 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 133/152 2024-08-18T23:33:26.153 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:26.153 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-18T23:33:26.153 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-18T23:33:26.153 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-18T23:33:26.153 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-18T23:33:26.204 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 134/152 2024-08-18T23:33:26.227 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1382.gde09d 134/152 2024-08-18T23:33:26.227 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:26.227 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-18T23:33:26.227 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-18T23:33:26.229 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 130/152 2024-08-18T23:33:26.271 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 135/152 2024-08-18T23:33:26.282 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 131/152 2024-08-18T23:33:26.294 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 135/152 2024-08-18T23:33:26.295 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:26.295 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-18T23:33:26.295 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-18T23:33:26.295 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-18T23:33:26.295 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-18T23:33:26.605 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 136/152 2024-08-18T23:33:26.629 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 136/152 2024-08-18T23:33:26.629 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:26.629 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-18T23:33:26.630 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-18T23:33:26.630 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-18T23:33:26.630 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-18T23:33:27.197 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-18T23:33:27.230 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 133/152 2024-08-18T23:33:27.254 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 133/152 2024-08-18T23:33:27.255 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:27.255 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-18T23:33:27.255 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-18T23:33:27.255 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-18T23:33:27.255 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:33:27.297 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 134/152 2024-08-18T23:33:27.320 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1382.gde09d 134/152 2024-08-18T23:33:27.320 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:27.320 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-18T23:33:27.320 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:33:27.373 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 135/152 2024-08-18T23:33:27.397 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 135/152 2024-08-18T23:33:27.397 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:27.397 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-18T23:33:27.397 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-18T23:33:27.397 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-18T23:33:27.397 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:33:27.699 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 136/152 2024-08-18T23:33:27.722 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 136/152 2024-08-18T23:33:27.723 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-18T23:33:27.723 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-18T23:33:27.723 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-18T23:33:27.723 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-18T23:33:27.723 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:33:29.117 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 137/152 2024-08-18T23:33:29.161 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 138/152 2024-08-18T23:33:29.206 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 139/152 2024-08-18T23:33:29.234 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 140/152 2024-08-18T23:33:29.270 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-18T23:33:29.458 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 142/152 2024-08-18T23:33:29.508 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-08-18T23:33:29.538 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 144/152 2024-08-18T23:33:30.197 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-18T23:33:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-18T23:33:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-18T23:33:31.114 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 146/152 2024-08-18T23:33:31.191 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 147/152 2024-08-18T23:33:31.192 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 137/152 2024-08-18T23:33:31.233 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 138/152 2024-08-18T23:33:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-18T23:33:31.277 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 139/152 2024-08-18T23:33:31.306 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 140/152 2024-08-18T23:33:31.341 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-18T23:33:31.346 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-18T23:33:31.379 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-18T23:33:31.379 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-18T23:33:31.471 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-18T23:33:31.471 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-18T23:33:31.533 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 142/152 2024-08-18T23:33:31.580 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-08-18T23:33:31.610 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 144/152 2024-08-18T23:33:32.179 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-18T23:33:32.223 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-18T23:33:32.244 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-18T23:33:32.277 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 137/152 2024-08-18T23:33:32.318 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 138/152 2024-08-18T23:33:32.368 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 139/152 2024-08-18T23:33:32.408 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 140/152 2024-08-18T23:33:32.435 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-18T23:33:32.624 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 142/152 2024-08-18T23:33:32.665 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-08-18T23:33:32.695 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 144/152 2024-08-18T23:33:33.071 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 146/152 2024-08-18T23:33:33.161 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 147/152 2024-08-18T23:33:33.211 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-18T23:33:33.298 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-18T23:33:33.332 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-18T23:33:33.332 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-18T23:33:33.348 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-18T23:33:33.384 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-18T23:33:33.406 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-08-18T23:33:33.424 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-18T23:33:33.424 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-18T23:33:34.252 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 146/152 2024-08-18T23:33:34.347 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 147/152 2024-08-18T23:33:34.401 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-18T23:33:34.513 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-18T23:33:34.551 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-18T23:33:34.552 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-18T23:33:34.652 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-18T23:33:34.652 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-18T23:33:37.384 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-18T23:33:37.384 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 1/152 2024-08-18T23:33:37.384 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2/152 2024-08-18T23:33:37.384 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 3/152 2024-08-18T23:33:37.385 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 4/152 2024-08-18T23:33:37.385 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 5/152 2024-08-18T23:33:37.385 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-18T23:33:37.385 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 7/152 2024-08-18T23:33:37.385 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 8/152 2024-08-18T23:33:37.385 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 9/152 2024-08-18T23:33:37.385 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 10/152 2024-08-18T23:33:37.385 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 11/152 2024-08-18T23:33:37.385 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-18T23:33:37.385 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 13/152 2024-08-18T23:33:37.385 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 14/152 2024-08-18T23:33:37.385 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 15/152 2024-08-18T23:33:37.386 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 16/152 2024-08-18T23:33:37.386 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 17/152 2024-08-18T23:33:37.386 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 18/152 2024-08-18T23:33:37.386 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 19/152 2024-08-18T23:33:37.386 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 20/152 2024-08-18T23:33:37.386 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 21/152 2024-08-18T23:33:37.386 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 22/152 2024-08-18T23:33:37.386 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 23/152 2024-08-18T23:33:37.386 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 24/152 2024-08-18T23:33:37.386 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 25/152 2024-08-18T23:33:37.386 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 26/152 2024-08-18T23:33:37.386 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 27/152 2024-08-18T23:33:37.397 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 28/152 2024-08-18T23:33:37.397 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 29/152 2024-08-18T23:33:37.398 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 30/152 2024-08-18T23:33:37.398 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 31/152 2024-08-18T23:33:37.398 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 32/152 2024-08-18T23:33:37.398 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 33/152 2024-08-18T23:33:37.398 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 34/152 2024-08-18T23:33:37.398 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 35/152 2024-08-18T23:33:37.398 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 36/152 2024-08-18T23:33:37.398 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 37/152 2024-08-18T23:33:37.398 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 38/152 2024-08-18T23:33:37.398 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 39/152 2024-08-18T23:33:37.398 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-18T23:33:37.399 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-08-18T23:33:37.399 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-08-18T23:33:37.399 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-18T23:33:37.399 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-18T23:33:37.399 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-18T23:33:37.399 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-18T23:33:37.399 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-18T23:33:37.399 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-18T23:33:37.399 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-18T23:33:37.399 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-18T23:33:37.399 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-18T23:33:37.400 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-18T23:33:37.400 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-18T23:33:37.400 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-18T23:33:37.400 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-18T23:33:37.400 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-18T23:33:37.400 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-18T23:33:37.400 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-18T23:33:37.400 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-18T23:33:37.400 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-18T23:33:37.400 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-18T23:33:37.400 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-08-18T23:33:37.400 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-18T23:33:37.401 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-18T23:33:37.401 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-18T23:33:37.401 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-08-18T23:33:37.401 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-08-18T23:33:37.401 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/152 2024-08-18T23:33:37.401 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-08-18T23:33:37.401 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-08-18T23:33:37.401 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-08-18T23:33:37.401 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-08-18T23:33:37.401 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-18T23:33:37.401 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-18T23:33:37.401 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-18T23:33:37.402 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-18T23:33:37.402 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-devel-3.9.19-4.el9.x86_64 77/152 2024-08-18T23:33:37.402 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-18T23:33:37.402 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-18T23:33:37.402 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-18T23:33:37.402 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-18T23:33:37.402 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-18T23:33:37.402 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-18T23:33:37.402 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-18T23:33:37.402 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-18T23:33:37.402 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-18T23:33:37.402 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-18T23:33:37.403 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-18T23:33:37.403 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-18T23:33:37.403 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-18T23:33:37.403 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-18T23:33:37.403 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-18T23:33:37.403 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-18T23:33:37.403 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-18T23:33:37.403 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-18T23:33:37.403 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-18T23:33:37.403 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-18T23:33:37.403 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-18T23:33:37.403 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-18T23:33:37.403 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-18T23:33:37.404 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/152 2024-08-18T23:33:37.404 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-18T23:33:37.404 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-18T23:33:37.404 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-18T23:33:37.404 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-18T23:33:37.404 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-18T23:33:37.404 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-18T23:33:37.404 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-18T23:33:37.404 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-18T23:33:37.404 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-18T23:33:37.404 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-18T23:33:37.404 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-18T23:33:37.405 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-18T23:33:37.405 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-18T23:33:37.405 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-18T23:33:37.405 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-18T23:33:37.405 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-18T23:33:37.405 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-18T23:33:37.405 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-18T23:33:37.405 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-18T23:33:37.405 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-18T23:33:37.405 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/152 2024-08-18T23:33:37.405 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-18T23:33:37.405 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-18T23:33:37.405 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-18T23:33:37.406 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-18T23:33:37.406 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-18T23:33:37.406 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-18T23:33:37.406 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-18T23:33:37.406 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-18T23:33:37.406 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-18T23:33:37.406 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-18T23:33:37.406 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-18T23:33:37.406 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-18T23:33:37.406 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-18T23:33:37.406 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-18T23:33:37.407 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-18T23:33:37.407 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-18T23:33:37.407 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-18T23:33:37.407 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-18T23:33:37.407 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-18T23:33:37.407 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-18T23:33:37.407 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-08-18T23:33:37.407 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-18T23:33:37.407 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-18T23:33:37.407 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-18T23:33:37.407 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-08-18T23:33:37.407 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-18T23:33:37.408 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 149/152 2024-08-18T23:33:37.408 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-18T23:33:37.408 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 151/152 2024-08-18T23:33:39.153 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-18T23:33:39.153 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 1/152 2024-08-18T23:33:39.153 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2/152 2024-08-18T23:33:39.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 3/152 2024-08-18T23:33:39.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 4/152 2024-08-18T23:33:39.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 5/152 2024-08-18T23:33:39.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-18T23:33:39.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 7/152 2024-08-18T23:33:39.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 8/152 2024-08-18T23:33:39.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 9/152 2024-08-18T23:33:39.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 10/152 2024-08-18T23:33:39.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 11/152 2024-08-18T23:33:39.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-18T23:33:39.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 13/152 2024-08-18T23:33:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 14/152 2024-08-18T23:33:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 15/152 2024-08-18T23:33:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 16/152 2024-08-18T23:33:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 17/152 2024-08-18T23:33:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 18/152 2024-08-18T23:33:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 19/152 2024-08-18T23:33:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 20/152 2024-08-18T23:33:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 21/152 2024-08-18T23:33:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 22/152 2024-08-18T23:33:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 23/152 2024-08-18T23:33:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 24/152 2024-08-18T23:33:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 25/152 2024-08-18T23:33:39.156 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 26/152 2024-08-18T23:33:39.156 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 27/152 2024-08-18T23:33:39.166 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 28/152 2024-08-18T23:33:39.166 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 29/152 2024-08-18T23:33:39.166 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 30/152 2024-08-18T23:33:39.166 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 31/152 2024-08-18T23:33:39.166 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 32/152 2024-08-18T23:33:39.166 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 33/152 2024-08-18T23:33:39.166 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 34/152 2024-08-18T23:33:39.166 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 35/152 2024-08-18T23:33:39.166 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 36/152 2024-08-18T23:33:39.167 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 37/152 2024-08-18T23:33:39.167 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 38/152 2024-08-18T23:33:39.167 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 39/152 2024-08-18T23:33:39.167 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-18T23:33:39.167 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-08-18T23:33:39.167 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-08-18T23:33:39.167 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-18T23:33:39.167 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-18T23:33:39.167 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-18T23:33:39.167 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-18T23:33:39.167 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-18T23:33:39.167 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-18T23:33:39.167 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-18T23:33:39.168 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-18T23:33:39.168 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-18T23:33:39.168 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-18T23:33:39.168 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-18T23:33:39.168 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-18T23:33:39.168 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-18T23:33:39.168 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-18T23:33:39.168 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-18T23:33:39.168 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-18T23:33:39.168 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-18T23:33:39.168 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-18T23:33:39.168 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-18T23:33:39.168 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-08-18T23:33:39.168 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-18T23:33:39.169 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-18T23:33:39.169 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-18T23:33:39.169 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-08-18T23:33:39.169 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-08-18T23:33:39.169 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/152 2024-08-18T23:33:39.169 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-08-18T23:33:39.169 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-08-18T23:33:39.169 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-08-18T23:33:39.169 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-08-18T23:33:39.169 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-18T23:33:39.169 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-18T23:33:39.169 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-18T23:33:39.169 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-18T23:33:39.170 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-devel-3.9.19-4.el9.x86_64 77/152 2024-08-18T23:33:39.170 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-18T23:33:39.170 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-18T23:33:39.170 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-18T23:33:39.170 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-18T23:33:39.170 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-18T23:33:39.170 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-18T23:33:39.170 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-18T23:33:39.170 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-18T23:33:39.170 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-18T23:33:39.170 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-18T23:33:39.170 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-18T23:33:39.170 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-18T23:33:39.170 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-18T23:33:39.171 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-18T23:33:39.171 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-18T23:33:39.171 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-18T23:33:39.171 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-18T23:33:39.171 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-18T23:33:39.171 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-18T23:33:39.171 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-18T23:33:39.171 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-18T23:33:39.171 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-18T23:33:39.171 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-18T23:33:39.171 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/152 2024-08-18T23:33:39.172 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-18T23:33:39.172 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-18T23:33:39.172 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-18T23:33:39.172 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-18T23:33:39.172 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-18T23:33:39.172 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-18T23:33:39.172 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-18T23:33:39.172 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-18T23:33:39.172 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-18T23:33:39.172 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-18T23:33:39.172 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-18T23:33:39.172 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-18T23:33:39.172 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-18T23:33:39.172 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-18T23:33:39.173 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-18T23:33:39.173 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-18T23:33:39.173 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-18T23:33:39.173 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-18T23:33:39.173 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-18T23:33:39.173 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-18T23:33:39.173 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/152 2024-08-18T23:33:39.173 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-18T23:33:39.173 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-18T23:33:39.173 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-18T23:33:39.173 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-18T23:33:39.173 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-18T23:33:39.173 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-18T23:33:39.173 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-18T23:33:39.174 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-18T23:33:39.174 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-18T23:33:39.174 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-18T23:33:39.174 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-18T23:33:39.174 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-18T23:33:39.174 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-18T23:33:39.174 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-18T23:33:39.174 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-18T23:33:39.174 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-18T23:33:39.174 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-18T23:33:39.174 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-18T23:33:39.174 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-18T23:33:39.175 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-18T23:33:39.175 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-08-18T23:33:39.175 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-18T23:33:39.175 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-18T23:33:39.175 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-18T23:33:39.175 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-08-18T23:33:39.175 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-18T23:33:39.175 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 149/152 2024-08-18T23:33:39.175 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-18T23:33:39.175 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 151/152 2024-08-18T23:33:39.604 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-18T23:33:39.605 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:33:39.605 INFO:teuthology.orchestra.run.smithi078.stdout:Upgraded: 2024-08-18T23:33:39.605 INFO:teuthology.orchestra.run.smithi078.stdout: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.605 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.605 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-08-18T23:33:39.605 INFO:teuthology.orchestra.run.smithi078.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-18T23:33:39.605 INFO:teuthology.orchestra.run.smithi078.stdout: automake-1.16.2-8.el9.noarch 2024-08-18T23:33:39.605 INFO:teuthology.orchestra.run.smithi078.stdout: bison-3.7.4-5.el9.x86_64 2024-08-18T23:33:39.605 INFO:teuthology.orchestra.run.smithi078.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-18T23:33:39.605 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.605 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.606 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.606 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.606 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:39.606 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.606 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.606 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.606 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:39.606 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:39.606 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:39.606 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:39.606 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:39.606 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.607 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.607 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:39.607 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.607 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.607 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.607 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:39.607 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:39.607 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.607 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.607 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:39.607 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-18T23:33:39.607 INFO:teuthology.orchestra.run.smithi078.stdout: flex-2.6.4-9.el9.x86_64 2024-08-18T23:33:39.607 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-18T23:33:39.608 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-18T23:33:39.608 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-18T23:33:39.608 INFO:teuthology.orchestra.run.smithi078.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-18T23:33:39.608 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-18T23:33:39.608 INFO:teuthology.orchestra.run.smithi078.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-18T23:33:39.608 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-18T23:33:39.608 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-18T23:33:39.608 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-18T23:33:39.608 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.608 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.608 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.608 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-18T23:33:39.608 INFO:teuthology.orchestra.run.smithi078.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-18T23:33:39.609 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-18T23:33:39.609 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-18T23:33:39.609 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-18T23:33:39.609 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.609 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.609 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-18T23:33:39.609 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.609 INFO:teuthology.orchestra.run.smithi078.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-18T23:33:39.609 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-18T23:33:39.609 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-18T23:33:39.609 INFO:teuthology.orchestra.run.smithi078.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-18T23:33:39.609 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-18T23:33:39.610 INFO:teuthology.orchestra.run.smithi078.stdout: lua-5.4.4-4.el9.x86_64 2024-08-18T23:33:39.610 INFO:teuthology.orchestra.run.smithi078.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-18T23:33:39.610 INFO:teuthology.orchestra.run.smithi078.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-18T23:33:39.610 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-18T23:33:39.610 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-18T23:33:39.610 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-18T23:33:39.610 INFO:teuthology.orchestra.run.smithi078.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-18T23:33:39.610 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-13.14-1.el9.x86_64 2024-08-18T23:33:39.610 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-08-18T23:33:39.610 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-08-18T23:33:39.610 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server-13.14-1.el9.x86_64 2024-08-18T23:33:39.611 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-18T23:33:39.611 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-18T23:33:39.611 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-18T23:33:39.611 INFO:teuthology.orchestra.run.smithi078.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-18T23:33:39.611 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-18T23:33:39.611 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-18T23:33:39.611 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-18T23:33:39.611 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.611 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.611 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.611 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-18T23:33:39.611 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-18T23:33:39.611 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-18T23:33:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-18T23:33:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-18T23:33:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-18T23:33:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-18T23:33:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel-3.9.19-4.el9.x86_64 2024-08-18T23:33:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-18T23:33:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-18T23:33:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-18T23:33:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-18T23:33:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-18T23:33:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-18T23:33:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-18T23:33:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-18T23:33:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-18T23:33:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-18T23:33:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-18T23:33:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-18T23:33:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-18T23:33:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-18T23:33:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-18T23:33:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-18T23:33:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-18T23:33:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-18T23:33:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-18T23:33:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-18T23:33:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-18T23:33:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-18T23:33:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply-3.11-14.el9.noarch 2024-08-18T23:33:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-18T23:33:39.614 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-18T23:33:39.614 INFO:teuthology.orchestra.run.smithi078.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-18T23:33:39.614 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-18T23:33:39.614 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-18T23:33:39.614 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-18T23:33:39.614 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-18T23:33:39.614 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-18T23:33:39.614 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-18T23:33:39.614 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.614 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.614 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-18T23:33:39.614 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-18T23:33:39.615 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-18T23:33:39.615 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.615 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-18T23:33:39.615 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-18T23:33:39.615 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-18T23:33:39.615 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-18T23:33:39.615 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-18T23:33:39.615 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-18T23:33:39.615 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-18T23:33:39.615 INFO:teuthology.orchestra.run.smithi078.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-18T23:33:39.615 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-18T23:33:39.615 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-18T23:33:39.615 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-18T23:33:39.615 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-18T23:33:39.616 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-18T23:33:39.616 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-18T23:33:39.616 INFO:teuthology.orchestra.run.smithi078.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-18T23:33:39.616 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.616 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.616 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:39.616 INFO:teuthology.orchestra.run.smithi078.stdout: re2-1:20211101-3.el9.x86_64 2024-08-18T23:33:39.616 INFO:teuthology.orchestra.run.smithi078.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-18T23:33:39.616 INFO:teuthology.orchestra.run.smithi078.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-18T23:33:39.616 INFO:teuthology.orchestra.run.smithi078.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-18T23:33:39.616 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-18T23:33:39.616 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-18T23:33:39.617 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-18T23:33:39.617 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:33:39.617 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-08-18T23:33:39.942 DEBUG:teuthology.parallel:result is None 2024-08-18T23:33:40.374 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-18T23:33:40.374 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 1/152 2024-08-18T23:33:40.375 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2/152 2024-08-18T23:33:40.375 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 3/152 2024-08-18T23:33:40.375 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 4/152 2024-08-18T23:33:40.375 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1382.gde09d 5/152 2024-08-18T23:33:40.375 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 6/152 2024-08-18T23:33:40.375 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 7/152 2024-08-18T23:33:40.375 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 8/152 2024-08-18T23:33:40.375 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 9/152 2024-08-18T23:33:40.375 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 10/152 2024-08-18T23:33:40.375 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 11/152 2024-08-18T23:33:40.375 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 12/152 2024-08-18T23:33:40.376 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 13/152 2024-08-18T23:33:40.376 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 14/152 2024-08-18T23:33:40.376 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_ 15/152 2024-08-18T23:33:40.376 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 16/152 2024-08-18T23:33:40.376 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 17/152 2024-08-18T23:33:40.376 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_6 18/152 2024-08-18T23:33:40.376 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86 19/152 2024-08-18T23:33:40.376 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 20/152 2024-08-18T23:33:40.376 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el 21/152 2024-08-18T23:33:40.376 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-common-2:19.1.0-1382.gde09dd01.el9. 22/152 2024-08-18T23:33:40.376 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_6 23/152 2024-08-18T23:33:40.376 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 24/152 2024-08-18T23:33:40.377 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 25/152 2024-08-18T23:33:40.377 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 26/152 2024-08-18T23:33:40.377 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 27/152 2024-08-18T23:33:40.377 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 28/152 2024-08-18T23:33:40.377 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 29/152 2024-08-18T23:33:40.377 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01. 30/152 2024-08-18T23:33:40.377 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noa 31/152 2024-08-18T23:33:40.377 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.n 32/152 2024-08-18T23:33:40.377 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1382.gde0 33/152 2024-08-18T23:33:40.377 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el 34/152 2024-08-18T23:33:40.377 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 35/152 2024-08-18T23:33:40.377 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.e 36/152 2024-08-18T23:33:40.378 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 37/152 2024-08-18T23:33:40.378 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 38/152 2024-08-18T23:33:40.378 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 39/152 2024-08-18T23:33:40.378 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-18T23:33:40.378 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-08-18T23:33:40.378 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-08-18T23:33:40.378 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-18T23:33:40.378 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-18T23:33:40.378 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-18T23:33:40.378 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-18T23:33:40.379 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-18T23:33:40.379 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-18T23:33:40.379 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-18T23:33:40.379 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-18T23:33:40.379 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-18T23:33:40.379 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-18T23:33:40.379 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-18T23:33:40.379 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-18T23:33:40.379 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-18T23:33:40.379 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-18T23:33:40.379 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-18T23:33:40.379 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-18T23:33:40.380 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-18T23:33:40.380 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-18T23:33:40.380 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-18T23:33:40.380 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-08-18T23:33:40.380 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-18T23:33:40.380 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-18T23:33:40.380 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-18T23:33:40.380 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-08-18T23:33:40.380 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-08-18T23:33:40.380 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/152 2024-08-18T23:33:40.380 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-08-18T23:33:40.380 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-08-18T23:33:40.381 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-08-18T23:33:40.381 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-08-18T23:33:40.381 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-18T23:33:40.381 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-18T23:33:40.381 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-18T23:33:40.381 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-18T23:33:40.381 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-devel-3.9.19-4.el9.x86_64 77/152 2024-08-18T23:33:40.382 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-18T23:33:40.382 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-18T23:33:40.382 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-18T23:33:40.382 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-18T23:33:40.382 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-18T23:33:40.382 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-18T23:33:40.382 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-18T23:33:40.382 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-18T23:33:40.382 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-18T23:33:40.382 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-18T23:33:40.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-18T23:33:40.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-18T23:33:40.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-18T23:33:40.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-18T23:33:40.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-18T23:33:40.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-18T23:33:40.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-18T23:33:40.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-18T23:33:40.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-18T23:33:40.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-18T23:33:40.383 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-18T23:33:40.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-18T23:33:40.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-18T23:33:40.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/152 2024-08-18T23:33:40.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-18T23:33:40.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-18T23:33:40.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-18T23:33:40.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-18T23:33:40.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-18T23:33:40.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-18T23:33:40.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-18T23:33:40.384 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-18T23:33:40.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-18T23:33:40.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-18T23:33:40.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-18T23:33:40.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-18T23:33:40.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-18T23:33:40.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-18T23:33:40.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-18T23:33:40.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-18T23:33:40.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-18T23:33:40.385 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-18T23:33:40.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-18T23:33:40.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-18T23:33:40.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/152 2024-08-18T23:33:40.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-18T23:33:40.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-18T23:33:40.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-18T23:33:40.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-18T23:33:40.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-18T23:33:40.386 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-18T23:33:40.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-18T23:33:40.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-18T23:33:40.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-18T23:33:40.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-18T23:33:40.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-18T23:33:40.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-18T23:33:40.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-18T23:33:40.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-18T23:33:40.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-18T23:33:40.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-18T23:33:40.387 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-18T23:33:40.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-18T23:33:40.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-18T23:33:40.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-18T23:33:40.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-08-18T23:33:40.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-18T23:33:40.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-18T23:33:40.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-18T23:33:40.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-08-18T23:33:40.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-18T23:33:40.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 149/152 2024-08-18T23:33:40.388 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-18T23:33:40.389 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 151/152 2024-08-18T23:33:41.377 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-18T23:33:41.377 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-18T23:33:41.377 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-08-18T23:33:41.378 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.378 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.378 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-08-18T23:33:41.378 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-18T23:33:41.378 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.2-8.el9.noarch 2024-08-18T23:33:41.378 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.7.4-5.el9.x86_64 2024-08-18T23:33:41.378 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-18T23:33:41.378 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.378 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.378 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.379 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.379 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:41.379 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.379 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.379 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.379 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:41.379 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:41.379 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:41.379 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:41.379 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:41.379 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.379 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.379 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:41.380 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.380 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.380 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.380 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:41.380 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:41.380 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.380 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.380 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:41.380 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-18T23:33:41.380 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.4-9.el9.x86_64 2024-08-18T23:33:41.380 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-18T23:33:41.380 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-18T23:33:41.380 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-18T23:33:41.381 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-18T23:33:41.381 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-18T23:33:41.381 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-18T23:33:41.381 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-18T23:33:41.381 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-18T23:33:41.381 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-18T23:33:41.381 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.381 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.381 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.381 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-18T23:33:41.381 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-18T23:33:41.381 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-18T23:33:41.381 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-18T23:33:41.382 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-18T23:33:41.382 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.382 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.382 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-18T23:33:41.382 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.382 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-18T23:33:41.382 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-18T23:33:41.382 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-18T23:33:41.382 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-18T23:33:41.382 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-18T23:33:41.382 INFO:teuthology.orchestra.run.smithi123.stdout: lua-5.4.4-4.el9.x86_64 2024-08-18T23:33:41.382 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-18T23:33:41.382 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-18T23:33:41.383 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-18T23:33:41.383 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-18T23:33:41.383 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-18T23:33:41.383 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-18T23:33:41.383 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-13.14-1.el9.x86_64 2024-08-18T23:33:41.383 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-08-18T23:33:41.383 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-08-18T23:33:41.383 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-13.14-1.el9.x86_64 2024-08-18T23:33:41.383 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-18T23:33:41.383 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-18T23:33:41.383 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-18T23:33:41.383 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-18T23:33:41.383 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-18T23:33:41.384 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-18T23:33:41.384 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-18T23:33:41.384 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.384 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.384 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.384 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-18T23:33:41.384 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-18T23:33:41.384 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-18T23:33:41.384 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-18T23:33:41.384 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-18T23:33:41.384 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-18T23:33:41.385 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-18T23:33:41.385 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel-3.9.19-4.el9.x86_64 2024-08-18T23:33:41.385 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-18T23:33:41.385 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-18T23:33:41.385 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-18T23:33:41.385 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-18T23:33:41.385 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-18T23:33:41.385 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-18T23:33:41.385 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-18T23:33:41.385 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-18T23:33:41.385 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-18T23:33:41.385 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-18T23:33:41.385 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-18T23:33:41.386 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-18T23:33:41.386 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-18T23:33:41.386 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-18T23:33:41.386 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-18T23:33:41.386 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-18T23:33:41.386 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-18T23:33:41.386 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-18T23:33:41.386 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-18T23:33:41.386 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-18T23:33:41.386 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-18T23:33:41.386 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-18T23:33:41.386 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply-3.11-14.el9.noarch 2024-08-18T23:33:41.386 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-18T23:33:41.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-18T23:33:41.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-18T23:33:41.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-18T23:33:41.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-18T23:33:41.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-18T23:33:41.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-18T23:33:41.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-18T23:33:41.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-18T23:33:41.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-18T23:33:41.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-18T23:33:41.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-18T23:33:41.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.388 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-18T23:33:41.388 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-18T23:33:41.388 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-18T23:33:41.388 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-18T23:33:41.388 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-18T23:33:41.388 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-18T23:33:41.388 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-18T23:33:41.388 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-18T23:33:41.388 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-18T23:33:41.388 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-18T23:33:41.388 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-18T23:33:41.388 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-18T23:33:41.388 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-18T23:33:41.388 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-18T23:33:41.389 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-18T23:33:41.389 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.389 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.389 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:41.389 INFO:teuthology.orchestra.run.smithi123.stdout: re2-1:20211101-3.el9.x86_64 2024-08-18T23:33:41.389 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-18T23:33:41.389 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-18T23:33:41.389 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-18T23:33:41.389 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-18T23:33:41.389 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-18T23:33:41.390 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-18T23:33:41.390 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-18T23:33:41.390 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-08-18T23:33:41.775 DEBUG:teuthology.parallel:result is None 2024-08-18T23:33:42.534 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-18T23:33:42.534 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:33:42.534 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2024-08-18T23:33:42.534 INFO:teuthology.orchestra.run.smithi161.stdout: librados2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.535 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.535 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-08-18T23:33:42.535 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-18T23:33:42.535 INFO:teuthology.orchestra.run.smithi161.stdout: automake-1.16.2-8.el9.noarch 2024-08-18T23:33:42.535 INFO:teuthology.orchestra.run.smithi161.stdout: bison-3.7.4-5.el9.x86_64 2024-08-18T23:33:42.535 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-18T23:33:42.535 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.535 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.535 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.535 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.535 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:42.536 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.536 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.536 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.536 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:42.536 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:42.536 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:42.536 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:42.536 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:42.536 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.536 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.536 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:42.536 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.536 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.537 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.537 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:42.537 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:42.537 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.537 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.537 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top-2:19.1.0-1382.gde09dd01.el9.noarch 2024-08-18T23:33:42.537 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-18T23:33:42.537 INFO:teuthology.orchestra.run.smithi161.stdout: flex-2.6.4-9.el9.x86_64 2024-08-18T23:33:42.537 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-18T23:33:42.537 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-18T23:33:42.537 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-18T23:33:42.537 INFO:teuthology.orchestra.run.smithi161.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-18T23:33:42.537 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-18T23:33:42.537 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-18T23:33:42.538 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-18T23:33:42.538 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-18T23:33:42.538 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-18T23:33:42.538 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.538 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.538 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.538 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-18T23:33:42.538 INFO:teuthology.orchestra.run.smithi161.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-18T23:33:42.538 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-18T23:33:42.538 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-18T23:33:42.538 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-18T23:33:42.538 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.538 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.539 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-18T23:33:42.539 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.539 INFO:teuthology.orchestra.run.smithi161.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-18T23:33:42.539 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-18T23:33:42.539 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-18T23:33:42.539 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-18T23:33:42.539 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-18T23:33:42.539 INFO:teuthology.orchestra.run.smithi161.stdout: lua-5.4.4-4.el9.x86_64 2024-08-18T23:33:42.539 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-18T23:33:42.539 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-18T23:33:42.539 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-18T23:33:42.539 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-18T23:33:42.540 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-18T23:33:42.540 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-18T23:33:42.540 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-13.14-1.el9.x86_64 2024-08-18T23:33:42.540 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-08-18T23:33:42.540 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-08-18T23:33:42.540 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server-13.14-1.el9.x86_64 2024-08-18T23:33:42.540 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-18T23:33:42.540 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-18T23:33:42.540 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-18T23:33:42.540 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-18T23:33:42.540 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-18T23:33:42.541 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-18T23:33:42.541 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-18T23:33:42.541 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.541 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.541 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.541 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-18T23:33:42.541 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-18T23:33:42.541 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-18T23:33:42.541 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-18T23:33:42.541 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-18T23:33:42.541 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-18T23:33:42.541 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-18T23:33:42.541 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel-3.9.19-4.el9.x86_64 2024-08-18T23:33:42.541 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-18T23:33:42.542 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-18T23:33:42.542 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-18T23:33:42.542 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-18T23:33:42.542 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-18T23:33:42.542 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-18T23:33:42.542 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-18T23:33:42.542 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-18T23:33:42.542 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-18T23:33:42.542 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-18T23:33:42.542 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-18T23:33:42.542 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-18T23:33:42.542 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-18T23:33:42.542 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-18T23:33:42.543 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-18T23:33:42.543 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-18T23:33:42.543 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-18T23:33:42.543 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-18T23:33:42.543 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-18T23:33:42.543 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-18T23:33:42.543 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-18T23:33:42.543 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-18T23:33:42.543 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply-3.11-14.el9.noarch 2024-08-18T23:33:42.543 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-18T23:33:42.543 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-18T23:33:42.543 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-18T23:33:42.544 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-18T23:33:42.544 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-18T23:33:42.544 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-18T23:33:42.544 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-18T23:33:42.544 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-18T23:33:42.544 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-18T23:33:42.544 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.544 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.544 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-18T23:33:42.544 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-18T23:33:42.544 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-18T23:33:42.544 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.544 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-18T23:33:42.544 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-18T23:33:42.544 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-18T23:33:42.545 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-18T23:33:42.545 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-18T23:33:42.545 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-18T23:33:42.545 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-18T23:33:42.545 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-18T23:33:42.545 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-18T23:33:42.545 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-18T23:33:42.545 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-18T23:33:42.545 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-18T23:33:42.545 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-18T23:33:42.545 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-18T23:33:42.545 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-18T23:33:42.545 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.545 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.546 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd-2:19.1.0-1382.gde09dd01.el9.x86_64 2024-08-18T23:33:42.546 INFO:teuthology.orchestra.run.smithi161.stdout: re2-1:20211101-3.el9.x86_64 2024-08-18T23:33:42.546 INFO:teuthology.orchestra.run.smithi161.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-18T23:33:42.546 INFO:teuthology.orchestra.run.smithi161.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-18T23:33:42.546 INFO:teuthology.orchestra.run.smithi161.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-18T23:33:42.546 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-18T23:33:42.546 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-18T23:33:42.546 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-18T23:33:42.547 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:33:42.547 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-08-18T23:33:42.871 DEBUG:teuthology.parallel:result is None 2024-08-18T23:33:42.871 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-18T23:33:43.025 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-18T23:33:43.048 INFO:teuthology.orchestra.run.smithi078.stdout:19.1.0-1382.gde09dd01.el9 2024-08-18T23:33:43.049 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1382.gde09dd01.el9 2024-08-18T23:33:43.049 INFO:teuthology.task.install:The correct ceph version 19.1.0-1382.gde09dd01 is installed. 2024-08-18T23:33:43.051 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-18T23:33:43.191 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-18T23:33:43.214 INFO:teuthology.orchestra.run.smithi123.stdout:19.1.0-1382.gde09dd01.el9 2024-08-18T23:33:43.214 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1382.gde09dd01.el9 2024-08-18T23:33:43.214 INFO:teuthology.task.install:The correct ceph version 19.1.0-1382.gde09dd01 is installed. 2024-08-18T23:33:43.216 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-18T23:33:43.352 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-18T23:33:43.375 INFO:teuthology.orchestra.run.smithi161.stdout:19.1.0-1382.gde09dd01.el9 2024-08-18T23:33:43.375 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1382.gde09dd01.el9 2024-08-18T23:33:43.375 INFO:teuthology.task.install:The correct ceph version 19.1.0-1382.gde09dd01 is installed. 2024-08-18T23:33:43.377 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-18T23:33:43.377 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:33:43.377 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-18T23:33:43.405 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-18T23:33:43.406 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-18T23:33:43.434 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-18T23:33:43.435 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-18T23:33:43.461 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-18T23:33:43.462 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:33:43.462 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/daemon-helper 2024-08-18T23:33:43.488 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-18T23:33:43.552 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-18T23:33:43.552 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-08-18T23:33:43.577 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-18T23:33:43.643 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-18T23:33:43.643 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2024-08-18T23:33:43.668 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-18T23:33:43.734 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-18T23:33:43.734 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:33:43.735 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-18T23:33:43.759 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-18T23:33:43.823 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-18T23:33:43.823 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-18T23:33:43.848 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-18T23:33:43.913 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-18T23:33:43.914 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-18T23:33:43.939 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-18T23:33:44.006 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-18T23:33:44.006 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:33:44.006 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/stdin-killer 2024-08-18T23:33:44.031 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-18T23:33:44.096 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-18T23:33:44.096 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-08-18T23:33:44.121 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-18T23:33:44.184 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-18T23:33:44.185 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2024-08-18T23:33:44.209 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-18T23:33:44.274 INFO:teuthology.run_tasks:Running task cephadm... 2024-08-18T23:33:44.359 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-08-18T23:33:44.359 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-18T23:33:44.360 INFO:tasks.cephadm:Cluster fsid is 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:33:44.360 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-08-18T23:33:44.360 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.78', 'mon.b': '172.21.15.123', 'mon.c': '172.21.15.161'} 2024-08-18T23:33:44.360 INFO:tasks.cephadm:First mon is mon.a on smithi078 2024-08-18T23:33:44.360 INFO:tasks.cephadm:First mgr is x 2024-08-18T23:33:44.360 INFO:tasks.cephadm:Normalizing hostnames... 2024-08-18T23:33:44.360 DEBUG:teuthology.orchestra.run.smithi078:> sudo hostname $(hostname -s) 2024-08-18T23:33:44.386 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname $(hostname -s) 2024-08-18T23:33:44.411 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname $(hostname -s) 2024-08-18T23:33:44.438 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-08-18T23:33:44.438 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-18T23:33:44.581 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'de09dd018de4baddfa6ca4d43bc8fb784cf75833', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55869/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.0-1382-gde09dd01', 'node_name': '172.21.5.40+adami10', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.0-1382.gde09dd01'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-08-18 16:22:46.893995', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-08-18T23:33:44.713 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 de09dd018de4baddfa6ca4d43bc8fb784cf75833 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-18T23:33:44.715 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/x86_64/flavors/default/cephadm 2024-08-18T23:33:44.715 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/x86_64/flavors/default/cephadm 2024-08-18T23:33:44.715 DEBUG:teuthology.orchestra.run.smithi078:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-18T23:33:45.077 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r--. 1 ubuntu ubuntu 784608 Aug 18 23:33 /home/ubuntu/cephtest/cephadm 2024-08-18T23:33:45.078 DEBUG:teuthology.orchestra.run.smithi123:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-18T23:33:45.452 INFO:teuthology.orchestra.run.smithi123.stdout:-rw-r--r--. 1 ubuntu ubuntu 784608 Aug 18 23:33 /home/ubuntu/cephtest/cephadm 2024-08-18T23:33:45.452 DEBUG:teuthology.orchestra.run.smithi161:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/de09dd018de4baddfa6ca4d43bc8fb784cf75833/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-18T23:33:45.833 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r--. 1 ubuntu ubuntu 784608 Aug 18 23:33 /home/ubuntu/cephtest/cephadm 2024-08-18T23:33:45.834 DEBUG:teuthology.orchestra.run.smithi078:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-18T23:33:45.854 DEBUG:teuthology.orchestra.run.smithi123:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-18T23:33:45.873 DEBUG:teuthology.orchestra.run.smithi161:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-18T23:33:45.905 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 on all hosts... 2024-08-18T23:33:45.905 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 pull 2024-08-18T23:33:45.907 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 pull 2024-08-18T23:33:45.916 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 pull 2024-08-18T23:33:47.504 INFO:teuthology.orchestra.run.smithi078.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833... 2024-08-18T23:33:47.570 INFO:teuthology.orchestra.run.smithi123.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833... 2024-08-18T23:33:48.349 INFO:teuthology.orchestra.run.smithi161.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833... 2024-08-18T23:34:14.588 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-08-18T23:34:14.588 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc)", 2024-08-18T23:34:14.588 INFO:teuthology.orchestra.run.smithi123.stdout: "image_id": "ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba", 2024-08-18T23:34:14.589 INFO:teuthology.orchestra.run.smithi123.stdout: "repo_digests": [ 2024-08-18T23:34:14.589 INFO:teuthology.orchestra.run.smithi123.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9" 2024-08-18T23:34:14.589 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-08-18T23:34:14.589 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-08-18T23:34:16.848 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-08-18T23:34:16.848 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc)", 2024-08-18T23:34:16.848 INFO:teuthology.orchestra.run.smithi161.stdout: "image_id": "ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba", 2024-08-18T23:34:16.848 INFO:teuthology.orchestra.run.smithi161.stdout: "repo_digests": [ 2024-08-18T23:34:16.848 INFO:teuthology.orchestra.run.smithi161.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9" 2024-08-18T23:34:16.849 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-08-18T23:34:16.849 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-08-18T23:34:18.092 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:34:18.092 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc)", 2024-08-18T23:34:18.092 INFO:teuthology.orchestra.run.smithi078.stdout: "image_id": "ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba", 2024-08-18T23:34:18.092 INFO:teuthology.orchestra.run.smithi078.stdout: "repo_digests": [ 2024-08-18T23:34:18.092 INFO:teuthology.orchestra.run.smithi078.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9" 2024-08-18T23:34:18.092 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-08-18T23:34:18.092 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:34:18.118 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph 2024-08-18T23:34:18.176 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph 2024-08-18T23:34:18.208 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph 2024-08-18T23:34:18.242 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 777 /etc/ceph 2024-08-18T23:34:18.267 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /etc/ceph 2024-08-18T23:34:18.295 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /etc/ceph 2024-08-18T23:34:18.322 INFO:tasks.cephadm:Writing seed config... 2024-08-18T23:34:18.323 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-08-18T23:34:18.323 INFO:tasks.cephadm: override: [client] debug client = 20 2024-08-18T23:34:18.323 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-08-18T23:34:18.324 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-08-18T23:34:18.324 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-08-18T23:34:18.324 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-08-18T23:34:18.324 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-08-18T23:34:18.324 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-08-18T23:34:18.324 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-08-18T23:34:18.324 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-08-18T23:34:18.324 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-08-18T23:34:18.324 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-08-18T23:34:18.324 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-08-18T23:34:18.324 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-08-18T23:34:18.324 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-08-18T23:34:18.324 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-08-18T23:34:18.324 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-08-18T23:34:18.324 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-08-18T23:34:18.325 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-08-18T23:34:18.325 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-08-18T23:34:18.325 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-08-18T23:34:18.325 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-08-18T23:34:18.325 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-08-18T23:34:18.325 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-08-18T23:34:18.325 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-08-18T23:34:18.325 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-08-18T23:34:18.325 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-08-18T23:34:18.325 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-08-18T23:34:18.325 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-08-18T23:34:18.325 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-08-18T23:34:18.325 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-08-18T23:34:18.325 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-08-18T23:34:18.325 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-08-18T23:34:18.326 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-08-18T23:34:18.326 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-08-18T23:34:18.326 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-08-18T23:34:18.326 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-08-18T23:34:18.326 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-08-18T23:34:18.326 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-08-18T23:34:18.326 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-08-18T23:34:18.326 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-08-18T23:34:18.326 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-08-18T23:34:18.326 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-08-18T23:34:18.326 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-08-18T23:34:18.326 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-08-18T23:34:18.326 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-08-18T23:34:18.327 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:34:18.327 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-08-18T23:34:18.342 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 4f1128e6-5dba-11ef-bccf-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-08-18T23:34:18.343 DEBUG:teuthology.orchestra.run.smithi078:mon.a> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mon.a.service 2024-08-18T23:34:18.385 DEBUG:teuthology.orchestra.run.smithi078:mgr.x> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mgr.x.service 2024-08-18T23:34:18.427 INFO:tasks.cephadm:Bootstrapping... 2024-08-18T23:34:18.428 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 -v bootstrap --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.78 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-08-18T23:34:18.629 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-08-18T23:34:18.629 INFO:teuthology.orchestra.run.smithi078.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833', '-v', 'bootstrap', '--fsid', '4f1128e6-5dba-11ef-bccf-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.78', '--skip-admin-label'] 2024-08-18T23:34:18.629 INFO:teuthology.orchestra.run.smithi078.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-08-18T23:34:18.629 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying podman|docker is present... 2024-08-18T23:34:18.657 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout 5.1.2 2024-08-18T23:34:18.657 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying lvm2 is present... 2024-08-18T23:34:18.657 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying time synchronization is in place... 2024-08-18T23:34:18.665 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-18T23:34:18.665 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-18T23:34:18.671 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-18T23:34:18.671 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-08-18T23:34:18.677 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout enabled 2024-08-18T23:34:18.684 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout active 2024-08-18T23:34:18.684 INFO:teuthology.orchestra.run.smithi078.stdout:Unit chronyd.service is enabled and running 2024-08-18T23:34:18.684 INFO:teuthology.orchestra.run.smithi078.stdout:Repeating the final host check... 2024-08-18T23:34:18.708 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout 5.1.2 2024-08-18T23:34:18.708 INFO:teuthology.orchestra.run.smithi078.stdout:podman (/bin/podman) version 5.1.2 is present 2024-08-18T23:34:18.708 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl is present 2024-08-18T23:34:18.708 INFO:teuthology.orchestra.run.smithi078.stdout:lvcreate is present 2024-08-18T23:34:18.714 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-18T23:34:18.715 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-18T23:34:18.720 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-18T23:34:18.721 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-08-18T23:34:18.727 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout enabled 2024-08-18T23:34:18.734 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout active 2024-08-18T23:34:18.734 INFO:teuthology.orchestra.run.smithi078.stdout:Unit chronyd.service is enabled and running 2024-08-18T23:34:18.734 INFO:teuthology.orchestra.run.smithi078.stdout:Host looks OK 2024-08-18T23:34:18.734 INFO:teuthology.orchestra.run.smithi078.stdout:Cluster fsid: 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:34:18.734 INFO:teuthology.orchestra.run.smithi078.stdout:Acquiring lock 139839833419936 on /run/cephadm/4f1128e6-5dba-11ef-bccf-c7b262605968.lock 2024-08-18T23:34:18.735 INFO:teuthology.orchestra.run.smithi078.stdout:Lock 139839833419936 acquired on /run/cephadm/4f1128e6-5dba-11ef-bccf-c7b262605968.lock 2024-08-18T23:34:18.735 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying IP 172.21.15.78 port 3300 ... 2024-08-18T23:34:18.735 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying IP 172.21.15.78 port 6789 ... 2024-08-18T23:34:18.735 INFO:teuthology.orchestra.run.smithi078.stdout:Base mon IP(s) is [172.21.15.78:3300, 172.21.15.78:6789], mon addrv is [v2:172.21.15.78:3300,v1:172.21.15.78:6789] 2024-08-18T23:34:18.738 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.78 metric 100 2024-08-18T23:34:18.738 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.78 metric 100 2024-08-18T23:34:18.741 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-08-18T23:34:18.741 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-08-18T23:34:18.741 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-08-18T23:34:18.743 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-08-18T23:34:18.743 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-08-18T23:34:18.743 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-18T23:34:18.743 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-08-18T23:34:18.744 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link noprefixroute 2024-08-18T23:34:18.744 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-18T23:34:18.744 INFO:teuthology.orchestra.run.smithi078.stdout:Mon IP `172.21.15.78` is in CIDR network `172.21.0.0/20` 2024-08-18T23:34:18.744 INFO:teuthology.orchestra.run.smithi078.stdout:Mon IP `172.21.15.78` is in CIDR network `172.21.0.0/20` 2024-08-18T23:34:18.744 INFO:teuthology.orchestra.run.smithi078.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-08-18T23:34:18.745 INFO:teuthology.orchestra.run.smithi078.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-08-18T23:34:18.746 INFO:teuthology.orchestra.run.smithi078.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833... 2024-08-18T23:34:19.227 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba 2024-08-18T23:34:19.227 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833... 2024-08-18T23:34:19.228 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Getting image source signatures 2024-08-18T23:34:19.228 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying blob sha256:7556061847b3cc957dc118ff2516cc1cd809c4272dcb16074c5c7fe28afc4774 2024-08-18T23:34:19.228 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-08-18T23:34:19.228 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying config sha256:ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba 2024-08-18T23:34:19.228 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Writing manifest to image destination 2024-08-18T23:34:20.156 INFO:teuthology.orchestra.run.smithi078.stdout:ceph: stdout ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc) 2024-08-18T23:34:20.157 INFO:teuthology.orchestra.run.smithi078.stdout:Ceph version: ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc) 2024-08-18T23:34:20.157 INFO:teuthology.orchestra.run.smithi078.stdout:Extracting ceph user uid/gid from container image... 2024-08-18T23:34:21.021 INFO:teuthology.orchestra.run.smithi078.stdout:stat: stdout 167 167 2024-08-18T23:34:21.021 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial keys... 2024-08-18T23:34:21.951 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQD9hMJmaVMFFRAAWeANAs82O3Lz77bi/ZbxVA== 2024-08-18T23:34:22.840 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQD+hMJmiPV3DxAAPXpAvP/gW7a2+JoMDFfRpA== 2024-08-18T23:34:23.630 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQD/hMJm9OjaCBAAjgvAMvShuMeWmsUelGFeJw== 2024-08-18T23:34:23.631 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial monmap... 2024-08-18T23:34:24.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-18T23:34:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-08-18T23:34:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:34:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-18T23:34:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:monmaptool for a [v2:172.21.15.78:3300,v1:172.21.15.78:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-18T23:34:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:setting min_mon_release = quincy 2024-08-18T23:34:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: set fsid to 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:34:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-18T23:34:24.504 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:34:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:Creating mon... 2024-08-18T23:34:29.138 INFO:teuthology.orchestra.run.smithi078.stdout:create mon.a on 2024-08-18T23:34:29.588 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-08-18T23:34:29.802 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-08-18T23:34:30.173 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-4f1128e6-5dba-11ef-bccf-c7b262605968.target → /etc/systemd/system/ceph-4f1128e6-5dba-11ef-bccf-c7b262605968.target. 2024-08-18T23:34:30.173 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-4f1128e6-5dba-11ef-bccf-c7b262605968.target → /etc/systemd/system/ceph-4f1128e6-5dba-11ef-bccf-c7b262605968.target. 2024-08-18T23:34:31.110 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mon.a 2024-08-18T23:34:31.111 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mon.a.service: Unit ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mon.a.service not loaded. 2024-08-18T23:34:31.473 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4f1128e6-5dba-11ef-bccf-c7b262605968.target.wants/ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mon.a.service → /etc/systemd/system/ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@.service. 2024-08-18T23:34:31.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:31 smithi078 podman[18051]: 2024-08-18 23:34:31.617426985 +0000 UTC m=+0.017209493 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-18T23:34:33.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:32 smithi078 podman[18051]: 2024-08-18 23:34:32.732325598 +0000 UTC m=+1.132108092 container create d2ced15041ac1a796bc914d908ca7bd94ab598608b8380e9feec806ee7d8a0a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-08-18T23:34:33.869 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:33 smithi078 podman[18051]: 2024-08-18 23:34:33.823194439 +0000 UTC m=+2.222976946 container init d2ced15041ac1a796bc914d908ca7bd94ab598608b8380e9feec806ee7d8a0a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-08-18T23:34:33.870 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:33 smithi078 podman[18051]: 2024-08-18 23:34:33.826565766 +0000 UTC m=+2.226348260 container start d2ced15041ac1a796bc914d908ca7bd94ab598608b8380e9feec806ee7d8a0a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-a, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-08-18T23:34:33.937 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-18T23:34:33.938 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-08-18T23:34:33.945 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-18T23:34:33.945 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-08-18T23:34:33.945 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-08-18T23:34:33.945 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to enable service . firewalld.service is not available 2024-08-18T23:34:33.945 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mon to start... 2024-08-18T23:34:33.945 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mon... 2024-08-18T23:34:34.170 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:33 smithi078 ceph-mon[18070]: mkfs 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:34:34.170 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:33 smithi078 ceph-mon[18070]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-18T23:34:34.170 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:33 smithi078 bash[18051]: d2ced15041ac1a796bc914d908ca7bd94ab598608b8380e9feec806ee7d8a0a3 2024-08-18T23:34:34.170 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:33 smithi078 systemd[1]: Started Ceph mon.a for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:34:35.048 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:34 smithi078 ceph-mon[18070]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-18T23:34:35.048 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:34 smithi078 ceph-mon[18070]: monmap epoch 1 2024-08-18T23:34:35.048 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:34 smithi078 ceph-mon[18070]: fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:34:35.048 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:34 smithi078 ceph-mon[18070]: last_changed 2024-08-18T23:34:23.952483+0000 2024-08-18T23:34:35.048 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:34 smithi078 ceph-mon[18070]: created 2024-08-18T23:34:23.952483+0000 2024-08-18T23:34:35.048 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:34 smithi078 ceph-mon[18070]: min_mon_release 19 (squid) 2024-08-18T23:34:35.048 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:34 smithi078 ceph-mon[18070]: election_strategy: 1 2024-08-18T23:34:35.049 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:34 smithi078 ceph-mon[18070]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-08-18T23:34:35.049 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:34 smithi078 ceph-mon[18070]: fsmap 2024-08-18T23:34:35.049 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:34 smithi078 ceph-mon[18070]: osdmap e1: 0 total, 0 up, 0 in 2024-08-18T23:34:35.049 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:34 smithi078 ceph-mon[18070]: mgrmap e1: no daemons active 2024-08-18T23:34:35.049 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:34 smithi078 ceph-mon[18070]: from='client.? 172.21.15.78:0/3890551084' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-08-18T23:34:35.050 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout cluster: 2024-08-18T23:34:35.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout id: 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:34:35.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-08-18T23:34:35.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-08-18T23:34:35.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout services: 2024-08-18T23:34:35.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.56979s) 2024-08-18T23:34:35.056 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-08-18T23:34:35.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-08-18T23:34:35.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-08-18T23:34:35.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout data: 2024-08-18T23:34:35.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-08-18T23:34:35.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-08-18T23:34:35.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-08-18T23:34:35.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout pgs: 2024-08-18T23:34:35.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-08-18T23:34:35.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.369+0000 7fabc3099640 1 Processor -- start 2024-08-18T23:34:35.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.369+0000 7fabc3099640 1 -- start start 2024-08-18T23:34:35.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.369+0000 7fabc3099640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabbc107e20 0x7fabbc10a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:35.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.369+0000 7fabc3099640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc07bd70 con 0x7fabbc107e20 2024-08-18T23:34:35.057 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.369+0000 7fabc0e35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabbc107e20 0x7fabbc10a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:35.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.369+0000 7fabc0e35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabbc107e20 0x7fabbc10a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44324/0 (socket says 172.21.15.78:44324) 2024-08-18T23:34:35.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.369+0000 7fabc0e35640 1 -- 172.21.15.78:0/40454390 learned_addr learned my addr 172.21.15.78:0/40454390 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:35.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.370+0000 7fabc0e35640 1 -- 172.21.15.78:0/40454390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabbc10a750 con 0x7fabbc107e20 2024-08-18T23:34:35.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.371+0000 7fabc0e35640 1 --2- 172.21.15.78:0/40454390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabbc107e20 0x7fabbc10a210 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009820 tx=0x7fabb002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1eed55f4670f23 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:35.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.372+0000 7fabbb7fe640 1 -- 172.21.15.78:0/40454390 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabb0037070 con 0x7fabbc107e20 2024-08-18T23:34:35.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.372+0000 7fabbb7fe640 1 -- 172.21.15.78:0/40454390 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb0032440 con 0x7fabbc107e20 2024-08-18T23:34:35.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.372+0000 7fabbb7fe640 1 -- 172.21.15.78:0/40454390 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabb0032740 con 0x7fabbc107e20 2024-08-18T23:34:35.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.372+0000 7fabc3099640 1 -- 172.21.15.78:0/40454390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabbc107e20 msgr2=0x7fabbc10a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:35.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.372+0000 7fabc3099640 1 --2- 172.21.15.78:0/40454390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabbc107e20 0x7fabbc10a210 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009820 tx=0x7fabb002a270 comp rx=0 tx=0).stop 2024-08-18T23:34:35.058 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.372+0000 7fabc3099640 1 -- 172.21.15.78:0/40454390 shutdown_connections 2024-08-18T23:34:35.059 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.372+0000 7fabc3099640 1 --2- 172.21.15.78:0/40454390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabbc107e20 0x7fabbc10a210 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:35.059 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.372+0000 7fabc3099640 1 -- 172.21.15.78:0/40454390 >> 172.21.15.78:0/40454390 conn(0x7fabbc1019a0 msgr2=0x7fabbc103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:35.059 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.372+0000 7fabc3099640 1 -- 172.21.15.78:0/40454390 shutdown_connections 2024-08-18T23:34:35.059 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.372+0000 7fabc3099640 1 -- 172.21.15.78:0/40454390 wait complete. 2024-08-18T23:34:35.059 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.373+0000 7fabc3099640 1 Processor -- start 2024-08-18T23:34:35.059 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.373+0000 7fabc3099640 1 -- start start 2024-08-18T23:34:35.059 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.373+0000 7fabc3099640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabbc107e20 0x7fabbc19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:35.059 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.373+0000 7fabc3099640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc10ab60 con 0x7fabbc107e20 2024-08-18T23:34:35.059 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.373+0000 7fabc0e35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabbc107e20 0x7fabbc19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:35.059 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.373+0000 7fabc0e35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabbc107e20 0x7fabbc19ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44328/0 (socket says 172.21.15.78:44328) 2024-08-18T23:34:35.059 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.373+0000 7fabc0e35640 1 -- 172.21.15.78:0/3890551084 learned_addr learned my addr 172.21.15.78:0/3890551084 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:35.059 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.373+0000 7fabc0e35640 1 -- 172.21.15.78:0/3890551084 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabbc19b2e0 con 0x7fabbc107e20 2024-08-18T23:34:35.059 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.374+0000 7fabc0e35640 1 --2- 172.21.15.78:0/3890551084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabbc107e20 0x7fabbc19ada0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fabb00090b0 tx=0x7fabb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:35.060 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.374+0000 7fabb9ffb640 1 -- 172.21.15.78:0/3890551084 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabb0041070 con 0x7fabbc107e20 2024-08-18T23:34:35.060 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.374+0000 7fabb9ffb640 1 -- 172.21.15.78:0/3890551084 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb003c7a0 con 0x7fabbc107e20 2024-08-18T23:34:35.060 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.374+0000 7fabb9ffb640 1 -- 172.21.15.78:0/3890551084 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabb0037040 con 0x7fabbc107e20 2024-08-18T23:34:35.060 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.374+0000 7fabc3099640 1 -- 172.21.15.78:0/3890551084 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabbc19b570 con 0x7fabbc107e20 2024-08-18T23:34:35.060 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.374+0000 7fabc3099640 1 -- 172.21.15.78:0/3890551084 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabbc19e260 con 0x7fabbc107e20 2024-08-18T23:34:35.060 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.374+0000 7fabb9ffb640 1 -- 172.21.15.78:0/3890551084 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fabb0044070 con 0x7fabbc107e20 2024-08-18T23:34:35.060 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.374+0000 7fabb9ffb640 1 -- 172.21.15.78:0/3890551084 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fabb003d3f0 con 0x7fabbc107e20 2024-08-18T23:34:35.060 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.375+0000 7fabc3099640 1 -- 172.21.15.78:0/3890551084 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabbc108280 con 0x7fabbc107e20 2024-08-18T23:34:35.060 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.377+0000 7fabb9ffb640 1 -- 172.21.15.78:0/3890551084 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7fabb0046450 con 0x7fabbc107e20 2024-08-18T23:34:35.060 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.435+0000 7fabc3099640 1 -- 172.21.15.78:0/3890551084 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fabbc02d5e0 con 0x7fabbc107e20 2024-08-18T23:34:35.060 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.435+0000 7fabb9ffb640 1 -- 172.21.15.78:0/3890551084 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+316 (secure 0 0 0) 0x7fabb0046630 con 0x7fabbc107e20 2024-08-18T23:34:35.060 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.436+0000 7fabc3099640 1 -- 172.21.15.78:0/3890551084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabbc107e20 msgr2=0x7fabbc19ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:35.060 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.436+0000 7fabc3099640 1 --2- 172.21.15.78:0/3890551084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabbc107e20 0x7fabbc19ada0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fabb00090b0 tx=0x7fabb00046c0 comp rx=0 tx=0).stop 2024-08-18T23:34:35.061 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.436+0000 7fabc3099640 1 -- 172.21.15.78:0/3890551084 shutdown_connections 2024-08-18T23:34:35.061 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.436+0000 7fabc3099640 1 --2- 172.21.15.78:0/3890551084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabbc107e20 0x7fabbc19ada0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:35.061 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.436+0000 7fabc3099640 1 -- 172.21.15.78:0/3890551084 >> 172.21.15.78:0/3890551084 conn(0x7fabbc1019a0 msgr2=0x7fabbc102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:35.061 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.436+0000 7fabc3099640 1 -- 172.21.15.78:0/3890551084 shutdown_connections 2024-08-18T23:34:35.061 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:34.436+0000 7fabc3099640 1 -- 172.21.15.78:0/3890551084 wait complete. 2024-08-18T23:34:35.061 INFO:teuthology.orchestra.run.smithi078.stdout:mon is available 2024-08-18T23:34:35.061 INFO:teuthology.orchestra.run.smithi078.stdout:Assimilating anything we can from ceph.conf... 2024-08-18T23:34:36.079 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-08-18T23:34:36.080 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [global] 2024-08-18T23:34:36.080 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout fsid = 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:34:36.080 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.78:3300,v1:172.21.15.78:6789] 2024-08-18T23:34:36.080 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-18T23:34:36.080 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-18T23:34:36.080 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-18T23:34:36.080 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-18T23:34:36.080 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-08-18T23:34:36.080 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-18T23:34:36.080 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-18T23:34:36.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-08-18T23:34:36.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [osd] 2024-08-18T23:34:36.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-18T23:34:36.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-18T23:34:36.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-18T23:34:36.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-18T23:34:36.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-18T23:34:36.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.434+0000 7f776b9cb640 1 Processor -- start 2024-08-18T23:34:36.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.434+0000 7f776b9cb640 1 -- start start 2024-08-18T23:34:36.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.434+0000 7f776b9cb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77641061a0 0x7f77641065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:36.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.434+0000 7f776b9cb640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7764106b70 con 0x7f77641061a0 2024-08-18T23:34:36.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.434+0000 7f7769767640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77641061a0 0x7f77641065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:36.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.434+0000 7f7769767640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77641061a0 0x7f77641065a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44336/0 (socket says 172.21.15.78:44336) 2024-08-18T23:34:36.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.435+0000 7f7769767640 1 -- 172.21.15.78:0/136176849 learned_addr learned my addr 172.21.15.78:0/136176849 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:36.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.435+0000 7f7769767640 1 -- 172.21.15.78:0/136176849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77641073a0 con 0x7f77641061a0 2024-08-18T23:34:36.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.435+0000 7f7769767640 1 --2- 172.21.15.78:0/136176849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77641061a0 0x7f77641065a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7760009820 tx=0x7f776002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284022b9744fa2b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:36.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.436+0000 7f775bfff640 1 -- 172.21.15.78:0/136176849 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7760037070 con 0x7f77641061a0 2024-08-18T23:34:36.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.436+0000 7f775bfff640 1 -- 172.21.15.78:0/136176849 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7760032440 con 0x7f77641061a0 2024-08-18T23:34:36.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.436+0000 7f775bfff640 1 -- 172.21.15.78:0/136176849 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7760032740 con 0x7f77641061a0 2024-08-18T23:34:36.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.436+0000 7f776b9cb640 1 -- 172.21.15.78:0/136176849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77641061a0 msgr2=0x7f77641065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:36.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.436+0000 7f776b9cb640 1 --2- 172.21.15.78:0/136176849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77641061a0 0x7f77641065a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7760009820 tx=0x7f776002a270 comp rx=0 tx=0).stop 2024-08-18T23:34:36.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.436+0000 7f776b9cb640 1 -- 172.21.15.78:0/136176849 shutdown_connections 2024-08-18T23:34:36.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.436+0000 7f776b9cb640 1 --2- 172.21.15.78:0/136176849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77641061a0 0x7f77641065a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:36.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.436+0000 7f776b9cb640 1 -- 172.21.15.78:0/136176849 >> 172.21.15.78:0/136176849 conn(0x7f77641019f0 msgr2=0x7f7764103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:36.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.436+0000 7f776b9cb640 1 -- 172.21.15.78:0/136176849 shutdown_connections 2024-08-18T23:34:36.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.436+0000 7f776b9cb640 1 -- 172.21.15.78:0/136176849 wait complete. 2024-08-18T23:34:36.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.437+0000 7f776b9cb640 1 Processor -- start 2024-08-18T23:34:36.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.437+0000 7f776b9cb640 1 -- start start 2024-08-18T23:34:36.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.437+0000 7f776b9cb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77641061a0 0x7f776407b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:36.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.437+0000 7f776b9cb640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7764107940 con 0x7f77641061a0 2024-08-18T23:34:36.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.437+0000 7f7769767640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77641061a0 0x7f776407b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:36.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.437+0000 7f7769767640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77641061a0 0x7f776407b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44338/0 (socket says 172.21.15.78:44338) 2024-08-18T23:34:36.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.437+0000 7f7769767640 1 -- 172.21.15.78:0/233096621 learned_addr learned my addr 172.21.15.78:0/233096621 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:36.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.437+0000 7f7769767640 1 -- 172.21.15.78:0/233096621 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776419daa0 con 0x7f77641061a0 2024-08-18T23:34:36.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.437+0000 7f7769767640 1 --2- 172.21.15.78:0/233096621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77641061a0 0x7f776407b9f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f77600090b0 tx=0x7f77600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:36.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.438+0000 7f775a7fc640 1 -- 172.21.15.78:0/233096621 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7760041070 con 0x7f77641061a0 2024-08-18T23:34:36.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.438+0000 7f775a7fc640 1 -- 172.21.15.78:0/233096621 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f776003c7a0 con 0x7f77641061a0 2024-08-18T23:34:36.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.438+0000 7f775a7fc640 1 -- 172.21.15.78:0/233096621 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7760037040 con 0x7f77641061a0 2024-08-18T23:34:36.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.438+0000 7f776b9cb640 1 -- 172.21.15.78:0/233096621 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776419dc70 con 0x7f77641061a0 2024-08-18T23:34:36.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.438+0000 7f776b9cb640 1 -- 172.21.15.78:0/233096621 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776419e010 con 0x7f77641061a0 2024-08-18T23:34:36.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.438+0000 7f775a7fc640 1 -- 172.21.15.78:0/233096621 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7760044070 con 0x7f77641061a0 2024-08-18T23:34:36.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.438+0000 7f775a7fc640 1 -- 172.21.15.78:0/233096621 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f776003d3f0 con 0x7f77641061a0 2024-08-18T23:34:36.085 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.439+0000 7f776b9cb640 1 -- 172.21.15.78:0/233096621 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7764107f90 con 0x7f77641061a0 2024-08-18T23:34:36.085 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.441+0000 7f775a7fc640 1 -- 172.21.15.78:0/233096621 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f7760046450 con 0x7f77641061a0 2024-08-18T23:34:36.085 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.497+0000 7f776b9cb640 1 -- 172.21.15.78:0/233096621 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f7764192570 con 0x7f77641061a0 2024-08-18T23:34:36.085 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.500+0000 7f775a7fc640 1 -- 172.21.15.78:0/233096621 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f7760046630 con 0x7f77641061a0 2024-08-18T23:34:36.085 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.501+0000 7f775a7fc640 1 -- 172.21.15.78:0/233096621 <== mon.0 v2:172.21.15.78:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f77600584b0 con 0x7f77641061a0 2024-08-18T23:34:36.085 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.502+0000 7f776b9cb640 1 -- 172.21.15.78:0/233096621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77641061a0 msgr2=0x7f776407b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:36.085 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.502+0000 7f776b9cb640 1 --2- 172.21.15.78:0/233096621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77641061a0 0x7f776407b9f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f77600090b0 tx=0x7f77600046c0 comp rx=0 tx=0).stop 2024-08-18T23:34:36.085 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.502+0000 7f776b9cb640 1 -- 172.21.15.78:0/233096621 shutdown_connections 2024-08-18T23:34:36.085 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.502+0000 7f776b9cb640 1 --2- 172.21.15.78:0/233096621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77641061a0 0x7f776407b9f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:36.086 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.502+0000 7f776b9cb640 1 -- 172.21.15.78:0/233096621 >> 172.21.15.78:0/233096621 conn(0x7f77641019f0 msgr2=0x7f7764190690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:36.086 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.502+0000 7f776b9cb640 1 -- 172.21.15.78:0/233096621 shutdown_connections 2024-08-18T23:34:36.086 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:35.502+0000 7f776b9cb640 1 -- 172.21.15.78:0/233096621 wait complete. 2024-08-18T23:34:36.086 INFO:teuthology.orchestra.run.smithi078.stdout:Generating new minimal ceph.conf... 2024-08-18T23:34:36.179 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:35 smithi078 ceph-mon[18070]: from='client.? 172.21.15.78:0/233096621' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-08-18T23:34:36.179 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:35 smithi078 ceph-mon[18070]: from='client.? 172.21.15.78:0/233096621' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-08-18T23:34:37.138 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:36 smithi078 ceph-mon[18070]: from='client.? 172.21.15.78:0/1190409900' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:34:37.378 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.452+0000 7f9ef2c9b640 1 Processor -- start 2024-08-18T23:34:37.378 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.452+0000 7f9ef2c9b640 1 -- start start 2024-08-18T23:34:37.378 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.453+0000 7f9ef2c9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eec1072c0 0x7f9eec1076c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:37.378 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.453+0000 7f9ef2c9b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec107c90 con 0x7f9eec1072c0 2024-08-18T23:34:37.378 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.453+0000 7f9ef0a37640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eec1072c0 0x7f9eec1076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:37.379 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.453+0000 7f9ef0a37640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eec1072c0 0x7f9eec1076c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39500/0 (socket says 172.21.15.78:39500) 2024-08-18T23:34:37.379 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.453+0000 7f9ef0a37640 1 -- 172.21.15.78:0/3391306778 learned_addr learned my addr 172.21.15.78:0/3391306778 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:37.379 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.453+0000 7f9ef0a37640 1 -- 172.21.15.78:0/3391306778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec108470 con 0x7f9eec1072c0 2024-08-18T23:34:37.379 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.454+0000 7f9ef0a37640 1 --2- 172.21.15.78:0/3391306778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eec1072c0 0x7f9eec1076c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009820 tx=0x7f9ee002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea695764aa2f7322 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:37.379 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.454+0000 7f9eeb7fe640 1 -- 172.21.15.78:0/3391306778 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee0037070 con 0x7f9eec1072c0 2024-08-18T23:34:37.379 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.454+0000 7f9eeb7fe640 1 -- 172.21.15.78:0/3391306778 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9ee0032900 con 0x7f9eec1072c0 2024-08-18T23:34:37.379 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.454+0000 7f9eeb7fe640 1 -- 172.21.15.78:0/3391306778 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee0030850 con 0x7f9eec1072c0 2024-08-18T23:34:37.379 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.454+0000 7f9ef2c9b640 1 -- 172.21.15.78:0/3391306778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eec1072c0 msgr2=0x7f9eec1076c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:37.379 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.454+0000 7f9ef2c9b640 1 --2- 172.21.15.78:0/3391306778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eec1072c0 0x7f9eec1076c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009820 tx=0x7f9ee002a270 comp rx=0 tx=0).stop 2024-08-18T23:34:37.379 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.455+0000 7f9ef2c9b640 1 -- 172.21.15.78:0/3391306778 shutdown_connections 2024-08-18T23:34:37.379 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.455+0000 7f9ef2c9b640 1 --2- 172.21.15.78:0/3391306778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eec1072c0 0x7f9eec1076c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:37.379 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.455+0000 7f9ef2c9b640 1 -- 172.21.15.78:0/3391306778 >> 172.21.15.78:0/3391306778 conn(0x7f9eec102ab0 msgr2=0x7f9eec104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:37.379 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.455+0000 7f9ef2c9b640 1 -- 172.21.15.78:0/3391306778 shutdown_connections 2024-08-18T23:34:37.379 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.455+0000 7f9ef2c9b640 1 -- 172.21.15.78:0/3391306778 wait complete. 2024-08-18T23:34:37.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.455+0000 7f9ef2c9b640 1 Processor -- start 2024-08-18T23:34:37.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.455+0000 7f9ef2c9b640 1 -- start start 2024-08-18T23:34:37.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.455+0000 7f9ef2c9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eec1072c0 0x7f9eec07ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:37.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.455+0000 7f9ef2c9b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec10b820 con 0x7f9eec1072c0 2024-08-18T23:34:37.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.455+0000 7f9ef0a37640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eec1072c0 0x7f9eec07ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:37.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.455+0000 7f9ef0a37640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eec1072c0 0x7f9eec07ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39504/0 (socket says 172.21.15.78:39504) 2024-08-18T23:34:37.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.455+0000 7f9ef0a37640 1 -- 172.21.15.78:0/1190409900 learned_addr learned my addr 172.21.15.78:0/1190409900 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:37.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.456+0000 7f9ef0a37640 1 -- 172.21.15.78:0/1190409900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec07cf40 con 0x7f9eec1072c0 2024-08-18T23:34:37.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.456+0000 7f9ef0a37640 1 --2- 172.21.15.78:0/1190409900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eec1072c0 0x7f9eec07ca00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00097f0 tx=0x7f9ee0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:37.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.456+0000 7f9ee9ffb640 1 -- 172.21.15.78:0/1190409900 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee0042070 con 0x7f9eec1072c0 2024-08-18T23:34:37.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.456+0000 7f9ee9ffb640 1 -- 172.21.15.78:0/1190409900 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9ee0032ca0 con 0x7f9eec1072c0 2024-08-18T23:34:37.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.456+0000 7f9ef2c9b640 1 -- 172.21.15.78:0/1190409900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eec07b130 con 0x7f9eec1072c0 2024-08-18T23:34:37.380 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.456+0000 7f9ee9ffb640 1 -- 172.21.15.78:0/1190409900 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee0037040 con 0x7f9eec1072c0 2024-08-18T23:34:37.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.456+0000 7f9ef2c9b640 1 -- 172.21.15.78:0/1190409900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eec07b670 con 0x7f9eec1072c0 2024-08-18T23:34:37.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.456+0000 7f9ee9ffb640 1 -- 172.21.15.78:0/1190409900 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9ee0045070 con 0x7f9eec1072c0 2024-08-18T23:34:37.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.456+0000 7f9ee9ffb640 1 -- 172.21.15.78:0/1190409900 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9ee0047ad0 con 0x7f9eec1072c0 2024-08-18T23:34:37.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.457+0000 7f9ef2c9b640 1 -- 172.21.15.78:0/1190409900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eec10b9a0 con 0x7f9eec1072c0 2024-08-18T23:34:37.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.459+0000 7f9ee9ffb640 1 -- 172.21.15.78:0/1190409900 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f9ee003da60 con 0x7f9eec1072c0 2024-08-18T23:34:37.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.516+0000 7f9ef2c9b640 1 -- 172.21.15.78:0/1190409900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f9eec107740 con 0x7f9eec1072c0 2024-08-18T23:34:37.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.517+0000 7f9ee9ffb640 1 -- 172.21.15.78:0/1190409900 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f9ee003d430 con 0x7f9eec1072c0 2024-08-18T23:34:37.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.517+0000 7f9ef2c9b640 1 -- 172.21.15.78:0/1190409900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eec1072c0 msgr2=0x7f9eec07ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:37.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.517+0000 7f9ef2c9b640 1 --2- 172.21.15.78:0/1190409900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eec1072c0 0x7f9eec07ca00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00097f0 tx=0x7f9ee0030ec0 comp rx=0 tx=0).stop 2024-08-18T23:34:37.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.518+0000 7f9ef2c9b640 1 -- 172.21.15.78:0/1190409900 shutdown_connections 2024-08-18T23:34:37.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.518+0000 7f9ef2c9b640 1 --2- 172.21.15.78:0/1190409900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eec1072c0 0x7f9eec07ca00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:37.381 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.518+0000 7f9ef2c9b640 1 -- 172.21.15.78:0/1190409900 >> 172.21.15.78:0/1190409900 conn(0x7f9eec102ab0 msgr2=0x7f9eec103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:37.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.518+0000 7f9ef2c9b640 1 -- 172.21.15.78:0/1190409900 shutdown_connections 2024-08-18T23:34:37.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:36.518+0000 7f9ef2c9b640 1 -- 172.21.15.78:0/1190409900 wait complete. 2024-08-18T23:34:37.382 INFO:teuthology.orchestra.run.smithi078.stdout:Restarting the monitor... 2024-08-18T23:34:37.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:37 smithi078 systemd[1]: Stopping Ceph mon.a for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:34:37.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:37 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-a[18066]: 2024-08-18T23:34:37.648+0000 7f90e1f75640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:34:37.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:37 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-a[18066]: 2024-08-18T23:34:37.648+0000 7f90e1f75640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-08-18T23:34:37.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:37 smithi078 podman[18315]: 2024-08-18 23:34:37.812548041 +0000 UTC m=+0.250568614 container died d2ced15041ac1a796bc914d908ca7bd94ab598608b8380e9feec806ee7d8a0a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-08-18T23:34:38.307 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 podman[18315]: 2024-08-18 23:34:38.10602996 +0000 UTC m=+0.544050536 container cleanup d2ced15041ac1a796bc914d908ca7bd94ab598608b8380e9feec806ee7d8a0a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-08-18T23:34:38.307 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 bash[18315]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-a 2024-08-18T23:34:38.659 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 podman[18329]: 2024-08-18 23:34:38.30694501 +0000 UTC m=+0.489164406 container remove d2ced15041ac1a796bc914d908ca7bd94ab598608b8380e9feec806ee7d8a0a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-a, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-08-18T23:34:38.659 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 systemd[1]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mon.a.service: Deactivated successfully. 2024-08-18T23:34:38.659 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 systemd[1]: Stopped Ceph mon.a for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:34:38.659 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 systemd[1]: Starting Ceph mon.a for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:34:38.911 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 podman[18421]: 2024-08-18 23:34:38.658231272 +0000 UTC m=+0.112890712 container create c87427028777e7408705cd076ec10fe9fc3ec5d95b63378328b597d4805ea38a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:34:38.911 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 podman[18421]: 2024-08-18 23:34:38.562843938 +0000 UTC m=+0.017503348 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-18T23:34:38.911 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 podman[18421]: 2024-08-18 23:34:38.907650577 +0000 UTC m=+0.362309983 container init c87427028777e7408705cd076ec10fe9fc3ec5d95b63378328b597d4805ea38a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-a, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-18T23:34:38.911 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 podman[18421]: 2024-08-18 23:34:38.911018165 +0000 UTC m=+0.365677569 container start c87427028777e7408705cd076ec10fe9fc3ec5d95b63378328b597d4805ea38a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-a, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0) 2024-08-18T23:34:39.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: set uid:gid to 167:167 (ceph:ceph) 2024-08-18T23:34:39.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc), process ceph-mon, pid 2 2024-08-18T23:34:39.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: pidfile_write: ignore empty --pid-file 2024-08-18T23:34:39.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: load: jerasure load: lrc 2024-08-18T23:34:39.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: RocksDB version: 7.9.2 2024-08-18T23:34:39.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Git sha 0 2024-08-18T23:34:39.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Compile date 2024-08-18 15:19:56 2024-08-18T23:34:39.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: DB SUMMARY 2024-08-18T23:34:39.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: DB Session ID: 49FA34ZIIAKRO8CCFVJV 2024-08-18T23:34:39.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: CURRENT file: CURRENT 2024-08-18T23:34:39.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: IDENTITY file: IDENTITY 2024-08-18T23:34:39.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-08-18T23:34:39.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-08-18T23:34:39.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104838 ; 2024-08-18T23:34:39.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.error_if_exists: 0 2024-08-18T23:34:39.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.create_if_missing: 0 2024-08-18T23:34:39.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.paranoid_checks: 1 2024-08-18T23:34:39.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-18T23:34:39.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-18T23:34:39.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-18T23:34:39.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.env: 0x559b966987a0 2024-08-18T23:34:39.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.fs: PosixFileSystem 2024-08-18T23:34:39.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.info_log: 0x559b98353500 2024-08-18T23:34:39.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_file_opening_threads: 16 2024-08-18T23:34:39.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.statistics: (nil) 2024-08-18T23:34:39.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.use_fsync: 0 2024-08-18T23:34:39.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_log_file_size: 0 2024-08-18T23:34:39.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-18T23:34:39.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-18T23:34:39.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.keep_log_file_num: 1000 2024-08-18T23:34:39.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.recycle_log_file_num: 0 2024-08-18T23:34:39.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.allow_fallocate: 1 2024-08-18T23:34:39.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.allow_mmap_reads: 0 2024-08-18T23:34:39.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.allow_mmap_writes: 0 2024-08-18T23:34:39.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.use_direct_reads: 0 2024-08-18T23:34:39.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-18T23:34:39.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.create_missing_column_families: 0 2024-08-18T23:34:39.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.db_log_dir: 2024-08-18T23:34:39.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.wal_dir: 2024-08-18T23:34:39.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-18T23:34:39.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-18T23:34:39.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-18T23:34:39.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-18T23:34:39.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-18T23:34:39.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-18T23:34:39.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.advise_random_on_open: 1 2024-08-18T23:34:39.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.db_write_buffer_size: 0 2024-08-18T23:34:39.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.write_buffer_manager: 0x559b98362dc0 2024-08-18T23:34:39.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-18T23:34:39.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-18T23:34:39.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-18T23:34:39.224 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.rate_limiter: (nil) 2024-08-18T23:34:39.224 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-18T23:34:39.224 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.wal_recovery_mode: 2 2024-08-18T23:34:39.224 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.enable_thread_tracking: 0 2024-08-18T23:34:39.224 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.enable_pipelined_write: 0 2024-08-18T23:34:39.224 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.unordered_write: 0 2024-08-18T23:34:39.224 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-18T23:34:39.224 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-18T23:34:39.224 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-18T23:34:39.224 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-18T23:34:39.224 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.row_cache: None 2024-08-18T23:34:39.224 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.wal_filter: None 2024-08-18T23:34:39.224 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-18T23:34:39.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.allow_ingest_behind: 0 2024-08-18T23:34:39.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.two_write_queues: 0 2024-08-18T23:34:39.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.manual_wal_flush: 0 2024-08-18T23:34:39.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.wal_compression: 0 2024-08-18T23:34:39.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.atomic_flush: 0 2024-08-18T23:34:39.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-18T23:34:39.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-18T23:34:39.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-18T23:34:39.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.log_readahead_size: 0 2024-08-18T23:34:39.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-18T23:34:39.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.best_efforts_recovery: 0 2024-08-18T23:34:39.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-18T23:34:39.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-18T23:34:39.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.allow_data_in_errors: 0 2024-08-18T23:34:39.226 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.db_host_id: __hostname__ 2024-08-18T23:34:39.226 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-18T23:34:39.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_background_jobs: 2 2024-08-18T23:34:39.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_background_compactions: -1 2024-08-18T23:34:39.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_subcompactions: 1 2024-08-18T23:34:39.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-18T23:34:39.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-18T23:34:39.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-18T23:34:39.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_total_wal_size: 0 2024-08-18T23:34:39.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-18T23:34:39.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-18T23:34:39.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-18T23:34:39.228 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-18T23:34:39.228 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_open_files: -1 2024-08-18T23:34:39.228 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.bytes_per_sync: 0 2024-08-18T23:34:39.228 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-18T23:34:39.228 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-18T23:34:39.228 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compaction_readahead_size: 0 2024-08-18T23:34:39.228 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_background_flushes: -1 2024-08-18T23:34:39.228 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Compression algorithms supported: 2024-08-18T23:34:39.228 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: kZSTD supported: 0 2024-08-18T23:34:39.228 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: kXpressCompression supported: 0 2024-08-18T23:34:39.228 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: kBZip2Compression supported: 0 2024-08-18T23:34:39.228 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-18T23:34:39.228 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: kLZ4Compression supported: 1 2024-08-18T23:34:39.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: kZlibCompression supported: 1 2024-08-18T23:34:39.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: kLZ4HCCompression supported: 1 2024-08-18T23:34:39.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: kSnappyCompression supported: 1 2024-08-18T23:34:39.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-18T23:34:39.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-18T23:34:39.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-08-18T23:34:39.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-18T23:34:39.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-18T23:34:39.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.merge_operator: 2024-08-18T23:34:39.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compaction_filter: None 2024-08-18T23:34:39.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compaction_filter_factory: None 2024-08-18T23:34:39.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.sst_partitioner_factory: None 2024-08-18T23:34:39.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-18T23:34:39.230 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-18T23:34:39.230 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559b98353120) 2024-08-18T23:34:39.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: cache_index_and_filter_blocks: 1 2024-08-18T23:34:39.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-18T23:34:39.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-18T23:34:39.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: pin_top_level_index_and_filter: 1 2024-08-18T23:34:39.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_type: 0 2024-08-18T23:34:39.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: data_block_index_type: 0 2024-08-18T23:34:39.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_shortening: 1 2024-08-18T23:34:39.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-18T23:34:39.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: checksum: 4 2024-08-18T23:34:39.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: no_block_cache: 0 2024-08-18T23:34:39.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache: 0x559b98347090 2024-08-18T23:34:39.231 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_name: BinnedLRUCache 2024-08-18T23:34:39.231 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_options: 2024-08-18T23:34:39.231 INFO:journalctl@ceph.mon.a.smithi078.stdout: capacity : 536870912 2024-08-18T23:34:39.231 INFO:journalctl@ceph.mon.a.smithi078.stdout: num_shard_bits : 4 2024-08-18T23:34:39.231 INFO:journalctl@ceph.mon.a.smithi078.stdout: strict_capacity_limit : 0 2024-08-18T23:34:39.231 INFO:journalctl@ceph.mon.a.smithi078.stdout: high_pri_pool_ratio: 0.000 2024-08-18T23:34:39.231 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_compressed: (nil) 2024-08-18T23:34:39.231 INFO:journalctl@ceph.mon.a.smithi078.stdout: persistent_cache: (nil) 2024-08-18T23:34:39.231 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_size: 4096 2024-08-18T23:34:39.231 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_size_deviation: 10 2024-08-18T23:34:39.231 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_restart_interval: 16 2024-08-18T23:34:39.231 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_block_restart_interval: 1 2024-08-18T23:34:39.231 INFO:journalctl@ceph.mon.a.smithi078.stdout: metadata_block_size: 4096 2024-08-18T23:34:39.231 INFO:journalctl@ceph.mon.a.smithi078.stdout: partition_filters: 0 2024-08-18T23:34:39.232 INFO:journalctl@ceph.mon.a.smithi078.stdout: use_delta_encoding: 1 2024-08-18T23:34:39.232 INFO:journalctl@ceph.mon.a.smithi078.stdout: filter_policy: bloomfilter 2024-08-18T23:34:39.232 INFO:journalctl@ceph.mon.a.smithi078.stdout: whole_key_filtering: 1 2024-08-18T23:34:39.232 INFO:journalctl@ceph.mon.a.smithi078.stdout: verify_compression: 0 2024-08-18T23:34:39.232 INFO:journalctl@ceph.mon.a.smithi078.stdout: read_amp_bytes_per_bit: 0 2024-08-18T23:34:39.232 INFO:journalctl@ceph.mon.a.smithi078.stdout: format_version: 5 2024-08-18T23:34:39.232 INFO:journalctl@ceph.mon.a.smithi078.stdout: enable_index_compression: 1 2024-08-18T23:34:39.232 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_align: 0 2024-08-18T23:34:39.232 INFO:journalctl@ceph.mon.a.smithi078.stdout: max_auto_readahead_size: 262144 2024-08-18T23:34:39.232 INFO:journalctl@ceph.mon.a.smithi078.stdout: prepopulate_block_cache: 0 2024-08-18T23:34:39.232 INFO:journalctl@ceph.mon.a.smithi078.stdout: initial_auto_readahead_size: 8192 2024-08-18T23:34:39.232 INFO:journalctl@ceph.mon.a.smithi078.stdout: num_file_reads_for_auto_readahead: 2 2024-08-18T23:34:39.232 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.write_buffer_size: 33554432 2024-08-18T23:34:39.233 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_write_buffer_number: 2 2024-08-18T23:34:39.233 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compression: NoCompression 2024-08-18T23:34:39.233 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.bottommost_compression: Disabled 2024-08-18T23:34:39.233 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.prefix_extractor: nullptr 2024-08-18T23:34:39.233 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-18T23:34:39.233 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.num_levels: 7 2024-08-18T23:34:39.233 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-18T23:34:39.233 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-18T23:34:39.233 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-18T23:34:39.233 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-18T23:34:39.233 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-18T23:34:39.233 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-18T23:34:39.233 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-18T23:34:39.233 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-18T23:34:39.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-18T23:34:39.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-18T23:34:39.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-18T23:34:39.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-18T23:34:39.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-18T23:34:39.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compression_opts.level: 32767 2024-08-18T23:34:39.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compression_opts.strategy: 0 2024-08-18T23:34:39.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-18T23:34:39.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-18T23:34:39.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-18T23:34:39.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-18T23:34:39.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compression_opts.enabled: false 2024-08-18T23:34:39.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-18T23:34:39.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-18T23:34:39.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-18T23:34:39.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-18T23:34:39.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.target_file_size_base: 67108864 2024-08-18T23:34:39.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-18T23:34:39.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-18T23:34:39.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-18T23:34:39.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-18T23:34:39.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-18T23:34:39.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-18T23:34:39.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-18T23:34:39.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-18T23:34:39.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-18T23:34:39.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-18T23:34:39.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-18T23:34:39.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-18T23:34:39.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-18T23:34:39.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-18T23:34:39.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.arena_block_size: 1048576 2024-08-18T23:34:39.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-18T23:34:39.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-18T23:34:39.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.disable_auto_compactions: 0 2024-08-18T23:34:39.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-18T23:34:39.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-18T23:34:39.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-18T23:34:39.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-18T23:34:39.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-18T23:34:39.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-18T23:34:39.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-18T23:34:39.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-18T23:34:39.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-18T23:34:39.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-18T23:34:39.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-18T23:34:39.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.inplace_update_support: 0 2024-08-18T23:34:39.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-18T23:34:39.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-18T23:34:39.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-18T23:34:39.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-18T23:34:39.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.bloom_locality: 0 2024-08-18T23:34:39.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.max_successive_merges: 0 2024-08-18T23:34:39.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-18T23:34:39.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.paranoid_file_checks: 0 2024-08-18T23:34:39.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.force_consistency_checks: 1 2024-08-18T23:34:39.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.report_bg_io_stats: 0 2024-08-18T23:34:39.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.ttl: 2592000 2024-08-18T23:34:39.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-18T23:34:39.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-18T23:34:39.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-18T23:34:39.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.enable_blob_files: false 2024-08-18T23:34:39.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.min_blob_size: 0 2024-08-18T23:34:39.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.blob_file_size: 268435456 2024-08-18T23:34:39.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-18T23:34:39.238 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-18T23:34:39.239 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-18T23:34:39.239 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-18T23:34:39.239 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-18T23:34:39.239 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.blob_file_starting_level: 0 2024-08-18T23:34:39.239 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-18T23:34:39.239 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-08-18T23:34:39.239 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-08-18T23:34:39.239 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 83e442c1-4930-49b3-8454-1cb179b1e321 2024-08-18T23:34:39.239 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724024078936580, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-08-18T23:34:39.239 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-08-18T23:34:39.239 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724024078937633, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99972, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97857, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14189, "raw_average_key_size": 49, "raw_value_size": 90154, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1724024078, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "83e442c1-4930-49b3-8454-1cb179b1e321", "db_session_id": "49FA34ZIIAKRO8CCFVJV", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-08-18T23:34:39.239 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724024078937727, "job": 1, "event": "recovery_finished"} 2024-08-18T23:34:39.239 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-08-18T23:34:39.240 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-18T23:34:39.240 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559b98374e00 2024-08-18T23:34:39.240 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: DB pointer 0x559b9846e000 2024-08-18T23:34:39.240 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-18T23:34:39.240 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-18T23:34:39.240 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** DB Stats ** 2024-08-18T23:34:39.240 INFO:journalctl@ceph.mon.a.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-18T23:34:39.240 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-18T23:34:39.240 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-18T23:34:39.240 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-18T23:34:39.240 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-18T23:34:39.240 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-18T23:34:39.240 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-18T23:34:39.241 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-08-18T23:34:39.241 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** Compaction Stats [default] ** 2024-08-18T23:34:39.241 INFO:journalctl@ceph.mon.a.smithi078.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-18T23:34:39.241 INFO:journalctl@ceph.mon.a.smithi078.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-18T23:34:39.241 INFO:journalctl@ceph.mon.a.smithi078.stdout: L0 2/0 99.49 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-18T23:34:39.241 INFO:journalctl@ceph.mon.a.smithi078.stdout: Sum 2/0 99.49 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-18T23:34:39.241 INFO:journalctl@ceph.mon.a.smithi078.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-18T23:34:39.241 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-08-18T23:34:39.241 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** Compaction Stats [default] ** 2024-08-18T23:34:39.241 INFO:journalctl@ceph.mon.a.smithi078.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-18T23:34:39.241 INFO:journalctl@ceph.mon.a.smithi078.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-18T23:34:39.241 INFO:journalctl@ceph.mon.a.smithi078.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 141.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-18T23:34:39.241 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-08-18T23:34:39.241 INFO:journalctl@ceph.mon.a.smithi078.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-18T23:34:39.242 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-08-18T23:34:39.242 INFO:journalctl@ceph.mon.a.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-18T23:34:39.242 INFO:journalctl@ceph.mon.a.smithi078.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-18T23:34:39.242 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-18T23:34:39.242 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-18T23:34:39.242 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-18T23:34:39.242 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-18T23:34:39.242 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative compaction: 0.00 GB write, 30.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-18T23:34:39.242 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval compaction: 0.00 GB write, 30.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-18T23:34:39.242 INFO:journalctl@ceph.mon.a.smithi078.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-18T23:34:39.242 INFO:journalctl@ceph.mon.a.smithi078.stdout: Block cache BinnedLRUCache@0x559b98347090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-08-18T23:34:39.242 INFO:journalctl@ceph.mon.a.smithi078.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-08-18T23:34:39.242 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-08-18T23:34:39.242 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-18T23:34:39.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: starting mon.a rank 0 at public addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] at bind addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:34:39.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: mon.a@-1(???) e1 preinit fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:34:39.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: mon.a@-1(???).mds e1 new map 2024-08-18T23:34:39.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: mon.a@-1(???).mds e1 print_map 2024-08-18T23:34:39.243 INFO:journalctl@ceph.mon.a.smithi078.stdout: e1 2024-08-18T23:34:39.243 INFO:journalctl@ceph.mon.a.smithi078.stdout: btime 2024-08-18T23:34:33:867054+0000 2024-08-18T23:34:39.243 INFO:journalctl@ceph.mon.a.smithi078.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-18T23:34:39.243 INFO:journalctl@ceph.mon.a.smithi078.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-18T23:34:39.243 INFO:journalctl@ceph.mon.a.smithi078.stdout: legacy client fscid: -1 2024-08-18T23:34:39.243 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-08-18T23:34:39.243 INFO:journalctl@ceph.mon.a.smithi078.stdout: No filesystems configured 2024-08-18T23:34:39.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-18T23:34:39.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-18T23:34:39.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-18T23:34:39.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-18T23:34:39.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:38 smithi078 ceph-mon[18440]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-08-18T23:34:39.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:39 smithi078 ceph-mon[18440]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-18T23:34:39.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:39 smithi078 ceph-mon[18440]: monmap epoch 1 2024-08-18T23:34:39.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:39 smithi078 ceph-mon[18440]: fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:34:39.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:39 smithi078 ceph-mon[18440]: last_changed 2024-08-18T23:34:23.952483+0000 2024-08-18T23:34:39.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:39 smithi078 ceph-mon[18440]: created 2024-08-18T23:34:23.952483+0000 2024-08-18T23:34:39.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:39 smithi078 ceph-mon[18440]: min_mon_release 19 (squid) 2024-08-18T23:34:39.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:39 smithi078 ceph-mon[18440]: election_strategy: 1 2024-08-18T23:34:39.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:39 smithi078 ceph-mon[18440]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-08-18T23:34:39.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:39 smithi078 ceph-mon[18440]: fsmap 2024-08-18T23:34:39.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:39 smithi078 ceph-mon[18440]: osdmap e1: 0 total, 0 up, 0 in 2024-08-18T23:34:39.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:39 smithi078 ceph-mon[18440]: mgrmap e1: no daemons active 2024-08-18T23:34:39.760 INFO:teuthology.orchestra.run.smithi078.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-08-18T23:34:40.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:39 smithi078 bash[18421]: c87427028777e7408705cd076ec10fe9fc3ec5d95b63378328b597d4805ea38a 2024-08-18T23:34:40.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:39 smithi078 systemd[1]: Started Ceph mon.a for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:34:43.292 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.674+0000 7f85a4747640 1 Processor -- start 2024-08-18T23:34:43.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.674+0000 7f85a4747640 1 -- start start 2024-08-18T23:34:43.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.675+0000 7f85a4747640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f859c106180 0x7f859c106580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:43.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.675+0000 7f85a4747640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f859c106b50 con 0x7f859c106180 2024-08-18T23:34:43.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.675+0000 7f85a24e3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f859c106180 0x7f859c106580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:43.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.675+0000 7f85a24e3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f859c106180 0x7f859c106580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39510/0 (socket says 172.21.15.78:39510) 2024-08-18T23:34:43.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.675+0000 7f85a24e3640 1 -- 172.21.15.78:0/3531123084 learned_addr learned my addr 172.21.15.78:0/3531123084 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:43.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.675+0000 7f85a24e3640 1 -- 172.21.15.78:0/3531123084 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859c107380 con 0x7f859c106180 2024-08-18T23:34:43.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.677+0000 7f85a24e3640 1 --2- 172.21.15.78:0/3531123084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f859c106180 0x7f859c106580 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f858c009820 tx=0x7f858c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a13689bf84c6eb server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:43.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.678+0000 7f85a14e1640 1 -- 172.21.15.78:0/3531123084 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f858c037070 con 0x7f859c106180 2024-08-18T23:34:43.293 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.678+0000 7f85a14e1640 1 -- 172.21.15.78:0/3531123084 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f858c032900 con 0x7f859c106180 2024-08-18T23:34:43.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.678+0000 7f85a14e1640 1 -- 172.21.15.78:0/3531123084 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f858c030850 con 0x7f859c106180 2024-08-18T23:34:43.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.678+0000 7f85a4747640 1 -- 172.21.15.78:0/3531123084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f859c106180 msgr2=0x7f859c106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:43.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.678+0000 7f85a4747640 1 --2- 172.21.15.78:0/3531123084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f859c106180 0x7f859c106580 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f858c009820 tx=0x7f858c02a270 comp rx=0 tx=0).stop 2024-08-18T23:34:43.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.678+0000 7f85a4747640 1 -- 172.21.15.78:0/3531123084 shutdown_connections 2024-08-18T23:34:43.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.678+0000 7f85a4747640 1 --2- 172.21.15.78:0/3531123084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f859c106180 0x7f859c106580 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:43.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.678+0000 7f85a4747640 1 -- 172.21.15.78:0/3531123084 >> 172.21.15.78:0/3531123084 conn(0x7f859c1018f0 msgr2=0x7f859c103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:43.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.678+0000 7f85a4747640 1 -- 172.21.15.78:0/3531123084 shutdown_connections 2024-08-18T23:34:43.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.678+0000 7f85a4747640 1 -- 172.21.15.78:0/3531123084 wait complete. 2024-08-18T23:34:43.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.679+0000 7f85a4747640 1 Processor -- start 2024-08-18T23:34:43.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.679+0000 7f85a4747640 1 -- start start 2024-08-18T23:34:43.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.679+0000 7f85a4747640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f859c106180 0x7f859c1913b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:43.294 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.679+0000 7f85a4747640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f859c10a730 con 0x7f859c106180 2024-08-18T23:34:43.295 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.679+0000 7f85a24e3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f859c106180 0x7f859c1913b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:43.295 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.679+0000 7f85a24e3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f859c106180 0x7f859c1913b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39514/0 (socket says 172.21.15.78:39514) 2024-08-18T23:34:43.295 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.679+0000 7f85a24e3640 1 -- 172.21.15.78:0/610917950 learned_addr learned my addr 172.21.15.78:0/610917950 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:43.295 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.680+0000 7f85a24e3640 1 -- 172.21.15.78:0/610917950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859c1918f0 con 0x7f859c106180 2024-08-18T23:34:43.295 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.680+0000 7f85a24e3640 1 --2- 172.21.15.78:0/610917950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f859c106180 0x7f859c1913b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f858c009940 tx=0x7f858c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:43.295 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.680+0000 7f85937fe640 1 -- 172.21.15.78:0/610917950 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f858c042070 con 0x7f859c106180 2024-08-18T23:34:43.295 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.680+0000 7f85937fe640 1 -- 172.21.15.78:0/610917950 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f858c032be0 con 0x7f859c106180 2024-08-18T23:34:43.295 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.680+0000 7f85a4747640 1 -- 172.21.15.78:0/610917950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f859c18fae0 con 0x7f859c106180 2024-08-18T23:34:43.295 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.680+0000 7f85937fe640 1 -- 172.21.15.78:0/610917950 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f858c037040 con 0x7f859c106180 2024-08-18T23:34:43.295 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.680+0000 7f85a4747640 1 -- 172.21.15.78:0/610917950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f859c190020 con 0x7f859c106180 2024-08-18T23:34:43.295 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.680+0000 7f85937fe640 1 -- 172.21.15.78:0/610917950 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f858c045070 con 0x7f859c106180 2024-08-18T23:34:43.295 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.680+0000 7f85937fe640 1 -- 172.21.15.78:0/610917950 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f858c0479f0 con 0x7f859c106180 2024-08-18T23:34:43.295 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.681+0000 7f85a4747640 1 -- 172.21.15.78:0/610917950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f859c10a8b0 con 0x7f859c106180 2024-08-18T23:34:43.296 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.683+0000 7f85937fe640 1 -- 172.21.15.78:0/610917950 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f858c032d90 con 0x7f859c106180 2024-08-18T23:34:43.296 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.740+0000 7f85a4747640 1 -- 172.21.15.78:0/610917950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f859c0682c0 con 0x7f859c106180 2024-08-18T23:34:43.296 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.741+0000 7f85937fe640 1 -- 172.21.15.78:0/610917950 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f858c03d430 con 0x7f859c106180 2024-08-18T23:34:43.296 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.742+0000 7f85a4747640 1 -- 172.21.15.78:0/610917950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f859c106180 msgr2=0x7f859c1913b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:43.296 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.742+0000 7f85a4747640 1 --2- 172.21.15.78:0/610917950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f859c106180 0x7f859c1913b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f858c009940 tx=0x7f858c0326e0 comp rx=0 tx=0).stop 2024-08-18T23:34:43.296 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.742+0000 7f85a4747640 1 -- 172.21.15.78:0/610917950 shutdown_connections 2024-08-18T23:34:43.296 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.742+0000 7f85a4747640 1 --2- 172.21.15.78:0/610917950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f859c106180 0x7f859c1913b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:43.296 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.742+0000 7f85a4747640 1 -- 172.21.15.78:0/610917950 >> 172.21.15.78:0/610917950 conn(0x7f859c1018f0 msgr2=0x7f859c1028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:43.296 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.742+0000 7f85a4747640 1 -- 172.21.15.78:0/610917950 shutdown_connections 2024-08-18T23:34:43.296 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:42.742+0000 7f85a4747640 1 -- 172.21.15.78:0/610917950 wait complete. 2024-08-18T23:34:43.296 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote config to /etc/ceph/ceph.conf 2024-08-18T23:34:43.319 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-08-18T23:34:43.319 INFO:teuthology.orchestra.run.smithi078.stdout:Creating mgr... 2024-08-18T23:34:43.319 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying port 0.0.0.0:9283 ... 2024-08-18T23:34:43.320 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying port 0.0.0.0:8765 ... 2024-08-18T23:34:43.608 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mgr.x 2024-08-18T23:34:43.608 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mgr.x.service: Unit ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mgr.x.service not loaded. 2024-08-18T23:34:43.754 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4f1128e6-5dba-11ef-bccf-c7b262605968.target.wants/ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@.service. 2024-08-18T23:34:43.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:43 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/610917950' entity='client.admin' 2024-08-18T23:34:43.969 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:43 smithi078 systemd[1]: Starting Ceph mgr.x for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:34:44.340 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:43 smithi078 podman[18656]: 2024-08-18 23:34:43.89512634 +0000 UTC m=+0.017293014 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 2024-08-18T23:34:44.341 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:44 smithi078 podman[18656]: 2024-08-18 23:34:44.011704977 +0000 UTC m=+0.133871650 container create 77b8af6f67c4b7ee1530806013871d533e9cbd45b6f327a51b56af57562fed2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0) 2024-08-18T23:34:44.341 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:44 smithi078 podman[18656]: 2024-08-18 23:34:44.223516 +0000 UTC m=+0.345682684 container init 77b8af6f67c4b7ee1530806013871d533e9cbd45b6f327a51b56af57562fed2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-18T23:34:44.341 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:44 smithi078 podman[18656]: 2024-08-18 23:34:44.226272299 +0000 UTC m=+0.348438973 container start 77b8af6f67c4b7ee1530806013871d533e9cbd45b6f327a51b56af57562fed2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0) 2024-08-18T23:34:44.341 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:44 smithi078 bash[18656]: 77b8af6f67c4b7ee1530806013871d533e9cbd45b6f327a51b56af57562fed2a 2024-08-18T23:34:44.354 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-18T23:34:44.354 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-08-18T23:34:44.362 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-18T23:34:44.362 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-08-18T23:34:44.362 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-08-18T23:34:44.362 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to enable service . firewalld.service is not available 2024-08-18T23:34:44.371 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-18T23:34:44.371 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-08-18T23:34:44.379 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-18T23:34:44.380 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-08-18T23:34:44.380 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-08-18T23:34:44.380 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-08-18T23:34:44.380 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr to start... 2024-08-18T23:34:44.380 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr... 2024-08-18T23:34:44.649 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:44 smithi078 systemd[1]: Started Ceph mgr.x for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:34:44.649 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:44 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:44.448+0000 7fb9fd1dd040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-18T23:34:44.649 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:44 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:44.514+0000 7fb9fd1dd040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-18T23:34:44.916 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:44 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:44.661+0000 7fb9fd1dd040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-18T23:34:44.916 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:44 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:44.822+0000 7fb9fd1dd040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-18T23:34:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:44 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2814172390' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-08-18T23:34:45.219 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:44 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:44.977+0000 7fb9fd1dd040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-18T23:34:45.219 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:45 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:45.101+0000 7fb9fd1dd040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-18T23:34:45.505 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:45 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:45.228+0000 7fb9fd1dd040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-18T23:34:45.505 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:45 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:45.286+0000 7fb9fd1dd040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-18T23:34:45.511 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-08-18T23:34:45.511 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-08-18T23:34:45.511 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "4f1128e6-5dba-11ef-bccf-c7b262605968", 2024-08-18T23:34:45.511 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-08-18T23:34:45.511 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-18T23:34:45.511 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-18T23:34:45.511 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-18T23:34:45.511 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:45.511 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-18T23:34:45.511 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-18T23:34:45.511 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-08-18T23:34:45.511 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-08-18T23:34:45.512 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-18T23:34:45.512 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-08-18T23:34:45.512 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-08-18T23:34:45.512 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-08-18T23:34:45.512 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-18T23:34:45.512 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-18T23:34:45.512 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-18T23:34:45.512 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-18T23:34:45.512 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:45.512 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-18T23:34:45.512 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-18T23:34:45.512 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-18T23:34:45.512 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-18T23:34:45.512 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-18T23:34:45.520 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-18T23:34:45.520 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-18T23:34:45.520 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-18T23:34:45.520 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:45.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-18T23:34:45.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-18T23:34:45.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-18T23:34:45.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-18T23:34:45.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-18T23:34:45.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-18T23:34:45.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-18T23:34:45.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-18T23:34:45.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-18T23:34:45.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:45.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-18T23:34:45.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-18T23:34:45.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "btime": "2024-08-18T23:34:33:867054+0000", 2024-08-18T23:34:45.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-18T23:34:45.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-18T23:34:45.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:45.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-18T23:34:45.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-18T23:34:45.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-18T23:34:45.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-18T23:34:45.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-18T23:34:45.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-18T23:34:45.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-08-18T23:34:45.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-08-18T23:34:45.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-18T23:34:45.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:45.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-18T23:34:45.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-18T23:34:45.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-08-18T23:34:33.867707+0000", 2024-08-18T23:34:45.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-18T23:34:45.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:45.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-18T23:34:45.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-08-18T23:34:45.527 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.846+0000 7f77cb8a2640 1 Processor -- start 2024-08-18T23:34:45.527 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.846+0000 7f77cb8a2640 1 -- start start 2024-08-18T23:34:45.527 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.846+0000 7f77cb8a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77c41055d0 0x7f77c41059d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:45.527 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.846+0000 7f77cb8a2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f77c4105fa0 con 0x7f77c41055d0 2024-08-18T23:34:45.527 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.847+0000 7f77c963e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77c41055d0 0x7f77c41059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:45.527 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.847+0000 7f77c963e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77c41055d0 0x7f77c41059d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39530/0 (socket says 172.21.15.78:39530) 2024-08-18T23:34:45.527 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.847+0000 7f77c963e640 1 -- 172.21.15.78:0/3862038723 learned_addr learned my addr 172.21.15.78:0/3862038723 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:45.527 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.847+0000 7f77c963e640 1 -- 172.21.15.78:0/3862038723 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c41085b0 con 0x7f77c41055d0 2024-08-18T23:34:45.527 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.848+0000 7f77c963e640 1 --2- 172.21.15.78:0/3862038723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77c41055d0 0x7f77c41059d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009820 tx=0x7f77c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9c41c786d673e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:45.528 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.848+0000 7f77bbfff640 1 -- 172.21.15.78:0/3862038723 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c0037070 con 0x7f77c41055d0 2024-08-18T23:34:45.528 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.848+0000 7f77bbfff640 1 -- 172.21.15.78:0/3862038723 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f77c0032900 con 0x7f77c41055d0 2024-08-18T23:34:45.528 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.848+0000 7f77bbfff640 1 -- 172.21.15.78:0/3862038723 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c0030850 con 0x7f77c41055d0 2024-08-18T23:34:45.528 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.849+0000 7f77cb8a2640 1 -- 172.21.15.78:0/3862038723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77c41055d0 msgr2=0x7f77c41059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:45.528 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.849+0000 7f77cb8a2640 1 --2- 172.21.15.78:0/3862038723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77c41055d0 0x7f77c41059d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009820 tx=0x7f77c002a270 comp rx=0 tx=0).stop 2024-08-18T23:34:45.528 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.849+0000 7f77cb8a2640 1 -- 172.21.15.78:0/3862038723 shutdown_connections 2024-08-18T23:34:45.528 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.849+0000 7f77cb8a2640 1 --2- 172.21.15.78:0/3862038723 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77c41055d0 0x7f77c41059d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:45.528 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.849+0000 7f77cb8a2640 1 -- 172.21.15.78:0/3862038723 >> 172.21.15.78:0/3862038723 conn(0x7f77c4101120 msgr2=0x7f77c4103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:45.528 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.849+0000 7f77cb8a2640 1 -- 172.21.15.78:0/3862038723 shutdown_connections 2024-08-18T23:34:45.528 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.849+0000 7f77cb8a2640 1 -- 172.21.15.78:0/3862038723 wait complete. 2024-08-18T23:34:45.528 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.850+0000 7f77cb8a2640 1 Processor -- start 2024-08-18T23:34:45.528 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.850+0000 7f77cb8a2640 1 -- start start 2024-08-18T23:34:45.529 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.850+0000 7f77cb8a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77c41055d0 0x7f77c419b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:45.529 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.850+0000 7f77cb8a2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f77c410b7e0 con 0x7f77c41055d0 2024-08-18T23:34:45.529 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.850+0000 7f77c963e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77c41055d0 0x7f77c419b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:45.529 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.850+0000 7f77c963e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77c41055d0 0x7f77c419b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39544/0 (socket says 172.21.15.78:39544) 2024-08-18T23:34:45.529 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.850+0000 7f77c963e640 1 -- 172.21.15.78:0/2814172390 learned_addr learned my addr 172.21.15.78:0/2814172390 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:45.529 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.850+0000 7f77c963e640 1 -- 172.21.15.78:0/2814172390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c419bbe0 con 0x7f77c41055d0 2024-08-18T23:34:45.529 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.850+0000 7f77c963e640 1 --2- 172.21.15.78:0/2814172390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77c41055d0 0x7f77c419b6a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009940 tx=0x7f77c00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:45.529 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.850+0000 7f77ba7fc640 1 -- 172.21.15.78:0/2814172390 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c0042070 con 0x7f77c41055d0 2024-08-18T23:34:45.529 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.850+0000 7f77ba7fc640 1 -- 172.21.15.78:0/2814172390 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f77c0032be0 con 0x7f77c41055d0 2024-08-18T23:34:45.529 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.850+0000 7f77ba7fc640 1 -- 172.21.15.78:0/2814172390 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c0037040 con 0x7f77c41055d0 2024-08-18T23:34:45.529 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.850+0000 7f77cb8a2640 1 -- 172.21.15.78:0/2814172390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c419be70 con 0x7f77c41055d0 2024-08-18T23:34:45.529 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.851+0000 7f77cb8a2640 1 -- 172.21.15.78:0/2814172390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c419eb60 con 0x7f77c41055d0 2024-08-18T23:34:45.530 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.851+0000 7f77ba7fc640 1 -- 172.21.15.78:0/2814172390 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f77c0045070 con 0x7f77c41055d0 2024-08-18T23:34:45.530 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.851+0000 7f77ba7fc640 1 -- 172.21.15.78:0/2814172390 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f77c0047a30 con 0x7f77c41055d0 2024-08-18T23:34:45.530 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.852+0000 7f77cb8a2640 1 -- 172.21.15.78:0/2814172390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c4105a50 con 0x7f77c41055d0 2024-08-18T23:34:45.530 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.854+0000 7f77ba7fc640 1 -- 172.21.15.78:0/2814172390 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f77c0032d90 con 0x7f77c41055d0 2024-08-18T23:34:45.530 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.914+0000 7f77cb8a2640 1 -- 172.21.15.78:0/2814172390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f77c41951c0 con 0x7f77c41055d0 2024-08-18T23:34:45.530 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.915+0000 7f77ba7fc640 1 -- 172.21.15.78:0/2814172390 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f77c003d430 con 0x7f77c41055d0 2024-08-18T23:34:45.530 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.916+0000 7f77cb8a2640 1 -- 172.21.15.78:0/2814172390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77c41055d0 msgr2=0x7f77c419b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:45.530 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.916+0000 7f77cb8a2640 1 --2- 172.21.15.78:0/2814172390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77c41055d0 0x7f77c419b6a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009940 tx=0x7f77c00326e0 comp rx=0 tx=0).stop 2024-08-18T23:34:45.530 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.916+0000 7f77cb8a2640 1 -- 172.21.15.78:0/2814172390 shutdown_connections 2024-08-18T23:34:45.530 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.916+0000 7f77cb8a2640 1 --2- 172.21.15.78:0/2814172390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77c41055d0 0x7f77c419b6a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:45.530 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.916+0000 7f77cb8a2640 1 -- 172.21.15.78:0/2814172390 >> 172.21.15.78:0/2814172390 conn(0x7f77c4101120 msgr2=0x7f77c4101f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:45.530 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.916+0000 7f77cb8a2640 1 -- 172.21.15.78:0/2814172390 shutdown_connections 2024-08-18T23:34:45.530 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:44.916+0000 7f77cb8a2640 1 -- 172.21.15.78:0/2814172390 wait complete. 2024-08-18T23:34:45.531 INFO:teuthology.orchestra.run.smithi078.stdout:mgr not available, waiting (1/15)... 2024-08-18T23:34:45.862 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:45 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:45.591+0000 7fb9fd1dd040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-18T23:34:45.862 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:45 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:45.650+0000 7fb9fd1dd040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-18T23:34:45.862 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:45 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:45.716+0000 7fb9fd1dd040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-18T23:34:45.863 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:45 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:45.861+0000 7fb9fd1dd040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-18T23:34:46.168 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:45 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:45.921+0000 7fb9fd1dd040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-18T23:34:46.169 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:45 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:45.986+0000 7fb9fd1dd040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-18T23:34:46.169 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:46 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:46.167+0000 7fb9fd1dd040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-18T23:34:46.470 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:46 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:46.228+0000 7fb9fd1dd040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-18T23:34:46.470 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:46 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:46.287+0000 7fb9fd1dd040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-18T23:34:46.471 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:46 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:46.353+0000 7fb9fd1dd040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-18T23:34:46.969 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:46 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:46.563+0000 7fb9fd1dd040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-18T23:34:46.969 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:46 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:46.621+0000 7fb9fd1dd040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-18T23:34:47.454 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:47 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:47.142+0000 7fb9fd1dd040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-18T23:34:47.719 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:47 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:47.452+0000 7fb9fd1dd040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-18T23:34:48.045 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:47 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:47.883+0000 7fb9fd1dd040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-18T23:34:48.045 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:47 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:47.943+0000 7fb9fd1dd040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-18T23:34:48.045 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:48 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:48.007+0000 7fb9fd1dd040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-18T23:34:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:48 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3312928148' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-08-18T23:34:48.649 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-08-18T23:34:48.650 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-08-18T23:34:48.650 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "4f1128e6-5dba-11ef-bccf-c7b262605968", 2024-08-18T23:34:48.650 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-08-18T23:34:48.650 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-18T23:34:48.650 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-18T23:34:48.650 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-18T23:34:48.650 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:48.650 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-18T23:34:48.650 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-18T23:34:48.650 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-08-18T23:34:48.650 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-08-18T23:34:48.650 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-18T23:34:48.650 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-08-18T23:34:48.651 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-08-18T23:34:48.651 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-08-18T23:34:48.651 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-18T23:34:48.651 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-18T23:34:48.651 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-18T23:34:48.651 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-18T23:34:48.651 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:48.651 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-18T23:34:48.651 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-18T23:34:48.651 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-18T23:34:48.651 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-18T23:34:48.651 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-18T23:34:48.651 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-18T23:34:48.652 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-18T23:34:48.652 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-18T23:34:48.652 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:48.652 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-18T23:34:48.652 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-18T23:34:48.652 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-18T23:34:48.652 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-18T23:34:48.652 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-18T23:34:48.652 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-18T23:34:48.652 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-18T23:34:48.652 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-18T23:34:48.652 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-18T23:34:48.652 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:48.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-18T23:34:48.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-18T23:34:48.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "btime": "2024-08-18T23:34:33:867054+0000", 2024-08-18T23:34:48.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-18T23:34:48.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-18T23:34:48.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:48.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-18T23:34:48.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-18T23:34:48.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-18T23:34:48.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-18T23:34:48.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-18T23:34:48.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-18T23:34:48.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-08-18T23:34:48.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-08-18T23:34:48.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-18T23:34:48.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:48.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-18T23:34:48.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-18T23:34:48.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-08-18T23:34:33.867707+0000", 2024-08-18T23:34:48.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-18T23:34:48.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:48.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-18T23:34:48.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-08-18T23:34:48.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.924+0000 7f7ea4a37640 1 Processor -- start 2024-08-18T23:34:48.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.924+0000 7f7ea4a37640 1 -- start start 2024-08-18T23:34:48.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.925+0000 7f7ea4a37640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea0107d40 0x7f7ea010a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:48.655 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.925+0000 7f7ea4a37640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea007be10 con 0x7f7ea0107d40 2024-08-18T23:34:48.655 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.925+0000 7f7e9e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea0107d40 0x7f7ea010a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:48.655 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.925+0000 7f7e9e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea0107d40 0x7f7ea010a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59278/0 (socket says 172.21.15.78:59278) 2024-08-18T23:34:48.655 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.925+0000 7f7e9e59c640 1 -- 172.21.15.78:0/3577844969 learned_addr learned my addr 172.21.15.78:0/3577844969 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:48.655 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.925+0000 7f7e9e59c640 1 -- 172.21.15.78:0/3577844969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea010a670 con 0x7f7ea0107d40 2024-08-18T23:34:48.655 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.926+0000 7f7e9e59c640 1 --2- 172.21.15.78:0/3577844969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea0107d40 0x7f7ea010a130 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7e90009820 tx=0x7f7e9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26143fa75e6d85b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:48.655 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.926+0000 7f7e9d59a640 1 -- 172.21.15.78:0/3577844969 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e90037070 con 0x7f7ea0107d40 2024-08-18T23:34:48.655 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.926+0000 7f7e9d59a640 1 -- 172.21.15.78:0/3577844969 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7e90032900 con 0x7f7ea0107d40 2024-08-18T23:34:48.655 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.926+0000 7f7e9d59a640 1 -- 172.21.15.78:0/3577844969 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e90030850 con 0x7f7ea0107d40 2024-08-18T23:34:48.655 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.927+0000 7f7ea4a37640 1 -- 172.21.15.78:0/3577844969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea0107d40 msgr2=0x7f7ea010a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:48.655 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.927+0000 7f7ea4a37640 1 --2- 172.21.15.78:0/3577844969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea0107d40 0x7f7ea010a130 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7e90009820 tx=0x7f7e9002a270 comp rx=0 tx=0).stop 2024-08-18T23:34:48.655 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.927+0000 7f7ea4a37640 1 -- 172.21.15.78:0/3577844969 shutdown_connections 2024-08-18T23:34:48.656 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.927+0000 7f7ea4a37640 1 --2- 172.21.15.78:0/3577844969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea0107d40 0x7f7ea010a130 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:48.656 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.927+0000 7f7ea4a37640 1 -- 172.21.15.78:0/3577844969 >> 172.21.15.78:0/3577844969 conn(0x7f7ea01019e0 msgr2=0x7f7ea0103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:48.656 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.927+0000 7f7ea4a37640 1 -- 172.21.15.78:0/3577844969 shutdown_connections 2024-08-18T23:34:48.656 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.927+0000 7f7ea4a37640 1 -- 172.21.15.78:0/3577844969 wait complete. 2024-08-18T23:34:48.656 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.928+0000 7f7ea4a37640 1 Processor -- start 2024-08-18T23:34:48.656 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.928+0000 7f7ea4a37640 1 -- start start 2024-08-18T23:34:48.656 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.928+0000 7f7ea4a37640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea0107d40 0x7f7ea019b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:48.656 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.928+0000 7f7ea4a37640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea010b6b0 con 0x7f7ea0107d40 2024-08-18T23:34:48.656 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.929+0000 7f7e9e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea0107d40 0x7f7ea019b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:48.656 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.929+0000 7f7e9e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea0107d40 0x7f7ea019b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59280/0 (socket says 172.21.15.78:59280) 2024-08-18T23:34:48.656 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.929+0000 7f7e9e59c640 1 -- 172.21.15.78:0/3312928148 learned_addr learned my addr 172.21.15.78:0/3312928148 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:48.656 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.929+0000 7f7e9e59c640 1 -- 172.21.15.78:0/3312928148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea019bb20 con 0x7f7ea0107d40 2024-08-18T23:34:48.656 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.929+0000 7f7e9e59c640 1 --2- 172.21.15.78:0/3312928148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea0107d40 0x7f7ea019b5e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7e900097f0 tx=0x7f7e90030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:48.657 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.929+0000 7f7e8f7fe640 1 -- 172.21.15.78:0/3312928148 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e90042070 con 0x7f7ea0107d40 2024-08-18T23:34:48.657 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.929+0000 7f7ea4a37640 1 -- 172.21.15.78:0/3312928148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea019bdb0 con 0x7f7ea0107d40 2024-08-18T23:34:48.657 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.929+0000 7f7e8f7fe640 1 -- 172.21.15.78:0/3312928148 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7e90032ca0 con 0x7f7ea0107d40 2024-08-18T23:34:48.657 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.929+0000 7f7e8f7fe640 1 -- 172.21.15.78:0/3312928148 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e90037040 con 0x7f7ea0107d40 2024-08-18T23:34:48.657 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.929+0000 7f7ea4a37640 1 -- 172.21.15.78:0/3312928148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea019eaa0 con 0x7f7ea0107d40 2024-08-18T23:34:48.657 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.930+0000 7f7e8f7fe640 1 -- 172.21.15.78:0/3312928148 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7e90045070 con 0x7f7ea0107d40 2024-08-18T23:34:48.657 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.930+0000 7f7e8f7fe640 1 -- 172.21.15.78:0/3312928148 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7e90047a90 con 0x7f7ea0107d40 2024-08-18T23:34:48.657 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.930+0000 7f7ea4a37640 1 -- 172.21.15.78:0/3312928148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea01081a0 con 0x7f7ea0107d40 2024-08-18T23:34:48.657 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.932+0000 7f7e8f7fe640 1 -- 172.21.15.78:0/3312928148 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f7e9003db10 con 0x7f7ea0107d40 2024-08-18T23:34:48.657 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.992+0000 7f7ea4a37640 1 -- 172.21.15.78:0/3312928148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7ea002d5e0 con 0x7f7ea0107d40 2024-08-18T23:34:48.657 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.993+0000 7f7e8f7fe640 1 -- 172.21.15.78:0/3312928148 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f7e9003d430 con 0x7f7ea0107d40 2024-08-18T23:34:48.657 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.993+0000 7f7ea4a37640 1 -- 172.21.15.78:0/3312928148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea0107d40 msgr2=0x7f7ea019b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:48.657 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.993+0000 7f7ea4a37640 1 --2- 172.21.15.78:0/3312928148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea0107d40 0x7f7ea019b5e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7e900097f0 tx=0x7f7e90030ec0 comp rx=0 tx=0).stop 2024-08-18T23:34:48.658 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.994+0000 7f7ea4a37640 1 -- 172.21.15.78:0/3312928148 shutdown_connections 2024-08-18T23:34:48.658 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.994+0000 7f7ea4a37640 1 --2- 172.21.15.78:0/3312928148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea0107d40 0x7f7ea019b5e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:48.658 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.994+0000 7f7ea4a37640 1 -- 172.21.15.78:0/3312928148 >> 172.21.15.78:0/3312928148 conn(0x7f7ea01019e0 msgr2=0x7f7ea01027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:48.658 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.994+0000 7f7ea4a37640 1 -- 172.21.15.78:0/3312928148 shutdown_connections 2024-08-18T23:34:48.658 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:47.994+0000 7f7ea4a37640 1 -- 172.21.15.78:0/3312928148 wait complete. 2024-08-18T23:34:48.658 INFO:teuthology.orchestra.run.smithi078.stdout:mgr not available, waiting (2/15)... 2024-08-18T23:34:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:49 smithi078 ceph-mon[18440]: Activating manager daemon x 2024-08-18T23:34:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:49 smithi078 ceph-mon[18440]: mgrmap e2: x(active, starting, since 0.00278128s) 2024-08-18T23:34:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:49 smithi078 ceph-mon[18440]: from='mgr.14100 172.21.15.78:0/3033211972' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-18T23:34:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:49 smithi078 ceph-mon[18440]: from='mgr.14100 172.21.15.78:0/3033211972' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-18T23:34:49.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:49 smithi078 ceph-mon[18440]: from='mgr.14100 172.21.15.78:0/3033211972' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-18T23:34:49.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:49 smithi078 ceph-mon[18440]: from='mgr.14100 172.21.15.78:0/3033211972' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-18T23:34:49.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:49 smithi078 ceph-mon[18440]: from='mgr.14100 172.21.15.78:0/3033211972' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-18T23:34:49.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:49 smithi078 ceph-mon[18440]: Manager daemon x is now available 2024-08-18T23:34:49.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:49 smithi078 ceph-mon[18440]: from='mgr.14100 172.21.15.78:0/3033211972' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-18T23:34:49.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:49 smithi078 ceph-mon[18440]: from='mgr.14100 172.21.15.78:0/3033211972' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-18T23:34:49.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:49 smithi078 ceph-mon[18440]: from='mgr.14100 172.21.15.78:0/3033211972' entity='mgr.x' 2024-08-18T23:34:49.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:49 smithi078 ceph-mon[18440]: from='mgr.14100 172.21.15.78:0/3033211972' entity='mgr.x' 2024-08-18T23:34:49.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:49 smithi078 ceph-mon[18440]: from='mgr.14100 172.21.15.78:0/3033211972' entity='mgr.x' 2024-08-18T23:34:51.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:51 smithi078 ceph-mon[18440]: mgrmap e3: x(active, since 1.00697s) 2024-08-18T23:34:51.846 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-08-18T23:34:51.847 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-08-18T23:34:51.847 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "4f1128e6-5dba-11ef-bccf-c7b262605968", 2024-08-18T23:34:51.847 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-08-18T23:34:51.847 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-18T23:34:51.847 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-18T23:34:51.847 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-18T23:34:51.847 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:51.847 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-18T23:34:51.847 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-18T23:34:51.847 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-08-18T23:34:51.847 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-08-18T23:34:51.848 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-18T23:34:51.848 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-08-18T23:34:51.848 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-08-18T23:34:51.848 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-08-18T23:34:51.848 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-18T23:34:51.848 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-18T23:34:51.848 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-18T23:34:51.848 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-18T23:34:51.848 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:51.848 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-18T23:34:51.848 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-18T23:34:51.848 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-18T23:34:51.849 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-18T23:34:51.849 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-18T23:34:51.849 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-18T23:34:51.849 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-18T23:34:51.849 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-18T23:34:51.849 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:51.849 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-18T23:34:51.849 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-18T23:34:51.849 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-18T23:34:51.849 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-18T23:34:51.849 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-18T23:34:51.849 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-18T23:34:51.849 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-18T23:34:51.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-18T23:34:51.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-18T23:34:51.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:51.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-18T23:34:51.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-18T23:34:51.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "btime": "2024-08-18T23:34:33:867054+0000", 2024-08-18T23:34:51.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-18T23:34:51.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-18T23:34:51.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:51.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-18T23:34:51.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-18T23:34:51.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-18T23:34:51.850 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-18T23:34:51.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-18T23:34:51.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-18T23:34:51.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-08-18T23:34:51.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-08-18T23:34:51.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-18T23:34:51.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:51.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-18T23:34:51.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-18T23:34:51.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-08-18T23:34:33.867707+0000", 2024-08-18T23:34:51.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-18T23:34:51.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-08-18T23:34:51.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-18T23:34:51.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-08-18T23:34:51.851 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:50.996+0000 7f1f44f42640 1 Processor -- start 2024-08-18T23:34:51.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:50.996+0000 7f1f44f42640 1 -- start start 2024-08-18T23:34:51.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:50.997+0000 7f1f44f42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f40107380 0x7f1f40107780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:51.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:50.997+0000 7f1f44f42640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1f40107d50 con 0x7f1f40107380 2024-08-18T23:34:51.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:50.997+0000 7f1f3e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f40107380 0x7f1f40107780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:51.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:50.997+0000 7f1f3e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f40107380 0x7f1f40107780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59364/0 (socket says 172.21.15.78:59364) 2024-08-18T23:34:51.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:50.997+0000 7f1f3e59c640 1 -- 172.21.15.78:0/3893118385 learned_addr learned my addr 172.21.15.78:0/3893118385 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:51.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:50.997+0000 7f1f3e59c640 1 -- 172.21.15.78:0/3893118385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f401084e0 con 0x7f1f40107380 2024-08-18T23:34:51.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:50.998+0000 7f1f3e59c640 1 --2- 172.21.15.78:0/3893118385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f40107380 0x7f1f40107780 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009820 tx=0x7f1f3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813132b8052f2140 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:51.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:50.998+0000 7f1f3d59a640 1 -- 172.21.15.78:0/3893118385 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f30037070 con 0x7f1f40107380 2024-08-18T23:34:51.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:50.998+0000 7f1f3d59a640 1 -- 172.21.15.78:0/3893118385 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1f30032900 con 0x7f1f40107380 2024-08-18T23:34:51.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:50.998+0000 7f1f3d59a640 1 -- 172.21.15.78:0/3893118385 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f30030850 con 0x7f1f40107380 2024-08-18T23:34:51.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:50.999+0000 7f1f44f42640 1 -- 172.21.15.78:0/3893118385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f40107380 msgr2=0x7f1f40107780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:51.852 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:50.999+0000 7f1f44f42640 1 --2- 172.21.15.78:0/3893118385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f40107380 0x7f1f40107780 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009820 tx=0x7f1f3002a270 comp rx=0 tx=0).stop 2024-08-18T23:34:51.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:50.999+0000 7f1f44f42640 1 -- 172.21.15.78:0/3893118385 shutdown_connections 2024-08-18T23:34:51.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:50.999+0000 7f1f44f42640 1 --2- 172.21.15.78:0/3893118385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f40107380 0x7f1f40107780 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:51.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:50.999+0000 7f1f44f42640 1 -- 172.21.15.78:0/3893118385 >> 172.21.15.78:0/3893118385 conn(0x7f1f40102b30 msgr2=0x7f1f40104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:51.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:50.999+0000 7f1f44f42640 1 -- 172.21.15.78:0/3893118385 shutdown_connections 2024-08-18T23:34:51.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:50.999+0000 7f1f44f42640 1 -- 172.21.15.78:0/3893118385 wait complete. 2024-08-18T23:34:51.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.000+0000 7f1f44f42640 1 Processor -- start 2024-08-18T23:34:51.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.000+0000 7f1f44f42640 1 -- start start 2024-08-18T23:34:51.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.000+0000 7f1f44f42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f40107380 0x7f1f4019b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:51.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.000+0000 7f1f44f42640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4010b890 con 0x7f1f40107380 2024-08-18T23:34:51.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.000+0000 7f1f3e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f40107380 0x7f1f4019b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:51.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.000+0000 7f1f3e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f40107380 0x7f1f4019b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59368/0 (socket says 172.21.15.78:59368) 2024-08-18T23:34:51.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.000+0000 7f1f3e59c640 1 -- 172.21.15.78:0/3428435816 learned_addr learned my addr 172.21.15.78:0/3428435816 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:51.853 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.001+0000 7f1f3e59c640 1 -- 172.21.15.78:0/3428435816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4019bbe0 con 0x7f1f40107380 2024-08-18T23:34:51.854 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.001+0000 7f1f3e59c640 1 --2- 172.21.15.78:0/3428435816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f40107380 0x7f1f4019b6a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1f300097f0 tx=0x7f1f30030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:51.854 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.001+0000 7f1f377fe640 1 -- 172.21.15.78:0/3428435816 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f30042070 con 0x7f1f40107380 2024-08-18T23:34:51.854 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.001+0000 7f1f377fe640 1 -- 172.21.15.78:0/3428435816 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1f30032ca0 con 0x7f1f40107380 2024-08-18T23:34:51.854 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.001+0000 7f1f377fe640 1 -- 172.21.15.78:0/3428435816 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f30037040 con 0x7f1f40107380 2024-08-18T23:34:51.854 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.001+0000 7f1f44f42640 1 -- 172.21.15.78:0/3428435816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4019be70 con 0x7f1f40107380 2024-08-18T23:34:51.854 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.001+0000 7f1f44f42640 1 -- 172.21.15.78:0/3428435816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f4019eb60 con 0x7f1f40107380 2024-08-18T23:34:51.854 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.002+0000 7f1f377fe640 1 -- 172.21.15.78:0/3428435816 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f1f30045070 con 0x7f1f40107380 2024-08-18T23:34:51.854 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.002+0000 7f1f377fe640 1 --2- 172.21.15.78:0/3428435816 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f1f2c038e60 0x7f1f2c03b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:51.854 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.002+0000 7f1f377fe640 1 -- 172.21.15.78:0/3428435816 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1f3006bdb0 con 0x7f1f40107380 2024-08-18T23:34:51.854 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.002+0000 7f1f44f42640 1 -- 172.21.15.78:0/3428435816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f4010ba10 con 0x7f1f40107380 2024-08-18T23:34:51.854 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.002+0000 7f1f3dd9b640 1 --2- 172.21.15.78:0/3428435816 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f1f2c038e60 0x7f1f2c03b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:51.854 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.003+0000 7f1f3dd9b640 1 --2- 172.21.15.78:0/3428435816 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f1f2c038e60 0x7f1f2c03b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1f280098c0 tx=0x7f1f28006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:51.854 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.007+0000 7f1f377fe640 1 -- 172.21.15.78:0/3428435816 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1f3003add0 con 0x7f1f40107380 2024-08-18T23:34:51.855 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.110+0000 7f1f377fe640 1 -- 172.21.15.78:0/3428435816 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f1f3003c6a0 con 0x7f1f40107380 2024-08-18T23:34:51.855 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.238+0000 7f1f44f42640 1 -- 172.21.15.78:0/3428435816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1f4002d5e0 con 0x7f1f40107380 2024-08-18T23:34:51.855 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.239+0000 7f1f377fe640 1 -- 172.21.15.78:0/3428435816 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f1f30047440 con 0x7f1f40107380 2024-08-18T23:34:51.855 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.241+0000 7f1f44f42640 1 -- 172.21.15.78:0/3428435816 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f1f2c038e60 msgr2=0x7f1f2c03b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:51.855 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.241+0000 7f1f44f42640 1 --2- 172.21.15.78:0/3428435816 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f1f2c038e60 0x7f1f2c03b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1f280098c0 tx=0x7f1f28006db0 comp rx=0 tx=0).stop 2024-08-18T23:34:51.855 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.241+0000 7f1f44f42640 1 -- 172.21.15.78:0/3428435816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f40107380 msgr2=0x7f1f4019b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:51.855 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.241+0000 7f1f44f42640 1 --2- 172.21.15.78:0/3428435816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f40107380 0x7f1f4019b6a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1f300097f0 tx=0x7f1f30030ec0 comp rx=0 tx=0).stop 2024-08-18T23:34:51.855 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.241+0000 7f1f44f42640 1 -- 172.21.15.78:0/3428435816 shutdown_connections 2024-08-18T23:34:51.855 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.241+0000 7f1f44f42640 1 --2- 172.21.15.78:0/3428435816 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f1f2c038e60 0x7f1f2c03b320 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:51.855 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.241+0000 7f1f44f42640 1 --2- 172.21.15.78:0/3428435816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f40107380 0x7f1f4019b6a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:51.855 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.241+0000 7f1f44f42640 1 -- 172.21.15.78:0/3428435816 >> 172.21.15.78:0/3428435816 conn(0x7f1f40102b30 msgr2=0x7f1f401038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:51.855 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.241+0000 7f1f44f42640 1 -- 172.21.15.78:0/3428435816 shutdown_connections 2024-08-18T23:34:51.855 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:51.241+0000 7f1f44f42640 1 -- 172.21.15.78:0/3428435816 wait complete. 2024-08-18T23:34:51.855 INFO:teuthology.orchestra.run.smithi078.stdout:mgr is available 2024-08-18T23:34:52.364 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:52 smithi078 ceph-mon[18440]: mgrmap e4: x(active, since 2s) 2024-08-18T23:34:52.364 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:52 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3428435816' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-08-18T23:34:52.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-08-18T23:34:52.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [global] 2024-08-18T23:34:52.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout fsid = 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:34:52.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.78:3300,v1:172.21.15.78:6789] 2024-08-18T23:34:52.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-18T23:34:52.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-18T23:34:52.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-18T23:34:52.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-18T23:34:52.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-08-18T23:34:52.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-18T23:34:52.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-18T23:34:52.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-08-18T23:34:52.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [osd] 2024-08-18T23:34:52.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-18T23:34:52.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-18T23:34:52.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-18T23:34:52.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-18T23:34:52.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-18T23:34:52.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.187+0000 7f459978a640 1 Processor -- start 2024-08-18T23:34:52.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.187+0000 7f459978a640 1 -- start start 2024-08-18T23:34:52.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.187+0000 7f459978a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4594107d80 0x7f459410a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:52.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.187+0000 7f459978a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f459407bdd0 con 0x7f4594107d80 2024-08-18T23:34:52.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.187+0000 7f4592ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4594107d80 0x7f459410a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:52.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.187+0000 7f4592ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4594107d80 0x7f459410a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59382/0 (socket says 172.21.15.78:59382) 2024-08-18T23:34:52.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.187+0000 7f4592ffd640 1 -- 172.21.15.78:0/425558458 learned_addr learned my addr 172.21.15.78:0/425558458 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:52.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.187+0000 7f4592ffd640 1 -- 172.21.15.78:0/425558458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459410a6b0 con 0x7f4594107d80 2024-08-18T23:34:52.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.188+0000 7f4592ffd640 1 --2- 172.21.15.78:0/425558458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4594107d80 0x7f459410a170 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4584009820 tx=0x7f458402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a68023f9b3825b6 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:52.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.188+0000 7f45927fc640 1 -- 172.21.15.78:0/425558458 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4584037070 con 0x7f4594107d80 2024-08-18T23:34:52.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.188+0000 7f45927fc640 1 -- 172.21.15.78:0/425558458 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4584032900 con 0x7f4594107d80 2024-08-18T23:34:52.901 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.188+0000 7f45927fc640 1 -- 172.21.15.78:0/425558458 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4584030850 con 0x7f4594107d80 2024-08-18T23:34:52.902 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.189+0000 7f459978a640 1 -- 172.21.15.78:0/425558458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4594107d80 msgr2=0x7f459410a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:52.902 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.189+0000 7f459978a640 1 --2- 172.21.15.78:0/425558458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4594107d80 0x7f459410a170 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4584009820 tx=0x7f458402a270 comp rx=0 tx=0).stop 2024-08-18T23:34:52.902 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.189+0000 7f459978a640 1 -- 172.21.15.78:0/425558458 shutdown_connections 2024-08-18T23:34:52.902 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.189+0000 7f459978a640 1 --2- 172.21.15.78:0/425558458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4594107d80 0x7f459410a170 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:52.902 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.189+0000 7f459978a640 1 -- 172.21.15.78:0/425558458 >> 172.21.15.78:0/425558458 conn(0x7f45941019e0 msgr2=0x7f4594103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:52.902 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.189+0000 7f459978a640 1 -- 172.21.15.78:0/425558458 shutdown_connections 2024-08-18T23:34:52.902 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.189+0000 7f459978a640 1 -- 172.21.15.78:0/425558458 wait complete. 2024-08-18T23:34:52.902 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.189+0000 7f459978a640 1 Processor -- start 2024-08-18T23:34:52.902 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.190+0000 7f459978a640 1 -- start start 2024-08-18T23:34:52.902 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.190+0000 7f459978a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4594107d80 0x7f4594107600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:52.902 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.190+0000 7f459978a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f459410b690 con 0x7f4594107d80 2024-08-18T23:34:52.902 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.190+0000 7f4592ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4594107d80 0x7f4594107600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:52.902 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.190+0000 7f4592ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4594107d80 0x7f4594107600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59392/0 (socket says 172.21.15.78:59392) 2024-08-18T23:34:52.903 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.190+0000 7f4592ffd640 1 -- 172.21.15.78:0/935941425 learned_addr learned my addr 172.21.15.78:0/935941425 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:52.903 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.190+0000 7f4592ffd640 1 -- 172.21.15.78:0/935941425 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4594107b40 con 0x7f4594107d80 2024-08-18T23:34:52.903 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.190+0000 7f4592ffd640 1 --2- 172.21.15.78:0/935941425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4594107d80 0x7f4594107600 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4584031000 tx=0x7f45840326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:52.903 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.190+0000 7f4590ff9640 1 -- 172.21.15.78:0/935941425 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4584042070 con 0x7f4594107d80 2024-08-18T23:34:52.903 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.190+0000 7f4590ff9640 1 -- 172.21.15.78:0/935941425 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4584032be0 con 0x7f4594107d80 2024-08-18T23:34:52.903 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.190+0000 7f459978a640 1 -- 172.21.15.78:0/935941425 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4594105d30 con 0x7f4594107d80 2024-08-18T23:34:52.903 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.190+0000 7f4590ff9640 1 -- 172.21.15.78:0/935941425 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4584037040 con 0x7f4594107d80 2024-08-18T23:34:52.903 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.190+0000 7f459978a640 1 -- 172.21.15.78:0/935941425 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4594106270 con 0x7f4594107d80 2024-08-18T23:34:52.903 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.191+0000 7f4590ff9640 1 -- 172.21.15.78:0/935941425 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f4584045070 con 0x7f4594107d80 2024-08-18T23:34:52.903 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.191+0000 7f4590ff9640 1 --2- 172.21.15.78:0/935941425 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f4580038f20 0x7f458003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:52.903 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.191+0000 7f459978a640 1 -- 172.21.15.78:0/935941425 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45941081e0 con 0x7f4594107d80 2024-08-18T23:34:52.903 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.191+0000 7f4590ff9640 1 -- 172.21.15.78:0/935941425 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f458406c120 con 0x7f4594107d80 2024-08-18T23:34:52.903 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.192+0000 7f458bfff640 1 --2- 172.21.15.78:0/935941425 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f4580038f20 0x7f458003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:52.904 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.192+0000 7f458bfff640 1 --2- 172.21.15.78:0/935941425 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f4580038f20 0x7f458003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f457c0098c0 tx=0x7f457c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:52.904 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.196+0000 7f4590ff9640 1 -- 172.21.15.78:0/935941425 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f458404c030 con 0x7f4594107d80 2024-08-18T23:34:52.904 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.363+0000 7f459978a640 1 -- 172.21.15.78:0/935941425 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f4594194d00 con 0x7f4594107d80 2024-08-18T23:34:52.904 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.364+0000 7f4590ff9640 1 -- 172.21.15.78:0/935941425 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f4584042210 con 0x7f4594107d80 2024-08-18T23:34:52.904 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.366+0000 7f459978a640 1 -- 172.21.15.78:0/935941425 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f4580038f20 msgr2=0x7f458003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:52.904 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.366+0000 7f459978a640 1 --2- 172.21.15.78:0/935941425 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f4580038f20 0x7f458003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f457c0098c0 tx=0x7f457c006db0 comp rx=0 tx=0).stop 2024-08-18T23:34:52.904 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.366+0000 7f459978a640 1 -- 172.21.15.78:0/935941425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4594107d80 msgr2=0x7f4594107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:52.904 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.366+0000 7f459978a640 1 --2- 172.21.15.78:0/935941425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4594107d80 0x7f4594107600 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4584031000 tx=0x7f45840326e0 comp rx=0 tx=0).stop 2024-08-18T23:34:52.904 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.366+0000 7f459978a640 1 -- 172.21.15.78:0/935941425 shutdown_connections 2024-08-18T23:34:52.904 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.366+0000 7f459978a640 1 --2- 172.21.15.78:0/935941425 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f4580038f20 0x7f458003b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:52.904 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.366+0000 7f459978a640 1 --2- 172.21.15.78:0/935941425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4594107d80 0x7f4594107600 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:52.904 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.366+0000 7f459978a640 1 -- 172.21.15.78:0/935941425 >> 172.21.15.78:0/935941425 conn(0x7f45941019e0 msgr2=0x7f4594102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:52.904 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.366+0000 7f459978a640 1 -- 172.21.15.78:0/935941425 shutdown_connections 2024-08-18T23:34:52.904 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:52.366+0000 7f459978a640 1 -- 172.21.15.78:0/935941425 wait complete. 2024-08-18T23:34:52.905 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling cephadm module... 2024-08-18T23:34:53.266 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:53 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/935941425' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-08-18T23:34:54.418 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:54 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/359706962' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-08-18T23:34:54.418 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:54 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: ignoring --setuser ceph since I am not root 2024-08-18T23:34:54.418 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:54 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: ignoring --setgroup ceph since I am not root 2024-08-18T23:34:54.418 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:54 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:54.350+0000 7f2a4104f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-18T23:34:54.418 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:54 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:54.416+0000 7f2a4104f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-18T23:34:54.696 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:54 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:54.561+0000 7f2a4104f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-18T23:34:54.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.262+0000 7f381e070640 1 Processor -- start 2024-08-18T23:34:54.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.262+0000 7f381e070640 1 -- start start 2024-08-18T23:34:54.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.263+0000 7f381e070640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3818106180 0x7f3818106580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:54.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.263+0000 7f381e070640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3818106b50 con 0x7f3818106180 2024-08-18T23:34:54.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.263+0000 7f38177fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3818106180 0x7f3818106580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:54.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.263+0000 7f38177fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3818106180 0x7f3818106580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59398/0 (socket says 172.21.15.78:59398) 2024-08-18T23:34:54.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.263+0000 7f38177fe640 1 -- 172.21.15.78:0/397068120 learned_addr learned my addr 172.21.15.78:0/397068120 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:54.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.263+0000 7f38177fe640 1 -- 172.21.15.78:0/397068120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3818107380 con 0x7f3818106180 2024-08-18T23:34:54.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.264+0000 7f38177fe640 1 --2- 172.21.15.78:0/397068120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3818106180 0x7f3818106580 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3808009820 tx=0x7f380802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184f0a0cf8fde958 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:54.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.264+0000 7f3816ffd640 1 -- 172.21.15.78:0/397068120 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3808037070 con 0x7f3818106180 2024-08-18T23:34:54.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.264+0000 7f3816ffd640 1 -- 172.21.15.78:0/397068120 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3808032900 con 0x7f3818106180 2024-08-18T23:34:54.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.264+0000 7f3816ffd640 1 -- 172.21.15.78:0/397068120 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3808030850 con 0x7f3818106180 2024-08-18T23:34:54.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.264+0000 7f381e070640 1 -- 172.21.15.78:0/397068120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3818106180 msgr2=0x7f3818106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:54.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.264+0000 7f381e070640 1 --2- 172.21.15.78:0/397068120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3818106180 0x7f3818106580 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3808009820 tx=0x7f380802a270 comp rx=0 tx=0).stop 2024-08-18T23:34:54.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.264+0000 7f381e070640 1 -- 172.21.15.78:0/397068120 shutdown_connections 2024-08-18T23:34:54.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.264+0000 7f381e070640 1 --2- 172.21.15.78:0/397068120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3818106180 0x7f3818106580 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:54.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.264+0000 7f381e070640 1 -- 172.21.15.78:0/397068120 >> 172.21.15.78:0/397068120 conn(0x7f38181018f0 msgr2=0x7f3818103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:54.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.265+0000 7f381e070640 1 -- 172.21.15.78:0/397068120 shutdown_connections 2024-08-18T23:34:54.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.265+0000 7f381e070640 1 -- 172.21.15.78:0/397068120 wait complete. 2024-08-18T23:34:54.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.265+0000 7f381e070640 1 Processor -- start 2024-08-18T23:34:54.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.265+0000 7f381e070640 1 -- start start 2024-08-18T23:34:54.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.265+0000 7f381e070640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3818106180 0x7f3818075150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:54.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.265+0000 7f381e070640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f381810a730 con 0x7f3818106180 2024-08-18T23:34:54.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.266+0000 7f38177fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3818106180 0x7f3818075150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:54.706 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.266+0000 7f38177fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3818106180 0x7f3818075150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59400/0 (socket says 172.21.15.78:59400) 2024-08-18T23:34:54.706 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.266+0000 7f38177fe640 1 -- 172.21.15.78:0/359706962 learned_addr learned my addr 172.21.15.78:0/359706962 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:54.706 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.266+0000 7f38177fe640 1 -- 172.21.15.78:0/359706962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381819e360 con 0x7f3818106180 2024-08-18T23:34:54.706 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.266+0000 7f38177fe640 1 --2- 172.21.15.78:0/359706962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3818106180 0x7f3818075150 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3808009940 tx=0x7f38080326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:54.706 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.266+0000 7f38157fa640 1 -- 172.21.15.78:0/359706962 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3808042070 con 0x7f3818106180 2024-08-18T23:34:54.706 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.266+0000 7f381e070640 1 -- 172.21.15.78:0/359706962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381819e530 con 0x7f3818106180 2024-08-18T23:34:54.706 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.266+0000 7f38157fa640 1 -- 172.21.15.78:0/359706962 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3808032be0 con 0x7f3818106180 2024-08-18T23:34:54.706 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.266+0000 7f38157fa640 1 -- 172.21.15.78:0/359706962 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3808037040 con 0x7f3818106180 2024-08-18T23:34:54.706 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.266+0000 7f381e070640 1 -- 172.21.15.78:0/359706962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f381819e8d0 con 0x7f3818106180 2024-08-18T23:34:54.706 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.267+0000 7f38157fa640 1 -- 172.21.15.78:0/359706962 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f3808045070 con 0x7f3818106180 2024-08-18T23:34:54.706 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.267+0000 7f38157fa640 1 --2- 172.21.15.78:0/359706962 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f3804038f20 0x7f380403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:54.706 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.267+0000 7f381e070640 1 -- 172.21.15.78:0/359706962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f381810a8b0 con 0x7f3818106180 2024-08-18T23:34:54.706 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.267+0000 7f38157fa640 1 -- 172.21.15.78:0/359706962 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f380806bec0 con 0x7f3818106180 2024-08-18T23:34:54.707 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.268+0000 7f380effd640 1 --2- 172.21.15.78:0/359706962 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f3804038f20 0x7f380403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:54.707 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.268+0000 7f380effd640 1 --2- 172.21.15.78:0/359706962 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f3804038f20 0x7f380403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f38000098c0 tx=0x7f3800006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:54.707 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.272+0000 7f38157fa640 1 -- 172.21.15.78:0/359706962 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f380806b8e0 con 0x7f3818106180 2024-08-18T23:34:54.707 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:53.482+0000 7f381e070640 1 -- 172.21.15.78:0/359706962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f381802d680 con 0x7f3818106180 2024-08-18T23:34:54.707 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:54.114+0000 7f38157fa640 1 -- 172.21.15.78:0/359706962 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f3808032d90 con 0x7f3818106180 2024-08-18T23:34:54.707 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:54.115+0000 7f38157fa640 1 -- 172.21.15.78:0/359706962 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f380803b330 con 0x7f3818106180 2024-08-18T23:34:54.707 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:54.117+0000 7f381e070640 1 -- 172.21.15.78:0/359706962 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f3804038f20 msgr2=0x7f380403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:54.707 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:54.117+0000 7f381e070640 1 --2- 172.21.15.78:0/359706962 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f3804038f20 0x7f380403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f38000098c0 tx=0x7f3800006db0 comp rx=0 tx=0).stop 2024-08-18T23:34:54.707 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:54.117+0000 7f381e070640 1 -- 172.21.15.78:0/359706962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3818106180 msgr2=0x7f3818075150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:54.707 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:54.117+0000 7f381e070640 1 --2- 172.21.15.78:0/359706962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3818106180 0x7f3818075150 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3808009940 tx=0x7f38080326e0 comp rx=0 tx=0).stop 2024-08-18T23:34:54.707 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:54.117+0000 7f381e070640 1 -- 172.21.15.78:0/359706962 shutdown_connections 2024-08-18T23:34:54.707 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:54.117+0000 7f381e070640 1 --2- 172.21.15.78:0/359706962 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f3804038f20 0x7f380403b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:54.707 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:54.117+0000 7f381e070640 1 --2- 172.21.15.78:0/359706962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3818106180 0x7f3818075150 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:54.708 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:54.117+0000 7f381e070640 1 -- 172.21.15.78:0/359706962 >> 172.21.15.78:0/359706962 conn(0x7f38181018f0 msgr2=0x7f3818102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:54.708 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:54.118+0000 7f381e070640 1 -- 172.21.15.78:0/359706962 shutdown_connections 2024-08-18T23:34:54.708 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:54.118+0000 7f381e070640 1 -- 172.21.15.78:0/359706962 wait complete. 2024-08-18T23:34:54.949 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:54 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:54.727+0000 7f2a4104f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-18T23:34:54.949 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:54 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:54.873+0000 7f2a4104f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-18T23:34:55.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:55 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/359706962' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-08-18T23:34:55.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:55 smithi078 ceph-mon[18440]: mgrmap e5: x(active, since 5s) 2024-08-18T23:34:55.219 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:55 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:55.002+0000 7f2a4104f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-18T23:34:55.220 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:55 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:55.132+0000 7f2a4104f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-18T23:34:55.220 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:55 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:55.191+0000 7f2a4104f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-18T23:34:55.631 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:55 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:55.501+0000 7f2a4104f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-18T23:34:55.631 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:55 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:55.561+0000 7f2a4104f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-18T23:34:55.631 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:55 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:55.629+0000 7f2a4104f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-18T23:34:55.905 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:55 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:55.777+0000 7f2a4104f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-18T23:34:55.905 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:55 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:55.837+0000 7f2a4104f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-18T23:34:55.905 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:55 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:55.904+0000 7f2a4104f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-18T23:34:55.943 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-08-18T23:34:55.944 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-08-18T23:34:55.944 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-18T23:34:55.944 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-18T23:34:55.944 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-18T23:34:55.944 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-08-18T23:34:55.944 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.116+0000 7f360fa93640 1 Processor -- start 2024-08-18T23:34:55.944 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.116+0000 7f360fa93640 1 -- start start 2024-08-18T23:34:55.944 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.117+0000 7f360fa93640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3608105b80 0x7f3608105f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:55.944 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.117+0000 7f360fa93640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f360807bd80 con 0x7f3608105b80 2024-08-18T23:34:55.944 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.117+0000 7f360d82f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3608105b80 0x7f3608105f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:55.944 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.117+0000 7f360d82f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3608105b80 0x7f3608105f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59420/0 (socket says 172.21.15.78:59420) 2024-08-18T23:34:55.944 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.117+0000 7f360d82f640 1 -- 172.21.15.78:0/3252101773 learned_addr learned my addr 172.21.15.78:0/3252101773 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:55.945 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.117+0000 7f360d82f640 1 -- 172.21.15.78:0/3252101773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36081064c0 con 0x7f3608105b80 2024-08-18T23:34:55.945 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.118+0000 7f360d82f640 1 --2- 172.21.15.78:0/3252101773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3608105b80 0x7f3608105f80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3604009820 tx=0x7f360402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569836f8ec22ae66 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:55.945 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.118+0000 7f360c82d640 1 -- 172.21.15.78:0/3252101773 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3604037070 con 0x7f3608105b80 2024-08-18T23:34:55.945 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.118+0000 7f360c82d640 1 -- 172.21.15.78:0/3252101773 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3604032900 con 0x7f3608105b80 2024-08-18T23:34:55.945 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.118+0000 7f360c82d640 1 -- 172.21.15.78:0/3252101773 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3604030850 con 0x7f3608105b80 2024-08-18T23:34:55.945 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.119+0000 7f360fa93640 1 -- 172.21.15.78:0/3252101773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3608105b80 msgr2=0x7f3608105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:55.945 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.119+0000 7f360fa93640 1 --2- 172.21.15.78:0/3252101773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3608105b80 0x7f3608105f80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3604009820 tx=0x7f360402a270 comp rx=0 tx=0).stop 2024-08-18T23:34:55.945 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.119+0000 7f360fa93640 1 -- 172.21.15.78:0/3252101773 shutdown_connections 2024-08-18T23:34:55.945 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.119+0000 7f360fa93640 1 --2- 172.21.15.78:0/3252101773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3608105b80 0x7f3608105f80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:55.945 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.119+0000 7f360fa93640 1 -- 172.21.15.78:0/3252101773 >> 172.21.15.78:0/3252101773 conn(0x7f36081019f0 msgr2=0x7f3608103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:55.945 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.119+0000 7f360fa93640 1 -- 172.21.15.78:0/3252101773 shutdown_connections 2024-08-18T23:34:55.945 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.119+0000 7f360fa93640 1 -- 172.21.15.78:0/3252101773 wait complete. 2024-08-18T23:34:55.946 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.119+0000 7f360fa93640 1 Processor -- start 2024-08-18T23:34:55.946 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.119+0000 7f360fa93640 1 -- start start 2024-08-18T23:34:55.946 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.119+0000 7f360fa93640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3608105b80 0x7f360819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:55.946 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.120+0000 7f360fa93640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3608107440 con 0x7f3608105b80 2024-08-18T23:34:55.946 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.120+0000 7f360d82f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3608105b80 0x7f360819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:34:55.946 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.120+0000 7f360d82f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3608105b80 0x7f360819d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59432/0 (socket says 172.21.15.78:59432) 2024-08-18T23:34:55.946 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.120+0000 7f360d82f640 1 -- 172.21.15.78:0/2096550764 learned_addr learned my addr 172.21.15.78:0/2096550764 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:34:55.946 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.120+0000 7f360d82f640 1 -- 172.21.15.78:0/2096550764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360819d990 con 0x7f3608105b80 2024-08-18T23:34:55.946 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.120+0000 7f360d82f640 1 --2- 172.21.15.78:0/2096550764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3608105b80 0x7f360819d450 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3604032ce0 tx=0x7f3604032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:34:55.946 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.120+0000 7f35feffd640 1 -- 172.21.15.78:0/2096550764 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3604042070 con 0x7f3608105b80 2024-08-18T23:34:55.946 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.120+0000 7f360fa93640 1 -- 172.21.15.78:0/2096550764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f360819dc20 con 0x7f3608105b80 2024-08-18T23:34:55.946 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.120+0000 7f35feffd640 1 -- 172.21.15.78:0/2096550764 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f36040308c0 con 0x7f3608105b80 2024-08-18T23:34:55.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.120+0000 7f35feffd640 1 -- 172.21.15.78:0/2096550764 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3604037040 con 0x7f3608105b80 2024-08-18T23:34:55.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.120+0000 7f360fa93640 1 -- 172.21.15.78:0/2096550764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36081a0910 con 0x7f3608105b80 2024-08-18T23:34:55.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.121+0000 7f360fa93640 1 -- 172.21.15.78:0/2096550764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3608107b10 con 0x7f3608105b80 2024-08-18T23:34:55.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.121+0000 7f35feffd640 1 -- 172.21.15.78:0/2096550764 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f3604045070 con 0x7f3608105b80 2024-08-18T23:34:55.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.122+0000 7f35feffd640 1 --2- 172.21.15.78:0/2096550764 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f35f4038f70 0x7f35f403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:34:55.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.122+0000 7f35feffd640 1 -- 172.21.15.78:0/2096550764 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f360406bff0 con 0x7f3608105b80 2024-08-18T23:34:55.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.122+0000 7f360d02e640 1 -- 172.21.15.78:0/2096550764 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f35f4038f70 msgr2=0x7f35f403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/2301978296 2024-08-18T23:34:55.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.122+0000 7f360d02e640 1 --2- 172.21.15.78:0/2096550764 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f35f4038f70 0x7f35f403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-18T23:34:55.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.126+0000 7f35feffd640 1 -- 172.21.15.78:0/2096550764 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f360404b170 con 0x7f3608105b80 2024-08-18T23:34:55.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.323+0000 7f360d02e640 1 -- 172.21.15.78:0/2096550764 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f35f4038f70 msgr2=0x7f35f403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/2301978296 2024-08-18T23:34:55.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.323+0000 7f360d02e640 1 --2- 172.21.15.78:0/2096550764 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f35f4038f70 0x7f35f403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-18T23:34:55.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.350+0000 7f360fa93640 1 -- 172.21.15.78:0/2096550764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f360802d860 con 0x7f3608105b80 2024-08-18T23:34:55.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.350+0000 7f35feffd640 1 -- 172.21.15.78:0/2096550764 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f360403cd50 con 0x7f3608105b80 2024-08-18T23:34:55.947 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.352+0000 7f360fa93640 1 -- 172.21.15.78:0/2096550764 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f35f4038f70 msgr2=0x7f35f403b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-18T23:34:55.948 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.352+0000 7f360fa93640 1 --2- 172.21.15.78:0/2096550764 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f35f4038f70 0x7f35f403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:55.948 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.352+0000 7f360fa93640 1 -- 172.21.15.78:0/2096550764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3608105b80 msgr2=0x7f360819d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:34:55.948 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.352+0000 7f360fa93640 1 --2- 172.21.15.78:0/2096550764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3608105b80 0x7f360819d450 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3604032ce0 tx=0x7f3604032d10 comp rx=0 tx=0).stop 2024-08-18T23:34:55.948 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.352+0000 7f360fa93640 1 -- 172.21.15.78:0/2096550764 shutdown_connections 2024-08-18T23:34:55.948 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.352+0000 7f360fa93640 1 --2- 172.21.15.78:0/2096550764 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7f35f4038f70 0x7f35f403b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:55.948 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.352+0000 7f360fa93640 1 --2- 172.21.15.78:0/2096550764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3608105b80 0x7f360819d450 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:34:55.948 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.352+0000 7f360fa93640 1 -- 172.21.15.78:0/2096550764 >> 172.21.15.78:0/2096550764 conn(0x7f36081019f0 msgr2=0x7f36081940d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:34:55.948 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.352+0000 7f360fa93640 1 -- 172.21.15.78:0/2096550764 shutdown_connections 2024-08-18T23:34:55.948 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:55.352+0000 7f360fa93640 1 -- 172.21.15.78:0/2096550764 wait complete. 2024-08-18T23:34:55.948 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for the mgr to restart... 2024-08-18T23:34:55.948 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr epoch 5... 2024-08-18T23:34:56.163 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:56 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2096550764' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-08-18T23:34:56.163 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:56 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:56.096+0000 7f2a4104f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-18T23:34:56.163 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:56 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:56.161+0000 7f2a4104f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-18T23:34:56.469 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:56 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:56.225+0000 7f2a4104f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-18T23:34:56.469 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:56 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:56.296+0000 7f2a4104f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-18T23:34:56.917 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:56 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:56.521+0000 7f2a4104f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-18T23:34:56.917 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:56 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:56.584+0000 7f2a4104f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-18T23:34:57.219 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:57 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:57.117+0000 7f2a4104f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-18T23:34:57.718 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:57 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:57.431+0000 7f2a4104f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-18T23:34:57.969 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:57 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:57.840+0000 7f2a4104f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-18T23:34:57.969 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:57 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:57.896+0000 7f2a4104f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-18T23:34:57.969 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:34:57 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:34:57.955+0000 7f2a4104f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-18T23:34:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:59 smithi078 ceph-mon[18440]: Active manager daemon x restarted 2024-08-18T23:34:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:59 smithi078 ceph-mon[18440]: Activating manager daemon x 2024-08-18T23:34:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:59 smithi078 ceph-mon[18440]: osdmap e2: 0 total, 0 up, 0 in 2024-08-18T23:34:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:59 smithi078 ceph-mon[18440]: mgrmap e6: x(active, starting, since 0.00380519s) 2024-08-18T23:34:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:59 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-18T23:34:59.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:59 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-18T23:34:59.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:59 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-18T23:34:59.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:59 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-18T23:34:59.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:59 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-18T23:34:59.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:59 smithi078 ceph-mon[18440]: Manager daemon x is now available 2024-08-18T23:34:59.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:59 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' 2024-08-18T23:34:59.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:59 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' 2024-08-18T23:34:59.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:59 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:34:59.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:59 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:34:59.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:59 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-18T23:34:59.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:34:59 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-18T23:35:00.341 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-mon[18440]: Found migration_current of "None". Setting to last migration. 2024-08-18T23:35:00.341 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' 2024-08-18T23:35:00.341 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' 2024-08-18T23:35:00.341 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-mon[18440]: mgrmap e7: x(active, since 1.00598s) 2024-08-18T23:35:00.636 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-08-18T23:35:00.643 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-08-18T23:35:00.643 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-18T23:35:00.643 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-08-18T23:35:00.643 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.308+0000 7fcf25d86640 1 Processor -- start 2024-08-18T23:35:00.643 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.308+0000 7fcf25d86640 1 -- start start 2024-08-18T23:35:00.643 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.308+0000 7fcf25d86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf20107cf0 0x7fcf2010a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:00.643 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.309+0000 7fcf25d86640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcf200755b0 con 0x7fcf20107cf0 2024-08-18T23:35:00.643 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.309+0000 7fcf1f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf20107cf0 0x7fcf2010a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:00.643 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.309+0000 7fcf1f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf20107cf0 0x7fcf2010a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48764/0 (socket says 172.21.15.78:48764) 2024-08-18T23:35:00.644 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.309+0000 7fcf1f7fe640 1 -- 172.21.15.78:0/3808359667 learned_addr learned my addr 172.21.15.78:0/3808359667 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:00.644 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.309+0000 7fcf1f7fe640 1 -- 172.21.15.78:0/3808359667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2010a620 con 0x7fcf20107cf0 2024-08-18T23:35:00.644 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.310+0000 7fcf1f7fe640 1 --2- 172.21.15.78:0/3808359667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf20107cf0 0x7fcf2010a0e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcf10009e80 tx=0x7fcf1002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8920241895c550c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:00.644 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.310+0000 7fcf1e7fc640 1 -- 172.21.15.78:0/3808359667 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf10036070 con 0x7fcf20107cf0 2024-08-18T23:35:00.644 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.310+0000 7fcf1e7fc640 1 -- 172.21.15.78:0/3808359667 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcf100318c0 con 0x7fcf20107cf0 2024-08-18T23:35:00.644 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.310+0000 7fcf1e7fc640 1 -- 172.21.15.78:0/3808359667 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf10007ab0 con 0x7fcf20107cf0 2024-08-18T23:35:00.644 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.310+0000 7fcf25d86640 1 -- 172.21.15.78:0/3808359667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf20107cf0 msgr2=0x7fcf2010a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:00.644 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.310+0000 7fcf25d86640 1 --2- 172.21.15.78:0/3808359667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf20107cf0 0x7fcf2010a0e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcf10009e80 tx=0x7fcf1002a7a0 comp rx=0 tx=0).stop 2024-08-18T23:35:00.644 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.311+0000 7fcf25d86640 1 -- 172.21.15.78:0/3808359667 shutdown_connections 2024-08-18T23:35:00.645 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.311+0000 7fcf25d86640 1 --2- 172.21.15.78:0/3808359667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf20107cf0 0x7fcf2010a0e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:00.645 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.311+0000 7fcf25d86640 1 -- 172.21.15.78:0/3808359667 >> 172.21.15.78:0/3808359667 conn(0x7fcf201018f0 msgr2=0x7fcf20103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:00.645 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.311+0000 7fcf25d86640 1 -- 172.21.15.78:0/3808359667 shutdown_connections 2024-08-18T23:35:00.645 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.311+0000 7fcf25d86640 1 -- 172.21.15.78:0/3808359667 wait complete. 2024-08-18T23:35:00.645 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.311+0000 7fcf25d86640 1 Processor -- start 2024-08-18T23:35:00.645 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.312+0000 7fcf25d86640 1 -- start start 2024-08-18T23:35:00.645 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.312+0000 7fcf25d86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf20107cf0 0x7fcf2019b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:00.645 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.312+0000 7fcf25d86640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2010b6c0 con 0x7fcf20107cf0 2024-08-18T23:35:00.645 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.312+0000 7fcf1f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf20107cf0 0x7fcf2019b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:00.645 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.312+0000 7fcf1f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf20107cf0 0x7fcf2019b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48766/0 (socket says 172.21.15.78:48766) 2024-08-18T23:35:00.645 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.312+0000 7fcf1f7fe640 1 -- 172.21.15.78:0/1421744197 learned_addr learned my addr 172.21.15.78:0/1421744197 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:00.645 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.312+0000 7fcf1f7fe640 1 -- 172.21.15.78:0/1421744197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2019bb00 con 0x7fcf20107cf0 2024-08-18T23:35:00.646 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.313+0000 7fcf1f7fe640 1 --2- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf20107cf0 0x7fcf2019b5c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcf10031ca0 tx=0x7fcf10031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:00.646 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.313+0000 7fcf1cff9640 1 -- 172.21.15.78:0/1421744197 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf10036050 con 0x7fcf20107cf0 2024-08-18T23:35:00.646 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.313+0000 7fcf25d86640 1 -- 172.21.15.78:0/1421744197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2019bd90 con 0x7fcf20107cf0 2024-08-18T23:35:00.646 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.313+0000 7fcf1cff9640 1 -- 172.21.15.78:0/1421744197 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcf10042070 con 0x7fcf20107cf0 2024-08-18T23:35:00.646 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.313+0000 7fcf1cff9640 1 -- 172.21.15.78:0/1421744197 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf1003d940 con 0x7fcf20107cf0 2024-08-18T23:35:00.646 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.313+0000 7fcf25d86640 1 -- 172.21.15.78:0/1421744197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf2019ea80 con 0x7fcf20107cf0 2024-08-18T23:35:00.646 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.314+0000 7fcf1cff9640 1 -- 172.21.15.78:0/1421744197 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fcf1003dc40 con 0x7fcf20107cf0 2024-08-18T23:35:00.646 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.314+0000 7fcf1cff9640 1 --2- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7fcf08038f70 0x7fcf0803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:00.646 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.314+0000 7fcf1cff9640 1 -- 172.21.15.78:0/1421744197 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcf10007e00 con 0x7fcf20107cf0 2024-08-18T23:35:00.646 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.314+0000 7fcf25d86640 1 -- 172.21.15.78:0/1421744197 --> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcf20108150 con 0x7fcf08038f70 2024-08-18T23:35:00.646 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.314+0000 7fcf1effd640 1 -- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7fcf08038f70 msgr2=0x7fcf0803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/2301978296 2024-08-18T23:35:00.646 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.314+0000 7fcf1effd640 1 --2- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7fcf08038f70 0x7fcf0803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-18T23:35:00.647 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.515+0000 7fcf1effd640 1 -- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7fcf08038f70 msgr2=0x7fcf0803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/2301978296 2024-08-18T23:35:00.647 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.515+0000 7fcf1effd640 1 --2- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7fcf08038f70 0x7fcf0803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-18T23:35:00.647 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.916+0000 7fcf1effd640 1 -- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7fcf08038f70 msgr2=0x7fcf0803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/2301978296 2024-08-18T23:35:00.647 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:56.916+0000 7fcf1effd640 1 --2- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7fcf08038f70 0x7fcf0803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-18T23:35:00.647 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:57.717+0000 7fcf1effd640 1 -- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7fcf08038f70 msgr2=0x7fcf0803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/2301978296 2024-08-18T23:35:00.647 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:57.717+0000 7fcf1effd640 1 --2- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7fcf08038f70 0x7fcf0803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-18T23:35:00.647 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:59.022+0000 7fcf1cff9640 1 -- 172.21.15.78:0/1421744197 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7fcf1006bdd0 con 0x7fcf20107cf0 2024-08-18T23:35:00.647 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:59.023+0000 7fcf1cff9640 1 -- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7fcf08038f70 msgr2=0x7fcf0803b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-18T23:35:00.647 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:34:59.023+0000 7fcf1cff9640 1 --2- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7fcf08038f70 0x7fcf0803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:00.647 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:00.024+0000 7fcf1cff9640 1 -- 172.21.15.78:0/1421744197 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fcf100463c0 con 0x7fcf20107cf0 2024-08-18T23:35:00.647 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:00.024+0000 7fcf1cff9640 1 --2- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7fcf0803c6b0 0x7fcf0803eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:00.647 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:00.024+0000 7fcf1cff9640 1 -- 172.21.15.78:0/1421744197 --> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcf10066570 con 0x7fcf0803c6b0 2024-08-18T23:35:00.647 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:00.024+0000 7fcf1effd640 1 --2- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7fcf0803c6b0 0x7fcf0803eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:00.648 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:00.025+0000 7fcf1effd640 1 --2- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7fcf0803c6b0 0x7fcf0803eaa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcf14003980 tx=0x7fcf14006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:00.648 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:00.025+0000 7fcf1cff9640 1 -- 172.21.15.78:0/1421744197 <== mgr.14118 v2:172.21.15.78:6800/311758909 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fcf10066570 con 0x7fcf0803c6b0 2024-08-18T23:35:00.648 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:00.032+0000 7fcf25d86640 1 -- 172.21.15.78:0/1421744197 --> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fcf200682c0 con 0x7fcf0803c6b0 2024-08-18T23:35:00.648 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:00.033+0000 7fcf1cff9640 1 -- 172.21.15.78:0/1421744197 <== mgr.14118 v2:172.21.15.78:6800/311758909 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fcf200682c0 con 0x7fcf0803c6b0 2024-08-18T23:35:00.648 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:00.033+0000 7fcf25d86640 1 -- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7fcf0803c6b0 msgr2=0x7fcf0803eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:00.648 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:00.033+0000 7fcf25d86640 1 --2- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7fcf0803c6b0 0x7fcf0803eaa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcf14003980 tx=0x7fcf14006e10 comp rx=0 tx=0).stop 2024-08-18T23:35:00.648 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:00.033+0000 7fcf25d86640 1 -- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf20107cf0 msgr2=0x7fcf2019b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:00.648 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:00.033+0000 7fcf25d86640 1 --2- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf20107cf0 0x7fcf2019b5c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcf10031ca0 tx=0x7fcf10031cd0 comp rx=0 tx=0).stop 2024-08-18T23:35:00.648 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:00.033+0000 7fcf25d86640 1 -- 172.21.15.78:0/1421744197 shutdown_connections 2024-08-18T23:35:00.648 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:00.033+0000 7fcf25d86640 1 --2- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7fcf0803c6b0 0x7fcf0803eaa0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:00.648 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:00.033+0000 7fcf25d86640 1 --2- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:6800/2301978296,v1:172.21.15.78:6801/2301978296] conn(0x7fcf08038f70 0x7fcf0803b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:00.648 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:00.033+0000 7fcf25d86640 1 --2- 172.21.15.78:0/1421744197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf20107cf0 0x7fcf2019b5c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:00.649 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:00.033+0000 7fcf25d86640 1 -- 172.21.15.78:0/1421744197 >> 172.21.15.78:0/1421744197 conn(0x7fcf201018f0 msgr2=0x7fcf20102710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:00.649 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:00.033+0000 7fcf25d86640 1 -- 172.21.15.78:0/1421744197 shutdown_connections 2024-08-18T23:35:00.649 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:00.033+0000 7fcf25d86640 1 -- 172.21.15.78:0/1421744197 wait complete. 2024-08-18T23:35:00.649 INFO:teuthology.orchestra.run.smithi078.stdout:mgr epoch 5 is available 2024-08-18T23:35:00.649 INFO:teuthology.orchestra.run.smithi078.stdout:Setting orchestrator backend to cephadm... 2024-08-18T23:35:00.952 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:00.853+0000 7f29a855d640 -1 log_channel(cephadm) log [ERR] : [18/Aug/2024:23:35:00] ENGINE Error in HTTPServer.serve 2024-08-18T23:35:00.953 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: Traceback (most recent call last): 2024-08-18T23:35:00.953 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-18T23:35:00.953 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: self._connections.run(self.expiration_interval) 2024-08-18T23:35:00.953 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-18T23:35:00.953 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: self._run(expiration_interval) 2024-08-18T23:35:00.953 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-18T23:35:00.953 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: new_conn = self._from_server_socket(self.server.socket) 2024-08-18T23:35:00.953 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-18T23:35:00.953 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-18T23:35:00.953 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-18T23:35:00.953 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: s = self.context.wrap_socket( 2024-08-18T23:35:00.953 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-18T23:35:00.954 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: return self.sslsocket_class._create( 2024-08-18T23:35:00.954 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-18T23:35:00.954 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: self.do_handshake() 2024-08-18T23:35:00.954 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-18T23:35:00.954 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: self._sslobj.do_handshake() 2024-08-18T23:35:00.954 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-18T23:35:00.954 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:01.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:01 smithi078 ceph-mon[18440]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-18T23:35:01.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:01 smithi078 ceph-mon[18440]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-08-18T23:35:01.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:01 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:35:01.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.030+0000 7f42b3008640 1 Processor -- start 2024-08-18T23:35:01.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.030+0000 7f42b3008640 1 -- start start 2024-08-18T23:35:01.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.030+0000 7f42b3008640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42ac1055d0 0x7f42ac1059d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:01.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.031+0000 7f42b3008640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac105fa0 con 0x7f42ac1055d0 2024-08-18T23:35:01.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.031+0000 7f42b0da4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42ac1055d0 0x7f42ac1059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:01.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.031+0000 7f42b0da4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42ac1055d0 0x7f42ac1059d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48842/0 (socket says 172.21.15.78:48842) 2024-08-18T23:35:01.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.031+0000 7f42b0da4640 1 -- 172.21.15.78:0/2705288312 learned_addr learned my addr 172.21.15.78:0/2705288312 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:01.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.031+0000 7f42b0da4640 1 -- 172.21.15.78:0/2705288312 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ac1085b0 con 0x7f42ac1055d0 2024-08-18T23:35:01.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.032+0000 7f42b0da4640 1 --2- 172.21.15.78:0/2705288312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42ac1055d0 0x7f42ac1059d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f429c009e80 tx=0x7f429c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf6897de7a1948f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:01.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.032+0000 7f42ab7fe640 1 -- 172.21.15.78:0/2705288312 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f429c036070 con 0x7f42ac1055d0 2024-08-18T23:35:01.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.032+0000 7f42ab7fe640 1 -- 172.21.15.78:0/2705288312 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f429c0318c0 con 0x7f42ac1055d0 2024-08-18T23:35:01.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.032+0000 7f42ab7fe640 1 -- 172.21.15.78:0/2705288312 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f429c007ab0 con 0x7f42ac1055d0 2024-08-18T23:35:01.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.032+0000 7f42b3008640 1 -- 172.21.15.78:0/2705288312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42ac1055d0 msgr2=0x7f42ac1059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:01.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.032+0000 7f42b3008640 1 --2- 172.21.15.78:0/2705288312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42ac1055d0 0x7f42ac1059d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f429c009e80 tx=0x7f429c02a7a0 comp rx=0 tx=0).stop 2024-08-18T23:35:01.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.032+0000 7f42b3008640 1 -- 172.21.15.78:0/2705288312 shutdown_connections 2024-08-18T23:35:01.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.032+0000 7f42b3008640 1 --2- 172.21.15.78:0/2705288312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42ac1055d0 0x7f42ac1059d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:01.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.032+0000 7f42b3008640 1 -- 172.21.15.78:0/2705288312 >> 172.21.15.78:0/2705288312 conn(0x7f42ac101120 msgr2=0x7f42ac103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:01.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.032+0000 7f42b3008640 1 -- 172.21.15.78:0/2705288312 shutdown_connections 2024-08-18T23:35:01.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.033+0000 7f42b3008640 1 -- 172.21.15.78:0/2705288312 wait complete. 2024-08-18T23:35:01.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.033+0000 7f42b3008640 1 Processor -- start 2024-08-18T23:35:01.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.033+0000 7f42b3008640 1 -- start start 2024-08-18T23:35:01.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.033+0000 7f42b3008640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42ac1055d0 0x7f42ac19b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:01.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.033+0000 7f42b3008640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac10b7e0 con 0x7f42ac1055d0 2024-08-18T23:35:01.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.033+0000 7f42b0da4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42ac1055d0 0x7f42ac19b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:01.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.033+0000 7f42b0da4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42ac1055d0 0x7f42ac19b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48858/0 (socket says 172.21.15.78:48858) 2024-08-18T23:35:01.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.033+0000 7f42b0da4640 1 -- 172.21.15.78:0/708627339 learned_addr learned my addr 172.21.15.78:0/708627339 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:01.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.034+0000 7f42b0da4640 1 -- 172.21.15.78:0/708627339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ac19bbe0 con 0x7f42ac1055d0 2024-08-18T23:35:01.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.034+0000 7f42b0da4640 1 --2- 172.21.15.78:0/708627339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42ac1055d0 0x7f42ac19b6a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f429c02acc0 tx=0x7f429c031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:01.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.034+0000 7f42a9ffb640 1 -- 172.21.15.78:0/708627339 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f429c036050 con 0x7f42ac1055d0 2024-08-18T23:35:01.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.034+0000 7f42b3008640 1 -- 172.21.15.78:0/708627339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42ac19be70 con 0x7f42ac1055d0 2024-08-18T23:35:01.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.034+0000 7f42b3008640 1 -- 172.21.15.78:0/708627339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42ac19eb60 con 0x7f42ac1055d0 2024-08-18T23:35:01.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.034+0000 7f42a9ffb640 1 -- 172.21.15.78:0/708627339 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f429c037040 con 0x7f42ac1055d0 2024-08-18T23:35:01.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.034+0000 7f42a9ffb640 1 -- 172.21.15.78:0/708627339 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f429c03d7e0 con 0x7f42ac1055d0 2024-08-18T23:35:01.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.035+0000 7f42a9ffb640 1 -- 172.21.15.78:0/708627339 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f429c044020 con 0x7f42ac1055d0 2024-08-18T23:35:01.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.035+0000 7f42a9ffb640 1 --2- 172.21.15.78:0/708627339 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f4298038e60 0x7f429803b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:01.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.035+0000 7f42b3008640 1 -- 172.21.15.78:0/708627339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f428c0052c0 con 0x7f42ac1055d0 2024-08-18T23:35:01.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.035+0000 7f42a9ffb640 1 -- 172.21.15.78:0/708627339 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f429c042020 con 0x7f42ac1055d0 2024-08-18T23:35:01.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.035+0000 7f42abfff640 1 --2- 172.21.15.78:0/708627339 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f4298038e60 0x7f429803b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:01.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.036+0000 7f42abfff640 1 --2- 172.21.15.78:0/708627339 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f4298038e60 0x7f429803b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42940098c0 tx=0x7f4294006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:01.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.040+0000 7f42a9ffb640 1 -- 172.21.15.78:0/708627339 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f429c030800 con 0x7f42ac1055d0 2024-08-18T23:35:01.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.074+0000 7f42a9ffb640 1 -- 172.21.15.78:0/708627339 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f429c03c730 con 0x7f42ac1055d0 2024-08-18T23:35:01.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.222+0000 7f42b3008640 1 -- 172.21.15.78:0/708627339 --> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f428c000bf0 con 0x7f4298038e60 2024-08-18T23:35:01.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.227+0000 7f42a9ffb640 1 -- 172.21.15.78:0/708627339 <== mgr.14118 v2:172.21.15.78:6800/311758909 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f428c000bf0 con 0x7f4298038e60 2024-08-18T23:35:01.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.229+0000 7f42b3008640 1 -- 172.21.15.78:0/708627339 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f4298038e60 msgr2=0x7f429803b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:01.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.229+0000 7f42b3008640 1 --2- 172.21.15.78:0/708627339 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f4298038e60 0x7f429803b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42940098c0 tx=0x7f4294006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:01.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.229+0000 7f42b3008640 1 -- 172.21.15.78:0/708627339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42ac1055d0 msgr2=0x7f42ac19b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:01.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.229+0000 7f42b3008640 1 --2- 172.21.15.78:0/708627339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42ac1055d0 0x7f42ac19b6a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f429c02acc0 tx=0x7f429c031ca0 comp rx=0 tx=0).stop 2024-08-18T23:35:01.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.229+0000 7f42b3008640 1 -- 172.21.15.78:0/708627339 shutdown_connections 2024-08-18T23:35:01.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.229+0000 7f42b3008640 1 --2- 172.21.15.78:0/708627339 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f4298038e60 0x7f429803b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:01.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.229+0000 7f42b3008640 1 --2- 172.21.15.78:0/708627339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f42ac1055d0 0x7f42ac19b6a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:01.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.229+0000 7f42b3008640 1 -- 172.21.15.78:0/708627339 >> 172.21.15.78:0/708627339 conn(0x7f42ac101120 msgr2=0x7f42ac101f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:01.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.229+0000 7f42b3008640 1 -- 172.21.15.78:0/708627339 shutdown_connections 2024-08-18T23:35:01.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:01.229+0000 7f42b3008640 1 -- 172.21.15.78:0/708627339 wait complete. 2024-08-18T23:35:02.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:02 smithi078 ceph-mon[18440]: [18/Aug/2024:23:35:00] ENGINE Bus STARTING 2024-08-18T23:35:02.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:02 smithi078 ceph-mon[18440]: [18/Aug/2024:23:35:00] ENGINE Serving on http://172.21.15.78:8765 2024-08-18T23:35:02.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:02 smithi078 ceph-mon[18440]: [18/Aug/2024:23:35:00] ENGINE Serving on https://172.21.15.78:7150 2024-08-18T23:35:02.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:02 smithi078 ceph-mon[18440]: [18/Aug/2024:23:35:00] ENGINE Bus STARTED 2024-08-18T23:35:02.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:02 smithi078 ceph-mon[18440]: [18/Aug/2024:23:35:00] ENGINE Error in HTTPServer.serve 2024-08-18T23:35:02.229 INFO:journalctl@ceph.mon.a.smithi078.stdout: Traceback (most recent call last): 2024-08-18T23:35:02.229 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-18T23:35:02.229 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._connections.run(self.expiration_interval) 2024-08-18T23:35:02.229 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-18T23:35:02.229 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._run(expiration_interval) 2024-08-18T23:35:02.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-18T23:35:02.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-18T23:35:02.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-18T23:35:02.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-18T23:35:02.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-18T23:35:02.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: s = self.context.wrap_socket( 2024-08-18T23:35:02.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-18T23:35:02.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: return self.sslsocket_class._create( 2024-08-18T23:35:02.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-18T23:35:02.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: self.do_handshake() 2024-08-18T23:35:02.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-18T23:35:02.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._sslobj.do_handshake() 2024-08-18T23:35:02.230 INFO:journalctl@ceph.mon.a.smithi078.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-18T23:35:02.231 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:02 smithi078 ceph-mon[18440]: mgrmap e8: x(active, since 2s) 2024-08-18T23:35:02.231 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:02 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' 2024-08-18T23:35:02.231 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:02 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:35:02.761 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout value unchanged 2024-08-18T23:35:02.761 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.050+0000 7f32ce15e640 1 Processor -- start 2024-08-18T23:35:02.761 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.050+0000 7f32ce15e640 1 -- start start 2024-08-18T23:35:02.761 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.051+0000 7f32ce15e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32c8107300 0x7f32c8107700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:02.761 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.051+0000 7f32ce15e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f32c8107c40 con 0x7f32c8107300 2024-08-18T23:35:02.761 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.051+0000 7f32c77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32c8107300 0x7f32c8107700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:02.761 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.051+0000 7f32c77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32c8107300 0x7f32c8107700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48862/0 (socket says 172.21.15.78:48862) 2024-08-18T23:35:02.761 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.051+0000 7f32c77fe640 1 -- 172.21.15.78:0/1379333105 learned_addr learned my addr 172.21.15.78:0/1379333105 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:02.761 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.051+0000 7f32c77fe640 1 -- 172.21.15.78:0/1379333105 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c8107dc0 con 0x7f32c8107300 2024-08-18T23:35:02.761 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.052+0000 7f32c77fe640 1 --2- 172.21.15.78:0/1379333105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32c8107300 0x7f32c8107700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009820 tx=0x7f32b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd4fb4c902b28b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:02.762 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.052+0000 7f32c67fc640 1 -- 172.21.15.78:0/1379333105 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32b8037070 con 0x7f32c8107300 2024-08-18T23:35:02.762 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.052+0000 7f32c67fc640 1 -- 172.21.15.78:0/1379333105 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f32b8032900 con 0x7f32c8107300 2024-08-18T23:35:02.762 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.052+0000 7f32c67fc640 1 -- 172.21.15.78:0/1379333105 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32b8030850 con 0x7f32c8107300 2024-08-18T23:35:02.762 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.053+0000 7f32ce15e640 1 -- 172.21.15.78:0/1379333105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32c8107300 msgr2=0x7f32c8107700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:02.762 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.053+0000 7f32ce15e640 1 --2- 172.21.15.78:0/1379333105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32c8107300 0x7f32c8107700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009820 tx=0x7f32b802a270 comp rx=0 tx=0).stop 2024-08-18T23:35:02.762 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.053+0000 7f32ce15e640 1 -- 172.21.15.78:0/1379333105 shutdown_connections 2024-08-18T23:35:02.762 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.053+0000 7f32ce15e640 1 --2- 172.21.15.78:0/1379333105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32c8107300 0x7f32c8107700 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:02.762 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.053+0000 7f32ce15e640 1 -- 172.21.15.78:0/1379333105 >> 172.21.15.78:0/1379333105 conn(0x7f32c8102b30 msgr2=0x7f32c8104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:02.762 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.053+0000 7f32ce15e640 1 -- 172.21.15.78:0/1379333105 shutdown_connections 2024-08-18T23:35:02.762 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.053+0000 7f32ce15e640 1 -- 172.21.15.78:0/1379333105 wait complete. 2024-08-18T23:35:02.762 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.053+0000 7f32ce15e640 1 Processor -- start 2024-08-18T23:35:02.762 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.053+0000 7f32ce15e640 1 -- start start 2024-08-18T23:35:02.763 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.054+0000 7f32ce15e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32c8107300 0x7f32c819b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:02.763 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.054+0000 7f32ce15e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f32c810b770 con 0x7f32c8107300 2024-08-18T23:35:02.763 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.054+0000 7f32c77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32c8107300 0x7f32c819b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:02.763 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.054+0000 7f32c77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32c8107300 0x7f32c819b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48868/0 (socket says 172.21.15.78:48868) 2024-08-18T23:35:02.763 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.054+0000 7f32c77fe640 1 -- 172.21.15.78:0/3568575316 learned_addr learned my addr 172.21.15.78:0/3568575316 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:02.763 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.054+0000 7f32c77fe640 1 -- 172.21.15.78:0/3568575316 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c819bb20 con 0x7f32c8107300 2024-08-18T23:35:02.763 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.054+0000 7f32c77fe640 1 --2- 172.21.15.78:0/3568575316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32c8107300 0x7f32c819b5e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f32b8031000 tx=0x7f32b80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:02.763 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.054+0000 7f32c4ff9640 1 -- 172.21.15.78:0/3568575316 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32b8042070 con 0x7f32c8107300 2024-08-18T23:35:02.763 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.054+0000 7f32c4ff9640 1 -- 172.21.15.78:0/3568575316 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f32b8032be0 con 0x7f32c8107300 2024-08-18T23:35:02.763 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.054+0000 7f32ce15e640 1 -- 172.21.15.78:0/3568575316 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c819bdb0 con 0x7f32c8107300 2024-08-18T23:35:02.763 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.054+0000 7f32c4ff9640 1 -- 172.21.15.78:0/3568575316 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32b8037040 con 0x7f32c8107300 2024-08-18T23:35:02.763 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.054+0000 7f32ce15e640 1 -- 172.21.15.78:0/3568575316 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c819eaa0 con 0x7f32c8107300 2024-08-18T23:35:02.763 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.055+0000 7f32c4ff9640 1 -- 172.21.15.78:0/3568575316 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f32b8045070 con 0x7f32c8107300 2024-08-18T23:35:02.764 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.055+0000 7f32c4ff9640 1 --2- 172.21.15.78:0/3568575316 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f32b4038fc0 0x7f32b403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:02.764 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.055+0000 7f32c4ff9640 1 -- 172.21.15.78:0/3568575316 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f32b806bf80 con 0x7f32c8107300 2024-08-18T23:35:02.764 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.055+0000 7f32ce15e640 1 -- 172.21.15.78:0/3568575316 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c8107780 con 0x7f32c8107300 2024-08-18T23:35:02.764 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.055+0000 7f32c6ffd640 1 --2- 172.21.15.78:0/3568575316 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f32b4038fc0 0x7f32b403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:02.764 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.056+0000 7f32c6ffd640 1 --2- 172.21.15.78:0/3568575316 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f32b4038fc0 0x7f32b403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f32b00098c0 tx=0x7f32b0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:02.764 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.060+0000 7f32c4ff9640 1 -- 172.21.15.78:0/3568575316 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f32b8042210 con 0x7f32c8107300 2024-08-18T23:35:02.764 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.227+0000 7f32ce15e640 1 -- 172.21.15.78:0/3568575316 --> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f32c8062360 con 0x7f32b4038fc0 2024-08-18T23:35:02.764 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.228+0000 7f32c4ff9640 1 -- 172.21.15.78:0/3568575316 <== mgr.14118 v2:172.21.15.78:6800/311758909 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f32c8062360 con 0x7f32b4038fc0 2024-08-18T23:35:02.764 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.230+0000 7f32ce15e640 1 -- 172.21.15.78:0/3568575316 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f32b4038fc0 msgr2=0x7f32b403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:02.764 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.230+0000 7f32ce15e640 1 --2- 172.21.15.78:0/3568575316 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f32b4038fc0 0x7f32b403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f32b00098c0 tx=0x7f32b0006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:02.764 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.230+0000 7f32ce15e640 1 -- 172.21.15.78:0/3568575316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32c8107300 msgr2=0x7f32c819b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:02.764 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.230+0000 7f32ce15e640 1 --2- 172.21.15.78:0/3568575316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32c8107300 0x7f32c819b5e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f32b8031000 tx=0x7f32b80326e0 comp rx=0 tx=0).stop 2024-08-18T23:35:02.764 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.230+0000 7f32ce15e640 1 -- 172.21.15.78:0/3568575316 shutdown_connections 2024-08-18T23:35:02.765 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.230+0000 7f32ce15e640 1 --2- 172.21.15.78:0/3568575316 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f32b4038fc0 0x7f32b403b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:02.765 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.230+0000 7f32ce15e640 1 --2- 172.21.15.78:0/3568575316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32c8107300 0x7f32c819b5e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:02.765 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.230+0000 7f32ce15e640 1 -- 172.21.15.78:0/3568575316 >> 172.21.15.78:0/3568575316 conn(0x7f32c8102b30 msgr2=0x7f32c8103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:02.765 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.230+0000 7f32ce15e640 1 -- 172.21.15.78:0/3568575316 shutdown_connections 2024-08-18T23:35:02.765 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:02.230+0000 7f32ce15e640 1 -- 172.21.15.78:0/3568575316 wait complete. 2024-08-18T23:35:02.765 INFO:teuthology.orchestra.run.smithi078.stdout:Generating ssh key... 2024-08-18T23:35:03.164 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:03 smithi078 ceph-mon[18440]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:35:03.720 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: Generating public/private rsa key pair. 2024-08-18T23:35:03.720 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: Your identification has been saved in /tmp/tmp4ke3y_p3/key 2024-08-18T23:35:03.720 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: Your public key has been saved in /tmp/tmp4ke3y_p3/key.pub 2024-08-18T23:35:03.720 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: The key fingerprint is: 2024-08-18T23:35:03.720 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: SHA256:3f9q7QAfkYeHRF7EbYgtriLe4enSVOOqTxahW4LAMjo ceph-4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:35:03.720 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: The key's randomart image is: 2024-08-18T23:35:03.720 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: +---[RSA 3072]----+ 2024-08-18T23:35:03.720 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: | +o++| 2024-08-18T23:35:03.720 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: | . ooo=+| 2024-08-18T23:35:03.720 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: |o o . . .*.o| 2024-08-18T23:35:03.721 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: |.o . . . oo.. + | 2024-08-18T23:35:03.721 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: |E . o So.oo . | 2024-08-18T23:35:03.721 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: | . .++.o + . | 2024-08-18T23:35:03.721 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: | ..*o= +. | 2024-08-18T23:35:03.721 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: | oo* .o.| 2024-08-18T23:35:03.721 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: | .=o ..oo| 2024-08-18T23:35:03.721 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: +----[SHA256]-----+ 2024-08-18T23:35:04.008 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.163+0000 7ff723794640 1 Processor -- start 2024-08-18T23:35:04.008 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.163+0000 7ff723794640 1 -- start start 2024-08-18T23:35:04.008 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.163+0000 7ff723794640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71c07dd30 0x7ff71c07c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:04.008 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.163+0000 7ff723794640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c07c7e0 con 0x7ff71c07dd30 2024-08-18T23:35:04.008 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.163+0000 7ff721530640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71c07dd30 0x7ff71c07c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:04.008 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.163+0000 7ff721530640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71c07dd30 0x7ff71c07c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48878/0 (socket says 172.21.15.78:48878) 2024-08-18T23:35:04.008 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.163+0000 7ff721530640 1 -- 172.21.15.78:0/3381824323 learned_addr learned my addr 172.21.15.78:0/3381824323 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:04.008 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.163+0000 7ff721530640 1 -- 172.21.15.78:0/3381824323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71c07c960 con 0x7ff71c07dd30 2024-08-18T23:35:04.008 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.164+0000 7ff721530640 1 --2- 172.21.15.78:0/3381824323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71c07dd30 0x7ff71c07c2a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff718009820 tx=0x7ff71802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432ae8055f22ebf0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:04.009 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.164+0000 7ff70ffff640 1 -- 172.21.15.78:0/3381824323 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff718037070 con 0x7ff71c07dd30 2024-08-18T23:35:04.009 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.164+0000 7ff70ffff640 1 -- 172.21.15.78:0/3381824323 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff718032900 con 0x7ff71c07dd30 2024-08-18T23:35:04.009 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.164+0000 7ff70ffff640 1 -- 172.21.15.78:0/3381824323 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff718030850 con 0x7ff71c07dd30 2024-08-18T23:35:04.009 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.165+0000 7ff723794640 1 -- 172.21.15.78:0/3381824323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71c07dd30 msgr2=0x7ff71c07c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:04.009 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.165+0000 7ff723794640 1 --2- 172.21.15.78:0/3381824323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71c07dd30 0x7ff71c07c2a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff718009820 tx=0x7ff71802a270 comp rx=0 tx=0).stop 2024-08-18T23:35:04.009 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.165+0000 7ff723794640 1 -- 172.21.15.78:0/3381824323 shutdown_connections 2024-08-18T23:35:04.009 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.165+0000 7ff723794640 1 --2- 172.21.15.78:0/3381824323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71c07dd30 0x7ff71c07c2a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:04.009 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.165+0000 7ff723794640 1 -- 172.21.15.78:0/3381824323 >> 172.21.15.78:0/3381824323 conn(0x7ff71c102b30 msgr2=0x7ff71c104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:04.009 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.165+0000 7ff723794640 1 -- 172.21.15.78:0/3381824323 shutdown_connections 2024-08-18T23:35:04.009 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.165+0000 7ff723794640 1 -- 172.21.15.78:0/3381824323 wait complete. 2024-08-18T23:35:04.009 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.165+0000 7ff723794640 1 Processor -- start 2024-08-18T23:35:04.009 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.166+0000 7ff723794640 1 -- start start 2024-08-18T23:35:04.009 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.166+0000 7ff723794640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71c07dd30 0x7ff71c19d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:04.009 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.166+0000 7ff723794640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c10d880 con 0x7ff71c07dd30 2024-08-18T23:35:04.009 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.166+0000 7ff721530640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71c07dd30 0x7ff71c19d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:04.010 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.166+0000 7ff721530640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71c07dd30 0x7ff71c19d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48886/0 (socket says 172.21.15.78:48886) 2024-08-18T23:35:04.010 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.166+0000 7ff721530640 1 -- 172.21.15.78:0/2405506508 learned_addr learned my addr 172.21.15.78:0/2405506508 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:04.010 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.166+0000 7ff721530640 1 -- 172.21.15.78:0/2405506508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71c19ddc0 con 0x7ff71c07dd30 2024-08-18T23:35:04.010 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.166+0000 7ff721530640 1 --2- 172.21.15.78:0/2405506508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71c07dd30 0x7ff71c19d880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff718031000 tx=0x7ff7180326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:04.010 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.166+0000 7ff70e7fc640 1 -- 172.21.15.78:0/2405506508 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff718042070 con 0x7ff71c07dd30 2024-08-18T23:35:04.010 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.166+0000 7ff723794640 1 -- 172.21.15.78:0/2405506508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71c19e050 con 0x7ff71c07dd30 2024-08-18T23:35:04.010 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.166+0000 7ff70e7fc640 1 -- 172.21.15.78:0/2405506508 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff718032be0 con 0x7ff71c07dd30 2024-08-18T23:35:04.010 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.166+0000 7ff70e7fc640 1 -- 172.21.15.78:0/2405506508 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff718037040 con 0x7ff71c07dd30 2024-08-18T23:35:04.010 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.166+0000 7ff723794640 1 -- 172.21.15.78:0/2405506508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff71c1a0d40 con 0x7ff71c07dd30 2024-08-18T23:35:04.010 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.167+0000 7ff70e7fc640 1 -- 172.21.15.78:0/2405506508 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff718045070 con 0x7ff71c07dd30 2024-08-18T23:35:04.010 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.167+0000 7ff723794640 1 -- 172.21.15.78:0/2405506508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff71c07c320 con 0x7ff71c07dd30 2024-08-18T23:35:04.010 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.167+0000 7ff70e7fc640 1 --2- 172.21.15.78:0/2405506508 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7ff708038f70 0x7ff70803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:04.010 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.168+0000 7ff70e7fc640 1 -- 172.21.15.78:0/2405506508 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff71806c070 con 0x7ff71c07dd30 2024-08-18T23:35:04.010 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.168+0000 7ff720d2f640 1 --2- 172.21.15.78:0/2405506508 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7ff708038f70 0x7ff70803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:04.011 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.168+0000 7ff720d2f640 1 --2- 172.21.15.78:0/2405506508 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7ff708038f70 0x7ff70803b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff7100098c0 tx=0x7ff710006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:04.011 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.172+0000 7ff70e7fc640 1 -- 172.21.15.78:0/2405506508 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff71803cc60 con 0x7ff71c07dd30 2024-08-18T23:35:04.011 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.342+0000 7ff723794640 1 -- 172.21.15.78:0/2405506508 --> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7ff71c062360 con 0x7ff708038f70 2024-08-18T23:35:04.011 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.472+0000 7ff70e7fc640 1 -- 172.21.15.78:0/2405506508 <== mgr.14118 v2:172.21.15.78:6800/311758909 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff71c062360 con 0x7ff708038f70 2024-08-18T23:35:04.011 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.474+0000 7ff723794640 1 -- 172.21.15.78:0/2405506508 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7ff708038f70 msgr2=0x7ff70803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:04.011 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.475+0000 7ff723794640 1 --2- 172.21.15.78:0/2405506508 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7ff708038f70 0x7ff70803b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff7100098c0 tx=0x7ff710006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:04.011 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.475+0000 7ff723794640 1 -- 172.21.15.78:0/2405506508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71c07dd30 msgr2=0x7ff71c19d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:04.011 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.475+0000 7ff723794640 1 --2- 172.21.15.78:0/2405506508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71c07dd30 0x7ff71c19d880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff718031000 tx=0x7ff7180326e0 comp rx=0 tx=0).stop 2024-08-18T23:35:04.011 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.475+0000 7ff723794640 1 -- 172.21.15.78:0/2405506508 shutdown_connections 2024-08-18T23:35:04.011 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.475+0000 7ff723794640 1 --2- 172.21.15.78:0/2405506508 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7ff708038f70 0x7ff70803b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:04.011 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.475+0000 7ff723794640 1 --2- 172.21.15.78:0/2405506508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff71c07dd30 0x7ff71c19d880 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:04.011 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.475+0000 7ff723794640 1 -- 172.21.15.78:0/2405506508 >> 172.21.15.78:0/2405506508 conn(0x7ff71c102b30 msgr2=0x7ff71c1039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:04.011 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.475+0000 7ff723794640 1 -- 172.21.15.78:0/2405506508 shutdown_connections 2024-08-18T23:35:04.011 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:03.475+0000 7ff723794640 1 -- 172.21.15.78:0/2405506508 wait complete. 2024-08-18T23:35:04.336 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:04 smithi078 ceph-mon[18440]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:35:04.337 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:04 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' 2024-08-18T23:35:04.337 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:04 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' 2024-08-18T23:35:05.189 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXqMaE+Nb8FG7cpy3luBBwAJwLn5EGz4p2o1aOY3cuTfjOR+tuLDg3dEhTmOzsePQ/2YEWym2qTBQJ4zu4QkX7fBLFNETgMqpCjQykA0pUJnFOSo42gVZ+fwa/1r1liNntjHqtYst0KDcb7MB8/XKzVECgHOtI6pcoC4JQogMcL62TL+51lGUdV5g7MLeoTsV7koshfEvUKkIo5hc+w4HB21EOqQJ+OZ7hbQtj/4yHOYV2cJwTSd3ULZ4jcQiMPK0LLyUaMRkod8CINFYo/0i32m1VcJYHrOKfPF04RiGxxw5QQsK3G1S9uWWTqCxekuwyxOTXRvRSP5/OfTUpg4SUVBZdOyN5V5wnkQjPi+UWAYYc3GnHrn7NSUSfQnpqvS9OWdzh4SFxm+qSodPntUIxGwQpRCCBNHi2Lke8uYJNxiHTUtj3BanH6d+y+tY2j2IKPS+gCzmelmOOQ1Q8NVCZR3cprRfuIs23APM029DBL0KSRDhPN/LXbKjdXb2dCHc= ceph-4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:35:05.189 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.411+0000 7f319df23640 1 Processor -- start 2024-08-18T23:35:05.189 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.411+0000 7f319df23640 1 -- start start 2024-08-18T23:35:05.189 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.411+0000 7f319df23640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31981061d0 0x7f31981065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.411+0000 7f319df23640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3198106ba0 con 0x7f31981061d0 2024-08-18T23:35:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.411+0000 7f31977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31981061d0 0x7f31981065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.411+0000 7f31977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31981061d0 0x7f31981065d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48902/0 (socket says 172.21.15.78:48902) 2024-08-18T23:35:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.411+0000 7f31977fe640 1 -- 172.21.15.78:0/3907153000 learned_addr learned my addr 172.21.15.78:0/3907153000 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.411+0000 7f31977fe640 1 -- 172.21.15.78:0/3907153000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31981073d0 con 0x7f31981061d0 2024-08-18T23:35:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.412+0000 7f31977fe640 1 --2- 172.21.15.78:0/3907153000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31981061d0 0x7f31981065d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3188009820 tx=0x7f318802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9caf3d3a76ff0eac server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.412+0000 7f31967fc640 1 -- 172.21.15.78:0/3907153000 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3188037070 con 0x7f31981061d0 2024-08-18T23:35:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.412+0000 7f31967fc640 1 -- 172.21.15.78:0/3907153000 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3188032900 con 0x7f31981061d0 2024-08-18T23:35:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.412+0000 7f31967fc640 1 -- 172.21.15.78:0/3907153000 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3188030850 con 0x7f31981061d0 2024-08-18T23:35:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.413+0000 7f319df23640 1 -- 172.21.15.78:0/3907153000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31981061d0 msgr2=0x7f31981065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.413+0000 7f319df23640 1 --2- 172.21.15.78:0/3907153000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31981061d0 0x7f31981065d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3188009820 tx=0x7f318802a270 comp rx=0 tx=0).stop 2024-08-18T23:35:05.191 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.413+0000 7f319df23640 1 -- 172.21.15.78:0/3907153000 shutdown_connections 2024-08-18T23:35:05.191 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.413+0000 7f319df23640 1 --2- 172.21.15.78:0/3907153000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31981061d0 0x7f31981065d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:05.191 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.413+0000 7f319df23640 1 -- 172.21.15.78:0/3907153000 >> 172.21.15.78:0/3907153000 conn(0x7f31981019a0 msgr2=0x7f3198103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:05.191 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.413+0000 7f319df23640 1 -- 172.21.15.78:0/3907153000 shutdown_connections 2024-08-18T23:35:05.191 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.413+0000 7f319df23640 1 -- 172.21.15.78:0/3907153000 wait complete. 2024-08-18T23:35:05.191 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.414+0000 7f319df23640 1 Processor -- start 2024-08-18T23:35:05.191 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.414+0000 7f319df23640 1 -- start start 2024-08-18T23:35:05.191 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.414+0000 7f319df23640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31981061d0 0x7f319819b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:05.191 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.414+0000 7f319df23640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f319810a780 con 0x7f31981061d0 2024-08-18T23:35:05.191 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.414+0000 7f31977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31981061d0 0x7f319819b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:05.191 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.414+0000 7f31977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31981061d0 0x7f319819b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48912/0 (socket says 172.21.15.78:48912) 2024-08-18T23:35:05.191 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.414+0000 7f31977fe640 1 -- 172.21.15.78:0/2551424571 learned_addr learned my addr 172.21.15.78:0/2551424571 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:05.191 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.414+0000 7f31977fe640 1 -- 172.21.15.78:0/2551424571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319819bb70 con 0x7f31981061d0 2024-08-18T23:35:05.191 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.414+0000 7f31977fe640 1 --2- 172.21.15.78:0/2551424571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31981061d0 0x7f319819b630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3188009940 tx=0x7f31880326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:05.192 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.415+0000 7f3194ff9640 1 -- 172.21.15.78:0/2551424571 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3188042070 con 0x7f31981061d0 2024-08-18T23:35:05.192 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.415+0000 7f3194ff9640 1 -- 172.21.15.78:0/2551424571 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3188032be0 con 0x7f31981061d0 2024-08-18T23:35:05.192 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.415+0000 7f3194ff9640 1 -- 172.21.15.78:0/2551424571 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3188037040 con 0x7f31981061d0 2024-08-18T23:35:05.192 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.415+0000 7f319df23640 1 -- 172.21.15.78:0/2551424571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319819be00 con 0x7f31981061d0 2024-08-18T23:35:05.192 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.415+0000 7f319df23640 1 -- 172.21.15.78:0/2551424571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f319819eaf0 con 0x7f31981061d0 2024-08-18T23:35:05.192 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.415+0000 7f3194ff9640 1 -- 172.21.15.78:0/2551424571 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3188045070 con 0x7f31981061d0 2024-08-18T23:35:05.192 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.416+0000 7f319df23640 1 -- 172.21.15.78:0/2551424571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f319810a900 con 0x7f31981061d0 2024-08-18T23:35:05.192 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.416+0000 7f3194ff9640 1 --2- 172.21.15.78:0/2551424571 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f3180038f70 0x7f318003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:05.192 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.416+0000 7f3194ff9640 1 -- 172.21.15.78:0/2551424571 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f318806bed0 con 0x7f31981061d0 2024-08-18T23:35:05.192 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.416+0000 7f3196ffd640 1 --2- 172.21.15.78:0/2551424571 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f3180038f70 0x7f318003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:05.192 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.416+0000 7f3196ffd640 1 --2- 172.21.15.78:0/2551424571 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f3180038f70 0x7f318003b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f318c0098c0 tx=0x7f318c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:05.192 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.421+0000 7f3194ff9640 1 -- 172.21.15.78:0/2551424571 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f318804b170 con 0x7f31981061d0 2024-08-18T23:35:05.192 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.587+0000 7f319df23640 1 -- 172.21.15.78:0/2551424571 --> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f3198062360 con 0x7f3180038f70 2024-08-18T23:35:05.192 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.588+0000 7f3194ff9640 1 -- 172.21.15.78:0/2551424571 <== mgr.14118 v2:172.21.15.78:6800/311758909 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f3198062360 con 0x7f3180038f70 2024-08-18T23:35:05.193 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.590+0000 7f319df23640 1 -- 172.21.15.78:0/2551424571 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f3180038f70 msgr2=0x7f318003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:05.193 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.590+0000 7f319df23640 1 --2- 172.21.15.78:0/2551424571 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f3180038f70 0x7f318003b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f318c0098c0 tx=0x7f318c006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:05.193 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.590+0000 7f319df23640 1 -- 172.21.15.78:0/2551424571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31981061d0 msgr2=0x7f319819b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:05.193 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.590+0000 7f319df23640 1 --2- 172.21.15.78:0/2551424571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31981061d0 0x7f319819b630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3188009940 tx=0x7f31880326e0 comp rx=0 tx=0).stop 2024-08-18T23:35:05.193 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.590+0000 7f319df23640 1 -- 172.21.15.78:0/2551424571 shutdown_connections 2024-08-18T23:35:05.193 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.590+0000 7f319df23640 1 --2- 172.21.15.78:0/2551424571 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f3180038f70 0x7f318003b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:05.193 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.590+0000 7f319df23640 1 --2- 172.21.15.78:0/2551424571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31981061d0 0x7f319819b630 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:05.193 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.590+0000 7f319df23640 1 -- 172.21.15.78:0/2551424571 >> 172.21.15.78:0/2551424571 conn(0x7f31981019a0 msgr2=0x7f3198102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:05.193 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.591+0000 7f319df23640 1 -- 172.21.15.78:0/2551424571 shutdown_connections 2024-08-18T23:35:05.193 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:04.591+0000 7f319df23640 1 -- 172.21.15.78:0/2551424571 wait complete. 2024-08-18T23:35:05.193 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-08-18T23:35:05.193 INFO:teuthology.orchestra.run.smithi078.stdout:Adding key to root@localhost authorized_keys... 2024-08-18T23:35:05.194 INFO:teuthology.orchestra.run.smithi078.stdout:Adding host smithi078... 2024-08-18T23:35:05.407 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:05 smithi078 ceph-mon[18440]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:35:05.407 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:05 smithi078 ceph-mon[18440]: Generating ssh key... 2024-08-18T23:35:06.303 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:06 smithi078 ceph-mon[18440]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:35:07.396 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:07 smithi078 ceph-mon[18440]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi078", "addr": "172.21.15.78", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:35:08.388 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:08 smithi078 ceph-mon[18440]: Deploying cephadm binary to smithi078 2024-08-18T23:35:09.539 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Added host 'smithi078' with addr '172.21.15.78' 2024-08-18T23:35:09.539 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.579+0000 7fcfabaaa640 1 Processor -- start 2024-08-18T23:35:09.539 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.579+0000 7fcfabaaa640 1 -- start start 2024-08-18T23:35:09.540 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.580+0000 7fcfabaaa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfa4107d50 0x7fcfa410a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:09.540 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.580+0000 7fcfabaaa640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa4075550 con 0x7fcfa4107d50 2024-08-18T23:35:09.540 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.580+0000 7fcfa9846640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfa4107d50 0x7fcfa410a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:09.540 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.580+0000 7fcfa9846640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfa4107d50 0x7fcfa410a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48918/0 (socket says 172.21.15.78:48918) 2024-08-18T23:35:09.540 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.580+0000 7fcfa9846640 1 -- 172.21.15.78:0/392751045 learned_addr learned my addr 172.21.15.78:0/392751045 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:09.540 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.580+0000 7fcfa9846640 1 -- 172.21.15.78:0/392751045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa410a680 con 0x7fcfa4107d50 2024-08-18T23:35:09.540 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.581+0000 7fcfa9846640 1 --2- 172.21.15.78:0/392751045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfa4107d50 0x7fcfa410a140 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009820 tx=0x7fcfa002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff8e208399e3001 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:09.540 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.581+0000 7fcfa8844640 1 -- 172.21.15.78:0/392751045 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfa0037070 con 0x7fcfa4107d50 2024-08-18T23:35:09.540 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.581+0000 7fcfa8844640 1 -- 172.21.15.78:0/392751045 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcfa0032900 con 0x7fcfa4107d50 2024-08-18T23:35:09.540 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.581+0000 7fcfa8844640 1 -- 172.21.15.78:0/392751045 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfa0030850 con 0x7fcfa4107d50 2024-08-18T23:35:09.540 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.581+0000 7fcfabaaa640 1 -- 172.21.15.78:0/392751045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfa4107d50 msgr2=0x7fcfa410a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:09.540 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.581+0000 7fcfabaaa640 1 --2- 172.21.15.78:0/392751045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfa4107d50 0x7fcfa410a140 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009820 tx=0x7fcfa002a270 comp rx=0 tx=0).stop 2024-08-18T23:35:09.540 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.582+0000 7fcfabaaa640 1 -- 172.21.15.78:0/392751045 shutdown_connections 2024-08-18T23:35:09.541 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.582+0000 7fcfabaaa640 1 --2- 172.21.15.78:0/392751045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfa4107d50 0x7fcfa410a140 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:09.541 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.582+0000 7fcfabaaa640 1 -- 172.21.15.78:0/392751045 >> 172.21.15.78:0/392751045 conn(0x7fcfa41018f0 msgr2=0x7fcfa4103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:09.541 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.582+0000 7fcfabaaa640 1 -- 172.21.15.78:0/392751045 shutdown_connections 2024-08-18T23:35:09.541 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.582+0000 7fcfabaaa640 1 -- 172.21.15.78:0/392751045 wait complete. 2024-08-18T23:35:09.541 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.582+0000 7fcfabaaa640 1 Processor -- start 2024-08-18T23:35:09.541 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.582+0000 7fcfabaaa640 1 -- start start 2024-08-18T23:35:09.541 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.582+0000 7fcfabaaa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfa4107d50 0x7fcfa419b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:09.541 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.582+0000 7fcfabaaa640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa410b6c0 con 0x7fcfa4107d50 2024-08-18T23:35:09.541 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.582+0000 7fcfa9846640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfa4107d50 0x7fcfa419b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:09.541 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.582+0000 7fcfa9846640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfa4107d50 0x7fcfa419b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48922/0 (socket says 172.21.15.78:48922) 2024-08-18T23:35:09.541 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.582+0000 7fcfa9846640 1 -- 172.21.15.78:0/689742734 learned_addr learned my addr 172.21.15.78:0/689742734 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:09.541 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.583+0000 7fcfa9846640 1 -- 172.21.15.78:0/689742734 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa419bb00 con 0x7fcfa4107d50 2024-08-18T23:35:09.542 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.583+0000 7fcfa9846640 1 --2- 172.21.15.78:0/689742734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfa4107d50 0x7fcfa419b5c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0031000 tx=0x7fcfa00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:09.542 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.583+0000 7fcf9affd640 1 -- 172.21.15.78:0/689742734 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfa0042070 con 0x7fcfa4107d50 2024-08-18T23:35:09.542 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.583+0000 7fcfabaaa640 1 -- 172.21.15.78:0/689742734 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa419bd90 con 0x7fcfa4107d50 2024-08-18T23:35:09.542 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.583+0000 7fcf9affd640 1 -- 172.21.15.78:0/689742734 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcfa0032be0 con 0x7fcfa4107d50 2024-08-18T23:35:09.542 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.583+0000 7fcf9affd640 1 -- 172.21.15.78:0/689742734 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfa0037040 con 0x7fcfa4107d50 2024-08-18T23:35:09.542 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.583+0000 7fcfabaaa640 1 -- 172.21.15.78:0/689742734 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa419ea80 con 0x7fcfa4107d50 2024-08-18T23:35:09.542 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.584+0000 7fcf9affd640 1 -- 172.21.15.78:0/689742734 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcfa0045070 con 0x7fcfa4107d50 2024-08-18T23:35:09.542 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.584+0000 7fcf9affd640 1 --2- 172.21.15.78:0/689742734 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7fcf90038f70 0x7fcf9003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:09.542 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.584+0000 7fcfabaaa640 1 -- 172.21.15.78:0/689742734 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa41081b0 con 0x7fcfa4107d50 2024-08-18T23:35:09.542 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.584+0000 7fcf9affd640 1 -- 172.21.15.78:0/689742734 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcfa006bf00 con 0x7fcfa4107d50 2024-08-18T23:35:09.542 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.584+0000 7fcfa9045640 1 --2- 172.21.15.78:0/689742734 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7fcf90038f70 0x7fcf9003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:09.543 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.585+0000 7fcfa9045640 1 --2- 172.21.15.78:0/689742734 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7fcf90038f70 0x7fcf9003b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcf940098c0 tx=0x7fcf94006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:09.543 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.589+0000 7fcf9affd640 1 -- 172.21.15.78:0/689742734 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fcfa005c070 con 0x7fcfa4107d50 2024-08-18T23:35:09.543 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:05.758+0000 7fcfabaaa640 1 -- 172.21.15.78:0/689742734 --> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi078", "addr": "172.21.15.78", "target": ["mon-mgr", ""]}) -- 0x7fcfa4062360 con 0x7fcf90038f70 2024-08-18T23:35:09.543 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:08.970+0000 7fcf9affd640 1 -- 172.21.15.78:0/689742734 <== mgr.14118 v2:172.21.15.78:6800/311758909 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fcfa4062360 con 0x7fcf90038f70 2024-08-18T23:35:09.543 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:08.972+0000 7fcfabaaa640 1 -- 172.21.15.78:0/689742734 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7fcf90038f70 msgr2=0x7fcf9003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:09.543 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:08.972+0000 7fcfabaaa640 1 --2- 172.21.15.78:0/689742734 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7fcf90038f70 0x7fcf9003b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcf940098c0 tx=0x7fcf94006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:09.543 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:08.972+0000 7fcfabaaa640 1 -- 172.21.15.78:0/689742734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfa4107d50 msgr2=0x7fcfa419b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:09.543 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:08.972+0000 7fcfabaaa640 1 --2- 172.21.15.78:0/689742734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfa4107d50 0x7fcfa419b5c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0031000 tx=0x7fcfa00326e0 comp rx=0 tx=0).stop 2024-08-18T23:35:09.543 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:08.972+0000 7fcfabaaa640 1 -- 172.21.15.78:0/689742734 shutdown_connections 2024-08-18T23:35:09.543 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:08.972+0000 7fcfabaaa640 1 --2- 172.21.15.78:0/689742734 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7fcf90038f70 0x7fcf9003b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:09.543 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:08.972+0000 7fcfabaaa640 1 --2- 172.21.15.78:0/689742734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfa4107d50 0x7fcfa419b5c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:09.543 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:08.972+0000 7fcfabaaa640 1 -- 172.21.15.78:0/689742734 >> 172.21.15.78:0/689742734 conn(0x7fcfa41018f0 msgr2=0x7fcfa4102710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:09.543 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:08.973+0000 7fcfabaaa640 1 -- 172.21.15.78:0/689742734 shutdown_connections 2024-08-18T23:35:09.543 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:08.973+0000 7fcfabaaa640 1 -- 172.21.15.78:0/689742734 wait complete. 2024-08-18T23:35:09.544 INFO:teuthology.orchestra.run.smithi078.stdout:Deploying unmanaged mon service... 2024-08-18T23:35:10.112 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:09 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' 2024-08-18T23:35:10.112 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:09 smithi078 ceph-mon[18440]: Added host smithi078 2024-08-18T23:35:10.113 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:09 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:35:10.883 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-08-18T23:35:10.883 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.905+0000 7f7439bd7640 1 Processor -- start 2024-08-18T23:35:10.883 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.905+0000 7f7439bd7640 1 -- start start 2024-08-18T23:35:10.883 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.906+0000 7f7439bd7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7434106180 0x7f7434106580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:10.883 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.906+0000 7f7439bd7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7434106b50 con 0x7f7434106180 2024-08-18T23:35:10.883 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.906+0000 7f74337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7434106180 0x7f7434106580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:10.883 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.906+0000 7f74337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7434106180 0x7f7434106580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45770/0 (socket says 172.21.15.78:45770) 2024-08-18T23:35:10.883 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.906+0000 7f74337fe640 1 -- 172.21.15.78:0/1466044587 learned_addr learned my addr 172.21.15.78:0/1466044587 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:10.883 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.906+0000 7f74337fe640 1 -- 172.21.15.78:0/1466044587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7434107380 con 0x7f7434106180 2024-08-18T23:35:10.883 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.907+0000 7f74337fe640 1 --2- 172.21.15.78:0/1466044587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7434106180 0x7f7434106580 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7424009820 tx=0x7f742402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d68081fbc3b1911 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:10.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.907+0000 7f7432ffd640 1 -- 172.21.15.78:0/1466044587 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7424037070 con 0x7f7434106180 2024-08-18T23:35:10.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.907+0000 7f7432ffd640 1 -- 172.21.15.78:0/1466044587 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7424032900 con 0x7f7434106180 2024-08-18T23:35:10.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.907+0000 7f7432ffd640 1 -- 172.21.15.78:0/1466044587 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7424030850 con 0x7f7434106180 2024-08-18T23:35:10.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.907+0000 7f7439bd7640 1 -- 172.21.15.78:0/1466044587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7434106180 msgr2=0x7f7434106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:10.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.907+0000 7f7439bd7640 1 --2- 172.21.15.78:0/1466044587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7434106180 0x7f7434106580 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7424009820 tx=0x7f742402a270 comp rx=0 tx=0).stop 2024-08-18T23:35:10.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.908+0000 7f7439bd7640 1 -- 172.21.15.78:0/1466044587 shutdown_connections 2024-08-18T23:35:10.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.908+0000 7f7439bd7640 1 --2- 172.21.15.78:0/1466044587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7434106180 0x7f7434106580 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:10.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.908+0000 7f7439bd7640 1 -- 172.21.15.78:0/1466044587 >> 172.21.15.78:0/1466044587 conn(0x7f74341018f0 msgr2=0x7f7434103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:10.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.908+0000 7f7439bd7640 1 -- 172.21.15.78:0/1466044587 shutdown_connections 2024-08-18T23:35:10.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.908+0000 7f7439bd7640 1 -- 172.21.15.78:0/1466044587 wait complete. 2024-08-18T23:35:10.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.908+0000 7f7439bd7640 1 Processor -- start 2024-08-18T23:35:10.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.908+0000 7f7439bd7640 1 -- start start 2024-08-18T23:35:10.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.909+0000 7f7439bd7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7434106180 0x7f74341913b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:10.884 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.909+0000 7f7439bd7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f743410a730 con 0x7f7434106180 2024-08-18T23:35:10.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.909+0000 7f74337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7434106180 0x7f74341913b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:10.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.909+0000 7f74337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7434106180 0x7f74341913b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45772/0 (socket says 172.21.15.78:45772) 2024-08-18T23:35:10.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.909+0000 7f74337fe640 1 -- 172.21.15.78:0/2101930135 learned_addr learned my addr 172.21.15.78:0/2101930135 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:10.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.909+0000 7f74337fe640 1 -- 172.21.15.78:0/2101930135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74341918f0 con 0x7f7434106180 2024-08-18T23:35:10.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.909+0000 7f74337fe640 1 --2- 172.21.15.78:0/2101930135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7434106180 0x7f74341913b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7424009940 tx=0x7f74240326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:10.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.909+0000 7f74317fa640 1 -- 172.21.15.78:0/2101930135 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7424042070 con 0x7f7434106180 2024-08-18T23:35:10.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.909+0000 7f74317fa640 1 -- 172.21.15.78:0/2101930135 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7424032be0 con 0x7f7434106180 2024-08-18T23:35:10.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.909+0000 7f7439bd7640 1 -- 172.21.15.78:0/2101930135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743418fae0 con 0x7f7434106180 2024-08-18T23:35:10.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.909+0000 7f74317fa640 1 -- 172.21.15.78:0/2101930135 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7424037040 con 0x7f7434106180 2024-08-18T23:35:10.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.909+0000 7f7439bd7640 1 -- 172.21.15.78:0/2101930135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7434190020 con 0x7f7434106180 2024-08-18T23:35:10.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.910+0000 7f74317fa640 1 -- 172.21.15.78:0/2101930135 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7424045070 con 0x7f7434106180 2024-08-18T23:35:10.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.910+0000 7f7439bd7640 1 -- 172.21.15.78:0/2101930135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f743410a8b0 con 0x7f7434106180 2024-08-18T23:35:10.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.910+0000 7f74317fa640 1 --2- 172.21.15.78:0/2101930135 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f7420038f70 0x7f742003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:10.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.910+0000 7f74317fa640 1 -- 172.21.15.78:0/2101930135 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f742406bf40 con 0x7f7434106180 2024-08-18T23:35:10.885 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.910+0000 7f742affd640 1 --2- 172.21.15.78:0/2101930135 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f7420038f70 0x7f742003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:10.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.911+0000 7f742affd640 1 --2- 172.21.15.78:0/2101930135 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f7420038f70 0x7f742003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f741c0098c0 tx=0x7f741c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:10.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:09.915+0000 7f74317fa640 1 -- 172.21.15.78:0/2101930135 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7424045e10 con 0x7f7434106180 2024-08-18T23:35:10.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:10.111+0000 7f7439bd7640 1 -- 172.21.15.78:0/2101930135 --> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f7434194ef0 con 0x7f7420038f70 2024-08-18T23:35:10.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:10.115+0000 7f74317fa640 1 -- 172.21.15.78:0/2101930135 <== mgr.14118 v2:172.21.15.78:6800/311758909 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7434194ef0 con 0x7f7420038f70 2024-08-18T23:35:10.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:10.117+0000 7f7439bd7640 1 -- 172.21.15.78:0/2101930135 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f7420038f70 msgr2=0x7f742003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:10.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:10.117+0000 7f7439bd7640 1 --2- 172.21.15.78:0/2101930135 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f7420038f70 0x7f742003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f741c0098c0 tx=0x7f741c006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:10.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:10.117+0000 7f7439bd7640 1 -- 172.21.15.78:0/2101930135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7434106180 msgr2=0x7f74341913b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:10.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:10.117+0000 7f7439bd7640 1 --2- 172.21.15.78:0/2101930135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7434106180 0x7f74341913b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7424009940 tx=0x7f74240326e0 comp rx=0 tx=0).stop 2024-08-18T23:35:10.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:10.118+0000 7f7439bd7640 1 -- 172.21.15.78:0/2101930135 shutdown_connections 2024-08-18T23:35:10.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:10.118+0000 7f7439bd7640 1 --2- 172.21.15.78:0/2101930135 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f7420038f70 0x7f742003b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:10.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:10.118+0000 7f7439bd7640 1 --2- 172.21.15.78:0/2101930135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7434106180 0x7f74341913b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:10.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:10.118+0000 7f7439bd7640 1 -- 172.21.15.78:0/2101930135 >> 172.21.15.78:0/2101930135 conn(0x7f74341018f0 msgr2=0x7f74341028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:10.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:10.118+0000 7f7439bd7640 1 -- 172.21.15.78:0/2101930135 shutdown_connections 2024-08-18T23:35:10.886 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:10.118+0000 7f7439bd7640 1 -- 172.21.15.78:0/2101930135 wait complete. 2024-08-18T23:35:10.887 INFO:teuthology.orchestra.run.smithi078.stdout:Deploying unmanaged mgr service... 2024-08-18T23:35:11.201 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:11 smithi078 ceph-mon[18440]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:35:11.202 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:11 smithi078 ceph-mon[18440]: Saving service mon spec with placement count:5 2024-08-18T23:35:11.202 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:11 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' 2024-08-18T23:35:12.080 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-08-18T23:35:12.080 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.283+0000 7f2ae28f0640 1 Processor -- start 2024-08-18T23:35:12.080 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.283+0000 7f2ae28f0640 1 -- start start 2024-08-18T23:35:12.080 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.283+0000 7f2ae28f0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2adc107d20 0x7f2adc10a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:12.080 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.283+0000 7f2ae28f0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc07be30 con 0x7f2adc107d20 2024-08-18T23:35:12.080 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.283+0000 7f2adbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2adc107d20 0x7f2adc10a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:12.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.283+0000 7f2adbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2adc107d20 0x7f2adc10a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45778/0 (socket says 172.21.15.78:45778) 2024-08-18T23:35:12.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.283+0000 7f2adbfff640 1 -- 172.21.15.78:0/902450234 learned_addr learned my addr 172.21.15.78:0/902450234 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:12.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.283+0000 7f2adbfff640 1 -- 172.21.15.78:0/902450234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2adc10a650 con 0x7f2adc107d20 2024-08-18T23:35:12.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.284+0000 7f2adbfff640 1 --2- 172.21.15.78:0/902450234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2adc107d20 0x7f2adc10a110 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009820 tx=0x7f2acc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189576ca96fe50e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:12.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.284+0000 7f2adb7fe640 1 -- 172.21.15.78:0/902450234 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2acc037070 con 0x7f2adc107d20 2024-08-18T23:35:12.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.284+0000 7f2adb7fe640 1 -- 172.21.15.78:0/902450234 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2acc032900 con 0x7f2adc107d20 2024-08-18T23:35:12.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.284+0000 7f2adb7fe640 1 -- 172.21.15.78:0/902450234 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2acc030850 con 0x7f2adc107d20 2024-08-18T23:35:12.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.285+0000 7f2ae28f0640 1 -- 172.21.15.78:0/902450234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2adc107d20 msgr2=0x7f2adc10a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:12.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.285+0000 7f2ae28f0640 1 --2- 172.21.15.78:0/902450234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2adc107d20 0x7f2adc10a110 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009820 tx=0x7f2acc02a270 comp rx=0 tx=0).stop 2024-08-18T23:35:12.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.285+0000 7f2ae28f0640 1 -- 172.21.15.78:0/902450234 shutdown_connections 2024-08-18T23:35:12.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.285+0000 7f2ae28f0640 1 --2- 172.21.15.78:0/902450234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2adc107d20 0x7f2adc10a110 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:12.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.285+0000 7f2ae28f0640 1 -- 172.21.15.78:0/902450234 >> 172.21.15.78:0/902450234 conn(0x7f2adc1019e0 msgr2=0x7f2adc103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:12.081 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.285+0000 7f2ae28f0640 1 -- 172.21.15.78:0/902450234 shutdown_connections 2024-08-18T23:35:12.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.285+0000 7f2ae28f0640 1 -- 172.21.15.78:0/902450234 wait complete. 2024-08-18T23:35:12.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.286+0000 7f2ae28f0640 1 Processor -- start 2024-08-18T23:35:12.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.286+0000 7f2ae28f0640 1 -- start start 2024-08-18T23:35:12.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.286+0000 7f2ae28f0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2adc107d20 0x7f2adc19b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:12.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.286+0000 7f2ae28f0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc10b620 con 0x7f2adc107d20 2024-08-18T23:35:12.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.286+0000 7f2adbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2adc107d20 0x7f2adc19b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:12.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.286+0000 7f2adbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2adc107d20 0x7f2adc19b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45784/0 (socket says 172.21.15.78:45784) 2024-08-18T23:35:12.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.286+0000 7f2adbfff640 1 -- 172.21.15.78:0/2728377783 learned_addr learned my addr 172.21.15.78:0/2728377783 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:12.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.286+0000 7f2adbfff640 1 -- 172.21.15.78:0/2728377783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2adc19bac0 con 0x7f2adc107d20 2024-08-18T23:35:12.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.286+0000 7f2adbfff640 1 --2- 172.21.15.78:0/2728377783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2adc107d20 0x7f2adc19b580 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2acc031000 tx=0x7f2acc0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:12.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.287+0000 7f2ad9ffb640 1 -- 172.21.15.78:0/2728377783 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2acc042070 con 0x7f2adc107d20 2024-08-18T23:35:12.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.287+0000 7f2ad9ffb640 1 -- 172.21.15.78:0/2728377783 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2acc032be0 con 0x7f2adc107d20 2024-08-18T23:35:12.082 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.287+0000 7f2ae28f0640 1 -- 172.21.15.78:0/2728377783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2adc19bdb0 con 0x7f2adc107d20 2024-08-18T23:35:12.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.287+0000 7f2ad9ffb640 1 -- 172.21.15.78:0/2728377783 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2acc037040 con 0x7f2adc107d20 2024-08-18T23:35:12.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.287+0000 7f2ae28f0640 1 -- 172.21.15.78:0/2728377783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2adc19ea40 con 0x7f2adc107d20 2024-08-18T23:35:12.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.288+0000 7f2ad9ffb640 1 -- 172.21.15.78:0/2728377783 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2acc045070 con 0x7f2adc107d20 2024-08-18T23:35:12.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.288+0000 7f2ae28f0640 1 -- 172.21.15.78:0/2728377783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2adc108180 con 0x7f2adc107d20 2024-08-18T23:35:12.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.288+0000 7f2ad9ffb640 1 --2- 172.21.15.78:0/2728377783 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f2ac8038f70 0x7f2ac803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:12.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.288+0000 7f2ad9ffb640 1 -- 172.21.15.78:0/2728377783 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2acc06bf30 con 0x7f2adc107d20 2024-08-18T23:35:12.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.288+0000 7f2ad3fff640 1 --2- 172.21.15.78:0/2728377783 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f2ac8038f70 0x7f2ac803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:12.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.288+0000 7f2ad3fff640 1 --2- 172.21.15.78:0/2728377783 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f2ac8038f70 0x7f2ac803b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40098c0 tx=0x7f2ac4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:12.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.292+0000 7f2ad9ffb640 1 -- 172.21.15.78:0/2728377783 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2acc042210 con 0x7f2adc107d20 2024-08-18T23:35:12.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.478+0000 7f2ae28f0640 1 -- 172.21.15.78:0/2728377783 --> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2adc02d720 con 0x7f2ac8038f70 2024-08-18T23:35:12.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.484+0000 7f2ad9ffb640 1 -- 172.21.15.78:0/2728377783 <== mgr.14118 v2:172.21.15.78:6800/311758909 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2adc02d720 con 0x7f2ac8038f70 2024-08-18T23:35:12.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.486+0000 7f2ae28f0640 1 -- 172.21.15.78:0/2728377783 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f2ac8038f70 msgr2=0x7f2ac803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:12.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.486+0000 7f2ae28f0640 1 --2- 172.21.15.78:0/2728377783 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f2ac8038f70 0x7f2ac803b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40098c0 tx=0x7f2ac4006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:12.083 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.486+0000 7f2ae28f0640 1 -- 172.21.15.78:0/2728377783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2adc107d20 msgr2=0x7f2adc19b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:12.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.486+0000 7f2ae28f0640 1 --2- 172.21.15.78:0/2728377783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2adc107d20 0x7f2adc19b580 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2acc031000 tx=0x7f2acc0326e0 comp rx=0 tx=0).stop 2024-08-18T23:35:12.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.486+0000 7f2ae28f0640 1 -- 172.21.15.78:0/2728377783 shutdown_connections 2024-08-18T23:35:12.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.486+0000 7f2ae28f0640 1 --2- 172.21.15.78:0/2728377783 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f2ac8038f70 0x7f2ac803b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:12.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.486+0000 7f2ae28f0640 1 --2- 172.21.15.78:0/2728377783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2adc107d20 0x7f2adc19b580 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:12.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.486+0000 7f2ae28f0640 1 -- 172.21.15.78:0/2728377783 >> 172.21.15.78:0/2728377783 conn(0x7f2adc1019e0 msgr2=0x7f2adc1027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:12.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.487+0000 7f2ae28f0640 1 -- 172.21.15.78:0/2728377783 shutdown_connections 2024-08-18T23:35:12.084 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:11.487+0000 7f2ae28f0640 1 -- 172.21.15.78:0/2728377783 wait complete. 2024-08-18T23:35:12.936 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:12 smithi078 ceph-mon[18440]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:35:12.936 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:12 smithi078 ceph-mon[18440]: Saving service mgr spec with placement count:2 2024-08-18T23:35:12.936 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:12 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' 2024-08-18T23:35:13.915 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.126+0000 7f314b717640 1 Processor -- start 2024-08-18T23:35:13.915 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.126+0000 7f314b717640 1 -- start start 2024-08-18T23:35:13.915 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.127+0000 7f314b717640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144107d70 0x7f314410a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:13.916 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.127+0000 7f314b717640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3144075530 con 0x7f3144107d70 2024-08-18T23:35:13.916 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.127+0000 7f31494b3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144107d70 0x7f314410a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:13.916 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.127+0000 7f31494b3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144107d70 0x7f314410a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45786/0 (socket says 172.21.15.78:45786) 2024-08-18T23:35:13.916 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.127+0000 7f31494b3640 1 -- 172.21.15.78:0/2026690348 learned_addr learned my addr 172.21.15.78:0/2026690348 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:13.916 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.127+0000 7f31494b3640 1 -- 172.21.15.78:0/2026690348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314410a6a0 con 0x7f3144107d70 2024-08-18T23:35:13.916 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.128+0000 7f31494b3640 1 --2- 172.21.15.78:0/2026690348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144107d70 0x7f314410a160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3134009820 tx=0x7f313402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ced6150d5d45a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:13.916 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.128+0000 7f313bfff640 1 -- 172.21.15.78:0/2026690348 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3134037070 con 0x7f3144107d70 2024-08-18T23:35:13.916 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.128+0000 7f313bfff640 1 -- 172.21.15.78:0/2026690348 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3134032920 con 0x7f3144107d70 2024-08-18T23:35:13.916 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.128+0000 7f313bfff640 1 -- 172.21.15.78:0/2026690348 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3134030810 con 0x7f3144107d70 2024-08-18T23:35:13.916 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.128+0000 7f314b717640 1 -- 172.21.15.78:0/2026690348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144107d70 msgr2=0x7f314410a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:13.916 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.128+0000 7f314b717640 1 --2- 172.21.15.78:0/2026690348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144107d70 0x7f314410a160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3134009820 tx=0x7f313402a270 comp rx=0 tx=0).stop 2024-08-18T23:35:13.916 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.129+0000 7f314b717640 1 -- 172.21.15.78:0/2026690348 shutdown_connections 2024-08-18T23:35:13.916 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.129+0000 7f314b717640 1 --2- 172.21.15.78:0/2026690348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144107d70 0x7f314410a160 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:13.917 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.129+0000 7f314b717640 1 -- 172.21.15.78:0/2026690348 >> 172.21.15.78:0/2026690348 conn(0x7f31441018f0 msgr2=0x7f3144103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:13.917 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.129+0000 7f314b717640 1 -- 172.21.15.78:0/2026690348 shutdown_connections 2024-08-18T23:35:13.917 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.129+0000 7f314b717640 1 -- 172.21.15.78:0/2026690348 wait complete. 2024-08-18T23:35:13.917 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.129+0000 7f314b717640 1 Processor -- start 2024-08-18T23:35:13.917 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.129+0000 7f314b717640 1 -- start start 2024-08-18T23:35:13.917 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.129+0000 7f314b717640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144107d70 0x7f31441075f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:13.917 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.129+0000 7f314b717640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f314410b6c0 con 0x7f3144107d70 2024-08-18T23:35:13.917 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.130+0000 7f31494b3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144107d70 0x7f31441075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:13.917 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.130+0000 7f31494b3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144107d70 0x7f31441075f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45790/0 (socket says 172.21.15.78:45790) 2024-08-18T23:35:13.917 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.130+0000 7f31494b3640 1 -- 172.21.15.78:0/168460380 learned_addr learned my addr 172.21.15.78:0/168460380 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:13.917 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.130+0000 7f31494b3640 1 -- 172.21.15.78:0/168460380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3144107b30 con 0x7f3144107d70 2024-08-18T23:35:13.917 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.130+0000 7f31494b3640 1 --2- 172.21.15.78:0/168460380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144107d70 0x7f31441075f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3134030420 tx=0x7f3134032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:13.918 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.130+0000 7f313a7fc640 1 -- 172.21.15.78:0/168460380 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3134037040 con 0x7f3144107d70 2024-08-18T23:35:13.918 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.130+0000 7f313a7fc640 1 -- 172.21.15.78:0/168460380 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3134040070 con 0x7f3144107d70 2024-08-18T23:35:13.918 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.130+0000 7f314b717640 1 -- 172.21.15.78:0/168460380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3144105d20 con 0x7f3144107d70 2024-08-18T23:35:13.918 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.130+0000 7f313a7fc640 1 -- 172.21.15.78:0/168460380 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f313403a6e0 con 0x7f3144107d70 2024-08-18T23:35:13.918 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.130+0000 7f314b717640 1 -- 172.21.15.78:0/168460380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3144106260 con 0x7f3144107d70 2024-08-18T23:35:13.918 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.131+0000 7f314b717640 1 -- 172.21.15.78:0/168460380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31441081d0 con 0x7f3144107d70 2024-08-18T23:35:13.918 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.131+0000 7f313a7fc640 1 -- 172.21.15.78:0/168460380 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f313403a9e0 con 0x7f3144107d70 2024-08-18T23:35:13.918 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.132+0000 7f313a7fc640 1 --2- 172.21.15.78:0/168460380 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f3130038f70 0x7f313003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:13.918 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.132+0000 7f313a7fc640 1 -- 172.21.15.78:0/168460380 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f313406c700 con 0x7f3144107d70 2024-08-18T23:35:13.918 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.132+0000 7f3148cb2640 1 --2- 172.21.15.78:0/168460380 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f3130038f70 0x7f313003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:13.918 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.132+0000 7f3148cb2640 1 --2- 172.21.15.78:0/168460380 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f3130038f70 0x7f313003b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f31400098c0 tx=0x7f3140006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:13.918 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.136+0000 7f313a7fc640 1 -- 172.21.15.78:0/168460380 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f313403add0 con 0x7f3144107d70 2024-08-18T23:35:13.919 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.318+0000 7f314b717640 1 -- 172.21.15.78:0/168460380 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f3144194cd0 con 0x7f3144107d70 2024-08-18T23:35:13.919 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.321+0000 7f313a7fc640 1 -- 172.21.15.78:0/168460380 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f3134043940 con 0x7f3144107d70 2024-08-18T23:35:13.919 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.324+0000 7f314b717640 1 -- 172.21.15.78:0/168460380 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f3130038f70 msgr2=0x7f313003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:13.919 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.324+0000 7f314b717640 1 --2- 172.21.15.78:0/168460380 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f3130038f70 0x7f313003b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f31400098c0 tx=0x7f3140006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:13.919 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.324+0000 7f314b717640 1 -- 172.21.15.78:0/168460380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144107d70 msgr2=0x7f31441075f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:13.919 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.324+0000 7f314b717640 1 --2- 172.21.15.78:0/168460380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144107d70 0x7f31441075f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3134030420 tx=0x7f3134032670 comp rx=0 tx=0).stop 2024-08-18T23:35:13.919 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.324+0000 7f314b717640 1 -- 172.21.15.78:0/168460380 shutdown_connections 2024-08-18T23:35:13.919 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.324+0000 7f314b717640 1 --2- 172.21.15.78:0/168460380 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f3130038f70 0x7f313003b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:13.919 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.324+0000 7f314b717640 1 --2- 172.21.15.78:0/168460380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3144107d70 0x7f31441075f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:13.919 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.324+0000 7f314b717640 1 -- 172.21.15.78:0/168460380 >> 172.21.15.78:0/168460380 conn(0x7f31441018f0 msgr2=0x7f31441026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:13.919 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.324+0000 7f314b717640 1 -- 172.21.15.78:0/168460380 shutdown_connections 2024-08-18T23:35:13.919 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:13.324+0000 7f314b717640 1 -- 172.21.15.78:0/168460380 wait complete. 2024-08-18T23:35:14.164 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:13 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' 2024-08-18T23:35:14.164 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:13 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' 2024-08-18T23:35:14.164 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:13 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/168460380' entity='client.admin' 2024-08-18T23:35:15.675 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:15 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1699427995' entity='client.admin' 2024-08-18T23:35:15.675 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:15 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' 2024-08-18T23:35:15.682 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.347+0000 7f743dda0640 1 Processor -- start 2024-08-18T23:35:15.682 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.347+0000 7f743dda0640 1 -- start start 2024-08-18T23:35:15.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.347+0000 7f743dda0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7438107300 0x7f7438107700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:15.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.347+0000 7f743dda0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7438107c40 con 0x7f7438107300 2024-08-18T23:35:15.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.347+0000 7f74377fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7438107300 0x7f7438107700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:15.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.347+0000 7f74377fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7438107300 0x7f7438107700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45802/0 (socket says 172.21.15.78:45802) 2024-08-18T23:35:15.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.347+0000 7f74377fe640 1 -- 172.21.15.78:0/3441542241 learned_addr learned my addr 172.21.15.78:0/3441542241 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:15.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.348+0000 7f74377fe640 1 -- 172.21.15.78:0/3441542241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7438107dc0 con 0x7f7438107300 2024-08-18T23:35:15.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.348+0000 7f74377fe640 1 --2- 172.21.15.78:0/3441542241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7438107300 0x7f7438107700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7428009820 tx=0x7f742802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1513dce941ff063 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:15.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.349+0000 7f74367fc640 1 -- 172.21.15.78:0/3441542241 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7428037070 con 0x7f7438107300 2024-08-18T23:35:15.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.349+0000 7f74367fc640 1 -- 172.21.15.78:0/3441542241 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7428032920 con 0x7f7438107300 2024-08-18T23:35:15.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.349+0000 7f74367fc640 1 -- 172.21.15.78:0/3441542241 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7428030810 con 0x7f7438107300 2024-08-18T23:35:15.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.349+0000 7f743dda0640 1 -- 172.21.15.78:0/3441542241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7438107300 msgr2=0x7f7438107700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:15.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.349+0000 7f743dda0640 1 --2- 172.21.15.78:0/3441542241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7438107300 0x7f7438107700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7428009820 tx=0x7f742802a270 comp rx=0 tx=0).stop 2024-08-18T23:35:15.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.349+0000 7f743dda0640 1 -- 172.21.15.78:0/3441542241 shutdown_connections 2024-08-18T23:35:15.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.349+0000 7f743dda0640 1 --2- 172.21.15.78:0/3441542241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7438107300 0x7f7438107700 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:15.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.349+0000 7f743dda0640 1 -- 172.21.15.78:0/3441542241 >> 172.21.15.78:0/3441542241 conn(0x7f7438102b30 msgr2=0x7f7438104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:15.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.349+0000 7f743dda0640 1 -- 172.21.15.78:0/3441542241 shutdown_connections 2024-08-18T23:35:15.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.349+0000 7f743dda0640 1 -- 172.21.15.78:0/3441542241 wait complete. 2024-08-18T23:35:15.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.350+0000 7f743dda0640 1 Processor -- start 2024-08-18T23:35:15.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.350+0000 7f743dda0640 1 -- start start 2024-08-18T23:35:15.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.350+0000 7f743dda0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7438107300 0x7f743819b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:15.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.350+0000 7f743dda0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f743810b770 con 0x7f7438107300 2024-08-18T23:35:15.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.350+0000 7f74377fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7438107300 0x7f743819b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:15.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.350+0000 7f74377fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7438107300 0x7f743819b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45814/0 (socket says 172.21.15.78:45814) 2024-08-18T23:35:15.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.350+0000 7f74377fe640 1 -- 172.21.15.78:0/1699427995 learned_addr learned my addr 172.21.15.78:0/1699427995 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:15.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.350+0000 7f74377fe640 1 -- 172.21.15.78:0/1699427995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743819bb40 con 0x7f7438107300 2024-08-18T23:35:15.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.351+0000 7f74377fe640 1 --2- 172.21.15.78:0/1699427995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7438107300 0x7f743819b600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7428030420 tx=0x7f7428032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:15.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.351+0000 7f7434ff9640 1 -- 172.21.15.78:0/1699427995 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7428037040 con 0x7f7438107300 2024-08-18T23:35:15.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.351+0000 7f7434ff9640 1 -- 172.21.15.78:0/1699427995 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7428040070 con 0x7f7438107300 2024-08-18T23:35:15.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.351+0000 7f743dda0640 1 -- 172.21.15.78:0/1699427995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743819bdd0 con 0x7f7438107300 2024-08-18T23:35:15.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.351+0000 7f7434ff9640 1 -- 172.21.15.78:0/1699427995 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f742803a6e0 con 0x7f7438107300 2024-08-18T23:35:15.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.351+0000 7f743dda0640 1 -- 172.21.15.78:0/1699427995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743819eac0 con 0x7f7438107300 2024-08-18T23:35:15.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.352+0000 7f7434ff9640 1 -- 172.21.15.78:0/1699427995 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f742803b420 con 0x7f7438107300 2024-08-18T23:35:15.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.352+0000 7f743dda0640 1 -- 172.21.15.78:0/1699427995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7438107780 con 0x7f7438107300 2024-08-18T23:35:15.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.352+0000 7f7434ff9640 1 --2- 172.21.15.78:0/1699427995 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f7420038f70 0x7f742003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:15.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.352+0000 7f7434ff9640 1 -- 172.21.15.78:0/1699427995 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f742806c5a0 con 0x7f7438107300 2024-08-18T23:35:15.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.352+0000 7f7436ffd640 1 --2- 172.21.15.78:0/1699427995 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f7420038f70 0x7f742003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:15.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.352+0000 7f7436ffd640 1 --2- 172.21.15.78:0/1699427995 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f7420038f70 0x7f742003b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f742c0098c0 tx=0x7f742c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:15.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.357+0000 7f7434ff9640 1 -- 172.21.15.78:0/1699427995 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f742803b710 con 0x7f7438107300 2024-08-18T23:35:15.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.538+0000 7f743dda0640 1 -- 172.21.15.78:0/1699427995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f743802d5e0 con 0x7f7438107300 2024-08-18T23:35:15.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.540+0000 7f7434ff9640 1 -- 172.21.15.78:0/1699427995 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f7428031e60 con 0x7f7438107300 2024-08-18T23:35:15.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.542+0000 7f743dda0640 1 -- 172.21.15.78:0/1699427995 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f7420038f70 msgr2=0x7f742003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:15.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.542+0000 7f743dda0640 1 --2- 172.21.15.78:0/1699427995 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f7420038f70 0x7f742003b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f742c0098c0 tx=0x7f742c006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:15.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.542+0000 7f743dda0640 1 -- 172.21.15.78:0/1699427995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7438107300 msgr2=0x7f743819b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:15.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.542+0000 7f743dda0640 1 --2- 172.21.15.78:0/1699427995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7438107300 0x7f743819b600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7428030420 tx=0x7f7428032670 comp rx=0 tx=0).stop 2024-08-18T23:35:15.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.543+0000 7f743dda0640 1 -- 172.21.15.78:0/1699427995 shutdown_connections 2024-08-18T23:35:15.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.543+0000 7f743dda0640 1 --2- 172.21.15.78:0/1699427995 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f7420038f70 0x7f742003b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:15.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.543+0000 7f743dda0640 1 --2- 172.21.15.78:0/1699427995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7438107300 0x7f743819b600 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:15.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.543+0000 7f743dda0640 1 -- 172.21.15.78:0/1699427995 >> 172.21.15.78:0/1699427995 conn(0x7f7438102b30 msgr2=0x7f74381038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:15.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.543+0000 7f743dda0640 1 -- 172.21.15.78:0/1699427995 shutdown_connections 2024-08-18T23:35:15.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:14.543+0000 7f743dda0640 1 -- 172.21.15.78:0/1699427995 wait complete. 2024-08-18T23:35:15.686 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling the dashboard module... 2024-08-18T23:35:17.464 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:17 smithi078 ceph-mon[18440]: from='mgr.14118 172.21.15.78:0/2111719543' entity='mgr.x' 2024-08-18T23:35:17.464 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:17 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3688914803' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-08-18T23:35:17.464 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:17 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: ignoring --setuser ceph since I am not root 2024-08-18T23:35:17.464 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:17 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: ignoring --setgroup ceph since I am not root 2024-08-18T23:35:17.464 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:17 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:17.396+0000 7f3b0261f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-18T23:35:17.464 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:17 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:17.462+0000 7f3b0261f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-18T23:35:17.719 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:17 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:17.609+0000 7f3b0261f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-18T23:35:17.872 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.086+0000 7fb4c4c0a640 1 Processor -- start 2024-08-18T23:35:17.872 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.086+0000 7fb4c4c0a640 1 -- start start 2024-08-18T23:35:17.872 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.087+0000 7fb4c4c0a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c0107d30 0x7fb4c010a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:17.872 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.087+0000 7fb4c4c0a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c0075570 con 0x7fb4c0107d30 2024-08-18T23:35:17.872 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.087+0000 7fb4be59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c0107d30 0x7fb4c010a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:17.872 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.087+0000 7fb4be59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c0107d30 0x7fb4c010a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45206/0 (socket says 172.21.15.78:45206) 2024-08-18T23:35:17.872 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.087+0000 7fb4be59c640 1 -- 172.21.15.78:0/3139569626 learned_addr learned my addr 172.21.15.78:0/3139569626 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:17.873 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.087+0000 7fb4be59c640 1 -- 172.21.15.78:0/3139569626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c010a660 con 0x7fb4c0107d30 2024-08-18T23:35:17.873 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.088+0000 7fb4be59c640 1 --2- 172.21.15.78:0/3139569626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c0107d30 0x7fb4c010a120 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009820 tx=0x7fb4b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec62dcef6ba63057 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:17.873 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.088+0000 7fb4bdd9b640 1 -- 172.21.15.78:0/3139569626 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4b0037070 con 0x7fb4c0107d30 2024-08-18T23:35:17.873 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.088+0000 7fb4bdd9b640 1 -- 172.21.15.78:0/3139569626 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4b0032920 con 0x7fb4c0107d30 2024-08-18T23:35:17.873 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.088+0000 7fb4bdd9b640 1 -- 172.21.15.78:0/3139569626 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4b0030810 con 0x7fb4c0107d30 2024-08-18T23:35:17.873 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.089+0000 7fb4c4c0a640 1 -- 172.21.15.78:0/3139569626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c0107d30 msgr2=0x7fb4c010a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:17.873 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.089+0000 7fb4c4c0a640 1 --2- 172.21.15.78:0/3139569626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c0107d30 0x7fb4c010a120 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009820 tx=0x7fb4b002a270 comp rx=0 tx=0).stop 2024-08-18T23:35:17.873 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.089+0000 7fb4c4c0a640 1 -- 172.21.15.78:0/3139569626 shutdown_connections 2024-08-18T23:35:17.873 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.089+0000 7fb4c4c0a640 1 --2- 172.21.15.78:0/3139569626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c0107d30 0x7fb4c010a120 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:17.873 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.089+0000 7fb4c4c0a640 1 -- 172.21.15.78:0/3139569626 >> 172.21.15.78:0/3139569626 conn(0x7fb4c01018f0 msgr2=0x7fb4c0103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:17.873 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.089+0000 7fb4c4c0a640 1 -- 172.21.15.78:0/3139569626 shutdown_connections 2024-08-18T23:35:17.873 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.089+0000 7fb4c4c0a640 1 -- 172.21.15.78:0/3139569626 wait complete. 2024-08-18T23:35:17.873 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.090+0000 7fb4c4c0a640 1 Processor -- start 2024-08-18T23:35:17.874 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.090+0000 7fb4c4c0a640 1 -- start start 2024-08-18T23:35:17.874 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.090+0000 7fb4c4c0a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c0107d30 0x7fb4c019b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:17.874 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.090+0000 7fb4c4c0a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c010b6e0 con 0x7fb4c0107d30 2024-08-18T23:35:17.874 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.090+0000 7fb4be59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c0107d30 0x7fb4c019b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:17.874 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.090+0000 7fb4be59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c0107d30 0x7fb4c019b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45220/0 (socket says 172.21.15.78:45220) 2024-08-18T23:35:17.874 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.090+0000 7fb4be59c640 1 -- 172.21.15.78:0/3688914803 learned_addr learned my addr 172.21.15.78:0/3688914803 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:17.874 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.090+0000 7fb4be59c640 1 -- 172.21.15.78:0/3688914803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c019bb20 con 0x7fb4c0107d30 2024-08-18T23:35:17.874 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.091+0000 7fb4be59c640 1 --2- 172.21.15.78:0/3688914803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c0107d30 0x7fb4c019b5e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0030420 tx=0x7fb4b0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:17.874 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.091+0000 7fb4b77fe640 1 -- 172.21.15.78:0/3688914803 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4b0037040 con 0x7fb4c0107d30 2024-08-18T23:35:17.874 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.091+0000 7fb4b77fe640 1 -- 172.21.15.78:0/3688914803 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4b0040070 con 0x7fb4c0107d30 2024-08-18T23:35:17.874 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.091+0000 7fb4c4c0a640 1 -- 172.21.15.78:0/3688914803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c019bdb0 con 0x7fb4c0107d30 2024-08-18T23:35:17.874 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.091+0000 7fb4b77fe640 1 -- 172.21.15.78:0/3688914803 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4b003a6e0 con 0x7fb4c0107d30 2024-08-18T23:35:17.875 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.091+0000 7fb4c4c0a640 1 -- 172.21.15.78:0/3688914803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c019eaa0 con 0x7fb4c0107d30 2024-08-18T23:35:17.875 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.092+0000 7fb4b77fe640 1 -- 172.21.15.78:0/3688914803 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb4b003a990 con 0x7fb4c0107d30 2024-08-18T23:35:17.875 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.092+0000 7fb4b77fe640 1 --2- 172.21.15.78:0/3688914803 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7fb4ac038fc0 0x7fb4ac03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:17.875 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.092+0000 7fb4b77fe640 1 -- 172.21.15.78:0/3688914803 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb4b006d560 con 0x7fb4c0107d30 2024-08-18T23:35:17.875 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.092+0000 7fb4c4c0a640 1 -- 172.21.15.78:0/3688914803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4c0108190 con 0x7fb4c0107d30 2024-08-18T23:35:17.875 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.092+0000 7fb4b7fff640 1 --2- 172.21.15.78:0/3688914803 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7fb4ac038fc0 0x7fb4ac03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:17.875 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.092+0000 7fb4b7fff640 1 --2- 172.21.15.78:0/3688914803 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7fb4ac038fc0 0x7fb4ac03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80098c0 tx=0x7fb4a8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:17.875 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.100+0000 7fb4b77fe640 1 -- 172.21.15.78:0/3688914803 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb4b0042e10 con 0x7fb4c0107d30 2024-08-18T23:35:17.875 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:16.325+0000 7fb4c4c0a640 1 -- 172.21.15.78:0/3688914803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fb4c00683e0 con 0x7fb4c0107d30 2024-08-18T23:35:17.876 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:17.155+0000 7fb4b77fe640 1 -- 172.21.15.78:0/3688914803 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fb4b0043b80 con 0x7fb4c0107d30 2024-08-18T23:35:17.876 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:17.157+0000 7fb4b77fe640 1 -- 172.21.15.78:0/3688914803 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fb4b0045980 con 0x7fb4c0107d30 2024-08-18T23:35:17.876 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:17.158+0000 7fb4c4c0a640 1 -- 172.21.15.78:0/3688914803 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7fb4ac038fc0 msgr2=0x7fb4ac03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:17.876 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:17.158+0000 7fb4c4c0a640 1 --2- 172.21.15.78:0/3688914803 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7fb4ac038fc0 0x7fb4ac03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80098c0 tx=0x7fb4a8006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:17.876 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:17.158+0000 7fb4c4c0a640 1 -- 172.21.15.78:0/3688914803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c0107d30 msgr2=0x7fb4c019b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:17.876 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:17.158+0000 7fb4c4c0a640 1 --2- 172.21.15.78:0/3688914803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c0107d30 0x7fb4c019b5e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0030420 tx=0x7fb4b0032670 comp rx=0 tx=0).stop 2024-08-18T23:35:17.876 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:17.158+0000 7fb4c4c0a640 1 -- 172.21.15.78:0/3688914803 shutdown_connections 2024-08-18T23:35:17.876 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:17.158+0000 7fb4c4c0a640 1 --2- 172.21.15.78:0/3688914803 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7fb4ac038fc0 0x7fb4ac03b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:17.876 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:17.158+0000 7fb4c4c0a640 1 --2- 172.21.15.78:0/3688914803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c0107d30 0x7fb4c019b5e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:17.876 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:17.158+0000 7fb4c4c0a640 1 -- 172.21.15.78:0/3688914803 >> 172.21.15.78:0/3688914803 conn(0x7fb4c01018f0 msgr2=0x7fb4c0102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:17.876 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:17.159+0000 7fb4c4c0a640 1 -- 172.21.15.78:0/3688914803 shutdown_connections 2024-08-18T23:35:17.876 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:17.159+0000 7fb4c4c0a640 1 -- 172.21.15.78:0/3688914803 wait complete. 2024-08-18T23:35:18.052 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:17 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:17.767+0000 7f3b0261f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-18T23:35:18.052 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:17 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:17.925+0000 7f3b0261f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-18T23:35:18.053 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:18 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:18.051+0000 7f3b0261f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-18T23:35:18.312 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:18 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3688914803' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-08-18T23:35:18.312 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:18 smithi078 ceph-mon[18440]: mgrmap e9: x(active, since 18s) 2024-08-18T23:35:18.313 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:18 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:18.180+0000 7f3b0261f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-18T23:35:18.313 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:18 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:18.237+0000 7f3b0261f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-18T23:35:18.612 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:18 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:18.546+0000 7f3b0261f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-18T23:35:18.886 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:18 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:18.611+0000 7f3b0261f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-18T23:35:18.886 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:18 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:18.679+0000 7f3b0261f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-18T23:35:18.887 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:18 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:18.826+0000 7f3b0261f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-18T23:35:18.887 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:18 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:18.885+0000 7f3b0261f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-18T23:35:19.139 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:18 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:18.952+0000 7f3b0261f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-18T23:35:19.174 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-08-18T23:35:19.180 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-08-18T23:35:19.180 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-18T23:35:19.180 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-18T23:35:19.180 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-18T23:35:19.180 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-08-18T23:35:19.181 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.311+0000 7f3917f17640 1 Processor -- start 2024-08-18T23:35:19.181 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.311+0000 7f3917f17640 1 -- start start 2024-08-18T23:35:19.181 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.311+0000 7f3917f17640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f391007dd20 0x7f391007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:19.181 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.311+0000 7f3917f17640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f391007c820 con 0x7f391007dd20 2024-08-18T23:35:19.181 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.312+0000 7f3915cb3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f391007dd20 0x7f391007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:19.181 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.312+0000 7f3915cb3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f391007dd20 0x7f391007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45252/0 (socket says 172.21.15.78:45252) 2024-08-18T23:35:19.181 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.312+0000 7f3915cb3640 1 -- 172.21.15.78:0/2203619381 learned_addr learned my addr 172.21.15.78:0/2203619381 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:19.181 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.312+0000 7f3915cb3640 1 -- 172.21.15.78:0/2203619381 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391007c9a0 con 0x7f391007dd20 2024-08-18T23:35:19.181 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.313+0000 7f3915cb3640 1 --2- 172.21.15.78:0/2203619381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f391007dd20 0x7f391007c2e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f390c009820 tx=0x7f390c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60196b94c98f3f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:19.181 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.313+0000 7f3914cb1640 1 -- 172.21.15.78:0/2203619381 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f390c037070 con 0x7f391007dd20 2024-08-18T23:35:19.181 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.313+0000 7f3914cb1640 1 -- 172.21.15.78:0/2203619381 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f390c032920 con 0x7f391007dd20 2024-08-18T23:35:19.181 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.313+0000 7f3914cb1640 1 -- 172.21.15.78:0/2203619381 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f390c030810 con 0x7f391007dd20 2024-08-18T23:35:19.181 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.314+0000 7f3917f17640 1 -- 172.21.15.78:0/2203619381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f391007dd20 msgr2=0x7f391007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:19.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.314+0000 7f3917f17640 1 --2- 172.21.15.78:0/2203619381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f391007dd20 0x7f391007c2e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f390c009820 tx=0x7f390c02a270 comp rx=0 tx=0).stop 2024-08-18T23:35:19.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.314+0000 7f3917f17640 1 -- 172.21.15.78:0/2203619381 shutdown_connections 2024-08-18T23:35:19.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.314+0000 7f3917f17640 1 --2- 172.21.15.78:0/2203619381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f391007dd20 0x7f391007c2e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:19.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.314+0000 7f3917f17640 1 -- 172.21.15.78:0/2203619381 >> 172.21.15.78:0/2203619381 conn(0x7f3910102b40 msgr2=0x7f3910104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:19.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.314+0000 7f3917f17640 1 -- 172.21.15.78:0/2203619381 shutdown_connections 2024-08-18T23:35:19.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.314+0000 7f3917f17640 1 -- 172.21.15.78:0/2203619381 wait complete. 2024-08-18T23:35:19.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.315+0000 7f3917f17640 1 Processor -- start 2024-08-18T23:35:19.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.315+0000 7f3917f17640 1 -- start start 2024-08-18T23:35:19.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.315+0000 7f3917f17640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f391007dd20 0x7f391019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:19.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.315+0000 7f3917f17640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f391010d800 con 0x7f391007dd20 2024-08-18T23:35:19.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.315+0000 7f3915cb3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f391007dd20 0x7f391019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:19.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.315+0000 7f3915cb3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f391007dd20 0x7f391019d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45256/0 (socket says 172.21.15.78:45256) 2024-08-18T23:35:19.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.315+0000 7f3915cb3640 1 -- 172.21.15.78:0/2282441773 learned_addr learned my addr 172.21.15.78:0/2282441773 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:19.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.315+0000 7f3915cb3640 1 -- 172.21.15.78:0/2282441773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391019d990 con 0x7f391007dd20 2024-08-18T23:35:19.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.316+0000 7f3915cb3640 1 --2- 172.21.15.78:0/2282441773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f391007dd20 0x7f391019d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f390c030380 tx=0x7f390c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:19.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.316+0000 7f3902ffd640 1 -- 172.21.15.78:0/2282441773 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f390c037040 con 0x7f391007dd20 2024-08-18T23:35:19.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.316+0000 7f3902ffd640 1 -- 172.21.15.78:0/2282441773 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f390c040070 con 0x7f391007dd20 2024-08-18T23:35:19.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.316+0000 7f3902ffd640 1 -- 172.21.15.78:0/2282441773 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f390c03a7b0 con 0x7f391007dd20 2024-08-18T23:35:19.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.316+0000 7f3917f17640 1 -- 172.21.15.78:0/2282441773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391019dc20 con 0x7f391007dd20 2024-08-18T23:35:19.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.316+0000 7f3917f17640 1 -- 172.21.15.78:0/2282441773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39101a0910 con 0x7f391007dd20 2024-08-18T23:35:19.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.317+0000 7f3902ffd640 1 -- 172.21.15.78:0/2282441773 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f390c03a9e0 con 0x7f391007dd20 2024-08-18T23:35:19.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.317+0000 7f3917f17640 1 -- 172.21.15.78:0/2282441773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f391007c360 con 0x7f391007dd20 2024-08-18T23:35:19.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.317+0000 7f3902ffd640 1 --2- 172.21.15.78:0/2282441773 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f38fc039020 0x7f38fc03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:19.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.317+0000 7f3902ffd640 1 -- 172.21.15.78:0/2282441773 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f390c06c5f0 con 0x7f391007dd20 2024-08-18T23:35:19.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.317+0000 7f39154b2640 1 -- 172.21.15.78:0/2282441773 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f38fc039020 msgr2=0x7f38fc03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/311758909 2024-08-18T23:35:19.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.317+0000 7f39154b2640 1 --2- 172.21.15.78:0/2282441773 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f38fc039020 0x7f38fc03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-18T23:35:19.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.322+0000 7f3902ffd640 1 -- 172.21.15.78:0/2282441773 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f390c031560 con 0x7f391007dd20 2024-08-18T23:35:19.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.518+0000 7f39154b2640 1 -- 172.21.15.78:0/2282441773 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f38fc039020 msgr2=0x7f38fc03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/311758909 2024-08-18T23:35:19.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.518+0000 7f39154b2640 1 --2- 172.21.15.78:0/2282441773 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f38fc039020 0x7f38fc03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-18T23:35:19.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.538+0000 7f3917f17640 1 -- 172.21.15.78:0/2282441773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f39101a0ae0 con 0x7f391007dd20 2024-08-18T23:35:19.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.539+0000 7f3902ffd640 1 -- 172.21.15.78:0/2282441773 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f39101a0ae0 con 0x7f391007dd20 2024-08-18T23:35:19.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.541+0000 7f3917f17640 1 -- 172.21.15.78:0/2282441773 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f38fc039020 msgr2=0x7f38fc03b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-18T23:35:19.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.541+0000 7f3917f17640 1 --2- 172.21.15.78:0/2282441773 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f38fc039020 0x7f38fc03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:19.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.541+0000 7f3917f17640 1 -- 172.21.15.78:0/2282441773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f391007dd20 msgr2=0x7f391019d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:19.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.541+0000 7f3917f17640 1 --2- 172.21.15.78:0/2282441773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f391007dd20 0x7f391019d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f390c030380 tx=0x7f390c032670 comp rx=0 tx=0).stop 2024-08-18T23:35:19.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.541+0000 7f3917f17640 1 -- 172.21.15.78:0/2282441773 shutdown_connections 2024-08-18T23:35:19.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.541+0000 7f3917f17640 1 --2- 172.21.15.78:0/2282441773 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f38fc039020 0x7f38fc03b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:19.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.541+0000 7f3917f17640 1 --2- 172.21.15.78:0/2282441773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f391007dd20 0x7f391019d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:19.185 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.541+0000 7f3917f17640 1 -- 172.21.15.78:0/2282441773 >> 172.21.15.78:0/2282441773 conn(0x7f3910102b40 msgr2=0x7f39101036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:19.185 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.541+0000 7f3917f17640 1 -- 172.21.15.78:0/2282441773 shutdown_connections 2024-08-18T23:35:19.185 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:18.541+0000 7f3917f17640 1 -- 172.21.15.78:0/2282441773 wait complete. 2024-08-18T23:35:19.185 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for the mgr to restart... 2024-08-18T23:35:19.185 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr epoch 9... 2024-08-18T23:35:19.442 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:19 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2282441773' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-08-18T23:35:19.442 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:19 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:19.138+0000 7f3b0261f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-18T23:35:19.442 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:19 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:19.205+0000 7f3b0261f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-18T23:35:19.442 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:19 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:19.266+0000 7f3b0261f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-18T23:35:19.442 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:19 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:19.335+0000 7f3b0261f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-18T23:35:19.719 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:19 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:19.549+0000 7f3b0261f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-18T23:35:19.719 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:19 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:19.611+0000 7f3b0261f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-18T23:35:20.431 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:20 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:20.124+0000 7f3b0261f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-18T23:35:20.719 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:20 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:20.430+0000 7f3b0261f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-18T23:35:21.219 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:20 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:20.841+0000 7f3b0261f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-18T23:35:21.219 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:20 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:20.897+0000 7f3b0261f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-18T23:35:21.219 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:20 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:20.956+0000 7f3b0261f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-18T23:35:22.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:22 smithi078 ceph-mon[18440]: Active manager daemon x restarted 2024-08-18T23:35:22.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:22 smithi078 ceph-mon[18440]: Activating manager daemon x 2024-08-18T23:35:22.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:22 smithi078 ceph-mon[18440]: osdmap e3: 0 total, 0 up, 0 in 2024-08-18T23:35:22.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:22 smithi078 ceph-mon[18440]: mgrmap e10: x(active, starting, since 0.00347249s) 2024-08-18T23:35:22.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-18T23:35:22.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-18T23:35:22.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-18T23:35:22.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-18T23:35:22.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-18T23:35:22.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:22 smithi078 ceph-mon[18440]: Manager daemon x is now available 2024-08-18T23:35:22.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:35:22.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-18T23:35:22.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-18T23:35:23.808 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-08-18T23:35:23.809 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-08-18T23:35:23.809 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:23.682+0000 7f3a6829c640 -1 log_channel(cephadm) log [ERR] : [18/Aug/2024:23:35:23] ENGINE Error in HTTPServer.serve 2024-08-18T23:35:23.809 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: Traceback (most recent call last): 2024-08-18T23:35:23.809 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-18T23:35:23.809 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: self._connections.run(self.expiration_interval) 2024-08-18T23:35:23.809 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-18T23:35:23.809 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: self._run(expiration_interval) 2024-08-18T23:35:23.809 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-18T23:35:23.809 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: new_conn = self._from_server_socket(self.server.socket) 2024-08-18T23:35:23.809 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-18T23:35:23.810 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-18T23:35:23.810 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-18T23:35:23.810 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: s = self.context.wrap_socket( 2024-08-18T23:35:23.810 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-18T23:35:23.810 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: return self.sslsocket_class._create( 2024-08-18T23:35:23.810 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-18T23:35:23.810 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: self.do_handshake() 2024-08-18T23:35:23.810 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-18T23:35:23.810 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: self._sslobj.do_handshake() 2024-08-18T23:35:23.810 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-18T23:35:23.810 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:35:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:35:23.817 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-18T23:35:23.817 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-08-18T23:35:23.817 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.627+0000 7f642fcb1640 1 Processor -- start 2024-08-18T23:35:23.817 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.627+0000 7f642fcb1640 1 -- start start 2024-08-18T23:35:23.817 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.627+0000 7f642fcb1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f642807cd00 0x7f642807b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:23.817 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.627+0000 7f642fcb1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f642807b780 con 0x7f642807cd00 2024-08-18T23:35:23.817 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.627+0000 7f642da4d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f642807cd00 0x7f642807b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:23.817 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.627+0000 7f642da4d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f642807cd00 0x7f642807b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45258/0 (socket says 172.21.15.78:45258) 2024-08-18T23:35:23.817 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.627+0000 7f642da4d640 1 -- 172.21.15.78:0/3929582640 learned_addr learned my addr 172.21.15.78:0/3929582640 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:23.817 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.627+0000 7f642da4d640 1 -- 172.21.15.78:0/3929582640 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642807b900 con 0x7f642807cd00 2024-08-18T23:35:23.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.628+0000 7f642da4d640 1 --2- 172.21.15.78:0/3929582640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f642807cd00 0x7f642807b1b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6424009820 tx=0x7f642402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd38d2544611dfce server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:23.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.628+0000 7f642ca4b640 1 -- 172.21.15.78:0/3929582640 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6424037070 con 0x7f642807cd00 2024-08-18T23:35:23.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.628+0000 7f642ca4b640 1 -- 172.21.15.78:0/3929582640 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6424032920 con 0x7f642807cd00 2024-08-18T23:35:23.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.628+0000 7f642ca4b640 1 -- 172.21.15.78:0/3929582640 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6424030810 con 0x7f642807cd00 2024-08-18T23:35:23.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.629+0000 7f642fcb1640 1 -- 172.21.15.78:0/3929582640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f642807cd00 msgr2=0x7f642807b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:23.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.629+0000 7f642fcb1640 1 --2- 172.21.15.78:0/3929582640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f642807cd00 0x7f642807b1b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6424009820 tx=0x7f642402a270 comp rx=0 tx=0).stop 2024-08-18T23:35:23.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.629+0000 7f642fcb1640 1 -- 172.21.15.78:0/3929582640 shutdown_connections 2024-08-18T23:35:23.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.629+0000 7f642fcb1640 1 --2- 172.21.15.78:0/3929582640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f642807cd00 0x7f642807b1b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:23.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.629+0000 7f642fcb1640 1 -- 172.21.15.78:0/3929582640 >> 172.21.15.78:0/3929582640 conn(0x7f6428102ab0 msgr2=0x7f6428104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:23.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.629+0000 7f642fcb1640 1 -- 172.21.15.78:0/3929582640 shutdown_connections 2024-08-18T23:35:23.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.629+0000 7f642fcb1640 1 -- 172.21.15.78:0/3929582640 wait complete. 2024-08-18T23:35:23.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.630+0000 7f642fcb1640 1 Processor -- start 2024-08-18T23:35:23.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.630+0000 7f642fcb1640 1 -- start start 2024-08-18T23:35:23.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.630+0000 7f642fcb1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f642807cd00 0x7f642819d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:23.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.630+0000 7f642fcb1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f642810d910 con 0x7f642807cd00 2024-08-18T23:35:23.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.630+0000 7f642da4d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f642807cd00 0x7f642819d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:23.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.630+0000 7f642da4d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f642807cd00 0x7f642819d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45260/0 (socket says 172.21.15.78:45260) 2024-08-18T23:35:23.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.630+0000 7f642da4d640 1 -- 172.21.15.78:0/934539977 learned_addr learned my addr 172.21.15.78:0/934539977 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:23.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.630+0000 7f642da4d640 1 -- 172.21.15.78:0/934539977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642819dd00 con 0x7f642807cd00 2024-08-18T23:35:23.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.630+0000 7f642da4d640 1 --2- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f642807cd00 0x7f642819d7c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f64240097f0 tx=0x7f64240327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:23.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.630+0000 7f641effd640 1 -- 172.21.15.78:0/934539977 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6424037040 con 0x7f642807cd00 2024-08-18T23:35:23.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.630+0000 7f641effd640 1 -- 172.21.15.78:0/934539977 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6424030c40 con 0x7f642807cd00 2024-08-18T23:35:23.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.630+0000 7f642fcb1640 1 -- 172.21.15.78:0/934539977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f642819df90 con 0x7f642807cd00 2024-08-18T23:35:23.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.630+0000 7f641effd640 1 -- 172.21.15.78:0/934539977 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f642403a5d0 con 0x7f642807cd00 2024-08-18T23:35:23.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.631+0000 7f642fcb1640 1 -- 172.21.15.78:0/934539977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64281a0c80 con 0x7f642807cd00 2024-08-18T23:35:23.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.631+0000 7f641effd640 1 -- 172.21.15.78:0/934539977 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f642403a7c0 con 0x7f642807cd00 2024-08-18T23:35:23.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.632+0000 7f641effd640 1 --2- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f6414039020 0x7f641403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:23.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.632+0000 7f642fcb1640 1 -- 172.21.15.78:0/934539977 --> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f642807b230 con 0x7f6414039020 2024-08-18T23:35:23.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.632+0000 7f641effd640 1 -- 172.21.15.78:0/934539977 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f642406c600 con 0x7f642807cd00 2024-08-18T23:35:23.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.632+0000 7f642d24c640 1 -- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f6414039020 msgr2=0x7f641403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/311758909 2024-08-18T23:35:23.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.632+0000 7f642d24c640 1 --2- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f6414039020 0x7f641403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-18T23:35:23.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.833+0000 7f642d24c640 1 -- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f6414039020 msgr2=0x7f641403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/311758909 2024-08-18T23:35:23.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:19.833+0000 7f642d24c640 1 --2- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f6414039020 0x7f641403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-18T23:35:23.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:20.234+0000 7f642d24c640 1 -- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f6414039020 msgr2=0x7f641403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/311758909 2024-08-18T23:35:23.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:20.234+0000 7f642d24c640 1 --2- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f6414039020 0x7f641403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-18T23:35:23.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:21.035+0000 7f642d24c640 1 -- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f6414039020 msgr2=0x7f641403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/311758909 2024-08-18T23:35:23.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:21.035+0000 7f642d24c640 1 --2- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f6414039020 0x7f641403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-18T23:35:23.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:22.037+0000 7f641effd640 1 -- 172.21.15.78:0/934539977 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f642406b0b0 con 0x7f642807cd00 2024-08-18T23:35:23.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:22.037+0000 7f641effd640 1 -- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f6414039020 msgr2=0x7f641403b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-18T23:35:23.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:22.037+0000 7f641effd640 1 --2- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f6414039020 0x7f641403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:23.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:23.039+0000 7f641effd640 1 -- 172.21.15.78:0/934539977 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f6424044920 con 0x7f642807cd00 2024-08-18T23:35:23.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:23.039+0000 7f641effd640 1 --2- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f641403c8f0 0x7f641403ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:23.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:23.039+0000 7f641effd640 1 -- 172.21.15.78:0/934539977 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f642403ce70 con 0x7f641403c8f0 2024-08-18T23:35:23.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:23.039+0000 7f642d24c640 1 --2- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f641403c8f0 0x7f641403ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:23.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:23.040+0000 7f642d24c640 1 --2- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f641403c8f0 0x7f641403ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6418003980 tx=0x7f64180092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:23.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:23.041+0000 7f641effd640 1 -- 172.21.15.78:0/934539977 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f642403ce70 con 0x7f641403c8f0 2024-08-18T23:35:23.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:23.048+0000 7f642fcb1640 1 -- 172.21.15.78:0/934539977 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f64280682c0 con 0x7f641403c8f0 2024-08-18T23:35:23.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:23.048+0000 7f641effd640 1 -- 172.21.15.78:0/934539977 <== mgr.14150 v2:172.21.15.78:6800/4000744943 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f64280682c0 con 0x7f641403c8f0 2024-08-18T23:35:23.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:23.048+0000 7f642fcb1640 1 -- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f641403c8f0 msgr2=0x7f641403ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:23.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:23.048+0000 7f642fcb1640 1 --2- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f641403c8f0 0x7f641403ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6418003980 tx=0x7f64180092b0 comp rx=0 tx=0).stop 2024-08-18T23:35:23.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:23.048+0000 7f642fcb1640 1 -- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f642807cd00 msgr2=0x7f642819d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:23.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:23.048+0000 7f642fcb1640 1 --2- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f642807cd00 0x7f642819d7c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f64240097f0 tx=0x7f64240327b0 comp rx=0 tx=0).stop 2024-08-18T23:35:23.822 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:23.048+0000 7f642fcb1640 1 -- 172.21.15.78:0/934539977 shutdown_connections 2024-08-18T23:35:23.822 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:23.048+0000 7f642fcb1640 1 --2- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f641403c8f0 0x7f641403ece0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:23.822 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:23.048+0000 7f642fcb1640 1 --2- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:6800/311758909,v1:172.21.15.78:6801/311758909] conn(0x7f6414039020 0x7f641403b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:23.822 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:23.048+0000 7f642fcb1640 1 --2- 172.21.15.78:0/934539977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f642807cd00 0x7f642819d7c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:23.822 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:23.048+0000 7f642fcb1640 1 -- 172.21.15.78:0/934539977 >> 172.21.15.78:0/934539977 conn(0x7f6428102ab0 msgr2=0x7f64281037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:23.822 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:23.048+0000 7f642fcb1640 1 -- 172.21.15.78:0/934539977 shutdown_connections 2024-08-18T23:35:23.822 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:23.048+0000 7f642fcb1640 1 -- 172.21.15.78:0/934539977 wait complete. 2024-08-18T23:35:23.822 INFO:teuthology.orchestra.run.smithi078.stdout:mgr epoch 9 is available 2024-08-18T23:35:23.822 INFO:teuthology.orchestra.run.smithi078.stdout:Generating a dashboard self-signed certificate... 2024-08-18T23:35:24.158 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:24 smithi078 ceph-mon[18440]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-18T23:35:24.159 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:24 smithi078 ceph-mon[18440]: mgrmap e11: x(active, since 1.00646s) 2024-08-18T23:35:24.159 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:24 smithi078 ceph-mon[18440]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-08-18T23:35:24.159 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:24 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:25.048 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-08-18T23:35:25.048 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.236+0000 7f7e36c99640 1 Processor -- start 2024-08-18T23:35:25.048 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.236+0000 7f7e36c99640 1 -- start start 2024-08-18T23:35:25.048 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.236+0000 7f7e36c99640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e30106710 0x7f7e30106b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:25.049 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.236+0000 7f7e36c99640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7e301070e0 con 0x7f7e30106710 2024-08-18T23:35:25.049 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.237+0000 7f7e34a35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e30106710 0x7f7e30106b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:25.049 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.237+0000 7f7e34a35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e30106710 0x7f7e30106b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45324/0 (socket says 172.21.15.78:45324) 2024-08-18T23:35:25.049 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.237+0000 7f7e34a35640 1 -- 172.21.15.78:0/3303151032 learned_addr learned my addr 172.21.15.78:0/3303151032 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:25.049 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.237+0000 7f7e34a35640 1 -- 172.21.15.78:0/3303151032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e30107430 con 0x7f7e30106710 2024-08-18T23:35:25.049 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.237+0000 7f7e34a35640 1 --2- 172.21.15.78:0/3303151032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e30106710 0x7f7e30106b10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009820 tx=0x7f7e2402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3c7eaf14512d4b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:25.049 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.238+0000 7f7e2f7fe640 1 -- 172.21.15.78:0/3303151032 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e24037070 con 0x7f7e30106710 2024-08-18T23:35:25.050 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.238+0000 7f7e2f7fe640 1 -- 172.21.15.78:0/3303151032 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e24032920 con 0x7f7e30106710 2024-08-18T23:35:25.050 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.238+0000 7f7e2f7fe640 1 -- 172.21.15.78:0/3303151032 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e24030810 con 0x7f7e30106710 2024-08-18T23:35:25.050 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.238+0000 7f7e36c99640 1 -- 172.21.15.78:0/3303151032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e30106710 msgr2=0x7f7e30106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:25.050 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.238+0000 7f7e36c99640 1 --2- 172.21.15.78:0/3303151032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e30106710 0x7f7e30106b10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009820 tx=0x7f7e2402a270 comp rx=0 tx=0).stop 2024-08-18T23:35:25.050 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.238+0000 7f7e36c99640 1 -- 172.21.15.78:0/3303151032 shutdown_connections 2024-08-18T23:35:25.050 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.238+0000 7f7e36c99640 1 --2- 172.21.15.78:0/3303151032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e30106710 0x7f7e30106b10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:25.050 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.238+0000 7f7e36c99640 1 -- 172.21.15.78:0/3303151032 >> 172.21.15.78:0/3303151032 conn(0x7f7e30101140 msgr2=0x7f7e30103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:25.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.238+0000 7f7e36c99640 1 -- 172.21.15.78:0/3303151032 shutdown_connections 2024-08-18T23:35:25.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.238+0000 7f7e36c99640 1 -- 172.21.15.78:0/3303151032 wait complete. 2024-08-18T23:35:25.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.239+0000 7f7e36c99640 1 Processor -- start 2024-08-18T23:35:25.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.239+0000 7f7e36c99640 1 -- start start 2024-08-18T23:35:25.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.239+0000 7f7e36c99640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e30106710 0x7f7e3019b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:25.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.239+0000 7f7e36c99640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3010a7e0 con 0x7f7e30106710 2024-08-18T23:35:25.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.239+0000 7f7e34a35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e30106710 0x7f7e3019b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:25.051 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.239+0000 7f7e34a35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e30106710 0x7f7e3019b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45326/0 (socket says 172.21.15.78:45326) 2024-08-18T23:35:25.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.239+0000 7f7e34a35640 1 -- 172.21.15.78:0/557453646 learned_addr learned my addr 172.21.15.78:0/557453646 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:25.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.239+0000 7f7e34a35640 1 -- 172.21.15.78:0/557453646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3019bb60 con 0x7f7e30106710 2024-08-18T23:35:25.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.240+0000 7f7e34a35640 1 --2- 172.21.15.78:0/557453646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e30106710 0x7f7e3019b620 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7e24030560 tx=0x7f7e24032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:25.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.240+0000 7f7e2dffb640 1 -- 172.21.15.78:0/557453646 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e24037040 con 0x7f7e30106710 2024-08-18T23:35:25.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.240+0000 7f7e2dffb640 1 -- 172.21.15.78:0/557453646 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e24040070 con 0x7f7e30106710 2024-08-18T23:35:25.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.240+0000 7f7e36c99640 1 -- 172.21.15.78:0/557453646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3019bdf0 con 0x7f7e30106710 2024-08-18T23:35:25.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.240+0000 7f7e2dffb640 1 -- 172.21.15.78:0/557453646 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e2403a740 con 0x7f7e30106710 2024-08-18T23:35:25.052 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.240+0000 7f7e36c99640 1 -- 172.21.15.78:0/557453646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3019eae0 con 0x7f7e30106710 2024-08-18T23:35:25.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.241+0000 7f7e2dffb640 1 -- 172.21.15.78:0/557453646 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f7e2403a8e0 con 0x7f7e30106710 2024-08-18T23:35:25.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.241+0000 7f7e2dffb640 1 --2- 172.21.15.78:0/557453646 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7e20038f00 0x7f7e2003b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:25.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.241+0000 7f7e36c99640 1 -- 172.21.15.78:0/557453646 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e30106b10 con 0x7f7e30106710 2024-08-18T23:35:25.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.241+0000 7f7e2dffb640 1 -- 172.21.15.78:0/557453646 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7e2406c4e0 con 0x7f7e30106710 2024-08-18T23:35:25.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.241+0000 7f7e2ffff640 1 --2- 172.21.15.78:0/557453646 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7e20038f00 0x7f7e2003b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:25.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.241+0000 7f7e2ffff640 1 --2- 172.21.15.78:0/557453646 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7e20038f00 0x7f7e2003b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0098c0 tx=0x7f7e1c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:25.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.246+0000 7f7e2dffb640 1 -- 172.21.15.78:0/557453646 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7e24044c70 con 0x7f7e30106710 2024-08-18T23:35:25.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.414+0000 7f7e2dffb640 1 -- 172.21.15.78:0/557453646 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f7e2406b010 con 0x7f7e30106710 2024-08-18T23:35:25.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.428+0000 7f7e36c99640 1 -- 172.21.15.78:0/557453646 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f7e30062360 con 0x7f7e20038f00 2024-08-18T23:35:25.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.494+0000 7f7e2dffb640 1 -- 172.21.15.78:0/557453646 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f7e30062360 con 0x7f7e20038f00 2024-08-18T23:35:25.053 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.496+0000 7f7e36c99640 1 -- 172.21.15.78:0/557453646 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7e20038f00 msgr2=0x7f7e2003b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:25.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.496+0000 7f7e36c99640 1 --2- 172.21.15.78:0/557453646 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7e20038f00 0x7f7e2003b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0098c0 tx=0x7f7e1c006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:25.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.496+0000 7f7e36c99640 1 -- 172.21.15.78:0/557453646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e30106710 msgr2=0x7f7e3019b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:25.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.496+0000 7f7e36c99640 1 --2- 172.21.15.78:0/557453646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e30106710 0x7f7e3019b620 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7e24030560 tx=0x7f7e24032670 comp rx=0 tx=0).stop 2024-08-18T23:35:25.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.496+0000 7f7e36c99640 1 -- 172.21.15.78:0/557453646 shutdown_connections 2024-08-18T23:35:25.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.496+0000 7f7e36c99640 1 --2- 172.21.15.78:0/557453646 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7e20038f00 0x7f7e2003b3c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:25.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.496+0000 7f7e36c99640 1 --2- 172.21.15.78:0/557453646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e30106710 0x7f7e3019b620 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:25.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.496+0000 7f7e36c99640 1 -- 172.21.15.78:0/557453646 >> 172.21.15.78:0/557453646 conn(0x7f7e30101140 msgr2=0x7f7e30101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:25.054 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.496+0000 7f7e36c99640 1 -- 172.21.15.78:0/557453646 shutdown_connections 2024-08-18T23:35:25.055 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:24.496+0000 7f7e36c99640 1 -- 172.21.15.78:0/557453646 wait complete. 2024-08-18T23:35:25.055 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial admin user... 2024-08-18T23:35:25.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:25 smithi078 ceph-mon[18440]: [18/Aug/2024:23:35:23] ENGINE Bus STARTING 2024-08-18T23:35:25.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:25 smithi078 ceph-mon[18440]: [18/Aug/2024:23:35:23] ENGINE Serving on http://172.21.15.78:8765 2024-08-18T23:35:25.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:25 smithi078 ceph-mon[18440]: [18/Aug/2024:23:35:23] ENGINE Serving on https://172.21.15.78:7150 2024-08-18T23:35:25.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:25 smithi078 ceph-mon[18440]: [18/Aug/2024:23:35:23] ENGINE Bus STARTED 2024-08-18T23:35:25.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:25 smithi078 ceph-mon[18440]: [18/Aug/2024:23:35:23] ENGINE Error in HTTPServer.serve 2024-08-18T23:35:25.220 INFO:journalctl@ceph.mon.a.smithi078.stdout: Traceback (most recent call last): 2024-08-18T23:35:25.220 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-18T23:35:25.220 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._connections.run(self.expiration_interval) 2024-08-18T23:35:25.220 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-18T23:35:25.220 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._run(expiration_interval) 2024-08-18T23:35:25.220 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-18T23:35:25.220 INFO:journalctl@ceph.mon.a.smithi078.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-18T23:35:25.220 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-18T23:35:25.220 INFO:journalctl@ceph.mon.a.smithi078.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-18T23:35:25.220 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-18T23:35:25.220 INFO:journalctl@ceph.mon.a.smithi078.stdout: s = self.context.wrap_socket( 2024-08-18T23:35:25.220 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-18T23:35:25.220 INFO:journalctl@ceph.mon.a.smithi078.stdout: return self.sslsocket_class._create( 2024-08-18T23:35:25.220 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-18T23:35:25.221 INFO:journalctl@ceph.mon.a.smithi078.stdout: self.do_handshake() 2024-08-18T23:35:25.221 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-18T23:35:25.221 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._sslobj.do_handshake() 2024-08-18T23:35:25.221 INFO:journalctl@ceph.mon.a.smithi078.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-18T23:35:25.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:25 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:25.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:25 smithi078 ceph-mon[18440]: mgrmap e12: x(active, since 2s) 2024-08-18T23:35:25.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:25 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:25.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:25 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:26.331 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:26 smithi078 ceph-mon[18440]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:35:26.331 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:26.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$5dq6eqjBIRUBDTFKns9Hu.LSqVt8TzOMFb/x1tnx.LYCRoCH4Ykbm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1724024125, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-08-18T23:35:26.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.423+0000 7f27d62d1640 1 Processor -- start 2024-08-18T23:35:26.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.423+0000 7f27d62d1640 1 -- start start 2024-08-18T23:35:26.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.423+0000 7f27d62d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d01061d0 0x7f27d01065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:26.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.423+0000 7f27d62d1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27d0106ba0 con 0x7f27d01061d0 2024-08-18T23:35:26.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.424+0000 7f27cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d01061d0 0x7f27d01065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:26.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.424+0000 7f27cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d01061d0 0x7f27d01065d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45328/0 (socket says 172.21.15.78:45328) 2024-08-18T23:35:26.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.424+0000 7f27cffff640 1 -- 172.21.15.78:0/2015006504 learned_addr learned my addr 172.21.15.78:0/2015006504 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:26.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.424+0000 7f27cffff640 1 -- 172.21.15.78:0/2015006504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d01073d0 con 0x7f27d01061d0 2024-08-18T23:35:26.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.425+0000 7f27cffff640 1 --2- 172.21.15.78:0/2015006504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d01061d0 0x7f27d01065d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009820 tx=0x7f27c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96db570fd8f6c0e9 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:26.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.425+0000 7f27ceffd640 1 -- 172.21.15.78:0/2015006504 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27c0037070 con 0x7f27d01061d0 2024-08-18T23:35:26.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.425+0000 7f27ceffd640 1 -- 172.21.15.78:0/2015006504 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c0032920 con 0x7f27d01061d0 2024-08-18T23:35:26.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.425+0000 7f27ceffd640 1 -- 172.21.15.78:0/2015006504 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27c0030810 con 0x7f27d01061d0 2024-08-18T23:35:26.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.426+0000 7f27d62d1640 1 -- 172.21.15.78:0/2015006504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d01061d0 msgr2=0x7f27d01065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:26.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.426+0000 7f27d62d1640 1 --2- 172.21.15.78:0/2015006504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d01061d0 0x7f27d01065d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009820 tx=0x7f27c002a270 comp rx=0 tx=0).stop 2024-08-18T23:35:26.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.426+0000 7f27d62d1640 1 -- 172.21.15.78:0/2015006504 shutdown_connections 2024-08-18T23:35:26.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.426+0000 7f27d62d1640 1 --2- 172.21.15.78:0/2015006504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d01061d0 0x7f27d01065d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:26.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.426+0000 7f27d62d1640 1 -- 172.21.15.78:0/2015006504 >> 172.21.15.78:0/2015006504 conn(0x7f27d01019a0 msgr2=0x7f27d0103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:26.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.426+0000 7f27d62d1640 1 -- 172.21.15.78:0/2015006504 shutdown_connections 2024-08-18T23:35:26.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.426+0000 7f27d62d1640 1 -- 172.21.15.78:0/2015006504 wait complete. 2024-08-18T23:35:26.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.427+0000 7f27d62d1640 1 Processor -- start 2024-08-18T23:35:26.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.427+0000 7f27d62d1640 1 -- start start 2024-08-18T23:35:26.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.427+0000 7f27d62d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d01061d0 0x7f27d007b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:26.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.427+0000 7f27d62d1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27d010a780 con 0x7f27d01061d0 2024-08-18T23:35:26.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.427+0000 7f27cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d01061d0 0x7f27d007b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:26.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.427+0000 7f27cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d01061d0 0x7f27d007b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45342/0 (socket says 172.21.15.78:45342) 2024-08-18T23:35:26.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.427+0000 7f27cffff640 1 -- 172.21.15.78:0/1381247519 learned_addr learned my addr 172.21.15.78:0/1381247519 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:26.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.428+0000 7f27cffff640 1 -- 172.21.15.78:0/1381247519 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d019c180 con 0x7f27d01061d0 2024-08-18T23:35:26.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.428+0000 7f27cffff640 1 --2- 172.21.15.78:0/1381247519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d01061d0 0x7f27d007b9f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f27c00097f0 tx=0x7f27c00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:26.970 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.428+0000 7f27cd7fa640 1 -- 172.21.15.78:0/1381247519 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27c0037040 con 0x7f27d01061d0 2024-08-18T23:35:26.970 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.428+0000 7f27cd7fa640 1 -- 172.21.15.78:0/1381247519 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c0030c40 con 0x7f27d01061d0 2024-08-18T23:35:26.970 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.428+0000 7f27cd7fa640 1 -- 172.21.15.78:0/1381247519 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27c003a5b0 con 0x7f27d01061d0 2024-08-18T23:35:26.970 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.428+0000 7f27d62d1640 1 -- 172.21.15.78:0/1381247519 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d01a0790 con 0x7f27d01061d0 2024-08-18T23:35:26.970 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.428+0000 7f27d62d1640 1 -- 172.21.15.78:0/1381247519 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d01a0b30 con 0x7f27d01061d0 2024-08-18T23:35:26.970 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.429+0000 7f27cd7fa640 1 -- 172.21.15.78:0/1381247519 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f27c003a7a0 con 0x7f27d01061d0 2024-08-18T23:35:26.970 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.429+0000 7f27cd7fa640 1 --2- 172.21.15.78:0/1381247519 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f27b8039020 0x7f27b803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:26.970 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.429+0000 7f27cd7fa640 1 -- 172.21.15.78:0/1381247519 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f27c006c4f0 con 0x7f27d01061d0 2024-08-18T23:35:26.970 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.430+0000 7f27d62d1640 1 -- 172.21.15.78:0/1381247519 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d0106650 con 0x7f27d01061d0 2024-08-18T23:35:26.970 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.430+0000 7f27cf7fe640 1 --2- 172.21.15.78:0/1381247519 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f27b8039020 0x7f27b803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:26.971 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.430+0000 7f27cf7fe640 1 --2- 172.21.15.78:0/1381247519 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f27b8039020 0x7f27b803b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f27c40098c0 tx=0x7f27c4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:26.971 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.435+0000 7f27cd7fa640 1 -- 172.21.15.78:0/1381247519 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f27c0030df0 con 0x7f27d01061d0 2024-08-18T23:35:26.971 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.625+0000 7f27d62d1640 1 -- 172.21.15.78:0/1381247519 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f27d0192990 con 0x7f27b8039020 2024-08-18T23:35:26.971 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.856+0000 7f27cd7fa640 1 -- 172.21.15.78:0/1381247519 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f27d0192990 con 0x7f27b8039020 2024-08-18T23:35:26.971 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.858+0000 7f27d62d1640 1 -- 172.21.15.78:0/1381247519 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f27b8039020 msgr2=0x7f27b803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:26.971 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.858+0000 7f27d62d1640 1 --2- 172.21.15.78:0/1381247519 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f27b8039020 0x7f27b803b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f27c40098c0 tx=0x7f27c4006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:26.971 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.858+0000 7f27d62d1640 1 -- 172.21.15.78:0/1381247519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d01061d0 msgr2=0x7f27d007b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:26.971 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.858+0000 7f27d62d1640 1 --2- 172.21.15.78:0/1381247519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d01061d0 0x7f27d007b9f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f27c00097f0 tx=0x7f27c00327b0 comp rx=0 tx=0).stop 2024-08-18T23:35:26.971 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.858+0000 7f27d62d1640 1 -- 172.21.15.78:0/1381247519 shutdown_connections 2024-08-18T23:35:26.971 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.858+0000 7f27d62d1640 1 --2- 172.21.15.78:0/1381247519 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f27b8039020 0x7f27b803b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:26.971 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.858+0000 7f27d62d1640 1 --2- 172.21.15.78:0/1381247519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27d01061d0 0x7f27d007b9f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:26.972 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.858+0000 7f27d62d1640 1 -- 172.21.15.78:0/1381247519 >> 172.21.15.78:0/1381247519 conn(0x7f27d01019a0 msgr2=0x7f27d01027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:26.972 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.859+0000 7f27d62d1640 1 -- 172.21.15.78:0/1381247519 shutdown_connections 2024-08-18T23:35:26.972 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:25.859+0000 7f27d62d1640 1 -- 172.21.15.78:0/1381247519 wait complete. 2024-08-18T23:35:26.972 INFO:teuthology.orchestra.run.smithi078.stdout:Fetching dashboard port number... 2024-08-18T23:35:27.211 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:27 smithi078 ceph-mon[18440]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:35:28.122 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 8443 2024-08-18T23:35:28.122 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.381+0000 7fb4d6af7640 1 Processor -- start 2024-08-18T23:35:28.122 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.381+0000 7fb4d6af7640 1 -- start start 2024-08-18T23:35:28.122 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.381+0000 7fb4d6af7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4d0107d90 0x7fb4d010a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:28.123 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.381+0000 7fb4d6af7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d007bdd0 con 0x7fb4d0107d90 2024-08-18T23:35:28.123 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.381+0000 7fb4d4893640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4d0107d90 0x7fb4d010a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:28.123 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.381+0000 7fb4d4893640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4d0107d90 0x7fb4d010a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39632/0 (socket says 172.21.15.78:39632) 2024-08-18T23:35:28.123 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.381+0000 7fb4d4893640 1 -- 172.21.15.78:0/3867330999 learned_addr learned my addr 172.21.15.78:0/3867330999 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:28.123 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.382+0000 7fb4d4893640 1 -- 172.21.15.78:0/3867330999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d010a6c0 con 0x7fb4d0107d90 2024-08-18T23:35:28.123 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.382+0000 7fb4d4893640 1 --2- 172.21.15.78:0/3867330999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4d0107d90 0x7fb4d010a180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4009820 tx=0x7fb4c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593088259db42f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:28.123 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.382+0000 7fb4cf7fe640 1 -- 172.21.15.78:0/3867330999 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4c4037070 con 0x7fb4d0107d90 2024-08-18T23:35:28.123 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.382+0000 7fb4cf7fe640 1 -- 172.21.15.78:0/3867330999 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4c4032920 con 0x7fb4d0107d90 2024-08-18T23:35:28.123 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.382+0000 7fb4cf7fe640 1 -- 172.21.15.78:0/3867330999 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4c4030810 con 0x7fb4d0107d90 2024-08-18T23:35:28.123 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.383+0000 7fb4d6af7640 1 -- 172.21.15.78:0/3867330999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4d0107d90 msgr2=0x7fb4d010a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:28.123 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.383+0000 7fb4d6af7640 1 --2- 172.21.15.78:0/3867330999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4d0107d90 0x7fb4d010a180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4009820 tx=0x7fb4c402a270 comp rx=0 tx=0).stop 2024-08-18T23:35:28.123 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.383+0000 7fb4d6af7640 1 -- 172.21.15.78:0/3867330999 shutdown_connections 2024-08-18T23:35:28.123 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.383+0000 7fb4d6af7640 1 --2- 172.21.15.78:0/3867330999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4d0107d90 0x7fb4d010a180 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:28.124 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.383+0000 7fb4d6af7640 1 -- 172.21.15.78:0/3867330999 >> 172.21.15.78:0/3867330999 conn(0x7fb4d01019f0 msgr2=0x7fb4d0103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:28.124 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.383+0000 7fb4d6af7640 1 -- 172.21.15.78:0/3867330999 shutdown_connections 2024-08-18T23:35:28.124 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.383+0000 7fb4d6af7640 1 -- 172.21.15.78:0/3867330999 wait complete. 2024-08-18T23:35:28.124 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.384+0000 7fb4d6af7640 1 Processor -- start 2024-08-18T23:35:28.124 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.384+0000 7fb4d6af7640 1 -- start start 2024-08-18T23:35:28.124 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.384+0000 7fb4d6af7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4d0107d90 0x7fb4d0107610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:28.124 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.384+0000 7fb4d6af7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d010b6e0 con 0x7fb4d0107d90 2024-08-18T23:35:28.124 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.384+0000 7fb4d4893640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4d0107d90 0x7fb4d0107610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:28.124 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.384+0000 7fb4d4893640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4d0107d90 0x7fb4d0107610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39638/0 (socket says 172.21.15.78:39638) 2024-08-18T23:35:28.124 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.384+0000 7fb4d4893640 1 -- 172.21.15.78:0/429829367 learned_addr learned my addr 172.21.15.78:0/429829367 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:28.124 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.384+0000 7fb4d4893640 1 -- 172.21.15.78:0/429829367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d0107b50 con 0x7fb4d0107d90 2024-08-18T23:35:28.124 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.384+0000 7fb4d4893640 1 --2- 172.21.15.78:0/429829367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4d0107d90 0x7fb4d0107610 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4030380 tx=0x7fb4c4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:28.125 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.384+0000 7fb4cdffb640 1 -- 172.21.15.78:0/429829367 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4c4037040 con 0x7fb4d0107d90 2024-08-18T23:35:28.125 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.384+0000 7fb4cdffb640 1 -- 172.21.15.78:0/429829367 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4c4040070 con 0x7fb4d0107d90 2024-08-18T23:35:28.125 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.385+0000 7fb4d6af7640 1 -- 172.21.15.78:0/429829367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d0105d40 con 0x7fb4d0107d90 2024-08-18T23:35:28.125 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.385+0000 7fb4cdffb640 1 -- 172.21.15.78:0/429829367 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4c403a7b0 con 0x7fb4d0107d90 2024-08-18T23:35:28.125 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.385+0000 7fb4d6af7640 1 -- 172.21.15.78:0/429829367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d0106280 con 0x7fb4d0107d90 2024-08-18T23:35:28.125 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.386+0000 7fb4cdffb640 1 -- 172.21.15.78:0/429829367 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fb4c403aa30 con 0x7fb4d0107d90 2024-08-18T23:35:28.125 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.386+0000 7fb4cdffb640 1 --2- 172.21.15.78:0/429829367 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4c0038fc0 0x7fb4c003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:28.125 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.386+0000 7fb4d6af7640 1 -- 172.21.15.78:0/429829367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d01081f0 con 0x7fb4d0107d90 2024-08-18T23:35:28.125 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.386+0000 7fb4cdffb640 1 -- 172.21.15.78:0/429829367 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb4c406c750 con 0x7fb4d0107d90 2024-08-18T23:35:28.125 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.386+0000 7fb4cffff640 1 --2- 172.21.15.78:0/429829367 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4c0038fc0 0x7fb4c003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:28.125 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.386+0000 7fb4cffff640 1 --2- 172.21.15.78:0/429829367 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4c0038fc0 0x7fb4c003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0098c0 tx=0x7fb4bc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:28.125 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.390+0000 7fb4cdffb640 1 -- 172.21.15.78:0/429829367 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb4c4044c70 con 0x7fb4d0107d90 2024-08-18T23:35:28.126 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.561+0000 7fb4d6af7640 1 -- 172.21.15.78:0/429829367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fb4d0194d00 con 0x7fb4d0107d90 2024-08-18T23:35:28.126 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.561+0000 7fb4cdffb640 1 -- 172.21.15.78:0/429829367 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fb4c4038280 con 0x7fb4d0107d90 2024-08-18T23:35:28.126 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.563+0000 7fb4d6af7640 1 -- 172.21.15.78:0/429829367 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4c0038fc0 msgr2=0x7fb4c003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:28.126 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.563+0000 7fb4d6af7640 1 --2- 172.21.15.78:0/429829367 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4c0038fc0 0x7fb4c003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0098c0 tx=0x7fb4bc006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:28.126 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.563+0000 7fb4d6af7640 1 -- 172.21.15.78:0/429829367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4d0107d90 msgr2=0x7fb4d0107610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:28.126 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.563+0000 7fb4d6af7640 1 --2- 172.21.15.78:0/429829367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4d0107d90 0x7fb4d0107610 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4030380 tx=0x7fb4c4032670 comp rx=0 tx=0).stop 2024-08-18T23:35:28.126 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.563+0000 7fb4d6af7640 1 -- 172.21.15.78:0/429829367 shutdown_connections 2024-08-18T23:35:28.126 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.563+0000 7fb4d6af7640 1 --2- 172.21.15.78:0/429829367 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4c0038fc0 0x7fb4c003b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:28.126 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.563+0000 7fb4d6af7640 1 --2- 172.21.15.78:0/429829367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4d0107d90 0x7fb4d0107610 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:28.126 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.563+0000 7fb4d6af7640 1 -- 172.21.15.78:0/429829367 >> 172.21.15.78:0/429829367 conn(0x7fb4d01019f0 msgr2=0x7fb4d0102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:28.126 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.564+0000 7fb4d6af7640 1 -- 172.21.15.78:0/429829367 shutdown_connections 2024-08-18T23:35:28.126 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:27.564+0000 7fb4d6af7640 1 -- 172.21.15.78:0/429829367 wait complete. 2024-08-18T23:35:28.129 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-18T23:35:28.130 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-08-18T23:35:28.137 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-18T23:35:28.137 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-08-18T23:35:28.137 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-08-18T23:35:28.137 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-08-18T23:35:28.138 INFO:teuthology.orchestra.run.smithi078.stdout:Ceph Dashboard is now available at: 2024-08-18T23:35:28.138 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:35:28.138 INFO:teuthology.orchestra.run.smithi078.stdout: URL: https://smithi078:8443/ 2024-08-18T23:35:28.139 INFO:teuthology.orchestra.run.smithi078.stdout: User: admin 2024-08-18T23:35:28.139 INFO:teuthology.orchestra.run.smithi078.stdout: Password: 62nlgob7cx 2024-08-18T23:35:28.139 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:35:28.139 INFO:teuthology.orchestra.run.smithi078.stdout:Saving cluster configuration to /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/config directory 2024-08-18T23:35:28.339 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:28 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/429829367' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-08-18T23:35:29.304 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:29 smithi078 ceph-mon[18440]: mgrmap e13: x(active, since 6s) 2024-08-18T23:35:29.304 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:29 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2827556044' entity='client.admin' 2024-08-18T23:35:29.312 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.494+0000 7faa99bbf640 1 Processor -- start 2024-08-18T23:35:29.312 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.494+0000 7faa99bbf640 1 -- start start 2024-08-18T23:35:29.312 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.495+0000 7faa99bbf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa94107d30 0x7faa9410a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:29.312 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.495+0000 7faa99bbf640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faa94075570 con 0x7faa94107d30 2024-08-18T23:35:29.312 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.495+0000 7faa937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa94107d30 0x7faa9410a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:29.312 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.495+0000 7faa937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa94107d30 0x7faa9410a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39644/0 (socket says 172.21.15.78:39644) 2024-08-18T23:35:29.312 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.495+0000 7faa937fe640 1 -- 172.21.15.78:0/1314692952 learned_addr learned my addr 172.21.15.78:0/1314692952 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:29.312 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.495+0000 7faa937fe640 1 -- 172.21.15.78:0/1314692952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9410a660 con 0x7faa94107d30 2024-08-18T23:35:29.312 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.496+0000 7faa937fe640 1 --2- 172.21.15.78:0/1314692952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa94107d30 0x7faa9410a120 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa84009820 tx=0x7faa8402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5f4398d324375a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:29.312 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.496+0000 7faa927fc640 1 -- 172.21.15.78:0/1314692952 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa84037070 con 0x7faa94107d30 2024-08-18T23:35:29.312 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.496+0000 7faa927fc640 1 -- 172.21.15.78:0/1314692952 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa84032920 con 0x7faa94107d30 2024-08-18T23:35:29.312 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.496+0000 7faa927fc640 1 -- 172.21.15.78:0/1314692952 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa84030810 con 0x7faa94107d30 2024-08-18T23:35:29.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.497+0000 7faa99bbf640 1 -- 172.21.15.78:0/1314692952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa94107d30 msgr2=0x7faa9410a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:29.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.497+0000 7faa99bbf640 1 --2- 172.21.15.78:0/1314692952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa94107d30 0x7faa9410a120 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa84009820 tx=0x7faa8402a270 comp rx=0 tx=0).stop 2024-08-18T23:35:29.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.497+0000 7faa99bbf640 1 -- 172.21.15.78:0/1314692952 shutdown_connections 2024-08-18T23:35:29.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.497+0000 7faa99bbf640 1 --2- 172.21.15.78:0/1314692952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa94107d30 0x7faa9410a120 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:29.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.497+0000 7faa99bbf640 1 -- 172.21.15.78:0/1314692952 >> 172.21.15.78:0/1314692952 conn(0x7faa941018f0 msgr2=0x7faa94103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:29.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.497+0000 7faa99bbf640 1 -- 172.21.15.78:0/1314692952 shutdown_connections 2024-08-18T23:35:29.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.497+0000 7faa99bbf640 1 -- 172.21.15.78:0/1314692952 wait complete. 2024-08-18T23:35:29.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.497+0000 7faa99bbf640 1 Processor -- start 2024-08-18T23:35:29.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.497+0000 7faa99bbf640 1 -- start start 2024-08-18T23:35:29.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.498+0000 7faa99bbf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa94107d30 0x7faa9419b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:29.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.498+0000 7faa99bbf640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faa9410b6e0 con 0x7faa94107d30 2024-08-18T23:35:29.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.498+0000 7faa937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa94107d30 0x7faa9419b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:29.313 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.498+0000 7faa937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa94107d30 0x7faa9419b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39646/0 (socket says 172.21.15.78:39646) 2024-08-18T23:35:29.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.498+0000 7faa937fe640 1 -- 172.21.15.78:0/2827556044 learned_addr learned my addr 172.21.15.78:0/2827556044 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:29.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.498+0000 7faa937fe640 1 -- 172.21.15.78:0/2827556044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9419bb20 con 0x7faa94107d30 2024-08-18T23:35:29.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.498+0000 7faa937fe640 1 --2- 172.21.15.78:0/2827556044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa94107d30 0x7faa9419b5e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faa840097f0 tx=0x7faa840327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:29.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.498+0000 7faa90ff9640 1 -- 172.21.15.78:0/2827556044 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa84037040 con 0x7faa94107d30 2024-08-18T23:35:29.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.498+0000 7faa90ff9640 1 -- 172.21.15.78:0/2827556044 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa84030c40 con 0x7faa94107d30 2024-08-18T23:35:29.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.498+0000 7faa99bbf640 1 -- 172.21.15.78:0/2827556044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9419bdb0 con 0x7faa94107d30 2024-08-18T23:35:29.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.498+0000 7faa90ff9640 1 -- 172.21.15.78:0/2827556044 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa8403a5d0 con 0x7faa94107d30 2024-08-18T23:35:29.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.498+0000 7faa99bbf640 1 -- 172.21.15.78:0/2827556044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa9419eaa0 con 0x7faa94107d30 2024-08-18T23:35:29.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.499+0000 7faa90ff9640 1 -- 172.21.15.78:0/2827556044 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7faa8403a870 con 0x7faa94107d30 2024-08-18T23:35:29.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.499+0000 7faa90ff9640 1 --2- 172.21.15.78:0/2827556044 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7faa7c039060 0x7faa7c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:29.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.500+0000 7faa90ff9640 1 -- 172.21.15.78:0/2827556044 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faa8406c730 con 0x7faa94107d30 2024-08-18T23:35:29.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.499+0000 7faa99bbf640 1 -- 172.21.15.78:0/2827556044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa94108190 con 0x7faa94107d30 2024-08-18T23:35:29.314 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.500+0000 7faa92ffd640 1 --2- 172.21.15.78:0/2827556044 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7faa7c039060 0x7faa7c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:29.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.500+0000 7faa92ffd640 1 --2- 172.21.15.78:0/2827556044 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7faa7c039060 0x7faa7c03b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faa880098c0 tx=0x7faa88006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:29.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.504+0000 7faa90ff9640 1 -- 172.21.15.78:0/2827556044 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7faa840392f0 con 0x7faa94107d30 2024-08-18T23:35:29.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.758+0000 7faa99bbf640 1 -- 172.21.15.78:0/2827556044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7faa94068410 con 0x7faa94107d30 2024-08-18T23:35:29.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.760+0000 7faa90ff9640 1 -- 172.21.15.78:0/2827556044 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7faa8403b9d0 con 0x7faa94107d30 2024-08-18T23:35:29.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-08-18T23:35:29.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.763+0000 7faa99bbf640 1 -- 172.21.15.78:0/2827556044 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7faa7c039060 msgr2=0x7faa7c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:29.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.763+0000 7faa99bbf640 1 --2- 172.21.15.78:0/2827556044 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7faa7c039060 0x7faa7c03b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faa880098c0 tx=0x7faa88006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:29.315 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.763+0000 7faa99bbf640 1 -- 172.21.15.78:0/2827556044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa94107d30 msgr2=0x7faa9419b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:29.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.763+0000 7faa99bbf640 1 --2- 172.21.15.78:0/2827556044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa94107d30 0x7faa9419b5e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faa840097f0 tx=0x7faa840327b0 comp rx=0 tx=0).stop 2024-08-18T23:35:29.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.763+0000 7faa99bbf640 1 -- 172.21.15.78:0/2827556044 shutdown_connections 2024-08-18T23:35:29.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.763+0000 7faa99bbf640 1 --2- 172.21.15.78:0/2827556044 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7faa7c039060 0x7faa7c03b520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:29.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.763+0000 7faa99bbf640 1 --2- 172.21.15.78:0/2827556044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa94107d30 0x7faa9419b5e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:29.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.763+0000 7faa99bbf640 1 -- 172.21.15.78:0/2827556044 >> 172.21.15.78:0/2827556044 conn(0x7faa941018f0 msgr2=0x7faa94102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:29.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.763+0000 7faa99bbf640 1 -- 172.21.15.78:0/2827556044 shutdown_connections 2024-08-18T23:35:29.316 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-08-18T23:35:28.763+0000 7faa99bbf640 1 -- 172.21.15.78:0/2827556044 wait complete. 2024-08-18T23:35:29.316 INFO:teuthology.orchestra.run.smithi078.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-08-18T23:35:29.316 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:35:29.316 INFO:teuthology.orchestra.run.smithi078.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-08-18T23:35:29.316 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:35:29.316 INFO:teuthology.orchestra.run.smithi078.stdout:Or, if you are only running a single cluster on this host: 2024-08-18T23:35:29.317 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:35:29.317 INFO:teuthology.orchestra.run.smithi078.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-08-18T23:35:29.317 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:35:29.317 INFO:teuthology.orchestra.run.smithi078.stdout:Please consider enabling telemetry to help improve Ceph: 2024-08-18T23:35:29.317 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:35:29.317 INFO:teuthology.orchestra.run.smithi078.stdout: ceph telemetry on 2024-08-18T23:35:29.317 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:35:29.317 INFO:teuthology.orchestra.run.smithi078.stdout:For more information see: 2024-08-18T23:35:29.317 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:35:29.317 INFO:teuthology.orchestra.run.smithi078.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-08-18T23:35:29.317 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:35:29.317 INFO:teuthology.orchestra.run.smithi078.stdout:Bootstrap complete. 2024-08-18T23:35:29.357 INFO:tasks.cephadm:Fetching config... 2024-08-18T23:35:29.357 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:35:29.357 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-08-18T23:35:29.374 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-08-18T23:35:29.374 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:35:29.374 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-08-18T23:35:29.431 INFO:tasks.cephadm:Fetching mon keyring... 2024-08-18T23:35:29.431 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:35:29.431 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/keyring of=/dev/stdout 2024-08-18T23:35:29.496 INFO:tasks.cephadm:Fetching pub ssh key... 2024-08-18T23:35:29.496 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:35:29.496 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-08-18T23:35:29.551 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-08-18T23:35:29.551 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXqMaE+Nb8FG7cpy3luBBwAJwLn5EGz4p2o1aOY3cuTfjOR+tuLDg3dEhTmOzsePQ/2YEWym2qTBQJ4zu4QkX7fBLFNETgMqpCjQykA0pUJnFOSo42gVZ+fwa/1r1liNntjHqtYst0KDcb7MB8/XKzVECgHOtI6pcoC4JQogMcL62TL+51lGUdV5g7MLeoTsV7koshfEvUKkIo5hc+w4HB21EOqQJ+OZ7hbQtj/4yHOYV2cJwTSd3ULZ4jcQiMPK0LLyUaMRkod8CINFYo/0i32m1VcJYHrOKfPF04RiGxxw5QQsK3G1S9uWWTqCxekuwyxOTXRvRSP5/OfTUpg4SUVBZdOyN5V5wnkQjPi+UWAYYc3GnHrn7NSUSfQnpqvS9OWdzh4SFxm+qSodPntUIxGwQpRCCBNHi2Lke8uYJNxiHTUtj3BanH6d+y+tY2j2IKPS+gCzmelmOOQ1Q8NVCZR3cprRfuIs23APM029DBL0KSRDhPN/LXbKjdXb2dCHc= ceph-4f1128e6-5dba-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-18T23:35:29.628 INFO:teuthology.orchestra.run.smithi078.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXqMaE+Nb8FG7cpy3luBBwAJwLn5EGz4p2o1aOY3cuTfjOR+tuLDg3dEhTmOzsePQ/2YEWym2qTBQJ4zu4QkX7fBLFNETgMqpCjQykA0pUJnFOSo42gVZ+fwa/1r1liNntjHqtYst0KDcb7MB8/XKzVECgHOtI6pcoC4JQogMcL62TL+51lGUdV5g7MLeoTsV7koshfEvUKkIo5hc+w4HB21EOqQJ+OZ7hbQtj/4yHOYV2cJwTSd3ULZ4jcQiMPK0LLyUaMRkod8CINFYo/0i32m1VcJYHrOKfPF04RiGxxw5QQsK3G1S9uWWTqCxekuwyxOTXRvRSP5/OfTUpg4SUVBZdOyN5V5wnkQjPi+UWAYYc3GnHrn7NSUSfQnpqvS9OWdzh4SFxm+qSodPntUIxGwQpRCCBNHi2Lke8uYJNxiHTUtj3BanH6d+y+tY2j2IKPS+gCzmelmOOQ1Q8NVCZR3cprRfuIs23APM029DBL0KSRDhPN/LXbKjdXb2dCHc= ceph-4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:35:29.639 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXqMaE+Nb8FG7cpy3luBBwAJwLn5EGz4p2o1aOY3cuTfjOR+tuLDg3dEhTmOzsePQ/2YEWym2qTBQJ4zu4QkX7fBLFNETgMqpCjQykA0pUJnFOSo42gVZ+fwa/1r1liNntjHqtYst0KDcb7MB8/XKzVECgHOtI6pcoC4JQogMcL62TL+51lGUdV5g7MLeoTsV7koshfEvUKkIo5hc+w4HB21EOqQJ+OZ7hbQtj/4yHOYV2cJwTSd3ULZ4jcQiMPK0LLyUaMRkod8CINFYo/0i32m1VcJYHrOKfPF04RiGxxw5QQsK3G1S9uWWTqCxekuwyxOTXRvRSP5/OfTUpg4SUVBZdOyN5V5wnkQjPi+UWAYYc3GnHrn7NSUSfQnpqvS9OWdzh4SFxm+qSodPntUIxGwQpRCCBNHi2Lke8uYJNxiHTUtj3BanH6d+y+tY2j2IKPS+gCzmelmOOQ1Q8NVCZR3cprRfuIs23APM029DBL0KSRDhPN/LXbKjdXb2dCHc= ceph-4f1128e6-5dba-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-18T23:35:29.726 INFO:teuthology.orchestra.run.smithi123.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXqMaE+Nb8FG7cpy3luBBwAJwLn5EGz4p2o1aOY3cuTfjOR+tuLDg3dEhTmOzsePQ/2YEWym2qTBQJ4zu4QkX7fBLFNETgMqpCjQykA0pUJnFOSo42gVZ+fwa/1r1liNntjHqtYst0KDcb7MB8/XKzVECgHOtI6pcoC4JQogMcL62TL+51lGUdV5g7MLeoTsV7koshfEvUKkIo5hc+w4HB21EOqQJ+OZ7hbQtj/4yHOYV2cJwTSd3ULZ4jcQiMPK0LLyUaMRkod8CINFYo/0i32m1VcJYHrOKfPF04RiGxxw5QQsK3G1S9uWWTqCxekuwyxOTXRvRSP5/OfTUpg4SUVBZdOyN5V5wnkQjPi+UWAYYc3GnHrn7NSUSfQnpqvS9OWdzh4SFxm+qSodPntUIxGwQpRCCBNHi2Lke8uYJNxiHTUtj3BanH6d+y+tY2j2IKPS+gCzmelmOOQ1Q8NVCZR3cprRfuIs23APM029DBL0KSRDhPN/LXbKjdXb2dCHc= ceph-4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:35:29.740 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXqMaE+Nb8FG7cpy3luBBwAJwLn5EGz4p2o1aOY3cuTfjOR+tuLDg3dEhTmOzsePQ/2YEWym2qTBQJ4zu4QkX7fBLFNETgMqpCjQykA0pUJnFOSo42gVZ+fwa/1r1liNntjHqtYst0KDcb7MB8/XKzVECgHOtI6pcoC4JQogMcL62TL+51lGUdV5g7MLeoTsV7koshfEvUKkIo5hc+w4HB21EOqQJ+OZ7hbQtj/4yHOYV2cJwTSd3ULZ4jcQiMPK0LLyUaMRkod8CINFYo/0i32m1VcJYHrOKfPF04RiGxxw5QQsK3G1S9uWWTqCxekuwyxOTXRvRSP5/OfTUpg4SUVBZdOyN5V5wnkQjPi+UWAYYc3GnHrn7NSUSfQnpqvS9OWdzh4SFxm+qSodPntUIxGwQpRCCBNHi2Lke8uYJNxiHTUtj3BanH6d+y+tY2j2IKPS+gCzmelmOOQ1Q8NVCZR3cprRfuIs23APM029DBL0KSRDhPN/LXbKjdXb2dCHc= ceph-4f1128e6-5dba-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-18T23:35:29.798 INFO:teuthology.orchestra.run.smithi161.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXqMaE+Nb8FG7cpy3luBBwAJwLn5EGz4p2o1aOY3cuTfjOR+tuLDg3dEhTmOzsePQ/2YEWym2qTBQJ4zu4QkX7fBLFNETgMqpCjQykA0pUJnFOSo42gVZ+fwa/1r1liNntjHqtYst0KDcb7MB8/XKzVECgHOtI6pcoC4JQogMcL62TL+51lGUdV5g7MLeoTsV7koshfEvUKkIo5hc+w4HB21EOqQJ+OZ7hbQtj/4yHOYV2cJwTSd3ULZ4jcQiMPK0LLyUaMRkod8CINFYo/0i32m1VcJYHrOKfPF04RiGxxw5QQsK3G1S9uWWTqCxekuwyxOTXRvRSP5/OfTUpg4SUVBZdOyN5V5wnkQjPi+UWAYYc3GnHrn7NSUSfQnpqvS9OWdzh4SFxm+qSodPntUIxGwQpRCCBNHi2Lke8uYJNxiHTUtj3BanH6d+y+tY2j2IKPS+gCzmelmOOQ1Q8NVCZR3cprRfuIs23APM029DBL0KSRDhPN/LXbKjdXb2dCHc= ceph-4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:35:29.812 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-08-18T23:35:30.062 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:35:30.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.542+0000 7fe7eb7b2640 1 -- 172.21.15.78:0/2682454362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e4100240 msgr2=0x7fe7e4100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:30.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.542+0000 7fe7eb7b2640 1 --2- 172.21.15.78:0/2682454362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e4100240 0x7fe7e4100640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009880 tx=0x7fe7d402a5d0 comp rx=0 tx=0).stop 2024-08-18T23:35:30.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.542+0000 7fe7eb7b2640 1 -- 172.21.15.78:0/2682454362 shutdown_connections 2024-08-18T23:35:30.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.542+0000 7fe7eb7b2640 1 --2- 172.21.15.78:0/2682454362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e4100240 0x7fe7e4100640 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:30.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.542+0000 7fe7eb7b2640 1 -- 172.21.15.78:0/2682454362 >> 172.21.15.78:0/2682454362 conn(0x7fe7e40fb9b0 msgr2=0x7fe7e40fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:30.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.542+0000 7fe7eb7b2640 1 -- 172.21.15.78:0/2682454362 shutdown_connections 2024-08-18T23:35:30.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.542+0000 7fe7eb7b2640 1 -- 172.21.15.78:0/2682454362 wait complete. 2024-08-18T23:35:30.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.543+0000 7fe7eb7b2640 1 Processor -- start 2024-08-18T23:35:30.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.543+0000 7fe7eb7b2640 1 -- start start 2024-08-18T23:35:30.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.543+0000 7fe7eb7b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e4100240 0x7fe7e40fd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:30.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.543+0000 7fe7eb7b2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e4106350 con 0x7fe7e4100240 2024-08-18T23:35:30.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.543+0000 7fe7e954e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e4100240 0x7fe7e40fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:30.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.543+0000 7fe7e954e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e4100240 0x7fe7e40fd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39668/0 (socket says 172.21.15.78:39668) 2024-08-18T23:35:30.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.543+0000 7fe7e954e640 1 -- 172.21.15.78:0/2589148739 learned_addr learned my addr 172.21.15.78:0/2589148739 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:30.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.544+0000 7fe7e954e640 1 -- 172.21.15.78:0/2589148739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e40fddd0 con 0x7fe7e4100240 2024-08-18T23:35:30.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.544+0000 7fe7e954e640 1 --2- 172.21.15.78:0/2589148739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e4100240 0x7fe7e40fd890 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4004f60 tx=0x7fe7d40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:30.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.544+0000 7fe7da7fc640 1 -- 172.21.15.78:0/2589148739 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7d403f070 con 0x7fe7e4100240 2024-08-18T23:35:30.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.544+0000 7fe7da7fc640 1 -- 172.21.15.78:0/2589148739 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7d4004d40 con 0x7fe7e4100240 2024-08-18T23:35:30.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.544+0000 7fe7eb7b2640 1 -- 172.21.15.78:0/2589148739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e40fbfc0 con 0x7fe7e4100240 2024-08-18T23:35:30.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.544+0000 7fe7da7fc640 1 -- 172.21.15.78:0/2589148739 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7d4035030 con 0x7fe7e4100240 2024-08-18T23:35:30.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.544+0000 7fe7eb7b2640 1 -- 172.21.15.78:0/2589148739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e40fc500 con 0x7fe7e4100240 2024-08-18T23:35:30.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.545+0000 7fe7da7fc640 1 -- 172.21.15.78:0/2589148739 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe7d4042070 con 0x7fe7e4100240 2024-08-18T23:35:30.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.545+0000 7fe7eb7b2640 1 -- 172.21.15.78:0/2589148739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e4106ad0 con 0x7fe7e4100240 2024-08-18T23:35:30.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.545+0000 7fe7da7fc640 1 --2- 172.21.15.78:0/2589148739 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe7d00390c0 0x7fe7d003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:30.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.545+0000 7fe7da7fc640 1 -- 172.21.15.78:0/2589148739 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe7d406cdb0 con 0x7fe7e4100240 2024-08-18T23:35:30.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.545+0000 7fe7e8d4d640 1 --2- 172.21.15.78:0/2589148739 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe7d00390c0 0x7fe7d003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:30.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.545+0000 7fe7e8d4d640 1 --2- 172.21.15.78:0/2589148739 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe7d00390c0 0x7fe7d003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00098c0 tx=0x7fe7e0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:30.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.550+0000 7fe7da7fc640 1 -- 172.21.15.78:0/2589148739 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe7d403be20 con 0x7fe7e4100240 2024-08-18T23:35:30.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.719+0000 7fe7eb7b2640 1 -- 172.21.15.78:0/2589148739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fe7e418ffb0 con 0x7fe7e4100240 2024-08-18T23:35:30.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.722+0000 7fe7da7fc640 1 -- 172.21.15.78:0/2589148739 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fe7d40303a0 con 0x7fe7e4100240 2024-08-18T23:35:30.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.724+0000 7fe7eb7b2640 1 -- 172.21.15.78:0/2589148739 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe7d00390c0 msgr2=0x7fe7d003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:30.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.724+0000 7fe7eb7b2640 1 --2- 172.21.15.78:0/2589148739 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe7d00390c0 0x7fe7d003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00098c0 tx=0x7fe7e0006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:30.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.724+0000 7fe7eb7b2640 1 -- 172.21.15.78:0/2589148739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e4100240 msgr2=0x7fe7e40fd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:30.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.724+0000 7fe7eb7b2640 1 --2- 172.21.15.78:0/2589148739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e4100240 0x7fe7e40fd890 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4004f60 tx=0x7fe7d40048f0 comp rx=0 tx=0).stop 2024-08-18T23:35:30.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.724+0000 7fe7eb7b2640 1 -- 172.21.15.78:0/2589148739 shutdown_connections 2024-08-18T23:35:30.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.724+0000 7fe7eb7b2640 1 --2- 172.21.15.78:0/2589148739 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe7d00390c0 0x7fe7d003b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:30.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.724+0000 7fe7eb7b2640 1 --2- 172.21.15.78:0/2589148739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7e4100240 0x7fe7e40fd890 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:30.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.724+0000 7fe7eb7b2640 1 -- 172.21.15.78:0/2589148739 >> 172.21.15.78:0/2589148739 conn(0x7fe7e40fb9b0 msgr2=0x7fe7e406a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:30.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.724+0000 7fe7eb7b2640 1 -- 172.21.15.78:0/2589148739 shutdown_connections 2024-08-18T23:35:30.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:30.724+0000 7fe7eb7b2640 1 -- 172.21.15.78:0/2589148739 wait complete. 2024-08-18T23:35:31.680 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-08-18T23:35:31.680 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-08-18T23:35:31.905 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:31 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2589148739' entity='client.admin' 2024-08-18T23:35:32.097 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:35:32.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.589+0000 7f6c9de5a640 1 -- 172.21.15.78:0/3734502774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c98101350 msgr2=0x7f6c98101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:32.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.589+0000 7f6c9de5a640 1 --2- 172.21.15.78:0/3734502774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c98101350 0x7f6c98101750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009880 tx=0x7f6c8802a5d0 comp rx=0 tx=0).stop 2024-08-18T23:35:32.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.589+0000 7f6c9de5a640 1 -- 172.21.15.78:0/3734502774 shutdown_connections 2024-08-18T23:35:32.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.589+0000 7f6c9de5a640 1 --2- 172.21.15.78:0/3734502774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c98101350 0x7f6c98101750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:32.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.589+0000 7f6c9de5a640 1 -- 172.21.15.78:0/3734502774 >> 172.21.15.78:0/3734502774 conn(0x7f6c980fcae0 msgr2=0x7f6c980fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:32.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.589+0000 7f6c9de5a640 1 -- 172.21.15.78:0/3734502774 shutdown_connections 2024-08-18T23:35:32.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.589+0000 7f6c9de5a640 1 -- 172.21.15.78:0/3734502774 wait complete. 2024-08-18T23:35:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.590+0000 7f6c9de5a640 1 Processor -- start 2024-08-18T23:35:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.590+0000 7f6c9de5a640 1 -- start start 2024-08-18T23:35:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.590+0000 7f6c9de5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c98101350 0x7f6c98075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.590+0000 7f6c9de5a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c98107460 con 0x7f6c98101350 2024-08-18T23:35:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.590+0000 7f6c977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c98101350 0x7f6c98075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.590+0000 7f6c977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c98101350 0x7f6c98075be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39680/0 (socket says 172.21.15.78:39680) 2024-08-18T23:35:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.590+0000 7f6c977fe640 1 -- 172.21.15.78:0/1328982472 learned_addr learned my addr 172.21.15.78:0/1328982472 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.590+0000 7f6c977fe640 1 -- 172.21.15.78:0/1328982472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c98076120 con 0x7f6c98101350 2024-08-18T23:35:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.590+0000 7f6c977fe640 1 --2- 172.21.15.78:0/1328982472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c98101350 0x7f6c98075be0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6c88004f60 tx=0x7f6c880048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.591+0000 7f6c957fa640 1 -- 172.21.15.78:0/1328982472 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c8803f070 con 0x7f6c98101350 2024-08-18T23:35:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.591+0000 7f6c957fa640 1 -- 172.21.15.78:0/1328982472 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c88004d40 con 0x7f6c98101350 2024-08-18T23:35:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.591+0000 7f6c957fa640 1 -- 172.21.15.78:0/1328982472 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c88035030 con 0x7f6c98101350 2024-08-18T23:35:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.591+0000 7f6c9de5a640 1 -- 172.21.15.78:0/1328982472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c98074310 con 0x7f6c98101350 2024-08-18T23:35:32.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.591+0000 7f6c9de5a640 1 -- 172.21.15.78:0/1328982472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c98074850 con 0x7f6c98101350 2024-08-18T23:35:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.592+0000 7f6c9de5a640 1 -- 172.21.15.78:0/1328982472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c98107be0 con 0x7f6c98101350 2024-08-18T23:35:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.592+0000 7f6c957fa640 1 -- 172.21.15.78:0/1328982472 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6c88042070 con 0x7f6c98101350 2024-08-18T23:35:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.592+0000 7f6c957fa640 1 --2- 172.21.15.78:0/1328982472 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6c84039060 0x7f6c8403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.592+0000 7f6c957fa640 1 -- 172.21.15.78:0/1328982472 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6c8806cd30 con 0x7f6c98101350 2024-08-18T23:35:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.592+0000 7f6c8ffff640 1 --2- 172.21.15.78:0/1328982472 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6c84039060 0x7f6c8403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:32.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.592+0000 7f6c8ffff640 1 --2- 172.21.15.78:0/1328982472 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6c84039060 0x7f6c8403b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6c800098c0 tx=0x7f6c80006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:32.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.597+0000 7f6c957fa640 1 -- 172.21.15.78:0/1328982472 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6c88042360 con 0x7f6c98101350 2024-08-18T23:35:32.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.779+0000 7f6c9de5a640 1 -- 172.21.15.78:0/1328982472 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f6c9818ff20 con 0x7f6c84039060 2024-08-18T23:35:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.782+0000 7f6c957fa640 1 -- 172.21.15.78:0/1328982472 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6c9818ff20 con 0x7f6c84039060 2024-08-18T23:35:32.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.784+0000 7f6c9de5a640 1 -- 172.21.15.78:0/1328982472 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6c84039060 msgr2=0x7f6c8403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:32.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.784+0000 7f6c9de5a640 1 --2- 172.21.15.78:0/1328982472 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6c84039060 0x7f6c8403b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6c800098c0 tx=0x7f6c80006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:32.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.784+0000 7f6c9de5a640 1 -- 172.21.15.78:0/1328982472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c98101350 msgr2=0x7f6c98075be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:32.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.784+0000 7f6c9de5a640 1 --2- 172.21.15.78:0/1328982472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c98101350 0x7f6c98075be0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6c88004f60 tx=0x7f6c880048f0 comp rx=0 tx=0).stop 2024-08-18T23:35:32.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.784+0000 7f6c9de5a640 1 -- 172.21.15.78:0/1328982472 shutdown_connections 2024-08-18T23:35:32.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.784+0000 7f6c9de5a640 1 --2- 172.21.15.78:0/1328982472 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6c84039060 0x7f6c8403b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:32.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.784+0000 7f6c9de5a640 1 --2- 172.21.15.78:0/1328982472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c98101350 0x7f6c98075be0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:32.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.784+0000 7f6c9de5a640 1 -- 172.21.15.78:0/1328982472 >> 172.21.15.78:0/1328982472 conn(0x7f6c980fcae0 msgr2=0x7f6c980fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:32.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.784+0000 7f6c9de5a640 1 -- 172.21.15.78:0/1328982472 shutdown_connections 2024-08-18T23:35:32.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:32.784+0000 7f6c9de5a640 1 -- 172.21.15.78:0/1328982472 wait complete. 2024-08-18T23:35:33.358 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi123 2024-08-18T23:35:33.358 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-18T23:35:33.358 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.conf 2024-08-18T23:35:33.377 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-18T23:35:33.377 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-18T23:35:33.435 INFO:tasks.cephadm:Adding host smithi123 to orchestrator... 2024-08-18T23:35:33.435 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch host add smithi123 2024-08-18T23:35:33.764 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:35:34.034 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:33 smithi078 ceph-mon[18440]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:35:34.034 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:34.034 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:34.034 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:34.034 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:35:34.034 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:35:34.035 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:35:34.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.361+0000 7f947eda3640 1 -- 172.21.15.78:0/2946248631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9478100270 msgr2=0x7f9478100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:34.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.361+0000 7f947eda3640 1 --2- 172.21.15.78:0/2946248631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9478100270 0x7f9478100670 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f946c009880 tx=0x7f946c02a5d0 comp rx=0 tx=0).stop 2024-08-18T23:35:34.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.361+0000 7f947eda3640 1 -- 172.21.15.78:0/2946248631 shutdown_connections 2024-08-18T23:35:34.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.361+0000 7f947eda3640 1 --2- 172.21.15.78:0/2946248631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9478100270 0x7f9478100670 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:34.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.361+0000 7f947eda3640 1 -- 172.21.15.78:0/2946248631 >> 172.21.15.78:0/2946248631 conn(0x7f94780fba00 msgr2=0x7f94780fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:34.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.361+0000 7f947eda3640 1 -- 172.21.15.78:0/2946248631 shutdown_connections 2024-08-18T23:35:34.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.361+0000 7f947eda3640 1 -- 172.21.15.78:0/2946248631 wait complete. 2024-08-18T23:35:34.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.361+0000 7f947eda3640 1 Processor -- start 2024-08-18T23:35:34.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.361+0000 7f947eda3640 1 -- start start 2024-08-18T23:35:34.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.361+0000 7f947eda3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9478100270 0x7f94780fd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:34.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.361+0000 7f947eda3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9478106320 con 0x7f9478100270 2024-08-18T23:35:34.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.362+0000 7f947cb3f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9478100270 0x7f94780fd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:34.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.362+0000 7f947cb3f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9478100270 0x7f94780fd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39700/0 (socket says 172.21.15.78:39700) 2024-08-18T23:35:34.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.362+0000 7f947cb3f640 1 -- 172.21.15.78:0/3826326282 learned_addr learned my addr 172.21.15.78:0/3826326282 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:34.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.362+0000 7f947cb3f640 1 -- 172.21.15.78:0/3826326282 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94780fde00 con 0x7f9478100270 2024-08-18T23:35:34.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.362+0000 7f947cb3f640 1 --2- 172.21.15.78:0/3826326282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9478100270 0x7f94780fd8c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f946c004f60 tx=0x7f946c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:34.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.362+0000 7f9475ffb640 1 -- 172.21.15.78:0/3826326282 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f946c03f070 con 0x7f9478100270 2024-08-18T23:35:34.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.362+0000 7f9475ffb640 1 -- 172.21.15.78:0/3826326282 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f946c004d40 con 0x7f9478100270 2024-08-18T23:35:34.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.362+0000 7f947eda3640 1 -- 172.21.15.78:0/3826326282 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94780fbff0 con 0x7f9478100270 2024-08-18T23:35:34.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.362+0000 7f9475ffb640 1 -- 172.21.15.78:0/3826326282 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f946c035030 con 0x7f9478100270 2024-08-18T23:35:34.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.363+0000 7f947eda3640 1 -- 172.21.15.78:0/3826326282 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94780fc530 con 0x7f9478100270 2024-08-18T23:35:34.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.363+0000 7f9475ffb640 1 -- 172.21.15.78:0/3826326282 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f946c042070 con 0x7f9478100270 2024-08-18T23:35:34.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.364+0000 7f947eda3640 1 -- 172.21.15.78:0/3826326282 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9478106aa0 con 0x7f9478100270 2024-08-18T23:35:34.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.364+0000 7f9475ffb640 1 --2- 172.21.15.78:0/3826326282 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94680390c0 0x7f946803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:34.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.364+0000 7f9475ffb640 1 -- 172.21.15.78:0/3826326282 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f946c06ccb0 con 0x7f9478100270 2024-08-18T23:35:34.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.364+0000 7f9477fff640 1 --2- 172.21.15.78:0/3826326282 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94680390c0 0x7f946803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:34.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.364+0000 7f9477fff640 1 --2- 172.21.15.78:0/3826326282 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94680390c0 0x7f946803b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f94640098c0 tx=0x7f9464006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:34.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.368+0000 7f9475ffb640 1 -- 172.21.15.78:0/3826326282 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f946c039350 con 0x7f9478100270 2024-08-18T23:35:34.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:34.560+0000 7f947eda3640 1 -- 172.21.15.78:0/3826326282 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}) -- 0x7f947818fff0 con 0x7f94680390c0 2024-08-18T23:35:34.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:34 smithi078 ceph-mon[18440]: Updating smithi078:/etc/ceph/ceph.conf 2024-08-18T23:35:34.971 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:34 smithi078 ceph-mon[18440]: Updating smithi078:/var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-18T23:35:34.971 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:34 smithi078 ceph-mon[18440]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-08-18T23:35:34.971 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:34 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:34.971 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:34 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:34.971 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:34 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:34.971 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:34 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:35:34.971 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:34 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:35:34.971 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:34 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:35:34.971 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:34 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:36.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:35 smithi078 ceph-mon[18440]: Updating smithi078:/var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-18T23:35:36.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:35 smithi078 ceph-mon[18440]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:35:37.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:36 smithi078 ceph-mon[18440]: Deploying cephadm binary to smithi123 2024-08-18T23:35:37.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:37.808+0000 7f9475ffb640 1 -- 172.21.15.78:0/3826326282 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f947818fff0 con 0x7f94680390c0 2024-08-18T23:35:37.810 INFO:teuthology.orchestra.run.smithi078.stdout:Added host 'smithi123' with addr '172.21.15.123' 2024-08-18T23:35:37.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:37.810+0000 7f947eda3640 1 -- 172.21.15.78:0/3826326282 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94680390c0 msgr2=0x7f946803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:37.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:37.810+0000 7f947eda3640 1 --2- 172.21.15.78:0/3826326282 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94680390c0 0x7f946803b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f94640098c0 tx=0x7f9464006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:37.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:37.810+0000 7f947eda3640 1 -- 172.21.15.78:0/3826326282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9478100270 msgr2=0x7f94780fd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:37.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:37.810+0000 7f947eda3640 1 --2- 172.21.15.78:0/3826326282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9478100270 0x7f94780fd8c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f946c004f60 tx=0x7f946c0048f0 comp rx=0 tx=0).stop 2024-08-18T23:35:37.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:37.811+0000 7f947eda3640 1 -- 172.21.15.78:0/3826326282 shutdown_connections 2024-08-18T23:35:37.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:37.811+0000 7f947eda3640 1 --2- 172.21.15.78:0/3826326282 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94680390c0 0x7f946803b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:37.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:37.811+0000 7f947eda3640 1 --2- 172.21.15.78:0/3826326282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9478100270 0x7f94780fd8c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:37.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:37.811+0000 7f947eda3640 1 -- 172.21.15.78:0/3826326282 >> 172.21.15.78:0/3826326282 conn(0x7f94780fba00 msgr2=0x7f9478069860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:37.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:37.811+0000 7f947eda3640 1 -- 172.21.15.78:0/3826326282 shutdown_connections 2024-08-18T23:35:37.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:37.811+0000 7f947eda3640 1 -- 172.21.15.78:0/3826326282 wait complete. 2024-08-18T23:35:38.420 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch host ls --format=json 2024-08-18T23:35:38.663 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:35:39.028 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:39.032 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:38 smithi078 ceph-mon[18440]: Added host smithi123 2024-08-18T23:35:39.032 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:35:39.032 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:39.032 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:39.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.107+0000 7f0e9f43c640 1 -- 172.21.15.78:0/1561319347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e98100420 msgr2=0x7f0e98100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:39.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.107+0000 7f0e9f43c640 1 --2- 172.21.15.78:0/1561319347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e98100420 0x7f0e98100800 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009880 tx=0x7f0e8c02a5d0 comp rx=0 tx=0).stop 2024-08-18T23:35:39.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.107+0000 7f0e9f43c640 1 -- 172.21.15.78:0/1561319347 shutdown_connections 2024-08-18T23:35:39.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.107+0000 7f0e9f43c640 1 --2- 172.21.15.78:0/1561319347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e98100420 0x7f0e98100800 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:39.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.107+0000 7f0e9f43c640 1 -- 172.21.15.78:0/1561319347 >> 172.21.15.78:0/1561319347 conn(0x7f0e980fbee0 msgr2=0x7f0e980fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:39.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.107+0000 7f0e9f43c640 1 -- 172.21.15.78:0/1561319347 shutdown_connections 2024-08-18T23:35:39.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.107+0000 7f0e9f43c640 1 -- 172.21.15.78:0/1561319347 wait complete. 2024-08-18T23:35:39.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.107+0000 7f0e9f43c640 1 Processor -- start 2024-08-18T23:35:39.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.108+0000 7f0e9f43c640 1 -- start start 2024-08-18T23:35:39.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.108+0000 7f0e9f43c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e98100420 0x7f0e98195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:39.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.108+0000 7f0e9f43c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9810a870 con 0x7f0e98100420 2024-08-18T23:35:39.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.108+0000 7f0e9d1d8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e98100420 0x7f0e98195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:39.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.108+0000 7f0e9d1d8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e98100420 0x7f0e98195da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37152/0 (socket says 172.21.15.78:37152) 2024-08-18T23:35:39.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.108+0000 7f0e9d1d8640 1 -- 172.21.15.78:0/1943889434 learned_addr learned my addr 172.21.15.78:0/1943889434 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:39.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.108+0000 7f0e9d1d8640 1 -- 172.21.15.78:0/1943889434 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e981962e0 con 0x7f0e98100420 2024-08-18T23:35:39.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.108+0000 7f0e9d1d8640 1 --2- 172.21.15.78:0/1943889434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e98100420 0x7f0e98195da0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009230 tx=0x7f0e8c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:39.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.108+0000 7f0e8a7fc640 1 -- 172.21.15.78:0/1943889434 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e8c03f070 con 0x7f0e98100420 2024-08-18T23:35:39.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.108+0000 7f0e8a7fc640 1 -- 172.21.15.78:0/1943889434 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e8c03bc80 con 0x7f0e98100420 2024-08-18T23:35:39.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.108+0000 7f0e9f43c640 1 -- 172.21.15.78:0/1943889434 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e981965d0 con 0x7f0e98100420 2024-08-18T23:35:39.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.108+0000 7f0e8a7fc640 1 -- 172.21.15.78:0/1943889434 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e8c035030 con 0x7f0e98100420 2024-08-18T23:35:39.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.108+0000 7f0e9f43c640 1 -- 172.21.15.78:0/1943889434 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e9819a670 con 0x7f0e98100420 2024-08-18T23:35:39.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.109+0000 7f0e8a7fc640 1 -- 172.21.15.78:0/1943889434 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0e8c042070 con 0x7f0e98100420 2024-08-18T23:35:39.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.109+0000 7f0e9f43c640 1 -- 172.21.15.78:0/1943889434 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e98101900 con 0x7f0e98100420 2024-08-18T23:35:39.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.110+0000 7f0e8a7fc640 1 --2- 172.21.15.78:0/1943889434 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0e84039060 0x7f0e8403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:39.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.110+0000 7f0e8a7fc640 1 -- 172.21.15.78:0/1943889434 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0e8c06ccc0 con 0x7f0e98100420 2024-08-18T23:35:39.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.110+0000 7f0e9c9d7640 1 --2- 172.21.15.78:0/1943889434 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0e84039060 0x7f0e8403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:39.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.110+0000 7f0e9c9d7640 1 --2- 172.21.15.78:0/1943889434 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0e84039060 0x7f0e8403b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0e940098c0 tx=0x7f0e94006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:39.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.114+0000 7f0e8a7fc640 1 -- 172.21.15.78:0/1943889434 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0e8c031e20 con 0x7f0e98100420 2024-08-18T23:35:39.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.287+0000 7f0e9f43c640 1 -- 172.21.15.78:0/1943889434 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e98197100 con 0x7f0e84039060 2024-08-18T23:35:39.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.288+0000 7f0e8a7fc640 1 -- 172.21.15.78:0/1943889434 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f0e98197100 con 0x7f0e84039060 2024-08-18T23:35:39.289 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:35:39.289 INFO:teuthology.orchestra.run.smithi078.stdout:[{"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}, {"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}] 2024-08-18T23:35:39.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.290+0000 7f0e9f43c640 1 -- 172.21.15.78:0/1943889434 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0e84039060 msgr2=0x7f0e8403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:39.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.290+0000 7f0e9f43c640 1 --2- 172.21.15.78:0/1943889434 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0e84039060 0x7f0e8403b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0e940098c0 tx=0x7f0e94006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:39.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.290+0000 7f0e9f43c640 1 -- 172.21.15.78:0/1943889434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e98100420 msgr2=0x7f0e98195da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:39.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.290+0000 7f0e9f43c640 1 --2- 172.21.15.78:0/1943889434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e98100420 0x7f0e98195da0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009230 tx=0x7f0e8c003d40 comp rx=0 tx=0).stop 2024-08-18T23:35:39.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.290+0000 7f0e9f43c640 1 -- 172.21.15.78:0/1943889434 shutdown_connections 2024-08-18T23:35:39.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.290+0000 7f0e9f43c640 1 --2- 172.21.15.78:0/1943889434 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0e84039060 0x7f0e8403b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:39.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.290+0000 7f0e9f43c640 1 --2- 172.21.15.78:0/1943889434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e98100420 0x7f0e98195da0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:39.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.290+0000 7f0e9f43c640 1 -- 172.21.15.78:0/1943889434 >> 172.21.15.78:0/1943889434 conn(0x7f0e980fbee0 msgr2=0x7f0e98107e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:39.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.290+0000 7f0e9f43c640 1 -- 172.21.15.78:0/1943889434 shutdown_connections 2024-08-18T23:35:39.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:39.290+0000 7f0e9f43c640 1 -- 172.21.15.78:0/1943889434 wait complete. 2024-08-18T23:35:39.899 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi161 2024-08-18T23:35:39.900 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-18T23:35:39.900 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.conf 2024-08-18T23:35:39.919 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-18T23:35:39.919 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-18T23:35:39.976 INFO:tasks.cephadm:Adding host smithi161 to orchestrator... 2024-08-18T23:35:39.976 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch host add smithi161 2024-08-18T23:35:40.218 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:35:40.452 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:40 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:40.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.654+0000 7f9e9a3c5640 1 -- 172.21.15.78:0/1840772174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e94101520 msgr2=0x7f9e94101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:40.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.654+0000 7f9e9a3c5640 1 --2- 172.21.15.78:0/1840772174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e94101520 0x7f9e94101900 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009880 tx=0x7f9e8402a5d0 comp rx=0 tx=0).stop 2024-08-18T23:35:40.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.654+0000 7f9e9a3c5640 1 -- 172.21.15.78:0/1840772174 shutdown_connections 2024-08-18T23:35:40.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.654+0000 7f9e9a3c5640 1 --2- 172.21.15.78:0/1840772174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e94101520 0x7f9e94101900 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:40.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.654+0000 7f9e9a3c5640 1 -- 172.21.15.78:0/1840772174 >> 172.21.15.78:0/1840772174 conn(0x7f9e940fcfc0 msgr2=0x7f9e940ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:40.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.654+0000 7f9e9a3c5640 1 -- 172.21.15.78:0/1840772174 shutdown_connections 2024-08-18T23:35:40.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.654+0000 7f9e9a3c5640 1 -- 172.21.15.78:0/1840772174 wait complete. 2024-08-18T23:35:40.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.655+0000 7f9e9a3c5640 1 Processor -- start 2024-08-18T23:35:40.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.655+0000 7f9e9a3c5640 1 -- start start 2024-08-18T23:35:40.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.655+0000 7f9e9a3c5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e94101520 0x7f9e94196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:40.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.655+0000 7f9e9a3c5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9410b970 con 0x7f9e94101520 2024-08-18T23:35:40.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.655+0000 7f9e93fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e94101520 0x7f9e94196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:40.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.655+0000 7f9e93fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e94101520 0x7f9e94196ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37178/0 (socket says 172.21.15.78:37178) 2024-08-18T23:35:40.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.655+0000 7f9e93fff640 1 -- 172.21.15.78:0/716582031 learned_addr learned my addr 172.21.15.78:0/716582031 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:40.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.656+0000 7f9e93fff640 1 -- 172.21.15.78:0/716582031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e941973e0 con 0x7f9e94101520 2024-08-18T23:35:40.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.656+0000 7f9e93fff640 1 --2- 172.21.15.78:0/716582031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e94101520 0x7f9e94196ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009230 tx=0x7f9e84003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:40.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.656+0000 7f9e91ffb640 1 -- 172.21.15.78:0/716582031 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e8403f070 con 0x7f9e94101520 2024-08-18T23:35:40.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.656+0000 7f9e91ffb640 1 -- 172.21.15.78:0/716582031 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e8403bc80 con 0x7f9e94101520 2024-08-18T23:35:40.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.656+0000 7f9e91ffb640 1 -- 172.21.15.78:0/716582031 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e84035030 con 0x7f9e94101520 2024-08-18T23:35:40.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.656+0000 7f9e9a3c5640 1 -- 172.21.15.78:0/716582031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e941976d0 con 0x7f9e94101520 2024-08-18T23:35:40.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.656+0000 7f9e9a3c5640 1 -- 172.21.15.78:0/716582031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e9419b770 con 0x7f9e94101520 2024-08-18T23:35:40.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.657+0000 7f9e91ffb640 1 -- 172.21.15.78:0/716582031 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9e84042070 con 0x7f9e94101520 2024-08-18T23:35:40.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.657+0000 7f9e91ffb640 1 --2- 172.21.15.78:0/716582031 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9e80039060 0x7f9e8003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:40.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.657+0000 7f9e9a3c5640 1 -- 172.21.15.78:0/716582031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e94102a00 con 0x7f9e94101520 2024-08-18T23:35:40.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.657+0000 7f9e91ffb640 1 -- 172.21.15.78:0/716582031 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9e8406ccc0 con 0x7f9e94101520 2024-08-18T23:35:40.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.657+0000 7f9e8bfff640 1 --2- 172.21.15.78:0/716582031 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9e80039060 0x7f9e8003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:40.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.658+0000 7f9e8bfff640 1 --2- 172.21.15.78:0/716582031 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9e80039060 0x7f9e8003b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c0098c0 tx=0x7f9e7c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:40.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.662+0000 7f9e91ffb640 1 -- 172.21.15.78:0/716582031 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9e840372d0 con 0x7f9e94101520 2024-08-18T23:35:40.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:40.834+0000 7f9e9a3c5640 1 -- 172.21.15.78:0/716582031 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}) -- 0x7f9e94198200 con 0x7f9e80039060 2024-08-18T23:35:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:41 smithi078 ceph-mon[18440]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:35:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:42 smithi078 ceph-mon[18440]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:35:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:42.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:42.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:35:42.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:35:42.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:35:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:43 smithi078 ceph-mon[18440]: Deploying cephadm binary to smithi161 2024-08-18T23:35:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:43 smithi078 ceph-mon[18440]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:35:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:43 smithi078 ceph-mon[18440]: Updating smithi123:/etc/ceph/ceph.conf 2024-08-18T23:35:43.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:43.967+0000 7f9e91ffb640 1 -- 172.21.15.78:0/716582031 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f9e94198200 con 0x7f9e80039060 2024-08-18T23:35:43.969 INFO:teuthology.orchestra.run.smithi078.stdout:Added host 'smithi161' with addr '172.21.15.161' 2024-08-18T23:35:43.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:43.969+0000 7f9e9a3c5640 1 -- 172.21.15.78:0/716582031 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9e80039060 msgr2=0x7f9e8003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:43.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:43.969+0000 7f9e9a3c5640 1 --2- 172.21.15.78:0/716582031 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9e80039060 0x7f9e8003b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c0098c0 tx=0x7f9e7c006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:43.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:43.969+0000 7f9e9a3c5640 1 -- 172.21.15.78:0/716582031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e94101520 msgr2=0x7f9e94196ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:43.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:43.969+0000 7f9e9a3c5640 1 --2- 172.21.15.78:0/716582031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e94101520 0x7f9e94196ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009230 tx=0x7f9e84003d40 comp rx=0 tx=0).stop 2024-08-18T23:35:43.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:43.970+0000 7f9e9a3c5640 1 -- 172.21.15.78:0/716582031 shutdown_connections 2024-08-18T23:35:43.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:43.970+0000 7f9e9a3c5640 1 --2- 172.21.15.78:0/716582031 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9e80039060 0x7f9e8003b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:43.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:43.970+0000 7f9e9a3c5640 1 --2- 172.21.15.78:0/716582031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e94101520 0x7f9e94196ea0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:43.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:43.970+0000 7f9e9a3c5640 1 -- 172.21.15.78:0/716582031 >> 172.21.15.78:0/716582031 conn(0x7f9e940fcfc0 msgr2=0x7f9e940fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:43.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:43.970+0000 7f9e9a3c5640 1 -- 172.21.15.78:0/716582031 shutdown_connections 2024-08-18T23:35:43.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:43.970+0000 7f9e9a3c5640 1 -- 172.21.15.78:0/716582031 wait complete. 2024-08-18T23:35:44.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:44 smithi078 ceph-mon[18440]: Updating smithi123:/var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-18T23:35:44.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:44 smithi078 ceph-mon[18440]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-08-18T23:35:44.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:44 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:44.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:44 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:44.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:44 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:44.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:44 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:44.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:44 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:35:44.574 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch host ls --format=json 2024-08-18T23:35:44.816 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:35:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:45 smithi078 ceph-mon[18440]: Updating smithi123:/var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-18T23:35:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:45 smithi078 ceph-mon[18440]: Added host smithi161 2024-08-18T23:35:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:45 smithi078 ceph-mon[18440]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:35:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:45 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:45 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:45.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.238+0000 7f496c5b2640 1 -- 172.21.15.78:0/638601954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4964101520 msgr2=0x7f4964101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:45.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.238+0000 7f496c5b2640 1 --2- 172.21.15.78:0/638601954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4964101520 0x7f4964101900 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4960009880 tx=0x7f496002a5d0 comp rx=0 tx=0).stop 2024-08-18T23:35:45.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.238+0000 7f496c5b2640 1 -- 172.21.15.78:0/638601954 shutdown_connections 2024-08-18T23:35:45.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.238+0000 7f496c5b2640 1 --2- 172.21.15.78:0/638601954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4964101520 0x7f4964101900 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:45.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.238+0000 7f496c5b2640 1 -- 172.21.15.78:0/638601954 >> 172.21.15.78:0/638601954 conn(0x7f49640fcfc0 msgr2=0x7f49640ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:45.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.238+0000 7f496c5b2640 1 -- 172.21.15.78:0/638601954 shutdown_connections 2024-08-18T23:35:45.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.238+0000 7f496c5b2640 1 -- 172.21.15.78:0/638601954 wait complete. 2024-08-18T23:35:45.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.239+0000 7f496c5b2640 1 Processor -- start 2024-08-18T23:35:45.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.239+0000 7f496c5b2640 1 -- start start 2024-08-18T23:35:45.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.239+0000 7f496c5b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4964101520 0x7f4964196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:45.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.239+0000 7f496c5b2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f496410b970 con 0x7f4964101520 2024-08-18T23:35:45.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.239+0000 7f496a34e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4964101520 0x7f4964196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:45.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.239+0000 7f496a34e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4964101520 0x7f4964196ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37192/0 (socket says 172.21.15.78:37192) 2024-08-18T23:35:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.239+0000 7f496a34e640 1 -- 172.21.15.78:0/3115875224 learned_addr learned my addr 172.21.15.78:0/3115875224 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.239+0000 7f496a34e640 1 -- 172.21.15.78:0/3115875224 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49641973e0 con 0x7f4964101520 2024-08-18T23:35:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.239+0000 7f496a34e640 1 --2- 172.21.15.78:0/3115875224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4964101520 0x7f4964196ea0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4960009230 tx=0x7f4960003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.239+0000 7f495b7fe640 1 -- 172.21.15.78:0/3115875224 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f496003f070 con 0x7f4964101520 2024-08-18T23:35:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.240+0000 7f495b7fe640 1 -- 172.21.15.78:0/3115875224 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f496003bc80 con 0x7f4964101520 2024-08-18T23:35:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.240+0000 7f496c5b2640 1 -- 172.21.15.78:0/3115875224 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49641976d0 con 0x7f4964101520 2024-08-18T23:35:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.240+0000 7f495b7fe640 1 -- 172.21.15.78:0/3115875224 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4960035030 con 0x7f4964101520 2024-08-18T23:35:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.240+0000 7f496c5b2640 1 -- 172.21.15.78:0/3115875224 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f496419b770 con 0x7f4964101520 2024-08-18T23:35:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.240+0000 7f495b7fe640 1 -- 172.21.15.78:0/3115875224 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4960042070 con 0x7f4964101520 2024-08-18T23:35:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.241+0000 7f496c5b2640 1 -- 172.21.15.78:0/3115875224 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4964102a00 con 0x7f4964101520 2024-08-18T23:35:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.241+0000 7f495b7fe640 1 --2- 172.21.15.78:0/3115875224 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4950039060 0x7f495003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:45.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.241+0000 7f495b7fe640 1 -- 172.21.15.78:0/3115875224 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f496006cd10 con 0x7f4964101520 2024-08-18T23:35:45.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.241+0000 7f4969b4d640 1 --2- 172.21.15.78:0/3115875224 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4950039060 0x7f495003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:45.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.241+0000 7f4969b4d640 1 --2- 172.21.15.78:0/3115875224 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4950039060 0x7f495003b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f49540098c0 tx=0x7f4954006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:45.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.245+0000 7f495b7fe640 1 -- 172.21.15.78:0/3115875224 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4960031e20 con 0x7f4964101520 2024-08-18T23:35:45.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.432+0000 7f496c5b2640 1 -- 172.21.15.78:0/3115875224 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4964198200 con 0x7f4950039060 2024-08-18T23:35:45.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.433+0000 7f495b7fe640 1 -- 172.21.15.78:0/3115875224 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f4964198200 con 0x7f4950039060 2024-08-18T23:35:45.434 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:35:45.435 INFO:teuthology.orchestra.run.smithi078.stdout:[{"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}, {"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}] 2024-08-18T23:35:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.435+0000 7f496c5b2640 1 -- 172.21.15.78:0/3115875224 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4950039060 msgr2=0x7f495003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.435+0000 7f496c5b2640 1 --2- 172.21.15.78:0/3115875224 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4950039060 0x7f495003b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f49540098c0 tx=0x7f4954006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.435+0000 7f496c5b2640 1 -- 172.21.15.78:0/3115875224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4964101520 msgr2=0x7f4964196ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.435+0000 7f496c5b2640 1 --2- 172.21.15.78:0/3115875224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4964101520 0x7f4964196ea0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4960009230 tx=0x7f4960003d40 comp rx=0 tx=0).stop 2024-08-18T23:35:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.435+0000 7f496c5b2640 1 -- 172.21.15.78:0/3115875224 shutdown_connections 2024-08-18T23:35:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.435+0000 7f496c5b2640 1 --2- 172.21.15.78:0/3115875224 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4950039060 0x7f495003b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.436+0000 7f496c5b2640 1 --2- 172.21.15.78:0/3115875224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4964101520 0x7f4964196ea0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.436+0000 7f496c5b2640 1 -- 172.21.15.78:0/3115875224 >> 172.21.15.78:0/3115875224 conn(0x7f49640fcfc0 msgr2=0x7f49640fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.436+0000 7f496c5b2640 1 -- 172.21.15.78:0/3115875224 shutdown_connections 2024-08-18T23:35:45.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:45.436+0000 7f496c5b2640 1 -- 172.21.15.78:0/3115875224 wait complete. 2024-08-18T23:35:46.039 INFO:tasks.cephadm:Setting crush tunables to default 2024-08-18T23:35:46.039 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd crush tunables default 2024-08-18T23:35:46.282 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:35:46.572 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:46 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:46.572 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:46 smithi078 ceph-mon[18440]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:35:46.572 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:46 smithi078 ceph-mon[18440]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:35:46.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.651+0000 7f76268b7640 1 -- 172.21.15.78:0/3898800685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7620100420 msgr2=0x7f7620100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:46.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.651+0000 7f76268b7640 1 --2- 172.21.15.78:0/3898800685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7620100420 0x7f7620100800 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7610009880 tx=0x7f761002a5d0 comp rx=0 tx=0).stop 2024-08-18T23:35:46.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.651+0000 7f76268b7640 1 -- 172.21.15.78:0/3898800685 shutdown_connections 2024-08-18T23:35:46.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.651+0000 7f76268b7640 1 --2- 172.21.15.78:0/3898800685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7620100420 0x7f7620100800 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:46.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.651+0000 7f76268b7640 1 -- 172.21.15.78:0/3898800685 >> 172.21.15.78:0/3898800685 conn(0x7f76200fbee0 msgr2=0x7f76200fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:46.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.651+0000 7f76268b7640 1 -- 172.21.15.78:0/3898800685 shutdown_connections 2024-08-18T23:35:46.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.651+0000 7f76268b7640 1 -- 172.21.15.78:0/3898800685 wait complete. 2024-08-18T23:35:46.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.651+0000 7f76268b7640 1 Processor -- start 2024-08-18T23:35:46.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.651+0000 7f76268b7640 1 -- start start 2024-08-18T23:35:46.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.652+0000 7f76268b7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7620100420 0x7f7620195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:46.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.652+0000 7f76268b7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f762010a870 con 0x7f7620100420 2024-08-18T23:35:46.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.652+0000 7f761ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7620100420 0x7f7620195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:46.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.652+0000 7f761ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7620100420 0x7f7620195da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34218/0 (socket says 172.21.15.78:34218) 2024-08-18T23:35:46.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.652+0000 7f761ffff640 1 -- 172.21.15.78:0/3214407833 learned_addr learned my addr 172.21.15.78:0/3214407833 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:35:46.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.652+0000 7f761ffff640 1 -- 172.21.15.78:0/3214407833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76201962e0 con 0x7f7620100420 2024-08-18T23:35:46.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.652+0000 7f761ffff640 1 --2- 172.21.15.78:0/3214407833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7620100420 0x7f7620195da0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7610009230 tx=0x7f7610003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:46.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.653+0000 7f761d7fa640 1 -- 172.21.15.78:0/3214407833 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7610046070 con 0x7f7620100420 2024-08-18T23:35:46.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.653+0000 7f761d7fa640 1 -- 172.21.15.78:0/3214407833 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7610038b60 con 0x7f7620100420 2024-08-18T23:35:46.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.653+0000 7f76268b7640 1 -- 172.21.15.78:0/3214407833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7620196570 con 0x7f7620100420 2024-08-18T23:35:46.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.653+0000 7f761d7fa640 1 -- 172.21.15.78:0/3214407833 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7610035070 con 0x7f7620100420 2024-08-18T23:35:46.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.653+0000 7f76268b7640 1 -- 172.21.15.78:0/3214407833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f762019a670 con 0x7f7620100420 2024-08-18T23:35:46.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.653+0000 7f761d7fa640 1 -- 172.21.15.78:0/3214407833 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7610049070 con 0x7f7620100420 2024-08-18T23:35:46.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.654+0000 7f76268b7640 1 -- 172.21.15.78:0/3214407833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7620101900 con 0x7f7620100420 2024-08-18T23:35:46.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.654+0000 7f761d7fa640 1 --2- 172.21.15.78:0/3214407833 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7608039060 0x7f760803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:46.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.654+0000 7f761d7fa640 1 -- 172.21.15.78:0/3214407833 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f761003e070 con 0x7f7620100420 2024-08-18T23:35:46.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.654+0000 7f761f7fe640 1 --2- 172.21.15.78:0/3214407833 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7608039060 0x7f760803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:46.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.654+0000 7f761f7fe640 1 --2- 172.21.15.78:0/3214407833 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7608039060 0x7f760803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f76140098c0 tx=0x7f7614006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:46.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.658+0000 7f761d7fa640 1 -- 172.21.15.78:0/3214407833 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7610031dc0 con 0x7f7620100420 2024-08-18T23:35:46.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:46.828+0000 7f76268b7640 1 -- 172.21.15.78:0/3214407833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f7620197100 con 0x7f7620100420 2024-08-18T23:35:47.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:47.313+0000 7f761d7fa640 1 -- 172.21.15.78:0/3214407833 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f7610050090 con 0x7f7620100420 2024-08-18T23:35:47.315 INFO:teuthology.orchestra.run.smithi078.stderr:adjusted tunables profile to default 2024-08-18T23:35:47.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:47.316+0000 7f76268b7640 1 -- 172.21.15.78:0/3214407833 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7608039060 msgr2=0x7f760803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:47.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:47.316+0000 7f76268b7640 1 --2- 172.21.15.78:0/3214407833 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7608039060 0x7f760803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f76140098c0 tx=0x7f7614006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:47.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:47.316+0000 7f76268b7640 1 -- 172.21.15.78:0/3214407833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7620100420 msgr2=0x7f7620195da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:47.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:47.316+0000 7f76268b7640 1 --2- 172.21.15.78:0/3214407833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7620100420 0x7f7620195da0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7610009230 tx=0x7f7610003d40 comp rx=0 tx=0).stop 2024-08-18T23:35:47.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:47.316+0000 7f76268b7640 1 -- 172.21.15.78:0/3214407833 shutdown_connections 2024-08-18T23:35:47.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:47.316+0000 7f76268b7640 1 --2- 172.21.15.78:0/3214407833 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7608039060 0x7f760803b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:47.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:47.316+0000 7f76268b7640 1 --2- 172.21.15.78:0/3214407833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7620100420 0x7f7620195da0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:47.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:47.316+0000 7f76268b7640 1 -- 172.21.15.78:0/3214407833 >> 172.21.15.78:0/3214407833 conn(0x7f76200fbee0 msgr2=0x7f76200fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:47.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:47.316+0000 7f76268b7640 1 -- 172.21.15.78:0/3214407833 shutdown_connections 2024-08-18T23:35:47.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:35:47.316+0000 7f76268b7640 1 -- 172.21.15.78:0/3214407833 wait complete. 2024-08-18T23:35:47.616 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:47 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3214407833' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-08-18T23:35:47.898 INFO:tasks.cephadm:Adding mon.a on smithi078 2024-08-18T23:35:47.898 INFO:tasks.cephadm:Adding mon.b on smithi123 2024-08-18T23:35:47.898 INFO:tasks.cephadm:Adding mon.c on smithi161 2024-08-18T23:35:47.898 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch apply mon '3;smithi078:172.21.15.78=a;smithi123:172.21.15.123=b;smithi161:172.21.15.161=c' 2024-08-18T23:35:48.166 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /etc/ceph/ceph.conf 2024-08-18T23:35:48.215 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /etc/ceph/ceph.conf 2024-08-18T23:35:48.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:48 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3214407833' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-18T23:35:48.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:48 smithi078 ceph-mon[18440]: osdmap e4: 0 total, 0 up, 0 in 2024-08-18T23:35:48.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:48 smithi078 ceph-mon[18440]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:35:48.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:48.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:48.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:48.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:48.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:35:48.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:35:48.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:35:48.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:48 smithi078 ceph-mon[18440]: Updating smithi161:/etc/ceph/ceph.conf 2024-08-18T23:35:48.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.704+0000 7f874eb92640 1 -- 172.21.15.161:0/2578953905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8748101310 msgr2=0x7f8748101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:48.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.704+0000 7f874eb92640 1 --2- 172.21.15.161:0/2578953905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8748101310 0x7f8748101710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8738009880 tx=0x7f873802a5d0 comp rx=0 tx=0).stop 2024-08-18T23:35:48.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.704+0000 7f874eb92640 1 -- 172.21.15.161:0/2578953905 shutdown_connections 2024-08-18T23:35:48.731 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.704+0000 7f874eb92640 1 --2- 172.21.15.161:0/2578953905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8748101310 0x7f8748101710 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:48.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.704+0000 7f874eb92640 1 -- 172.21.15.161:0/2578953905 >> 172.21.15.161:0/2578953905 conn(0x7f87480fcaa0 msgr2=0x7f87480feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:48.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.704+0000 7f874eb92640 1 -- 172.21.15.161:0/2578953905 shutdown_connections 2024-08-18T23:35:48.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.704+0000 7f874eb92640 1 -- 172.21.15.161:0/2578953905 wait complete. 2024-08-18T23:35:48.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.704+0000 7f874eb92640 1 Processor -- start 2024-08-18T23:35:48.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.704+0000 7f874eb92640 1 -- start start 2024-08-18T23:35:48.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.705+0000 7f874eb92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8748101310 0x7f8748196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:48.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.705+0000 7f874eb92640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8748107420 con 0x7f8748101310 2024-08-18T23:35:48.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.705+0000 7f874c92e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8748101310 0x7f8748196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:48.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.705+0000 7f874c92e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8748101310 0x7f8748196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.161:37852/0 (socket says 172.21.15.161:37852) 2024-08-18T23:35:48.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.705+0000 7f874c92e640 1 -- 172.21.15.161:0/1291681526 learned_addr learned my addr 172.21.15.161:0/1291681526 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-18T23:35:48.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.705+0000 7f874c92e640 1 -- 172.21.15.161:0/1291681526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8748196fd0 con 0x7f8748101310 2024-08-18T23:35:48.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.705+0000 7f874c92e640 1 --2- 172.21.15.161:0/1291681526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8748101310 0x7f8748196a90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8738004f60 tx=0x7f87380048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:48.732 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.705+0000 7f8745ffb640 1 -- 172.21.15.161:0/1291681526 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f873803f070 con 0x7f8748101310 2024-08-18T23:35:48.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.705+0000 7f874eb92640 1 -- 172.21.15.161:0/1291681526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8748197260 con 0x7f8748101310 2024-08-18T23:35:48.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.705+0000 7f8745ffb640 1 -- 172.21.15.161:0/1291681526 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8738004d40 con 0x7f8748101310 2024-08-18T23:35:48.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.706+0000 7f8745ffb640 1 -- 172.21.15.161:0/1291681526 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8738035030 con 0x7f8748101310 2024-08-18T23:35:48.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.706+0000 7f874eb92640 1 -- 172.21.15.161:0/1291681526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8748199f50 con 0x7f8748101310 2024-08-18T23:35:48.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.706+0000 7f8745ffb640 1 -- 172.21.15.161:0/1291681526 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8738042070 con 0x7f8748101310 2024-08-18T23:35:48.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.707+0000 7f874eb92640 1 -- 172.21.15.161:0/1291681526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8748107ba0 con 0x7f8748101310 2024-08-18T23:35:48.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.707+0000 7f8745ffb640 1 --2- 172.21.15.161:0/1291681526 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8734039060 0x7f873403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:48.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.707+0000 7f8745ffb640 1 -- 172.21.15.161:0/1291681526 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f873806cdd0 con 0x7f8748101310 2024-08-18T23:35:48.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.707+0000 7f8747fff640 1 --2- 172.21.15.161:0/1291681526 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8734039060 0x7f873403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:48.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.707+0000 7f8747fff640 1 --2- 172.21.15.161:0/1291681526 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8734039060 0x7f873403b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f87300098c0 tx=0x7f8730006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:48.733 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.712+0000 7f8745ffb640 1 -- 172.21.15.161:0/1291681526 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f873804a260 con 0x7f8748101310 2024-08-18T23:35:48.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.901+0000 7f874eb92640 1 -- 172.21.15.161:0/1291681526 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi123:172.21.15.123=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}) -- 0x7f874802d650 con 0x7f8734039060 2024-08-18T23:35:48.908 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.906+0000 7f8745ffb640 1 -- 172.21.15.161:0/1291681526 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f874802d650 con 0x7f8734039060 2024-08-18T23:35:48.908 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mon update... 2024-08-18T23:35:48.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.909+0000 7f874eb92640 1 -- 172.21.15.161:0/1291681526 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8734039060 msgr2=0x7f873403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:48.910 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.909+0000 7f874eb92640 1 --2- 172.21.15.161:0/1291681526 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8734039060 0x7f873403b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f87300098c0 tx=0x7f8730006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:48.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.909+0000 7f874eb92640 1 -- 172.21.15.161:0/1291681526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8748101310 msgr2=0x7f8748196a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:48.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.909+0000 7f874eb92640 1 --2- 172.21.15.161:0/1291681526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8748101310 0x7f8748196a90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8738004f60 tx=0x7f87380048f0 comp rx=0 tx=0).stop 2024-08-18T23:35:48.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.909+0000 7f874eb92640 1 -- 172.21.15.161:0/1291681526 shutdown_connections 2024-08-18T23:35:48.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.909+0000 7f874eb92640 1 --2- 172.21.15.161:0/1291681526 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8734039060 0x7f873403b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:48.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.909+0000 7f874eb92640 1 --2- 172.21.15.161:0/1291681526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8748101310 0x7f8748196a90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:48.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.909+0000 7f874eb92640 1 -- 172.21.15.161:0/1291681526 >> 172.21.15.161:0/1291681526 conn(0x7f87480fcaa0 msgr2=0x7f87480fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:48.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.909+0000 7f874eb92640 1 -- 172.21.15.161:0/1291681526 shutdown_connections 2024-08-18T23:35:48.911 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:48.909+0000 7f874eb92640 1 -- 172.21.15.161:0/1291681526 wait complete. 2024-08-18T23:35:49.553 DEBUG:teuthology.orchestra.run.smithi123:mon.b> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mon.b.service 2024-08-18T23:35:49.556 DEBUG:teuthology.orchestra.run.smithi161:mon.c> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mon.c.service 2024-08-18T23:35:49.559 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-18T23:35:49.559 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-18T23:35:50.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:49 smithi078 ceph-mon[18440]: Updating smithi161:/var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-18T23:35:50.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:49 smithi078 ceph-mon[18440]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-08-18T23:35:50.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:49 smithi078 ceph-mon[18440]: Updating smithi161:/var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-18T23:35:50.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:49 smithi078 ceph-mon[18440]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi123:172.21.15.123=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:35:50.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:49 smithi078 ceph-mon[18440]: Saving service mon spec with placement smithi078:172.21.15.78=a;smithi123:172.21.15.123=b;smithi161:172.21.15.161=c;count:3 2024-08-18T23:35:50.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:50.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:50.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:50.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:50.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-18T23:35:50.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:35:50.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:49 smithi078 ceph-mon[18440]: Deploying daemon mon.c on smithi161 2024-08-18T23:35:50.600 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-18T23:35:51.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:50 smithi078 ceph-mon[18440]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:35:51.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.904+0000 7fab0e83b640 1 -- 172.21.15.161:0/2796375344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab08102460 msgr2=0x7fab08102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:51.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.904+0000 7fab0e83b640 1 --2- 172.21.15.161:0/2796375344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab08102460 0x7fab08102840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009810 tx=0x7faaf802a540 comp rx=0 tx=0).stop 2024-08-18T23:35:51.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.904+0000 7fab0e83b640 1 -- 172.21.15.161:0/2796375344 shutdown_connections 2024-08-18T23:35:51.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.904+0000 7fab0e83b640 1 --2- 172.21.15.161:0/2796375344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab08102460 0x7fab08102840 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:51.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.904+0000 7fab0e83b640 1 -- 172.21.15.161:0/2796375344 >> 172.21.15.161:0/2796375344 conn(0x7fab08077040 msgr2=0x7fab080fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:51.942 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.904+0000 7fab0e83b640 1 -- 172.21.15.161:0/2796375344 shutdown_connections 2024-08-18T23:35:51.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.904+0000 7fab0e83b640 1 -- 172.21.15.161:0/2796375344 wait complete. 2024-08-18T23:35:51.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.905+0000 7fab0e83b640 1 Processor -- start 2024-08-18T23:35:51.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.905+0000 7fab0e83b640 1 -- start start 2024-08-18T23:35:51.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.905+0000 7fab0e83b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab08102460 0x7fab081000c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:51.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.905+0000 7fab0e83b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fab0810b9b0 con 0x7fab08102460 2024-08-18T23:35:51.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.905+0000 7fab07fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab08102460 0x7fab081000c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:51.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.905+0000 7fab07fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab08102460 0x7fab081000c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.161:37878/0 (socket says 172.21.15.161:37878) 2024-08-18T23:35:51.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.905+0000 7fab07fff640 1 -- 172.21.15.161:0/1629823251 learned_addr learned my addr 172.21.15.161:0/1629823251 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-18T23:35:51.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.906+0000 7fab07fff640 1 -- 172.21.15.161:0/1629823251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab08101fb0 con 0x7fab08102460 2024-08-18T23:35:51.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.906+0000 7fab07fff640 1 --2- 172.21.15.161:0/1629823251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab08102460 0x7fab081000c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faaf80045e0 tx=0x7faaf8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:51.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.906+0000 7fab057fa640 1 -- 172.21.15.161:0/1629823251 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faaf8030e70 con 0x7fab08102460 2024-08-18T23:35:51.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.906+0000 7fab0e83b640 1 -- 172.21.15.161:0/1629823251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab08100600 con 0x7fab08102460 2024-08-18T23:35:51.943 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.906+0000 7fab057fa640 1 -- 172.21.15.161:0/1629823251 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf80318e0 con 0x7fab08102460 2024-08-18T23:35:51.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.906+0000 7fab0e83b640 1 -- 172.21.15.161:0/1629823251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab08100ac0 con 0x7fab08102460 2024-08-18T23:35:51.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.906+0000 7fab057fa640 1 -- 172.21.15.161:0/1629823251 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faaf8049060 con 0x7fab08102460 2024-08-18T23:35:51.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.908+0000 7fab0e83b640 1 -- 172.21.15.161:0/1629823251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab08069cc0 con 0x7fab08102460 2024-08-18T23:35:51.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.908+0000 7fab057fa640 1 -- 172.21.15.161:0/1629823251 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7faaf8039040 con 0x7fab08102460 2024-08-18T23:35:51.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.908+0000 7fab057fa640 1 --2- 172.21.15.161:0/1629823251 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7faaf0039060 0x7faaf003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:51.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.908+0000 7fab057fa640 1 -- 172.21.15.161:0/1629823251 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7faaf806d340 con 0x7fab08102460 2024-08-18T23:35:51.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.908+0000 7fab077fe640 1 --2- 172.21.15.161:0/1629823251 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7faaf0039060 0x7faaf003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:51.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.909+0000 7fab077fe640 1 --2- 172.21.15.161:0/1629823251 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7faaf0039060 0x7faaf003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faafc0098c0 tx=0x7faafc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:51.944 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:51.913+0000 7fab057fa640 1 -- 172.21.15.161:0/1629823251 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7faaf8040030 con 0x7fab08102460 2024-08-18T23:35:52.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:52.159+0000 7fab0e83b640 1 -- 172.21.15.161:0/1629823251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fab081015f0 con 0x7fab08102460 2024-08-18T23:35:52.160 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:52.159+0000 7fab057fa640 1 -- 172.21.15.161:0/1629823251 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7faaf80433b0 con 0x7fab08102460 2024-08-18T23:35:52.160 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:35:52.160 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":1,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","modified":"2024-08-18T23:34:23.952483Z","created":"2024-08-18T23:34:23.952483Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-08-18T23:35:52.161 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 1 2024-08-18T23:35:52.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:52.162+0000 7fab0e83b640 1 -- 172.21.15.161:0/1629823251 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7faaf0039060 msgr2=0x7faaf003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:52.163 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:52.162+0000 7fab0e83b640 1 --2- 172.21.15.161:0/1629823251 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7faaf0039060 0x7faaf003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faafc0098c0 tx=0x7faafc006db0 comp rx=0 tx=0).stop 2024-08-18T23:35:52.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:52.162+0000 7fab0e83b640 1 -- 172.21.15.161:0/1629823251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab08102460 msgr2=0x7fab081000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:52.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:52.162+0000 7fab0e83b640 1 --2- 172.21.15.161:0/1629823251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab08102460 0x7fab081000c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faaf80045e0 tx=0x7faaf8004610 comp rx=0 tx=0).stop 2024-08-18T23:35:52.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:52.162+0000 7fab0e83b640 1 -- 172.21.15.161:0/1629823251 shutdown_connections 2024-08-18T23:35:52.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:52.162+0000 7fab0e83b640 1 --2- 172.21.15.161:0/1629823251 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7faaf0039060 0x7faaf003b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:52.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:52.162+0000 7fab0e83b640 1 --2- 172.21.15.161:0/1629823251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab08102460 0x7fab081000c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:52.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:52.162+0000 7fab0e83b640 1 -- 172.21.15.161:0/1629823251 >> 172.21.15.161:0/1629823251 conn(0x7fab08077040 msgr2=0x7fab081051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:52.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:52.162+0000 7fab0e83b640 1 -- 172.21.15.161:0/1629823251 shutdown_connections 2024-08-18T23:35:52.164 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:52.163+0000 7fab0e83b640 1 -- 172.21.15.161:0/1629823251 wait complete. 2024-08-18T23:35:52.871 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:52 smithi161 systemd[1]: Starting Ceph mon.c for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:35:53.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 podman[22008]: 2024-08-18 23:35:52.998810289 +0000 UTC m=+0.017097212 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:35:53.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 podman[22008]: 2024-08-18 23:35:53.104913193 +0000 UTC m=+0.123200103 container create bbc41949c7923301bade0f04fa212c11248143ba5606428d916a10f3ac576f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0) 2024-08-18T23:35:53.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:53 smithi078 ceph-mon[18440]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:35:53.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:53 smithi078 ceph-mon[18440]: from='client.? 172.21.15.161:0/1629823251' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-18T23:35:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 podman[22008]: 2024-08-18 23:35:53.30717139 +0000 UTC m=+0.325458313 container init bbc41949c7923301bade0f04fa212c11248143ba5606428d916a10f3ac576f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-c, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-08-18T23:35:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 podman[22008]: 2024-08-18 23:35:53.310211464 +0000 UTC m=+0.328498373 container start bbc41949c7923301bade0f04fa212c11248143ba5606428d916a10f3ac576f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-c, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:35:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: set uid:gid to 167:167 (ceph:ceph) 2024-08-18T23:35:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc), process ceph-mon, pid 2 2024-08-18T23:35:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: pidfile_write: ignore empty --pid-file 2024-08-18T23:35:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: load: jerasure load: lrc 2024-08-18T23:35:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: RocksDB version: 7.9.2 2024-08-18T23:35:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Git sha 0 2024-08-18T23:35:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Compile date 2024-08-18 15:19:56 2024-08-18T23:35:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: DB SUMMARY 2024-08-18T23:35:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: DB Session ID: AIAL7S7LWMTU2NDW1SSR 2024-08-18T23:35:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: CURRENT file: CURRENT 2024-08-18T23:35:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: IDENTITY file: IDENTITY 2024-08-18T23:35:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-08-18T23:35:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-08-18T23:35:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-08-18T23:35:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.error_if_exists: 0 2024-08-18T23:35:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.create_if_missing: 0 2024-08-18T23:35:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.paranoid_checks: 1 2024-08-18T23:35:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-18T23:35:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-18T23:35:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-18T23:35:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.env: 0x5587e380b7a0 2024-08-18T23:35:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.fs: PosixFileSystem 2024-08-18T23:35:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.info_log: 0x5587e49bd500 2024-08-18T23:35:53.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_file_opening_threads: 16 2024-08-18T23:35:53.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.statistics: (nil) 2024-08-18T23:35:53.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.use_fsync: 0 2024-08-18T23:35:53.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_log_file_size: 0 2024-08-18T23:35:53.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-18T23:35:53.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-18T23:35:53.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.keep_log_file_num: 1000 2024-08-18T23:35:53.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.recycle_log_file_num: 0 2024-08-18T23:35:53.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.allow_fallocate: 1 2024-08-18T23:35:53.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.allow_mmap_reads: 0 2024-08-18T23:35:53.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.allow_mmap_writes: 0 2024-08-18T23:35:53.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.use_direct_reads: 0 2024-08-18T23:35:53.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-18T23:35:53.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.create_missing_column_families: 0 2024-08-18T23:35:53.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.db_log_dir: 2024-08-18T23:35:53.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.wal_dir: 2024-08-18T23:35:53.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-18T23:35:53.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-18T23:35:53.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-18T23:35:53.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-18T23:35:53.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-18T23:35:53.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-18T23:35:53.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.advise_random_on_open: 1 2024-08-18T23:35:53.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.db_write_buffer_size: 0 2024-08-18T23:35:53.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.write_buffer_manager: 0x5587e49ccdc0 2024-08-18T23:35:53.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-18T23:35:53.681 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-18T23:35:53.682 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-18T23:35:53.682 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.rate_limiter: (nil) 2024-08-18T23:35:53.682 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-18T23:35:53.682 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.wal_recovery_mode: 2 2024-08-18T23:35:53.682 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.enable_thread_tracking: 0 2024-08-18T23:35:53.682 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.enable_pipelined_write: 0 2024-08-18T23:35:53.682 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.unordered_write: 0 2024-08-18T23:35:53.682 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-18T23:35:53.682 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-18T23:35:53.682 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-18T23:35:53.682 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-18T23:35:53.682 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.row_cache: None 2024-08-18T23:35:53.682 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.wal_filter: None 2024-08-18T23:35:53.684 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-18T23:35:53.684 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.allow_ingest_behind: 0 2024-08-18T23:35:53.684 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.two_write_queues: 0 2024-08-18T23:35:53.684 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.manual_wal_flush: 0 2024-08-18T23:35:53.684 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.wal_compression: 0 2024-08-18T23:35:53.684 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.atomic_flush: 0 2024-08-18T23:35:53.684 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-18T23:35:53.684 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-18T23:35:53.685 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-18T23:35:53.685 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.log_readahead_size: 0 2024-08-18T23:35:53.685 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-18T23:35:53.685 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.best_efforts_recovery: 0 2024-08-18T23:35:53.685 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-18T23:35:53.685 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-18T23:35:53.685 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.allow_data_in_errors: 0 2024-08-18T23:35:53.685 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.db_host_id: __hostname__ 2024-08-18T23:35:53.685 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-18T23:35:53.685 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_background_jobs: 2 2024-08-18T23:35:53.685 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_background_compactions: -1 2024-08-18T23:35:53.685 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_subcompactions: 1 2024-08-18T23:35:53.685 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-18T23:35:53.685 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-18T23:35:53.686 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-18T23:35:53.686 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_total_wal_size: 0 2024-08-18T23:35:53.686 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-18T23:35:53.686 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-18T23:35:53.686 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-18T23:35:53.686 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-18T23:35:53.686 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_open_files: -1 2024-08-18T23:35:53.686 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.bytes_per_sync: 0 2024-08-18T23:35:53.686 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-18T23:35:53.686 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-18T23:35:53.686 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compaction_readahead_size: 0 2024-08-18T23:35:53.686 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_background_flushes: -1 2024-08-18T23:35:53.686 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Compression algorithms supported: 2024-08-18T23:35:53.687 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: kZSTD supported: 0 2024-08-18T23:35:53.687 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: kXpressCompression supported: 0 2024-08-18T23:35:53.687 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: kBZip2Compression supported: 0 2024-08-18T23:35:53.687 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-18T23:35:53.687 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: kLZ4Compression supported: 1 2024-08-18T23:35:53.687 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: kZlibCompression supported: 1 2024-08-18T23:35:53.687 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: kLZ4HCCompression supported: 1 2024-08-18T23:35:53.687 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: kSnappyCompression supported: 1 2024-08-18T23:35:53.687 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-18T23:35:53.687 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-18T23:35:53.687 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-08-18T23:35:53.687 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-18T23:35:53.687 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-18T23:35:53.687 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.merge_operator: 2024-08-18T23:35:53.688 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compaction_filter: None 2024-08-18T23:35:53.688 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compaction_filter_factory: None 2024-08-18T23:35:53.688 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.sst_partitioner_factory: None 2024-08-18T23:35:53.688 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-18T23:35:53.688 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-18T23:35:53.688 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5587e49bd120) 2024-08-18T23:35:53.688 INFO:journalctl@ceph.mon.c.smithi161.stdout: cache_index_and_filter_blocks: 1 2024-08-18T23:35:53.688 INFO:journalctl@ceph.mon.c.smithi161.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-18T23:35:53.688 INFO:journalctl@ceph.mon.c.smithi161.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-18T23:35:53.688 INFO:journalctl@ceph.mon.c.smithi161.stdout: pin_top_level_index_and_filter: 1 2024-08-18T23:35:53.688 INFO:journalctl@ceph.mon.c.smithi161.stdout: index_type: 0 2024-08-18T23:35:53.688 INFO:journalctl@ceph.mon.c.smithi161.stdout: data_block_index_type: 0 2024-08-18T23:35:53.688 INFO:journalctl@ceph.mon.c.smithi161.stdout: index_shortening: 1 2024-08-18T23:35:53.689 INFO:journalctl@ceph.mon.c.smithi161.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-18T23:35:53.689 INFO:journalctl@ceph.mon.c.smithi161.stdout: checksum: 4 2024-08-18T23:35:53.689 INFO:journalctl@ceph.mon.c.smithi161.stdout: no_block_cache: 0 2024-08-18T23:35:53.689 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_cache: 0x5587e49b1090 2024-08-18T23:35:53.689 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_cache_name: BinnedLRUCache 2024-08-18T23:35:53.689 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_cache_options: 2024-08-18T23:35:53.689 INFO:journalctl@ceph.mon.c.smithi161.stdout: capacity : 536870912 2024-08-18T23:35:53.689 INFO:journalctl@ceph.mon.c.smithi161.stdout: num_shard_bits : 4 2024-08-18T23:35:53.689 INFO:journalctl@ceph.mon.c.smithi161.stdout: strict_capacity_limit : 0 2024-08-18T23:35:53.689 INFO:journalctl@ceph.mon.c.smithi161.stdout: high_pri_pool_ratio: 0.000 2024-08-18T23:35:53.689 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_cache_compressed: (nil) 2024-08-18T23:35:53.689 INFO:journalctl@ceph.mon.c.smithi161.stdout: persistent_cache: (nil) 2024-08-18T23:35:53.689 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_size: 4096 2024-08-18T23:35:53.690 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_size_deviation: 10 2024-08-18T23:35:53.690 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_restart_interval: 16 2024-08-18T23:35:53.690 INFO:journalctl@ceph.mon.c.smithi161.stdout: index_block_restart_interval: 1 2024-08-18T23:35:53.690 INFO:journalctl@ceph.mon.c.smithi161.stdout: metadata_block_size: 4096 2024-08-18T23:35:53.690 INFO:journalctl@ceph.mon.c.smithi161.stdout: partition_filters: 0 2024-08-18T23:35:53.690 INFO:journalctl@ceph.mon.c.smithi161.stdout: use_delta_encoding: 1 2024-08-18T23:35:53.690 INFO:journalctl@ceph.mon.c.smithi161.stdout: filter_policy: bloomfilter 2024-08-18T23:35:53.690 INFO:journalctl@ceph.mon.c.smithi161.stdout: whole_key_filtering: 1 2024-08-18T23:35:53.690 INFO:journalctl@ceph.mon.c.smithi161.stdout: verify_compression: 0 2024-08-18T23:35:53.690 INFO:journalctl@ceph.mon.c.smithi161.stdout: read_amp_bytes_per_bit: 0 2024-08-18T23:35:53.690 INFO:journalctl@ceph.mon.c.smithi161.stdout: format_version: 5 2024-08-18T23:35:53.690 INFO:journalctl@ceph.mon.c.smithi161.stdout: enable_index_compression: 1 2024-08-18T23:35:53.690 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_align: 0 2024-08-18T23:35:53.690 INFO:journalctl@ceph.mon.c.smithi161.stdout: max_auto_readahead_size: 262144 2024-08-18T23:35:53.691 INFO:journalctl@ceph.mon.c.smithi161.stdout: prepopulate_block_cache: 0 2024-08-18T23:35:53.691 INFO:journalctl@ceph.mon.c.smithi161.stdout: initial_auto_readahead_size: 8192 2024-08-18T23:35:53.691 INFO:journalctl@ceph.mon.c.smithi161.stdout: num_file_reads_for_auto_readahead: 2 2024-08-18T23:35:53.691 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.write_buffer_size: 33554432 2024-08-18T23:35:53.691 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_write_buffer_number: 2 2024-08-18T23:35:53.691 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compression: NoCompression 2024-08-18T23:35:53.691 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.bottommost_compression: Disabled 2024-08-18T23:35:53.691 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.prefix_extractor: nullptr 2024-08-18T23:35:53.691 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-18T23:35:53.691 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.num_levels: 7 2024-08-18T23:35:53.691 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-18T23:35:53.691 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-18T23:35:53.691 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-18T23:35:53.692 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-18T23:35:53.692 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-18T23:35:53.692 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-18T23:35:53.692 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-18T23:35:53.692 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-18T23:35:53.692 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-18T23:35:53.692 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-18T23:35:53.692 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-18T23:35:53.692 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-18T23:35:53.692 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-18T23:35:53.692 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compression_opts.level: 32767 2024-08-18T23:35:53.692 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compression_opts.strategy: 0 2024-08-18T23:35:53.692 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-18T23:35:53.693 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-18T23:35:53.693 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-18T23:35:53.693 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-18T23:35:53.693 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compression_opts.enabled: false 2024-08-18T23:35:53.693 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-18T23:35:53.693 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-18T23:35:53.693 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-18T23:35:53.693 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-18T23:35:53.693 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.target_file_size_base: 67108864 2024-08-18T23:35:53.693 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-18T23:35:53.693 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-18T23:35:53.693 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-18T23:35:53.693 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-18T23:35:53.694 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-18T23:35:53.694 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-18T23:35:53.694 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-18T23:35:53.694 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-18T23:35:53.694 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-18T23:35:53.694 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-18T23:35:53.694 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-18T23:35:53.694 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-18T23:35:53.694 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-18T23:35:53.694 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-18T23:35:53.694 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.arena_block_size: 1048576 2024-08-18T23:35:53.694 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-18T23:35:53.694 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-18T23:35:53.694 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.disable_auto_compactions: 0 2024-08-18T23:35:53.695 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-18T23:35:53.695 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-18T23:35:53.695 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-18T23:35:53.695 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-18T23:35:53.695 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-18T23:35:53.695 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-18T23:35:53.695 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-18T23:35:53.695 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-18T23:35:53.695 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-18T23:35:53.695 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-18T23:35:53.695 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-18T23:35:53.695 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.inplace_update_support: 0 2024-08-18T23:35:53.695 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-18T23:35:53.695 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-18T23:35:53.696 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-18T23:35:53.696 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-18T23:35:53.696 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.bloom_locality: 0 2024-08-18T23:35:53.696 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.max_successive_merges: 0 2024-08-18T23:35:53.696 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-18T23:35:53.696 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.paranoid_file_checks: 0 2024-08-18T23:35:53.696 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.force_consistency_checks: 1 2024-08-18T23:35:53.696 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.report_bg_io_stats: 0 2024-08-18T23:35:53.696 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.ttl: 2592000 2024-08-18T23:35:53.696 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-18T23:35:53.696 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-18T23:35:53.696 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-18T23:35:53.696 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.enable_blob_files: false 2024-08-18T23:35:53.697 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.min_blob_size: 0 2024-08-18T23:35:53.697 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.blob_file_size: 268435456 2024-08-18T23:35:53.697 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-18T23:35:53.697 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-18T23:35:53.697 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-18T23:35:53.697 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-18T23:35:53.697 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-18T23:35:53.697 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.blob_file_starting_level: 0 2024-08-18T23:35:53.697 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-18T23:35:53.697 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-08-18T23:35:53.697 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-08-18T23:35:53.697 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3f8ccea8-5aec-45e8-b4e9-46325557d68e 2024-08-18T23:35:53.697 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724024153337285, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-08-18T23:35:53.697 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-08-18T23:35:53.698 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724024153337852, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1724024153, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3f8ccea8-5aec-45e8-b4e9-46325557d68e", "db_session_id": "AIAL7S7LWMTU2NDW1SSR", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-08-18T23:35:53.698 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724024153337977, "job": 1, "event": "recovery_finished"} 2024-08-18T23:35:53.698 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-08-18T23:35:53.698 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-18T23:35:53.698 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5587e49dee00 2024-08-18T23:35:53.698 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: DB pointer 0x5587e4ad8000 2024-08-18T23:35:53.698 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-18T23:35:53.698 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-18T23:35:53.698 INFO:journalctl@ceph.mon.c.smithi161.stdout: ** DB Stats ** 2024-08-18T23:35:53.698 INFO:journalctl@ceph.mon.c.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-18T23:35:53.698 INFO:journalctl@ceph.mon.c.smithi161.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-18T23:35:53.698 INFO:journalctl@ceph.mon.c.smithi161.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-18T23:35:53.698 INFO:journalctl@ceph.mon.c.smithi161.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-18T23:35:53.699 INFO:journalctl@ceph.mon.c.smithi161.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-18T23:35:53.699 INFO:journalctl@ceph.mon.c.smithi161.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-18T23:35:53.699 INFO:journalctl@ceph.mon.c.smithi161.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-18T23:35:53.699 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2024-08-18T23:35:53.699 INFO:journalctl@ceph.mon.c.smithi161.stdout: ** Compaction Stats [default] ** 2024-08-18T23:35:53.699 INFO:journalctl@ceph.mon.c.smithi161.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-18T23:35:53.699 INFO:journalctl@ceph.mon.c.smithi161.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-18T23:35:53.699 INFO:journalctl@ceph.mon.c.smithi161.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-18T23:35:53.699 INFO:journalctl@ceph.mon.c.smithi161.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-18T23:35:53.699 INFO:journalctl@ceph.mon.c.smithi161.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-18T23:35:53.699 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2024-08-18T23:35:53.699 INFO:journalctl@ceph.mon.c.smithi161.stdout: ** Compaction Stats [default] ** 2024-08-18T23:35:53.699 INFO:journalctl@ceph.mon.c.smithi161.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-18T23:35:53.700 INFO:journalctl@ceph.mon.c.smithi161.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-18T23:35:53.700 INFO:journalctl@ceph.mon.c.smithi161.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-18T23:35:53.700 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2024-08-18T23:35:53.700 INFO:journalctl@ceph.mon.c.smithi161.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-18T23:35:53.700 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2024-08-18T23:35:53.700 INFO:journalctl@ceph.mon.c.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-18T23:35:53.700 INFO:journalctl@ceph.mon.c.smithi161.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-18T23:35:53.700 INFO:journalctl@ceph.mon.c.smithi161.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-18T23:35:53.700 INFO:journalctl@ceph.mon.c.smithi161.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-18T23:35:53.700 INFO:journalctl@ceph.mon.c.smithi161.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-18T23:35:53.700 INFO:journalctl@ceph.mon.c.smithi161.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-18T23:35:53.700 INFO:journalctl@ceph.mon.c.smithi161.stdout: Cumulative compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-18T23:35:53.700 INFO:journalctl@ceph.mon.c.smithi161.stdout: Interval compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-18T23:35:53.700 INFO:journalctl@ceph.mon.c.smithi161.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-18T23:35:53.701 INFO:journalctl@ceph.mon.c.smithi161.stdout: Block cache BinnedLRUCache@0x5587e49b1090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-08-18T23:35:53.701 INFO:journalctl@ceph.mon.c.smithi161.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-08-18T23:35:53.701 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2024-08-18T23:35:53.701 INFO:journalctl@ceph.mon.c.smithi161.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-18T23:35:53.701 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-08-18T23:35:53.701 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: using public_addr v2:172.21.15.161:0/0 -> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] 2024-08-18T23:35:53.701 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: starting mon.c rank -1 at public addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] at bind addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:35:53.701 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: mon.c@-1(???) e0 preinit fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:35:53.701 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: mon.c@-1(synchronizing).mds e1 new map 2024-08-18T23:35:53.701 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: mon.c@-1(synchronizing).mds e1 print_map 2024-08-18T23:35:53.701 INFO:journalctl@ceph.mon.c.smithi161.stdout: e1 2024-08-18T23:35:53.701 INFO:journalctl@ceph.mon.c.smithi161.stdout: btime 2024-08-18T23:34:33:867054+0000 2024-08-18T23:35:53.701 INFO:journalctl@ceph.mon.c.smithi161.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-18T23:35:53.702 INFO:journalctl@ceph.mon.c.smithi161.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-18T23:35:53.702 INFO:journalctl@ceph.mon.c.smithi161.stdout: legacy client fscid: -1 2024-08-18T23:35:53.702 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2024-08-18T23:35:53.702 INFO:journalctl@ceph.mon.c.smithi161.stdout: No filesystems configured 2024-08-18T23:35:53.702 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-08-18T23:35:53.702 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-08-18T23:35:53.702 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-08-18T23:35:53.702 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-08-18T23:35:53.702 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-08-18T23:35:53.702 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-08-18T23:35:53.702 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-18T23:35:53.702 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-18T23:35:53.702 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-18T23:35:53.703 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-18T23:35:53.703 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: Updating smithi123:/var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-18T23:35:53.703 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-08-18T23:35:53.703 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:53.703 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:53.703 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:53.703 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:53.703 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:35:53.703 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: Updating smithi123:/var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-18T23:35:53.703 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: Added host smithi161 2024-08-18T23:35:53.703 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:35:53.703 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:53.703 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:53.703 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:53.704 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:35:53.704 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:35:53.704 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3214407833' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-08-18T23:35:53.704 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3214407833' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-18T23:35:53.704 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: osdmap e4: 0 total, 0 up, 0 in 2024-08-18T23:35:53.704 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:35:53.704 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:53.704 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:53.704 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:53.704 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:53.704 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:35:53.704 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:35:53.704 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:35:53.704 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: Updating smithi161:/etc/ceph/ceph.conf 2024-08-18T23:35:53.705 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: Updating smithi161:/var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-18T23:35:53.705 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-08-18T23:35:53.705 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: Updating smithi161:/var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-18T23:35:53.705 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi123:172.21.15.123=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:35:53.705 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: Saving service mon spec with placement smithi078:172.21.15.78=a;smithi123:172.21.15.123=b;smithi161:172.21.15.161=c;count:3 2024-08-18T23:35:53.705 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:53.705 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:53.705 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:53.705 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:53.705 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-18T23:35:53.705 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:35:53.705 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: Deploying daemon mon.c on smithi161 2024-08-18T23:35:53.705 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:35:53.706 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:35:53.706 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: from='client.? 172.21.15.161:0/1629823251' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-18T23:35:53.706 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 ceph-mon[22027]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-08-18T23:35:53.706 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 bash[22008]: bbc41949c7923301bade0f04fa212c11248143ba5606428d916a10f3ac576f32 2024-08-18T23:35:53.706 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:53 smithi161 systemd[1]: Started Ceph mon.c for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:35:53.900 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-18T23:35:53.901 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-18T23:35:54.149 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.c/config 2024-08-18T23:35:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-18T23:35:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:35:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: mon.a calling monitor election 2024-08-18T23:35:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:35:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:35:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:35:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: mon.c calling monitor election 2024-08-18T23:35:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:35:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:35:58.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:35:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:35:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:35:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-18T23:35:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: monmap epoch 2 2024-08-18T23:35:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:35:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: last_changed 2024-08-18T23:35:53.364656+0000 2024-08-18T23:35:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: created 2024-08-18T23:34:23.952483+0000 2024-08-18T23:35:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: min_mon_release 19 (squid) 2024-08-18T23:35:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: election_strategy: 1 2024-08-18T23:35:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-08-18T23:35:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-08-18T23:35:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: fsmap 2024-08-18T23:35:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: osdmap e4: 0 total, 0 up, 0 in 2024-08-18T23:35:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: mgrmap e13: x(active, since 36s) 2024-08-18T23:35:58.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: overall HEALTH_OK 2024-08-18T23:35:58.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:58.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:58.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:58.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-18T23:35:58.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:58 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:35:58.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-18T23:35:58.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:35:58.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: mon.a calling monitor election 2024-08-18T23:35:58.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:35:58.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:35:58.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:35:58.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: mon.c calling monitor election 2024-08-18T23:35:58.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:35:58.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:35:58.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:35:58.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:35:58.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:35:58.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-18T23:35:58.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: monmap epoch 2 2024-08-18T23:35:58.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:35:58.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: last_changed 2024-08-18T23:35:53.364656+0000 2024-08-18T23:35:58.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: created 2024-08-18T23:34:23.952483+0000 2024-08-18T23:35:58.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: min_mon_release 19 (squid) 2024-08-18T23:35:58.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: election_strategy: 1 2024-08-18T23:35:58.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-08-18T23:35:58.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-08-18T23:35:58.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: fsmap 2024-08-18T23:35:58.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: osdmap e4: 0 total, 0 up, 0 in 2024-08-18T23:35:58.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: mgrmap e13: x(active, since 36s) 2024-08-18T23:35:58.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: overall HEALTH_OK 2024-08-18T23:35:58.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:58.929 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:58.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:35:58.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-18T23:35:58.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:58 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:35:58.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.961+0000 7fe9e3214640 1 -- 172.21.15.161:0/861393410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc003530 msgr2=0x7fe9cc0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:58.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.961+0000 7fe9e3214640 1 --2- 172.21.15.161:0/861393410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc003530 0x7fe9cc0059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc101770 tx=0x7fe9d002c490 comp rx=0 tx=0).stop 2024-08-18T23:35:58.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.961+0000 7fe9e3214640 1 -- 172.21.15.161:0/861393410 shutdown_connections 2024-08-18T23:35:58.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.961+0000 7fe9e3214640 1 --2- 172.21.15.161:0/861393410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc003530 0x7fe9cc0059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:58.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.961+0000 7fe9e3214640 1 -- 172.21.15.161:0/861393410 >> 172.21.15.161:0/861393410 conn(0x7fe9dc0fc000 msgr2=0x7fe9dc0fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:58.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.961+0000 7fe9e3214640 1 -- 172.21.15.161:0/861393410 shutdown_connections 2024-08-18T23:35:58.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.961+0000 7fe9e3214640 1 -- 172.21.15.161:0/861393410 wait complete. 2024-08-18T23:35:58.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.962+0000 7fe9e3214640 1 Processor -- start 2024-08-18T23:35:58.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.962+0000 7fe9e3214640 1 -- start start 2024-08-18T23:35:58.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.962+0000 7fe9e3214640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc003530 0x7fe9dc0fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:58.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.962+0000 7fe9e3214640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9dc0fe5f0 0x7fe9dc0fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:58.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.963+0000 7fe9e3214640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc1083f0 con 0x7fe9cc003530 2024-08-18T23:35:58.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.963+0000 7fe9e3214640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc108270 con 0x7fe9dc0fe5f0 2024-08-18T23:35:58.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.963+0000 7fe9e0fb0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc003530 0x7fe9dc0fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:58.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.963+0000 7fe9dbfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9dc0fe5f0 0x7fe9dc0fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:58.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.963+0000 7fe9e0fb0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc003530 0x7fe9dc0fc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.161:48450/0 (socket says 172.21.15.161:48450) 2024-08-18T23:35:58.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.963+0000 7fe9e0fb0640 1 -- 172.21.15.161:0/1263832166 learned_addr learned my addr 172.21.15.161:0/1263832166 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-18T23:35:58.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.963+0000 7fe9dbfff640 1 -- 172.21.15.161:0/1263832166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9dc0fe5f0 msgr2=0x7fe9dc0fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-08-18T23:35:58.965 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.963+0000 7fe9dbfff640 1 -- 172.21.15.161:0/1263832166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9dc0fe5f0 msgr2=0x7fe9dc0fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-18T23:35:58.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.963+0000 7fe9dbfff640 1 --2- 172.21.15.161:0/1263832166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9dc0fe5f0 0x7fe9dc0fcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-18T23:35:58.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.963+0000 7fe9dbfff640 1 --2- 172.21.15.161:0/1263832166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9dc0fe5f0 0x7fe9dc0fcef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-18T23:35:58.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.963+0000 7fe9e0fb0640 1 -- 172.21.15.161:0/1263832166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9dc0fe5f0 msgr2=0x7fe9dc0fcef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-18T23:35:58.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.963+0000 7fe9e0fb0640 1 --2- 172.21.15.161:0/1263832166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9dc0fe5f0 0x7fe9dc0fcef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:58.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.963+0000 7fe9e0fb0640 1 -- 172.21.15.161:0/1263832166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9dc0fd430 con 0x7fe9cc003530 2024-08-18T23:35:58.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.963+0000 7fe9e0fb0640 1 --2- 172.21.15.161:0/1263832166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc003530 0x7fe9dc0fc9b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe9d002ca80 tx=0x7fe9d0032b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:58.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.963+0000 7fe9d9ffb640 1 -- 172.21.15.161:0/1263832166 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe9d0038040 con 0x7fe9cc003530 2024-08-18T23:35:58.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.964+0000 7fe9d9ffb640 1 -- 172.21.15.161:0/1263832166 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9d003c410 con 0x7fe9cc003530 2024-08-18T23:35:58.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.964+0000 7fe9e3214640 1 -- 172.21.15.161:0/1263832166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9dc0fd6c0 con 0x7fe9cc003530 2024-08-18T23:35:58.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.964+0000 7fe9d9ffb640 1 -- 172.21.15.161:0/1263832166 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe9d003b370 con 0x7fe9cc003530 2024-08-18T23:35:58.966 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.964+0000 7fe9e3214640 1 -- 172.21.15.161:0/1263832166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9dc1a7ba0 con 0x7fe9cc003530 2024-08-18T23:35:58.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.965+0000 7fe9d9ffb640 1 -- 172.21.15.161:0/1263832166 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe9d0043070 con 0x7fe9cc003530 2024-08-18T23:35:58.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.965+0000 7fe9e3214640 1 -- 172.21.15.161:0/1263832166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9dc101a20 con 0x7fe9cc003530 2024-08-18T23:35:58.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.965+0000 7fe9d9ffb640 1 --2- 172.21.15.161:0/1263832166 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe9c0038e00 0x7fe9c003b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:35:58.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.965+0000 7fe9d9ffb640 1 -- 172.21.15.161:0/1263832166 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe9d006e460 con 0x7fe9cc003530 2024-08-18T23:35:58.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.965+0000 7fe9dbfff640 1 --2- 172.21.15.161:0/1263832166 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe9c0038e00 0x7fe9c003b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:35:58.967 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.966+0000 7fe9dbfff640 1 --2- 172.21.15.161:0/1263832166 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe9c0038e00 0x7fe9c003b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8006fd0 tx=0x7fe9c8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:35:58.975 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:58.974+0000 7fe9d9ffb640 1 -- 172.21.15.161:0/1263832166 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe9d0044570 con 0x7fe9cc003530 2024-08-18T23:35:59.207 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:59.206+0000 7fe9e3214640 1 -- 172.21.15.161:0/1263832166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe9dc190650 con 0x7fe9cc003530 2024-08-18T23:35:59.208 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:59.207+0000 7fe9d9ffb640 1 -- 172.21.15.161:0/1263832166 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fe9d00447d0 con 0x7fe9cc003530 2024-08-18T23:35:59.208 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:35:59.208 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":2,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","modified":"2024-08-18T23:35:53.364656Z","created":"2024-08-18T23:34:23.952483Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-18T23:35:59.208 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 2 2024-08-18T23:35:59.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:59.209+0000 7fe9e3214640 1 -- 172.21.15.161:0/1263832166 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe9c0038e00 msgr2=0x7fe9c003b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:59.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:59.209+0000 7fe9e3214640 1 --2- 172.21.15.161:0/1263832166 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe9c0038e00 0x7fe9c003b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8006fd0 tx=0x7fe9c8008040 comp rx=0 tx=0).stop 2024-08-18T23:35:59.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:59.209+0000 7fe9e3214640 1 -- 172.21.15.161:0/1263832166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc003530 msgr2=0x7fe9dc0fc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:35:59.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:59.209+0000 7fe9e3214640 1 --2- 172.21.15.161:0/1263832166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc003530 0x7fe9dc0fc9b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe9d002ca80 tx=0x7fe9d0032b50 comp rx=0 tx=0).stop 2024-08-18T23:35:59.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:59.209+0000 7fe9e3214640 1 -- 172.21.15.161:0/1263832166 shutdown_connections 2024-08-18T23:35:59.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:59.209+0000 7fe9e3214640 1 --2- 172.21.15.161:0/1263832166 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe9c0038e00 0x7fe9c003b2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:59.210 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:59.209+0000 7fe9e3214640 1 --2- 172.21.15.161:0/1263832166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9dc0fe5f0 0x7fe9dc0fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:59.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:59.209+0000 7fe9e3214640 1 --2- 172.21.15.161:0/1263832166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc003530 0x7fe9dc0fc9b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:35:59.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:59.209+0000 7fe9e3214640 1 -- 172.21.15.161:0/1263832166 >> 172.21.15.161:0/1263832166 conn(0x7fe9dc0fc000 msgr2=0x7fe9dc06a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:35:59.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:59.209+0000 7fe9e3214640 1 -- 172.21.15.161:0/1263832166 shutdown_connections 2024-08-18T23:35:59.211 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:35:59.209+0000 7fe9e3214640 1 -- 172.21.15.161:0/1263832166 wait complete. 2024-08-18T23:35:59.530 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:59 smithi161 ceph-mon[22027]: Deploying daemon mon.b on smithi123 2024-08-18T23:35:59.530 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:59 smithi161 ceph-mon[22027]: from='client.? 172.21.15.161:0/1263832166' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-18T23:35:59.530 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:35:59 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:35:59.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:59 smithi078 ceph-mon[18440]: Deploying daemon mon.b on smithi123 2024-08-18T23:35:59.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:59 smithi078 ceph-mon[18440]: from='client.? 172.21.15.161:0/1263832166' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-18T23:35:59.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:35:59 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:36:00.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:00 smithi078 ceph-mon[18440]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:00.719 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:36:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:36:00.364+0000 7f3aa145c640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-08-18T23:36:00.829 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-18T23:36:00.830 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-18T23:36:00.858 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:00 smithi161 ceph-mon[22027]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:01.086 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.c/config 2024-08-18T23:36:01.524 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.521+0000 7f4fd9ee6640 1 -- 172.21.15.161:0/2541875156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd4100340 msgr2=0x7f4fd4100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:01.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.521+0000 7f4fd9ee6640 1 --2- 172.21.15.161:0/2541875156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd4100340 0x7f4fd4100720 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009880 tx=0x7f4fc402a670 comp rx=0 tx=0).stop 2024-08-18T23:36:01.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.521+0000 7f4fd9ee6640 1 -- 172.21.15.161:0/2541875156 shutdown_connections 2024-08-18T23:36:01.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.521+0000 7f4fd9ee6640 1 --2- 172.21.15.161:0/2541875156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd4100340 0x7f4fd4100720 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:01.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.521+0000 7f4fd9ee6640 1 -- 172.21.15.161:0/2541875156 >> 172.21.15.161:0/2541875156 conn(0x7f4fd40fbf50 msgr2=0x7f4fd40fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:36:01.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.521+0000 7f4fd9ee6640 1 -- 172.21.15.161:0/2541875156 shutdown_connections 2024-08-18T23:36:01.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.521+0000 7f4fd9ee6640 1 -- 172.21.15.161:0/2541875156 wait complete. 2024-08-18T23:36:01.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.521+0000 7f4fd9ee6640 1 Processor -- start 2024-08-18T23:36:01.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.522+0000 7f4fd9ee6640 1 -- start start 2024-08-18T23:36:01.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.522+0000 7f4fd9ee6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fd4100340 0x7f4fd419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:01.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.522+0000 7f4fd9ee6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd419dc80 0x7f4fd41a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:01.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.522+0000 7f4fd9ee6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41120a0 con 0x7f4fd419dc80 2024-08-18T23:36:01.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.522+0000 7f4fd9ee6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd4111f20 con 0x7f4fd4100340 2024-08-18T23:36:01.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.522+0000 7f4fd37fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fd4100340 0x7f4fd419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:01.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.522+0000 7f4fd37fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fd4100340 0x7f4fd419d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:56004/0 (socket says 172.21.15.161:56004) 2024-08-18T23:36:01.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.522+0000 7f4fd37fe640 1 -- 172.21.15.161:0/3592476280 learned_addr learned my addr 172.21.15.161:0/3592476280 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-18T23:36:01.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.522+0000 7f4fd2ffd640 1 --2- 172.21.15.161:0/3592476280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd419dc80 0x7f4fd41a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:01.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.522+0000 7f4fd37fe640 1 -- 172.21.15.161:0/3592476280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fd4100340 msgr2=0x7f4fd419d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-08-18T23:36:01.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.522+0000 7f4fd37fe640 1 -- 172.21.15.161:0/3592476280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fd4100340 msgr2=0x7f4fd419d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-18T23:36:01.526 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.522+0000 7f4fd37fe640 1 --2- 172.21.15.161:0/3592476280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fd4100340 0x7f4fd419d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-18T23:36:01.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.522+0000 7f4fd37fe640 1 --2- 172.21.15.161:0/3592476280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fd4100340 0x7f4fd419d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-18T23:36:01.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.522+0000 7f4fd2ffd640 1 -- 172.21.15.161:0/3592476280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fd4100340 msgr2=0x7f4fd419d740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-18T23:36:01.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.522+0000 7f4fd2ffd640 1 --2- 172.21.15.161:0/3592476280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fd4100340 0x7f4fd419d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:01.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.522+0000 7f4fd2ffd640 1 -- 172.21.15.161:0/3592476280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd419e2a0 con 0x7f4fd419dc80 2024-08-18T23:36:01.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.523+0000 7f4fd2ffd640 1 --2- 172.21.15.161:0/3592476280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd419dc80 0x7f4fd41a2010 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800b3d0 tx=0x7f4fc800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:36:01.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.523+0000 7f4fd0ff9640 1 -- 172.21.15.161:0/3592476280 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4fc8011630 con 0x7f4fd419dc80 2024-08-18T23:36:01.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.523+0000 7f4fd0ff9640 1 -- 172.21.15.161:0/3592476280 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fc8011cb0 con 0x7f4fd419dc80 2024-08-18T23:36:01.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.523+0000 7f4fd9ee6640 1 -- 172.21.15.161:0/3592476280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd41a2550 con 0x7f4fd419dc80 2024-08-18T23:36:01.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.523+0000 7f4fd0ff9640 1 -- 172.21.15.161:0/3592476280 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4fc8010df0 con 0x7f4fd419dc80 2024-08-18T23:36:01.527 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.523+0000 7f4fd9ee6640 1 -- 172.21.15.161:0/3592476280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd41a2a90 con 0x7f4fd419dc80 2024-08-18T23:36:01.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.524+0000 7f4fd0ff9640 1 -- 172.21.15.161:0/3592476280 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4fc8018020 con 0x7f4fd419dc80 2024-08-18T23:36:01.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.524+0000 7f4fd9ee6640 1 -- 172.21.15.161:0/3592476280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd406aa40 con 0x7f4fd419dc80 2024-08-18T23:36:01.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.524+0000 7f4fd0ff9640 1 --2- 172.21.15.161:0/3592476280 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4fbc038e00 0x7f4fbc03b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:01.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.524+0000 7f4fd0ff9640 1 -- 172.21.15.161:0/3592476280 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4fc804bcb0 con 0x7f4fd419dc80 2024-08-18T23:36:01.528 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.524+0000 7f4fd37fe640 1 --2- 172.21.15.161:0/3592476280 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4fbc038e00 0x7f4fbc03b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:01.529 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.525+0000 7f4fd37fe640 1 --2- 172.21.15.161:0/3592476280 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4fbc038e00 0x7f4fbc03b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40096c0 tx=0x7f4fc4005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:36:01.530 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.529+0000 7f4fd0ff9640 1 -- 172.21.15.161:0/3592476280 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4fc801f020 con 0x7f4fd419dc80 2024-08-18T23:36:01.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.761+0000 7f4fd9ee6640 1 -- 172.21.15.161:0/3592476280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4fd41a2da0 con 0x7f4fd419dc80 2024-08-18T23:36:01.762 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.761+0000 7f4fd0ff9640 1 -- 172.21.15.161:0/3592476280 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f4fc80172c0 con 0x7f4fd419dc80 2024-08-18T23:36:01.762 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:36:01.763 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":2,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","modified":"2024-08-18T23:35:53.364656Z","created":"2024-08-18T23:34:23.952483Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-18T23:36:01.763 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 2 2024-08-18T23:36:01.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.764+0000 7f4fd9ee6640 1 -- 172.21.15.161:0/3592476280 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4fbc038e00 msgr2=0x7f4fbc03b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:01.764 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.764+0000 7f4fd9ee6640 1 --2- 172.21.15.161:0/3592476280 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4fbc038e00 0x7f4fbc03b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40096c0 tx=0x7f4fc4005a40 comp rx=0 tx=0).stop 2024-08-18T23:36:01.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.764+0000 7f4fd9ee6640 1 -- 172.21.15.161:0/3592476280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd419dc80 msgr2=0x7f4fd41a2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:01.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.764+0000 7f4fd9ee6640 1 --2- 172.21.15.161:0/3592476280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd419dc80 0x7f4fd41a2010 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800b3d0 tx=0x7f4fc800b890 comp rx=0 tx=0).stop 2024-08-18T23:36:01.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.764+0000 7f4fd9ee6640 1 -- 172.21.15.161:0/3592476280 shutdown_connections 2024-08-18T23:36:01.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.764+0000 7f4fd9ee6640 1 --2- 172.21.15.161:0/3592476280 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4fbc038e00 0x7f4fbc03b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:01.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.764+0000 7f4fd9ee6640 1 --2- 172.21.15.161:0/3592476280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd419dc80 0x7f4fd41a2010 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:01.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.764+0000 7f4fd9ee6640 1 --2- 172.21.15.161:0/3592476280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4fd4100340 0x7f4fd419d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:01.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.764+0000 7f4fd9ee6640 1 -- 172.21.15.161:0/3592476280 >> 172.21.15.161:0/3592476280 conn(0x7f4fd40fbf50 msgr2=0x7f4fd40fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:36:01.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.764+0000 7f4fd9ee6640 1 -- 172.21.15.161:0/3592476280 shutdown_connections 2024-08-18T23:36:01.765 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:01.764+0000 7f4fd9ee6640 1 -- 172.21.15.161:0/3592476280 wait complete. 2024-08-18T23:36:02.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:01 smithi123 podman[21789]: 2024-08-18 23:36:01.862958868 +0000 UTC m=+0.023433291 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:36:02.053 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:01 smithi161 ceph-mon[22027]: from='client.? 172.21.15.161:0/3592476280' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-18T23:36:02.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:01 smithi078 ceph-mon[18440]: from='client.? 172.21.15.161:0/3592476280' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-18T23:36:02.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 podman[21789]: 2024-08-18 23:36:02.162608035 +0000 UTC m=+0.323082467 container init 12b66fb5d4d319c4ab5863569d8fd16a1f76932ceaf37d81522a500898337e05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-b, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-08-18T23:36:02.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 podman[21789]: 2024-08-18 23:36:02.166054868 +0000 UTC m=+0.326529277 container start 12b66fb5d4d319c4ab5863569d8fd16a1f76932ceaf37d81522a500898337e05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-08-18T23:36:02.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: set uid:gid to 167:167 (ceph:ceph) 2024-08-18T23:36:02.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc), process ceph-mon, pid 2 2024-08-18T23:36:02.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: pidfile_write: ignore empty --pid-file 2024-08-18T23:36:02.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: load: jerasure load: lrc 2024-08-18T23:36:02.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: RocksDB version: 7.9.2 2024-08-18T23:36:02.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Git sha 0 2024-08-18T23:36:02.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Compile date 2024-08-18 15:19:56 2024-08-18T23:36:02.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: DB SUMMARY 2024-08-18T23:36:02.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: DB Session ID: 6WDCAUNQYH8ZN257TY6K 2024-08-18T23:36:02.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: CURRENT file: CURRENT 2024-08-18T23:36:02.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: IDENTITY file: IDENTITY 2024-08-18T23:36:02.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-08-18T23:36:02.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-08-18T23:36:02.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-08-18T23:36:02.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.error_if_exists: 0 2024-08-18T23:36:02.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.create_if_missing: 0 2024-08-18T23:36:02.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.paranoid_checks: 1 2024-08-18T23:36:02.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-18T23:36:02.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-18T23:36:02.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-18T23:36:02.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.env: 0x55745b5f07a0 2024-08-18T23:36:02.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.fs: PosixFileSystem 2024-08-18T23:36:02.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.info_log: 0x55745d8bb500 2024-08-18T23:36:02.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_file_opening_threads: 16 2024-08-18T23:36:02.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.statistics: (nil) 2024-08-18T23:36:02.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.use_fsync: 0 2024-08-18T23:36:02.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_log_file_size: 0 2024-08-18T23:36:02.552 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-18T23:36:02.552 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-18T23:36:02.552 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.keep_log_file_num: 1000 2024-08-18T23:36:02.552 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.recycle_log_file_num: 0 2024-08-18T23:36:02.552 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.allow_fallocate: 1 2024-08-18T23:36:02.552 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.allow_mmap_reads: 0 2024-08-18T23:36:02.552 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.allow_mmap_writes: 0 2024-08-18T23:36:02.552 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.use_direct_reads: 0 2024-08-18T23:36:02.552 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-18T23:36:02.552 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.create_missing_column_families: 0 2024-08-18T23:36:02.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.db_log_dir: 2024-08-18T23:36:02.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.wal_dir: 2024-08-18T23:36:02.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-18T23:36:02.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-18T23:36:02.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-18T23:36:02.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-18T23:36:02.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-18T23:36:02.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-18T23:36:02.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.advise_random_on_open: 1 2024-08-18T23:36:02.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.db_write_buffer_size: 0 2024-08-18T23:36:02.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.write_buffer_manager: 0x55745d8cadc0 2024-08-18T23:36:02.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-18T23:36:02.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-18T23:36:02.554 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-18T23:36:02.554 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.rate_limiter: (nil) 2024-08-18T23:36:02.554 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-18T23:36:02.554 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.wal_recovery_mode: 2 2024-08-18T23:36:02.554 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.enable_thread_tracking: 0 2024-08-18T23:36:02.554 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.enable_pipelined_write: 0 2024-08-18T23:36:02.554 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.unordered_write: 0 2024-08-18T23:36:02.554 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-18T23:36:02.554 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-18T23:36:02.554 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-18T23:36:02.554 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-18T23:36:02.554 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.row_cache: None 2024-08-18T23:36:02.555 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.wal_filter: None 2024-08-18T23:36:02.555 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-18T23:36:02.555 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.allow_ingest_behind: 0 2024-08-18T23:36:02.555 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.two_write_queues: 0 2024-08-18T23:36:02.555 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.manual_wal_flush: 0 2024-08-18T23:36:02.555 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.wal_compression: 0 2024-08-18T23:36:02.555 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.atomic_flush: 0 2024-08-18T23:36:02.555 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-18T23:36:02.555 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-18T23:36:02.555 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-18T23:36:02.555 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.log_readahead_size: 0 2024-08-18T23:36:02.555 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-18T23:36:02.555 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.best_efforts_recovery: 0 2024-08-18T23:36:02.555 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-18T23:36:02.556 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-18T23:36:02.556 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.allow_data_in_errors: 0 2024-08-18T23:36:02.556 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.db_host_id: __hostname__ 2024-08-18T23:36:02.556 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-18T23:36:02.556 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_background_jobs: 2 2024-08-18T23:36:02.556 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_background_compactions: -1 2024-08-18T23:36:02.556 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_subcompactions: 1 2024-08-18T23:36:02.556 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-18T23:36:02.556 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-18T23:36:02.556 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-18T23:36:02.556 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_total_wal_size: 0 2024-08-18T23:36:02.556 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-18T23:36:02.556 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-18T23:36:02.556 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-18T23:36:02.557 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-18T23:36:02.557 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_open_files: -1 2024-08-18T23:36:02.557 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.bytes_per_sync: 0 2024-08-18T23:36:02.557 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-18T23:36:02.557 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-18T23:36:02.557 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compaction_readahead_size: 0 2024-08-18T23:36:02.557 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_background_flushes: -1 2024-08-18T23:36:02.557 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Compression algorithms supported: 2024-08-18T23:36:02.557 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: kZSTD supported: 0 2024-08-18T23:36:02.557 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: kXpressCompression supported: 0 2024-08-18T23:36:02.557 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: kBZip2Compression supported: 0 2024-08-18T23:36:02.557 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-18T23:36:02.557 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: kLZ4Compression supported: 1 2024-08-18T23:36:02.557 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: kZlibCompression supported: 1 2024-08-18T23:36:02.558 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: kLZ4HCCompression supported: 1 2024-08-18T23:36:02.558 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: kSnappyCompression supported: 1 2024-08-18T23:36:02.558 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-18T23:36:02.558 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-18T23:36:02.558 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-08-18T23:36:02.558 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-18T23:36:02.558 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-18T23:36:02.558 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.merge_operator: 2024-08-18T23:36:02.558 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compaction_filter: None 2024-08-18T23:36:02.558 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compaction_filter_factory: None 2024-08-18T23:36:02.558 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.sst_partitioner_factory: None 2024-08-18T23:36:02.558 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-18T23:36:02.558 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-18T23:36:02.559 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55745d8bb120) 2024-08-18T23:36:02.559 INFO:journalctl@ceph.mon.b.smithi123.stdout: cache_index_and_filter_blocks: 1 2024-08-18T23:36:02.559 INFO:journalctl@ceph.mon.b.smithi123.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-18T23:36:02.559 INFO:journalctl@ceph.mon.b.smithi123.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-18T23:36:02.559 INFO:journalctl@ceph.mon.b.smithi123.stdout: pin_top_level_index_and_filter: 1 2024-08-18T23:36:02.559 INFO:journalctl@ceph.mon.b.smithi123.stdout: index_type: 0 2024-08-18T23:36:02.559 INFO:journalctl@ceph.mon.b.smithi123.stdout: data_block_index_type: 0 2024-08-18T23:36:02.559 INFO:journalctl@ceph.mon.b.smithi123.stdout: index_shortening: 1 2024-08-18T23:36:02.559 INFO:journalctl@ceph.mon.b.smithi123.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-18T23:36:02.559 INFO:journalctl@ceph.mon.b.smithi123.stdout: checksum: 4 2024-08-18T23:36:02.559 INFO:journalctl@ceph.mon.b.smithi123.stdout: no_block_cache: 0 2024-08-18T23:36:02.559 INFO:journalctl@ceph.mon.b.smithi123.stdout: block_cache: 0x55745d8af090 2024-08-18T23:36:02.559 INFO:journalctl@ceph.mon.b.smithi123.stdout: block_cache_name: BinnedLRUCache 2024-08-18T23:36:02.559 INFO:journalctl@ceph.mon.b.smithi123.stdout: block_cache_options: 2024-08-18T23:36:02.560 INFO:journalctl@ceph.mon.b.smithi123.stdout: capacity : 536870912 2024-08-18T23:36:02.560 INFO:journalctl@ceph.mon.b.smithi123.stdout: num_shard_bits : 4 2024-08-18T23:36:02.560 INFO:journalctl@ceph.mon.b.smithi123.stdout: strict_capacity_limit : 0 2024-08-18T23:36:02.560 INFO:journalctl@ceph.mon.b.smithi123.stdout: high_pri_pool_ratio: 0.000 2024-08-18T23:36:02.560 INFO:journalctl@ceph.mon.b.smithi123.stdout: block_cache_compressed: (nil) 2024-08-18T23:36:02.560 INFO:journalctl@ceph.mon.b.smithi123.stdout: persistent_cache: (nil) 2024-08-18T23:36:02.560 INFO:journalctl@ceph.mon.b.smithi123.stdout: block_size: 4096 2024-08-18T23:36:02.560 INFO:journalctl@ceph.mon.b.smithi123.stdout: block_size_deviation: 10 2024-08-18T23:36:02.560 INFO:journalctl@ceph.mon.b.smithi123.stdout: block_restart_interval: 16 2024-08-18T23:36:02.560 INFO:journalctl@ceph.mon.b.smithi123.stdout: index_block_restart_interval: 1 2024-08-18T23:36:02.560 INFO:journalctl@ceph.mon.b.smithi123.stdout: metadata_block_size: 4096 2024-08-18T23:36:02.560 INFO:journalctl@ceph.mon.b.smithi123.stdout: partition_filters: 0 2024-08-18T23:36:02.560 INFO:journalctl@ceph.mon.b.smithi123.stdout: use_delta_encoding: 1 2024-08-18T23:36:02.560 INFO:journalctl@ceph.mon.b.smithi123.stdout: filter_policy: bloomfilter 2024-08-18T23:36:02.561 INFO:journalctl@ceph.mon.b.smithi123.stdout: whole_key_filtering: 1 2024-08-18T23:36:02.561 INFO:journalctl@ceph.mon.b.smithi123.stdout: verify_compression: 0 2024-08-18T23:36:02.561 INFO:journalctl@ceph.mon.b.smithi123.stdout: read_amp_bytes_per_bit: 0 2024-08-18T23:36:02.561 INFO:journalctl@ceph.mon.b.smithi123.stdout: format_version: 5 2024-08-18T23:36:02.561 INFO:journalctl@ceph.mon.b.smithi123.stdout: enable_index_compression: 1 2024-08-18T23:36:02.561 INFO:journalctl@ceph.mon.b.smithi123.stdout: block_align: 0 2024-08-18T23:36:02.561 INFO:journalctl@ceph.mon.b.smithi123.stdout: max_auto_readahead_size: 262144 2024-08-18T23:36:02.561 INFO:journalctl@ceph.mon.b.smithi123.stdout: prepopulate_block_cache: 0 2024-08-18T23:36:02.561 INFO:journalctl@ceph.mon.b.smithi123.stdout: initial_auto_readahead_size: 8192 2024-08-18T23:36:02.561 INFO:journalctl@ceph.mon.b.smithi123.stdout: num_file_reads_for_auto_readahead: 2 2024-08-18T23:36:02.561 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.write_buffer_size: 33554432 2024-08-18T23:36:02.561 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_write_buffer_number: 2 2024-08-18T23:36:02.561 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compression: NoCompression 2024-08-18T23:36:02.561 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.bottommost_compression: Disabled 2024-08-18T23:36:02.562 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.prefix_extractor: nullptr 2024-08-18T23:36:02.562 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-18T23:36:02.562 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.num_levels: 7 2024-08-18T23:36:02.562 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-18T23:36:02.562 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-18T23:36:02.562 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-18T23:36:02.562 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-18T23:36:02.562 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-18T23:36:02.562 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-18T23:36:02.562 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-18T23:36:02.562 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-18T23:36:02.562 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-18T23:36:02.562 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-18T23:36:02.562 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-18T23:36:02.563 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-18T23:36:02.563 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-18T23:36:02.563 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compression_opts.level: 32767 2024-08-18T23:36:02.563 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compression_opts.strategy: 0 2024-08-18T23:36:02.563 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-18T23:36:02.563 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-18T23:36:02.563 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-18T23:36:02.563 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-18T23:36:02.563 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compression_opts.enabled: false 2024-08-18T23:36:02.563 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-18T23:36:02.563 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-18T23:36:02.563 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-18T23:36:02.563 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-18T23:36:02.563 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.target_file_size_base: 67108864 2024-08-18T23:36:02.564 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-18T23:36:02.564 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-18T23:36:02.564 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-18T23:36:02.564 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-18T23:36:02.564 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-18T23:36:02.564 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-18T23:36:02.564 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-18T23:36:02.564 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-18T23:36:02.564 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-18T23:36:02.564 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-18T23:36:02.564 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-18T23:36:02.564 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-18T23:36:02.564 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-18T23:36:02.564 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-18T23:36:02.565 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.arena_block_size: 1048576 2024-08-18T23:36:02.565 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-18T23:36:02.565 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-18T23:36:02.565 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.disable_auto_compactions: 0 2024-08-18T23:36:02.565 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-18T23:36:02.565 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-18T23:36:02.565 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-18T23:36:02.565 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-18T23:36:02.565 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-18T23:36:02.565 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-18T23:36:02.565 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-18T23:36:02.565 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-18T23:36:02.565 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-18T23:36:02.565 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-18T23:36:02.566 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-18T23:36:02.566 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.inplace_update_support: 0 2024-08-18T23:36:02.566 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-18T23:36:02.566 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-18T23:36:02.566 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-18T23:36:02.566 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-18T23:36:02.566 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.bloom_locality: 0 2024-08-18T23:36:02.566 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.max_successive_merges: 0 2024-08-18T23:36:02.566 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-18T23:36:02.566 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.paranoid_file_checks: 0 2024-08-18T23:36:02.566 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.force_consistency_checks: 1 2024-08-18T23:36:02.566 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.report_bg_io_stats: 0 2024-08-18T23:36:02.566 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.ttl: 2592000 2024-08-18T23:36:02.566 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-18T23:36:02.567 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-18T23:36:02.567 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-18T23:36:02.567 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.enable_blob_files: false 2024-08-18T23:36:02.567 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.min_blob_size: 0 2024-08-18T23:36:02.567 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.blob_file_size: 268435456 2024-08-18T23:36:02.567 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-18T23:36:02.567 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-18T23:36:02.567 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-18T23:36:02.567 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-18T23:36:02.567 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-18T23:36:02.567 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.blob_file_starting_level: 0 2024-08-18T23:36:02.567 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-18T23:36:02.567 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-08-18T23:36:02.567 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-08-18T23:36:02.568 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 31ae0ce9-2f48-4431-90ae-2e3e6b4f0974 2024-08-18T23:36:02.568 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724024162193341, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-08-18T23:36:02.568 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-08-18T23:36:02.568 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724024162193735, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1724024162, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "31ae0ce9-2f48-4431-90ae-2e3e6b4f0974", "db_session_id": "6WDCAUNQYH8ZN257TY6K", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-08-18T23:36:02.568 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724024162193836, "job": 1, "event": "recovery_finished"} 2024-08-18T23:36:02.568 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-08-18T23:36:02.568 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-18T23:36:02.568 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55745d8dce00 2024-08-18T23:36:02.568 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: DB pointer 0x55745d9d6000 2024-08-18T23:36:02.569 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-18T23:36:02.569 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-18T23:36:02.569 INFO:journalctl@ceph.mon.b.smithi123.stdout: ** DB Stats ** 2024-08-18T23:36:02.569 INFO:journalctl@ceph.mon.b.smithi123.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-18T23:36:02.569 INFO:journalctl@ceph.mon.b.smithi123.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-18T23:36:02.569 INFO:journalctl@ceph.mon.b.smithi123.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-18T23:36:02.569 INFO:journalctl@ceph.mon.b.smithi123.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-18T23:36:02.569 INFO:journalctl@ceph.mon.b.smithi123.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-18T23:36:02.569 INFO:journalctl@ceph.mon.b.smithi123.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-18T23:36:02.569 INFO:journalctl@ceph.mon.b.smithi123.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-18T23:36:02.569 INFO:journalctl@ceph.mon.b.smithi123.stdout: 2024-08-18T23:36:02.569 INFO:journalctl@ceph.mon.b.smithi123.stdout: ** Compaction Stats [default] ** 2024-08-18T23:36:02.569 INFO:journalctl@ceph.mon.b.smithi123.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-18T23:36:02.570 INFO:journalctl@ceph.mon.b.smithi123.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-18T23:36:02.570 INFO:journalctl@ceph.mon.b.smithi123.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-18T23:36:02.570 INFO:journalctl@ceph.mon.b.smithi123.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-18T23:36:02.570 INFO:journalctl@ceph.mon.b.smithi123.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-18T23:36:02.570 INFO:journalctl@ceph.mon.b.smithi123.stdout: 2024-08-18T23:36:02.570 INFO:journalctl@ceph.mon.b.smithi123.stdout: ** Compaction Stats [default] ** 2024-08-18T23:36:02.570 INFO:journalctl@ceph.mon.b.smithi123.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-18T23:36:02.570 INFO:journalctl@ceph.mon.b.smithi123.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-18T23:36:02.570 INFO:journalctl@ceph.mon.b.smithi123.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-18T23:36:02.570 INFO:journalctl@ceph.mon.b.smithi123.stdout: 2024-08-18T23:36:02.570 INFO:journalctl@ceph.mon.b.smithi123.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-18T23:36:02.570 INFO:journalctl@ceph.mon.b.smithi123.stdout: 2024-08-18T23:36:02.570 INFO:journalctl@ceph.mon.b.smithi123.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-18T23:36:02.571 INFO:journalctl@ceph.mon.b.smithi123.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-18T23:36:02.571 INFO:journalctl@ceph.mon.b.smithi123.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-18T23:36:02.571 INFO:journalctl@ceph.mon.b.smithi123.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-18T23:36:02.571 INFO:journalctl@ceph.mon.b.smithi123.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-18T23:36:02.571 INFO:journalctl@ceph.mon.b.smithi123.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-18T23:36:02.571 INFO:journalctl@ceph.mon.b.smithi123.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-18T23:36:02.571 INFO:journalctl@ceph.mon.b.smithi123.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-18T23:36:02.571 INFO:journalctl@ceph.mon.b.smithi123.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-18T23:36:02.571 INFO:journalctl@ceph.mon.b.smithi123.stdout: Block cache BinnedLRUCache@0x55745d8af090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-08-18T23:36:02.571 INFO:journalctl@ceph.mon.b.smithi123.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-08-18T23:36:02.571 INFO:journalctl@ceph.mon.b.smithi123.stdout: 2024-08-18T23:36:02.571 INFO:journalctl@ceph.mon.b.smithi123.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-18T23:36:02.571 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-08-18T23:36:02.571 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: using public_addr v2:172.21.15.123:0/0 -> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] 2024-08-18T23:36:02.572 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: starting mon.b rank -1 at public addrs [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] at bind addrs [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:36:02.572 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: mon.b@-1(???) e0 preinit fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:36:02.572 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: mon.b@-1(synchronizing).mds e1 new map 2024-08-18T23:36:02.572 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: mon.b@-1(synchronizing).mds e1 print_map 2024-08-18T23:36:02.572 INFO:journalctl@ceph.mon.b.smithi123.stdout: e1 2024-08-18T23:36:02.572 INFO:journalctl@ceph.mon.b.smithi123.stdout: btime 2024-08-18T23:34:33:867054+0000 2024-08-18T23:36:02.572 INFO:journalctl@ceph.mon.b.smithi123.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-18T23:36:02.572 INFO:journalctl@ceph.mon.b.smithi123.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-18T23:36:02.572 INFO:journalctl@ceph.mon.b.smithi123.stdout: legacy client fscid: -1 2024-08-18T23:36:02.572 INFO:journalctl@ceph.mon.b.smithi123.stdout: 2024-08-18T23:36:02.572 INFO:journalctl@ceph.mon.b.smithi123.stdout: No filesystems configured 2024-08-18T23:36:02.572 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-08-18T23:36:02.572 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-08-18T23:36:02.572 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-08-18T23:36:02.573 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-08-18T23:36:02.573 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-08-18T23:36:02.573 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-08-18T23:36:02.573 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-18T23:36:02.573 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-18T23:36:02.573 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-18T23:36:02.573 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-18T23:36:02.573 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: Updating smithi123:/var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-18T23:36:02.573 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-08-18T23:36:02.573 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:02.573 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:02.573 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:02.573 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:02.573 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:02.574 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: Updating smithi123:/var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-18T23:36:02.574 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: Added host smithi161 2024-08-18T23:36:02.574 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:02.574 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:02.574 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:02.574 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:02.574 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:36:02.574 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:02.574 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3214407833' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-08-18T23:36:02.574 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3214407833' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-18T23:36:02.574 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: osdmap e4: 0 total, 0 up, 0 in 2024-08-18T23:36:02.574 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:02.574 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:02.574 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:02.575 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:02.575 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:02.575 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:36:02.575 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:02.575 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:36:02.575 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: Updating smithi161:/etc/ceph/ceph.conf 2024-08-18T23:36:02.575 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: Updating smithi161:/var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-18T23:36:02.575 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-08-18T23:36:02.575 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: Updating smithi161:/var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-18T23:36:02.575 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi123:172.21.15.123=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:36:02.575 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: Saving service mon spec with placement smithi078:172.21.15.78=a;smithi123:172.21.15.123=b;smithi161:172.21.15.161=c;count:3 2024-08-18T23:36:02.575 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:02.575 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:02.576 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:02.576 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:02.576 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-18T23:36:02.576 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:02.576 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: Deploying daemon mon.c on smithi161 2024-08-18T23:36:02.576 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:02.576 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:02.576 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='client.? 172.21.15.161:0/1629823251' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-18T23:36:02.576 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-18T23:36:02.576 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:36:02.576 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: mon.a calling monitor election 2024-08-18T23:36:02.576 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:02.576 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:36:02.576 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:36:02.577 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: mon.c calling monitor election 2024-08-18T23:36:02.577 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:02.577 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:36:02.577 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:36:02.577 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:02.577 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:36:02.577 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-18T23:36:02.577 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: monmap epoch 2 2024-08-18T23:36:02.577 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:36:02.578 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: last_changed 2024-08-18T23:35:53.364656+0000 2024-08-18T23:36:02.578 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: created 2024-08-18T23:34:23.952483+0000 2024-08-18T23:36:02.578 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: min_mon_release 19 (squid) 2024-08-18T23:36:02.578 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: election_strategy: 1 2024-08-18T23:36:02.578 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-08-18T23:36:02.578 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-08-18T23:36:02.578 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: fsmap 2024-08-18T23:36:02.578 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: osdmap e4: 0 total, 0 up, 0 in 2024-08-18T23:36:02.578 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: mgrmap e13: x(active, since 36s) 2024-08-18T23:36:02.578 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: overall HEALTH_OK 2024-08-18T23:36:02.578 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:02.578 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:02.578 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:02.578 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-18T23:36:02.579 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:02.579 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: Deploying daemon mon.b on smithi123 2024-08-18T23:36:02.579 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='client.? 172.21.15.161:0/1263832166' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-18T23:36:02.579 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:36:02.579 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:02.579 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: from='client.? 172.21.15.161:0/3592476280' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-18T23:36:02.579 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 ceph-mon[21808]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-08-18T23:36:02.579 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 bash[21789]: 12b66fb5d4d319c4ab5863569d8fd16a1f76932ceaf37d81522a500898337e05 2024-08-18T23:36:02.579 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:02 smithi123 systemd[1]: Started Ceph mon.b for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:36:03.349 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-18T23:36:03.349 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-18T23:36:03.603 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.c/config 2024-08-18T23:36:07.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.256+0000 7fc61cab5640 1 -- 172.21.15.161:0/2864412296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc604003530 msgr2=0x7fc6040059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:07.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.256+0000 7fc61cab5640 1 --2- 172.21.15.161:0/2864412296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc604003530 0x7fc6040059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc618101720 tx=0x7fc60002c150 comp rx=0 tx=0).stop 2024-08-18T23:36:07.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.256+0000 7fc61cab5640 1 -- 172.21.15.161:0/2864412296 shutdown_connections 2024-08-18T23:36:07.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.256+0000 7fc61cab5640 1 --2- 172.21.15.161:0/2864412296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc604003530 0x7fc6040059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:07.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.256+0000 7fc61cab5640 1 -- 172.21.15.161:0/2864412296 >> 172.21.15.161:0/2864412296 conn(0x7fc6180fbf50 msgr2=0x7fc6180fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:36:07.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.256+0000 7fc61cab5640 1 -- 172.21.15.161:0/2864412296 shutdown_connections 2024-08-18T23:36:07.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.256+0000 7fc61cab5640 1 -- 172.21.15.161:0/2864412296 wait complete. 2024-08-18T23:36:07.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.257+0000 7fc61cab5640 1 Processor -- start 2024-08-18T23:36:07.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.257+0000 7fc61cab5640 1 -- start start 2024-08-18T23:36:07.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc61cab5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc604003530 0x7fc618101f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:07.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc61cab5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6181024d0 0x7fc618102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:07.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc61cab5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6181a3220 0x7fc6181a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:07.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc61cab5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc618108350 con 0x7fc6181024d0 2024-08-18T23:36:07.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc61cab5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6181081d0 con 0x7fc604003530 2024-08-18T23:36:07.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc61cab5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc6181084d0 con 0x7fc6181a3220 2024-08-18T23:36:07.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc616d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6181a3220 0x7fc6181a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:07.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc616d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6181a3220 0x7fc6181a5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43838/0 (socket says 172.21.15.161:43838) 2024-08-18T23:36:07.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc616d9d640 1 -- 172.21.15.161:0/3235906406 learned_addr learned my addr 172.21.15.161:0/3235906406 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-18T23:36:07.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc615d9b640 1 --2- 172.21.15.161:0/3235906406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6181024d0 0x7fc618102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:07.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc61659c640 1 --2- 172.21.15.161:0/3235906406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc604003530 0x7fc618101f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:07.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc616d9d640 1 -- 172.21.15.161:0/3235906406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6181a3220 msgr2=0x7fc6181a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-08-18T23:36:07.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc616d9d640 1 -- 172.21.15.161:0/3235906406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6181a3220 msgr2=0x7fc6181a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-18T23:36:07.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc616d9d640 1 --2- 172.21.15.161:0/3235906406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6181a3220 0x7fc6181a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-18T23:36:07.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc616d9d640 1 --2- 172.21.15.161:0/3235906406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6181a3220 0x7fc6181a5610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-18T23:36:07.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc61659c640 1 -- 172.21.15.161:0/3235906406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc604003530 msgr2=0x7fc618101f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-08-18T23:36:07.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc61659c640 1 -- 172.21.15.161:0/3235906406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc604003530 msgr2=0x7fc618101f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-18T23:36:07.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc61659c640 1 --2- 172.21.15.161:0/3235906406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc604003530 0x7fc618101f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-18T23:36:07.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc61659c640 1 --2- 172.21.15.161:0/3235906406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc604003530 0x7fc618101f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-18T23:36:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc615d9b640 1 -- 172.21.15.161:0/3235906406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6181a3220 msgr2=0x7fc6181a5610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-18T23:36:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc615d9b640 1 --2- 172.21.15.161:0/3235906406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6181a3220 0x7fc6181a5610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc615d9b640 1 -- 172.21.15.161:0/3235906406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc604003530 msgr2=0x7fc618101f90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-18T23:36:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc615d9b640 1 --2- 172.21.15.161:0/3235906406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc604003530 0x7fc618101f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.258+0000 7fc615d9b640 1 -- 172.21.15.161:0/3235906406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6181a5ce0 con 0x7fc6181024d0 2024-08-18T23:36:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.259+0000 7fc615d9b640 1 --2- 172.21.15.161:0/3235906406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6181024d0 0x7fc618102930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc60800b450 tx=0x7fc60800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:36:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.259+0000 7fc60f7fe640 1 -- 172.21.15.161:0/3235906406 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608004230 con 0x7fc6181024d0 2024-08-18T23:36:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.259+0000 7fc60f7fe640 1 -- 172.21.15.161:0/3235906406 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6080043d0 con 0x7fc6181024d0 2024-08-18T23:36:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.259+0000 7fc61cab5640 1 -- 172.21.15.161:0/3235906406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6181a5fd0 con 0x7fc6181024d0 2024-08-18T23:36:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.259+0000 7fc60f7fe640 1 -- 172.21.15.161:0/3235906406 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608003cb0 con 0x7fc6181024d0 2024-08-18T23:36:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.259+0000 7fc61cab5640 1 -- 172.21.15.161:0/3235906406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6181a64e0 con 0x7fc6181024d0 2024-08-18T23:36:07.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.260+0000 7fc60f7fe640 1 -- 172.21.15.161:0/3235906406 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc60800f050 con 0x7fc6181024d0 2024-08-18T23:36:07.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.260+0000 7fc60f7fe640 1 --2- 172.21.15.161:0/3235906406 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc5f80394f0 0x7fc5f803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:07.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.260+0000 7fc60f7fe640 1 -- 172.21.15.161:0/3235906406 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc60804ce90 con 0x7fc6181024d0 2024-08-18T23:36:07.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.260+0000 7fc61cab5640 1 -- 172.21.15.161:0/3235906406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc618109480 con 0x7fc6181024d0 2024-08-18T23:36:07.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.261+0000 7fc61659c640 1 --2- 172.21.15.161:0/3235906406 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc5f80394f0 0x7fc5f803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:07.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.261+0000 7fc61659c640 1 --2- 172.21.15.161:0/3235906406 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc5f80394f0 0x7fc5f803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc600005c80 tx=0x7fc600005d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:36:07.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.266+0000 7fc60f7fe640 1 -- 172.21.15.161:0/3235906406 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc60800d040 con 0x7fc6181024d0 2024-08-18T23:36:07.512 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.512+0000 7fc61cab5640 1 -- 172.21.15.161:0/3235906406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc618190ac0 con 0x7fc6181024d0 2024-08-18T23:36:07.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:07.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-18T23:36:07.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: mon.c calling monitor election 2024-08-18T23:36:07.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: mon.a calling monitor election 2024-08-18T23:36:07.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:07.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:36:07.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:07.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:07.514 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:07.514 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:07.514 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:07.514 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:07.514 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:07.514 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-18T23:36:07.514 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: monmap epoch 3 2024-08-18T23:36:07.514 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:36:07.514 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: last_changed 2024-08-18T23:36:02.218902+0000 2024-08-18T23:36:07.514 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: created 2024-08-18T23:34:23.952483+0000 2024-08-18T23:36:07.514 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: min_mon_release 19 (squid) 2024-08-18T23:36:07.514 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: election_strategy: 1 2024-08-18T23:36:07.514 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-08-18T23:36:07.515 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-08-18T23:36:07.515 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-08-18T23:36:07.515 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: fsmap 2024-08-18T23:36:07.515 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: osdmap e4: 0 total, 0 up, 0 in 2024-08-18T23:36:07.515 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: mgrmap e13: x(active, since 45s) 2024-08-18T23:36:07.515 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: overall HEALTH_OK 2024-08-18T23:36:07.515 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:07.515 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:07.515 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:07.515 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:07.515 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.512+0000 7fc60f7fe640 1 -- 172.21.15.161:0/3235906406 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7fc608011d00 con 0x7fc6181024d0 2024-08-18T23:36:07.516 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:36:07.516 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":3,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","modified":"2024-08-18T23:36:02.218902Z","created":"2024-08-18T23:34:23.952483Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-18T23:36:07.517 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 3 2024-08-18T23:36:07.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.515+0000 7fc61cab5640 1 -- 172.21.15.161:0/3235906406 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc5f80394f0 msgr2=0x7fc5f803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:07.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.515+0000 7fc61cab5640 1 --2- 172.21.15.161:0/3235906406 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc5f80394f0 0x7fc5f803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc600005c80 tx=0x7fc600005d10 comp rx=0 tx=0).stop 2024-08-18T23:36:07.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.515+0000 7fc61cab5640 1 -- 172.21.15.161:0/3235906406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6181024d0 msgr2=0x7fc618102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:07.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.515+0000 7fc61cab5640 1 --2- 172.21.15.161:0/3235906406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6181024d0 0x7fc618102930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc60800b450 tx=0x7fc60800b910 comp rx=0 tx=0).stop 2024-08-18T23:36:07.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.515+0000 7fc61cab5640 1 -- 172.21.15.161:0/3235906406 shutdown_connections 2024-08-18T23:36:07.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.515+0000 7fc61cab5640 1 --2- 172.21.15.161:0/3235906406 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc5f80394f0 0x7fc5f803b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:07.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.515+0000 7fc61cab5640 1 --2- 172.21.15.161:0/3235906406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6181a3220 0x7fc6181a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:07.517 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.515+0000 7fc61cab5640 1 --2- 172.21.15.161:0/3235906406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6181024d0 0x7fc618102930 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:07.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.515+0000 7fc61cab5640 1 --2- 172.21.15.161:0/3235906406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc604003530 0x7fc618101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:07.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.515+0000 7fc61cab5640 1 -- 172.21.15.161:0/3235906406 >> 172.21.15.161:0/3235906406 conn(0x7fc6180fbf50 msgr2=0x7fc61806a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:36:07.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.515+0000 7fc61cab5640 1 -- 172.21.15.161:0/3235906406 shutdown_connections 2024-08-18T23:36:07.518 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:07.515+0000 7fc61cab5640 1 -- 172.21.15.161:0/3235906406 wait complete. 2024-08-18T23:36:07.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:07.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-18T23:36:07.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: mon.c calling monitor election 2024-08-18T23:36:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: mon.a calling monitor election 2024-08-18T23:36:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:36:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-18T23:36:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: monmap epoch 3 2024-08-18T23:36:07.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:36:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: last_changed 2024-08-18T23:36:02.218902+0000 2024-08-18T23:36:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: created 2024-08-18T23:34:23.952483+0000 2024-08-18T23:36:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: min_mon_release 19 (squid) 2024-08-18T23:36:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: election_strategy: 1 2024-08-18T23:36:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-08-18T23:36:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-08-18T23:36:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-08-18T23:36:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: fsmap 2024-08-18T23:36:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: osdmap e4: 0 total, 0 up, 0 in 2024-08-18T23:36:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: mgrmap e13: x(active, since 45s) 2024-08-18T23:36:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: overall HEALTH_OK 2024-08-18T23:36:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:07.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:07.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:08.152 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-08-18T23:36:08.153 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph config generate-minimal-conf 2024-08-18T23:36:08.413 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:36:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:08 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:08.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:08 smithi161 ceph-mon[22027]: from='client.? 172.21.15.161:0/3235906406' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-18T23:36:08.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:08 smithi161 ceph-mon[22027]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:08.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:08 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:08.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:08 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:08.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:08 smithi078 ceph-mon[18440]: from='client.? 172.21.15.161:0/3235906406' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-18T23:36:08.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:08 smithi078 ceph-mon[18440]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:08.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:08 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:08.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.866+0000 7f8768d16640 1 -- 172.21.15.78:0/2423545241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8764100420 msgr2=0x7f8764100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:08.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.866+0000 7f8768d16640 1 --2- 172.21.15.78:0/2423545241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8764100420 0x7f8764100800 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f875c009880 tx=0x7f875c02a5d0 comp rx=0 tx=0).stop 2024-08-18T23:36:08.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.866+0000 7f8768d16640 1 -- 172.21.15.78:0/2423545241 shutdown_connections 2024-08-18T23:36:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.866+0000 7f8768d16640 1 --2- 172.21.15.78:0/2423545241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8764100420 0x7f8764100800 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.866+0000 7f8768d16640 1 -- 172.21.15.78:0/2423545241 >> 172.21.15.78:0/2423545241 conn(0x7f87640fbee0 msgr2=0x7f87640fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:36:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.866+0000 7f8768d16640 1 -- 172.21.15.78:0/2423545241 shutdown_connections 2024-08-18T23:36:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.866+0000 7f8768d16640 1 -- 172.21.15.78:0/2423545241 wait complete. 2024-08-18T23:36:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.867+0000 7f8768d16640 1 Processor -- start 2024-08-18T23:36:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.867+0000 7f8768d16640 1 -- start start 2024-08-18T23:36:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.867+0000 7f8768d16640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8764100420 0x7f87641971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.867+0000 7f8768d16640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87641976e0 0x7f876419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.867+0000 7f8768d16640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f876419bf10 0x7f876419c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.867+0000 7f8768d16640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f876410a820 con 0x7f8764100420 2024-08-18T23:36:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.867+0000 7f8768d16640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f876410a6a0 con 0x7f876419bf10 2024-08-18T23:36:08.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.867+0000 7f8768d16640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f876410a9a0 con 0x7f87641976e0 2024-08-18T23:36:08.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.867+0000 7f876259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8764100420 0x7f87641971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:08.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.867+0000 7f876259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8764100420 0x7f87641971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43932/0 (socket says 172.21.15.78:43932) 2024-08-18T23:36:08.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.867+0000 7f876259c640 1 -- 172.21.15.78:0/1315936299 learned_addr learned my addr 172.21.15.78:0/1315936299 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:36:08.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.867+0000 7f8761d9b640 1 --2- 172.21.15.78:0/1315936299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87641976e0 0x7f876419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:08.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.867+0000 7f8762d9d640 1 --2- 172.21.15.78:0/1315936299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f876419bf10 0x7f876419c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:08.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.867+0000 7f876259c640 1 -- 172.21.15.78:0/1315936299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87641976e0 msgr2=0x7f876419b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:08.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.867+0000 7f876259c640 1 --2- 172.21.15.78:0/1315936299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87641976e0 0x7f876419b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:08.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.867+0000 7f876259c640 1 -- 172.21.15.78:0/1315936299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f876419bf10 msgr2=0x7f876419c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:08.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.868+0000 7f876259c640 1 --2- 172.21.15.78:0/1315936299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f876419bf10 0x7f876419c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:08.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.868+0000 7f876259c640 1 -- 172.21.15.78:0/1315936299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87641a3c80 con 0x7f8764100420 2024-08-18T23:36:08.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.868+0000 7f876259c640 1 --2- 172.21.15.78:0/1315936299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8764100420 0x7f87641971a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f875c031550 tx=0x7f875c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:36:08.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.868+0000 7f87537fe640 1 -- 172.21.15.78:0/1315936299 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c03f030 con 0x7f8764100420 2024-08-18T23:36:08.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.868+0000 7f8768d16640 1 -- 172.21.15.78:0/1315936299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87641a3e50 con 0x7f8764100420 2024-08-18T23:36:08.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.868+0000 7f87537fe640 1 -- 172.21.15.78:0/1315936299 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f875c003ce0 con 0x7f8764100420 2024-08-18T23:36:08.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.868+0000 7f87537fe640 1 -- 172.21.15.78:0/1315936299 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c035070 con 0x7f8764100420 2024-08-18T23:36:08.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.868+0000 7f8768d16640 1 -- 172.21.15.78:0/1315936299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87641a42b0 con 0x7f8764100420 2024-08-18T23:36:08.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.869+0000 7f87537fe640 1 -- 172.21.15.78:0/1315936299 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f875c042030 con 0x7f8764100420 2024-08-18T23:36:08.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.869+0000 7f8768d16640 1 -- 172.21.15.78:0/1315936299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8764100880 con 0x7f8764100420 2024-08-18T23:36:08.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.869+0000 7f87537fe640 1 --2- 172.21.15.78:0/1315936299 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f874c0394f0 0x7f874c03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:08.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.869+0000 7f87537fe640 1 -- 172.21.15.78:0/1315936299 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f875c06ce20 con 0x7f8764100420 2024-08-18T23:36:08.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.869+0000 7f8761d9b640 1 --2- 172.21.15.78:0/1315936299 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f874c0394f0 0x7f874c03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:08.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.870+0000 7f8761d9b640 1 --2- 172.21.15.78:0/1315936299 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f874c0394f0 0x7f874c03b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f875800e250 tx=0x7f875800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:36:08.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:08.874+0000 7f87537fe640 1 -- 172.21.15.78:0/1315936299 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f875c03bce0 con 0x7f8764100420 2024-08-18T23:36:09.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:09.051+0000 7f8768d16640 1 -- 172.21.15.78:0/1315936299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f87641a4600 con 0x7f8764100420 2024-08-18T23:36:09.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:09.051+0000 7f87537fe640 1 -- 172.21.15.78:0/1315936299 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f875c044930 con 0x7f8764100420 2024-08-18T23:36:09.053 INFO:teuthology.orchestra.run.smithi078.stdout:# minimal ceph.conf for 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:36:09.053 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2024-08-18T23:36:09.053 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:36:09.053 INFO:teuthology.orchestra.run.smithi078.stdout: mon_host = [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] 2024-08-18T23:36:09.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:09.053+0000 7f8768d16640 1 -- 172.21.15.78:0/1315936299 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f874c0394f0 msgr2=0x7f874c03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:09.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:09.053+0000 7f8768d16640 1 --2- 172.21.15.78:0/1315936299 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f874c0394f0 0x7f874c03b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f875800e250 tx=0x7f875800b040 comp rx=0 tx=0).stop 2024-08-18T23:36:09.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:09.053+0000 7f8768d16640 1 -- 172.21.15.78:0/1315936299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8764100420 msgr2=0x7f87641971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:09.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:09.053+0000 7f8768d16640 1 --2- 172.21.15.78:0/1315936299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8764100420 0x7f87641971a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f875c031550 tx=0x7f875c0042f0 comp rx=0 tx=0).stop 2024-08-18T23:36:09.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:09.053+0000 7f8768d16640 1 -- 172.21.15.78:0/1315936299 shutdown_connections 2024-08-18T23:36:09.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:09.053+0000 7f8768d16640 1 --2- 172.21.15.78:0/1315936299 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f874c0394f0 0x7f874c03b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:09.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:09.053+0000 7f8768d16640 1 --2- 172.21.15.78:0/1315936299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f876419bf10 0x7f876419c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:09.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:09.053+0000 7f8768d16640 1 --2- 172.21.15.78:0/1315936299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87641976e0 0x7f876419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:09.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:09.053+0000 7f8768d16640 1 --2- 172.21.15.78:0/1315936299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8764100420 0x7f87641971a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:09.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:09.053+0000 7f8768d16640 1 -- 172.21.15.78:0/1315936299 >> 172.21.15.78:0/1315936299 conn(0x7f87640fbee0 msgr2=0x7f8764069c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:36:09.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:09.054+0000 7f8768d16640 1 -- 172.21.15.78:0/1315936299 shutdown_connections 2024-08-18T23:36:09.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:09.054+0000 7f8768d16640 1 -- 172.21.15.78:0/1315936299 wait complete. 2024-08-18T23:36:09.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:09.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-18T23:36:09.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: mon.c calling monitor election 2024-08-18T23:36:09.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: mon.a calling monitor election 2024-08-18T23:36:09.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:09.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:36:09.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:09.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:09.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:09.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:09.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:09.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:09.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:09.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-18T23:36:09.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: monmap epoch 3 2024-08-18T23:36:09.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:36:09.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: last_changed 2024-08-18T23:36:02.218902+0000 2024-08-18T23:36:09.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: created 2024-08-18T23:34:23.952483+0000 2024-08-18T23:36:09.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: min_mon_release 19 (squid) 2024-08-18T23:36:09.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: election_strategy: 1 2024-08-18T23:36:09.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-08-18T23:36:09.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-08-18T23:36:09.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-08-18T23:36:09.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: fsmap 2024-08-18T23:36:09.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: osdmap e4: 0 total, 0 up, 0 in 2024-08-18T23:36:09.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: mgrmap e13: x(active, since 45s) 2024-08-18T23:36:09.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: overall HEALTH_OK 2024-08-18T23:36:09.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:09.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:09.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:09.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:09.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:09.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='client.? 172.21.15.161:0/3235906406' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-18T23:36:09.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:09.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:09.740 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-08-18T23:36:09.740 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:36:09.740 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.conf 2024-08-18T23:36:09.768 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:36:09.769 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-18T23:36:09.834 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-18T23:36:09.834 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.conf 2024-08-18T23:36:09.862 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-18T23:36:09.862 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-18T23:36:09.928 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-18T23:36:09.928 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.conf 2024-08-18T23:36:09.955 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-18T23:36:09.956 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-18T23:36:10.021 INFO:tasks.cephadm:Adding mgr.x on smithi078 2024-08-18T23:36:10.022 INFO:tasks.cephadm:Adding mgr.y on smithi123 2024-08-18T23:36:10.022 INFO:tasks.cephadm:Adding mgr.z on smithi161 2024-08-18T23:36:10.022 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch apply mgr '3;smithi078=x;smithi123=y;smithi161=z' 2024-08-18T23:36:10.153 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: mon.b calling monitor election 2024-08-18T23:36:10.153 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: mon.b calling monitor election 2024-08-18T23:36:10.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: mon.c calling monitor election 2024-08-18T23:36:10.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: mon.a calling monitor election 2024-08-18T23:36:10.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-18T23:36:10.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: monmap epoch 3 2024-08-18T23:36:10.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:36:10.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: last_changed 2024-08-18T23:36:02.218902+0000 2024-08-18T23:36:10.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: created 2024-08-18T23:34:23.952483+0000 2024-08-18T23:36:10.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: min_mon_release 19 (squid) 2024-08-18T23:36:10.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: election_strategy: 1 2024-08-18T23:36:10.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-08-18T23:36:10.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-08-18T23:36:10.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-08-18T23:36:10.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: fsmap 2024-08-18T23:36:10.155 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: osdmap e4: 0 total, 0 up, 0 in 2024-08-18T23:36:10.155 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: mgrmap e13: x(active, since 47s) 2024-08-18T23:36:10.155 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: overall HEALTH_OK 2024-08-18T23:36:10.155 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.155 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.155 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.155 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.155 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.155 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.156 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.156 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.156 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.156 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.156 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.156 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-18T23:36:10.156 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-18T23:36:10.156 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:10.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: mon.b calling monitor election 2024-08-18T23:36:10.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: mon.b calling monitor election 2024-08-18T23:36:10.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: mon.c calling monitor election 2024-08-18T23:36:10.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: mon.a calling monitor election 2024-08-18T23:36:10.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-18T23:36:10.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: monmap epoch 3 2024-08-18T23:36:10.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:36:10.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: last_changed 2024-08-18T23:36:02.218902+0000 2024-08-18T23:36:10.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: created 2024-08-18T23:34:23.952483+0000 2024-08-18T23:36:10.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: min_mon_release 19 (squid) 2024-08-18T23:36:10.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: election_strategy: 1 2024-08-18T23:36:10.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-08-18T23:36:10.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-08-18T23:36:10.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-08-18T23:36:10.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: fsmap 2024-08-18T23:36:10.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: osdmap e4: 0 total, 0 up, 0 in 2024-08-18T23:36:10.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: mgrmap e13: x(active, since 47s) 2024-08-18T23:36:10.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: overall HEALTH_OK 2024-08-18T23:36:10.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-18T23:36:10.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-18T23:36:10.180 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:10.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: mon.b calling monitor election 2024-08-18T23:36:10.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: mon.b calling monitor election 2024-08-18T23:36:10.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: mon.c calling monitor election 2024-08-18T23:36:10.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: mon.a calling monitor election 2024-08-18T23:36:10.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-18T23:36:10.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: monmap epoch 3 2024-08-18T23:36:10.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:36:10.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: last_changed 2024-08-18T23:36:02.218902+0000 2024-08-18T23:36:10.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: created 2024-08-18T23:34:23.952483+0000 2024-08-18T23:36:10.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: min_mon_release 19 (squid) 2024-08-18T23:36:10.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: election_strategy: 1 2024-08-18T23:36:10.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-08-18T23:36:10.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-08-18T23:36:10.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-08-18T23:36:10.301 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: fsmap 2024-08-18T23:36:10.301 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: osdmap e4: 0 total, 0 up, 0 in 2024-08-18T23:36:10.301 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: mgrmap e13: x(active, since 47s) 2024-08-18T23:36:10.301 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: overall HEALTH_OK 2024-08-18T23:36:10.301 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.301 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.302 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.302 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.302 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.302 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.302 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.302 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.302 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.302 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.302 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:10.302 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-18T23:36:10.302 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-18T23:36:10.302 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:10.319 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.c/config 2024-08-18T23:36:10.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.737+0000 7f0858063640 1 -- 172.21.15.161:0/2443332985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0850100310 msgr2=0x7f0850100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:10.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.737+0000 7f0858063640 1 --2- 172.21.15.161:0/2443332985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0850100310 0x7f0850100710 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f084c009880 tx=0x7f084c02a5d0 comp rx=0 tx=0).stop 2024-08-18T23:36:10.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.737+0000 7f0858063640 1 -- 172.21.15.161:0/2443332985 shutdown_connections 2024-08-18T23:36:10.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.737+0000 7f0858063640 1 --2- 172.21.15.161:0/2443332985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0850100310 0x7f0850100710 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:10.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.737+0000 7f0858063640 1 -- 172.21.15.161:0/2443332985 >> 172.21.15.161:0/2443332985 conn(0x7f08500fbae0 msgr2=0x7f08500fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:36:10.739 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.738+0000 7f0858063640 1 -- 172.21.15.161:0/2443332985 shutdown_connections 2024-08-18T23:36:10.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.738+0000 7f0858063640 1 -- 172.21.15.161:0/2443332985 wait complete. 2024-08-18T23:36:10.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.738+0000 7f0858063640 1 Processor -- start 2024-08-18T23:36:10.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.738+0000 7f0858063640 1 -- start start 2024-08-18T23:36:10.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.738+0000 7f0858063640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0850100310 0x7f0850196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:10.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.738+0000 7f0858063640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08501972d0 0x7f085019c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:10.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.738+0000 7f0858063640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0850197750 0x7f085019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:10.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.738+0000 7f0858063640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0850106400 con 0x7f0850197750 2024-08-18T23:36:10.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.738+0000 7f0858063640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0850106280 con 0x7f0850100310 2024-08-18T23:36:10.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.738+0000 7f0858063640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0850106580 con 0x7f08501972d0 2024-08-18T23:36:10.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.738+0000 7f08555fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08501972d0 0x7f085019c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:10.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.739+0000 7f08555fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08501972d0 0x7f085019c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:43854/0 (socket says 172.21.15.161:43854) 2024-08-18T23:36:10.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.739+0000 7f08555fe640 1 -- 172.21.15.161:0/2791461757 learned_addr learned my addr 172.21.15.161:0/2791461757 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-18T23:36:10.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.739+0000 7f0855dff640 1 --2- 172.21.15.161:0/2791461757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0850100310 0x7f0850196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:10.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.739+0000 7f0856600640 1 --2- 172.21.15.161:0/2791461757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0850197750 0x7f085019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:10.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.739+0000 7f08555fe640 1 -- 172.21.15.161:0/2791461757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0850100310 msgr2=0x7f0850196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:10.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.739+0000 7f08555fe640 1 --2- 172.21.15.161:0/2791461757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0850100310 0x7f0850196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:10.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.739+0000 7f08555fe640 1 -- 172.21.15.161:0/2791461757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0850197750 msgr2=0x7f085019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:10.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.739+0000 7f08555fe640 1 --2- 172.21.15.161:0/2791461757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0850197750 0x7f085019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:10.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.739+0000 7f08555fe640 1 -- 172.21.15.161:0/2791461757 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085019f150 con 0x7f08501972d0 2024-08-18T23:36:10.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.739+0000 7f08555fe640 1 --2- 172.21.15.161:0/2791461757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08501972d0 0x7f085019c300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f084000b520 tx=0x7f084000b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:36:10.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.739+0000 7f0846ffd640 1 -- 172.21.15.161:0/2791461757 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0840004230 con 0x7f08501972d0 2024-08-18T23:36:10.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.739+0000 7f0846ffd640 1 -- 172.21.15.161:0/2791461757 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08400043d0 con 0x7f08501972d0 2024-08-18T23:36:10.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.739+0000 7f0846ffd640 1 -- 172.21.15.161:0/2791461757 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0840003d60 con 0x7f08501972d0 2024-08-18T23:36:10.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.739+0000 7f0858063640 1 -- 172.21.15.161:0/2791461757 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085019f440 con 0x7f08501972d0 2024-08-18T23:36:10.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.740+0000 7f0858063640 1 -- 172.21.15.161:0/2791461757 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f085019fa00 con 0x7f08501972d0 2024-08-18T23:36:10.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.740+0000 7f0846ffd640 1 -- 172.21.15.161:0/2791461757 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f084000e070 con 0x7f08501972d0 2024-08-18T23:36:10.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.740+0000 7f0846ffd640 1 --2- 172.21.15.161:0/2791461757 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f083c0394f0 0x7f083c03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:10.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.740+0000 7f0846ffd640 1 -- 172.21.15.161:0/2791461757 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f084004ba70 con 0x7f08501972d0 2024-08-18T23:36:10.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.741+0000 7f0858063640 1 -- 172.21.15.161:0/2791461757 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0850190590 con 0x7f08501972d0 2024-08-18T23:36:10.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.741+0000 7f0855dff640 1 --2- 172.21.15.161:0/2791461757 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f083c0394f0 0x7f083c03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:10.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.741+0000 7f0855dff640 1 --2- 172.21.15.161:0/2791461757 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f083c0394f0 0x7f083c03b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f084c0314e0 tx=0x7f084c0315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:36:10.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.746+0000 7f0846ffd640 1 -- 172.21.15.161:0/2791461757 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0840014020 con 0x7f08501972d0 2024-08-18T23:36:10.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.930+0000 7f0858063640 1 -- 172.21.15.161:0/2791461757 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi123=y;smithi161=z", "target": ["mon-mgr", ""]}) -- 0x7f08500623f0 con 0x7f083c0394f0 2024-08-18T23:36:10.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.935+0000 7f0846ffd640 1 -- 172.21.15.161:0/2791461757 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f08500623f0 con 0x7f083c0394f0 2024-08-18T23:36:10.936 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mgr update... 2024-08-18T23:36:10.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.937+0000 7f0858063640 1 -- 172.21.15.161:0/2791461757 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f083c0394f0 msgr2=0x7f083c03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:10.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.937+0000 7f0858063640 1 --2- 172.21.15.161:0/2791461757 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f083c0394f0 0x7f083c03b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f084c0314e0 tx=0x7f084c0315d0 comp rx=0 tx=0).stop 2024-08-18T23:36:10.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.937+0000 7f0858063640 1 -- 172.21.15.161:0/2791461757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08501972d0 msgr2=0x7f085019c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:10.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.937+0000 7f0858063640 1 --2- 172.21.15.161:0/2791461757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08501972d0 0x7f085019c300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f084000b520 tx=0x7f084000b9e0 comp rx=0 tx=0).stop 2024-08-18T23:36:10.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.937+0000 7f0858063640 1 -- 172.21.15.161:0/2791461757 shutdown_connections 2024-08-18T23:36:10.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.937+0000 7f0858063640 1 --2- 172.21.15.161:0/2791461757 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f083c0394f0 0x7f083c03b9b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:10.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.937+0000 7f0858063640 1 --2- 172.21.15.161:0/2791461757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0850197750 0x7f085019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:10.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.937+0000 7f0858063640 1 --2- 172.21.15.161:0/2791461757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08501972d0 0x7f085019c300 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:10.938 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.937+0000 7f0858063640 1 --2- 172.21.15.161:0/2791461757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0850100310 0x7f0850196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:10.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.937+0000 7f0858063640 1 -- 172.21.15.161:0/2791461757 >> 172.21.15.161:0/2791461757 conn(0x7f08500fbae0 msgr2=0x7f0850069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:36:10.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.937+0000 7f0858063640 1 -- 172.21.15.161:0/2791461757 shutdown_connections 2024-08-18T23:36:10.939 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:10.937+0000 7f0858063640 1 -- 172.21.15.161:0/2791461757 wait complete. 2024-08-18T23:36:11.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:10 smithi161 ceph-mon[22027]: Reconfiguring mon.a (unknown last config time)... 2024-08-18T23:36:11.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:10 smithi161 ceph-mon[22027]: Reconfiguring daemon mon.a on smithi078 2024-08-18T23:36:11.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:10 smithi161 ceph-mon[22027]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:11.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:10 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:11.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:10 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:11.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:10 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:11.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:10 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-18T23:36:11.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:10 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-18T23:36:11.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:10 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:11.188 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:10 smithi123 ceph-mon[21808]: Reconfiguring mon.a (unknown last config time)... 2024-08-18T23:36:11.188 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:10 smithi123 ceph-mon[21808]: Reconfiguring daemon mon.a on smithi078 2024-08-18T23:36:11.188 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:10 smithi123 ceph-mon[21808]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:11.188 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:10 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:11.188 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:10 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:11.188 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:10 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:11.188 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:10 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-18T23:36:11.188 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:10 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-18T23:36:11.188 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:10 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:11.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:10 smithi078 ceph-mon[18440]: Reconfiguring mon.a (unknown last config time)... 2024-08-18T23:36:11.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:10 smithi078 ceph-mon[18440]: Reconfiguring daemon mon.a on smithi078 2024-08-18T23:36:11.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:10 smithi078 ceph-mon[18440]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:11.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:10 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:36:11.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:10 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:11.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:10 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:11.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:10 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-18T23:36:11.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:10 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-18T23:36:11.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:10 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:11.515 DEBUG:teuthology.orchestra.run.smithi123:mgr.y> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mgr.y.service 2024-08-18T23:36:11.518 DEBUG:teuthology.orchestra.run.smithi161:mgr.z> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mgr.z.service 2024-08-18T23:36:11.520 INFO:tasks.cephadm:Deploying OSDs... 2024-08-18T23:36:11.520 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:36:11.520 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2024-08-18T23:36:11.538 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-18T23:36:11.539 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_1 2024-08-18T23:36:11.595 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-18T23:36:11.595 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T23:36:11.595 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-08-18T23:36:11.595 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T23:36:11.596 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-08-18T23:36:11.596 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-08-18 23:35:32.425905821 +0000 2024-08-18T23:36:11.596 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-08-18 23:31:34.176545153 +0000 2024-08-18T23:36:11.596 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-08-18 23:31:34.176545153 +0000 2024-08-18T23:36:11.596 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-08-18 23:31:34.176545153 +0000 2024-08-18T23:36:11.596 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-18T23:36:11.659 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:36:11 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:36:11.218+0000 7f3aa145c640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-08-18T23:36:11.661 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-08-18T23:36:11.661 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-08-18T23:36:11.662 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000198576 s, 2.6 MB/s 2024-08-18T23:36:11.663 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-18T23:36:11.720 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_2 2024-08-18T23:36:11.777 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-18T23:36:11.777 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T23:36:11.777 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-08-18T23:36:11.777 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T23:36:11.777 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-08-18T23:36:11.777 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-08-18 23:35:32.451905414 +0000 2024-08-18T23:36:11.777 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-08-18 23:31:34.523544264 +0000 2024-08-18T23:36:11.777 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-08-18 23:31:34.523544264 +0000 2024-08-18T23:36:11.777 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-08-18 23:31:34.523544264 +0000 2024-08-18T23:36:11.778 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-18T23:36:11.840 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-08-18T23:36:11.841 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-08-18T23:36:11.841 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000205261 s, 2.5 MB/s 2024-08-18T23:36:11.842 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-18T23:36:11.898 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_3 2024-08-18T23:36:11.954 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-18T23:36:11.954 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T23:36:11.954 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-08-18T23:36:11.954 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T23:36:11.955 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-08-18T23:36:11.955 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-08-18 23:35:32.477905007 +0000 2024-08-18T23:36:11.955 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-08-18 23:31:34.842543448 +0000 2024-08-18T23:36:11.955 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-08-18 23:31:34.842543448 +0000 2024-08-18T23:36:11.955 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-08-18 23:31:34.842543448 +0000 2024-08-18T23:36:11.955 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-18T23:36:12.018 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-08-18T23:36:12.018 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-08-18T23:36:12.018 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000201207 s, 2.5 MB/s 2024-08-18T23:36:12.019 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-18T23:36:12.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:11 smithi123 ceph-mon[21808]: Reconfiguring mon.b (monmap changed)... 2024-08-18T23:36:12.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:11 smithi123 ceph-mon[21808]: Reconfiguring daemon mon.b on smithi123 2024-08-18T23:36:12.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:11 smithi123 ceph-mon[21808]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi123=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:36:12.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:11 smithi123 ceph-mon[21808]: Saving service mgr spec with placement smithi078=x;smithi123=y;smithi161=z;count:3 2024-08-18T23:36:12.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:11 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:12.076 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_4 2024-08-18T23:36:12.133 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-18T23:36:12.133 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T23:36:12.133 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-08-18T23:36:12.133 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T23:36:12.133 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-08-18T23:36:12.133 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-08-18 23:35:32.503904599 +0000 2024-08-18T23:36:12.133 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-08-18 23:31:35.158542640 +0000 2024-08-18T23:36:12.133 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-08-18 23:31:35.158542640 +0000 2024-08-18T23:36:12.133 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-08-18 23:31:35.158542640 +0000 2024-08-18T23:36:12.134 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-18T23:36:12.194 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:11 smithi078 ceph-mon[18440]: Reconfiguring mon.b (monmap changed)... 2024-08-18T23:36:12.194 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:11 smithi078 ceph-mon[18440]: Reconfiguring daemon mon.b on smithi123 2024-08-18T23:36:12.194 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:11 smithi078 ceph-mon[18440]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi123=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:36:12.194 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:11 smithi078 ceph-mon[18440]: Saving service mgr spec with placement smithi078=x;smithi123=y;smithi161=z;count:3 2024-08-18T23:36:12.194 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:11 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:12.196 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-08-18T23:36:12.196 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-08-18T23:36:12.196 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000227114 s, 2.3 MB/s 2024-08-18T23:36:12.197 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-18T23:36:12.254 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-08-18T23:36:12.254 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-08-18T23:36:12.270 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-18T23:36:12.270 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-08-18T23:36:12.327 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-18T23:36:12.327 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T23:36:12.327 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-08-18T23:36:12.327 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T23:36:12.327 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-08-18T23:36:12.327 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-08-18 23:36:01.643906927 +0000 2024-08-18T23:36:12.327 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-08-18 23:31:52.511123138 +0000 2024-08-18T23:36:12.327 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-08-18 23:31:52.511123138 +0000 2024-08-18T23:36:12.327 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-08-18 23:31:52.511123138 +0000 2024-08-18T23:36:12.328 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-18T23:36:12.392 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-08-18T23:36:12.485 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-08-18T23:36:12.485 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000269075 s, 1.9 MB/s 2024-08-18T23:36:12.486 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:11 smithi161 ceph-mon[22027]: Reconfiguring mon.b (monmap changed)... 2024-08-18T23:36:12.486 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:11 smithi161 ceph-mon[22027]: Reconfiguring daemon mon.b on smithi123 2024-08-18T23:36:12.486 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:11 smithi161 ceph-mon[22027]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi123=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:36:12.486 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:11 smithi161 ceph-mon[22027]: Saving service mgr spec with placement smithi078=x;smithi123=y;smithi161=z;count:3 2024-08-18T23:36:12.486 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:11 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:12.487 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-18T23:36:12.546 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-08-18T23:36:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-18T23:36:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T23:36:12.602 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-08-18T23:36:12.602 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T23:36:12.602 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-08-18T23:36:12.602 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-08-18 23:36:01.643906927 +0000 2024-08-18T23:36:12.602 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-08-18 23:31:52.824116437 +0000 2024-08-18T23:36:12.603 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-08-18 23:31:52.824116437 +0000 2024-08-18T23:36:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-08-18 23:31:52.824116437 +0000 2024-08-18T23:36:12.603 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-18T23:36:12.666 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-08-18T23:36:12.666 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-08-18T23:36:12.666 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000231793 s, 2.2 MB/s 2024-08-18T23:36:12.668 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-18T23:36:12.724 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-08-18T23:36:12.778 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-18T23:36:12.779 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T23:36:12.779 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-08-18T23:36:12.779 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T23:36:12.779 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-08-18T23:36:12.779 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-08-18 23:36:01.644906903 +0000 2024-08-18T23:36:12.779 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-08-18 23:31:53.145109566 +0000 2024-08-18T23:36:12.779 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-08-18 23:31:53.145109566 +0000 2024-08-18T23:36:12.779 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-08-18 23:31:53.145109566 +0000 2024-08-18T23:36:12.779 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-18T23:36:12.841 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-08-18T23:36:12.842 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-08-18T23:36:12.842 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000173694 s, 2.9 MB/s 2024-08-18T23:36:12.843 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-18T23:36:12.899 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-08-18T23:36:12.954 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-18T23:36:12.954 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T23:36:12.954 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-08-18T23:36:12.954 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T23:36:12.955 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-08-18T23:36:12.955 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-08-18 23:36:01.644906903 +0000 2024-08-18T23:36:12.955 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-08-18 23:31:53.470102609 +0000 2024-08-18T23:36:12.955 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-08-18 23:31:53.470102609 +0000 2024-08-18T23:36:12.955 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-08-18 23:31:53.470102609 +0000 2024-08-18T23:36:12.955 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-18T23:36:13.018 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-08-18T23:36:13.018 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-08-18T23:36:13.018 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000181809 s, 2.8 MB/s 2024-08-18T23:36:13.019 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-18T23:36:13.076 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-18T23:36:13.076 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2024-08-18T23:36:13.094 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-18T23:36:13.094 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2024-08-18T23:36:13.151 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-18T23:36:13.153 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T23:36:13.153 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-08-18T23:36:13.153 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T23:36:13.153 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-08-18T23:36:13.153 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-08-18 23:35:52.620582659 +0000 2024-08-18T23:36:13.153 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-08-18 23:31:53.334192452 +0000 2024-08-18T23:36:13.154 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-08-18 23:31:53.334192452 +0000 2024-08-18T23:36:13.154 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-08-18 23:31:53.334192452 +0000 2024-08-18T23:36:13.154 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-18T23:36:13.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:13 smithi161 ceph-mon[22027]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:13.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:13 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:13.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:13 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:13.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:13 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:13.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:13 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-18T23:36:13.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:13 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-18T23:36:13.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:13 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:13.202 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-08-18T23:36:13.202 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-08-18T23:36:13.202 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.00036332 s, 1.4 MB/s 2024-08-18T23:36:13.203 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-18T23:36:13.261 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2024-08-18T23:36:13.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:13 smithi123 ceph-mon[21808]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:13.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:13 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:13.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:13 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:13.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:13 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:13.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:13 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-18T23:36:13.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:13 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-18T23:36:13.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:13 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:13.316 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-18T23:36:13.317 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T23:36:13.317 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-08-18T23:36:13.317 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T23:36:13.317 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-08-18T23:36:13.317 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-08-18 23:35:52.620582659 +0000 2024-08-18T23:36:13.317 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-08-18 23:31:53.662186836 +0000 2024-08-18T23:36:13.317 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-08-18 23:31:53.662186836 +0000 2024-08-18T23:36:13.317 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-08-18 23:31:53.662186836 +0000 2024-08-18T23:36:13.317 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-18T23:36:13.380 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-08-18T23:36:13.380 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-08-18T23:36:13.380 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000216304 s, 2.4 MB/s 2024-08-18T23:36:13.382 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-18T23:36:13.439 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2024-08-18T23:36:13.532 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:13 smithi078 ceph-mon[18440]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:13.532 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:13 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:13.532 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:13 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:13.532 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:13 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:13.532 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:13 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-18T23:36:13.532 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:13 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-18T23:36:13.532 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:13 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:13.547 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-18T23:36:13.547 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T23:36:13.547 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-08-18T23:36:13.547 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T23:36:13.547 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-08-18T23:36:13.548 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-08-18 23:35:52.621582637 +0000 2024-08-18T23:36:13.548 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-08-18 23:31:53.981181373 +0000 2024-08-18T23:36:13.548 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-08-18 23:31:53.981181373 +0000 2024-08-18T23:36:13.548 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-08-18 23:31:53.981181373 +0000 2024-08-18T23:36:13.548 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-18T23:36:13.612 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-08-18T23:36:13.612 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-08-18T23:36:13.612 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000235376 s, 2.2 MB/s 2024-08-18T23:36:13.613 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-18T23:36:13.670 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2024-08-18T23:36:13.726 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-18T23:36:13.726 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-18T23:36:13.726 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-08-18T23:36:13.726 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T23:36:13.726 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-08-18T23:36:13.726 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-08-18 23:35:52.621582637 +0000 2024-08-18T23:36:13.727 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-08-18 23:31:54.317175620 +0000 2024-08-18T23:36:13.727 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-08-18 23:31:54.317175620 +0000 2024-08-18T23:36:13.727 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-08-18 23:31:54.317175620 +0000 2024-08-18T23:36:13.727 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-18T23:36:13.792 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-08-18T23:36:13.792 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-08-18T23:36:13.792 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000203953 s, 2.5 MB/s 2024-08-18T23:36:13.793 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-18T23:36:13.851 INFO:tasks.cephadm:Deploying osd.0 on smithi078 with /dev/vg_nvme/lv_4... 2024-08-18T23:36:13.851 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-18T23:36:14.104 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:36:14.199 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:14 smithi078 ceph-mon[18440]: Reconfiguring mon.c (monmap changed)... 2024-08-18T23:36:14.199 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:14 smithi078 ceph-mon[18440]: Reconfiguring daemon mon.c on smithi161 2024-08-18T23:36:14.199 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:14.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:14.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:14.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:14.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:36:14.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:14.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-18T23:36:14.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-18T23:36:14.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-18T23:36:14.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:14.214 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:14 smithi161 ceph-mon[22027]: Reconfiguring mon.c (monmap changed)... 2024-08-18T23:36:14.214 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:14 smithi161 ceph-mon[22027]: Reconfiguring daemon mon.c on smithi161 2024-08-18T23:36:14.215 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:14.215 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:14.215 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:14.215 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:14.215 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:36:14.215 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:14.215 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-18T23:36:14.215 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-18T23:36:14.215 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-18T23:36:14.215 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:14.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:14 smithi123 ceph-mon[21808]: Reconfiguring mon.c (monmap changed)... 2024-08-18T23:36:14.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:14 smithi123 ceph-mon[21808]: Reconfiguring daemon mon.c on smithi161 2024-08-18T23:36:14.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:14.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:14.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:14.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:14.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:36:14.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:14.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-18T23:36:14.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-18T23:36:14.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-18T23:36:14.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:15.341 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:15 smithi078 ceph-mon[18440]: Deploying daemon mgr.z on smithi161 2024-08-18T23:36:15.341 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:15 smithi078 ceph-mon[18440]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:15.424 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:15 smithi161 ceph-mon[22027]: Deploying daemon mgr.z on smithi161 2024-08-18T23:36:15.424 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:15 smithi161 ceph-mon[22027]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:15.424 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:15 smithi161 systemd[1]: Starting Ceph mgr.z for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:36:15.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:15 smithi123 ceph-mon[21808]: Deploying daemon mgr.z on smithi161 2024-08-18T23:36:15.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:15 smithi123 ceph-mon[21808]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:15.677 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:15 smithi161 podman[23838]: 2024-08-18 23:36:15.325809744 +0000 UTC m=+0.016981592 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:36:15.677 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:15 smithi161 podman[23838]: 2024-08-18 23:36:15.452983723 +0000 UTC m=+0.144155564 container create faf5c890d84059c9d1a896940c1091d492e7fb19e6fd1b815bb761a36156bf1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240731) 2024-08-18T23:36:15.677 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:15 smithi161 podman[23838]: 2024-08-18 23:36:15.655176924 +0000 UTC m=+0.346348766 container init faf5c890d84059c9d1a896940c1091d492e7fb19e6fd1b815bb761a36156bf1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-08-18T23:36:15.678 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:15 smithi161 podman[23838]: 2024-08-18 23:36:15.65785858 +0000 UTC m=+0.349030422 container start faf5c890d84059c9d1a896940c1091d492e7fb19e6fd1b815bb761a36156bf1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-08-18T23:36:15.940 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:15 smithi161 ceph-mgr[23857]: -- 172.21.15.161:0/1049845248 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5653825670e0 con 0x565382545000 2024-08-18T23:36:15.940 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:15 smithi161 bash[23838]: faf5c890d84059c9d1a896940c1091d492e7fb19e6fd1b815bb761a36156bf1f 2024-08-18T23:36:15.940 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:15 smithi161 systemd[1]: Started Ceph mgr.z for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:36:15.940 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:15 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:15.874+0000 7fb9971bd040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-18T23:36:15.940 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:15 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:15.938+0000 7fb9971bd040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-18T23:36:16.236 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:16 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:16.080+0000 7fb9971bd040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-18T23:36:16.236 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:16 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:16.235+0000 7fb9971bd040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-18T23:36:16.390 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:36:16.416 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_4 2024-08-18T23:36:16.500 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:16 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:16.376+0000 7fb9971bd040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-18T23:36:16.500 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:16 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:16.499+0000 7fb9971bd040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-18T23:36:16.662 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:36:16.825 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:16 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:16.622+0000 7fb9971bd040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-18T23:36:16.825 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:16 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:16.678+0000 7fb9971bd040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-18T23:36:17.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:16 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:17.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:16 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:17.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:16 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:17.049 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:16 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-18T23:36:17.049 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:16 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-18T23:36:17.049 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:16 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-18T23:36:17.049 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:16 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:17.049 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:16 smithi123 ceph-mon[21808]: Deploying daemon mgr.y on smithi123 2024-08-18T23:36:17.049 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:16 smithi123 ceph-mon[21808]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:17.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.063+0000 7f1bdbb2e640 1 -- 172.21.15.78:0/882471305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bd4102b30 msgr2=0x7f1bd4102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.063+0000 7f1bdbb2e640 1 --2- 172.21.15.78:0/882471305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bd4102b30 0x7f1bd4102fb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009900 tx=0x7f1bc802a410 comp rx=0 tx=0).stop 2024-08-18T23:36:17.065 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:16 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:17.065 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:16 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:17.065 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:16 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:17.065 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:16 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-18T23:36:17.065 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:16 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-18T23:36:17.066 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:16 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-18T23:36:17.066 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:16 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:17.066 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:16 smithi078 ceph-mon[18440]: Deploying daemon mgr.y on smithi123 2024-08-18T23:36:17.066 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:16 smithi078 ceph-mon[18440]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:17.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.063+0000 7f1bdbb2e640 1 -- 172.21.15.78:0/882471305 shutdown_connections 2024-08-18T23:36:17.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.063+0000 7f1bdbb2e640 1 --2- 172.21.15.78:0/882471305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bd41034f0 0x7f1bd4109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:17.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.063+0000 7f1bdbb2e640 1 --2- 172.21.15.78:0/882471305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bd4102b30 0x7f1bd4102fb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:17.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.063+0000 7f1bdbb2e640 1 --2- 172.21.15.78:0/882471305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bd4101930 0x7f1bd4101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:17.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.063+0000 7f1bdbb2e640 1 -- 172.21.15.78:0/882471305 >> 172.21.15.78:0/882471305 conn(0x7f1bd40fd0c0 msgr2=0x7f1bd40ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:36:17.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.063+0000 7f1bdbb2e640 1 -- 172.21.15.78:0/882471305 shutdown_connections 2024-08-18T23:36:17.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.063+0000 7f1bdbb2e640 1 -- 172.21.15.78:0/882471305 wait complete. 2024-08-18T23:36:17.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.064+0000 7f1bdbb2e640 1 Processor -- start 2024-08-18T23:36:17.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.064+0000 7f1bdbb2e640 1 -- start start 2024-08-18T23:36:17.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.064+0000 7f1bdbb2e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bd4101930 0x7f1bd4197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:17.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.064+0000 7f1bdbb2e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bd4102b30 0x7f1bd4198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:17.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.064+0000 7f1bdbb2e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bd41034f0 0x7f1bd419f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:17.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.064+0000 7f1bdbb2e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd410d5b0 con 0x7f1bd4102b30 2024-08-18T23:36:17.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.064+0000 7f1bdbb2e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd410d430 con 0x7f1bd41034f0 2024-08-18T23:36:17.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.064+0000 7f1bdbb2e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd410d730 con 0x7f1bd4101930 2024-08-18T23:36:17.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.064+0000 7f1bd90c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bd4102b30 0x7f1bd4198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:17.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.064+0000 7f1bd98ca640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bd4101930 0x7f1bd4197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:17.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.064+0000 7f1bd90c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bd4102b30 0x7f1bd4198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35744/0 (socket says 172.21.15.78:35744) 2024-08-18T23:36:17.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.064+0000 7f1bd90c9640 1 -- 172.21.15.78:0/2595802010 learned_addr learned my addr 172.21.15.78:0/2595802010 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:36:17.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.064+0000 7f1bda0cb640 1 --2- 172.21.15.78:0/2595802010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bd41034f0 0x7f1bd419f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:17.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.065+0000 7f1bd90c9640 1 -- 172.21.15.78:0/2595802010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bd4101930 msgr2=0x7f1bd4197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:17.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.065+0000 7f1bd90c9640 1 --2- 172.21.15.78:0/2595802010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bd4101930 0x7f1bd4197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:17.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.065+0000 7f1bd90c9640 1 -- 172.21.15.78:0/2595802010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bd41034f0 msgr2=0x7f1bd419f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:17.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.065+0000 7f1bd90c9640 1 --2- 172.21.15.78:0/2595802010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bd41034f0 0x7f1bd419f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:17.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.065+0000 7f1bd90c9640 1 -- 172.21.15.78:0/2595802010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd419f870 con 0x7f1bd4102b30 2024-08-18T23:36:17.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.065+0000 7f1bd98ca640 1 --2- 172.21.15.78:0/2595802010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bd4101930 0x7f1bd4197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:36:17.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.065+0000 7f1bda0cb640 1 --2- 172.21.15.78:0/2595802010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bd41034f0 0x7f1bd419f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:36:17.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.065+0000 7f1bd90c9640 1 --2- 172.21.15.78:0/2595802010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bd4102b30 0x7f1bd4198190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009230 tx=0x7f1bc8031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:36:17.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.065+0000 7f1bd2ffd640 1 -- 172.21.15.78:0/2595802010 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc803f030 con 0x7f1bd4102b30 2024-08-18T23:36:17.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.065+0000 7f1bd2ffd640 1 -- 172.21.15.78:0/2595802010 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bc8003ad0 con 0x7f1bd4102b30 2024-08-18T23:36:17.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.065+0000 7f1bd2ffd640 1 -- 172.21.15.78:0/2595802010 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8035070 con 0x7f1bd4102b30 2024-08-18T23:36:17.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.065+0000 7f1bdbb2e640 1 -- 172.21.15.78:0/2595802010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd419fb00 con 0x7f1bd4102b30 2024-08-18T23:36:17.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.065+0000 7f1bdbb2e640 1 -- 172.21.15.78:0/2595802010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd41a0030 con 0x7f1bd4102b30 2024-08-18T23:36:17.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.066+0000 7f1bd2ffd640 1 -- 172.21.15.78:0/2595802010 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1bc8042030 con 0x7f1bd4102b30 2024-08-18T23:36:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.066+0000 7f1bd2ffd640 1 --2- 172.21.15.78:0/2595802010 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1bbc039290 0x7f1bbc03b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.066+0000 7f1bdbb2e640 1 -- 172.21.15.78:0/2595802010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd4108cd0 con 0x7f1bd4102b30 2024-08-18T23:36:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.066+0000 7f1bd2ffd640 1 -- 172.21.15.78:0/2595802010 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1bc806c900 con 0x7f1bd4102b30 2024-08-18T23:36:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.066+0000 7f1bd98ca640 1 --2- 172.21.15.78:0/2595802010 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1bbc039290 0x7f1bbc03b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.067+0000 7f1bd98ca640 1 --2- 172.21.15.78:0/2595802010 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1bbc039290 0x7f1bbc03b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4199130 tx=0x7f1bcc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:36:17.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.071+0000 7f1bd2ffd640 1 -- 172.21.15.78:0/2595802010 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1bc804cd90 con 0x7f1bd4102b30 2024-08-18T23:36:17.106 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:16 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:17.107 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:16 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:17.107 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:16 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:17.107 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:16 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-18T23:36:17.107 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:16 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-18T23:36:17.107 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:16 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-18T23:36:17.107 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:16 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:17.107 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:16 smithi161 ceph-mon[22027]: Deploying daemon mgr.y on smithi123 2024-08-18T23:36:17.107 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:16 smithi161 ceph-mon[22027]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:17.108 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:16 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:16.977+0000 7fb9971bd040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-18T23:36:17.108 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:17 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:17.037+0000 7fb9971bd040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-18T23:36:17.108 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:17 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:17.105+0000 7fb9971bd040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-18T23:36:17.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:17.254+0000 7f1bdbb2e640 1 -- 172.21.15.78:0/2595802010 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1bd4062360 con 0x7f1bbc039290 2024-08-18T23:36:17.377 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:17 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:17.252+0000 7fb9971bd040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-18T23:36:17.378 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:17 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:17.311+0000 7fb9971bd040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-18T23:36:17.378 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:17 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:17.376+0000 7fb9971bd040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-18T23:36:17.677 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:17 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:17.558+0000 7fb9971bd040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-18T23:36:17.677 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:17 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:17.617+0000 7fb9971bd040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-18T23:36:17.677 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:17 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:17.676+0000 7fb9971bd040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-18T23:36:17.798 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:17 smithi123 podman[23165]: 2024-08-18 23:36:17.729882886 +0000 UTC m=+0.120216301 container create 4ec69d36f5b08888ad749144b9dca9c04e2bb6dcf7de611ee25c6695bb3a3fb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-08-18T23:36:17.927 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:17 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:17.742+0000 7fb9971bd040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-18T23:36:17.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:17 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:36:17.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:17 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:36:17.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:17 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:17.942 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:17 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:36:17.942 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:17 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:36:17.942 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:17 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:18.150 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:17 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:36:18.150 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:17 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:36:18.150 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:17 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:18.151 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:17 smithi123 podman[23165]: 2024-08-18 23:36:17.923349021 +0000 UTC m=+0.313682374 container init 4ec69d36f5b08888ad749144b9dca9c04e2bb6dcf7de611ee25c6695bb3a3fb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-08-18T23:36:18.151 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:17 smithi123 podman[23165]: 2024-08-18 23:36:17.926218009 +0000 UTC m=+0.316551364 container start 4ec69d36f5b08888ad749144b9dca9c04e2bb6dcf7de611ee25c6695bb3a3fb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-08-18T23:36:18.151 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:17 smithi123 ceph-mgr[23184]: -- 172.21.15.123:0/521880831 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f83d63f0e0 con 0x55f83d61d400 2024-08-18T23:36:18.151 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:17 smithi123 bash[23165]: 4ec69d36f5b08888ad749144b9dca9c04e2bb6dcf7de611ee25c6695bb3a3fb2 2024-08-18T23:36:18.151 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:17 smithi123 systemd[1]: Started Ceph mgr.y for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:36:18.427 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:17 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:17.953+0000 7fb9971bd040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-18T23:36:18.427 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:18 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:18.011+0000 7fb9971bd040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-18T23:36:18.546 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:18 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:18.149+0000 7fd0b30ca040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-18T23:36:18.546 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:18 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:18.218+0000 7fd0b30ca040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-18T23:36:18.546 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:18 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:18.366+0000 7fd0b30ca040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-18T23:36:18.797 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:18 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:18.545+0000 7fd0b30ca040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-18T23:36:18.797 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:18 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:18.692+0000 7fd0b30ca040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-18T23:36:18.892 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:18 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:18.571+0000 7fb9971bd040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-18T23:36:18.893 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:18 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:18.891+0000 7fb9971bd040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-18T23:36:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:19 smithi161 ceph-mon[22027]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:36:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:19 smithi161 ceph-mon[22027]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:19.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:19.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:19.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:19 smithi078 ceph-mon[18440]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:36:19.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:19 smithi078 ceph-mon[18440]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:19.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:19.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:19.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:19.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:19.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:19.297 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:18 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:18.817+0000 7fd0b30ca040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-18T23:36:19.297 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:18 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:18.956+0000 7fd0b30ca040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-18T23:36:19.298 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:19 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:19.013+0000 7fd0b30ca040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-18T23:36:19.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:19 smithi123 ceph-mon[21808]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:36:19.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:19 smithi123 ceph-mon[21808]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:19.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:19.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:19.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:19.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:19.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:19.606 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:19 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:19.328+0000 7fd0b30ca040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-18T23:36:19.606 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:19 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:19.392+0000 7fd0b30ca040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-18T23:36:19.606 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:19 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:19.459+0000 7fd0b30ca040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-18T23:36:19.606 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:19 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:19.605+0000 7fd0b30ca040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-18T23:36:19.677 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:19 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:19.324+0000 7fb9971bd040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-18T23:36:19.677 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:19 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:19.383+0000 7fb9971bd040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-18T23:36:19.677 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:36:19 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z[23853]: 2024-08-18T23:36:19.445+0000 7fb9971bd040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-18T23:36:19.909 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:19 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:19.663+0000 7fd0b30ca040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-18T23:36:19.910 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:19 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:19.727+0000 7fd0b30ca040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-18T23:36:19.910 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:19 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:19.908+0000 7fd0b30ca040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-18T23:36:20.266 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:19 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:19.968+0000 7fd0b30ca040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-18T23:36:20.266 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:20.026+0000 7fd0b30ca040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-18T23:36:20.267 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:20.092+0000 7fd0b30ca040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-18T23:36:20.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:20.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:20.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:20.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:20.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:20.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:36:20.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:20.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:20 smithi078 ceph-mon[18440]: Reconfiguring mgr.x (unknown last config time)... 2024-08-18T23:36:20.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-18T23:36:20.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-18T23:36:20.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:20.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:20 smithi078 ceph-mon[18440]: Reconfiguring daemon mgr.x on smithi078 2024-08-18T23:36:20.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:20 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1244342994' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4de1c78-6ebc-49a9-b9c5-85d43535fab7"}]: dispatch 2024-08-18T23:36:20.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:20 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1244342994' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4de1c78-6ebc-49a9-b9c5-85d43535fab7"}]': finished 2024-08-18T23:36:20.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:20 smithi078 ceph-mon[18440]: osdmap e5: 1 total, 0 up, 1 in 2024-08-18T23:36:20.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-18T23:36:20.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:20 smithi078 ceph-mon[18440]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:20.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:20 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/4178669627' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:36:20.547 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:20.301+0000 7fd0b30ca040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-18T23:36:20.548 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:20.359+0000 7fd0b30ca040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-18T23:36:20.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:20.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:20.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:20.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:20.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:20.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:36:20.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:20.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-mon[21808]: Reconfiguring mgr.x (unknown last config time)... 2024-08-18T23:36:20.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-18T23:36:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-18T23:36:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-mon[21808]: Reconfiguring daemon mgr.x on smithi078 2024-08-18T23:36:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1244342994' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4de1c78-6ebc-49a9-b9c5-85d43535fab7"}]: dispatch 2024-08-18T23:36:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1244342994' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4de1c78-6ebc-49a9-b9c5-85d43535fab7"}]': finished 2024-08-18T23:36:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-mon[21808]: osdmap e5: 1 total, 0 up, 1 in 2024-08-18T23:36:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-18T23:36:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-mon[21808]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/4178669627' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:36:20.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:20.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:20.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:20.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:20.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:36:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:20 smithi161 ceph-mon[22027]: Reconfiguring mgr.x (unknown last config time)... 2024-08-18T23:36:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-18T23:36:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-18T23:36:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:20 smithi161 ceph-mon[22027]: Reconfiguring daemon mgr.x on smithi078 2024-08-18T23:36:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:20 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1244342994' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4de1c78-6ebc-49a9-b9c5-85d43535fab7"}]: dispatch 2024-08-18T23:36:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:20 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1244342994' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4de1c78-6ebc-49a9-b9c5-85d43535fab7"}]': finished 2024-08-18T23:36:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:20 smithi161 ceph-mon[22027]: osdmap e5: 1 total, 0 up, 1 in 2024-08-18T23:36:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-18T23:36:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:20 smithi161 ceph-mon[22027]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:20.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:20 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/4178669627' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:36:21.176 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:20 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:20.871+0000 7fd0b30ca040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-18T23:36:21.176 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:21 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:21.175+0000 7fd0b30ca040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-18T23:36:21.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:21.276+0000 7f1bd2ffd640 1 -- 172.21.15.78:0/2595802010 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f1bc8003c80 con 0x7f1bd4102b30 2024-08-18T23:36:21.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:21 smithi123 ceph-mon[21808]: Standby manager daemon z started 2024-08-18T23:36:21.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:21 smithi123 ceph-mon[21808]: from='mgr.? 172.21.15.161:0/2208713462' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-18T23:36:21.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:21 smithi123 ceph-mon[21808]: from='mgr.? 172.21.15.161:0/2208713462' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-18T23:36:21.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:21 smithi123 ceph-mon[21808]: from='mgr.? 172.21.15.161:0/2208713462' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-18T23:36:21.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:21 smithi123 ceph-mon[21808]: from='mgr.? 172.21.15.161:0/2208713462' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-18T23:36:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:21 smithi161 ceph-mon[22027]: Standby manager daemon z started 2024-08-18T23:36:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:21 smithi161 ceph-mon[22027]: from='mgr.? 172.21.15.161:0/2208713462' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-18T23:36:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:21 smithi161 ceph-mon[22027]: from='mgr.? 172.21.15.161:0/2208713462' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-18T23:36:21.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:21 smithi161 ceph-mon[22027]: from='mgr.? 172.21.15.161:0/2208713462' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-18T23:36:21.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:21 smithi161 ceph-mon[22027]: from='mgr.? 172.21.15.161:0/2208713462' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-18T23:36:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:21 smithi078 ceph-mon[18440]: Standby manager daemon z started 2024-08-18T23:36:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:21 smithi078 ceph-mon[18440]: from='mgr.? 172.21.15.161:0/2208713462' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-18T23:36:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:21 smithi078 ceph-mon[18440]: from='mgr.? 172.21.15.161:0/2208713462' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-18T23:36:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:21 smithi078 ceph-mon[18440]: from='mgr.? 172.21.15.161:0/2208713462' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-18T23:36:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:21 smithi078 ceph-mon[18440]: from='mgr.? 172.21.15.161:0/2208713462' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-18T23:36:22.047 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:21 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:21.584+0000 7fd0b30ca040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-18T23:36:22.047 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:21 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:21.639+0000 7fd0b30ca040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-18T23:36:22.048 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:36:21 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y[23180]: 2024-08-18T23:36:21.696+0000 7fd0b30ca040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-18T23:36:22.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:22 smithi123 ceph-mon[21808]: mgrmap e14: x(active, since 59s), standbys: z 2024-08-18T23:36:22.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-18T23:36:22.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:22 smithi123 ceph-mon[21808]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:22.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:22.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:22 smithi161 ceph-mon[22027]: mgrmap e14: x(active, since 59s), standbys: z 2024-08-18T23:36:22.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-18T23:36:22.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:22 smithi161 ceph-mon[22027]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:22.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:22.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:22 smithi078 ceph-mon[18440]: mgrmap e14: x(active, since 59s), standbys: z 2024-08-18T23:36:22.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-18T23:36:22.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:22 smithi078 ceph-mon[18440]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:22.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:23.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:23.289+0000 7f1bd2ffd640 1 -- 172.21.15.78:0/2595802010 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f1bc80a0ac0 con 0x7f1bd4102b30 2024-08-18T23:36:23.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:23 smithi123 ceph-mon[21808]: Standby manager daemon y started 2024-08-18T23:36:23.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:23 smithi123 ceph-mon[21808]: from='mgr.? 172.21.15.123:0/1502296588' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-18T23:36:23.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:23 smithi123 ceph-mon[21808]: from='mgr.? 172.21.15.123:0/1502296588' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-18T23:36:23.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:23 smithi123 ceph-mon[21808]: from='mgr.? 172.21.15.123:0/1502296588' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-18T23:36:23.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:23 smithi123 ceph-mon[21808]: from='mgr.? 172.21.15.123:0/1502296588' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-18T23:36:23.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:23 smithi078 ceph-mon[18440]: Standby manager daemon y started 2024-08-18T23:36:23.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:23 smithi078 ceph-mon[18440]: from='mgr.? 172.21.15.123:0/1502296588' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-18T23:36:23.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:23 smithi078 ceph-mon[18440]: from='mgr.? 172.21.15.123:0/1502296588' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-18T23:36:23.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:23 smithi078 ceph-mon[18440]: from='mgr.? 172.21.15.123:0/1502296588' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-18T23:36:23.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:23 smithi078 ceph-mon[18440]: from='mgr.? 172.21.15.123:0/1502296588' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-18T23:36:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:23 smithi161 ceph-mon[22027]: Standby manager daemon y started 2024-08-18T23:36:23.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:23 smithi161 ceph-mon[22027]: from='mgr.? 172.21.15.123:0/1502296588' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-18T23:36:23.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:23 smithi161 ceph-mon[22027]: from='mgr.? 172.21.15.123:0/1502296588' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-18T23:36:23.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:23 smithi161 ceph-mon[22027]: from='mgr.? 172.21.15.123:0/1502296588' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-18T23:36:23.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:23 smithi161 ceph-mon[22027]: from='mgr.? 172.21.15.123:0/1502296588' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-18T23:36:24.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:24 smithi123 ceph-mon[21808]: mgrmap e15: x(active, since 61s), standbys: z, y 2024-08-18T23:36:24.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:24 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-18T23:36:24.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:24 smithi123 ceph-mon[21808]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:24.553 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:24 smithi078 ceph-mon[18440]: mgrmap e15: x(active, since 61s), standbys: z, y 2024-08-18T23:36:24.553 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:24 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-18T23:36:24.553 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:24 smithi078 ceph-mon[18440]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:24 smithi161 ceph-mon[22027]: mgrmap e15: x(active, since 61s), standbys: z, y 2024-08-18T23:36:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:24 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-18T23:36:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:24 smithi161 ceph-mon[22027]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:25.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:25 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:25.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:25 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:25.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:25 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:25.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:25 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:25.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:25 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:36:25.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:25 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:26.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:25 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:26.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:25 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:26.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:25 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:26.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:25 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:26.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:25 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:36:26.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:25 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:26.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:25 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:26.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:25 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:26.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:25 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:26.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:25 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:26.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:25 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:36:26.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:25 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:27.048 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:26 smithi078 ceph-mon[18440]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:27.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:26 smithi161 ceph-mon[22027]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:27.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:26 smithi123 ceph-mon[21808]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:28.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:27 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-08-18T23:36:28.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:27 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:28.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:27 smithi161 ceph-mon[22027]: Deploying daemon osd.0 on smithi078 2024-08-18T23:36:28.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:27 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-08-18T23:36:28.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:27 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:28.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:27 smithi078 ceph-mon[18440]: Deploying daemon osd.0 on smithi078 2024-08-18T23:36:28.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:27 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-08-18T23:36:28.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:27 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:28.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:27 smithi123 ceph-mon[21808]: Deploying daemon osd.0 on smithi078 2024-08-18T23:36:29.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:28 smithi161 ceph-mon[22027]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:29.198 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:28 smithi123 ceph-mon[21808]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:29.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:28 smithi078 ceph-mon[18440]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:31.327 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:31 smithi078 ceph-mon[18440]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:31 smithi161 ceph-mon[22027]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:31.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:31 smithi123 ceph-mon[21808]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:32.368 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:32 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:32.368 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:32 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:32.368 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:32 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:32.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:32 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:32.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:32 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:32.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:32 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:32.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:32 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:32.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:32 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:32.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:32 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:33.294 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:33 smithi078 ceph-mon[18440]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:33.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:33 smithi161 ceph-mon[22027]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:33.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:33 smithi123 ceph-mon[21808]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:35.141+0000 7f1bd2ffd640 1 -- 172.21.15.78:0/2595802010 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1bd4062360 con 0x7f1bbc039290 2024-08-18T23:36:35.143 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 0 on host 'smithi078' 2024-08-18T23:36:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:35.143+0000 7f1bdbb2e640 1 -- 172.21.15.78:0/2595802010 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1bbc039290 msgr2=0x7f1bbc03b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:35.143+0000 7f1bdbb2e640 1 --2- 172.21.15.78:0/2595802010 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1bbc039290 0x7f1bbc03b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4199130 tx=0x7f1bcc005f90 comp rx=0 tx=0).stop 2024-08-18T23:36:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:35.143+0000 7f1bdbb2e640 1 -- 172.21.15.78:0/2595802010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bd4102b30 msgr2=0x7f1bd4198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:35.143+0000 7f1bdbb2e640 1 --2- 172.21.15.78:0/2595802010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bd4102b30 0x7f1bd4198190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009230 tx=0x7f1bc8031b60 comp rx=0 tx=0).stop 2024-08-18T23:36:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:35.143+0000 7f1bdbb2e640 1 -- 172.21.15.78:0/2595802010 shutdown_connections 2024-08-18T23:36:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:35.143+0000 7f1bdbb2e640 1 --2- 172.21.15.78:0/2595802010 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1bbc039290 0x7f1bbc03b750 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:35.143+0000 7f1bdbb2e640 1 --2- 172.21.15.78:0/2595802010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bd41034f0 0x7f1bd419f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:35.143+0000 7f1bdbb2e640 1 --2- 172.21.15.78:0/2595802010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bd4102b30 0x7f1bd4198190 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:35.143+0000 7f1bdbb2e640 1 --2- 172.21.15.78:0/2595802010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bd4101930 0x7f1bd4197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:35.143+0000 7f1bdbb2e640 1 -- 172.21.15.78:0/2595802010 >> 172.21.15.78:0/2595802010 conn(0x7f1bd40fd0c0 msgr2=0x7f1bd4105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:36:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:35.143+0000 7f1bdbb2e640 1 -- 172.21.15.78:0/2595802010 shutdown_connections 2024-08-18T23:36:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:36:35.143+0000 7f1bdbb2e640 1 -- 172.21.15.78:0/2595802010 wait complete. 2024-08-18T23:36:35.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:35 smithi078 ceph-mon[18440]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:35.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:35 smithi161 ceph-mon[22027]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:35.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:35 smithi123 ceph-mon[21808]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:35.813 DEBUG:teuthology.orchestra.run.smithi078:osd.0> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.0.service 2024-08-18T23:36:35.857 INFO:tasks.cephadm:Deploying osd.1 on smithi123 with /dev/vg_nvme/lv_4... 2024-08-18T23:36:35.857 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-18T23:36:36.110 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.b/config 2024-08-18T23:36:36.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:36 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:36.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:36 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:36.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:36 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:36.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:36 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:36.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:36 smithi161 ceph-mon[22027]: from='osd.0 [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-18T23:36:36.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:36 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:36.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:36 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:36.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:36 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:36.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:36 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:36.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:36 smithi123 ceph-mon[21808]: from='osd.0 [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-18T23:36:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:36 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:36 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:36 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:36 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:36.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:36 smithi078 ceph-mon[18440]: from='osd.0 [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-18T23:36:37.167 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:37 smithi078 ceph-mon[18440]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:37.167 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:37 smithi078 ceph-mon[18440]: from='osd.0 [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-18T23:36:37.167 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:37 smithi078 ceph-mon[18440]: osdmap e6: 1 total, 0 up, 1 in 2024-08-18T23:36:37.167 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:37 smithi078 ceph-mon[18440]: from='osd.0 [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-08-18T23:36:37.167 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:37 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-18T23:36:37.167 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:36:37 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-0[32351]: 2024-08-18T23:36:37.159+0000 7fcfdfade640 -1 osd.0 0 waiting for initial osdmap 2024-08-18T23:36:37.167 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:36:37 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-0[32351]: 2024-08-18T23:36:37.164+0000 7fcfdb906640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T23:36:37.244 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:37 smithi123 ceph-mon[21808]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:37.245 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:37 smithi123 ceph-mon[21808]: from='osd.0 [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-18T23:36:37.245 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:37 smithi123 ceph-mon[21808]: osdmap e6: 1 total, 0 up, 1 in 2024-08-18T23:36:37.245 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:37 smithi123 ceph-mon[21808]: from='osd.0 [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-08-18T23:36:37.245 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:37 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-18T23:36:37.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:37 smithi161 ceph-mon[22027]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:37.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:37 smithi161 ceph-mon[22027]: from='osd.0 [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-18T23:36:37.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:37 smithi161 ceph-mon[22027]: osdmap e6: 1 total, 0 up, 1 in 2024-08-18T23:36:37.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:37 smithi161 ceph-mon[22027]: from='osd.0 [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-08-18T23:36:37.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:37 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-18T23:36:38.277 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-18T23:36:38.302 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_4 2024-08-18T23:36:38.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:38 smithi161 ceph-mon[22027]: from='osd.0 [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-08-18T23:36:38.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:38 smithi161 ceph-mon[22027]: osdmap e7: 1 total, 0 up, 1 in 2024-08-18T23:36:38.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:38 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-18T23:36:38.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:38 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-18T23:36:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:38 smithi078 ceph-mon[18440]: from='osd.0 [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-08-18T23:36:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:38 smithi078 ceph-mon[18440]: osdmap e7: 1 total, 0 up, 1 in 2024-08-18T23:36:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-18T23:36:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-18T23:36:38.545 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.b/config 2024-08-18T23:36:38.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:38 smithi123 ceph-mon[21808]: from='osd.0 [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-08-18T23:36:38.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:38 smithi123 ceph-mon[21808]: osdmap e7: 1 total, 0 up, 1 in 2024-08-18T23:36:38.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:38 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-18T23:36:38.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:38 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-18T23:36:38.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.975+0000 7f002fc40640 1 -- 172.21.15.123:0/3563852093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0028102400 msgr2=0x7f0028108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:38.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.975+0000 7f002fc40640 1 --2- 172.21.15.123:0/3563852093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0028102400 0x7f0028108c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0020009ca0 tx=0x7f002002a8c0 comp rx=0 tx=0).stop 2024-08-18T23:36:38.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.975+0000 7f002fc40640 1 -- 172.21.15.123:0/3563852093 shutdown_connections 2024-08-18T23:36:38.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.975+0000 7f002fc40640 1 --2- 172.21.15.123:0/3563852093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0028102400 0x7f0028108c90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:38.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.975+0000 7f002fc40640 1 --2- 172.21.15.123:0/3563852093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0028101a40 0x7f0028101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:38.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.975+0000 7f002fc40640 1 --2- 172.21.15.123:0/3563852093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0028100840 0x7f0028100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:38.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.975+0000 7f002fc40640 1 -- 172.21.15.123:0/3563852093 >> 172.21.15.123:0/3563852093 conn(0x7f00280fbfd0 msgr2=0x7f00280fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:36:38.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.975+0000 7f002fc40640 1 -- 172.21.15.123:0/3563852093 shutdown_connections 2024-08-18T23:36:38.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.975+0000 7f002fc40640 1 -- 172.21.15.123:0/3563852093 wait complete. 2024-08-18T23:36:38.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.975+0000 7f002fc40640 1 Processor -- start 2024-08-18T23:36:38.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.975+0000 7f002fc40640 1 -- start start 2024-08-18T23:36:38.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002fc40640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0028100840 0x7f0028196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:38.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002fc40640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0028101a40 0x7f0028197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:38.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002fc40640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0028102400 0x7f002819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:38.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002fc40640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f002810c4c0 con 0x7f0028102400 2024-08-18T23:36:38.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002fc40640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f002810c340 con 0x7f0028101a40 2024-08-18T23:36:38.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002fc40640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f002810c640 con 0x7f0028100840 2024-08-18T23:36:38.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002d1db640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0028101a40 0x7f0028197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:38.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002d1db640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0028101a40 0x7f0028197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50106/0 (socket says 172.21.15.123:50106) 2024-08-18T23:36:38.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002d1db640 1 -- 172.21.15.123:0/1497808152 learned_addr learned my addr 172.21.15.123:0/1497808152 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-08-18T23:36:38.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002e1dd640 1 --2- 172.21.15.123:0/1497808152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0028102400 0x7f002819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:38.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002d9dc640 1 --2- 172.21.15.123:0/1497808152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0028100840 0x7f0028196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:38.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002d1db640 1 -- 172.21.15.123:0/1497808152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0028100840 msgr2=0x7f0028196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:38.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002d1db640 1 --2- 172.21.15.123:0/1497808152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0028100840 0x7f0028196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:38.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002d1db640 1 -- 172.21.15.123:0/1497808152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0028102400 msgr2=0x7f002819e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:38.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002d1db640 1 --2- 172.21.15.123:0/1497808152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0028102400 0x7f002819e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:38.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002d1db640 1 -- 172.21.15.123:0/1497808152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f002819e720 con 0x7f0028101a40 2024-08-18T23:36:38.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002e1dd640 1 --2- 172.21.15.123:0/1497808152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0028102400 0x7f002819e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:36:38.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002d1db640 1 --2- 172.21.15.123:0/1497808152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0028101a40 0x7f0028197030 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f00180078f0 tx=0x7f0018007db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:36:38.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f001effd640 1 -- 172.21.15.123:0/1497808152 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0018019020 con 0x7f0028101a40 2024-08-18T23:36:38.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002fc40640 1 -- 172.21.15.123:0/1497808152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f002806a520 con 0x7f0028101a40 2024-08-18T23:36:38.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f001effd640 1 -- 172.21.15.123:0/1497808152 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0018005500 con 0x7f0028101a40 2024-08-18T23:36:38.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f001effd640 1 -- 172.21.15.123:0/1497808152 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001800e890 con 0x7f0028101a40 2024-08-18T23:36:38.980 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.976+0000 7f002fc40640 1 -- 172.21.15.123:0/1497808152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f002806aa10 con 0x7f0028101a40 2024-08-18T23:36:38.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.977+0000 7f002fc40640 1 -- 172.21.15.123:0/1497808152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0028100c40 con 0x7f0028101a40 2024-08-18T23:36:38.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.979+0000 7f001effd640 1 -- 172.21.15.123:0/1497808152 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f00180046c0 con 0x7f0028101a40 2024-08-18T23:36:38.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.979+0000 7f001effd640 1 --2- 172.21.15.123:0/1497808152 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f00140a2b20 0x7f00140a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:38.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.979+0000 7f001effd640 1 -- 172.21.15.123:0/1497808152 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f00180cf260 con 0x7f0028101a40 2024-08-18T23:36:38.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.980+0000 7f002d9dc640 1 --2- 172.21.15.123:0/1497808152 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f00140a2b20 0x7f00140a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:38.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.980+0000 7f002d9dc640 1 --2- 172.21.15.123:0/1497808152 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f00140a2b20 0x7f00140a4fe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0024005fd0 tx=0x7f0024005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:36:38.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:38.983+0000 7f001effd640 1 -- 172.21.15.123:0/1497808152 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0018023020 con 0x7f0028101a40 2024-08-18T23:36:39.159 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:39.158+0000 7f002fc40640 1 -- 172.21.15.123:0/1497808152 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f002806ac70 con 0x7f00140a2b20 2024-08-18T23:36:39.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: purged_snaps scrub starts 2024-08-18T23:36:39.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: purged_snaps scrub ok 2024-08-18T23:36:39.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:39.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-18T23:36:39.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: osd.0 [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311] boot 2024-08-18T23:36:39.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: osdmap e8: 1 total, 1 up, 1 in 2024-08-18T23:36:39.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-18T23:36:39.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:39.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:39.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:36:39.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:39.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:39.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:36:39.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:39.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:39.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:39.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:36:39.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:39.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:36:39.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:36:39.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:39.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: purged_snaps scrub starts 2024-08-18T23:36:39.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: purged_snaps scrub ok 2024-08-18T23:36:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-18T23:36:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: osd.0 [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311] boot 2024-08-18T23:36:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: osdmap e8: 1 total, 1 up, 1 in 2024-08-18T23:36:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-18T23:36:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:36:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:36:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:39.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:39.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:39.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:36:39.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:39.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:36:39.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:36:39.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: purged_snaps scrub starts 2024-08-18T23:36:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: purged_snaps scrub ok 2024-08-18T23:36:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-18T23:36:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-18T23:36:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: osd.0 [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311] boot 2024-08-18T23:36:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: osdmap e8: 1 total, 1 up, 1 in 2024-08-18T23:36:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-18T23:36:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:36:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:36:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:39.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:39.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:36:39.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:39.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:36:39.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:36:39.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:40.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:40 smithi161 ceph-mon[22027]: Detected new or changed devices on smithi078 2024-08-18T23:36:40.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:40 smithi161 ceph-mon[22027]: Adjusting osd_memory_target on smithi078 to 17088M 2024-08-18T23:36:40.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:40 smithi161 ceph-mon[22027]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:36:40.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:40 smithi161 ceph-mon[22027]: osdmap e9: 1 total, 1 up, 1 in 2024-08-18T23:36:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:40 smithi078 ceph-mon[18440]: Detected new or changed devices on smithi078 2024-08-18T23:36:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:40 smithi078 ceph-mon[18440]: Adjusting osd_memory_target on smithi078 to 17088M 2024-08-18T23:36:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:40 smithi078 ceph-mon[18440]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:36:40.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:40 smithi078 ceph-mon[18440]: osdmap e9: 1 total, 1 up, 1 in 2024-08-18T23:36:40.475 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:40 smithi123 ceph-mon[21808]: Detected new or changed devices on smithi078 2024-08-18T23:36:40.475 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:40 smithi123 ceph-mon[21808]: Adjusting osd_memory_target on smithi078 to 17088M 2024-08-18T23:36:40.475 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:40 smithi123 ceph-mon[21808]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:36:40.475 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:40 smithi123 ceph-mon[21808]: osdmap e9: 1 total, 1 up, 1 in 2024-08-18T23:36:41.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:41 smithi161 ceph-mon[22027]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:41 smithi078 ceph-mon[18440]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:41.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:41 smithi123 ceph-mon[21808]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:42.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:42 smithi123 ceph-mon[21808]: from='client.? 172.21.15.123:0/3784782559' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f8e0c3d-472b-4230-ad23-2b0a34861251"}]: dispatch 2024-08-18T23:36:42.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:42 smithi123 ceph-mon[21808]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f8e0c3d-472b-4230-ad23-2b0a34861251"}]: dispatch 2024-08-18T23:36:42.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:42 smithi123 ceph-mon[21808]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f8e0c3d-472b-4230-ad23-2b0a34861251"}]': finished 2024-08-18T23:36:42.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:42 smithi123 ceph-mon[21808]: osdmap e10: 2 total, 1 up, 2 in 2024-08-18T23:36:42.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:42 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-18T23:36:42.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:42 smithi123 ceph-mon[21808]: from='client.? 172.21.15.123:0/1088689979' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:36:42.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:42 smithi161 ceph-mon[22027]: from='client.? 172.21.15.123:0/3784782559' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f8e0c3d-472b-4230-ad23-2b0a34861251"}]: dispatch 2024-08-18T23:36:42.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:42 smithi161 ceph-mon[22027]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f8e0c3d-472b-4230-ad23-2b0a34861251"}]: dispatch 2024-08-18T23:36:42.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:42 smithi161 ceph-mon[22027]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f8e0c3d-472b-4230-ad23-2b0a34861251"}]': finished 2024-08-18T23:36:42.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:42 smithi161 ceph-mon[22027]: osdmap e10: 2 total, 1 up, 2 in 2024-08-18T23:36:42.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:42 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-18T23:36:42.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:42 smithi161 ceph-mon[22027]: from='client.? 172.21.15.123:0/1088689979' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:36:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:42 smithi078 ceph-mon[18440]: from='client.? 172.21.15.123:0/3784782559' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f8e0c3d-472b-4230-ad23-2b0a34861251"}]: dispatch 2024-08-18T23:36:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:42 smithi078 ceph-mon[18440]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f8e0c3d-472b-4230-ad23-2b0a34861251"}]: dispatch 2024-08-18T23:36:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:42 smithi078 ceph-mon[18440]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f8e0c3d-472b-4230-ad23-2b0a34861251"}]': finished 2024-08-18T23:36:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:42 smithi078 ceph-mon[18440]: osdmap e10: 2 total, 1 up, 2 in 2024-08-18T23:36:42.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-18T23:36:42.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:42 smithi078 ceph-mon[18440]: from='client.? 172.21.15.123:0/1088689979' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:36:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:43 smithi078 ceph-mon[18440]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:43.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:43 smithi123 ceph-mon[21808]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:43 smithi161 ceph-mon[22027]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:45.467 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:45 smithi123 ceph-mon[21808]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:45.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:45 smithi078 ceph-mon[18440]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:45 smithi161 ceph-mon[22027]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:47.416 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:47 smithi123 ceph-mon[21808]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:47.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:47 smithi078 ceph-mon[18440]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:47 smithi161 ceph-mon[22027]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-08-18T23:36:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:48.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:48 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-08-18T23:36:48.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:48 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:48.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:48 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-08-18T23:36:48.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:48 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:49.467 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:49 smithi123 ceph-mon[21808]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:49.467 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:49 smithi123 ceph-mon[21808]: Deploying daemon osd.1 on smithi123 2024-08-18T23:36:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:49 smithi078 ceph-mon[18440]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:49 smithi078 ceph-mon[18440]: Deploying daemon osd.1 on smithi123 2024-08-18T23:36:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:49 smithi161 ceph-mon[22027]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:49 smithi161 ceph-mon[22027]: Deploying daemon osd.1 on smithi123 2024-08-18T23:36:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:51 smithi161 ceph-mon[22027]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:51.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:51 smithi078 ceph-mon[18440]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:51.536 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:51 smithi123 ceph-mon[21808]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:53.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:53 smithi078 ceph-mon[18440]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:53.472 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:53 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:53.472 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:53 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:53.472 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:53 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:53.512 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:53 smithi123 ceph-mon[21808]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:53.512 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:53 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:53.512 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:53 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:53.512 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:53 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:53 smithi161 ceph-mon[22027]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:55.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:55 smithi123 ceph-mon[21808]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:55.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:55 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:55.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:55 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:55.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:55 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:55.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:55 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:36:55.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:55 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:55 smithi078 ceph-mon[18440]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:55 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:55 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:55 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:55 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:36:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:55 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:55 smithi161 ceph-mon[22027]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:55 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:55.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:55 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:55.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:55 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:36:55.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:55 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:36:55.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:55 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:56.012 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 1 on host 'smithi123' 2024-08-18T23:36:56.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:56.010+0000 7f001effd640 1 -- 172.21.15.123:0/1497808152 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f002806ac70 con 0x7f00140a2b20 2024-08-18T23:36:56.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:56.012+0000 7f002fc40640 1 -- 172.21.15.123:0/1497808152 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f00140a2b20 msgr2=0x7f00140a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:56.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:56.012+0000 7f002fc40640 1 --2- 172.21.15.123:0/1497808152 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f00140a2b20 0x7f00140a4fe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0024005fd0 tx=0x7f0024005850 comp rx=0 tx=0).stop 2024-08-18T23:36:56.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:56.012+0000 7f002fc40640 1 -- 172.21.15.123:0/1497808152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0028101a40 msgr2=0x7f0028197030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:56.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:56.012+0000 7f002fc40640 1 --2- 172.21.15.123:0/1497808152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0028101a40 0x7f0028197030 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f00180078f0 tx=0x7f0018007db0 comp rx=0 tx=0).stop 2024-08-18T23:36:56.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:56.012+0000 7f002fc40640 1 -- 172.21.15.123:0/1497808152 shutdown_connections 2024-08-18T23:36:56.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:56.012+0000 7f002fc40640 1 --2- 172.21.15.123:0/1497808152 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f00140a2b20 0x7f00140a4fe0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:56.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:56.012+0000 7f002fc40640 1 --2- 172.21.15.123:0/1497808152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0028102400 0x7f002819e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:56.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:56.012+0000 7f002fc40640 1 --2- 172.21.15.123:0/1497808152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0028101a40 0x7f0028197030 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:56.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:56.012+0000 7f002fc40640 1 --2- 172.21.15.123:0/1497808152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0028100840 0x7f0028196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:56.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:56.012+0000 7f002fc40640 1 -- 172.21.15.123:0/1497808152 >> 172.21.15.123:0/1497808152 conn(0x7f00280fbfd0 msgr2=0x7f0028104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:36:56.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:56.013+0000 7f002fc40640 1 -- 172.21.15.123:0/1497808152 shutdown_connections 2024-08-18T23:36:56.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:36:56.013+0000 7f002fc40640 1 -- 172.21.15.123:0/1497808152 wait complete. 2024-08-18T23:36:56.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:56 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:56.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:56 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:56.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:56 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:56.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:56 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:56.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:56 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:56.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:56 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:56.621 DEBUG:teuthology.orchestra.run.smithi123:osd.1> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.1.service 2024-08-18T23:36:56.624 INFO:tasks.cephadm:Deploying osd.2 on smithi161 with /dev/vg_nvme/lv_4... 2024-08-18T23:36:56.624 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-18T23:36:56.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:56 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:36:56.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:56 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:56.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:56 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:36:56.857 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:36:56 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-1[26149]: 2024-08-18T23:36:56.693+0000 7f58e1d36640 -1 osd.1 0 log_to_monitors true 2024-08-18T23:36:56.882 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.c/config 2024-08-18T23:36:57.325 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:57 smithi161 ceph-mon[22027]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:57.325 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:57 smithi161 ceph-mon[22027]: from='osd.1 [v2:172.21.15.123:6800/4199446368,v1:172.21.15.123:6801/4199446368]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-18T23:36:57.325 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:57 smithi161 ceph-mon[22027]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-18T23:36:57.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:57 smithi123 ceph-mon[21808]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:57.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:57 smithi123 ceph-mon[21808]: from='osd.1 [v2:172.21.15.123:6800/4199446368,v1:172.21.15.123:6801/4199446368]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-18T23:36:57.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:57 smithi123 ceph-mon[21808]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-18T23:36:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:57 smithi078 ceph-mon[18440]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:57 smithi078 ceph-mon[18440]: from='osd.1 [v2:172.21.15.123:6800/4199446368,v1:172.21.15.123:6801/4199446368]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-18T23:36:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:57 smithi078 ceph-mon[18440]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-18T23:36:58.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:58 smithi123 ceph-mon[21808]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-18T23:36:58.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:58 smithi123 ceph-mon[21808]: osdmap e11: 2 total, 1 up, 2 in 2024-08-18T23:36:58.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:58 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-18T23:36:58.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:58 smithi123 ceph-mon[21808]: from='osd.1 [v2:172.21.15.123:6800/4199446368,v1:172.21.15.123:6801/4199446368]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:36:58.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:58 smithi123 ceph-mon[21808]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:36:58.374 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:58 smithi161 ceph-mon[22027]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-18T23:36:58.375 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:58 smithi161 ceph-mon[22027]: osdmap e11: 2 total, 1 up, 2 in 2024-08-18T23:36:58.375 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:58 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-18T23:36:58.375 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:58 smithi161 ceph-mon[22027]: from='osd.1 [v2:172.21.15.123:6800/4199446368,v1:172.21.15.123:6801/4199446368]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:36:58.375 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:58 smithi161 ceph-mon[22027]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:36:58.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:58 smithi078 ceph-mon[18440]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-18T23:36:58.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:58 smithi078 ceph-mon[18440]: osdmap e11: 2 total, 1 up, 2 in 2024-08-18T23:36:58.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:58 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-18T23:36:58.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:58 smithi078 ceph-mon[18440]: from='osd.1 [v2:172.21.15.123:6800/4199446368,v1:172.21.15.123:6801/4199446368]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:36:58.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:58 smithi078 ceph-mon[18440]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:36:59.029 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:36:59.054 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_4 2024-08-18T23:36:59.199 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:36:58 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-1[26149]: 2024-08-18T23:36:58.913+0000 7f58ddcb9640 -1 osd.1 0 waiting for initial osdmap 2024-08-18T23:36:59.199 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:36:58 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-1[26149]: 2024-08-18T23:36:58.918+0000 7f58d5ad9640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T23:36:59.322 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:59 smithi161 ceph-mon[22027]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:59.322 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:59 smithi161 ceph-mon[22027]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-08-18T23:36:59.322 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:59 smithi161 ceph-mon[22027]: osdmap e12: 2 total, 1 up, 2 in 2024-08-18T23:36:59.322 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:59 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-18T23:36:59.322 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:59 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-18T23:36:59.322 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:36:59 smithi161 ceph-mon[22027]: from='osd.1 ' entity='osd.1' 2024-08-18T23:36:59.348 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.c/config 2024-08-18T23:36:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:59 smithi078 ceph-mon[18440]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:59 smithi078 ceph-mon[18440]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-08-18T23:36:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:59 smithi078 ceph-mon[18440]: osdmap e12: 2 total, 1 up, 2 in 2024-08-18T23:36:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:59 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-18T23:36:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:59 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-18T23:36:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:36:59 smithi078 ceph-mon[18440]: from='osd.1 ' entity='osd.1' 2024-08-18T23:36:59.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:59 smithi123 ceph-mon[21808]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:36:59.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:59 smithi123 ceph-mon[21808]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-08-18T23:36:59.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:59 smithi123 ceph-mon[21808]: osdmap e12: 2 total, 1 up, 2 in 2024-08-18T23:36:59.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:59 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-18T23:36:59.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:59 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-18T23:36:59.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:36:59 smithi123 ceph-mon[21808]: from='osd.1 ' entity='osd.1' 2024-08-18T23:36:59.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.830+0000 7fbe6779f640 1 -- 172.21.15.161:0/3687219226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe60101930 msgr2=0x7fbe60101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:59.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.830+0000 7fbe6779f640 1 --2- 172.21.15.161:0/3687219226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe60101930 0x7fbe60101d30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbe50009900 tx=0x7fbe5002a410 comp rx=0 tx=0).stop 2024-08-18T23:36:59.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.831+0000 7fbe6779f640 1 -- 172.21.15.161:0/3687219226 shutdown_connections 2024-08-18T23:36:59.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.831+0000 7fbe6779f640 1 --2- 172.21.15.161:0/3687219226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe601034f0 0x7fbe60109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:59.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.831+0000 7fbe6779f640 1 --2- 172.21.15.161:0/3687219226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe60102b30 0x7fbe60102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:59.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.831+0000 7fbe6779f640 1 --2- 172.21.15.161:0/3687219226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe60101930 0x7fbe60101d30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:59.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.831+0000 7fbe6779f640 1 -- 172.21.15.161:0/3687219226 >> 172.21.15.161:0/3687219226 conn(0x7fbe600fd0c0 msgr2=0x7fbe600ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:36:59.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.831+0000 7fbe6779f640 1 -- 172.21.15.161:0/3687219226 shutdown_connections 2024-08-18T23:36:59.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.831+0000 7fbe6779f640 1 -- 172.21.15.161:0/3687219226 wait complete. 2024-08-18T23:36:59.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.831+0000 7fbe6779f640 1 Processor -- start 2024-08-18T23:36:59.846 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.831+0000 7fbe6779f640 1 -- start start 2024-08-18T23:36:59.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.831+0000 7fbe6779f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe60101930 0x7fbe60197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:59.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.831+0000 7fbe6779f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe60102b30 0x7fbe601981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:59.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.831+0000 7fbe6779f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe601034f0 0x7fbe6019f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:59.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.831+0000 7fbe6779f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6010d5c0 con 0x7fbe60102b30 2024-08-18T23:36:59.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.831+0000 7fbe6779f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6010d440 con 0x7fbe60101930 2024-08-18T23:36:59.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.831+0000 7fbe6779f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6010d740 con 0x7fbe601034f0 2024-08-18T23:36:59.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.832+0000 7fbe65d3c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe601034f0 0x7fbe6019f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:59.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.832+0000 7fbe65d3c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe601034f0 0x7fbe6019f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55392/0 (socket says 172.21.15.161:55392) 2024-08-18T23:36:59.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.832+0000 7fbe65d3c640 1 -- 172.21.15.161:0/425603711 learned_addr learned my addr 172.21.15.161:0/425603711 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-18T23:36:59.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.832+0000 7fbe6553b640 1 --2- 172.21.15.161:0/425603711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe60101930 0x7fbe60197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:59.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.832+0000 7fbe64d3a640 1 --2- 172.21.15.161:0/425603711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe60102b30 0x7fbe601981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:59.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.832+0000 7fbe65d3c640 1 -- 172.21.15.161:0/425603711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe60101930 msgr2=0x7fbe60197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:59.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.832+0000 7fbe65d3c640 1 --2- 172.21.15.161:0/425603711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe60101930 0x7fbe60197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:59.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.832+0000 7fbe65d3c640 1 -- 172.21.15.161:0/425603711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe60102b30 msgr2=0x7fbe601981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:36:59.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.832+0000 7fbe65d3c640 1 --2- 172.21.15.161:0/425603711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe60102b30 0x7fbe601981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:36:59.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.832+0000 7fbe65d3c640 1 -- 172.21.15.161:0/425603711 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6019f880 con 0x7fbe601034f0 2024-08-18T23:36:59.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.832+0000 7fbe65d3c640 1 --2- 172.21.15.161:0/425603711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe601034f0 0x7fbe6019f1e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800eaa0 tx=0x7fbe5800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:36:59.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.832+0000 7fbe567fc640 1 -- 172.21.15.161:0/425603711 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5800cdb0 con 0x7fbe601034f0 2024-08-18T23:36:59.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.832+0000 7fbe567fc640 1 -- 172.21.15.161:0/425603711 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe58005020 con 0x7fbe601034f0 2024-08-18T23:36:59.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.832+0000 7fbe6779f640 1 -- 172.21.15.161:0/425603711 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6019fb70 con 0x7fbe601034f0 2024-08-18T23:36:59.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.832+0000 7fbe567fc640 1 -- 172.21.15.161:0/425603711 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe580116b0 con 0x7fbe601034f0 2024-08-18T23:36:59.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.832+0000 7fbe6779f640 1 -- 172.21.15.161:0/425603711 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe601a0080 con 0x7fbe601034f0 2024-08-18T23:36:59.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.833+0000 7fbe6779f640 1 -- 172.21.15.161:0/425603711 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe60108d80 con 0x7fbe601034f0 2024-08-18T23:36:59.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.834+0000 7fbe567fc640 1 -- 172.21.15.161:0/425603711 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fbe58005560 con 0x7fbe601034f0 2024-08-18T23:36:59.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.835+0000 7fbe567fc640 1 --2- 172.21.15.161:0/425603711 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbe4c0a2af0 0x7fbe4c0a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:36:59.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.835+0000 7fbe567fc640 1 -- 172.21.15.161:0/425603711 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fbe580cd860 con 0x7fbe601034f0 2024-08-18T23:36:59.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.835+0000 7fbe6553b640 1 --2- 172.21.15.161:0/425603711 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbe4c0a2af0 0x7fbe4c0a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:36:59.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.836+0000 7fbe6553b640 1 --2- 172.21.15.161:0/425603711 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbe4c0a2af0 0x7fbe4c0a4fb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbe50005ab0 tx=0x7fbe50000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:36:59.849 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:36:59.840+0000 7fbe567fc640 1 -- 172.21.15.161:0/425603711 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbe58016030 con 0x7fbe601034f0 2024-08-18T23:37:00.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:37:00.024+0000 7fbe6779f640 1 -- 172.21.15.161:0/425603711 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbe60062360 con 0x7fbe4c0a2af0 2024-08-18T23:37:00.283 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:00 smithi161 ceph-mon[22027]: purged_snaps scrub starts 2024-08-18T23:37:00.283 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:00 smithi161 ceph-mon[22027]: purged_snaps scrub ok 2024-08-18T23:37:00.283 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:00 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-18T23:37:00.283 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:00 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:00.284 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:00 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:00.284 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:00 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:37:00.284 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:00 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:00.284 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:00 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:00.284 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:00 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:37:00.284 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:00 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:00.284 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:00 smithi161 ceph-mon[22027]: osd.1 [v2:172.21.15.123:6800/4199446368,v1:172.21.15.123:6801/4199446368] boot 2024-08-18T23:37:00.284 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:00 smithi161 ceph-mon[22027]: osdmap e13: 2 total, 2 up, 2 in 2024-08-18T23:37:00.284 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:00 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-18T23:37:00.284 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:00 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:37:00.284 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:00 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:37:00.284 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:00 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:00.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:00 smithi078 ceph-mon[18440]: purged_snaps scrub starts 2024-08-18T23:37:00.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:00 smithi078 ceph-mon[18440]: purged_snaps scrub ok 2024-08-18T23:37:00.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:00 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-18T23:37:00.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:00 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:00.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:00 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:00.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:00 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:37:00.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:00 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:00.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:00 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:00.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:00 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:37:00.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:00 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:00.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:00 smithi078 ceph-mon[18440]: osd.1 [v2:172.21.15.123:6800/4199446368,v1:172.21.15.123:6801/4199446368] boot 2024-08-18T23:37:00.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:00 smithi078 ceph-mon[18440]: osdmap e13: 2 total, 2 up, 2 in 2024-08-18T23:37:00.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:00 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-18T23:37:00.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:00 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:37:00.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:00 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:37:00.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:00 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:00.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:00 smithi123 ceph-mon[21808]: purged_snaps scrub starts 2024-08-18T23:37:00.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:00 smithi123 ceph-mon[21808]: purged_snaps scrub ok 2024-08-18T23:37:00.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:00 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-18T23:37:00.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:00 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:00.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:00 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:00.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:00 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:37:00.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:00 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:00.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:00 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:00.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:00 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:37:00.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:00 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:00.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:00 smithi123 ceph-mon[21808]: osd.1 [v2:172.21.15.123:6800/4199446368,v1:172.21.15.123:6801/4199446368] boot 2024-08-18T23:37:00.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:00 smithi123 ceph-mon[21808]: osdmap e13: 2 total, 2 up, 2 in 2024-08-18T23:37:00.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:00 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-18T23:37:00.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:00 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:37:00.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:00 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:37:00.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:00 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:01.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:01 smithi078 ceph-mon[18440]: Detected new or changed devices on smithi123 2024-08-18T23:37:01.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:01 smithi078 ceph-mon[18440]: Adjusting osd_memory_target on smithi123 to 17088M 2024-08-18T23:37:01.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:01 smithi078 ceph-mon[18440]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:37:01.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:01 smithi078 ceph-mon[18440]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:37:01.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:01 smithi161 ceph-mon[22027]: Detected new or changed devices on smithi123 2024-08-18T23:37:01.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:01 smithi161 ceph-mon[22027]: Adjusting osd_memory_target on smithi123 to 17088M 2024-08-18T23:37:01.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:01 smithi161 ceph-mon[22027]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:37:01.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:01 smithi161 ceph-mon[22027]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:37:01.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:01 smithi123 ceph-mon[21808]: Detected new or changed devices on smithi123 2024-08-18T23:37:01.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:01 smithi123 ceph-mon[21808]: Adjusting osd_memory_target on smithi123 to 17088M 2024-08-18T23:37:01.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:01 smithi123 ceph-mon[21808]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:37:01.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:01 smithi123 ceph-mon[21808]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-18T23:37:02.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:02 smithi123 ceph-mon[21808]: osdmap e14: 2 total, 2 up, 2 in 2024-08-18T23:37:02.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:02 smithi123 ceph-mon[21808]: from='client.? 172.21.15.161:0/4208994427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27ef3314-bb6f-413d-be05-e10a105dec9f"}]: dispatch 2024-08-18T23:37:02.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:02 smithi123 ceph-mon[21808]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27ef3314-bb6f-413d-be05-e10a105dec9f"}]: dispatch 2024-08-18T23:37:02.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:02 smithi123 ceph-mon[21808]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27ef3314-bb6f-413d-be05-e10a105dec9f"}]': finished 2024-08-18T23:37:02.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:02 smithi123 ceph-mon[21808]: osdmap e15: 3 total, 2 up, 3 in 2024-08-18T23:37:02.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-18T23:37:02.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:02 smithi161 ceph-mon[22027]: osdmap e14: 2 total, 2 up, 2 in 2024-08-18T23:37:02.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:02 smithi161 ceph-mon[22027]: from='client.? 172.21.15.161:0/4208994427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27ef3314-bb6f-413d-be05-e10a105dec9f"}]: dispatch 2024-08-18T23:37:02.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:02 smithi161 ceph-mon[22027]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27ef3314-bb6f-413d-be05-e10a105dec9f"}]: dispatch 2024-08-18T23:37:02.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:02 smithi161 ceph-mon[22027]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27ef3314-bb6f-413d-be05-e10a105dec9f"}]': finished 2024-08-18T23:37:02.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:02 smithi161 ceph-mon[22027]: osdmap e15: 3 total, 2 up, 3 in 2024-08-18T23:37:02.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:02 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-18T23:37:02.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:02 smithi078 ceph-mon[18440]: osdmap e14: 2 total, 2 up, 2 in 2024-08-18T23:37:02.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:02 smithi078 ceph-mon[18440]: from='client.? 172.21.15.161:0/4208994427' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27ef3314-bb6f-413d-be05-e10a105dec9f"}]: dispatch 2024-08-18T23:37:02.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:02 smithi078 ceph-mon[18440]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27ef3314-bb6f-413d-be05-e10a105dec9f"}]: dispatch 2024-08-18T23:37:02.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:02 smithi078 ceph-mon[18440]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27ef3314-bb6f-413d-be05-e10a105dec9f"}]': finished 2024-08-18T23:37:02.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:02 smithi078 ceph-mon[18440]: osdmap e15: 3 total, 2 up, 3 in 2024-08-18T23:37:02.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:02 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-18T23:37:03.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:03 smithi123 ceph-mon[21808]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:03.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:03 smithi123 ceph-mon[21808]: from='client.? 172.21.15.161:0/463551070' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:37:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:03 smithi161 ceph-mon[22027]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:03 smithi161 ceph-mon[22027]: from='client.? 172.21.15.161:0/463551070' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:37:03.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:03 smithi078 ceph-mon[18440]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:03.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:03 smithi078 ceph-mon[18440]: from='client.? 172.21.15.161:0/463551070' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:37:04.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:04 smithi123 ceph-mon[21808]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:04 smithi161 ceph-mon[22027]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:04.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:04 smithi078 ceph-mon[18440]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:07.280 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:07 smithi161 ceph-mon[22027]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:07 smithi078 ceph-mon[18440]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:07.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:07 smithi123 ceph-mon[21808]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:09.331 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:09 smithi161 ceph-mon[22027]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:09.331 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-08-18T23:37:09.331 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:09 smithi078 ceph-mon[18440]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-08-18T23:37:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:09.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:09 smithi123 ceph-mon[21808]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:09.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-08-18T23:37:09.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:10.338 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:10 smithi161 ceph-mon[22027]: Deploying daemon osd.2 on smithi161 2024-08-18T23:37:10.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:10 smithi078 ceph-mon[18440]: Deploying daemon osd.2 on smithi161 2024-08-18T23:37:10.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:10 smithi123 ceph-mon[21808]: Deploying daemon osd.2 on smithi161 2024-08-18T23:37:11.361 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:11 smithi161 ceph-mon[22027]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:11.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:11 smithi078 ceph-mon[18440]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:11.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:11 smithi123 ceph-mon[21808]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:13.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:13 smithi161 ceph-mon[22027]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:13.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:13 smithi123 ceph-mon[21808]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:13 smithi078 ceph-mon[18440]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:14.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:37:14.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:14.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:37:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:14.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:37:14.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:14.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:15.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:15 smithi161 ceph-mon[22027]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:15 smithi078 ceph-mon[18440]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:15.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:15 smithi123 ceph-mon[21808]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:16.120 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:37:16.119+0000 7fbe567fc640 1 -- 172.21.15.161:0/425603711 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbe60062360 con 0x7fbe4c0a2af0 2024-08-18T23:37:16.121 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 2 on host 'smithi161' 2024-08-18T23:37:16.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:37:16.122+0000 7fbe6779f640 1 -- 172.21.15.161:0/425603711 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbe4c0a2af0 msgr2=0x7fbe4c0a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:37:16.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:37:16.122+0000 7fbe6779f640 1 --2- 172.21.15.161:0/425603711 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbe4c0a2af0 0x7fbe4c0a4fb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbe50005ab0 tx=0x7fbe50000bc0 comp rx=0 tx=0).stop 2024-08-18T23:37:16.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:37:16.122+0000 7fbe6779f640 1 -- 172.21.15.161:0/425603711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe601034f0 msgr2=0x7fbe6019f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:37:16.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:37:16.122+0000 7fbe6779f640 1 --2- 172.21.15.161:0/425603711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe601034f0 0x7fbe6019f1e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800eaa0 tx=0x7fbe5800ef60 comp rx=0 tx=0).stop 2024-08-18T23:37:16.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:37:16.122+0000 7fbe6779f640 1 -- 172.21.15.161:0/425603711 shutdown_connections 2024-08-18T23:37:16.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:37:16.122+0000 7fbe6779f640 1 --2- 172.21.15.161:0/425603711 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbe4c0a2af0 0x7fbe4c0a4fb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:37:16.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:37:16.122+0000 7fbe6779f640 1 --2- 172.21.15.161:0/425603711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe601034f0 0x7fbe6019f1e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:37:16.123 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:37:16.122+0000 7fbe6779f640 1 --2- 172.21.15.161:0/425603711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe60102b30 0x7fbe601981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:37:16.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:37:16.122+0000 7fbe6779f640 1 --2- 172.21.15.161:0/425603711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe60101930 0x7fbe60197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:37:16.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:37:16.122+0000 7fbe6779f640 1 -- 172.21.15.161:0/425603711 >> 172.21.15.161:0/425603711 conn(0x7fbe600fd0c0 msgr2=0x7fbe60105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:37:16.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:37:16.122+0000 7fbe6779f640 1 -- 172.21.15.161:0/425603711 shutdown_connections 2024-08-18T23:37:16.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:37:16.122+0000 7fbe6779f640 1 -- 172.21.15.161:0/425603711 wait complete. 2024-08-18T23:37:17.004 DEBUG:teuthology.orchestra.run.smithi161:osd.2> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.2.service 2024-08-18T23:37:17.007 INFO:tasks.cephadm:Deploying osd.3 on smithi078 with /dev/vg_nvme/lv_3... 2024-08-18T23:37:17.007 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-18T23:37:17.252 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:17 smithi161 ceph-mon[22027]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:17.252 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:17 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:17.253 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:17 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:17.253 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:17 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:17.253 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:17 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:17.269 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:37:17.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:17 smithi078 ceph-mon[18440]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:17.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:17 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:17.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:17 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:17.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:17 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:17.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:17 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:17.539 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:37:17 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-2[26978]: 2024-08-18T23:37:17.328+0000 7f2a162f3640 -1 osd.2 0 log_to_monitors true 2024-08-18T23:37:17.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:17 smithi123 ceph-mon[21808]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:17.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:17 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:17.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:17 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:17.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:17 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:17.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:17 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:18.401 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:18 smithi161 ceph-mon[22027]: from='osd.2 [v2:172.21.15.161:6800/2582810057,v1:172.21.15.161:6801/2582810057]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-18T23:37:18.401 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:18 smithi161 ceph-mon[22027]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-18T23:37:18.415 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:18 smithi078 ceph-mon[18440]: from='osd.2 [v2:172.21.15.161:6800/2582810057,v1:172.21.15.161:6801/2582810057]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-18T23:37:18.415 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:18 smithi078 ceph-mon[18440]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-18T23:37:18.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:18 smithi123 ceph-mon[21808]: from='osd.2 [v2:172.21.15.161:6800/2582810057,v1:172.21.15.161:6801/2582810057]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-18T23:37:18.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:18 smithi123 ceph-mon[21808]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-18T23:37:19.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:19 smithi078 ceph-mon[18440]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:19.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:19 smithi078 ceph-mon[18440]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-18T23:37:19.382 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:19 smithi078 ceph-mon[18440]: from='osd.2 [v2:172.21.15.161:6800/2582810057,v1:172.21.15.161:6801/2582810057]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:37:19.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:19 smithi078 ceph-mon[18440]: osdmap e16: 3 total, 2 up, 3 in 2024-08-18T23:37:19.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-18T23:37:19.383 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:19 smithi078 ceph-mon[18440]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:37:19.428 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:37:19 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-2[26978]: 2024-08-18T23:37:19.127+0000 7f2a12276640 -1 osd.2 0 waiting for initial osdmap 2024-08-18T23:37:19.428 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:37:19 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-2[26978]: 2024-08-18T23:37:19.131+0000 7f2a0e09e640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T23:37:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:19 smithi161 ceph-mon[22027]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:19 smithi161 ceph-mon[22027]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-18T23:37:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:19 smithi161 ceph-mon[22027]: from='osd.2 [v2:172.21.15.161:6800/2582810057,v1:172.21.15.161:6801/2582810057]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:37:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:19 smithi161 ceph-mon[22027]: osdmap e16: 3 total, 2 up, 3 in 2024-08-18T23:37:19.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-18T23:37:19.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:19 smithi161 ceph-mon[22027]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:37:19.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:19 smithi123 ceph-mon[21808]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:19.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:19 smithi123 ceph-mon[21808]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-18T23:37:19.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:19 smithi123 ceph-mon[21808]: from='osd.2 [v2:172.21.15.161:6800/2582810057,v1:172.21.15.161:6801/2582810057]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:37:19.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:19 smithi123 ceph-mon[21808]: osdmap e16: 3 total, 2 up, 3 in 2024-08-18T23:37:19.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-18T23:37:19.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:19 smithi123 ceph-mon[21808]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:37:19.778 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:37:19.804 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_3 2024-08-18T23:37:20.077 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:37:20.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:20 smithi161 ceph-mon[22027]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-18T23:37:20.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:20 smithi161 ceph-mon[22027]: osdmap e17: 3 total, 2 up, 3 in 2024-08-18T23:37:20.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-18T23:37:20.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-18T23:37:20.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:20.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:20.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:37:20.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:20.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:20.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:37:20.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:20.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:37:20.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:20.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:37:20.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:20.432 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:20 smithi078 ceph-mon[18440]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-18T23:37:20.432 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:20 smithi078 ceph-mon[18440]: osdmap e17: 3 total, 2 up, 3 in 2024-08-18T23:37:20.432 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-18T23:37:20.432 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-18T23:37:20.432 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:20.432 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:20.432 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:37:20.433 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:20.433 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:20.433 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:37:20.433 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:20.433 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:37:20.433 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:20.433 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:37:20.433 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:20.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:20 smithi123 ceph-mon[21808]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-18T23:37:20.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:20 smithi123 ceph-mon[21808]: osdmap e17: 3 total, 2 up, 3 in 2024-08-18T23:37:20.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-18T23:37:20.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-18T23:37:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:37:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:37:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:37:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:37:20.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:20.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.635+0000 7f29afee7640 1 -- 172.21.15.78:0/967876582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8102400 msgr2=0x7f29a8108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:37:20.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.635+0000 7f29afee7640 1 --2- 172.21.15.78:0/967876582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8102400 0x7f29a8108c90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009ca0 tx=0x7f29a002a8c0 comp rx=0 tx=0).stop 2024-08-18T23:37:20.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.635+0000 7f29afee7640 1 -- 172.21.15.78:0/967876582 shutdown_connections 2024-08-18T23:37:20.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.635+0000 7f29afee7640 1 --2- 172.21.15.78:0/967876582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8102400 0x7f29a8108c90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:37:20.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.635+0000 7f29afee7640 1 --2- 172.21.15.78:0/967876582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a8101a40 0x7f29a8101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:37:20.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.635+0000 7f29afee7640 1 --2- 172.21.15.78:0/967876582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29a8100840 0x7f29a8100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:37:20.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.635+0000 7f29afee7640 1 -- 172.21.15.78:0/967876582 >> 172.21.15.78:0/967876582 conn(0x7f29a80fbfd0 msgr2=0x7f29a80fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:37:20.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.635+0000 7f29afee7640 1 -- 172.21.15.78:0/967876582 shutdown_connections 2024-08-18T23:37:20.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.635+0000 7f29afee7640 1 -- 172.21.15.78:0/967876582 wait complete. 2024-08-18T23:37:20.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29afee7640 1 Processor -- start 2024-08-18T23:37:20.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29afee7640 1 -- start start 2024-08-18T23:37:20.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29afee7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29a8100840 0x7f29a8199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:37:20.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29afee7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8101a40 0x7f29a819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:37:20.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29afee7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a8102400 0x7f29a81a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:37:20.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29afee7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29a810d6d0 con 0x7f29a8101a40 2024-08-18T23:37:20.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29afee7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29a810d550 con 0x7f29a8102400 2024-08-18T23:37:20.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29afee7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f29a810d850 con 0x7f29a8100840 2024-08-18T23:37:20.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29ad482640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8101a40 0x7f29a819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:37:20.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29ad482640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8101a40 0x7f29a819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52870/0 (socket says 172.21.15.78:52870) 2024-08-18T23:37:20.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29ad482640 1 -- 172.21.15.78:0/3933185410 learned_addr learned my addr 172.21.15.78:0/3933185410 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:37:20.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29adc83640 1 --2- 172.21.15.78:0/3933185410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29a8100840 0x7f29a8199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:37:20.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29ad482640 1 -- 172.21.15.78:0/3933185410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29a8100840 msgr2=0x7f29a8199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:37:20.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29ad482640 1 --2- 172.21.15.78:0/3933185410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29a8100840 0x7f29a8199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:37:20.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29ad482640 1 -- 172.21.15.78:0/3933185410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a8102400 msgr2=0x7f29a81a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:37:20.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29ae484640 1 --2- 172.21.15.78:0/3933185410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a8102400 0x7f29a81a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:37:20.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29ad482640 1 --2- 172.21.15.78:0/3933185410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a8102400 0x7f29a81a1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:37:20.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29ad482640 1 -- 172.21.15.78:0/3933185410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a81a1be0 con 0x7f29a8101a40 2024-08-18T23:37:20.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29adc83640 1 --2- 172.21.15.78:0/3933185410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29a8100840 0x7f29a8199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:37:20.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.636+0000 7f29ad482640 1 --2- 172.21.15.78:0/3933185410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8101a40 0x7f29a819a4f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f29a400b4a0 tx=0x7f29a400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:37:20.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.637+0000 7f299affd640 1 -- 172.21.15.78:0/3933185410 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4014070 con 0x7f29a8101a40 2024-08-18T23:37:20.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.637+0000 7f299affd640 1 -- 172.21.15.78:0/3933185410 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29a4010af0 con 0x7f29a8101a40 2024-08-18T23:37:20.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.637+0000 7f29afee7640 1 -- 172.21.15.78:0/3933185410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a81a1ed0 con 0x7f29a8101a40 2024-08-18T23:37:20.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.637+0000 7f299affd640 1 -- 172.21.15.78:0/3933185410 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4017d10 con 0x7f29a8101a40 2024-08-18T23:37:20.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.637+0000 7f29afee7640 1 -- 172.21.15.78:0/3933185410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a81a2400 con 0x7f29a8101a40 2024-08-18T23:37:20.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.638+0000 7f29afee7640 1 -- 172.21.15.78:0/3933185410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a81a2690 con 0x7f29a8101a40 2024-08-18T23:37:20.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.639+0000 7f299affd640 1 -- 172.21.15.78:0/3933185410 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f29a4017070 con 0x7f29a8101a40 2024-08-18T23:37:20.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.639+0000 7f299affd640 1 --2- 172.21.15.78:0/3933185410 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f29940a2b20 0x7f29940a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:37:20.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.639+0000 7f29adc83640 1 --2- 172.21.15.78:0/3933185410 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f29940a2b20 0x7f29940a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:37:20.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.639+0000 7f299affd640 1 -- 172.21.15.78:0/3933185410 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7f29a40cd160 con 0x7f29a8101a40 2024-08-18T23:37:20.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.640+0000 7f29adc83640 1 --2- 172.21.15.78:0/3933185410 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f29940a2b20 0x7f29940a4fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f299c006fd0 tx=0x7f299c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:37:20.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.643+0000 7f299affd640 1 -- 172.21.15.78:0/3933185410 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f29a409b6f0 con 0x7f29a8101a40 2024-08-18T23:37:20.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:20.819+0000 7f29afee7640 1 -- 172.21.15.78:0/3933185410 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f29a81a2860 con 0x7f29940a2b20 2024-08-18T23:37:21.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:21 smithi078 ceph-mon[18440]: purged_snaps scrub starts 2024-08-18T23:37:21.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:21 smithi078 ceph-mon[18440]: purged_snaps scrub ok 2024-08-18T23:37:21.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:21 smithi078 ceph-mon[18440]: Detected new or changed devices on smithi161 2024-08-18T23:37:21.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:21 smithi078 ceph-mon[18440]: Adjusting osd_memory_target on smithi161 to 17088M 2024-08-18T23:37:21.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:21 smithi078 ceph-mon[18440]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:21.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:21 smithi078 ceph-mon[18440]: osd.2 [v2:172.21.15.161:6800/2582810057,v1:172.21.15.161:6801/2582810057] boot 2024-08-18T23:37:21.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:21 smithi078 ceph-mon[18440]: osdmap e18: 3 total, 3 up, 3 in 2024-08-18T23:37:21.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:21 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-18T23:37:21.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:21 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:37:21.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:21 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:37:21.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:21 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:21.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:21 smithi161 ceph-mon[22027]: purged_snaps scrub starts 2024-08-18T23:37:21.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:21 smithi161 ceph-mon[22027]: purged_snaps scrub ok 2024-08-18T23:37:21.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:21 smithi161 ceph-mon[22027]: Detected new or changed devices on smithi161 2024-08-18T23:37:21.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:21 smithi161 ceph-mon[22027]: Adjusting osd_memory_target on smithi161 to 17088M 2024-08-18T23:37:21.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:21 smithi161 ceph-mon[22027]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:21.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:21 smithi161 ceph-mon[22027]: osd.2 [v2:172.21.15.161:6800/2582810057,v1:172.21.15.161:6801/2582810057] boot 2024-08-18T23:37:21.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:21 smithi161 ceph-mon[22027]: osdmap e18: 3 total, 3 up, 3 in 2024-08-18T23:37:21.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:21 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-18T23:37:21.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:21 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:37:21.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:21 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:37:21.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:21 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:21.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:21 smithi123 ceph-mon[21808]: purged_snaps scrub starts 2024-08-18T23:37:21.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:21 smithi123 ceph-mon[21808]: purged_snaps scrub ok 2024-08-18T23:37:21.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:21 smithi123 ceph-mon[21808]: Detected new or changed devices on smithi161 2024-08-18T23:37:21.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:21 smithi123 ceph-mon[21808]: Adjusting osd_memory_target on smithi161 to 17088M 2024-08-18T23:37:21.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:21 smithi123 ceph-mon[21808]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-18T23:37:21.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:21 smithi123 ceph-mon[21808]: osd.2 [v2:172.21.15.161:6800/2582810057,v1:172.21.15.161:6801/2582810057] boot 2024-08-18T23:37:21.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:21 smithi123 ceph-mon[21808]: osdmap e18: 3 total, 3 up, 3 in 2024-08-18T23:37:21.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:21 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-18T23:37:21.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:21 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:37:21.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:21 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:37:21.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:21 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:22.571 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:22 smithi078 ceph-mon[18440]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:37:22.571 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:22 smithi078 ceph-mon[18440]: osdmap e19: 3 total, 3 up, 3 in 2024-08-18T23:37:22.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:22 smithi161 ceph-mon[22027]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:37:22.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:22 smithi161 ceph-mon[22027]: osdmap e19: 3 total, 3 up, 3 in 2024-08-18T23:37:22.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:22 smithi123 ceph-mon[21808]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:37:22.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:22 smithi123 ceph-mon[21808]: osdmap e19: 3 total, 3 up, 3 in 2024-08-18T23:37:23.612 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:23 smithi078 ceph-mon[18440]: pgmap v69: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-18T23:37:23.612 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-08-18T23:37:23.612 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-18T23:37:23.612 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:23 smithi078 ceph-mon[18440]: osdmap e20: 3 total, 3 up, 3 in 2024-08-18T23:37:23.612 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-08-18T23:37:23.612 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:23 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1758758954' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9e52662-5ef9-4d70-bdd7-2b92165822ee"}]: dispatch 2024-08-18T23:37:23.612 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-18T23:37:23.612 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:23 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1758758954' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9e52662-5ef9-4d70-bdd7-2b92165822ee"}]': finished 2024-08-18T23:37:23.612 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:23 smithi078 ceph-mon[18440]: osdmap e21: 4 total, 3 up, 4 in 2024-08-18T23:37:23.612 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-18T23:37:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:23 smithi161 ceph-mon[22027]: pgmap v69: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-18T23:37:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-08-18T23:37:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-18T23:37:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:23 smithi161 ceph-mon[22027]: osdmap e20: 3 total, 3 up, 3 in 2024-08-18T23:37:23.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-08-18T23:37:23.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:23 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1758758954' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9e52662-5ef9-4d70-bdd7-2b92165822ee"}]: dispatch 2024-08-18T23:37:23.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-18T23:37:23.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:23 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1758758954' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9e52662-5ef9-4d70-bdd7-2b92165822ee"}]': finished 2024-08-18T23:37:23.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:23 smithi161 ceph-mon[22027]: osdmap e21: 4 total, 3 up, 4 in 2024-08-18T23:37:23.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-18T23:37:23.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:23 smithi123 ceph-mon[21808]: pgmap v69: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-18T23:37:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-08-18T23:37:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-18T23:37:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:23 smithi123 ceph-mon[21808]: osdmap e20: 3 total, 3 up, 3 in 2024-08-18T23:37:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-08-18T23:37:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:23 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1758758954' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9e52662-5ef9-4d70-bdd7-2b92165822ee"}]: dispatch 2024-08-18T23:37:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-18T23:37:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:23 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1758758954' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9e52662-5ef9-4d70-bdd7-2b92165822ee"}]': finished 2024-08-18T23:37:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:23 smithi123 ceph-mon[21808]: osdmap e21: 4 total, 3 up, 4 in 2024-08-18T23:37:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-18T23:37:23.867 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:37:23 smithi078 sudo[36694]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-18T23:37:23.867 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:37:23 smithi078 sudo[36694]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-18T23:37:23.867 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:37:23 smithi078 sudo[36694]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-18T23:37:23.868 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:37:23 smithi078 sudo[36694]: pam_unix(sudo:session): session closed for user root 2024-08-18T23:37:24.219 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:37:23 smithi078 sudo[36703]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-18T23:37:24.219 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:37:23 smithi078 sudo[36703]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-18T23:37:24.219 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:37:23 smithi078 sudo[36703]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-18T23:37:24.219 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:37:23 smithi078 sudo[36703]: pam_unix(sudo:session): session closed for user root 2024-08-18T23:37:24.427 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:37:24 smithi123 sudo[30372]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-18T23:37:24.427 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:37:24 smithi123 sudo[30372]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-18T23:37:24.427 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:37:24 smithi123 sudo[30372]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-18T23:37:24.427 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:37:24 smithi123 sudo[30372]: pam_unix(sudo:session): session closed for user root 2024-08-18T23:37:24.428 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:24 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2611360751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:37:24.428 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:24 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:24.428 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:24 smithi123 ceph-mon[21808]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-18T23:37:24.428 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:24 smithi123 ceph-mon[21808]: osdmap e22: 4 total, 3 up, 4 in 2024-08-18T23:37:24.428 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:24 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-18T23:37:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:24 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2611360751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:37:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:24 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:24 smithi161 ceph-mon[22027]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-18T23:37:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:24 smithi161 ceph-mon[22027]: osdmap e22: 4 total, 3 up, 4 in 2024-08-18T23:37:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:24 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-18T23:37:24.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:24 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2611360751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:37:24.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:24 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:24.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:24 smithi078 ceph-mon[18440]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-18T23:37:24.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:24 smithi078 ceph-mon[18440]: osdmap e22: 4 total, 3 up, 4 in 2024-08-18T23:37:24.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:24 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-18T23:37:24.797 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:37:24 smithi123 sudo[30375]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-18T23:37:24.798 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:37:24 smithi123 sudo[30375]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-18T23:37:24.798 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:37:24 smithi123 sudo[30375]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-18T23:37:24.798 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:37:24 smithi123 sudo[30375]: pam_unix(sudo:session): session closed for user root 2024-08-18T23:37:24.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:24.938+0000 7f299affd640 1 -- 172.21.15.78:0/3933185410 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f29a4097660 con 0x7f29a8101a40 2024-08-18T23:37:24.955 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:37:24 smithi161 sudo[30576]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-18T23:37:24.955 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:37:24 smithi161 sudo[30576]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-18T23:37:24.955 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:37:24 smithi161 sudo[30576]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-18T23:37:24.955 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:37:24 smithi161 sudo[30576]: pam_unix(sudo:session): session closed for user root 2024-08-18T23:37:25.427 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:37:24 smithi161 sudo[30579]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-18T23:37:25.427 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:37:24 smithi161 sudo[30579]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-18T23:37:25.427 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:37:24 smithi161 sudo[30579]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-18T23:37:25.427 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:37:24 smithi161 sudo[30579]: pam_unix(sudo:session): session closed for user root 2024-08-18T23:37:25.482 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:25 smithi078 sudo[37115]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-18T23:37:25.482 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:25 smithi078 sudo[37115]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-18T23:37:25.482 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:25 smithi078 sudo[37115]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-18T23:37:25.482 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:25 smithi078 sudo[37115]: pam_unix(sudo:session): session closed for user root 2024-08-18T23:37:25.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:25 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:25.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:25 smithi161 ceph-mon[22027]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-18T23:37:25.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:25 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:25.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:25 smithi161 ceph-mon[22027]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-18T23:37:25.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:25 smithi161 sudo[30582]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-18T23:37:25.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:25 smithi161 sudo[30582]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-18T23:37:25.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:25 smithi161 sudo[30582]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-18T23:37:25.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:25 smithi161 sudo[30582]: pam_unix(sudo:session): session closed for user root 2024-08-18T23:37:25.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:25 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:25.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:25 smithi123 ceph-mon[21808]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-18T23:37:25.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:25 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:25.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:25 smithi123 ceph-mon[21808]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-18T23:37:25.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:25 smithi078 sudo[37303]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-18T23:37:25.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:25 smithi078 sudo[37303]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-18T23:37:25.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:25 smithi078 sudo[37303]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-18T23:37:25.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:25 smithi078 sudo[37303]: pam_unix(sudo:session): session closed for user root 2024-08-18T23:37:25.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:25 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:25.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:25 smithi078 ceph-mon[18440]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-18T23:37:25.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:25 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:25.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:25 smithi078 ceph-mon[18440]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-18T23:37:26.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:25 smithi161 sudo[30585]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-18T23:37:26.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:25 smithi161 sudo[30585]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-18T23:37:26.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:25 smithi161 sudo[30585]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-18T23:37:26.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:25 smithi161 sudo[30585]: pam_unix(sudo:session): session closed for user root 2024-08-18T23:37:26.485 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 sudo[30378]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-18T23:37:26.486 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 sudo[30378]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-18T23:37:26.486 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 sudo[30378]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-18T23:37:26.486 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 sudo[30378]: pam_unix(sudo:session): session closed for user root 2024-08-18T23:37:26.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 sudo[30381]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-18T23:37:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 sudo[30381]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-18T23:37:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 sudo[30381]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-18T23:37:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 sudo[30381]: pam_unix(sudo:session): session closed for user root 2024-08-18T23:37:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 ceph-mon[21808]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-18T23:37:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-18T23:37:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:37:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:37:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 ceph-mon[21808]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-18T23:37:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-18T23:37:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:37:26.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:37:26.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 ceph-mon[21808]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-18T23:37:26.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-18T23:37:26.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:37:26.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:37:26.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:26 smithi123 ceph-mon[21808]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-08-18T23:37:26.799 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:26 smithi078 ceph-mon[18440]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-18T23:37:26.799 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-18T23:37:26.799 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:37:26.800 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:37:26.800 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:26 smithi078 ceph-mon[18440]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-18T23:37:26.800 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-18T23:37:26.800 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:37:26.800 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:37:26.800 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:26 smithi078 ceph-mon[18440]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-18T23:37:26.800 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-18T23:37:26.800 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:37:26.800 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:37:26.800 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:26 smithi078 ceph-mon[18440]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-08-18T23:37:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:26 smithi161 ceph-mon[22027]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-18T23:37:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-18T23:37:26.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:37:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:37:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:26 smithi161 ceph-mon[22027]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-18T23:37:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-18T23:37:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:37:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:37:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:26 smithi161 ceph-mon[22027]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-18T23:37:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-18T23:37:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-18T23:37:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-18T23:37:26.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:26 smithi161 ceph-mon[22027]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-08-18T23:37:27.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:27 smithi123 ceph-mon[21808]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-18T23:37:27.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:27 smithi123 ceph-mon[21808]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-18T23:37:27.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:27 smithi161 ceph-mon[22027]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-18T23:37:27.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:27 smithi161 ceph-mon[22027]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-18T23:37:27.932 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:27 smithi078 ceph-mon[18440]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-18T23:37:27.932 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:27 smithi078 ceph-mon[18440]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-18T23:37:28.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:28 smithi123 ceph-mon[21808]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-08-18T23:37:28.917 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:28 smithi078 ceph-mon[18440]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-08-18T23:37:28.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:28 smithi161 ceph-mon[22027]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-08-18T23:37:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:31 smithi161 ceph-mon[22027]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-08-18T23:37:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:31 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-08-18T23:37:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:31 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:31.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:31 smithi078 ceph-mon[18440]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-08-18T23:37:31.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:31 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-08-18T23:37:31.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:31 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:31.502 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:31 smithi123 ceph-mon[21808]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-08-18T23:37:31.502 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:31 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-08-18T23:37:31.502 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:31 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:32.317 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:32 smithi078 ceph-mon[18440]: Deploying daemon osd.3 on smithi078 2024-08-18T23:37:32.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:32 smithi161 ceph-mon[22027]: Deploying daemon osd.3 on smithi078 2024-08-18T23:37:32.516 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:32 smithi123 ceph-mon[21808]: Deploying daemon osd.3 on smithi078 2024-08-18T23:37:33.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:33 smithi161 ceph-mon[22027]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-08-18T23:37:33.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:33 smithi078 ceph-mon[18440]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-08-18T23:37:33.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:33 smithi123 ceph-mon[21808]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-08-18T23:37:35.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:35 smithi161 ceph-mon[22027]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-08-18T23:37:35.440 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:35 smithi078 ceph-mon[18440]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-08-18T23:37:35.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:35 smithi123 ceph-mon[21808]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-08-18T23:37:36.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:36 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:37:36.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:36 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:36.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:36 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:36.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:36 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:37:36.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:36 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:36.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:36 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:36.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:36 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:37:36.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:36 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:36.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:36 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:37.358 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:37 smithi078 ceph-mon[18440]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-08-18T23:37:37.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:37 smithi161 ceph-mon[22027]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-08-18T23:37:37.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:37 smithi123 ceph-mon[21808]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-08-18T23:37:38.982 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 3 on host 'smithi078' 2024-08-18T23:37:38.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:38.979+0000 7f299affd640 1 -- 172.21.15.78:0/3933185410 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f29a81a2860 con 0x7f29940a2b20 2024-08-18T23:37:38.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:38.981+0000 7f29afee7640 1 -- 172.21.15.78:0/3933185410 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f29940a2b20 msgr2=0x7f29940a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:37:38.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:38.981+0000 7f29afee7640 1 --2- 172.21.15.78:0/3933185410 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f29940a2b20 0x7f29940a4fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f299c006fd0 tx=0x7f299c008040 comp rx=0 tx=0).stop 2024-08-18T23:37:38.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:38.981+0000 7f29afee7640 1 -- 172.21.15.78:0/3933185410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8101a40 msgr2=0x7f29a819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:37:38.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:38.981+0000 7f29afee7640 1 --2- 172.21.15.78:0/3933185410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8101a40 0x7f29a819a4f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f29a400b4a0 tx=0x7f29a400b960 comp rx=0 tx=0).stop 2024-08-18T23:37:38.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:38.981+0000 7f29afee7640 1 -- 172.21.15.78:0/3933185410 shutdown_connections 2024-08-18T23:37:38.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:38.981+0000 7f29afee7640 1 --2- 172.21.15.78:0/3933185410 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f29940a2b20 0x7f29940a4fe0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:37:38.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:38.981+0000 7f29afee7640 1 --2- 172.21.15.78:0/3933185410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a8102400 0x7f29a81a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:37:38.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:38.981+0000 7f29afee7640 1 --2- 172.21.15.78:0/3933185410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8101a40 0x7f29a819a4f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:37:38.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:38.981+0000 7f29afee7640 1 --2- 172.21.15.78:0/3933185410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29a8100840 0x7f29a8199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:37:38.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:38.981+0000 7f29afee7640 1 -- 172.21.15.78:0/3933185410 >> 172.21.15.78:0/3933185410 conn(0x7f29a80fbfd0 msgr2=0x7f29a8107210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:37:38.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:38.982+0000 7f29afee7640 1 -- 172.21.15.78:0/3933185410 shutdown_connections 2024-08-18T23:37:38.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:37:38.982+0000 7f29afee7640 1 -- 172.21.15.78:0/3933185410 wait complete. 2024-08-18T23:37:39.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:39 smithi078 ceph-mon[18440]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-08-18T23:37:39.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:39.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:39.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:39.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:37:39.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:39.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:37:39.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:39.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:39.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:39 smithi161 ceph-mon[22027]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-08-18T23:37:39.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:39.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:39.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:37:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:37:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:39.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:39.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:39 smithi123 ceph-mon[21808]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-08-18T23:37:39.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:39.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:39.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:39.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:37:39.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:39.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:37:39.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:39.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:39.581 DEBUG:teuthology.orchestra.run.smithi078:osd.3> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.3.service 2024-08-18T23:37:39.583 INFO:tasks.cephadm:Deploying osd.4 on smithi123 with /dev/vg_nvme/lv_3... 2024-08-18T23:37:39.583 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-18T23:37:39.853 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.b/config 2024-08-18T23:37:40.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:40 smithi123 ceph-mon[21808]: from='osd.3 [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-08-18T23:37:40.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:40 smithi161 ceph-mon[22027]: from='osd.3 [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-08-18T23:37:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:40 smithi078 ceph-mon[18440]: from='osd.3 [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-08-18T23:37:41.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:41 smithi123 ceph-mon[21808]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-08-18T23:37:41.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:41 smithi123 ceph-mon[21808]: from='osd.3 [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-18T23:37:41.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:41 smithi123 ceph-mon[21808]: osdmap e23: 4 total, 3 up, 4 in 2024-08-18T23:37:41.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:41 smithi123 ceph-mon[21808]: from='osd.3 [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-08-18T23:37:41.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:41 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-18T23:37:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:41 smithi078 ceph-mon[18440]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-08-18T23:37:41.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:41 smithi078 ceph-mon[18440]: from='osd.3 [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-18T23:37:41.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:41 smithi078 ceph-mon[18440]: osdmap e23: 4 total, 3 up, 4 in 2024-08-18T23:37:41.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:41 smithi078 ceph-mon[18440]: from='osd.3 [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-08-18T23:37:41.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:41 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-18T23:37:41.599 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:41 smithi161 ceph-mon[22027]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-08-18T23:37:41.599 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:41 smithi161 ceph-mon[22027]: from='osd.3 [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-18T23:37:41.599 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:41 smithi161 ceph-mon[22027]: osdmap e23: 4 total, 3 up, 4 in 2024-08-18T23:37:41.599 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:41 smithi161 ceph-mon[22027]: from='osd.3 [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-08-18T23:37:41.599 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:41 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-18T23:37:42.185 INFO:journalctl@ceph.osd.3.smithi078.stdout:Aug 18 23:37:41 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-3[38685]: 2024-08-18T23:37:41.864+0000 7f641f909640 -1 osd.3 0 waiting for initial osdmap 2024-08-18T23:37:42.185 INFO:journalctl@ceph.osd.3.smithi078.stdout:Aug 18 23:37:41 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-3[38685]: 2024-08-18T23:37:41.870+0000 7f6416f15640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T23:37:42.255 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-18T23:37:42.278 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_3 2024-08-18T23:37:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:42 smithi078 ceph-mon[18440]: from='osd.3 [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-08-18T23:37:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:42 smithi078 ceph-mon[18440]: osdmap e24: 4 total, 3 up, 4 in 2024-08-18T23:37:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-18T23:37:42.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-18T23:37:42.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:42 smithi078 ceph-mon[18440]: from='osd.3 [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521]' entity='osd.3' 2024-08-18T23:37:42.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:42 smithi078 ceph-mon[18440]: osd.3 [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521] boot 2024-08-18T23:37:42.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:42 smithi078 ceph-mon[18440]: osdmap e25: 4 total, 4 up, 4 in 2024-08-18T23:37:42.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-18T23:37:42.517 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:42 smithi161 ceph-mon[22027]: from='osd.3 [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-08-18T23:37:42.518 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:42 smithi161 ceph-mon[22027]: osdmap e24: 4 total, 3 up, 4 in 2024-08-18T23:37:42.518 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:42 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-18T23:37:42.518 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:42 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-18T23:37:42.518 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:42 smithi161 ceph-mon[22027]: from='osd.3 [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521]' entity='osd.3' 2024-08-18T23:37:42.518 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:42 smithi161 ceph-mon[22027]: osd.3 [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521] boot 2024-08-18T23:37:42.518 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:42 smithi161 ceph-mon[22027]: osdmap e25: 4 total, 4 up, 4 in 2024-08-18T23:37:42.518 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:42 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-18T23:37:42.523 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.b/config 2024-08-18T23:37:42.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:42 smithi123 ceph-mon[21808]: from='osd.3 [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-08-18T23:37:42.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:42 smithi123 ceph-mon[21808]: osdmap e24: 4 total, 3 up, 4 in 2024-08-18T23:37:42.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:42 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-18T23:37:42.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:42 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-18T23:37:42.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:42 smithi123 ceph-mon[21808]: from='osd.3 [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521]' entity='osd.3' 2024-08-18T23:37:42.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:42 smithi123 ceph-mon[21808]: osd.3 [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521] boot 2024-08-18T23:37:42.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:42 smithi123 ceph-mon[21808]: osdmap e25: 4 total, 4 up, 4 in 2024-08-18T23:37:42.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:42 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-18T23:37:42.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.996+0000 7f41b46a4640 1 -- 172.21.15.123:0/2715172812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ac101a40 msgr2=0x7f41ac101ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:37:42.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.996+0000 7f41b46a4640 1 --2- 172.21.15.123:0/2715172812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ac101a40 0x7f41ac101ec0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f41a8009900 tx=0x7f41a802a410 comp rx=0 tx=0).stop 2024-08-18T23:37:42.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.996+0000 7f41b46a4640 1 -- 172.21.15.123:0/2715172812 shutdown_connections 2024-08-18T23:37:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.996+0000 7f41b46a4640 1 --2- 172.21.15.123:0/2715172812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ac102400 0x7f41ac108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:37:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.996+0000 7f41b46a4640 1 --2- 172.21.15.123:0/2715172812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ac101a40 0x7f41ac101ec0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:37:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.996+0000 7f41b46a4640 1 --2- 172.21.15.123:0/2715172812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41ac100840 0x7f41ac100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:37:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.996+0000 7f41b46a4640 1 -- 172.21.15.123:0/2715172812 >> 172.21.15.123:0/2715172812 conn(0x7f41ac0fbfd0 msgr2=0x7f41ac0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:37:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.996+0000 7f41b46a4640 1 -- 172.21.15.123:0/2715172812 shutdown_connections 2024-08-18T23:37:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.996+0000 7f41b46a4640 1 -- 172.21.15.123:0/2715172812 wait complete. 2024-08-18T23:37:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.997+0000 7f41b46a4640 1 Processor -- start 2024-08-18T23:37:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.997+0000 7f41b46a4640 1 -- start start 2024-08-18T23:37:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.997+0000 7f41b46a4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41ac100840 0x7f41ac06b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:37:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.997+0000 7f41b46a4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ac101a40 0x7f41ac19ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:37:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.997+0000 7f41b46a4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ac102400 0x7f41ac19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:37:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.997+0000 7f41b46a4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac10c520 con 0x7f41ac100840 2024-08-18T23:37:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.997+0000 7f41b46a4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac10c3a0 con 0x7f41ac101a40 2024-08-18T23:37:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.997+0000 7f41b46a4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac10c6a0 con 0x7f41ac102400 2024-08-18T23:37:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.997+0000 7f41b1c3f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ac101a40 0x7f41ac19ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:37:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.997+0000 7f41b1c3f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ac101a40 0x7f41ac19ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41096/0 (socket says 172.21.15.123:41096) 2024-08-18T23:37:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.997+0000 7f41b1c3f640 1 -- 172.21.15.123:0/198372830 learned_addr learned my addr 172.21.15.123:0/198372830 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-08-18T23:37:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.997+0000 7f41b2440640 1 --2- 172.21.15.123:0/198372830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41ac100840 0x7f41ac06b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:37:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.997+0000 7f41b2c41640 1 --2- 172.21.15.123:0/198372830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ac102400 0x7f41ac19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:37:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.997+0000 7f41b1c3f640 1 -- 172.21.15.123:0/198372830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ac102400 msgr2=0x7f41ac19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:37:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.997+0000 7f41b1c3f640 1 --2- 172.21.15.123:0/198372830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ac102400 0x7f41ac19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:37:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.997+0000 7f41b1c3f640 1 -- 172.21.15.123:0/198372830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41ac100840 msgr2=0x7f41ac06b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:37:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.997+0000 7f41b1c3f640 1 --2- 172.21.15.123:0/198372830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41ac100840 0x7f41ac06b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:37:43.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.997+0000 7f41b1c3f640 1 -- 172.21.15.123:0/198372830 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ac19f930 con 0x7f41ac101a40 2024-08-18T23:37:43.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.997+0000 7f41b2c41640 1 --2- 172.21.15.123:0/198372830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ac102400 0x7f41ac19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:37:43.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.998+0000 7f41b1c3f640 1 --2- 172.21.15.123:0/198372830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ac101a40 0x7f41ac19ca00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f41a8009230 tx=0x7f41a8031b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:37:43.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.998+0000 7f419f7fe640 1 -- 172.21.15.123:0/198372830 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a803f030 con 0x7f41ac101a40 2024-08-18T23:37:43.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.998+0000 7f419f7fe640 1 -- 172.21.15.123:0/198372830 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a8003ad0 con 0x7f41ac101a40 2024-08-18T23:37:43.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.998+0000 7f419f7fe640 1 -- 172.21.15.123:0/198372830 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a8035070 con 0x7f41ac101a40 2024-08-18T23:37:43.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.998+0000 7f41b46a4640 1 -- 172.21.15.123:0/198372830 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ac19fb60 con 0x7f41ac101a40 2024-08-18T23:37:43.002 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.998+0000 7f41b46a4640 1 -- 172.21.15.123:0/198372830 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ac1a0040 con 0x7f41ac101a40 2024-08-18T23:37:43.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:42.999+0000 7f41b46a4640 1 -- 172.21.15.123:0/198372830 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41ac107c90 con 0x7f41ac101a40 2024-08-18T23:37:43.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:43.000+0000 7f419f7fe640 1 -- 172.21.15.123:0/198372830 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f41a8042030 con 0x7f41ac101a40 2024-08-18T23:37:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:43.001+0000 7f419f7fe640 1 --2- 172.21.15.123:0/198372830 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f41980a2b80 0x7f41980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:37:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:43.001+0000 7f419f7fe640 1 -- 172.21.15.123:0/198372830 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3702+0+0 (secure 0 0 0) 0x7f41a80ee3e0 con 0x7f41ac101a40 2024-08-18T23:37:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:43.001+0000 7f41b2440640 1 --2- 172.21.15.123:0/198372830 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f41980a2b80 0x7f41980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:37:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:43.001+0000 7f41b2440640 1 --2- 172.21.15.123:0/198372830 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f41980a2b80 0x7f41980a5040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f41ac1018a0 tx=0x7f41a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:37:43.006 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:43.005+0000 7f419f7fe640 1 -- 172.21.15.123:0/198372830 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f41a80bc540 con 0x7f41ac101a40 2024-08-18T23:37:43.181 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:37:43.180+0000 7f41b46a4640 1 -- 172.21.15.123:0/198372830 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f41ac062360 con 0x7f41980a2b80 2024-08-18T23:37:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:43 smithi078 ceph-mon[18440]: purged_snaps scrub starts 2024-08-18T23:37:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:43 smithi078 ceph-mon[18440]: purged_snaps scrub ok 2024-08-18T23:37:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:43 smithi078 ceph-mon[18440]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-08-18T23:37:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:43 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:43 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:43.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:43 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:37:43.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:43 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:37:43.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:43 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:43.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:43 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:43.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:43 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:37:43.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:43 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:43.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:43 smithi078 ceph-mon[18440]: osdmap e26: 4 total, 4 up, 4 in 2024-08-18T23:37:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:43 smithi123 ceph-mon[21808]: purged_snaps scrub starts 2024-08-18T23:37:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:43 smithi123 ceph-mon[21808]: purged_snaps scrub ok 2024-08-18T23:37:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:43 smithi123 ceph-mon[21808]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-08-18T23:37:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:43 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:43 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:43 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:37:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:43 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:37:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:43 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:43 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:43 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:37:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:43 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:43.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:43 smithi123 ceph-mon[21808]: osdmap e26: 4 total, 4 up, 4 in 2024-08-18T23:37:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:43 smithi161 ceph-mon[22027]: purged_snaps scrub starts 2024-08-18T23:37:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:43 smithi161 ceph-mon[22027]: purged_snaps scrub ok 2024-08-18T23:37:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:43 smithi161 ceph-mon[22027]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-08-18T23:37:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:43 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:43.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:43 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:43.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:43 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:37:43.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:43 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:37:43.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:43 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:43.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:43 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:43.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:43 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:37:43.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:43 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:43.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:43 smithi161 ceph-mon[22027]: osdmap e26: 4 total, 4 up, 4 in 2024-08-18T23:37:44.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:44 smithi078 ceph-mon[18440]: Detected new or changed devices on smithi078 2024-08-18T23:37:44.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:44 smithi078 ceph-mon[18440]: Adjusting osd_memory_target on smithi078 to 8544M 2024-08-18T23:37:44.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:44 smithi078 ceph-mon[18440]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:37:44.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:44 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:37:44.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:44 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:37:44.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:44 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:44.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:44 smithi078 ceph-mon[18440]: osdmap e27: 4 total, 4 up, 4 in 2024-08-18T23:37:44.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:44 smithi123 ceph-mon[21808]: Detected new or changed devices on smithi078 2024-08-18T23:37:44.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:44 smithi123 ceph-mon[21808]: Adjusting osd_memory_target on smithi078 to 8544M 2024-08-18T23:37:44.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:44 smithi123 ceph-mon[21808]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:37:44.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:44 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:37:44.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:44 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:37:44.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:44 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:44.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:44 smithi123 ceph-mon[21808]: osdmap e27: 4 total, 4 up, 4 in 2024-08-18T23:37:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:44 smithi161 ceph-mon[22027]: Detected new or changed devices on smithi078 2024-08-18T23:37:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:44 smithi161 ceph-mon[22027]: Adjusting osd_memory_target on smithi078 to 8544M 2024-08-18T23:37:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:44 smithi161 ceph-mon[22027]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:37:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:44 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:37:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:44 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:37:44.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:44 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:44.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:44 smithi161 ceph-mon[22027]: osdmap e27: 4 total, 4 up, 4 in 2024-08-18T23:37:45.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:45 smithi078 ceph-mon[18440]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 357 GiB / 358 GiB avail 2024-08-18T23:37:45.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:45 smithi123 ceph-mon[21808]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 357 GiB / 358 GiB avail 2024-08-18T23:37:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:45 smithi161 ceph-mon[22027]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 357 GiB / 358 GiB avail 2024-08-18T23:37:46.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:46 smithi078 ceph-mon[18440]: osdmap e28: 4 total, 4 up, 4 in 2024-08-18T23:37:46.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:46 smithi078 ceph-mon[18440]: from='client.? 172.21.15.123:0/3559671048' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ed26775-79e3-4b24-9557-69fcf5e8e824"}]: dispatch 2024-08-18T23:37:46.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:46 smithi078 ceph-mon[18440]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ed26775-79e3-4b24-9557-69fcf5e8e824"}]: dispatch 2024-08-18T23:37:46.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:46 smithi078 ceph-mon[18440]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ed26775-79e3-4b24-9557-69fcf5e8e824"}]': finished 2024-08-18T23:37:46.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:46 smithi078 ceph-mon[18440]: osdmap e29: 5 total, 4 up, 5 in 2024-08-18T23:37:46.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:46 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-18T23:37:46.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:46 smithi078 ceph-mon[18440]: from='client.? 172.21.15.123:0/1140572545' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:37:46.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:46 smithi123 ceph-mon[21808]: osdmap e28: 4 total, 4 up, 4 in 2024-08-18T23:37:46.620 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:46 smithi123 ceph-mon[21808]: from='client.? 172.21.15.123:0/3559671048' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ed26775-79e3-4b24-9557-69fcf5e8e824"}]: dispatch 2024-08-18T23:37:46.621 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:46 smithi123 ceph-mon[21808]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ed26775-79e3-4b24-9557-69fcf5e8e824"}]: dispatch 2024-08-18T23:37:46.621 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:46 smithi123 ceph-mon[21808]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ed26775-79e3-4b24-9557-69fcf5e8e824"}]': finished 2024-08-18T23:37:46.621 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:46 smithi123 ceph-mon[21808]: osdmap e29: 5 total, 4 up, 5 in 2024-08-18T23:37:46.621 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:46 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-18T23:37:46.621 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:46 smithi123 ceph-mon[21808]: from='client.? 172.21.15.123:0/1140572545' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:37:46.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:46 smithi161 ceph-mon[22027]: osdmap e28: 4 total, 4 up, 4 in 2024-08-18T23:37:46.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:46 smithi161 ceph-mon[22027]: from='client.? 172.21.15.123:0/3559671048' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ed26775-79e3-4b24-9557-69fcf5e8e824"}]: dispatch 2024-08-18T23:37:46.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:46 smithi161 ceph-mon[22027]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ed26775-79e3-4b24-9557-69fcf5e8e824"}]: dispatch 2024-08-18T23:37:46.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:46 smithi161 ceph-mon[22027]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ed26775-79e3-4b24-9557-69fcf5e8e824"}]': finished 2024-08-18T23:37:46.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:46 smithi161 ceph-mon[22027]: osdmap e29: 5 total, 4 up, 5 in 2024-08-18T23:37:46.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:46 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-18T23:37:46.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:46 smithi161 ceph-mon[22027]: from='client.? 172.21.15.123:0/1140572545' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:37:47.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:47 smithi078 ceph-mon[18440]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 384 MiB used, 357 GiB / 358 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-08-18T23:37:47.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:47 smithi123 ceph-mon[21808]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 384 MiB used, 357 GiB / 358 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-08-18T23:37:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:47 smithi161 ceph-mon[22027]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 384 MiB used, 357 GiB / 358 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-08-18T23:37:49.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:49 smithi123 ceph-mon[21808]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-08-18T23:37:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:49 smithi078 ceph-mon[18440]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-08-18T23:37:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:49 smithi161 ceph-mon[22027]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-08-18T23:37:51.371 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:51 smithi123 ceph-mon[21808]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-18T23:37:51.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:51 smithi078 ceph-mon[18440]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-18T23:37:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:51 smithi161 ceph-mon[22027]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-18T23:37:53.416 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:53 smithi123 ceph-mon[21808]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-08-18T23:37:53.416 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:53 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-08-18T23:37:53.417 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:53 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:53.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:53 smithi078 ceph-mon[18440]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-08-18T23:37:53.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:53 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-08-18T23:37:53.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:53 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:53 smithi161 ceph-mon[22027]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-08-18T23:37:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-08-18T23:37:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:37:54.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:54 smithi078 ceph-mon[18440]: Deploying daemon osd.4 on smithi123 2024-08-18T23:37:54.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:54 smithi123 ceph-mon[21808]: Deploying daemon osd.4 on smithi123 2024-08-18T23:37:54.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:54 smithi161 ceph-mon[22027]: Deploying daemon osd.4 on smithi123 2024-08-18T23:37:55.522 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:55 smithi123 ceph-mon[21808]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-08-18T23:37:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:55 smithi161 ceph-mon[22027]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-08-18T23:37:55.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:55 smithi078 ceph-mon[18440]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-08-18T23:37:57.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:57 smithi123 ceph-mon[21808]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-18T23:37:57.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:37:57.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:57.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:57 smithi161 ceph-mon[22027]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-18T23:37:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:37:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:57.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:57 smithi078 ceph-mon[18440]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-18T23:37:57.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:37:57.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:57.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:37:59.456 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:37:59 smithi123 ceph-mon[21808]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-18T23:37:59.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:37:59 smithi078 ceph-mon[18440]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-18T23:37:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:37:59 smithi161 ceph-mon[22027]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-18T23:38:00.536 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:00 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:00.536 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:00 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:00.536 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:00 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:00.536 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:00 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:38:00.536 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:00 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:00.578 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:00.576+0000 7f419f7fe640 1 -- 172.21.15.123:0/198372830 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f41ac062360 con 0x7f41980a2b80 2024-08-18T23:38:00.578 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 4 on host 'smithi123' 2024-08-18T23:38:00.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:00.579+0000 7f41b46a4640 1 -- 172.21.15.123:0/198372830 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f41980a2b80 msgr2=0x7f41980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:38:00.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:00.579+0000 7f41b46a4640 1 --2- 172.21.15.123:0/198372830 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f41980a2b80 0x7f41980a5040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f41ac1018a0 tx=0x7f41a0005f90 comp rx=0 tx=0).stop 2024-08-18T23:38:00.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:00.579+0000 7f41b46a4640 1 -- 172.21.15.123:0/198372830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ac101a40 msgr2=0x7f41ac19ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:38:00.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:00.579+0000 7f41b46a4640 1 --2- 172.21.15.123:0/198372830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ac101a40 0x7f41ac19ca00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f41a8009230 tx=0x7f41a8031b60 comp rx=0 tx=0).stop 2024-08-18T23:38:00.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:00.579+0000 7f41b46a4640 1 -- 172.21.15.123:0/198372830 shutdown_connections 2024-08-18T23:38:00.580 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:00.579+0000 7f41b46a4640 1 --2- 172.21.15.123:0/198372830 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f41980a2b80 0x7f41980a5040 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:00.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:00.579+0000 7f41b46a4640 1 --2- 172.21.15.123:0/198372830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41ac102400 0x7f41ac19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:00.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:00.579+0000 7f41b46a4640 1 --2- 172.21.15.123:0/198372830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ac101a40 0x7f41ac19ca00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:00.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:00.579+0000 7f41b46a4640 1 --2- 172.21.15.123:0/198372830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41ac100840 0x7f41ac06b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:00.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:00.579+0000 7f41b46a4640 1 -- 172.21.15.123:0/198372830 >> 172.21.15.123:0/198372830 conn(0x7f41ac0fbfd0 msgr2=0x7f41ac104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:38:00.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:00.579+0000 7f41b46a4640 1 -- 172.21.15.123:0/198372830 shutdown_connections 2024-08-18T23:38:00.581 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:00.579+0000 7f41b46a4640 1 -- 172.21.15.123:0/198372830 wait complete. 2024-08-18T23:38:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:00 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:00 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:00 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:00.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:00 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:38:00.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:00 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:00.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:00 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:00.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:00 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:00.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:00 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:00.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:00 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:38:00.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:00 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:01.208 DEBUG:teuthology.orchestra.run.smithi123:osd.4> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.4.service 2024-08-18T23:38:01.210 INFO:tasks.cephadm:Deploying osd.5 on smithi161 with /dev/vg_nvme/lv_3... 2024-08-18T23:38:01.211 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-18T23:38:01.471 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.c/config 2024-08-18T23:38:01.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:01 smithi123 ceph-mon[21808]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-18T23:38:01.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:01 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:38:01.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:01 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:01.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:01 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:01.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:01 smithi123 ceph-mon[21808]: from='osd.4 [v2:172.21.15.123:6808/1281511569,v1:172.21.15.123:6809/1281511569]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-18T23:38:01.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:01 smithi123 ceph-mon[21808]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-18T23:38:01.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:01 smithi161 ceph-mon[22027]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-18T23:38:01.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:01 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:38:01.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:01 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:01.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:01 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:01.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:01 smithi161 ceph-mon[22027]: from='osd.4 [v2:172.21.15.123:6808/1281511569,v1:172.21.15.123:6809/1281511569]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-18T23:38:01.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:01 smithi161 ceph-mon[22027]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-18T23:38:01.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:01 smithi078 ceph-mon[18440]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-18T23:38:01.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:01 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:38:01.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:01 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:01.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:01 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:01.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:01 smithi078 ceph-mon[18440]: from='osd.4 [v2:172.21.15.123:6808/1281511569,v1:172.21.15.123:6809/1281511569]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-18T23:38:01.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:01 smithi078 ceph-mon[18440]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-18T23:38:02.672 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:02 smithi161 ceph-mon[22027]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-18T23:38:02.672 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:02 smithi161 ceph-mon[22027]: from='osd.4 [v2:172.21.15.123:6808/1281511569,v1:172.21.15.123:6809/1281511569]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:38:02.672 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:02 smithi161 ceph-mon[22027]: osdmap e30: 5 total, 4 up, 5 in 2024-08-18T23:38:02.672 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:02 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-18T23:38:02.672 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:02 smithi161 ceph-mon[22027]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:38:02.672 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:02 smithi161 ceph-mon[22027]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-18T23:38:02.732 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:02 smithi123 ceph-mon[21808]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-18T23:38:02.732 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:02 smithi123 ceph-mon[21808]: from='osd.4 [v2:172.21.15.123:6808/1281511569,v1:172.21.15.123:6809/1281511569]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:38:02.732 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:02 smithi123 ceph-mon[21808]: osdmap e30: 5 total, 4 up, 5 in 2024-08-18T23:38:02.733 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-18T23:38:02.733 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:02 smithi123 ceph-mon[21808]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:38:02.733 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:02 smithi123 ceph-mon[21808]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-18T23:38:02.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:02 smithi078 ceph-mon[18440]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-18T23:38:02.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:02 smithi078 ceph-mon[18440]: from='osd.4 [v2:172.21.15.123:6808/1281511569,v1:172.21.15.123:6809/1281511569]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:38:02.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:02 smithi078 ceph-mon[18440]: osdmap e30: 5 total, 4 up, 5 in 2024-08-18T23:38:02.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:02 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-18T23:38:02.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:02 smithi078 ceph-mon[18440]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:38:02.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:02 smithi078 ceph-mon[18440]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-18T23:38:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:03 smithi161 ceph-mon[22027]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-08-18T23:38:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:03 smithi161 ceph-mon[22027]: osdmap e31: 5 total, 4 up, 5 in 2024-08-18T23:38:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:03 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-18T23:38:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:03 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-18T23:38:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:03 smithi161 ceph-mon[22027]: from='osd.4 ' entity='osd.4' 2024-08-18T23:38:03.724 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:38:03.750 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_3 2024-08-18T23:38:03.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:03 smithi123 ceph-mon[21808]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-08-18T23:38:03.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:03 smithi123 ceph-mon[21808]: osdmap e31: 5 total, 4 up, 5 in 2024-08-18T23:38:03.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:03 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-18T23:38:03.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:03 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-18T23:38:03.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:03 smithi123 ceph-mon[21808]: from='osd.4 ' entity='osd.4' 2024-08-18T23:38:03.798 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:38:03 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-4[32991]: 2024-08-18T23:38:03.402+0000 7f4dea20a640 -1 osd.4 0 waiting for initial osdmap 2024-08-18T23:38:03.798 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:38:03 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-4[32991]: 2024-08-18T23:38:03.409+0000 7f4de1816640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T23:38:03.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:03 smithi078 ceph-mon[18440]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-08-18T23:38:03.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:03 smithi078 ceph-mon[18440]: osdmap e31: 5 total, 4 up, 5 in 2024-08-18T23:38:03.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:03 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-18T23:38:03.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:03 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-18T23:38:03.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:03 smithi078 ceph-mon[18440]: from='osd.4 ' entity='osd.4' 2024-08-18T23:38:04.000 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.c/config 2024-08-18T23:38:04.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.434+0000 7f8cb9bc2640 1 -- 172.21.15.161:0/1821045439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cb4100840 msgr2=0x7f8cb4100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:38:04.465 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.434+0000 7f8cb9bc2640 1 --2- 172.21.15.161:0/1821045439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cb4100840 0x7f8cb4100c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4009900 tx=0x7f8ca402a410 comp rx=0 tx=0).stop 2024-08-18T23:38:04.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.434+0000 7f8cb9bc2640 1 -- 172.21.15.161:0/1821045439 shutdown_connections 2024-08-18T23:38:04.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.434+0000 7f8cb9bc2640 1 --2- 172.21.15.161:0/1821045439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb4102400 0x7f8cb4108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:04.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.434+0000 7f8cb9bc2640 1 --2- 172.21.15.161:0/1821045439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cb4101a40 0x7f8cb4101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:04.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.434+0000 7f8cb9bc2640 1 --2- 172.21.15.161:0/1821045439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cb4100840 0x7f8cb4100c40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:04.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.434+0000 7f8cb9bc2640 1 -- 172.21.15.161:0/1821045439 >> 172.21.15.161:0/1821045439 conn(0x7f8cb40fbfd0 msgr2=0x7f8cb40fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:38:04.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.434+0000 7f8cb9bc2640 1 -- 172.21.15.161:0/1821045439 shutdown_connections 2024-08-18T23:38:04.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.434+0000 7f8cb9bc2640 1 -- 172.21.15.161:0/1821045439 wait complete. 2024-08-18T23:38:04.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.434+0000 7f8cb9bc2640 1 Processor -- start 2024-08-18T23:38:04.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.434+0000 7f8cb9bc2640 1 -- start start 2024-08-18T23:38:04.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.434+0000 7f8cb9bc2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cb4100840 0x7f8cb406b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:38:04.466 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.434+0000 7f8cb9bc2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cb4101a40 0x7f8cb419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:38:04.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cb9bc2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb4102400 0x7f8cb419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:38:04.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cb9bc2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb410c540 con 0x7f8cb4101a40 2024-08-18T23:38:04.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cb9bc2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb410c3c0 con 0x7f8cb4102400 2024-08-18T23:38:04.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cb9bc2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb410c6c0 con 0x7f8cb4100840 2024-08-18T23:38:04.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cb37fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cb4100840 0x7f8cb406b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:38:04.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cb37fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cb4100840 0x7f8cb406b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:56954/0 (socket says 172.21.15.161:56954) 2024-08-18T23:38:04.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cb37fe640 1 -- 172.21.15.161:0/287244829 learned_addr learned my addr 172.21.15.161:0/287244829 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-18T23:38:04.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cb3fff640 1 --2- 172.21.15.161:0/287244829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb4102400 0x7f8cb419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:38:04.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cabfff640 1 --2- 172.21.15.161:0/287244829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cb4101a40 0x7f8cb419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:38:04.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cb37fe640 1 -- 172.21.15.161:0/287244829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb4102400 msgr2=0x7f8cb419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:38:04.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cb37fe640 1 --2- 172.21.15.161:0/287244829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb4102400 0x7f8cb419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:04.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cb37fe640 1 -- 172.21.15.161:0/287244829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cb4101a40 msgr2=0x7f8cb419ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:38:04.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cb37fe640 1 --2- 172.21.15.161:0/287244829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cb4101a40 0x7f8cb419ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:04.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cb37fe640 1 -- 172.21.15.161:0/287244829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb419f950 con 0x7f8cb4100840 2024-08-18T23:38:04.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cabfff640 1 --2- 172.21.15.161:0/287244829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cb4101a40 0x7f8cb419ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:38:04.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cb37fe640 1 --2- 172.21.15.161:0/287244829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cb4100840 0x7f8cb406b020 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4031610 tx=0x7f8ca40043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:38:04.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cb17fa640 1 -- 172.21.15.161:0/287244829 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca403f030 con 0x7f8cb4100840 2024-08-18T23:38:04.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cb17fa640 1 -- 172.21.15.161:0/287244829 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ca4003ce0 con 0x7f8cb4100840 2024-08-18T23:38:04.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cb9bc2640 1 -- 172.21.15.161:0/287244829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb419fb80 con 0x7f8cb4100840 2024-08-18T23:38:04.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cb17fa640 1 -- 172.21.15.161:0/287244829 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4035070 con 0x7f8cb4100840 2024-08-18T23:38:04.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.435+0000 7f8cb9bc2640 1 -- 172.21.15.161:0/287244829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb41a0060 con 0x7f8cb4100840 2024-08-18T23:38:04.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.436+0000 7f8cb9bc2640 1 -- 172.21.15.161:0/287244829 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cb4107be0 con 0x7f8cb4100840 2024-08-18T23:38:04.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.437+0000 7f8cb17fa640 1 -- 172.21.15.161:0/287244829 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8ca4042030 con 0x7f8cb4100840 2024-08-18T23:38:04.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.438+0000 7f8cb17fa640 1 --2- 172.21.15.161:0/287244829 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8ca00a2b80 0x7f8ca00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:38:04.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.438+0000 7f8cb17fa640 1 -- 172.21.15.161:0/287244829 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7f8ca40ee950 con 0x7f8cb4100840 2024-08-18T23:38:04.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.438+0000 7f8cabfff640 1 --2- 172.21.15.161:0/287244829 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8ca00a2b80 0x7f8ca00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:38:04.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.439+0000 7f8cabfff640 1 --2- 172.21.15.161:0/287244829 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8ca00a2b80 0x7f8ca00a5040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8cb406b5a0 tx=0x7f8c9c005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:38:04.469 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.441+0000 7f8cb17fa640 1 -- 172.21.15.161:0/287244829 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8ca40bc920 con 0x7f8cb4100840 2024-08-18T23:38:04.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:04.619+0000 7f8cb9bc2640 1 -- 172.21.15.161:0/287244829 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8cb4062360 con 0x7f8ca00a2b80 2024-08-18T23:38:04.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:04 smithi161 ceph-mon[22027]: purged_snaps scrub starts 2024-08-18T23:38:04.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:04 smithi161 ceph-mon[22027]: purged_snaps scrub ok 2024-08-18T23:38:04.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-18T23:38:04.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:04 smithi161 ceph-mon[22027]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-18T23:38:04.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:04 smithi161 ceph-mon[22027]: Detected new or changed devices on smithi123 2024-08-18T23:38:04.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:04 smithi161 ceph-mon[22027]: Adjusting osd_memory_target on smithi123 to 8544M 2024-08-18T23:38:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:38:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:04 smithi161 ceph-mon[22027]: osd.4 [v2:172.21.15.123:6808/1281511569,v1:172.21.15.123:6809/1281511569] boot 2024-08-18T23:38:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:04 smithi161 ceph-mon[22027]: osdmap e32: 5 total, 5 up, 5 in 2024-08-18T23:38:04.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-18T23:38:04.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:04 smithi078 ceph-mon[18440]: purged_snaps scrub starts 2024-08-18T23:38:04.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:04 smithi078 ceph-mon[18440]: purged_snaps scrub ok 2024-08-18T23:38:04.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-18T23:38:04.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:04 smithi078 ceph-mon[18440]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-18T23:38:04.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:04 smithi078 ceph-mon[18440]: Detected new or changed devices on smithi123 2024-08-18T23:38:04.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:04.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:04.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:04.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:04.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:04 smithi078 ceph-mon[18440]: Adjusting osd_memory_target on smithi123 to 8544M 2024-08-18T23:38:04.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:04.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:04.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:38:04.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:04.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:04 smithi078 ceph-mon[18440]: osd.4 [v2:172.21.15.123:6808/1281511569,v1:172.21.15.123:6809/1281511569] boot 2024-08-18T23:38:04.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:04 smithi078 ceph-mon[18440]: osdmap e32: 5 total, 5 up, 5 in 2024-08-18T23:38:04.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-18T23:38:05.049 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:04 smithi123 ceph-mon[21808]: purged_snaps scrub starts 2024-08-18T23:38:05.050 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:04 smithi123 ceph-mon[21808]: purged_snaps scrub ok 2024-08-18T23:38:05.050 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-18T23:38:05.050 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:04 smithi123 ceph-mon[21808]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-18T23:38:05.050 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:04 smithi123 ceph-mon[21808]: Detected new or changed devices on smithi123 2024-08-18T23:38:05.050 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:05.050 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:05.050 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:05.050 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:05.051 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:04 smithi123 ceph-mon[21808]: Adjusting osd_memory_target on smithi123 to 8544M 2024-08-18T23:38:05.051 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:05.051 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:05.051 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:38:05.051 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:05.051 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:04 smithi123 ceph-mon[21808]: osd.4 [v2:172.21.15.123:6808/1281511569,v1:172.21.15.123:6809/1281511569] boot 2024-08-18T23:38:05.051 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:04 smithi123 ceph-mon[21808]: osdmap e32: 5 total, 5 up, 5 in 2024-08-18T23:38:05.051 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-18T23:38:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:05 smithi161 ceph-mon[22027]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:38:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:05 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:38:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:05 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:38:05.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:05 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:05.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:05 smithi078 ceph-mon[18440]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:38:05.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:05 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:38:05.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:05 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:38:05.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:05 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:06.050 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:05 smithi123 ceph-mon[21808]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:38:06.050 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:05 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:38:06.050 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:05 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:38:06.050 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:05 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:06.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:06 smithi161 ceph-mon[22027]: osdmap e33: 5 total, 5 up, 5 in 2024-08-18T23:38:06.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:06 smithi161 ceph-mon[22027]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:06.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:06 smithi078 ceph-mon[18440]: osdmap e33: 5 total, 5 up, 5 in 2024-08-18T23:38:06.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:06 smithi078 ceph-mon[18440]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:07.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:06 smithi123 ceph-mon[21808]: osdmap e33: 5 total, 5 up, 5 in 2024-08-18T23:38:07.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:06 smithi123 ceph-mon[21808]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:07.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:07 smithi161 ceph-mon[22027]: from='client.? 172.21.15.161:0/1067750721' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9793b1d-5613-48a4-938e-adb718f4fa4e"}]: dispatch 2024-08-18T23:38:07.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:07 smithi161 ceph-mon[22027]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9793b1d-5613-48a4-938e-adb718f4fa4e"}]: dispatch 2024-08-18T23:38:07.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:07 smithi161 ceph-mon[22027]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9793b1d-5613-48a4-938e-adb718f4fa4e"}]': finished 2024-08-18T23:38:07.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:07 smithi161 ceph-mon[22027]: osdmap e34: 6 total, 5 up, 6 in 2024-08-18T23:38:07.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-18T23:38:07.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:07 smithi161 ceph-mon[22027]: from='client.? 172.21.15.161:0/1161420579' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:38:07.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:07 smithi078 ceph-mon[18440]: from='client.? 172.21.15.161:0/1067750721' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9793b1d-5613-48a4-938e-adb718f4fa4e"}]: dispatch 2024-08-18T23:38:07.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:07 smithi078 ceph-mon[18440]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9793b1d-5613-48a4-938e-adb718f4fa4e"}]: dispatch 2024-08-18T23:38:07.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:07 smithi078 ceph-mon[18440]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9793b1d-5613-48a4-938e-adb718f4fa4e"}]': finished 2024-08-18T23:38:07.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:07 smithi078 ceph-mon[18440]: osdmap e34: 6 total, 5 up, 6 in 2024-08-18T23:38:07.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-18T23:38:07.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:07 smithi078 ceph-mon[18440]: from='client.? 172.21.15.161:0/1161420579' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:38:08.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:07 smithi123 ceph-mon[21808]: from='client.? 172.21.15.161:0/1067750721' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9793b1d-5613-48a4-938e-adb718f4fa4e"}]: dispatch 2024-08-18T23:38:08.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:07 smithi123 ceph-mon[21808]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9793b1d-5613-48a4-938e-adb718f4fa4e"}]: dispatch 2024-08-18T23:38:08.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:07 smithi123 ceph-mon[21808]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9793b1d-5613-48a4-938e-adb718f4fa4e"}]': finished 2024-08-18T23:38:08.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:07 smithi123 ceph-mon[21808]: osdmap e34: 6 total, 5 up, 6 in 2024-08-18T23:38:08.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:07 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-18T23:38:08.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:07 smithi123 ceph-mon[21808]: from='client.? 172.21.15.161:0/1161420579' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:38:08.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:08 smithi161 ceph-mon[22027]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:08.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:08 smithi078 ceph-mon[18440]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:09.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:08 smithi123 ceph-mon[21808]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:11.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:11 smithi161 ceph-mon[22027]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:11.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:11 smithi078 ceph-mon[18440]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:11.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:11 smithi123 ceph-mon[21808]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:13 smithi161 ceph-mon[22027]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:13 smithi078 ceph-mon[18440]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:13.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:13 smithi123 ceph-mon[21808]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:14.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-08-18T23:38:14.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-08-18T23:38:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:14.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-08-18T23:38:14.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:15.422 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:15 smithi161 ceph-mon[22027]: Deploying daemon osd.5 on smithi161 2024-08-18T23:38:15.422 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:15 smithi161 ceph-mon[22027]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:15 smithi078 ceph-mon[18440]: Deploying daemon osd.5 on smithi161 2024-08-18T23:38:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:15 smithi078 ceph-mon[18440]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:15.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:15 smithi123 ceph-mon[21808]: Deploying daemon osd.5 on smithi161 2024-08-18T23:38:15.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:15 smithi123 ceph-mon[21808]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:17.369 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:17 smithi161 ceph-mon[22027]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:17 smithi078 ceph-mon[18440]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:17.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:17 smithi123 ceph-mon[21808]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:19 smithi161 ceph-mon[22027]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:38:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:19.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:19 smithi078 ceph-mon[18440]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:38:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:19.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:19.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:19 smithi123 ceph-mon[21808]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:19.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:38:19.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:19.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:21.399 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:21 smithi161 ceph-mon[22027]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:21.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:21 smithi078 ceph-mon[18440]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:21.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:21 smithi123 ceph-mon[21808]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:22.482 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 5 on host 'smithi161' 2024-08-18T23:38:22.482 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:22.480+0000 7f8cb17fa640 1 -- 172.21.15.161:0/287244829 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8cb4062360 con 0x7f8ca00a2b80 2024-08-18T23:38:22.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:22.483+0000 7f8cb9bc2640 1 -- 172.21.15.161:0/287244829 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8ca00a2b80 msgr2=0x7f8ca00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:38:22.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:22.483+0000 7f8cb9bc2640 1 --2- 172.21.15.161:0/287244829 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8ca00a2b80 0x7f8ca00a5040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8cb406b5a0 tx=0x7f8c9c005d30 comp rx=0 tx=0).stop 2024-08-18T23:38:22.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:22.483+0000 7f8cb9bc2640 1 -- 172.21.15.161:0/287244829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cb4100840 msgr2=0x7f8cb406b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:38:22.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:22.483+0000 7f8cb9bc2640 1 --2- 172.21.15.161:0/287244829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cb4100840 0x7f8cb406b020 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4031610 tx=0x7f8ca40043b0 comp rx=0 tx=0).stop 2024-08-18T23:38:22.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:22.483+0000 7f8cb9bc2640 1 -- 172.21.15.161:0/287244829 shutdown_connections 2024-08-18T23:38:22.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:22.483+0000 7f8cb9bc2640 1 --2- 172.21.15.161:0/287244829 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8ca00a2b80 0x7f8ca00a5040 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:22.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:22.483+0000 7f8cb9bc2640 1 --2- 172.21.15.161:0/287244829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb4102400 0x7f8cb419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:22.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:22.483+0000 7f8cb9bc2640 1 --2- 172.21.15.161:0/287244829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cb4101a40 0x7f8cb419ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:22.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:22.483+0000 7f8cb9bc2640 1 --2- 172.21.15.161:0/287244829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cb4100840 0x7f8cb406b020 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:22.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:22.483+0000 7f8cb9bc2640 1 -- 172.21.15.161:0/287244829 >> 172.21.15.161:0/287244829 conn(0x7f8cb40fbfd0 msgr2=0x7f8cb4104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:38:22.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:22.483+0000 7f8cb9bc2640 1 -- 172.21.15.161:0/287244829 shutdown_connections 2024-08-18T23:38:22.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:38:22.483+0000 7f8cb9bc2640 1 -- 172.21.15.161:0/287244829 wait complete. 2024-08-18T23:38:22.763 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:22.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:22.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:22.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:38:22.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:22.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:22 smithi161 ceph-mon[22027]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:22.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:38:22.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:22.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:22.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:22.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:22.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:22.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:38:22.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:22.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:22 smithi078 ceph-mon[18440]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:22.971 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:38:22.971 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:22.971 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:23.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:23.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:23.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:23.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:38:23.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:23.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:22 smithi123 ceph-mon[21808]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:23.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:38:23.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:23.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:23.059 DEBUG:teuthology.orchestra.run.smithi161:osd.5> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.5.service 2024-08-18T23:38:23.061 INFO:tasks.cephadm:Deploying osd.6 on smithi078 with /dev/vg_nvme/lv_2... 2024-08-18T23:38:23.061 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-18T23:38:23.331 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:38:23.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:23 smithi078 ceph-mon[18440]: from='osd.5 [v2:172.21.15.161:6808/2751796117,v1:172.21.15.161:6809/2751796117]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-18T23:38:23.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:23 smithi078 ceph-mon[18440]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-18T23:38:23.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:23 smithi161 ceph-mon[22027]: from='osd.5 [v2:172.21.15.161:6808/2751796117,v1:172.21.15.161:6809/2751796117]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-18T23:38:23.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:23 smithi161 ceph-mon[22027]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-18T23:38:24.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:23 smithi123 ceph-mon[21808]: from='osd.5 [v2:172.21.15.161:6808/2751796117,v1:172.21.15.161:6809/2751796117]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-18T23:38:24.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:23 smithi123 ceph-mon[21808]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-18T23:38:24.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:24 smithi161 ceph-mon[22027]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-18T23:38:24.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:24 smithi161 ceph-mon[22027]: from='osd.5 [v2:172.21.15.161:6808/2751796117,v1:172.21.15.161:6809/2751796117]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:38:24.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:24 smithi161 ceph-mon[22027]: osdmap e35: 6 total, 5 up, 6 in 2024-08-18T23:38:24.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:24 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-18T23:38:24.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:24 smithi161 ceph-mon[22027]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:38:24.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:24 smithi161 ceph-mon[22027]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:24.929 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:38:24 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:38:24.689+0000 7f50cb076640 -1 osd.5 0 waiting for initial osdmap 2024-08-18T23:38:24.929 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:38:24 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:38:24.695+0000 7f50c6e9e640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T23:38:24.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:24 smithi078 ceph-mon[18440]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-18T23:38:24.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:24 smithi078 ceph-mon[18440]: from='osd.5 [v2:172.21.15.161:6808/2751796117,v1:172.21.15.161:6809/2751796117]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:38:24.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:24 smithi078 ceph-mon[18440]: osdmap e35: 6 total, 5 up, 6 in 2024-08-18T23:38:24.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:24 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-18T23:38:24.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:24 smithi078 ceph-mon[18440]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:38:24.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:24 smithi078 ceph-mon[18440]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:25.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:24 smithi123 ceph-mon[21808]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-18T23:38:25.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:24 smithi123 ceph-mon[21808]: from='osd.5 [v2:172.21.15.161:6808/2751796117,v1:172.21.15.161:6809/2751796117]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:38:25.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:24 smithi123 ceph-mon[21808]: osdmap e35: 6 total, 5 up, 6 in 2024-08-18T23:38:25.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:24 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-18T23:38:25.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:24 smithi123 ceph-mon[21808]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:38:25.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:24 smithi123 ceph-mon[21808]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:25.783 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:38:25.809 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_2 2024-08-18T23:38:25.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:25 smithi161 ceph-mon[22027]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-18T23:38:25.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:25 smithi161 ceph-mon[22027]: osdmap e36: 6 total, 5 up, 6 in 2024-08-18T23:38:25.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:25 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-18T23:38:25.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:25 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-18T23:38:25.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:25 smithi078 ceph-mon[18440]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-18T23:38:25.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:25 smithi078 ceph-mon[18440]: osdmap e36: 6 total, 5 up, 6 in 2024-08-18T23:38:25.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:25 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-18T23:38:25.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:25 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-18T23:38:26.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:25 smithi123 ceph-mon[21808]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-18T23:38:26.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:25 smithi123 ceph-mon[21808]: osdmap e36: 6 total, 5 up, 6 in 2024-08-18T23:38:26.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:25 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-18T23:38:26.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:25 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-18T23:38:26.073 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:38:26.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.547+0000 7f18aaff5640 1 -- 172.21.15.78:0/2466519350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18a4100b10 msgr2=0x7f18a4100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:38:26.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.547+0000 7f18aaff5640 1 --2- 172.21.15.78:0/2466519350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18a4100b10 0x7f18a4100f90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1890009900 tx=0x7f189002a410 comp rx=0 tx=0).stop 2024-08-18T23:38:26.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.547+0000 7f18aaff5640 1 -- 172.21.15.78:0/2466519350 shutdown_connections 2024-08-18T23:38:26.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.547+0000 7f18aaff5640 1 --2- 172.21.15.78:0/2466519350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18a41014d0 0x7f18a4108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:26.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.547+0000 7f18aaff5640 1 --2- 172.21.15.78:0/2466519350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18a4100b10 0x7f18a4100f90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:26.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.547+0000 7f18aaff5640 1 --2- 172.21.15.78:0/2466519350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18a4101dd0 0x7f18a41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:26.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.547+0000 7f18aaff5640 1 -- 172.21.15.78:0/2466519350 >> 172.21.15.78:0/2466519350 conn(0x7f18a40fb770 msgr2=0x7f18a40fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:38:26.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.547+0000 7f18aaff5640 1 -- 172.21.15.78:0/2466519350 shutdown_connections 2024-08-18T23:38:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.547+0000 7f18aaff5640 1 -- 172.21.15.78:0/2466519350 wait complete. 2024-08-18T23:38:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18aaff5640 1 Processor -- start 2024-08-18T23:38:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18aaff5640 1 -- start start 2024-08-18T23:38:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18aaff5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18a4100b10 0x7f18a4196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:38:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18aaff5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18a41014d0 0x7f18a4197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:38:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18aaff5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18a4101dd0 0x7f18a419e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:38:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18aaff5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f18a410c4d0 con 0x7f18a41014d0 2024-08-18T23:38:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18aaff5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f18a410c350 con 0x7f18a4101dd0 2024-08-18T23:38:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18aaff5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f18a410c650 con 0x7f18a4100b10 2024-08-18T23:38:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18a3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18a41014d0 0x7f18a4197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:38:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18a3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18a41014d0 0x7f18a4197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40432/0 (socket says 172.21.15.78:40432) 2024-08-18T23:38:26.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18a3fff640 1 -- 172.21.15.78:0/3962913467 learned_addr learned my addr 172.21.15.78:0/3962913467 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:38:26.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18a8d91640 1 --2- 172.21.15.78:0/3962913467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18a4100b10 0x7f18a4196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:38:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18a9592640 1 --2- 172.21.15.78:0/3962913467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18a4101dd0 0x7f18a419e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:38:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18a3fff640 1 -- 172.21.15.78:0/3962913467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18a4100b10 msgr2=0x7f18a4196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:38:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18a3fff640 1 --2- 172.21.15.78:0/3962913467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18a4100b10 0x7f18a4196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18a3fff640 1 -- 172.21.15.78:0/3962913467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18a4101dd0 msgr2=0x7f18a419e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:38:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18a3fff640 1 --2- 172.21.15.78:0/3962913467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18a4101dd0 0x7f18a419e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18a3fff640 1 -- 172.21.15.78:0/3962913467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a419e750 con 0x7f18a41014d0 2024-08-18T23:38:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18a8d91640 1 --2- 172.21.15.78:0/3962913467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18a4100b10 0x7f18a4196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:38:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.548+0000 7f18a9592640 1 --2- 172.21.15.78:0/3962913467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18a4101dd0 0x7f18a419e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:38:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.549+0000 7f18a3fff640 1 --2- 172.21.15.78:0/3962913467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18a41014d0 0x7f18a4197060 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1890005940 tx=0x7f18900042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:38:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.549+0000 7f18a1ffb640 1 -- 172.21.15.78:0/3962913467 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890046070 con 0x7f18a41014d0 2024-08-18T23:38:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.549+0000 7f18aaff5640 1 -- 172.21.15.78:0/3962913467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a40ffa00 con 0x7f18a41014d0 2024-08-18T23:38:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.549+0000 7f18a1ffb640 1 -- 172.21.15.78:0/3962913467 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18900044d0 con 0x7f18a41014d0 2024-08-18T23:38:26.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.549+0000 7f18a1ffb640 1 -- 172.21.15.78:0/3962913467 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890035050 con 0x7f18a41014d0 2024-08-18T23:38:26.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.549+0000 7f18aaff5640 1 -- 172.21.15.78:0/3962913467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a40ffeb0 con 0x7f18a41014d0 2024-08-18T23:38:26.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.550+0000 7f18aaff5640 1 -- 172.21.15.78:0/3962913467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a4100f90 con 0x7f18a41014d0 2024-08-18T23:38:26.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.551+0000 7f18a1ffb640 1 -- 172.21.15.78:0/3962913467 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1890049070 con 0x7f18a41014d0 2024-08-18T23:38:26.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.552+0000 7f18a1ffb640 1 --2- 172.21.15.78:0/3962913467 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f18940a2bb0 0x7f18940a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:38:26.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.552+0000 7f18a8d91640 1 --2- 172.21.15.78:0/3962913467 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f18940a2bb0 0x7f18940a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:38:26.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.552+0000 7f18a1ffb640 1 -- 172.21.15.78:0/3962913467 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4566+0+0 (secure 0 0 0) 0x7f18900efe60 con 0x7f18a41014d0 2024-08-18T23:38:26.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.552+0000 7f18a8d91640 1 --2- 172.21.15.78:0/3962913467 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f18940a2bb0 0x7f18940a5070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f18a406a240 tx=0x7f189800a5e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:38:26.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.556+0000 7f18a1ffb640 1 -- 172.21.15.78:0/3962913467 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f189007ddf0 con 0x7f18a41014d0 2024-08-18T23:38:26.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:26 smithi078 ceph-mon[18440]: purged_snaps scrub starts 2024-08-18T23:38:26.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:26 smithi078 ceph-mon[18440]: purged_snaps scrub ok 2024-08-18T23:38:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-18T23:38:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:26 smithi078 ceph-mon[18440]: osd.5 [v2:172.21.15.161:6808/2751796117,v1:172.21.15.161:6809/2751796117] boot 2024-08-18T23:38:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:26 smithi078 ceph-mon[18440]: osdmap e37: 6 total, 6 up, 6 in 2024-08-18T23:38:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-18T23:38:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:26 smithi078 ceph-mon[18440]: Detected new or changed devices on smithi161 2024-08-18T23:38:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:26 smithi078 ceph-mon[18440]: Adjusting osd_memory_target on smithi161 to 8544M 2024-08-18T23:38:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:38:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:26.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:26 smithi078 ceph-mon[18440]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:26.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:26.734+0000 7f18aaff5640 1 -- 172.21.15.78:0/3962913467 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f18a41001d0 con 0x7f18940a2bb0 2024-08-18T23:38:27.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:26 smithi123 ceph-mon[21808]: purged_snaps scrub starts 2024-08-18T23:38:27.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:26 smithi123 ceph-mon[21808]: purged_snaps scrub ok 2024-08-18T23:38:27.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-18T23:38:27.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:26 smithi123 ceph-mon[21808]: osd.5 [v2:172.21.15.161:6808/2751796117,v1:172.21.15.161:6809/2751796117] boot 2024-08-18T23:38:27.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:26 smithi123 ceph-mon[21808]: osdmap e37: 6 total, 6 up, 6 in 2024-08-18T23:38:27.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-18T23:38:27.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:26 smithi123 ceph-mon[21808]: Detected new or changed devices on smithi161 2024-08-18T23:38:27.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:27.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:27.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:27.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:27.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:26 smithi123 ceph-mon[21808]: Adjusting osd_memory_target on smithi161 to 8544M 2024-08-18T23:38:27.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:27.049 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:27.049 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:38:27.049 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:27.049 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:26 smithi123 ceph-mon[21808]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:27.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:26 smithi161 ceph-mon[22027]: purged_snaps scrub starts 2024-08-18T23:38:27.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:26 smithi161 ceph-mon[22027]: purged_snaps scrub ok 2024-08-18T23:38:27.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-18T23:38:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:26 smithi161 ceph-mon[22027]: osd.5 [v2:172.21.15.161:6808/2751796117,v1:172.21.15.161:6809/2751796117] boot 2024-08-18T23:38:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:26 smithi161 ceph-mon[22027]: osdmap e37: 6 total, 6 up, 6 in 2024-08-18T23:38:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-18T23:38:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:26 smithi161 ceph-mon[22027]: Detected new or changed devices on smithi161 2024-08-18T23:38:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:26 smithi161 ceph-mon[22027]: Adjusting osd_memory_target on smithi161 to 8544M 2024-08-18T23:38:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:27.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:38:27.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:27.179 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:26 smithi161 ceph-mon[22027]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-18T23:38:27.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:27 smithi078 ceph-mon[18440]: osdmap e38: 6 total, 6 up, 6 in 2024-08-18T23:38:27.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:27 smithi078 ceph-mon[18440]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:38:27.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:27 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:38:27.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:27 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:38:27.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:27 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:28.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:27 smithi123 ceph-mon[21808]: osdmap e38: 6 total, 6 up, 6 in 2024-08-18T23:38:28.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:27 smithi123 ceph-mon[21808]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:38:28.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:27 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:38:28.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:27 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:38:28.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:27 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:28.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:27 smithi161 ceph-mon[22027]: osdmap e38: 6 total, 6 up, 6 in 2024-08-18T23:38:28.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:27 smithi161 ceph-mon[22027]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:38:28.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:27 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:38:28.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:27 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:38:28.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:27 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:28.852 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:28 smithi078 ceph-mon[18440]: osdmap e39: 6 total, 6 up, 6 in 2024-08-18T23:38:28.852 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:28 smithi078 ceph-mon[18440]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-18T23:38:29.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:28 smithi123 ceph-mon[21808]: osdmap e39: 6 total, 6 up, 6 in 2024-08-18T23:38:29.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:28 smithi123 ceph-mon[21808]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-18T23:38:29.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:28 smithi161 ceph-mon[22027]: osdmap e39: 6 total, 6 up, 6 in 2024-08-18T23:38:29.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:28 smithi161 ceph-mon[22027]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-18T23:38:29.758 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:29 smithi078 ceph-mon[18440]: osdmap e40: 6 total, 6 up, 6 in 2024-08-18T23:38:29.758 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:29 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3459231733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dbc5e9c-a14f-4b71-910a-1e6d4827bb32"}]: dispatch 2024-08-18T23:38:29.758 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:29 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3459231733' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5dbc5e9c-a14f-4b71-910a-1e6d4827bb32"}]': finished 2024-08-18T23:38:29.758 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:29 smithi078 ceph-mon[18440]: osdmap e41: 7 total, 6 up, 7 in 2024-08-18T23:38:29.758 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:29 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-18T23:38:30.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:29 smithi123 ceph-mon[21808]: osdmap e40: 6 total, 6 up, 6 in 2024-08-18T23:38:30.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:29 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3459231733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dbc5e9c-a14f-4b71-910a-1e6d4827bb32"}]: dispatch 2024-08-18T23:38:30.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:29 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3459231733' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5dbc5e9c-a14f-4b71-910a-1e6d4827bb32"}]': finished 2024-08-18T23:38:30.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:29 smithi123 ceph-mon[21808]: osdmap e41: 7 total, 6 up, 7 in 2024-08-18T23:38:30.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:29 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-18T23:38:30.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:29 smithi161 ceph-mon[22027]: osdmap e40: 6 total, 6 up, 6 in 2024-08-18T23:38:30.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:29 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3459231733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dbc5e9c-a14f-4b71-910a-1e6d4827bb32"}]: dispatch 2024-08-18T23:38:30.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:29 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3459231733' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5dbc5e9c-a14f-4b71-910a-1e6d4827bb32"}]': finished 2024-08-18T23:38:30.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:29 smithi161 ceph-mon[22027]: osdmap e41: 7 total, 6 up, 7 in 2024-08-18T23:38:30.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:29 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-18T23:38:30.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:30 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2731877077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:38:30.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:30 smithi078 ceph-mon[18440]: pgmap v125: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-18T23:38:31.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:30 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2731877077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:38:31.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:30 smithi123 ceph-mon[21808]: pgmap v125: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-18T23:38:31.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:30 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2731877077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:38:31.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:30 smithi161 ceph-mon[22027]: pgmap v125: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-18T23:38:33.406 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:33 smithi078 ceph-mon[18440]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-08-18T23:38:33.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:33 smithi161 ceph-mon[22027]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-08-18T23:38:33.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:33 smithi123 ceph-mon[21808]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-08-18T23:38:35.267 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:35 smithi078 ceph-mon[18440]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-08-18T23:38:35.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:35 smithi161 ceph-mon[22027]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-08-18T23:38:35.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:35 smithi123 ceph-mon[21808]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-08-18T23:38:37.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:37 smithi078 ceph-mon[18440]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-18T23:38:37.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:37 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-08-18T23:38:37.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:37 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:37.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:37 smithi078 ceph-mon[18440]: Deploying daemon osd.6 on smithi078 2024-08-18T23:38:37.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:37 smithi161 ceph-mon[22027]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-18T23:38:37.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:37 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-08-18T23:38:37.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:37 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:37.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:37 smithi161 ceph-mon[22027]: Deploying daemon osd.6 on smithi078 2024-08-18T23:38:37.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:37 smithi123 ceph-mon[21808]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-18T23:38:37.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:37 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-08-18T23:38:37.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:37 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:37.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:37 smithi123 ceph-mon[21808]: Deploying daemon osd.6 on smithi078 2024-08-18T23:38:39.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:39 smithi161 ceph-mon[22027]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-08-18T23:38:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:39 smithi078 ceph-mon[18440]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-08-18T23:38:39.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:39 smithi123 ceph-mon[21808]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-08-18T23:38:41.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:41 smithi161 ceph-mon[22027]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-08-18T23:38:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:41 smithi078 ceph-mon[18440]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-08-18T23:38:41.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:41 smithi123 ceph-mon[21808]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-08-18T23:38:42.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:42 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:38:42.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:42 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:42.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:42 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:38:42.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:42.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:42.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:42 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:38:42.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:42 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:42.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:42 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:43.412 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:43 smithi161 ceph-mon[22027]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-18T23:38:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:43 smithi078 ceph-mon[18440]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-18T23:38:43.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:43 smithi123 ceph-mon[21808]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-18T23:38:44.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:44.942+0000 7f18a1ffb640 1 -- 172.21.15.78:0/3962913467 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f18a41001d0 con 0x7f18940a2bb0 2024-08-18T23:38:44.945 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 6 on host 'smithi078' 2024-08-18T23:38:44.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:44.944+0000 7f18aaff5640 1 -- 172.21.15.78:0/3962913467 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f18940a2bb0 msgr2=0x7f18940a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:38:44.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:44.944+0000 7f18aaff5640 1 --2- 172.21.15.78:0/3962913467 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f18940a2bb0 0x7f18940a5070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f18a406a240 tx=0x7f189800a5e0 comp rx=0 tx=0).stop 2024-08-18T23:38:44.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:44.944+0000 7f18aaff5640 1 -- 172.21.15.78:0/3962913467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18a41014d0 msgr2=0x7f18a4197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:38:44.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:44.944+0000 7f18aaff5640 1 --2- 172.21.15.78:0/3962913467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18a41014d0 0x7f18a4197060 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1890005940 tx=0x7f18900042e0 comp rx=0 tx=0).stop 2024-08-18T23:38:44.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:44.944+0000 7f18aaff5640 1 -- 172.21.15.78:0/3962913467 shutdown_connections 2024-08-18T23:38:44.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:44.944+0000 7f18aaff5640 1 --2- 172.21.15.78:0/3962913467 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f18940a2bb0 0x7f18940a5070 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:44.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:44.944+0000 7f18aaff5640 1 --2- 172.21.15.78:0/3962913467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18a4101dd0 0x7f18a419e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:44.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:44.944+0000 7f18aaff5640 1 --2- 172.21.15.78:0/3962913467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f18a41014d0 0x7f18a4197060 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:44.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:44.945+0000 7f18aaff5640 1 --2- 172.21.15.78:0/3962913467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18a4100b10 0x7f18a4196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:44.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:44.945+0000 7f18aaff5640 1 -- 172.21.15.78:0/3962913467 >> 172.21.15.78:0/3962913467 conn(0x7f18a40fb770 msgr2=0x7f18a4104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:38:44.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:44.945+0000 7f18aaff5640 1 -- 172.21.15.78:0/3962913467 shutdown_connections 2024-08-18T23:38:44.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:38:44.945+0000 7f18aaff5640 1 -- 172.21.15.78:0/3962913467 wait complete. 2024-08-18T23:38:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:45 smithi078 ceph-mon[18440]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-18T23:38:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:45 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:45 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:45 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:45.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:45 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:45.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:45 smithi161 ceph-mon[22027]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-18T23:38:45.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:45 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:45.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:45 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:45.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:45 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:45.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:45 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:45.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:45 smithi123 ceph-mon[21808]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-18T23:38:45.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:45 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:45.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:45 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:45.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:45 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:45.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:45 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:45.616 DEBUG:teuthology.orchestra.run.smithi078:osd.6> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.6.service 2024-08-18T23:38:45.618 INFO:tasks.cephadm:Deploying osd.7 on smithi123 with /dev/vg_nvme/lv_2... 2024-08-18T23:38:45.618 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-18T23:38:45.885 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.b/config 2024-08-18T23:38:46.296 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:46 smithi123 ceph-mon[21808]: from='osd.6 [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-08-18T23:38:46.364 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:46 smithi078 ceph-mon[18440]: from='osd.6 [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-08-18T23:38:46.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:46 smithi161 ceph-mon[22027]: from='osd.6 [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-08-18T23:38:47.238 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:47 smithi123 ceph-mon[21808]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-18T23:38:47.238 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:47 smithi123 ceph-mon[21808]: from='osd.6 [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-18T23:38:47.238 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:47 smithi123 ceph-mon[21808]: osdmap e42: 7 total, 6 up, 7 in 2024-08-18T23:38:47.238 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:47 smithi123 ceph-mon[21808]: from='osd.6 [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-08-18T23:38:47.239 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:47 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-18T23:38:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:47 smithi161 ceph-mon[22027]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-18T23:38:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:47 smithi161 ceph-mon[22027]: from='osd.6 [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-18T23:38:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:47 smithi161 ceph-mon[22027]: osdmap e42: 7 total, 6 up, 7 in 2024-08-18T23:38:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:47 smithi161 ceph-mon[22027]: from='osd.6 [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-08-18T23:38:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:47 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-18T23:38:47.443 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:47 smithi078 ceph-mon[18440]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-18T23:38:47.443 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:47 smithi078 ceph-mon[18440]: from='osd.6 [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-18T23:38:47.443 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:47 smithi078 ceph-mon[18440]: osdmap e42: 7 total, 6 up, 7 in 2024-08-18T23:38:47.443 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:47 smithi078 ceph-mon[18440]: from='osd.6 [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-08-18T23:38:47.443 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:47 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-18T23:38:48.132 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:38:47 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:38:47.807+0000 7fc615eb4640 -1 osd.6 0 waiting for initial osdmap 2024-08-18T23:38:48.132 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:38:47 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:38:47.815+0000 7fc60dcd4640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T23:38:48.289 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:48 smithi123 ceph-mon[21808]: from='osd.6 [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-08-18T23:38:48.289 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:48 smithi123 ceph-mon[21808]: osdmap e43: 7 total, 6 up, 7 in 2024-08-18T23:38:48.289 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:48 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-18T23:38:48.290 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:48 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-18T23:38:48.290 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:48 smithi123 ceph-mon[21808]: from='osd.6 [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223]' entity='osd.6' 2024-08-18T23:38:48.290 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:48 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-18T23:38:48.292 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-18T23:38:48.320 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_2 2024-08-18T23:38:48.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:48 smithi161 ceph-mon[22027]: from='osd.6 [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-08-18T23:38:48.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:48 smithi161 ceph-mon[22027]: osdmap e43: 7 total, 6 up, 7 in 2024-08-18T23:38:48.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:48 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-18T23:38:48.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:48 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-18T23:38:48.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:48 smithi161 ceph-mon[22027]: from='osd.6 [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223]' entity='osd.6' 2024-08-18T23:38:48.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:48 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-18T23:38:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:48 smithi078 ceph-mon[18440]: from='osd.6 [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-08-18T23:38:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:48 smithi078 ceph-mon[18440]: osdmap e43: 7 total, 6 up, 7 in 2024-08-18T23:38:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-18T23:38:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-18T23:38:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:48 smithi078 ceph-mon[18440]: from='osd.6 [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223]' entity='osd.6' 2024-08-18T23:38:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-18T23:38:48.578 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.b/config 2024-08-18T23:38:49.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.071+0000 7fcbdba27640 1 -- 172.21.15.123:0/113410726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbd41023c0 msgr2=0x7fcbd41047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:38:49.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.071+0000 7fcbdba27640 1 --2- 172.21.15.123:0/113410726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbd41023c0 0x7fcbd41047b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009900 tx=0x7fcbd002a410 comp rx=0 tx=0).stop 2024-08-18T23:38:49.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.071+0000 7fcbdba27640 1 -- 172.21.15.123:0/113410726 shutdown_connections 2024-08-18T23:38:49.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.071+0000 7fcbdba27640 1 --2- 172.21.15.123:0/113410726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbd4107620 0x7fcbd4109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:49.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.071+0000 7fcbdba27640 1 --2- 172.21.15.123:0/113410726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbd4104cf0 0x7fcbd41070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:49.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.071+0000 7fcbdba27640 1 --2- 172.21.15.123:0/113410726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbd41023c0 0x7fcbd41047b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:49.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.071+0000 7fcbdba27640 1 -- 172.21.15.123:0/113410726 >> 172.21.15.123:0/113410726 conn(0x7fcbd40fbfa0 msgr2=0x7fcbd40fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:38:49.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.071+0000 7fcbdba27640 1 -- 172.21.15.123:0/113410726 shutdown_connections 2024-08-18T23:38:49.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.071+0000 7fcbdba27640 1 -- 172.21.15.123:0/113410726 wait complete. 2024-08-18T23:38:49.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.072+0000 7fcbdba27640 1 Processor -- start 2024-08-18T23:38:49.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.072+0000 7fcbdba27640 1 -- start start 2024-08-18T23:38:49.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.072+0000 7fcbdba27640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbd41023c0 0x7fcbd4101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:38:49.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.072+0000 7fcbdba27640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbd4104cf0 0x7fcbd4100370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:38:49.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.072+0000 7fcbdba27640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbd4107620 0x7fcbd41008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:38:49.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.072+0000 7fcbdba27640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd410d5f0 con 0x7fcbd4107620 2024-08-18T23:38:49.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.072+0000 7fcbdba27640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd410d470 con 0x7fcbd41023c0 2024-08-18T23:38:49.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.072+0000 7fcbdba27640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd410d770 con 0x7fcbd4104cf0 2024-08-18T23:38:49.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.072+0000 7fcbd97c3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbd41023c0 0x7fcbd4101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:38:49.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.072+0000 7fcbd97c3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbd41023c0 0x7fcbd4101d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36956/0 (socket says 172.21.15.123:36956) 2024-08-18T23:38:49.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.072+0000 7fcbd97c3640 1 -- 172.21.15.123:0/3443596149 learned_addr learned my addr 172.21.15.123:0/3443596149 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-08-18T23:38:49.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.073+0000 7fcbd9fc4640 1 --2- 172.21.15.123:0/3443596149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbd4107620 0x7fcbd41008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:38:49.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.073+0000 7fcbd8fc2640 1 --2- 172.21.15.123:0/3443596149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbd4104cf0 0x7fcbd4100370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:38:49.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.073+0000 7fcbd97c3640 1 -- 172.21.15.123:0/3443596149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbd4104cf0 msgr2=0x7fcbd4100370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:38:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.073+0000 7fcbd97c3640 1 --2- 172.21.15.123:0/3443596149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbd4104cf0 0x7fcbd4100370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.073+0000 7fcbd97c3640 1 -- 172.21.15.123:0/3443596149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbd4107620 msgr2=0x7fcbd41008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:38:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.073+0000 7fcbd97c3640 1 --2- 172.21.15.123:0/3443596149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbd4107620 0x7fcbd41008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:38:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.073+0000 7fcbd97c3640 1 -- 172.21.15.123:0/3443596149 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd4100df0 con 0x7fcbd41023c0 2024-08-18T23:38:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.073+0000 7fcbd97c3640 1 --2- 172.21.15.123:0/3443596149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbd41023c0 0x7fcbd4101d20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0005940 tx=0x7fcbd0031b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:38:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.073+0000 7fcbca7fc640 1 -- 172.21.15.123:0/3443596149 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd003f030 con 0x7fcbd41023c0 2024-08-18T23:38:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.073+0000 7fcbdba27640 1 -- 172.21.15.123:0/3443596149 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd4101080 con 0x7fcbd41023c0 2024-08-18T23:38:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.073+0000 7fcbca7fc640 1 -- 172.21.15.123:0/3443596149 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbd0003ad0 con 0x7fcbd41023c0 2024-08-18T23:38:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.073+0000 7fcbca7fc640 1 -- 172.21.15.123:0/3443596149 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0035070 con 0x7fcbd41023c0 2024-08-18T23:38:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.073+0000 7fcbdba27640 1 -- 172.21.15.123:0/3443596149 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd41a6760 con 0x7fcbd41023c0 2024-08-18T23:38:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.075+0000 7fcbdba27640 1 -- 172.21.15.123:0/3443596149 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd41050d0 con 0x7fcbd41023c0 2024-08-18T23:38:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.075+0000 7fcbca7fc640 1 -- 172.21.15.123:0/3443596149 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcbd0042030 con 0x7fcbd41023c0 2024-08-18T23:38:49.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.076+0000 7fcbca7fc640 1 --2- 172.21.15.123:0/3443596149 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcbc00a2b80 0x7fcbc00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:38:49.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.076+0000 7fcbd8fc2640 1 --2- 172.21.15.123:0/3443596149 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcbc00a2b80 0x7fcbc00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:38:49.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.076+0000 7fcbca7fc640 1 -- 172.21.15.123:0/3443596149 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7fcbd00ee840 con 0x7fcbd41023c0 2024-08-18T23:38:49.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.077+0000 7fcbd8fc2640 1 --2- 172.21.15.123:0/3443596149 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcbc00a2b80 0x7fcbc00a5040 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4101aa0 tx=0x7fcbc4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:38:49.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.081+0000 7fcbca7fc640 1 -- 172.21.15.123:0/3443596149 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fcbd00bbe60 con 0x7fcbd41023c0 2024-08-18T23:38:49.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:49 smithi078 ceph-mon[18440]: purged_snaps scrub starts 2024-08-18T23:38:49.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:49 smithi078 ceph-mon[18440]: purged_snaps scrub ok 2024-08-18T23:38:49.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:49 smithi078 ceph-mon[18440]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-18T23:38:49.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:49 smithi078 ceph-mon[18440]: osd.6 [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223] boot 2024-08-18T23:38:49.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:49 smithi078 ceph-mon[18440]: osdmap e44: 7 total, 7 up, 7 in 2024-08-18T23:38:49.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-18T23:38:49.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:49.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:49.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:49.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:49.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:49.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:49.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:49.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:38:49.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:49.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:38:49.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:49.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:38:49.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:49.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:38:49.268+0000 7fcbdba27640 1 -- 172.21.15.123:0/3443596149 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fcbd4062360 con 0x7fcbc00a2b80 2024-08-18T23:38:49.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:49 smithi161 ceph-mon[22027]: purged_snaps scrub starts 2024-08-18T23:38:49.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:49 smithi161 ceph-mon[22027]: purged_snaps scrub ok 2024-08-18T23:38:49.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:49 smithi161 ceph-mon[22027]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-18T23:38:49.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:49 smithi161 ceph-mon[22027]: osd.6 [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223] boot 2024-08-18T23:38:49.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:49 smithi161 ceph-mon[22027]: osdmap e44: 7 total, 7 up, 7 in 2024-08-18T23:38:49.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:49 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-18T23:38:49.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:49 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:49.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:49 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:49.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:49 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:49.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:49 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:49.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:49 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:49.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:49 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:49.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:49 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:49.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:49 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:38:49.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:49 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:49.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:49 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:38:49.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:49 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:49.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:49 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:38:49.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:49 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:49.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:49 smithi123 ceph-mon[21808]: purged_snaps scrub starts 2024-08-18T23:38:49.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:49 smithi123 ceph-mon[21808]: purged_snaps scrub ok 2024-08-18T23:38:49.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:49 smithi123 ceph-mon[21808]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-18T23:38:49.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:49 smithi123 ceph-mon[21808]: osd.6 [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223] boot 2024-08-18T23:38:49.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:49 smithi123 ceph-mon[21808]: osdmap e44: 7 total, 7 up, 7 in 2024-08-18T23:38:49.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:49 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-18T23:38:49.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:49 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:49.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:49 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:49.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:49 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:49.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:49 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:49.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:49 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:38:49.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:49 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:49.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:49 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:49.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:49 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:38:49.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:49 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:49.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:49 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:38:49.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:49 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:49.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:49 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:38:49.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:49 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:38:50.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:50 smithi161 ceph-mon[22027]: Detected new or changed devices on smithi078 2024-08-18T23:38:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:50 smithi161 ceph-mon[22027]: Adjusting osd_memory_target on smithi078 to 5696M 2024-08-18T23:38:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:50 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:38:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:50 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:38:50.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:50 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:50.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:50 smithi078 ceph-mon[18440]: Detected new or changed devices on smithi078 2024-08-18T23:38:50.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:50 smithi078 ceph-mon[18440]: Adjusting osd_memory_target on smithi078 to 5696M 2024-08-18T23:38:50.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:50 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:38:50.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:50 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:38:50.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:50 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:50.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:50 smithi123 ceph-mon[21808]: Detected new or changed devices on smithi078 2024-08-18T23:38:50.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:50 smithi123 ceph-mon[21808]: Adjusting osd_memory_target on smithi078 to 5696M 2024-08-18T23:38:50.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:50 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:38:50.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:50 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:38:50.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:50 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:51.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:51 smithi123 ceph-mon[21808]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:38:51.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:51 smithi123 ceph-mon[21808]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:38:51.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:51 smithi123 ceph-mon[21808]: osdmap e45: 7 total, 7 up, 7 in 2024-08-18T23:38:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:51 smithi161 ceph-mon[22027]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:38:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:51 smithi161 ceph-mon[22027]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:38:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:51 smithi161 ceph-mon[22027]: osdmap e45: 7 total, 7 up, 7 in 2024-08-18T23:38:51.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:51 smithi078 ceph-mon[18440]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:38:51.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:51 smithi078 ceph-mon[18440]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:38:51.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:51 smithi078 ceph-mon[18440]: osdmap e45: 7 total, 7 up, 7 in 2024-08-18T23:38:52.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:52 smithi123 ceph-mon[21808]: from='client.? 172.21.15.123:0/239763576' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26fb7b69-131f-42ee-b745-6fe8b4f65391"}]: dispatch 2024-08-18T23:38:52.645 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:52 smithi123 ceph-mon[21808]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26fb7b69-131f-42ee-b745-6fe8b4f65391"}]: dispatch 2024-08-18T23:38:52.645 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:52 smithi123 ceph-mon[21808]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26fb7b69-131f-42ee-b745-6fe8b4f65391"}]': finished 2024-08-18T23:38:52.645 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:52 smithi123 ceph-mon[21808]: osdmap e46: 8 total, 7 up, 8 in 2024-08-18T23:38:52.645 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:52 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-18T23:38:52.646 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:52 smithi078 ceph-mon[18440]: from='client.? 172.21.15.123:0/239763576' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26fb7b69-131f-42ee-b745-6fe8b4f65391"}]: dispatch 2024-08-18T23:38:52.646 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:52 smithi078 ceph-mon[18440]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26fb7b69-131f-42ee-b745-6fe8b4f65391"}]: dispatch 2024-08-18T23:38:52.646 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:52 smithi078 ceph-mon[18440]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26fb7b69-131f-42ee-b745-6fe8b4f65391"}]': finished 2024-08-18T23:38:52.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:52 smithi078 ceph-mon[18440]: osdmap e46: 8 total, 7 up, 8 in 2024-08-18T23:38:52.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:52 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-18T23:38:52.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:52 smithi161 ceph-mon[22027]: from='client.? 172.21.15.123:0/239763576' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26fb7b69-131f-42ee-b745-6fe8b4f65391"}]: dispatch 2024-08-18T23:38:52.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:52 smithi161 ceph-mon[22027]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26fb7b69-131f-42ee-b745-6fe8b4f65391"}]: dispatch 2024-08-18T23:38:52.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:52 smithi161 ceph-mon[22027]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26fb7b69-131f-42ee-b745-6fe8b4f65391"}]': finished 2024-08-18T23:38:52.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:52 smithi161 ceph-mon[22027]: osdmap e46: 8 total, 7 up, 8 in 2024-08-18T23:38:52.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:52 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-18T23:38:53.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:53 smithi161 ceph-mon[22027]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:38:53.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:53 smithi161 ceph-mon[22027]: from='client.? 172.21.15.123:0/2167904797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:38:53.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:53 smithi078 ceph-mon[18440]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:38:53.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:53 smithi078 ceph-mon[18440]: from='client.? 172.21.15.123:0/2167904797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:38:53.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:53 smithi123 ceph-mon[21808]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:38:53.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:53 smithi123 ceph-mon[21808]: from='client.? 172.21.15.123:0/2167904797' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:38:55.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:55 smithi161 ceph-mon[22027]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:38:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:55 smithi078 ceph-mon[18440]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:38:55.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:55 smithi123 ceph-mon[21808]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:38:57.390 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:57 smithi123 ceph-mon[21808]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:38:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:57 smithi161 ceph-mon[22027]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:38:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:57 smithi078 ceph-mon[18440]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:38:59.287 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:59 smithi123 ceph-mon[21808]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:38:59.287 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:59 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-08-18T23:38:59.287 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:38:59 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:59 smithi161 ceph-mon[22027]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:38:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:59 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-08-18T23:38:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:38:59 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:38:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:59 smithi078 ceph-mon[18440]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:38:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:59 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-08-18T23:38:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:38:59 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:00.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:00 smithi123 ceph-mon[21808]: Deploying daemon osd.7 on smithi123 2024-08-18T23:39:00.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:00 smithi161 ceph-mon[22027]: Deploying daemon osd.7 on smithi123 2024-08-18T23:39:00.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:00 smithi078 ceph-mon[18440]: Deploying daemon osd.7 on smithi123 2024-08-18T23:39:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:01 smithi161 ceph-mon[22027]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:39:01.453 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:01 smithi078 ceph-mon[18440]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:39:01.480 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:01 smithi123 ceph-mon[21808]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:39:03.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:03 smithi123 ceph-mon[21808]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:39:03.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:03 smithi161 ceph-mon[22027]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:39:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:03 smithi078 ceph-mon[18440]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:39:04.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:39:04.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:04.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:04.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:39:04.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:04.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:04.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:39:04.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:04.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:05.419 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:05 smithi123 ceph-mon[21808]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:39:05.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:05 smithi161 ceph-mon[22027]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:39:05.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:05 smithi078 ceph-mon[18440]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:39:07.153 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:07.151+0000 7fcbca7fc640 1 -- 172.21.15.123:0/3443596149 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcbd4062360 con 0x7fcbc00a2b80 2024-08-18T23:39:07.153 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 7 on host 'smithi123' 2024-08-18T23:39:07.154 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:07.153+0000 7fcbdba27640 1 -- 172.21.15.123:0/3443596149 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcbc00a2b80 msgr2=0x7fcbc00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:39:07.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:07.153+0000 7fcbdba27640 1 --2- 172.21.15.123:0/3443596149 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcbc00a2b80 0x7fcbc00a5040 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4101aa0 tx=0x7fcbc4005f90 comp rx=0 tx=0).stop 2024-08-18T23:39:07.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:07.153+0000 7fcbdba27640 1 -- 172.21.15.123:0/3443596149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbd41023c0 msgr2=0x7fcbd4101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:39:07.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:07.153+0000 7fcbdba27640 1 --2- 172.21.15.123:0/3443596149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbd41023c0 0x7fcbd4101d20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0005940 tx=0x7fcbd0031b60 comp rx=0 tx=0).stop 2024-08-18T23:39:07.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:07.154+0000 7fcbdba27640 1 -- 172.21.15.123:0/3443596149 shutdown_connections 2024-08-18T23:39:07.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:07.154+0000 7fcbdba27640 1 --2- 172.21.15.123:0/3443596149 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcbc00a2b80 0x7fcbc00a5040 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:07.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:07.154+0000 7fcbdba27640 1 --2- 172.21.15.123:0/3443596149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbd4107620 0x7fcbd41008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:07.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:07.154+0000 7fcbdba27640 1 --2- 172.21.15.123:0/3443596149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbd4104cf0 0x7fcbd4100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:07.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:07.154+0000 7fcbdba27640 1 --2- 172.21.15.123:0/3443596149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbd41023c0 0x7fcbd4101d20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:07.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:07.154+0000 7fcbdba27640 1 -- 172.21.15.123:0/3443596149 >> 172.21.15.123:0/3443596149 conn(0x7fcbd40fbfa0 msgr2=0x7fcbd4102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:39:07.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:07.154+0000 7fcbdba27640 1 -- 172.21.15.123:0/3443596149 shutdown_connections 2024-08-18T23:39:07.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:07.154+0000 7fcbdba27640 1 -- 172.21.15.123:0/3443596149 wait complete. 2024-08-18T23:39:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:07 smithi161 ceph-mon[22027]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:39:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:07.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:07.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:07.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:39:07.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:07.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:39:07.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:07.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:07.447 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:07 smithi123 ceph-mon[21808]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:39:07.448 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:07 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:07.448 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:07 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:07.448 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:07 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:07.448 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:07 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:39:07.448 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:07 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:07.448 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:07 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:39:07.448 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:07 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:07.448 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:07 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:07 smithi078 ceph-mon[18440]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:39:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:39:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:07.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:39:07.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:07.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:07.783 DEBUG:teuthology.orchestra.run.smithi123:osd.7> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.7.service 2024-08-18T23:39:07.786 INFO:tasks.cephadm:Deploying osd.8 on smithi161 with /dev/vg_nvme/lv_2... 2024-08-18T23:39:07.786 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-18T23:39:08.046 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.c/config 2024-08-18T23:39:08.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:08 smithi123 ceph-mon[21808]: from='osd.7 [v2:172.21.15.123:6816/3440305156,v1:172.21.15.123:6817/3440305156]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-18T23:39:08.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:08 smithi123 ceph-mon[21808]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-18T23:39:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:08 smithi161 ceph-mon[22027]: from='osd.7 [v2:172.21.15.123:6816/3440305156,v1:172.21.15.123:6817/3440305156]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-18T23:39:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:08 smithi161 ceph-mon[22027]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-18T23:39:08.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:08 smithi078 ceph-mon[18440]: from='osd.7 [v2:172.21.15.123:6816/3440305156,v1:172.21.15.123:6817/3440305156]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-18T23:39:08.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:08 smithi078 ceph-mon[18440]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-18T23:39:09.283 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:09 smithi123 ceph-mon[21808]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:39:09.283 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:09 smithi123 ceph-mon[21808]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-18T23:39:09.283 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:09 smithi123 ceph-mon[21808]: osdmap e47: 8 total, 7 up, 8 in 2024-08-18T23:39:09.283 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-18T23:39:09.283 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:09 smithi123 ceph-mon[21808]: from='osd.7 [v2:172.21.15.123:6816/3440305156,v1:172.21.15.123:6817/3440305156]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:39:09.283 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:09 smithi123 ceph-mon[21808]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:39:09.449 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:09 smithi161 ceph-mon[22027]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:39:09.449 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:09 smithi161 ceph-mon[22027]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-18T23:39:09.449 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:09 smithi161 ceph-mon[22027]: osdmap e47: 8 total, 7 up, 8 in 2024-08-18T23:39:09.449 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-18T23:39:09.449 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:09 smithi161 ceph-mon[22027]: from='osd.7 [v2:172.21.15.123:6816/3440305156,v1:172.21.15.123:6817/3440305156]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:39:09.449 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:09 smithi161 ceph-mon[22027]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:39:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:09 smithi078 ceph-mon[18440]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:39:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:09 smithi078 ceph-mon[18440]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-18T23:39:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:09 smithi078 ceph-mon[18440]: osdmap e47: 8 total, 7 up, 8 in 2024-08-18T23:39:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-18T23:39:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:09 smithi078 ceph-mon[18440]: from='osd.7 [v2:172.21.15.123:6816/3440305156,v1:172.21.15.123:6817/3440305156]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:39:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:09 smithi078 ceph-mon[18440]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:39:10.191 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:39:09 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:39:09.906+0000 7ff1d442c640 -1 osd.7 0 waiting for initial osdmap 2024-08-18T23:39:10.191 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:39:09 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:39:09.914+0000 7ff1cc24c640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T23:39:10.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:10 smithi161 ceph-mon[22027]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-08-18T23:39:10.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:10 smithi161 ceph-mon[22027]: osdmap e48: 8 total, 7 up, 8 in 2024-08-18T23:39:10.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:10 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-18T23:39:10.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:10 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-18T23:39:10.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:10 smithi161 ceph-mon[22027]: from='osd.7 ' entity='osd.7' 2024-08-18T23:39:10.446 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:10 smithi123 ceph-mon[21808]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-08-18T23:39:10.446 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:10 smithi123 ceph-mon[21808]: osdmap e48: 8 total, 7 up, 8 in 2024-08-18T23:39:10.446 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:10 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-18T23:39:10.447 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:10 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-18T23:39:10.447 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:10 smithi123 ceph-mon[21808]: from='osd.7 ' entity='osd.7' 2024-08-18T23:39:10.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:10 smithi078 ceph-mon[18440]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-08-18T23:39:10.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:10 smithi078 ceph-mon[18440]: osdmap e48: 8 total, 7 up, 8 in 2024-08-18T23:39:10.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:10 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-18T23:39:10.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:10 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-18T23:39:10.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:10 smithi078 ceph-mon[18440]: from='osd.7 ' entity='osd.7' 2024-08-18T23:39:10.642 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:39:10.669 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_2 2024-08-18T23:39:10.931 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.c/config 2024-08-18T23:39:11.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:11 smithi078 ceph-mon[18440]: purged_snaps scrub starts 2024-08-18T23:39:11.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:11 smithi078 ceph-mon[18440]: purged_snaps scrub ok 2024-08-18T23:39:11.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:11 smithi078 ceph-mon[18440]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:39:11.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:11 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-18T23:39:11.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:11 smithi078 ceph-mon[18440]: osd.7 [v2:172.21.15.123:6816/3440305156,v1:172.21.15.123:6817/3440305156] boot 2024-08-18T23:39:11.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:11 smithi078 ceph-mon[18440]: osdmap e49: 8 total, 8 up, 8 in 2024-08-18T23:39:11.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:11 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-18T23:39:11.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:11 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:11.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:11 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:11.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:11 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:11.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:11 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:11.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:11 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:11.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:11 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:11.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:11 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:11.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:11 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:39:11.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:11 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:11.372 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:11 smithi161 ceph-mon[22027]: purged_snaps scrub starts 2024-08-18T23:39:11.372 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:11 smithi161 ceph-mon[22027]: purged_snaps scrub ok 2024-08-18T23:39:11.372 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:11 smithi161 ceph-mon[22027]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:39:11.372 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:11 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-18T23:39:11.372 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:11 smithi161 ceph-mon[22027]: osd.7 [v2:172.21.15.123:6816/3440305156,v1:172.21.15.123:6817/3440305156] boot 2024-08-18T23:39:11.372 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:11 smithi161 ceph-mon[22027]: osdmap e49: 8 total, 8 up, 8 in 2024-08-18T23:39:11.372 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:11 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-18T23:39:11.372 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:11 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:11.372 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:11 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:11.372 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:11 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:11.372 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:11 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:11.372 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:11 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:11.373 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:11 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:11.373 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:11 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:11.373 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:11 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:39:11.373 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:11 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:11.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.454+0000 7f678fd56640 1 -- 172.21.15.161:0/315007890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6788101a00 msgr2=0x7f6788101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:39:11.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.454+0000 7f678fd56640 1 --2- 172.21.15.161:0/315007890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6788101a00 0x7f6788101e80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6778009900 tx=0x7f677802a410 comp rx=0 tx=0).stop 2024-08-18T23:39:11.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.454+0000 7f678fd56640 1 -- 172.21.15.161:0/315007890 shutdown_connections 2024-08-18T23:39:11.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.454+0000 7f678fd56640 1 --2- 172.21.15.161:0/315007890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67881023c0 0x7f6788108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:11.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.454+0000 7f678fd56640 1 --2- 172.21.15.161:0/315007890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6788101a00 0x7f6788101e80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:11.484 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.454+0000 7f678fd56640 1 --2- 172.21.15.161:0/315007890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6788100800 0x7f6788100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:11.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.454+0000 7f678fd56640 1 -- 172.21.15.161:0/315007890 >> 172.21.15.161:0/315007890 conn(0x7f67880fbfb0 msgr2=0x7f67880fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:39:11.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.454+0000 7f678fd56640 1 -- 172.21.15.161:0/315007890 shutdown_connections 2024-08-18T23:39:11.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.454+0000 7f678fd56640 1 -- 172.21.15.161:0/315007890 wait complete. 2024-08-18T23:39:11.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.455+0000 7f678fd56640 1 Processor -- start 2024-08-18T23:39:11.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.455+0000 7f678fd56640 1 -- start start 2024-08-18T23:39:11.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.455+0000 7f678fd56640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6788100800 0x7f678806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:39:11.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.455+0000 7f678fd56640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6788101a00 0x7f678819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:39:11.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.455+0000 7f678fd56640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67881023c0 0x7f678819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:39:11.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.455+0000 7f678fd56640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f678810c480 con 0x7f67881023c0 2024-08-18T23:39:11.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.455+0000 7f678fd56640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f678810c300 con 0x7f6788100800 2024-08-18T23:39:11.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.455+0000 7f678fd56640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f678810c600 con 0x7f6788101a00 2024-08-18T23:39:11.485 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.455+0000 7f678d2f1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6788101a00 0x7f678819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:39:11.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.455+0000 7f678d2f1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6788101a00 0x7f678819c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39078/0 (socket says 172.21.15.161:39078) 2024-08-18T23:39:11.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.455+0000 7f678d2f1640 1 -- 172.21.15.161:0/1686997118 learned_addr learned my addr 172.21.15.161:0/1686997118 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-18T23:39:11.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.455+0000 7f678e2f3640 1 --2- 172.21.15.161:0/1686997118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67881023c0 0x7f678819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:39:11.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.455+0000 7f678daf2640 1 --2- 172.21.15.161:0/1686997118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6788100800 0x7f678806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:39:11.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.455+0000 7f678d2f1640 1 -- 172.21.15.161:0/1686997118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6788100800 msgr2=0x7f678806a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:39:11.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.455+0000 7f678d2f1640 1 --2- 172.21.15.161:0/1686997118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6788100800 0x7f678806a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:11.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.455+0000 7f678d2f1640 1 -- 172.21.15.161:0/1686997118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67881023c0 msgr2=0x7f678819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:39:11.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.455+0000 7f678d2f1640 1 --2- 172.21.15.161:0/1686997118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67881023c0 0x7f678819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:11.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.455+0000 7f678d2f1640 1 -- 172.21.15.161:0/1686997118 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678819f840 con 0x7f6788101a00 2024-08-18T23:39:11.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.455+0000 7f678d2f1640 1 --2- 172.21.15.161:0/1686997118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6788101a00 0x7f678819c930 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6778009230 tx=0x7f6778031b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:39:11.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.456+0000 7f677effd640 1 -- 172.21.15.161:0/1686997118 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677803f030 con 0x7f6788101a00 2024-08-18T23:39:11.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.456+0000 7f678fd56640 1 -- 172.21.15.161:0/1686997118 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f678819fad0 con 0x7f6788101a00 2024-08-18T23:39:11.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.456+0000 7f677effd640 1 -- 172.21.15.161:0/1686997118 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6778003ad0 con 0x7f6788101a00 2024-08-18T23:39:11.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.456+0000 7f677effd640 1 -- 172.21.15.161:0/1686997118 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778035070 con 0x7f6788101a00 2024-08-18T23:39:11.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.456+0000 7f678fd56640 1 -- 172.21.15.161:0/1686997118 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f678819ffb0 con 0x7f6788101a00 2024-08-18T23:39:11.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.457+0000 7f678fd56640 1 -- 172.21.15.161:0/1686997118 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6788107bf0 con 0x7f6788101a00 2024-08-18T23:39:11.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.459+0000 7f677effd640 1 -- 172.21.15.161:0/1686997118 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6778042030 con 0x7f6788101a00 2024-08-18T23:39:11.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.459+0000 7f677effd640 1 --2- 172.21.15.161:0/1686997118 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f67740a2b80 0x7f67740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:39:11.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.459+0000 7f677effd640 1 -- 172.21.15.161:0/1686997118 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5426+0+0 (secure 0 0 0) 0x7f67780eee80 con 0x7f6788101a00 2024-08-18T23:39:11.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.459+0000 7f678daf2640 1 --2- 172.21.15.161:0/1686997118 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f67740a2b80 0x7f67740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:39:11.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.460+0000 7f678daf2640 1 --2- 172.21.15.161:0/1686997118 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f67740a2b80 0x7f67740a5040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f678806ad80 tx=0x7f6784005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:39:11.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.462+0000 7f677effd640 1 -- 172.21.15.161:0/1686997118 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f67780bc920 con 0x7f6788101a00 2024-08-18T23:39:11.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:11 smithi123 ceph-mon[21808]: purged_snaps scrub starts 2024-08-18T23:39:11.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:11 smithi123 ceph-mon[21808]: purged_snaps scrub ok 2024-08-18T23:39:11.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:11 smithi123 ceph-mon[21808]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-08-18T23:39:11.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:11 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-18T23:39:11.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:11 smithi123 ceph-mon[21808]: osd.7 [v2:172.21.15.123:6816/3440305156,v1:172.21.15.123:6817/3440305156] boot 2024-08-18T23:39:11.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:11 smithi123 ceph-mon[21808]: osdmap e49: 8 total, 8 up, 8 in 2024-08-18T23:39:11.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:11 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-18T23:39:11.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:11 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:11.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:11 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:11.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:11 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:11.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:11 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:11.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:11 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:11.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:11 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:11.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:11 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:11.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:11 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:39:11.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:11 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:11.640 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:11.639+0000 7f678fd56640 1 -- 172.21.15.161:0/1686997118 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6788062360 con 0x7f67740a2b80 2024-08-18T23:39:12.303 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:12 smithi161 ceph-mon[22027]: Detected new or changed devices on smithi123 2024-08-18T23:39:12.303 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:12 smithi161 ceph-mon[22027]: Adjusting osd_memory_target on smithi123 to 5696M 2024-08-18T23:39:12.303 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:12 smithi161 ceph-mon[22027]: osdmap e50: 8 total, 8 up, 8 in 2024-08-18T23:39:12.304 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:12 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:39:12.304 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:12 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:39:12.304 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:12 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:12.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:12 smithi078 ceph-mon[18440]: Detected new or changed devices on smithi123 2024-08-18T23:39:12.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:12 smithi078 ceph-mon[18440]: Adjusting osd_memory_target on smithi123 to 5696M 2024-08-18T23:39:12.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:12 smithi078 ceph-mon[18440]: osdmap e50: 8 total, 8 up, 8 in 2024-08-18T23:39:12.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:12 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:39:12.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:12 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:39:12.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:12 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:12.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:12 smithi123 ceph-mon[21808]: Detected new or changed devices on smithi123 2024-08-18T23:39:12.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:12 smithi123 ceph-mon[21808]: Adjusting osd_memory_target on smithi123 to 5696M 2024-08-18T23:39:12.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:12 smithi123 ceph-mon[21808]: osdmap e50: 8 total, 8 up, 8 in 2024-08-18T23:39:12.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:12 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:39:12.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:12 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:39:12.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:12 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:13.294 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:13 smithi161 ceph-mon[22027]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:39:13.294 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:13 smithi161 ceph-mon[22027]: pgmap v155: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:13.295 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:13 smithi161 ceph-mon[22027]: osdmap e51: 8 total, 8 up, 8 in 2024-08-18T23:39:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:13 smithi078 ceph-mon[18440]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:39:13.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:13 smithi078 ceph-mon[18440]: pgmap v155: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:13.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:13 smithi078 ceph-mon[18440]: osdmap e51: 8 total, 8 up, 8 in 2024-08-18T23:39:13.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:13 smithi123 ceph-mon[21808]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:39:13.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:13 smithi123 ceph-mon[21808]: pgmap v155: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:13.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:13 smithi123 ceph-mon[21808]: osdmap e51: 8 total, 8 up, 8 in 2024-08-18T23:39:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:14 smithi078 ceph-mon[18440]: osdmap e52: 8 total, 8 up, 8 in 2024-08-18T23:39:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:14 smithi078 ceph-mon[18440]: from='client.? 172.21.15.161:0/3423875004' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e267bc50-0c29-4d7b-b368-b1621e29b2df"}]: dispatch 2024-08-18T23:39:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:14 smithi078 ceph-mon[18440]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e267bc50-0c29-4d7b-b368-b1621e29b2df"}]: dispatch 2024-08-18T23:39:14.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:14 smithi078 ceph-mon[18440]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e267bc50-0c29-4d7b-b368-b1621e29b2df"}]': finished 2024-08-18T23:39:14.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:14 smithi078 ceph-mon[18440]: osdmap e53: 9 total, 8 up, 9 in 2024-08-18T23:39:14.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-18T23:39:14.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:14 smithi123 ceph-mon[21808]: osdmap e52: 8 total, 8 up, 8 in 2024-08-18T23:39:14.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:14 smithi123 ceph-mon[21808]: from='client.? 172.21.15.161:0/3423875004' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e267bc50-0c29-4d7b-b368-b1621e29b2df"}]: dispatch 2024-08-18T23:39:14.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:14 smithi123 ceph-mon[21808]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e267bc50-0c29-4d7b-b368-b1621e29b2df"}]: dispatch 2024-08-18T23:39:14.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:14 smithi123 ceph-mon[21808]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e267bc50-0c29-4d7b-b368-b1621e29b2df"}]': finished 2024-08-18T23:39:14.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:14 smithi123 ceph-mon[21808]: osdmap e53: 9 total, 8 up, 9 in 2024-08-18T23:39:14.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-18T23:39:14.570 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:14 smithi161 ceph-mon[22027]: osdmap e52: 8 total, 8 up, 8 in 2024-08-18T23:39:14.570 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:14 smithi161 ceph-mon[22027]: from='client.? 172.21.15.161:0/3423875004' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e267bc50-0c29-4d7b-b368-b1621e29b2df"}]: dispatch 2024-08-18T23:39:14.570 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:14 smithi161 ceph-mon[22027]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e267bc50-0c29-4d7b-b368-b1621e29b2df"}]: dispatch 2024-08-18T23:39:14.570 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:14 smithi161 ceph-mon[22027]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e267bc50-0c29-4d7b-b368-b1621e29b2df"}]': finished 2024-08-18T23:39:14.570 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:14 smithi161 ceph-mon[22027]: osdmap e53: 9 total, 8 up, 9 in 2024-08-18T23:39:14.570 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-18T23:39:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:15 smithi078 ceph-mon[18440]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:15 smithi078 ceph-mon[18440]: from='client.? 172.21.15.161:0/2093221092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:39:15.487 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:15 smithi123 ceph-mon[21808]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:15.487 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:15 smithi123 ceph-mon[21808]: from='client.? 172.21.15.161:0/2093221092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:39:15.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:15 smithi161 ceph-mon[22027]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:15.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:15 smithi161 ceph-mon[22027]: from='client.? 172.21.15.161:0/2093221092' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:39:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:17 smithi078 ceph-mon[18440]: pgmap v160: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:17.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:17 smithi161 ceph-mon[22027]: pgmap v160: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:17.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:17 smithi123 ceph-mon[21808]: pgmap v160: 1 pgs: 1 remapped+peering; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:19.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:19 smithi161 ceph-mon[22027]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:19.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:19 smithi123 ceph-mon[21808]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:19.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:19 smithi078 ceph-mon[18440]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:21.483 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:21 smithi161 ceph-mon[22027]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:21.483 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:21 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-08-18T23:39:21.483 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:21 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:21.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:21 smithi123 ceph-mon[21808]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:21.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:21 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-08-18T23:39:21.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:21 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:21 smithi078 ceph-mon[18440]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:21 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-08-18T23:39:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:21 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:22.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:22 smithi161 ceph-mon[22027]: Deploying daemon osd.8 on smithi161 2024-08-18T23:39:22.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:22 smithi123 ceph-mon[21808]: Deploying daemon osd.8 on smithi161 2024-08-18T23:39:22.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:22 smithi078 ceph-mon[18440]: Deploying daemon osd.8 on smithi161 2024-08-18T23:39:23.337 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:23 smithi161 ceph-mon[22027]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:23.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:23 smithi123 ceph-mon[21808]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:23.727 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:23 smithi078 ceph-mon[18440]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:25.530 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:25 smithi161 ceph-mon[22027]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:25.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:25 smithi123 ceph-mon[21808]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:25.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:25 smithi078 ceph-mon[18440]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:26.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:39:26.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:26.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:26.556 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:39:26.556 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:26.556 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:26.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:39:26.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:26.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:27.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:27 smithi123 ceph-mon[21808]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:27.565 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:27 smithi161 ceph-mon[22027]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:27.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:27 smithi078 ceph-mon[18440]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:29.325 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:29.323+0000 7f677effd640 1 -- 172.21.15.161:0/1686997118 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6788062360 con 0x7f67740a2b80 2024-08-18T23:39:29.325 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 8 on host 'smithi161' 2024-08-18T23:39:29.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:29.326+0000 7f678fd56640 1 -- 172.21.15.161:0/1686997118 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f67740a2b80 msgr2=0x7f67740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:39:29.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:29.326+0000 7f678fd56640 1 --2- 172.21.15.161:0/1686997118 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f67740a2b80 0x7f67740a5040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f678806ad80 tx=0x7f6784005f90 comp rx=0 tx=0).stop 2024-08-18T23:39:29.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:29.326+0000 7f678fd56640 1 -- 172.21.15.161:0/1686997118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6788101a00 msgr2=0x7f678819c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:39:29.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:29.326+0000 7f678fd56640 1 --2- 172.21.15.161:0/1686997118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6788101a00 0x7f678819c930 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6778009230 tx=0x7f6778031b60 comp rx=0 tx=0).stop 2024-08-18T23:39:29.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:29.326+0000 7f678fd56640 1 -- 172.21.15.161:0/1686997118 shutdown_connections 2024-08-18T23:39:29.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:29.326+0000 7f678fd56640 1 --2- 172.21.15.161:0/1686997118 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f67740a2b80 0x7f67740a5040 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:29.330 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:29.326+0000 7f678fd56640 1 --2- 172.21.15.161:0/1686997118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67881023c0 0x7f678819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:29.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:29.326+0000 7f678fd56640 1 --2- 172.21.15.161:0/1686997118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6788101a00 0x7f678819c930 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:29.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:29.326+0000 7f678fd56640 1 --2- 172.21.15.161:0/1686997118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6788100800 0x7f678806a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:29.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:29.326+0000 7f678fd56640 1 -- 172.21.15.161:0/1686997118 >> 172.21.15.161:0/1686997118 conn(0x7f67880fbfb0 msgr2=0x7f6788104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:39:29.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:29.326+0000 7f678fd56640 1 -- 172.21.15.161:0/1686997118 shutdown_connections 2024-08-18T23:39:29.331 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:39:29.326+0000 7f678fd56640 1 -- 172.21.15.161:0/1686997118 wait complete. 2024-08-18T23:39:29.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:29 smithi078 ceph-mon[18440]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:29.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:29 smithi123 ceph-mon[21808]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:29.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:29 smithi161 ceph-mon[22027]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:29.973 DEBUG:teuthology.orchestra.run.smithi161:osd.8> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.8.service 2024-08-18T23:39:29.975 INFO:tasks.cephadm:Deploying osd.9 on smithi078 with /dev/vg_nvme/lv_1... 2024-08-18T23:39:29.975 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-18T23:39:30.249 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:39:30.318 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:39:30 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:39:30.055+0000 7f512d8f5640 -1 osd.8 0 log_to_monitors true 2024-08-18T23:39:30.572 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:30 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:30.572 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:30 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:30.573 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:30 smithi078 ceph-mon[18440]: from='osd.8 [v2:172.21.15.161:6816/1485008778,v1:172.21.15.161:6817/1485008778]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-18T23:39:30.573 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:30 smithi078 ceph-mon[18440]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-18T23:39:30.573 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:30 smithi078 ceph-mon[18440]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:30.573 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:30 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:30.573 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:30 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:30.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:30 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:30.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:30 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:30.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:30 smithi161 ceph-mon[22027]: from='osd.8 [v2:172.21.15.161:6816/1485008778,v1:172.21.15.161:6817/1485008778]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-18T23:39:30.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:30 smithi161 ceph-mon[22027]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-18T23:39:30.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:30 smithi161 ceph-mon[22027]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:30.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:30 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:30.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:30 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:30.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:30 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:30.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:30 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:30.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:30 smithi123 ceph-mon[21808]: from='osd.8 [v2:172.21.15.161:6816/1485008778,v1:172.21.15.161:6817/1485008778]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-18T23:39:30.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:30 smithi123 ceph-mon[21808]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-18T23:39:30.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:30 smithi123 ceph-mon[21808]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:30.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:30 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:30.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:30 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:31.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:31 smithi161 ceph-mon[22027]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-18T23:39:31.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:31 smithi161 ceph-mon[22027]: osdmap e54: 9 total, 8 up, 9 in 2024-08-18T23:39:31.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:31 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-18T23:39:31.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:31 smithi161 ceph-mon[22027]: from='osd.8 [v2:172.21.15.161:6816/1485008778,v1:172.21.15.161:6817/1485008778]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:39:31.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:31 smithi161 ceph-mon[22027]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:39:31.589 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:39:31 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:39:31.335+0000 7f5129878640 -1 osd.8 0 waiting for initial osdmap 2024-08-18T23:39:31.590 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:39:31 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:39:31.343+0000 7f51256a0640 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T23:39:31.611 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:31 smithi078 ceph-mon[18440]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-18T23:39:31.611 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:31 smithi078 ceph-mon[18440]: osdmap e54: 9 total, 8 up, 9 in 2024-08-18T23:39:31.611 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:31 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-18T23:39:31.611 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:31 smithi078 ceph-mon[18440]: from='osd.8 [v2:172.21.15.161:6816/1485008778,v1:172.21.15.161:6817/1485008778]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:39:31.612 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:31 smithi078 ceph-mon[18440]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:39:31.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:31 smithi123 ceph-mon[21808]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-18T23:39:31.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:31 smithi123 ceph-mon[21808]: osdmap e54: 9 total, 8 up, 9 in 2024-08-18T23:39:31.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:31 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-18T23:39:31.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:31 smithi123 ceph-mon[21808]: from='osd.8 [v2:172.21.15.161:6816/1485008778,v1:172.21.15.161:6817/1485008778]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:39:31.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:31 smithi123 ceph-mon[21808]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:39:32.580 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:32 smithi078 ceph-mon[18440]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-18T23:39:32.580 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:32 smithi078 ceph-mon[18440]: osdmap e55: 9 total, 8 up, 9 in 2024-08-18T23:39:32.580 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:32 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-18T23:39:32.580 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:32 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-18T23:39:32.581 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:32 smithi078 ceph-mon[18440]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:32.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:32 smithi161 ceph-mon[22027]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-18T23:39:32.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:32 smithi161 ceph-mon[22027]: osdmap e55: 9 total, 8 up, 9 in 2024-08-18T23:39:32.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:32 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-18T23:39:32.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:32 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-18T23:39:32.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:32 smithi161 ceph-mon[22027]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:32.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:32 smithi123 ceph-mon[21808]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-18T23:39:32.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:32 smithi123 ceph-mon[21808]: osdmap e55: 9 total, 8 up, 9 in 2024-08-18T23:39:32.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:32 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-18T23:39:32.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:32 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-18T23:39:32.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:32 smithi123 ceph-mon[21808]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-18T23:39:32.841 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:39:32.866 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_1 2024-08-18T23:39:33.115 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:39:33.542 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:33 smithi078 ceph-mon[18440]: purged_snaps scrub starts 2024-08-18T23:39:33.543 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:33 smithi078 ceph-mon[18440]: purged_snaps scrub ok 2024-08-18T23:39:33.543 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-18T23:39:33.543 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:33 smithi078 ceph-mon[18440]: osd.8 [v2:172.21.15.161:6816/1485008778,v1:172.21.15.161:6817/1485008778] boot 2024-08-18T23:39:33.543 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:33 smithi078 ceph-mon[18440]: osdmap e56: 9 total, 9 up, 9 in 2024-08-18T23:39:33.543 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-18T23:39:33.543 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:33.543 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:33.543 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:33.543 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:33.544 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:33.544 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:33.544 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:33.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.624+0000 7fbee93b0640 1 -- 172.21.15.78:0/576948915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee4102b30 msgr2=0x7fbee4102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:39:33.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.624+0000 7fbee93b0640 1 --2- 172.21.15.78:0/576948915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee4102b30 0x7fbee4102fb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009900 tx=0x7fbecc02a410 comp rx=0 tx=0).stop 2024-08-18T23:39:33.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.624+0000 7fbee93b0640 1 -- 172.21.15.78:0/576948915 shutdown_connections 2024-08-18T23:39:33.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.624+0000 7fbee93b0640 1 --2- 172.21.15.78:0/576948915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee41034f0 0x7fbee4109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:33.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.624+0000 7fbee93b0640 1 --2- 172.21.15.78:0/576948915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee4102b30 0x7fbee4102fb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:33.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.624+0000 7fbee93b0640 1 --2- 172.21.15.78:0/576948915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbee4101930 0x7fbee4101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:33.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.624+0000 7fbee93b0640 1 -- 172.21.15.78:0/576948915 >> 172.21.15.78:0/576948915 conn(0x7fbee40fd0c0 msgr2=0x7fbee40ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:39:33.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.624+0000 7fbee93b0640 1 -- 172.21.15.78:0/576948915 shutdown_connections 2024-08-18T23:39:33.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.624+0000 7fbee93b0640 1 -- 172.21.15.78:0/576948915 wait complete. 2024-08-18T23:39:33.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.625+0000 7fbee93b0640 1 Processor -- start 2024-08-18T23:39:33.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.625+0000 7fbee93b0640 1 -- start start 2024-08-18T23:39:33.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.625+0000 7fbee93b0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee4101930 0x7fbee4197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:39:33.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.625+0000 7fbee93b0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbee4102b30 0x7fbee4198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:39:33.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.625+0000 7fbee93b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee41034f0 0x7fbee419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:39:33.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.625+0000 7fbee93b0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbee410d5e0 con 0x7fbee41034f0 2024-08-18T23:39:33.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.625+0000 7fbee93b0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbee410d460 con 0x7fbee4102b30 2024-08-18T23:39:33.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.625+0000 7fbee93b0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbee410d760 con 0x7fbee4101930 2024-08-18T23:39:33.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.625+0000 7fbee37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee41034f0 0x7fbee419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:39:33.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.625+0000 7fbee37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee41034f0 0x7fbee419f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40914/0 (socket says 172.21.15.78:40914) 2024-08-18T23:39:33.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.625+0000 7fbee37fe640 1 -- 172.21.15.78:0/542447110 learned_addr learned my addr 172.21.15.78:0/542447110 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:39:33.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.625+0000 7fbee2ffd640 1 --2- 172.21.15.78:0/542447110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee4101930 0x7fbee4197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:39:33.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.625+0000 7fbee27fc640 1 --2- 172.21.15.78:0/542447110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbee4102b30 0x7fbee4198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:39:33.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.625+0000 7fbee37fe640 1 -- 172.21.15.78:0/542447110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee4101930 msgr2=0x7fbee4197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:39:33.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.625+0000 7fbee37fe640 1 --2- 172.21.15.78:0/542447110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee4101930 0x7fbee4197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:33.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.625+0000 7fbee37fe640 1 -- 172.21.15.78:0/542447110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbee4102b30 msgr2=0x7fbee4198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:39:33.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.625+0000 7fbee37fe640 1 --2- 172.21.15.78:0/542447110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbee4102b30 0x7fbee4198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:33.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.625+0000 7fbee37fe640 1 -- 172.21.15.78:0/542447110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee419f810 con 0x7fbee41034f0 2024-08-18T23:39:33.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.625+0000 7fbee27fc640 1 --2- 172.21.15.78:0/542447110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbee4102b30 0x7fbee4198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:39:33.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.626+0000 7fbee37fe640 1 --2- 172.21.15.78:0/542447110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee41034f0 0x7fbee419f1a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00eaa0 tx=0x7fbedc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:39:33.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.626+0000 7fbedbfff640 1 -- 172.21.15.78:0/542447110 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc00cdb0 con 0x7fbee41034f0 2024-08-18T23:39:33.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.626+0000 7fbedbfff640 1 -- 172.21.15.78:0/542447110 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbedc005020 con 0x7fbee41034f0 2024-08-18T23:39:33.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.626+0000 7fbee93b0640 1 -- 172.21.15.78:0/542447110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee419fb00 con 0x7fbee41034f0 2024-08-18T23:39:33.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.626+0000 7fbedbfff640 1 -- 172.21.15.78:0/542447110 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0116b0 con 0x7fbee41034f0 2024-08-18T23:39:33.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.626+0000 7fbee93b0640 1 -- 172.21.15.78:0/542447110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee419fff0 con 0x7fbee41034f0 2024-08-18T23:39:33.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.627+0000 7fbee93b0640 1 -- 172.21.15.78:0/542447110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee4108ce0 con 0x7fbee41034f0 2024-08-18T23:39:33.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.628+0000 7fbedbfff640 1 -- 172.21.15.78:0/542447110 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbedc005560 con 0x7fbee41034f0 2024-08-18T23:39:33.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.629+0000 7fbedbfff640 1 --2- 172.21.15.78:0/542447110 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbed00a2bb0 0x7fbed00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:39:33.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.629+0000 7fbedbfff640 1 -- 172.21.15.78:0/542447110 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5858+0+0 (secure 0 0 0) 0x7fbedc0ce990 con 0x7fbee41034f0 2024-08-18T23:39:33.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.629+0000 7fbee2ffd640 1 --2- 172.21.15.78:0/542447110 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbed00a2bb0 0x7fbed00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:39:33.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.629+0000 7fbee2ffd640 1 --2- 172.21.15.78:0/542447110 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbed00a2bb0 0x7fbed00a5070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbed4005db0 tx=0x7fbed4005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:39:33.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.633+0000 7fbedbfff640 1 -- 172.21.15.78:0/542447110 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbedc016030 con 0x7fbee41034f0 2024-08-18T23:39:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:33 smithi161 ceph-mon[22027]: purged_snaps scrub starts 2024-08-18T23:39:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:33 smithi161 ceph-mon[22027]: purged_snaps scrub ok 2024-08-18T23:39:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:33 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-18T23:39:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:33 smithi161 ceph-mon[22027]: osd.8 [v2:172.21.15.161:6816/1485008778,v1:172.21.15.161:6817/1485008778] boot 2024-08-18T23:39:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:33 smithi161 ceph-mon[22027]: osdmap e56: 9 total, 9 up, 9 in 2024-08-18T23:39:33.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:33 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-18T23:39:33.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:33 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:33.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:33 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:33.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:33 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:33.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:33 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:33.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:33 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:33.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:33 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:33.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:33 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:33.698 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:33 smithi123 ceph-mon[21808]: purged_snaps scrub starts 2024-08-18T23:39:33.698 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:33 smithi123 ceph-mon[21808]: purged_snaps scrub ok 2024-08-18T23:39:33.699 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:33 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-18T23:39:33.699 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:33 smithi123 ceph-mon[21808]: osd.8 [v2:172.21.15.161:6816/1485008778,v1:172.21.15.161:6817/1485008778] boot 2024-08-18T23:39:33.699 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:33 smithi123 ceph-mon[21808]: osdmap e56: 9 total, 9 up, 9 in 2024-08-18T23:39:33.699 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:33 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-18T23:39:33.699 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:33 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:33.699 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:33 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:33.699 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:33 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:33.699 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:33 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:33.699 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:33 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:33.699 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:33 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:33.699 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:33 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:33.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:33.809+0000 7fbee93b0640 1 -- 172.21.15.78:0/542447110 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbee4062360 con 0x7fbed00a2bb0 2024-08-18T23:39:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:34 smithi078 ceph-mon[18440]: Detected new or changed devices on smithi161 2024-08-18T23:39:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:34 smithi078 ceph-mon[18440]: Adjusting osd_memory_target on smithi161 to 5696M 2024-08-18T23:39:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:34 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:39:34.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:34 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:34.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:34 smithi078 ceph-mon[18440]: osdmap e57: 9 total, 9 up, 9 in 2024-08-18T23:39:34.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:34 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:39:34.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:34 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:34.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:34 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:39:34.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:34 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:34.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:34 smithi078 ceph-mon[18440]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:39:34.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:34 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:39:34.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:34 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:39:34.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:34 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:34.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:34 smithi078 ceph-mon[18440]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:34.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:34 smithi161 ceph-mon[22027]: Detected new or changed devices on smithi161 2024-08-18T23:39:34.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:34 smithi161 ceph-mon[22027]: Adjusting osd_memory_target on smithi161 to 5696M 2024-08-18T23:39:34.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:34 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:39:34.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:34 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:34.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:34 smithi161 ceph-mon[22027]: osdmap e57: 9 total, 9 up, 9 in 2024-08-18T23:39:34.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:34 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:39:34.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:34 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:34.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:34 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:39:34.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:34 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:34.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:34 smithi161 ceph-mon[22027]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:39:34.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:34 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:39:34.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:34 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:39:34.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:34 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:34.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:34 smithi161 ceph-mon[22027]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:34.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:34 smithi123 ceph-mon[21808]: Detected new or changed devices on smithi161 2024-08-18T23:39:34.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:34 smithi123 ceph-mon[21808]: Adjusting osd_memory_target on smithi161 to 5696M 2024-08-18T23:39:34.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:34 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:39:34.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:34 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:34.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:34 smithi123 ceph-mon[21808]: osdmap e57: 9 total, 9 up, 9 in 2024-08-18T23:39:34.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:34 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:39:34.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:34 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:34.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:34 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:39:34.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:34 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:34.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:34 smithi123 ceph-mon[21808]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:39:34.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:34 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:39:34.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:34 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:39:34.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:34 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:34.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:34 smithi123 ceph-mon[21808]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:35.629 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:35 smithi078 ceph-mon[18440]: osdmap e58: 9 total, 9 up, 9 in 2024-08-18T23:39:35.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:35 smithi161 ceph-mon[22027]: osdmap e58: 9 total, 9 up, 9 in 2024-08-18T23:39:35.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:35 smithi123 ceph-mon[21808]: osdmap e58: 9 total, 9 up, 9 in 2024-08-18T23:39:36.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:36 smithi123 ceph-mon[21808]: osdmap e59: 9 total, 9 up, 9 in 2024-08-18T23:39:36.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:36 smithi123 ceph-mon[21808]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:36.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:36 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2538030257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7654818-b767-4fe6-8211-346297895f5e"}]: dispatch 2024-08-18T23:39:36.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:36 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2538030257' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7654818-b767-4fe6-8211-346297895f5e"}]': finished 2024-08-18T23:39:36.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:36 smithi123 ceph-mon[21808]: osdmap e60: 10 total, 9 up, 10 in 2024-08-18T23:39:36.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:36 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-18T23:39:36.908 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:36 smithi078 ceph-mon[18440]: osdmap e59: 9 total, 9 up, 9 in 2024-08-18T23:39:36.908 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:36 smithi078 ceph-mon[18440]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:36.908 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:36 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2538030257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7654818-b767-4fe6-8211-346297895f5e"}]: dispatch 2024-08-18T23:39:36.908 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:36 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2538030257' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7654818-b767-4fe6-8211-346297895f5e"}]': finished 2024-08-18T23:39:36.908 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:36 smithi078 ceph-mon[18440]: osdmap e60: 10 total, 9 up, 10 in 2024-08-18T23:39:36.908 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:36 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-18T23:39:36.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:36 smithi161 ceph-mon[22027]: osdmap e59: 9 total, 9 up, 9 in 2024-08-18T23:39:36.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:36 smithi161 ceph-mon[22027]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:36.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:36 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2538030257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7654818-b767-4fe6-8211-346297895f5e"}]: dispatch 2024-08-18T23:39:36.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:36 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2538030257' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7654818-b767-4fe6-8211-346297895f5e"}]': finished 2024-08-18T23:39:36.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:36 smithi161 ceph-mon[22027]: osdmap e60: 10 total, 9 up, 10 in 2024-08-18T23:39:36.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:36 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-18T23:39:37.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:37 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1439758494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:39:37.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:37 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1439758494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:39:37.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:37 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1439758494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:39:38.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:38 smithi123 ceph-mon[21808]: pgmap v178: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-08-18T23:39:38.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:38 smithi161 ceph-mon[22027]: pgmap v178: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-08-18T23:39:38.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:38 smithi078 ceph-mon[18440]: pgmap v178: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-08-18T23:39:41.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:41 smithi161 ceph-mon[22027]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-18T23:39:41.445 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:41 smithi078 ceph-mon[18440]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-18T23:39:41.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:41 smithi123 ceph-mon[21808]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-18T23:39:43.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:43 smithi161 ceph-mon[22027]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-08-18T23:39:43.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:43 smithi078 ceph-mon[18440]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-08-18T23:39:43.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:43 smithi123 ceph-mon[21808]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-08-18T23:39:44.359 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:44 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-08-18T23:39:44.359 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:44 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:44.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:44 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-08-18T23:39:44.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:44 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:44.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:44 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-08-18T23:39:44.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:44 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:45.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:45 smithi123 ceph-mon[21808]: Deploying daemon osd.9 on smithi078 2024-08-18T23:39:45.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:45 smithi123 ceph-mon[21808]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-08-18T23:39:45.424 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:45 smithi078 ceph-mon[18440]: Deploying daemon osd.9 on smithi078 2024-08-18T23:39:45.424 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:45 smithi078 ceph-mon[18440]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-08-18T23:39:45.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:45 smithi161 ceph-mon[22027]: Deploying daemon osd.9 on smithi078 2024-08-18T23:39:45.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:45 smithi161 ceph-mon[22027]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-08-18T23:39:47.414 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:47 smithi078 ceph-mon[18440]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-18T23:39:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:47 smithi161 ceph-mon[22027]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-18T23:39:47.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:47 smithi123 ceph-mon[21808]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-18T23:39:49.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:49 smithi161 ceph-mon[22027]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:49 smithi078 ceph-mon[18440]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:49.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:49 smithi123 ceph-mon[21808]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:50.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:50 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:39:50.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:50 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:50.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:50 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:50.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:50 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:39:50.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:50 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:50.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:50 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:50.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:50 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:39:50.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:50 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:50.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:50 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:51 smithi161 ceph-mon[22027]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:51.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:51 smithi078 ceph-mon[18440]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:51.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:51 smithi123 ceph-mon[21808]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:53.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:53.122+0000 7fbedbfff640 1 -- 172.21.15.78:0/542447110 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbee4062360 con 0x7fbed00a2bb0 2024-08-18T23:39:53.123 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 9 on host 'smithi078' 2024-08-18T23:39:53.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:53.124+0000 7fbee93b0640 1 -- 172.21.15.78:0/542447110 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbed00a2bb0 msgr2=0x7fbed00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:39:53.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:53.124+0000 7fbee93b0640 1 --2- 172.21.15.78:0/542447110 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbed00a2bb0 0x7fbed00a5070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbed4005db0 tx=0x7fbed4005d40 comp rx=0 tx=0).stop 2024-08-18T23:39:53.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:53.124+0000 7fbee93b0640 1 -- 172.21.15.78:0/542447110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee41034f0 msgr2=0x7fbee419f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:39:53.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:53.124+0000 7fbee93b0640 1 --2- 172.21.15.78:0/542447110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee41034f0 0x7fbee419f1a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00eaa0 tx=0x7fbedc00ef60 comp rx=0 tx=0).stop 2024-08-18T23:39:53.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:53.124+0000 7fbee93b0640 1 -- 172.21.15.78:0/542447110 shutdown_connections 2024-08-18T23:39:53.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:53.124+0000 7fbee93b0640 1 --2- 172.21.15.78:0/542447110 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbed00a2bb0 0x7fbed00a5070 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:53.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:53.124+0000 7fbee93b0640 1 --2- 172.21.15.78:0/542447110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee41034f0 0x7fbee419f1a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:53.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:53.124+0000 7fbee93b0640 1 --2- 172.21.15.78:0/542447110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbee4102b30 0x7fbee4198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:53.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:53.124+0000 7fbee93b0640 1 --2- 172.21.15.78:0/542447110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee4101930 0x7fbee4197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:53.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:53.124+0000 7fbee93b0640 1 -- 172.21.15.78:0/542447110 >> 172.21.15.78:0/542447110 conn(0x7fbee40fd0c0 msgr2=0x7fbee4105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:39:53.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:53.125+0000 7fbee93b0640 1 -- 172.21.15.78:0/542447110 shutdown_connections 2024-08-18T23:39:53.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:39:53.125+0000 7fbee93b0640 1 -- 172.21.15.78:0/542447110 wait complete. 2024-08-18T23:39:53.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:53 smithi161 ceph-mon[22027]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:53.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:53.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:53.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:53.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:39:53.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:53 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:53.454 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:53 smithi078 ceph-mon[18440]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:53.454 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:53 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:53.454 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:53 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:53.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:53 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:53.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:53 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:39:53.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:53 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:53.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:53 smithi123 ceph-mon[21808]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:53.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:53 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:53.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:53 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:53.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:53 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:53.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:53 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:39:53.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:53 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:53.794 DEBUG:teuthology.orchestra.run.smithi078:osd.9> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.9.service 2024-08-18T23:39:53.797 INFO:tasks.cephadm:Deploying osd.10 on smithi123 with /dev/vg_nvme/lv_1... 2024-08-18T23:39:53.797 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-18T23:39:54.058 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.b/config 2024-08-18T23:39:54.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:54 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:39:54.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:54 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:54.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:54 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:54.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:54 smithi161 ceph-mon[22027]: from='osd.9 [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-18T23:39:54.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:54 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:39:54.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:54 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:54.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:54 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:54.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:54 smithi123 ceph-mon[21808]: from='osd.9 [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-18T23:39:54.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:54 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:39:54.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:54 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:54.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:54 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:54.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:54 smithi078 ceph-mon[18440]: from='osd.9 [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-18T23:39:55.393 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:55 smithi078 ceph-mon[18440]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:55.393 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:55 smithi078 ceph-mon[18440]: from='osd.9 [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-18T23:39:55.393 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:55 smithi078 ceph-mon[18440]: osdmap e61: 10 total, 9 up, 10 in 2024-08-18T23:39:55.393 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:55 smithi078 ceph-mon[18440]: from='osd.9 [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-08-18T23:39:55.394 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:55 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-18T23:39:55.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:55 smithi161 ceph-mon[22027]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:55.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:55 smithi161 ceph-mon[22027]: from='osd.9 [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-18T23:39:55.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:55 smithi161 ceph-mon[22027]: osdmap e61: 10 total, 9 up, 10 in 2024-08-18T23:39:55.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:55 smithi161 ceph-mon[22027]: from='osd.9 [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-08-18T23:39:55.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:55 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-18T23:39:55.434 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:55 smithi123 ceph-mon[21808]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:55.434 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:55 smithi123 ceph-mon[21808]: from='osd.9 [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-18T23:39:55.434 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:55 smithi123 ceph-mon[21808]: osdmap e61: 10 total, 9 up, 10 in 2024-08-18T23:39:55.434 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:55 smithi123 ceph-mon[21808]: from='osd.9 [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-08-18T23:39:55.434 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:55 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-18T23:39:56.131 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:39:55 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:39:55.796+0000 7ffa8d9fa640 -1 osd.9 0 waiting for initial osdmap 2024-08-18T23:39:56.131 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:39:55 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:39:55.806+0000 7ffa8581a640 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T23:39:56.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:56 smithi161 ceph-mon[22027]: purged_snaps scrub starts 2024-08-18T23:39:56.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:56 smithi161 ceph-mon[22027]: purged_snaps scrub ok 2024-08-18T23:39:56.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:56 smithi161 ceph-mon[22027]: from='osd.9 [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-08-18T23:39:56.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:56 smithi161 ceph-mon[22027]: osdmap e62: 10 total, 9 up, 10 in 2024-08-18T23:39:56.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:56 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-18T23:39:56.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:56 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-18T23:39:56.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:56 smithi161 ceph-mon[22027]: from='osd.9 [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661]' entity='osd.9' 2024-08-18T23:39:56.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:56 smithi078 ceph-mon[18440]: purged_snaps scrub starts 2024-08-18T23:39:56.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:56 smithi078 ceph-mon[18440]: purged_snaps scrub ok 2024-08-18T23:39:56.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:56 smithi078 ceph-mon[18440]: from='osd.9 [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-08-18T23:39:56.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:56 smithi078 ceph-mon[18440]: osdmap e62: 10 total, 9 up, 10 in 2024-08-18T23:39:56.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:56 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-18T23:39:56.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:56 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-18T23:39:56.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:56 smithi078 ceph-mon[18440]: from='osd.9 [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661]' entity='osd.9' 2024-08-18T23:39:56.485 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:56 smithi123 ceph-mon[21808]: purged_snaps scrub starts 2024-08-18T23:39:56.485 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:56 smithi123 ceph-mon[21808]: purged_snaps scrub ok 2024-08-18T23:39:56.485 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:56 smithi123 ceph-mon[21808]: from='osd.9 [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-08-18T23:39:56.485 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:56 smithi123 ceph-mon[21808]: osdmap e62: 10 total, 9 up, 10 in 2024-08-18T23:39:56.485 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:56 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-18T23:39:56.485 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:56 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-18T23:39:56.485 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:56 smithi123 ceph-mon[21808]: from='osd.9 [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661]' entity='osd.9' 2024-08-18T23:39:56.488 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-08-18T23:39:56.515 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_1 2024-08-18T23:39:56.777 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.b/config 2024-08-18T23:39:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.257+0000 7fb57c2df640 1 -- 172.21.15.123:0/1990394029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5741023c0 msgr2=0x7fb574108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:39:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.257+0000 7fb57c2df640 1 --2- 172.21.15.123:0/1990394029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5741023c0 0x7fb574108c50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009ca0 tx=0x7fb56c02a8c0 comp rx=0 tx=0).stop 2024-08-18T23:39:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.257+0000 7fb57c2df640 1 -- 172.21.15.123:0/1990394029 shutdown_connections 2024-08-18T23:39:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.257+0000 7fb57c2df640 1 --2- 172.21.15.123:0/1990394029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5741023c0 0x7fb574108c50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.257+0000 7fb57c2df640 1 --2- 172.21.15.123:0/1990394029 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb574101a00 0x7fb574101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.257+0000 7fb57c2df640 1 --2- 172.21.15.123:0/1990394029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb574100800 0x7fb574100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.257+0000 7fb57c2df640 1 -- 172.21.15.123:0/1990394029 >> 172.21.15.123:0/1990394029 conn(0x7fb5740fbfb0 msgr2=0x7fb5740fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:39:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.257+0000 7fb57c2df640 1 -- 172.21.15.123:0/1990394029 shutdown_connections 2024-08-18T23:39:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.257+0000 7fb57c2df640 1 -- 172.21.15.123:0/1990394029 wait complete. 2024-08-18T23:39:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.258+0000 7fb57c2df640 1 Processor -- start 2024-08-18T23:39:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.258+0000 7fb57c2df640 1 -- start start 2024-08-18T23:39:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.258+0000 7fb57c2df640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb574100800 0x7fb57406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:39:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.258+0000 7fb57c2df640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb574101a00 0x7fb57419c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:39:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.258+0000 7fb57c2df640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5741023c0 0x7fb57419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:39:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.258+0000 7fb57c2df640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb57410c460 con 0x7fb5741023c0 2024-08-18T23:39:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.258+0000 7fb57c2df640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb57410c2e0 con 0x7fb574100800 2024-08-18T23:39:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.258+0000 7fb57c2df640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb57410c5e0 con 0x7fb574101a00 2024-08-18T23:39:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.258+0000 7fb57a07b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb574100800 0x7fb57406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:39:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.258+0000 7fb57a07b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb574100800 0x7fb57406a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:57816/0 (socket says 172.21.15.123:57816) 2024-08-18T23:39:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.258+0000 7fb57a07b640 1 -- 172.21.15.123:0/2718621461 learned_addr learned my addr 172.21.15.123:0/2718621461 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-08-18T23:39:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.258+0000 7fb57987a640 1 --2- 172.21.15.123:0/2718621461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb574101a00 0x7fb57419c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:39:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.258+0000 7fb57a07b640 1 -- 172.21.15.123:0/2718621461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb574101a00 msgr2=0x7fb57419c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:39:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.258+0000 7fb57a07b640 1 --2- 172.21.15.123:0/2718621461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb574101a00 0x7fb57419c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:57.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.258+0000 7fb57a07b640 1 -- 172.21.15.123:0/2718621461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5741023c0 msgr2=0x7fb57419ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:39:57.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.258+0000 7fb57a07b640 1 --2- 172.21.15.123:0/2718621461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5741023c0 0x7fb57419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:39:57.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.258+0000 7fb57a07b640 1 -- 172.21.15.123:0/2718621461 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57419f720 con 0x7fb574100800 2024-08-18T23:39:57.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.258+0000 7fb57987a640 1 --2- 172.21.15.123:0/2718621461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb574101a00 0x7fb57419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:39:57.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.259+0000 7fb57a07b640 1 --2- 172.21.15.123:0/2718621461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb574100800 0x7fb57406a840 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb57000e9c0 tx=0x7fb57000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:39:57.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.259+0000 7fb56b7fe640 1 -- 172.21.15.123:0/2718621461 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57000cd80 con 0x7fb574100800 2024-08-18T23:39:57.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.259+0000 7fb56b7fe640 1 -- 172.21.15.123:0/2718621461 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5700035f0 con 0x7fb574100800 2024-08-18T23:39:57.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.259+0000 7fb57c2df640 1 -- 172.21.15.123:0/2718621461 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57419fa10 con 0x7fb574100800 2024-08-18T23:39:57.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.259+0000 7fb56b7fe640 1 -- 172.21.15.123:0/2718621461 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570010690 con 0x7fb574100800 2024-08-18T23:39:57.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.259+0000 7fb57c2df640 1 -- 172.21.15.123:0/2718621461 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb57419ff50 con 0x7fb574100800 2024-08-18T23:39:57.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.260+0000 7fb57c2df640 1 -- 172.21.15.123:0/2718621461 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb574107bb0 con 0x7fb574100800 2024-08-18T23:39:57.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.261+0000 7fb56b7fe640 1 -- 172.21.15.123:0/2718621461 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb5700037a0 con 0x7fb574100800 2024-08-18T23:39:57.283 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.262+0000 7fb56b7fe640 1 --2- 172.21.15.123:0/2718621461 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb5600a2b80 0x7fb5600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:39:57.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.262+0000 7fb56b7fe640 1 -- 172.21.15.123:0/2718621461 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6261+0+0 (secure 0 0 0) 0x7fb5700cedb0 con 0x7fb574100800 2024-08-18T23:39:57.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.262+0000 7fb57987a640 1 --2- 172.21.15.123:0/2718621461 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb5600a2b80 0x7fb5600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:39:57.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.263+0000 7fb57987a640 1 --2- 172.21.15.123:0/2718621461 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb5600a2b80 0x7fb5600a5040 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb57406a5c0 tx=0x7fb564005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:39:57.284 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.266+0000 7fb56b7fe640 1 -- 172.21.15.123:0/2718621461 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb57009c510 con 0x7fb574100800 2024-08-18T23:39:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:57 smithi161 ceph-mon[22027]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-18T23:39:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:57 smithi161 ceph-mon[22027]: osd.9 [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661] boot 2024-08-18T23:39:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:57 smithi161 ceph-mon[22027]: osdmap e63: 10 total, 10 up, 10 in 2024-08-18T23:39:57.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-18T23:39:57.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:57.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:57.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:57.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:57.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:57.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:57.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:57.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:57.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:39:57.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:57.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:39:57.448+0000 7fb57c2df640 1 -- 172.21.15.123:0/2718621461 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb574062360 con 0x7fb5600a2b80 2024-08-18T23:39:57.449 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:57 smithi123 ceph-mon[21808]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:57.449 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-18T23:39:57.449 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:57 smithi123 ceph-mon[21808]: osd.9 [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661] boot 2024-08-18T23:39:57.449 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:57 smithi123 ceph-mon[21808]: osdmap e63: 10 total, 10 up, 10 in 2024-08-18T23:39:57.449 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-18T23:39:57.449 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:57.449 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:57.449 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:57.449 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:57.449 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:57.449 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:57.450 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:57.450 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:57.450 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:39:57.450 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:57 smithi078 ceph-mon[18440]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-08-18T23:39:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-18T23:39:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:57 smithi078 ceph-mon[18440]: osd.9 [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661] boot 2024-08-18T23:39:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:57 smithi078 ceph-mon[18440]: osdmap e63: 10 total, 10 up, 10 in 2024-08-18T23:39:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-18T23:39:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:57.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:57.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:57.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:39:57.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:57.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:57.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:39:57.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:39:58.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:58 smithi123 ceph-mon[21808]: Detected new or changed devices on smithi078 2024-08-18T23:39:58.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:58 smithi123 ceph-mon[21808]: Adjusting osd_memory_target on smithi078 to 4272M 2024-08-18T23:39:58.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:58 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:39:58.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:58 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:39:58.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:58 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:58.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:58 smithi161 ceph-mon[22027]: Detected new or changed devices on smithi078 2024-08-18T23:39:58.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:58 smithi161 ceph-mon[22027]: Adjusting osd_memory_target on smithi078 to 4272M 2024-08-18T23:39:58.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:58 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:39:58.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:58 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:39:58.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:58 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:58.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:58 smithi078 ceph-mon[18440]: Detected new or changed devices on smithi078 2024-08-18T23:39:58.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:58 smithi078 ceph-mon[18440]: Adjusting osd_memory_target on smithi078 to 4272M 2024-08-18T23:39:58.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:58 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:39:58.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:58 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:39:58.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:58 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:39:59.139 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:59 smithi123 ceph-mon[21808]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:39:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:59 smithi161 ceph-mon[22027]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:39:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:59 smithi161 ceph-mon[22027]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:39:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:39:59 smithi161 ceph-mon[22027]: osdmap e64: 10 total, 10 up, 10 in 2024-08-18T23:39:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:59 smithi078 ceph-mon[18440]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:39:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:59 smithi078 ceph-mon[18440]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:39:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:39:59 smithi078 ceph-mon[18440]: osdmap e64: 10 total, 10 up, 10 in 2024-08-18T23:39:59.537 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:59 smithi123 ceph-mon[21808]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:39:59.537 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:39:59 smithi123 ceph-mon[21808]: osdmap e64: 10 total, 10 up, 10 in 2024-08-18T23:40:00.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:00 smithi161 ceph-mon[22027]: from='client.? 172.21.15.123:0/3224808807' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59aa32ff-510e-4b52-8170-08a34a0b6b57"}]: dispatch 2024-08-18T23:40:00.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:00 smithi161 ceph-mon[22027]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59aa32ff-510e-4b52-8170-08a34a0b6b57"}]: dispatch 2024-08-18T23:40:00.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:00 smithi161 ceph-mon[22027]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59aa32ff-510e-4b52-8170-08a34a0b6b57"}]': finished 2024-08-18T23:40:00.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:00 smithi161 ceph-mon[22027]: osdmap e65: 11 total, 10 up, 11 in 2024-08-18T23:40:00.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:00 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-18T23:40:00.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:00 smithi161 ceph-mon[22027]: overall HEALTH_OK 2024-08-18T23:40:00.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:00 smithi123 ceph-mon[21808]: from='client.? 172.21.15.123:0/3224808807' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59aa32ff-510e-4b52-8170-08a34a0b6b57"}]: dispatch 2024-08-18T23:40:00.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:00 smithi123 ceph-mon[21808]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59aa32ff-510e-4b52-8170-08a34a0b6b57"}]: dispatch 2024-08-18T23:40:00.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:00 smithi123 ceph-mon[21808]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59aa32ff-510e-4b52-8170-08a34a0b6b57"}]': finished 2024-08-18T23:40:00.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:00 smithi123 ceph-mon[21808]: osdmap e65: 11 total, 10 up, 11 in 2024-08-18T23:40:00.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:00 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-18T23:40:00.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:00 smithi123 ceph-mon[21808]: overall HEALTH_OK 2024-08-18T23:40:00.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:00 smithi078 ceph-mon[18440]: from='client.? 172.21.15.123:0/3224808807' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59aa32ff-510e-4b52-8170-08a34a0b6b57"}]: dispatch 2024-08-18T23:40:00.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:00 smithi078 ceph-mon[18440]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59aa32ff-510e-4b52-8170-08a34a0b6b57"}]: dispatch 2024-08-18T23:40:00.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:00 smithi078 ceph-mon[18440]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59aa32ff-510e-4b52-8170-08a34a0b6b57"}]': finished 2024-08-18T23:40:00.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:00 smithi078 ceph-mon[18440]: osdmap e65: 11 total, 10 up, 11 in 2024-08-18T23:40:00.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:00 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-18T23:40:00.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:00 smithi078 ceph-mon[18440]: overall HEALTH_OK 2024-08-18T23:40:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:01 smithi161 ceph-mon[22027]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:01.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:01 smithi161 ceph-mon[22027]: from='client.? 172.21.15.123:0/3916119064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:40:01.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:01 smithi078 ceph-mon[18440]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:01.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:01 smithi078 ceph-mon[18440]: from='client.? 172.21.15.123:0/3916119064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:40:01.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:01 smithi123 ceph-mon[21808]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:01.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:01 smithi123 ceph-mon[21808]: from='client.? 172.21.15.123:0/3916119064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:40:03.422 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:03 smithi123 ceph-mon[21808]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:03.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:03 smithi161 ceph-mon[22027]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:03 smithi078 ceph-mon[18440]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:05.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:05 smithi161 ceph-mon[22027]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:05.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:05 smithi078 ceph-mon[18440]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:05.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:05 smithi123 ceph-mon[21808]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:07 smithi161 ceph-mon[22027]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-08-18T23:40:07.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:07 smithi078 ceph-mon[18440]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-08-18T23:40:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:07.515 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:07 smithi123 ceph-mon[21808]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:07.515 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:07 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-08-18T23:40:07.515 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:07 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:08.261 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:08 smithi123 ceph-mon[21808]: Deploying daemon osd.10 on smithi123 2024-08-18T23:40:08.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:08 smithi161 ceph-mon[22027]: Deploying daemon osd.10 on smithi123 2024-08-18T23:40:08.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:08 smithi078 ceph-mon[18440]: Deploying daemon osd.10 on smithi123 2024-08-18T23:40:09.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:09 smithi161 ceph-mon[22027]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:09.460 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:09 smithi123 ceph-mon[21808]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:09.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:09 smithi078 ceph-mon[18440]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:11.353 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:11 smithi123 ceph-mon[21808]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:11.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:11 smithi161 ceph-mon[22027]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:11.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:11 smithi078 ceph-mon[18440]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:12 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:40:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:12 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:12 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:12.428 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:12 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:40:12.428 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:12 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:12.428 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:12 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:12.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:12 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:40:12.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:12 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:12.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:12 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:13.397 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:13 smithi123 ceph-mon[21808]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:13 smithi161 ceph-mon[22027]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:13 smithi078 ceph-mon[18440]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:15.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:15 smithi161 ceph-mon[22027]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:15.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:15 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:15.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:15 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:15 smithi078 ceph-mon[18440]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:15 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:15 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:15.509 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:15 smithi123 ceph-mon[21808]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:15.509 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:15 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:15.509 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:15 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:15.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:40:15.550+0000 7fb56b7fe640 1 -- 172.21.15.123:0/2718621461 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb574062360 con 0x7fb5600a2b80 2024-08-18T23:40:15.552 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 10 on host 'smithi123' 2024-08-18T23:40:15.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:40:15.553+0000 7fb57c2df640 1 -- 172.21.15.123:0/2718621461 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb5600a2b80 msgr2=0x7fb5600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:15.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:40:15.553+0000 7fb57c2df640 1 --2- 172.21.15.123:0/2718621461 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb5600a2b80 0x7fb5600a5040 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb57406a5c0 tx=0x7fb564005d90 comp rx=0 tx=0).stop 2024-08-18T23:40:15.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:40:15.553+0000 7fb57c2df640 1 -- 172.21.15.123:0/2718621461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb574100800 msgr2=0x7fb57406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:15.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:40:15.553+0000 7fb57c2df640 1 --2- 172.21.15.123:0/2718621461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb574100800 0x7fb57406a840 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb57000e9c0 tx=0x7fb57000ee80 comp rx=0 tx=0).stop 2024-08-18T23:40:15.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:40:15.553+0000 7fb57c2df640 1 -- 172.21.15.123:0/2718621461 shutdown_connections 2024-08-18T23:40:15.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:40:15.553+0000 7fb57c2df640 1 --2- 172.21.15.123:0/2718621461 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb5600a2b80 0x7fb5600a5040 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:15.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:40:15.553+0000 7fb57c2df640 1 --2- 172.21.15.123:0/2718621461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5741023c0 0x7fb57419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:15.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:40:15.553+0000 7fb57c2df640 1 --2- 172.21.15.123:0/2718621461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb574101a00 0x7fb57419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:15.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:40:15.553+0000 7fb57c2df640 1 --2- 172.21.15.123:0/2718621461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb574100800 0x7fb57406a840 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:15.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:40:15.553+0000 7fb57c2df640 1 -- 172.21.15.123:0/2718621461 >> 172.21.15.123:0/2718621461 conn(0x7fb5740fbfb0 msgr2=0x7fb574104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:40:15.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:40:15.553+0000 7fb57c2df640 1 -- 172.21.15.123:0/2718621461 shutdown_connections 2024-08-18T23:40:15.556 INFO:teuthology.orchestra.run.smithi123.stderr:2024-08-18T23:40:15.553+0000 7fb57c2df640 1 -- 172.21.15.123:0/2718621461 wait complete. 2024-08-18T23:40:16.251 DEBUG:teuthology.orchestra.run.smithi123:osd.10> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.10.service 2024-08-18T23:40:16.253 INFO:tasks.cephadm:Deploying osd.11 on smithi161 with /dev/vg_nvme/lv_1... 2024-08-18T23:40:16.253 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-18T23:40:16.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:16 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:16.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:16 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:40:16.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:16 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:16.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:16 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:40:16.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:16 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:16.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:16 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:16.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:16 smithi161 ceph-mon[22027]: from='osd.10 [v2:172.21.15.123:6824/122226286,v1:172.21.15.123:6825/122226286]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-18T23:40:16.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:16 smithi161 ceph-mon[22027]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-18T23:40:16.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:16 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:16.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:16 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:40:16.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:16 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:16.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:16 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:40:16.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:16 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:16.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:16 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:16.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:16 smithi078 ceph-mon[18440]: from='osd.10 [v2:172.21.15.123:6824/122226286,v1:172.21.15.123:6825/122226286]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-18T23:40:16.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:16 smithi078 ceph-mon[18440]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-18T23:40:16.515 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.c/config 2024-08-18T23:40:16.528 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:16 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:16.528 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:16 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:40:16.528 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:16 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:16.528 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:16 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:40:16.528 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:16 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:16.528 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:16 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:16.528 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:16 smithi123 ceph-mon[21808]: from='osd.10 [v2:172.21.15.123:6824/122226286,v1:172.21.15.123:6825/122226286]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-18T23:40:16.528 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:16 smithi123 ceph-mon[21808]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-18T23:40:17.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:17 smithi161 ceph-mon[22027]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:17.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:17 smithi161 ceph-mon[22027]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-18T23:40:17.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:17 smithi161 ceph-mon[22027]: from='osd.10 [v2:172.21.15.123:6824/122226286,v1:172.21.15.123:6825/122226286]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:40:17.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:17 smithi161 ceph-mon[22027]: osdmap e66: 11 total, 10 up, 11 in 2024-08-18T23:40:17.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:17 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-18T23:40:17.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:17 smithi161 ceph-mon[22027]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:40:17.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:17 smithi123 ceph-mon[21808]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:17.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:17 smithi123 ceph-mon[21808]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-18T23:40:17.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:17 smithi123 ceph-mon[21808]: from='osd.10 [v2:172.21.15.123:6824/122226286,v1:172.21.15.123:6825/122226286]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:40:17.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:17 smithi123 ceph-mon[21808]: osdmap e66: 11 total, 10 up, 11 in 2024-08-18T23:40:17.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:17 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-18T23:40:17.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:17 smithi123 ceph-mon[21808]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:40:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:17 smithi078 ceph-mon[18440]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:17 smithi078 ceph-mon[18440]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-18T23:40:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:17 smithi078 ceph-mon[18440]: from='osd.10 [v2:172.21.15.123:6824/122226286,v1:172.21.15.123:6825/122226286]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:40:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:17 smithi078 ceph-mon[18440]: osdmap e66: 11 total, 10 up, 11 in 2024-08-18T23:40:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:17 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-18T23:40:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:17 smithi078 ceph-mon[18440]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-08-18T23:40:17.716 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:40:17 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:40:17.559+0000 7f3f86d98640 -1 osd.10 0 waiting for initial osdmap 2024-08-18T23:40:17.716 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:40:17 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:40:17.572+0000 7f3f82bc0640 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T23:40:18.612 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:18 smithi161 ceph-mon[22027]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-08-18T23:40:18.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:18 smithi161 ceph-mon[22027]: osdmap e67: 11 total, 10 up, 11 in 2024-08-18T23:40:18.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:18 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-18T23:40:18.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:18 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-18T23:40:18.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:18 smithi161 ceph-mon[22027]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:18.712 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:18 smithi123 ceph-mon[21808]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-08-18T23:40:18.712 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:18 smithi123 ceph-mon[21808]: osdmap e67: 11 total, 10 up, 11 in 2024-08-18T23:40:18.712 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:18 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-18T23:40:18.712 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:18 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-18T23:40:18.712 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:18 smithi123 ceph-mon[21808]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:18.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:18 smithi078 ceph-mon[18440]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-08-18T23:40:18.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:18 smithi078 ceph-mon[18440]: osdmap e67: 11 total, 10 up, 11 in 2024-08-18T23:40:18.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:18 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-18T23:40:18.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:18 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-18T23:40:18.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:18 smithi078 ceph-mon[18440]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-18T23:40:19.106 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-18T23:40:19.131 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_1 2024-08-18T23:40:19.390 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.c/config 2024-08-18T23:40:19.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:19 smithi161 ceph-mon[22027]: purged_snaps scrub starts 2024-08-18T23:40:19.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:19 smithi161 ceph-mon[22027]: purged_snaps scrub ok 2024-08-18T23:40:19.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:19 smithi161 ceph-mon[22027]: osd.10 [v2:172.21.15.123:6824/122226286,v1:172.21.15.123:6825/122226286] boot 2024-08-18T23:40:19.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:19 smithi161 ceph-mon[22027]: osdmap e68: 11 total, 11 up, 11 in 2024-08-18T23:40:19.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-18T23:40:19.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:19 smithi161 ceph-mon[22027]: osdmap e69: 11 total, 11 up, 11 in 2024-08-18T23:40:19.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:19 smithi161 ceph-mon[22027]: Detected new or changed devices on smithi123 2024-08-18T23:40:19.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:19.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:19.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:19.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:19.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:19.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:19.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:19 smithi161 ceph-mon[22027]: Adjusting osd_memory_target on smithi123 to 4272M 2024-08-18T23:40:19.766 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:19.766 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:19.766 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:40:19.766 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:19 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:19.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.847+0000 7fdc3805c640 1 -- 172.21.15.161:0/4006240415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc30103500 msgr2=0x7fdc30109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:19.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.847+0000 7fdc3805c640 1 --2- 172.21.15.161:0/4006240415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc30103500 0x7fdc30109d90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdc28009900 tx=0x7fdc2802a410 comp rx=0 tx=0).stop 2024-08-18T23:40:19.864 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.847+0000 7fdc3805c640 1 -- 172.21.15.161:0/4006240415 shutdown_connections 2024-08-18T23:40:19.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.847+0000 7fdc3805c640 1 --2- 172.21.15.161:0/4006240415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc30103500 0x7fdc30109d90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:19.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.847+0000 7fdc3805c640 1 --2- 172.21.15.161:0/4006240415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc30102b40 0x7fdc30102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:19.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.847+0000 7fdc3805c640 1 --2- 172.21.15.161:0/4006240415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc30101940 0x7fdc30101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:19.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.847+0000 7fdc3805c640 1 -- 172.21.15.161:0/4006240415 >> 172.21.15.161:0/4006240415 conn(0x7fdc300fd0f0 msgr2=0x7fdc300ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:40:19.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.847+0000 7fdc3805c640 1 -- 172.21.15.161:0/4006240415 shutdown_connections 2024-08-18T23:40:19.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.847+0000 7fdc3805c640 1 -- 172.21.15.161:0/4006240415 wait complete. 2024-08-18T23:40:19.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.847+0000 7fdc3805c640 1 Processor -- start 2024-08-18T23:40:19.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.847+0000 7fdc3805c640 1 -- start start 2024-08-18T23:40:19.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc3805c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc30101940 0x7fdc30197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:19.865 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc3805c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc30102b40 0x7fdc301981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc3805c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc30103500 0x7fdc3019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc3805c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3010d640 con 0x7fdc30103500 2024-08-18T23:40:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc3805c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3010d4c0 con 0x7fdc30102b40 2024-08-18T23:40:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc3805c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3010d7c0 con 0x7fdc30101940 2024-08-18T23:40:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc35df8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc30101940 0x7fdc30197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc35df8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc30101940 0x7fdc30197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:35744/0 (socket says 172.21.15.161:35744) 2024-08-18T23:40:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc35df8640 1 -- 172.21.15.161:0/962271819 learned_addr learned my addr 172.21.15.161:0/962271819 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-18T23:40:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc365f9640 1 --2- 172.21.15.161:0/962271819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc30103500 0x7fdc3019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc355f7640 1 --2- 172.21.15.161:0/962271819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc30102b40 0x7fdc301981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc35df8640 1 -- 172.21.15.161:0/962271819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc30102b40 msgr2=0x7fdc301981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc35df8640 1 --2- 172.21.15.161:0/962271819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc30102b40 0x7fdc301981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc35df8640 1 -- 172.21.15.161:0/962271819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc30103500 msgr2=0x7fdc3019f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:19.866 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc35df8640 1 --2- 172.21.15.161:0/962271819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc30103500 0x7fdc3019f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:19.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc35df8640 1 -- 172.21.15.161:0/962271819 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc3019f880 con 0x7fdc30101940 2024-08-18T23:40:19.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc35df8640 1 --2- 172.21.15.161:0/962271819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc30101940 0x7fdc30197c60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdc2000eaa0 tx=0x7fdc2000eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:40:19.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc2effd640 1 -- 172.21.15.161:0/962271819 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20016070 con 0x7fdc30101940 2024-08-18T23:40:19.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc2effd640 1 -- 172.21.15.161:0/962271819 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc2000ce20 con 0x7fdc30101940 2024-08-18T23:40:19.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc3805c640 1 -- 172.21.15.161:0/962271819 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc3019fb70 con 0x7fdc30101940 2024-08-18T23:40:19.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc2effd640 1 -- 172.21.15.161:0/962271819 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20011470 con 0x7fdc30101940 2024-08-18T23:40:19.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.848+0000 7fdc3805c640 1 -- 172.21.15.161:0/962271819 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc301a0060 con 0x7fdc30101940 2024-08-18T23:40:19.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.849+0000 7fdc3805c640 1 -- 172.21.15.161:0/962271819 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc30108d30 con 0x7fdc30101940 2024-08-18T23:40:19.867 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.850+0000 7fdc2effd640 1 -- 172.21.15.161:0/962271819 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdc20019070 con 0x7fdc30101940 2024-08-18T23:40:19.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.851+0000 7fdc2effd640 1 --2- 172.21.15.161:0/962271819 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdc180a2b80 0x7fdc180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:19.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.851+0000 7fdc2effd640 1 -- 172.21.15.161:0/962271819 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6723+0+0 (secure 0 0 0) 0x7fdc200ceae0 con 0x7fdc30101940 2024-08-18T23:40:19.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.851+0000 7fdc355f7640 1 --2- 172.21.15.161:0/962271819 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdc180a2b80 0x7fdc180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:19.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.852+0000 7fdc355f7640 1 --2- 172.21.15.161:0/962271819 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdc180a2b80 0x7fdc180a5040 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdc30199170 tx=0x7fdc24009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:40:19.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:19.855+0000 7fdc2effd640 1 -- 172.21.15.161:0/962271819 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdc2009c070 con 0x7fdc30101940 2024-08-18T23:40:19.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:19 smithi078 ceph-mon[18440]: purged_snaps scrub starts 2024-08-18T23:40:19.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:19 smithi078 ceph-mon[18440]: purged_snaps scrub ok 2024-08-18T23:40:19.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:19 smithi078 ceph-mon[18440]: osd.10 [v2:172.21.15.123:6824/122226286,v1:172.21.15.123:6825/122226286] boot 2024-08-18T23:40:19.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:19 smithi078 ceph-mon[18440]: osdmap e68: 11 total, 11 up, 11 in 2024-08-18T23:40:19.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-18T23:40:19.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:19 smithi078 ceph-mon[18440]: osdmap e69: 11 total, 11 up, 11 in 2024-08-18T23:40:19.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:19 smithi078 ceph-mon[18440]: Detected new or changed devices on smithi123 2024-08-18T23:40:19.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:19.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:19.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:19.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:19.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:19.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:19.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:19 smithi078 ceph-mon[18440]: Adjusting osd_memory_target on smithi123 to 4272M 2024-08-18T23:40:19.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:19.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:19.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:40:19.971 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:19 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:20.033 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:20.032+0000 7fdc3805c640 1 -- 172.21.15.161:0/962271819 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdc30062360 con 0x7fdc180a2b80 2024-08-18T23:40:20.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:19 smithi123 ceph-mon[21808]: purged_snaps scrub starts 2024-08-18T23:40:20.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:19 smithi123 ceph-mon[21808]: purged_snaps scrub ok 2024-08-18T23:40:20.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:19 smithi123 ceph-mon[21808]: osd.10 [v2:172.21.15.123:6824/122226286,v1:172.21.15.123:6825/122226286] boot 2024-08-18T23:40:20.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:19 smithi123 ceph-mon[21808]: osdmap e68: 11 total, 11 up, 11 in 2024-08-18T23:40:20.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-18T23:40:20.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:19 smithi123 ceph-mon[21808]: osdmap e69: 11 total, 11 up, 11 in 2024-08-18T23:40:20.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:19 smithi123 ceph-mon[21808]: Detected new or changed devices on smithi123 2024-08-18T23:40:20.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:20.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:20.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:20.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:20.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:20.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:20.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:19 smithi123 ceph-mon[21808]: Adjusting osd_memory_target on smithi123 to 4272M 2024-08-18T23:40:20.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:20.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:20.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:40:20.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:19 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:21.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:21 smithi123 ceph-mon[21808]: osdmap e70: 11 total, 11 up, 11 in 2024-08-18T23:40:21.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:21 smithi123 ceph-mon[21808]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:40:21.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:21 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:40:21.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:21 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:40:21.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:21 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:21.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:21 smithi123 ceph-mon[21808]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:21.422 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:21 smithi161 ceph-mon[22027]: osdmap e70: 11 total, 11 up, 11 in 2024-08-18T23:40:21.422 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:21 smithi161 ceph-mon[22027]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:40:21.422 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:21 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:40:21.422 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:21 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:40:21.422 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:21 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:21.422 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:21 smithi161 ceph-mon[22027]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:21.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:21 smithi078 ceph-mon[18440]: osdmap e70: 11 total, 11 up, 11 in 2024-08-18T23:40:21.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:21 smithi078 ceph-mon[18440]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:40:21.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:21 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-18T23:40:21.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:21 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-18T23:40:21.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:21 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:21.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:21 smithi078 ceph-mon[18440]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:22.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:22 smithi123 ceph-mon[21808]: osdmap e71: 11 total, 11 up, 11 in 2024-08-18T23:40:22.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:22 smithi161 ceph-mon[22027]: osdmap e71: 11 total, 11 up, 11 in 2024-08-18T23:40:22.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:22 smithi078 ceph-mon[18440]: osdmap e71: 11 total, 11 up, 11 in 2024-08-18T23:40:23.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:23 smithi123 ceph-mon[21808]: osdmap e72: 11 total, 11 up, 11 in 2024-08-18T23:40:23.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:23 smithi123 ceph-mon[21808]: pgmap v212: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 163 KiB/s, 0 objects/s recovering 2024-08-18T23:40:23.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:23 smithi123 ceph-mon[21808]: from='client.? 172.21.15.161:0/1408055333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67bacab4-d078-4c5c-b4ff-9f6a8a00f507"}]: dispatch 2024-08-18T23:40:23.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:23 smithi123 ceph-mon[21808]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67bacab4-d078-4c5c-b4ff-9f6a8a00f507"}]: dispatch 2024-08-18T23:40:23.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:23 smithi123 ceph-mon[21808]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67bacab4-d078-4c5c-b4ff-9f6a8a00f507"}]': finished 2024-08-18T23:40:23.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:23 smithi123 ceph-mon[21808]: osdmap e73: 12 total, 11 up, 12 in 2024-08-18T23:40:23.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-18T23:40:23.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:23 smithi161 ceph-mon[22027]: osdmap e72: 11 total, 11 up, 11 in 2024-08-18T23:40:23.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:23 smithi161 ceph-mon[22027]: pgmap v212: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 163 KiB/s, 0 objects/s recovering 2024-08-18T23:40:23.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:23 smithi161 ceph-mon[22027]: from='client.? 172.21.15.161:0/1408055333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67bacab4-d078-4c5c-b4ff-9f6a8a00f507"}]: dispatch 2024-08-18T23:40:23.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:23 smithi161 ceph-mon[22027]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67bacab4-d078-4c5c-b4ff-9f6a8a00f507"}]: dispatch 2024-08-18T23:40:23.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:23 smithi161 ceph-mon[22027]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67bacab4-d078-4c5c-b4ff-9f6a8a00f507"}]': finished 2024-08-18T23:40:23.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:23 smithi161 ceph-mon[22027]: osdmap e73: 12 total, 11 up, 12 in 2024-08-18T23:40:23.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-18T23:40:23.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:23 smithi078 ceph-mon[18440]: osdmap e72: 11 total, 11 up, 11 in 2024-08-18T23:40:23.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:23 smithi078 ceph-mon[18440]: pgmap v212: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 163 KiB/s, 0 objects/s recovering 2024-08-18T23:40:23.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:23 smithi078 ceph-mon[18440]: from='client.? 172.21.15.161:0/1408055333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67bacab4-d078-4c5c-b4ff-9f6a8a00f507"}]: dispatch 2024-08-18T23:40:23.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:23 smithi078 ceph-mon[18440]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67bacab4-d078-4c5c-b4ff-9f6a8a00f507"}]: dispatch 2024-08-18T23:40:23.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:23 smithi078 ceph-mon[18440]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67bacab4-d078-4c5c-b4ff-9f6a8a00f507"}]': finished 2024-08-18T23:40:23.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:23 smithi078 ceph-mon[18440]: osdmap e73: 12 total, 11 up, 12 in 2024-08-18T23:40:23.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-18T23:40:24.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:24 smithi123 ceph-mon[21808]: from='client.? 172.21.15.161:0/3347351142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:40:24.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:24 smithi161 ceph-mon[22027]: from='client.? 172.21.15.161:0/3347351142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:40:24.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:24 smithi078 ceph-mon[18440]: from='client.? 172.21.15.161:0/3347351142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-18T23:40:25.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:25 smithi123 ceph-mon[21808]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-08-18T23:40:25.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:25 smithi161 ceph-mon[22027]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-08-18T23:40:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:25 smithi078 ceph-mon[18440]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-08-18T23:40:27.376 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:27 smithi161 ceph-mon[22027]: pgmap v215: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-18T23:40:27.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:27 smithi078 ceph-mon[18440]: pgmap v215: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-18T23:40:27.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:27 smithi123 ceph-mon[21808]: pgmap v215: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-18T23:40:29.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:29 smithi161 ceph-mon[22027]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-08-18T23:40:29.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:29 smithi078 ceph-mon[18440]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-08-18T23:40:29.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:29 smithi123 ceph-mon[21808]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-08-18T23:40:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:30 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-08-18T23:40:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:30 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:30.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:30 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-08-18T23:40:30.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:30 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:30.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:30 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-08-18T23:40:30.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:30 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:31 smithi161 ceph-mon[22027]: Deploying daemon osd.11 on smithi161 2024-08-18T23:40:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:31 smithi161 ceph-mon[22027]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:31.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:31 smithi078 ceph-mon[18440]: Deploying daemon osd.11 on smithi161 2024-08-18T23:40:31.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:31 smithi078 ceph-mon[18440]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:31.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:31 smithi123 ceph-mon[21808]: Deploying daemon osd.11 on smithi161 2024-08-18T23:40:31.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:31 smithi123 ceph-mon[21808]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:33.421 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:33 smithi161 ceph-mon[22027]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:33.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:33 smithi078 ceph-mon[18440]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:33.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:33 smithi123 ceph-mon[21808]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:35.385 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:35 smithi161 ceph-mon[22027]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:35.385 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:35 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:40:35.385 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:35 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:35.385 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:35 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:35.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:35 smithi078 ceph-mon[18440]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:35.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:35 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:40:35.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:35 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:35.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:35 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:35.523 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:35 smithi123 ceph-mon[21808]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:35.523 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:35 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:40:35.523 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:35 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:35.523 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:35 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:37.255 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:37 smithi161 ceph-mon[22027]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:37.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:37 smithi078 ceph-mon[18440]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:37.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:37 smithi123 ceph-mon[21808]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:38.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:38.323+0000 7fdc2effd640 1 -- 172.21.15.161:0/962271819 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fdc30062360 con 0x7fdc180a2b80 2024-08-18T23:40:38.324 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 11 on host 'smithi161' 2024-08-18T23:40:38.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:38.325+0000 7fdc3805c640 1 -- 172.21.15.161:0/962271819 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdc180a2b80 msgr2=0x7fdc180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:38.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:38.325+0000 7fdc3805c640 1 --2- 172.21.15.161:0/962271819 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdc180a2b80 0x7fdc180a5040 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdc30199170 tx=0x7fdc24009110 comp rx=0 tx=0).stop 2024-08-18T23:40:38.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:38.325+0000 7fdc3805c640 1 -- 172.21.15.161:0/962271819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc30101940 msgr2=0x7fdc30197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:38.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:38.325+0000 7fdc3805c640 1 --2- 172.21.15.161:0/962271819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc30101940 0x7fdc30197c60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdc2000eaa0 tx=0x7fdc2000eea0 comp rx=0 tx=0).stop 2024-08-18T23:40:38.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:38.326+0000 7fdc3805c640 1 -- 172.21.15.161:0/962271819 shutdown_connections 2024-08-18T23:40:38.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:38.326+0000 7fdc3805c640 1 --2- 172.21.15.161:0/962271819 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdc180a2b80 0x7fdc180a5040 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:38.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:38.326+0000 7fdc3805c640 1 --2- 172.21.15.161:0/962271819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc30103500 0x7fdc3019f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:38.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:38.326+0000 7fdc3805c640 1 --2- 172.21.15.161:0/962271819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc30102b40 0x7fdc301981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:38.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:38.326+0000 7fdc3805c640 1 --2- 172.21.15.161:0/962271819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc30101940 0x7fdc30197c60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:38.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:38.326+0000 7fdc3805c640 1 -- 172.21.15.161:0/962271819 >> 172.21.15.161:0/962271819 conn(0x7fdc300fd0f0 msgr2=0x7fdc30105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:40:38.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:38.326+0000 7fdc3805c640 1 -- 172.21.15.161:0/962271819 shutdown_connections 2024-08-18T23:40:38.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:38.326+0000 7fdc3805c640 1 -- 172.21.15.161:0/962271819 wait complete. 2024-08-18T23:40:38.576 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:38 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:38.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:38 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:38.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:38 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:38.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:38 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:40:38.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:38 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:38.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:38 smithi161 ceph-mon[22027]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:38.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:38 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:40:38.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:38 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:38.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:38 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:38.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:38.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:38.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:38.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:40:38.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:38.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:38 smithi078 ceph-mon[18440]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:38.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:40:38.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:38.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:38.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:38 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:38.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:38 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:38.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:38 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:38.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:38 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:40:38.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:38 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:38.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:38 smithi123 ceph-mon[21808]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:38.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:38 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:40:38.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:38 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:38.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:38 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:38.975 DEBUG:teuthology.orchestra.run.smithi161:osd.11> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.11.service 2024-08-18T23:40:38.977 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-08-18T23:40:38.978 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd stat -f json 2024-08-18T23:40:39.250 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:40:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:39 smithi161 ceph-mon[22027]: from='osd.11 [v2:172.21.15.161:6824/2037560985,v1:172.21.15.161:6825/2037560985]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-18T23:40:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:39 smithi161 ceph-mon[22027]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-18T23:40:39.688 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:39 smithi078 ceph-mon[18440]: from='osd.11 [v2:172.21.15.161:6824/2037560985,v1:172.21.15.161:6825/2037560985]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-18T23:40:39.688 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:39 smithi078 ceph-mon[18440]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-18T23:40:39.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:39 smithi123 ceph-mon[21808]: from='osd.11 [v2:172.21.15.161:6824/2037560985,v1:172.21.15.161:6825/2037560985]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-18T23:40:39.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:39 smithi123 ceph-mon[21808]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-18T23:40:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.770+0000 7f6c32444640 1 -- 172.21.15.78:0/2110515268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c101a30 msgr2=0x7f6c2c101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.770+0000 7f6c32444640 1 --2- 172.21.15.78:0/2110515268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c101a30 0x7f6c2c101eb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009900 tx=0x7f6c2002a410 comp rx=0 tx=0).stop 2024-08-18T23:40:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.770+0000 7f6c32444640 1 -- 172.21.15.78:0/2110515268 shutdown_connections 2024-08-18T23:40:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.770+0000 7f6c32444640 1 --2- 172.21.15.78:0/2110515268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c2c1023f0 0x7f6c2c108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.770+0000 7f6c32444640 1 --2- 172.21.15.78:0/2110515268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c101a30 0x7f6c2c101eb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.770+0000 7f6c32444640 1 --2- 172.21.15.78:0/2110515268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c2c100830 0x7f6c2c100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.770+0000 7f6c32444640 1 -- 172.21.15.78:0/2110515268 >> 172.21.15.78:0/2110515268 conn(0x7f6c2c0fbfa0 msgr2=0x7f6c2c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:40:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.770+0000 7f6c32444640 1 -- 172.21.15.78:0/2110515268 shutdown_connections 2024-08-18T23:40:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.771+0000 7f6c32444640 1 -- 172.21.15.78:0/2110515268 wait complete. 2024-08-18T23:40:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.771+0000 7f6c32444640 1 Processor -- start 2024-08-18T23:40:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.771+0000 7f6c32444640 1 -- start start 2024-08-18T23:40:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c32444640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c2c100830 0x7f6c2c197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c32444640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c2c101a30 0x7f6c2c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c32444640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c1023f0 0x7f6c2c19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c32444640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c10d5f0 con 0x7f6c2c1023f0 2024-08-18T23:40:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c32444640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c10d470 con 0x7f6c2c101a30 2024-08-18T23:40:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c32444640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c10d770 con 0x7f6c2c100830 2024-08-18T23:40:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c309e1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c1023f0 0x7f6c2c19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c2bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c2c100830 0x7f6c2c197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c2bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c2c100830 0x7f6c2c197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:47074/0 (socket says 172.21.15.78:47074) 2024-08-18T23:40:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c309e1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c1023f0 0x7f6c2c19f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54676/0 (socket says 172.21.15.78:54676) 2024-08-18T23:40:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c2bfff640 1 -- 172.21.15.78:0/3737903740 learned_addr learned my addr 172.21.15.78:0/3737903740 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:40:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c2b7fe640 1 --2- 172.21.15.78:0/3737903740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c2c101a30 0x7f6c2c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c2bfff640 1 -- 172.21.15.78:0/3737903740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c2c101a30 msgr2=0x7f6c2c198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c2bfff640 1 --2- 172.21.15.78:0/3737903740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c2c101a30 0x7f6c2c198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c2bfff640 1 -- 172.21.15.78:0/3737903740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c1023f0 msgr2=0x7f6c2c19f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c2bfff640 1 --2- 172.21.15.78:0/3737903740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c1023f0 0x7f6c2c19f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c2bfff640 1 -- 172.21.15.78:0/3737903740 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2c19f900 con 0x7f6c2c100830 2024-08-18T23:40:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c309e1640 1 --2- 172.21.15.78:0/3737903740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c1023f0 0x7f6c2c19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:40:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c2bfff640 1 --2- 172.21.15.78:0/3737903740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c2c100830 0x7f6c2c197ce0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00d950 tx=0x7f6c1c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:40:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c297fa640 1 -- 172.21.15.78:0/3737903740 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c017070 con 0x7f6c2c100830 2024-08-18T23:40:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.772+0000 7f6c297fa640 1 -- 172.21.15.78:0/3737903740 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c1c009c30 con 0x7f6c2c100830 2024-08-18T23:40:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.773+0000 7f6c32444640 1 -- 172.21.15.78:0/3737903740 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c2c19fbf0 con 0x7f6c2c100830 2024-08-18T23:40:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.773+0000 7f6c297fa640 1 -- 172.21.15.78:0/3737903740 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c00f600 con 0x7f6c2c100830 2024-08-18T23:40:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.773+0000 7f6c32444640 1 -- 172.21.15.78:0/3737903740 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c2c1a00e0 con 0x7f6c2c100830 2024-08-18T23:40:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.773+0000 7f6c32444640 1 -- 172.21.15.78:0/3737903740 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c2c107cf0 con 0x7f6c2c100830 2024-08-18T23:40:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.775+0000 7f6c297fa640 1 -- 172.21.15.78:0/3737903740 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6c1c01a070 con 0x7f6c2c100830 2024-08-18T23:40:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.775+0000 7f6c297fa640 1 --2- 172.21.15.78:0/3737903740 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6c140a2b80 0x7f6c140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.775+0000 7f6c2b7fe640 1 --2- 172.21.15.78:0/3737903740 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6c140a2b80 0x7f6c140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.775+0000 7f6c297fa640 1 -- 172.21.15.78:0/3737903740 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6831+0+0 (secure 0 0 0) 0x7f6c1c0ce470 con 0x7f6c2c100830 2024-08-18T23:40:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.776+0000 7f6c2b7fe640 1 --2- 172.21.15.78:0/3737903740 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6c140a2b80 0x7f6c140a5040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6c20000c00 tx=0x7f6c200315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:40:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.779+0000 7f6c297fa640 1 -- 172.21.15.78:0/3737903740 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6c1c009de0 con 0x7f6c2c100830 2024-08-18T23:40:39.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.956+0000 7f6c32444640 1 -- 172.21.15.78:0/3737903740 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f6c2c02d890 con 0x7f6c2c100830 2024-08-18T23:40:39.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.956+0000 7f6c297fa640 1 -- 172.21.15.78:0/3737903740 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f6c1c095e20 con 0x7f6c2c100830 2024-08-18T23:40:39.957 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:40:39.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.958+0000 7f6c32444640 1 -- 172.21.15.78:0/3737903740 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6c140a2b80 msgr2=0x7f6c140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:39.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.958+0000 7f6c32444640 1 --2- 172.21.15.78:0/3737903740 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6c140a2b80 0x7f6c140a5040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6c20000c00 tx=0x7f6c200315b0 comp rx=0 tx=0).stop 2024-08-18T23:40:39.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.958+0000 7f6c32444640 1 -- 172.21.15.78:0/3737903740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c2c100830 msgr2=0x7f6c2c197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:39.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.958+0000 7f6c32444640 1 --2- 172.21.15.78:0/3737903740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c2c100830 0x7f6c2c197ce0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00d950 tx=0x7f6c1c00de10 comp rx=0 tx=0).stop 2024-08-18T23:40:39.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.959+0000 7f6c32444640 1 -- 172.21.15.78:0/3737903740 shutdown_connections 2024-08-18T23:40:39.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.959+0000 7f6c32444640 1 --2- 172.21.15.78:0/3737903740 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6c140a2b80 0x7f6c140a5040 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:39.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.959+0000 7f6c32444640 1 --2- 172.21.15.78:0/3737903740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c1023f0 0x7f6c2c19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:39.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.959+0000 7f6c32444640 1 --2- 172.21.15.78:0/3737903740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c2c101a30 0x7f6c2c198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:39.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.959+0000 7f6c32444640 1 --2- 172.21.15.78:0/3737903740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c2c100830 0x7f6c2c197ce0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:39.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.959+0000 7f6c32444640 1 -- 172.21.15.78:0/3737903740 >> 172.21.15.78:0/3737903740 conn(0x7f6c2c0fbfa0 msgr2=0x7f6c2c104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:40:39.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.959+0000 7f6c32444640 1 -- 172.21.15.78:0/3737903740 shutdown_connections 2024-08-18T23:40:39.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:39.959+0000 7f6c32444640 1 -- 172.21.15.78:0/3737903740 wait complete. 2024-08-18T23:40:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:40 smithi078 ceph-mon[18440]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-18T23:40:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:40 smithi078 ceph-mon[18440]: osdmap e74: 12 total, 11 up, 12 in 2024-08-18T23:40:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:40 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-18T23:40:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:40 smithi078 ceph-mon[18440]: from='osd.11 [v2:172.21.15.161:6824/2037560985,v1:172.21.15.161:6825/2037560985]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:40:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:40 smithi078 ceph-mon[18440]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:40:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:40 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3737903740' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-18T23:40:40.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:40 smithi078 ceph-mon[18440]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:40.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:40 smithi123 ceph-mon[21808]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-18T23:40:40.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:40 smithi123 ceph-mon[21808]: osdmap e74: 12 total, 11 up, 12 in 2024-08-18T23:40:40.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:40 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-18T23:40:40.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:40 smithi123 ceph-mon[21808]: from='osd.11 [v2:172.21.15.161:6824/2037560985,v1:172.21.15.161:6825/2037560985]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:40:40.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:40 smithi123 ceph-mon[21808]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:40:40.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:40 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3737903740' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-18T23:40:40.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:40 smithi123 ceph-mon[21808]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:40.804 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1724024418,"num_in_osds":12,"osd_in_since":1724024422,"num_remapped_pgs":0} 2024-08-18T23:40:40.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:40 smithi161 ceph-mon[22027]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-18T23:40:40.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:40 smithi161 ceph-mon[22027]: osdmap e74: 12 total, 11 up, 12 in 2024-08-18T23:40:40.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:40 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-18T23:40:40.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:40 smithi161 ceph-mon[22027]: from='osd.11 [v2:172.21.15.161:6824/2037560985,v1:172.21.15.161:6825/2037560985]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:40:40.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:40 smithi161 ceph-mon[22027]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-18T23:40:40.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:40 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3737903740' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-18T23:40:40.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:40 smithi161 ceph-mon[22027]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:41.466 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:40:41 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:40:41.197+0000 7f53285ad640 -1 osd.11 0 waiting for initial osdmap 2024-08-18T23:40:41.466 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:40:41 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:40:41.211+0000 7f53203cd640 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-18T23:40:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:41 smithi078 ceph-mon[18440]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-18T23:40:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:41 smithi078 ceph-mon[18440]: osdmap e75: 12 total, 11 up, 12 in 2024-08-18T23:40:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:41 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-18T23:40:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:41 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-18T23:40:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:41 smithi078 ceph-mon[18440]: from='osd.11 ' entity='osd.11' 2024-08-18T23:40:41.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:41 smithi123 ceph-mon[21808]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-18T23:40:41.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:41 smithi123 ceph-mon[21808]: osdmap e75: 12 total, 11 up, 12 in 2024-08-18T23:40:41.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:41 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-18T23:40:41.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:41 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-18T23:40:41.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:41 smithi123 ceph-mon[21808]: from='osd.11 ' entity='osd.11' 2024-08-18T23:40:41.805 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd stat -f json 2024-08-18T23:40:41.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:41 smithi161 ceph-mon[22027]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-18T23:40:41.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:41 smithi161 ceph-mon[22027]: osdmap e75: 12 total, 11 up, 12 in 2024-08-18T23:40:41.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:41 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-18T23:40:41.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:41 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-18T23:40:41.863 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:41 smithi161 ceph-mon[22027]: from='osd.11 ' entity='osd.11' 2024-08-18T23:40:42.066 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:40:42.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.570+0000 7f34ddfdb640 1 -- 172.21.15.78:0/3884462105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34d8101a00 msgr2=0x7f34d8101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:42.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.570+0000 7f34ddfdb640 1 --2- 172.21.15.78:0/3884462105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34d8101a00 0x7f34d8101e80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009900 tx=0x7f34cc02a410 comp rx=0 tx=0).stop 2024-08-18T23:40:42.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.570+0000 7f34ddfdb640 1 -- 172.21.15.78:0/3884462105 shutdown_connections 2024-08-18T23:40:42.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.570+0000 7f34ddfdb640 1 --2- 172.21.15.78:0/3884462105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34d81023c0 0x7f34d8108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:42.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.570+0000 7f34ddfdb640 1 --2- 172.21.15.78:0/3884462105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34d8101a00 0x7f34d8101e80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:42.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.570+0000 7f34ddfdb640 1 --2- 172.21.15.78:0/3884462105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34d8100800 0x7f34d8100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:42.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.570+0000 7f34ddfdb640 1 -- 172.21.15.78:0/3884462105 >> 172.21.15.78:0/3884462105 conn(0x7f34d80fbfb0 msgr2=0x7f34d80fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:40:42.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.570+0000 7f34ddfdb640 1 -- 172.21.15.78:0/3884462105 shutdown_connections 2024-08-18T23:40:42.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.570+0000 7f34ddfdb640 1 -- 172.21.15.78:0/3884462105 wait complete. 2024-08-18T23:40:42.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.570+0000 7f34ddfdb640 1 Processor -- start 2024-08-18T23:40:42.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.571+0000 7f34ddfdb640 1 -- start start 2024-08-18T23:40:42.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.571+0000 7f34ddfdb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34d8100800 0x7f34d806a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:42.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.571+0000 7f34ddfdb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34d8101a00 0x7f34d819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:42.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.571+0000 7f34ddfdb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34d81023c0 0x7f34d819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:42.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.571+0000 7f34ddfdb640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34d810c580 con 0x7f34d8101a00 2024-08-18T23:40:42.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.571+0000 7f34ddfdb640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f34d810c400 con 0x7f34d81023c0 2024-08-18T23:40:42.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.571+0000 7f34ddfdb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f34d810c700 con 0x7f34d8100800 2024-08-18T23:40:42.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.571+0000 7f34d6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34d8101a00 0x7f34d819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:42.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.571+0000 7f34d6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34d8101a00 0x7f34d819cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54700/0 (socket says 172.21.15.78:54700) 2024-08-18T23:40:42.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.571+0000 7f34d6ffd640 1 -- 172.21.15.78:0/147837420 learned_addr learned my addr 172.21.15.78:0/147837420 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:40:42.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.571+0000 7f34d77fe640 1 --2- 172.21.15.78:0/147837420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34d8100800 0x7f34d806a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:42.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.571+0000 7f34d7fff640 1 --2- 172.21.15.78:0/147837420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34d81023c0 0x7f34d819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:42.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.571+0000 7f34d6ffd640 1 -- 172.21.15.78:0/147837420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34d8100800 msgr2=0x7f34d806a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.571+0000 7f34d6ffd640 1 --2- 172.21.15.78:0/147837420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34d8100800 0x7f34d806a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.571+0000 7f34d6ffd640 1 -- 172.21.15.78:0/147837420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34d81023c0 msgr2=0x7f34d819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.571+0000 7f34d6ffd640 1 --2- 172.21.15.78:0/147837420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34d81023c0 0x7f34d819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.571+0000 7f34d6ffd640 1 -- 172.21.15.78:0/147837420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d819fa50 con 0x7f34d8101a00 2024-08-18T23:40:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.572+0000 7f34d6ffd640 1 --2- 172.21.15.78:0/147837420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34d8101a00 0x7f34d819cb20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009230 tx=0x7f34cc004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:40:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.572+0000 7f34d4ff9640 1 -- 172.21.15.78:0/147837420 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc03f030 con 0x7f34d8101a00 2024-08-18T23:40:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.572+0000 7f34ddfdb640 1 -- 172.21.15.78:0/147837420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d81968f0 con 0x7f34d8101a00 2024-08-18T23:40:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.572+0000 7f34d4ff9640 1 -- 172.21.15.78:0/147837420 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34cc003ce0 con 0x7f34d8101a00 2024-08-18T23:40:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.572+0000 7f34d4ff9640 1 -- 172.21.15.78:0/147837420 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc035070 con 0x7f34d8101a00 2024-08-18T23:40:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.572+0000 7f34ddfdb640 1 -- 172.21.15.78:0/147837420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d8196ea0 con 0x7f34d8101a00 2024-08-18T23:40:42.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.573+0000 7f34ddfdb640 1 -- 172.21.15.78:0/147837420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34d8107c90 con 0x7f34d8101a00 2024-08-18T23:40:42.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.574+0000 7f34d4ff9640 1 -- 172.21.15.78:0/147837420 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f34cc042030 con 0x7f34d8101a00 2024-08-18T23:40:42.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.574+0000 7f34d4ff9640 1 --2- 172.21.15.78:0/147837420 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f34c00a2b80 0x7f34c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:42.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.575+0000 7f34d77fe640 1 --2- 172.21.15.78:0/147837420 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f34c00a2b80 0x7f34c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:42.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.575+0000 7f34d4ff9640 1 -- 172.21.15.78:0/147837420 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f34cc0eef90 con 0x7f34d8101a00 2024-08-18T23:40:42.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.575+0000 7f34d77fe640 1 --2- 172.21.15.78:0/147837420 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f34c00a2b80 0x7f34c00a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f34d8101860 tx=0x7f34c800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:40:42.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.579+0000 7f34d4ff9640 1 -- 172.21.15.78:0/147837420 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f34cc0bc390 con 0x7f34d8101a00 2024-08-18T23:40:42.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:42 smithi078 ceph-mon[18440]: purged_snaps scrub starts 2024-08-18T23:40:42.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:42 smithi078 ceph-mon[18440]: purged_snaps scrub ok 2024-08-18T23:40:42.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-18T23:40:42.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:42 smithi078 ceph-mon[18440]: Detected new or changed devices on smithi161 2024-08-18T23:40:42.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:42.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:42.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:42.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:42.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:42.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:42.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:42 smithi078 ceph-mon[18440]: Adjusting osd_memory_target on smithi161 to 4272M 2024-08-18T23:40:42.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:42.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:42.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:40:42.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:42.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:42 smithi078 ceph-mon[18440]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:42.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:42 smithi078 ceph-mon[18440]: osd.11 [v2:172.21.15.161:6824/2037560985,v1:172.21.15.161:6825/2037560985] boot 2024-08-18T23:40:42.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:42 smithi078 ceph-mon[18440]: osdmap e76: 12 total, 12 up, 12 in 2024-08-18T23:40:42.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:42 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-18T23:40:42.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.746+0000 7f34ddfdb640 1 -- 172.21.15.78:0/147837420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f34d802d7e0 con 0x7f34d8101a00 2024-08-18T23:40:42.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.746+0000 7f34d4ff9640 1 -- 172.21.15.78:0/147837420 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7f34cc0bbc90 con 0x7f34d8101a00 2024-08-18T23:40:42.747 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:40:42.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.748+0000 7f34ddfdb640 1 -- 172.21.15.78:0/147837420 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f34c00a2b80 msgr2=0x7f34c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:42.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.748+0000 7f34ddfdb640 1 --2- 172.21.15.78:0/147837420 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f34c00a2b80 0x7f34c00a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f34d8101860 tx=0x7f34c800d040 comp rx=0 tx=0).stop 2024-08-18T23:40:42.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.748+0000 7f34ddfdb640 1 -- 172.21.15.78:0/147837420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34d8101a00 msgr2=0x7f34d819cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:42.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.748+0000 7f34ddfdb640 1 --2- 172.21.15.78:0/147837420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34d8101a00 0x7f34d819cb20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009230 tx=0x7f34cc004430 comp rx=0 tx=0).stop 2024-08-18T23:40:42.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.748+0000 7f34ddfdb640 1 -- 172.21.15.78:0/147837420 shutdown_connections 2024-08-18T23:40:42.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.748+0000 7f34ddfdb640 1 --2- 172.21.15.78:0/147837420 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f34c00a2b80 0x7f34c00a5040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:42.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.748+0000 7f34ddfdb640 1 --2- 172.21.15.78:0/147837420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34d81023c0 0x7f34d819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:42.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.748+0000 7f34ddfdb640 1 --2- 172.21.15.78:0/147837420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34d8101a00 0x7f34d819cb20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:42.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.748+0000 7f34ddfdb640 1 --2- 172.21.15.78:0/147837420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34d8100800 0x7f34d806a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:42.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.748+0000 7f34ddfdb640 1 -- 172.21.15.78:0/147837420 >> 172.21.15.78:0/147837420 conn(0x7f34d80fbfb0 msgr2=0x7f34d8104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:40:42.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.748+0000 7f34ddfdb640 1 -- 172.21.15.78:0/147837420 shutdown_connections 2024-08-18T23:40:42.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:42.748+0000 7f34ddfdb640 1 -- 172.21.15.78:0/147837420 wait complete. 2024-08-18T23:40:42.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:42 smithi123 ceph-mon[21808]: purged_snaps scrub starts 2024-08-18T23:40:42.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:42 smithi123 ceph-mon[21808]: purged_snaps scrub ok 2024-08-18T23:40:42.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:42 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-18T23:40:42.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:42 smithi123 ceph-mon[21808]: Detected new or changed devices on smithi161 2024-08-18T23:40:42.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:42 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:42.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:42 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:42.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:42 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:42.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:42 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:42.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:42 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:42.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:42 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:42.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:42 smithi123 ceph-mon[21808]: Adjusting osd_memory_target on smithi161 to 4272M 2024-08-18T23:40:42.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:42 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:42.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:42 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:42.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:42 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:40:42.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:42 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:42.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:42 smithi123 ceph-mon[21808]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:42.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:42 smithi123 ceph-mon[21808]: osd.11 [v2:172.21.15.161:6824/2037560985,v1:172.21.15.161:6825/2037560985] boot 2024-08-18T23:40:42.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:42 smithi123 ceph-mon[21808]: osdmap e76: 12 total, 12 up, 12 in 2024-08-18T23:40:42.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:42 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-18T23:40:42.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:42 smithi161 ceph-mon[22027]: purged_snaps scrub starts 2024-08-18T23:40:42.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:42 smithi161 ceph-mon[22027]: purged_snaps scrub ok 2024-08-18T23:40:42.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:42 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-18T23:40:42.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:42 smithi161 ceph-mon[22027]: Detected new or changed devices on smithi161 2024-08-18T23:40:42.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:42 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:42 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:42 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:42 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:42 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:42 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:40:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:42 smithi161 ceph-mon[22027]: Adjusting osd_memory_target on smithi161 to 4272M 2024-08-18T23:40:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:42 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:42 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:42 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:40:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:42 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:42 smithi161 ceph-mon[22027]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-18T23:40:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:42 smithi161 ceph-mon[22027]: osd.11 [v2:172.21.15.161:6824/2037560985,v1:172.21.15.161:6825/2037560985] boot 2024-08-18T23:40:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:42 smithi161 ceph-mon[22027]: osdmap e76: 12 total, 12 up, 12 in 2024-08-18T23:40:42.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:42 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-18T23:40:43.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:43 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/147837420' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-18T23:40:43.536 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1724024442,"num_in_osds":12,"osd_in_since":1724024422,"num_remapped_pgs":0} 2024-08-18T23:40:43.537 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-18T23:40:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:43 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/147837420' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-18T23:40:43.784 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:40:43.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:43 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/147837420' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-18T23:40:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.275+0000 7f37abfe1640 1 -- 172.21.15.78:0/2604455667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a41007b0 msgr2=0x7f37a4100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.275+0000 7f37abfe1640 1 --2- 172.21.15.78:0/2604455667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a41007b0 0x7f37a4100c10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3794009930 tx=0x7f379402a420 comp rx=0 tx=0).stop 2024-08-18T23:40:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.275+0000 7f37abfe1640 1 -- 172.21.15.78:0/2604455667 shutdown_connections 2024-08-18T23:40:44.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.275+0000 7f37abfe1640 1 --2- 172.21.15.78:0/2604455667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a4101150 0x7f37a410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.275+0000 7f37abfe1640 1 --2- 172.21.15.78:0/2604455667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a41007b0 0x7f37a4100c10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.275+0000 7f37abfe1640 1 --2- 172.21.15.78:0/2604455667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37a41067b0 0x7f37a4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.275+0000 7f37abfe1640 1 -- 172.21.15.78:0/2604455667 >> 172.21.15.78:0/2604455667 conn(0x7f37a40fc4d0 msgr2=0x7f37a40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:40:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.275+0000 7f37abfe1640 1 -- 172.21.15.78:0/2604455667 shutdown_connections 2024-08-18T23:40:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.275+0000 7f37abfe1640 1 -- 172.21.15.78:0/2604455667 wait complete. 2024-08-18T23:40:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.275+0000 7f37abfe1640 1 Processor -- start 2024-08-18T23:40:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37abfe1640 1 -- start start 2024-08-18T23:40:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37abfe1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a41007b0 0x7f37a4199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37abfe1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37a4101150 0x7f37a41995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37abfe1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a41067b0 0x7f37a41a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37abfe1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f37a4110bd0 con 0x7f37a41007b0 2024-08-18T23:40:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37abfe1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37a4110a50 con 0x7f37a41067b0 2024-08-18T23:40:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37abfe1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f37a4110d50 con 0x7f37a4101150 2024-08-18T23:40:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37a9d7d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a41007b0 0x7f37a4199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37a9d7d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a41007b0 0x7f37a4199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54718/0 (socket says 172.21.15.78:54718) 2024-08-18T23:40:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37a9d7d640 1 -- 172.21.15.78:0/1873858229 learned_addr learned my addr 172.21.15.78:0/1873858229 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:40:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37a957c640 1 --2- 172.21.15.78:0/1873858229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37a4101150 0x7f37a41995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37aa57e640 1 --2- 172.21.15.78:0/1873858229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a41067b0 0x7f37a41a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37a9d7d640 1 -- 172.21.15.78:0/1873858229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37a4101150 msgr2=0x7f37a41995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37a9d7d640 1 --2- 172.21.15.78:0/1873858229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37a4101150 0x7f37a41995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37a9d7d640 1 -- 172.21.15.78:0/1873858229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a41067b0 msgr2=0x7f37a41a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37a9d7d640 1 --2- 172.21.15.78:0/1873858229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a41067b0 0x7f37a41a4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37a9d7d640 1 -- 172.21.15.78:0/1873858229 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a41a4640 con 0x7f37a41007b0 2024-08-18T23:40:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37a957c640 1 --2- 172.21.15.78:0/1873858229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37a4101150 0x7f37a41995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:40:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37a9d7d640 1 --2- 172.21.15.78:0/1873858229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a41007b0 0x7f37a4199080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f37a000b4a0 tx=0x7f37a000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:40:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f379affd640 1 -- 172.21.15.78:0/1873858229 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0014070 con 0x7f37a41007b0 2024-08-18T23:40:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f379affd640 1 -- 172.21.15.78:0/1873858229 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a0010af0 con 0x7f37a41007b0 2024-08-18T23:40:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37abfe1640 1 -- 172.21.15.78:0/1873858229 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a41a4810 con 0x7f37a41007b0 2024-08-18T23:40:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f379affd640 1 -- 172.21.15.78:0/1873858229 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0017d10 con 0x7f37a41007b0 2024-08-18T23:40:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.276+0000 7f37abfe1640 1 -- 172.21.15.78:0/1873858229 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a41a4cd0 con 0x7f37a41007b0 2024-08-18T23:40:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.277+0000 7f37abfe1640 1 -- 172.21.15.78:0/1873858229 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37a4100c10 con 0x7f37a41007b0 2024-08-18T23:40:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.278+0000 7f379affd640 1 -- 172.21.15.78:0/1873858229 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f37a0017070 con 0x7f37a41007b0 2024-08-18T23:40:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.279+0000 7f379affd640 1 --2- 172.21.15.78:0/1873858229 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f37900a2b80 0x7f37900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.279+0000 7f37a957c640 1 --2- 172.21.15.78:0/1873858229 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f37900a2b80 0x7f37900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.279+0000 7f379affd640 1 -- 172.21.15.78:0/1873858229 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f37a00ce090 con 0x7f37a41007b0 2024-08-18T23:40:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.280+0000 7f37a957c640 1 --2- 172.21.15.78:0/1873858229 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f37900a2b80 0x7f37900a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3794000c00 tx=0x7f3794005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:40:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.283+0000 7f379affd640 1 -- 172.21.15.78:0/1873858229 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f37a009b490 con 0x7f37a41007b0 2024-08-18T23:40:44.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.451+0000 7f37abfe1640 1 -- 172.21.15.78:0/1873858229 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37a4108d40 con 0x7f37a41007b0 2024-08-18T23:40:44.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.452+0000 7f379affd640 1 -- 172.21.15.78:0/1873858229 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+18454 (secure 0 0 0) 0x7f37a009ad90 con 0x7f37a41007b0 2024-08-18T23:40:44.453 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:40:44.453 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":77,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","created":"2024-08-18T23:34:33.867246+0000","modified":"2024-08-18T23:40:43.466788+0000","last_up_change":"2024-08-18T23:40:42.200381+0000","last_in_change":"2024-08-18T23:40:22.425813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T23:37:22.338211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4de1c78-6ebc-49a9-b9c5-85d43535fab7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6803","nonce":3639645311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6805","nonce":3639645311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6809","nonce":3639645311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6807","nonce":3639645311}]},"public_addr":"172.21.15.78:6803/3639645311","cluster_addr":"172.21.15.78:6805/3639645311","heartbeat_back_addr":"172.21.15.78:6809/3639645311","heartbeat_front_addr":"172.21.15.78:6807/3639645311","state":["exists","up"]},{"osd":1,"uuid":"4f8e0c3d-472b-4230-ad23-2b0a34861251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6801","nonce":4199446368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6803","nonce":4199446368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6807","nonce":4199446368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6805","nonce":4199446368}]},"public_addr":"172.21.15.123:6801/4199446368","cluster_addr":"172.21.15.123:6803/4199446368","heartbeat_back_addr":"172.21.15.123:6807/4199446368","heartbeat_front_addr":"172.21.15.123:6805/4199446368","state":["exists","up"]},{"osd":2,"uuid":"27ef3314-bb6f-413d-be05-e10a105dec9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6801","nonce":2582810057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6803","nonce":2582810057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6807","nonce":2582810057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6805","nonce":2582810057}]},"public_addr":"172.21.15.161:6801/2582810057","cluster_addr":"172.21.15.161:6803/2582810057","heartbeat_back_addr":"172.21.15.161:6807/2582810057","heartbeat_front_addr":"172.21.15.161:6805/2582810057","state":["exists","up"]},{"osd":3,"uuid":"e9e52662-5ef9-4d70-bdd7-2b92165822ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6811","nonce":2697347521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6813","nonce":2697347521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6817","nonce":2697347521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6815","nonce":2697347521}]},"public_addr":"172.21.15.78:6811/2697347521","cluster_addr":"172.21.15.78:6813/2697347521","heartbeat_back_addr":"172.21.15.78:6817/2697347521","heartbeat_front_addr":"172.21.15.78:6815/2697347521","state":["exists","up"]},{"osd":4,"uuid":"5ed26775-79e3-4b24-9557-69fcf5e8e824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6809","nonce":1281511569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6811","nonce":1281511569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6815","nonce":1281511569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6813","nonce":1281511569}]},"public_addr":"172.21.15.123:6809/1281511569","cluster_addr":"172.21.15.123:6811/1281511569","heartbeat_back_addr":"172.21.15.123:6815/1281511569","heartbeat_front_addr":"172.21.15.123:6813/1281511569","state":["exists","up"]},{"osd":5,"uuid":"a9793b1d-5613-48a4-938e-adb718f4fa4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6809","nonce":2751796117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6811","nonce":2751796117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6815","nonce":2751796117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6813","nonce":2751796117}]},"public_addr":"172.21.15.161:6809/2751796117","cluster_addr":"172.21.15.161:6811/2751796117","heartbeat_back_addr":"172.21.15.161:6815/2751796117","heartbeat_front_addr":"172.21.15.161:6813/2751796117","state":["exists","up"]},{"osd":6,"uuid":"5dbc5e9c-a14f-4b71-910a-1e6d4827bb32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6819","nonce":95383223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6821","nonce":95383223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6825","nonce":95383223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6823","nonce":95383223}]},"public_addr":"172.21.15.78:6819/95383223","cluster_addr":"172.21.15.78:6821/95383223","heartbeat_back_addr":"172.21.15.78:6825/95383223","heartbeat_front_addr":"172.21.15.78:6823/95383223","state":["exists","up"]},{"osd":7,"uuid":"26fb7b69-131f-42ee-b745-6fe8b4f65391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6817","nonce":3440305156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6819","nonce":3440305156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6823","nonce":3440305156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6821","nonce":3440305156}]},"public_addr":"172.21.15.123:6817/3440305156","cluster_addr":"172.21.15.123:6819/3440305156","heartbeat_back_addr":"172.21.15.123:6823/3440305156","heartbeat_front_addr":"172.21.15.123:6821/3440305156","state":["exists","up"]},{"osd":8,"uuid":"e267bc50-0c29-4d7b-b368-b1621e29b2df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6817","nonce":1485008778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6819","nonce":1485008778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6823","nonce":1485008778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6821","nonce":1485008778}]},"public_addr":"172.21.15.161:6817/1485008778","cluster_addr":"172.21.15.161:6819/1485008778","heartbeat_back_addr":"172.21.15.161:6823/1485008778","heartbeat_front_addr":"172.21.15.161:6821/1485008778","state":["exists","up"]},{"osd":9,"uuid":"b7654818-b767-4fe6-8211-346297895f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6827","nonce":2884475661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6829","nonce":2884475661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6833","nonce":2884475661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6831","nonce":2884475661}]},"public_addr":"172.21.15.78:6827/2884475661","cluster_addr":"172.21.15.78:6829/2884475661","heartbeat_back_addr":"172.21.15.78:6833/2884475661","heartbeat_front_addr":"172.21.15.78:6831/2884475661","state":["exists","up"]},{"osd":10,"uuid":"59aa32ff-510e-4b52-8170-08a34a0b6b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6825","nonce":122226286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6827","nonce":122226286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6831","nonce":122226286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6829","nonce":122226286}]},"public_addr":"172.21.15.123:6825/122226286","cluster_addr":"172.21.15.123:6827/122226286","heartbeat_back_addr":"172.21.15.123:6831/122226286","heartbeat_front_addr":"172.21.15.123:6829/122226286","state":["exists","up"]},{"osd":11,"uuid":"67bacab4-d078-4c5c-b4ff-9f6a8a00f507","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6825","nonce":2037560985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6827","nonce":2037560985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6831","nonce":2037560985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6829","nonce":2037560985}]},"public_addr":"172.21.15.161:6825/2037560985","cluster_addr":"172.21.15.161:6827/2037560985","heartbeat_back_addr":"172.21.15.161:6831/2037560985","heartbeat_front_addr":"172.21.15.161:6829/2037560985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:36.826044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:57.688421+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:18.297424+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:40.355215+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:02.167902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:23.968759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:46.254083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:08.522038+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:31.028119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:54.119807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:16.847034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:39.607886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6800/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3040499240":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6801/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:6800/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/757409165":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/2562072965":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/636184437":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/1570708294":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3137806628":"2024-08-19T23:34:59.021165+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T23:40:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.454+0000 7f37abfe1640 1 -- 172.21.15.78:0/1873858229 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f37900a2b80 msgr2=0x7f37900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.454+0000 7f37abfe1640 1 --2- 172.21.15.78:0/1873858229 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f37900a2b80 0x7f37900a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3794000c00 tx=0x7f3794005a20 comp rx=0 tx=0).stop 2024-08-18T23:40:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.454+0000 7f37abfe1640 1 -- 172.21.15.78:0/1873858229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a41007b0 msgr2=0x7f37a4199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.454+0000 7f37abfe1640 1 --2- 172.21.15.78:0/1873858229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a41007b0 0x7f37a4199080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f37a000b4a0 tx=0x7f37a000b960 comp rx=0 tx=0).stop 2024-08-18T23:40:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.454+0000 7f37abfe1640 1 -- 172.21.15.78:0/1873858229 shutdown_connections 2024-08-18T23:40:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.454+0000 7f37abfe1640 1 --2- 172.21.15.78:0/1873858229 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f37900a2b80 0x7f37900a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.454+0000 7f37abfe1640 1 --2- 172.21.15.78:0/1873858229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a41067b0 0x7f37a41a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.454+0000 7f37abfe1640 1 --2- 172.21.15.78:0/1873858229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37a4101150 0x7f37a41995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.454+0000 7f37abfe1640 1 --2- 172.21.15.78:0/1873858229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a41007b0 0x7f37a4199080 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:44.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.454+0000 7f37abfe1640 1 -- 172.21.15.78:0/1873858229 >> 172.21.15.78:0/1873858229 conn(0x7f37a40fc4d0 msgr2=0x7f37a40fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:40:44.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.454+0000 7f37abfe1640 1 -- 172.21.15.78:0/1873858229 shutdown_connections 2024-08-18T23:40:44.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:44.454+0000 7f37abfe1640 1 -- 172.21.15.78:0/1873858229 wait complete. 2024-08-18T23:40:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:44 smithi161 ceph-mon[22027]: osdmap e77: 12 total, 12 up, 12 in 2024-08-18T23:40:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:44 smithi161 ceph-mon[22027]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:44 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1873858229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:40:44.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:44 smithi078 ceph-mon[18440]: osdmap e77: 12 total, 12 up, 12 in 2024-08-18T23:40:44.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:44 smithi078 ceph-mon[18440]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:44.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:44 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1873858229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:40:44.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:44 smithi123 ceph-mon[21808]: osdmap e77: 12 total, 12 up, 12 in 2024-08-18T23:40:44.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:44 smithi123 ceph-mon[21808]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:44.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:44 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1873858229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:40:45.187 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-18T23:37:22.338211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-18T23:40:45.187 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-08-18T23:40:45.433 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:40:45.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.927+0000 7fbe38aaf640 1 -- 172.21.15.78:0/2981994879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe341078b0 msgr2=0x7fbe34107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:45.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.927+0000 7fbe38aaf640 1 --2- 172.21.15.78:0/2981994879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe341078b0 0x7fbe34107c90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009930 tx=0x7fbe2402a420 comp rx=0 tx=0).stop 2024-08-18T23:40:45.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.927+0000 7fbe38aaf640 1 -- 172.21.15.78:0/2981994879 shutdown_connections 2024-08-18T23:40:45.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.927+0000 7fbe38aaf640 1 --2- 172.21.15.78:0/2981994879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe34102250 0x7fbe3410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:45.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.927+0000 7fbe38aaf640 1 --2- 172.21.15.78:0/2981994879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe341018b0 0x7fbe34101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:45.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.927+0000 7fbe38aaf640 1 --2- 172.21.15.78:0/2981994879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe341078b0 0x7fbe34107c90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:45.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.927+0000 7fbe38aaf640 1 -- 172.21.15.78:0/2981994879 >> 172.21.15.78:0/2981994879 conn(0x7fbe340fd5f0 msgr2=0x7fbe340ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:40:45.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.927+0000 7fbe38aaf640 1 -- 172.21.15.78:0/2981994879 shutdown_connections 2024-08-18T23:40:45.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.927+0000 7fbe38aaf640 1 -- 172.21.15.78:0/2981994879 wait complete. 2024-08-18T23:40:45.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.928+0000 7fbe38aaf640 1 Processor -- start 2024-08-18T23:40:45.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.928+0000 7fbe38aaf640 1 -- start start 2024-08-18T23:40:45.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.928+0000 7fbe38aaf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe341018b0 0x7fbe34198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:45.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.928+0000 7fbe38aaf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe34102250 0x7fbe341988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:45.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.928+0000 7fbe38aaf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe341078b0 0x7fbe3419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:45.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.928+0000 7fbe38aaf640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbe34111d50 con 0x7fbe34102250 2024-08-18T23:40:45.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.928+0000 7fbe38aaf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe34111bd0 con 0x7fbe341018b0 2024-08-18T23:40:45.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.928+0000 7fbe38aaf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbe34111ed0 con 0x7fbe341078b0 2024-08-18T23:40:45.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.928+0000 7fbe29d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe34102250 0x7fbe341988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:45.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.928+0000 7fbe29d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe34102250 0x7fbe341988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44906/0 (socket says 172.21.15.78:44906) 2024-08-18T23:40:45.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.928+0000 7fbe29d9b640 1 -- 172.21.15.78:0/4287782443 learned_addr learned my addr 172.21.15.78:0/4287782443 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:40:45.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.929+0000 7fbe3259c640 1 --2- 172.21.15.78:0/4287782443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe341018b0 0x7fbe34198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:45.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.929+0000 7fbe32d9d640 1 --2- 172.21.15.78:0/4287782443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe341078b0 0x7fbe3419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:45.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.929+0000 7fbe29d9b640 1 -- 172.21.15.78:0/4287782443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe341078b0 msgr2=0x7fbe3419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:45.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.929+0000 7fbe29d9b640 1 --2- 172.21.15.78:0/4287782443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe341078b0 0x7fbe3419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:45.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.929+0000 7fbe29d9b640 1 -- 172.21.15.78:0/4287782443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe341018b0 msgr2=0x7fbe34198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:45.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.929+0000 7fbe29d9b640 1 --2- 172.21.15.78:0/4287782443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe341018b0 0x7fbe34198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:45.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.929+0000 7fbe29d9b640 1 -- 172.21.15.78:0/4287782443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3419d320 con 0x7fbe34102250 2024-08-18T23:40:45.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.929+0000 7fbe3259c640 1 --2- 172.21.15.78:0/4287782443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe341018b0 0x7fbe34198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:40:45.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.929+0000 7fbe29d9b640 1 --2- 172.21.15.78:0/4287782443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe34102250 0x7fbe341988b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00b4a0 tx=0x7fbe1c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:40:45.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.929+0000 7fbe2bfff640 1 -- 172.21.15.78:0/4287782443 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c014070 con 0x7fbe34102250 2024-08-18T23:40:45.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.929+0000 7fbe2bfff640 1 -- 172.21.15.78:0/4287782443 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe1c010af0 con 0x7fbe34102250 2024-08-18T23:40:45.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.929+0000 7fbe38aaf640 1 -- 172.21.15.78:0/4287782443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3419d610 con 0x7fbe34102250 2024-08-18T23:40:45.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.929+0000 7fbe2bfff640 1 -- 172.21.15.78:0/4287782443 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c017d10 con 0x7fbe34102250 2024-08-18T23:40:45.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.929+0000 7fbe38aaf640 1 -- 172.21.15.78:0/4287782443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe341a4ef0 con 0x7fbe34102250 2024-08-18T23:40:45.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.930+0000 7fbe38aaf640 1 -- 172.21.15.78:0/4287782443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe34101d90 con 0x7fbe34102250 2024-08-18T23:40:45.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.931+0000 7fbe2bfff640 1 -- 172.21.15.78:0/4287782443 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbe1c017070 con 0x7fbe34102250 2024-08-18T23:40:45.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.932+0000 7fbe2bfff640 1 --2- 172.21.15.78:0/4287782443 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbe200a2b80 0x7fbe200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:45.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.932+0000 7fbe3259c640 1 --2- 172.21.15.78:0/4287782443 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbe200a2b80 0x7fbe200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:45.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.932+0000 7fbe2bfff640 1 -- 172.21.15.78:0/4287782443 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fbe1c0ce090 con 0x7fbe34102250 2024-08-18T23:40:45.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.933+0000 7fbe3259c640 1 --2- 172.21.15.78:0/4287782443 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbe200a2b80 0x7fbe200a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbe24000c00 tx=0x7fbe24005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:40:45.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:45.936+0000 7fbe2bfff640 1 -- 172.21.15.78:0/4287782443 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbe1c09b490 con 0x7fbe34102250 2024-08-18T23:40:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:46.102+0000 7fbe38aaf640 1 -- 172.21.15.78:0/4287782443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbe3402d5e0 con 0x7fbe34102250 2024-08-18T23:40:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:46.102+0000 7fbe2bfff640 1 -- 172.21.15.78:0/4287782443 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7fbe1c09ad90 con 0x7fbe34102250 2024-08-18T23:40:46.103 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-08-18T23:40:46.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:46.104+0000 7fbe38aaf640 1 -- 172.21.15.78:0/4287782443 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbe200a2b80 msgr2=0x7fbe200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:46.104+0000 7fbe38aaf640 1 --2- 172.21.15.78:0/4287782443 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbe200a2b80 0x7fbe200a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbe24000c00 tx=0x7fbe24005a20 comp rx=0 tx=0).stop 2024-08-18T23:40:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:46.104+0000 7fbe38aaf640 1 -- 172.21.15.78:0/4287782443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe34102250 msgr2=0x7fbe341988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:46.104+0000 7fbe38aaf640 1 --2- 172.21.15.78:0/4287782443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe34102250 0x7fbe341988b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00b4a0 tx=0x7fbe1c00b960 comp rx=0 tx=0).stop 2024-08-18T23:40:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:46.104+0000 7fbe38aaf640 1 -- 172.21.15.78:0/4287782443 shutdown_connections 2024-08-18T23:40:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:46.104+0000 7fbe38aaf640 1 --2- 172.21.15.78:0/4287782443 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbe200a2b80 0x7fbe200a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:46.104+0000 7fbe38aaf640 1 --2- 172.21.15.78:0/4287782443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe341078b0 0x7fbe3419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:46.104+0000 7fbe38aaf640 1 --2- 172.21.15.78:0/4287782443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe34102250 0x7fbe341988b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:46.104+0000 7fbe38aaf640 1 --2- 172.21.15.78:0/4287782443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe341018b0 0x7fbe34198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:46.104+0000 7fbe38aaf640 1 -- 172.21.15.78:0/4287782443 >> 172.21.15.78:0/4287782443 conn(0x7fbe340fd5f0 msgr2=0x7fbe340fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:40:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:46.105+0000 7fbe38aaf640 1 -- 172.21.15.78:0/4287782443 shutdown_connections 2024-08-18T23:40:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:46.105+0000 7fbe38aaf640 1 -- 172.21.15.78:0/4287782443 wait complete. 2024-08-18T23:40:46.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:46 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/4287782443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-18T23:40:46.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:46 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/4287782443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-18T23:40:46.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:46 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/4287782443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-18T23:40:46.866 INFO:tasks.cephadm:Adding mds.a on smithi078 2024-08-18T23:40:46.866 INFO:tasks.cephadm:Adding mds.d on smithi078 2024-08-18T23:40:46.866 INFO:tasks.cephadm:Adding mds.g on smithi078 2024-08-18T23:40:46.866 INFO:tasks.cephadm:Adding mds.j on smithi078 2024-08-18T23:40:46.867 INFO:tasks.cephadm:Adding mds.b on smithi123 2024-08-18T23:40:46.867 INFO:tasks.cephadm:Adding mds.e on smithi123 2024-08-18T23:40:46.867 INFO:tasks.cephadm:Adding mds.h on smithi123 2024-08-18T23:40:46.867 INFO:tasks.cephadm:Adding mds.k on smithi123 2024-08-18T23:40:46.867 INFO:tasks.cephadm:Adding mds.c on smithi161 2024-08-18T23:40:46.867 INFO:tasks.cephadm:Adding mds.f on smithi161 2024-08-18T23:40:46.867 INFO:tasks.cephadm:Adding mds.i on smithi161 2024-08-18T23:40:46.867 INFO:tasks.cephadm:Adding mds.l on smithi161 2024-08-18T23:40:46.867 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph orch apply mds all '12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l' 2024-08-18T23:40:47.133 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.c/config 2024-08-18T23:40:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:47 smithi161 ceph-mon[22027]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:47.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:47 smithi078 ceph-mon[18440]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:47.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:47 smithi123 ceph-mon[21808]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:47.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.651+0000 7f7387886640 1 -- 172.21.15.161:0/4226676640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73801023d0 msgr2=0x7f73801047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:47.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.651+0000 7f7387886640 1 --2- 172.21.15.161:0/4226676640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73801023d0 0x7f73801047c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f737c009900 tx=0x7f737c02a3f0 comp rx=0 tx=0).stop 2024-08-18T23:40:47.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.651+0000 7f7387886640 1 -- 172.21.15.161:0/4226676640 shutdown_connections 2024-08-18T23:40:47.654 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.651+0000 7f7387886640 1 --2- 172.21.15.161:0/4226676640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7380107630 0x7f7380109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:47.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.651+0000 7f7387886640 1 --2- 172.21.15.161:0/4226676640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7380104d00 0x7f73801070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:47.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.651+0000 7f7387886640 1 --2- 172.21.15.161:0/4226676640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73801023d0 0x7f73801047c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:47.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.651+0000 7f7387886640 1 -- 172.21.15.161:0/4226676640 >> 172.21.15.161:0/4226676640 conn(0x7f73800fbfd0 msgr2=0x7f73800fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:40:47.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.651+0000 7f7387886640 1 -- 172.21.15.161:0/4226676640 shutdown_connections 2024-08-18T23:40:47.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.651+0000 7f7387886640 1 -- 172.21.15.161:0/4226676640 wait complete. 2024-08-18T23:40:47.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7387886640 1 Processor -- start 2024-08-18T23:40:47.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7387886640 1 -- start start 2024-08-18T23:40:47.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7387886640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73801023d0 0x7f7380197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:47.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7387886640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7380104d00 0x7f7380198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:47.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7387886640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7380107630 0x7f738019f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:47.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7387886640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f738010d5e0 con 0x7f7380104d00 2024-08-18T23:40:47.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7387886640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f738010d460 con 0x7f7380107630 2024-08-18T23:40:47.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7387886640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f738010d760 con 0x7f73801023d0 2024-08-18T23:40:47.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7385622640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73801023d0 0x7f7380197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:47.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7385622640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73801023d0 0x7f7380197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:52258/0 (socket says 172.21.15.161:52258) 2024-08-18T23:40:47.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7385622640 1 -- 172.21.15.161:0/2249146891 learned_addr learned my addr 172.21.15.161:0/2249146891 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-18T23:40:47.657 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7384e21640 1 --2- 172.21.15.161:0/2249146891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7380104d00 0x7f7380198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:47.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7385e23640 1 --2- 172.21.15.161:0/2249146891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7380107630 0x7f738019f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:47.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7385622640 1 -- 172.21.15.161:0/2249146891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7380107630 msgr2=0x7f738019f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:47.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7385622640 1 --2- 172.21.15.161:0/2249146891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7380107630 0x7f738019f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:47.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7385622640 1 -- 172.21.15.161:0/2249146891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7380104d00 msgr2=0x7f7380198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:47.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7385622640 1 --2- 172.21.15.161:0/2249146891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7380104d00 0x7f7380198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:47.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7385622640 1 -- 172.21.15.161:0/2249146891 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738019f7f0 con 0x7f73801023d0 2024-08-18T23:40:47.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7385e23640 1 --2- 172.21.15.161:0/2249146891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7380107630 0x7f738019f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:40:47.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.652+0000 7f7384e21640 1 --2- 172.21.15.161:0/2249146891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7380104d00 0x7f7380198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:40:47.658 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.653+0000 7f7385622640 1 --2- 172.21.15.161:0/2249146891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73801023d0 0x7f7380197bc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f737c009230 tx=0x7f737c004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:40:47.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.653+0000 7f73767fc640 1 -- 172.21.15.161:0/2249146891 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c03f030 con 0x7f73801023d0 2024-08-18T23:40:47.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.653+0000 7f73767fc640 1 -- 172.21.15.161:0/2249146891 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f737c003ce0 con 0x7f73801023d0 2024-08-18T23:40:47.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.653+0000 7f7387886640 1 -- 172.21.15.161:0/2249146891 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73801001c0 con 0x7f73801023d0 2024-08-18T23:40:47.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.653+0000 7f73767fc640 1 -- 172.21.15.161:0/2249146891 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c035070 con 0x7f73801023d0 2024-08-18T23:40:47.659 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.653+0000 7f7387886640 1 -- 172.21.15.161:0/2249146891 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73801006a0 con 0x7f73801023d0 2024-08-18T23:40:47.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.654+0000 7f7387886640 1 -- 172.21.15.161:0/2249146891 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73800fdac0 con 0x7f73801023d0 2024-08-18T23:40:47.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.655+0000 7f73767fc640 1 -- 172.21.15.161:0/2249146891 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f737c042030 con 0x7f73801023d0 2024-08-18T23:40:47.660 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.655+0000 7f73767fc640 1 --2- 172.21.15.161:0/2249146891 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f736c0a2bb0 0x7f736c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:47.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.656+0000 7f73767fc640 1 -- 172.21.15.161:0/2249146891 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f737c0ef2d0 con 0x7f73801023d0 2024-08-18T23:40:47.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.656+0000 7f7384e21640 1 --2- 172.21.15.161:0/2249146891 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f736c0a2bb0 0x7f736c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:47.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.656+0000 7f7384e21640 1 --2- 172.21.15.161:0/2249146891 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f736c0a2bb0 0x7f736c0a5070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f73700058c0 tx=0x7f7370005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:40:47.661 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.660+0000 7f73767fc640 1 -- 172.21.15.161:0/2249146891 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f737c0bc6d0 con 0x7f73801023d0 2024-08-18T23:40:47.838 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.836+0000 7f7387886640 1 -- 172.21.15.161:0/2249146891 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}) -- 0x7f7380101cb0 con 0x7f736c0a2bb0 2024-08-18T23:40:47.845 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.844+0000 7f73767fc640 1 -- 172.21.15.161:0/2249146891 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f7380101cb0 con 0x7f736c0a2bb0 2024-08-18T23:40:47.845 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mds.all update... 2024-08-18T23:40:47.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.846+0000 7f7387886640 1 -- 172.21.15.161:0/2249146891 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f736c0a2bb0 msgr2=0x7f736c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:47.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.846+0000 7f7387886640 1 --2- 172.21.15.161:0/2249146891 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f736c0a2bb0 0x7f736c0a5070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f73700058c0 tx=0x7f7370005850 comp rx=0 tx=0).stop 2024-08-18T23:40:47.847 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.846+0000 7f7387886640 1 -- 172.21.15.161:0/2249146891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73801023d0 msgr2=0x7f7380197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:47.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.846+0000 7f7387886640 1 --2- 172.21.15.161:0/2249146891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73801023d0 0x7f7380197bc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f737c009230 tx=0x7f737c004360 comp rx=0 tx=0).stop 2024-08-18T23:40:47.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.846+0000 7f7387886640 1 -- 172.21.15.161:0/2249146891 shutdown_connections 2024-08-18T23:40:47.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.846+0000 7f7387886640 1 --2- 172.21.15.161:0/2249146891 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f736c0a2bb0 0x7f736c0a5070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:47.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.846+0000 7f7387886640 1 --2- 172.21.15.161:0/2249146891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7380107630 0x7f738019f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:47.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.846+0000 7f7387886640 1 --2- 172.21.15.161:0/2249146891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7380104d00 0x7f7380198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:47.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.846+0000 7f7387886640 1 --2- 172.21.15.161:0/2249146891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73801023d0 0x7f7380197bc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:47.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.846+0000 7f7387886640 1 -- 172.21.15.161:0/2249146891 >> 172.21.15.161:0/2249146891 conn(0x7f73800fbfd0 msgr2=0x7f7380102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:40:47.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.846+0000 7f7387886640 1 -- 172.21.15.161:0/2249146891 shutdown_connections 2024-08-18T23:40:47.848 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-18T23:40:47.847+0000 7f7387886640 1 -- 172.21.15.161:0/2249146891 wait complete. 2024-08-18T23:40:48.763 DEBUG:teuthology.orchestra.run.smithi078:mds.a> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.a.service 2024-08-18T23:40:48.766 DEBUG:teuthology.orchestra.run.smithi078:mds.d> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.d.service 2024-08-18T23:40:48.809 DEBUG:teuthology.orchestra.run.smithi078:mds.g> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.g.service 2024-08-18T23:40:48.852 DEBUG:teuthology.orchestra.run.smithi078:mds.j> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.j.service 2024-08-18T23:40:48.895 DEBUG:teuthology.orchestra.run.smithi123:mds.b> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.b.service 2024-08-18T23:40:48.898 DEBUG:teuthology.orchestra.run.smithi123:mds.e> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.e.service 2024-08-18T23:40:48.941 DEBUG:teuthology.orchestra.run.smithi123:mds.h> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.h.service 2024-08-18T23:40:48.983 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:48 smithi161 ceph-mon[22027]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:40:48.983 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:48 smithi161 ceph-mon[22027]: Saving service mds.all spec with placement smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-08-18T23:40:48.983 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:48 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:48.983 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:48 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:40:48.983 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:48 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:48.983 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:48 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:40:48.983 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:48 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:48.984 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:48 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:40:48.984 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:48 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:40:48.984 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:48 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:48.984 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:48 smithi161 ceph-mon[22027]: Deploying daemon mds.l on smithi161 2024-08-18T23:40:48.984 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:48 smithi161 ceph-mon[22027]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:48.984 DEBUG:teuthology.orchestra.run.smithi123:mds.k> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.k.service 2024-08-18T23:40:49.027 DEBUG:teuthology.orchestra.run.smithi161:mds.c> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.c.service 2024-08-18T23:40:49.030 DEBUG:teuthology.orchestra.run.smithi161:mds.f> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.f.service 2024-08-18T23:40:49.073 DEBUG:teuthology.orchestra.run.smithi161:mds.i> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.i.service 2024-08-18T23:40:49.116 DEBUG:teuthology.orchestra.run.smithi161:mds.l> sudo journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.l.service 2024-08-18T23:40:49.159 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-08-18T23:40:49.160 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:40:49.182 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:48 smithi078 ceph-mon[18440]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:40:49.182 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:48 smithi078 ceph-mon[18440]: Saving service mds.all spec with placement smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-08-18T23:40:49.182 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:49.182 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:40:49.182 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:49.183 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:40:49.183 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:49.183 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:40:49.183 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:40:49.183 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:48 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:49.183 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:48 smithi078 ceph-mon[18440]: Deploying daemon mds.l on smithi161 2024-08-18T23:40:49.183 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:48 smithi078 ceph-mon[18440]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:49.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:48 smithi123 ceph-mon[21808]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:40:49.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:48 smithi123 ceph-mon[21808]: Saving service mds.all spec with placement smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-08-18T23:40:49.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:48 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:49.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:48 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:40:49.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:48 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:49.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:48 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:40:49.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:48 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:49.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:48 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:40:49.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:48 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:40:49.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:48 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:49.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:48 smithi123 ceph-mon[21808]: Deploying daemon mds.l on smithi161 2024-08-18T23:40:49.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:48 smithi123 ceph-mon[21808]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:40:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.874+0000 7fc6a1db0640 1 -- 172.21.15.78:0/223680721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc69c102a70 msgr2=0x7fc69c102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.874+0000 7fc6a1db0640 1 --2- 172.21.15.78:0/223680721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc69c102a70 0x7fc69c102e50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009900 tx=0x7fc68c02a480 comp rx=0 tx=0).stop 2024-08-18T23:40:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.874+0000 7fc6a1db0640 1 -- 172.21.15.78:0/223680721 shutdown_connections 2024-08-18T23:40:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.874+0000 7fc6a1db0640 1 --2- 172.21.15.78:0/223680721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc69c10bea0 0x7fc69c10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.874+0000 7fc6a1db0640 1 --2- 172.21.15.78:0/223680721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc69c103390 0x7fc69c10b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.874+0000 7fc6a1db0640 1 --2- 172.21.15.78:0/223680721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc69c102a70 0x7fc69c102e50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:49.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.874+0000 7fc6a1db0640 1 -- 172.21.15.78:0/223680721 >> 172.21.15.78:0/223680721 conn(0x7fc69c0fc4d0 msgr2=0x7fc69c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:40:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.874+0000 7fc6a1db0640 1 -- 172.21.15.78:0/223680721 shutdown_connections 2024-08-18T23:40:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.874+0000 7fc6a1db0640 1 -- 172.21.15.78:0/223680721 wait complete. 2024-08-18T23:40:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.875+0000 7fc6a1db0640 1 Processor -- start 2024-08-18T23:40:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.875+0000 7fc6a1db0640 1 -- start start 2024-08-18T23:40:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.875+0000 7fc6a1db0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc69c102a70 0x7fc69c198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.875+0000 7fc6a1db0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc69c103390 0x7fc69c198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.875+0000 7fc6a1db0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc69c10bea0 0x7fc69c19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.875+0000 7fc6a1db0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c111ce0 con 0x7fc69c102a70 2024-08-18T23:40:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.875+0000 7fc6a1db0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c111b60 con 0x7fc69c103390 2024-08-18T23:40:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.875+0000 7fc6a1db0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c111e60 con 0x7fc69c10bea0 2024-08-18T23:40:49.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.875+0000 7fc69b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc69c102a70 0x7fc69c198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.875+0000 7fc69b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc69c102a70 0x7fc69c198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44920/0 (socket says 172.21.15.78:44920) 2024-08-18T23:40:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.875+0000 7fc69b7fe640 1 -- 172.21.15.78:0/1083227297 learned_addr learned my addr 172.21.15.78:0/1083227297 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:40:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.876+0000 7fc69bfff640 1 --2- 172.21.15.78:0/1083227297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc69c10bea0 0x7fc69c19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.876+0000 7fc69affd640 1 --2- 172.21.15.78:0/1083227297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc69c103390 0x7fc69c198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.876+0000 7fc69b7fe640 1 -- 172.21.15.78:0/1083227297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc69c10bea0 msgr2=0x7fc69c19cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.876+0000 7fc69b7fe640 1 --2- 172.21.15.78:0/1083227297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc69c10bea0 0x7fc69c19cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.876+0000 7fc69b7fe640 1 -- 172.21.15.78:0/1083227297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc69c103390 msgr2=0x7fc69c198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.876+0000 7fc69b7fe640 1 --2- 172.21.15.78:0/1083227297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc69c103390 0x7fc69c198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.876+0000 7fc69b7fe640 1 -- 172.21.15.78:0/1083227297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c19d1f0 con 0x7fc69c102a70 2024-08-18T23:40:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.876+0000 7fc69bfff640 1 --2- 172.21.15.78:0/1083227297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc69c10bea0 0x7fc69c19cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:40:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.876+0000 7fc69affd640 1 --2- 172.21.15.78:0/1083227297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc69c103390 0x7fc69c198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:40:49.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.876+0000 7fc69b7fe640 1 --2- 172.21.15.78:0/1083227297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc69c102a70 0x7fc69c198240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc68c005ab0 tx=0x7fc68c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:40:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.876+0000 7fc698ff9640 1 -- 172.21.15.78:0/1083227297 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c03f030 con 0x7fc69c102a70 2024-08-18T23:40:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.876+0000 7fc6a1db0640 1 -- 172.21.15.78:0/1083227297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69c19d480 con 0x7fc69c102a70 2024-08-18T23:40:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.876+0000 7fc698ff9640 1 -- 172.21.15.78:0/1083227297 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc68c003ce0 con 0x7fc69c102a70 2024-08-18T23:40:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.876+0000 7fc698ff9640 1 -- 172.21.15.78:0/1083227297 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c035070 con 0x7fc69c102a70 2024-08-18T23:40:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.876+0000 7fc6a1db0640 1 -- 172.21.15.78:0/1083227297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69c1a4dc0 con 0x7fc69c102a70 2024-08-18T23:40:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.877+0000 7fc6a1db0640 1 -- 172.21.15.78:0/1083227297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc69c102e50 con 0x7fc69c102a70 2024-08-18T23:40:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.879+0000 7fc698ff9640 1 -- 172.21.15.78:0/1083227297 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc68c042030 con 0x7fc69c102a70 2024-08-18T23:40:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.879+0000 7fc698ff9640 1 --2- 172.21.15.78:0/1083227297 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc6840a2b80 0x7fc6840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.879+0000 7fc69affd640 1 --2- 172.21.15.78:0/1083227297 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc6840a2b80 0x7fc6840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.879+0000 7fc698ff9640 1 -- 172.21.15.78:0/1083227297 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fc68c0ef790 con 0x7fc69c102a70 2024-08-18T23:40:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.880+0000 7fc69affd640 1 --2- 172.21.15.78:0/1083227297 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc6840a2b80 0x7fc6840a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc690006fd0 tx=0x7fc690008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:40:49.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:49.883+0000 7fc698ff9640 1 -- 172.21.15.78:0/1083227297 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc68c0bcb90 con 0x7fc69c102a70 2024-08-18T23:40:50.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:50.080+0000 7fc6a1db0640 1 -- 172.21.15.78:0/1083227297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc69c106f10 con 0x7fc69c102a70 2024-08-18T23:40:50.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:50.081+0000 7fc698ff9640 1 -- 172.21.15.78:0/1083227297 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fc68c0bc490 con 0x7fc69c102a70 2024-08-18T23:40:50.083 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:40:50.083 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"btime":"2024-08-18T23:34:33:867054+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-08-18T23:40:50.083 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 1 2024-08-18T23:40:50.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:50.083+0000 7fc6a1db0640 1 -- 172.21.15.78:0/1083227297 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc6840a2b80 msgr2=0x7fc6840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:50.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:50.083+0000 7fc6a1db0640 1 --2- 172.21.15.78:0/1083227297 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc6840a2b80 0x7fc6840a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc690006fd0 tx=0x7fc690008040 comp rx=0 tx=0).stop 2024-08-18T23:40:50.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:50.083+0000 7fc6a1db0640 1 -- 172.21.15.78:0/1083227297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc69c102a70 msgr2=0x7fc69c198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:50.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:50.083+0000 7fc6a1db0640 1 --2- 172.21.15.78:0/1083227297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc69c102a70 0x7fc69c198240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc68c005ab0 tx=0x7fc68c0042f0 comp rx=0 tx=0).stop 2024-08-18T23:40:50.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:50.083+0000 7fc6a1db0640 1 -- 172.21.15.78:0/1083227297 shutdown_connections 2024-08-18T23:40:50.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:50.083+0000 7fc6a1db0640 1 --2- 172.21.15.78:0/1083227297 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc6840a2b80 0x7fc6840a5040 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:50.083+0000 7fc6a1db0640 1 --2- 172.21.15.78:0/1083227297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc69c10bea0 0x7fc69c19cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:50.083+0000 7fc6a1db0640 1 --2- 172.21.15.78:0/1083227297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc69c103390 0x7fc69c198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:50.083+0000 7fc6a1db0640 1 --2- 172.21.15.78:0/1083227297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc69c102a70 0x7fc69c198240 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:50.083+0000 7fc6a1db0640 1 -- 172.21.15.78:0/1083227297 >> 172.21.15.78:0/1083227297 conn(0x7fc69c0fc4d0 msgr2=0x7fc69c0fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:40:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:50.083+0000 7fc6a1db0640 1 -- 172.21.15.78:0/1083227297 shutdown_connections 2024-08-18T23:40:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:50.083+0000 7fc6a1db0640 1 -- 172.21.15.78:0/1083227297 wait complete. 2024-08-18T23:40:50.379 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:50 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1083227297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:40:50.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:50 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1083227297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:40:50.427 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 18 23:40:50 smithi161 systemd[1]: Starting Ceph mds.l for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:40:50.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:50 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1083227297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:40:50.819 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 18 23:40:50 smithi161 podman[51275]: 2024-08-18 23:40:50.460133893 +0000 UTC m=+0.017284310 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:40:50.819 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 18 23:40:50 smithi161 podman[51275]: 2024-08-18 23:40:50.574596497 +0000 UTC m=+0.131746899 container create ef763974721e39e5c4818b31a4e30fe9a1c739f2a29c8ce6cd0e1f5f97679f99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-l, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-08-18T23:40:50.819 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 18 23:40:50 smithi161 podman[51275]: 2024-08-18 23:40:50.793150137 +0000 UTC m=+0.350300538 container init ef763974721e39e5c4818b31a4e30fe9a1c739f2a29c8ce6cd0e1f5f97679f99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-08-18T23:40:50.819 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 18 23:40:50 smithi161 podman[51275]: 2024-08-18 23:40:50.795892238 +0000 UTC m=+0.353042642 container start ef763974721e39e5c4818b31a4e30fe9a1c739f2a29c8ce6cd0e1f5f97679f99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-l, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-08-18T23:40:51.154 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 18 23:40:50 smithi161 ceph-mds[51294]: -- 172.21.15.161:0/530937548 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560cdcf61860 con 0x560cdcf3e800 2024-08-18T23:40:51.155 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 18 23:40:50 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-l[51290]: starting mds.l at 2024-08-18T23:40:51.155 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 18 23:40:50 smithi161 bash[51275]: ef763974721e39e5c4818b31a4e30fe9a1c739f2a29c8ce6cd0e1f5f97679f99 2024-08-18T23:40:51.155 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 18 23:40:50 smithi161 systemd[1]: Started Ceph mds.l for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:40:51.402 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:51 smithi123 ceph-mon[21808]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:51.403 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:51 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.161:6832/3196205121,v1:172.21.15.161:6833/3196205121] up:boot 2024-08-18T23:40:51.403 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:51 smithi123 ceph-mon[21808]: fsmap 1 up:standby 2024-08-18T23:40:51.403 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:51 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-18T23:40:51.403 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:51 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:51.403 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:51 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:51.403 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:51 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:51.403 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:51 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:40:51.403 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:51 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:40:51.404 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:51 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:51 smithi161 ceph-mon[22027]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:51 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.161:6832/3196205121,v1:172.21.15.161:6833/3196205121] up:boot 2024-08-18T23:40:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:51 smithi161 ceph-mon[22027]: fsmap 1 up:standby 2024-08-18T23:40:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:51 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-18T23:40:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:51 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:51 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:51 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:51 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:40:51.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:51 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:40:51.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:51 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:51.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:51 smithi078 ceph-mon[18440]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:51.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:51 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.161:6832/3196205121,v1:172.21.15.161:6833/3196205121] up:boot 2024-08-18T23:40:51.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:51 smithi078 ceph-mon[18440]: fsmap 1 up:standby 2024-08-18T23:40:51.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:51 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-18T23:40:51.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:51 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:51.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:51 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:51.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:51 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:51.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:51 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:40:51.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:51 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:40:51.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:51 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:52.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:52 smithi161 ceph-mon[22027]: Deploying daemon mds.k on smithi123 2024-08-18T23:40:52.446 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:52 smithi123 ceph-mon[21808]: Deploying daemon mds.k on smithi123 2024-08-18T23:40:52.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:52 smithi078 ceph-mon[18440]: Deploying daemon mds.k on smithi123 2024-08-18T23:40:52.881 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:40:53.140 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:40:53.160 INFO:journalctl@ceph.mds.k.smithi123.stdout:Aug 18 23:40:53 smithi123 systemd[1]: Starting Ceph mds.k for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:40:53.426 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:53 smithi078 ceph-mon[18440]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:53.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:53 smithi161 ceph-mon[22027]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:53.483 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:53 smithi123 ceph-mon[21808]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:53.484 INFO:journalctl@ceph.mds.k.smithi123.stdout:Aug 18 23:40:53 smithi123 podman[50605]: 2024-08-18 23:40:53.123342414 +0000 UTC m=+0.017258927 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:40:53.484 INFO:journalctl@ceph.mds.k.smithi123.stdout:Aug 18 23:40:53 smithi123 podman[50605]: 2024-08-18 23:40:53.247966612 +0000 UTC m=+0.141883119 container create 8268bfa218662cc1e2d596352d8dc198311b172d873bcc070b0df38d67ae7394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-18T23:40:53.484 INFO:journalctl@ceph.mds.k.smithi123.stdout:Aug 18 23:40:53 smithi123 podman[50605]: 2024-08-18 23:40:53.48371904 +0000 UTC m=+0.377635561 container init 8268bfa218662cc1e2d596352d8dc198311b172d873bcc070b0df38d67ae7394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-k, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-18T23:40:53.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.614+0000 7f39e6fc7640 1 -- 172.21.15.78:0/1417126560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39e0102250 msgr2=0x7f39e010e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:53.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.614+0000 7f39e6fc7640 1 --2- 172.21.15.78:0/1417126560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39e0102250 0x7f39e010e780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009cd0 tx=0x7f39d802a8e0 comp rx=0 tx=0).stop 2024-08-18T23:40:53.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.614+0000 7f39e6fc7640 1 -- 172.21.15.78:0/1417126560 shutdown_connections 2024-08-18T23:40:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.614+0000 7f39e6fc7640 1 --2- 172.21.15.78:0/1417126560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39e0102250 0x7f39e010e780 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.614+0000 7f39e6fc7640 1 --2- 172.21.15.78:0/1417126560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39e01018b0 0x7f39e0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.614+0000 7f39e6fc7640 1 --2- 172.21.15.78:0/1417126560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e01078b0 0x7f39e0107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.614+0000 7f39e6fc7640 1 -- 172.21.15.78:0/1417126560 >> 172.21.15.78:0/1417126560 conn(0x7f39e00fd5f0 msgr2=0x7f39e00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:40:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.614+0000 7f39e6fc7640 1 -- 172.21.15.78:0/1417126560 shutdown_connections 2024-08-18T23:40:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.614+0000 7f39e6fc7640 1 -- 172.21.15.78:0/1417126560 wait complete. 2024-08-18T23:40:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.615+0000 7f39e6fc7640 1 Processor -- start 2024-08-18T23:40:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.615+0000 7f39e6fc7640 1 -- start start 2024-08-18T23:40:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.615+0000 7f39e6fc7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39e01018b0 0x7f39e0074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.615+0000 7f39e6fc7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e0102250 0x7f39e0074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.615+0000 7f39e6fc7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39e01078b0 0x7f39e0074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:53.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.615+0000 7f39e6fc7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39e0111cb0 con 0x7f39e01078b0 2024-08-18T23:40:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.615+0000 7f39e6fc7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f39e0111b30 con 0x7f39e0102250 2024-08-18T23:40:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.615+0000 7f39e6fc7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f39e0111e30 con 0x7f39e01018b0 2024-08-18T23:40:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.615+0000 7f39e5564640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39e01078b0 0x7f39e0074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.615+0000 7f39e5564640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39e01078b0 0x7f39e0074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44950/0 (socket says 172.21.15.78:44950) 2024-08-18T23:40:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.615+0000 7f39e5564640 1 -- 172.21.15.78:0/3062971298 learned_addr learned my addr 172.21.15.78:0/3062971298 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:40:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.615+0000 7f39e4d63640 1 --2- 172.21.15.78:0/3062971298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39e01018b0 0x7f39e0074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.615+0000 7f39d7fff640 1 --2- 172.21.15.78:0/3062971298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e0102250 0x7f39e0074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.616+0000 7f39e5564640 1 -- 172.21.15.78:0/3062971298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39e01018b0 msgr2=0x7f39e0074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.616+0000 7f39e5564640 1 --2- 172.21.15.78:0/3062971298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39e01018b0 0x7f39e0074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.616+0000 7f39e5564640 1 -- 172.21.15.78:0/3062971298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e0102250 msgr2=0x7f39e0074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.616+0000 7f39e5564640 1 --2- 172.21.15.78:0/3062971298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e0102250 0x7f39e0074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.616+0000 7f39e5564640 1 -- 172.21.15.78:0/3062971298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e01a65b0 con 0x7f39e01078b0 2024-08-18T23:40:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.616+0000 7f39e5564640 1 --2- 172.21.15.78:0/3062971298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39e01078b0 0x7f39e0074ca0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f39d8006010 tx=0x7f39d8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:40:53.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.616+0000 7f39d5ffb640 1 -- 172.21.15.78:0/3062971298 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8004450 con 0x7f39e01078b0 2024-08-18T23:40:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.616+0000 7f39e6fc7640 1 -- 172.21.15.78:0/3062971298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e01a6840 con 0x7f39e01078b0 2024-08-18T23:40:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.616+0000 7f39d5ffb640 1 -- 172.21.15.78:0/3062971298 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d8003e10 con 0x7f39e01078b0 2024-08-18T23:40:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.616+0000 7f39d5ffb640 1 -- 172.21.15.78:0/3062971298 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8046480 con 0x7f39e01078b0 2024-08-18T23:40:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.616+0000 7f39e6fc7640 1 -- 172.21.15.78:0/3062971298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e01a6d80 con 0x7f39e01078b0 2024-08-18T23:40:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.617+0000 7f39e6fc7640 1 -- 172.21.15.78:0/3062971298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e0101d10 con 0x7f39e01078b0 2024-08-18T23:40:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.618+0000 7f39d5ffb640 1 -- 172.21.15.78:0/3062971298 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f39d80045f0 con 0x7f39e01078b0 2024-08-18T23:40:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.619+0000 7f39d5ffb640 1 --2- 172.21.15.78:0/3062971298 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f39c80a2b80 0x7f39c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.619+0000 7f39e4d63640 1 --2- 172.21.15.78:0/3062971298 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f39c80a2b80 0x7f39c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.619+0000 7f39d5ffb640 1 -- 172.21.15.78:0/3062971298 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f39d802e030 con 0x7f39e01078b0 2024-08-18T23:40:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.620+0000 7f39e4d63640 1 --2- 172.21.15.78:0/3062971298 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f39c80a2b80 0x7f39c80a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00ab60 tx=0x7f39dc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:40:53.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.622+0000 7f39d5ffb640 1 -- 172.21.15.78:0/3062971298 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f39d804b060 con 0x7f39e01078b0 2024-08-18T23:40:53.797 INFO:journalctl@ceph.mds.k.smithi123.stdout:Aug 18 23:40:53 smithi123 podman[50605]: 2024-08-18 23:40:53.48677043 +0000 UTC m=+0.380686940 container start 8268bfa218662cc1e2d596352d8dc198311b172d873bcc070b0df38d67ae7394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-08-18T23:40:53.798 INFO:journalctl@ceph.mds.k.smithi123.stdout:Aug 18 23:40:53 smithi123 ceph-mds[50624]: -- 172.21.15.123:0/2539991719 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a309f7d860 con 0x55a309f5b400 2024-08-18T23:40:53.798 INFO:journalctl@ceph.mds.k.smithi123.stdout:Aug 18 23:40:53 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-k[50620]: starting mds.k at 2024-08-18T23:40:53.798 INFO:journalctl@ceph.mds.k.smithi123.stdout:Aug 18 23:40:53 smithi123 bash[50605]: 8268bfa218662cc1e2d596352d8dc198311b172d873bcc070b0df38d67ae7394 2024-08-18T23:40:53.798 INFO:journalctl@ceph.mds.k.smithi123.stdout:Aug 18 23:40:53 smithi123 systemd[1]: Started Ceph mds.k for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:40:53.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.845+0000 7f39e6fc7640 1 -- 172.21.15.78:0/3062971298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39e01a6fe0 con 0x7f39e01078b0 2024-08-18T23:40:53.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.846+0000 7f39d5ffb640 1 -- 172.21.15.78:0/3062971298 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f39d80bda20 con 0x7f39e01078b0 2024-08-18T23:40:53.848 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:40:53.848 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":2,"btime":"2024-08-18T23:40:50:823748+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-08-18T23:40:53.848 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 2 2024-08-18T23:40:53.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.849+0000 7f39e6fc7640 1 -- 172.21.15.78:0/3062971298 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f39c80a2b80 msgr2=0x7f39c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:53.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.849+0000 7f39e6fc7640 1 --2- 172.21.15.78:0/3062971298 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f39c80a2b80 0x7f39c80a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00ab60 tx=0x7f39dc00a2d0 comp rx=0 tx=0).stop 2024-08-18T23:40:53.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.849+0000 7f39e6fc7640 1 -- 172.21.15.78:0/3062971298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39e01078b0 msgr2=0x7f39e0074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:53.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.849+0000 7f39e6fc7640 1 --2- 172.21.15.78:0/3062971298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39e01078b0 0x7f39e0074ca0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f39d8006010 tx=0x7f39d8003c20 comp rx=0 tx=0).stop 2024-08-18T23:40:53.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.849+0000 7f39e6fc7640 1 -- 172.21.15.78:0/3062971298 shutdown_connections 2024-08-18T23:40:53.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.849+0000 7f39e6fc7640 1 --2- 172.21.15.78:0/3062971298 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f39c80a2b80 0x7f39c80a5040 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:53.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.849+0000 7f39e6fc7640 1 --2- 172.21.15.78:0/3062971298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39e01078b0 0x7f39e0074ca0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:53.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.849+0000 7f39e6fc7640 1 --2- 172.21.15.78:0/3062971298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e0102250 0x7f39e0074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:53.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.849+0000 7f39e6fc7640 1 --2- 172.21.15.78:0/3062971298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39e01018b0 0x7f39e0074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:53.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.849+0000 7f39e6fc7640 1 -- 172.21.15.78:0/3062971298 >> 172.21.15.78:0/3062971298 conn(0x7f39e00fd5f0 msgr2=0x7f39e00fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:40:53.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.849+0000 7f39e6fc7640 1 -- 172.21.15.78:0/3062971298 shutdown_connections 2024-08-18T23:40:53.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:53.849+0000 7f39e6fc7640 1 -- 172.21.15.78:0/3062971298 wait complete. 2024-08-18T23:40:54.852 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:54 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:54.852 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:54 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:54.853 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:54 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:54.853 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:54 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:40:54.853 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:54 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:40:54.853 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:54 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:54.853 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:54 smithi078 ceph-mon[18440]: Deploying daemon mds.j on smithi078 2024-08-18T23:40:54.853 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:54 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3062971298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:40:54.853 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:54 smithi078 ceph-mon[18440]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:54.853 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:54 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.123:6832/3595058071,v1:172.21.15.123:6833/3595058071] up:boot 2024-08-18T23:40:54.853 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:54 smithi078 ceph-mon[18440]: fsmap 2 up:standby 2024-08-18T23:40:54.853 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:54 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-18T23:40:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:54 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:54 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:54 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:54 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:40:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:54 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:40:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:54 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:54.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:54 smithi161 ceph-mon[22027]: Deploying daemon mds.j on smithi078 2024-08-18T23:40:54.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:54 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3062971298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:40:54.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:54 smithi161 ceph-mon[22027]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:54.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:54 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.123:6832/3595058071,v1:172.21.15.123:6833/3595058071] up:boot 2024-08-18T23:40:54.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:54 smithi161 ceph-mon[22027]: fsmap 2 up:standby 2024-08-18T23:40:54.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:54 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-18T23:40:55.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:54 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:55.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:54 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:55.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:54 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:55.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:54 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:40:55.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:54 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:40:55.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:54 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:55.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:54 smithi123 ceph-mon[21808]: Deploying daemon mds.j on smithi078 2024-08-18T23:40:55.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:54 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3062971298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:40:55.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:54 smithi123 ceph-mon[21808]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:55.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:54 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.123:6832/3595058071,v1:172.21.15.123:6833/3595058071] up:boot 2024-08-18T23:40:55.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:54 smithi123 ceph-mon[21808]: fsmap 2 up:standby 2024-08-18T23:40:55.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:54 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-18T23:40:56.452 INFO:journalctl@ceph.mds.j.smithi078.stdout:Aug 18 23:40:56 smithi078 systemd[1]: Starting Ceph mds.j for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:40:56.561 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:40:56.704 INFO:journalctl@ceph.mds.j.smithi078.stdout:Aug 18 23:40:56 smithi078 podman[57470]: 2024-08-18 23:40:56.35318042 +0000 UTC m=+0.017422817 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:40:56.705 INFO:journalctl@ceph.mds.j.smithi078.stdout:Aug 18 23:40:56 smithi078 podman[57470]: 2024-08-18 23:40:56.467950098 +0000 UTC m=+0.132192486 container create 97eba7693b06f7d7220d2bf0694ecb789e64ef45e34637fb2c3ab32c6e94bf07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:40:56.705 INFO:journalctl@ceph.mds.j.smithi078.stdout:Aug 18 23:40:56 smithi078 podman[57470]: 2024-08-18 23:40:56.704112799 +0000 UTC m=+0.368355199 container init 97eba7693b06f7d7220d2bf0694ecb789e64ef45e34637fb2c3ab32c6e94bf07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-j, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-18T23:40:56.908 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:40:56.969 INFO:journalctl@ceph.mds.j.smithi078.stdout:Aug 18 23:40:56 smithi078 podman[57470]: 2024-08-18 23:40:56.708157026 +0000 UTC m=+0.372399417 container start 97eba7693b06f7d7220d2bf0694ecb789e64ef45e34637fb2c3ab32c6e94bf07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-j, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0) 2024-08-18T23:40:56.969 INFO:journalctl@ceph.mds.j.smithi078.stdout:Aug 18 23:40:56 smithi078 ceph-mds[57507]: -- 172.21.15.78:0/2897617264 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55de3180d860 con 0x55de317eb000 2024-08-18T23:40:56.969 INFO:journalctl@ceph.mds.j.smithi078.stdout:Aug 18 23:40:56 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-j[57502]: starting mds.j at 2024-08-18T23:40:56.969 INFO:journalctl@ceph.mds.j.smithi078.stdout:Aug 18 23:40:56 smithi078 bash[57470]: 97eba7693b06f7d7220d2bf0694ecb789e64ef45e34637fb2c3ab32c6e94bf07 2024-08-18T23:40:56.969 INFO:journalctl@ceph.mds.j.smithi078.stdout:Aug 18 23:40:56 smithi078 systemd[1]: Started Ceph mds.j for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:40:57.325 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:57 smithi078 ceph-mon[18440]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:57.325 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:57.325 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:57.325 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:57.325 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:40:57.325 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:40:57.325 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:57 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:57 smithi161 ceph-mon[22027]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:57.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:40:57.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:40:57.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:57 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.407+0000 7f90c8abf640 1 -- 172.21.15.78:0/2308818984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90c4100b50 msgr2=0x7f90c4100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.407+0000 7f90c8abf640 1 --2- 172.21.15.78:0/2308818984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90c4100b50 0x7f90c4100f30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009900 tx=0x7f90ac02a480 comp rx=0 tx=0).stop 2024-08-18T23:40:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.407+0000 7f90c8abf640 1 -- 172.21.15.78:0/2308818984 shutdown_connections 2024-08-18T23:40:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.407+0000 7f90c8abf640 1 --2- 172.21.15.78:0/2308818984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90c4102c20 0x7f90c410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.407+0000 7f90c8abf640 1 --2- 172.21.15.78:0/2308818984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90c41022a0 0x7f90c41026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.407+0000 7f90c8abf640 1 --2- 172.21.15.78:0/2308818984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90c4100b50 0x7f90c4100f30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.407+0000 7f90c8abf640 1 -- 172.21.15.78:0/2308818984 >> 172.21.15.78:0/2308818984 conn(0x7f90c40fc500 msgr2=0x7f90c40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:40:57.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.407+0000 7f90c8abf640 1 -- 172.21.15.78:0/2308818984 shutdown_connections 2024-08-18T23:40:57.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.407+0000 7f90c8abf640 1 -- 172.21.15.78:0/2308818984 wait complete. 2024-08-18T23:40:57.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.408+0000 7f90c8abf640 1 Processor -- start 2024-08-18T23:40:57.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.408+0000 7f90c8abf640 1 -- start start 2024-08-18T23:40:57.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.408+0000 7f90c8abf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90c4100b50 0x7f90c4198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:57.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.408+0000 7f90c8abf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90c41022a0 0x7f90c4198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:57.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.408+0000 7f90c8abf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90c4102c20 0x7f90c419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:57.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.408+0000 7f90c8abf640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f90c4110cb0 con 0x7f90c4100b50 2024-08-18T23:40:57.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.408+0000 7f90c8abf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f90c4110b30 con 0x7f90c4102c20 2024-08-18T23:40:57.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.408+0000 7f90c8abf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f90c4110e30 con 0x7f90c41022a0 2024-08-18T23:40:57.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.408+0000 7f90c259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90c4100b50 0x7f90c4198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:57.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.408+0000 7f90c259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90c4100b50 0x7f90c4198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56296/0 (socket says 172.21.15.78:56296) 2024-08-18T23:40:57.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.408+0000 7f90c259c640 1 -- 172.21.15.78:0/13082497 learned_addr learned my addr 172.21.15.78:0/13082497 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:40:57.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.408+0000 7f90c1d9b640 1 --2- 172.21.15.78:0/13082497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90c41022a0 0x7f90c4198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:57.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.408+0000 7f90c259c640 1 -- 172.21.15.78:0/13082497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90c41022a0 msgr2=0x7f90c4198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:57.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.408+0000 7f90c259c640 1 --2- 172.21.15.78:0/13082497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90c41022a0 0x7f90c4198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:57.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.408+0000 7f90c259c640 1 -- 172.21.15.78:0/13082497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90c4102c20 msgr2=0x7f90c419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:57.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.408+0000 7f90c2d9d640 1 --2- 172.21.15.78:0/13082497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90c4102c20 0x7f90c419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:57.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.408+0000 7f90c259c640 1 --2- 172.21.15.78:0/13082497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90c4102c20 0x7f90c419cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:57.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.408+0000 7f90c259c640 1 -- 172.21.15.78:0/13082497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c419d1e0 con 0x7f90c4100b50 2024-08-18T23:40:57.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.408+0000 7f90c2d9d640 1 --2- 172.21.15.78:0/13082497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90c4102c20 0x7f90c419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:40:57.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.409+0000 7f90c259c640 1 --2- 172.21.15.78:0/13082497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90c4100b50 0x7f90c4198230 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f90ac006010 tx=0x7f90ac003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:40:57.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.409+0000 7f90bb7fe640 1 -- 172.21.15.78:0/13082497 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac03f070 con 0x7f90c4100b50 2024-08-18T23:40:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.409+0000 7f90c8abf640 1 -- 172.21.15.78:0/13082497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c419d470 con 0x7f90c4100b50 2024-08-18T23:40:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.409+0000 7f90bb7fe640 1 -- 172.21.15.78:0/13082497 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90ac031c30 con 0x7f90c4100b50 2024-08-18T23:40:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.409+0000 7f90bb7fe640 1 -- 172.21.15.78:0/13082497 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac035030 con 0x7f90c4100b50 2024-08-18T23:40:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.409+0000 7f90c8abf640 1 -- 172.21.15.78:0/13082497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c41a4db0 con 0x7f90c4100b50 2024-08-18T23:40:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.410+0000 7f90c8abf640 1 -- 172.21.15.78:0/13082497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90c4100f30 con 0x7f90c4100b50 2024-08-18T23:40:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.411+0000 7f90bb7fe640 1 -- 172.21.15.78:0/13082497 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f90ac042070 con 0x7f90c4100b50 2024-08-18T23:40:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.412+0000 7f90bb7fe640 1 --2- 172.21.15.78:0/13082497 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f90b00a2b80 0x7f90b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:40:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.412+0000 7f90c1d9b640 1 --2- 172.21.15.78:0/13082497 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f90b00a2b80 0x7f90b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:40:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.412+0000 7f90bb7fe640 1 -- 172.21.15.78:0/13082497 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f90ac0efa00 con 0x7f90c4100b50 2024-08-18T23:40:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.412+0000 7f90c1d9b640 1 --2- 172.21.15.78:0/13082497 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f90b00a2b80 0x7f90b00a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f90b4005fd0 tx=0x7f90b4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:40:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.415+0000 7f90bb7fe640 1 -- 172.21.15.78:0/13082497 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f90ac031de0 con 0x7f90c4100b50 2024-08-18T23:40:57.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:57 smithi123 ceph-mon[21808]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:57.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:57.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:57.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:40:57.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:40:57.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:40:57.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:57 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:40:57.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.629+0000 7f90c8abf640 1 -- 172.21.15.78:0/13082497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90c41a4f80 con 0x7f90c4100b50 2024-08-18T23:40:57.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.629+0000 7f90bb7fe640 1 -- 172.21.15.78:0/13082497 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7f90ac0bcec0 con 0x7f90c4100b50 2024-08-18T23:40:57.630 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:40:57.631 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":4,"btime":"2024-08-18T23:40:57:152046+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-08-18T23:40:57.631 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 4 2024-08-18T23:40:57.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.631+0000 7f90c8abf640 1 -- 172.21.15.78:0/13082497 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f90b00a2b80 msgr2=0x7f90b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:57.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.631+0000 7f90c8abf640 1 --2- 172.21.15.78:0/13082497 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f90b00a2b80 0x7f90b00a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f90b4005fd0 tx=0x7f90b4005850 comp rx=0 tx=0).stop 2024-08-18T23:40:57.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.631+0000 7f90c8abf640 1 -- 172.21.15.78:0/13082497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90c4100b50 msgr2=0x7f90c4198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:40:57.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.631+0000 7f90c8abf640 1 --2- 172.21.15.78:0/13082497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90c4100b50 0x7f90c4198230 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f90ac006010 tx=0x7f90ac003c10 comp rx=0 tx=0).stop 2024-08-18T23:40:57.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.631+0000 7f90c8abf640 1 -- 172.21.15.78:0/13082497 shutdown_connections 2024-08-18T23:40:57.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.631+0000 7f90c8abf640 1 --2- 172.21.15.78:0/13082497 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f90b00a2b80 0x7f90b00a5040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:57.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.631+0000 7f90c8abf640 1 --2- 172.21.15.78:0/13082497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90c4102c20 0x7f90c419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:57.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.631+0000 7f90c8abf640 1 --2- 172.21.15.78:0/13082497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90c41022a0 0x7f90c4198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:57.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.631+0000 7f90c8abf640 1 --2- 172.21.15.78:0/13082497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90c4100b50 0x7f90c4198230 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:40:57.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.631+0000 7f90c8abf640 1 -- 172.21.15.78:0/13082497 >> 172.21.15.78:0/13082497 conn(0x7f90c40fc500 msgr2=0x7f90c4069f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:40:57.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.631+0000 7f90c8abf640 1 -- 172.21.15.78:0/13082497 shutdown_connections 2024-08-18T23:40:57.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:40:57.631+0000 7f90c8abf640 1 -- 172.21.15.78:0/13082497 wait complete. 2024-08-18T23:40:58.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:58 smithi161 ceph-mon[22027]: Deploying daemon mds.d on smithi078 2024-08-18T23:40:58.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:58 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.78:6834/3927701910,v1:172.21.15.78:6835/3927701910] up:boot 2024-08-18T23:40:58.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:58 smithi161 ceph-mon[22027]: fsmap 3 up:standby 2024-08-18T23:40:58.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:58 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-18T23:40:58.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:58 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/13082497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:40:58.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:58 smithi078 ceph-mon[18440]: Deploying daemon mds.d on smithi078 2024-08-18T23:40:58.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:58 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.78:6834/3927701910,v1:172.21.15.78:6835/3927701910] up:boot 2024-08-18T23:40:58.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:58 smithi078 ceph-mon[18440]: fsmap 3 up:standby 2024-08-18T23:40:58.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:58 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-18T23:40:58.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:58 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/13082497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:40:58.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:58 smithi123 ceph-mon[21808]: Deploying daemon mds.d on smithi078 2024-08-18T23:40:58.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:58 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.78:6834/3927701910,v1:172.21.15.78:6835/3927701910] up:boot 2024-08-18T23:40:58.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:58 smithi123 ceph-mon[21808]: fsmap 3 up:standby 2024-08-18T23:40:58.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:58 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-18T23:40:58.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:58 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/13082497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:40:59.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:40:59 smithi078 ceph-mon[18440]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:40:59 smithi161 ceph-mon[22027]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:59.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:40:59 smithi123 ceph-mon[21808]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:40:59.969 INFO:journalctl@ceph.mds.d.smithi078.stdout:Aug 18 23:40:59 smithi078 systemd[1]: Starting Ceph mds.d for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:41:00.266 INFO:journalctl@ceph.mds.d.smithi078.stdout:Aug 18 23:40:59 smithi078 podman[57885]: 2024-08-18 23:40:59.881436271 +0000 UTC m=+0.017350171 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:41:00.266 INFO:journalctl@ceph.mds.d.smithi078.stdout:Aug 18 23:41:00 smithi078 podman[57885]: 2024-08-18 23:41:00.006668995 +0000 UTC m=+0.142582890 container create d984b200609eb62ae59d4d5446ac5eea776911e3d40191254b89655bd9c2127c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-08-18T23:41:00.403 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:41:00.569 INFO:journalctl@ceph.mds.d.smithi078.stdout:Aug 18 23:41:00 smithi078 podman[57885]: 2024-08-18 23:41:00.265972497 +0000 UTC m=+0.401886392 container init d984b200609eb62ae59d4d5446ac5eea776911e3d40191254b89655bd9c2127c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-d, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-08-18T23:41:00.569 INFO:journalctl@ceph.mds.d.smithi078.stdout:Aug 18 23:41:00 smithi078 podman[57885]: 2024-08-18 23:41:00.269188969 +0000 UTC m=+0.405102863 container start d984b200609eb62ae59d4d5446ac5eea776911e3d40191254b89655bd9c2127c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-08-18T23:41:00.569 INFO:journalctl@ceph.mds.d.smithi078.stdout:Aug 18 23:41:00 smithi078 ceph-mds[57905]: -- 172.21.15.78:0/2535984300 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5570c868b860 con 0x5570c8669400 2024-08-18T23:41:00.569 INFO:journalctl@ceph.mds.d.smithi078.stdout:Aug 18 23:41:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-d[57900]: starting mds.d at 2024-08-18T23:41:00.569 INFO:journalctl@ceph.mds.d.smithi078.stdout:Aug 18 23:41:00 smithi078 bash[57885]: d984b200609eb62ae59d4d5446ac5eea776911e3d40191254b89655bd9c2127c 2024-08-18T23:41:00.569 INFO:journalctl@ceph.mds.d.smithi078.stdout:Aug 18 23:41:00 smithi078 systemd[1]: Started Ceph mds.d for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:41:00.674 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:01.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.301+0000 7f6fd2e91640 1 -- 172.21.15.78:0/779578548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc100b50 msgr2=0x7f6fcc100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:01.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.301+0000 7f6fd2e91640 1 --2- 172.21.15.78:0/779578548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc100b50 0x7f6fcc100f30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009ca0 tx=0x7f6fc402a950 comp rx=0 tx=0).stop 2024-08-18T23:41:01.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.301+0000 7f6fd2e91640 1 -- 172.21.15.78:0/779578548 shutdown_connections 2024-08-18T23:41:01.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.301+0000 7f6fd2e91640 1 --2- 172.21.15.78:0/779578548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fcc102c20 0x7f6fcc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:01.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.301+0000 7f6fd2e91640 1 --2- 172.21.15.78:0/779578548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fcc1022a0 0x7f6fcc1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:01.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.301+0000 7f6fd2e91640 1 --2- 172.21.15.78:0/779578548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc100b50 0x7f6fcc100f30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:01.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.301+0000 7f6fd2e91640 1 -- 172.21.15.78:0/779578548 >> 172.21.15.78:0/779578548 conn(0x7f6fcc0fc500 msgr2=0x7f6fcc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:01.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.301+0000 7f6fd2e91640 1 -- 172.21.15.78:0/779578548 shutdown_connections 2024-08-18T23:41:01.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.301+0000 7f6fd2e91640 1 -- 172.21.15.78:0/779578548 wait complete. 2024-08-18T23:41:01.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.302+0000 7f6fd2e91640 1 Processor -- start 2024-08-18T23:41:01.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.302+0000 7f6fd2e91640 1 -- start start 2024-08-18T23:41:01.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.302+0000 7f6fd2e91640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fcc100b50 0x7f6fcc1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:01.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.302+0000 7f6fd2e91640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc1022a0 0x7f6fcc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:01.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.302+0000 7f6fd2e91640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fcc102c20 0x7f6fcc19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:01.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.302+0000 7f6fd2e91640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc110cb0 con 0x7f6fcc1022a0 2024-08-18T23:41:01.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.302+0000 7f6fd2e91640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc110b30 con 0x7f6fcc102c20 2024-08-18T23:41:01.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.302+0000 7f6fd2e91640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc110e30 con 0x7f6fcc100b50 2024-08-18T23:41:01.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.302+0000 7f6fcbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc1022a0 0x7f6fcc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:01.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.302+0000 7f6fcbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc1022a0 0x7f6fcc1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56320/0 (socket says 172.21.15.78:56320) 2024-08-18T23:41:01.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.302+0000 7f6fcbfff640 1 -- 172.21.15.78:0/3791271679 learned_addr learned my addr 172.21.15.78:0/3791271679 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:01.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.303+0000 7f6fd142e640 1 --2- 172.21.15.78:0/3791271679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fcc102c20 0x7f6fcc19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:01.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.303+0000 7f6fd0c2d640 1 --2- 172.21.15.78:0/3791271679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fcc100b50 0x7f6fcc1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:01.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.303+0000 7f6fcbfff640 1 -- 172.21.15.78:0/3791271679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fcc100b50 msgr2=0x7f6fcc1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:01.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.303+0000 7f6fcbfff640 1 --2- 172.21.15.78:0/3791271679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fcc100b50 0x7f6fcc1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:01.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.303+0000 7f6fcbfff640 1 -- 172.21.15.78:0/3791271679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fcc102c20 msgr2=0x7f6fcc19ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:01.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.303+0000 7f6fcbfff640 1 --2- 172.21.15.78:0/3791271679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fcc102c20 0x7f6fcc19ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:01.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.303+0000 7f6fcbfff640 1 -- 172.21.15.78:0/3791271679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fcc19d150 con 0x7f6fcc1022a0 2024-08-18T23:41:01.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.303+0000 7f6fd0c2d640 1 --2- 172.21.15.78:0/3791271679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fcc100b50 0x7f6fcc1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:41:01.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.303+0000 7f6fcbfff640 1 --2- 172.21.15.78:0/3791271679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc1022a0 0x7f6fcc1986e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00d950 tx=0x7f6fbc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:01.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.303+0000 7f6fc9ffb640 1 -- 172.21.15.78:0/3791271679 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc017070 con 0x7f6fcc1022a0 2024-08-18T23:41:01.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.303+0000 7f6fc9ffb640 1 -- 172.21.15.78:0/3791271679 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fbc009c30 con 0x7f6fcc1022a0 2024-08-18T23:41:01.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.303+0000 7f6fc9ffb640 1 -- 172.21.15.78:0/3791271679 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc00f6b0 con 0x7f6fcc1022a0 2024-08-18T23:41:01.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.303+0000 7f6fd2e91640 1 -- 172.21.15.78:0/3791271679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fcc19d440 con 0x7f6fcc1022a0 2024-08-18T23:41:01.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.303+0000 7f6fd2e91640 1 -- 172.21.15.78:0/3791271679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fcc1a4d20 con 0x7f6fcc1022a0 2024-08-18T23:41:01.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.305+0000 7f6fd2e91640 1 -- 172.21.15.78:0/3791271679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fcc100f30 con 0x7f6fcc1022a0 2024-08-18T23:41:01.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.305+0000 7f6fc9ffb640 1 -- 172.21.15.78:0/3791271679 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6fbc0109d0 con 0x7f6fcc1022a0 2024-08-18T23:41:01.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.306+0000 7f6fc9ffb640 1 --2- 172.21.15.78:0/3791271679 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6fb80a2c00 0x7f6fb80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:01.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.306+0000 7f6fd0c2d640 1 --2- 172.21.15.78:0/3791271679 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6fb80a2c00 0x7f6fb80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:01.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.306+0000 7f6fc9ffb640 1 -- 172.21.15.78:0/3791271679 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f6fbc0ce790 con 0x7f6fcc1022a0 2024-08-18T23:41:01.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.306+0000 7f6fd0c2d640 1 --2- 172.21.15.78:0/3791271679 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6fb80a2c00 0x7f6fb80a50c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4030040 tx=0x7f6fc4005c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:01.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.310+0000 7f6fc9ffb640 1 -- 172.21.15.78:0/3791271679 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6fbc009de0 con 0x7f6fcc1022a0 2024-08-18T23:41:01.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:01 smithi078 ceph-mon[18440]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:01.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:01 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:01.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:01 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:01.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:01 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:01.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:01 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:01.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:01 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:01.372 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:01 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:01.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.532+0000 7f6fd2e91640 1 -- 172.21.15.78:0/3791271679 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fcc1a4ef0 con 0x7f6fcc1022a0 2024-08-18T23:41:01.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.532+0000 7f6fc9ffb640 1 -- 172.21.15.78:0/3791271679 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3916 (secure 0 0 0) 0x7f6fbc09bb90 con 0x7f6fcc1022a0 2024-08-18T23:41:01.534 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:41:01.534 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":5,"btime":"2024-08-18T23:41:01:183629+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-08-18T23:41:01.534 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 5 2024-08-18T23:41:01.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.534+0000 7f6fd2e91640 1 -- 172.21.15.78:0/3791271679 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6fb80a2c00 msgr2=0x7f6fb80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:01.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.534+0000 7f6fd2e91640 1 --2- 172.21.15.78:0/3791271679 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6fb80a2c00 0x7f6fb80a50c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4030040 tx=0x7f6fc4005c30 comp rx=0 tx=0).stop 2024-08-18T23:41:01.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.534+0000 7f6fd2e91640 1 -- 172.21.15.78:0/3791271679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc1022a0 msgr2=0x7f6fcc1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:01.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.534+0000 7f6fd2e91640 1 --2- 172.21.15.78:0/3791271679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc1022a0 0x7f6fcc1986e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00d950 tx=0x7f6fbc00de10 comp rx=0 tx=0).stop 2024-08-18T23:41:01.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.534+0000 7f6fd2e91640 1 -- 172.21.15.78:0/3791271679 shutdown_connections 2024-08-18T23:41:01.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.534+0000 7f6fd2e91640 1 --2- 172.21.15.78:0/3791271679 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6fb80a2c00 0x7f6fb80a50c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:01.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.534+0000 7f6fd2e91640 1 --2- 172.21.15.78:0/3791271679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fcc102c20 0x7f6fcc19ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:01.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.534+0000 7f6fd2e91640 1 --2- 172.21.15.78:0/3791271679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc1022a0 0x7f6fcc1986e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:01.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.534+0000 7f6fd2e91640 1 --2- 172.21.15.78:0/3791271679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fcc100b50 0x7f6fcc1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:01.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.534+0000 7f6fd2e91640 1 -- 172.21.15.78:0/3791271679 >> 172.21.15.78:0/3791271679 conn(0x7f6fcc0fc500 msgr2=0x7f6fcc105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:01.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.535+0000 7f6fd2e91640 1 -- 172.21.15.78:0/3791271679 shutdown_connections 2024-08-18T23:41:01.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:01.535+0000 7f6fd2e91640 1 -- 172.21.15.78:0/3791271679 wait complete. 2024-08-18T23:41:01.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:01 smithi123 ceph-mon[21808]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:01.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:01 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:01.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:01 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:01.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:01 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:01.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:01 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:01.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:01 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:01.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:01 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:01 smithi161 ceph-mon[22027]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:01 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:01 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:01 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:01 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:01.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:01 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:01.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:01 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:02.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:02 smithi078 ceph-mon[18440]: Deploying daemon mds.e on smithi123 2024-08-18T23:41:02.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:02 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.78:6836/1656541536,v1:172.21.15.78:6837/1656541536] up:boot 2024-08-18T23:41:02.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:02 smithi078 ceph-mon[18440]: fsmap 4 up:standby 2024-08-18T23:41:02.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:02 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-18T23:41:02.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:02 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3791271679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:02.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:02 smithi123 ceph-mon[21808]: Deploying daemon mds.e on smithi123 2024-08-18T23:41:02.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:02 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.78:6836/1656541536,v1:172.21.15.78:6837/1656541536] up:boot 2024-08-18T23:41:02.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:02 smithi123 ceph-mon[21808]: fsmap 4 up:standby 2024-08-18T23:41:02.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:02 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-18T23:41:02.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:02 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3791271679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:02.548 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:41:02 smithi123 systemd[1]: Starting Ceph mds.e for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:41:02.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:02 smithi161 ceph-mon[22027]: Deploying daemon mds.e on smithi123 2024-08-18T23:41:02.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:02 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.78:6836/1656541536,v1:172.21.15.78:6837/1656541536] up:boot 2024-08-18T23:41:02.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:02 smithi161 ceph-mon[22027]: fsmap 4 up:standby 2024-08-18T23:41:02.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:02 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-18T23:41:02.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:02 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3791271679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:02.939 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:41:02 smithi123 podman[50906]: 2024-08-18 23:41:02.549246193 +0000 UTC m=+0.017213803 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:41:02.939 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:41:02 smithi123 podman[50906]: 2024-08-18 23:41:02.686425741 +0000 UTC m=+0.154393341 container create a311a99fbf6242e4a3cbc08f96fec3f7566d1c8c964c0d3d3b4bd206459e52ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, ceph=True, org.label-schema.build-date=20240731) 2024-08-18T23:41:03.204 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:41:02 smithi123 podman[50906]: 2024-08-18 23:41:02.939368884 +0000 UTC m=+0.407336484 container init a311a99fbf6242e4a3cbc08f96fec3f7566d1c8c964c0d3d3b4bd206459e52ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:41:03.205 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:41:02 smithi123 podman[50906]: 2024-08-18 23:41:02.942426057 +0000 UTC m=+0.410393662 container start a311a99fbf6242e4a3cbc08f96fec3f7566d1c8c964c0d3d3b4bd206459e52ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-08-18T23:41:03.205 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:41:02 smithi123 ceph-mds[50926]: -- 172.21.15.123:0/3068329642 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559b6a2f5860 con 0x559b6a2d3000 2024-08-18T23:41:03.205 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:41:02 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-e[50922]: starting mds.e at 2024-08-18T23:41:03.205 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:41:03 smithi123 bash[50906]: a311a99fbf6242e4a3cbc08f96fec3f7566d1c8c964c0d3d3b4bd206459e52ec 2024-08-18T23:41:03.205 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:41:03 smithi123 systemd[1]: Started Ceph mds.e for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:41:03.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:03 smithi161 ceph-mon[22027]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:03.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:03 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:03.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:03 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:03.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:03 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:03.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:03 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:03.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:03 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:03.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:03 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:03 smithi078 ceph-mon[18440]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:03 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:03 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:03 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:03 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:03.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:03 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:03.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:03 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:03.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:03 smithi123 ceph-mon[21808]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:03.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:03 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:03.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:03 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:03.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:03 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:03.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:03 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:03.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:03 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:03.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:03 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:04.261 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:41:04.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:04 smithi161 ceph-mon[22027]: Deploying daemon mds.f on smithi161 2024-08-18T23:41:04.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:04 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.123:6834/2459199316,v1:172.21.15.123:6835/2459199316] up:boot 2024-08-18T23:41:04.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:04 smithi161 ceph-mon[22027]: fsmap 5 up:standby 2024-08-18T23:41:04.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-18T23:41:04.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:04 smithi078 ceph-mon[18440]: Deploying daemon mds.f on smithi161 2024-08-18T23:41:04.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:04 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.123:6834/2459199316,v1:172.21.15.123:6835/2459199316] up:boot 2024-08-18T23:41:04.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:04 smithi078 ceph-mon[18440]: fsmap 5 up:standby 2024-08-18T23:41:04.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-18T23:41:04.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:04 smithi123 ceph-mon[21808]: Deploying daemon mds.f on smithi161 2024-08-18T23:41:04.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:04 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.123:6834/2459199316,v1:172.21.15.123:6835/2459199316] up:boot 2024-08-18T23:41:04.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:04 smithi123 ceph-mon[21808]: fsmap 5 up:standby 2024-08-18T23:41:04.472 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-18T23:41:04.534 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:05.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.030+0000 7ff048f9d640 1 -- 172.21.15.78:0/193061455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0441018b0 msgr2=0x7ff044101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:05.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.030+0000 7ff048f9d640 1 --2- 172.21.15.78:0/193061455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0441018b0 0x7ff044101d10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff038009930 tx=0x7ff03802a420 comp rx=0 tx=0).stop 2024-08-18T23:41:05.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.030+0000 7ff048f9d640 1 -- 172.21.15.78:0/193061455 shutdown_connections 2024-08-18T23:41:05.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.030+0000 7ff048f9d640 1 --2- 172.21.15.78:0/193061455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff044102250 0x7ff04410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:05.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.030+0000 7ff048f9d640 1 --2- 172.21.15.78:0/193061455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0441018b0 0x7ff044101d10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:05.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.030+0000 7ff048f9d640 1 --2- 172.21.15.78:0/193061455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0441078b0 0x7ff044107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:05.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.030+0000 7ff048f9d640 1 -- 172.21.15.78:0/193061455 >> 172.21.15.78:0/193061455 conn(0x7ff0440fd5f0 msgr2=0x7ff0440ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:05.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.030+0000 7ff048f9d640 1 -- 172.21.15.78:0/193061455 shutdown_connections 2024-08-18T23:41:05.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.030+0000 7ff048f9d640 1 -- 172.21.15.78:0/193061455 wait complete. 2024-08-18T23:41:05.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.031+0000 7ff048f9d640 1 Processor -- start 2024-08-18T23:41:05.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.031+0000 7ff048f9d640 1 -- start start 2024-08-18T23:41:05.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.031+0000 7ff048f9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0441018b0 0x7ff044198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:05.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.031+0000 7ff048f9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff044102250 0x7ff044198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:05.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.031+0000 7ff048f9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0441078b0 0x7ff04419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:05.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.031+0000 7ff048f9d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff044111cd0 con 0x7ff0441078b0 2024-08-18T23:41:05.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.031+0000 7ff048f9d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff044111b50 con 0x7ff044102250 2024-08-18T23:41:05.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.031+0000 7ff048f9d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff044111e50 con 0x7ff0441018b0 2024-08-18T23:41:05.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.032+0000 7ff041d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff044102250 0x7ff044198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:05.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.032+0000 7ff042d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0441078b0 0x7ff04419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.032+0000 7ff041d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff044102250 0x7ff044198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:51294/0 (socket says 172.21.15.78:51294) 2024-08-18T23:41:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.032+0000 7ff041d9b640 1 -- 172.21.15.78:0/2476288279 learned_addr learned my addr 172.21.15.78:0/2476288279 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.032+0000 7ff04259c640 1 --2- 172.21.15.78:0/2476288279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0441018b0 0x7ff044198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.032+0000 7ff042d9d640 1 -- 172.21.15.78:0/2476288279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0441018b0 msgr2=0x7ff044198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.032+0000 7ff042d9d640 1 --2- 172.21.15.78:0/2476288279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0441018b0 0x7ff044198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.032+0000 7ff042d9d640 1 -- 172.21.15.78:0/2476288279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff044102250 msgr2=0x7ff044198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.032+0000 7ff042d9d640 1 --2- 172.21.15.78:0/2476288279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff044102250 0x7ff044198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.032+0000 7ff042d9d640 1 -- 172.21.15.78:0/2476288279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04419d1c0 con 0x7ff0441078b0 2024-08-18T23:41:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.032+0000 7ff041d9b640 1 --2- 172.21.15.78:0/2476288279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff044102250 0x7ff044198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:41:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.032+0000 7ff042d9d640 1 --2- 172.21.15.78:0/2476288279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0441078b0 0x7ff04419cae0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00be20 tx=0x7ff03c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.032+0000 7ff0337fe640 1 -- 172.21.15.78:0/2476288279 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c019070 con 0x7ff0441078b0 2024-08-18T23:41:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.032+0000 7ff048f9d640 1 -- 172.21.15.78:0/2476288279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04419d4b0 con 0x7ff0441078b0 2024-08-18T23:41:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.032+0000 7ff0337fe640 1 -- 172.21.15.78:0/2476288279 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff03c014920 con 0x7ff0441078b0 2024-08-18T23:41:05.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.032+0000 7ff0337fe640 1 -- 172.21.15.78:0/2476288279 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c012920 con 0x7ff0441078b0 2024-08-18T23:41:05.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.032+0000 7ff048f9d640 1 -- 172.21.15.78:0/2476288279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0441a4d90 con 0x7ff0441078b0 2024-08-18T23:41:05.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.033+0000 7ff048f9d640 1 -- 172.21.15.78:0/2476288279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff044101d10 con 0x7ff0441078b0 2024-08-18T23:41:05.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.034+0000 7ff0337fe640 1 -- 172.21.15.78:0/2476288279 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff03c01c070 con 0x7ff0441078b0 2024-08-18T23:41:05.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.035+0000 7ff0337fe640 1 --2- 172.21.15.78:0/2476288279 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff02c0a2b80 0x7ff02c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:05.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.035+0000 7ff04259c640 1 --2- 172.21.15.78:0/2476288279 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff02c0a2b80 0x7ff02c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:05.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.035+0000 7ff0337fe640 1 -- 172.21.15.78:0/2476288279 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7ff03c0ce400 con 0x7ff0441078b0 2024-08-18T23:41:05.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.036+0000 7ff04259c640 1 --2- 172.21.15.78:0/2476288279 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff02c0a2b80 0x7ff02c0a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff034009bd0 tx=0x7ff034005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:05.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.040+0000 7ff0337fe640 1 -- 172.21.15.78:0/2476288279 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff03c09b1b0 con 0x7ff0441078b0 2024-08-18T23:41:05.177 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 18 23:41:04 smithi161 systemd[1]: Starting Ceph mds.f for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:41:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.241+0000 7ff048f9d640 1 -- 172.21.15.78:0/2476288279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff044109e70 con 0x7ff0441078b0 2024-08-18T23:41:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.242+0000 7ff0337fe640 1 -- 172.21.15.78:0/2476288279 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4744 (secure 0 0 0) 0x7ff03c09f4b0 con 0x7ff0441078b0 2024-08-18T23:41:05.243 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:41:05.243 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":6,"btime":"2024-08-18T23:41:03:194901+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-08-18T23:41:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 6 2024-08-18T23:41:05.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.244+0000 7ff048f9d640 1 -- 172.21.15.78:0/2476288279 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff02c0a2b80 msgr2=0x7ff02c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:05.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.244+0000 7ff048f9d640 1 --2- 172.21.15.78:0/2476288279 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff02c0a2b80 0x7ff02c0a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff034009bd0 tx=0x7ff034005f90 comp rx=0 tx=0).stop 2024-08-18T23:41:05.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.244+0000 7ff048f9d640 1 -- 172.21.15.78:0/2476288279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0441078b0 msgr2=0x7ff04419cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:05.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.244+0000 7ff048f9d640 1 --2- 172.21.15.78:0/2476288279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0441078b0 0x7ff04419cae0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00be20 tx=0x7ff03c00bf10 comp rx=0 tx=0).stop 2024-08-18T23:41:05.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.244+0000 7ff048f9d640 1 -- 172.21.15.78:0/2476288279 shutdown_connections 2024-08-18T23:41:05.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.244+0000 7ff048f9d640 1 --2- 172.21.15.78:0/2476288279 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff02c0a2b80 0x7ff02c0a5040 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:05.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.244+0000 7ff048f9d640 1 --2- 172.21.15.78:0/2476288279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0441078b0 0x7ff04419cae0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:05.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.244+0000 7ff048f9d640 1 --2- 172.21.15.78:0/2476288279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff044102250 0x7ff044198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:05.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.244+0000 7ff048f9d640 1 --2- 172.21.15.78:0/2476288279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0441018b0 0x7ff044198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:05.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.244+0000 7ff048f9d640 1 -- 172.21.15.78:0/2476288279 >> 172.21.15.78:0/2476288279 conn(0x7ff0440fd5f0 msgr2=0x7ff0440fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:05.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.244+0000 7ff048f9d640 1 -- 172.21.15.78:0/2476288279 shutdown_connections 2024-08-18T23:41:05.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:05.244+0000 7ff048f9d640 1 -- 172.21.15.78:0/2476288279 wait complete. 2024-08-18T23:41:05.465 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:05 smithi161 ceph-mon[22027]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:05.466 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 18 23:41:05 smithi161 podman[51575]: 2024-08-18 23:41:05.095387945 +0000 UTC m=+0.017223308 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:41:05.466 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 18 23:41:05 smithi161 podman[51575]: 2024-08-18 23:41:05.194735695 +0000 UTC m=+0.116571107 container create 4c0696181a07b24049ca8db2905e2795e76823fd450b44c6b259d8905ff2a017 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-08-18T23:41:05.466 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 18 23:41:05 smithi161 podman[51575]: 2024-08-18 23:41:05.4394112 +0000 UTC m=+0.361246573 container init 4c0696181a07b24049ca8db2905e2795e76823fd450b44c6b259d8905ff2a017 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-08-18T23:41:05.466 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 18 23:41:05 smithi161 podman[51575]: 2024-08-18 23:41:05.442204178 +0000 UTC m=+0.364039542 container start 4c0696181a07b24049ca8db2905e2795e76823fd450b44c6b259d8905ff2a017 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-08-18T23:41:05.466 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 18 23:41:05 smithi161 ceph-mds[51594]: -- 172.21.15.161:0/1028074206 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5576c39a1860 con 0x5576c397e800 2024-08-18T23:41:05.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:05 smithi078 ceph-mon[18440]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:05.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:05 smithi123 ceph-mon[21808]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:05.927 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 18 23:41:05 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-f[51590]: starting mds.f at 2024-08-18T23:41:05.927 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 18 23:41:05 smithi161 bash[51575]: 4c0696181a07b24049ca8db2905e2795e76823fd450b44c6b259d8905ff2a017 2024-08-18T23:41:05.927 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 18 23:41:05 smithi161 systemd[1]: Started Ceph mds.f for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:41:06.275 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:06 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2476288279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:06.275 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:06 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:06.275 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:06 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:06.275 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:06 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:06.275 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:06 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:06.275 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:06 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:06.275 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:06 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:06.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:06 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2476288279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:06.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:06 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:06.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:06 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:06.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:06 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:06.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:06 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:06.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:06 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:06.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:06 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:06.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:06 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2476288279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:06.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:06 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:06.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:06 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:06.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:06 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:06.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:06 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:06.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:06 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:06.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:06 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:07.267 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:07 smithi161 ceph-mon[22027]: Deploying daemon mds.c on smithi161 2024-08-18T23:41:07.268 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:07 smithi161 ceph-mon[22027]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:07.268 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:07 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.161:6834/4100464644,v1:172.21.15.161:6835/4100464644] up:boot 2024-08-18T23:41:07.268 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:07 smithi161 ceph-mon[22027]: fsmap 6 up:standby 2024-08-18T23:41:07.268 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:07 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-18T23:41:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:07 smithi078 ceph-mon[18440]: Deploying daemon mds.c on smithi161 2024-08-18T23:41:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:07 smithi078 ceph-mon[18440]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:07 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.161:6834/4100464644,v1:172.21.15.161:6835/4100464644] up:boot 2024-08-18T23:41:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:07 smithi078 ceph-mon[18440]: fsmap 6 up:standby 2024-08-18T23:41:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:07 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-18T23:41:07.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:07 smithi123 ceph-mon[21808]: Deploying daemon mds.c on smithi161 2024-08-18T23:41:07.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:07 smithi123 ceph-mon[21808]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:07.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:07 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.161:6834/4100464644,v1:172.21.15.161:6835/4100464644] up:boot 2024-08-18T23:41:07.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:07 smithi123 ceph-mon[21808]: fsmap 6 up:standby 2024-08-18T23:41:07.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:07 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-18T23:41:07.673 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:41:07 smithi161 systemd[1]: Starting Ceph mds.c for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:41:07.927 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:41:07 smithi161 podman[51876]: 2024-08-18 23:41:07.575788093 +0000 UTC m=+0.017339013 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:41:07.927 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:41:07 smithi161 podman[51876]: 2024-08-18 23:41:07.674284473 +0000 UTC m=+0.115835396 container create 89609bf43c692ca6b987939e63ad4b104103909f56c1af28a134adbe5f17f800 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-c, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, ceph=True) 2024-08-18T23:41:07.959 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:41:08.243 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:08.427 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:41:07 smithi161 podman[51876]: 2024-08-18 23:41:07.93535084 +0000 UTC m=+0.376901770 container init 89609bf43c692ca6b987939e63ad4b104103909f56c1af28a134adbe5f17f800 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-08-18T23:41:08.427 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:41:07 smithi161 podman[51876]: 2024-08-18 23:41:07.938208718 +0000 UTC m=+0.379759638 container start 89609bf43c692ca6b987939e63ad4b104103909f56c1af28a134adbe5f17f800 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0) 2024-08-18T23:41:08.427 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:41:07 smithi161 ceph-mds[51895]: -- 172.21.15.161:0/3290255552 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b60c41b860 con 0x55b60c3f9000 2024-08-18T23:41:08.428 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:41:07 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-c[51891]: starting mds.c at 2024-08-18T23:41:08.428 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:41:08 smithi161 bash[51876]: 89609bf43c692ca6b987939e63ad4b104103909f56c1af28a134adbe5f17f800 2024-08-18T23:41:08.428 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:41:08 smithi161 systemd[1]: Started Ceph mds.c for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:41:08.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.730+0000 7f1d24f4a640 1 -- 172.21.15.78:0/1157944213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d201018b0 msgr2=0x7f1d20101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:08.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.730+0000 7f1d24f4a640 1 --2- 172.21.15.78:0/1157944213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d201018b0 0x7f1d20101d10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009930 tx=0x7f1d0802a420 comp rx=0 tx=0).stop 2024-08-18T23:41:08.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.730+0000 7f1d24f4a640 1 -- 172.21.15.78:0/1157944213 shutdown_connections 2024-08-18T23:41:08.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.730+0000 7f1d24f4a640 1 --2- 172.21.15.78:0/1157944213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d20102250 0x7f1d2010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.730+0000 7f1d24f4a640 1 --2- 172.21.15.78:0/1157944213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d201018b0 0x7f1d20101d10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.730+0000 7f1d24f4a640 1 --2- 172.21.15.78:0/1157944213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d201078b0 0x7f1d20107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.730+0000 7f1d24f4a640 1 -- 172.21.15.78:0/1157944213 >> 172.21.15.78:0/1157944213 conn(0x7f1d200fd5f0 msgr2=0x7f1d200ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.730+0000 7f1d24f4a640 1 -- 172.21.15.78:0/1157944213 shutdown_connections 2024-08-18T23:41:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.730+0000 7f1d24f4a640 1 -- 172.21.15.78:0/1157944213 wait complete. 2024-08-18T23:41:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d24f4a640 1 Processor -- start 2024-08-18T23:41:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d24f4a640 1 -- start start 2024-08-18T23:41:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d24f4a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d201018b0 0x7f1d20074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d24f4a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d20102250 0x7f1d20074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d24f4a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d201078b0 0x7f1d20074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d24f4a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1d20111cb0 con 0x7f1d201018b0 2024-08-18T23:41:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d24f4a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d20111b30 con 0x7f1d201078b0 2024-08-18T23:41:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d24f4a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1d20111e30 con 0x7f1d20102250 2024-08-18T23:41:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d1e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d201018b0 0x7f1d20074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d1e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d201018b0 0x7f1d20074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59464/0 (socket says 172.21.15.78:59464) 2024-08-18T23:41:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d1e59c640 1 -- 172.21.15.78:0/1222486587 learned_addr learned my addr 172.21.15.78:0/1222486587 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d1dd9b640 1 --2- 172.21.15.78:0/1222486587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d20102250 0x7f1d20074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d1e59c640 1 -- 172.21.15.78:0/1222486587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d20102250 msgr2=0x7f1d20074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d1e59c640 1 --2- 172.21.15.78:0/1222486587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d20102250 0x7f1d20074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d1e59c640 1 -- 172.21.15.78:0/1222486587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d201078b0 msgr2=0x7f1d20074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d1ed9d640 1 --2- 172.21.15.78:0/1222486587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d201078b0 0x7f1d20074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d1e59c640 1 --2- 172.21.15.78:0/1222486587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d201078b0 0x7f1d20074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d1e59c640 1 -- 172.21.15.78:0/1222486587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d201a65d0 con 0x7f1d201018b0 2024-08-18T23:41:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d1dd9b640 1 --2- 172.21.15.78:0/1222486587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d20102250 0x7f1d20074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:41:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.731+0000 7f1d1ed9d640 1 --2- 172.21.15.78:0/1222486587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d201078b0 0x7f1d20074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:41:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.732+0000 7f1d1e59c640 1 --2- 172.21.15.78:0/1222486587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d201018b0 0x7f1d20074220 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800cce0 tx=0x7f1d1800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.732+0000 7f1d177fe640 1 -- 172.21.15.78:0/1222486587 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1800f650 con 0x7f1d201018b0 2024-08-18T23:41:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.732+0000 7f1d24f4a640 1 -- 172.21.15.78:0/1222486587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d201a68c0 con 0x7f1d201018b0 2024-08-18T23:41:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.732+0000 7f1d177fe640 1 -- 172.21.15.78:0/1222486587 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d1800fcd0 con 0x7f1d201018b0 2024-08-18T23:41:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.732+0000 7f1d177fe640 1 -- 172.21.15.78:0/1222486587 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18018c70 con 0x7f1d201018b0 2024-08-18T23:41:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.732+0000 7f1d24f4a640 1 -- 172.21.15.78:0/1222486587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d201a6dd0 con 0x7f1d201018b0 2024-08-18T23:41:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.733+0000 7f1d24f4a640 1 -- 172.21.15.78:0/1222486587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d20101d10 con 0x7f1d201018b0 2024-08-18T23:41:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.734+0000 7f1d177fe640 1 -- 172.21.15.78:0/1222486587 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1d1800f7f0 con 0x7f1d201018b0 2024-08-18T23:41:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.734+0000 7f1d177fe640 1 --2- 172.21.15.78:0/1222486587 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1d0c0a2b80 0x7f1d0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.735+0000 7f1d1dd9b640 1 --2- 172.21.15.78:0/1222486587 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1d0c0a2b80 0x7f1d0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.735+0000 7f1d177fe640 1 -- 172.21.15.78:0/1222486587 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f1d180cf090 con 0x7f1d201018b0 2024-08-18T23:41:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.735+0000 7f1d1dd9b640 1 --2- 172.21.15.78:0/1222486587 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1d0c0a2b80 0x7f1d0c0a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1d08000c00 tx=0x7f1d08005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.738+0000 7f1d177fe640 1 -- 172.21.15.78:0/1222486587 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1d18014030 con 0x7f1d201018b0 2024-08-18T23:41:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.938+0000 7f1d24f4a640 1 -- 172.21.15.78:0/1222486587 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d201a7090 con 0x7f1d201018b0 2024-08-18T23:41:08.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.938+0000 7f1d177fe640 1 -- 172.21.15.78:0/1222486587 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6400 (secure 0 0 0) 0x7f1d1809c490 con 0x7f1d201018b0 2024-08-18T23:41:08.940 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:41:08.940 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"btime":"2024-08-18T23:41:08:215502+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-08-18T23:41:08.940 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2024-08-18T23:41:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.940+0000 7f1d24f4a640 1 -- 172.21.15.78:0/1222486587 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1d0c0a2b80 msgr2=0x7f1d0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:08.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.940+0000 7f1d24f4a640 1 --2- 172.21.15.78:0/1222486587 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1d0c0a2b80 0x7f1d0c0a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1d08000c00 tx=0x7f1d08005a20 comp rx=0 tx=0).stop 2024-08-18T23:41:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.940+0000 7f1d24f4a640 1 -- 172.21.15.78:0/1222486587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d201018b0 msgr2=0x7f1d20074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.940+0000 7f1d24f4a640 1 --2- 172.21.15.78:0/1222486587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d201018b0 0x7f1d20074220 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800cce0 tx=0x7f1d1800a620 comp rx=0 tx=0).stop 2024-08-18T23:41:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.940+0000 7f1d24f4a640 1 -- 172.21.15.78:0/1222486587 shutdown_connections 2024-08-18T23:41:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.940+0000 7f1d24f4a640 1 --2- 172.21.15.78:0/1222486587 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1d0c0a2b80 0x7f1d0c0a5040 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.940+0000 7f1d24f4a640 1 --2- 172.21.15.78:0/1222486587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d201078b0 0x7f1d20074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.940+0000 7f1d24f4a640 1 --2- 172.21.15.78:0/1222486587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d20102250 0x7f1d20074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.940+0000 7f1d24f4a640 1 --2- 172.21.15.78:0/1222486587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d201018b0 0x7f1d20074220 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.940+0000 7f1d24f4a640 1 -- 172.21.15.78:0/1222486587 >> 172.21.15.78:0/1222486587 conn(0x7f1d200fd5f0 msgr2=0x7f1d200fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.940+0000 7f1d24f4a640 1 -- 172.21.15.78:0/1222486587 shutdown_connections 2024-08-18T23:41:08.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:08.941+0000 7f1d24f4a640 1 -- 172.21.15.78:0/1222486587 wait complete. 2024-08-18T23:41:09.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:09.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:09.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:09.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:09.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:09.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:09 smithi078 ceph-mon[18440]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:09.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:09.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:09 smithi078 ceph-mon[18440]: Deploying daemon mds.h on smithi123 2024-08-18T23:41:09.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:09 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.161:6836/2017185021,v1:172.21.15.161:6837/2017185021] up:boot 2024-08-18T23:41:09.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:09 smithi078 ceph-mon[18440]: fsmap 7 up:standby 2024-08-18T23:41:09.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:09 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-18T23:41:09.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:09 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1222486587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:09.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:09.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:09.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:09.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:09.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:09.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:09 smithi123 ceph-mon[21808]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:09.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:09.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:09 smithi123 ceph-mon[21808]: Deploying daemon mds.h on smithi123 2024-08-18T23:41:09.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:09 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.161:6836/2017185021,v1:172.21.15.161:6837/2017185021] up:boot 2024-08-18T23:41:09.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:09 smithi123 ceph-mon[21808]: fsmap 7 up:standby 2024-08-18T23:41:09.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:09 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-18T23:41:09.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:09 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1222486587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:09.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:09.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:09.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:09.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:09.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:09.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:09 smithi161 ceph-mon[22027]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:09.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:09.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:09 smithi161 ceph-mon[22027]: Deploying daemon mds.h on smithi123 2024-08-18T23:41:09.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:09 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.161:6836/2017185021,v1:172.21.15.161:6837/2017185021] up:boot 2024-08-18T23:41:09.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:09 smithi161 ceph-mon[22027]: fsmap 7 up:standby 2024-08-18T23:41:09.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:09 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-18T23:41:09.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:09 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1222486587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:10.282 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:41:10 smithi123 systemd[1]: Starting Ceph mds.h for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:41:10.547 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:41:10 smithi123 podman[51220]: 2024-08-18 23:41:10.184159392 +0000 UTC m=+0.017409220 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:41:10.548 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:41:10 smithi123 podman[51220]: 2024-08-18 23:41:10.316739621 +0000 UTC m=+0.149989429 container create 42145008a6800ea476f5ce246cce063069f967691d7fbc9a67ef3ade412044d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-18T23:41:11.047 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:41:10 smithi123 podman[51220]: 2024-08-18 23:41:10.600391844 +0000 UTC m=+0.433641666 container init 42145008a6800ea476f5ce246cce063069f967691d7fbc9a67ef3ade412044d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0) 2024-08-18T23:41:11.047 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:41:10 smithi123 podman[51220]: 2024-08-18 23:41:10.603137563 +0000 UTC m=+0.436387373 container start 42145008a6800ea476f5ce246cce063069f967691d7fbc9a67ef3ade412044d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-h, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:41:11.048 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:41:10 smithi123 ceph-mds[51240]: -- 172.21.15.123:0/3468360632 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564914bfb860 con 0x564914bd9000 2024-08-18T23:41:11.048 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:41:10 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-h[51236]: starting mds.h at 2024-08-18T23:41:11.048 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:41:10 smithi123 bash[51220]: 42145008a6800ea476f5ce246cce063069f967691d7fbc9a67ef3ade412044d0 2024-08-18T23:41:11.048 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:41:10 smithi123 systemd[1]: Started Ceph mds.h for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:41:11.423 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:11 smithi161 ceph-mon[22027]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:11.423 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:11 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:11.423 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:11 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:11.423 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:11 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:11.423 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:11 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:11.424 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:11 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:11.424 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:11 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:11.441 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:11 smithi078 ceph-mon[18440]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:11.441 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:11 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:11.441 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:11 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:11.441 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:11 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:11.441 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:11 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:11.441 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:11 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:11.441 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:11 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:11.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:11 smithi123 ceph-mon[21808]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:11.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:11 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:11.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:11 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:11.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:11 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:11.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:11 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:11.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:11 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:11.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:11 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:11.736 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:41:11.995 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:12.412 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:12 smithi078 ceph-mon[18440]: Deploying daemon mds.i on smithi161 2024-08-18T23:41:12.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:12 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] up:boot 2024-08-18T23:41:12.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:12 smithi078 ceph-mon[18440]: fsmap 8 up:standby 2024-08-18T23:41:12.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:12 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-18T23:41:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:12 smithi161 ceph-mon[22027]: Deploying daemon mds.i on smithi161 2024-08-18T23:41:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:12 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] up:boot 2024-08-18T23:41:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:12 smithi161 ceph-mon[22027]: fsmap 8 up:standby 2024-08-18T23:41:12.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:12 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-18T23:41:12.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.493+0000 7f407dc81640 1 -- 172.21.15.78:0/229950690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4078102240 msgr2=0x7f407810e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:12.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.493+0000 7f407dc81640 1 --2- 172.21.15.78:0/229950690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4078102240 0x7f407810e770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4070009cd0 tx=0x7f407002a8e0 comp rx=0 tx=0).stop 2024-08-18T23:41:12.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.493+0000 7f407dc81640 1 -- 172.21.15.78:0/229950690 shutdown_connections 2024-08-18T23:41:12.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.493+0000 7f407dc81640 1 --2- 172.21.15.78:0/229950690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4078102240 0x7f407810e770 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:12.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.493+0000 7f407dc81640 1 --2- 172.21.15.78:0/229950690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40781018a0 0x7f4078101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:12.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.493+0000 7f407dc81640 1 --2- 172.21.15.78:0/229950690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40781078a0 0x7f4078107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:12.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.493+0000 7f407dc81640 1 -- 172.21.15.78:0/229950690 >> 172.21.15.78:0/229950690 conn(0x7f40780fd5e0 msgr2=0x7f40780ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:12.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.493+0000 7f407dc81640 1 -- 172.21.15.78:0/229950690 shutdown_connections 2024-08-18T23:41:12.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.493+0000 7f407dc81640 1 -- 172.21.15.78:0/229950690 wait complete. 2024-08-18T23:41:12.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.493+0000 7f407dc81640 1 Processor -- start 2024-08-18T23:41:12.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.494+0000 7f407dc81640 1 -- start start 2024-08-18T23:41:12.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.494+0000 7f407dc81640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40781018a0 0x7f4078074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:12.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.494+0000 7f407dc81640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4078102240 0x7f4078074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:12.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.494+0000 7f407dc81640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40781078a0 0x7f4078074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:12.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.494+0000 7f407dc81640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4078111ca0 con 0x7f40781078a0 2024-08-18T23:41:12.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.494+0000 7f407dc81640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4078111b20 con 0x7f40781018a0 2024-08-18T23:41:12.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.494+0000 7f407dc81640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4078111e20 con 0x7f4078102240 2024-08-18T23:41:12.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.494+0000 7f4077fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40781078a0 0x7f4078074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:12.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.494+0000 7f4077fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40781078a0 0x7f4078074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59478/0 (socket says 172.21.15.78:59478) 2024-08-18T23:41:12.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.494+0000 7f4077fff640 1 -- 172.21.15.78:0/845208351 learned_addr learned my addr 172.21.15.78:0/845208351 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:12.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.494+0000 7f4076ffd640 1 --2- 172.21.15.78:0/845208351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4078102240 0x7f4078074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:12.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.494+0000 7f40777fe640 1 --2- 172.21.15.78:0/845208351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40781018a0 0x7f4078074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:12.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.494+0000 7f4077fff640 1 -- 172.21.15.78:0/845208351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4078102240 msgr2=0x7f4078074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:12.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.494+0000 7f4077fff640 1 --2- 172.21.15.78:0/845208351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4078102240 0x7f4078074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:12.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.494+0000 7f4077fff640 1 -- 172.21.15.78:0/845208351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40781018a0 msgr2=0x7f4078074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:12.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.494+0000 7f4077fff640 1 --2- 172.21.15.78:0/845208351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40781018a0 0x7f4078074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:12.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.494+0000 7f4077fff640 1 -- 172.21.15.78:0/845208351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40781a6660 con 0x7f40781078a0 2024-08-18T23:41:12.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.494+0000 7f4076ffd640 1 --2- 172.21.15.78:0/845208351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4078102240 0x7f4078074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:41:12.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.495+0000 7f4077fff640 1 --2- 172.21.15.78:0/845208351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40781078a0 0x7f4078074ca0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4070005cc0 tx=0x7f4070032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:12.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.495+0000 7f4074ff9640 1 -- 172.21.15.78:0/845208351 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070004040 con 0x7f40781078a0 2024-08-18T23:41:12.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.495+0000 7f407dc81640 1 -- 172.21.15.78:0/845208351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40781a68f0 con 0x7f40781078a0 2024-08-18T23:41:12.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.495+0000 7f4074ff9640 1 -- 172.21.15.78:0/845208351 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40700046c0 con 0x7f40781078a0 2024-08-18T23:41:12.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.495+0000 7f4074ff9640 1 -- 172.21.15.78:0/845208351 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070030e00 con 0x7f40781078a0 2024-08-18T23:41:12.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.495+0000 7f407dc81640 1 -- 172.21.15.78:0/845208351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40781a6e80 con 0x7f40781078a0 2024-08-18T23:41:12.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.496+0000 7f407dc81640 1 -- 172.21.15.78:0/845208351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4078101d00 con 0x7f40781078a0 2024-08-18T23:41:12.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.498+0000 7f4074ff9640 1 -- 172.21.15.78:0/845208351 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f40700305a0 con 0x7f40781078a0 2024-08-18T23:41:12.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.498+0000 7f4074ff9640 1 --2- 172.21.15.78:0/845208351 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f40600a2b80 0x7f40600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:12.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.498+0000 7f40777fe640 1 --2- 172.21.15.78:0/845208351 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f40600a2b80 0x7f40600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:12.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.498+0000 7f4074ff9640 1 -- 172.21.15.78:0/845208351 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f407002e030 con 0x7f40781078a0 2024-08-18T23:41:12.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.499+0000 7f40777fe640 1 --2- 172.21.15.78:0/845208351 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f40600a2b80 0x7f40600a5040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4068009ae0 tx=0x7f4068005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:12.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.507+0000 7f4074ff9640 1 -- 172.21.15.78:0/845208351 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f407004b060 con 0x7f40781078a0 2024-08-18T23:41:12.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:12 smithi123 ceph-mon[21808]: Deploying daemon mds.i on smithi161 2024-08-18T23:41:12.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:12 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] up:boot 2024-08-18T23:41:12.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:12 smithi123 ceph-mon[21808]: fsmap 8 up:standby 2024-08-18T23:41:12.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:12 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-18T23:41:12.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.714+0000 7f407dc81640 1 -- 172.21.15.78:0/845208351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4078109e10 con 0x7f40781078a0 2024-08-18T23:41:12.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.714+0000 7f4074ff9640 1 -- 172.21.15.78:0/845208351 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7228 (secure 0 0 0) 0x7f40700bd7c0 con 0x7f40781078a0 2024-08-18T23:41:12.716 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:41:12.716 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":9,"btime":"2024-08-18T23:41:11:155174+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-08-18T23:41:12.716 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 9 2024-08-18T23:41:12.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.716+0000 7f407dc81640 1 -- 172.21.15.78:0/845208351 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f40600a2b80 msgr2=0x7f40600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:12.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.716+0000 7f407dc81640 1 --2- 172.21.15.78:0/845208351 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f40600a2b80 0x7f40600a5040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4068009ae0 tx=0x7f4068005f90 comp rx=0 tx=0).stop 2024-08-18T23:41:12.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.716+0000 7f407dc81640 1 -- 172.21.15.78:0/845208351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40781078a0 msgr2=0x7f4078074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:12.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.716+0000 7f407dc81640 1 --2- 172.21.15.78:0/845208351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40781078a0 0x7f4078074ca0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4070005cc0 tx=0x7f4070032f40 comp rx=0 tx=0).stop 2024-08-18T23:41:12.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.717+0000 7f407dc81640 1 -- 172.21.15.78:0/845208351 shutdown_connections 2024-08-18T23:41:12.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.717+0000 7f407dc81640 1 --2- 172.21.15.78:0/845208351 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f40600a2b80 0x7f40600a5040 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:12.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.717+0000 7f407dc81640 1 --2- 172.21.15.78:0/845208351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40781078a0 0x7f4078074ca0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:12.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.717+0000 7f407dc81640 1 --2- 172.21.15.78:0/845208351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4078102240 0x7f4078074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:12.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.717+0000 7f407dc81640 1 --2- 172.21.15.78:0/845208351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40781018a0 0x7f4078074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:12.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.717+0000 7f407dc81640 1 -- 172.21.15.78:0/845208351 >> 172.21.15.78:0/845208351 conn(0x7f40780fd5e0 msgr2=0x7f40780fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:12.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.717+0000 7f407dc81640 1 -- 172.21.15.78:0/845208351 shutdown_connections 2024-08-18T23:41:12.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:12.717+0000 7f407dc81640 1 -- 172.21.15.78:0/845208351 wait complete. 2024-08-18T23:41:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:13 smithi161 ceph-mon[22027]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:13.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:13 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/845208351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:13.427 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 18 23:41:13 smithi161 systemd[1]: Started Ceph mds.i for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:41:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:13 smithi078 ceph-mon[18440]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:13 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/845208351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:13.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:13 smithi123 ceph-mon[21808]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:13.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:13 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/845208351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:14 smithi161 ceph-mon[22027]: Deploying daemon mds.g on smithi078 2024-08-18T23:41:14.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:14 smithi161 ceph-mon[22027]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:14.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:14 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] up:boot 2024-08-18T23:41:14.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:14 smithi161 ceph-mon[22027]: fsmap 9 up:standby 2024-08-18T23:41:14.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:14 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-18T23:41:14.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:14.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:14.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:14.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:14.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:14.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:14.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:14 smithi078 ceph-mon[18440]: Deploying daemon mds.g on smithi078 2024-08-18T23:41:14.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:14 smithi078 ceph-mon[18440]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:14.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:14 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] up:boot 2024-08-18T23:41:14.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:14 smithi078 ceph-mon[18440]: fsmap 9 up:standby 2024-08-18T23:41:14.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:14 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-18T23:41:14.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:14.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:14.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:14.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:14.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:14.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:14.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:14 smithi123 ceph-mon[21808]: Deploying daemon mds.g on smithi078 2024-08-18T23:41:14.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:14 smithi123 ceph-mon[21808]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:14.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:14 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] up:boot 2024-08-18T23:41:14.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:14 smithi123 ceph-mon[21808]: fsmap 9 up:standby 2024-08-18T23:41:14.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:14 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-18T23:41:15.532 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:41:15.558 INFO:journalctl@ceph.mds.g.smithi078.stdout:Aug 18 23:41:15 smithi078 systemd[1]: Starting Ceph mds.g for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:41:15.855 INFO:journalctl@ceph.mds.g.smithi078.stdout:Aug 18 23:41:15 smithi078 podman[58683]: 2024-08-18 23:41:15.604986717 +0000 UTC m=+0.017680111 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:41:15.855 INFO:journalctl@ceph.mds.g.smithi078.stdout:Aug 18 23:41:15 smithi078 podman[58683]: 2024-08-18 23:41:15.737201058 +0000 UTC m=+0.149894439 container create 241c7df2c510972dff317b4ee6cbc2a8dd03e49f2563a2ed0ec0401c5ec4bc50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-g, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-18T23:41:16.113 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:16.219 INFO:journalctl@ceph.mds.g.smithi078.stdout:Aug 18 23:41:15 smithi078 podman[58683]: 2024-08-18 23:41:15.97308298 +0000 UTC m=+0.385776363 container init 241c7df2c510972dff317b4ee6cbc2a8dd03e49f2563a2ed0ec0401c5ec4bc50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:41:16.219 INFO:journalctl@ceph.mds.g.smithi078.stdout:Aug 18 23:41:15 smithi078 podman[58683]: 2024-08-18 23:41:15.975998891 +0000 UTC m=+0.388692290 container start 241c7df2c510972dff317b4ee6cbc2a8dd03e49f2563a2ed0ec0401c5ec4bc50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-18T23:41:16.219 INFO:journalctl@ceph.mds.g.smithi078.stdout:Aug 18 23:41:15 smithi078 ceph-mds[58728]: -- 172.21.15.78:0/3298095431 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5562cc48d860 con 0x5562cc46b400 2024-08-18T23:41:16.219 INFO:journalctl@ceph.mds.g.smithi078.stdout:Aug 18 23:41:16 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-g[58724]: starting mds.g at 2024-08-18T23:41:16.219 INFO:journalctl@ceph.mds.g.smithi078.stdout:Aug 18 23:41:16 smithi078 bash[58683]: 241c7df2c510972dff317b4ee6cbc2a8dd03e49f2563a2ed0ec0401c5ec4bc50 2024-08-18T23:41:16.220 INFO:journalctl@ceph.mds.g.smithi078.stdout:Aug 18 23:41:16 smithi078 systemd[1]: Started Ceph mds.g for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:41:16.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.616+0000 7fc9f72bf640 1 -- 172.21.15.78:0/88403075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f0100080 msgr2=0x7fc9f0100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.616+0000 7fc9f72bf640 1 --2- 172.21.15.78:0/88403075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f0100080 0x7fc9f0100460 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec009930 tx=0x7fc9ec02a420 comp rx=0 tx=0).stop 2024-08-18T23:41:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.616+0000 7fc9f72bf640 1 -- 172.21.15.78:0/88403075 shutdown_connections 2024-08-18T23:41:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.616+0000 7fc9f72bf640 1 --2- 172.21.15.78:0/88403075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9f0106c80 0x7fc9f010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.616+0000 7fc9f72bf640 1 --2- 172.21.15.78:0/88403075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f0100a30 0x7fc9f0106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.616+0000 7fc9f72bf640 1 --2- 172.21.15.78:0/88403075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f0100080 0x7fc9f0100460 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.616+0000 7fc9f72bf640 1 -- 172.21.15.78:0/88403075 >> 172.21.15.78:0/88403075 conn(0x7fc9f0076f40 msgr2=0x7fc9f00fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.616+0000 7fc9f72bf640 1 -- 172.21.15.78:0/88403075 shutdown_connections 2024-08-18T23:41:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.616+0000 7fc9f72bf640 1 -- 172.21.15.78:0/88403075 wait complete. 2024-08-18T23:41:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.617+0000 7fc9f72bf640 1 Processor -- start 2024-08-18T23:41:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.617+0000 7fc9f72bf640 1 -- start start 2024-08-18T23:41:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.617+0000 7fc9f72bf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9f0100080 0x7fc9f0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.617+0000 7fc9f72bf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f0100a30 0x7fc9f01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.617+0000 7fc9f72bf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f0106c80 0x7fc9f019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.617+0000 7fc9f72bf640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f0111db0 con 0x7fc9f0100a30 2024-08-18T23:41:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.617+0000 7fc9f72bf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f0111c30 con 0x7fc9f0100080 2024-08-18T23:41:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.617+0000 7fc9f72bf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f0111f30 con 0x7fc9f0106c80 2024-08-18T23:41:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.617+0000 7fc9f485a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f0100a30 0x7fc9f01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.617+0000 7fc9f485a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f0100a30 0x7fc9f01988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49294/0 (socket says 172.21.15.78:49294) 2024-08-18T23:41:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.617+0000 7fc9f485a640 1 -- 172.21.15.78:0/312539412 learned_addr learned my addr 172.21.15.78:0/312539412 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.618+0000 7fc9f505b640 1 --2- 172.21.15.78:0/312539412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9f0100080 0x7fc9f0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.618+0000 7fc9f585c640 1 --2- 172.21.15.78:0/312539412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f0106c80 0x7fc9f019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.618+0000 7fc9f485a640 1 -- 172.21.15.78:0/312539412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f0106c80 msgr2=0x7fc9f019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.618+0000 7fc9f485a640 1 --2- 172.21.15.78:0/312539412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f0106c80 0x7fc9f019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.618+0000 7fc9f485a640 1 -- 172.21.15.78:0/312539412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9f0100080 msgr2=0x7fc9f0198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.618+0000 7fc9f485a640 1 --2- 172.21.15.78:0/312539412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9f0100080 0x7fc9f0198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.618+0000 7fc9f485a640 1 -- 172.21.15.78:0/312539412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f019d320 con 0x7fc9f0100a30 2024-08-18T23:41:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.618+0000 7fc9f585c640 1 --2- 172.21.15.78:0/312539412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f0106c80 0x7fc9f019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:41:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.618+0000 7fc9f485a640 1 --2- 172.21.15.78:0/312539412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f0100a30 0x7fc9f01988b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000b450 tx=0x7fc9e000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.618+0000 7fc9e67fc640 1 -- 172.21.15.78:0/312539412 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0015020 con 0x7fc9f0100a30 2024-08-18T23:41:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.618+0000 7fc9f72bf640 1 -- 172.21.15.78:0/312539412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f019d610 con 0x7fc9f0100a30 2024-08-18T23:41:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.618+0000 7fc9e67fc640 1 -- 172.21.15.78:0/312539412 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9e0011d70 con 0x7fc9f0100a30 2024-08-18T23:41:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.618+0000 7fc9e67fc640 1 -- 172.21.15.78:0/312539412 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e000f340 con 0x7fc9f0100a30 2024-08-18T23:41:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.618+0000 7fc9f72bf640 1 -- 172.21.15.78:0/312539412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f01a4ef0 con 0x7fc9f0100a30 2024-08-18T23:41:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.619+0000 7fc9f72bf640 1 -- 172.21.15.78:0/312539412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f01004e0 con 0x7fc9f0100a30 2024-08-18T23:41:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.620+0000 7fc9e67fc640 1 -- 172.21.15.78:0/312539412 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc9e0018020 con 0x7fc9f0100a30 2024-08-18T23:41:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.621+0000 7fc9e67fc640 1 --2- 172.21.15.78:0/312539412 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc9dc0a2b80 0x7fc9dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.621+0000 7fc9f505b640 1 --2- 172.21.15.78:0/312539412 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc9dc0a2b80 0x7fc9dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:16.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.621+0000 7fc9e67fc640 1 -- 172.21.15.78:0/312539412 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fc9e00ce700 con 0x7fc9f0100a30 2024-08-18T23:41:16.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.621+0000 7fc9f505b640 1 --2- 172.21.15.78:0/312539412 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc9dc0a2b80 0x7fc9dc0a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec000c00 tx=0x7fc9ec031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:16.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.625+0000 7fc9e67fc640 1 -- 172.21.15.78:0/312539412 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc9e001f020 con 0x7fc9f0100a30 2024-08-18T23:41:16.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.836+0000 7fc9f72bf640 1 -- 172.21.15.78:0/312539412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9f002d5e0 con 0x7fc9f0100a30 2024-08-18T23:41:16.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.836+0000 7fc9e67fc640 1 -- 172.21.15.78:0/312539412 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8884 (secure 0 0 0) 0x7fc9e000d040 con 0x7fc9f0100a30 2024-08-18T23:41:16.838 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:41:16.838 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":11,"btime":"2024-08-18T23:41:16:054218+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-08-18T23:41:16.838 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 11 2024-08-18T23:41:16.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.838+0000 7fc9f72bf640 1 -- 172.21.15.78:0/312539412 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc9dc0a2b80 msgr2=0x7fc9dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:16.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.838+0000 7fc9f72bf640 1 --2- 172.21.15.78:0/312539412 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc9dc0a2b80 0x7fc9dc0a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec000c00 tx=0x7fc9ec031490 comp rx=0 tx=0).stop 2024-08-18T23:41:16.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.838+0000 7fc9f72bf640 1 -- 172.21.15.78:0/312539412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f0100a30 msgr2=0x7fc9f01988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:16.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.838+0000 7fc9f72bf640 1 --2- 172.21.15.78:0/312539412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f0100a30 0x7fc9f01988b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000b450 tx=0x7fc9e000b910 comp rx=0 tx=0).stop 2024-08-18T23:41:16.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.838+0000 7fc9f72bf640 1 -- 172.21.15.78:0/312539412 shutdown_connections 2024-08-18T23:41:16.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.838+0000 7fc9f72bf640 1 --2- 172.21.15.78:0/312539412 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc9dc0a2b80 0x7fc9dc0a5040 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:16.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.838+0000 7fc9f72bf640 1 --2- 172.21.15.78:0/312539412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f0106c80 0x7fc9f019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:16.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.838+0000 7fc9f72bf640 1 --2- 172.21.15.78:0/312539412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc9f0100a30 0x7fc9f01988b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:16.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.838+0000 7fc9f72bf640 1 --2- 172.21.15.78:0/312539412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9f0100080 0x7fc9f0198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:16.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.838+0000 7fc9f72bf640 1 -- 172.21.15.78:0/312539412 >> 172.21.15.78:0/312539412 conn(0x7fc9f0076f40 msgr2=0x7fc9f0104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:16.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.838+0000 7fc9f72bf640 1 -- 172.21.15.78:0/312539412 shutdown_connections 2024-08-18T23:41:16.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:16.838+0000 7fc9f72bf640 1 -- 172.21.15.78:0/312539412 wait complete. 2024-08-18T23:41:17.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:17 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.78:6838/2648770058,v1:172.21.15.78:6839/2648770058] up:boot 2024-08-18T23:41:17.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:17 smithi161 ceph-mon[22027]: fsmap 10 up:standby 2024-08-18T23:41:17.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:17 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-18T23:41:17.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:17 smithi161 ceph-mon[22027]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:17.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:17 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:17.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:17 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:17.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:17 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:17.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:17 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:17.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:17 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:17.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:17 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:17.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:17 smithi161 ceph-mon[22027]: Deploying daemon mds.a on smithi078 2024-08-18T23:41:17.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:17 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/312539412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:17 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.78:6838/2648770058,v1:172.21.15.78:6839/2648770058] up:boot 2024-08-18T23:41:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:17 smithi078 ceph-mon[18440]: fsmap 10 up:standby 2024-08-18T23:41:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:17 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-18T23:41:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:17 smithi078 ceph-mon[18440]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:17 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:17.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:17 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:17.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:17 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:17.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:17 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:17.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:17 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:17.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:17 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:17.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:17 smithi078 ceph-mon[18440]: Deploying daemon mds.a on smithi078 2024-08-18T23:41:17.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:17 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/312539412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:17.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:17 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.78:6838/2648770058,v1:172.21.15.78:6839/2648770058] up:boot 2024-08-18T23:41:17.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:17 smithi123 ceph-mon[21808]: fsmap 10 up:standby 2024-08-18T23:41:17.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:17 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-18T23:41:17.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:17 smithi123 ceph-mon[21808]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:17.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:17 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:17.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:17 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:17.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:17 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:17.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:17 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:17.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:17 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:17.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:17 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:17.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:17 smithi123 ceph-mon[21808]: Deploying daemon mds.a on smithi078 2024-08-18T23:41:17.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:17 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/312539412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:19.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:19 smithi078 ceph-mon[18440]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:19.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:19 smithi161 ceph-mon[22027]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:19.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:19 smithi123 ceph-mon[21808]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:19.691 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:41:19.955 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:19.969 INFO:journalctl@ceph.mds.a.smithi078.stdout:Aug 18 23:41:19 smithi078 systemd[1]: Started Ceph mds.a for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:41:20.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.428+0000 7fd2e9189640 1 -- 172.21.15.78:0/888998865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e41018b0 msgr2=0x7fd2e4101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:20.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.428+0000 7fd2e9189640 1 --2- 172.21.15.78:0/888998865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e41018b0 0x7fd2e4101d10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009930 tx=0x7fd2d802a420 comp rx=0 tx=0).stop 2024-08-18T23:41:20.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.429+0000 7fd2e9189640 1 -- 172.21.15.78:0/888998865 shutdown_connections 2024-08-18T23:41:20.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.429+0000 7fd2e9189640 1 --2- 172.21.15.78:0/888998865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e4102250 0x7fd2e410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:20.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.429+0000 7fd2e9189640 1 --2- 172.21.15.78:0/888998865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e41018b0 0x7fd2e4101d10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:20.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.429+0000 7fd2e9189640 1 --2- 172.21.15.78:0/888998865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2e41078b0 0x7fd2e4107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:20.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.429+0000 7fd2e9189640 1 -- 172.21.15.78:0/888998865 >> 172.21.15.78:0/888998865 conn(0x7fd2e40fd5f0 msgr2=0x7fd2e40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:20.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.429+0000 7fd2e9189640 1 -- 172.21.15.78:0/888998865 shutdown_connections 2024-08-18T23:41:20.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.429+0000 7fd2e9189640 1 -- 172.21.15.78:0/888998865 wait complete. 2024-08-18T23:41:20.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.429+0000 7fd2e9189640 1 Processor -- start 2024-08-18T23:41:20.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.429+0000 7fd2e9189640 1 -- start start 2024-08-18T23:41:20.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.430+0000 7fd2e9189640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e41018b0 0x7fd2e419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:20.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.430+0000 7fd2e9189640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e4102250 0x7fd2e419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:20.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.430+0000 7fd2e9189640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2e41078b0 0x7fd2e419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:20.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.430+0000 7fd2e9189640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4111d50 con 0x7fd2e4102250 2024-08-18T23:41:20.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.430+0000 7fd2e9189640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4111bd0 con 0x7fd2e41018b0 2024-08-18T23:41:20.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.430+0000 7fd2e9189640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4111ed0 con 0x7fd2e41078b0 2024-08-18T23:41:20.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.430+0000 7fd2e259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e4102250 0x7fd2e419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:20.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.430+0000 7fd2e259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e4102250 0x7fd2e419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49348/0 (socket says 172.21.15.78:49348) 2024-08-18T23:41:20.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.430+0000 7fd2e259c640 1 -- 172.21.15.78:0/2712895962 learned_addr learned my addr 172.21.15.78:0/2712895962 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:20.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.430+0000 7fd2e2d9d640 1 --2- 172.21.15.78:0/2712895962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e41018b0 0x7fd2e419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:20.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.430+0000 7fd2e359e640 1 --2- 172.21.15.78:0/2712895962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2e41078b0 0x7fd2e419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:20.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.430+0000 7fd2e259c640 1 -- 172.21.15.78:0/2712895962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2e41078b0 msgr2=0x7fd2e419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:20.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.430+0000 7fd2e259c640 1 --2- 172.21.15.78:0/2712895962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2e41078b0 0x7fd2e419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:20.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.430+0000 7fd2e259c640 1 -- 172.21.15.78:0/2712895962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e41018b0 msgr2=0x7fd2e419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:20.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.430+0000 7fd2e259c640 1 --2- 172.21.15.78:0/2712895962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e41018b0 0x7fd2e419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:20.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.430+0000 7fd2e259c640 1 -- 172.21.15.78:0/2712895962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e419f500 con 0x7fd2e4102250 2024-08-18T23:41:20.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.430+0000 7fd2e259c640 1 --2- 172.21.15.78:0/2712895962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e4102250 0x7fd2e419aa90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009a50 tx=0x7fd2d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:20.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.430+0000 7fd2cbfff640 1 -- 172.21.15.78:0/2712895962 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d803f030 con 0x7fd2e4102250 2024-08-18T23:41:20.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.431+0000 7fd2cbfff640 1 -- 172.21.15.78:0/2712895962 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2d8003ad0 con 0x7fd2e4102250 2024-08-18T23:41:20.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.431+0000 7fd2e9189640 1 -- 172.21.15.78:0/2712895962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e419f790 con 0x7fd2e4102250 2024-08-18T23:41:20.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.431+0000 7fd2cbfff640 1 -- 172.21.15.78:0/2712895962 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8035070 con 0x7fd2e4102250 2024-08-18T23:41:20.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.431+0000 7fd2e9189640 1 -- 172.21.15.78:0/2712895962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e41a70d0 con 0x7fd2e4102250 2024-08-18T23:41:20.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.432+0000 7fd2e9189640 1 -- 172.21.15.78:0/2712895962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e4102ff0 con 0x7fd2e4102250 2024-08-18T23:41:20.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.433+0000 7fd2cbfff640 1 -- 172.21.15.78:0/2712895962 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd2d8042030 con 0x7fd2e4102250 2024-08-18T23:41:20.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.433+0000 7fd2cbfff640 1 --2- 172.21.15.78:0/2712895962 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2cc0a2b80 0x7fd2cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:20.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.433+0000 7fd2e2d9d640 1 --2- 172.21.15.78:0/2712895962 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2cc0a2b80 0x7fd2cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:20.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.434+0000 7fd2cbfff640 1 -- 172.21.15.78:0/2712895962 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fd2d80ef380 con 0x7fd2e4102250 2024-08-18T23:41:20.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.434+0000 7fd2e2d9d640 1 --2- 172.21.15.78:0/2712895962 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2cc0a2b80 0x7fd2cc0a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009990 tx=0x7fd2d4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:20.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.438+0000 7fd2cbfff640 1 -- 172.21.15.78:0/2712895962 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd2d80bc780 con 0x7fd2e4102250 2024-08-18T23:41:20.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.638+0000 7fd2e9189640 1 -- 172.21.15.78:0/2712895962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2e41a72a0 con 0x7fd2e4102250 2024-08-18T23:41:20.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.639+0000 7fd2cbfff640 1 -- 172.21.15.78:0/2712895962 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9710 (secure 0 0 0) 0x7fd2d80bc080 con 0x7fd2e4102250 2024-08-18T23:41:20.640 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:41:20.640 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":12,"btime":"2024-08-18T23:41:20:158242+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-08-18T23:41:20.641 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 12 2024-08-18T23:41:20.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.641+0000 7fd2e9189640 1 -- 172.21.15.78:0/2712895962 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2cc0a2b80 msgr2=0x7fd2cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:20.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.641+0000 7fd2e9189640 1 --2- 172.21.15.78:0/2712895962 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2cc0a2b80 0x7fd2cc0a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009990 tx=0x7fd2d4009110 comp rx=0 tx=0).stop 2024-08-18T23:41:20.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.641+0000 7fd2e9189640 1 -- 172.21.15.78:0/2712895962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e4102250 msgr2=0x7fd2e419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:20.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.641+0000 7fd2e9189640 1 --2- 172.21.15.78:0/2712895962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e4102250 0x7fd2e419aa90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009a50 tx=0x7fd2d8004270 comp rx=0 tx=0).stop 2024-08-18T23:41:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.641+0000 7fd2e9189640 1 -- 172.21.15.78:0/2712895962 shutdown_connections 2024-08-18T23:41:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.641+0000 7fd2e9189640 1 --2- 172.21.15.78:0/2712895962 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2cc0a2b80 0x7fd2cc0a5040 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.641+0000 7fd2e9189640 1 --2- 172.21.15.78:0/2712895962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2e41078b0 0x7fd2e419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.641+0000 7fd2e9189640 1 --2- 172.21.15.78:0/2712895962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e4102250 0x7fd2e419aa90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.641+0000 7fd2e9189640 1 --2- 172.21.15.78:0/2712895962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e41018b0 0x7fd2e419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.641+0000 7fd2e9189640 1 -- 172.21.15.78:0/2712895962 >> 172.21.15.78:0/2712895962 conn(0x7fd2e40fd5f0 msgr2=0x7fd2e410cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.641+0000 7fd2e9189640 1 -- 172.21.15.78:0/2712895962 shutdown_connections 2024-08-18T23:41:20.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:20.641+0000 7fd2e9189640 1 -- 172.21.15.78:0/2712895962 wait complete. 2024-08-18T23:41:20.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:20.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:20.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:20.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:20.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:20.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:20.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:20 smithi078 ceph-mon[18440]: Deploying daemon mds.b on smithi123 2024-08-18T23:41:20.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:20 smithi078 ceph-mon[18440]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:20.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:20 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.78:6840/2010800750,v1:172.21.15.78:6841/2010800750] up:boot 2024-08-18T23:41:20.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:20 smithi078 ceph-mon[18440]: fsmap 11 up:standby 2024-08-18T23:41:20.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:20 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-18T23:41:20.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:20 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2712895962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:21.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:21.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:21.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:21.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:21.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:21.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:21.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:20 smithi161 ceph-mon[22027]: Deploying daemon mds.b on smithi123 2024-08-18T23:41:21.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:20 smithi161 ceph-mon[22027]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:21.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:20 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.78:6840/2010800750,v1:172.21.15.78:6841/2010800750] up:boot 2024-08-18T23:41:21.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:20 smithi161 ceph-mon[22027]: fsmap 11 up:standby 2024-08-18T23:41:21.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:20 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-18T23:41:21.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:20 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2712895962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:21.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:21.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:21.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:21.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-18T23:41:21.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-18T23:41:21.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:21.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:20 smithi123 ceph-mon[21808]: Deploying daemon mds.b on smithi123 2024-08-18T23:41:21.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:20 smithi123 ceph-mon[21808]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:21.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:20 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.78:6840/2010800750,v1:172.21.15.78:6841/2010800750] up:boot 2024-08-18T23:41:21.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:20 smithi123 ceph-mon[21808]: fsmap 11 up:standby 2024-08-18T23:41:21.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:20 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-18T23:41:21.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:20 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2712895962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:22.047 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:41:21 smithi123 systemd[1]: Starting Ceph mds.b for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:41:22.450 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:41:22 smithi123 podman[51531]: 2024-08-18 23:41:22.080627662 +0000 UTC m=+0.017240749 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:41:22.450 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:41:22 smithi123 podman[51531]: 2024-08-18 23:41:22.188112304 +0000 UTC m=+0.124725380 container create b7423c35eb0bb5ce8bccff6e915983afd0196b0fd97a067a1002764a78e3247d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , ceph=True) 2024-08-18T23:41:22.797 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:41:22 smithi123 podman[51531]: 2024-08-18 23:41:22.449329163 +0000 UTC m=+0.385942236 container init b7423c35eb0bb5ce8bccff6e915983afd0196b0fd97a067a1002764a78e3247d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-b, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-18T23:41:22.797 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:41:22 smithi123 podman[51531]: 2024-08-18 23:41:22.453080532 +0000 UTC m=+0.389693612 container start b7423c35eb0bb5ce8bccff6e915983afd0196b0fd97a067a1002764a78e3247d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-b, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-08-18T23:41:22.798 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:41:22 smithi123 ceph-mds[51551]: -- 172.21.15.123:0/722558903 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f0e01cb860 con 0x55f0e01a9400 2024-08-18T23:41:22.798 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:41:22 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-b[51547]: starting mds.b at 2024-08-18T23:41:22.798 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:41:22 smithi123 bash[51531]: b7423c35eb0bb5ce8bccff6e915983afd0196b0fd97a067a1002764a78e3247d 2024-08-18T23:41:22.798 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:41:22 smithi123 systemd[1]: Started Ceph mds.b for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:41:23.419 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:41:23.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:23 smithi161 ceph-mon[22027]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:23.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:23.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:23.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:23.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:23.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:23.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:41:23.442 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:23 smithi078 ceph-mon[18440]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:23.442 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:23.442 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:23.442 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:23.442 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:23.442 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:23.443 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:41:23.480 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:23 smithi123 ceph-mon[21808]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:23.480 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:23.480 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:23.480 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:23.480 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:23.481 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:23.481 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:41:23.905 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:24.289 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:24 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] up:boot 2024-08-18T23:41:24.289 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:24 smithi078 ceph-mon[18440]: fsmap 12 up:standby 2024-08-18T23:41:24.289 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:24 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-18T23:41:24.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:24 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] up:boot 2024-08-18T23:41:24.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:24 smithi161 ceph-mon[22027]: fsmap 12 up:standby 2024-08-18T23:41:24.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:24 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-18T23:41:24.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.373+0000 7f0a2e339640 1 -- 172.21.15.78:0/74153603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281022a0 msgr2=0x7f0a281026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:24.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.373+0000 7f0a2e339640 1 --2- 172.21.15.78:0/74153603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281022a0 0x7f0a281026e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009900 tx=0x7f0a1c02a480 comp rx=0 tx=0).stop 2024-08-18T23:41:24.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.373+0000 7f0a2e339640 1 -- 172.21.15.78:0/74153603 shutdown_connections 2024-08-18T23:41:24.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.373+0000 7f0a2e339640 1 --2- 172.21.15.78:0/74153603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a28102c20 0x7f0a2810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:24.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.373+0000 7f0a2e339640 1 --2- 172.21.15.78:0/74153603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281022a0 0x7f0a281026e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:24.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.373+0000 7f0a2e339640 1 --2- 172.21.15.78:0/74153603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a28100b50 0x7f0a28100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:24.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.373+0000 7f0a2e339640 1 -- 172.21.15.78:0/74153603 >> 172.21.15.78:0/74153603 conn(0x7f0a280fc500 msgr2=0x7f0a280fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:24.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.373+0000 7f0a2e339640 1 -- 172.21.15.78:0/74153603 shutdown_connections 2024-08-18T23:41:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.373+0000 7f0a2e339640 1 -- 172.21.15.78:0/74153603 wait complete. 2024-08-18T23:41:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.373+0000 7f0a2e339640 1 Processor -- start 2024-08-18T23:41:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.373+0000 7f0a2e339640 1 -- start start 2024-08-18T23:41:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.374+0000 7f0a2e339640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a28100b50 0x7f0a28196050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.374+0000 7f0a2e339640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281022a0 0x7f0a28196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.374+0000 7f0a2e339640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a28102c20 0x7f0a2819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.374+0000 7f0a2e339640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2810fbb0 con 0x7f0a281022a0 2024-08-18T23:41:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.374+0000 7f0a2e339640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2810fa30 con 0x7f0a28100b50 2024-08-18T23:41:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.374+0000 7f0a2e339640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2810fd30 con 0x7f0a28102c20 2024-08-18T23:41:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.374+0000 7f0a277fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281022a0 0x7f0a28196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.374+0000 7f0a277fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281022a0 0x7f0a28196590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49368/0 (socket says 172.21.15.78:49368) 2024-08-18T23:41:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.374+0000 7f0a277fe640 1 -- 172.21.15.78:0/1463942844 learned_addr learned my addr 172.21.15.78:0/1463942844 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.374+0000 7f0a27fff640 1 --2- 172.21.15.78:0/1463942844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a28100b50 0x7f0a28196050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.374+0000 7f0a2c8d6640 1 --2- 172.21.15.78:0/1463942844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a28102c20 0x7f0a2819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:24.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.374+0000 7f0a277fe640 1 -- 172.21.15.78:0/1463942844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a28102c20 msgr2=0x7f0a2819a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:24.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.374+0000 7f0a277fe640 1 --2- 172.21.15.78:0/1463942844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a28102c20 0x7f0a2819a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:24.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.374+0000 7f0a277fe640 1 -- 172.21.15.78:0/1463942844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a28100b50 msgr2=0x7f0a28196050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:24.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.374+0000 7f0a277fe640 1 --2- 172.21.15.78:0/1463942844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a28100b50 0x7f0a28196050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:24.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.374+0000 7f0a277fe640 1 -- 172.21.15.78:0/1463942844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2819b000 con 0x7f0a281022a0 2024-08-18T23:41:24.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.374+0000 7f0a27fff640 1 --2- 172.21.15.78:0/1463942844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a28100b50 0x7f0a28196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:41:24.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.374+0000 7f0a277fe640 1 --2- 172.21.15.78:0/1463942844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281022a0 0x7f0a28196590 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c006010 tx=0x7f0a1c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:24.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.375+0000 7f0a257fa640 1 -- 172.21.15.78:0/1463942844 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c03f070 con 0x7f0a281022a0 2024-08-18T23:41:24.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.375+0000 7f0a2e339640 1 -- 172.21.15.78:0/1463942844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2819b290 con 0x7f0a281022a0 2024-08-18T23:41:24.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.375+0000 7f0a257fa640 1 -- 172.21.15.78:0/1463942844 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a1c031c30 con 0x7f0a281022a0 2024-08-18T23:41:24.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.375+0000 7f0a257fa640 1 -- 172.21.15.78:0/1463942844 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c035030 con 0x7f0a281022a0 2024-08-18T23:41:24.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.375+0000 7f0a2e339640 1 -- 172.21.15.78:0/1463942844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a281a2bd0 con 0x7f0a281022a0 2024-08-18T23:41:24.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.376+0000 7f0a2e339640 1 -- 172.21.15.78:0/1463942844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a28101ee0 con 0x7f0a281022a0 2024-08-18T23:41:24.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.377+0000 7f0a257fa640 1 -- 172.21.15.78:0/1463942844 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0a1c042070 con 0x7f0a281022a0 2024-08-18T23:41:24.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.377+0000 7f0a257fa640 1 --2- 172.21.15.78:0/1463942844 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0a100a2b80 0x7f0a100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:24.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.377+0000 7f0a27fff640 1 --2- 172.21.15.78:0/1463942844 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0a100a2b80 0x7f0a100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:24.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.377+0000 7f0a257fa640 1 -- 172.21.15.78:0/1463942844 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f0a1c0ef7c0 con 0x7f0a281022a0 2024-08-18T23:41:24.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.378+0000 7f0a27fff640 1 --2- 172.21.15.78:0/1463942844 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0a100a2b80 0x7f0a100a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0a18009fc0 tx=0x7f0a18005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:24.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.381+0000 7f0a257fa640 1 -- 172.21.15.78:0/1463942844 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0a1c031de0 con 0x7f0a281022a0 2024-08-18T23:41:24.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:24 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] up:boot 2024-08-18T23:41:24.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:24 smithi123 ceph-mon[21808]: fsmap 12 up:standby 2024-08-18T23:41:24.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:24 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-18T23:41:24.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.588+0000 7f0a2e339640 1 -- 172.21.15.78:0/1463942844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a28108dc0 con 0x7f0a281022a0 2024-08-18T23:41:24.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.589+0000 7f0a257fa640 1 -- 172.21.15.78:0/1463942844 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10539 (secure 0 0 0) 0x7f0a1c0bc570 con 0x7f0a281022a0 2024-08-18T23:41:24.590 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:41:24.590 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":13,"btime":"2024-08-18T23:41:23:157778+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24452,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-08-18T23:41:24.590 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 13 2024-08-18T23:41:24.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.592+0000 7f0a2e339640 1 -- 172.21.15.78:0/1463942844 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0a100a2b80 msgr2=0x7f0a100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:24.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.592+0000 7f0a2e339640 1 --2- 172.21.15.78:0/1463942844 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0a100a2b80 0x7f0a100a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0a18009fc0 tx=0x7f0a18005f90 comp rx=0 tx=0).stop 2024-08-18T23:41:24.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.592+0000 7f0a2e339640 1 -- 172.21.15.78:0/1463942844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281022a0 msgr2=0x7f0a28196590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:24.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.592+0000 7f0a2e339640 1 --2- 172.21.15.78:0/1463942844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281022a0 0x7f0a28196590 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c006010 tx=0x7f0a1c003c10 comp rx=0 tx=0).stop 2024-08-18T23:41:24.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.592+0000 7f0a2e339640 1 -- 172.21.15.78:0/1463942844 shutdown_connections 2024-08-18T23:41:24.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.592+0000 7f0a2e339640 1 --2- 172.21.15.78:0/1463942844 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0a100a2b80 0x7f0a100a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:24.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.592+0000 7f0a2e339640 1 --2- 172.21.15.78:0/1463942844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a28102c20 0x7f0a2819a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:24.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.592+0000 7f0a2e339640 1 --2- 172.21.15.78:0/1463942844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a281022a0 0x7f0a28196590 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:24.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.592+0000 7f0a2e339640 1 --2- 172.21.15.78:0/1463942844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a28100b50 0x7f0a28196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:24.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.592+0000 7f0a2e339640 1 -- 172.21.15.78:0/1463942844 >> 172.21.15.78:0/1463942844 conn(0x7f0a280fc500 msgr2=0x7f0a280fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.592+0000 7f0a2e339640 1 -- 172.21.15.78:0/1463942844 shutdown_connections 2024-08-18T23:41:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:24.592+0000 7f0a2e339640 1 -- 172.21.15.78:0/1463942844 wait complete. 2024-08-18T23:41:25.366 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-08-18T23:41:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:25 smithi078 ceph-mon[18440]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:25 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:25 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:25 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:25 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:25 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1463942844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:25.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:25 smithi123 ceph-mon[21808]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:25.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:25 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:25.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:25 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:25.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:25 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:25.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:25 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:25.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:25 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1463942844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:25.671 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:25.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:25 smithi161 ceph-mon[22027]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:25.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:25 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:25.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:25 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:25.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:25 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:25.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:25 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:25.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:25 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1463942844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:26.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.162+0000 7f49d0360640 1 -- 172.21.15.78:0/3217324434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49c8102a60 msgr2=0x7f49c8102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:26.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.162+0000 7f49d0360640 1 --2- 172.21.15.78:0/3217324434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49c8102a60 0x7f49c8102e40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009930 tx=0x7f49c402a420 comp rx=0 tx=0).stop 2024-08-18T23:41:26.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.162+0000 7f49d0360640 1 -- 172.21.15.78:0/3217324434 shutdown_connections 2024-08-18T23:41:26.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.162+0000 7f49d0360640 1 --2- 172.21.15.78:0/3217324434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c8107fd0 0x7f49c810e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:26.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.162+0000 7f49d0360640 1 --2- 172.21.15.78:0/3217324434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49c8103410 0x7f49c81078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:26.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.162+0000 7f49d0360640 1 --2- 172.21.15.78:0/3217324434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49c8102a60 0x7f49c8102e40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:26.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.162+0000 7f49d0360640 1 -- 172.21.15.78:0/3217324434 >> 172.21.15.78:0/3217324434 conn(0x7f49c80fc500 msgr2=0x7f49c80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:26.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.162+0000 7f49d0360640 1 -- 172.21.15.78:0/3217324434 shutdown_connections 2024-08-18T23:41:26.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.162+0000 7f49d0360640 1 -- 172.21.15.78:0/3217324434 wait complete. 2024-08-18T23:41:26.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.163+0000 7f49d0360640 1 Processor -- start 2024-08-18T23:41:26.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.163+0000 7f49d0360640 1 -- start start 2024-08-18T23:41:26.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.163+0000 7f49d0360640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49c8102a60 0x7f49c81982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:26.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.163+0000 7f49d0360640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c8103410 0x7f49c8198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:26.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.163+0000 7f49d0360640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49c8107fd0 0x7f49c819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:26.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.163+0000 7f49d0360640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f49c8111dd0 con 0x7f49c8102a60 2024-08-18T23:41:26.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.163+0000 7f49d0360640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f49c8111c50 con 0x7f49c8107fd0 2024-08-18T23:41:26.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.163+0000 7f49d0360640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f49c8111f50 con 0x7f49c8103410 2024-08-18T23:41:26.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.163+0000 7f49ce0fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49c8102a60 0x7f49c81982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:26.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.163+0000 7f49ce0fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49c8102a60 0x7f49c81982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50936/0 (socket says 172.21.15.78:50936) 2024-08-18T23:41:26.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.163+0000 7f49ce0fc640 1 -- 172.21.15.78:0/4252121581 learned_addr learned my addr 172.21.15.78:0/4252121581 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:26.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.163+0000 7f49ce8fd640 1 --2- 172.21.15.78:0/4252121581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49c8107fd0 0x7f49c819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:26.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.163+0000 7f49cd8fb640 1 --2- 172.21.15.78:0/4252121581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c8103410 0x7f49c8198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:26.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.164+0000 7f49ce0fc640 1 -- 172.21.15.78:0/4252121581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c8103410 msgr2=0x7f49c8198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:26.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.164+0000 7f49ce0fc640 1 --2- 172.21.15.78:0/4252121581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c8103410 0x7f49c8198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:26.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.164+0000 7f49ce0fc640 1 -- 172.21.15.78:0/4252121581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49c8107fd0 msgr2=0x7f49c819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:26.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.164+0000 7f49ce0fc640 1 --2- 172.21.15.78:0/4252121581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49c8107fd0 0x7f49c819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:26.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.164+0000 7f49ce0fc640 1 -- 172.21.15.78:0/4252121581 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c819d280 con 0x7f49c8102a60 2024-08-18T23:41:26.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.164+0000 7f49ce0fc640 1 --2- 172.21.15.78:0/4252121581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49c8102a60 0x7f49c81982d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009230 tx=0x7f49c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:26.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.164+0000 7f49bb7fe640 1 -- 172.21.15.78:0/4252121581 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c403f030 con 0x7f49c8102a60 2024-08-18T23:41:26.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.164+0000 7f49bb7fe640 1 -- 172.21.15.78:0/4252121581 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c4003ad0 con 0x7f49c8102a60 2024-08-18T23:41:26.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.164+0000 7f49d0360640 1 -- 172.21.15.78:0/4252121581 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c819d510 con 0x7f49c8102a60 2024-08-18T23:41:26.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.164+0000 7f49bb7fe640 1 -- 172.21.15.78:0/4252121581 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c4035070 con 0x7f49c8102a60 2024-08-18T23:41:26.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.164+0000 7f49d0360640 1 -- 172.21.15.78:0/4252121581 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c81a4e50 con 0x7f49c8102a60 2024-08-18T23:41:26.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.165+0000 7f49d0360640 1 -- 172.21.15.78:0/4252121581 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c8102e40 con 0x7f49c8102a60 2024-08-18T23:41:26.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.166+0000 7f49bb7fe640 1 -- 172.21.15.78:0/4252121581 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f49c4042030 con 0x7f49c8102a60 2024-08-18T23:41:26.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.167+0000 7f49bb7fe640 1 --2- 172.21.15.78:0/4252121581 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f49b40a2c00 0x7f49b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:26.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.167+0000 7f49cd8fb640 1 --2- 172.21.15.78:0/4252121581 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f49b40a2c00 0x7f49b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:26.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.167+0000 7f49bb7fe640 1 -- 172.21.15.78:0/4252121581 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f49c40eef40 con 0x7f49c8102a60 2024-08-18T23:41:26.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.167+0000 7f49cd8fb640 1 --2- 172.21.15.78:0/4252121581 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f49b40a2c00 0x7f49b40a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f49c80feb80 tx=0x7f49bc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:26.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.171+0000 7f49bb7fe640 1 -- 172.21.15.78:0/4252121581 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f49c40bbcf0 con 0x7f49c8102a60 2024-08-18T23:41:26.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.364+0000 7f49d0360640 1 -- 172.21.15.78:0/4252121581 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f49c8109fe0 con 0x7f49c8102a60 2024-08-18T23:41:26.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.364+0000 7f49bb7fe640 1 -- 172.21.15.78:0/4252121581 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7f49c40bfff0 con 0x7f49c8102a60 2024-08-18T23:41:26.365 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:41:26.366 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-08-18T23:41:26.366 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:41:26.366 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2024-08-18T23:41:26.366 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2024-08-18T23:41:26.366 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:41:26.366 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-08-18T23:41:26.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.367+0000 7f49d0360640 1 -- 172.21.15.78:0/4252121581 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f49b40a2c00 msgr2=0x7f49b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:26.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.367+0000 7f49d0360640 1 --2- 172.21.15.78:0/4252121581 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f49b40a2c00 0x7f49b40a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f49c80feb80 tx=0x7f49bc00b390 comp rx=0 tx=0).stop 2024-08-18T23:41:26.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.367+0000 7f49d0360640 1 -- 172.21.15.78:0/4252121581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49c8102a60 msgr2=0x7f49c81982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:26.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.367+0000 7f49d0360640 1 --2- 172.21.15.78:0/4252121581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49c8102a60 0x7f49c81982d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009230 tx=0x7f49c4004270 comp rx=0 tx=0).stop 2024-08-18T23:41:26.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.367+0000 7f49d0360640 1 -- 172.21.15.78:0/4252121581 shutdown_connections 2024-08-18T23:41:26.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.367+0000 7f49d0360640 1 --2- 172.21.15.78:0/4252121581 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f49b40a2c00 0x7f49b40a50c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.367+0000 7f49d0360640 1 --2- 172.21.15.78:0/4252121581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49c8107fd0 0x7f49c819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.367+0000 7f49d0360640 1 --2- 172.21.15.78:0/4252121581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49c8103410 0x7f49c8198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.367+0000 7f49d0360640 1 --2- 172.21.15.78:0/4252121581 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49c8102a60 0x7f49c81982d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.367+0000 7f49d0360640 1 -- 172.21.15.78:0/4252121581 >> 172.21.15.78:0/4252121581 conn(0x7f49c80fc500 msgr2=0x7f49c8105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.367+0000 7f49d0360640 1 -- 172.21.15.78:0/4252121581 shutdown_connections 2024-08-18T23:41:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:26.367+0000 7f49d0360640 1 -- 172.21.15.78:0/4252121581 wait complete. 2024-08-18T23:41:26.684 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:26.684 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:26.684 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:26.684 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:41:26.685 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:26.685 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:26 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:41:26.685 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:26 smithi123 ceph-mon[21808]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:26.685 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:26 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/4252121581' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-08-18T23:41:26.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:26.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:26.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:26.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:41:26.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:26 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:41:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:26 smithi078 ceph-mon[18440]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:26 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/4252121581' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-08-18T23:41:26.758 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:26.758 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:26.758 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:26.758 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:41:26.759 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:26.759 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:26 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:41:26.759 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:26 smithi161 ceph-mon[22027]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:26.759 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:26 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/4252121581' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-08-18T23:41:27.369 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-18T23:41:27.369 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-18T23:41:27.960 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:28.403 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:28 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:28.403 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:28 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:28.403 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:28 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:28.403 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:28 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:28.403 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:28 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:28.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:28 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:28.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:28 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:28.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:28 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:28.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:28 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:28.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:28 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:28.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.485+0000 7f6c2d244640 1 -- 172.21.15.78:0/354920863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c28101150 msgr2=0x7f6c2810d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:28.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.485+0000 7f6c2d244640 1 --2- 172.21.15.78:0/354920863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c28101150 0x7f6c2810d680 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009cd0 tx=0x7f6c2002a8e0 comp rx=0 tx=0).stop 2024-08-18T23:41:28.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.485+0000 7f6c2d244640 1 -- 172.21.15.78:0/354920863 shutdown_connections 2024-08-18T23:41:28.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.485+0000 7f6c2d244640 1 --2- 172.21.15.78:0/354920863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c28101150 0x7f6c2810d680 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:28.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.485+0000 7f6c2d244640 1 --2- 172.21.15.78:0/354920863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c281007b0 0x7f6c28100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:28.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.485+0000 7f6c2d244640 1 --2- 172.21.15.78:0/354920863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c281067b0 0x7f6c28106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:28.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.485+0000 7f6c2d244640 1 -- 172.21.15.78:0/354920863 >> 172.21.15.78:0/354920863 conn(0x7f6c280fc4d0 msgr2=0x7f6c280fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:28.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.485+0000 7f6c2d244640 1 -- 172.21.15.78:0/354920863 shutdown_connections 2024-08-18T23:41:28.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.485+0000 7f6c2d244640 1 -- 172.21.15.78:0/354920863 wait complete. 2024-08-18T23:41:28.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.485+0000 7f6c2d244640 1 Processor -- start 2024-08-18T23:41:28.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c2d244640 1 -- start start 2024-08-18T23:41:28.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c2d244640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c281007b0 0x7f6c28199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:28.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c2d244640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c28101150 0x7f6c28199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:28.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c2d244640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c281067b0 0x7f6c281a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:28.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c2d244640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c28110bf0 con 0x7f6c281067b0 2024-08-18T23:41:28.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c2d244640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c28110a70 con 0x7f6c281007b0 2024-08-18T23:41:28.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c2d244640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6c28110d70 con 0x7f6c28101150 2024-08-18T23:41:28.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c2759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c281067b0 0x7f6c281a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:28.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c2759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c281067b0 0x7f6c281a40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50952/0 (socket says 172.21.15.78:50952) 2024-08-18T23:41:28.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c2759e640 1 -- 172.21.15.78:0/476122172 learned_addr learned my addr 172.21.15.78:0/476122172 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:28.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c2659c640 1 --2- 172.21.15.78:0/476122172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c28101150 0x7f6c28199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:28.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c26d9d640 1 --2- 172.21.15.78:0/476122172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c281007b0 0x7f6c28199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:28.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c2759e640 1 -- 172.21.15.78:0/476122172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c28101150 msgr2=0x7f6c28199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:28.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c2759e640 1 --2- 172.21.15.78:0/476122172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c28101150 0x7f6c28199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:28.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c2759e640 1 -- 172.21.15.78:0/476122172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c281007b0 msgr2=0x7f6c28199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:28.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c2759e640 1 --2- 172.21.15.78:0/476122172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c281007b0 0x7f6c28199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:28.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c2759e640 1 -- 172.21.15.78:0/476122172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c281a4600 con 0x7f6c281067b0 2024-08-18T23:41:28.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c26d9d640 1 --2- 172.21.15.78:0/476122172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c281007b0 0x7f6c28199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:41:28.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c2759e640 1 --2- 172.21.15.78:0/476122172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c281067b0 0x7f6c281a40c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6c20005cc0 tx=0x7f6c20003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:28.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.486+0000 7f6c0ffff640 1 -- 172.21.15.78:0/476122172 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c20004040 con 0x7f6c281067b0 2024-08-18T23:41:28.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.487+0000 7f6c2d244640 1 -- 172.21.15.78:0/476122172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c281a47d0 con 0x7f6c281067b0 2024-08-18T23:41:28.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.487+0000 7f6c0ffff640 1 -- 172.21.15.78:0/476122172 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c200046c0 con 0x7f6c281067b0 2024-08-18T23:41:28.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.487+0000 7f6c0ffff640 1 -- 172.21.15.78:0/476122172 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c20046480 con 0x7f6c281067b0 2024-08-18T23:41:28.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.487+0000 7f6c2d244640 1 -- 172.21.15.78:0/476122172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c281a4cb0 con 0x7f6c281067b0 2024-08-18T23:41:28.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.488+0000 7f6c2d244640 1 -- 172.21.15.78:0/476122172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c28100c10 con 0x7f6c281067b0 2024-08-18T23:41:28.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.489+0000 7f6c0ffff640 1 -- 172.21.15.78:0/476122172 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6c200305a0 con 0x7f6c281067b0 2024-08-18T23:41:28.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.490+0000 7f6c0ffff640 1 --2- 172.21.15.78:0/476122172 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6c100a2b80 0x7f6c100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:28.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.490+0000 7f6c26d9d640 1 --2- 172.21.15.78:0/476122172 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6c100a2b80 0x7f6c100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:28.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.490+0000 7f6c0ffff640 1 -- 172.21.15.78:0/476122172 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f6c2002e030 con 0x7f6c281067b0 2024-08-18T23:41:28.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.490+0000 7f6c26d9d640 1 --2- 172.21.15.78:0/476122172 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6c100a2b80 0x7f6c100a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6c18006fd0 tx=0x7f6c18008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:28.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.494+0000 7f6c0ffff640 1 -- 172.21.15.78:0/476122172 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6c2004b060 con 0x7f6c281067b0 2024-08-18T23:41:28.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:28 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:28.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:28 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:28.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:28 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:28.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:28 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:28.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:28 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:28.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:28.675+0000 7f6c2d244640 1 -- 172.21.15.78:0/476122172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6c28108d40 con 0x7f6c281067b0 2024-08-18T23:41:29.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:29.106+0000 7f6c0ffff640 1 -- 172.21.15.78:0/476122172 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7f6c200bd870 con 0x7f6c281067b0 2024-08-18T23:41:29.107 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2024-08-18T23:41:29.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:29.108+0000 7f6c2d244640 1 -- 172.21.15.78:0/476122172 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6c100a2b80 msgr2=0x7f6c100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:29.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:29.108+0000 7f6c2d244640 1 --2- 172.21.15.78:0/476122172 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6c100a2b80 0x7f6c100a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6c18006fd0 tx=0x7f6c18008040 comp rx=0 tx=0).stop 2024-08-18T23:41:29.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:29.108+0000 7f6c2d244640 1 -- 172.21.15.78:0/476122172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c281067b0 msgr2=0x7f6c281a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:29.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:29.108+0000 7f6c2d244640 1 --2- 172.21.15.78:0/476122172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c281067b0 0x7f6c281a40c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6c20005cc0 tx=0x7f6c20003970 comp rx=0 tx=0).stop 2024-08-18T23:41:29.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:29.108+0000 7f6c2d244640 1 -- 172.21.15.78:0/476122172 shutdown_connections 2024-08-18T23:41:29.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:29.108+0000 7f6c2d244640 1 --2- 172.21.15.78:0/476122172 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6c100a2b80 0x7f6c100a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:29.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:29.108+0000 7f6c2d244640 1 --2- 172.21.15.78:0/476122172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c281067b0 0x7f6c281a40c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:29.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:29.108+0000 7f6c2d244640 1 --2- 172.21.15.78:0/476122172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c28101150 0x7f6c28199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:29.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:29.108+0000 7f6c2d244640 1 --2- 172.21.15.78:0/476122172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c281007b0 0x7f6c28199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:29.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:29.108+0000 7f6c2d244640 1 -- 172.21.15.78:0/476122172 >> 172.21.15.78:0/476122172 conn(0x7f6c280fc4d0 msgr2=0x7f6c280fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:29.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:29.109+0000 7f6c2d244640 1 -- 172.21.15.78:0/476122172 shutdown_connections 2024-08-18T23:41:29.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:29.109+0000 7f6c2d244640 1 -- 172.21.15.78:0/476122172 wait complete. 2024-08-18T23:41:29.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:29 smithi161 ceph-mon[22027]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:29.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:29 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/476122172' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-18T23:41:29.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:29 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:29.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:29 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:29.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:29 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:29.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:29 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:41:29.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:29 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:29.440 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:29 smithi078 ceph-mon[18440]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:29.440 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:29 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/476122172' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-18T23:41:29.440 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:29 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:29.440 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:29 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:29.440 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:29 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:29.440 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:29 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:41:29.441 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:29 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:29.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:29 smithi123 ceph-mon[21808]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:29.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:29 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/476122172' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-18T23:41:29.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:29 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:29.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:29 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:29.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:29 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:41:29.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:29 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:41:29.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:29 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:41:29.886 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-08-18T23:41:30.116 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:30 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/476122172' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-18T23:41:30.116 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:30 smithi078 ceph-mon[18440]: osdmap e78: 12 total, 12 up, 12 in 2024-08-18T23:41:30.142 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:30 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/476122172' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-18T23:41:30.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:30 smithi161 ceph-mon[22027]: osdmap e78: 12 total, 12 up, 12 in 2024-08-18T23:41:30.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:30 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/476122172' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-18T23:41:30.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:30 smithi123 ceph-mon[21808]: osdmap e78: 12 total, 12 up, 12 in 2024-08-18T23:41:30.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.613+0000 7fa505109640 1 -- 172.21.15.78:0/1494438476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5001007b0 msgr2=0x7fa500100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:30.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.613+0000 7fa505109640 1 --2- 172.21.15.78:0/1494438476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5001007b0 0x7fa500100c10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8009930 tx=0x7fa4e802a420 comp rx=0 tx=0).stop 2024-08-18T23:41:30.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.613+0000 7fa505109640 1 -- 172.21.15.78:0/1494438476 shutdown_connections 2024-08-18T23:41:30.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.613+0000 7fa505109640 1 --2- 172.21.15.78:0/1494438476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa500101150 0x7fa50010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:30.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.613+0000 7fa505109640 1 --2- 172.21.15.78:0/1494438476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5001007b0 0x7fa500100c10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:30.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.613+0000 7fa505109640 1 --2- 172.21.15.78:0/1494438476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5001067b0 0x7fa500106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:30.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.613+0000 7fa505109640 1 -- 172.21.15.78:0/1494438476 >> 172.21.15.78:0/1494438476 conn(0x7fa5000fc4d0 msgr2=0x7fa5000fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:30.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.613+0000 7fa505109640 1 -- 172.21.15.78:0/1494438476 shutdown_connections 2024-08-18T23:41:30.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.613+0000 7fa505109640 1 -- 172.21.15.78:0/1494438476 wait complete. 2024-08-18T23:41:30.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.614+0000 7fa505109640 1 Processor -- start 2024-08-18T23:41:30.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.614+0000 7fa505109640 1 -- start start 2024-08-18T23:41:30.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.614+0000 7fa505109640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5001007b0 0x7fa500197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:30.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.614+0000 7fa505109640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa500101150 0x7fa5001975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:30.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.614+0000 7fa505109640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5001067b0 0x7fa50019b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:30.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.614+0000 7fa505109640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa500110bd0 con 0x7fa5001007b0 2024-08-18T23:41:30.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.614+0000 7fa505109640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa500110a50 con 0x7fa500101150 2024-08-18T23:41:30.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.614+0000 7fa505109640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa500110d50 con 0x7fa5001067b0 2024-08-18T23:41:30.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.614+0000 7fa4fed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5001007b0 0x7fa500197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:30.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.615+0000 7fa4fed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5001007b0 0x7fa500197090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50968/0 (socket says 172.21.15.78:50968) 2024-08-18T23:41:30.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.615+0000 7fa4fed9d640 1 -- 172.21.15.78:0/4062649226 learned_addr learned my addr 172.21.15.78:0/4062649226 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:30.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.615+0000 7fa4f659c640 1 --2- 172.21.15.78:0/4062649226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa500101150 0x7fa5001975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:30.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.615+0000 7fa4ff59e640 1 --2- 172.21.15.78:0/4062649226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5001067b0 0x7fa50019b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:30.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.615+0000 7fa4fed9d640 1 -- 172.21.15.78:0/4062649226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5001067b0 msgr2=0x7fa50019b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:30.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.615+0000 7fa4fed9d640 1 --2- 172.21.15.78:0/4062649226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5001067b0 0x7fa50019b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:30.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.615+0000 7fa4fed9d640 1 -- 172.21.15.78:0/4062649226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa500101150 msgr2=0x7fa5001975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:30.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.615+0000 7fa4fed9d640 1 --2- 172.21.15.78:0/4062649226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa500101150 0x7fa5001975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:30.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.615+0000 7fa4fed9d640 1 -- 172.21.15.78:0/4062649226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50019c040 con 0x7fa5001007b0 2024-08-18T23:41:30.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.615+0000 7fa4ff59e640 1 --2- 172.21.15.78:0/4062649226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5001067b0 0x7fa50019b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:41:30.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.615+0000 7fa4fed9d640 1 --2- 172.21.15.78:0/4062649226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5001007b0 0x7fa500197090 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000c920 tx=0x7fa4f000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:30.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.615+0000 7fa4fcd99640 1 -- 172.21.15.78:0/4062649226 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0007c70 con 0x7fa5001007b0 2024-08-18T23:41:30.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.615+0000 7fa505109640 1 -- 172.21.15.78:0/4062649226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa50019c330 con 0x7fa5001007b0 2024-08-18T23:41:30.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.615+0000 7fa4fcd99640 1 -- 172.21.15.78:0/4062649226 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f0007e10 con 0x7fa5001007b0 2024-08-18T23:41:30.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.615+0000 7fa4fcd99640 1 -- 172.21.15.78:0/4062649226 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0018890 con 0x7fa5001007b0 2024-08-18T23:41:30.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.615+0000 7fa505109640 1 -- 172.21.15.78:0/4062649226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa50006a4c0 con 0x7fa5001007b0 2024-08-18T23:41:30.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.616+0000 7fa505109640 1 -- 172.21.15.78:0/4062649226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa500101ef0 con 0x7fa5001007b0 2024-08-18T23:41:30.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.617+0000 7fa4fcd99640 1 -- 172.21.15.78:0/4062649226 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa4f0018a50 con 0x7fa5001007b0 2024-08-18T23:41:30.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.618+0000 7fa4fcd99640 1 --2- 172.21.15.78:0/4062649226 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa4e00a2b80 0x7fa4e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:30.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.618+0000 7fa4f659c640 1 --2- 172.21.15.78:0/4062649226 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa4e00a2b80 0x7fa4e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:30.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.619+0000 7fa4fcd99640 1 -- 172.21.15.78:0/4062649226 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7497+0+0 (secure 0 0 0) 0x7fa4f00cf190 con 0x7fa5001007b0 2024-08-18T23:41:30.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.619+0000 7fa4f659c640 1 --2- 172.21.15.78:0/4062649226 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa4e00a2b80 0x7fa4e00a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8009900 tx=0x7fa4e80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:30.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.622+0000 7fa4fcd99640 1 -- 172.21.15.78:0/4062649226 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa4f0014030 con 0x7fa5001007b0 2024-08-18T23:41:30.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:30.789+0000 7fa505109640 1 -- 172.21.15.78:0/4062649226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fa500100c10 con 0x7fa5001007b0 2024-08-18T23:41:31.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:31.115+0000 7fa4fcd99640 1 -- 172.21.15.78:0/4062649226 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v80) ==== 171+0+0 (secure 0 0 0) 0x7fa4f009c410 con 0x7fa5001007b0 2024-08-18T23:41:31.117 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2024-08-18T23:41:31.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:31.117+0000 7fa505109640 1 -- 172.21.15.78:0/4062649226 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa4e00a2b80 msgr2=0x7fa4e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:31.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:31.118+0000 7fa505109640 1 --2- 172.21.15.78:0/4062649226 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa4e00a2b80 0x7fa4e00a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8009900 tx=0x7fa4e80058d0 comp rx=0 tx=0).stop 2024-08-18T23:41:31.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:31.118+0000 7fa505109640 1 -- 172.21.15.78:0/4062649226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5001007b0 msgr2=0x7fa500197090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:31.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:31.118+0000 7fa505109640 1 --2- 172.21.15.78:0/4062649226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5001007b0 0x7fa500197090 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000c920 tx=0x7fa4f000cde0 comp rx=0 tx=0).stop 2024-08-18T23:41:31.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:31.118+0000 7fa505109640 1 -- 172.21.15.78:0/4062649226 shutdown_connections 2024-08-18T23:41:31.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:31.118+0000 7fa505109640 1 --2- 172.21.15.78:0/4062649226 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa4e00a2b80 0x7fa4e00a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:31.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:31.118+0000 7fa505109640 1 --2- 172.21.15.78:0/4062649226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5001067b0 0x7fa50019b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:31.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:31.118+0000 7fa505109640 1 --2- 172.21.15.78:0/4062649226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa500101150 0x7fa5001975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:31.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:31.118+0000 7fa505109640 1 --2- 172.21.15.78:0/4062649226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5001007b0 0x7fa500197090 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:31.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:31.118+0000 7fa505109640 1 -- 172.21.15.78:0/4062649226 >> 172.21.15.78:0/4062649226 conn(0x7fa5000fc4d0 msgr2=0x7fa5000fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:31.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:31.118+0000 7fa505109640 1 -- 172.21.15.78:0/4062649226 shutdown_connections 2024-08-18T23:41:31.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:31.118+0000 7fa505109640 1 -- 172.21.15.78:0/4062649226 wait complete. 2024-08-18T23:41:31.408 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:31 smithi078 ceph-mon[18440]: pgmap v252: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:31.408 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:31 smithi078 ceph-mon[18440]: osdmap e79: 12 total, 12 up, 12 in 2024-08-18T23:41:31.408 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:31 smithi078 ceph-mon[18440]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-18T23:41:31.408 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:31 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/4062649226' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-08-18T23:41:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:31 smithi161 ceph-mon[22027]: pgmap v252: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:31 smithi161 ceph-mon[22027]: osdmap e79: 12 total, 12 up, 12 in 2024-08-18T23:41:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:31 smithi161 ceph-mon[22027]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-18T23:41:31.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:31 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/4062649226' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-08-18T23:41:31.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:31 smithi123 ceph-mon[21808]: pgmap v252: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:31.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:31 smithi123 ceph-mon[21808]: osdmap e79: 12 total, 12 up, 12 in 2024-08-18T23:41:31.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:31 smithi123 ceph-mon[21808]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-18T23:41:31.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:31 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/4062649226' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-08-18T23:41:31.979 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-18T23:41:32.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:32 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/4062649226' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-08-18T23:41:32.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:32 smithi078 ceph-mon[18440]: osdmap e80: 12 total, 12 up, 12 in 2024-08-18T23:41:32.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:32 smithi078 ceph-mon[18440]: osdmap e81: 12 total, 12 up, 12 in 2024-08-18T23:41:32.255 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:32.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:32 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/4062649226' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-08-18T23:41:32.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:32 smithi161 ceph-mon[22027]: osdmap e80: 12 total, 12 up, 12 in 2024-08-18T23:41:32.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:32 smithi161 ceph-mon[22027]: osdmap e81: 12 total, 12 up, 12 in 2024-08-18T23:41:32.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:32 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/4062649226' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-08-18T23:41:32.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:32 smithi123 ceph-mon[21808]: osdmap e80: 12 total, 12 up, 12 in 2024-08-18T23:41:32.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:32 smithi123 ceph-mon[21808]: osdmap e81: 12 total, 12 up, 12 in 2024-08-18T23:41:32.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.727+0000 7f2bb87f5640 1 -- 172.21.15.78:0/304591935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bb0101150 msgr2=0x7f2bb010d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:32.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.727+0000 7f2bb87f5640 1 --2- 172.21.15.78:0/304591935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bb0101150 0x7f2bb010d680 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009cd0 tx=0x7f2ba802a8e0 comp rx=0 tx=0).stop 2024-08-18T23:41:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.727+0000 7f2bb87f5640 1 -- 172.21.15.78:0/304591935 shutdown_connections 2024-08-18T23:41:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.727+0000 7f2bb87f5640 1 --2- 172.21.15.78:0/304591935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bb0101150 0x7f2bb010d680 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.727+0000 7f2bb87f5640 1 --2- 172.21.15.78:0/304591935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bb01007b0 0x7f2bb0100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.727+0000 7f2bb87f5640 1 --2- 172.21.15.78:0/304591935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb01067b0 0x7f2bb0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.727+0000 7f2bb87f5640 1 -- 172.21.15.78:0/304591935 >> 172.21.15.78:0/304591935 conn(0x7f2bb00fc4d0 msgr2=0x7f2bb00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.727+0000 7f2bb87f5640 1 -- 172.21.15.78:0/304591935 shutdown_connections 2024-08-18T23:41:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.727+0000 7f2bb87f5640 1 -- 172.21.15.78:0/304591935 wait complete. 2024-08-18T23:41:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.728+0000 7f2bb87f5640 1 Processor -- start 2024-08-18T23:41:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.728+0000 7f2bb87f5640 1 -- start start 2024-08-18T23:41:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.728+0000 7f2bb87f5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bb01007b0 0x7f2bb01970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.728+0000 7f2bb87f5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb0101150 0x7f2bb01975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:32.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.728+0000 7f2bb87f5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bb01067b0 0x7f2bb019b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:32.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.728+0000 7f2bb87f5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb0110bd0 con 0x7f2bb01007b0 2024-08-18T23:41:32.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.728+0000 7f2bb87f5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb0110a50 con 0x7f2bb0101150 2024-08-18T23:41:32.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.728+0000 7f2bb87f5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb0110d50 con 0x7f2bb01067b0 2024-08-18T23:41:32.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.728+0000 7f2bb6591640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bb01007b0 0x7f2bb01970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:32.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.728+0000 7f2bb6591640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bb01007b0 0x7f2bb01970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50996/0 (socket says 172.21.15.78:50996) 2024-08-18T23:41:32.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.728+0000 7f2bb6591640 1 -- 172.21.15.78:0/3018625443 learned_addr learned my addr 172.21.15.78:0/3018625443 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:32.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.729+0000 7f2bb6d92640 1 --2- 172.21.15.78:0/3018625443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bb01067b0 0x7f2bb019b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:32.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.729+0000 7f2bb5d90640 1 --2- 172.21.15.78:0/3018625443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb0101150 0x7f2bb01975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:32.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.729+0000 7f2bb6591640 1 -- 172.21.15.78:0/3018625443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bb01067b0 msgr2=0x7f2bb019b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:32.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.729+0000 7f2bb6591640 1 --2- 172.21.15.78:0/3018625443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bb01067b0 0x7f2bb019b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:32.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.729+0000 7f2bb6591640 1 -- 172.21.15.78:0/3018625443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb0101150 msgr2=0x7f2bb01975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:32.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.729+0000 7f2bb6591640 1 --2- 172.21.15.78:0/3018625443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb0101150 0x7f2bb01975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:32.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.729+0000 7f2bb6591640 1 -- 172.21.15.78:0/3018625443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb019c060 con 0x7f2bb01007b0 2024-08-18T23:41:32.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.729+0000 7f2bb6d92640 1 --2- 172.21.15.78:0/3018625443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bb01067b0 0x7f2bb019b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:41:32.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.729+0000 7f2bb5d90640 1 --2- 172.21.15.78:0/3018625443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb0101150 0x7f2bb01975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:41:32.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.729+0000 7f2bb6591640 1 --2- 172.21.15.78:0/3018625443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bb01007b0 0x7f2bb01970b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00d950 tx=0x7f2bac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:32.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.729+0000 7f2ba77fe640 1 -- 172.21.15.78:0/3018625443 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac017070 con 0x7f2bb01007b0 2024-08-18T23:41:32.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.729+0000 7f2ba77fe640 1 -- 172.21.15.78:0/3018625443 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bac009c30 con 0x7f2bb01007b0 2024-08-18T23:41:32.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.729+0000 7f2bb87f5640 1 -- 172.21.15.78:0/3018625443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb019c350 con 0x7f2bb01007b0 2024-08-18T23:41:32.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.729+0000 7f2ba77fe640 1 -- 172.21.15.78:0/3018625443 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00f6b0 con 0x7f2bb01007b0 2024-08-18T23:41:32.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.729+0000 7f2bb87f5640 1 -- 172.21.15.78:0/3018625443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb01a3c30 con 0x7f2bb01007b0 2024-08-18T23:41:32.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.730+0000 7f2bb87f5640 1 -- 172.21.15.78:0/3018625443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb0100c10 con 0x7f2bb01007b0 2024-08-18T23:41:32.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.731+0000 7f2ba77fe640 1 -- 172.21.15.78:0/3018625443 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2bac0109d0 con 0x7f2bb01007b0 2024-08-18T23:41:32.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.732+0000 7f2ba77fe640 1 --2- 172.21.15.78:0/3018625443 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2b9c0a2b80 0x7f2b9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:32.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.732+0000 7f2bb5d90640 1 --2- 172.21.15.78:0/3018625443 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2b9c0a2b80 0x7f2b9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:32.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.732+0000 7f2ba77fe640 1 -- 172.21.15.78:0/3018625443 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7879+0+0 (secure 0 0 0) 0x7f2bac0cebd0 con 0x7f2bb01007b0 2024-08-18T23:41:32.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.733+0000 7f2bb5d90640 1 --2- 172.21.15.78:0/3018625443 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2b9c0a2b80 0x7f2b9c0a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2bb00feaa0 tx=0x7f2ba0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:32.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.737+0000 7f2ba77fe640 1 -- 172.21.15.78:0/3018625443 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2bac009de0 con 0x7f2bb01007b0 2024-08-18T23:41:32.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:32.940+0000 7f2bb87f5640 1 -- 172.21.15.78:0/3018625443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2bb002d5e0 con 0x7f2bb01007b0 2024-08-18T23:41:33.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:33.139+0000 7f2ba77fe640 1 -- 172.21.15.78:0/3018625443 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f2bac09bce0 con 0x7f2bb01007b0 2024-08-18T23:41:33.140 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 2 and data pool 3 2024-08-18T23:41:33.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:33.141+0000 7f2bb87f5640 1 -- 172.21.15.78:0/3018625443 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2b9c0a2b80 msgr2=0x7f2b9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:33.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:33.141+0000 7f2bb87f5640 1 --2- 172.21.15.78:0/3018625443 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2b9c0a2b80 0x7f2b9c0a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2bb00feaa0 tx=0x7f2ba0008040 comp rx=0 tx=0).stop 2024-08-18T23:41:33.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:33.141+0000 7f2bb87f5640 1 -- 172.21.15.78:0/3018625443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bb01007b0 msgr2=0x7f2bb01970b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:33.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:33.141+0000 7f2bb87f5640 1 --2- 172.21.15.78:0/3018625443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bb01007b0 0x7f2bb01970b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00d950 tx=0x7f2bac00de10 comp rx=0 tx=0).stop 2024-08-18T23:41:33.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:33.141+0000 7f2bb87f5640 1 -- 172.21.15.78:0/3018625443 shutdown_connections 2024-08-18T23:41:33.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:33.141+0000 7f2bb87f5640 1 --2- 172.21.15.78:0/3018625443 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2b9c0a2b80 0x7f2b9c0a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:33.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:33.141+0000 7f2bb87f5640 1 --2- 172.21.15.78:0/3018625443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bb01067b0 0x7f2bb019b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:33.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:33.141+0000 7f2bb87f5640 1 --2- 172.21.15.78:0/3018625443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb0101150 0x7f2bb01975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:33.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:33.141+0000 7f2bb87f5640 1 --2- 172.21.15.78:0/3018625443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bb01007b0 0x7f2bb01970b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:33.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:33.141+0000 7f2bb87f5640 1 -- 172.21.15.78:0/3018625443 >> 172.21.15.78:0/3018625443 conn(0x7f2bb00fc4d0 msgr2=0x7f2bb00fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:33.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:33.141+0000 7f2bb87f5640 1 -- 172.21.15.78:0/3018625443 shutdown_connections 2024-08-18T23:41:33.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:33.142+0000 7f2bb87f5640 1 -- 172.21.15.78:0/3018625443 wait complete. 2024-08-18T23:41:33.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:33 smithi078 ceph-mon[18440]: pgmap v255: 97 pgs: 26 creating+peering, 58 unknown, 13 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:33.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:33 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3018625443' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-18T23:41:33.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:33 smithi078 ceph-mon[18440]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-18T23:41:33.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:33 smithi161 ceph-mon[22027]: pgmap v255: 97 pgs: 26 creating+peering, 58 unknown, 13 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:33.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:33 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3018625443' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-18T23:41:33.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:33 smithi161 ceph-mon[22027]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-18T23:41:33.449 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:33 smithi123 ceph-mon[21808]: pgmap v255: 97 pgs: 26 creating+peering, 58 unknown, 13 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:33.449 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:33 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3018625443' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-18T23:41:33.449 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:33 smithi123 ceph-mon[21808]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-18T23:41:33.957 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-08-18T23:41:33.957 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-08-18T23:41:34.212 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:34.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:34 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3018625443' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-18T23:41:34.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:34 smithi078 ceph-mon[18440]: osdmap e82: 12 total, 12 up, 12 in 2024-08-18T23:41:34.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:34 smithi078 ceph-mon[18440]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-18T23:41:34.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:34 smithi078 ceph-mon[18440]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-18T23:41:34.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:34 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] up:active 2024-08-18T23:41:34.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:34 smithi078 ceph-mon[18440]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-18T23:41:34.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:34 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3018625443' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-18T23:41:34.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:34 smithi161 ceph-mon[22027]: osdmap e82: 12 total, 12 up, 12 in 2024-08-18T23:41:34.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:34 smithi161 ceph-mon[22027]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-18T23:41:34.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:34 smithi161 ceph-mon[22027]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-18T23:41:34.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:34 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] up:active 2024-08-18T23:41:34.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:34 smithi161 ceph-mon[22027]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-18T23:41:34.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:34 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3018625443' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-18T23:41:34.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:34 smithi123 ceph-mon[21808]: osdmap e82: 12 total, 12 up, 12 in 2024-08-18T23:41:34.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:34 smithi123 ceph-mon[21808]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-18T23:41:34.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:34 smithi123 ceph-mon[21808]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-18T23:41:34.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:34 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] up:active 2024-08-18T23:41:34.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:34 smithi123 ceph-mon[21808]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-18T23:41:34.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.811+0000 7f21f94a5640 1 -- 172.21.15.78:0/568824682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f4102260 msgr2=0x7f21f410e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:34.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.811+0000 7f21f94a5640 1 --2- 172.21.15.78:0/568824682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f4102260 0x7f21f410e790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009930 tx=0x7f21ec02a420 comp rx=0 tx=0).stop 2024-08-18T23:41:34.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.811+0000 7f21f94a5640 1 -- 172.21.15.78:0/568824682 shutdown_connections 2024-08-18T23:41:34.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.811+0000 7f21f94a5640 1 --2- 172.21.15.78:0/568824682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f4102260 0x7f21f410e790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:34.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.811+0000 7f21f94a5640 1 --2- 172.21.15.78:0/568824682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21f41018c0 0x7f21f4101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:34.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.811+0000 7f21f94a5640 1 --2- 172.21.15.78:0/568824682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21f41078c0 0x7f21f4107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:34.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.811+0000 7f21f94a5640 1 -- 172.21.15.78:0/568824682 >> 172.21.15.78:0/568824682 conn(0x7f21f40fd620 msgr2=0x7f21f40ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:34.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.811+0000 7f21f94a5640 1 -- 172.21.15.78:0/568824682 shutdown_connections 2024-08-18T23:41:34.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.811+0000 7f21f94a5640 1 -- 172.21.15.78:0/568824682 wait complete. 2024-08-18T23:41:34.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.811+0000 7f21f94a5640 1 Processor -- start 2024-08-18T23:41:34.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.811+0000 7f21f94a5640 1 -- start start 2024-08-18T23:41:34.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f94a5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f41018c0 0x7f21f4198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:34.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f94a5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21f4102260 0x7f21f4198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:34.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f94a5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21f41078c0 0x7f21f419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:34.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f94a5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4111ce0 con 0x7f21f41018c0 2024-08-18T23:41:34.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f94a5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4111b60 con 0x7f21f4102260 2024-08-18T23:41:34.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f94a5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4111e60 con 0x7f21f41078c0 2024-08-18T23:41:34.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f41018c0 0x7f21f4198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:34.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f41018c0 0x7f21f4198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51010/0 (socket says 172.21.15.78:51010) 2024-08-18T23:41:34.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f2ffd640 1 -- 172.21.15.78:0/675704481 learned_addr learned my addr 172.21.15.78:0/675704481 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:34.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f37fe640 1 --2- 172.21.15.78:0/675704481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21f41078c0 0x7f21f419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:34.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f2ffd640 1 -- 172.21.15.78:0/675704481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21f41078c0 msgr2=0x7f21f419cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:34.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f2ffd640 1 --2- 172.21.15.78:0/675704481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21f41078c0 0x7f21f419cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:34.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f27fc640 1 --2- 172.21.15.78:0/675704481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21f4102260 0x7f21f4198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:34.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f2ffd640 1 -- 172.21.15.78:0/675704481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21f4102260 msgr2=0x7f21f4198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:34.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f2ffd640 1 --2- 172.21.15.78:0/675704481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21f4102260 0x7f21f4198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:34.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f2ffd640 1 -- 172.21.15.78:0/675704481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f419d1f0 con 0x7f21f41018c0 2024-08-18T23:41:34.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f37fe640 1 --2- 172.21.15.78:0/675704481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21f41078c0 0x7f21f419cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:41:34.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f27fc640 1 --2- 172.21.15.78:0/675704481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21f4102260 0x7f21f4198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:41:34.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f2ffd640 1 --2- 172.21.15.78:0/675704481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f41018c0 0x7f21f4198240 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f21e400eab0 tx=0x7f21e400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:34.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21dbfff640 1 -- 172.21.15.78:0/675704481 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e400ce40 con 0x7f21f41018c0 2024-08-18T23:41:34.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21dbfff640 1 -- 172.21.15.78:0/675704481 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e4011950 con 0x7f21f41018c0 2024-08-18T23:41:34.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21f94a5640 1 -- 172.21.15.78:0/675704481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f419d4e0 con 0x7f21f41018c0 2024-08-18T23:41:34.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.812+0000 7f21dbfff640 1 -- 172.21.15.78:0/675704481 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4010ae0 con 0x7f21f41018c0 2024-08-18T23:41:34.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.813+0000 7f21f94a5640 1 -- 172.21.15.78:0/675704481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f41a4dc0 con 0x7f21f41018c0 2024-08-18T23:41:34.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.813+0000 7f21f94a5640 1 -- 172.21.15.78:0/675704481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f4101d20 con 0x7f21f41018c0 2024-08-18T23:41:34.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.814+0000 7f21dbfff640 1 -- 172.21.15.78:0/675704481 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f21e4011b00 con 0x7f21f41018c0 2024-08-18T23:41:34.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.815+0000 7f21dbfff640 1 --2- 172.21.15.78:0/675704481 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f21dc0a2b80 0x7f21dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:34.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.815+0000 7f21f27fc640 1 --2- 172.21.15.78:0/675704481 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f21dc0a2b80 0x7f21dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:34.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.816+0000 7f21dbfff640 1 -- 172.21.15.78:0/675704481 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f21e40cf6e0 con 0x7f21f41018c0 2024-08-18T23:41:34.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.816+0000 7f21f27fc640 1 --2- 172.21.15.78:0/675704481 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f21dc0a2b80 0x7f21dc0a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f21e8006fd0 tx=0x7f21e8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:34.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.820+0000 7f21dbfff640 1 -- 172.21.15.78:0/675704481 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f21e4016030 con 0x7f21f41018c0 2024-08-18T23:41:34.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:34.994+0000 7f21f94a5640 1 -- 172.21.15.78:0/675704481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f21f41995a0 con 0x7f21f41018c0 2024-08-18T23:41:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:35.148+0000 7f21dbfff640 1 -- 172.21.15.78:0/675704481 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v83) ==== 157+0+0 (secure 0 0 0) 0x7f21e409c790 con 0x7f21f41018c0 2024-08-18T23:41:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:35.150+0000 7f21f94a5640 1 -- 172.21.15.78:0/675704481 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f21dc0a2b80 msgr2=0x7f21dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:35.150+0000 7f21f94a5640 1 --2- 172.21.15.78:0/675704481 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f21dc0a2b80 0x7f21dc0a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f21e8006fd0 tx=0x7f21e8008040 comp rx=0 tx=0).stop 2024-08-18T23:41:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:35.150+0000 7f21f94a5640 1 -- 172.21.15.78:0/675704481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f41018c0 msgr2=0x7f21f4198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:35.150+0000 7f21f94a5640 1 --2- 172.21.15.78:0/675704481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f41018c0 0x7f21f4198240 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f21e400eab0 tx=0x7f21e400ef70 comp rx=0 tx=0).stop 2024-08-18T23:41:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:35.150+0000 7f21f94a5640 1 -- 172.21.15.78:0/675704481 shutdown_connections 2024-08-18T23:41:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:35.150+0000 7f21f94a5640 1 --2- 172.21.15.78:0/675704481 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f21dc0a2b80 0x7f21dc0a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:35.150+0000 7f21f94a5640 1 --2- 172.21.15.78:0/675704481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21f41078c0 0x7f21f419cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:35.150+0000 7f21f94a5640 1 --2- 172.21.15.78:0/675704481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21f4102260 0x7f21f4198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:35.150+0000 7f21f94a5640 1 --2- 172.21.15.78:0/675704481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21f41018c0 0x7f21f4198240 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:35.150+0000 7f21f94a5640 1 -- 172.21.15.78:0/675704481 >> 172.21.15.78:0/675704481 conn(0x7f21f40fd620 msgr2=0x7f21f40fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:35.150+0000 7f21f94a5640 1 -- 172.21.15.78:0/675704481 shutdown_connections 2024-08-18T23:41:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:35.150+0000 7f21f94a5640 1 -- 172.21.15.78:0/675704481 wait complete. 2024-08-18T23:41:35.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:35 smithi161 ceph-mon[22027]: pgmap v258: 97 pgs: 1 creating+activating, 26 creating+peering, 26 unknown, 44 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-08-18T23:41:35.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:35 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/675704481' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-08-18T23:41:35.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:35 smithi161 ceph-mon[22027]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-18T23:41:35.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:35 smithi161 ceph-mon[22027]: Cluster is now healthy 2024-08-18T23:41:35.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:35 smithi078 ceph-mon[18440]: pgmap v258: 97 pgs: 1 creating+activating, 26 creating+peering, 26 unknown, 44 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-08-18T23:41:35.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:35 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/675704481' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-08-18T23:41:35.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:35 smithi078 ceph-mon[18440]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-18T23:41:35.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:35 smithi078 ceph-mon[18440]: Cluster is now healthy 2024-08-18T23:41:35.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:35 smithi123 ceph-mon[21808]: pgmap v258: 97 pgs: 1 creating+activating, 26 creating+peering, 26 unknown, 44 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-08-18T23:41:35.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:35 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/675704481' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-08-18T23:41:35.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:35 smithi123 ceph-mon[21808]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-18T23:41:35.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:35 smithi123 ceph-mon[21808]: Cluster is now healthy 2024-08-18T23:41:35.972 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-08-18T23:41:36.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:36 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/675704481' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-08-18T23:41:36.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:36 smithi078 ceph-mon[18440]: osdmap e83: 12 total, 12 up, 12 in 2024-08-18T23:41:36.222 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:36.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:36 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/675704481' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-08-18T23:41:36.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:36 smithi161 ceph-mon[22027]: osdmap e83: 12 total, 12 up, 12 in 2024-08-18T23:41:36.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:36 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/675704481' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-08-18T23:41:36.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:36 smithi123 ceph-mon[21808]: osdmap e83: 12 total, 12 up, 12 in 2024-08-18T23:41:36.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.716+0000 7fac93035640 1 -- 172.21.15.78:0/1281826725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac8c101150 msgr2=0x7fac8c10d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:36.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.716+0000 7fac93035640 1 --2- 172.21.15.78:0/1281826725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac8c101150 0x7fac8c10d680 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fac84009cd0 tx=0x7fac8402a8e0 comp rx=0 tx=0).stop 2024-08-18T23:41:36.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.716+0000 7fac93035640 1 -- 172.21.15.78:0/1281826725 shutdown_connections 2024-08-18T23:41:36.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.716+0000 7fac93035640 1 --2- 172.21.15.78:0/1281826725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac8c101150 0x7fac8c10d680 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:36.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.716+0000 7fac93035640 1 --2- 172.21.15.78:0/1281826725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac8c1007b0 0x7fac8c100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:36.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.716+0000 7fac93035640 1 --2- 172.21.15.78:0/1281826725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac8c1067b0 0x7fac8c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:36.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.716+0000 7fac93035640 1 -- 172.21.15.78:0/1281826725 >> 172.21.15.78:0/1281826725 conn(0x7fac8c0fc4d0 msgr2=0x7fac8c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.716+0000 7fac93035640 1 -- 172.21.15.78:0/1281826725 shutdown_connections 2024-08-18T23:41:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.716+0000 7fac93035640 1 -- 172.21.15.78:0/1281826725 wait complete. 2024-08-18T23:41:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.717+0000 7fac93035640 1 Processor -- start 2024-08-18T23:41:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.717+0000 7fac93035640 1 -- start start 2024-08-18T23:41:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.717+0000 7fac93035640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac8c1007b0 0x7fac8c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.717+0000 7fac93035640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac8c101150 0x7fac8c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.717+0000 7fac93035640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac8c1067b0 0x7fac8c19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.717+0000 7fac93035640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c110bb0 con 0x7fac8c1067b0 2024-08-18T23:41:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.717+0000 7fac93035640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c110a30 con 0x7fac8c101150 2024-08-18T23:41:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.717+0000 7fac93035640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c110d30 con 0x7fac8c1007b0 2024-08-18T23:41:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.717+0000 7fac915d2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac8c1067b0 0x7fac8c19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.717+0000 7fac915d2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac8c1067b0 0x7fac8c19ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36170/0 (socket says 172.21.15.78:36170) 2024-08-18T23:41:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.717+0000 7fac915d2640 1 -- 172.21.15.78:0/2213236619 learned_addr learned my addr 172.21.15.78:0/2213236619 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.718+0000 7fac90dd1640 1 --2- 172.21.15.78:0/2213236619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac8c1007b0 0x7fac8c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.718+0000 7fac8bfff640 1 --2- 172.21.15.78:0/2213236619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac8c101150 0x7fac8c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.718+0000 7fac915d2640 1 -- 172.21.15.78:0/2213236619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac8c1007b0 msgr2=0x7fac8c19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.718+0000 7fac915d2640 1 --2- 172.21.15.78:0/2213236619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac8c1007b0 0x7fac8c19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.718+0000 7fac915d2640 1 -- 172.21.15.78:0/2213236619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac8c101150 msgr2=0x7fac8c19a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.718+0000 7fac915d2640 1 --2- 172.21.15.78:0/2213236619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac8c101150 0x7fac8c19a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.718+0000 7fac915d2640 1 -- 172.21.15.78:0/2213236619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8c19f2e0 con 0x7fac8c1067b0 2024-08-18T23:41:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.718+0000 7fac8bfff640 1 --2- 172.21.15.78:0/2213236619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac8c101150 0x7fac8c19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:41:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.718+0000 7fac915d2640 1 --2- 172.21.15.78:0/2213236619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac8c1067b0 0x7fac8c19ec00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fac84005cc0 tx=0x7fac84032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.718+0000 7fac89ffb640 1 -- 172.21.15.78:0/2213236619 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84004040 con 0x7fac8c1067b0 2024-08-18T23:41:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.718+0000 7fac89ffb640 1 -- 172.21.15.78:0/2213236619 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac840046c0 con 0x7fac8c1067b0 2024-08-18T23:41:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.718+0000 7fac93035640 1 -- 172.21.15.78:0/2213236619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac8c19f570 con 0x7fac8c1067b0 2024-08-18T23:41:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.718+0000 7fac89ffb640 1 -- 172.21.15.78:0/2213236619 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84030e00 con 0x7fac8c1067b0 2024-08-18T23:41:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.718+0000 7fac93035640 1 -- 172.21.15.78:0/2213236619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac8c1a6eb0 con 0x7fac8c1067b0 2024-08-18T23:41:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.719+0000 7fac93035640 1 -- 172.21.15.78:0/2213236619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac8c100c10 con 0x7fac8c1067b0 2024-08-18T23:41:36.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.720+0000 7fac89ffb640 1 -- 172.21.15.78:0/2213236619 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fac840305a0 con 0x7fac8c1067b0 2024-08-18T23:41:36.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.721+0000 7fac89ffb640 1 --2- 172.21.15.78:0/2213236619 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fac7c0a2b80 0x7fac7c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:36.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.721+0000 7fac90dd1640 1 --2- 172.21.15.78:0/2213236619 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fac7c0a2b80 0x7fac7c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:36.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.721+0000 7fac89ffb640 1 -- 172.21.15.78:0/2213236619 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8277+0+0 (secure 0 0 0) 0x7fac8402e030 con 0x7fac8c1067b0 2024-08-18T23:41:36.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.721+0000 7fac90dd1640 1 --2- 172.21.15.78:0/2213236619 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fac7c0a2b80 0x7fac7c0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fac80009ae0 tx=0x7fac80005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:36.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.725+0000 7fac89ffb640 1 -- 172.21.15.78:0/2213236619 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fac8404b060 con 0x7fac8c1067b0 2024-08-18T23:41:36.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:36.896+0000 7fac93035640 1 -- 172.21.15.78:0/2213236619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fac8c1a7080 con 0x7fac8c1067b0 2024-08-18T23:41:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:37 smithi161 ceph-mon[22027]: pgmap v260: 97 pgs: 1 creating+activating, 17 creating+peering, 79 active+clean; 578 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 821 B/s wr, 6 op/s 2024-08-18T23:41:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:37 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2213236619' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-18T23:41:37.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:37 smithi078 ceph-mon[18440]: pgmap v260: 97 pgs: 1 creating+activating, 17 creating+peering, 79 active+clean; 578 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 821 B/s wr, 6 op/s 2024-08-18T23:41:37.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:37 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2213236619' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-18T23:41:37.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:37 smithi123 ceph-mon[21808]: pgmap v260: 97 pgs: 1 creating+activating, 17 creating+peering, 79 active+clean; 578 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 821 B/s wr, 6 op/s 2024-08-18T23:41:37.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:37 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2213236619' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-18T23:41:38.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:38.165+0000 7fac89ffb640 1 -- 172.21.15.78:0/2213236619 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v85) ==== 228+0+0 (secure 0 0 0) 0x7fac840bd890 con 0x7fac8c1067b0 2024-08-18T23:41:38.166 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data_ec' created 2024-08-18T23:41:38.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:38.167+0000 7fac93035640 1 -- 172.21.15.78:0/2213236619 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fac7c0a2b80 msgr2=0x7fac7c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:38.167+0000 7fac93035640 1 --2- 172.21.15.78:0/2213236619 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fac7c0a2b80 0x7fac7c0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fac80009ae0 tx=0x7fac80005f90 comp rx=0 tx=0).stop 2024-08-18T23:41:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:38.167+0000 7fac93035640 1 -- 172.21.15.78:0/2213236619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac8c1067b0 msgr2=0x7fac8c19ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:38.167+0000 7fac93035640 1 --2- 172.21.15.78:0/2213236619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac8c1067b0 0x7fac8c19ec00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fac84005cc0 tx=0x7fac84032f40 comp rx=0 tx=0).stop 2024-08-18T23:41:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:38.167+0000 7fac93035640 1 -- 172.21.15.78:0/2213236619 shutdown_connections 2024-08-18T23:41:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:38.167+0000 7fac93035640 1 --2- 172.21.15.78:0/2213236619 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fac7c0a2b80 0x7fac7c0a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:38.167+0000 7fac93035640 1 --2- 172.21.15.78:0/2213236619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac8c1067b0 0x7fac8c19ec00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:38.167+0000 7fac93035640 1 --2- 172.21.15.78:0/2213236619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac8c101150 0x7fac8c19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:38.167+0000 7fac93035640 1 --2- 172.21.15.78:0/2213236619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac8c1007b0 0x7fac8c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:38.167+0000 7fac93035640 1 -- 172.21.15.78:0/2213236619 >> 172.21.15.78:0/2213236619 conn(0x7fac8c0fc4d0 msgr2=0x7fac8c06a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:38.168+0000 7fac93035640 1 -- 172.21.15.78:0/2213236619 shutdown_connections 2024-08-18T23:41:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:38.168+0000 7fac93035640 1 -- 172.21.15.78:0/2213236619 wait complete. 2024-08-18T23:41:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:38 smithi078 ceph-mon[18440]: osdmap e84: 12 total, 12 up, 12 in 2024-08-18T23:41:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:38 smithi078 ceph-mon[18440]: pgmap v262: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-08-18T23:41:38.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:38 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2213236619' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-18T23:41:38.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:38 smithi078 ceph-mon[18440]: osdmap e85: 12 total, 12 up, 12 in 2024-08-18T23:41:38.547 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:41:38 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-b[51547]: 2024-08-18T23:41:38.164+0000 7f7b3120b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-18T23:41:38.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:38 smithi123 ceph-mon[21808]: osdmap e84: 12 total, 12 up, 12 in 2024-08-18T23:41:38.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:38 smithi123 ceph-mon[21808]: pgmap v262: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-08-18T23:41:38.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:38 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2213236619' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-18T23:41:38.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:38 smithi123 ceph-mon[21808]: osdmap e85: 12 total, 12 up, 12 in 2024-08-18T23:41:38.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:38 smithi161 ceph-mon[22027]: osdmap e84: 12 total, 12 up, 12 in 2024-08-18T23:41:38.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:38 smithi161 ceph-mon[22027]: pgmap v262: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-08-18T23:41:38.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:38 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2213236619' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-18T23:41:38.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:38 smithi161 ceph-mon[22027]: osdmap e85: 12 total, 12 up, 12 in 2024-08-18T23:41:38.979 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-08-18T23:41:39.244 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:39.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.722+0000 7fe826bbd640 1 -- 172.21.15.78:0/2088309008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe820102c20 msgr2=0x7fe82010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:39.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.722+0000 7fe826bbd640 1 --2- 172.21.15.78:0/2088309008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe820102c20 0x7fe82010d770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe818009ca0 tx=0x7fe81802a950 comp rx=0 tx=0).stop 2024-08-18T23:41:39.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.722+0000 7fe826bbd640 1 -- 172.21.15.78:0/2088309008 shutdown_connections 2024-08-18T23:41:39.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.722+0000 7fe826bbd640 1 --2- 172.21.15.78:0/2088309008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe820102c20 0x7fe82010d770 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:39.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.722+0000 7fe826bbd640 1 --2- 172.21.15.78:0/2088309008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8201022a0 0x7fe8201026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:39.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.722+0000 7fe826bbd640 1 --2- 172.21.15.78:0/2088309008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe820100b50 0x7fe820100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:39.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.722+0000 7fe826bbd640 1 -- 172.21.15.78:0/2088309008 >> 172.21.15.78:0/2088309008 conn(0x7fe8200fc500 msgr2=0x7fe8200fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:39.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.722+0000 7fe826bbd640 1 -- 172.21.15.78:0/2088309008 shutdown_connections 2024-08-18T23:41:39.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.722+0000 7fe826bbd640 1 -- 172.21.15.78:0/2088309008 wait complete. 2024-08-18T23:41:39.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.723+0000 7fe826bbd640 1 Processor -- start 2024-08-18T23:41:39.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.723+0000 7fe826bbd640 1 -- start start 2024-08-18T23:41:39.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.723+0000 7fe826bbd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe820100b50 0x7fe820197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:39.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.723+0000 7fe826bbd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8201022a0 0x7fe8201975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:39.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.723+0000 7fe826bbd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe820102c20 0x7fe82019b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:39.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.723+0000 7fe826bbd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe820110d00 con 0x7fe820100b50 2024-08-18T23:41:39.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.723+0000 7fe826bbd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe820110b80 con 0x7fe820102c20 2024-08-18T23:41:39.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.723+0000 7fe826bbd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe820110e80 con 0x7fe8201022a0 2024-08-18T23:41:39.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.723+0000 7fe824959640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe820100b50 0x7fe820197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:39.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.723+0000 7fe824959640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe820100b50 0x7fe820197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36180/0 (socket says 172.21.15.78:36180) 2024-08-18T23:41:39.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.723+0000 7fe824959640 1 -- 172.21.15.78:0/3428170459 learned_addr learned my addr 172.21.15.78:0/3428170459 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:39.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.723+0000 7fe81ffff640 1 --2- 172.21.15.78:0/3428170459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8201022a0 0x7fe8201975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:39.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.723+0000 7fe82515a640 1 --2- 172.21.15.78:0/3428170459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe820102c20 0x7fe82019b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:39.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.724+0000 7fe824959640 1 -- 172.21.15.78:0/3428170459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8201022a0 msgr2=0x7fe8201975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:39.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.724+0000 7fe824959640 1 --2- 172.21.15.78:0/3428170459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8201022a0 0x7fe8201975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:39.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.724+0000 7fe824959640 1 -- 172.21.15.78:0/3428170459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe820102c20 msgr2=0x7fe82019b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:39.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.724+0000 7fe824959640 1 --2- 172.21.15.78:0/3428170459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe820102c20 0x7fe82019b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.724+0000 7fe824959640 1 -- 172.21.15.78:0/3428170459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82019c030 con 0x7fe820100b50 2024-08-18T23:41:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.724+0000 7fe82515a640 1 --2- 172.21.15.78:0/3428170459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe820102c20 0x7fe82019b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:41:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.724+0000 7fe824959640 1 --2- 172.21.15.78:0/3428170459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe820100b50 0x7fe820197080 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe81400d770 tx=0x7fe81400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.724+0000 7fe81dffb640 1 -- 172.21.15.78:0/3428170459 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814017070 con 0x7fe820100b50 2024-08-18T23:41:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.724+0000 7fe81dffb640 1 -- 172.21.15.78:0/3428170459 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe814005500 con 0x7fe820100b50 2024-08-18T23:41:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.724+0000 7fe826bbd640 1 -- 172.21.15.78:0/3428170459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe82019c320 con 0x7fe820100b50 2024-08-18T23:41:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.724+0000 7fe81dffb640 1 -- 172.21.15.78:0/3428170459 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814004e00 con 0x7fe820100b50 2024-08-18T23:41:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.724+0000 7fe826bbd640 1 -- 172.21.15.78:0/3428170459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8201a3c00 con 0x7fe820100b50 2024-08-18T23:41:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.725+0000 7fe826bbd640 1 -- 172.21.15.78:0/3428170459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe820100f30 con 0x7fe820100b50 2024-08-18T23:41:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.726+0000 7fe81dffb640 1 -- 172.21.15.78:0/3428170459 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe814004fa0 con 0x7fe820100b50 2024-08-18T23:41:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.727+0000 7fe81dffb640 1 --2- 172.21.15.78:0/3428170459 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe8100a2b80 0x7fe8100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.727+0000 7fe81ffff640 1 --2- 172.21.15.78:0/3428170459 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe8100a2b80 0x7fe8100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:39.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.727+0000 7fe81dffb640 1 -- 172.21.15.78:0/3428170459 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8770+0+0 (secure 0 0 0) 0x7fe8140cec90 con 0x7fe820100b50 2024-08-18T23:41:39.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.727+0000 7fe81ffff640 1 --2- 172.21.15.78:0/3428170459 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe8100a2b80 0x7fe8100a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe8200fe9d0 tx=0x7fe80c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:39.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.731+0000 7fe81dffb640 1 -- 172.21.15.78:0/3428170459 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe81409b9d0 con 0x7fe820100b50 2024-08-18T23:41:39.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:39.899+0000 7fe826bbd640 1 -- 172.21.15.78:0/3428170459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fe8201a3dd0 con 0x7fe820100b50 2024-08-18T23:41:40.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:40.173+0000 7fe81dffb640 1 -- 172.21.15.78:0/3428170459 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) ==== 169+0+0 (secure 0 0 0) 0x7fe81409b2d0 con 0x7fe820100b50 2024-08-18T23:41:40.174 INFO:teuthology.orchestra.run.smithi078.stderr:set pool 4 allow_ec_overwrites to true 2024-08-18T23:41:40.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:40.176+0000 7fe826bbd640 1 -- 172.21.15.78:0/3428170459 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe8100a2b80 msgr2=0x7fe8100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:40.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:40.176+0000 7fe826bbd640 1 --2- 172.21.15.78:0/3428170459 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe8100a2b80 0x7fe8100a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe8200fe9d0 tx=0x7fe80c00b390 comp rx=0 tx=0).stop 2024-08-18T23:41:40.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:40.176+0000 7fe826bbd640 1 -- 172.21.15.78:0/3428170459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe820100b50 msgr2=0x7fe820197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:40.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:40.176+0000 7fe826bbd640 1 --2- 172.21.15.78:0/3428170459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe820100b50 0x7fe820197080 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe81400d770 tx=0x7fe81400dc30 comp rx=0 tx=0).stop 2024-08-18T23:41:40.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:40.176+0000 7fe826bbd640 1 -- 172.21.15.78:0/3428170459 shutdown_connections 2024-08-18T23:41:40.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:40.176+0000 7fe826bbd640 1 --2- 172.21.15.78:0/3428170459 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe8100a2b80 0x7fe8100a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:40.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:40.176+0000 7fe826bbd640 1 --2- 172.21.15.78:0/3428170459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe820102c20 0x7fe82019b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:40.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:40.176+0000 7fe826bbd640 1 --2- 172.21.15.78:0/3428170459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8201022a0 0x7fe8201975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:40.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:40.176+0000 7fe826bbd640 1 --2- 172.21.15.78:0/3428170459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe820100b50 0x7fe820197080 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:40.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:40.176+0000 7fe826bbd640 1 -- 172.21.15.78:0/3428170459 >> 172.21.15.78:0/3428170459 conn(0x7fe8200fc500 msgr2=0x7fe820106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:40.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:40.176+0000 7fe826bbd640 1 -- 172.21.15.78:0/3428170459 shutdown_connections 2024-08-18T23:41:40.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:40.176+0000 7fe826bbd640 1 -- 172.21.15.78:0/3428170459 wait complete. 2024-08-18T23:41:40.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:40 smithi161 ceph-mon[22027]: osdmap e86: 12 total, 12 up, 12 in 2024-08-18T23:41:40.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:40 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3428170459' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-08-18T23:41:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:40 smithi078 ceph-mon[18440]: osdmap e86: 12 total, 12 up, 12 in 2024-08-18T23:41:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:40 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3428170459' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-08-18T23:41:40.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:40 smithi123 ceph-mon[21808]: osdmap e86: 12 total, 12 up, 12 in 2024-08-18T23:41:40.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:40 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3428170459' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-08-18T23:41:40.840 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-08-18T23:41:41.090 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:41.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:41 smithi161 ceph-mon[22027]: pgmap v265: 129 pgs: 2 creating+activating, 4 creating+peering, 19 unknown, 104 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2024-08-18T23:41:41.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:41 smithi161 ceph-mon[22027]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-18T23:41:41.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:41 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3428170459' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-08-18T23:41:41.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:41 smithi161 ceph-mon[22027]: osdmap e87: 12 total, 12 up, 12 in 2024-08-18T23:41:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:41 smithi078 ceph-mon[18440]: pgmap v265: 129 pgs: 2 creating+activating, 4 creating+peering, 19 unknown, 104 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2024-08-18T23:41:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:41 smithi078 ceph-mon[18440]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-18T23:41:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:41 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3428170459' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-08-18T23:41:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:41 smithi078 ceph-mon[18440]: osdmap e87: 12 total, 12 up, 12 in 2024-08-18T23:41:41.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:41 smithi123 ceph-mon[21808]: pgmap v265: 129 pgs: 2 creating+activating, 4 creating+peering, 19 unknown, 104 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2024-08-18T23:41:41.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:41 smithi123 ceph-mon[21808]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-18T23:41:41.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:41 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3428170459' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-08-18T23:41:41.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:41 smithi123 ceph-mon[21808]: osdmap e87: 12 total, 12 up, 12 in 2024-08-18T23:41:41.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.597+0000 7f38ac8da640 1 -- 172.21.15.78:0/3706711425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38a8101150 msgr2=0x7f38a810d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:41.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.597+0000 7f38ac8da640 1 --2- 172.21.15.78:0/3706711425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38a8101150 0x7f38a810d680 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3898009930 tx=0x7f389802a420 comp rx=0 tx=0).stop 2024-08-18T23:41:41.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.598+0000 7f38ac8da640 1 -- 172.21.15.78:0/3706711425 shutdown_connections 2024-08-18T23:41:41.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.598+0000 7f38ac8da640 1 --2- 172.21.15.78:0/3706711425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38a8101150 0x7f38a810d680 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:41.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.598+0000 7f38ac8da640 1 --2- 172.21.15.78:0/3706711425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38a81007b0 0x7f38a8100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:41.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.598+0000 7f38ac8da640 1 --2- 172.21.15.78:0/3706711425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38a81067b0 0x7f38a8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:41.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.598+0000 7f38ac8da640 1 -- 172.21.15.78:0/3706711425 >> 172.21.15.78:0/3706711425 conn(0x7f38a80fc4d0 msgr2=0x7f38a80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:41.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.598+0000 7f38ac8da640 1 -- 172.21.15.78:0/3706711425 shutdown_connections 2024-08-18T23:41:41.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.598+0000 7f38ac8da640 1 -- 172.21.15.78:0/3706711425 wait complete. 2024-08-18T23:41:41.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.598+0000 7f38ac8da640 1 Processor -- start 2024-08-18T23:41:41.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.598+0000 7f38ac8da640 1 -- start start 2024-08-18T23:41:41.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.598+0000 7f38ac8da640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38a81007b0 0x7f38a8198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:41.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f38ac8da640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38a8101150 0x7f38a8198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:41.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f38ac8da640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38a81067b0 0x7f38a819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:41.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f38ac8da640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8110c00 con 0x7f38a8101150 2024-08-18T23:41:41.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f38ac8da640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8110a80 con 0x7f38a81067b0 2024-08-18T23:41:41.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f38ac8da640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8110d80 con 0x7f38a81007b0 2024-08-18T23:41:41.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f38a5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38a8101150 0x7f38a8198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:41.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f38a5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38a8101150 0x7f38a8198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36198/0 (socket says 172.21.15.78:36198) 2024-08-18T23:41:41.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f38a5d9b640 1 -- 172.21.15.78:0/2704706696 learned_addr learned my addr 172.21.15.78:0/2704706696 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:41.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f38a659c640 1 --2- 172.21.15.78:0/2704706696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38a81007b0 0x7f38a8198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:41.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f38a6d9d640 1 --2- 172.21.15.78:0/2704706696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38a81067b0 0x7f38a819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:41.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f38a5d9b640 1 -- 172.21.15.78:0/2704706696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38a81007b0 msgr2=0x7f38a8198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:41.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f38a5d9b640 1 --2- 172.21.15.78:0/2704706696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38a81007b0 0x7f38a8198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:41.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f38a5d9b640 1 -- 172.21.15.78:0/2704706696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38a81067b0 msgr2=0x7f38a819c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:41.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f38a5d9b640 1 --2- 172.21.15.78:0/2704706696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38a81067b0 0x7f38a819c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:41.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f38a5d9b640 1 -- 172.21.15.78:0/2704706696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a819d0c0 con 0x7f38a8101150 2024-08-18T23:41:41.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f38a6d9d640 1 --2- 172.21.15.78:0/2704706696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38a81067b0 0x7f38a819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:41:41.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f38a5d9b640 1 --2- 172.21.15.78:0/2704706696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38a8101150 0x7f38a8198650 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f389000d950 tx=0x7f389000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:41.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f389f7fe640 1 -- 172.21.15.78:0/2704706696 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890017030 con 0x7f38a8101150 2024-08-18T23:41:41.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f389f7fe640 1 -- 172.21.15.78:0/2704706696 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3890004020 con 0x7f38a8101150 2024-08-18T23:41:41.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f38ac8da640 1 -- 172.21.15.78:0/2704706696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a819d3b0 con 0x7f38a8101150 2024-08-18T23:41:41.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f389f7fe640 1 -- 172.21.15.78:0/2704706696 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38900047a0 con 0x7f38a8101150 2024-08-18T23:41:41.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.599+0000 7f38ac8da640 1 -- 172.21.15.78:0/2704706696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a81a4c90 con 0x7f38a8101150 2024-08-18T23:41:41.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.600+0000 7f38ac8da640 1 -- 172.21.15.78:0/2704706696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a8100c10 con 0x7f38a8101150 2024-08-18T23:41:41.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.601+0000 7f389f7fe640 1 -- 172.21.15.78:0/2704706696 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f38900041d0 con 0x7f38a8101150 2024-08-18T23:41:41.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.602+0000 7f389f7fe640 1 --2- 172.21.15.78:0/2704706696 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f38940a2c00 0x7f38940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:41.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.602+0000 7f38a659c640 1 --2- 172.21.15.78:0/2704706696 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f38940a2c00 0x7f38940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:41.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.602+0000 7f389f7fe640 1 -- 172.21.15.78:0/2704706696 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7f38900cec80 con 0x7f38a8101150 2024-08-18T23:41:41.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.602+0000 7f38a659c640 1 --2- 172.21.15.78:0/2704706696 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f38940a2c00 0x7f38940a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f38a0005fd0 tx=0x7f38a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:41.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.606+0000 7f389f7fe640 1 -- 172.21.15.78:0/2704706696 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f389009b9c0 con 0x7f38a8101150 2024-08-18T23:41:41.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:41.809+0000 7f38ac8da640 1 -- 172.21.15.78:0/2704706696 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f38a81a5020 con 0x7f38a8101150 2024-08-18T23:41:42.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:42.188+0000 7f389f7fe640 1 -- 172.21.15.78:0/2704706696 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7f389009b2c0 con 0x7f38a8101150 2024-08-18T23:41:42.189 INFO:teuthology.orchestra.run.smithi078.stderr:added data pool 4 to fsmap 2024-08-18T23:41:42.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:42.190+0000 7f38ac8da640 1 -- 172.21.15.78:0/2704706696 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f38940a2c00 msgr2=0x7f38940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:42.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:42.190+0000 7f38ac8da640 1 --2- 172.21.15.78:0/2704706696 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f38940a2c00 0x7f38940a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f38a0005fd0 tx=0x7f38a0009260 comp rx=0 tx=0).stop 2024-08-18T23:41:42.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:42.190+0000 7f38ac8da640 1 -- 172.21.15.78:0/2704706696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38a8101150 msgr2=0x7f38a8198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:42.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:42.190+0000 7f38ac8da640 1 --2- 172.21.15.78:0/2704706696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38a8101150 0x7f38a8198650 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f389000d950 tx=0x7f389000de10 comp rx=0 tx=0).stop 2024-08-18T23:41:42.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:42.190+0000 7f38ac8da640 1 -- 172.21.15.78:0/2704706696 shutdown_connections 2024-08-18T23:41:42.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:42.190+0000 7f38ac8da640 1 --2- 172.21.15.78:0/2704706696 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f38940a2c00 0x7f38940a50c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:42.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:42.190+0000 7f38ac8da640 1 --2- 172.21.15.78:0/2704706696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38a81067b0 0x7f38a819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:42.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:42.190+0000 7f38ac8da640 1 --2- 172.21.15.78:0/2704706696 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38a8101150 0x7f38a8198650 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:42.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:42.190+0000 7f38ac8da640 1 --2- 172.21.15.78:0/2704706696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38a81007b0 0x7f38a8198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:42.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:42.190+0000 7f38ac8da640 1 -- 172.21.15.78:0/2704706696 >> 172.21.15.78:0/2704706696 conn(0x7f38a80fc4d0 msgr2=0x7f38a806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:42.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:42.191+0000 7f38ac8da640 1 -- 172.21.15.78:0/2704706696 shutdown_connections 2024-08-18T23:41:42.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:42.191+0000 7f38ac8da640 1 -- 172.21.15.78:0/2704706696 wait complete. 2024-08-18T23:41:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:42 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2704706696' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-08-18T23:41:42.524 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:42 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2704706696' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-08-18T23:41:42.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:42 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2704706696' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-08-18T23:41:43.014 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:41:43.263 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:43.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:43 smithi078 ceph-mon[18440]: pgmap v267: 129 pgs: 2 creating+activating, 4 creating+peering, 9 unknown, 114 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:43.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:43 smithi078 ceph-mon[18440]: osdmap e88: 12 total, 12 up, 12 in 2024-08-18T23:41:43.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:43 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2704706696' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-08-18T23:41:43.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:43 smithi078 ceph-mon[18440]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-18T23:41:43.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:43 smithi123 ceph-mon[21808]: pgmap v267: 129 pgs: 2 creating+activating, 4 creating+peering, 9 unknown, 114 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:43 smithi123 ceph-mon[21808]: osdmap e88: 12 total, 12 up, 12 in 2024-08-18T23:41:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:43 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2704706696' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-08-18T23:41:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:43 smithi123 ceph-mon[21808]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-18T23:41:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:43 smithi161 ceph-mon[22027]: pgmap v267: 129 pgs: 2 creating+activating, 4 creating+peering, 9 unknown, 114 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:43 smithi161 ceph-mon[22027]: osdmap e88: 12 total, 12 up, 12 in 2024-08-18T23:41:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:43 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2704706696' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-08-18T23:41:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:43 smithi161 ceph-mon[22027]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-18T23:41:43.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.743+0000 7f46b9743640 1 -- 172.21.15.78:0/3189021625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46b4100b50 msgr2=0x7f46b4100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:43.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.743+0000 7f46b9743640 1 --2- 172.21.15.78:0/3189021625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46b4100b50 0x7f46b4100f30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009900 tx=0x7f46a402a480 comp rx=0 tx=0).stop 2024-08-18T23:41:43.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.743+0000 7f46b9743640 1 -- 172.21.15.78:0/3189021625 shutdown_connections 2024-08-18T23:41:43.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.743+0000 7f46b9743640 1 --2- 172.21.15.78:0/3189021625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b4102c20 0x7f46b410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:43.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.743+0000 7f46b9743640 1 --2- 172.21.15.78:0/3189021625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46b41022a0 0x7f46b41026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:43.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.743+0000 7f46b9743640 1 --2- 172.21.15.78:0/3189021625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46b4100b50 0x7f46b4100f30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:43.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.743+0000 7f46b9743640 1 -- 172.21.15.78:0/3189021625 >> 172.21.15.78:0/3189021625 conn(0x7f46b40fc500 msgr2=0x7f46b40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:43.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.743+0000 7f46b9743640 1 -- 172.21.15.78:0/3189021625 shutdown_connections 2024-08-18T23:41:43.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.743+0000 7f46b9743640 1 -- 172.21.15.78:0/3189021625 wait complete. 2024-08-18T23:41:43.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.743+0000 7f46b9743640 1 Processor -- start 2024-08-18T23:41:43.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.744+0000 7f46b9743640 1 -- start start 2024-08-18T23:41:43.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.744+0000 7f46b9743640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b4100b50 0x7f46b4197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:43.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.744+0000 7f46b9743640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46b41022a0 0x7f46b41976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:43.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.744+0000 7f46b9743640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46b4102c20 0x7f46b419ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:43.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.744+0000 7f46b9743640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f46b4110cf0 con 0x7f46b41022a0 2024-08-18T23:41:43.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.744+0000 7f46b9743640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46b4110b70 con 0x7f46b4100b50 2024-08-18T23:41:43.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.744+0000 7f46b9743640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f46b4110e70 con 0x7f46b4102c20 2024-08-18T23:41:43.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.744+0000 7f46aa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46b41022a0 0x7f46b41976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:43.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.744+0000 7f46aa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46b41022a0 0x7f46b41976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36212/0 (socket says 172.21.15.78:36212) 2024-08-18T23:41:43.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.744+0000 7f46aa7fc640 1 -- 172.21.15.78:0/3437294489 learned_addr learned my addr 172.21.15.78:0/3437294489 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.744+0000 7f46b37fe640 1 --2- 172.21.15.78:0/3437294489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46b4102c20 0x7f46b419ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.744+0000 7f46aa7fc640 1 -- 172.21.15.78:0/3437294489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46b4102c20 msgr2=0x7f46b419ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.744+0000 7f46aa7fc640 1 --2- 172.21.15.78:0/3437294489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46b4102c20 0x7f46b419ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.744+0000 7f46aa7fc640 1 -- 172.21.15.78:0/3437294489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b4100b50 msgr2=0x7f46b4197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.744+0000 7f46b2ffd640 1 --2- 172.21.15.78:0/3437294489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b4100b50 0x7f46b4197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.744+0000 7f46aa7fc640 1 --2- 172.21.15.78:0/3437294489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b4100b50 0x7f46b4197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.744+0000 7f46aa7fc640 1 -- 172.21.15.78:0/3437294489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b419c110 con 0x7f46b41022a0 2024-08-18T23:41:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.745+0000 7f46aa7fc640 1 --2- 172.21.15.78:0/3437294489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46b41022a0 0x7f46b41976a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f469c00d770 tx=0x7f469c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.745+0000 7f46b0ff9640 1 -- 172.21.15.78:0/3437294489 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c017070 con 0x7f46b41022a0 2024-08-18T23:41:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.745+0000 7f46b9743640 1 -- 172.21.15.78:0/3437294489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b419c400 con 0x7f46b41022a0 2024-08-18T23:41:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.745+0000 7f46b0ff9640 1 -- 172.21.15.78:0/3437294489 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f469c005500 con 0x7f46b41022a0 2024-08-18T23:41:43.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.745+0000 7f46b0ff9640 1 -- 172.21.15.78:0/3437294489 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c004e00 con 0x7f46b41022a0 2024-08-18T23:41:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.745+0000 7f46b9743640 1 -- 172.21.15.78:0/3437294489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b41a3ce0 con 0x7f46b41022a0 2024-08-18T23:41:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.746+0000 7f46b9743640 1 -- 172.21.15.78:0/3437294489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b4100f30 con 0x7f46b41022a0 2024-08-18T23:41:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.747+0000 7f46b0ff9640 1 -- 172.21.15.78:0/3437294489 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f469c004fa0 con 0x7f46b41022a0 2024-08-18T23:41:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.748+0000 7f46b0ff9640 1 --2- 172.21.15.78:0/3437294489 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f46a00a2c00 0x7f46a00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.748+0000 7f46b2ffd640 1 --2- 172.21.15.78:0/3437294489 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f46a00a2c00 0x7f46a00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.748+0000 7f46b0ff9640 1 -- 172.21.15.78:0/3437294489 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f469c0cecf0 con 0x7f46b41022a0 2024-08-18T23:41:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.748+0000 7f46b2ffd640 1 --2- 172.21.15.78:0/3437294489 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f46a00a2c00 0x7f46a00a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f46a40096c0 tx=0x7f46a4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:43.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.751+0000 7f46b0ff9640 1 -- 172.21.15.78:0/3437294489 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f469c0d3070 con 0x7f46b41022a0 2024-08-18T23:41:43.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.951+0000 7f46b9743640 1 -- 172.21.15.78:0/3437294489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46b4108dc0 con 0x7f46b41022a0 2024-08-18T23:41:43.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.951+0000 7f46b0ff9640 1 -- 172.21.15.78:0/3437294489 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11857 (secure 0 0 0) 0x7f469c09ba10 con 0x7f46b41022a0 2024-08-18T23:41:43.953 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:41:43.953 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":16,"btime":"2024-08-18T23:41:42:176701+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:41:41.811239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24452,"qdb_cluster":[24452]},"id":1}]} 2024-08-18T23:41:43.953 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 16 2024-08-18T23:41:43.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.954+0000 7f46b9743640 1 -- 172.21.15.78:0/3437294489 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f46a00a2c00 msgr2=0x7f46a00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:43.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.954+0000 7f46b9743640 1 --2- 172.21.15.78:0/3437294489 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f46a00a2c00 0x7f46a00a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f46a40096c0 tx=0x7f46a4000bc0 comp rx=0 tx=0).stop 2024-08-18T23:41:43.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.954+0000 7f46b9743640 1 -- 172.21.15.78:0/3437294489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46b41022a0 msgr2=0x7f46b41976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:43.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.954+0000 7f46b9743640 1 --2- 172.21.15.78:0/3437294489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46b41022a0 0x7f46b41976a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f469c00d770 tx=0x7f469c00dc30 comp rx=0 tx=0).stop 2024-08-18T23:41:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.954+0000 7f46b9743640 1 -- 172.21.15.78:0/3437294489 shutdown_connections 2024-08-18T23:41:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.954+0000 7f46b9743640 1 --2- 172.21.15.78:0/3437294489 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f46a00a2c00 0x7f46a00a50c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.954+0000 7f46b9743640 1 --2- 172.21.15.78:0/3437294489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46b4102c20 0x7f46b419ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.954+0000 7f46b9743640 1 --2- 172.21.15.78:0/3437294489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46b41022a0 0x7f46b41976a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.954+0000 7f46b9743640 1 --2- 172.21.15.78:0/3437294489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b4100b50 0x7f46b4197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.954+0000 7f46b9743640 1 -- 172.21.15.78:0/3437294489 >> 172.21.15.78:0/3437294489 conn(0x7f46b40fc500 msgr2=0x7f46b40fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.954+0000 7f46b9743640 1 -- 172.21.15.78:0/3437294489 shutdown_connections 2024-08-18T23:41:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:43.954+0000 7f46b9743640 1 -- 172.21.15.78:0/3437294489 wait complete. 2024-08-18T23:41:44.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:44 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3437294489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:44.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:44 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3437294489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:44.613 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-18T23:41:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:44 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3437294489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:44.873 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:45.407 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:45 smithi078 ceph-mon[18440]: pgmap v269: 129 pgs: 2 creating+activating, 3 creating+peering, 124 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:45.408 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:45 smithi078 ceph-mon[18440]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-08-18T23:41:45.408 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:45 smithi078 ceph-mon[18440]: Cluster is now healthy 2024-08-18T23:41:45.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.364+0000 7fb4b0358640 1 -- 172.21.15.78:0/851439456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4a8102260 msgr2=0x7fb4a810e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:45.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.364+0000 7fb4b0358640 1 --2- 172.21.15.78:0/851439456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4a8102260 0x7fb4a810e790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009cd0 tx=0x7fb4a002a8e0 comp rx=0 tx=0).stop 2024-08-18T23:41:45.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.365+0000 7fb4b0358640 1 -- 172.21.15.78:0/851439456 shutdown_connections 2024-08-18T23:41:45.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.365+0000 7fb4b0358640 1 --2- 172.21.15.78:0/851439456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4a8102260 0x7fb4a810e790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.365+0000 7fb4b0358640 1 --2- 172.21.15.78:0/851439456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4a81018c0 0x7fb4a8101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.365+0000 7fb4b0358640 1 --2- 172.21.15.78:0/851439456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4a81078c0 0x7fb4a8107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.365+0000 7fb4b0358640 1 -- 172.21.15.78:0/851439456 >> 172.21.15.78:0/851439456 conn(0x7fb4a80fd620 msgr2=0x7fb4a80ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.365+0000 7fb4b0358640 1 -- 172.21.15.78:0/851439456 shutdown_connections 2024-08-18T23:41:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.365+0000 7fb4b0358640 1 -- 172.21.15.78:0/851439456 wait complete. 2024-08-18T23:41:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.365+0000 7fb4b0358640 1 Processor -- start 2024-08-18T23:41:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.365+0000 7fb4b0358640 1 -- start start 2024-08-18T23:41:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.365+0000 7fb4b0358640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4a81018c0 0x7fb4a8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.365+0000 7fb4b0358640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4a8102260 0x7fb4a8198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.365+0000 7fb4b0358640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4a81078c0 0x7fb4a819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.366+0000 7fb4b0358640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8111d60 con 0x7fb4a8102260 2024-08-18T23:41:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.366+0000 7fb4b0358640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8111be0 con 0x7fb4a81018c0 2024-08-18T23:41:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.366+0000 7fb4b0358640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8111ee0 con 0x7fb4a81078c0 2024-08-18T23:41:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.366+0000 7fb4ad8f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4a8102260 0x7fb4a8198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.366+0000 7fb4ad8f3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4a8102260 0x7fb4a8198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36238/0 (socket says 172.21.15.78:36238) 2024-08-18T23:41:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.366+0000 7fb4ad8f3640 1 -- 172.21.15.78:0/3155013915 learned_addr learned my addr 172.21.15.78:0/3155013915 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.366+0000 7fb4ae0f4640 1 --2- 172.21.15.78:0/3155013915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4a81018c0 0x7fb4a8198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.366+0000 7fb4ae8f5640 1 --2- 172.21.15.78:0/3155013915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4a81078c0 0x7fb4a819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.366+0000 7fb4ad8f3640 1 -- 172.21.15.78:0/3155013915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4a81078c0 msgr2=0x7fb4a819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.366+0000 7fb4ad8f3640 1 --2- 172.21.15.78:0/3155013915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4a81078c0 0x7fb4a819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.366+0000 7fb4ad8f3640 1 -- 172.21.15.78:0/3155013915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4a81018c0 msgr2=0x7fb4a8198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.366+0000 7fb4ad8f3640 1 --2- 172.21.15.78:0/3155013915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4a81018c0 0x7fb4a8198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.366+0000 7fb4ad8f3640 1 -- 172.21.15.78:0/3155013915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a819d2d0 con 0x7fb4a8102260 2024-08-18T23:41:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.366+0000 7fb4ad8f3640 1 --2- 172.21.15.78:0/3155013915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4a8102260 0x7fb4a8198860 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb49800b4a0 tx=0x7fb49800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:45.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.366+0000 7fb49f7fe640 1 -- 172.21.15.78:0/3155013915 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498015020 con 0x7fb4a8102260 2024-08-18T23:41:45.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.366+0000 7fb4b0358640 1 -- 172.21.15.78:0/3155013915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a819d5c0 con 0x7fb4a8102260 2024-08-18T23:41:45.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.366+0000 7fb49f7fe640 1 -- 172.21.15.78:0/3155013915 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb498011d10 con 0x7fb4a8102260 2024-08-18T23:41:45.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.366+0000 7fb49f7fe640 1 -- 172.21.15.78:0/3155013915 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49800f2e0 con 0x7fb4a8102260 2024-08-18T23:41:45.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.366+0000 7fb4b0358640 1 -- 172.21.15.78:0/3155013915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a81a4ea0 con 0x7fb4a8102260 2024-08-18T23:41:45.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.367+0000 7fb4b0358640 1 -- 172.21.15.78:0/3155013915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a8101da0 con 0x7fb4a8102260 2024-08-18T23:41:45.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.369+0000 7fb49f7fe640 1 -- 172.21.15.78:0/3155013915 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb498018020 con 0x7fb4a8102260 2024-08-18T23:41:45.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.369+0000 7fb49f7fe640 1 --2- 172.21.15.78:0/3155013915 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4940a2b80 0x7fb4940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:45.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.369+0000 7fb4ae0f4640 1 --2- 172.21.15.78:0/3155013915 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4940a2b80 0x7fb4940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:45.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.369+0000 7fb49f7fe640 1 -- 172.21.15.78:0/3155013915 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fb4980cf140 con 0x7fb4a8102260 2024-08-18T23:41:45.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.370+0000 7fb4ae0f4640 1 --2- 172.21.15.78:0/3155013915 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4940a2b80 0x7fb4940a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009990 tx=0x7fb4a4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:45.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.377+0000 7fb49f7fe640 1 -- 172.21.15.78:0/3155013915 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb49801f020 con 0x7fb4a8102260 2024-08-18T23:41:45.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:45 smithi123 ceph-mon[21808]: pgmap v269: 129 pgs: 2 creating+activating, 3 creating+peering, 124 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:45.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:45 smithi123 ceph-mon[21808]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-08-18T23:41:45.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:45 smithi123 ceph-mon[21808]: Cluster is now healthy 2024-08-18T23:41:45.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.550+0000 7fb4b0358640 1 -- 172.21.15.78:0/3155013915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4a81098c0 con 0x7fb4a8102260 2024-08-18T23:41:45.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.552+0000 7fb49f7fe640 1 -- 172.21.15.78:0/3155013915 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24126 (secure 0 0 0) 0x7fb49809be60 con 0x7fb4a8102260 2024-08-18T23:41:45.553 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:41:45.553 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":88,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","created":"2024-08-18T23:34:33.867246+0000","modified":"2024-08-18T23:41:41.811101+0000","last_up_change":"2024-08-18T23:40:42.200381+0000","last_in_change":"2024-08-18T23:40:22.425813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T23:37:22.338211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T23:41:28.920802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T23:41:31.026094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T23:41:37.403238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4de1c78-6ebc-49a9-b9c5-85d43535fab7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6803","nonce":3639645311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6805","nonce":3639645311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6809","nonce":3639645311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6807","nonce":3639645311}]},"public_addr":"172.21.15.78:6803/3639645311","cluster_addr":"172.21.15.78:6805/3639645311","heartbeat_back_addr":"172.21.15.78:6809/3639645311","heartbeat_front_addr":"172.21.15.78:6807/3639645311","state":["exists","up"]},{"osd":1,"uuid":"4f8e0c3d-472b-4230-ad23-2b0a34861251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6801","nonce":4199446368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6803","nonce":4199446368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6807","nonce":4199446368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6805","nonce":4199446368}]},"public_addr":"172.21.15.123:6801/4199446368","cluster_addr":"172.21.15.123:6803/4199446368","heartbeat_back_addr":"172.21.15.123:6807/4199446368","heartbeat_front_addr":"172.21.15.123:6805/4199446368","state":["exists","up"]},{"osd":2,"uuid":"27ef3314-bb6f-413d-be05-e10a105dec9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6801","nonce":2582810057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6803","nonce":2582810057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6807","nonce":2582810057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6805","nonce":2582810057}]},"public_addr":"172.21.15.161:6801/2582810057","cluster_addr":"172.21.15.161:6803/2582810057","heartbeat_back_addr":"172.21.15.161:6807/2582810057","heartbeat_front_addr":"172.21.15.161:6805/2582810057","state":["exists","up"]},{"osd":3,"uuid":"e9e52662-5ef9-4d70-bdd7-2b92165822ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6811","nonce":2697347521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6813","nonce":2697347521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6817","nonce":2697347521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6815","nonce":2697347521}]},"public_addr":"172.21.15.78:6811/2697347521","cluster_addr":"172.21.15.78:6813/2697347521","heartbeat_back_addr":"172.21.15.78:6817/2697347521","heartbeat_front_addr":"172.21.15.78:6815/2697347521","state":["exists","up"]},{"osd":4,"uuid":"5ed26775-79e3-4b24-9557-69fcf5e8e824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6809","nonce":1281511569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6811","nonce":1281511569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6815","nonce":1281511569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6813","nonce":1281511569}]},"public_addr":"172.21.15.123:6809/1281511569","cluster_addr":"172.21.15.123:6811/1281511569","heartbeat_back_addr":"172.21.15.123:6815/1281511569","heartbeat_front_addr":"172.21.15.123:6813/1281511569","state":["exists","up"]},{"osd":5,"uuid":"a9793b1d-5613-48a4-938e-adb718f4fa4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6809","nonce":2751796117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6811","nonce":2751796117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6815","nonce":2751796117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6813","nonce":2751796117}]},"public_addr":"172.21.15.161:6809/2751796117","cluster_addr":"172.21.15.161:6811/2751796117","heartbeat_back_addr":"172.21.15.161:6815/2751796117","heartbeat_front_addr":"172.21.15.161:6813/2751796117","state":["exists","up"]},{"osd":6,"uuid":"5dbc5e9c-a14f-4b71-910a-1e6d4827bb32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6819","nonce":95383223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6821","nonce":95383223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6825","nonce":95383223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6823","nonce":95383223}]},"public_addr":"172.21.15.78:6819/95383223","cluster_addr":"172.21.15.78:6821/95383223","heartbeat_back_addr":"172.21.15.78:6825/95383223","heartbeat_front_addr":"172.21.15.78:6823/95383223","state":["exists","up"]},{"osd":7,"uuid":"26fb7b69-131f-42ee-b745-6fe8b4f65391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6817","nonce":3440305156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6819","nonce":3440305156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6823","nonce":3440305156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6821","nonce":3440305156}]},"public_addr":"172.21.15.123:6817/3440305156","cluster_addr":"172.21.15.123:6819/3440305156","heartbeat_back_addr":"172.21.15.123:6823/3440305156","heartbeat_front_addr":"172.21.15.123:6821/3440305156","state":["exists","up"]},{"osd":8,"uuid":"e267bc50-0c29-4d7b-b368-b1621e29b2df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6817","nonce":1485008778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6819","nonce":1485008778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6823","nonce":1485008778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6821","nonce":1485008778}]},"public_addr":"172.21.15.161:6817/1485008778","cluster_addr":"172.21.15.161:6819/1485008778","heartbeat_back_addr":"172.21.15.161:6823/1485008778","heartbeat_front_addr":"172.21.15.161:6821/1485008778","state":["exists","up"]},{"osd":9,"uuid":"b7654818-b767-4fe6-8211-346297895f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6827","nonce":2884475661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6829","nonce":2884475661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6833","nonce":2884475661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6831","nonce":2884475661}]},"public_addr":"172.21.15.78:6827/2884475661","cluster_addr":"172.21.15.78:6829/2884475661","heartbeat_back_addr":"172.21.15.78:6833/2884475661","heartbeat_front_addr":"172.21.15.78:6831/2884475661","state":["exists","up"]},{"osd":10,"uuid":"59aa32ff-510e-4b52-8170-08a34a0b6b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6825","nonce":122226286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6827","nonce":122226286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6831","nonce":122226286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6829","nonce":122226286}]},"public_addr":"172.21.15.123:6825/122226286","cluster_addr":"172.21.15.123:6827/122226286","heartbeat_back_addr":"172.21.15.123:6831/122226286","heartbeat_front_addr":"172.21.15.123:6829/122226286","state":["exists","up"]},{"osd":11,"uuid":"67bacab4-d078-4c5c-b4ff-9f6a8a00f507","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6825","nonce":2037560985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6827","nonce":2037560985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6831","nonce":2037560985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6829","nonce":2037560985}]},"public_addr":"172.21.15.161:6825/2037560985","cluster_addr":"172.21.15.161:6827/2037560985","heartbeat_back_addr":"172.21.15.161:6831/2037560985","heartbeat_front_addr":"172.21.15.161:6829/2037560985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:36.826044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:57.688421+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:18.297424+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:40.355215+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:02.167902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:23.968759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:46.254083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:08.522038+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:31.028119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:54.119807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:16.847034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:39.607886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6800/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3040499240":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6801/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:6800/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/757409165":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/2562072965":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/636184437":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/1570708294":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3137806628":"2024-08-19T23:34:59.021165+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T23:41:45.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.554+0000 7fb4b0358640 1 -- 172.21.15.78:0/3155013915 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4940a2b80 msgr2=0x7fb4940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:45.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.554+0000 7fb4b0358640 1 --2- 172.21.15.78:0/3155013915 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4940a2b80 0x7fb4940a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009990 tx=0x7fb4a4009110 comp rx=0 tx=0).stop 2024-08-18T23:41:45.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.554+0000 7fb4b0358640 1 -- 172.21.15.78:0/3155013915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4a8102260 msgr2=0x7fb4a8198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:45.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.554+0000 7fb4b0358640 1 --2- 172.21.15.78:0/3155013915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4a8102260 0x7fb4a8198860 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb49800b4a0 tx=0x7fb49800b8a0 comp rx=0 tx=0).stop 2024-08-18T23:41:45.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.554+0000 7fb4b0358640 1 -- 172.21.15.78:0/3155013915 shutdown_connections 2024-08-18T23:41:45.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.554+0000 7fb4b0358640 1 --2- 172.21.15.78:0/3155013915 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4940a2b80 0x7fb4940a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:45.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.554+0000 7fb4b0358640 1 --2- 172.21.15.78:0/3155013915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4a81078c0 0x7fb4a819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:45.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.554+0000 7fb4b0358640 1 --2- 172.21.15.78:0/3155013915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4a8102260 0x7fb4a8198860 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:45.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.554+0000 7fb4b0358640 1 --2- 172.21.15.78:0/3155013915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4a81018c0 0x7fb4a8198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:45.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.554+0000 7fb4b0358640 1 -- 172.21.15.78:0/3155013915 >> 172.21.15.78:0/3155013915 conn(0x7fb4a80fd620 msgr2=0x7fb4a80fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:45.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.554+0000 7fb4b0358640 1 -- 172.21.15.78:0/3155013915 shutdown_connections 2024-08-18T23:41:45.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:45.554+0000 7fb4b0358640 1 -- 172.21.15.78:0/3155013915 wait complete. 2024-08-18T23:41:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:45 smithi161 ceph-mon[22027]: pgmap v269: 129 pgs: 2 creating+activating, 3 creating+peering, 124 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:45 smithi161 ceph-mon[22027]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-08-18T23:41:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:45 smithi161 ceph-mon[22027]: Cluster is now healthy 2024-08-18T23:41:46.324 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-18T23:41:46.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:46 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3155013915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:41:46.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:46 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3155013915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:41:46.575 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:46.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:46 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3155013915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:41:47.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.035+0000 7f46415c8640 1 -- 172.21.15.78:0/3501945604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f463c102240 msgr2=0x7f463c10e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:47.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.035+0000 7f46415c8640 1 --2- 172.21.15.78:0/3501945604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f463c102240 0x7f463c10e770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4634009930 tx=0x7f463402a420 comp rx=0 tx=0).stop 2024-08-18T23:41:47.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.035+0000 7f46415c8640 1 -- 172.21.15.78:0/3501945604 shutdown_connections 2024-08-18T23:41:47.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.035+0000 7f46415c8640 1 --2- 172.21.15.78:0/3501945604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f463c102240 0x7f463c10e770 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:47.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.035+0000 7f46415c8640 1 --2- 172.21.15.78:0/3501945604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f463c1018a0 0x7f463c101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:47.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.035+0000 7f46415c8640 1 --2- 172.21.15.78:0/3501945604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f463c1078a0 0x7f463c107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:47.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.035+0000 7f46415c8640 1 -- 172.21.15.78:0/3501945604 >> 172.21.15.78:0/3501945604 conn(0x7f463c0fd5e0 msgr2=0x7f463c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:47.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.035+0000 7f46415c8640 1 -- 172.21.15.78:0/3501945604 shutdown_connections 2024-08-18T23:41:47.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.035+0000 7f46415c8640 1 -- 172.21.15.78:0/3501945604 wait complete. 2024-08-18T23:41:47.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.035+0000 7f46415c8640 1 Processor -- start 2024-08-18T23:41:47.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.036+0000 7f46415c8640 1 -- start start 2024-08-18T23:41:47.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.036+0000 7f46415c8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f463c1018a0 0x7f463c074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:47.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.036+0000 7f46415c8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f463c102240 0x7f463c074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:47.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.036+0000 7f46415c8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f463c1078a0 0x7f463c074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.036+0000 7f46415c8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f463c111cc0 con 0x7f463c1078a0 2024-08-18T23:41:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.036+0000 7f46415c8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f463c111b40 con 0x7f463c1018a0 2024-08-18T23:41:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.036+0000 7f46415c8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f463c111e40 con 0x7f463c102240 2024-08-18T23:41:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.036+0000 7f463b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f463c1078a0 0x7f463c074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.036+0000 7f463b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f463c1078a0 0x7f463c074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44116/0 (socket says 172.21.15.78:44116) 2024-08-18T23:41:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.036+0000 7f463b7fe640 1 -- 172.21.15.78:0/1466941722 learned_addr learned my addr 172.21.15.78:0/1466941722 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.036+0000 7f463affd640 1 --2- 172.21.15.78:0/1466941722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f463c1018a0 0x7f463c074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.036+0000 7f463b7fe640 1 -- 172.21.15.78:0/1466941722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f463c102240 msgr2=0x7f463c074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.036+0000 7f463b7fe640 1 --2- 172.21.15.78:0/1466941722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f463c102240 0x7f463c074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.036+0000 7f463b7fe640 1 -- 172.21.15.78:0/1466941722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f463c1018a0 msgr2=0x7f463c074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.036+0000 7f463b7fe640 1 --2- 172.21.15.78:0/1466941722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f463c1018a0 0x7f463c074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:47.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.036+0000 7f463b7fe640 1 -- 172.21.15.78:0/1466941722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c1a6670 con 0x7f463c1078a0 2024-08-18T23:41:47.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.036+0000 7f463affd640 1 --2- 172.21.15.78:0/1466941722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f463c1018a0 0x7f463c074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:41:47.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.036+0000 7f463b7fe640 1 --2- 172.21.15.78:0/1466941722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f463c1078a0 0x7f463c074ca0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4634005ab0 tx=0x7f46340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:47.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.037+0000 7f4633fff640 1 -- 172.21.15.78:0/1466941722 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463403f030 con 0x7f463c1078a0 2024-08-18T23:41:47.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.037+0000 7f4633fff640 1 -- 172.21.15.78:0/1466941722 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4634003ce0 con 0x7f463c1078a0 2024-08-18T23:41:47.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.037+0000 7f46415c8640 1 -- 172.21.15.78:0/1466941722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463c1a6900 con 0x7f463c1078a0 2024-08-18T23:41:47.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.037+0000 7f4633fff640 1 -- 172.21.15.78:0/1466941722 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634035070 con 0x7f463c1078a0 2024-08-18T23:41:47.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.037+0000 7f46415c8640 1 -- 172.21.15.78:0/1466941722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f463c1a6e30 con 0x7f463c1078a0 2024-08-18T23:41:47.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.038+0000 7f46415c8640 1 -- 172.21.15.78:0/1466941722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f463c101d00 con 0x7f463c1078a0 2024-08-18T23:41:47.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.039+0000 7f4633fff640 1 -- 172.21.15.78:0/1466941722 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4634042030 con 0x7f463c1078a0 2024-08-18T23:41:47.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.040+0000 7f4633fff640 1 --2- 172.21.15.78:0/1466941722 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f46280a2c00 0x7f46280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:47.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.040+0000 7f463affd640 1 --2- 172.21.15.78:0/1466941722 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f46280a2c00 0x7f46280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:47.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.040+0000 7f4633fff640 1 -- 172.21.15.78:0/1466941722 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f46340ef890 con 0x7f463c1078a0 2024-08-18T23:41:47.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.040+0000 7f463affd640 1 --2- 172.21.15.78:0/1466941722 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f46280a2c00 0x7f46280a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f462c007eb0 tx=0x7f462c0073c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:47.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.044+0000 7f4633fff640 1 -- 172.21.15.78:0/1466941722 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f46340bc550 con 0x7f463c1078a0 2024-08-18T23:41:47.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.216+0000 7f46415c8640 1 -- 172.21.15.78:0/1466941722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f463c1098a0 con 0x7f463c1078a0 2024-08-18T23:41:47.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:47 smithi078 ceph-mon[18440]: pgmap v270: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:47.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.218+0000 7f4633fff640 1 -- 172.21.15.78:0/1466941722 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24126 (secure 0 0 0) 0x7f46340bbe50 con 0x7f463c1078a0 2024-08-18T23:41:47.220 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:41:47.220 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":88,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","created":"2024-08-18T23:34:33.867246+0000","modified":"2024-08-18T23:41:41.811101+0000","last_up_change":"2024-08-18T23:40:42.200381+0000","last_in_change":"2024-08-18T23:40:22.425813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T23:37:22.338211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T23:41:28.920802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T23:41:31.026094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T23:41:37.403238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4de1c78-6ebc-49a9-b9c5-85d43535fab7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6803","nonce":3639645311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6805","nonce":3639645311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6809","nonce":3639645311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6807","nonce":3639645311}]},"public_addr":"172.21.15.78:6803/3639645311","cluster_addr":"172.21.15.78:6805/3639645311","heartbeat_back_addr":"172.21.15.78:6809/3639645311","heartbeat_front_addr":"172.21.15.78:6807/3639645311","state":["exists","up"]},{"osd":1,"uuid":"4f8e0c3d-472b-4230-ad23-2b0a34861251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6801","nonce":4199446368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6803","nonce":4199446368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6807","nonce":4199446368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6805","nonce":4199446368}]},"public_addr":"172.21.15.123:6801/4199446368","cluster_addr":"172.21.15.123:6803/4199446368","heartbeat_back_addr":"172.21.15.123:6807/4199446368","heartbeat_front_addr":"172.21.15.123:6805/4199446368","state":["exists","up"]},{"osd":2,"uuid":"27ef3314-bb6f-413d-be05-e10a105dec9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6801","nonce":2582810057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6803","nonce":2582810057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6807","nonce":2582810057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6805","nonce":2582810057}]},"public_addr":"172.21.15.161:6801/2582810057","cluster_addr":"172.21.15.161:6803/2582810057","heartbeat_back_addr":"172.21.15.161:6807/2582810057","heartbeat_front_addr":"172.21.15.161:6805/2582810057","state":["exists","up"]},{"osd":3,"uuid":"e9e52662-5ef9-4d70-bdd7-2b92165822ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6811","nonce":2697347521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6813","nonce":2697347521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6817","nonce":2697347521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6815","nonce":2697347521}]},"public_addr":"172.21.15.78:6811/2697347521","cluster_addr":"172.21.15.78:6813/2697347521","heartbeat_back_addr":"172.21.15.78:6817/2697347521","heartbeat_front_addr":"172.21.15.78:6815/2697347521","state":["exists","up"]},{"osd":4,"uuid":"5ed26775-79e3-4b24-9557-69fcf5e8e824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6809","nonce":1281511569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6811","nonce":1281511569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6815","nonce":1281511569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6813","nonce":1281511569}]},"public_addr":"172.21.15.123:6809/1281511569","cluster_addr":"172.21.15.123:6811/1281511569","heartbeat_back_addr":"172.21.15.123:6815/1281511569","heartbeat_front_addr":"172.21.15.123:6813/1281511569","state":["exists","up"]},{"osd":5,"uuid":"a9793b1d-5613-48a4-938e-adb718f4fa4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6809","nonce":2751796117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6811","nonce":2751796117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6815","nonce":2751796117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6813","nonce":2751796117}]},"public_addr":"172.21.15.161:6809/2751796117","cluster_addr":"172.21.15.161:6811/2751796117","heartbeat_back_addr":"172.21.15.161:6815/2751796117","heartbeat_front_addr":"172.21.15.161:6813/2751796117","state":["exists","up"]},{"osd":6,"uuid":"5dbc5e9c-a14f-4b71-910a-1e6d4827bb32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6819","nonce":95383223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6821","nonce":95383223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6825","nonce":95383223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6823","nonce":95383223}]},"public_addr":"172.21.15.78:6819/95383223","cluster_addr":"172.21.15.78:6821/95383223","heartbeat_back_addr":"172.21.15.78:6825/95383223","heartbeat_front_addr":"172.21.15.78:6823/95383223","state":["exists","up"]},{"osd":7,"uuid":"26fb7b69-131f-42ee-b745-6fe8b4f65391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6817","nonce":3440305156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6819","nonce":3440305156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6823","nonce":3440305156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6821","nonce":3440305156}]},"public_addr":"172.21.15.123:6817/3440305156","cluster_addr":"172.21.15.123:6819/3440305156","heartbeat_back_addr":"172.21.15.123:6823/3440305156","heartbeat_front_addr":"172.21.15.123:6821/3440305156","state":["exists","up"]},{"osd":8,"uuid":"e267bc50-0c29-4d7b-b368-b1621e29b2df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6817","nonce":1485008778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6819","nonce":1485008778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6823","nonce":1485008778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6821","nonce":1485008778}]},"public_addr":"172.21.15.161:6817/1485008778","cluster_addr":"172.21.15.161:6819/1485008778","heartbeat_back_addr":"172.21.15.161:6823/1485008778","heartbeat_front_addr":"172.21.15.161:6821/1485008778","state":["exists","up"]},{"osd":9,"uuid":"b7654818-b767-4fe6-8211-346297895f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6827","nonce":2884475661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6829","nonce":2884475661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6833","nonce":2884475661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6831","nonce":2884475661}]},"public_addr":"172.21.15.78:6827/2884475661","cluster_addr":"172.21.15.78:6829/2884475661","heartbeat_back_addr":"172.21.15.78:6833/2884475661","heartbeat_front_addr":"172.21.15.78:6831/2884475661","state":["exists","up"]},{"osd":10,"uuid":"59aa32ff-510e-4b52-8170-08a34a0b6b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6825","nonce":122226286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6827","nonce":122226286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6831","nonce":122226286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6829","nonce":122226286}]},"public_addr":"172.21.15.123:6825/122226286","cluster_addr":"172.21.15.123:6827/122226286","heartbeat_back_addr":"172.21.15.123:6831/122226286","heartbeat_front_addr":"172.21.15.123:6829/122226286","state":["exists","up"]},{"osd":11,"uuid":"67bacab4-d078-4c5c-b4ff-9f6a8a00f507","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6825","nonce":2037560985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6827","nonce":2037560985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6831","nonce":2037560985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6829","nonce":2037560985}]},"public_addr":"172.21.15.161:6825/2037560985","cluster_addr":"172.21.15.161:6827/2037560985","heartbeat_back_addr":"172.21.15.161:6831/2037560985","heartbeat_front_addr":"172.21.15.161:6829/2037560985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:36.826044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:57.688421+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:18.297424+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:40.355215+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:02.167902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:23.968759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:46.254083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:08.522038+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:31.028119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:54.119807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:16.847034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:39.607886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6800/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3040499240":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6801/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:6800/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/757409165":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/2562072965":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/636184437":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/1570708294":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3137806628":"2024-08-19T23:34:59.021165+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T23:41:47.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.220+0000 7f46415c8640 1 -- 172.21.15.78:0/1466941722 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f46280a2c00 msgr2=0x7f46280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:47.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.220+0000 7f46415c8640 1 --2- 172.21.15.78:0/1466941722 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f46280a2c00 0x7f46280a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f462c007eb0 tx=0x7f462c0073c0 comp rx=0 tx=0).stop 2024-08-18T23:41:47.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.220+0000 7f46415c8640 1 -- 172.21.15.78:0/1466941722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f463c1078a0 msgr2=0x7f463c074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:47.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.220+0000 7f46415c8640 1 --2- 172.21.15.78:0/1466941722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f463c1078a0 0x7f463c074ca0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4634005ab0 tx=0x7f46340042f0 comp rx=0 tx=0).stop 2024-08-18T23:41:47.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.220+0000 7f46415c8640 1 -- 172.21.15.78:0/1466941722 shutdown_connections 2024-08-18T23:41:47.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.220+0000 7f46415c8640 1 --2- 172.21.15.78:0/1466941722 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f46280a2c00 0x7f46280a50c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:47.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.220+0000 7f46415c8640 1 --2- 172.21.15.78:0/1466941722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f463c1078a0 0x7f463c074ca0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:47.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.220+0000 7f46415c8640 1 --2- 172.21.15.78:0/1466941722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f463c102240 0x7f463c074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:47.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.220+0000 7f46415c8640 1 --2- 172.21.15.78:0/1466941722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f463c1018a0 0x7f463c074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:47.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.220+0000 7f46415c8640 1 -- 172.21.15.78:0/1466941722 >> 172.21.15.78:0/1466941722 conn(0x7f463c0fd5e0 msgr2=0x7f463c0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:47.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.221+0000 7f46415c8640 1 -- 172.21.15.78:0/1466941722 shutdown_connections 2024-08-18T23:41:47.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:47.221+0000 7f46415c8640 1 -- 172.21.15.78:0/1466941722 wait complete. 2024-08-18T23:41:47.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:47 smithi123 ceph-mon[21808]: pgmap v270: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:47 smithi161 ceph-mon[22027]: pgmap v270: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:47.958 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:41:48.213 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:48.335 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:48 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1466941722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:41:48.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:48 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1466941722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:41:48.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:48 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1466941722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:41:48.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.727+0000 7fb639dd1640 1 -- 172.21.15.78:0/4146682807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb634102a60 msgr2=0x7fb634102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:48.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.727+0000 7fb639dd1640 1 --2- 172.21.15.78:0/4146682807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb634102a60 0x7fb634102e40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb624009930 tx=0x7fb62402a420 comp rx=0 tx=0).stop 2024-08-18T23:41:48.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.727+0000 7fb639dd1640 1 -- 172.21.15.78:0/4146682807 shutdown_connections 2024-08-18T23:41:48.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.727+0000 7fb639dd1640 1 --2- 172.21.15.78:0/4146682807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb634107fd0 0x7fb63410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:48.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.727+0000 7fb639dd1640 1 --2- 172.21.15.78:0/4146682807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb634103410 0x7fb6341078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:48.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.727+0000 7fb639dd1640 1 --2- 172.21.15.78:0/4146682807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb634102a60 0x7fb634102e40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:48.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.727+0000 7fb639dd1640 1 -- 172.21.15.78:0/4146682807 >> 172.21.15.78:0/4146682807 conn(0x7fb6340fc500 msgr2=0x7fb6340fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:48.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.727+0000 7fb639dd1640 1 -- 172.21.15.78:0/4146682807 shutdown_connections 2024-08-18T23:41:48.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.727+0000 7fb639dd1640 1 -- 172.21.15.78:0/4146682807 wait complete. 2024-08-18T23:41:48.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.728+0000 7fb639dd1640 1 Processor -- start 2024-08-18T23:41:48.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.728+0000 7fb639dd1640 1 -- start start 2024-08-18T23:41:48.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.728+0000 7fb639dd1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb634102a60 0x7fb634198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:48.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.728+0000 7fb639dd1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb634103410 0x7fb634198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:48.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.728+0000 7fb639dd1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb634107fd0 0x7fb63419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:48.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.728+0000 7fb639dd1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb634111e30 con 0x7fb634107fd0 2024-08-18T23:41:48.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.728+0000 7fb639dd1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb634111cb0 con 0x7fb634102a60 2024-08-18T23:41:48.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.728+0000 7fb639dd1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb634111fb0 con 0x7fb634103410 2024-08-18T23:41:48.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.728+0000 7fb633fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb634107fd0 0x7fb63419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:48.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.728+0000 7fb6337fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb634102a60 0x7fb634198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:48.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.728+0000 7fb633fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb634107fd0 0x7fb63419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44140/0 (socket says 172.21.15.78:44140) 2024-08-18T23:41:48.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.728+0000 7fb633fff640 1 -- 172.21.15.78:0/3009523308 learned_addr learned my addr 172.21.15.78:0/3009523308 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:48.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.728+0000 7fb632ffd640 1 --2- 172.21.15.78:0/3009523308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb634103410 0x7fb634198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:48.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.728+0000 7fb633fff640 1 -- 172.21.15.78:0/3009523308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb634103410 msgr2=0x7fb634198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:48.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.728+0000 7fb633fff640 1 --2- 172.21.15.78:0/3009523308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb634103410 0x7fb634198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:48.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.728+0000 7fb633fff640 1 -- 172.21.15.78:0/3009523308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb634102a60 msgr2=0x7fb634198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:48.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.728+0000 7fb633fff640 1 --2- 172.21.15.78:0/3009523308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb634102a60 0x7fb634198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:48.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.728+0000 7fb633fff640 1 -- 172.21.15.78:0/3009523308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63419d2e0 con 0x7fb634107fd0 2024-08-18T23:41:48.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.728+0000 7fb632ffd640 1 --2- 172.21.15.78:0/3009523308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb634103410 0x7fb634198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:41:48.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.729+0000 7fb633fff640 1 --2- 172.21.15.78:0/3009523308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb634107fd0 0x7fb63419cc00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00b970 tx=0x7fb62c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:48.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.729+0000 7fb630ff9640 1 -- 172.21.15.78:0/3009523308 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c019070 con 0x7fb634107fd0 2024-08-18T23:41:48.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.729+0000 7fb639dd1640 1 -- 172.21.15.78:0/3009523308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63419d5d0 con 0x7fb634107fd0 2024-08-18T23:41:48.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.729+0000 7fb630ff9640 1 -- 172.21.15.78:0/3009523308 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb62c00cb40 con 0x7fb634107fd0 2024-08-18T23:41:48.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.729+0000 7fb630ff9640 1 -- 172.21.15.78:0/3009523308 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c007650 con 0x7fb634107fd0 2024-08-18T23:41:48.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.729+0000 7fb639dd1640 1 -- 172.21.15.78:0/3009523308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6341a4eb0 con 0x7fb634107fd0 2024-08-18T23:41:48.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.730+0000 7fb639dd1640 1 -- 172.21.15.78:0/3009523308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb63406a580 con 0x7fb634107fd0 2024-08-18T23:41:48.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.731+0000 7fb630ff9640 1 -- 172.21.15.78:0/3009523308 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb62c01c070 con 0x7fb634107fd0 2024-08-18T23:41:48.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.732+0000 7fb630ff9640 1 --2- 172.21.15.78:0/3009523308 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb61c0a2b80 0x7fb61c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:48.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.732+0000 7fb6337fe640 1 --2- 172.21.15.78:0/3009523308 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb61c0a2b80 0x7fb61c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:48.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.732+0000 7fb630ff9640 1 -- 172.21.15.78:0/3009523308 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fb62c0ced20 con 0x7fb634107fd0 2024-08-18T23:41:48.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.732+0000 7fb6337fe640 1 --2- 172.21.15.78:0/3009523308 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb61c0a2b80 0x7fb61c0a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb624009900 tx=0x7fb6240058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:48.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.735+0000 7fb630ff9640 1 -- 172.21.15.78:0/3009523308 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb62c09b3f0 con 0x7fb634107fd0 2024-08-18T23:41:48.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.935+0000 7fb639dd1640 1 -- 172.21.15.78:0/3009523308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb634102e40 con 0x7fb634107fd0 2024-08-18T23:41:48.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.936+0000 7fb630ff9640 1 -- 172.21.15.78:0/3009523308 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11857 (secure 0 0 0) 0x7fb634102e40 con 0x7fb634107fd0 2024-08-18T23:41:48.937 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:41:48.937 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":16,"btime":"2024-08-18T23:41:42:176701+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24403,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:41:41.811239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24452,"qdb_cluster":[24452]},"id":1}]} 2024-08-18T23:41:48.937 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 16 2024-08-18T23:41:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.938+0000 7fb639dd1640 1 -- 172.21.15.78:0/3009523308 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb61c0a2b80 msgr2=0x7fb61c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.938+0000 7fb639dd1640 1 --2- 172.21.15.78:0/3009523308 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb61c0a2b80 0x7fb61c0a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb624009900 tx=0x7fb6240058d0 comp rx=0 tx=0).stop 2024-08-18T23:41:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.938+0000 7fb639dd1640 1 -- 172.21.15.78:0/3009523308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb634107fd0 msgr2=0x7fb63419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.938+0000 7fb639dd1640 1 --2- 172.21.15.78:0/3009523308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb634107fd0 0x7fb63419cc00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00b970 tx=0x7fb62c00be30 comp rx=0 tx=0).stop 2024-08-18T23:41:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.938+0000 7fb639dd1640 1 -- 172.21.15.78:0/3009523308 shutdown_connections 2024-08-18T23:41:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.938+0000 7fb639dd1640 1 --2- 172.21.15.78:0/3009523308 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb61c0a2b80 0x7fb61c0a5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.938+0000 7fb639dd1640 1 --2- 172.21.15.78:0/3009523308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb634107fd0 0x7fb63419cc00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.938+0000 7fb639dd1640 1 --2- 172.21.15.78:0/3009523308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb634103410 0x7fb634198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.938+0000 7fb639dd1640 1 --2- 172.21.15.78:0/3009523308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb634102a60 0x7fb634198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.938+0000 7fb639dd1640 1 -- 172.21.15.78:0/3009523308 >> 172.21.15.78:0/3009523308 conn(0x7fb6340fc500 msgr2=0x7fb634105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.938+0000 7fb639dd1640 1 -- 172.21.15.78:0/3009523308 shutdown_connections 2024-08-18T23:41:48.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:48.938+0000 7fb639dd1640 1 -- 172.21.15.78:0/3009523308 wait complete. 2024-08-18T23:41:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:49 smithi078 ceph-mon[18440]: pgmap v271: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:49 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3009523308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:49.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:49 smithi123 ceph-mon[21808]: pgmap v271: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:49.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:49 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3009523308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:49 smithi161 ceph-mon[22027]: pgmap v271: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:49 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3009523308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:41:49.726 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-18T23:41:49.977 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:50.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.460+0000 7f9289f55640 1 -- 172.21.15.78:0/1798699681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9284100b50 msgr2=0x7f9284100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:50.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.460+0000 7f9289f55640 1 --2- 172.21.15.78:0/1798699681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9284100b50 0x7f9284100f30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9274009900 tx=0x7f927402a480 comp rx=0 tx=0).stop 2024-08-18T23:41:50.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.460+0000 7f9289f55640 1 -- 172.21.15.78:0/1798699681 shutdown_connections 2024-08-18T23:41:50.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.460+0000 7f9289f55640 1 --2- 172.21.15.78:0/1798699681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9284102c20 0x7f928410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:50.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.460+0000 7f9289f55640 1 --2- 172.21.15.78:0/1798699681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92841022a0 0x7f92841026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:50.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.460+0000 7f9289f55640 1 --2- 172.21.15.78:0/1798699681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9284100b50 0x7f9284100f30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:50.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.460+0000 7f9289f55640 1 -- 172.21.15.78:0/1798699681 >> 172.21.15.78:0/1798699681 conn(0x7f92840fc500 msgr2=0x7f92840fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:50.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.460+0000 7f9289f55640 1 -- 172.21.15.78:0/1798699681 shutdown_connections 2024-08-18T23:41:50.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.460+0000 7f9289f55640 1 -- 172.21.15.78:0/1798699681 wait complete. 2024-08-18T23:41:50.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.460+0000 7f9289f55640 1 Processor -- start 2024-08-18T23:41:50.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.461+0000 7f9289f55640 1 -- start start 2024-08-18T23:41:50.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.461+0000 7f9289f55640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9284100b50 0x7f92841972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:50.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.461+0000 7f9289f55640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92841022a0 0x7f92841977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:50.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.461+0000 7f9289f55640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9284102c20 0x7f928419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:50.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.461+0000 7f9289f55640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9284110d50 con 0x7f9284102c20 2024-08-18T23:41:50.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.461+0000 7f9289f55640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9284110bd0 con 0x7f9284100b50 2024-08-18T23:41:50.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.461+0000 7f9289f55640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9284110ed0 con 0x7f92841022a0 2024-08-18T23:41:50.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.461+0000 7f9283fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9284102c20 0x7f928419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:50.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.461+0000 7f9283fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9284102c20 0x7f928419bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44160/0 (socket says 172.21.15.78:44160) 2024-08-18T23:41:50.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.461+0000 7f9283fff640 1 -- 172.21.15.78:0/3088200168 learned_addr learned my addr 172.21.15.78:0/3088200168 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:50.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.461+0000 7f92837fe640 1 --2- 172.21.15.78:0/3088200168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9284100b50 0x7f92841972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:50.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.461+0000 7f9282ffd640 1 --2- 172.21.15.78:0/3088200168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92841022a0 0x7f92841977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:50.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.461+0000 7f9283fff640 1 -- 172.21.15.78:0/3088200168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92841022a0 msgr2=0x7f92841977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:50.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.461+0000 7f9283fff640 1 --2- 172.21.15.78:0/3088200168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92841022a0 0x7f92841977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:50.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.461+0000 7f9283fff640 1 -- 172.21.15.78:0/3088200168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9284100b50 msgr2=0x7f92841972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:50.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.461+0000 7f9283fff640 1 --2- 172.21.15.78:0/3088200168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9284100b50 0x7f92841972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:50.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.461+0000 7f9283fff640 1 -- 172.21.15.78:0/3088200168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928419c260 con 0x7f9284102c20 2024-08-18T23:41:50.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.461+0000 7f9282ffd640 1 --2- 172.21.15.78:0/3088200168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92841022a0 0x7f92841977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:41:50.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.462+0000 7f9283fff640 1 --2- 172.21.15.78:0/3088200168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9284102c20 0x7f928419bb80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f927c00b970 tx=0x7f927c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:50.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.462+0000 7f9280ff9640 1 -- 172.21.15.78:0/3088200168 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c019070 con 0x7f9284102c20 2024-08-18T23:41:50.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.462+0000 7f9289f55640 1 -- 172.21.15.78:0/3088200168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f928419c550 con 0x7f9284102c20 2024-08-18T23:41:50.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.462+0000 7f9280ff9640 1 -- 172.21.15.78:0/3088200168 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f927c00cb40 con 0x7f9284102c20 2024-08-18T23:41:50.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.462+0000 7f9280ff9640 1 -- 172.21.15.78:0/3088200168 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c007650 con 0x7f9284102c20 2024-08-18T23:41:50.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.462+0000 7f9289f55640 1 -- 172.21.15.78:0/3088200168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92841a3e30 con 0x7f9284102c20 2024-08-18T23:41:50.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.463+0000 7f9289f55640 1 -- 172.21.15.78:0/3088200168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9284100fb0 con 0x7f9284102c20 2024-08-18T23:41:50.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.464+0000 7f9280ff9640 1 -- 172.21.15.78:0/3088200168 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f927c01c070 con 0x7f9284102c20 2024-08-18T23:41:50.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.464+0000 7f9280ff9640 1 --2- 172.21.15.78:0/3088200168 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f92700a2b80 0x7f92700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:50.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.464+0000 7f92837fe640 1 --2- 172.21.15.78:0/3088200168 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f92700a2b80 0x7f92700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:50.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.465+0000 7f9280ff9640 1 -- 172.21.15.78:0/3088200168 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f927c0ce920 con 0x7f9284102c20 2024-08-18T23:41:50.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.465+0000 7f92837fe640 1 --2- 172.21.15.78:0/3088200168 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f92700a2b80 0x7f92700a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9274005ab0 tx=0x7f9274005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:50.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.469+0000 7f9280ff9640 1 -- 172.21.15.78:0/3088200168 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f927c09b640 con 0x7f9284102c20 2024-08-18T23:41:50.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.635+0000 7f9289f55640 1 -- 172.21.15.78:0/3088200168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9284108dc0 con 0x7f9284102c20 2024-08-18T23:41:50.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.637+0000 7f9280ff9640 1 -- 172.21.15.78:0/3088200168 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24126 (secure 0 0 0) 0x7f927c09af40 con 0x7f9284102c20 2024-08-18T23:41:50.639 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:41:50.639 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":88,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","created":"2024-08-18T23:34:33.867246+0000","modified":"2024-08-18T23:41:41.811101+0000","last_up_change":"2024-08-18T23:40:42.200381+0000","last_in_change":"2024-08-18T23:40:22.425813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T23:37:22.338211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T23:41:28.920802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T23:41:31.026094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T23:41:37.403238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4de1c78-6ebc-49a9-b9c5-85d43535fab7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6803","nonce":3639645311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6805","nonce":3639645311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6809","nonce":3639645311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6807","nonce":3639645311}]},"public_addr":"172.21.15.78:6803/3639645311","cluster_addr":"172.21.15.78:6805/3639645311","heartbeat_back_addr":"172.21.15.78:6809/3639645311","heartbeat_front_addr":"172.21.15.78:6807/3639645311","state":["exists","up"]},{"osd":1,"uuid":"4f8e0c3d-472b-4230-ad23-2b0a34861251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6801","nonce":4199446368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6803","nonce":4199446368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6807","nonce":4199446368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6805","nonce":4199446368}]},"public_addr":"172.21.15.123:6801/4199446368","cluster_addr":"172.21.15.123:6803/4199446368","heartbeat_back_addr":"172.21.15.123:6807/4199446368","heartbeat_front_addr":"172.21.15.123:6805/4199446368","state":["exists","up"]},{"osd":2,"uuid":"27ef3314-bb6f-413d-be05-e10a105dec9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6801","nonce":2582810057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6803","nonce":2582810057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6807","nonce":2582810057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6805","nonce":2582810057}]},"public_addr":"172.21.15.161:6801/2582810057","cluster_addr":"172.21.15.161:6803/2582810057","heartbeat_back_addr":"172.21.15.161:6807/2582810057","heartbeat_front_addr":"172.21.15.161:6805/2582810057","state":["exists","up"]},{"osd":3,"uuid":"e9e52662-5ef9-4d70-bdd7-2b92165822ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6811","nonce":2697347521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6813","nonce":2697347521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6817","nonce":2697347521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6815","nonce":2697347521}]},"public_addr":"172.21.15.78:6811/2697347521","cluster_addr":"172.21.15.78:6813/2697347521","heartbeat_back_addr":"172.21.15.78:6817/2697347521","heartbeat_front_addr":"172.21.15.78:6815/2697347521","state":["exists","up"]},{"osd":4,"uuid":"5ed26775-79e3-4b24-9557-69fcf5e8e824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6809","nonce":1281511569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6811","nonce":1281511569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6815","nonce":1281511569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6813","nonce":1281511569}]},"public_addr":"172.21.15.123:6809/1281511569","cluster_addr":"172.21.15.123:6811/1281511569","heartbeat_back_addr":"172.21.15.123:6815/1281511569","heartbeat_front_addr":"172.21.15.123:6813/1281511569","state":["exists","up"]},{"osd":5,"uuid":"a9793b1d-5613-48a4-938e-adb718f4fa4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6809","nonce":2751796117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6811","nonce":2751796117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6815","nonce":2751796117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6813","nonce":2751796117}]},"public_addr":"172.21.15.161:6809/2751796117","cluster_addr":"172.21.15.161:6811/2751796117","heartbeat_back_addr":"172.21.15.161:6815/2751796117","heartbeat_front_addr":"172.21.15.161:6813/2751796117","state":["exists","up"]},{"osd":6,"uuid":"5dbc5e9c-a14f-4b71-910a-1e6d4827bb32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6819","nonce":95383223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6821","nonce":95383223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6825","nonce":95383223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6823","nonce":95383223}]},"public_addr":"172.21.15.78:6819/95383223","cluster_addr":"172.21.15.78:6821/95383223","heartbeat_back_addr":"172.21.15.78:6825/95383223","heartbeat_front_addr":"172.21.15.78:6823/95383223","state":["exists","up"]},{"osd":7,"uuid":"26fb7b69-131f-42ee-b745-6fe8b4f65391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6817","nonce":3440305156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6819","nonce":3440305156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6823","nonce":3440305156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6821","nonce":3440305156}]},"public_addr":"172.21.15.123:6817/3440305156","cluster_addr":"172.21.15.123:6819/3440305156","heartbeat_back_addr":"172.21.15.123:6823/3440305156","heartbeat_front_addr":"172.21.15.123:6821/3440305156","state":["exists","up"]},{"osd":8,"uuid":"e267bc50-0c29-4d7b-b368-b1621e29b2df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6817","nonce":1485008778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6819","nonce":1485008778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6823","nonce":1485008778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6821","nonce":1485008778}]},"public_addr":"172.21.15.161:6817/1485008778","cluster_addr":"172.21.15.161:6819/1485008778","heartbeat_back_addr":"172.21.15.161:6823/1485008778","heartbeat_front_addr":"172.21.15.161:6821/1485008778","state":["exists","up"]},{"osd":9,"uuid":"b7654818-b767-4fe6-8211-346297895f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6827","nonce":2884475661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6829","nonce":2884475661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6833","nonce":2884475661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6831","nonce":2884475661}]},"public_addr":"172.21.15.78:6827/2884475661","cluster_addr":"172.21.15.78:6829/2884475661","heartbeat_back_addr":"172.21.15.78:6833/2884475661","heartbeat_front_addr":"172.21.15.78:6831/2884475661","state":["exists","up"]},{"osd":10,"uuid":"59aa32ff-510e-4b52-8170-08a34a0b6b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6825","nonce":122226286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6827","nonce":122226286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6831","nonce":122226286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6829","nonce":122226286}]},"public_addr":"172.21.15.123:6825/122226286","cluster_addr":"172.21.15.123:6827/122226286","heartbeat_back_addr":"172.21.15.123:6831/122226286","heartbeat_front_addr":"172.21.15.123:6829/122226286","state":["exists","up"]},{"osd":11,"uuid":"67bacab4-d078-4c5c-b4ff-9f6a8a00f507","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6825","nonce":2037560985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6827","nonce":2037560985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6831","nonce":2037560985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6829","nonce":2037560985}]},"public_addr":"172.21.15.161:6825/2037560985","cluster_addr":"172.21.15.161:6827/2037560985","heartbeat_back_addr":"172.21.15.161:6831/2037560985","heartbeat_front_addr":"172.21.15.161:6829/2037560985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:36.826044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:57.688421+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:18.297424+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:40.355215+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:02.167902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:23.968759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:46.254083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:08.522038+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:31.028119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:54.119807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:16.847034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:39.607886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6800/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3040499240":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6801/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:6800/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/757409165":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/2562072965":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/636184437":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/1570708294":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3137806628":"2024-08-19T23:34:59.021165+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T23:41:50.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.639+0000 7f9289f55640 1 -- 172.21.15.78:0/3088200168 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f92700a2b80 msgr2=0x7f92700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:50.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.639+0000 7f9289f55640 1 --2- 172.21.15.78:0/3088200168 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f92700a2b80 0x7f92700a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9274005ab0 tx=0x7f9274005a20 comp rx=0 tx=0).stop 2024-08-18T23:41:50.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.639+0000 7f9289f55640 1 -- 172.21.15.78:0/3088200168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9284102c20 msgr2=0x7f928419bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:50.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.639+0000 7f9289f55640 1 --2- 172.21.15.78:0/3088200168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9284102c20 0x7f928419bb80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f927c00b970 tx=0x7f927c00be30 comp rx=0 tx=0).stop 2024-08-18T23:41:50.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.639+0000 7f9289f55640 1 -- 172.21.15.78:0/3088200168 shutdown_connections 2024-08-18T23:41:50.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.639+0000 7f9289f55640 1 --2- 172.21.15.78:0/3088200168 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f92700a2b80 0x7f92700a5040 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:50.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.639+0000 7f9289f55640 1 --2- 172.21.15.78:0/3088200168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9284102c20 0x7f928419bb80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:50.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.639+0000 7f9289f55640 1 --2- 172.21.15.78:0/3088200168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92841022a0 0x7f92841977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:50.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.639+0000 7f9289f55640 1 --2- 172.21.15.78:0/3088200168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9284100b50 0x7f92841972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:50.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.639+0000 7f9289f55640 1 -- 172.21.15.78:0/3088200168 >> 172.21.15.78:0/3088200168 conn(0x7f92840fc500 msgr2=0x7f92840fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:50.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.639+0000 7f9289f55640 1 -- 172.21.15.78:0/3088200168 shutdown_connections 2024-08-18T23:41:50.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:50.639+0000 7f9289f55640 1 -- 172.21.15.78:0/3088200168 wait complete. 2024-08-18T23:41:51.356 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-08-18T23:41:32.941573+0000', 'modified': '2024-08-18T23:41:41.811239+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24452}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24452': {'gid': 24452, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.123:6839/2656330956', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 2656330956}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 2656330956}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24452, 'qdb_cluster': [24452]} 2024-08-18T23:41:51.356 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-08-18T23:41:51.356 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-18T23:41:51.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:51 smithi078 ceph-mon[18440]: pgmap v272: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:51.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:51 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3088200168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:41:51.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:51 smithi123 ceph-mon[21808]: pgmap v272: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:51.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:51 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3088200168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:41:51.607 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:51 smithi161 ceph-mon[22027]: pgmap v272: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:51 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3088200168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:41:52.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.076+0000 7f740cffa640 1 -- 172.21.15.78:0/2955729795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74081018b0 msgr2=0x7f7408101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:52.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.076+0000 7f740cffa640 1 --2- 172.21.15.78:0/2955729795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74081018b0 0x7f7408101d10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009930 tx=0x7f73f002a420 comp rx=0 tx=0).stop 2024-08-18T23:41:52.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.076+0000 7f740cffa640 1 -- 172.21.15.78:0/2955729795 shutdown_connections 2024-08-18T23:41:52.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.076+0000 7f740cffa640 1 --2- 172.21.15.78:0/2955729795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7408102250 0x7f740810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.076+0000 7f740cffa640 1 --2- 172.21.15.78:0/2955729795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74081018b0 0x7f7408101d10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.076+0000 7f740cffa640 1 --2- 172.21.15.78:0/2955729795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74081078b0 0x7f7408107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.076+0000 7f740cffa640 1 -- 172.21.15.78:0/2955729795 >> 172.21.15.78:0/2955729795 conn(0x7f74080fd5f0 msgr2=0x7f74080ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:52.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.076+0000 7f740cffa640 1 -- 172.21.15.78:0/2955729795 shutdown_connections 2024-08-18T23:41:52.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.076+0000 7f740cffa640 1 -- 172.21.15.78:0/2955729795 wait complete. 2024-08-18T23:41:52.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.077+0000 7f740cffa640 1 Processor -- start 2024-08-18T23:41:52.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.077+0000 7f740cffa640 1 -- start start 2024-08-18T23:41:52.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.077+0000 7f740cffa640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74081018b0 0x7f7408198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:52.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.077+0000 7f740cffa640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7408102250 0x7f7408198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:52.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.077+0000 7f740cffa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74081078b0 0x7f740819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:52.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.077+0000 7f740cffa640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7408111d40 con 0x7f74081078b0 2024-08-18T23:41:52.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.077+0000 7f740cffa640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7408111bc0 con 0x7f74081018b0 2024-08-18T23:41:52.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.077+0000 7f740cffa640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7408111ec0 con 0x7f7408102250 2024-08-18T23:41:52.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.077+0000 7f7406d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74081078b0 0x7f740819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:52.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.077+0000 7f7406d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74081078b0 0x7f740819cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44172/0 (socket says 172.21.15.78:44172) 2024-08-18T23:41:52.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.077+0000 7f7406d9d640 1 -- 172.21.15.78:0/1205219090 learned_addr learned my addr 172.21.15.78:0/1205219090 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:52.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.077+0000 7f740659c640 1 --2- 172.21.15.78:0/1205219090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74081018b0 0x7f7408198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:52.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.077+0000 7f7406d9d640 1 -- 172.21.15.78:0/1205219090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7408102250 msgr2=0x7f7408198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:52.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.077+0000 7f7406d9d640 1 --2- 172.21.15.78:0/1205219090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7408102250 0x7f7408198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.077+0000 7f7406d9d640 1 -- 172.21.15.78:0/1205219090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74081018b0 msgr2=0x7f7408198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:52.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.077+0000 7f7406d9d640 1 --2- 172.21.15.78:0/1205219090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74081018b0 0x7f7408198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.077+0000 7f7406d9d640 1 -- 172.21.15.78:0/1205219090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740819d1f0 con 0x7f74081078b0 2024-08-18T23:41:52.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.077+0000 7f740659c640 1 --2- 172.21.15.78:0/1205219090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74081018b0 0x7f7408198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:41:52.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.078+0000 7f7406d9d640 1 --2- 172.21.15.78:0/1205219090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74081078b0 0x7f740819cb10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f740000b970 tx=0x7f740000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:52.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.078+0000 7f73ff7fe640 1 -- 172.21.15.78:0/1205219090 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400019070 con 0x7f74081078b0 2024-08-18T23:41:52.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.078+0000 7f73ff7fe640 1 -- 172.21.15.78:0/1205219090 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f740000cb40 con 0x7f74081078b0 2024-08-18T23:41:52.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.078+0000 7f740cffa640 1 -- 172.21.15.78:0/1205219090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740819d4e0 con 0x7f74081078b0 2024-08-18T23:41:52.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.078+0000 7f73ff7fe640 1 -- 172.21.15.78:0/1205219090 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400007650 con 0x7f74081078b0 2024-08-18T23:41:52.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.078+0000 7f740cffa640 1 -- 172.21.15.78:0/1205219090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74081a4dc0 con 0x7f74081078b0 2024-08-18T23:41:52.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.081+0000 7f73ff7fe640 1 -- 172.21.15.78:0/1205219090 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f740001c070 con 0x7f74081078b0 2024-08-18T23:41:52.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.081+0000 7f740f3f8740 1 Processor -- start 2024-08-18T23:41:52.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.081+0000 7f740f3f8740 1 -- start start 2024-08-18T23:41:52.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.081+0000 7f740f3f8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558df2b40630 0x558df2b40a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:52.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.081+0000 7f740f3f8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 0x558df2b4b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:52.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.081+0000 7f740f3f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b4b810 0x558df2b4dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:52.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.081+0000 7f740f3f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558df2aa5710 con 0x558df2b4b810 2024-08-18T23:41:52.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.081+0000 7f740f3f8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558df2a3d770 con 0x558df2b40630 2024-08-18T23:41:52.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.081+0000 7f740f3f8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558df2aa16d0 con 0x558df2b40f50 2024-08-18T23:41:52.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.081+0000 7f73fffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b4b810 0x558df2b4dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:52.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.081+0000 7f73fffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b4b810 0x558df2b4dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44176/0 (socket says 172.21.15.78:44176) 2024-08-18T23:41:52.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.081+0000 7f73fffff640 1 -- 172.21.15.78:0/2701466257 learned_addr learned my addr 172.21.15.78:0/2701466257 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:52.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.081+0000 7f73ff7fe640 1 --2- 172.21.15.78:0/1205219090 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f73f40a2b80 0x7f73f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:52.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.082+0000 7f73fffff640 1 --2- 172.21.15.78:0/2701466257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558df2b40630 0x558df2b40a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:52.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.082+0000 7f740659c640 1 --2- 172.21.15.78:0/2701466257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 0x558df2b4b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:52.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.082+0000 7f7406d9d640 1 --2- 172.21.15.78:0/1205219090 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f73f40a2b80 0x7f73f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:52.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.082+0000 7f73ff7fe640 1 -- 172.21.15.78:0/1205219090 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f74000ce920 con 0x7f74081078b0 2024-08-18T23:41:52.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.082+0000 7f73fffff640 1 -- 172.21.15.78:0/2701466257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 msgr2=0x558df2b4b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:41:52.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.082+0000 7f73fffff640 1 --2- 172.21.15.78:0/2701466257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 0x558df2b4b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.082+0000 7f73fffff640 1 -- 172.21.15.78:0/2701466257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558df2b40630 msgr2=0x558df2b40a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:41:52.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.082+0000 7f73fffff640 1 --2- 172.21.15.78:0/2701466257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558df2b40630 0x558df2b40a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.082+0000 7f73fffff640 1 -- 172.21.15.78:0/2701466257 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df2a443e0 con 0x558df2b4b810 2024-08-18T23:41:52.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.082+0000 7f73fffff640 1 --2- 172.21.15.78:0/2701466257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b4b810 0x558df2b4dc00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f73f0031c10 tx=0x7f73f00056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2384c138334029eb server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:52.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.082+0000 7f7406d9d640 1 --2- 172.21.15.78:0/1205219090 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f73f40a2b80 0x7f73f40a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f740009ac40 tx=0x7f7400090420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:52.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.082+0000 7f73fcff9640 1 -- 172.21.15.78:0/2701466257 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0035070 con 0x558df2b4b810 2024-08-18T23:41:52.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.082+0000 7f73fcff9640 1 -- 172.21.15.78:0/2701466257 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f0004230 con 0x558df2b4b810 2024-08-18T23:41:52.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.082+0000 7f73fcff9640 1 -- 172.21.15.78:0/2701466257 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f003c670 con 0x558df2b4b810 2024-08-18T23:41:52.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.083+0000 7f740f3f8740 1 -- 172.21.15.78:0/2701466257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b4b810 msgr2=0x558df2b4dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:52.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.083+0000 7f740f3f8740 1 --2- 172.21.15.78:0/2701466257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b4b810 0x558df2b4dc00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f73f0031c10 tx=0x7f73f00056d0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.083+0000 7f740f3f8740 1 -- 172.21.15.78:0/2701466257 shutdown_connections 2024-08-18T23:41:52.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.083+0000 7f740f3f8740 1 --2- 172.21.15.78:0/2701466257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b4b810 0x558df2b4dc00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.083+0000 7f740f3f8740 1 --2- 172.21.15.78:0/2701466257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 0x558df2b4b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.083+0000 7f740f3f8740 1 --2- 172.21.15.78:0/2701466257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558df2b40630 0x558df2b40a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.083+0000 7f740f3f8740 1 -- 172.21.15.78:0/2701466257 >> 172.21.15.78:0/2701466257 conn(0x558df2b3cbc0 msgr2=0x558df2b3cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:52.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.083+0000 7f740f3f8740 1 -- 172.21.15.78:0/2701466257 shutdown_connections 2024-08-18T23:41:52.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.083+0000 7f740f3f8740 1 -- 172.21.15.78:0/2701466257 wait complete. 2024-08-18T23:41:52.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.083+0000 7f740f3f8740 1 -- start start 2024-08-18T23:41:52.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.084+0000 7f740f3f8740 1 Processor -- start 2024-08-18T23:41:52.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.084+0000 7f740f3f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b40630 0x558df2b487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:52.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.084+0000 7f740f3f8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 0x558df2b48d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:52.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.084+0000 7f740f3f8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558df2b4b810 0x558df2b49240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:52.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.084+0000 7f740f3f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558df2b4e3e0 con 0x558df2b40630 2024-08-18T23:41:52.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.084+0000 7f740f3f8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558df2a60970 con 0x558df2b4b810 2024-08-18T23:41:52.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.084+0000 7f740f3f8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558df2b4e560 con 0x558df2b40f50 2024-08-18T23:41:52.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.084+0000 7f740f3f8740 10 client.0 _ll_register_callbacks cb 0x558df27b47b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:41:52.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.084+0000 7f73fffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b40630 0x558df2b487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:52.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.084+0000 7f73fffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b40630 0x558df2b487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44186/0 (socket says 172.21.15.78:44186) 2024-08-18T23:41:52.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.084+0000 7f73fffff640 1 -- 172.21.15.78:0/2020827059 learned_addr learned my addr 172.21.15.78:0/2020827059 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:52.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.084+0000 7f73fffff640 1 -- 172.21.15.78:0/2020827059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 msgr2=0x558df2b48d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:41:52.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.084+0000 7f73fffff640 1 --2- 172.21.15.78:0/2020827059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 0x558df2b48d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.084+0000 7f73fffff640 1 -- 172.21.15.78:0/2020827059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558df2b4b810 msgr2=0x558df2b49240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:41:52.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.084+0000 7f73fffff640 1 --2- 172.21.15.78:0/2020827059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558df2b4b810 0x558df2b49240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.084+0000 7f73fffff640 1 -- 172.21.15.78:0/2020827059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df28172b0 con 0x558df2b40630 2024-08-18T23:41:52.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.085+0000 7f73fffff640 1 --2- 172.21.15.78:0/2020827059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b40630 0x558df2b487c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f73f0005ec0 tx=0x7f73f0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975e6fb694791510 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:52.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.085+0000 7f73edffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:41:52.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.085+0000 7f73edffb640 1 -- 172.21.15.78:0/2020827059 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0035040 con 0x558df2b40630 2024-08-18T23:41:52.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.085+0000 7f73edffb640 1 -- 172.21.15.78:0/2020827059 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f73f00037f0 con 0x558df2b40630 2024-08-18T23:41:52.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.085+0000 7f740f3f8740 1 -- 172.21.15.78:0/2020827059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73f0004230 con 0x558df2b40630 2024-08-18T23:41:52.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.085+0000 7f73edffb640 1 -- 172.21.15.78:0/2020827059 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f73f0058050 con 0x558df2b40630 2024-08-18T23:41:52.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.085+0000 7f740f3f8740 10 client.14712 fetch_fsmap learned FSMap version 16 2024-08-18T23:41:52.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.085+0000 7f740f3f8740 1 -- 172.21.15.78:0/2020827059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x558df2b5a0c0 con 0x558df2b40630 2024-08-18T23:41:52.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.085+0000 7f73edffb640 1 -- 172.21.15.78:0/2020827059 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f0004230 con 0x558df2b40630 2024-08-18T23:41:52.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.085+0000 7f73edffb640 1 -- 172.21.15.78:0/2020827059 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7f73f003a040 con 0x558df2b40630 2024-08-18T23:41:52.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.085+0000 7f740f3f8740 10 client.14712 fetch_fsmap finished waiting for FSMap version 16 2024-08-18T23:41:52.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.085+0000 7f740f3f8740 10 client.14712 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-18T23:41:52.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.085+0000 7f740f3f8740 20 client.14712 populate_metadata read hostname 'smithi078' 2024-08-18T23:41:52.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.085+0000 7f740f3f8740 1 --2- 172.21.15.78:0/2020827059 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558df2b5b410 0x558df2b5d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:52.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.085+0000 7f740f3f8740 4 client.14712 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:41:52.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.085+0000 7f740f3f8740 1 -- 172.21.15.78:0/2020827059 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558df2ab2c80 con 0x558df2b5b410 2024-08-18T23:41:52.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.086+0000 7f740659c640 1 --2- 172.21.15.78:0/2020827059 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558df2b5b410 0x558df2b5d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:52.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.086+0000 7f73edffb640 10 client.14712 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:41:52.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.086+0000 7f740659c640 1 --2- 172.21.15.78:0/2020827059 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558df2b5b410 0x558df2b5d5e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94f4ac4542e3c0ed server_cookie=8c4267a3063cb988 in_seq=0 out_seq=0 2024-08-18T23:41:52.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.087+0000 7f73edffb640 1 -- 172.21.15.78:0/2020827059 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558df2ab2c80 con 0x558df2b5b410 2024-08-18T23:41:52.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.087+0000 7f73edffb640 10 client.14712 handle_command_reply: tid=0 2024-08-18T23:41:52.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.087+0000 7f740f3f8740 1 client.14712 shutdown 2024-08-18T23:41:52.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.087+0000 7f740f3f8740 1 -- 172.21.15.78:0/2020827059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b40630 msgr2=0x558df2b487c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:52.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.087+0000 7f740f3f8740 1 --2- 172.21.15.78:0/2020827059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b40630 0x558df2b487c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f73f0005ec0 tx=0x7f73f0009560 comp rx=0 tx=0).stop 2024-08-18T23:41:52.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.087+0000 7f740f3f8740 1 -- 172.21.15.78:0/2020827059 shutdown_connections 2024-08-18T23:41:52.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.087+0000 7f740f3f8740 1 --2- 172.21.15.78:0/2020827059 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558df2b5b410 0x558df2b5d5e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.087+0000 7f740f3f8740 1 --2- 172.21.15.78:0/2020827059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558df2b4b810 0x558df2b49240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.087+0000 7f740f3f8740 1 --2- 172.21.15.78:0/2020827059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 0x558df2b48d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.087+0000 7f740f3f8740 1 --2- 172.21.15.78:0/2020827059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b40630 0x558df2b487c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.087+0000 7f73edffb640 0 client.14712 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:41:52.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.087+0000 7f740f3f8740 1 -- 172.21.15.78:0/2020827059 >> 172.21.15.78:0/2020827059 conn(0x558df2b3cbc0 msgr2=0x558df2b428a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:52.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.087+0000 7f740f3f8740 1 -- 172.21.15.78:0/2020827059 shutdown_connections 2024-08-18T23:41:52.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.087+0000 7f740f3f8740 1 -- 172.21.15.78:0/2020827059 wait complete. 2024-08-18T23:41:52.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.087+0000 7f740f3f8740 20 client.14712 trim_cache size 0 max 16384 2024-08-18T23:41:52.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.104+0000 7f740f3f8740 1 Processor -- start 2024-08-18T23:41:52.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.104+0000 7f740f3f8740 1 -- start start 2024-08-18T23:41:52.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f740f3f8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558df2b40630 0x558df2b43090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:52.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f740f3f8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 0x558df2b49090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:52.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f740f3f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b4b810 0x558df2b3bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:52.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f740f3f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558df2b5b920 con 0x558df2b4b810 2024-08-18T23:41:52.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f740f3f8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558df2b5bff0 con 0x558df2b40630 2024-08-18T23:41:52.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f740f3f8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558df2b5c6c0 con 0x558df2b40f50 2024-08-18T23:41:52.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f73fffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b4b810 0x558df2b3bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:52.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f73fffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b4b810 0x558df2b3bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44200/0 (socket says 172.21.15.78:44200) 2024-08-18T23:41:52.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f73fffff640 1 -- 172.21.15.78:0/2541581956 learned_addr learned my addr 172.21.15.78:0/2541581956 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:52.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f73fffff640 1 --2- 172.21.15.78:0/2541581956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558df2b40630 0x558df2b43090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:52.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f740659c640 1 --2- 172.21.15.78:0/2541581956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 0x558df2b49090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:52.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f73fffff640 1 -- 172.21.15.78:0/2541581956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 msgr2=0x558df2b49090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:41:52.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f73fffff640 1 --2- 172.21.15.78:0/2541581956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 0x558df2b49090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f73fffff640 1 -- 172.21.15.78:0/2541581956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558df2b40630 msgr2=0x558df2b43090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:41:52.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f73fffff640 1 --2- 172.21.15.78:0/2541581956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558df2b40630 0x558df2b43090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f73fffff640 1 -- 172.21.15.78:0/2541581956 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df2b47770 con 0x558df2b4b810 2024-08-18T23:41:52.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f740659c640 1 --2- 172.21.15.78:0/2541581956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 0x558df2b49090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:41:52.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f73fffff640 1 --2- 172.21.15.78:0/2541581956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b4b810 0x558df2b3bc30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f73f003d4b0 tx=0x7f73f00056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c4b25bf2240e5b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:52.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f73effff640 1 -- 172.21.15.78:0/2541581956 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0041070 con 0x558df2b4b810 2024-08-18T23:41:52.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f73effff640 1 -- 172.21.15.78:0/2541581956 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f004f070 con 0x558df2b4b810 2024-08-18T23:41:52.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.105+0000 7f73effff640 1 -- 172.21.15.78:0/2541581956 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f003dac0 con 0x558df2b4b810 2024-08-18T23:41:52.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.106+0000 7f740f3f8740 1 -- 172.21.15.78:0/2541581956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b4b810 msgr2=0x558df2b3bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:52.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.106+0000 7f740f3f8740 1 --2- 172.21.15.78:0/2541581956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b4b810 0x558df2b3bc30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f73f003d4b0 tx=0x7f73f00056d0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.106+0000 7f740f3f8740 1 -- 172.21.15.78:0/2541581956 shutdown_connections 2024-08-18T23:41:52.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.106+0000 7f740f3f8740 1 --2- 172.21.15.78:0/2541581956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b4b810 0x558df2b3bc30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.106+0000 7f740f3f8740 1 --2- 172.21.15.78:0/2541581956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 0x558df2b49090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.106+0000 7f740f3f8740 1 --2- 172.21.15.78:0/2541581956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558df2b40630 0x558df2b43090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.106+0000 7f740f3f8740 1 -- 172.21.15.78:0/2541581956 >> 172.21.15.78:0/2541581956 conn(0x558df2b3cbc0 msgr2=0x558df2b3b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:52.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.106+0000 7f740f3f8740 1 -- 172.21.15.78:0/2541581956 shutdown_connections 2024-08-18T23:41:52.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.106+0000 7f740f3f8740 1 -- 172.21.15.78:0/2541581956 wait complete. 2024-08-18T23:41:52.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.106+0000 7f740f3f8740 1 -- start start 2024-08-18T23:41:52.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.106+0000 7f740f3f8740 1 Processor -- start 2024-08-18T23:41:52.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.107+0000 7f740f3f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b40630 0x558df2b65e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:52.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.107+0000 7f740f3f8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 0x558df2b5f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:52.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.107+0000 7f740f3f8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558df2b4b810 0x558df2b5f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:52.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.107+0000 7f740f3f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558df2b5d460 con 0x558df2b40630 2024-08-18T23:41:52.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.107+0000 7f740f3f8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558df2b5cd90 con 0x558df2b4b810 2024-08-18T23:41:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.107+0000 7f740f3f8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558df2b5db30 con 0x558df2b40f50 2024-08-18T23:41:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.107+0000 7f740f3f8740 10 client.0 _ll_register_callbacks cb 0x7f73f00315f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:41:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.107+0000 7f73fffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b40630 0x558df2b65e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.107+0000 7f73fffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b40630 0x558df2b65e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44206/0 (socket says 172.21.15.78:44206) 2024-08-18T23:41:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.107+0000 7f73fffff640 1 -- 172.21.15.78:0/3133662922 learned_addr learned my addr 172.21.15.78:0/3133662922 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.107+0000 7f740659c640 1 --2- 172.21.15.78:0/3133662922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 0x558df2b5f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.107+0000 7f73fffff640 1 -- 172.21.15.78:0/3133662922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 msgr2=0x558df2b5f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:41:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.107+0000 7f73fffff640 1 --2- 172.21.15.78:0/3133662922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 0x558df2b5f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.107+0000 7f73fffff640 1 -- 172.21.15.78:0/3133662922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558df2b4b810 msgr2=0x558df2b5f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:41:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.107+0000 7f73fffff640 1 --2- 172.21.15.78:0/3133662922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558df2b4b810 0x558df2b5f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.107+0000 7f73fffff640 1 -- 172.21.15.78:0/3133662922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df2b5fd40 con 0x558df2b40630 2024-08-18T23:41:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.107+0000 7f73fffff640 1 --2- 172.21.15.78:0/3133662922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b40630 0x558df2b65e40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f73f005f040 tx=0x7f73f003d770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa66f651f1de34d server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.107+0000 7f73edffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:41:52.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.108+0000 7f73edffb640 1 -- 172.21.15.78:0/3133662922 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f006a060 con 0x558df2b40630 2024-08-18T23:41:52.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.108+0000 7f73edffb640 1 -- 172.21.15.78:0/3133662922 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f73f0003a00 con 0x558df2b40630 2024-08-18T23:41:52.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.108+0000 7f740f3f8740 1 -- 172.21.15.78:0/3133662922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73f004f070 con 0x558df2b40630 2024-08-18T23:41:52.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.108+0000 7f73edffb640 1 -- 172.21.15.78:0/3133662922 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f73f0035030 con 0x558df2b40630 2024-08-18T23:41:52.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.108+0000 7f740f3f8740 10 client.14718 fetch_fsmap learned FSMap version 16 2024-08-18T23:41:52.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.108+0000 7f740f3f8740 1 -- 172.21.15.78:0/3133662922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x558df2b64720 con 0x558df2b40630 2024-08-18T23:41:52.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.108+0000 7f73edffb640 1 -- 172.21.15.78:0/3133662922 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f004f070 con 0x558df2b40630 2024-08-18T23:41:52.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.108+0000 7f73edffb640 1 -- 172.21.15.78:0/3133662922 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7f73f004a5b0 con 0x558df2b40630 2024-08-18T23:41:52.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.108+0000 7f740f3f8740 10 client.14718 fetch_fsmap finished waiting for FSMap version 16 2024-08-18T23:41:52.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.108+0000 7f740f3f8740 10 client.14718 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-18T23:41:52.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.108+0000 7f740f3f8740 20 client.14718 populate_metadata read hostname 'smithi078' 2024-08-18T23:41:52.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.108+0000 7f740f3f8740 1 --2- 172.21.15.78:0/3133662922 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558df2bcac60 0x558df2bc9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:52.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.108+0000 7f740f3f8740 4 client.14718 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-18T23:41:52.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.108+0000 7f740f3f8740 1 -- 172.21.15.78:0/3133662922 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "status"}) -- 0x558df2aad190 con 0x558df2bcac60 2024-08-18T23:41:52.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.108+0000 7f740659c640 1 --2- 172.21.15.78:0/3133662922 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558df2bcac60 0x558df2bc9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:52.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.109+0000 7f73edffb640 10 client.14718 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:41:52.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.109+0000 7f740659c640 1 --2- 172.21.15.78:0/3133662922 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558df2bcac60 0x558df2bc9970 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d3d6e8b83540887 server_cookie=8d68e99a8fdff2c1 in_seq=0 out_seq=0 2024-08-18T23:41:52.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.109+0000 7f73edffb640 1 -- 172.21.15.78:0/3133662922 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558df2aad190 con 0x558df2bcac60 2024-08-18T23:41:52.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.109+0000 7f73edffb640 10 client.14718 handle_command_reply: tid=0 2024-08-18T23:41:52.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.109+0000 7f740f3f8740 1 client.14718 shutdown 2024-08-18T23:41:52.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.109+0000 7f740f3f8740 1 -- 172.21.15.78:0/3133662922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b40630 msgr2=0x558df2b65e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:52.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.109+0000 7f740f3f8740 1 --2- 172.21.15.78:0/3133662922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b40630 0x558df2b65e40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f73f005f040 tx=0x7f73f003d770 comp rx=0 tx=0).stop 2024-08-18T23:41:52.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.109+0000 7f740f3f8740 1 -- 172.21.15.78:0/3133662922 shutdown_connections 2024-08-18T23:41:52.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.109+0000 7f740f3f8740 1 --2- 172.21.15.78:0/3133662922 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558df2bcac60 0x558df2bc9970 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.109+0000 7f740f3f8740 1 --2- 172.21.15.78:0/3133662922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558df2b4b810 0x558df2b5f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.109+0000 7f740f3f8740 1 --2- 172.21.15.78:0/3133662922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558df2b40f50 0x558df2b5f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.109+0000 7f73edffb640 0 client.14718 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:41:52.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.109+0000 7f740f3f8740 1 --2- 172.21.15.78:0/3133662922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558df2b40630 0x558df2b65e40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.109+0000 7f740f3f8740 1 -- 172.21.15.78:0/3133662922 >> 172.21.15.78:0/3133662922 conn(0x558df2b3cbc0 msgr2=0x558df2b3bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:52.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.109+0000 7f740f3f8740 1 -- 172.21.15.78:0/3133662922 shutdown_connections 2024-08-18T23:41:52.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.109+0000 7f740f3f8740 1 -- 172.21.15.78:0/3133662922 wait complete. 2024-08-18T23:41:52.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.109+0000 7f740f3f8740 20 client.14718 trim_cache size 0 max 16384 2024-08-18T23:41:52.126 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:41:52.126 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "4f1128e6-5dba-11ef-bccf-c7b262605968", 2024-08-18T23:41:52.126 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-08-18T23:41:52.126 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24452, 2024-08-18T23:41:52.126 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-08-18T23:41:52.126 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-08-18T23:41:52.126 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-08-18T23:41:52.126 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 18.969342701999999, 2024-08-18T23:41:52.126 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 16, 2024-08-18T23:41:52.127 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 88, 2024-08-18T23:41:52.127 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 88, 2024-08-18T23:41:52.127 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 29.632323839000001 2024-08-18T23:41:52.127 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:41:52.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.110+0000 7f740cffa640 1 -- 172.21.15.78:0/1205219090 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f73f40a2b80 msgr2=0x7f73f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:52.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.110+0000 7f740cffa640 1 --2- 172.21.15.78:0/1205219090 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f73f40a2b80 0x7f73f40a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f740009ac40 tx=0x7f7400090420 comp rx=0 tx=0).stop 2024-08-18T23:41:52.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.110+0000 7f740cffa640 1 -- 172.21.15.78:0/1205219090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74081078b0 msgr2=0x7f740819cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:52.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.110+0000 7f740cffa640 1 --2- 172.21.15.78:0/1205219090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74081078b0 0x7f740819cb10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f740000b970 tx=0x7f740000be30 comp rx=0 tx=0).stop 2024-08-18T23:41:52.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.110+0000 7f740cffa640 1 -- 172.21.15.78:0/1205219090 shutdown_connections 2024-08-18T23:41:52.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.110+0000 7f740cffa640 1 --2- 172.21.15.78:0/1205219090 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f73f40a2b80 0x7f73f40a5040 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.110+0000 7f740cffa640 1 --2- 172.21.15.78:0/1205219090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74081078b0 0x7f740819cb10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.110+0000 7f740cffa640 1 --2- 172.21.15.78:0/1205219090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7408102250 0x7f7408198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.110+0000 7f740cffa640 1 --2- 172.21.15.78:0/1205219090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74081018b0 0x7f7408198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:52.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.110+0000 7f740cffa640 1 -- 172.21.15.78:0/1205219090 >> 172.21.15.78:0/1205219090 conn(0x7f74080fd5f0 msgr2=0x7f74080fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:52.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.110+0000 7f740cffa640 1 -- 172.21.15.78:0/1205219090 shutdown_connections 2024-08-18T23:41:52.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:52.110+0000 7f740cffa640 1 -- 172.21.15.78:0/1205219090 wait complete. 2024-08-18T23:41:52.907 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-08-18T23:41:52.907 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-08-18T23:41:52.908 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs ls 2024-08-18T23:41:53.156 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:53.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:53 smithi078 ceph-mon[18440]: pgmap v273: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:53.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:53 smithi123 ceph-mon[21808]: pgmap v273: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:53.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.669+0000 7f2230b21640 1 -- 172.21.15.78:0/1629975973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f222c1067b0 msgr2=0x7f222c106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:53.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.669+0000 7f2230b21640 1 --2- 172.21.15.78:0/1629975973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f222c1067b0 0x7f222c106b90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f221c009930 tx=0x7f221c02a420 comp rx=0 tx=0).stop 2024-08-18T23:41:53.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.670+0000 7f2230b21640 1 -- 172.21.15.78:0/1629975973 shutdown_connections 2024-08-18T23:41:53.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.670+0000 7f2230b21640 1 --2- 172.21.15.78:0/1629975973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f222c101150 0x7f222c10d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:53.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.670+0000 7f2230b21640 1 --2- 172.21.15.78:0/1629975973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f222c1007b0 0x7f222c100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:53.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.670+0000 7f2230b21640 1 --2- 172.21.15.78:0/1629975973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f222c1067b0 0x7f222c106b90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:53.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.670+0000 7f2230b21640 1 -- 172.21.15.78:0/1629975973 >> 172.21.15.78:0/1629975973 conn(0x7f222c0fc4d0 msgr2=0x7f222c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:53.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.670+0000 7f2230b21640 1 -- 172.21.15.78:0/1629975973 shutdown_connections 2024-08-18T23:41:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.670+0000 7f2230b21640 1 -- 172.21.15.78:0/1629975973 wait complete. 2024-08-18T23:41:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.670+0000 7f2230b21640 1 Processor -- start 2024-08-18T23:41:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.670+0000 7f2230b21640 1 -- start start 2024-08-18T23:41:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f2230b21640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f222c1007b0 0x7f222c197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f2230b21640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f222c101150 0x7f222c197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f2230b21640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f222c1067b0 0x7f222c19bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f2230b21640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f222c110c50 con 0x7f222c1067b0 2024-08-18T23:41:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f2230b21640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f222c110ad0 con 0x7f222c1007b0 2024-08-18T23:41:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f2230b21640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f222c110dd0 con 0x7f222c101150 2024-08-18T23:41:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f222ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f222c1067b0 0x7f222c19bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:53.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f222ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f222c1067b0 0x7f222c19bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44230/0 (socket says 172.21.15.78:44230) 2024-08-18T23:41:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f222ad9d640 1 -- 172.21.15.78:0/3985193118 learned_addr learned my addr 172.21.15.78:0/3985193118 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f222a59c640 1 --2- 172.21.15.78:0/3985193118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f222c1007b0 0x7f222c197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f2229d9b640 1 --2- 172.21.15.78:0/3985193118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f222c101150 0x7f222c197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f222ad9d640 1 -- 172.21.15.78:0/3985193118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f222c101150 msgr2=0x7f222c197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f222ad9d640 1 --2- 172.21.15.78:0/3985193118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f222c101150 0x7f222c197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f222ad9d640 1 -- 172.21.15.78:0/3985193118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f222c1007b0 msgr2=0x7f222c197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f222ad9d640 1 --2- 172.21.15.78:0/3985193118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f222c1007b0 0x7f222c197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f222ad9d640 1 -- 172.21.15.78:0/3985193118 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222c19c1c0 con 0x7f222c1067b0 2024-08-18T23:41:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f222a59c640 1 --2- 172.21.15.78:0/3985193118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f222c1007b0 0x7f222c197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:41:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f2229d9b640 1 --2- 172.21.15.78:0/3985193118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f222c101150 0x7f222c197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:41:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f222ad9d640 1 --2- 172.21.15.78:0/3985193118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f222c1067b0 0x7f222c19bae0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f222400b970 tx=0x7f222400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f22237fe640 1 -- 172.21.15.78:0/3985193118 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2224019070 con 0x7f222c1067b0 2024-08-18T23:41:53.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f22237fe640 1 -- 172.21.15.78:0/3985193118 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f222400cb40 con 0x7f222c1067b0 2024-08-18T23:41:53.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f2230b21640 1 -- 172.21.15.78:0/3985193118 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f222c19c4b0 con 0x7f222c1067b0 2024-08-18T23:41:53.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.671+0000 7f22237fe640 1 -- 172.21.15.78:0/3985193118 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2224007650 con 0x7f222c1067b0 2024-08-18T23:41:53.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.672+0000 7f2230b21640 1 -- 172.21.15.78:0/3985193118 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f222c1a3d90 con 0x7f222c1067b0 2024-08-18T23:41:53.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.673+0000 7f2230b21640 1 -- 172.21.15.78:0/3985193118 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f222c100c90 con 0x7f222c1067b0 2024-08-18T23:41:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:53 smithi161 ceph-mon[22027]: pgmap v273: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:53.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.674+0000 7f22237fe640 1 -- 172.21.15.78:0/3985193118 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f222401c070 con 0x7f222c1067b0 2024-08-18T23:41:53.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.675+0000 7f22237fe640 1 --2- 172.21.15.78:0/3985193118 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f22180a2c00 0x7f22180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:53.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.675+0000 7f222a59c640 1 --2- 172.21.15.78:0/3985193118 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f22180a2c00 0x7f22180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:53.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.675+0000 7f22237fe640 1 -- 172.21.15.78:0/3985193118 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f22240ce8b0 con 0x7f222c1067b0 2024-08-18T23:41:53.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.675+0000 7f222a59c640 1 --2- 172.21.15.78:0/3985193118 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f22180a2c00 0x7f22180a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f221c000c00 tx=0x7f221c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:53.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.679+0000 7f22237fe640 1 -- 172.21.15.78:0/3985193118 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f222409b5d0 con 0x7f222c1067b0 2024-08-18T23:41:53.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.881+0000 7f2230b21640 1 -- 172.21.15.78:0/3985193118 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f222c02d5e0 con 0x7f222c1067b0 2024-08-18T23:41:53.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.881+0000 7f22237fe640 1 -- 172.21.15.78:0/3985193118 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7f222409aed0 con 0x7f222c1067b0 2024-08-18T23:41:53.883 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-08-18T23:41:53.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.883+0000 7f2230b21640 1 -- 172.21.15.78:0/3985193118 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f22180a2c00 msgr2=0x7f22180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:53.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.883+0000 7f2230b21640 1 --2- 172.21.15.78:0/3985193118 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f22180a2c00 0x7f22180a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f221c000c00 tx=0x7f221c005a20 comp rx=0 tx=0).stop 2024-08-18T23:41:53.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.883+0000 7f2230b21640 1 -- 172.21.15.78:0/3985193118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f222c1067b0 msgr2=0x7f222c19bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:53.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.883+0000 7f2230b21640 1 --2- 172.21.15.78:0/3985193118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f222c1067b0 0x7f222c19bae0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f222400b970 tx=0x7f222400be30 comp rx=0 tx=0).stop 2024-08-18T23:41:53.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.883+0000 7f2230b21640 1 -- 172.21.15.78:0/3985193118 shutdown_connections 2024-08-18T23:41:53.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.883+0000 7f2230b21640 1 --2- 172.21.15.78:0/3985193118 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f22180a2c00 0x7f22180a50c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:53.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.883+0000 7f2230b21640 1 --2- 172.21.15.78:0/3985193118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f222c1067b0 0x7f222c19bae0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:53.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.883+0000 7f2230b21640 1 --2- 172.21.15.78:0/3985193118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f222c101150 0x7f222c197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:53.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.883+0000 7f2230b21640 1 --2- 172.21.15.78:0/3985193118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f222c1007b0 0x7f222c197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:53.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.883+0000 7f2230b21640 1 -- 172.21.15.78:0/3985193118 >> 172.21.15.78:0/3985193118 conn(0x7f222c0fc4d0 msgr2=0x7f222c0fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:53.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.883+0000 7f2230b21640 1 -- 172.21.15.78:0/3985193118 shutdown_connections 2024-08-18T23:41:53.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:53.883+0000 7f2230b21640 1 -- 172.21.15.78:0/3985193118 wait complete. 2024-08-18T23:41:54.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:54 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3985193118' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-18T23:41:54.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:54 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3985193118' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-18T23:41:54.590 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-08-18T23:41:54.591 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-18T23:41:54.591 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-08-18T23:41:54.591 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-08-18T23:41:54.591 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-08-18T23:41:54.591 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-08-18T23:41:54.591 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-18T23:41:54.591 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-18T23:41:54.591 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-08-18T23:41:54.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:41:54.591 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-08-18T23:41:54.607 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-18T23:41:54.608 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-18T23:41:54.608 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-08-18T23:41:54.608 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-08-18T23:41:54.608 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-08-18T23:41:54.608 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-08-18T23:41:54.608 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-18T23:41:54.608 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-08-18T23:41:54.608 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-08-18T23:41:54.608 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-18T23:41:54.608 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-08-18T23:41:54.608 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-08-18T23:41:54.609 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-18T23:41:54.609 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-08-18T23:41:54.609 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-18T23:41:54.609 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-08-18T23:41:54.609 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-08-18T23:41:54.609 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42424sec preferred_lft 42424sec 2024-08-18T23:41:54.609 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link noprefixroute 2024-08-18T23:41:54.609 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-08-18T23:41:54.610 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-18T23:41:54.610 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-18T23:41:54.610 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-08-18T23:41:54.610 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add name ceph-brx type bridge 2024-08-18T23:41:54.610 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr flush dev ceph-brx 2024-08-18T23:41:54.610 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set ceph-brx up 2024-08-18T23:41:54.610 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-18T23:41:54.610 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-08-18T23:41:54.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:54 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3985193118' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-18T23:41:54.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-18T23:41:54.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-18T23:41:54.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:41:54.809 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-18T23:41:54.834 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-08-18T23:41:54.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:41:54.835 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-08-18T23:41:54.902 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-08-18T23:41:54.902 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-18T23:41:54.902 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-18T23:41:54.902 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-08-18T23:41:54.902 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-18T23:41:54.903 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-18T23:41:54.903 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-08-18T23:41:54.903 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-18T23:41:54.903 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-18T23:41:54.903 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-18T23:41:54.903 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-08-18T23:41:54.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-18T23:41:55.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-18T23:41:55.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:41:55.170 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-08-18T23:41:55.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:41:55.187 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list-id 2024-08-18T23:41:55.243 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-18T23:41:55.243 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-08-18T23:41:55.243 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-18T23:41:55.243 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-08-18T23:41:55.243 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-08-18T23:41:55.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-18T23:41:55.342 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:55 smithi078 ceph-mon[18440]: pgmap v274: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:55.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-18T23:41:55.363 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-08-18T23:41:55.364 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-18T23:41:55.364 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-08-18T23:41:55.364 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-08-18T23:41:55.364 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-18T23:41:55.364 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-08-18T23:41:55.364 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-08-18T23:41:55.364 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-08-18T23:41:55.364 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-08-18T23:41:55.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-18T23:41:55.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-18T23:41:55.487 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-18T23:41:55.487 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-08-18T23:41:55.487 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-08-18T23:41:55.487 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set dev brx.0 master ceph-brx 2024-08-18T23:41:55.487 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-08-18T23:41:55.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-18T23:41:55.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:55 smithi123 ceph-mon[21808]: pgmap v274: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:55.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-18T23:41:55.563 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-08-18T23:41:55.563 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-18T23:41:55.563 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-08-18T23:41:55.578 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-08-18T23:41:55.579 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-18T23:41:55.579 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-08-18T23:41:55.634 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2024-08-18T23:41:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:55 smithi161 ceph-mon[22027]: pgmap v274: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:55.699 DEBUG:teuthology.orchestra.run.smithi078:> cat /proc/self/mounts | awk '{print $2}' 2024-08-18T23:41:55.755 INFO:teuthology.orchestra.run.smithi078.stdout:/proc 2024-08-18T23:41:55.755 INFO:teuthology.orchestra.run.smithi078.stdout:/sys 2024-08-18T23:41:55.755 INFO:teuthology.orchestra.run.smithi078.stdout:/dev 2024-08-18T23:41:55.755 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/security 2024-08-18T23:41:55.755 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/shm 2024-08-18T23:41:55.755 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/pts 2024-08-18T23:41:55.756 INFO:teuthology.orchestra.run.smithi078.stdout:/run 2024-08-18T23:41:55.756 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/cgroup 2024-08-18T23:41:55.756 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/pstore 2024-08-18T23:41:55.756 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/bpf 2024-08-18T23:41:55.756 INFO:teuthology.orchestra.run.smithi078.stdout:/ 2024-08-18T23:41:55.756 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/selinux 2024-08-18T23:41:55.756 INFO:teuthology.orchestra.run.smithi078.stdout:/proc/sys/fs/binfmt_misc 2024-08-18T23:41:55.756 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/mqueue 2024-08-18T23:41:55.756 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/hugepages 2024-08-18T23:41:55.756 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/debug 2024-08-18T23:41:55.756 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/tracing 2024-08-18T23:41:55.756 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-sysusers.service 2024-08-18T23:41:55.756 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/config 2024-08-18T23:41:55.757 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/fuse/connections 2024-08-18T23:41:55.757 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-sysctl.service 2024-08-18T23:41:55.757 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-08-18T23:41:55.757 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-08-18T23:41:55.757 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/nfs/rpc_pipefs 2024-08-18T23:41:55.757 INFO:teuthology.orchestra.run.smithi078.stdout:/run/user/1000 2024-08-18T23:41:55.757 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/ceph 2024-08-18T23:41:55.757 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay 2024-08-18T23:41:55.757 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/070ae3505dd78d3407db8e5a84a82d413120aa9a9c34ed5041061a7960705f7c/merged 2024-08-18T23:41:55.757 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/1ba720aba2e9d35647456a56597767593b9681701af14c6123daf889d31b0725/merged 2024-08-18T23:41:55.757 INFO:teuthology.orchestra.run.smithi078.stdout:/run/user/0 2024-08-18T23:41:55.757 INFO:teuthology.orchestra.run.smithi078.stdout:/proc/sys/fs/binfmt_misc 2024-08-18T23:41:55.757 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/10cd3444c69a39965d6be5d573af31fc499e0bdf2cd08c2379ef7500e946c2a2/merged 2024-08-18T23:41:55.758 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/c946d44b4cc84a058cd622462f48cf869ab921c6baf275e97bc9a1719cab920d/merged 2024-08-18T23:41:55.758 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/e3a0c3c1fbe7d8013fd3f2fff9e26ec53a2488bce769f00bf29222e8d65b3434/merged 2024-08-18T23:41:55.758 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/cd76571cf9a7996ec05b22c2c7238a1d2d646fbbb913371e48e3a3879017719c/merged 2024-08-18T23:41:55.758 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/5eadbbd6bc08b6321df00cc8b020ec25050d3d0163247bd5defc1a757e96694e/merged 2024-08-18T23:41:55.758 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/881e018cc655b63a7ef5a9b6ab9115b7ca0c594e5ee9098177315e0040f1ebfd/merged 2024-08-18T23:41:55.758 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/782db86b571d9c485b9610e37e7ddadcf91dc6b3289960e78d1c242d4301c98d/merged 2024-08-18T23:41:55.758 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/3dd371d1ddae73c5e1fc5cc8108f7cc29f5919281027302eedf71eb868f17104/merged 2024-08-18T23:41:55.758 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns 2024-08-18T23:41:55.758 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-18T23:41:55.758 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-18T23:41:55.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:41:55.759 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2024-08-18T23:41:55.811 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-18T23:41:55.811 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-08-18T23:41:55.853 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2024-08-18T23:41:55.878 DEBUG:teuthology.orchestra.run.smithi078:> cat /proc/self/mounts | awk '{print $2}' 2024-08-18T23:41:55.934 INFO:teuthology.orchestra.run.smithi078.stdout:/proc 2024-08-18T23:41:55.935 INFO:teuthology.orchestra.run.smithi078.stdout:/sys 2024-08-18T23:41:55.935 INFO:teuthology.orchestra.run.smithi078.stdout:/dev 2024-08-18T23:41:55.935 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/security 2024-08-18T23:41:55.935 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/shm 2024-08-18T23:41:55.935 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/pts 2024-08-18T23:41:55.935 INFO:teuthology.orchestra.run.smithi078.stdout:/run 2024-08-18T23:41:55.935 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/cgroup 2024-08-18T23:41:55.935 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/pstore 2024-08-18T23:41:55.935 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/bpf 2024-08-18T23:41:55.935 INFO:teuthology.orchestra.run.smithi078.stdout:/ 2024-08-18T23:41:55.935 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/selinux 2024-08-18T23:41:55.935 INFO:teuthology.orchestra.run.smithi078.stdout:/proc/sys/fs/binfmt_misc 2024-08-18T23:41:55.936 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/mqueue 2024-08-18T23:41:55.936 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/hugepages 2024-08-18T23:41:55.936 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/debug 2024-08-18T23:41:55.936 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/tracing 2024-08-18T23:41:55.936 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-sysusers.service 2024-08-18T23:41:55.936 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/config 2024-08-18T23:41:55.936 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/fuse/connections 2024-08-18T23:41:55.936 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-sysctl.service 2024-08-18T23:41:55.936 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-08-18T23:41:55.936 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-08-18T23:41:55.936 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/nfs/rpc_pipefs 2024-08-18T23:41:55.936 INFO:teuthology.orchestra.run.smithi078.stdout:/run/user/1000 2024-08-18T23:41:55.936 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/ceph 2024-08-18T23:41:55.937 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay 2024-08-18T23:41:55.937 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/070ae3505dd78d3407db8e5a84a82d413120aa9a9c34ed5041061a7960705f7c/merged 2024-08-18T23:41:55.937 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/1ba720aba2e9d35647456a56597767593b9681701af14c6123daf889d31b0725/merged 2024-08-18T23:41:55.937 INFO:teuthology.orchestra.run.smithi078.stdout:/run/user/0 2024-08-18T23:41:55.937 INFO:teuthology.orchestra.run.smithi078.stdout:/proc/sys/fs/binfmt_misc 2024-08-18T23:41:55.937 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/10cd3444c69a39965d6be5d573af31fc499e0bdf2cd08c2379ef7500e946c2a2/merged 2024-08-18T23:41:55.937 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/c946d44b4cc84a058cd622462f48cf869ab921c6baf275e97bc9a1719cab920d/merged 2024-08-18T23:41:55.937 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/e3a0c3c1fbe7d8013fd3f2fff9e26ec53a2488bce769f00bf29222e8d65b3434/merged 2024-08-18T23:41:55.937 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/cd76571cf9a7996ec05b22c2c7238a1d2d646fbbb913371e48e3a3879017719c/merged 2024-08-18T23:41:55.937 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/5eadbbd6bc08b6321df00cc8b020ec25050d3d0163247bd5defc1a757e96694e/merged 2024-08-18T23:41:55.937 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/881e018cc655b63a7ef5a9b6ab9115b7ca0c594e5ee9098177315e0040f1ebfd/merged 2024-08-18T23:41:55.937 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/782db86b571d9c485b9610e37e7ddadcf91dc6b3289960e78d1c242d4301c98d/merged 2024-08-18T23:41:55.937 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/3dd371d1ddae73c5e1fc5cc8108f7cc29f5919281027302eedf71eb868f17104/merged 2024-08-18T23:41:55.938 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns 2024-08-18T23:41:55.938 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-18T23:41:55.938 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-18T23:41:55.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:41:55.938 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2024-08-18T23:41:55.951 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi078.stderr:2024-08-18T23:41:55.949+0000 7f8fcc11c480 -1 init, newargv = 0x55fea98d39c0 newargc=15 2024-08-18T23:41:55.951 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi078.stderr:ceph-fuse[62362]: starting ceph client 2024-08-18T23:41:55.958 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi078.stderr:ceph-fuse[62362]: starting fuse 2024-08-18T23:41:55.967 INFO:teuthology.orchestra.run.smithi078.stdout:121 2024-08-18T23:41:55.967 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2024-08-18T23:41:55.967 DEBUG:teuthology.orchestra.run.smithi078:> sudo stdin-killer -- python3 -c ' 2024-08-18T23:41:55.967 DEBUG:teuthology.orchestra.run.smithi078:> import glob 2024-08-18T23:41:55.967 DEBUG:teuthology.orchestra.run.smithi078:> import re 2024-08-18T23:41:55.967 DEBUG:teuthology.orchestra.run.smithi078:> import os 2024-08-18T23:41:55.967 DEBUG:teuthology.orchestra.run.smithi078:> import subprocess 2024-08-18T23:41:55.967 DEBUG:teuthology.orchestra.run.smithi078:> 2024-08-18T23:41:55.967 DEBUG:teuthology.orchestra.run.smithi078:> def _find_admin_socket(client_name): 2024-08-18T23:41:55.968 DEBUG:teuthology.orchestra.run.smithi078:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-08-18T23:41:55.968 DEBUG:teuthology.orchestra.run.smithi078:> files = glob.glob(asok_path) 2024-08-18T23:41:55.968 DEBUG:teuthology.orchestra.run.smithi078:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-08-18T23:41:55.968 DEBUG:teuthology.orchestra.run.smithi078:> 2024-08-18T23:41:55.968 DEBUG:teuthology.orchestra.run.smithi078:> # Given a non-glob path, it better be there 2024-08-18T23:41:55.968 DEBUG:teuthology.orchestra.run.smithi078:> if "*" not in asok_path: 2024-08-18T23:41:55.968 DEBUG:teuthology.orchestra.run.smithi078:> assert(len(files) == 1) 2024-08-18T23:41:55.968 DEBUG:teuthology.orchestra.run.smithi078:> return files[0] 2024-08-18T23:41:55.968 DEBUG:teuthology.orchestra.run.smithi078:> 2024-08-18T23:41:55.968 DEBUG:teuthology.orchestra.run.smithi078:> for f in files: 2024-08-18T23:41:55.968 DEBUG:teuthology.orchestra.run.smithi078:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-18T23:41:55.968 DEBUG:teuthology.orchestra.run.smithi078:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-18T23:41:55.968 DEBUG:teuthology.orchestra.run.smithi078:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-18T23:41:55.968 DEBUG:teuthology.orchestra.run.smithi078:> contents = proc_f.read() 2024-08-18T23:41:55.968 DEBUG:teuthology.orchestra.run.smithi078:> if mountpoint in contents: 2024-08-18T23:41:55.969 DEBUG:teuthology.orchestra.run.smithi078:> return f 2024-08-18T23:41:55.969 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-18T23:41:55.969 DEBUG:teuthology.orchestra.run.smithi078:> 2024-08-18T23:41:55.969 DEBUG:teuthology.orchestra.run.smithi078:> print(_find_admin_socket("client.admin")) 2024-08-18T23:41:55.969 DEBUG:teuthology.orchestra.run.smithi078:> ' 2024-08-18T23:41:56.083 INFO:teuthology.orchestra.run.smithi078.stdout:/var/run/ceph/ceph-client.admin.62362.asok 2024-08-18T23:41:56.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-18T23:41:56.093 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.62362.asok 2024-08-18T23:41:56.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:41:56.094 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.62362.asok status 2024-08-18T23:41:56.229 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:41:56.229 INFO:teuthology.orchestra.run.smithi078.stdout: "metadata": { 2024-08-18T23:41:56.229 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_sha1": "de09dd018de4baddfa6ca4d43bc8fb784cf75833", 2024-08-18T23:41:56.229 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 19.1.0-1382-gde09dd01 (de09dd018de4baddfa6ca4d43bc8fb784cf75833) squid (rc)", 2024-08-18T23:41:56.229 INFO:teuthology.orchestra.run.smithi078.stdout: "entity_id": "admin", 2024-08-18T23:41:56.230 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": "smithi078", 2024-08-18T23:41:56.230 INFO:teuthology.orchestra.run.smithi078.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-08-18T23:41:56.230 INFO:teuthology.orchestra.run.smithi078.stdout: "pid": "62362", 2024-08-18T23:41:56.230 INFO:teuthology.orchestra.run.smithi078.stdout: "root": "/" 2024-08-18T23:41:56.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:41:56.230 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_count": 0, 2024-08-18T23:41:56.230 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_pinned_count": 0, 2024-08-18T23:41:56.230 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 14730, 2024-08-18T23:41:56.230 INFO:teuthology.orchestra.run.smithi078.stdout: "inst": { 2024-08-18T23:41:56.230 INFO:teuthology.orchestra.run.smithi078.stdout: "name": { 2024-08-18T23:41:56.230 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "client", 2024-08-18T23:41:56.230 INFO:teuthology.orchestra.run.smithi078.stdout: "num": 14730 2024-08-18T23:41:56.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:41:56.231 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2024-08-18T23:41:56.231 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2024-08-18T23:41:56.231 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2024-08-18T23:41:56.231 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 216526462 2024-08-18T23:41:56.231 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:41:56.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:41:56.231 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2024-08-18T23:41:56.231 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2024-08-18T23:41:56.231 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2024-08-18T23:41:56.231 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 216526462 2024-08-18T23:41:56.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:41:56.231 INFO:teuthology.orchestra.run.smithi078.stdout: "inst_str": "client.14730 192.168.0.1:0/216526462", 2024-08-18T23:41:56.231 INFO:teuthology.orchestra.run.smithi078.stdout: "addr_str": "192.168.0.1:0/216526462", 2024-08-18T23:41:56.231 INFO:teuthology.orchestra.run.smithi078.stdout: "inode_count": 1, 2024-08-18T23:41:56.232 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_epoch": 16, 2024-08-18T23:41:56.232 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch": 88, 2024-08-18T23:41:56.232 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch_barrier": 0, 2024-08-18T23:41:56.232 INFO:teuthology.orchestra.run.smithi078.stdout: "blocklisted": false, 2024-08-18T23:41:56.232 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs" 2024-08-18T23:41:56.232 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:41:56.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:41:56.238 DEBUG:teuthology.orchestra.run.smithi078:> stat --file-system '--printf=%T 2024-08-18T23:41:56.238 DEBUG:teuthology.orchestra.run.smithi078:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-08-18T23:41:56.254 INFO:teuthology.orchestra.run.smithi078.stdout:fuseblk 2024-08-18T23:41:56.254 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-08-18T23:41:56.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:41:56.254 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-08-18T23:41:56.319 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-08-18T23:41:56.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-18T23:41:56.433 INFO:teuthology.orchestra.run.smithi078.stdout:# file: . 2024-08-18T23:41:56.433 INFO:teuthology.orchestra.run.smithi078.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-08-18T23:41:56.433 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:41:56.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-18T23:41:56.440 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:41:56.440 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-18T23:41:56.457 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd blocklist ls 2024-08-18T23:41:56.747 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:57.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.282+0000 7f94fc5fa640 1 -- 172.21.15.78:0/3401874804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94f41067b0 msgr2=0x7f94f4106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:57.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.282+0000 7f94fc5fa640 1 --2- 172.21.15.78:0/3401874804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94f41067b0 0x7f94f4106b90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f94e8009900 tx=0x7f94e802a410 comp rx=0 tx=0).stop 2024-08-18T23:41:57.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.282+0000 7f94fc5fa640 1 -- 172.21.15.78:0/3401874804 shutdown_connections 2024-08-18T23:41:57.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.282+0000 7f94fc5fa640 1 --2- 172.21.15.78:0/3401874804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94f4101150 0x7f94f410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:57.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.282+0000 7f94fc5fa640 1 --2- 172.21.15.78:0/3401874804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94f41007b0 0x7f94f4100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:57.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.282+0000 7f94fc5fa640 1 --2- 172.21.15.78:0/3401874804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94f41067b0 0x7f94f4106b90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:57.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.282+0000 7f94fc5fa640 1 -- 172.21.15.78:0/3401874804 >> 172.21.15.78:0/3401874804 conn(0x7f94f40fc4d0 msgr2=0x7f94f40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:57.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.282+0000 7f94fc5fa640 1 -- 172.21.15.78:0/3401874804 shutdown_connections 2024-08-18T23:41:57.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.282+0000 7f94fc5fa640 1 -- 172.21.15.78:0/3401874804 wait complete. 2024-08-18T23:41:57.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94fc5fa640 1 Processor -- start 2024-08-18T23:41:57.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94fc5fa640 1 -- start start 2024-08-18T23:41:57.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94fc5fa640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94f41007b0 0x7f94f4198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:57.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94fc5fa640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94f4101150 0x7f94f4198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:57.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94fc5fa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94f41067b0 0x7f94f419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:57.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94fc5fa640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94f4111cd0 con 0x7f94f41067b0 2024-08-18T23:41:57.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94fc5fa640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94f4111b50 con 0x7f94f41007b0 2024-08-18T23:41:57.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94fc5fa640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f94f4111e50 con 0x7f94f4101150 2024-08-18T23:41:57.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94fab97640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94f41067b0 0x7f94f419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:57.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94fab97640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94f41067b0 0x7f94f419ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35706/0 (socket says 172.21.15.78:35706) 2024-08-18T23:41:57.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94fab97640 1 -- 172.21.15.78:0/2247168632 learned_addr learned my addr 172.21.15.78:0/2247168632 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:57.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94f9b95640 1 --2- 172.21.15.78:0/2247168632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94f4101150 0x7f94f4198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:57.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94fa396640 1 --2- 172.21.15.78:0/2247168632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94f41007b0 0x7f94f4198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:57.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94fab97640 1 -- 172.21.15.78:0/2247168632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94f4101150 msgr2=0x7f94f4198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:57.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94fab97640 1 --2- 172.21.15.78:0/2247168632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94f4101150 0x7f94f4198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:57.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94fab97640 1 -- 172.21.15.78:0/2247168632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94f41007b0 msgr2=0x7f94f4198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:57.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94fab97640 1 --2- 172.21.15.78:0/2247168632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94f41007b0 0x7f94f4198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94fab97640 1 -- 172.21.15.78:0/2247168632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f419d0e0 con 0x7f94f41067b0 2024-08-18T23:41:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94f9b95640 1 --2- 172.21.15.78:0/2247168632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94f4101150 0x7f94f4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:41:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.283+0000 7f94fa396640 1 --2- 172.21.15.78:0/2247168632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94f41007b0 0x7f94f4198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:41:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.284+0000 7f94fab97640 1 --2- 172.21.15.78:0/2247168632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94f41067b0 0x7f94f419ca00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00b970 tx=0x7f94ec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.284+0000 7f94f37fe640 1 -- 172.21.15.78:0/2247168632 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec019070 con 0x7f94f41067b0 2024-08-18T23:41:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.284+0000 7f94fc5fa640 1 -- 172.21.15.78:0/2247168632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f419d3d0 con 0x7f94f41067b0 2024-08-18T23:41:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.284+0000 7f94f37fe640 1 -- 172.21.15.78:0/2247168632 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94ec00cb40 con 0x7f94f41067b0 2024-08-18T23:41:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.284+0000 7f94f37fe640 1 -- 172.21.15.78:0/2247168632 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec007610 con 0x7f94f41067b0 2024-08-18T23:41:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.284+0000 7f94fc5fa640 1 -- 172.21.15.78:0/2247168632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f41a4cb0 con 0x7f94f41067b0 2024-08-18T23:41:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.285+0000 7f94fc5fa640 1 -- 172.21.15.78:0/2247168632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94f4100c10 con 0x7f94f41067b0 2024-08-18T23:41:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.286+0000 7f94f37fe640 1 -- 172.21.15.78:0/2247168632 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f94ec01c070 con 0x7f94f41067b0 2024-08-18T23:41:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.286+0000 7f94f37fe640 1 --2- 172.21.15.78:0/2247168632 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94dc0a2b80 0x7f94dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.287+0000 7f94fa396640 1 --2- 172.21.15.78:0/2247168632 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94dc0a2b80 0x7f94dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.287+0000 7f94f37fe640 1 -- 172.21.15.78:0/2247168632 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f94ec0ced70 con 0x7f94f41067b0 2024-08-18T23:41:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.287+0000 7f94fa396640 1 --2- 172.21.15.78:0/2247168632 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94dc0a2b80 0x7f94dc0a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f94e8000c00 tx=0x7f94e8032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.290+0000 7f94f37fe640 1 -- 172.21.15.78:0/2247168632 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f94ec09ba90 con 0x7f94f41067b0 2024-08-18T23:41:57.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.467+0000 7f94fc5fa640 1 -- 172.21.15.78:0/2247168632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f94f41a4fb0 con 0x7f94f41067b0 2024-08-18T23:41:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:57 smithi078 ceph-mon[18440]: pgmap v275: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:57.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.467+0000 7f94f37fe640 1 -- 172.21.15.78:0/2247168632 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+588 (secure 0 0 0) 0x7f94ec09b390 con 0x7f94f41067b0 2024-08-18T23:41:57.469 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/311758909 2024-08-19T23:35:22.036055+0000 2024-08-18T23:41:57.470 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/311758909 2024-08-19T23:35:22.036055+0000 2024-08-18T23:41:57.470 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/3040499240 2024-08-19T23:35:22.036055+0000 2024-08-18T23:41:57.470 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/2301978296 2024-08-19T23:34:59.021165+0000 2024-08-18T23:41:57.470 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/2301978296 2024-08-19T23:34:59.021165+0000 2024-08-18T23:41:57.470 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/757409165 2024-08-19T23:34:59.021165+0000 2024-08-18T23:41:57.470 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2562072965 2024-08-19T23:35:22.036055+0000 2024-08-18T23:41:57.470 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/636184437 2024-08-19T23:34:59.021165+0000 2024-08-18T23:41:57.470 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/1570708294 2024-08-19T23:35:22.036055+0000 2024-08-18T23:41:57.470 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/3137806628 2024-08-19T23:34:59.021165+0000 2024-08-18T23:41:57.471 INFO:teuthology.orchestra.run.smithi078.stderr:listed 10 entries 2024-08-18T23:41:57.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.469+0000 7f94fc5fa640 1 -- 172.21.15.78:0/2247168632 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94dc0a2b80 msgr2=0x7f94dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:57.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.469+0000 7f94fc5fa640 1 --2- 172.21.15.78:0/2247168632 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94dc0a2b80 0x7f94dc0a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f94e8000c00 tx=0x7f94e8032d50 comp rx=0 tx=0).stop 2024-08-18T23:41:57.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.469+0000 7f94fc5fa640 1 -- 172.21.15.78:0/2247168632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94f41067b0 msgr2=0x7f94f419ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:57.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.469+0000 7f94fc5fa640 1 --2- 172.21.15.78:0/2247168632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94f41067b0 0x7f94f419ca00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00b970 tx=0x7f94ec00be30 comp rx=0 tx=0).stop 2024-08-18T23:41:57.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.469+0000 7f94fc5fa640 1 -- 172.21.15.78:0/2247168632 shutdown_connections 2024-08-18T23:41:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.469+0000 7f94fc5fa640 1 --2- 172.21.15.78:0/2247168632 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94dc0a2b80 0x7f94dc0a5040 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.469+0000 7f94fc5fa640 1 --2- 172.21.15.78:0/2247168632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94f41067b0 0x7f94f419ca00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.469+0000 7f94fc5fa640 1 --2- 172.21.15.78:0/2247168632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94f4101150 0x7f94f4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.469+0000 7f94fc5fa640 1 --2- 172.21.15.78:0/2247168632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94f41007b0 0x7f94f4198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.469+0000 7f94fc5fa640 1 -- 172.21.15.78:0/2247168632 >> 172.21.15.78:0/2247168632 conn(0x7f94f40fc4d0 msgr2=0x7f94f40fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.469+0000 7f94fc5fa640 1 -- 172.21.15.78:0/2247168632 shutdown_connections 2024-08-18T23:41:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:57.469+0000 7f94fc5fa640 1 -- 172.21.15.78:0/2247168632 wait complete. 2024-08-18T23:41:57.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:57 smithi123 ceph-mon[21808]: pgmap v275: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:57 smithi161 ceph-mon[22027]: pgmap v275: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:58.153 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:41:58.153 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-18T23:41:58.172 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd blocklist ls 2024-08-18T23:41:58.464 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:41:58.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:58 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2247168632' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-18T23:41:58.587 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:58 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2247168632' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-18T23:41:58.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:58 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2247168632' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-18T23:41:58.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.956+0000 7fe88b348640 1 -- 172.21.15.78:0/2104797742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe884106810 msgr2=0x7fe884106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:58.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.956+0000 7fe88b348640 1 --2- 172.21.15.78:0/2104797742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe884106810 0x7fe884106bf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe880009930 tx=0x7fe88002a420 comp rx=0 tx=0).stop 2024-08-18T23:41:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.956+0000 7fe88b348640 1 -- 172.21.15.78:0/2104797742 shutdown_connections 2024-08-18T23:41:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.956+0000 7fe88b348640 1 --2- 172.21.15.78:0/2104797742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe884101900 0x7fe88410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.956+0000 7fe88b348640 1 --2- 172.21.15.78:0/2104797742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe884100f40 0x7fe8841013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.956+0000 7fe88b348640 1 --2- 172.21.15.78:0/2104797742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe884106810 0x7fe884106bf0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.956+0000 7fe88b348640 1 -- 172.21.15.78:0/2104797742 >> 172.21.15.78:0/2104797742 conn(0x7fe884076f60 msgr2=0x7fe8840fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.956+0000 7fe88b348640 1 -- 172.21.15.78:0/2104797742 shutdown_connections 2024-08-18T23:41:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.956+0000 7fe88b348640 1 -- 172.21.15.78:0/2104797742 wait complete. 2024-08-18T23:41:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.957+0000 7fe88b348640 1 Processor -- start 2024-08-18T23:41:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.957+0000 7fe88b348640 1 -- start start 2024-08-18T23:41:58.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.957+0000 7fe88b348640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe884100f40 0x7fe884199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.957+0000 7fe88b348640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe884101900 0x7fe884199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.957+0000 7fe88b348640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe884106810 0x7fe8841a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.957+0000 7fe88b348640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe884110b90 con 0x7fe884101900 2024-08-18T23:41:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.957+0000 7fe88b348640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe884110a10 con 0x7fe884100f40 2024-08-18T23:41:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.957+0000 7fe88b348640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe884110d10 con 0x7fe884106810 2024-08-18T23:41:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.957+0000 7fe8888e3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe884101900 0x7fe884199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.957+0000 7fe8888e3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe884101900 0x7fe884199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35720/0 (socket says 172.21.15.78:35720) 2024-08-18T23:41:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.957+0000 7fe8888e3640 1 -- 172.21.15.78:0/3576008217 learned_addr learned my addr 172.21.15.78:0/3576008217 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:41:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.957+0000 7fe8890e4640 1 --2- 172.21.15.78:0/3576008217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe884100f40 0x7fe884199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.958+0000 7fe8898e5640 1 --2- 172.21.15.78:0/3576008217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe884106810 0x7fe8841a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.958+0000 7fe8888e3640 1 -- 172.21.15.78:0/3576008217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe884106810 msgr2=0x7fe8841a4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.958+0000 7fe8888e3640 1 --2- 172.21.15.78:0/3576008217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe884106810 0x7fe8841a4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.958+0000 7fe8888e3640 1 -- 172.21.15.78:0/3576008217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe884100f40 msgr2=0x7fe884199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.958+0000 7fe8888e3640 1 --2- 172.21.15.78:0/3576008217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe884100f40 0x7fe884199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.958+0000 7fe8888e3640 1 -- 172.21.15.78:0/3576008217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8841a45c0 con 0x7fe884101900 2024-08-18T23:41:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.958+0000 7fe8890e4640 1 --2- 172.21.15.78:0/3576008217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe884100f40 0x7fe884199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:41:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.958+0000 7fe8888e3640 1 --2- 172.21.15.78:0/3576008217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe884101900 0x7fe884199540 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe87400b4a0 tx=0x7fe87400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.958+0000 7fe87a7fc640 1 -- 172.21.15.78:0/3576008217 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874014070 con 0x7fe884101900 2024-08-18T23:41:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.958+0000 7fe87a7fc640 1 -- 172.21.15.78:0/3576008217 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe874010af0 con 0x7fe884101900 2024-08-18T23:41:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.958+0000 7fe88b348640 1 -- 172.21.15.78:0/3576008217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8841a4790 con 0x7fe884101900 2024-08-18T23:41:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.958+0000 7fe87a7fc640 1 -- 172.21.15.78:0/3576008217 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874017d10 con 0x7fe884101900 2024-08-18T23:41:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.958+0000 7fe88b348640 1 -- 172.21.15.78:0/3576008217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8841a4ca0 con 0x7fe884101900 2024-08-18T23:41:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.959+0000 7fe88b348640 1 -- 172.21.15.78:0/3576008217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8841013c0 con 0x7fe884101900 2024-08-18T23:41:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.959+0000 7fe87a7fc640 1 -- 172.21.15.78:0/3576008217 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe874017070 con 0x7fe884101900 2024-08-18T23:41:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.961+0000 7fe87a7fc640 1 --2- 172.21.15.78:0/3576008217 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe8700a2c00 0x7fe8700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:41:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.961+0000 7fe87a7fc640 1 -- 172.21.15.78:0/3576008217 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fe8740ce690 con 0x7fe884101900 2024-08-18T23:41:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.961+0000 7fe8890e4640 1 --2- 172.21.15.78:0/3576008217 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe8700a2c00 0x7fe8700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:41:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.961+0000 7fe8890e4640 1 --2- 172.21.15.78:0/3576008217 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe8700a2c00 0x7fe8700a50c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe880000c00 tx=0x7fe880005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:41:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:58.964+0000 7fe87a7fc640 1 -- 172.21.15.78:0/3576008217 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe87409b3b0 con 0x7fe884101900 2024-08-18T23:41:59.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:59.144+0000 7fe88b348640 1 -- 172.21.15.78:0/3576008217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe884108d20 con 0x7fe884101900 2024-08-18T23:41:59.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:59.144+0000 7fe87a7fc640 1 -- 172.21.15.78:0/3576008217 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+588 (secure 0 0 0) 0x7fe87409acb0 con 0x7fe884101900 2024-08-18T23:41:59.145 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/311758909 2024-08-19T23:35:22.036055+0000 2024-08-18T23:41:59.146 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/311758909 2024-08-19T23:35:22.036055+0000 2024-08-18T23:41:59.146 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/3040499240 2024-08-19T23:35:22.036055+0000 2024-08-18T23:41:59.146 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/2301978296 2024-08-19T23:34:59.021165+0000 2024-08-18T23:41:59.146 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/2301978296 2024-08-19T23:34:59.021165+0000 2024-08-18T23:41:59.146 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/757409165 2024-08-19T23:34:59.021165+0000 2024-08-18T23:41:59.146 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2562072965 2024-08-19T23:35:22.036055+0000 2024-08-18T23:41:59.146 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/636184437 2024-08-19T23:34:59.021165+0000 2024-08-18T23:41:59.146 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/1570708294 2024-08-19T23:35:22.036055+0000 2024-08-18T23:41:59.146 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/3137806628 2024-08-19T23:34:59.021165+0000 2024-08-18T23:41:59.147 INFO:teuthology.orchestra.run.smithi078.stderr:listed 10 entries 2024-08-18T23:41:59.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:59.146+0000 7fe88b348640 1 -- 172.21.15.78:0/3576008217 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe8700a2c00 msgr2=0x7fe8700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:59.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:59.146+0000 7fe88b348640 1 --2- 172.21.15.78:0/3576008217 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe8700a2c00 0x7fe8700a50c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe880000c00 tx=0x7fe880005a20 comp rx=0 tx=0).stop 2024-08-18T23:41:59.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:59.146+0000 7fe88b348640 1 -- 172.21.15.78:0/3576008217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe884101900 msgr2=0x7fe884199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:41:59.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:59.146+0000 7fe88b348640 1 --2- 172.21.15.78:0/3576008217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe884101900 0x7fe884199540 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe87400b4a0 tx=0x7fe87400b960 comp rx=0 tx=0).stop 2024-08-18T23:41:59.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:59.146+0000 7fe88b348640 1 -- 172.21.15.78:0/3576008217 shutdown_connections 2024-08-18T23:41:59.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:59.146+0000 7fe88b348640 1 --2- 172.21.15.78:0/3576008217 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe8700a2c00 0x7fe8700a50c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:59.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:59.146+0000 7fe88b348640 1 --2- 172.21.15.78:0/3576008217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe884106810 0x7fe8841a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:59.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:59.146+0000 7fe88b348640 1 --2- 172.21.15.78:0/3576008217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe884101900 0x7fe884199540 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:59.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:59.146+0000 7fe88b348640 1 --2- 172.21.15.78:0/3576008217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe884100f40 0x7fe884199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:41:59.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:59.146+0000 7fe88b348640 1 -- 172.21.15.78:0/3576008217 >> 172.21.15.78:0/3576008217 conn(0x7fe884076f60 msgr2=0x7fe8840fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:41:59.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:59.147+0000 7fe88b348640 1 -- 172.21.15.78:0/3576008217 shutdown_connections 2024-08-18T23:41:59.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:41:59.147+0000 7fe88b348640 1 -- 172.21.15.78:0/3576008217 wait complete. 2024-08-18T23:41:59.423 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:59 smithi078 ceph-mon[18440]: pgmap v276: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:59.424 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:41:59 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3576008217' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-18T23:41:59.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:59 smithi123 ceph-mon[21808]: pgmap v276: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:59.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:41:59 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3576008217' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-18T23:41:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:59 smithi161 ceph-mon[22027]: pgmap v276: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:41:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:41:59 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3576008217' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-18T23:41:59.819 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi078.front.sepia.ceph.com... 2024-08-18T23:41:59.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:41:59.819 DEBUG:teuthology.orchestra.run.smithi078:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-08-18T23:41:59.845 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi078.stderr:ceph-fuse[62362]: fuse finished with error 0 and tester_r 0 2024-08-18T23:41:59.848 INFO:teuthology.orchestra.run:waiting for 300 2024-08-18T23:41:59.944 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-08-18T23:41:59.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:41:59.944 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-08-18T23:41:59.973 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-18T23:42:00.274 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:00.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.774+0000 7fb4cdb2d640 1 -- 172.21.15.78:0/3882123224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c8103390 msgr2=0x7fb4c810b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:00.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.774+0000 7fb4cdb2d640 1 --2- 172.21.15.78:0/3882123224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c8103390 0x7fb4c810b880 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009900 tx=0x7fb4bc02a480 comp rx=0 tx=0).stop 2024-08-18T23:42:00.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.774+0000 7fb4cdb2d640 1 -- 172.21.15.78:0/3882123224 shutdown_connections 2024-08-18T23:42:00.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.774+0000 7fb4cdb2d640 1 --2- 172.21.15.78:0/3882123224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4c810bea0 0x7fb4c810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:00.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.774+0000 7fb4cdb2d640 1 --2- 172.21.15.78:0/3882123224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c8103390 0x7fb4c810b880 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:00.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.774+0000 7fb4cdb2d640 1 --2- 172.21.15.78:0/3882123224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4c8102a70 0x7fb4c8102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:00.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.774+0000 7fb4cdb2d640 1 -- 172.21.15.78:0/3882123224 >> 172.21.15.78:0/3882123224 conn(0x7fb4c80fc4d0 msgr2=0x7fb4c80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:00.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.774+0000 7fb4cdb2d640 1 -- 172.21.15.78:0/3882123224 shutdown_connections 2024-08-18T23:42:00.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.774+0000 7fb4cdb2d640 1 -- 172.21.15.78:0/3882123224 wait complete. 2024-08-18T23:42:00.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.774+0000 7fb4cdb2d640 1 Processor -- start 2024-08-18T23:42:00.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.774+0000 7fb4cdb2d640 1 -- start start 2024-08-18T23:42:00.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4cdb2d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c8102a70 0x7fb4c81006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:00.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4cdb2d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4c8103390 0x7fb4c8100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:00.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4cdb2d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4c810bea0 0x7fb4c8101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:00.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4cdb2d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c8111ca0 con 0x7fb4c8102a70 2024-08-18T23:42:00.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4cdb2d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c8111b20 con 0x7fb4c810bea0 2024-08-18T23:42:00.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4cdb2d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c8111e20 con 0x7fb4c8103390 2024-08-18T23:42:00.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4c77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c8102a70 0x7fb4c81006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:00.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4c77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c8102a70 0x7fb4c81006d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35742/0 (socket says 172.21.15.78:35742) 2024-08-18T23:42:00.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4c77fe640 1 -- 172.21.15.78:0/2490399435 learned_addr learned my addr 172.21.15.78:0/2490399435 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:00.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4c6ffd640 1 --2- 172.21.15.78:0/2490399435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4c8103390 0x7fb4c8100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:00.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4c7fff640 1 --2- 172.21.15.78:0/2490399435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4c810bea0 0x7fb4c8101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:00.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4c77fe640 1 -- 172.21.15.78:0/2490399435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4c8103390 msgr2=0x7fb4c8100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:00.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4c77fe640 1 --2- 172.21.15.78:0/2490399435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4c8103390 0x7fb4c8100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:00.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4c77fe640 1 -- 172.21.15.78:0/2490399435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4c810bea0 msgr2=0x7fb4c8101150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:00.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4c77fe640 1 --2- 172.21.15.78:0/2490399435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4c810bea0 0x7fb4c8101150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:00.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4c77fe640 1 -- 172.21.15.78:0/2490399435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c81a6660 con 0x7fb4c8102a70 2024-08-18T23:42:00.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4c6ffd640 1 --2- 172.21.15.78:0/2490399435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4c8103390 0x7fb4c8100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:42:00.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4c7fff640 1 --2- 172.21.15.78:0/2490399435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4c810bea0 0x7fb4c8101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:42:00.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4c77fe640 1 --2- 172.21.15.78:0/2490399435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c8102a70 0x7fb4c81006d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800ea80 tx=0x7fb4b800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:00.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4c4ff9640 1 -- 172.21.15.78:0/2490399435 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b800ce40 con 0x7fb4c8102a70 2024-08-18T23:42:00.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4c4ff9640 1 -- 172.21.15.78:0/2490399435 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4b8005020 con 0x7fb4c8102a70 2024-08-18T23:42:00.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4cdb2d640 1 -- 172.21.15.78:0/2490399435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c81a6950 con 0x7fb4c8102a70 2024-08-18T23:42:00.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4c4ff9640 1 -- 172.21.15.78:0/2490399435 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8011760 con 0x7fb4c8102a70 2024-08-18T23:42:00.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.775+0000 7fb4cdb2d640 1 -- 172.21.15.78:0/2490399435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c81a6e90 con 0x7fb4c8102a70 2024-08-18T23:42:00.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.776+0000 7fb4cdb2d640 1 -- 172.21.15.78:0/2490399435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4c8102e50 con 0x7fb4c8102a70 2024-08-18T23:42:00.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.777+0000 7fb4c4ff9640 1 -- 172.21.15.78:0/2490399435 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb4b80051d0 con 0x7fb4c8102a70 2024-08-18T23:42:00.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.778+0000 7fb4c4ff9640 1 --2- 172.21.15.78:0/2490399435 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4b00a2c00 0x7fb4b00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:00.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.778+0000 7fb4c6ffd640 1 --2- 172.21.15.78:0/2490399435 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4b00a2c00 0x7fb4b00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:00.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.778+0000 7fb4c4ff9640 1 -- 172.21.15.78:0/2490399435 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fb4b80cf8d0 con 0x7fb4c8102a70 2024-08-18T23:42:00.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.779+0000 7fb4c6ffd640 1 --2- 172.21.15.78:0/2490399435 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4b00a2c00 0x7fb4b00a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc000c00 tx=0x7fb4bc032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:00.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.783+0000 7fb4c4ff9640 1 -- 172.21.15.78:0/2490399435 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb4b8016030 con 0x7fb4c8102a70 2024-08-18T23:42:00.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.950+0000 7fb4cdb2d640 1 -- 172.21.15.78:0/2490399435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4c802d5e0 con 0x7fb4c8102a70 2024-08-18T23:42:00.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.952+0000 7fb4c4ff9640 1 -- 172.21.15.78:0/2490399435 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24126 (secure 0 0 0) 0x7fb4b809c5f0 con 0x7fb4c8102a70 2024-08-18T23:42:00.953 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:42:00.953 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":88,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","created":"2024-08-18T23:34:33.867246+0000","modified":"2024-08-18T23:41:41.811101+0000","last_up_change":"2024-08-18T23:40:42.200381+0000","last_in_change":"2024-08-18T23:40:22.425813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T23:37:22.338211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T23:41:28.920802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T23:41:31.026094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T23:41:37.403238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4de1c78-6ebc-49a9-b9c5-85d43535fab7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6803","nonce":3639645311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6805","nonce":3639645311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6809","nonce":3639645311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6807","nonce":3639645311}]},"public_addr":"172.21.15.78:6803/3639645311","cluster_addr":"172.21.15.78:6805/3639645311","heartbeat_back_addr":"172.21.15.78:6809/3639645311","heartbeat_front_addr":"172.21.15.78:6807/3639645311","state":["exists","up"]},{"osd":1,"uuid":"4f8e0c3d-472b-4230-ad23-2b0a34861251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6801","nonce":4199446368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6803","nonce":4199446368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6807","nonce":4199446368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6805","nonce":4199446368}]},"public_addr":"172.21.15.123:6801/4199446368","cluster_addr":"172.21.15.123:6803/4199446368","heartbeat_back_addr":"172.21.15.123:6807/4199446368","heartbeat_front_addr":"172.21.15.123:6805/4199446368","state":["exists","up"]},{"osd":2,"uuid":"27ef3314-bb6f-413d-be05-e10a105dec9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6801","nonce":2582810057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6803","nonce":2582810057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6807","nonce":2582810057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6805","nonce":2582810057}]},"public_addr":"172.21.15.161:6801/2582810057","cluster_addr":"172.21.15.161:6803/2582810057","heartbeat_back_addr":"172.21.15.161:6807/2582810057","heartbeat_front_addr":"172.21.15.161:6805/2582810057","state":["exists","up"]},{"osd":3,"uuid":"e9e52662-5ef9-4d70-bdd7-2b92165822ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6811","nonce":2697347521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6813","nonce":2697347521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6817","nonce":2697347521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6815","nonce":2697347521}]},"public_addr":"172.21.15.78:6811/2697347521","cluster_addr":"172.21.15.78:6813/2697347521","heartbeat_back_addr":"172.21.15.78:6817/2697347521","heartbeat_front_addr":"172.21.15.78:6815/2697347521","state":["exists","up"]},{"osd":4,"uuid":"5ed26775-79e3-4b24-9557-69fcf5e8e824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6809","nonce":1281511569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6811","nonce":1281511569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6815","nonce":1281511569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6813","nonce":1281511569}]},"public_addr":"172.21.15.123:6809/1281511569","cluster_addr":"172.21.15.123:6811/1281511569","heartbeat_back_addr":"172.21.15.123:6815/1281511569","heartbeat_front_addr":"172.21.15.123:6813/1281511569","state":["exists","up"]},{"osd":5,"uuid":"a9793b1d-5613-48a4-938e-adb718f4fa4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6809","nonce":2751796117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6811","nonce":2751796117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6815","nonce":2751796117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6813","nonce":2751796117}]},"public_addr":"172.21.15.161:6809/2751796117","cluster_addr":"172.21.15.161:6811/2751796117","heartbeat_back_addr":"172.21.15.161:6815/2751796117","heartbeat_front_addr":"172.21.15.161:6813/2751796117","state":["exists","up"]},{"osd":6,"uuid":"5dbc5e9c-a14f-4b71-910a-1e6d4827bb32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6819","nonce":95383223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6821","nonce":95383223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6825","nonce":95383223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6823","nonce":95383223}]},"public_addr":"172.21.15.78:6819/95383223","cluster_addr":"172.21.15.78:6821/95383223","heartbeat_back_addr":"172.21.15.78:6825/95383223","heartbeat_front_addr":"172.21.15.78:6823/95383223","state":["exists","up"]},{"osd":7,"uuid":"26fb7b69-131f-42ee-b745-6fe8b4f65391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6817","nonce":3440305156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6819","nonce":3440305156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6823","nonce":3440305156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6821","nonce":3440305156}]},"public_addr":"172.21.15.123:6817/3440305156","cluster_addr":"172.21.15.123:6819/3440305156","heartbeat_back_addr":"172.21.15.123:6823/3440305156","heartbeat_front_addr":"172.21.15.123:6821/3440305156","state":["exists","up"]},{"osd":8,"uuid":"e267bc50-0c29-4d7b-b368-b1621e29b2df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6817","nonce":1485008778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6819","nonce":1485008778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6823","nonce":1485008778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6821","nonce":1485008778}]},"public_addr":"172.21.15.161:6817/1485008778","cluster_addr":"172.21.15.161:6819/1485008778","heartbeat_back_addr":"172.21.15.161:6823/1485008778","heartbeat_front_addr":"172.21.15.161:6821/1485008778","state":["exists","up"]},{"osd":9,"uuid":"b7654818-b767-4fe6-8211-346297895f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6827","nonce":2884475661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6829","nonce":2884475661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6833","nonce":2884475661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6831","nonce":2884475661}]},"public_addr":"172.21.15.78:6827/2884475661","cluster_addr":"172.21.15.78:6829/2884475661","heartbeat_back_addr":"172.21.15.78:6833/2884475661","heartbeat_front_addr":"172.21.15.78:6831/2884475661","state":["exists","up"]},{"osd":10,"uuid":"59aa32ff-510e-4b52-8170-08a34a0b6b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6825","nonce":122226286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6827","nonce":122226286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6831","nonce":122226286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6829","nonce":122226286}]},"public_addr":"172.21.15.123:6825/122226286","cluster_addr":"172.21.15.123:6827/122226286","heartbeat_back_addr":"172.21.15.123:6831/122226286","heartbeat_front_addr":"172.21.15.123:6829/122226286","state":["exists","up"]},{"osd":11,"uuid":"67bacab4-d078-4c5c-b4ff-9f6a8a00f507","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6825","nonce":2037560985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6827","nonce":2037560985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6831","nonce":2037560985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6829","nonce":2037560985}]},"public_addr":"172.21.15.161:6825/2037560985","cluster_addr":"172.21.15.161:6827/2037560985","heartbeat_back_addr":"172.21.15.161:6831/2037560985","heartbeat_front_addr":"172.21.15.161:6829/2037560985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:36.826044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:57.688421+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:18.297424+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:40.355215+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:02.167902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:23.968759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:46.254083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:08.522038+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:31.028119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:54.119807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:16.847034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:39.607886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6800/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3040499240":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6801/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:6800/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/757409165":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/2562072965":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/636184437":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/1570708294":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3137806628":"2024-08-19T23:34:59.021165+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T23:42:00.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.954+0000 7fb4cdb2d640 1 -- 172.21.15.78:0/2490399435 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4b00a2c00 msgr2=0x7fb4b00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:00.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.954+0000 7fb4cdb2d640 1 --2- 172.21.15.78:0/2490399435 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4b00a2c00 0x7fb4b00a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc000c00 tx=0x7fb4bc032c70 comp rx=0 tx=0).stop 2024-08-18T23:42:00.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.954+0000 7fb4cdb2d640 1 -- 172.21.15.78:0/2490399435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c8102a70 msgr2=0x7fb4c81006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:00.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.954+0000 7fb4cdb2d640 1 --2- 172.21.15.78:0/2490399435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c8102a70 0x7fb4c81006d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800ea80 tx=0x7fb4b800ef40 comp rx=0 tx=0).stop 2024-08-18T23:42:00.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.954+0000 7fb4cdb2d640 1 -- 172.21.15.78:0/2490399435 shutdown_connections 2024-08-18T23:42:00.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.954+0000 7fb4cdb2d640 1 --2- 172.21.15.78:0/2490399435 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4b00a2c00 0x7fb4b00a50c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:00.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.954+0000 7fb4cdb2d640 1 --2- 172.21.15.78:0/2490399435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4c810bea0 0x7fb4c8101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:00.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.954+0000 7fb4cdb2d640 1 --2- 172.21.15.78:0/2490399435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4c8103390 0x7fb4c8100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:00.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.954+0000 7fb4cdb2d640 1 --2- 172.21.15.78:0/2490399435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4c8102a70 0x7fb4c81006d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:00.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.954+0000 7fb4cdb2d640 1 -- 172.21.15.78:0/2490399435 >> 172.21.15.78:0/2490399435 conn(0x7fb4c80fc4d0 msgr2=0x7fb4c80fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:00.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.954+0000 7fb4cdb2d640 1 -- 172.21.15.78:0/2490399435 shutdown_connections 2024-08-18T23:42:00.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:00.954+0000 7fb4cdb2d640 1 -- 172.21.15.78:0/2490399435 wait complete. 2024-08-18T23:42:01.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:01 smithi123 ceph-mon[21808]: pgmap v277: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-08-18T23:42:01.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:01 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2490399435' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:42:01.582 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:01 smithi078 ceph-mon[18440]: pgmap v277: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-08-18T23:42:01.582 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:01 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2490399435' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:42:01.611 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-18T23:42:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:01 smithi161 ceph-mon[22027]: pgmap v277: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-08-18T23:42:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:01 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2490399435' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:42:01.861 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:02.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.357+0000 7fe5a004b640 1 -- 172.21.15.78:0/1312699355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe598100b60 msgr2=0x7fe598104bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:02.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.357+0000 7fe5a004b640 1 --2- 172.21.15.78:0/1312699355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe598100b60 0x7fe598104bb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009980 tx=0x7fe58c02c620 comp rx=0 tx=0).stop 2024-08-18T23:42:02.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.357+0000 7fe5a004b640 1 -- 172.21.15.78:0/1312699355 shutdown_connections 2024-08-18T23:42:02.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.357+0000 7fe5a004b640 1 --2- 172.21.15.78:0/1312699355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5981052e0 0x7fe59810ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:02.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.357+0000 7fe5a004b640 1 --2- 172.21.15.78:0/1312699355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe598100b60 0x7fe598104bb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:02.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.357+0000 7fe5a004b640 1 --2- 172.21.15.78:0/1312699355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5981001b0 0x7fe598100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:02.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.357+0000 7fe5a004b640 1 -- 172.21.15.78:0/1312699355 >> 172.21.15.78:0/1312699355 conn(0x7fe598076f40 msgr2=0x7fe5980fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:02.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.357+0000 7fe5a004b640 1 -- 172.21.15.78:0/1312699355 shutdown_connections 2024-08-18T23:42:02.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.357+0000 7fe5a004b640 1 -- 172.21.15.78:0/1312699355 wait complete. 2024-08-18T23:42:02.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.357+0000 7fe5a004b640 1 Processor -- start 2024-08-18T23:42:02.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.357+0000 7fe5a004b640 1 -- start start 2024-08-18T23:42:02.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.358+0000 7fe5a004b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5981001b0 0x7fe598195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:02.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.358+0000 7fe5a004b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe598100b60 0x7fe5981963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:02.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.358+0000 7fe5a004b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5981052e0 0x7fe59819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:02.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.358+0000 7fe5a004b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe59810fac0 con 0x7fe5981001b0 2024-08-18T23:42:02.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.358+0000 7fe5a004b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe59810f940 con 0x7fe5981052e0 2024-08-18T23:42:02.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.358+0000 7fe5a004b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe59810fc40 con 0x7fe598100b60 2024-08-18T23:42:02.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.358+0000 7fe59dde7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5981001b0 0x7fe598195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:02.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.358+0000 7fe59dde7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5981001b0 0x7fe598195eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35770/0 (socket says 172.21.15.78:35770) 2024-08-18T23:42:02.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.358+0000 7fe59dde7640 1 -- 172.21.15.78:0/1221438572 learned_addr learned my addr 172.21.15.78:0/1221438572 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:02.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.358+0000 7fe59e5e8640 1 --2- 172.21.15.78:0/1221438572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5981052e0 0x7fe59819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:02.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.358+0000 7fe59d5e6640 1 --2- 172.21.15.78:0/1221438572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe598100b60 0x7fe5981963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:02.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.358+0000 7fe59dde7640 1 -- 172.21.15.78:0/1221438572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe598100b60 msgr2=0x7fe5981963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:02.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.358+0000 7fe59dde7640 1 --2- 172.21.15.78:0/1221438572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe598100b60 0x7fe5981963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:02.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.358+0000 7fe59dde7640 1 -- 172.21.15.78:0/1221438572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5981052e0 msgr2=0x7fe59819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:02.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.358+0000 7fe59dde7640 1 --2- 172.21.15.78:0/1221438572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5981052e0 0x7fe59819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:02.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.358+0000 7fe59dde7640 1 -- 172.21.15.78:0/1221438572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59819ae60 con 0x7fe5981001b0 2024-08-18T23:42:02.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.358+0000 7fe59e5e8640 1 --2- 172.21.15.78:0/1221438572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5981052e0 0x7fe59819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:42:02.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.358+0000 7fe59dde7640 1 --2- 172.21.15.78:0/1221438572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5981001b0 0x7fe598195eb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe58800cce0 tx=0x7fe58800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:02.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.358+0000 7fe596ffd640 1 -- 172.21.15.78:0/1221438572 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58800f650 con 0x7fe5981001b0 2024-08-18T23:42:02.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.359+0000 7fe5a004b640 1 -- 172.21.15.78:0/1221438572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59819b150 con 0x7fe5981001b0 2024-08-18T23:42:02.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.359+0000 7fe596ffd640 1 -- 172.21.15.78:0/1221438572 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe58800fcd0 con 0x7fe5981001b0 2024-08-18T23:42:02.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.359+0000 7fe596ffd640 1 -- 172.21.15.78:0/1221438572 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588018c70 con 0x7fe5981001b0 2024-08-18T23:42:02.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.359+0000 7fe5a004b640 1 -- 172.21.15.78:0/1221438572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5981a2a30 con 0x7fe5981001b0 2024-08-18T23:42:02.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.360+0000 7fe5a004b640 1 -- 172.21.15.78:0/1221438572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe598100590 con 0x7fe5981001b0 2024-08-18T23:42:02.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.361+0000 7fe596ffd640 1 -- 172.21.15.78:0/1221438572 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe58800f7f0 con 0x7fe5981001b0 2024-08-18T23:42:02.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.361+0000 7fe596ffd640 1 --2- 172.21.15.78:0/1221438572 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe5800a2b80 0x7fe5800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:02.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.361+0000 7fe59d5e6640 1 --2- 172.21.15.78:0/1221438572 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe5800a2b80 0x7fe5800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:02.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.361+0000 7fe596ffd640 1 -- 172.21.15.78:0/1221438572 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fe5880cf840 con 0x7fe5981001b0 2024-08-18T23:42:02.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.362+0000 7fe59d5e6640 1 --2- 172.21.15.78:0/1221438572 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe5800a2b80 0x7fe5800a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe598102a50 tx=0x7fe58c0334d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:02.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.365+0000 7fe596ffd640 1 -- 172.21.15.78:0/1221438572 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe588014030 con 0x7fe5981001b0 2024-08-18T23:42:02.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.533+0000 7fe5a004b640 1 -- 172.21.15.78:0/1221438572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe598197210 con 0x7fe5981001b0 2024-08-18T23:42:02.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.534+0000 7fe596ffd640 1 -- 172.21.15.78:0/1221438572 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24126 (secure 0 0 0) 0x7fe58809c500 con 0x7fe5981001b0 2024-08-18T23:42:02.536 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:42:02.536 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":88,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","created":"2024-08-18T23:34:33.867246+0000","modified":"2024-08-18T23:41:41.811101+0000","last_up_change":"2024-08-18T23:40:42.200381+0000","last_in_change":"2024-08-18T23:40:22.425813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T23:37:22.338211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T23:41:28.920802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T23:41:31.026094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T23:41:37.403238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4de1c78-6ebc-49a9-b9c5-85d43535fab7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6803","nonce":3639645311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6805","nonce":3639645311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6809","nonce":3639645311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6807","nonce":3639645311}]},"public_addr":"172.21.15.78:6803/3639645311","cluster_addr":"172.21.15.78:6805/3639645311","heartbeat_back_addr":"172.21.15.78:6809/3639645311","heartbeat_front_addr":"172.21.15.78:6807/3639645311","state":["exists","up"]},{"osd":1,"uuid":"4f8e0c3d-472b-4230-ad23-2b0a34861251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6801","nonce":4199446368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6803","nonce":4199446368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6807","nonce":4199446368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6805","nonce":4199446368}]},"public_addr":"172.21.15.123:6801/4199446368","cluster_addr":"172.21.15.123:6803/4199446368","heartbeat_back_addr":"172.21.15.123:6807/4199446368","heartbeat_front_addr":"172.21.15.123:6805/4199446368","state":["exists","up"]},{"osd":2,"uuid":"27ef3314-bb6f-413d-be05-e10a105dec9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6801","nonce":2582810057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6803","nonce":2582810057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6807","nonce":2582810057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6805","nonce":2582810057}]},"public_addr":"172.21.15.161:6801/2582810057","cluster_addr":"172.21.15.161:6803/2582810057","heartbeat_back_addr":"172.21.15.161:6807/2582810057","heartbeat_front_addr":"172.21.15.161:6805/2582810057","state":["exists","up"]},{"osd":3,"uuid":"e9e52662-5ef9-4d70-bdd7-2b92165822ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6811","nonce":2697347521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6813","nonce":2697347521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6817","nonce":2697347521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6815","nonce":2697347521}]},"public_addr":"172.21.15.78:6811/2697347521","cluster_addr":"172.21.15.78:6813/2697347521","heartbeat_back_addr":"172.21.15.78:6817/2697347521","heartbeat_front_addr":"172.21.15.78:6815/2697347521","state":["exists","up"]},{"osd":4,"uuid":"5ed26775-79e3-4b24-9557-69fcf5e8e824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6809","nonce":1281511569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6811","nonce":1281511569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6815","nonce":1281511569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6813","nonce":1281511569}]},"public_addr":"172.21.15.123:6809/1281511569","cluster_addr":"172.21.15.123:6811/1281511569","heartbeat_back_addr":"172.21.15.123:6815/1281511569","heartbeat_front_addr":"172.21.15.123:6813/1281511569","state":["exists","up"]},{"osd":5,"uuid":"a9793b1d-5613-48a4-938e-adb718f4fa4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6809","nonce":2751796117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6811","nonce":2751796117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6815","nonce":2751796117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6813","nonce":2751796117}]},"public_addr":"172.21.15.161:6809/2751796117","cluster_addr":"172.21.15.161:6811/2751796117","heartbeat_back_addr":"172.21.15.161:6815/2751796117","heartbeat_front_addr":"172.21.15.161:6813/2751796117","state":["exists","up"]},{"osd":6,"uuid":"5dbc5e9c-a14f-4b71-910a-1e6d4827bb32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6819","nonce":95383223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6821","nonce":95383223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6825","nonce":95383223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6823","nonce":95383223}]},"public_addr":"172.21.15.78:6819/95383223","cluster_addr":"172.21.15.78:6821/95383223","heartbeat_back_addr":"172.21.15.78:6825/95383223","heartbeat_front_addr":"172.21.15.78:6823/95383223","state":["exists","up"]},{"osd":7,"uuid":"26fb7b69-131f-42ee-b745-6fe8b4f65391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6817","nonce":3440305156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6819","nonce":3440305156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6823","nonce":3440305156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6821","nonce":3440305156}]},"public_addr":"172.21.15.123:6817/3440305156","cluster_addr":"172.21.15.123:6819/3440305156","heartbeat_back_addr":"172.21.15.123:6823/3440305156","heartbeat_front_addr":"172.21.15.123:6821/3440305156","state":["exists","up"]},{"osd":8,"uuid":"e267bc50-0c29-4d7b-b368-b1621e29b2df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6817","nonce":1485008778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6819","nonce":1485008778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6823","nonce":1485008778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6821","nonce":1485008778}]},"public_addr":"172.21.15.161:6817/1485008778","cluster_addr":"172.21.15.161:6819/1485008778","heartbeat_back_addr":"172.21.15.161:6823/1485008778","heartbeat_front_addr":"172.21.15.161:6821/1485008778","state":["exists","up"]},{"osd":9,"uuid":"b7654818-b767-4fe6-8211-346297895f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6827","nonce":2884475661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6829","nonce":2884475661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6833","nonce":2884475661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6831","nonce":2884475661}]},"public_addr":"172.21.15.78:6827/2884475661","cluster_addr":"172.21.15.78:6829/2884475661","heartbeat_back_addr":"172.21.15.78:6833/2884475661","heartbeat_front_addr":"172.21.15.78:6831/2884475661","state":["exists","up"]},{"osd":10,"uuid":"59aa32ff-510e-4b52-8170-08a34a0b6b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6825","nonce":122226286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6827","nonce":122226286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6831","nonce":122226286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6829","nonce":122226286}]},"public_addr":"172.21.15.123:6825/122226286","cluster_addr":"172.21.15.123:6827/122226286","heartbeat_back_addr":"172.21.15.123:6831/122226286","heartbeat_front_addr":"172.21.15.123:6829/122226286","state":["exists","up"]},{"osd":11,"uuid":"67bacab4-d078-4c5c-b4ff-9f6a8a00f507","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6825","nonce":2037560985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6827","nonce":2037560985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6831","nonce":2037560985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6829","nonce":2037560985}]},"public_addr":"172.21.15.161:6825/2037560985","cluster_addr":"172.21.15.161:6827/2037560985","heartbeat_back_addr":"172.21.15.161:6831/2037560985","heartbeat_front_addr":"172.21.15.161:6829/2037560985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:36.826044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:57.688421+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:18.297424+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:40.355215+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:02.167902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:23.968759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:46.254083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:08.522038+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:31.028119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:54.119807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:16.847034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:39.607886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6800/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3040499240":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6801/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:6800/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/757409165":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/2562072965":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/636184437":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/1570708294":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3137806628":"2024-08-19T23:34:59.021165+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T23:42:02.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.536+0000 7fe5a004b640 1 -- 172.21.15.78:0/1221438572 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe5800a2b80 msgr2=0x7fe5800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.536+0000 7fe5a004b640 1 --2- 172.21.15.78:0/1221438572 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe5800a2b80 0x7fe5800a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe598102a50 tx=0x7fe58c0334d0 comp rx=0 tx=0).stop 2024-08-18T23:42:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.536+0000 7fe5a004b640 1 -- 172.21.15.78:0/1221438572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5981001b0 msgr2=0x7fe598195eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.536+0000 7fe5a004b640 1 --2- 172.21.15.78:0/1221438572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5981001b0 0x7fe598195eb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe58800cce0 tx=0x7fe58800a620 comp rx=0 tx=0).stop 2024-08-18T23:42:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.537+0000 7fe5a004b640 1 -- 172.21.15.78:0/1221438572 shutdown_connections 2024-08-18T23:42:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.537+0000 7fe5a004b640 1 --2- 172.21.15.78:0/1221438572 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe5800a2b80 0x7fe5800a5040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.537+0000 7fe5a004b640 1 --2- 172.21.15.78:0/1221438572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5981052e0 0x7fe59819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.537+0000 7fe5a004b640 1 --2- 172.21.15.78:0/1221438572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe598100b60 0x7fe5981963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.537+0000 7fe5a004b640 1 --2- 172.21.15.78:0/1221438572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5981001b0 0x7fe598195eb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.537+0000 7fe5a004b640 1 -- 172.21.15.78:0/1221438572 >> 172.21.15.78:0/1221438572 conn(0x7fe598076f40 msgr2=0x7fe598103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.537+0000 7fe5a004b640 1 -- 172.21.15.78:0/1221438572 shutdown_connections 2024-08-18T23:42:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:02.537+0000 7fe5a004b640 1 -- 172.21.15.78:0/1221438572 wait complete. 2024-08-18T23:42:03.269 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-08-18T23:42:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:03.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:03 smithi123 ceph-mon[21808]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-08-18T23:42:03.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:03 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1221438572' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:42:03.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:03 smithi078 ceph-mon[18440]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-08-18T23:42:03.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:03 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1221438572' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:42:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:03 smithi161 ceph-mon[22027]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-08-18T23:42:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:03 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1221438572' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:42:04.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.009+0000 7fc093637640 1 -- 172.21.15.78:0/4107437978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c1007b0 msgr2=0x7fc08c100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:04.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.009+0000 7fc093637640 1 --2- 172.21.15.78:0/4107437978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c1007b0 0x7fc08c100c10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc080009930 tx=0x7fc08002a420 comp rx=0 tx=0).stop 2024-08-18T23:42:04.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.009+0000 7fc093637640 1 -- 172.21.15.78:0/4107437978 shutdown_connections 2024-08-18T23:42:04.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.009+0000 7fc093637640 1 --2- 172.21.15.78:0/4107437978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc08c101150 0x7fc08c10d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:04.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.009+0000 7fc093637640 1 --2- 172.21.15.78:0/4107437978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c1007b0 0x7fc08c100c10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:04.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.009+0000 7fc093637640 1 --2- 172.21.15.78:0/4107437978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc08c1067b0 0x7fc08c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:04.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.009+0000 7fc093637640 1 -- 172.21.15.78:0/4107437978 >> 172.21.15.78:0/4107437978 conn(0x7fc08c0fc4d0 msgr2=0x7fc08c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:04.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.009+0000 7fc093637640 1 -- 172.21.15.78:0/4107437978 shutdown_connections 2024-08-18T23:42:04.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.009+0000 7fc093637640 1 -- 172.21.15.78:0/4107437978 wait complete. 2024-08-18T23:42:04.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.010+0000 7fc093637640 1 Processor -- start 2024-08-18T23:42:04.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.010+0000 7fc093637640 1 -- start start 2024-08-18T23:42:04.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.010+0000 7fc093637640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc08c1007b0 0x7fc08c1970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:04.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.010+0000 7fc093637640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c101150 0x7fc08c197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:04.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.010+0000 7fc093637640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc08c1067b0 0x7fc08c19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:04.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.010+0000 7fc093637640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c110bd0 con 0x7fc08c101150 2024-08-18T23:42:04.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.010+0000 7fc093637640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c110a50 con 0x7fc08c1067b0 2024-08-18T23:42:04.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.010+0000 7fc093637640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c110d50 con 0x7fc08c1007b0 2024-08-18T23:42:04.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.010+0000 7fc090bd2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c101150 0x7fc08c197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:04.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.010+0000 7fc090bd2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c101150 0x7fc08c197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35786/0 (socket says 172.21.15.78:35786) 2024-08-18T23:42:04.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.010+0000 7fc090bd2640 1 -- 172.21.15.78:0/3164464364 learned_addr learned my addr 172.21.15.78:0/3164464364 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:04.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.010+0000 7fc091bd4640 1 --2- 172.21.15.78:0/3164464364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc08c1067b0 0x7fc08c19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:04.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.010+0000 7fc0913d3640 1 --2- 172.21.15.78:0/3164464364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc08c1007b0 0x7fc08c1970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:04.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.010+0000 7fc090bd2640 1 -- 172.21.15.78:0/3164464364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc08c1007b0 msgr2=0x7fc08c1970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:04.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.010+0000 7fc090bd2640 1 --2- 172.21.15.78:0/3164464364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc08c1007b0 0x7fc08c1970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:04.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.010+0000 7fc090bd2640 1 -- 172.21.15.78:0/3164464364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc08c1067b0 msgr2=0x7fc08c19b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:04.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.010+0000 7fc090bd2640 1 --2- 172.21.15.78:0/3164464364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc08c1067b0 0x7fc08c19b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:04.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.010+0000 7fc090bd2640 1 -- 172.21.15.78:0/3164464364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08c19c0a0 con 0x7fc08c101150 2024-08-18T23:42:04.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.011+0000 7fc090bd2640 1 --2- 172.21.15.78:0/3164464364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c101150 0x7fc08c197630 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc080009230 tx=0x7fc080004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:04.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.011+0000 7fc08a7fc640 1 -- 172.21.15.78:0/3164464364 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08003f030 con 0x7fc08c101150 2024-08-18T23:42:04.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.011+0000 7fc08a7fc640 1 -- 172.21.15.78:0/3164464364 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc080003ad0 con 0x7fc08c101150 2024-08-18T23:42:04.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.011+0000 7fc093637640 1 -- 172.21.15.78:0/3164464364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08c19c330 con 0x7fc08c101150 2024-08-18T23:42:04.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.011+0000 7fc08a7fc640 1 -- 172.21.15.78:0/3164464364 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080035070 con 0x7fc08c101150 2024-08-18T23:42:04.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.011+0000 7fc093637640 1 -- 172.21.15.78:0/3164464364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08c1a3c70 con 0x7fc08c101150 2024-08-18T23:42:04.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.012+0000 7fc093637640 1 -- 172.21.15.78:0/3164464364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc08c101ef0 con 0x7fc08c101150 2024-08-18T23:42:04.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.013+0000 7fc08a7fc640 1 -- 172.21.15.78:0/3164464364 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc080042030 con 0x7fc08c101150 2024-08-18T23:42:04.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.014+0000 7fc08a7fc640 1 --2- 172.21.15.78:0/3164464364 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc0740a2b80 0x7fc0740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:04.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.014+0000 7fc0913d3640 1 --2- 172.21.15.78:0/3164464364 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc0740a2b80 0x7fc0740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:04.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.014+0000 7fc08a7fc640 1 -- 172.21.15.78:0/3164464364 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fc0800efbb0 con 0x7fc08c101150 2024-08-18T23:42:04.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.014+0000 7fc0913d3640 1 --2- 172.21.15.78:0/3164464364 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc0740a2b80 0x7fc0740a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc084005fd0 tx=0x7fc08400b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:04.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.017+0000 7fc08a7fc640 1 -- 172.21.15.78:0/3164464364 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc0800bc8d0 con 0x7fc08c101150 2024-08-18T23:42:04.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.225+0000 7fc093637640 1 -- 172.21.15.78:0/3164464364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc08c108d40 con 0x7fc08c101150 2024-08-18T23:42:04.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.234+0000 7fc08a7fc640 1 -- 172.21.15.78:0/3164464364 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fc0800bc1d0 con 0x7fc08c101150 2024-08-18T23:42:04.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.236+0000 7fc093637640 1 -- 172.21.15.78:0/3164464364 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc0740a2b80 msgr2=0x7fc0740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:04.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.236+0000 7fc093637640 1 --2- 172.21.15.78:0/3164464364 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc0740a2b80 0x7fc0740a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc084005fd0 tx=0x7fc08400b590 comp rx=0 tx=0).stop 2024-08-18T23:42:04.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.236+0000 7fc093637640 1 -- 172.21.15.78:0/3164464364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c101150 msgr2=0x7fc08c197630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:04.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.236+0000 7fc093637640 1 --2- 172.21.15.78:0/3164464364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c101150 0x7fc08c197630 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc080009230 tx=0x7fc080004270 comp rx=0 tx=0).stop 2024-08-18T23:42:04.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.236+0000 7fc093637640 1 -- 172.21.15.78:0/3164464364 shutdown_connections 2024-08-18T23:42:04.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.236+0000 7fc093637640 1 --2- 172.21.15.78:0/3164464364 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc0740a2b80 0x7fc0740a5040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:04.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.236+0000 7fc093637640 1 --2- 172.21.15.78:0/3164464364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc08c1067b0 0x7fc08c19b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:04.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.236+0000 7fc093637640 1 --2- 172.21.15.78:0/3164464364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc08c101150 0x7fc08c197630 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:04.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.236+0000 7fc093637640 1 --2- 172.21.15.78:0/3164464364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc08c1007b0 0x7fc08c1970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:04.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.236+0000 7fc093637640 1 -- 172.21.15.78:0/3164464364 >> 172.21.15.78:0/3164464364 conn(0x7fc08c0fc4d0 msgr2=0x7fc08c0fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:04.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.237+0000 7fc093637640 1 -- 172.21.15.78:0/3164464364 shutdown_connections 2024-08-18T23:42:04.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:04.237+0000 7fc093637640 1 -- 172.21.15.78:0/3164464364 wait complete. 2024-08-18T23:42:04.534 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:04 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3164464364' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-18T23:42:04.623 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:04 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3164464364' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-18T23:42:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:04 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3164464364' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-18T23:42:04.978 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-08-18T23:42:04.978 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-08-18T23:42:05.230 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:05.500 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:05 smithi078 ceph-mon[18440]: pgmap v279: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-08-18T23:42:05.500 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:05 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3164464364' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-18T23:42:05.500 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:05 smithi078 ceph-mon[18440]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-18T23:42:05.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:05 smithi123 ceph-mon[21808]: pgmap v279: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-08-18T23:42:05.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:05 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3164464364' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-18T23:42:05.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:05 smithi123 ceph-mon[21808]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-18T23:42:05.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:05 smithi161 ceph-mon[22027]: pgmap v279: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-08-18T23:42:05.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:05 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3164464364' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-18T23:42:05.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:05 smithi161 ceph-mon[22027]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-18T23:42:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.860+0000 7f056f745640 1 -- 172.21.15.78:0/1932626208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05681018b0 msgr2=0x7f0568101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.860+0000 7f056f745640 1 --2- 172.21.15.78:0/1932626208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05681018b0 0x7f0568101d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0558009930 tx=0x7f055802a420 comp rx=0 tx=0).stop 2024-08-18T23:42:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.860+0000 7f056f745640 1 -- 172.21.15.78:0/1932626208 shutdown_connections 2024-08-18T23:42:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.860+0000 7f056f745640 1 --2- 172.21.15.78:0/1932626208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0568102250 0x7f056810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.860+0000 7f056f745640 1 --2- 172.21.15.78:0/1932626208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05681018b0 0x7f0568101d10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.860+0000 7f056f745640 1 --2- 172.21.15.78:0/1932626208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05681078b0 0x7f0568107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.860+0000 7f056f745640 1 -- 172.21.15.78:0/1932626208 >> 172.21.15.78:0/1932626208 conn(0x7f05680fd5f0 msgr2=0x7f05680ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.860+0000 7f056f745640 1 -- 172.21.15.78:0/1932626208 shutdown_connections 2024-08-18T23:42:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.860+0000 7f056f745640 1 -- 172.21.15.78:0/1932626208 wait complete. 2024-08-18T23:42:05.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.861+0000 7f056f745640 1 Processor -- start 2024-08-18T23:42:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.861+0000 7f056f745640 1 -- start start 2024-08-18T23:42:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.861+0000 7f056f745640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05681018b0 0x7f05681981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.861+0000 7f056f745640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0568102250 0x7f0568198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.861+0000 7f056f745640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05681078b0 0x7f056819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.861+0000 7f056f745640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0568111c70 con 0x7f0568102250 2024-08-18T23:42:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.861+0000 7f056f745640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0568111af0 con 0x7f05681018b0 2024-08-18T23:42:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.861+0000 7f056f745640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0568111df0 con 0x7f05681078b0 2024-08-18T23:42:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.861+0000 7f056cce0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0568102250 0x7f0568198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.861+0000 7f056cce0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0568102250 0x7f0568198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45088/0 (socket says 172.21.15.78:45088) 2024-08-18T23:42:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.861+0000 7f056cce0640 1 -- 172.21.15.78:0/4136218275 learned_addr learned my addr 172.21.15.78:0/4136218275 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.861+0000 7f056dce2640 1 --2- 172.21.15.78:0/4136218275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05681078b0 0x7f056819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:05.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.861+0000 7f056cce0640 1 -- 172.21.15.78:0/4136218275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05681078b0 msgr2=0x7f056819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:05.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.861+0000 7f056cce0640 1 --2- 172.21.15.78:0/4136218275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05681078b0 0x7f056819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:05.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.861+0000 7f056cce0640 1 -- 172.21.15.78:0/4136218275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05681018b0 msgr2=0x7f05681981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:05.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.861+0000 7f056cce0640 1 --2- 172.21.15.78:0/4136218275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05681018b0 0x7f05681981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:05.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.861+0000 7f056cce0640 1 -- 172.21.15.78:0/4136218275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056819d180 con 0x7f0568102250 2024-08-18T23:42:05.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.861+0000 7f056dce2640 1 --2- 172.21.15.78:0/4136218275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05681078b0 0x7f056819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:42:05.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.862+0000 7f056cce0640 1 --2- 172.21.15.78:0/4136218275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0568102250 0x7f0568198710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0558009230 tx=0x7f0558004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:05.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.862+0000 7f055e7fc640 1 -- 172.21.15.78:0/4136218275 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055803f030 con 0x7f0568102250 2024-08-18T23:42:05.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.862+0000 7f055e7fc640 1 -- 172.21.15.78:0/4136218275 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0558003ad0 con 0x7f0568102250 2024-08-18T23:42:05.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.862+0000 7f056f745640 1 -- 172.21.15.78:0/4136218275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056819d410 con 0x7f0568102250 2024-08-18T23:42:05.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.862+0000 7f055e7fc640 1 -- 172.21.15.78:0/4136218275 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0558035070 con 0x7f0568102250 2024-08-18T23:42:05.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.862+0000 7f056f745640 1 -- 172.21.15.78:0/4136218275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05681a4d50 con 0x7f0568102250 2024-08-18T23:42:05.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.863+0000 7f056f745640 1 -- 172.21.15.78:0/4136218275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0568102ff0 con 0x7f0568102250 2024-08-18T23:42:05.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.864+0000 7f055e7fc640 1 -- 172.21.15.78:0/4136218275 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0558042030 con 0x7f0568102250 2024-08-18T23:42:05.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.864+0000 7f055e7fc640 1 --2- 172.21.15.78:0/4136218275 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f05540a2b80 0x7f05540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:05.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.865+0000 7f056d4e1640 1 --2- 172.21.15.78:0/4136218275 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f05540a2b80 0x7f05540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:05.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.865+0000 7f055e7fc640 1 -- 172.21.15.78:0/4136218275 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f05580ef800 con 0x7f0568102250 2024-08-18T23:42:05.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.865+0000 7f056d4e1640 1 --2- 172.21.15.78:0/4136218275 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f05540a2b80 0x7f05540a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0564005d80 tx=0x7f0564005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:05.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:05.869+0000 7f055e7fc640 1 -- 172.21.15.78:0/4136218275 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f05580bc520 con 0x7f0568102250 2024-08-18T23:42:06.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:06.072+0000 7f056f745640 1 -- 172.21.15.78:0/4136218275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f0568109e40 con 0x7f0568102250 2024-08-18T23:42:06.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:06.241+0000 7f055e7fc640 1 -- 172.21.15.78:0/4136218275 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f05580bbe20 con 0x7f0568102250 2024-08-18T23:42:06.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:06.243+0000 7f056f745640 1 -- 172.21.15.78:0/4136218275 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f05540a2b80 msgr2=0x7f05540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:06.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:06.243+0000 7f056f745640 1 --2- 172.21.15.78:0/4136218275 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f05540a2b80 0x7f05540a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0564005d80 tx=0x7f0564005c90 comp rx=0 tx=0).stop 2024-08-18T23:42:06.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:06.243+0000 7f056f745640 1 -- 172.21.15.78:0/4136218275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0568102250 msgr2=0x7f0568198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:06.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:06.243+0000 7f056f745640 1 --2- 172.21.15.78:0/4136218275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0568102250 0x7f0568198710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0558009230 tx=0x7f0558004270 comp rx=0 tx=0).stop 2024-08-18T23:42:06.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:06.243+0000 7f056f745640 1 -- 172.21.15.78:0/4136218275 shutdown_connections 2024-08-18T23:42:06.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:06.243+0000 7f056f745640 1 --2- 172.21.15.78:0/4136218275 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f05540a2b80 0x7f05540a5040 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:06.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:06.243+0000 7f056f745640 1 --2- 172.21.15.78:0/4136218275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05681078b0 0x7f056819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:06.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:06.243+0000 7f056f745640 1 --2- 172.21.15.78:0/4136218275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0568102250 0x7f0568198710 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:06.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:06.243+0000 7f056f745640 1 --2- 172.21.15.78:0/4136218275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05681018b0 0x7f05681981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:06.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:06.243+0000 7f056f745640 1 -- 172.21.15.78:0/4136218275 >> 172.21.15.78:0/4136218275 conn(0x7f05680fd5f0 msgr2=0x7f05680fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:06.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:06.243+0000 7f056f745640 1 -- 172.21.15.78:0/4136218275 shutdown_connections 2024-08-18T23:42:06.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:06.243+0000 7f056f745640 1 -- 172.21.15.78:0/4136218275 wait complete. 2024-08-18T23:42:06.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:06 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/4136218275' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-08-18T23:42:06.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:06 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/4136218275' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-08-18T23:42:06.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:06 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/4136218275' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-08-18T23:42:06.902 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-08-18T23:42:07.152 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:07.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:07 smithi123 ceph-mon[21808]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-18T23:42:07.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:07 smithi123 ceph-mon[21808]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-18T23:42:07.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:07 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/4136218275' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-18T23:42:07.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:07 smithi123 ceph-mon[21808]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-18T23:42:07.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:07 smithi123 ceph-mon[21808]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-18T23:42:07.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:07 smithi123 ceph-mon[21808]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-08-18T23:42:07.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:07 smithi123 ceph-mon[21808]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-08-18T23:42:07.553 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:07 smithi078 ceph-mon[18440]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-18T23:42:07.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:07 smithi078 ceph-mon[18440]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-18T23:42:07.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:07 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/4136218275' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-18T23:42:07.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:07 smithi078 ceph-mon[18440]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-18T23:42:07.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:07 smithi078 ceph-mon[18440]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-18T23:42:07.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:07 smithi078 ceph-mon[18440]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-08-18T23:42:07.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:07 smithi078 ceph-mon[18440]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-08-18T23:42:07.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.634+0000 7f77433ad640 1 -- 172.21.15.78:0/4225366759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f773c102250 msgr2=0x7f773c10e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:07.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.634+0000 7f77433ad640 1 --2- 172.21.15.78:0/4225366759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f773c102250 0x7f773c10e780 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7734009930 tx=0x7f773402a420 comp rx=0 tx=0).stop 2024-08-18T23:42:07.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.634+0000 7f77433ad640 1 -- 172.21.15.78:0/4225366759 shutdown_connections 2024-08-18T23:42:07.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.634+0000 7f77433ad640 1 --2- 172.21.15.78:0/4225366759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f773c102250 0x7f773c10e780 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:07.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.634+0000 7f77433ad640 1 --2- 172.21.15.78:0/4225366759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f773c1018b0 0x7f773c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:07.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.634+0000 7f77433ad640 1 --2- 172.21.15.78:0/4225366759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f773c1078b0 0x7f773c107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:07.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.634+0000 7f77433ad640 1 -- 172.21.15.78:0/4225366759 >> 172.21.15.78:0/4225366759 conn(0x7f773c0fd5f0 msgr2=0x7f773c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:07.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.634+0000 7f77433ad640 1 -- 172.21.15.78:0/4225366759 shutdown_connections 2024-08-18T23:42:07.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.634+0000 7f77433ad640 1 -- 172.21.15.78:0/4225366759 wait complete. 2024-08-18T23:42:07.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.635+0000 7f77433ad640 1 Processor -- start 2024-08-18T23:42:07.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.635+0000 7f77433ad640 1 -- start start 2024-08-18T23:42:07.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.635+0000 7f77433ad640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f773c1018b0 0x7f773c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:07.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.635+0000 7f77433ad640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f773c102250 0x7f773c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:07.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.635+0000 7f77433ad640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f773c1078b0 0x7f773c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:07.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.635+0000 7f77433ad640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f773c111cd0 con 0x7f773c102250 2024-08-18T23:42:07.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.635+0000 7f77433ad640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f773c111b50 con 0x7f773c1078b0 2024-08-18T23:42:07.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.635+0000 7f77433ad640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f773c111e50 con 0x7f773c1018b0 2024-08-18T23:42:07.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.635+0000 7f7740948640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f773c102250 0x7f773c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:07.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.635+0000 7f7740948640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f773c102250 0x7f773c198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45104/0 (socket says 172.21.15.78:45104) 2024-08-18T23:42:07.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.635+0000 7f7740948640 1 -- 172.21.15.78:0/2693228929 learned_addr learned my addr 172.21.15.78:0/2693228929 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:07.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.636+0000 7f7741149640 1 --2- 172.21.15.78:0/2693228929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f773c1018b0 0x7f773c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:07.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.636+0000 7f774194a640 1 --2- 172.21.15.78:0/2693228929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f773c1078b0 0x7f773c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:07.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.636+0000 7f7740948640 1 -- 172.21.15.78:0/2693228929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f773c1018b0 msgr2=0x7f773c198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:07.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.636+0000 7f7740948640 1 --2- 172.21.15.78:0/2693228929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f773c1018b0 0x7f773c198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:07.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.636+0000 7f7740948640 1 -- 172.21.15.78:0/2693228929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f773c1078b0 msgr2=0x7f773c19c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:07.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.636+0000 7f7740948640 1 --2- 172.21.15.78:0/2693228929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f773c1078b0 0x7f773c19c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:07.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.636+0000 7f7740948640 1 -- 172.21.15.78:0/2693228929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773c19d0c0 con 0x7f773c102250 2024-08-18T23:42:07.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.636+0000 7f774194a640 1 --2- 172.21.15.78:0/2693228929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f773c1078b0 0x7f773c19c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:42:07.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.636+0000 7f7740948640 1 --2- 172.21.15.78:0/2693228929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f773c102250 0x7f773c198650 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f772c00c920 tx=0x7f772c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:07.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.636+0000 7f77327fc640 1 -- 172.21.15.78:0/2693228929 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c007c70 con 0x7f773c102250 2024-08-18T23:42:07.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.636+0000 7f77433ad640 1 -- 172.21.15.78:0/2693228929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f773c19d3b0 con 0x7f773c102250 2024-08-18T23:42:07.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.636+0000 7f77327fc640 1 -- 172.21.15.78:0/2693228929 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f772c007e10 con 0x7f773c102250 2024-08-18T23:42:07.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.636+0000 7f77327fc640 1 -- 172.21.15.78:0/2693228929 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c018890 con 0x7f773c102250 2024-08-18T23:42:07.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.636+0000 7f77433ad640 1 -- 172.21.15.78:0/2693228929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f773c1a4c90 con 0x7f773c102250 2024-08-18T23:42:07.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.637+0000 7f77433ad640 1 -- 172.21.15.78:0/2693228929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f773c101d10 con 0x7f773c102250 2024-08-18T23:42:07.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.638+0000 7f77327fc640 1 -- 172.21.15.78:0/2693228929 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f772c018a70 con 0x7f773c102250 2024-08-18T23:42:07.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.639+0000 7f77327fc640 1 --2- 172.21.15.78:0/2693228929 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f77280a2b80 0x7f77280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:07.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.639+0000 7f7741149640 1 --2- 172.21.15.78:0/2693228929 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f77280a2b80 0x7f77280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:07.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.639+0000 7f77327fc640 1 -- 172.21.15.78:0/2693228929 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f772c0cf480 con 0x7f773c102250 2024-08-18T23:42:07.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.639+0000 7f7741149640 1 --2- 172.21.15.78:0/2693228929 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f77280a2b80 0x7f77280a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7738009f20 tx=0x7f77380094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:07.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.643+0000 7f77327fc640 1 -- 172.21.15.78:0/2693228929 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f772c014030 con 0x7f773c102250 2024-08-18T23:42:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:07 smithi161 ceph-mon[22027]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-18T23:42:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:07 smithi161 ceph-mon[22027]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-18T23:42:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:07 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/4136218275' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-18T23:42:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:07 smithi161 ceph-mon[22027]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-18T23:42:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:07 smithi161 ceph-mon[22027]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-18T23:42:07.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:07 smithi161 ceph-mon[22027]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-08-18T23:42:07.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:07 smithi161 ceph-mon[22027]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-08-18T23:42:07.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:07.844+0000 7f77433ad640 1 -- 172.21.15.78:0/2693228929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f773c1a5020 con 0x7f773c102250 2024-08-18T23:42:08.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:08.257+0000 7f77327fc640 1 -- 172.21.15.78:0/2693228929 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f772c00de20 con 0x7f773c102250 2024-08-18T23:42:08.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:08.259+0000 7f77433ad640 1 -- 172.21.15.78:0/2693228929 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f77280a2b80 msgr2=0x7f77280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:08.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:08.259+0000 7f77433ad640 1 --2- 172.21.15.78:0/2693228929 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f77280a2b80 0x7f77280a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7738009f20 tx=0x7f77380094f0 comp rx=0 tx=0).stop 2024-08-18T23:42:08.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:08.259+0000 7f77433ad640 1 -- 172.21.15.78:0/2693228929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f773c102250 msgr2=0x7f773c198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:08.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:08.259+0000 7f77433ad640 1 --2- 172.21.15.78:0/2693228929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f773c102250 0x7f773c198650 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f772c00c920 tx=0x7f772c00cde0 comp rx=0 tx=0).stop 2024-08-18T23:42:08.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:08.259+0000 7f77433ad640 1 -- 172.21.15.78:0/2693228929 shutdown_connections 2024-08-18T23:42:08.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:08.259+0000 7f77433ad640 1 --2- 172.21.15.78:0/2693228929 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f77280a2b80 0x7f77280a5040 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:08.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:08.259+0000 7f77433ad640 1 --2- 172.21.15.78:0/2693228929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f773c1078b0 0x7f773c19c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:08.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:08.259+0000 7f77433ad640 1 --2- 172.21.15.78:0/2693228929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f773c102250 0x7f773c198650 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:08.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:08.259+0000 7f77433ad640 1 --2- 172.21.15.78:0/2693228929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f773c1018b0 0x7f773c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:08.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:08.259+0000 7f77433ad640 1 -- 172.21.15.78:0/2693228929 >> 172.21.15.78:0/2693228929 conn(0x7f773c0fd5f0 msgr2=0x7f773c0fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:08.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:08.259+0000 7f77433ad640 1 -- 172.21.15.78:0/2693228929 shutdown_connections 2024-08-18T23:42:08.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:08.260+0000 7f77433ad640 1 -- 172.21.15.78:0/2693228929 wait complete. 2024-08-18T23:42:08.547 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:42:08 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-b[51547]: 2024-08-18T23:42:08.167+0000 7f7b3120b640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-08-18T23:42:08.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:08 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] up:active 2024-08-18T23:42:08.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:08 smithi123 ceph-mon[21808]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-18T23:42:08.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:08 smithi123 ceph-mon[21808]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-18T23:42:08.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:08 smithi123 ceph-mon[21808]: Cluster is now healthy 2024-08-18T23:42:08.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:08 smithi123 ceph-mon[21808]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-08-18T23:42:08.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:08 smithi123 ceph-mon[21808]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-08-18T23:42:08.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:08 smithi123 ceph-mon[21808]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-08-18T23:42:08.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:08 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2693228929' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-18T23:42:08.635 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:08 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] up:active 2024-08-18T23:42:08.635 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:08 smithi078 ceph-mon[18440]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-18T23:42:08.635 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:08 smithi078 ceph-mon[18440]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-18T23:42:08.635 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:08 smithi078 ceph-mon[18440]: Cluster is now healthy 2024-08-18T23:42:08.635 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:08 smithi078 ceph-mon[18440]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-08-18T23:42:08.635 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:08 smithi078 ceph-mon[18440]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-08-18T23:42:08.635 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:08 smithi078 ceph-mon[18440]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-08-18T23:42:08.635 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:08 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2693228929' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-18T23:42:08.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:08 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] up:active 2024-08-18T23:42:08.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:08 smithi161 ceph-mon[22027]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-18T23:42:08.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:08 smithi161 ceph-mon[22027]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-18T23:42:08.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:08 smithi161 ceph-mon[22027]: Cluster is now healthy 2024-08-18T23:42:08.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:08 smithi161 ceph-mon[22027]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-08-18T23:42:08.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:08 smithi161 ceph-mon[22027]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-08-18T23:42:08.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:08 smithi161 ceph-mon[22027]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-08-18T23:42:08.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:08 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2693228929' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-18T23:42:09.003 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-08-18T23:42:09.275 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:09.538 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:09 smithi078 ceph-mon[18440]: pgmap v281: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-08-18T23:42:09.538 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:09 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2693228929' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-18T23:42:09.538 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:09 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] up:active 2024-08-18T23:42:09.538 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:09 smithi078 ceph-mon[18440]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-08-18T23:42:09.538 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:09 smithi078 ceph-mon[18440]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:42:09.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:09 smithi123 ceph-mon[21808]: pgmap v281: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-08-18T23:42:09.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:09 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2693228929' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-18T23:42:09.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:09 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] up:active 2024-08-18T23:42:09.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:09 smithi123 ceph-mon[21808]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-08-18T23:42:09.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:09 smithi123 ceph-mon[21808]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:42:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:09 smithi161 ceph-mon[22027]: pgmap v281: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-08-18T23:42:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:09 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2693228929' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-18T23:42:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:09 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] up:active 2024-08-18T23:42:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:09 smithi161 ceph-mon[22027]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-08-18T23:42:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:09 smithi161 ceph-mon[22027]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:42:09.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.760+0000 7f9606775640 1 -- 172.21.15.78:0/2985596086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96001078b0 msgr2=0x7f9600107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:09.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.760+0000 7f9606775640 1 --2- 172.21.15.78:0/2985596086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96001078b0 0x7f9600107c90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009930 tx=0x7f95f002a420 comp rx=0 tx=0).stop 2024-08-18T23:42:09.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.760+0000 7f9606775640 1 -- 172.21.15.78:0/2985596086 shutdown_connections 2024-08-18T23:42:09.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.760+0000 7f9606775640 1 --2- 172.21.15.78:0/2985596086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9600102250 0x7f960010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:09.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.760+0000 7f9606775640 1 --2- 172.21.15.78:0/2985596086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96001018b0 0x7f9600101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:09.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.760+0000 7f9606775640 1 --2- 172.21.15.78:0/2985596086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96001078b0 0x7f9600107c90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:09.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.760+0000 7f9606775640 1 -- 172.21.15.78:0/2985596086 >> 172.21.15.78:0/2985596086 conn(0x7f96000fd5f0 msgr2=0x7f96000ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:09.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.760+0000 7f9606775640 1 -- 172.21.15.78:0/2985596086 shutdown_connections 2024-08-18T23:42:09.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.760+0000 7f9606775640 1 -- 172.21.15.78:0/2985596086 wait complete. 2024-08-18T23:42:09.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.761+0000 7f9606775640 1 Processor -- start 2024-08-18T23:42:09.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.761+0000 7f9606775640 1 -- start start 2024-08-18T23:42:09.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.761+0000 7f9606775640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96001018b0 0x7f9600198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:09.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.761+0000 7f9606775640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9600102250 0x7f9600198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:09.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.761+0000 7f9606775640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96001078b0 0x7f960019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:09.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.761+0000 7f9606775640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9600111d50 con 0x7f9600102250 2024-08-18T23:42:09.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.761+0000 7f9606775640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9600111bd0 con 0x7f96001018b0 2024-08-18T23:42:09.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.761+0000 7f9606775640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9600111ed0 con 0x7f96001078b0 2024-08-18T23:42:09.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.762+0000 7f95ff7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9600102250 0x7f9600198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:09.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.762+0000 7f95ff7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9600102250 0x7f9600198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45122/0 (socket says 172.21.15.78:45122) 2024-08-18T23:42:09.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.762+0000 7f95ff7fe640 1 -- 172.21.15.78:0/514202435 learned_addr learned my addr 172.21.15.78:0/514202435 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:09.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.762+0000 7f95fffff640 1 --2- 172.21.15.78:0/514202435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96001018b0 0x7f9600198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:09.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.762+0000 7f9604d12640 1 --2- 172.21.15.78:0/514202435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96001078b0 0x7f960019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:09.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.762+0000 7f95ff7fe640 1 -- 172.21.15.78:0/514202435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96001078b0 msgr2=0x7f960019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:09.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.762+0000 7f95ff7fe640 1 --2- 172.21.15.78:0/514202435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96001078b0 0x7f960019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:09.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.762+0000 7f95ff7fe640 1 -- 172.21.15.78:0/514202435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96001018b0 msgr2=0x7f9600198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:09.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.762+0000 7f95ff7fe640 1 --2- 172.21.15.78:0/514202435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96001018b0 0x7f9600198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:09.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.762+0000 7f95ff7fe640 1 -- 172.21.15.78:0/514202435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960019d2c0 con 0x7f9600102250 2024-08-18T23:42:09.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.762+0000 7f95ff7fe640 1 --2- 172.21.15.78:0/514202435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9600102250 0x7f9600198850 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f95f400b4a0 tx=0x7f95f400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:09.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.762+0000 7f95fd7fa640 1 -- 172.21.15.78:0/514202435 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4014070 con 0x7f9600102250 2024-08-18T23:42:09.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.762+0000 7f9606775640 1 -- 172.21.15.78:0/514202435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960019d5b0 con 0x7f9600102250 2024-08-18T23:42:09.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.762+0000 7f95fd7fa640 1 -- 172.21.15.78:0/514202435 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95f4010af0 con 0x7f9600102250 2024-08-18T23:42:09.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.762+0000 7f95fd7fa640 1 -- 172.21.15.78:0/514202435 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4017d10 con 0x7f9600102250 2024-08-18T23:42:09.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.762+0000 7f9606775640 1 -- 172.21.15.78:0/514202435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96001a4e90 con 0x7f9600102250 2024-08-18T23:42:09.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.763+0000 7f9606775640 1 -- 172.21.15.78:0/514202435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9600101d90 con 0x7f9600102250 2024-08-18T23:42:09.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.764+0000 7f95fd7fa640 1 -- 172.21.15.78:0/514202435 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f95f4017070 con 0x7f9600102250 2024-08-18T23:42:09.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.765+0000 7f95fd7fa640 1 --2- 172.21.15.78:0/514202435 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f95e80a2b80 0x7f95e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:09.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.765+0000 7f95fffff640 1 --2- 172.21.15.78:0/514202435 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f95e80a2b80 0x7f95e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:09.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.765+0000 7f95fd7fa640 1 -- 172.21.15.78:0/514202435 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f95f40ce5a0 con 0x7f9600102250 2024-08-18T23:42:09.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.765+0000 7f95fffff640 1 --2- 172.21.15.78:0/514202435 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f95e80a2b80 0x7f95e80a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009900 tx=0x7f95f00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:09.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.769+0000 7f95fd7fa640 1 -- 172.21.15.78:0/514202435 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f95f409ac70 con 0x7f9600102250 2024-08-18T23:42:09.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:09.973+0000 7f9606775640 1 -- 172.21.15.78:0/514202435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f960002d5e0 con 0x7f9600102250 2024-08-18T23:42:10.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:10.263+0000 7f95fd7fa640 1 -- 172.21.15.78:0/514202435 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7f95f409ef70 con 0x7f9600102250 2024-08-18T23:42:10.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:10.266+0000 7f9606775640 1 -- 172.21.15.78:0/514202435 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f95e80a2b80 msgr2=0x7f95e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:10.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:10.266+0000 7f9606775640 1 --2- 172.21.15.78:0/514202435 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f95e80a2b80 0x7f95e80a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009900 tx=0x7f95f00058d0 comp rx=0 tx=0).stop 2024-08-18T23:42:10.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:10.266+0000 7f9606775640 1 -- 172.21.15.78:0/514202435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9600102250 msgr2=0x7f9600198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:10.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:10.266+0000 7f9606775640 1 --2- 172.21.15.78:0/514202435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9600102250 0x7f9600198850 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f95f400b4a0 tx=0x7f95f400b960 comp rx=0 tx=0).stop 2024-08-18T23:42:10.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:10.266+0000 7f9606775640 1 -- 172.21.15.78:0/514202435 shutdown_connections 2024-08-18T23:42:10.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:10.266+0000 7f9606775640 1 --2- 172.21.15.78:0/514202435 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f95e80a2b80 0x7f95e80a5040 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:10.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:10.266+0000 7f9606775640 1 --2- 172.21.15.78:0/514202435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96001078b0 0x7f960019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:10.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:10.266+0000 7f9606775640 1 --2- 172.21.15.78:0/514202435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9600102250 0x7f9600198850 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:10.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:10.266+0000 7f9606775640 1 --2- 172.21.15.78:0/514202435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96001018b0 0x7f9600198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:10.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:10.266+0000 7f9606775640 1 -- 172.21.15.78:0/514202435 >> 172.21.15.78:0/514202435 conn(0x7f96000fd5f0 msgr2=0x7f96000fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:10.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:10.266+0000 7f9606775640 1 -- 172.21.15.78:0/514202435 shutdown_connections 2024-08-18T23:42:10.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:10.266+0000 7f9606775640 1 -- 172.21.15.78:0/514202435 wait complete. 2024-08-18T23:42:10.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:10 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/514202435' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-08-18T23:42:10.643 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:10 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/514202435' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-08-18T23:42:10.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:10 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/514202435' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-08-18T23:42:10.988 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-08-18T23:42:10.988 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-08-18T23:42:11.241 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:11.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:11 smithi078 ceph-mon[18440]: pgmap v282: 129 pgs: 129 active+clean; 583 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-08-18T23:42:11.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:11 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/514202435' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-18T23:42:11.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:11 smithi078 ceph-mon[18440]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:42:11.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:11 smithi123 ceph-mon[21808]: pgmap v282: 129 pgs: 129 active+clean; 583 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-08-18T23:42:11.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:11 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/514202435' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-18T23:42:11.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:11 smithi123 ceph-mon[21808]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:42:11.578 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:11 smithi161 ceph-mon[22027]: pgmap v282: 129 pgs: 129 active+clean; 583 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-08-18T23:42:11.578 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:11 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/514202435' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-18T23:42:11.579 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:11 smithi161 ceph-mon[22027]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:42:11.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.713+0000 7fc8abd1f640 1 -- 172.21.15.78:0/3671533018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a41007b0 msgr2=0x7fc8a4100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:11.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.713+0000 7fc8abd1f640 1 --2- 172.21.15.78:0/3671533018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a41007b0 0x7fc8a4100c10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc894009930 tx=0x7fc89402a420 comp rx=0 tx=0).stop 2024-08-18T23:42:11.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.713+0000 7fc8abd1f640 1 -- 172.21.15.78:0/3671533018 shutdown_connections 2024-08-18T23:42:11.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.713+0000 7fc8abd1f640 1 --2- 172.21.15.78:0/3671533018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8a4101150 0x7fc8a410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:11.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.713+0000 7fc8abd1f640 1 --2- 172.21.15.78:0/3671533018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a41007b0 0x7fc8a4100c10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:11.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.713+0000 7fc8abd1f640 1 --2- 172.21.15.78:0/3671533018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8a41067b0 0x7fc8a4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:11.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.714+0000 7fc8abd1f640 1 -- 172.21.15.78:0/3671533018 >> 172.21.15.78:0/3671533018 conn(0x7fc8a40fc4d0 msgr2=0x7fc8a40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:11.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.714+0000 7fc8abd1f640 1 -- 172.21.15.78:0/3671533018 shutdown_connections 2024-08-18T23:42:11.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.714+0000 7fc8abd1f640 1 -- 172.21.15.78:0/3671533018 wait complete. 2024-08-18T23:42:11.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.714+0000 7fc8abd1f640 1 Processor -- start 2024-08-18T23:42:11.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.714+0000 7fc8abd1f640 1 -- start start 2024-08-18T23:42:11.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.714+0000 7fc8abd1f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8a41007b0 0x7fc8a4196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:11.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.714+0000 7fc8abd1f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a4101150 0x7fc8a4197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:11.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.714+0000 7fc8abd1f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8a41067b0 0x7fc8a419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:11.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.714+0000 7fc8abd1f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4110bb0 con 0x7fc8a4101150 2024-08-18T23:42:11.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.714+0000 7fc8abd1f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4110a30 con 0x7fc8a41067b0 2024-08-18T23:42:11.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.714+0000 7fc8abd1f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4110d30 con 0x7fc8a41007b0 2024-08-18T23:42:11.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.714+0000 7fc8a92ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a4101150 0x7fc8a4197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:11.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.715+0000 7fc8a92ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a4101150 0x7fc8a4197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45146/0 (socket says 172.21.15.78:45146) 2024-08-18T23:42:11.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.715+0000 7fc8a92ba640 1 -- 172.21.15.78:0/1559836517 learned_addr learned my addr 172.21.15.78:0/1559836517 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:11.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.715+0000 7fc8a9abb640 1 --2- 172.21.15.78:0/1559836517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8a41007b0 0x7fc8a4196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:11.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.715+0000 7fc8aa2bc640 1 --2- 172.21.15.78:0/1559836517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8a41067b0 0x7fc8a419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:11.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.715+0000 7fc8a92ba640 1 -- 172.21.15.78:0/1559836517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8a41007b0 msgr2=0x7fc8a4196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:11.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.715+0000 7fc8a92ba640 1 --2- 172.21.15.78:0/1559836517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8a41007b0 0x7fc8a4196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:11.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.715+0000 7fc8a92ba640 1 -- 172.21.15.78:0/1559836517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8a41067b0 msgr2=0x7fc8a419b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:11.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.715+0000 7fc8a92ba640 1 --2- 172.21.15.78:0/1559836517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8a41067b0 0x7fc8a419b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:11.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.715+0000 7fc8a92ba640 1 -- 172.21.15.78:0/1559836517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a419bfa0 con 0x7fc8a4101150 2024-08-18T23:42:11.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.715+0000 7fc8aa2bc640 1 --2- 172.21.15.78:0/1559836517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8a41067b0 0x7fc8a419b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:42:11.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.715+0000 7fc8a92ba640 1 --2- 172.21.15.78:0/1559836517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a4101150 0x7fc8a4197530 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc894009230 tx=0x7fc894004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:11.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.715+0000 7fc89affd640 1 -- 172.21.15.78:0/1559836517 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894046070 con 0x7fc8a4101150 2024-08-18T23:42:11.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.715+0000 7fc89affd640 1 -- 172.21.15.78:0/1559836517 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc894003ad0 con 0x7fc8a4101150 2024-08-18T23:42:11.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.715+0000 7fc8abd1f640 1 -- 172.21.15.78:0/1559836517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a419c230 con 0x7fc8a4101150 2024-08-18T23:42:11.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.715+0000 7fc89affd640 1 -- 172.21.15.78:0/1559836517 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894035050 con 0x7fc8a4101150 2024-08-18T23:42:11.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.715+0000 7fc8abd1f640 1 -- 172.21.15.78:0/1559836517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a406a4c0 con 0x7fc8a4101150 2024-08-18T23:42:11.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.716+0000 7fc8abd1f640 1 -- 172.21.15.78:0/1559836517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a4101ef0 con 0x7fc8a4101150 2024-08-18T23:42:11.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.718+0000 7fc89affd640 1 -- 172.21.15.78:0/1559836517 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc894049070 con 0x7fc8a4101150 2024-08-18T23:42:11.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.718+0000 7fc89affd640 1 --2- 172.21.15.78:0/1559836517 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc8900a2b80 0x7fc8900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:11.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.718+0000 7fc8a9abb640 1 --2- 172.21.15.78:0/1559836517 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc8900a2b80 0x7fc8900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:11.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.718+0000 7fc89affd640 1 -- 172.21.15.78:0/1559836517 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fc8940f0f60 con 0x7fc8a4101150 2024-08-18T23:42:11.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.719+0000 7fc8a9abb640 1 --2- 172.21.15.78:0/1559836517 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc8900a2b80 0x7fc8900a5040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0007e40 tx=0x7fc8a00075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:11.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.721+0000 7fc89affd640 1 -- 172.21.15.78:0/1559836517 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc894050080 con 0x7fc8a4101150 2024-08-18T23:42:11.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.911+0000 7fc8abd1f640 1 -- 172.21.15.78:0/1559836517 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc8a40fd240 con 0x7fc8900a2b80 2024-08-18T23:42:11.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.939+0000 7fc89affd640 1 -- 172.21.15.78:0/1559836517 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc8a40fd240 con 0x7fc8900a2b80 2024-08-18T23:42:11.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.941+0000 7fc8abd1f640 1 -- 172.21.15.78:0/1559836517 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc8900a2b80 msgr2=0x7fc8900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:11.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.941+0000 7fc8abd1f640 1 --2- 172.21.15.78:0/1559836517 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc8900a2b80 0x7fc8900a5040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0007e40 tx=0x7fc8a00075b0 comp rx=0 tx=0).stop 2024-08-18T23:42:11.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.941+0000 7fc8abd1f640 1 -- 172.21.15.78:0/1559836517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a4101150 msgr2=0x7fc8a4197530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:11.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.941+0000 7fc8abd1f640 1 --2- 172.21.15.78:0/1559836517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a4101150 0x7fc8a4197530 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc894009230 tx=0x7fc894004270 comp rx=0 tx=0).stop 2024-08-18T23:42:11.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.941+0000 7fc8abd1f640 1 -- 172.21.15.78:0/1559836517 shutdown_connections 2024-08-18T23:42:11.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.941+0000 7fc8abd1f640 1 --2- 172.21.15.78:0/1559836517 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc8900a2b80 0x7fc8900a5040 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:11.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.941+0000 7fc8abd1f640 1 --2- 172.21.15.78:0/1559836517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8a41067b0 0x7fc8a419b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:11.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.941+0000 7fc8abd1f640 1 --2- 172.21.15.78:0/1559836517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8a4101150 0x7fc8a4197530 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:11.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.941+0000 7fc8abd1f640 1 --2- 172.21.15.78:0/1559836517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8a41007b0 0x7fc8a4196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:11.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.941+0000 7fc8abd1f640 1 -- 172.21.15.78:0/1559836517 >> 172.21.15.78:0/1559836517 conn(0x7fc8a40fc4d0 msgr2=0x7fc8a40fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:11.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.941+0000 7fc8abd1f640 1 -- 172.21.15.78:0/1559836517 shutdown_connections 2024-08-18T23:42:11.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:11.941+0000 7fc8abd1f640 1 -- 172.21.15.78:0/1559836517 wait complete. 2024-08-18T23:42:12.219 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:42:11 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:42:11.916+0000 7f3a6e368640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-18T23:42:12.219 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:42:11 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:42:11.916+0000 7f3a6e368640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-18T23:42:12.220 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:42:11 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:42:11.916+0000 7f3a6e368640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-18T23:42:12.220 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:42:11 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:42:11.916+0000 7f3a6e368640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-18T23:42:12.220 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:42:11 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:42:11.916+0000 7f3a6e368640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-18T23:42:12.655 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-08-18T23:42:12.905 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:13.338 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:13 smithi078 ceph-mon[18440]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:42:13.339 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:13 smithi078 ceph-mon[18440]: pgmap v283: 129 pgs: 129 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-08-18T23:42:13.339 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:13 smithi078 ceph-mon[18440]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-08-18T23:42:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.424+0000 7fdab93c7640 1 -- 172.21.15.78:0/2805630606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdab4108010 msgr2=0x7fdab410e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.424+0000 7fdab93c7640 1 --2- 172.21.15.78:0/2805630606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdab4108010 0x7fdab410e980 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009cd0 tx=0x7fdaac02a910 comp rx=0 tx=0).stop 2024-08-18T23:42:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.424+0000 7fdab93c7640 1 -- 172.21.15.78:0/2805630606 shutdown_connections 2024-08-18T23:42:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.424+0000 7fdab93c7640 1 --2- 172.21.15.78:0/2805630606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdab4108010 0x7fdab410e980 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.424+0000 7fdab93c7640 1 --2- 172.21.15.78:0/2805630606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab4103490 0x7fdab41078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.424+0000 7fdab93c7640 1 --2- 172.21.15.78:0/2805630606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab4102ae0 0x7fdab4102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.424+0000 7fdab93c7640 1 -- 172.21.15.78:0/2805630606 >> 172.21.15.78:0/2805630606 conn(0x7fdab40fc500 msgr2=0x7fdab40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.424+0000 7fdab93c7640 1 -- 172.21.15.78:0/2805630606 shutdown_connections 2024-08-18T23:42:13.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.424+0000 7fdab93c7640 1 -- 172.21.15.78:0/2805630606 wait complete. 2024-08-18T23:42:13.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.424+0000 7fdab93c7640 1 Processor -- start 2024-08-18T23:42:13.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.425+0000 7fdab93c7640 1 -- start start 2024-08-18T23:42:13.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.425+0000 7fdab93c7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdab4102ae0 0x7fdab4100740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:13.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.425+0000 7fdab93c7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab4103490 0x7fdab4100c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:13.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.425+0000 7fdab93c7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab4108010 0x7fdab41011c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:13.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.425+0000 7fdab93c7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdab4111ea0 con 0x7fdab4102ae0 2024-08-18T23:42:13.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.425+0000 7fdab93c7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdab4111d20 con 0x7fdab4108010 2024-08-18T23:42:13.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.425+0000 7fdab93c7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdab4112020 con 0x7fdab4103490 2024-08-18T23:42:13.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.425+0000 7fdab2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdab4102ae0 0x7fdab4100740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:13.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.425+0000 7fdab2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdab4102ae0 0x7fdab4100740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45192/0 (socket says 172.21.15.78:45192) 2024-08-18T23:42:13.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.425+0000 7fdab2ffd640 1 -- 172.21.15.78:0/3640452471 learned_addr learned my addr 172.21.15.78:0/3640452471 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:13.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.425+0000 7fdab37fe640 1 --2- 172.21.15.78:0/3640452471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab4108010 0x7fdab41011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:13.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.425+0000 7fdaabfff640 1 --2- 172.21.15.78:0/3640452471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab4103490 0x7fdab4100c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:13.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.425+0000 7fdab2ffd640 1 -- 172.21.15.78:0/3640452471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab4103490 msgr2=0x7fdab4100c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:13.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.425+0000 7fdab2ffd640 1 --2- 172.21.15.78:0/3640452471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab4103490 0x7fdab4100c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:13.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.425+0000 7fdab2ffd640 1 -- 172.21.15.78:0/3640452471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab4108010 msgr2=0x7fdab41011c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:13.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.425+0000 7fdab2ffd640 1 --2- 172.21.15.78:0/3640452471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab4108010 0x7fdab41011c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:13.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.425+0000 7fdab2ffd640 1 -- 172.21.15.78:0/3640452471 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab41a68d0 con 0x7fdab4102ae0 2024-08-18T23:42:13.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.426+0000 7fdab2ffd640 1 --2- 172.21.15.78:0/3640452471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdab4102ae0 0x7fdab4100740 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400b4a0 tx=0x7fdaa400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:13.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.426+0000 7fdab0ff9640 1 -- 172.21.15.78:0/3640452471 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4015020 con 0x7fdab4102ae0 2024-08-18T23:42:13.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.426+0000 7fdab0ff9640 1 -- 172.21.15.78:0/3640452471 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaa4011d10 con 0x7fdab4102ae0 2024-08-18T23:42:13.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.426+0000 7fdab93c7640 1 -- 172.21.15.78:0/3640452471 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab41a6bc0 con 0x7fdab4102ae0 2024-08-18T23:42:13.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.426+0000 7fdab0ff9640 1 -- 172.21.15.78:0/3640452471 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa400f2e0 con 0x7fdab4102ae0 2024-08-18T23:42:13.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.426+0000 7fdab93c7640 1 -- 172.21.15.78:0/3640452471 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab41a70b0 con 0x7fdab4102ae0 2024-08-18T23:42:13.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.427+0000 7fdab93c7640 1 -- 172.21.15.78:0/3640452471 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab406a500 con 0x7fdab4102ae0 2024-08-18T23:42:13.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.428+0000 7fdab0ff9640 1 -- 172.21.15.78:0/3640452471 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdaa4018020 con 0x7fdab4102ae0 2024-08-18T23:42:13.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.428+0000 7fdab0ff9640 1 --2- 172.21.15.78:0/3640452471 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdaa00a2c10 0x7fdaa00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:13.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.429+0000 7fdab0ff9640 1 -- 172.21.15.78:0/3640452471 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fdaa40ceba0 con 0x7fdab4102ae0 2024-08-18T23:42:13.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.429+0000 7fdaabfff640 1 --2- 172.21.15.78:0/3640452471 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdaa00a2c10 0x7fdaa00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:13.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.429+0000 7fdaabfff640 1 --2- 172.21.15.78:0/3640452471 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdaa00a2c10 0x7fdaa00a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdab41021d0 tx=0x7fda9c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:13.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.433+0000 7fdab0ff9640 1 -- 172.21.15.78:0/3640452471 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdaa401f020 con 0x7fdab4102ae0 2024-08-18T23:42:13.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:13 smithi123 ceph-mon[21808]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:42:13.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:13 smithi123 ceph-mon[21808]: pgmap v283: 129 pgs: 129 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-08-18T23:42:13.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:13 smithi123 ceph-mon[21808]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-08-18T23:42:13.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.643+0000 7fdab93c7640 1 -- 172.21.15.78:0/3640452471 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fdab4062360 con 0x7fdaa00a2c10 2024-08-18T23:42:13.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.667+0000 7fdab0ff9640 1 -- 172.21.15.78:0/3640452471 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdab4062360 con 0x7fdaa00a2c10 2024-08-18T23:42:13.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.669+0000 7fdab93c7640 1 -- 172.21.15.78:0/3640452471 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdaa00a2c10 msgr2=0x7fdaa00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:13.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.669+0000 7fdab93c7640 1 --2- 172.21.15.78:0/3640452471 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdaa00a2c10 0x7fdaa00a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdab41021d0 tx=0x7fda9c009110 comp rx=0 tx=0).stop 2024-08-18T23:42:13.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.669+0000 7fdab93c7640 1 -- 172.21.15.78:0/3640452471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdab4102ae0 msgr2=0x7fdab4100740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:13.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.669+0000 7fdab93c7640 1 --2- 172.21.15.78:0/3640452471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdab4102ae0 0x7fdab4100740 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400b4a0 tx=0x7fdaa400b8a0 comp rx=0 tx=0).stop 2024-08-18T23:42:13.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.669+0000 7fdab93c7640 1 -- 172.21.15.78:0/3640452471 shutdown_connections 2024-08-18T23:42:13.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.669+0000 7fdab93c7640 1 --2- 172.21.15.78:0/3640452471 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdaa00a2c10 0x7fdaa00a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:13.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.669+0000 7fdab93c7640 1 --2- 172.21.15.78:0/3640452471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdab4108010 0x7fdab41011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:13.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.669+0000 7fdab93c7640 1 --2- 172.21.15.78:0/3640452471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdab4103490 0x7fdab4100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:13.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.669+0000 7fdab93c7640 1 --2- 172.21.15.78:0/3640452471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdab4102ae0 0x7fdab4100740 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:13.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.669+0000 7fdab93c7640 1 -- 172.21.15.78:0/3640452471 >> 172.21.15.78:0/3640452471 conn(0x7fdab40fc500 msgr2=0x7fdab4105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:13.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.669+0000 7fdab93c7640 1 -- 172.21.15.78:0/3640452471 shutdown_connections 2024-08-18T23:42:13.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:13.669+0000 7fdab93c7640 1 -- 172.21.15.78:0/3640452471 wait complete. 2024-08-18T23:42:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:13 smithi161 ceph-mon[22027]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:42:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:13 smithi161 ceph-mon[22027]: pgmap v283: 129 pgs: 129 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-08-18T23:42:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:13 smithi161 ceph-mon[22027]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-08-18T23:42:14.380 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-08-18T23:42:14.638 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:15.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.156+0000 7f114eb61640 1 -- 172.21.15.78:0/4238712963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11481078b0 msgr2=0x7f1148107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:15.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.156+0000 7f114eb61640 1 --2- 172.21.15.78:0/4238712963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11481078b0 0x7f1148107c90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f113c009930 tx=0x7f113c02a420 comp rx=0 tx=0).stop 2024-08-18T23:42:15.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.156+0000 7f114eb61640 1 -- 172.21.15.78:0/4238712963 shutdown_connections 2024-08-18T23:42:15.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.156+0000 7f114eb61640 1 --2- 172.21.15.78:0/4238712963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1148102250 0x7f114810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:15.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.156+0000 7f114eb61640 1 --2- 172.21.15.78:0/4238712963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11481018b0 0x7f1148101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:15.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.156+0000 7f114eb61640 1 --2- 172.21.15.78:0/4238712963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11481078b0 0x7f1148107c90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:15.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.156+0000 7f114eb61640 1 -- 172.21.15.78:0/4238712963 >> 172.21.15.78:0/4238712963 conn(0x7f11480fd5f0 msgr2=0x7f11480ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:15.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.156+0000 7f114eb61640 1 -- 172.21.15.78:0/4238712963 shutdown_connections 2024-08-18T23:42:15.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.156+0000 7f114eb61640 1 -- 172.21.15.78:0/4238712963 wait complete. 2024-08-18T23:42:15.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.157+0000 7f114eb61640 1 Processor -- start 2024-08-18T23:42:15.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.157+0000 7f114eb61640 1 -- start start 2024-08-18T23:42:15.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.157+0000 7f114eb61640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11481018b0 0x7f1148074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:15.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.157+0000 7f114eb61640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1148102250 0x7f1148074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:15.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.157+0000 7f114eb61640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11481078b0 0x7f1148074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:15.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.157+0000 7f114eb61640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1148111cd0 con 0x7f1148102250 2024-08-18T23:42:15.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.157+0000 7f114eb61640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1148111b50 con 0x7f11481018b0 2024-08-18T23:42:15.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.157+0000 7f114eb61640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1148111e50 con 0x7f11481078b0 2024-08-18T23:42:15.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.157+0000 7f1147fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1148102250 0x7f1148074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:15.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.157+0000 7f1147fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1148102250 0x7f1148074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45214/0 (socket says 172.21.15.78:45214) 2024-08-18T23:42:15.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.157+0000 7f1147fff640 1 -- 172.21.15.78:0/4267909039 learned_addr learned my addr 172.21.15.78:0/4267909039 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:15.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.157+0000 7f114c8fd640 1 --2- 172.21.15.78:0/4267909039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11481018b0 0x7f1148074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:15.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.157+0000 7f114d0fe640 1 --2- 172.21.15.78:0/4267909039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11481078b0 0x7f1148074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:15.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.158+0000 7f1147fff640 1 -- 172.21.15.78:0/4267909039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11481078b0 msgr2=0x7f1148074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:15.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.158+0000 7f1147fff640 1 --2- 172.21.15.78:0/4267909039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11481078b0 0x7f1148074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:15.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.158+0000 7f1147fff640 1 -- 172.21.15.78:0/4267909039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11481018b0 msgr2=0x7f1148074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:15.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.158+0000 7f1147fff640 1 --2- 172.21.15.78:0/4267909039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11481018b0 0x7f1148074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:15.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.158+0000 7f1147fff640 1 -- 172.21.15.78:0/4267909039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11481a66b0 con 0x7f1148102250 2024-08-18T23:42:15.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.158+0000 7f1147fff640 1 --2- 172.21.15.78:0/4267909039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1148102250 0x7f1148074760 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f113400b4a0 tx=0x7f113400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:15.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.158+0000 7f1145ffb640 1 -- 172.21.15.78:0/4267909039 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134014070 con 0x7f1148102250 2024-08-18T23:42:15.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.158+0000 7f1145ffb640 1 -- 172.21.15.78:0/4267909039 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1134010af0 con 0x7f1148102250 2024-08-18T23:42:15.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.158+0000 7f114eb61640 1 -- 172.21.15.78:0/4267909039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11481a69a0 con 0x7f1148102250 2024-08-18T23:42:15.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.158+0000 7f1145ffb640 1 -- 172.21.15.78:0/4267909039 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134017d10 con 0x7f1148102250 2024-08-18T23:42:15.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.158+0000 7f114eb61640 1 -- 172.21.15.78:0/4267909039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11481a6ee0 con 0x7f1148102250 2024-08-18T23:42:15.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.159+0000 7f114eb61640 1 -- 172.21.15.78:0/4267909039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1148101d10 con 0x7f1148102250 2024-08-18T23:42:15.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.160+0000 7f1145ffb640 1 -- 172.21.15.78:0/4267909039 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1134017070 con 0x7f1148102250 2024-08-18T23:42:15.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.161+0000 7f1145ffb640 1 --2- 172.21.15.78:0/4267909039 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f11380a2c90 0x7f11380a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:15.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.161+0000 7f114c8fd640 1 --2- 172.21.15.78:0/4267909039 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f11380a2c90 0x7f11380a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:15.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.161+0000 7f1145ffb640 1 -- 172.21.15.78:0/4267909039 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f11340ce670 con 0x7f1148102250 2024-08-18T23:42:15.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.161+0000 7f114c8fd640 1 --2- 172.21.15.78:0/4267909039 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f11380a2c90 0x7f11380a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f113c009900 tx=0x7f113c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:15.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.165+0000 7f1145ffb640 1 -- 172.21.15.78:0/4267909039 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f113409b340 con 0x7f1148102250 2024-08-18T23:42:15.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.357+0000 7f114eb61640 1 -- 172.21.15.78:0/4267909039 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f1148062360 con 0x7f11380a2c90 2024-08-18T23:42:15.359 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:15 smithi078 ceph-mon[18440]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:42:15.359 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:15 smithi078 ceph-mon[18440]: pgmap v284: 129 pgs: 129 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-08-18T23:42:15.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.362+0000 7f1145ffb640 1 -- 172.21.15.78:0/4267909039 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1148062360 con 0x7f11380a2c90 2024-08-18T23:42:15.364 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/qa/sv_0/6c29c5fd-e5b0-4594-8be0-0b4e172f5a06 2024-08-18T23:42:15.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.365+0000 7f114eb61640 1 -- 172.21.15.78:0/4267909039 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f11380a2c90 msgr2=0x7f11380a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:15.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.365+0000 7f114eb61640 1 --2- 172.21.15.78:0/4267909039 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f11380a2c90 0x7f11380a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f113c009900 tx=0x7f113c0058d0 comp rx=0 tx=0).stop 2024-08-18T23:42:15.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.365+0000 7f114eb61640 1 -- 172.21.15.78:0/4267909039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1148102250 msgr2=0x7f1148074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:15.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.365+0000 7f114eb61640 1 --2- 172.21.15.78:0/4267909039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1148102250 0x7f1148074760 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f113400b4a0 tx=0x7f113400b960 comp rx=0 tx=0).stop 2024-08-18T23:42:15.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.365+0000 7f114eb61640 1 -- 172.21.15.78:0/4267909039 shutdown_connections 2024-08-18T23:42:15.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.365+0000 7f114eb61640 1 --2- 172.21.15.78:0/4267909039 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f11380a2c90 0x7f11380a5150 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:15.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.365+0000 7f114eb61640 1 --2- 172.21.15.78:0/4267909039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11481078b0 0x7f1148074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:15.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.365+0000 7f114eb61640 1 --2- 172.21.15.78:0/4267909039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1148102250 0x7f1148074760 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:15.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.365+0000 7f114eb61640 1 --2- 172.21.15.78:0/4267909039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11481018b0 0x7f1148074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:15.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.365+0000 7f114eb61640 1 -- 172.21.15.78:0/4267909039 >> 172.21.15.78:0/4267909039 conn(0x7f11480fd5f0 msgr2=0x7f11480fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:15.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.365+0000 7f114eb61640 1 -- 172.21.15.78:0/4267909039 shutdown_connections 2024-08-18T23:42:15.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:15.365+0000 7f114eb61640 1 -- 172.21.15.78:0/4267909039 wait complete. 2024-08-18T23:42:15.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:15 smithi123 ceph-mon[21808]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:42:15.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:15 smithi123 ceph-mon[21808]: pgmap v284: 129 pgs: 129 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-08-18T23:42:15.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:15 smithi161 ceph-mon[22027]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:42:15.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:15 smithi161 ceph-mon[22027]: pgmap v284: 129 pgs: 129 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-08-18T23:42:16.071 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-08-18T23:42:16.326 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:16.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.805+0000 7fb172b25640 1 -- 172.21.15.78:0/1805779833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c100b50 msgr2=0x7fb16c100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:16.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.805+0000 7fb172b25640 1 --2- 172.21.15.78:0/1805779833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c100b50 0x7fb16c100f30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb160009900 tx=0x7fb16002a480 comp rx=0 tx=0).stop 2024-08-18T23:42:16.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.805+0000 7fb172b25640 1 -- 172.21.15.78:0/1805779833 shutdown_connections 2024-08-18T23:42:16.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.805+0000 7fb172b25640 1 --2- 172.21.15.78:0/1805779833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb16c102c20 0x7fb16c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:16.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.805+0000 7fb172b25640 1 --2- 172.21.15.78:0/1805779833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb16c1022a0 0x7fb16c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:16.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.805+0000 7fb172b25640 1 --2- 172.21.15.78:0/1805779833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c100b50 0x7fb16c100f30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:16.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.805+0000 7fb172b25640 1 -- 172.21.15.78:0/1805779833 >> 172.21.15.78:0/1805779833 conn(0x7fb16c0fc500 msgr2=0x7fb16c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:16.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.805+0000 7fb172b25640 1 -- 172.21.15.78:0/1805779833 shutdown_connections 2024-08-18T23:42:16.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.805+0000 7fb172b25640 1 -- 172.21.15.78:0/1805779833 wait complete. 2024-08-18T23:42:16.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.805+0000 7fb172b25640 1 Processor -- start 2024-08-18T23:42:16.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.805+0000 7fb172b25640 1 -- start start 2024-08-18T23:42:16.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.806+0000 7fb172b25640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c100b50 0x7fb16c1990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:16.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.806+0000 7fb172b25640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb16c1022a0 0x7fb16c1995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:16.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.806+0000 7fb172b25640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb16c102c20 0x7fb16c1a4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:16.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.806+0000 7fb172b25640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c110cd0 con 0x7fb16c100b50 2024-08-18T23:42:16.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.806+0000 7fb172b25640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c110b50 con 0x7fb16c102c20 2024-08-18T23:42:16.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.806+0000 7fb172b25640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c110e50 con 0x7fb16c1022a0 2024-08-18T23:42:16.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.806+0000 7fb1708c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c100b50 0x7fb16c1990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:16.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.806+0000 7fb1708c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c100b50 0x7fb16c1990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36498/0 (socket says 172.21.15.78:36498) 2024-08-18T23:42:16.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.806+0000 7fb1708c1640 1 -- 172.21.15.78:0/414847588 learned_addr learned my addr 172.21.15.78:0/414847588 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:16.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.806+0000 7fb16bfff640 1 --2- 172.21.15.78:0/414847588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb16c1022a0 0x7fb16c1995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:16.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.806+0000 7fb1710c2640 1 --2- 172.21.15.78:0/414847588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb16c102c20 0x7fb16c1a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:16.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.806+0000 7fb1708c1640 1 -- 172.21.15.78:0/414847588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb16c1022a0 msgr2=0x7fb16c1995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:16.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.806+0000 7fb1708c1640 1 --2- 172.21.15.78:0/414847588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb16c1022a0 0x7fb16c1995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:16.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.806+0000 7fb1708c1640 1 -- 172.21.15.78:0/414847588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb16c102c20 msgr2=0x7fb16c1a4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:16.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.806+0000 7fb1708c1640 1 --2- 172.21.15.78:0/414847588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb16c102c20 0x7fb16c1a4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:16.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.806+0000 7fb1708c1640 1 -- 172.21.15.78:0/414847588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16c1a4660 con 0x7fb16c100b50 2024-08-18T23:42:16.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.806+0000 7fb16bfff640 1 --2- 172.21.15.78:0/414847588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb16c1022a0 0x7fb16c1995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:42:16.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.806+0000 7fb1708c1640 1 --2- 172.21.15.78:0/414847588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c100b50 0x7fb16c1990a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb160006010 tx=0x7fb160003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:16.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.806+0000 7fb169ffb640 1 -- 172.21.15.78:0/414847588 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16003f070 con 0x7fb16c100b50 2024-08-18T23:42:16.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.807+0000 7fb169ffb640 1 -- 172.21.15.78:0/414847588 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb160031e30 con 0x7fb16c100b50 2024-08-18T23:42:16.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.807+0000 7fb172b25640 1 -- 172.21.15.78:0/414847588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16c1a4830 con 0x7fb16c100b50 2024-08-18T23:42:16.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.807+0000 7fb169ffb640 1 -- 172.21.15.78:0/414847588 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160035030 con 0x7fb16c100b50 2024-08-18T23:42:16.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.807+0000 7fb172b25640 1 -- 172.21.15.78:0/414847588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16c1a4c80 con 0x7fb16c100b50 2024-08-18T23:42:16.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.808+0000 7fb172b25640 1 -- 172.21.15.78:0/414847588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb16c100f30 con 0x7fb16c100b50 2024-08-18T23:42:16.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.809+0000 7fb169ffb640 1 -- 172.21.15.78:0/414847588 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb160042070 con 0x7fb16c100b50 2024-08-18T23:42:16.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.810+0000 7fb169ffb640 1 --2- 172.21.15.78:0/414847588 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb15c0a2c10 0x7fb15c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:16.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.810+0000 7fb16bfff640 1 --2- 172.21.15.78:0/414847588 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb15c0a2c10 0x7fb15c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:16.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.810+0000 7fb169ffb640 1 -- 172.21.15.78:0/414847588 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fb1600efe10 con 0x7fb16c100b50 2024-08-18T23:42:16.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.810+0000 7fb16bfff640 1 --2- 172.21.15.78:0/414847588 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb15c0a2c10 0x7fb15c0a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb16c06a7f0 tx=0x7fb158008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:16.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:16.814+0000 7fb169ffb640 1 -- 172.21.15.78:0/414847588 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb1600bc9d0 con 0x7fb16c100b50 2024-08-18T23:42:17.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:17.006+0000 7fb172b25640 1 -- 172.21.15.78:0/414847588 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb16c106ca0 con 0x7fb15c0a2c10 2024-08-18T23:42:17.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:17.033+0000 7fb169ffb640 1 -- 172.21.15.78:0/414847588 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb16c106ca0 con 0x7fb15c0a2c10 2024-08-18T23:42:17.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:17.035+0000 7fb172b25640 1 -- 172.21.15.78:0/414847588 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb15c0a2c10 msgr2=0x7fb15c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:17.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:17.035+0000 7fb172b25640 1 --2- 172.21.15.78:0/414847588 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb15c0a2c10 0x7fb15c0a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb16c06a7f0 tx=0x7fb158008040 comp rx=0 tx=0).stop 2024-08-18T23:42:17.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:17.035+0000 7fb172b25640 1 -- 172.21.15.78:0/414847588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c100b50 msgr2=0x7fb16c1990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:17.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:17.035+0000 7fb172b25640 1 --2- 172.21.15.78:0/414847588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c100b50 0x7fb16c1990a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb160006010 tx=0x7fb160003c80 comp rx=0 tx=0).stop 2024-08-18T23:42:17.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:17.035+0000 7fb172b25640 1 -- 172.21.15.78:0/414847588 shutdown_connections 2024-08-18T23:42:17.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:17.035+0000 7fb172b25640 1 --2- 172.21.15.78:0/414847588 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb15c0a2c10 0x7fb15c0a50d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:17.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:17.035+0000 7fb172b25640 1 --2- 172.21.15.78:0/414847588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb16c102c20 0x7fb16c1a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:17.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:17.035+0000 7fb172b25640 1 --2- 172.21.15.78:0/414847588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb16c1022a0 0x7fb16c1995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:17.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:17.035+0000 7fb172b25640 1 --2- 172.21.15.78:0/414847588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb16c100b50 0x7fb16c1990a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:17.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:17.035+0000 7fb172b25640 1 -- 172.21.15.78:0/414847588 >> 172.21.15.78:0/414847588 conn(0x7fb16c0fc500 msgr2=0x7fb16c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:17.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:17.035+0000 7fb172b25640 1 -- 172.21.15.78:0/414847588 shutdown_connections 2024-08-18T23:42:17.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:17.035+0000 7fb172b25640 1 -- 172.21.15.78:0/414847588 wait complete. 2024-08-18T23:42:17.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:17 smithi123 ceph-mon[21808]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:42:17.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:17 smithi123 ceph-mon[21808]: pgmap v285: 129 pgs: 129 active+clean; 604 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-08-18T23:42:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:17 smithi161 ceph-mon[22027]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:42:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:17 smithi161 ceph-mon[22027]: pgmap v285: 129 pgs: 129 active+clean; 604 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-08-18T23:42:17.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:17 smithi078 ceph-mon[18440]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:42:17.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:17 smithi078 ceph-mon[18440]: pgmap v285: 129 pgs: 129 active+clean; 604 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-08-18T23:42:17.838 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-08-18T23:42:18.088 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:18.464 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:18 smithi078 ceph-mon[18440]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:42:18.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:18 smithi123 ceph-mon[21808]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:42:18.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.548+0000 7f931b20b640 1 -- 172.21.15.78:0/369119477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9314100b50 msgr2=0x7f9314100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:18.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.548+0000 7f931b20b640 1 --2- 172.21.15.78:0/369119477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9314100b50 0x7f9314100f30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9304009ca0 tx=0x7f930402a950 comp rx=0 tx=0).stop 2024-08-18T23:42:18.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.549+0000 7f931b20b640 1 -- 172.21.15.78:0/369119477 shutdown_connections 2024-08-18T23:42:18.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.549+0000 7f931b20b640 1 --2- 172.21.15.78:0/369119477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9314102c20 0x7f931410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:18.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.549+0000 7f931b20b640 1 --2- 172.21.15.78:0/369119477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93141022a0 0x7f93141026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:18.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.549+0000 7f931b20b640 1 --2- 172.21.15.78:0/369119477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9314100b50 0x7f9314100f30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:18.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.549+0000 7f931b20b640 1 -- 172.21.15.78:0/369119477 >> 172.21.15.78:0/369119477 conn(0x7f93140fc500 msgr2=0x7f93140fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:18.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.549+0000 7f931b20b640 1 -- 172.21.15.78:0/369119477 shutdown_connections 2024-08-18T23:42:18.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.549+0000 7f931b20b640 1 -- 172.21.15.78:0/369119477 wait complete. 2024-08-18T23:42:18.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.549+0000 7f931b20b640 1 Processor -- start 2024-08-18T23:42:18.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.549+0000 7f931b20b640 1 -- start start 2024-08-18T23:42:18.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.549+0000 7f931b20b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9314100b50 0x7f93141990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:18.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.549+0000 7f931b20b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93141022a0 0x7f93141995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:18.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.549+0000 7f931b20b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9314102c20 0x7f93141a4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:18.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.549+0000 7f931b20b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9314110cd0 con 0x7f9314100b50 2024-08-18T23:42:18.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.549+0000 7f931b20b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9314110b50 con 0x7f9314102c20 2024-08-18T23:42:18.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.549+0000 7f931b20b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9314110e50 con 0x7f93141022a0 2024-08-18T23:42:18.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.550+0000 7f9318fa7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9314100b50 0x7f93141990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:18.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.550+0000 7f9318fa7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9314100b50 0x7f93141990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36518/0 (socket says 172.21.15.78:36518) 2024-08-18T23:42:18.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.550+0000 7f9318fa7640 1 -- 172.21.15.78:0/2806641030 learned_addr learned my addr 172.21.15.78:0/2806641030 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:18.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.550+0000 7f93197a8640 1 --2- 172.21.15.78:0/2806641030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9314102c20 0x7f93141a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:18.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.550+0000 7f9313fff640 1 --2- 172.21.15.78:0/2806641030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93141022a0 0x7f93141995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:18.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.550+0000 7f9318fa7640 1 -- 172.21.15.78:0/2806641030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93141022a0 msgr2=0x7f93141995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:18.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.550+0000 7f9318fa7640 1 --2- 172.21.15.78:0/2806641030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93141022a0 0x7f93141995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:18.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.550+0000 7f9318fa7640 1 -- 172.21.15.78:0/2806641030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9314102c20 msgr2=0x7f93141a4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:18.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.550+0000 7f9318fa7640 1 --2- 172.21.15.78:0/2806641030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9314102c20 0x7f93141a4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:18.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.550+0000 7f9318fa7640 1 -- 172.21.15.78:0/2806641030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93141a4660 con 0x7f9314100b50 2024-08-18T23:42:18.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.550+0000 7f9313fff640 1 --2- 172.21.15.78:0/2806641030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93141022a0 0x7f93141995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:42:18.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.550+0000 7f9318fa7640 1 --2- 172.21.15.78:0/2806641030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9314100b50 0x7f93141990a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9304005cc0 tx=0x7f930402f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:18.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.550+0000 7f9311ffb640 1 -- 172.21.15.78:0/2806641030 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304034040 con 0x7f9314100b50 2024-08-18T23:42:18.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.550+0000 7f9311ffb640 1 -- 172.21.15.78:0/2806641030 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9304003e40 con 0x7f9314100b50 2024-08-18T23:42:18.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.550+0000 7f931b20b640 1 -- 172.21.15.78:0/2806641030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93141a4830 con 0x7f9314100b50 2024-08-18T23:42:18.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.550+0000 7f9311ffb640 1 -- 172.21.15.78:0/2806641030 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304045480 con 0x7f9314100b50 2024-08-18T23:42:18.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.550+0000 7f931b20b640 1 -- 172.21.15.78:0/2806641030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93141a4ce0 con 0x7f9314100b50 2024-08-18T23:42:18.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.551+0000 7f931b20b640 1 -- 172.21.15.78:0/2806641030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9314100f30 con 0x7f9314100b50 2024-08-18T23:42:18.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.553+0000 7f9311ffb640 1 -- 172.21.15.78:0/2806641030 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9304037020 con 0x7f9314100b50 2024-08-18T23:42:18.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.554+0000 7f9311ffb640 1 --2- 172.21.15.78:0/2806641030 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f92fc0a2c90 0x7f92fc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:18.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.554+0000 7f9311ffb640 1 -- 172.21.15.78:0/2806641030 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f93040f0680 con 0x7f9314100b50 2024-08-18T23:42:18.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.554+0000 7f9313fff640 1 --2- 172.21.15.78:0/2806641030 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f92fc0a2c90 0x7f92fc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:18.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.554+0000 7f9313fff640 1 --2- 172.21.15.78:0/2806641030 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f92fc0a2c90 0x7f92fc0a5150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f931406a7f0 tx=0x7f9308008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:18.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.557+0000 7f9311ffb640 1 -- 172.21.15.78:0/2806641030 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9304080df0 con 0x7f9314100b50 2024-08-18T23:42:18.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:18 smithi161 ceph-mon[22027]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:42:18.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.750+0000 7f931b20b640 1 -- 172.21.15.78:0/2806641030 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f9314106ca0 con 0x7f92fc0a2c90 2024-08-18T23:42:18.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.755+0000 7f9311ffb640 1 -- 172.21.15.78:0/2806641030 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9314106ca0 con 0x7f92fc0a2c90 2024-08-18T23:42:18.756 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/qa/sv_1/5f1302ca-91a6-4860-a5ba-eb68e67b3c86 2024-08-18T23:42:18.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.757+0000 7f931b20b640 1 -- 172.21.15.78:0/2806641030 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f92fc0a2c90 msgr2=0x7f92fc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:18.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.757+0000 7f931b20b640 1 --2- 172.21.15.78:0/2806641030 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f92fc0a2c90 0x7f92fc0a5150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f931406a7f0 tx=0x7f9308008040 comp rx=0 tx=0).stop 2024-08-18T23:42:18.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.757+0000 7f931b20b640 1 -- 172.21.15.78:0/2806641030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9314100b50 msgr2=0x7f93141990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:18.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.757+0000 7f931b20b640 1 --2- 172.21.15.78:0/2806641030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9314100b50 0x7f93141990a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9304005cc0 tx=0x7f930402f6c0 comp rx=0 tx=0).stop 2024-08-18T23:42:18.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.757+0000 7f931b20b640 1 -- 172.21.15.78:0/2806641030 shutdown_connections 2024-08-18T23:42:18.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.757+0000 7f931b20b640 1 --2- 172.21.15.78:0/2806641030 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f92fc0a2c90 0x7f92fc0a5150 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:18.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.757+0000 7f931b20b640 1 --2- 172.21.15.78:0/2806641030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9314102c20 0x7f93141a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:18.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.757+0000 7f931b20b640 1 --2- 172.21.15.78:0/2806641030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93141022a0 0x7f93141995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:18.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.757+0000 7f931b20b640 1 --2- 172.21.15.78:0/2806641030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9314100b50 0x7f93141990a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:18.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.757+0000 7f931b20b640 1 -- 172.21.15.78:0/2806641030 >> 172.21.15.78:0/2806641030 conn(0x7f93140fc500 msgr2=0x7f93140fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:18.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.757+0000 7f931b20b640 1 -- 172.21.15.78:0/2806641030 shutdown_connections 2024-08-18T23:42:18.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:18.757+0000 7f931b20b640 1 -- 172.21.15.78:0/2806641030 wait complete. 2024-08-18T23:42:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:19 smithi078 ceph-mon[18440]: pgmap v286: 129 pgs: 129 active+clean; 605 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-08-18T23:42:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:19 smithi078 ceph-mon[18440]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:42:19.521 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:42:19.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:19 smithi123 ceph-mon[21808]: pgmap v286: 129 pgs: 129 active+clean; 605 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-08-18T23:42:19.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:19 smithi123 ceph-mon[21808]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:42:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:19 smithi161 ceph-mon[22027]: pgmap v286: 129 pgs: 129 active+clean; 605 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-08-18T23:42:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:19 smithi161 ceph-mon[22027]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:42:19.772 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:20.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.286+0000 7f524c4ce640 1 -- 172.21.15.78:0/2517586745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5244103390 msgr2=0x7f524410b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:20.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.286+0000 7f524c4ce640 1 --2- 172.21.15.78:0/2517586745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5244103390 0x7f524410b880 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5234009900 tx=0x7f523402a480 comp rx=0 tx=0).stop 2024-08-18T23:42:20.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.286+0000 7f524c4ce640 1 -- 172.21.15.78:0/2517586745 shutdown_connections 2024-08-18T23:42:20.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.286+0000 7f524c4ce640 1 --2- 172.21.15.78:0/2517586745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f524410bea0 0x7f524410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:20.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.286+0000 7f524c4ce640 1 --2- 172.21.15.78:0/2517586745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5244103390 0x7f524410b880 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:20.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.286+0000 7f524c4ce640 1 --2- 172.21.15.78:0/2517586745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5244102a70 0x7f5244102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:20.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.286+0000 7f524c4ce640 1 -- 172.21.15.78:0/2517586745 >> 172.21.15.78:0/2517586745 conn(0x7f52440fc4d0 msgr2=0x7f52440fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:20.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.286+0000 7f524c4ce640 1 -- 172.21.15.78:0/2517586745 shutdown_connections 2024-08-18T23:42:20.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.286+0000 7f524c4ce640 1 -- 172.21.15.78:0/2517586745 wait complete. 2024-08-18T23:42:20.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.287+0000 7f524c4ce640 1 Processor -- start 2024-08-18T23:42:20.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.287+0000 7f524c4ce640 1 -- start start 2024-08-18T23:42:20.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.287+0000 7f524c4ce640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5244102a70 0x7f52441006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:20.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.287+0000 7f524c4ce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5244103390 0x7f5244100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:20.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.287+0000 7f524c4ce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524410bea0 0x7f5244101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:20.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.287+0000 7f524c4ce640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5244111ce0 con 0x7f524410bea0 2024-08-18T23:42:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.287+0000 7f524c4ce640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5244111b60 con 0x7f5244102a70 2024-08-18T23:42:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.287+0000 7f524c4ce640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5244111e60 con 0x7f5244103390 2024-08-18T23:42:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.287+0000 7f524aa6b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524410bea0 0x7f5244101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.287+0000 7f524aa6b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524410bea0 0x7f5244101150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36536/0 (socket says 172.21.15.78:36536) 2024-08-18T23:42:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.287+0000 7f524aa6b640 1 -- 172.21.15.78:0/688123999 learned_addr learned my addr 172.21.15.78:0/688123999 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.287+0000 7f5249a69640 1 --2- 172.21.15.78:0/688123999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5244103390 0x7f5244100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.287+0000 7f524a26a640 1 --2- 172.21.15.78:0/688123999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5244102a70 0x7f52441006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.287+0000 7f524aa6b640 1 -- 172.21.15.78:0/688123999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5244103390 msgr2=0x7f5244100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.287+0000 7f524aa6b640 1 --2- 172.21.15.78:0/688123999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5244103390 0x7f5244100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.287+0000 7f524aa6b640 1 -- 172.21.15.78:0/688123999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5244102a70 msgr2=0x7f52441006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.287+0000 7f524aa6b640 1 --2- 172.21.15.78:0/688123999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5244102a70 0x7f52441006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.287+0000 7f524aa6b640 1 -- 172.21.15.78:0/688123999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52441a6690 con 0x7f524410bea0 2024-08-18T23:42:20.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.288+0000 7f524aa6b640 1 --2- 172.21.15.78:0/688123999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524410bea0 0x7f5244101150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f523c00b970 tx=0x7f523c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.289+0000 7f523b7fe640 1 -- 172.21.15.78:0/688123999 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c019070 con 0x7f524410bea0 2024-08-18T23:42:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.289+0000 7f524c4ce640 1 -- 172.21.15.78:0/688123999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52441a6980 con 0x7f524410bea0 2024-08-18T23:42:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.289+0000 7f523b7fe640 1 -- 172.21.15.78:0/688123999 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f523c00cb40 con 0x7f524410bea0 2024-08-18T23:42:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.289+0000 7f523b7fe640 1 -- 172.21.15.78:0/688123999 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c007650 con 0x7f524410bea0 2024-08-18T23:42:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.289+0000 7f524c4ce640 1 -- 172.21.15.78:0/688123999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52441a6e90 con 0x7f524410bea0 2024-08-18T23:42:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.290+0000 7f524c4ce640 1 -- 172.21.15.78:0/688123999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5244102e50 con 0x7f524410bea0 2024-08-18T23:42:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.291+0000 7f523b7fe640 1 -- 172.21.15.78:0/688123999 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f523c01c070 con 0x7f524410bea0 2024-08-18T23:42:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.292+0000 7f523b7fe640 1 --2- 172.21.15.78:0/688123999 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f52300a2c90 0x7f52300a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.292+0000 7f524a26a640 1 --2- 172.21.15.78:0/688123999 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f52300a2c90 0x7f52300a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.292+0000 7f523b7fe640 1 -- 172.21.15.78:0/688123999 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f523c0ce850 con 0x7f524410bea0 2024-08-18T23:42:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.292+0000 7f524a26a640 1 --2- 172.21.15.78:0/688123999 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f52300a2c90 0x7f52300a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f52400058c0 tx=0x7f524000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:20.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.296+0000 7f523b7fe640 1 -- 172.21.15.78:0/688123999 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f523c09aed0 con 0x7f524410bea0 2024-08-18T23:42:20.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:20 smithi078 ceph-mon[18440]: pgmap v287: 129 pgs: 129 active+clean; 629 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2024-08-18T23:42:20.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.498+0000 7f524c4ce640 1 -- 172.21.15.78:0/688123999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5244106f10 con 0x7f524410bea0 2024-08-18T23:42:20.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.499+0000 7f523b7fe640 1 -- 172.21.15.78:0/688123999 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11930 (secure 0 0 0) 0x7f523c09f1d0 con 0x7f524410bea0 2024-08-18T23:42:20.500 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:42:20.500 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":24,"btime":"2024-08-18T23:42:10:259464+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:42:09.974847+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:42:20.501 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 24 2024-08-18T23:42:20.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.501+0000 7f524c4ce640 1 -- 172.21.15.78:0/688123999 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f52300a2c90 msgr2=0x7f52300a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:20.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.501+0000 7f524c4ce640 1 --2- 172.21.15.78:0/688123999 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f52300a2c90 0x7f52300a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f52400058c0 tx=0x7f524000a2d0 comp rx=0 tx=0).stop 2024-08-18T23:42:20.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.501+0000 7f524c4ce640 1 -- 172.21.15.78:0/688123999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524410bea0 msgr2=0x7f5244101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:20.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.501+0000 7f524c4ce640 1 --2- 172.21.15.78:0/688123999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524410bea0 0x7f5244101150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f523c00b970 tx=0x7f523c00be30 comp rx=0 tx=0).stop 2024-08-18T23:42:20.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.501+0000 7f524c4ce640 1 -- 172.21.15.78:0/688123999 shutdown_connections 2024-08-18T23:42:20.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.501+0000 7f524c4ce640 1 --2- 172.21.15.78:0/688123999 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f52300a2c90 0x7f52300a5150 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:20.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.501+0000 7f524c4ce640 1 --2- 172.21.15.78:0/688123999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f524410bea0 0x7f5244101150 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:20.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.501+0000 7f524c4ce640 1 --2- 172.21.15.78:0/688123999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5244103390 0x7f5244100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:20.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.501+0000 7f524c4ce640 1 --2- 172.21.15.78:0/688123999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5244102a70 0x7f52441006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:20.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.501+0000 7f524c4ce640 1 -- 172.21.15.78:0/688123999 >> 172.21.15.78:0/688123999 conn(0x7f52440fc4d0 msgr2=0x7f52440fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:20.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.501+0000 7f524c4ce640 1 -- 172.21.15.78:0/688123999 shutdown_connections 2024-08-18T23:42:20.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:20.501+0000 7f524c4ce640 1 -- 172.21.15.78:0/688123999 wait complete. 2024-08-18T23:42:20.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:20 smithi161 ceph-mon[22027]: pgmap v287: 129 pgs: 129 active+clean; 629 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2024-08-18T23:42:20.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:20 smithi123 ceph-mon[21808]: pgmap v287: 129 pgs: 129 active+clean; 629 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2024-08-18T23:42:21.215 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-18T23:42:21.465 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:21.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:21 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/688123999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:42:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:21 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/688123999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:42:21.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:21 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/688123999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:42:21.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.916+0000 7f83a4a5c640 1 -- 172.21.15.78:0/2095352479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83a0102a70 msgr2=0x7f83a0102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:21.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.916+0000 7f83a4a5c640 1 --2- 172.21.15.78:0/2095352479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83a0102a70 0x7f83a0102e50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8390009900 tx=0x7f839002a480 comp rx=0 tx=0).stop 2024-08-18T23:42:21.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.916+0000 7f83a4a5c640 1 -- 172.21.15.78:0/2095352479 shutdown_connections 2024-08-18T23:42:21.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.916+0000 7f83a4a5c640 1 --2- 172.21.15.78:0/2095352479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83a010bea0 0x7f83a010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:21.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.916+0000 7f83a4a5c640 1 --2- 172.21.15.78:0/2095352479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83a0103390 0x7f83a010b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:21.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.916+0000 7f83a4a5c640 1 --2- 172.21.15.78:0/2095352479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83a0102a70 0x7f83a0102e50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:21.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.916+0000 7f83a4a5c640 1 -- 172.21.15.78:0/2095352479 >> 172.21.15.78:0/2095352479 conn(0x7f83a00fc4d0 msgr2=0x7f83a00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:21.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.916+0000 7f83a4a5c640 1 -- 172.21.15.78:0/2095352479 shutdown_connections 2024-08-18T23:42:21.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.916+0000 7f83a4a5c640 1 -- 172.21.15.78:0/2095352479 wait complete. 2024-08-18T23:42:21.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.917+0000 7f83a4a5c640 1 Processor -- start 2024-08-18T23:42:21.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.917+0000 7f83a4a5c640 1 -- start start 2024-08-18T23:42:21.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.917+0000 7f83a4a5c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83a0102a70 0x7f83a01981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:21.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.917+0000 7f83a4a5c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83a0103390 0x7f83a0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:21.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.917+0000 7f83a4a5c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83a010bea0 0x7f83a019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:21.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.917+0000 7f83a4a5c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f83a0111ce0 con 0x7f83a010bea0 2024-08-18T23:42:21.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.917+0000 7f83a4a5c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83a0111b60 con 0x7f83a0103390 2024-08-18T23:42:21.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.917+0000 7f83a4a5c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f83a0111e60 con 0x7f83a0102a70 2024-08-18T23:42:21.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.917+0000 7f839ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83a010bea0 0x7f83a019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:21.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.917+0000 7f839ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83a010bea0 0x7f83a019ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36542/0 (socket says 172.21.15.78:36542) 2024-08-18T23:42:21.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.917+0000 7f839ed9d640 1 -- 172.21.15.78:0/1541290549 learned_addr learned my addr 172.21.15.78:0/1541290549 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:21.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.918+0000 7f8397fff640 1 --2- 172.21.15.78:0/1541290549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83a0103390 0x7f83a0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:21.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.918+0000 7f839e59c640 1 --2- 172.21.15.78:0/1541290549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83a0102a70 0x7f83a01981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:21.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.918+0000 7f839ed9d640 1 -- 172.21.15.78:0/1541290549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83a0102a70 msgr2=0x7f83a01981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:21.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.918+0000 7f839ed9d640 1 --2- 172.21.15.78:0/1541290549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83a0102a70 0x7f83a01981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:21.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.918+0000 7f839ed9d640 1 -- 172.21.15.78:0/1541290549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83a0103390 msgr2=0x7f83a0198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:21.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.918+0000 7f839ed9d640 1 --2- 172.21.15.78:0/1541290549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83a0103390 0x7f83a0198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:21.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.918+0000 7f839ed9d640 1 -- 172.21.15.78:0/1541290549 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a019d170 con 0x7f83a010bea0 2024-08-18T23:42:21.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.918+0000 7f8397fff640 1 --2- 172.21.15.78:0/1541290549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83a0103390 0x7f83a0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:42:21.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.918+0000 7f839e59c640 1 --2- 172.21.15.78:0/1541290549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83a0102a70 0x7f83a01981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:42:21.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.918+0000 7f839ed9d640 1 --2- 172.21.15.78:0/1541290549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83a010bea0 0x7f83a019ca90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f839800b970 tx=0x7f839800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:21.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.918+0000 7f83977fe640 1 -- 172.21.15.78:0/1541290549 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398019070 con 0x7f83a010bea0 2024-08-18T23:42:21.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.918+0000 7f83a4a5c640 1 -- 172.21.15.78:0/1541290549 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a019d460 con 0x7f83a010bea0 2024-08-18T23:42:21.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.918+0000 7f83977fe640 1 -- 172.21.15.78:0/1541290549 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f839800cb40 con 0x7f83a010bea0 2024-08-18T23:42:21.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.918+0000 7f83977fe640 1 -- 172.21.15.78:0/1541290549 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398007610 con 0x7f83a010bea0 2024-08-18T23:42:21.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.918+0000 7f83a4a5c640 1 -- 172.21.15.78:0/1541290549 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a01a4d40 con 0x7f83a010bea0 2024-08-18T23:42:21.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.919+0000 7f83a4a5c640 1 -- 172.21.15.78:0/1541290549 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a0102e50 con 0x7f83a010bea0 2024-08-18T23:42:21.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.920+0000 7f83977fe640 1 -- 172.21.15.78:0/1541290549 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f839801c070 con 0x7f83a010bea0 2024-08-18T23:42:21.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.921+0000 7f83977fe640 1 --2- 172.21.15.78:0/1541290549 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f838c0a2c10 0x7f838c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:21.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.921+0000 7f839e59c640 1 --2- 172.21.15.78:0/1541290549 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f838c0a2c10 0x7f838c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:21.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.921+0000 7f83977fe640 1 -- 172.21.15.78:0/1541290549 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f83980ce940 con 0x7f83a010bea0 2024-08-18T23:42:21.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.922+0000 7f839e59c640 1 --2- 172.21.15.78:0/1541290549 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f838c0a2c10 0x7f838c0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8390005ab0 tx=0x7f8390005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:21.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:21.925+0000 7f83977fe640 1 -- 172.21.15.78:0/1541290549 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f839809b610 con 0x7f83a010bea0 2024-08-18T23:42:22.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:22.093+0000 7f83a4a5c640 1 -- 172.21.15.78:0/1541290549 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83a0106fa0 con 0x7f83a010bea0 2024-08-18T23:42:22.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:22.095+0000 7f83977fe640 1 -- 172.21.15.78:0/1541290549 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24126 (secure 0 0 0) 0x7f839809af10 con 0x7f83a010bea0 2024-08-18T23:42:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:42:22.097 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":88,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","created":"2024-08-18T23:34:33.867246+0000","modified":"2024-08-18T23:41:41.811101+0000","last_up_change":"2024-08-18T23:40:42.200381+0000","last_in_change":"2024-08-18T23:40:22.425813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T23:37:22.338211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T23:41:28.920802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T23:41:31.026094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T23:41:37.403238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4de1c78-6ebc-49a9-b9c5-85d43535fab7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6803","nonce":3639645311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6805","nonce":3639645311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6809","nonce":3639645311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6807","nonce":3639645311}]},"public_addr":"172.21.15.78:6803/3639645311","cluster_addr":"172.21.15.78:6805/3639645311","heartbeat_back_addr":"172.21.15.78:6809/3639645311","heartbeat_front_addr":"172.21.15.78:6807/3639645311","state":["exists","up"]},{"osd":1,"uuid":"4f8e0c3d-472b-4230-ad23-2b0a34861251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6801","nonce":4199446368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6803","nonce":4199446368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6807","nonce":4199446368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6805","nonce":4199446368}]},"public_addr":"172.21.15.123:6801/4199446368","cluster_addr":"172.21.15.123:6803/4199446368","heartbeat_back_addr":"172.21.15.123:6807/4199446368","heartbeat_front_addr":"172.21.15.123:6805/4199446368","state":["exists","up"]},{"osd":2,"uuid":"27ef3314-bb6f-413d-be05-e10a105dec9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6801","nonce":2582810057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6803","nonce":2582810057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6807","nonce":2582810057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6805","nonce":2582810057}]},"public_addr":"172.21.15.161:6801/2582810057","cluster_addr":"172.21.15.161:6803/2582810057","heartbeat_back_addr":"172.21.15.161:6807/2582810057","heartbeat_front_addr":"172.21.15.161:6805/2582810057","state":["exists","up"]},{"osd":3,"uuid":"e9e52662-5ef9-4d70-bdd7-2b92165822ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6811","nonce":2697347521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6813","nonce":2697347521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6817","nonce":2697347521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6815","nonce":2697347521}]},"public_addr":"172.21.15.78:6811/2697347521","cluster_addr":"172.21.15.78:6813/2697347521","heartbeat_back_addr":"172.21.15.78:6817/2697347521","heartbeat_front_addr":"172.21.15.78:6815/2697347521","state":["exists","up"]},{"osd":4,"uuid":"5ed26775-79e3-4b24-9557-69fcf5e8e824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6809","nonce":1281511569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6811","nonce":1281511569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6815","nonce":1281511569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6813","nonce":1281511569}]},"public_addr":"172.21.15.123:6809/1281511569","cluster_addr":"172.21.15.123:6811/1281511569","heartbeat_back_addr":"172.21.15.123:6815/1281511569","heartbeat_front_addr":"172.21.15.123:6813/1281511569","state":["exists","up"]},{"osd":5,"uuid":"a9793b1d-5613-48a4-938e-adb718f4fa4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6809","nonce":2751796117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6811","nonce":2751796117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6815","nonce":2751796117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6813","nonce":2751796117}]},"public_addr":"172.21.15.161:6809/2751796117","cluster_addr":"172.21.15.161:6811/2751796117","heartbeat_back_addr":"172.21.15.161:6815/2751796117","heartbeat_front_addr":"172.21.15.161:6813/2751796117","state":["exists","up"]},{"osd":6,"uuid":"5dbc5e9c-a14f-4b71-910a-1e6d4827bb32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6819","nonce":95383223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6821","nonce":95383223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6825","nonce":95383223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6823","nonce":95383223}]},"public_addr":"172.21.15.78:6819/95383223","cluster_addr":"172.21.15.78:6821/95383223","heartbeat_back_addr":"172.21.15.78:6825/95383223","heartbeat_front_addr":"172.21.15.78:6823/95383223","state":["exists","up"]},{"osd":7,"uuid":"26fb7b69-131f-42ee-b745-6fe8b4f65391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6817","nonce":3440305156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6819","nonce":3440305156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6823","nonce":3440305156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6821","nonce":3440305156}]},"public_addr":"172.21.15.123:6817/3440305156","cluster_addr":"172.21.15.123:6819/3440305156","heartbeat_back_addr":"172.21.15.123:6823/3440305156","heartbeat_front_addr":"172.21.15.123:6821/3440305156","state":["exists","up"]},{"osd":8,"uuid":"e267bc50-0c29-4d7b-b368-b1621e29b2df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6817","nonce":1485008778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6819","nonce":1485008778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6823","nonce":1485008778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6821","nonce":1485008778}]},"public_addr":"172.21.15.161:6817/1485008778","cluster_addr":"172.21.15.161:6819/1485008778","heartbeat_back_addr":"172.21.15.161:6823/1485008778","heartbeat_front_addr":"172.21.15.161:6821/1485008778","state":["exists","up"]},{"osd":9,"uuid":"b7654818-b767-4fe6-8211-346297895f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6827","nonce":2884475661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6829","nonce":2884475661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6833","nonce":2884475661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6831","nonce":2884475661}]},"public_addr":"172.21.15.78:6827/2884475661","cluster_addr":"172.21.15.78:6829/2884475661","heartbeat_back_addr":"172.21.15.78:6833/2884475661","heartbeat_front_addr":"172.21.15.78:6831/2884475661","state":["exists","up"]},{"osd":10,"uuid":"59aa32ff-510e-4b52-8170-08a34a0b6b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6825","nonce":122226286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6827","nonce":122226286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6831","nonce":122226286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6829","nonce":122226286}]},"public_addr":"172.21.15.123:6825/122226286","cluster_addr":"172.21.15.123:6827/122226286","heartbeat_back_addr":"172.21.15.123:6831/122226286","heartbeat_front_addr":"172.21.15.123:6829/122226286","state":["exists","up"]},{"osd":11,"uuid":"67bacab4-d078-4c5c-b4ff-9f6a8a00f507","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6825","nonce":2037560985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6827","nonce":2037560985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6831","nonce":2037560985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6829","nonce":2037560985}]},"public_addr":"172.21.15.161:6825/2037560985","cluster_addr":"172.21.15.161:6827/2037560985","heartbeat_back_addr":"172.21.15.161:6831/2037560985","heartbeat_front_addr":"172.21.15.161:6829/2037560985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:36.826044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:57.688421+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:18.297424+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:40.355215+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:02.167902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:23.968759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:46.254083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:08.522038+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:31.028119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:54.119807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:16.847034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:39.607886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6800/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3040499240":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6801/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:6800/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/757409165":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/2562072965":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/636184437":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/1570708294":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3137806628":"2024-08-19T23:34:59.021165+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T23:42:22.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:22.097+0000 7f83a4a5c640 1 -- 172.21.15.78:0/1541290549 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f838c0a2c10 msgr2=0x7f838c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:22.097+0000 7f83a4a5c640 1 --2- 172.21.15.78:0/1541290549 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f838c0a2c10 0x7f838c0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8390005ab0 tx=0x7f8390005a20 comp rx=0 tx=0).stop 2024-08-18T23:42:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:22.097+0000 7f83a4a5c640 1 -- 172.21.15.78:0/1541290549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83a010bea0 msgr2=0x7f83a019ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:22.097+0000 7f83a4a5c640 1 --2- 172.21.15.78:0/1541290549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83a010bea0 0x7f83a019ca90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f839800b970 tx=0x7f839800be30 comp rx=0 tx=0).stop 2024-08-18T23:42:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:22.097+0000 7f83a4a5c640 1 -- 172.21.15.78:0/1541290549 shutdown_connections 2024-08-18T23:42:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:22.097+0000 7f83a4a5c640 1 --2- 172.21.15.78:0/1541290549 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f838c0a2c10 0x7f838c0a50d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:22.097+0000 7f83a4a5c640 1 --2- 172.21.15.78:0/1541290549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83a010bea0 0x7f83a019ca90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:22.097+0000 7f83a4a5c640 1 --2- 172.21.15.78:0/1541290549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83a0103390 0x7f83a0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:22.097+0000 7f83a4a5c640 1 --2- 172.21.15.78:0/1541290549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83a0102a70 0x7f83a01981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:22.097+0000 7f83a4a5c640 1 -- 172.21.15.78:0/1541290549 >> 172.21.15.78:0/1541290549 conn(0x7f83a00fc4d0 msgr2=0x7f83a00fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:22.097+0000 7f83a4a5c640 1 -- 172.21.15.78:0/1541290549 shutdown_connections 2024-08-18T23:42:22.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:22.097+0000 7f83a4a5c640 1 -- 172.21.15.78:0/1541290549 wait complete. 2024-08-18T23:42:22.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-08-18T23:42:22.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-08-18T23:42:22.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-08-18T23:42:22.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-08-18T23:42:22.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-08-18T23:42:22.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-08-18T23:42:22.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2024-08-18T23:42:22.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-08-18T23:42:22.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-08-18T23:42:22.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:22 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2024-08-18T23:42:22.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:22 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1541290549' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:42:22.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:22 smithi078 ceph-mon[18440]: pgmap v288: 129 pgs: 129 active+clean; 629 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2024-08-18T23:42:22.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-08-18T23:42:22.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-08-18T23:42:22.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-08-18T23:42:22.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-08-18T23:42:22.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-08-18T23:42:22.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-08-18T23:42:22.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2024-08-18T23:42:22.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-08-18T23:42:22.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-08-18T23:42:22.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:22 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2024-08-18T23:42:22.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:22 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1541290549' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:42:22.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:22 smithi161 ceph-mon[22027]: pgmap v288: 129 pgs: 129 active+clean; 629 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2024-08-18T23:42:22.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-08-18T23:42:22.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-08-18T23:42:22.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-08-18T23:42:22.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-08-18T23:42:22.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-08-18T23:42:22.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-08-18T23:42:22.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2024-08-18T23:42:22.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-08-18T23:42:22.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-08-18T23:42:22.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:22 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2024-08-18T23:42:22.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:22 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1541290549' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:42:22.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:22 smithi123 ceph-mon[21808]: pgmap v288: 129 pgs: 129 active+clean; 629 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2024-08-18T23:42:22.941 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-18T23:42:22.942 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-18T23:42:23.191 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:23.615 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-08-18T23:42:23.615 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-08-18T23:42:23.615 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-08-18T23:42:23.615 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-08-18T23:42:23.615 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-08-18T23:42:23.615 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-08-18T23:42:23.615 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2024-08-18T23:42:23.615 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-08-18T23:42:23.615 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-08-18T23:42:23.615 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2024-08-18T23:42:23.615 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:23 smithi078 ceph-mon[18440]: osdmap e89: 12 total, 12 up, 12 in 2024-08-18T23:42:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-08-18T23:42:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-08-18T23:42:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-08-18T23:42:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-08-18T23:42:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-08-18T23:42:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-08-18T23:42:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2024-08-18T23:42:23.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-08-18T23:42:23.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-08-18T23:42:23.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2024-08-18T23:42:23.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:23 smithi161 ceph-mon[22027]: osdmap e89: 12 total, 12 up, 12 in 2024-08-18T23:42:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.692+0000 7f0dc432b640 1 -- 172.21.15.78:0/1372198778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dbc100b50 msgr2=0x7f0dbc100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.692+0000 7f0dc432b640 1 --2- 172.21.15.78:0/1372198778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dbc100b50 0x7f0dbc100f30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009900 tx=0x7f0db802a480 comp rx=0 tx=0).stop 2024-08-18T23:42:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.692+0000 7f0dc432b640 1 -- 172.21.15.78:0/1372198778 shutdown_connections 2024-08-18T23:42:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.692+0000 7f0dc432b640 1 --2- 172.21.15.78:0/1372198778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dbc102c20 0x7f0dbc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.692+0000 7f0dc432b640 1 --2- 172.21.15.78:0/1372198778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0dbc1022a0 0x7f0dbc1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.692+0000 7f0dc432b640 1 --2- 172.21.15.78:0/1372198778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dbc100b50 0x7f0dbc100f30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.692+0000 7f0dc432b640 1 -- 172.21.15.78:0/1372198778 >> 172.21.15.78:0/1372198778 conn(0x7f0dbc0fc500 msgr2=0x7f0dbc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.692+0000 7f0dc432b640 1 -- 172.21.15.78:0/1372198778 shutdown_connections 2024-08-18T23:42:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.692+0000 7f0dc432b640 1 -- 172.21.15.78:0/1372198778 wait complete. 2024-08-18T23:42:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc432b640 1 Processor -- start 2024-08-18T23:42:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc432b640 1 -- start start 2024-08-18T23:42:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc432b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dbc100b50 0x7f0dbc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc432b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0dbc1022a0 0x7f0dbc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:23.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc432b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dbc102c20 0x7f0dbc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:23.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc432b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc110d50 con 0x7f0dbc100b50 2024-08-18T23:42:23.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc432b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc110bd0 con 0x7f0dbc102c20 2024-08-18T23:42:23.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc432b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc110ed0 con 0x7f0dbc1022a0 2024-08-18T23:42:23.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc20c7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dbc100b50 0x7f0dbc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:23.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc20c7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dbc100b50 0x7f0dbc1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36566/0 (socket says 172.21.15.78:36566) 2024-08-18T23:42:23.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc20c7640 1 -- 172.21.15.78:0/1072678627 learned_addr learned my addr 172.21.15.78:0/1072678627 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:23.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc18c6640 1 --2- 172.21.15.78:0/1072678627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0dbc1022a0 0x7f0dbc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:23.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc20c7640 1 -- 172.21.15.78:0/1072678627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0dbc1022a0 msgr2=0x7f0dbc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:23.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc20c7640 1 --2- 172.21.15.78:0/1072678627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0dbc1022a0 0x7f0dbc198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:23.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc20c7640 1 -- 172.21.15.78:0/1072678627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dbc102c20 msgr2=0x7f0dbc19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:23.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc28c8640 1 --2- 172.21.15.78:0/1072678627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dbc102c20 0x7f0dbc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:23.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc20c7640 1 --2- 172.21.15.78:0/1072678627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dbc102c20 0x7f0dbc19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:23.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc20c7640 1 -- 172.21.15.78:0/1072678627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dbc19d380 con 0x7f0dbc100b50 2024-08-18T23:42:23.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc18c6640 1 --2- 172.21.15.78:0/1072678627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0dbc1022a0 0x7f0dbc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:42:23.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.693+0000 7f0dc28c8640 1 --2- 172.21.15.78:0/1072678627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dbc102c20 0x7f0dbc19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:42:23.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.694+0000 7f0dc20c7640 1 --2- 172.21.15.78:0/1072678627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dbc100b50 0x7f0dbc1983d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0db8006010 tx=0x7f0db8003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:23.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.694+0000 7f0db37fe640 1 -- 172.21.15.78:0/1072678627 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db803f070 con 0x7f0dbc100b50 2024-08-18T23:42:23.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.694+0000 7f0db37fe640 1 -- 172.21.15.78:0/1072678627 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0db8031e30 con 0x7f0dbc100b50 2024-08-18T23:42:23.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.694+0000 7f0dc432b640 1 -- 172.21.15.78:0/1072678627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dbc19d610 con 0x7f0dbc100b50 2024-08-18T23:42:23.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.694+0000 7f0db37fe640 1 -- 172.21.15.78:0/1072678627 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8035030 con 0x7f0dbc100b50 2024-08-18T23:42:23.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.694+0000 7f0dc432b640 1 -- 172.21.15.78:0/1072678627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dbc1a4f50 con 0x7f0dbc100b50 2024-08-18T23:42:23.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.695+0000 7f0dc432b640 1 -- 172.21.15.78:0/1072678627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dbc100fb0 con 0x7f0dbc100b50 2024-08-18T23:42:23.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.696+0000 7f0db37fe640 1 -- 172.21.15.78:0/1072678627 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0db8042070 con 0x7f0dbc100b50 2024-08-18T23:42:23.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.697+0000 7f0db37fe640 1 --2- 172.21.15.78:0/1072678627 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0da80a2c10 0x7f0da80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:23.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.697+0000 7f0dc18c6640 1 --2- 172.21.15.78:0/1072678627 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0da80a2c10 0x7f0da80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:23.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.697+0000 7f0db37fe640 1 -- 172.21.15.78:0/1072678627 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f0db80f04d0 con 0x7f0dbc100b50 2024-08-18T23:42:23.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.697+0000 7f0dc18c6640 1 --2- 172.21.15.78:0/1072678627 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0da80a2c10 0x7f0da80a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0dac006fd0 tx=0x7f0dac008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:23.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.700+0000 7f0db37fe640 1 -- 172.21.15.78:0/1072678627 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0db80bd000 con 0x7f0dbc100b50 2024-08-18T23:42:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-08-18T23:42:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-08-18T23:42:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-08-18T23:42:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-08-18T23:42:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-08-18T23:42:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-08-18T23:42:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2024-08-18T23:42:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-08-18T23:42:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-08-18T23:42:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2024-08-18T23:42:23.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:23 smithi123 ceph-mon[21808]: osdmap e89: 12 total, 12 up, 12 in 2024-08-18T23:42:23.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.863+0000 7f0dc432b640 1 -- 172.21.15.78:0/1072678627 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0dbc062360 con 0x7f0da80a2c10 2024-08-18T23:42:23.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.871+0000 7f0db37fe640 1 -- 172.21.15.78:0/1072678627 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340291 (secure 0 0 0) 0x7f0dbc062360 con 0x7f0da80a2c10 2024-08-18T23:42:23.876 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:42:23.878 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-08-18T23:42:23.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.875+0000 7f0dc432b640 1 -- 172.21.15.78:0/1072678627 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0da80a2c10 msgr2=0x7f0da80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:23.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.875+0000 7f0dc432b640 1 --2- 172.21.15.78:0/1072678627 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0da80a2c10 0x7f0da80a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0dac006fd0 tx=0x7f0dac008040 comp rx=0 tx=0).stop 2024-08-18T23:42:23.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.875+0000 7f0dc432b640 1 -- 172.21.15.78:0/1072678627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dbc100b50 msgr2=0x7f0dbc1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:23.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.875+0000 7f0dc432b640 1 --2- 172.21.15.78:0/1072678627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dbc100b50 0x7f0dbc1983d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0db8006010 tx=0x7f0db8003c80 comp rx=0 tx=0).stop 2024-08-18T23:42:23.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.875+0000 7f0dc432b640 1 -- 172.21.15.78:0/1072678627 shutdown_connections 2024-08-18T23:42:23.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.875+0000 7f0dc432b640 1 --2- 172.21.15.78:0/1072678627 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0da80a2c10 0x7f0da80a50d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:23.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.875+0000 7f0dc432b640 1 --2- 172.21.15.78:0/1072678627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dbc102c20 0x7f0dbc19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:23.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.875+0000 7f0dc432b640 1 --2- 172.21.15.78:0/1072678627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0dbc1022a0 0x7f0dbc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:23.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.875+0000 7f0dc432b640 1 --2- 172.21.15.78:0/1072678627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dbc100b50 0x7f0dbc1983d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:23.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.875+0000 7f0dc432b640 1 -- 172.21.15.78:0/1072678627 >> 172.21.15.78:0/1072678627 conn(0x7f0dbc0fc500 msgr2=0x7f0dbc069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:23.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.875+0000 7f0dc432b640 1 -- 172.21.15.78:0/1072678627 shutdown_connections 2024-08-18T23:42:23.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:23.875+0000 7f0dc432b640 1 -- 172.21.15.78:0/1072678627 wait complete. 2024-08-18T23:42:24.538 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-08-18T23:42:23.312851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644084,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":225,"num_read_kb":215,"num_write":340,"num_write_kb":3040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387728,"kb_used_data":13616,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554096,"statfs":{"total":1151940427776,"available":1151543394304,"internally_reserved":0,"allocated":13942784,"data_stored":7741286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46265,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":78,"num_write":30,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.194542"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:07.263169+0000","last_change":"2024-08-18T23:41:30.120580+0000","last_active":"2024-08-18T23:42:07.263169+0000","last_peered":"2024-08-18T23:42:07.263169+0000","last_clean":"2024-08-18T23:42:07.263169+0000","last_became_active":"2024-08-18T23:41:30.120266+0000","last_became_peered":"2024-08-18T23:41:30.120266+0000","last_unstale":"2024-08-18T23:42:07.263169+0000","last_undegraded":"2024-08-18T23:42:07.263169+0000","last_fullsized":"2024-08-18T23:42:07.263169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:06:40.997070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436535+0000","last_change":"2024-08-18T23:41:32.143232+0000","last_active":"2024-08-18T23:41:42.436535+0000","last_peered":"2024-08-18T23:41:42.436535+0000","last_clean":"2024-08-18T23:41:42.436535+0000","last_became_active":"2024-08-18T23:41:32.142916+0000","last_became_peered":"2024-08-18T23:41:32.142916+0000","last_unstale":"2024-08-18T23:41:42.436535+0000","last_undegraded":"2024-08-18T23:41:42.436535+0000","last_fullsized":"2024-08-18T23:41:42.436535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:50:54.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438428+0000","last_change":"2024-08-18T23:41:39.188772+0000","last_active":"2024-08-18T23:41:42.438428+0000","last_peered":"2024-08-18T23:41:42.438428+0000","last_clean":"2024-08-18T23:41:42.438428+0000","last_became_active":"2024-08-18T23:41:39.188362+0000","last_became_peered":"2024-08-18T23:41:39.188362+0000","last_unstale":"2024-08-18T23:41:42.438428+0000","last_undegraded":"2024-08-18T23:41:42.438428+0000","last_fullsized":"2024-08-18T23:41:42.438428+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:47:50.255543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436883+0000","last_change":"2024-08-18T23:41:30.137179+0000","last_active":"2024-08-18T23:41:42.436883+0000","last_peered":"2024-08-18T23:41:42.436883+0000","last_clean":"2024-08-18T23:41:42.436883+0000","last_became_active":"2024-08-18T23:41:30.136618+0000","last_became_peered":"2024-08-18T23:41:30.136618+0000","last_unstale":"2024-08-18T23:41:42.436883+0000","last_undegraded":"2024-08-18T23:41:42.436883+0000","last_fullsized":"2024-08-18T23:41:42.436883+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:59.826364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437188+0000","last_change":"2024-08-18T23:41:32.142075+0000","last_active":"2024-08-18T23:41:42.437188+0000","last_peered":"2024-08-18T23:41:42.437188+0000","last_clean":"2024-08-18T23:41:42.437188+0000","last_became_active":"2024-08-18T23:41:32.141709+0000","last_became_peered":"2024-08-18T23:41:32.141709+0000","last_unstale":"2024-08-18T23:41:42.437188+0000","last_undegraded":"2024-08-18T23:41:42.437188+0000","last_fullsized":"2024-08-18T23:41:42.437188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:08.324627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437258+0000","last_change":"2024-08-18T23:41:39.184575+0000","last_active":"2024-08-18T23:41:42.437258+0000","last_peered":"2024-08-18T23:41:42.437258+0000","last_clean":"2024-08-18T23:41:42.437258+0000","last_became_active":"2024-08-18T23:41:39.184169+0000","last_became_peered":"2024-08-18T23:41:39.184169+0000","last_unstale":"2024-08-18T23:41:42.437258+0000","last_undegraded":"2024-08-18T23:41:42.437258+0000","last_fullsized":"2024-08-18T23:41:42.437258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:32.218172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:07.261289+0000","last_change":"2024-08-18T23:41:30.121638+0000","last_active":"2024-08-18T23:42:07.261289+0000","last_peered":"2024-08-18T23:42:07.261289+0000","last_clean":"2024-08-18T23:42:07.261289+0000","last_became_active":"2024-08-18T23:41:30.121290+0000","last_became_peered":"2024-08-18T23:41:30.121290+0000","last_unstale":"2024-08-18T23:42:07.261289+0000","last_undegraded":"2024-08-18T23:42:07.261289+0000","last_fullsized":"2024-08-18T23:42:07.261289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:43:55.638780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437662+0000","last_change":"2024-08-18T23:41:32.142177+0000","last_active":"2024-08-18T23:41:42.437662+0000","last_peered":"2024-08-18T23:41:42.437662+0000","last_clean":"2024-08-18T23:41:42.437662+0000","last_became_active":"2024-08-18T23:41:32.141731+0000","last_became_peered":"2024-08-18T23:41:32.141731+0000","last_unstale":"2024-08-18T23:41:42.437662+0000","last_undegraded":"2024-08-18T23:41:42.437662+0000","last_fullsized":"2024-08-18T23:41:42.437662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:36.283299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438115+0000","last_change":"2024-08-18T23:41:39.182931+0000","last_active":"2024-08-18T23:41:42.438115+0000","last_peered":"2024-08-18T23:41:42.438115+0000","last_clean":"2024-08-18T23:41:42.438115+0000","last_became_active":"2024-08-18T23:41:39.182008+0000","last_became_peered":"2024-08-18T23:41:39.182008+0000","last_unstale":"2024-08-18T23:41:42.438115+0000","last_undegraded":"2024-08-18T23:41:42.438115+0000","last_fullsized":"2024-08-18T23:41:42.438115+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:58:38.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":2,"reported_epoch":89,"state":"peering","last_fresh":"2024-08-18T23:41:33.156065+0000","last_change":"2024-08-18T23:42:22.476106+0000","last_active":"2024-08-18T23:42:22.476106+0000","last_peered":"2024-08-18T23:41:33.156065+0000","last_clean":"2024-08-18T23:41:33.156065+0000","last_became_active":"2024-08-18T23:41:30.119986+0000","last_became_peered":"2024-08-18T23:41:30.119986+0000","last_unstale":"2024-08-18T23:41:33.156065+0000","last_undegraded":"2024-08-18T23:41:33.156065+0000","last_fullsized":"2024-08-18T23:41:33.156065+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438033+0000","last_change":"2024-08-18T23:41:32.148662+0000","last_active":"2024-08-18T23:41:42.438033+0000","last_peered":"2024-08-18T23:41:42.438033+0000","last_clean":"2024-08-18T23:41:42.438033+0000","last_became_active":"2024-08-18T23:41:32.148318+0000","last_became_peered":"2024-08-18T23:41:32.148318+0000","last_unstale":"2024-08-18T23:41:42.438033+0000","last_undegraded":"2024-08-18T23:41:42.438033+0000","last_fullsized":"2024-08-18T23:41:42.438033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:58.668669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437251+0000","last_change":"2024-08-18T23:41:39.181114+0000","last_active":"2024-08-18T23:41:42.437251+0000","last_peered":"2024-08-18T23:41:42.437251+0000","last_clean":"2024-08-18T23:41:42.437251+0000","last_became_active":"2024-08-18T23:41:39.180599+0000","last_became_peered":"2024-08-18T23:41:39.180599+0000","last_unstale":"2024-08-18T23:41:42.437251+0000","last_undegraded":"2024-08-18T23:41:42.437251+0000","last_fullsized":"2024-08-18T23:41:42.437251+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:36:08.749143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437425+0000","last_change":"2024-08-18T23:41:32.144326+0000","last_active":"2024-08-18T23:41:42.437425+0000","last_peered":"2024-08-18T23:41:42.437425+0000","last_clean":"2024-08-18T23:41:42.437425+0000","last_became_active":"2024-08-18T23:41:32.144004+0000","last_became_peered":"2024-08-18T23:41:32.144004+0000","last_unstale":"2024-08-18T23:41:42.437425+0000","last_undegraded":"2024-08-18T23:41:42.437425+0000","last_fullsized":"2024-08-18T23:41:42.437425+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:10.918624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:06.259302+0000","last_change":"2024-08-18T23:41:30.125160+0000","last_active":"2024-08-18T23:42:06.259302+0000","last_peered":"2024-08-18T23:42:06.259302+0000","last_clean":"2024-08-18T23:42:06.259302+0000","last_became_active":"2024-08-18T23:41:30.124685+0000","last_became_peered":"2024-08-18T23:41:30.124685+0000","last_unstale":"2024-08-18T23:42:06.259302+0000","last_undegraded":"2024-08-18T23:42:06.259302+0000","last_fullsized":"2024-08-18T23:42:06.259302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:52:48.997902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437021+0000","last_change":"2024-08-18T23:41:39.681481+0000","last_active":"2024-08-18T23:41:42.437021+0000","last_peered":"2024-08-18T23:41:42.437021+0000","last_clean":"2024-08-18T23:41:42.437021+0000","last_became_active":"2024-08-18T23:41:39.680204+0000","last_became_peered":"2024-08-18T23:41:39.680204+0000","last_unstale":"2024-08-18T23:41:42.437021+0000","last_undegraded":"2024-08-18T23:41:42.437021+0000","last_fullsized":"2024-08-18T23:41:42.437021+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:00:05.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436405+0000","last_change":"2024-08-18T23:41:32.151335+0000","last_active":"2024-08-18T23:41:42.436405+0000","last_peered":"2024-08-18T23:41:42.436405+0000","last_clean":"2024-08-18T23:41:42.436405+0000","last_became_active":"2024-08-18T23:41:32.151040+0000","last_became_peered":"2024-08-18T23:41:32.151040+0000","last_unstale":"2024-08-18T23:41:42.436405+0000","last_undegraded":"2024-08-18T23:41:42.436405+0000","last_fullsized":"2024-08-18T23:41:42.436405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:16.108555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436860+0000","last_change":"2024-08-18T23:41:30.122301+0000","last_active":"2024-08-18T23:41:42.436860+0000","last_peered":"2024-08-18T23:41:42.436860+0000","last_clean":"2024-08-18T23:41:42.436860+0000","last_became_active":"2024-08-18T23:41:30.121117+0000","last_became_peered":"2024-08-18T23:41:30.121117+0000","last_unstale":"2024-08-18T23:41:42.436860+0000","last_undegraded":"2024-08-18T23:41:42.436860+0000","last_fullsized":"2024-08-18T23:41:42.436860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:40:12.678803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437653+0000","last_change":"2024-08-18T23:41:39.182395+0000","last_active":"2024-08-18T23:41:42.437653+0000","last_peered":"2024-08-18T23:41:42.437653+0000","last_clean":"2024-08-18T23:41:42.437653+0000","last_became_active":"2024-08-18T23:41:39.181979+0000","last_became_peered":"2024-08-18T23:41:39.181979+0000","last_unstale":"2024-08-18T23:41:42.437653+0000","last_undegraded":"2024-08-18T23:41:42.437653+0000","last_fullsized":"2024-08-18T23:41:42.437653+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:04.674503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.310565+0000","last_change":"2024-08-18T23:41:32.140464+0000","last_active":"2024-08-18T23:42:22.310565+0000","last_peered":"2024-08-18T23:42:22.310565+0000","last_clean":"2024-08-18T23:42:22.310565+0000","last_became_active":"2024-08-18T23:41:32.139153+0000","last_became_peered":"2024-08-18T23:41:32.139153+0000","last_unstale":"2024-08-18T23:42:22.310565+0000","last_undegraded":"2024-08-18T23:42:22.310565+0000","last_fullsized":"2024-08-18T23:42:22.310565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:27.387680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:06.256815+0000","last_change":"2024-08-18T23:41:30.122787+0000","last_active":"2024-08-18T23:42:06.256815+0000","last_peered":"2024-08-18T23:42:06.256815+0000","last_clean":"2024-08-18T23:42:06.256815+0000","last_became_active":"2024-08-18T23:41:30.122462+0000","last_became_peered":"2024-08-18T23:41:30.122462+0000","last_unstale":"2024-08-18T23:42:06.256815+0000","last_undegraded":"2024-08-18T23:42:06.256815+0000","last_fullsized":"2024-08-18T23:42:06.256815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:55.568831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.184346+0000","last_change":"2024-08-18T23:41:39.190511+0000","last_active":"2024-08-18T23:41:42.184346+0000","last_peered":"2024-08-18T23:41:42.184346+0000","last_clean":"2024-08-18T23:41:42.184346+0000","last_became_active":"2024-08-18T23:41:39.190154+0000","last_became_peered":"2024-08-18T23:41:39.190154+0000","last_unstale":"2024-08-18T23:41:42.184346+0000","last_undegraded":"2024-08-18T23:41:42.184346+0000","last_fullsized":"2024-08-18T23:41:42.184346+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:33:34.514827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436720+0000","last_change":"2024-08-18T23:41:32.143867+0000","last_active":"2024-08-18T23:41:42.436720+0000","last_peered":"2024-08-18T23:41:42.436720+0000","last_clean":"2024-08-18T23:41:42.436720+0000","last_became_active":"2024-08-18T23:41:32.143527+0000","last_became_peered":"2024-08-18T23:41:32.143527+0000","last_unstale":"2024-08-18T23:41:42.436720+0000","last_undegraded":"2024-08-18T23:41:42.436720+0000","last_fullsized":"2024-08-18T23:41:42.436720+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:40:25.902133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:06.257456+0000","last_change":"2024-08-18T23:41:30.136519+0000","last_active":"2024-08-18T23:42:06.257456+0000","last_peered":"2024-08-18T23:42:06.257456+0000","last_clean":"2024-08-18T23:42:06.257456+0000","last_became_active":"2024-08-18T23:41:30.136193+0000","last_became_peered":"2024-08-18T23:41:30.136193+0000","last_unstale":"2024-08-18T23:42:06.257456+0000","last_undegraded":"2024-08-18T23:42:06.257456+0000","last_fullsized":"2024-08-18T23:42:06.257456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:25:18.061897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436916+0000","last_change":"2024-08-18T23:41:39.680848+0000","last_active":"2024-08-18T23:41:42.436916+0000","last_peered":"2024-08-18T23:41:42.436916+0000","last_clean":"2024-08-18T23:41:42.436916+0000","last_became_active":"2024-08-18T23:41:39.680375+0000","last_became_peered":"2024-08-18T23:41:39.680375+0000","last_unstale":"2024-08-18T23:41:42.436916+0000","last_undegraded":"2024-08-18T23:41:42.436916+0000","last_fullsized":"2024-08-18T23:41:42.436916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:50.041110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436985+0000","last_change":"2024-08-18T23:41:32.139025+0000","last_active":"2024-08-18T23:41:42.436985+0000","last_peered":"2024-08-18T23:41:42.436985+0000","last_clean":"2024-08-18T23:41:42.436985+0000","last_became_active":"2024-08-18T23:41:32.138733+0000","last_became_peered":"2024-08-18T23:41:32.138733+0000","last_unstale":"2024-08-18T23:41:42.436985+0000","last_undegraded":"2024-08-18T23:41:42.436985+0000","last_fullsized":"2024-08-18T23:41:42.436985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:01:39.826391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:06.259829+0000","last_change":"2024-08-18T23:41:30.477359+0000","last_active":"2024-08-18T23:42:06.259829+0000","last_peered":"2024-08-18T23:42:06.259829+0000","last_clean":"2024-08-18T23:42:06.259829+0000","last_became_active":"2024-08-18T23:41:30.477011+0000","last_became_peered":"2024-08-18T23:41:30.477011+0000","last_unstale":"2024-08-18T23:42:06.259829+0000","last_undegraded":"2024-08-18T23:42:06.259829+0000","last_fullsized":"2024-08-18T23:42:06.259829+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:53:10.052859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438585+0000","last_change":"2024-08-18T23:41:39.182140+0000","last_active":"2024-08-18T23:41:42.438585+0000","last_peered":"2024-08-18T23:41:42.438585+0000","last_clean":"2024-08-18T23:41:42.438585+0000","last_became_active":"2024-08-18T23:41:39.181667+0000","last_became_peered":"2024-08-18T23:41:39.181667+0000","last_unstale":"2024-08-18T23:41:42.438585+0000","last_undegraded":"2024-08-18T23:41:42.438585+0000","last_fullsized":"2024-08-18T23:41:42.438585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:02:18.128028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436969+0000","last_change":"2024-08-18T23:41:32.137064+0000","last_active":"2024-08-18T23:41:42.436969+0000","last_peered":"2024-08-18T23:41:42.436969+0000","last_clean":"2024-08-18T23:41:42.436969+0000","last_became_active":"2024-08-18T23:41:32.136738+0000","last_became_peered":"2024-08-18T23:41:32.136738+0000","last_unstale":"2024-08-18T23:41:42.436969+0000","last_undegraded":"2024-08-18T23:41:42.436969+0000","last_fullsized":"2024-08-18T23:41:42.436969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:15:11.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'9","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:19.153483+0000","last_change":"2024-08-18T23:41:30.120042+0000","last_active":"2024-08-18T23:42:19.153483+0000","last_peered":"2024-08-18T23:42:19.153483+0000","last_clean":"2024-08-18T23:42:19.153483+0000","last_became_active":"2024-08-18T23:41:30.119719+0000","last_became_peered":"2024-08-18T23:41:30.119719+0000","last_unstale":"2024-08-18T23:42:19.153483+0000","last_undegraded":"2024-08-18T23:42:19.153483+0000","last_fullsized":"2024-08-18T23:42:19.153483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:35:44.466330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436182+0000","last_change":"2024-08-18T23:41:39.680405+0000","last_active":"2024-08-18T23:41:42.436182+0000","last_peered":"2024-08-18T23:41:42.436182+0000","last_clean":"2024-08-18T23:41:42.436182+0000","last_became_active":"2024-08-18T23:41:39.679940+0000","last_became_peered":"2024-08-18T23:41:39.679940+0000","last_unstale":"2024-08-18T23:41:42.436182+0000","last_undegraded":"2024-08-18T23:41:42.436182+0000","last_fullsized":"2024-08-18T23:41:42.436182+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:15:03.942116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481785+0000","last_change":"2024-08-18T23:41:32.139688+0000","last_active":"2024-08-18T23:41:42.481785+0000","last_peered":"2024-08-18T23:41:42.481785+0000","last_clean":"2024-08-18T23:41:42.481785+0000","last_became_active":"2024-08-18T23:41:32.139397+0000","last_became_peered":"2024-08-18T23:41:32.139397+0000","last_unstale":"2024-08-18T23:41:42.481785+0000","last_undegraded":"2024-08-18T23:41:42.481785+0000","last_fullsized":"2024-08-18T23:41:42.481785+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:03:57.017488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:06.258771+0000","last_change":"2024-08-18T23:41:30.137578+0000","last_active":"2024-08-18T23:42:06.258771+0000","last_peered":"2024-08-18T23:42:06.258771+0000","last_clean":"2024-08-18T23:42:06.258771+0000","last_became_active":"2024-08-18T23:41:30.137246+0000","last_became_peered":"2024-08-18T23:41:30.137246+0000","last_unstale":"2024-08-18T23:42:06.258771+0000","last_undegraded":"2024-08-18T23:42:06.258771+0000","last_fullsized":"2024-08-18T23:42:06.258771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:48:54.839432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.309910+0000","last_change":"2024-08-18T23:41:39.186360+0000","last_active":"2024-08-18T23:42:22.309910+0000","last_peered":"2024-08-18T23:42:22.309910+0000","last_clean":"2024-08-18T23:42:22.309910+0000","last_became_active":"2024-08-18T23:41:39.185808+0000","last_became_peered":"2024-08-18T23:41:39.185808+0000","last_unstale":"2024-08-18T23:42:22.309910+0000","last_undegraded":"2024-08-18T23:42:22.309910+0000","last_fullsized":"2024-08-18T23:42:22.309910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:53.891061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.181975+0000","last_change":"2024-08-18T23:41:32.146070+0000","last_active":"2024-08-18T23:41:42.181975+0000","last_peered":"2024-08-18T23:41:42.181975+0000","last_clean":"2024-08-18T23:41:42.181975+0000","last_became_active":"2024-08-18T23:41:32.145640+0000","last_became_peered":"2024-08-18T23:41:32.145640+0000","last_unstale":"2024-08-18T23:41:42.181975+0000","last_undegraded":"2024-08-18T23:41:42.181975+0000","last_fullsized":"2024-08-18T23:41:42.181975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:56:21.368762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.184447+0000","last_change":"2024-08-18T23:41:30.138565+0000","last_active":"2024-08-18T23:41:42.184447+0000","last_peered":"2024-08-18T23:41:42.184447+0000","last_clean":"2024-08-18T23:41:42.184447+0000","last_became_active":"2024-08-18T23:41:30.138210+0000","last_became_peered":"2024-08-18T23:41:30.138210+0000","last_unstale":"2024-08-18T23:41:42.184447+0000","last_undegraded":"2024-08-18T23:41:42.184447+0000","last_fullsized":"2024-08-18T23:41:42.184447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:42:14.928884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437024+0000","last_change":"2024-08-18T23:41:39.680656+0000","last_active":"2024-08-18T23:41:42.437024+0000","last_peered":"2024-08-18T23:41:42.437024+0000","last_clean":"2024-08-18T23:41:42.437024+0000","last_became_active":"2024-08-18T23:41:39.680203+0000","last_became_peered":"2024-08-18T23:41:39.680203+0000","last_unstale":"2024-08-18T23:41:42.437024+0000","last_undegraded":"2024-08-18T23:41:42.437024+0000","last_fullsized":"2024-08-18T23:41:42.437024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:55:43.138869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.181940+0000","last_change":"2024-08-18T23:41:32.130222+0000","last_active":"2024-08-18T23:41:42.181940+0000","last_peered":"2024-08-18T23:41:42.181940+0000","last_clean":"2024-08-18T23:41:42.181940+0000","last_became_active":"2024-08-18T23:41:32.129844+0000","last_became_peered":"2024-08-18T23:41:32.129844+0000","last_unstale":"2024-08-18T23:41:42.181940+0000","last_undegraded":"2024-08-18T23:41:42.181940+0000","last_fullsized":"2024-08-18T23:41:42.181940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:39:21.106343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437643+0000","last_change":"2024-08-18T23:41:30.123142+0000","last_active":"2024-08-18T23:41:42.437643+0000","last_peered":"2024-08-18T23:41:42.437643+0000","last_clean":"2024-08-18T23:41:42.437643+0000","last_became_active":"2024-08-18T23:41:30.121645+0000","last_became_peered":"2024-08-18T23:41:30.121645+0000","last_unstale":"2024-08-18T23:41:42.437643+0000","last_undegraded":"2024-08-18T23:41:42.437643+0000","last_fullsized":"2024-08-18T23:41:42.437643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:07:24.912259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481697+0000","last_change":"2024-08-18T23:41:39.186033+0000","last_active":"2024-08-18T23:41:42.481697+0000","last_peered":"2024-08-18T23:41:42.481697+0000","last_clean":"2024-08-18T23:41:42.481697+0000","last_became_active":"2024-08-18T23:41:39.185451+0000","last_became_peered":"2024-08-18T23:41:39.185451+0000","last_unstale":"2024-08-18T23:41:42.481697+0000","last_undegraded":"2024-08-18T23:41:42.481697+0000","last_fullsized":"2024-08-18T23:41:42.481697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:52:13.556760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437517+0000","last_change":"2024-08-18T23:41:32.141065+0000","last_active":"2024-08-18T23:41:42.437517+0000","last_peered":"2024-08-18T23:41:42.437517+0000","last_clean":"2024-08-18T23:41:42.437517+0000","last_became_active":"2024-08-18T23:41:32.140732+0000","last_became_peered":"2024-08-18T23:41:32.140732+0000","last_unstale":"2024-08-18T23:41:42.437517+0000","last_undegraded":"2024-08-18T23:41:42.437517+0000","last_fullsized":"2024-08-18T23:41:42.437517+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:08:32.287835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:17.286692+0000","last_change":"2024-08-18T23:41:30.136838+0000","last_active":"2024-08-18T23:42:17.286692+0000","last_peered":"2024-08-18T23:42:17.286692+0000","last_clean":"2024-08-18T23:42:17.286692+0000","last_became_active":"2024-08-18T23:41:30.136523+0000","last_became_peered":"2024-08-18T23:41:30.136523+0000","last_unstale":"2024-08-18T23:42:17.286692+0000","last_undegraded":"2024-08-18T23:42:17.286692+0000","last_fullsized":"2024-08-18T23:42:17.286692+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:56:17.750535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481937+0000","last_change":"2024-08-18T23:41:39.680314+0000","last_active":"2024-08-18T23:41:42.481937+0000","last_peered":"2024-08-18T23:41:42.481937+0000","last_clean":"2024-08-18T23:41:42.481937+0000","last_became_active":"2024-08-18T23:41:39.679920+0000","last_became_peered":"2024-08-18T23:41:39.679920+0000","last_unstale":"2024-08-18T23:41:42.481937+0000","last_undegraded":"2024-08-18T23:41:42.481937+0000","last_fullsized":"2024-08-18T23:41:42.481937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:26:29.866081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481393+0000","last_change":"2024-08-18T23:41:32.139752+0000","last_active":"2024-08-18T23:41:42.481393+0000","last_peered":"2024-08-18T23:41:42.481393+0000","last_clean":"2024-08-18T23:41:42.481393+0000","last_became_active":"2024-08-18T23:41:32.139451+0000","last_became_peered":"2024-08-18T23:41:32.139451+0000","last_unstale":"2024-08-18T23:41:42.481393+0000","last_undegraded":"2024-08-18T23:41:42.481393+0000","last_fullsized":"2024-08-18T23:41:42.481393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:15:13.390529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:07.263221+0000","last_change":"2024-08-18T23:41:30.126513+0000","last_active":"2024-08-18T23:42:07.263221+0000","last_peered":"2024-08-18T23:42:07.263221+0000","last_clean":"2024-08-18T23:42:07.263221+0000","last_became_active":"2024-08-18T23:41:30.126046+0000","last_became_peered":"2024-08-18T23:41:30.126046+0000","last_unstale":"2024-08-18T23:42:07.263221+0000","last_undegraded":"2024-08-18T23:42:07.263221+0000","last_fullsized":"2024-08-18T23:42:07.263221+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:49:16.565565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438321+0000","last_change":"2024-08-18T23:41:39.681006+0000","last_active":"2024-08-18T23:41:42.438321+0000","last_peered":"2024-08-18T23:41:42.438321+0000","last_clean":"2024-08-18T23:41:42.438321+0000","last_became_active":"2024-08-18T23:41:39.680409+0000","last_became_peered":"2024-08-18T23:41:39.680409+0000","last_unstale":"2024-08-18T23:41:42.438321+0000","last_undegraded":"2024-08-18T23:41:42.438321+0000","last_fullsized":"2024-08-18T23:41:42.438321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:47.778866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436780+0000","last_change":"2024-08-18T23:41:32.147545+0000","last_active":"2024-08-18T23:41:42.436780+0000","last_peered":"2024-08-18T23:41:42.436780+0000","last_clean":"2024-08-18T23:41:42.436780+0000","last_became_active":"2024-08-18T23:41:32.147238+0000","last_became_peered":"2024-08-18T23:41:32.147238+0000","last_unstale":"2024-08-18T23:41:42.436780+0000","last_undegraded":"2024-08-18T23:41:42.436780+0000","last_fullsized":"2024-08-18T23:41:42.436780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:12:13.589535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:08.269170+0000","last_change":"2024-08-18T23:41:30.120650+0000","last_active":"2024-08-18T23:42:08.269170+0000","last_peered":"2024-08-18T23:42:08.269170+0000","last_clean":"2024-08-18T23:42:08.269170+0000","last_became_active":"2024-08-18T23:41:30.120196+0000","last_became_peered":"2024-08-18T23:41:30.120196+0000","last_unstale":"2024-08-18T23:42:08.269170+0000","last_undegraded":"2024-08-18T23:42:08.269170+0000","last_fullsized":"2024-08-18T23:42:08.269170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:40:23.370088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437295+0000","last_change":"2024-08-18T23:40:22.013519+0000","last_active":"2024-08-18T23:41:42.437295+0000","last_peered":"2024-08-18T23:41:42.437295+0000","last_clean":"2024-08-18T23:41:42.437295+0000","last_became_active":"2024-08-18T23:40:22.012986+0000","last_became_peered":"2024-08-18T23:40:22.012986+0000","last_unstale":"2024-08-18T23:41:42.437295+0000","last_undegraded":"2024-08-18T23:41:42.437295+0000","last_fullsized":"2024-08-18T23:41:42.437295+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_clean_scrub_stamp":"2024-08-18T23:37:22.388424+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:19.874356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438358+0000","last_change":"2024-08-18T23:41:39.188400+0000","last_active":"2024-08-18T23:41:42.438358+0000","last_peered":"2024-08-18T23:41:42.438358+0000","last_clean":"2024-08-18T23:41:42.438358+0000","last_became_active":"2024-08-18T23:41:39.188023+0000","last_became_peered":"2024-08-18T23:41:39.188023+0000","last_unstale":"2024-08-18T23:41:42.438358+0000","last_undegraded":"2024-08-18T23:41:42.438358+0000","last_fullsized":"2024-08-18T23:41:42.438358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:07:33.845809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438418+0000","last_change":"2024-08-18T23:41:32.139907+0000","last_active":"2024-08-18T23:41:42.438418+0000","last_peered":"2024-08-18T23:41:42.438418+0000","last_clean":"2024-08-18T23:41:42.438418+0000","last_became_active":"2024-08-18T23:41:32.137956+0000","last_became_peered":"2024-08-18T23:41:32.137956+0000","last_unstale":"2024-08-18T23:41:42.438418+0000","last_undegraded":"2024-08-18T23:41:42.438418+0000","last_fullsized":"2024-08-18T23:41:42.438418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:10:13.223328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:07.263532+0000","last_change":"2024-08-18T23:41:30.477857+0000","last_active":"2024-08-18T23:42:07.263532+0000","last_peered":"2024-08-18T23:42:07.263532+0000","last_clean":"2024-08-18T23:42:07.263532+0000","last_became_active":"2024-08-18T23:41:30.477376+0000","last_became_peered":"2024-08-18T23:41:30.477376+0000","last_unstale":"2024-08-18T23:42:07.263532+0000","last_undegraded":"2024-08-18T23:42:07.263532+0000","last_fullsized":"2024-08-18T23:42:07.263532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:57:08.059818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436966+0000","last_change":"2024-08-18T23:41:39.185917+0000","last_active":"2024-08-18T23:41:42.436966+0000","last_peered":"2024-08-18T23:41:42.436966+0000","last_clean":"2024-08-18T23:41:42.436966+0000","last_became_active":"2024-08-18T23:41:39.185427+0000","last_became_peered":"2024-08-18T23:41:39.185427+0000","last_unstale":"2024-08-18T23:41:42.436966+0000","last_undegraded":"2024-08-18T23:41:42.436966+0000","last_fullsized":"2024-08-18T23:41:42.436966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:52:13.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437928+0000","last_change":"2024-08-18T23:41:32.147530+0000","last_active":"2024-08-18T23:41:42.437928+0000","last_peered":"2024-08-18T23:41:42.437928+0000","last_clean":"2024-08-18T23:41:42.437928+0000","last_became_active":"2024-08-18T23:41:32.147216+0000","last_became_peered":"2024-08-18T23:41:32.147216+0000","last_unstale":"2024-08-18T23:41:42.437928+0000","last_undegraded":"2024-08-18T23:41:42.437928+0000","last_fullsized":"2024-08-18T23:41:42.437928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:04:06.421490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438376+0000","last_change":"2024-08-18T23:41:30.121307+0000","last_active":"2024-08-18T23:41:42.438376+0000","last_peered":"2024-08-18T23:41:42.438376+0000","last_clean":"2024-08-18T23:41:42.438376+0000","last_became_active":"2024-08-18T23:41:30.120932+0000","last_became_peered":"2024-08-18T23:41:30.120932+0000","last_unstale":"2024-08-18T23:41:42.438376+0000","last_undegraded":"2024-08-18T23:41:42.438376+0000","last_fullsized":"2024-08-18T23:41:42.438376+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:06:07.742880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436674+0000","last_change":"2024-08-18T23:41:39.184647+0000","last_active":"2024-08-18T23:41:42.436674+0000","last_peered":"2024-08-18T23:41:42.436674+0000","last_clean":"2024-08-18T23:41:42.436674+0000","last_became_active":"2024-08-18T23:41:39.184251+0000","last_became_peered":"2024-08-18T23:41:39.184251+0000","last_unstale":"2024-08-18T23:41:42.436674+0000","last_undegraded":"2024-08-18T23:41:42.436674+0000","last_fullsized":"2024-08-18T23:41:42.436674+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:55.200801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437485+0000","last_change":"2024-08-18T23:41:32.137007+0000","last_active":"2024-08-18T23:41:42.437485+0000","last_peered":"2024-08-18T23:41:42.437485+0000","last_clean":"2024-08-18T23:41:42.437485+0000","last_became_active":"2024-08-18T23:41:32.136684+0000","last_became_peered":"2024-08-18T23:41:32.136684+0000","last_unstale":"2024-08-18T23:41:42.437485+0000","last_undegraded":"2024-08-18T23:41:42.437485+0000","last_fullsized":"2024-08-18T23:41:42.437485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:21:26.584303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481898+0000","last_change":"2024-08-18T23:41:30.121183+0000","last_active":"2024-08-18T23:41:42.481898+0000","last_peered":"2024-08-18T23:41:42.481898+0000","last_clean":"2024-08-18T23:41:42.481898+0000","last_became_active":"2024-08-18T23:41:30.120847+0000","last_became_peered":"2024-08-18T23:41:30.120847+0000","last_unstale":"2024-08-18T23:41:42.481898+0000","last_undegraded":"2024-08-18T23:41:42.481898+0000","last_fullsized":"2024-08-18T23:41:42.481898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:51.025821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437287+0000","last_change":"2024-08-18T23:41:39.183375+0000","last_active":"2024-08-18T23:41:42.437287+0000","last_peered":"2024-08-18T23:41:42.437287+0000","last_clean":"2024-08-18T23:41:42.437287+0000","last_became_active":"2024-08-18T23:41:39.182339+0000","last_became_peered":"2024-08-18T23:41:39.182339+0000","last_unstale":"2024-08-18T23:41:42.437287+0000","last_undegraded":"2024-08-18T23:41:42.437287+0000","last_fullsized":"2024-08-18T23:41:42.437287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:26.119056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.313184+0000","last_change":"2024-08-18T23:41:32.140748+0000","last_active":"2024-08-18T23:42:22.313184+0000","last_peered":"2024-08-18T23:42:22.313184+0000","last_clean":"2024-08-18T23:42:22.313184+0000","last_became_active":"2024-08-18T23:41:32.140306+0000","last_became_peered":"2024-08-18T23:41:32.140306+0000","last_unstale":"2024-08-18T23:42:22.313184+0000","last_undegraded":"2024-08-18T23:42:22.313184+0000","last_fullsized":"2024-08-18T23:42:22.313184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:23:02.371529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:08.270449+0000","last_change":"2024-08-18T23:41:30.121617+0000","last_active":"2024-08-18T23:42:08.270449+0000","last_peered":"2024-08-18T23:42:08.270449+0000","last_clean":"2024-08-18T23:42:08.270449+0000","last_became_active":"2024-08-18T23:41:30.121257+0000","last_became_peered":"2024-08-18T23:41:30.121257+0000","last_unstale":"2024-08-18T23:42:08.270449+0000","last_undegraded":"2024-08-18T23:42:08.270449+0000","last_fullsized":"2024-08-18T23:42:08.270449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:23:42.091553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438472+0000","last_change":"2024-08-18T23:41:39.183833+0000","last_active":"2024-08-18T23:41:42.438472+0000","last_peered":"2024-08-18T23:41:42.438472+0000","last_clean":"2024-08-18T23:41:42.438472+0000","last_became_active":"2024-08-18T23:41:39.183176+0000","last_became_peered":"2024-08-18T23:41:39.183176+0000","last_unstale":"2024-08-18T23:41:42.438472+0000","last_undegraded":"2024-08-18T23:41:42.438472+0000","last_fullsized":"2024-08-18T23:41:42.438472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:36:09.827378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438671+0000","last_change":"2024-08-18T23:41:32.147507+0000","last_active":"2024-08-18T23:41:42.438671+0000","last_peered":"2024-08-18T23:41:42.438671+0000","last_clean":"2024-08-18T23:41:42.438671+0000","last_became_active":"2024-08-18T23:41:32.146042+0000","last_became_peered":"2024-08-18T23:41:32.146042+0000","last_unstale":"2024-08-18T23:41:42.438671+0000","last_undegraded":"2024-08-18T23:41:42.438671+0000","last_fullsized":"2024-08-18T23:41:42.438671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:48.941442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:07.263798+0000","last_change":"2024-08-18T23:41:30.137755+0000","last_active":"2024-08-18T23:42:07.263798+0000","last_peered":"2024-08-18T23:42:07.263798+0000","last_clean":"2024-08-18T23:42:07.263798+0000","last_became_active":"2024-08-18T23:41:30.137194+0000","last_became_peered":"2024-08-18T23:41:30.137194+0000","last_unstale":"2024-08-18T23:42:07.263798+0000","last_undegraded":"2024-08-18T23:42:07.263798+0000","last_fullsized":"2024-08-18T23:42:07.263798+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:15:49.681405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481932+0000","last_change":"2024-08-18T23:41:39.183986+0000","last_active":"2024-08-18T23:41:42.481932+0000","last_peered":"2024-08-18T23:41:42.481932+0000","last_clean":"2024-08-18T23:41:42.481932+0000","last_became_active":"2024-08-18T23:41:39.183384+0000","last_became_peered":"2024-08-18T23:41:39.183384+0000","last_unstale":"2024-08-18T23:41:42.481932+0000","last_undegraded":"2024-08-18T23:41:42.481932+0000","last_fullsized":"2024-08-18T23:41:42.481932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:04:28.920804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436902+0000","last_change":"2024-08-18T23:41:32.134729+0000","last_active":"2024-08-18T23:41:42.436902+0000","last_peered":"2024-08-18T23:41:42.436902+0000","last_clean":"2024-08-18T23:41:42.436902+0000","last_became_active":"2024-08-18T23:41:32.134366+0000","last_became_peered":"2024-08-18T23:41:32.134366+0000","last_unstale":"2024-08-18T23:41:42.436902+0000","last_undegraded":"2024-08-18T23:41:42.436902+0000","last_fullsized":"2024-08-18T23:41:42.436902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:38.610561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:18.288731+0000","last_change":"2024-08-18T23:41:30.126443+0000","last_active":"2024-08-18T23:42:18.288731+0000","last_peered":"2024-08-18T23:42:18.288731+0000","last_clean":"2024-08-18T23:42:18.288731+0000","last_became_active":"2024-08-18T23:41:30.124264+0000","last_became_peered":"2024-08-18T23:41:30.124264+0000","last_unstale":"2024-08-18T23:42:18.288731+0000","last_undegraded":"2024-08-18T23:42:18.288731+0000","last_fullsized":"2024-08-18T23:42:18.288731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:43.727114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.184435+0000","last_change":"2024-08-18T23:41:39.192164+0000","last_active":"2024-08-18T23:41:42.184435+0000","last_peered":"2024-08-18T23:41:42.184435+0000","last_clean":"2024-08-18T23:41:42.184435+0000","last_became_active":"2024-08-18T23:41:39.191833+0000","last_became_peered":"2024-08-18T23:41:39.191833+0000","last_unstale":"2024-08-18T23:41:42.184435+0000","last_undegraded":"2024-08-18T23:41:42.184435+0000","last_fullsized":"2024-08-18T23:41:42.184435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:33:50.243930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437783+0000","last_change":"2024-08-18T23:41:32.137384+0000","last_active":"2024-08-18T23:41:42.437783+0000","last_peered":"2024-08-18T23:41:42.437783+0000","last_clean":"2024-08-18T23:41:42.437783+0000","last_became_active":"2024-08-18T23:41:32.137051+0000","last_became_peered":"2024-08-18T23:41:32.137051+0000","last_unstale":"2024-08-18T23:41:42.437783+0000","last_undegraded":"2024-08-18T23:41:42.437783+0000","last_fullsized":"2024-08-18T23:41:42.437783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:22:04.367548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:06.259611+0000","last_change":"2024-08-18T23:41:30.124071+0000","last_active":"2024-08-18T23:42:06.259611+0000","last_peered":"2024-08-18T23:42:06.259611+0000","last_clean":"2024-08-18T23:42:06.259611+0000","last_became_active":"2024-08-18T23:41:30.123118+0000","last_became_peered":"2024-08-18T23:41:30.123118+0000","last_unstale":"2024-08-18T23:42:06.259611+0000","last_undegraded":"2024-08-18T23:42:06.259611+0000","last_fullsized":"2024-08-18T23:42:06.259611+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:21:39.015187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.310039+0000","last_change":"2024-08-18T23:41:39.184038+0000","last_active":"2024-08-18T23:42:22.310039+0000","last_peered":"2024-08-18T23:42:22.310039+0000","last_clean":"2024-08-18T23:42:22.310039+0000","last_became_active":"2024-08-18T23:41:39.183611+0000","last_became_peered":"2024-08-18T23:41:39.183611+0000","last_unstale":"2024-08-18T23:42:22.310039+0000","last_undegraded":"2024-08-18T23:42:22.310039+0000","last_fullsized":"2024-08-18T23:42:22.310039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:32:26.029151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481536+0000","last_change":"2024-08-18T23:41:32.143631+0000","last_active":"2024-08-18T23:41:42.481536+0000","last_peered":"2024-08-18T23:41:42.481536+0000","last_clean":"2024-08-18T23:41:42.481536+0000","last_became_active":"2024-08-18T23:41:32.143350+0000","last_became_peered":"2024-08-18T23:41:32.143350+0000","last_unstale":"2024-08-18T23:41:42.481536+0000","last_undegraded":"2024-08-18T23:41:42.481536+0000","last_fullsized":"2024-08-18T23:41:42.481536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:41:23.713467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:07.261492+0000","last_change":"2024-08-18T23:41:30.123383+0000","last_active":"2024-08-18T23:42:07.261492+0000","last_peered":"2024-08-18T23:42:07.261492+0000","last_clean":"2024-08-18T23:42:07.261492+0000","last_became_active":"2024-08-18T23:41:30.123037+0000","last_became_peered":"2024-08-18T23:41:30.123037+0000","last_unstale":"2024-08-18T23:42:07.261492+0000","last_undegraded":"2024-08-18T23:42:07.261492+0000","last_fullsized":"2024-08-18T23:42:07.261492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:27.702478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437947+0000","last_change":"2024-08-18T23:41:39.184302+0000","last_active":"2024-08-18T23:41:42.437947+0000","last_peered":"2024-08-18T23:41:42.437947+0000","last_clean":"2024-08-18T23:41:42.437947+0000","last_became_active":"2024-08-18T23:41:39.183172+0000","last_became_peered":"2024-08-18T23:41:39.183172+0000","last_unstale":"2024-08-18T23:41:42.437947+0000","last_undegraded":"2024-08-18T23:41:42.437947+0000","last_fullsized":"2024-08-18T23:41:42.437947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:23:26.655865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437282+0000","last_change":"2024-08-18T23:41:32.137426+0000","last_active":"2024-08-18T23:41:42.437282+0000","last_peered":"2024-08-18T23:41:42.437282+0000","last_clean":"2024-08-18T23:41:42.437282+0000","last_became_active":"2024-08-18T23:41:32.137106+0000","last_became_peered":"2024-08-18T23:41:32.137106+0000","last_unstale":"2024-08-18T23:41:42.437282+0000","last_undegraded":"2024-08-18T23:41:42.437282+0000","last_fullsized":"2024-08-18T23:41:42.437282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:34:51.889663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'26","reported_seq":86,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:19.291434+0000","last_change":"2024-08-18T23:41:30.124320+0000","last_active":"2024-08-18T23:42:19.291434+0000","last_peered":"2024-08-18T23:42:19.291434+0000","last_clean":"2024-08-18T23:42:19.291434+0000","last_became_active":"2024-08-18T23:41:30.123979+0000","last_became_peered":"2024-08-18T23:41:30.123979+0000","last_unstale":"2024-08-18T23:42:19.291434+0000","last_undegraded":"2024-08-18T23:42:19.291434+0000","last_fullsized":"2024-08-18T23:42:19.291434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:31:03.689720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49092,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":63,"num_write":27,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.184049+0000","last_change":"2024-08-18T23:41:39.190481+0000","last_active":"2024-08-18T23:41:42.184049+0000","last_peered":"2024-08-18T23:41:42.184049+0000","last_clean":"2024-08-18T23:41:42.184049+0000","last_became_active":"2024-08-18T23:41:39.190076+0000","last_became_peered":"2024-08-18T23:41:39.190076+0000","last_unstale":"2024-08-18T23:41:42.184049+0000","last_undegraded":"2024-08-18T23:41:42.184049+0000","last_fullsized":"2024-08-18T23:41:42.184049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:47:39.313142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437826+0000","last_change":"2024-08-18T23:41:32.145422+0000","last_active":"2024-08-18T23:41:42.437826+0000","last_peered":"2024-08-18T23:41:42.437826+0000","last_clean":"2024-08-18T23:41:42.437826+0000","last_became_active":"2024-08-18T23:41:32.145047+0000","last_became_peered":"2024-08-18T23:41:32.145047+0000","last_unstale":"2024-08-18T23:41:42.437826+0000","last_undegraded":"2024-08-18T23:41:42.437826+0000","last_fullsized":"2024-08-18T23:41:42.437826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:31.905248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:06.257992+0000","last_change":"2024-08-18T23:41:30.137364+0000","last_active":"2024-08-18T23:42:06.257992+0000","last_peered":"2024-08-18T23:42:06.257992+0000","last_clean":"2024-08-18T23:42:06.257992+0000","last_became_active":"2024-08-18T23:41:30.137004+0000","last_became_peered":"2024-08-18T23:41:30.137004+0000","last_unstale":"2024-08-18T23:42:06.257992+0000","last_undegraded":"2024-08-18T23:42:06.257992+0000","last_fullsized":"2024-08-18T23:42:06.257992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:20:56.392149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438360+0000","last_change":"2024-08-18T23:41:39.184029+0000","last_active":"2024-08-18T23:41:42.438360+0000","last_peered":"2024-08-18T23:41:42.438360+0000","last_clean":"2024-08-18T23:41:42.438360+0000","last_became_active":"2024-08-18T23:41:39.183159+0000","last_became_peered":"2024-08-18T23:41:39.183159+0000","last_unstale":"2024-08-18T23:41:42.438360+0000","last_undegraded":"2024-08-18T23:41:42.438360+0000","last_fullsized":"2024-08-18T23:41:42.438360+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:51:47.608798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437221+0000","last_change":"2024-08-18T23:41:32.142424+0000","last_active":"2024-08-18T23:41:42.437221+0000","last_peered":"2024-08-18T23:41:42.437221+0000","last_clean":"2024-08-18T23:41:42.437221+0000","last_became_active":"2024-08-18T23:41:32.141936+0000","last_became_peered":"2024-08-18T23:41:32.141936+0000","last_unstale":"2024-08-18T23:41:42.437221+0000","last_undegraded":"2024-08-18T23:41:42.437221+0000","last_fullsized":"2024-08-18T23:41:42.437221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:05:47.192928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:06.259464+0000","last_change":"2024-08-18T23:41:30.124430+0000","last_active":"2024-08-18T23:42:06.259464+0000","last_peered":"2024-08-18T23:42:06.259464+0000","last_clean":"2024-08-18T23:42:06.259464+0000","last_became_active":"2024-08-18T23:41:30.123626+0000","last_became_peered":"2024-08-18T23:41:30.123626+0000","last_unstale":"2024-08-18T23:42:06.259464+0000","last_undegraded":"2024-08-18T23:42:06.259464+0000","last_fullsized":"2024-08-18T23:42:06.259464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:26:18.426010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.183907+0000","last_change":"2024-08-18T23:41:39.192765+0000","last_active":"2024-08-18T23:41:42.183907+0000","last_peered":"2024-08-18T23:41:42.183907+0000","last_clean":"2024-08-18T23:41:42.183907+0000","last_became_active":"2024-08-18T23:41:39.192250+0000","last_became_peered":"2024-08-18T23:41:39.192250+0000","last_unstale":"2024-08-18T23:41:42.183907+0000","last_undegraded":"2024-08-18T23:41:42.183907+0000","last_fullsized":"2024-08-18T23:41:42.183907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:51:16.794404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438562+0000","last_change":"2024-08-18T23:41:32.141966+0000","last_active":"2024-08-18T23:41:42.438562+0000","last_peered":"2024-08-18T23:41:42.438562+0000","last_clean":"2024-08-18T23:41:42.438562+0000","last_became_active":"2024-08-18T23:41:32.141561+0000","last_became_peered":"2024-08-18T23:41:32.141561+0000","last_unstale":"2024-08-18T23:41:42.438562+0000","last_undegraded":"2024-08-18T23:41:42.438562+0000","last_fullsized":"2024-08-18T23:41:42.438562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:20.696089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:18.288089+0000","last_change":"2024-08-18T23:41:30.121598+0000","last_active":"2024-08-18T23:42:18.288089+0000","last_peered":"2024-08-18T23:42:18.288089+0000","last_clean":"2024-08-18T23:42:18.288089+0000","last_became_active":"2024-08-18T23:41:30.120533+0000","last_became_peered":"2024-08-18T23:41:30.120533+0000","last_unstale":"2024-08-18T23:42:18.288089+0000","last_undegraded":"2024-08-18T23:42:18.288089+0000","last_fullsized":"2024-08-18T23:42:18.288089+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:06:37.458272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436352+0000","last_change":"2024-08-18T23:41:39.178877+0000","last_active":"2024-08-18T23:41:42.436352+0000","last_peered":"2024-08-18T23:41:42.436352+0000","last_clean":"2024-08-18T23:41:42.436352+0000","last_became_active":"2024-08-18T23:41:39.178381+0000","last_became_peered":"2024-08-18T23:41:39.178381+0000","last_unstale":"2024-08-18T23:41:42.436352+0000","last_undegraded":"2024-08-18T23:41:42.436352+0000","last_fullsized":"2024-08-18T23:41:42.436352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:27.275893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436858+0000","last_change":"2024-08-18T23:41:39.188381+0000","last_active":"2024-08-18T23:41:42.436858+0000","last_peered":"2024-08-18T23:41:42.436858+0000","last_clean":"2024-08-18T23:41:42.436858+0000","last_became_active":"2024-08-18T23:41:39.187983+0000","last_became_peered":"2024-08-18T23:41:39.187983+0000","last_unstale":"2024-08-18T23:41:42.436858+0000","last_undegraded":"2024-08-18T23:41:42.436858+0000","last_fullsized":"2024-08-18T23:41:42.436858+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:04:45.460576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.183985+0000","last_change":"2024-08-18T23:41:32.144637+0000","last_active":"2024-08-18T23:41:42.183985+0000","last_peered":"2024-08-18T23:41:42.183985+0000","last_clean":"2024-08-18T23:41:42.183985+0000","last_became_active":"2024-08-18T23:41:32.144329+0000","last_became_peered":"2024-08-18T23:41:32.144329+0000","last_unstale":"2024-08-18T23:41:42.183985+0000","last_undegraded":"2024-08-18T23:41:42.183985+0000","last_fullsized":"2024-08-18T23:41:42.183985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:29:39.791264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437701+0000","last_change":"2024-08-18T23:41:30.122325+0000","last_active":"2024-08-18T23:41:42.437701+0000","last_peered":"2024-08-18T23:41:42.437701+0000","last_clean":"2024-08-18T23:41:42.437701+0000","last_became_active":"2024-08-18T23:41:30.121480+0000","last_became_peered":"2024-08-18T23:41:30.121480+0000","last_unstale":"2024-08-18T23:41:42.437701+0000","last_undegraded":"2024-08-18T23:41:42.437701+0000","last_fullsized":"2024-08-18T23:41:42.437701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:32:12.810650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"peering","last_fresh":"2024-08-18T23:41:42.471784+0000","last_change":"2024-08-18T23:42:22.309631+0000","last_active":"2024-08-18T23:42:22.309631+0000","last_peered":"2024-08-18T23:41:42.471784+0000","last_clean":"2024-08-18T23:41:42.471784+0000","last_became_active":"2024-08-18T23:41:39.185811+0000","last_became_peered":"2024-08-18T23:41:39.185811+0000","last_unstale":"2024-08-18T23:41:42.471784+0000","last_undegraded":"2024-08-18T23:41:42.471784+0000","last_fullsized":"2024-08-18T23:41:42.471784+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4,9,10],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436290+0000","last_change":"2024-08-18T23:41:32.145545+0000","last_active":"2024-08-18T23:41:42.436290+0000","last_peered":"2024-08-18T23:41:42.436290+0000","last_clean":"2024-08-18T23:41:42.436290+0000","last_became_active":"2024-08-18T23:41:32.145183+0000","last_became_peered":"2024-08-18T23:41:32.145183+0000","last_unstale":"2024-08-18T23:41:42.436290+0000","last_undegraded":"2024-08-18T23:41:42.436290+0000","last_fullsized":"2024-08-18T23:41:42.436290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:03:04.111658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:07.261928+0000","last_change":"2024-08-18T23:41:30.118776+0000","last_active":"2024-08-18T23:42:07.261928+0000","last_peered":"2024-08-18T23:42:07.261928+0000","last_clean":"2024-08-18T23:42:07.261928+0000","last_became_active":"2024-08-18T23:41:30.118436+0000","last_became_peered":"2024-08-18T23:41:30.118436+0000","last_unstale":"2024-08-18T23:42:07.261928+0000","last_undegraded":"2024-08-18T23:42:07.261928+0000","last_fullsized":"2024-08-18T23:42:07.261928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:13.980927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:07.262089+0000","last_change":"2024-08-18T23:41:30.125201+0000","last_active":"2024-08-18T23:42:07.262089+0000","last_peered":"2024-08-18T23:42:07.262089+0000","last_clean":"2024-08-18T23:42:07.262089+0000","last_became_active":"2024-08-18T23:41:30.124894+0000","last_became_peered":"2024-08-18T23:41:30.124894+0000","last_unstale":"2024-08-18T23:42:07.262089+0000","last_undegraded":"2024-08-18T23:42:07.262089+0000","last_fullsized":"2024-08-18T23:42:07.262089+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:30:17.721829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437803+0000","last_change":"2024-08-18T23:41:32.145452+0000","last_active":"2024-08-18T23:41:42.437803+0000","last_peered":"2024-08-18T23:41:42.437803+0000","last_clean":"2024-08-18T23:41:42.437803+0000","last_became_active":"2024-08-18T23:41:32.145151+0000","last_became_peered":"2024-08-18T23:41:32.145151+0000","last_unstale":"2024-08-18T23:41:42.437803+0000","last_undegraded":"2024-08-18T23:41:42.437803+0000","last_fullsized":"2024-08-18T23:41:42.437803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:08:08.713154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438790+0000","last_change":"2024-08-18T23:41:39.193334+0000","last_active":"2024-08-18T23:41:42.438790+0000","last_peered":"2024-08-18T23:41:42.438790+0000","last_clean":"2024-08-18T23:41:42.438790+0000","last_became_active":"2024-08-18T23:41:39.192924+0000","last_became_peered":"2024-08-18T23:41:39.192924+0000","last_unstale":"2024-08-18T23:41:42.438790+0000","last_undegraded":"2024-08-18T23:41:42.438790+0000","last_fullsized":"2024-08-18T23:41:42.438790+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:07.016300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:20.291145+0000","last_change":"2024-08-18T23:41:30.125217+0000","last_active":"2024-08-18T23:42:20.291145+0000","last_peered":"2024-08-18T23:42:20.291145+0000","last_clean":"2024-08-18T23:42:20.291145+0000","last_became_active":"2024-08-18T23:41:30.124889+0000","last_became_peered":"2024-08-18T23:41:30.124889+0000","last_unstale":"2024-08-18T23:42:20.291145+0000","last_undegraded":"2024-08-18T23:42:20.291145+0000","last_fullsized":"2024-08-18T23:42:20.291145+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:43.389537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437179+0000","last_change":"2024-08-18T23:41:32.142585+0000","last_active":"2024-08-18T23:41:42.437179+0000","last_peered":"2024-08-18T23:41:42.437179+0000","last_clean":"2024-08-18T23:41:42.437179+0000","last_became_active":"2024-08-18T23:41:32.142207+0000","last_became_peered":"2024-08-18T23:41:32.142207+0000","last_unstale":"2024-08-18T23:41:42.437179+0000","last_undegraded":"2024-08-18T23:41:42.437179+0000","last_fullsized":"2024-08-18T23:41:42.437179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:16:58.755444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437736+0000","last_change":"2024-08-18T23:41:39.185618+0000","last_active":"2024-08-18T23:41:42.437736+0000","last_peered":"2024-08-18T23:41:42.437736+0000","last_clean":"2024-08-18T23:41:42.437736+0000","last_became_active":"2024-08-18T23:41:39.185222+0000","last_became_peered":"2024-08-18T23:41:39.185222+0000","last_unstale":"2024-08-18T23:41:42.437736+0000","last_undegraded":"2024-08-18T23:41:42.437736+0000","last_fullsized":"2024-08-18T23:41:42.437736+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:01:27.290998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.309157+0000","last_change":"2024-08-18T23:41:32.140719+0000","last_active":"2024-08-18T23:42:22.309157+0000","last_peered":"2024-08-18T23:42:22.309157+0000","last_clean":"2024-08-18T23:42:22.309157+0000","last_became_active":"2024-08-18T23:41:32.140131+0000","last_became_peered":"2024-08-18T23:41:32.140131+0000","last_unstale":"2024-08-18T23:42:22.309157+0000","last_undegraded":"2024-08-18T23:42:22.309157+0000","last_fullsized":"2024-08-18T23:42:22.309157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:48:26.778094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438674+0000","last_change":"2024-08-18T23:41:32.138778+0000","last_active":"2024-08-18T23:41:42.438674+0000","last_peered":"2024-08-18T23:41:42.438674+0000","last_clean":"2024-08-18T23:41:42.438674+0000","last_became_active":"2024-08-18T23:41:32.138408+0000","last_became_peered":"2024-08-18T23:41:32.138408+0000","last_unstale":"2024-08-18T23:41:42.438674+0000","last_undegraded":"2024-08-18T23:41:42.438674+0000","last_fullsized":"2024-08-18T23:41:42.438674+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:29:41.621518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436698+0000","last_change":"2024-08-18T23:41:32.149522+0000","last_active":"2024-08-18T23:41:42.436698+0000","last_peered":"2024-08-18T23:41:42.436698+0000","last_clean":"2024-08-18T23:41:42.436698+0000","last_became_active":"2024-08-18T23:41:32.149182+0000","last_became_peered":"2024-08-18T23:41:32.149182+0000","last_unstale":"2024-08-18T23:41:42.436698+0000","last_undegraded":"2024-08-18T23:41:42.436698+0000","last_fullsized":"2024-08-18T23:41:42.436698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:30:16.945045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436641+0000","last_change":"2024-08-18T23:41:32.149853+0000","last_active":"2024-08-18T23:41:42.436641+0000","last_peered":"2024-08-18T23:41:42.436641+0000","last_clean":"2024-08-18T23:41:42.436641+0000","last_became_active":"2024-08-18T23:41:32.141907+0000","last_became_peered":"2024-08-18T23:41:32.141907+0000","last_unstale":"2024-08-18T23:41:42.436641+0000","last_undegraded":"2024-08-18T23:41:42.436641+0000","last_fullsized":"2024-08-18T23:41:42.436641+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:24:37.323567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481741+0000","last_change":"2024-08-18T23:41:32.136179+0000","last_active":"2024-08-18T23:41:42.481741+0000","last_peered":"2024-08-18T23:41:42.481741+0000","last_clean":"2024-08-18T23:41:42.481741+0000","last_became_active":"2024-08-18T23:41:32.135939+0000","last_became_peered":"2024-08-18T23:41:32.135939+0000","last_unstale":"2024-08-18T23:41:42.481741+0000","last_undegraded":"2024-08-18T23:41:42.481741+0000","last_fullsized":"2024-08-18T23:41:42.481741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:15.392061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.310009+0000","last_change":"2024-08-18T23:41:32.137509+0000","last_active":"2024-08-18T23:42:22.310009+0000","last_peered":"2024-08-18T23:42:22.310009+0000","last_clean":"2024-08-18T23:42:22.310009+0000","last_became_active":"2024-08-18T23:41:32.136380+0000","last_became_peered":"2024-08-18T23:41:32.136380+0000","last_unstale":"2024-08-18T23:42:22.310009+0000","last_undegraded":"2024-08-18T23:42:22.310009+0000","last_fullsized":"2024-08-18T23:42:22.310009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:49.776225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438165+0000","last_change":"2024-08-18T23:41:32.139271+0000","last_active":"2024-08-18T23:41:42.438165+0000","last_peered":"2024-08-18T23:41:42.438165+0000","last_clean":"2024-08-18T23:41:42.438165+0000","last_became_active":"2024-08-18T23:41:32.138868+0000","last_became_peered":"2024-08-18T23:41:32.138868+0000","last_unstale":"2024-08-18T23:41:42.438165+0000","last_undegraded":"2024-08-18T23:41:42.438165+0000","last_fullsized":"2024-08-18T23:41:42.438165+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:49:27.890857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436507+0000","last_change":"2024-08-18T23:41:32.144460+0000","last_active":"2024-08-18T23:41:42.436507+0000","last_peered":"2024-08-18T23:41:42.436507+0000","last_clean":"2024-08-18T23:41:42.436507+0000","last_became_active":"2024-08-18T23:41:32.144178+0000","last_became_peered":"2024-08-18T23:41:32.144178+0000","last_unstale":"2024-08-18T23:41:42.436507+0000","last_undegraded":"2024-08-18T23:41:42.436507+0000","last_fullsized":"2024-08-18T23:41:42.436507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:56.252215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437434+0000","last_change":"2024-08-18T23:41:32.147973+0000","last_active":"2024-08-18T23:41:42.437434+0000","last_peered":"2024-08-18T23:41:42.437434+0000","last_clean":"2024-08-18T23:41:42.437434+0000","last_became_active":"2024-08-18T23:41:32.147621+0000","last_became_peered":"2024-08-18T23:41:32.147621+0000","last_unstale":"2024-08-18T23:41:42.437434+0000","last_undegraded":"2024-08-18T23:41:42.437434+0000","last_fullsized":"2024-08-18T23:41:42.437434+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:14.176569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436418+0000","last_change":"2024-08-18T23:41:32.146029+0000","last_active":"2024-08-18T23:41:42.436418+0000","last_peered":"2024-08-18T23:41:42.436418+0000","last_clean":"2024-08-18T23:41:42.436418+0000","last_became_active":"2024-08-18T23:41:32.145584+0000","last_became_peered":"2024-08-18T23:41:32.145584+0000","last_unstale":"2024-08-18T23:41:42.436418+0000","last_undegraded":"2024-08-18T23:41:42.436418+0000","last_fullsized":"2024-08-18T23:41:42.436418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:25:33.884906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437194+0000","last_change":"2024-08-18T23:41:32.145950+0000","last_active":"2024-08-18T23:41:42.437194+0000","last_peered":"2024-08-18T23:41:42.437194+0000","last_clean":"2024-08-18T23:41:42.437194+0000","last_became_active":"2024-08-18T23:41:32.145618+0000","last_became_peered":"2024-08-18T23:41:32.145618+0000","last_unstale":"2024-08-18T23:41:42.437194+0000","last_undegraded":"2024-08-18T23:41:42.437194+0000","last_fullsized":"2024-08-18T23:41:42.437194+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:05:58.889240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481678+0000","last_change":"2024-08-18T23:41:32.130684+0000","last_active":"2024-08-18T23:41:42.481678+0000","last_peered":"2024-08-18T23:41:42.481678+0000","last_clean":"2024-08-18T23:41:42.481678+0000","last_became_active":"2024-08-18T23:41:32.130369+0000","last_became_peered":"2024-08-18T23:41:32.130369+0000","last_unstale":"2024-08-18T23:41:42.481678+0000","last_undegraded":"2024-08-18T23:41:42.481678+0000","last_fullsized":"2024-08-18T23:41:42.481678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:22.537018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.310146+0000","last_change":"2024-08-18T23:41:32.137291+0000","last_active":"2024-08-18T23:42:22.310146+0000","last_peered":"2024-08-18T23:42:22.310146+0000","last_clean":"2024-08-18T23:42:22.310146+0000","last_became_active":"2024-08-18T23:41:32.136977+0000","last_became_peered":"2024-08-18T23:41:32.136977+0000","last_unstale":"2024-08-18T23:42:22.310146+0000","last_undegraded":"2024-08-18T23:42:22.310146+0000","last_fullsized":"2024-08-18T23:42:22.310146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:18:45.544760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437895+0000","last_change":"2024-08-18T23:41:32.144434+0000","last_active":"2024-08-18T23:41:42.437895+0000","last_peered":"2024-08-18T23:41:42.437895+0000","last_clean":"2024-08-18T23:41:42.437895+0000","last_became_active":"2024-08-18T23:41:32.144119+0000","last_became_peered":"2024-08-18T23:41:32.144119+0000","last_unstale":"2024-08-18T23:41:42.437895+0000","last_undegraded":"2024-08-18T23:41:42.437895+0000","last_fullsized":"2024-08-18T23:41:42.437895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:50:54.528630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.184493+0000","last_change":"2024-08-18T23:41:32.135034+0000","last_active":"2024-08-18T23:41:42.184493+0000","last_peered":"2024-08-18T23:41:42.184493+0000","last_clean":"2024-08-18T23:41:42.184493+0000","last_became_active":"2024-08-18T23:41:32.134700+0000","last_became_peered":"2024-08-18T23:41:32.134700+0000","last_unstale":"2024-08-18T23:41:42.184493+0000","last_undegraded":"2024-08-18T23:41:42.184493+0000","last_fullsized":"2024-08-18T23:41:42.184493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:06:31.263114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.310319+0000","last_change":"2024-08-18T23:41:32.141191+0000","last_active":"2024-08-18T23:42:22.310319+0000","last_peered":"2024-08-18T23:42:22.310319+0000","last_clean":"2024-08-18T23:42:22.310319+0000","last_became_active":"2024-08-18T23:41:32.140901+0000","last_became_peered":"2024-08-18T23:41:32.140901+0000","last_unstale":"2024-08-18T23:42:22.310319+0000","last_undegraded":"2024-08-18T23:42:22.310319+0000","last_fullsized":"2024-08-18T23:42:22.310319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:24.849985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437507+0000","last_change":"2024-08-18T23:41:32.142543+0000","last_active":"2024-08-18T23:41:42.437507+0000","last_peered":"2024-08-18T23:41:42.437507+0000","last_clean":"2024-08-18T23:41:42.437507+0000","last_became_active":"2024-08-18T23:41:32.142084+0000","last_became_peered":"2024-08-18T23:41:32.142084+0000","last_unstale":"2024-08-18T23:41:42.437507+0000","last_undegraded":"2024-08-18T23:41:42.437507+0000","last_fullsized":"2024-08-18T23:41:42.437507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:54:18.200016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436457+0000","last_change":"2024-08-18T23:41:32.148968+0000","last_active":"2024-08-18T23:41:42.436457+0000","last_peered":"2024-08-18T23:41:42.436457+0000","last_clean":"2024-08-18T23:41:42.436457+0000","last_became_active":"2024-08-18T23:41:32.148625+0000","last_became_peered":"2024-08-18T23:41:32.148625+0000","last_unstale":"2024-08-18T23:41:42.436457+0000","last_undegraded":"2024-08-18T23:41:42.436457+0000","last_fullsized":"2024-08-18T23:41:42.436457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:11:40.172139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436410+0000","last_change":"2024-08-18T23:41:32.133478+0000","last_active":"2024-08-18T23:41:42.436410+0000","last_peered":"2024-08-18T23:41:42.436410+0000","last_clean":"2024-08-18T23:41:42.436410+0000","last_became_active":"2024-08-18T23:41:32.133149+0000","last_became_peered":"2024-08-18T23:41:32.133149+0000","last_unstale":"2024-08-18T23:41:42.436410+0000","last_undegraded":"2024-08-18T23:41:42.436410+0000","last_fullsized":"2024-08-18T23:41:42.436410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:57:53.713309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481649+0000","last_change":"2024-08-18T23:41:32.138206+0000","last_active":"2024-08-18T23:41:42.481649+0000","last_peered":"2024-08-18T23:41:42.481649+0000","last_clean":"2024-08-18T23:41:42.481649+0000","last_became_active":"2024-08-18T23:41:32.137903+0000","last_became_peered":"2024-08-18T23:41:32.137903+0000","last_unstale":"2024-08-18T23:41:42.481649+0000","last_undegraded":"2024-08-18T23:41:42.481649+0000","last_fullsized":"2024-08-18T23:41:42.481649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:18:48.600658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437946+0000","last_change":"2024-08-18T23:41:32.136303+0000","last_active":"2024-08-18T23:41:42.437946+0000","last_peered":"2024-08-18T23:41:42.437946+0000","last_clean":"2024-08-18T23:41:42.437946+0000","last_became_active":"2024-08-18T23:41:32.134628+0000","last_became_peered":"2024-08-18T23:41:32.134628+0000","last_unstale":"2024-08-18T23:41:42.437946+0000","last_undegraded":"2024-08-18T23:41:42.437946+0000","last_fullsized":"2024-08-18T23:41:42.437946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:28:51.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438368+0000","last_change":"2024-08-18T23:41:32.137512+0000","last_active":"2024-08-18T23:41:42.438368+0000","last_peered":"2024-08-18T23:41:42.438368+0000","last_clean":"2024-08-18T23:41:42.438368+0000","last_became_active":"2024-08-18T23:41:32.137150+0000","last_became_peered":"2024-08-18T23:41:32.137150+0000","last_unstale":"2024-08-18T23:41:42.438368+0000","last_undegraded":"2024-08-18T23:41:42.438368+0000","last_fullsized":"2024-08-18T23:41:42.438368+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:44:08.020780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.182271+0000","last_change":"2024-08-18T23:41:32.139229+0000","last_active":"2024-08-18T23:41:42.182271+0000","last_peered":"2024-08-18T23:41:42.182271+0000","last_clean":"2024-08-18T23:41:42.182271+0000","last_became_active":"2024-08-18T23:41:32.138864+0000","last_became_peered":"2024-08-18T23:41:32.138864+0000","last_unstale":"2024-08-18T23:41:42.182271+0000","last_undegraded":"2024-08-18T23:41:42.182271+0000","last_fullsized":"2024-08-18T23:41:42.182271+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:54:21.701750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437965+0000","last_change":"2024-08-18T23:41:32.148727+0000","last_active":"2024-08-18T23:41:42.437965+0000","last_peered":"2024-08-18T23:41:42.437965+0000","last_clean":"2024-08-18T23:41:42.437965+0000","last_became_active":"2024-08-18T23:41:32.146828+0000","last_became_peered":"2024-08-18T23:41:32.146828+0000","last_unstale":"2024-08-18T23:41:42.437965+0000","last_undegraded":"2024-08-18T23:41:42.437965+0000","last_fullsized":"2024-08-18T23:41:42.437965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:51:49.175150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436255+0000","last_change":"2024-08-18T23:41:32.144796+0000","last_active":"2024-08-18T23:41:42.436255+0000","last_peered":"2024-08-18T23:41:42.436255+0000","last_clean":"2024-08-18T23:41:42.436255+0000","last_became_active":"2024-08-18T23:41:32.141931+0000","last_became_peered":"2024-08-18T23:41:32.141931+0000","last_unstale":"2024-08-18T23:41:42.436255+0000","last_undegraded":"2024-08-18T23:41:42.436255+0000","last_fullsized":"2024-08-18T23:41:42.436255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:48:08.428845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438077+0000","last_change":"2024-08-18T23:41:32.135265+0000","last_active":"2024-08-18T23:41:42.438077+0000","last_peered":"2024-08-18T23:41:42.438077+0000","last_clean":"2024-08-18T23:41:42.438077+0000","last_became_active":"2024-08-18T23:41:32.134952+0000","last_became_peered":"2024-08-18T23:41:32.134952+0000","last_unstale":"2024-08-18T23:41:42.438077+0000","last_undegraded":"2024-08-18T23:41:42.438077+0000","last_fullsized":"2024-08-18T23:41:42.438077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:35:14.099536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.181837+0000","last_change":"2024-08-18T23:41:32.135941+0000","last_active":"2024-08-18T23:41:42.181837+0000","last_peered":"2024-08-18T23:41:42.181837+0000","last_clean":"2024-08-18T23:41:42.181837+0000","last_became_active":"2024-08-18T23:41:32.135623+0000","last_became_peered":"2024-08-18T23:41:32.135623+0000","last_unstale":"2024-08-18T23:41:42.181837+0000","last_undegraded":"2024-08-18T23:41:42.181837+0000","last_fullsized":"2024-08-18T23:41:42.181837+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:43:36.504738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436949+0000","last_change":"2024-08-18T23:41:32.141643+0000","last_active":"2024-08-18T23:41:42.436949+0000","last_peered":"2024-08-18T23:41:42.436949+0000","last_clean":"2024-08-18T23:41:42.436949+0000","last_became_active":"2024-08-18T23:41:32.141301+0000","last_became_peered":"2024-08-18T23:41:32.141301+0000","last_unstale":"2024-08-18T23:41:42.436949+0000","last_undegraded":"2024-08-18T23:41:42.436949+0000","last_fullsized":"2024-08-18T23:41:42.436949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:23:53.100740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437849+0000","last_change":"2024-08-18T23:41:32.146730+0000","last_active":"2024-08-18T23:41:42.437849+0000","last_peered":"2024-08-18T23:41:42.437849+0000","last_clean":"2024-08-18T23:41:42.437849+0000","last_became_active":"2024-08-18T23:41:32.146252+0000","last_became_peered":"2024-08-18T23:41:32.146252+0000","last_unstale":"2024-08-18T23:41:42.437849+0000","last_undegraded":"2024-08-18T23:41:42.437849+0000","last_fullsized":"2024-08-18T23:41:42.437849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:41.481761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436851+0000","last_change":"2024-08-18T23:41:32.149569+0000","last_active":"2024-08-18T23:41:42.436851+0000","last_peered":"2024-08-18T23:41:42.436851+0000","last_clean":"2024-08-18T23:41:42.436851+0000","last_became_active":"2024-08-18T23:41:32.149277+0000","last_became_peered":"2024-08-18T23:41:32.149277+0000","last_unstale":"2024-08-18T23:41:42.436851+0000","last_undegraded":"2024-08-18T23:41:42.436851+0000","last_fullsized":"2024-08-18T23:41:42.436851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:34:00.547909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436812+0000","last_change":"2024-08-18T23:41:32.137889+0000","last_active":"2024-08-18T23:41:42.436812+0000","last_peered":"2024-08-18T23:41:42.436812+0000","last_clean":"2024-08-18T23:41:42.436812+0000","last_became_active":"2024-08-18T23:41:32.137578+0000","last_became_peered":"2024-08-18T23:41:32.137578+0000","last_unstale":"2024-08-18T23:41:42.436812+0000","last_undegraded":"2024-08-18T23:41:42.436812+0000","last_fullsized":"2024-08-18T23:41:42.436812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:18:09.987838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.310210+0000","last_change":"2024-08-18T23:41:32.142084+0000","last_active":"2024-08-18T23:42:22.310210+0000","last_peered":"2024-08-18T23:42:22.310210+0000","last_clean":"2024-08-18T23:42:22.310210+0000","last_became_active":"2024-08-18T23:41:32.141670+0000","last_became_peered":"2024-08-18T23:41:32.141670+0000","last_unstale":"2024-08-18T23:42:22.310210+0000","last_undegraded":"2024-08-18T23:42:22.310210+0000","last_fullsized":"2024-08-18T23:42:22.310210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:03.990351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53482,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":99,"num_read_kb":106,"num_write":157,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":160446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514519,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":486092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776155,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1662976,"data_stored":1122842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939680,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":490424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168612,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":486248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397544,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36896,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708256,"statfs":{"total":95995035648,"available":95957254144,"internally_reserved":0,"allocated":1597440,"data_stored":1082546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561069,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":481504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790002,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1605632,"data_stored":1077744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953526,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":486544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":481698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":532246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":530506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":482892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":49092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":50832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":51060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T23:42:24.545 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-18T23:42:24.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:24 smithi078 ceph-mon[18440]: osdmap e90: 12 total, 12 up, 12 in 2024-08-18T23:42:24.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:24 smithi078 ceph-mon[18440]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:42:24.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:24 smithi078 ceph-mon[18440]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 629 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-08-18T23:42:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:24 smithi161 ceph-mon[22027]: osdmap e90: 12 total, 12 up, 12 in 2024-08-18T23:42:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:24 smithi161 ceph-mon[22027]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:42:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:24 smithi161 ceph-mon[22027]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 629 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-08-18T23:42:24.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:24 smithi123 ceph-mon[21808]: osdmap e90: 12 total, 12 up, 12 in 2024-08-18T23:42:24.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:24 smithi123 ceph-mon[21808]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:42:24.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:24 smithi123 ceph-mon[21808]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 629 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-08-18T23:42:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:25.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.260+0000 7fc4852f7640 1 -- 172.21.15.78:0/2983099826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc480102c20 msgr2=0x7fc48010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:25.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.260+0000 7fc4852f7640 1 --2- 172.21.15.78:0/2983099826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc480102c20 0x7fc48010d770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc478009ca0 tx=0x7fc47802a950 comp rx=0 tx=0).stop 2024-08-18T23:42:25.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.260+0000 7fc4852f7640 1 -- 172.21.15.78:0/2983099826 shutdown_connections 2024-08-18T23:42:25.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.260+0000 7fc4852f7640 1 --2- 172.21.15.78:0/2983099826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc480102c20 0x7fc48010d770 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:25.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.260+0000 7fc4852f7640 1 --2- 172.21.15.78:0/2983099826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4801022a0 0x7fc4801026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:25.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.260+0000 7fc4852f7640 1 --2- 172.21.15.78:0/2983099826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc480100b50 0x7fc480100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:25.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.260+0000 7fc4852f7640 1 -- 172.21.15.78:0/2983099826 >> 172.21.15.78:0/2983099826 conn(0x7fc4800fc500 msgr2=0x7fc4800fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:25.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.260+0000 7fc4852f7640 1 -- 172.21.15.78:0/2983099826 shutdown_connections 2024-08-18T23:42:25.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.260+0000 7fc4852f7640 1 -- 172.21.15.78:0/2983099826 wait complete. 2024-08-18T23:42:25.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.261+0000 7fc4852f7640 1 Processor -- start 2024-08-18T23:42:25.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.261+0000 7fc4852f7640 1 -- start start 2024-08-18T23:42:25.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.261+0000 7fc4852f7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc480100b50 0x7fc480198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:25.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.261+0000 7fc4852f7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4801022a0 0x7fc480198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:25.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.261+0000 7fc4852f7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc480102c20 0x7fc48019c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:25.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.261+0000 7fc4852f7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc480110cb0 con 0x7fc480102c20 2024-08-18T23:42:25.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.261+0000 7fc4852f7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc480110b30 con 0x7fc480100b50 2024-08-18T23:42:25.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.261+0000 7fc4852f7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc480110e30 con 0x7fc4801022a0 2024-08-18T23:42:25.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.261+0000 7fc47f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc480102c20 0x7fc48019c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:25.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.261+0000 7fc47f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc480102c20 0x7fc48019c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36588/0 (socket says 172.21.15.78:36588) 2024-08-18T23:42:25.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.261+0000 7fc47f7fe640 1 -- 172.21.15.78:0/3550793162 learned_addr learned my addr 172.21.15.78:0/3550793162 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:25.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.261+0000 7fc4767fc640 1 --2- 172.21.15.78:0/3550793162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4801022a0 0x7fc480198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:25.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.262+0000 7fc47effd640 1 --2- 172.21.15.78:0/3550793162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc480100b50 0x7fc480198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:25.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.262+0000 7fc47f7fe640 1 -- 172.21.15.78:0/3550793162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4801022a0 msgr2=0x7fc480198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:25.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.262+0000 7fc47f7fe640 1 --2- 172.21.15.78:0/3550793162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4801022a0 0x7fc480198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:25.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.262+0000 7fc47f7fe640 1 -- 172.21.15.78:0/3550793162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc480100b50 msgr2=0x7fc480198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:25.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.262+0000 7fc47f7fe640 1 --2- 172.21.15.78:0/3550793162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc480100b50 0x7fc480198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:25.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.262+0000 7fc47f7fe640 1 -- 172.21.15.78:0/3550793162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48019d0d0 con 0x7fc480102c20 2024-08-18T23:42:25.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.262+0000 7fc47effd640 1 --2- 172.21.15.78:0/3550793162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc480100b50 0x7fc480198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:42:25.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.262+0000 7fc47f7fe640 1 --2- 172.21.15.78:0/3550793162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc480102c20 0x7fc48019c9f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc47802ae70 tx=0x7fc47802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:25.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.262+0000 7fc47cff9640 1 -- 172.21.15.78:0/3550793162 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478034040 con 0x7fc480102c20 2024-08-18T23:42:25.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.262+0000 7fc4852f7640 1 -- 172.21.15.78:0/3550793162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48019d360 con 0x7fc480102c20 2024-08-18T23:42:25.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.262+0000 7fc47cff9640 1 -- 172.21.15.78:0/3550793162 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc478030920 con 0x7fc480102c20 2024-08-18T23:42:25.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.262+0000 7fc47cff9640 1 -- 172.21.15.78:0/3550793162 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478045480 con 0x7fc480102c20 2024-08-18T23:42:25.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.262+0000 7fc4852f7640 1 -- 172.21.15.78:0/3550793162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4801a4ca0 con 0x7fc480102c20 2024-08-18T23:42:25.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.263+0000 7fc4852f7640 1 -- 172.21.15.78:0/3550793162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc480101ee0 con 0x7fc480102c20 2024-08-18T23:42:25.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.264+0000 7fc47cff9640 1 -- 172.21.15.78:0/3550793162 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc478037020 con 0x7fc480102c20 2024-08-18T23:42:25.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.265+0000 7fc47cff9640 1 --2- 172.21.15.78:0/3550793162 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc46c0a2c10 0x7fc46c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:25.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.265+0000 7fc47effd640 1 --2- 172.21.15.78:0/3550793162 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc46c0a2c10 0x7fc46c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:25.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.265+0000 7fc47cff9640 1 -- 172.21.15.78:0/3550793162 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc4780f0330 con 0x7fc480102c20 2024-08-18T23:42:25.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.265+0000 7fc47effd640 1 --2- 172.21.15.78:0/3550793162 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc46c0a2c10 0x7fc46c0a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc470009b40 tx=0x7fc470005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:25.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.269+0000 7fc47cff9640 1 -- 172.21.15.78:0/3550793162 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc4780bcf80 con 0x7fc480102c20 2024-08-18T23:42:25.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.432+0000 7fc4852f7640 1 -- 172.21.15.78:0/3550793162 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc480062360 con 0x7fc46c0a2c10 2024-08-18T23:42:25.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.440+0000 7fc47cff9640 1 -- 172.21.15.78:0/3550793162 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340303 (secure 0 0 0) 0x7fc480062360 con 0x7fc46c0a2c10 2024-08-18T23:42:25.446 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:42:25.448 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-08-18T23:42:25.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.443+0000 7fc4852f7640 1 -- 172.21.15.78:0/3550793162 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc46c0a2c10 msgr2=0x7fc46c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:25.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.443+0000 7fc4852f7640 1 --2- 172.21.15.78:0/3550793162 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc46c0a2c10 0x7fc46c0a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc470009b40 tx=0x7fc470005e50 comp rx=0 tx=0).stop 2024-08-18T23:42:25.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.443+0000 7fc4852f7640 1 -- 172.21.15.78:0/3550793162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc480102c20 msgr2=0x7fc48019c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:25.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.443+0000 7fc4852f7640 1 --2- 172.21.15.78:0/3550793162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc480102c20 0x7fc48019c9f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc47802ae70 tx=0x7fc47802f5d0 comp rx=0 tx=0).stop 2024-08-18T23:42:25.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.444+0000 7fc4852f7640 1 -- 172.21.15.78:0/3550793162 shutdown_connections 2024-08-18T23:42:25.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.444+0000 7fc4852f7640 1 --2- 172.21.15.78:0/3550793162 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc46c0a2c10 0x7fc46c0a50d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:25.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.444+0000 7fc4852f7640 1 --2- 172.21.15.78:0/3550793162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc480102c20 0x7fc48019c9f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:25.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.444+0000 7fc4852f7640 1 --2- 172.21.15.78:0/3550793162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4801022a0 0x7fc480198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:25.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.444+0000 7fc4852f7640 1 --2- 172.21.15.78:0/3550793162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc480100b50 0x7fc480198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:25.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.444+0000 7fc4852f7640 1 -- 172.21.15.78:0/3550793162 >> 172.21.15.78:0/3550793162 conn(0x7fc4800fc500 msgr2=0x7fc480069e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:25.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.444+0000 7fc4852f7640 1 -- 172.21.15.78:0/3550793162 shutdown_connections 2024-08-18T23:42:25.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:25.444+0000 7fc4852f7640 1 -- 172.21.15.78:0/3550793162 wait complete. 2024-08-18T23:42:26.367 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-08-18T23:42:24.120903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644084,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":225,"num_read_kb":215,"num_write":340,"num_write_kb":3040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387748,"kb_used_data":13632,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554076,"statfs":{"total":1151940427776,"available":1151543373824,"internally_reserved":0,"allocated":13959168,"data_stored":7750868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25142,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":41,"num_write":21,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002124"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:07.263169+0000","last_change":"2024-08-18T23:41:30.120580+0000","last_active":"2024-08-18T23:42:07.263169+0000","last_peered":"2024-08-18T23:42:07.263169+0000","last_clean":"2024-08-18T23:42:07.263169+0000","last_became_active":"2024-08-18T23:41:30.120266+0000","last_became_peered":"2024-08-18T23:41:30.120266+0000","last_unstale":"2024-08-18T23:42:07.263169+0000","last_undegraded":"2024-08-18T23:42:07.263169+0000","last_fullsized":"2024-08-18T23:42:07.263169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:06:40.997070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436535+0000","last_change":"2024-08-18T23:41:32.143232+0000","last_active":"2024-08-18T23:41:42.436535+0000","last_peered":"2024-08-18T23:41:42.436535+0000","last_clean":"2024-08-18T23:41:42.436535+0000","last_became_active":"2024-08-18T23:41:32.142916+0000","last_became_peered":"2024-08-18T23:41:32.142916+0000","last_unstale":"2024-08-18T23:41:42.436535+0000","last_undegraded":"2024-08-18T23:41:42.436535+0000","last_fullsized":"2024-08-18T23:41:42.436535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:50:54.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438428+0000","last_change":"2024-08-18T23:41:39.188772+0000","last_active":"2024-08-18T23:41:42.438428+0000","last_peered":"2024-08-18T23:41:42.438428+0000","last_clean":"2024-08-18T23:41:42.438428+0000","last_became_active":"2024-08-18T23:41:39.188362+0000","last_became_peered":"2024-08-18T23:41:39.188362+0000","last_unstale":"2024-08-18T23:41:42.438428+0000","last_undegraded":"2024-08-18T23:41:42.438428+0000","last_fullsized":"2024-08-18T23:41:42.438428+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:47:50.255543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436883+0000","last_change":"2024-08-18T23:41:30.137179+0000","last_active":"2024-08-18T23:41:42.436883+0000","last_peered":"2024-08-18T23:41:42.436883+0000","last_clean":"2024-08-18T23:41:42.436883+0000","last_became_active":"2024-08-18T23:41:30.136618+0000","last_became_peered":"2024-08-18T23:41:30.136618+0000","last_unstale":"2024-08-18T23:41:42.436883+0000","last_undegraded":"2024-08-18T23:41:42.436883+0000","last_fullsized":"2024-08-18T23:41:42.436883+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:59.826364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437188+0000","last_change":"2024-08-18T23:41:32.142075+0000","last_active":"2024-08-18T23:41:42.437188+0000","last_peered":"2024-08-18T23:41:42.437188+0000","last_clean":"2024-08-18T23:41:42.437188+0000","last_became_active":"2024-08-18T23:41:32.141709+0000","last_became_peered":"2024-08-18T23:41:32.141709+0000","last_unstale":"2024-08-18T23:41:42.437188+0000","last_undegraded":"2024-08-18T23:41:42.437188+0000","last_fullsized":"2024-08-18T23:41:42.437188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:08.324627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437258+0000","last_change":"2024-08-18T23:41:39.184575+0000","last_active":"2024-08-18T23:41:42.437258+0000","last_peered":"2024-08-18T23:41:42.437258+0000","last_clean":"2024-08-18T23:41:42.437258+0000","last_became_active":"2024-08-18T23:41:39.184169+0000","last_became_peered":"2024-08-18T23:41:39.184169+0000","last_unstale":"2024-08-18T23:41:42.437258+0000","last_undegraded":"2024-08-18T23:41:42.437258+0000","last_fullsized":"2024-08-18T23:41:42.437258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:32.218172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324756+0000","last_change":"2024-08-18T23:41:30.121638+0000","last_active":"2024-08-18T23:42:23.324756+0000","last_peered":"2024-08-18T23:42:23.324756+0000","last_clean":"2024-08-18T23:42:23.324756+0000","last_became_active":"2024-08-18T23:41:30.121290+0000","last_became_peered":"2024-08-18T23:41:30.121290+0000","last_unstale":"2024-08-18T23:42:23.324756+0000","last_undegraded":"2024-08-18T23:42:23.324756+0000","last_fullsized":"2024-08-18T23:42:23.324756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:43:55.638780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437662+0000","last_change":"2024-08-18T23:41:32.142177+0000","last_active":"2024-08-18T23:41:42.437662+0000","last_peered":"2024-08-18T23:41:42.437662+0000","last_clean":"2024-08-18T23:41:42.437662+0000","last_became_active":"2024-08-18T23:41:32.141731+0000","last_became_peered":"2024-08-18T23:41:32.141731+0000","last_unstale":"2024-08-18T23:41:42.437662+0000","last_undegraded":"2024-08-18T23:41:42.437662+0000","last_fullsized":"2024-08-18T23:41:42.437662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:36.283299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438115+0000","last_change":"2024-08-18T23:41:39.182931+0000","last_active":"2024-08-18T23:41:42.438115+0000","last_peered":"2024-08-18T23:41:42.438115+0000","last_clean":"2024-08-18T23:41:42.438115+0000","last_became_active":"2024-08-18T23:41:39.182008+0000","last_became_peered":"2024-08-18T23:41:39.182008+0000","last_unstale":"2024-08-18T23:41:42.438115+0000","last_undegraded":"2024-08-18T23:41:42.438115+0000","last_fullsized":"2024-08-18T23:41:42.438115+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:58:38.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":2,"reported_epoch":89,"state":"peering","last_fresh":"2024-08-18T23:41:33.156065+0000","last_change":"2024-08-18T23:42:22.476106+0000","last_active":"2024-08-18T23:42:22.476106+0000","last_peered":"2024-08-18T23:41:33.156065+0000","last_clean":"2024-08-18T23:41:33.156065+0000","last_became_active":"2024-08-18T23:41:30.119986+0000","last_became_peered":"2024-08-18T23:41:30.119986+0000","last_unstale":"2024-08-18T23:41:33.156065+0000","last_undegraded":"2024-08-18T23:41:33.156065+0000","last_fullsized":"2024-08-18T23:41:33.156065+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438033+0000","last_change":"2024-08-18T23:41:32.148662+0000","last_active":"2024-08-18T23:41:42.438033+0000","last_peered":"2024-08-18T23:41:42.438033+0000","last_clean":"2024-08-18T23:41:42.438033+0000","last_became_active":"2024-08-18T23:41:32.148318+0000","last_became_peered":"2024-08-18T23:41:32.148318+0000","last_unstale":"2024-08-18T23:41:42.438033+0000","last_undegraded":"2024-08-18T23:41:42.438033+0000","last_fullsized":"2024-08-18T23:41:42.438033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:58.668669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437251+0000","last_change":"2024-08-18T23:41:39.181114+0000","last_active":"2024-08-18T23:41:42.437251+0000","last_peered":"2024-08-18T23:41:42.437251+0000","last_clean":"2024-08-18T23:41:42.437251+0000","last_became_active":"2024-08-18T23:41:39.180599+0000","last_became_peered":"2024-08-18T23:41:39.180599+0000","last_unstale":"2024-08-18T23:41:42.437251+0000","last_undegraded":"2024-08-18T23:41:42.437251+0000","last_fullsized":"2024-08-18T23:41:42.437251+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:36:08.749143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437425+0000","last_change":"2024-08-18T23:41:32.144326+0000","last_active":"2024-08-18T23:41:42.437425+0000","last_peered":"2024-08-18T23:41:42.437425+0000","last_clean":"2024-08-18T23:41:42.437425+0000","last_became_active":"2024-08-18T23:41:32.144004+0000","last_became_peered":"2024-08-18T23:41:32.144004+0000","last_unstale":"2024-08-18T23:41:42.437425+0000","last_undegraded":"2024-08-18T23:41:42.437425+0000","last_fullsized":"2024-08-18T23:41:42.437425+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:10.918624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324188+0000","last_change":"2024-08-18T23:41:30.125160+0000","last_active":"2024-08-18T23:42:23.324188+0000","last_peered":"2024-08-18T23:42:23.324188+0000","last_clean":"2024-08-18T23:42:23.324188+0000","last_became_active":"2024-08-18T23:41:30.124685+0000","last_became_peered":"2024-08-18T23:41:30.124685+0000","last_unstale":"2024-08-18T23:42:23.324188+0000","last_undegraded":"2024-08-18T23:42:23.324188+0000","last_fullsized":"2024-08-18T23:42:23.324188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:52:48.997902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437021+0000","last_change":"2024-08-18T23:41:39.681481+0000","last_active":"2024-08-18T23:41:42.437021+0000","last_peered":"2024-08-18T23:41:42.437021+0000","last_clean":"2024-08-18T23:41:42.437021+0000","last_became_active":"2024-08-18T23:41:39.680204+0000","last_became_peered":"2024-08-18T23:41:39.680204+0000","last_unstale":"2024-08-18T23:41:42.437021+0000","last_undegraded":"2024-08-18T23:41:42.437021+0000","last_fullsized":"2024-08-18T23:41:42.437021+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:00:05.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436405+0000","last_change":"2024-08-18T23:41:32.151335+0000","last_active":"2024-08-18T23:41:42.436405+0000","last_peered":"2024-08-18T23:41:42.436405+0000","last_clean":"2024-08-18T23:41:42.436405+0000","last_became_active":"2024-08-18T23:41:32.151040+0000","last_became_peered":"2024-08-18T23:41:32.151040+0000","last_unstale":"2024-08-18T23:41:42.436405+0000","last_undegraded":"2024-08-18T23:41:42.436405+0000","last_fullsized":"2024-08-18T23:41:42.436405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:16.108555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436860+0000","last_change":"2024-08-18T23:41:30.122301+0000","last_active":"2024-08-18T23:41:42.436860+0000","last_peered":"2024-08-18T23:41:42.436860+0000","last_clean":"2024-08-18T23:41:42.436860+0000","last_became_active":"2024-08-18T23:41:30.121117+0000","last_became_peered":"2024-08-18T23:41:30.121117+0000","last_unstale":"2024-08-18T23:41:42.436860+0000","last_undegraded":"2024-08-18T23:41:42.436860+0000","last_fullsized":"2024-08-18T23:41:42.436860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:40:12.678803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437653+0000","last_change":"2024-08-18T23:41:39.182395+0000","last_active":"2024-08-18T23:41:42.437653+0000","last_peered":"2024-08-18T23:41:42.437653+0000","last_clean":"2024-08-18T23:41:42.437653+0000","last_became_active":"2024-08-18T23:41:39.181979+0000","last_became_peered":"2024-08-18T23:41:39.181979+0000","last_unstale":"2024-08-18T23:41:42.437653+0000","last_undegraded":"2024-08-18T23:41:42.437653+0000","last_fullsized":"2024-08-18T23:41:42.437653+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:04.674503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.310565+0000","last_change":"2024-08-18T23:41:32.140464+0000","last_active":"2024-08-18T23:42:22.310565+0000","last_peered":"2024-08-18T23:42:22.310565+0000","last_clean":"2024-08-18T23:42:22.310565+0000","last_became_active":"2024-08-18T23:41:32.139153+0000","last_became_peered":"2024-08-18T23:41:32.139153+0000","last_unstale":"2024-08-18T23:42:22.310565+0000","last_undegraded":"2024-08-18T23:42:22.310565+0000","last_fullsized":"2024-08-18T23:42:22.310565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:27.387680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:06.256815+0000","last_change":"2024-08-18T23:41:30.122787+0000","last_active":"2024-08-18T23:42:06.256815+0000","last_peered":"2024-08-18T23:42:06.256815+0000","last_clean":"2024-08-18T23:42:06.256815+0000","last_became_active":"2024-08-18T23:41:30.122462+0000","last_became_peered":"2024-08-18T23:41:30.122462+0000","last_unstale":"2024-08-18T23:42:06.256815+0000","last_undegraded":"2024-08-18T23:42:06.256815+0000","last_fullsized":"2024-08-18T23:42:06.256815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:55.568831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.184346+0000","last_change":"2024-08-18T23:41:39.190511+0000","last_active":"2024-08-18T23:41:42.184346+0000","last_peered":"2024-08-18T23:41:42.184346+0000","last_clean":"2024-08-18T23:41:42.184346+0000","last_became_active":"2024-08-18T23:41:39.190154+0000","last_became_peered":"2024-08-18T23:41:39.190154+0000","last_unstale":"2024-08-18T23:41:42.184346+0000","last_undegraded":"2024-08-18T23:41:42.184346+0000","last_fullsized":"2024-08-18T23:41:42.184346+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:33:34.514827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323413+0000","last_change":"2024-08-18T23:41:32.143867+0000","last_active":"2024-08-18T23:42:23.323413+0000","last_peered":"2024-08-18T23:42:23.323413+0000","last_clean":"2024-08-18T23:42:23.323413+0000","last_became_active":"2024-08-18T23:41:32.143527+0000","last_became_peered":"2024-08-18T23:41:32.143527+0000","last_unstale":"2024-08-18T23:42:23.323413+0000","last_undegraded":"2024-08-18T23:42:23.323413+0000","last_fullsized":"2024-08-18T23:42:23.323413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:40:25.902133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:06.257456+0000","last_change":"2024-08-18T23:41:30.136519+0000","last_active":"2024-08-18T23:42:06.257456+0000","last_peered":"2024-08-18T23:42:06.257456+0000","last_clean":"2024-08-18T23:42:06.257456+0000","last_became_active":"2024-08-18T23:41:30.136193+0000","last_became_peered":"2024-08-18T23:41:30.136193+0000","last_unstale":"2024-08-18T23:42:06.257456+0000","last_undegraded":"2024-08-18T23:42:06.257456+0000","last_fullsized":"2024-08-18T23:42:06.257456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:25:18.061897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436916+0000","last_change":"2024-08-18T23:41:39.680848+0000","last_active":"2024-08-18T23:41:42.436916+0000","last_peered":"2024-08-18T23:41:42.436916+0000","last_clean":"2024-08-18T23:41:42.436916+0000","last_became_active":"2024-08-18T23:41:39.680375+0000","last_became_peered":"2024-08-18T23:41:39.680375+0000","last_unstale":"2024-08-18T23:41:42.436916+0000","last_undegraded":"2024-08-18T23:41:42.436916+0000","last_fullsized":"2024-08-18T23:41:42.436916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:50.041110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436985+0000","last_change":"2024-08-18T23:41:32.139025+0000","last_active":"2024-08-18T23:41:42.436985+0000","last_peered":"2024-08-18T23:41:42.436985+0000","last_clean":"2024-08-18T23:41:42.436985+0000","last_became_active":"2024-08-18T23:41:32.138733+0000","last_became_peered":"2024-08-18T23:41:32.138733+0000","last_unstale":"2024-08-18T23:41:42.436985+0000","last_undegraded":"2024-08-18T23:41:42.436985+0000","last_fullsized":"2024-08-18T23:41:42.436985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:01:39.826391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:06.259829+0000","last_change":"2024-08-18T23:41:30.477359+0000","last_active":"2024-08-18T23:42:06.259829+0000","last_peered":"2024-08-18T23:42:06.259829+0000","last_clean":"2024-08-18T23:42:06.259829+0000","last_became_active":"2024-08-18T23:41:30.477011+0000","last_became_peered":"2024-08-18T23:41:30.477011+0000","last_unstale":"2024-08-18T23:42:06.259829+0000","last_undegraded":"2024-08-18T23:42:06.259829+0000","last_fullsized":"2024-08-18T23:42:06.259829+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:53:10.052859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438585+0000","last_change":"2024-08-18T23:41:39.182140+0000","last_active":"2024-08-18T23:41:42.438585+0000","last_peered":"2024-08-18T23:41:42.438585+0000","last_clean":"2024-08-18T23:41:42.438585+0000","last_became_active":"2024-08-18T23:41:39.181667+0000","last_became_peered":"2024-08-18T23:41:39.181667+0000","last_unstale":"2024-08-18T23:41:42.438585+0000","last_undegraded":"2024-08-18T23:41:42.438585+0000","last_fullsized":"2024-08-18T23:41:42.438585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:02:18.128028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436969+0000","last_change":"2024-08-18T23:41:32.137064+0000","last_active":"2024-08-18T23:41:42.436969+0000","last_peered":"2024-08-18T23:41:42.436969+0000","last_clean":"2024-08-18T23:41:42.436969+0000","last_became_active":"2024-08-18T23:41:32.136738+0000","last_became_peered":"2024-08-18T23:41:32.136738+0000","last_unstale":"2024-08-18T23:41:42.436969+0000","last_undegraded":"2024-08-18T23:41:42.436969+0000","last_fullsized":"2024-08-18T23:41:42.436969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:15:11.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'9","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:19.153483+0000","last_change":"2024-08-18T23:41:30.120042+0000","last_active":"2024-08-18T23:42:19.153483+0000","last_peered":"2024-08-18T23:42:19.153483+0000","last_clean":"2024-08-18T23:42:19.153483+0000","last_became_active":"2024-08-18T23:41:30.119719+0000","last_became_peered":"2024-08-18T23:41:30.119719+0000","last_unstale":"2024-08-18T23:42:19.153483+0000","last_undegraded":"2024-08-18T23:42:19.153483+0000","last_fullsized":"2024-08-18T23:42:19.153483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:35:44.466330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436182+0000","last_change":"2024-08-18T23:41:39.680405+0000","last_active":"2024-08-18T23:41:42.436182+0000","last_peered":"2024-08-18T23:41:42.436182+0000","last_clean":"2024-08-18T23:41:42.436182+0000","last_became_active":"2024-08-18T23:41:39.679940+0000","last_became_peered":"2024-08-18T23:41:39.679940+0000","last_unstale":"2024-08-18T23:41:42.436182+0000","last_undegraded":"2024-08-18T23:41:42.436182+0000","last_fullsized":"2024-08-18T23:41:42.436182+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:15:03.942116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481785+0000","last_change":"2024-08-18T23:41:32.139688+0000","last_active":"2024-08-18T23:41:42.481785+0000","last_peered":"2024-08-18T23:41:42.481785+0000","last_clean":"2024-08-18T23:41:42.481785+0000","last_became_active":"2024-08-18T23:41:32.139397+0000","last_became_peered":"2024-08-18T23:41:32.139397+0000","last_unstale":"2024-08-18T23:41:42.481785+0000","last_undegraded":"2024-08-18T23:41:42.481785+0000","last_fullsized":"2024-08-18T23:41:42.481785+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:03:57.017488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324876+0000","last_change":"2024-08-18T23:41:30.137578+0000","last_active":"2024-08-18T23:42:23.324876+0000","last_peered":"2024-08-18T23:42:23.324876+0000","last_clean":"2024-08-18T23:42:23.324876+0000","last_became_active":"2024-08-18T23:41:30.137246+0000","last_became_peered":"2024-08-18T23:41:30.137246+0000","last_unstale":"2024-08-18T23:42:23.324876+0000","last_undegraded":"2024-08-18T23:42:23.324876+0000","last_fullsized":"2024-08-18T23:42:23.324876+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:48:54.839432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.309910+0000","last_change":"2024-08-18T23:41:39.186360+0000","last_active":"2024-08-18T23:42:22.309910+0000","last_peered":"2024-08-18T23:42:22.309910+0000","last_clean":"2024-08-18T23:42:22.309910+0000","last_became_active":"2024-08-18T23:41:39.185808+0000","last_became_peered":"2024-08-18T23:41:39.185808+0000","last_unstale":"2024-08-18T23:42:22.309910+0000","last_undegraded":"2024-08-18T23:42:22.309910+0000","last_fullsized":"2024-08-18T23:42:22.309910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:53.891061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.181975+0000","last_change":"2024-08-18T23:41:32.146070+0000","last_active":"2024-08-18T23:41:42.181975+0000","last_peered":"2024-08-18T23:41:42.181975+0000","last_clean":"2024-08-18T23:41:42.181975+0000","last_became_active":"2024-08-18T23:41:32.145640+0000","last_became_peered":"2024-08-18T23:41:32.145640+0000","last_unstale":"2024-08-18T23:41:42.181975+0000","last_undegraded":"2024-08-18T23:41:42.181975+0000","last_fullsized":"2024-08-18T23:41:42.181975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:56:21.368762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.184447+0000","last_change":"2024-08-18T23:41:30.138565+0000","last_active":"2024-08-18T23:41:42.184447+0000","last_peered":"2024-08-18T23:41:42.184447+0000","last_clean":"2024-08-18T23:41:42.184447+0000","last_became_active":"2024-08-18T23:41:30.138210+0000","last_became_peered":"2024-08-18T23:41:30.138210+0000","last_unstale":"2024-08-18T23:41:42.184447+0000","last_undegraded":"2024-08-18T23:41:42.184447+0000","last_fullsized":"2024-08-18T23:41:42.184447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:42:14.928884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323149+0000","last_change":"2024-08-18T23:41:39.680656+0000","last_active":"2024-08-18T23:42:23.323149+0000","last_peered":"2024-08-18T23:42:23.323149+0000","last_clean":"2024-08-18T23:42:23.323149+0000","last_became_active":"2024-08-18T23:41:39.680203+0000","last_became_peered":"2024-08-18T23:41:39.680203+0000","last_unstale":"2024-08-18T23:42:23.323149+0000","last_undegraded":"2024-08-18T23:42:23.323149+0000","last_fullsized":"2024-08-18T23:42:23.323149+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:55:43.138869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.181940+0000","last_change":"2024-08-18T23:41:32.130222+0000","last_active":"2024-08-18T23:41:42.181940+0000","last_peered":"2024-08-18T23:41:42.181940+0000","last_clean":"2024-08-18T23:41:42.181940+0000","last_became_active":"2024-08-18T23:41:32.129844+0000","last_became_peered":"2024-08-18T23:41:32.129844+0000","last_unstale":"2024-08-18T23:41:42.181940+0000","last_undegraded":"2024-08-18T23:41:42.181940+0000","last_fullsized":"2024-08-18T23:41:42.181940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:39:21.106343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437643+0000","last_change":"2024-08-18T23:41:30.123142+0000","last_active":"2024-08-18T23:41:42.437643+0000","last_peered":"2024-08-18T23:41:42.437643+0000","last_clean":"2024-08-18T23:41:42.437643+0000","last_became_active":"2024-08-18T23:41:30.121645+0000","last_became_peered":"2024-08-18T23:41:30.121645+0000","last_unstale":"2024-08-18T23:41:42.437643+0000","last_undegraded":"2024-08-18T23:41:42.437643+0000","last_fullsized":"2024-08-18T23:41:42.437643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:07:24.912259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481697+0000","last_change":"2024-08-18T23:41:39.186033+0000","last_active":"2024-08-18T23:41:42.481697+0000","last_peered":"2024-08-18T23:41:42.481697+0000","last_clean":"2024-08-18T23:41:42.481697+0000","last_became_active":"2024-08-18T23:41:39.185451+0000","last_became_peered":"2024-08-18T23:41:39.185451+0000","last_unstale":"2024-08-18T23:41:42.481697+0000","last_undegraded":"2024-08-18T23:41:42.481697+0000","last_fullsized":"2024-08-18T23:41:42.481697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:52:13.556760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437517+0000","last_change":"2024-08-18T23:41:32.141065+0000","last_active":"2024-08-18T23:41:42.437517+0000","last_peered":"2024-08-18T23:41:42.437517+0000","last_clean":"2024-08-18T23:41:42.437517+0000","last_became_active":"2024-08-18T23:41:32.140732+0000","last_became_peered":"2024-08-18T23:41:32.140732+0000","last_unstale":"2024-08-18T23:41:42.437517+0000","last_undegraded":"2024-08-18T23:41:42.437517+0000","last_fullsized":"2024-08-18T23:41:42.437517+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:08:32.287835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:17.286692+0000","last_change":"2024-08-18T23:41:30.136838+0000","last_active":"2024-08-18T23:42:17.286692+0000","last_peered":"2024-08-18T23:42:17.286692+0000","last_clean":"2024-08-18T23:42:17.286692+0000","last_became_active":"2024-08-18T23:41:30.136523+0000","last_became_peered":"2024-08-18T23:41:30.136523+0000","last_unstale":"2024-08-18T23:42:17.286692+0000","last_undegraded":"2024-08-18T23:42:17.286692+0000","last_fullsized":"2024-08-18T23:42:17.286692+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:56:17.750535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481937+0000","last_change":"2024-08-18T23:41:39.680314+0000","last_active":"2024-08-18T23:41:42.481937+0000","last_peered":"2024-08-18T23:41:42.481937+0000","last_clean":"2024-08-18T23:41:42.481937+0000","last_became_active":"2024-08-18T23:41:39.679920+0000","last_became_peered":"2024-08-18T23:41:39.679920+0000","last_unstale":"2024-08-18T23:41:42.481937+0000","last_undegraded":"2024-08-18T23:41:42.481937+0000","last_fullsized":"2024-08-18T23:41:42.481937+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:26:29.866081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481393+0000","last_change":"2024-08-18T23:41:32.139752+0000","last_active":"2024-08-18T23:41:42.481393+0000","last_peered":"2024-08-18T23:41:42.481393+0000","last_clean":"2024-08-18T23:41:42.481393+0000","last_became_active":"2024-08-18T23:41:32.139451+0000","last_became_peered":"2024-08-18T23:41:32.139451+0000","last_unstale":"2024-08-18T23:41:42.481393+0000","last_undegraded":"2024-08-18T23:41:42.481393+0000","last_fullsized":"2024-08-18T23:41:42.481393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:15:13.390529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:07.263221+0000","last_change":"2024-08-18T23:41:30.126513+0000","last_active":"2024-08-18T23:42:07.263221+0000","last_peered":"2024-08-18T23:42:07.263221+0000","last_clean":"2024-08-18T23:42:07.263221+0000","last_became_active":"2024-08-18T23:41:30.126046+0000","last_became_peered":"2024-08-18T23:41:30.126046+0000","last_unstale":"2024-08-18T23:42:07.263221+0000","last_undegraded":"2024-08-18T23:42:07.263221+0000","last_fullsized":"2024-08-18T23:42:07.263221+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:49:16.565565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438321+0000","last_change":"2024-08-18T23:41:39.681006+0000","last_active":"2024-08-18T23:41:42.438321+0000","last_peered":"2024-08-18T23:41:42.438321+0000","last_clean":"2024-08-18T23:41:42.438321+0000","last_became_active":"2024-08-18T23:41:39.680409+0000","last_became_peered":"2024-08-18T23:41:39.680409+0000","last_unstale":"2024-08-18T23:41:42.438321+0000","last_undegraded":"2024-08-18T23:41:42.438321+0000","last_fullsized":"2024-08-18T23:41:42.438321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:47.778866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436780+0000","last_change":"2024-08-18T23:41:32.147545+0000","last_active":"2024-08-18T23:41:42.436780+0000","last_peered":"2024-08-18T23:41:42.436780+0000","last_clean":"2024-08-18T23:41:42.436780+0000","last_became_active":"2024-08-18T23:41:32.147238+0000","last_became_peered":"2024-08-18T23:41:32.147238+0000","last_unstale":"2024-08-18T23:41:42.436780+0000","last_undegraded":"2024-08-18T23:41:42.436780+0000","last_fullsized":"2024-08-18T23:41:42.436780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:12:13.589535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:08.269170+0000","last_change":"2024-08-18T23:41:30.120650+0000","last_active":"2024-08-18T23:42:08.269170+0000","last_peered":"2024-08-18T23:42:08.269170+0000","last_clean":"2024-08-18T23:42:08.269170+0000","last_became_active":"2024-08-18T23:41:30.120196+0000","last_became_peered":"2024-08-18T23:41:30.120196+0000","last_unstale":"2024-08-18T23:42:08.269170+0000","last_undegraded":"2024-08-18T23:42:08.269170+0000","last_fullsized":"2024-08-18T23:42:08.269170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:40:23.370088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437295+0000","last_change":"2024-08-18T23:40:22.013519+0000","last_active":"2024-08-18T23:41:42.437295+0000","last_peered":"2024-08-18T23:41:42.437295+0000","last_clean":"2024-08-18T23:41:42.437295+0000","last_became_active":"2024-08-18T23:40:22.012986+0000","last_became_peered":"2024-08-18T23:40:22.012986+0000","last_unstale":"2024-08-18T23:41:42.437295+0000","last_undegraded":"2024-08-18T23:41:42.437295+0000","last_fullsized":"2024-08-18T23:41:42.437295+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_clean_scrub_stamp":"2024-08-18T23:37:22.388424+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:19.874356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438358+0000","last_change":"2024-08-18T23:41:39.188400+0000","last_active":"2024-08-18T23:41:42.438358+0000","last_peered":"2024-08-18T23:41:42.438358+0000","last_clean":"2024-08-18T23:41:42.438358+0000","last_became_active":"2024-08-18T23:41:39.188023+0000","last_became_peered":"2024-08-18T23:41:39.188023+0000","last_unstale":"2024-08-18T23:41:42.438358+0000","last_undegraded":"2024-08-18T23:41:42.438358+0000","last_fullsized":"2024-08-18T23:41:42.438358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:07:33.845809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438418+0000","last_change":"2024-08-18T23:41:32.139907+0000","last_active":"2024-08-18T23:41:42.438418+0000","last_peered":"2024-08-18T23:41:42.438418+0000","last_clean":"2024-08-18T23:41:42.438418+0000","last_became_active":"2024-08-18T23:41:32.137956+0000","last_became_peered":"2024-08-18T23:41:32.137956+0000","last_unstale":"2024-08-18T23:41:42.438418+0000","last_undegraded":"2024-08-18T23:41:42.438418+0000","last_fullsized":"2024-08-18T23:41:42.438418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:10:13.223328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:07.263532+0000","last_change":"2024-08-18T23:41:30.477857+0000","last_active":"2024-08-18T23:42:07.263532+0000","last_peered":"2024-08-18T23:42:07.263532+0000","last_clean":"2024-08-18T23:42:07.263532+0000","last_became_active":"2024-08-18T23:41:30.477376+0000","last_became_peered":"2024-08-18T23:41:30.477376+0000","last_unstale":"2024-08-18T23:42:07.263532+0000","last_undegraded":"2024-08-18T23:42:07.263532+0000","last_fullsized":"2024-08-18T23:42:07.263532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:57:08.059818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436966+0000","last_change":"2024-08-18T23:41:39.185917+0000","last_active":"2024-08-18T23:41:42.436966+0000","last_peered":"2024-08-18T23:41:42.436966+0000","last_clean":"2024-08-18T23:41:42.436966+0000","last_became_active":"2024-08-18T23:41:39.185427+0000","last_became_peered":"2024-08-18T23:41:39.185427+0000","last_unstale":"2024-08-18T23:41:42.436966+0000","last_undegraded":"2024-08-18T23:41:42.436966+0000","last_fullsized":"2024-08-18T23:41:42.436966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:52:13.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437928+0000","last_change":"2024-08-18T23:41:32.147530+0000","last_active":"2024-08-18T23:41:42.437928+0000","last_peered":"2024-08-18T23:41:42.437928+0000","last_clean":"2024-08-18T23:41:42.437928+0000","last_became_active":"2024-08-18T23:41:32.147216+0000","last_became_peered":"2024-08-18T23:41:32.147216+0000","last_unstale":"2024-08-18T23:41:42.437928+0000","last_undegraded":"2024-08-18T23:41:42.437928+0000","last_fullsized":"2024-08-18T23:41:42.437928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:04:06.421490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438376+0000","last_change":"2024-08-18T23:41:30.121307+0000","last_active":"2024-08-18T23:41:42.438376+0000","last_peered":"2024-08-18T23:41:42.438376+0000","last_clean":"2024-08-18T23:41:42.438376+0000","last_became_active":"2024-08-18T23:41:30.120932+0000","last_became_peered":"2024-08-18T23:41:30.120932+0000","last_unstale":"2024-08-18T23:41:42.438376+0000","last_undegraded":"2024-08-18T23:41:42.438376+0000","last_fullsized":"2024-08-18T23:41:42.438376+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:06:07.742880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436674+0000","last_change":"2024-08-18T23:41:39.184647+0000","last_active":"2024-08-18T23:41:42.436674+0000","last_peered":"2024-08-18T23:41:42.436674+0000","last_clean":"2024-08-18T23:41:42.436674+0000","last_became_active":"2024-08-18T23:41:39.184251+0000","last_became_peered":"2024-08-18T23:41:39.184251+0000","last_unstale":"2024-08-18T23:41:42.436674+0000","last_undegraded":"2024-08-18T23:41:42.436674+0000","last_fullsized":"2024-08-18T23:41:42.436674+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:55.200801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437485+0000","last_change":"2024-08-18T23:41:32.137007+0000","last_active":"2024-08-18T23:41:42.437485+0000","last_peered":"2024-08-18T23:41:42.437485+0000","last_clean":"2024-08-18T23:41:42.437485+0000","last_became_active":"2024-08-18T23:41:32.136684+0000","last_became_peered":"2024-08-18T23:41:32.136684+0000","last_unstale":"2024-08-18T23:41:42.437485+0000","last_undegraded":"2024-08-18T23:41:42.437485+0000","last_fullsized":"2024-08-18T23:41:42.437485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:21:26.584303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481898+0000","last_change":"2024-08-18T23:41:30.121183+0000","last_active":"2024-08-18T23:41:42.481898+0000","last_peered":"2024-08-18T23:41:42.481898+0000","last_clean":"2024-08-18T23:41:42.481898+0000","last_became_active":"2024-08-18T23:41:30.120847+0000","last_became_peered":"2024-08-18T23:41:30.120847+0000","last_unstale":"2024-08-18T23:41:42.481898+0000","last_undegraded":"2024-08-18T23:41:42.481898+0000","last_fullsized":"2024-08-18T23:41:42.481898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:51.025821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437287+0000","last_change":"2024-08-18T23:41:39.183375+0000","last_active":"2024-08-18T23:41:42.437287+0000","last_peered":"2024-08-18T23:41:42.437287+0000","last_clean":"2024-08-18T23:41:42.437287+0000","last_became_active":"2024-08-18T23:41:39.182339+0000","last_became_peered":"2024-08-18T23:41:39.182339+0000","last_unstale":"2024-08-18T23:41:42.437287+0000","last_undegraded":"2024-08-18T23:41:42.437287+0000","last_fullsized":"2024-08-18T23:41:42.437287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:26.119056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.313184+0000","last_change":"2024-08-18T23:41:32.140748+0000","last_active":"2024-08-18T23:42:22.313184+0000","last_peered":"2024-08-18T23:42:22.313184+0000","last_clean":"2024-08-18T23:42:22.313184+0000","last_became_active":"2024-08-18T23:41:32.140306+0000","last_became_peered":"2024-08-18T23:41:32.140306+0000","last_unstale":"2024-08-18T23:42:22.313184+0000","last_undegraded":"2024-08-18T23:42:22.313184+0000","last_fullsized":"2024-08-18T23:42:22.313184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:23:02.371529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324632+0000","last_change":"2024-08-18T23:41:30.121617+0000","last_active":"2024-08-18T23:42:23.324632+0000","last_peered":"2024-08-18T23:42:23.324632+0000","last_clean":"2024-08-18T23:42:23.324632+0000","last_became_active":"2024-08-18T23:41:30.121257+0000","last_became_peered":"2024-08-18T23:41:30.121257+0000","last_unstale":"2024-08-18T23:42:23.324632+0000","last_undegraded":"2024-08-18T23:42:23.324632+0000","last_fullsized":"2024-08-18T23:42:23.324632+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:23:42.091553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438472+0000","last_change":"2024-08-18T23:41:39.183833+0000","last_active":"2024-08-18T23:41:42.438472+0000","last_peered":"2024-08-18T23:41:42.438472+0000","last_clean":"2024-08-18T23:41:42.438472+0000","last_became_active":"2024-08-18T23:41:39.183176+0000","last_became_peered":"2024-08-18T23:41:39.183176+0000","last_unstale":"2024-08-18T23:41:42.438472+0000","last_undegraded":"2024-08-18T23:41:42.438472+0000","last_fullsized":"2024-08-18T23:41:42.438472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:36:09.827378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438671+0000","last_change":"2024-08-18T23:41:32.147507+0000","last_active":"2024-08-18T23:41:42.438671+0000","last_peered":"2024-08-18T23:41:42.438671+0000","last_clean":"2024-08-18T23:41:42.438671+0000","last_became_active":"2024-08-18T23:41:32.146042+0000","last_became_peered":"2024-08-18T23:41:32.146042+0000","last_unstale":"2024-08-18T23:41:42.438671+0000","last_undegraded":"2024-08-18T23:41:42.438671+0000","last_fullsized":"2024-08-18T23:41:42.438671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:48.941442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323009+0000","last_change":"2024-08-18T23:41:30.137755+0000","last_active":"2024-08-18T23:42:23.323009+0000","last_peered":"2024-08-18T23:42:23.323009+0000","last_clean":"2024-08-18T23:42:23.323009+0000","last_became_active":"2024-08-18T23:41:30.137194+0000","last_became_peered":"2024-08-18T23:41:30.137194+0000","last_unstale":"2024-08-18T23:42:23.323009+0000","last_undegraded":"2024-08-18T23:42:23.323009+0000","last_fullsized":"2024-08-18T23:42:23.323009+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:15:49.681405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481932+0000","last_change":"2024-08-18T23:41:39.183986+0000","last_active":"2024-08-18T23:41:42.481932+0000","last_peered":"2024-08-18T23:41:42.481932+0000","last_clean":"2024-08-18T23:41:42.481932+0000","last_became_active":"2024-08-18T23:41:39.183384+0000","last_became_peered":"2024-08-18T23:41:39.183384+0000","last_unstale":"2024-08-18T23:41:42.481932+0000","last_undegraded":"2024-08-18T23:41:42.481932+0000","last_fullsized":"2024-08-18T23:41:42.481932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:04:28.920804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436902+0000","last_change":"2024-08-18T23:41:32.134729+0000","last_active":"2024-08-18T23:41:42.436902+0000","last_peered":"2024-08-18T23:41:42.436902+0000","last_clean":"2024-08-18T23:41:42.436902+0000","last_became_active":"2024-08-18T23:41:32.134366+0000","last_became_peered":"2024-08-18T23:41:32.134366+0000","last_unstale":"2024-08-18T23:41:42.436902+0000","last_undegraded":"2024-08-18T23:41:42.436902+0000","last_fullsized":"2024-08-18T23:41:42.436902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:38.610561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:18.288731+0000","last_change":"2024-08-18T23:41:30.126443+0000","last_active":"2024-08-18T23:42:18.288731+0000","last_peered":"2024-08-18T23:42:18.288731+0000","last_clean":"2024-08-18T23:42:18.288731+0000","last_became_active":"2024-08-18T23:41:30.124264+0000","last_became_peered":"2024-08-18T23:41:30.124264+0000","last_unstale":"2024-08-18T23:42:18.288731+0000","last_undegraded":"2024-08-18T23:42:18.288731+0000","last_fullsized":"2024-08-18T23:42:18.288731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:43.727114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.184435+0000","last_change":"2024-08-18T23:41:39.192164+0000","last_active":"2024-08-18T23:41:42.184435+0000","last_peered":"2024-08-18T23:41:42.184435+0000","last_clean":"2024-08-18T23:41:42.184435+0000","last_became_active":"2024-08-18T23:41:39.191833+0000","last_became_peered":"2024-08-18T23:41:39.191833+0000","last_unstale":"2024-08-18T23:41:42.184435+0000","last_undegraded":"2024-08-18T23:41:42.184435+0000","last_fullsized":"2024-08-18T23:41:42.184435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:33:50.243930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437783+0000","last_change":"2024-08-18T23:41:32.137384+0000","last_active":"2024-08-18T23:41:42.437783+0000","last_peered":"2024-08-18T23:41:42.437783+0000","last_clean":"2024-08-18T23:41:42.437783+0000","last_became_active":"2024-08-18T23:41:32.137051+0000","last_became_peered":"2024-08-18T23:41:32.137051+0000","last_unstale":"2024-08-18T23:41:42.437783+0000","last_undegraded":"2024-08-18T23:41:42.437783+0000","last_fullsized":"2024-08-18T23:41:42.437783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:22:04.367548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:06.259611+0000","last_change":"2024-08-18T23:41:30.124071+0000","last_active":"2024-08-18T23:42:06.259611+0000","last_peered":"2024-08-18T23:42:06.259611+0000","last_clean":"2024-08-18T23:42:06.259611+0000","last_became_active":"2024-08-18T23:41:30.123118+0000","last_became_peered":"2024-08-18T23:41:30.123118+0000","last_unstale":"2024-08-18T23:42:06.259611+0000","last_undegraded":"2024-08-18T23:42:06.259611+0000","last_fullsized":"2024-08-18T23:42:06.259611+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:21:39.015187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.310039+0000","last_change":"2024-08-18T23:41:39.184038+0000","last_active":"2024-08-18T23:42:22.310039+0000","last_peered":"2024-08-18T23:42:22.310039+0000","last_clean":"2024-08-18T23:42:22.310039+0000","last_became_active":"2024-08-18T23:41:39.183611+0000","last_became_peered":"2024-08-18T23:41:39.183611+0000","last_unstale":"2024-08-18T23:42:22.310039+0000","last_undegraded":"2024-08-18T23:42:22.310039+0000","last_fullsized":"2024-08-18T23:42:22.310039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:32:26.029151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481536+0000","last_change":"2024-08-18T23:41:32.143631+0000","last_active":"2024-08-18T23:41:42.481536+0000","last_peered":"2024-08-18T23:41:42.481536+0000","last_clean":"2024-08-18T23:41:42.481536+0000","last_became_active":"2024-08-18T23:41:32.143350+0000","last_became_peered":"2024-08-18T23:41:32.143350+0000","last_unstale":"2024-08-18T23:41:42.481536+0000","last_undegraded":"2024-08-18T23:41:42.481536+0000","last_fullsized":"2024-08-18T23:41:42.481536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:41:23.713467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:07.261492+0000","last_change":"2024-08-18T23:41:30.123383+0000","last_active":"2024-08-18T23:42:07.261492+0000","last_peered":"2024-08-18T23:42:07.261492+0000","last_clean":"2024-08-18T23:42:07.261492+0000","last_became_active":"2024-08-18T23:41:30.123037+0000","last_became_peered":"2024-08-18T23:41:30.123037+0000","last_unstale":"2024-08-18T23:42:07.261492+0000","last_undegraded":"2024-08-18T23:42:07.261492+0000","last_fullsized":"2024-08-18T23:42:07.261492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:27.702478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437947+0000","last_change":"2024-08-18T23:41:39.184302+0000","last_active":"2024-08-18T23:41:42.437947+0000","last_peered":"2024-08-18T23:41:42.437947+0000","last_clean":"2024-08-18T23:41:42.437947+0000","last_became_active":"2024-08-18T23:41:39.183172+0000","last_became_peered":"2024-08-18T23:41:39.183172+0000","last_unstale":"2024-08-18T23:41:42.437947+0000","last_undegraded":"2024-08-18T23:41:42.437947+0000","last_fullsized":"2024-08-18T23:41:42.437947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:23:26.655865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324579+0000","last_change":"2024-08-18T23:41:32.137426+0000","last_active":"2024-08-18T23:42:23.324579+0000","last_peered":"2024-08-18T23:42:23.324579+0000","last_clean":"2024-08-18T23:42:23.324579+0000","last_became_active":"2024-08-18T23:41:32.137106+0000","last_became_peered":"2024-08-18T23:41:32.137106+0000","last_unstale":"2024-08-18T23:42:23.324579+0000","last_undegraded":"2024-08-18T23:42:23.324579+0000","last_fullsized":"2024-08-18T23:42:23.324579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:34:51.889663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'26","reported_seq":86,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:19.291434+0000","last_change":"2024-08-18T23:41:30.124320+0000","last_active":"2024-08-18T23:42:19.291434+0000","last_peered":"2024-08-18T23:42:19.291434+0000","last_clean":"2024-08-18T23:42:19.291434+0000","last_became_active":"2024-08-18T23:41:30.123979+0000","last_became_peered":"2024-08-18T23:41:30.123979+0000","last_unstale":"2024-08-18T23:42:19.291434+0000","last_undegraded":"2024-08-18T23:42:19.291434+0000","last_fullsized":"2024-08-18T23:42:19.291434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:31:03.689720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49092,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":63,"num_write":27,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.184049+0000","last_change":"2024-08-18T23:41:39.190481+0000","last_active":"2024-08-18T23:41:42.184049+0000","last_peered":"2024-08-18T23:41:42.184049+0000","last_clean":"2024-08-18T23:41:42.184049+0000","last_became_active":"2024-08-18T23:41:39.190076+0000","last_became_peered":"2024-08-18T23:41:39.190076+0000","last_unstale":"2024-08-18T23:41:42.184049+0000","last_undegraded":"2024-08-18T23:41:42.184049+0000","last_fullsized":"2024-08-18T23:41:42.184049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:47:39.313142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437826+0000","last_change":"2024-08-18T23:41:32.145422+0000","last_active":"2024-08-18T23:41:42.437826+0000","last_peered":"2024-08-18T23:41:42.437826+0000","last_clean":"2024-08-18T23:41:42.437826+0000","last_became_active":"2024-08-18T23:41:32.145047+0000","last_became_peered":"2024-08-18T23:41:32.145047+0000","last_unstale":"2024-08-18T23:41:42.437826+0000","last_undegraded":"2024-08-18T23:41:42.437826+0000","last_fullsized":"2024-08-18T23:41:42.437826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:31.905248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:06.257992+0000","last_change":"2024-08-18T23:41:30.137364+0000","last_active":"2024-08-18T23:42:06.257992+0000","last_peered":"2024-08-18T23:42:06.257992+0000","last_clean":"2024-08-18T23:42:06.257992+0000","last_became_active":"2024-08-18T23:41:30.137004+0000","last_became_peered":"2024-08-18T23:41:30.137004+0000","last_unstale":"2024-08-18T23:42:06.257992+0000","last_undegraded":"2024-08-18T23:42:06.257992+0000","last_fullsized":"2024-08-18T23:42:06.257992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:20:56.392149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438360+0000","last_change":"2024-08-18T23:41:39.184029+0000","last_active":"2024-08-18T23:41:42.438360+0000","last_peered":"2024-08-18T23:41:42.438360+0000","last_clean":"2024-08-18T23:41:42.438360+0000","last_became_active":"2024-08-18T23:41:39.183159+0000","last_became_peered":"2024-08-18T23:41:39.183159+0000","last_unstale":"2024-08-18T23:41:42.438360+0000","last_undegraded":"2024-08-18T23:41:42.438360+0000","last_fullsized":"2024-08-18T23:41:42.438360+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:51:47.608798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437221+0000","last_change":"2024-08-18T23:41:32.142424+0000","last_active":"2024-08-18T23:41:42.437221+0000","last_peered":"2024-08-18T23:41:42.437221+0000","last_clean":"2024-08-18T23:41:42.437221+0000","last_became_active":"2024-08-18T23:41:32.141936+0000","last_became_peered":"2024-08-18T23:41:32.141936+0000","last_unstale":"2024-08-18T23:41:42.437221+0000","last_undegraded":"2024-08-18T23:41:42.437221+0000","last_fullsized":"2024-08-18T23:41:42.437221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:05:47.192928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:06.259464+0000","last_change":"2024-08-18T23:41:30.124430+0000","last_active":"2024-08-18T23:42:06.259464+0000","last_peered":"2024-08-18T23:42:06.259464+0000","last_clean":"2024-08-18T23:42:06.259464+0000","last_became_active":"2024-08-18T23:41:30.123626+0000","last_became_peered":"2024-08-18T23:41:30.123626+0000","last_unstale":"2024-08-18T23:42:06.259464+0000","last_undegraded":"2024-08-18T23:42:06.259464+0000","last_fullsized":"2024-08-18T23:42:06.259464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:26:18.426010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.183907+0000","last_change":"2024-08-18T23:41:39.192765+0000","last_active":"2024-08-18T23:41:42.183907+0000","last_peered":"2024-08-18T23:41:42.183907+0000","last_clean":"2024-08-18T23:41:42.183907+0000","last_became_active":"2024-08-18T23:41:39.192250+0000","last_became_peered":"2024-08-18T23:41:39.192250+0000","last_unstale":"2024-08-18T23:41:42.183907+0000","last_undegraded":"2024-08-18T23:41:42.183907+0000","last_fullsized":"2024-08-18T23:41:42.183907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:51:16.794404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438562+0000","last_change":"2024-08-18T23:41:32.141966+0000","last_active":"2024-08-18T23:41:42.438562+0000","last_peered":"2024-08-18T23:41:42.438562+0000","last_clean":"2024-08-18T23:41:42.438562+0000","last_became_active":"2024-08-18T23:41:32.141561+0000","last_became_peered":"2024-08-18T23:41:32.141561+0000","last_unstale":"2024-08-18T23:41:42.438562+0000","last_undegraded":"2024-08-18T23:41:42.438562+0000","last_fullsized":"2024-08-18T23:41:42.438562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:20.696089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:18.288089+0000","last_change":"2024-08-18T23:41:30.121598+0000","last_active":"2024-08-18T23:42:18.288089+0000","last_peered":"2024-08-18T23:42:18.288089+0000","last_clean":"2024-08-18T23:42:18.288089+0000","last_became_active":"2024-08-18T23:41:30.120533+0000","last_became_peered":"2024-08-18T23:41:30.120533+0000","last_unstale":"2024-08-18T23:42:18.288089+0000","last_undegraded":"2024-08-18T23:42:18.288089+0000","last_fullsized":"2024-08-18T23:42:18.288089+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:06:37.458272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436352+0000","last_change":"2024-08-18T23:41:39.178877+0000","last_active":"2024-08-18T23:41:42.436352+0000","last_peered":"2024-08-18T23:41:42.436352+0000","last_clean":"2024-08-18T23:41:42.436352+0000","last_became_active":"2024-08-18T23:41:39.178381+0000","last_became_peered":"2024-08-18T23:41:39.178381+0000","last_unstale":"2024-08-18T23:41:42.436352+0000","last_undegraded":"2024-08-18T23:41:42.436352+0000","last_fullsized":"2024-08-18T23:41:42.436352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:27.275893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436858+0000","last_change":"2024-08-18T23:41:39.188381+0000","last_active":"2024-08-18T23:41:42.436858+0000","last_peered":"2024-08-18T23:41:42.436858+0000","last_clean":"2024-08-18T23:41:42.436858+0000","last_became_active":"2024-08-18T23:41:39.187983+0000","last_became_peered":"2024-08-18T23:41:39.187983+0000","last_unstale":"2024-08-18T23:41:42.436858+0000","last_undegraded":"2024-08-18T23:41:42.436858+0000","last_fullsized":"2024-08-18T23:41:42.436858+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:04:45.460576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.183985+0000","last_change":"2024-08-18T23:41:32.144637+0000","last_active":"2024-08-18T23:41:42.183985+0000","last_peered":"2024-08-18T23:41:42.183985+0000","last_clean":"2024-08-18T23:41:42.183985+0000","last_became_active":"2024-08-18T23:41:32.144329+0000","last_became_peered":"2024-08-18T23:41:32.144329+0000","last_unstale":"2024-08-18T23:41:42.183985+0000","last_undegraded":"2024-08-18T23:41:42.183985+0000","last_fullsized":"2024-08-18T23:41:42.183985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:29:39.791264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437701+0000","last_change":"2024-08-18T23:41:30.122325+0000","last_active":"2024-08-18T23:41:42.437701+0000","last_peered":"2024-08-18T23:41:42.437701+0000","last_clean":"2024-08-18T23:41:42.437701+0000","last_became_active":"2024-08-18T23:41:30.121480+0000","last_became_peered":"2024-08-18T23:41:30.121480+0000","last_unstale":"2024-08-18T23:41:42.437701+0000","last_undegraded":"2024-08-18T23:41:42.437701+0000","last_fullsized":"2024-08-18T23:41:42.437701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:32:12.810650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"peering","last_fresh":"2024-08-18T23:41:42.471784+0000","last_change":"2024-08-18T23:42:22.309631+0000","last_active":"2024-08-18T23:42:22.309631+0000","last_peered":"2024-08-18T23:41:42.471784+0000","last_clean":"2024-08-18T23:41:42.471784+0000","last_became_active":"2024-08-18T23:41:39.185811+0000","last_became_peered":"2024-08-18T23:41:39.185811+0000","last_unstale":"2024-08-18T23:41:42.471784+0000","last_undegraded":"2024-08-18T23:41:42.471784+0000","last_fullsized":"2024-08-18T23:41:42.471784+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4,9,10],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436290+0000","last_change":"2024-08-18T23:41:32.145545+0000","last_active":"2024-08-18T23:41:42.436290+0000","last_peered":"2024-08-18T23:41:42.436290+0000","last_clean":"2024-08-18T23:41:42.436290+0000","last_became_active":"2024-08-18T23:41:32.145183+0000","last_became_peered":"2024-08-18T23:41:32.145183+0000","last_unstale":"2024-08-18T23:41:42.436290+0000","last_undegraded":"2024-08-18T23:41:42.436290+0000","last_fullsized":"2024-08-18T23:41:42.436290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:03:04.111658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:07.261928+0000","last_change":"2024-08-18T23:41:30.118776+0000","last_active":"2024-08-18T23:42:07.261928+0000","last_peered":"2024-08-18T23:42:07.261928+0000","last_clean":"2024-08-18T23:42:07.261928+0000","last_became_active":"2024-08-18T23:41:30.118436+0000","last_became_peered":"2024-08-18T23:41:30.118436+0000","last_unstale":"2024-08-18T23:42:07.261928+0000","last_undegraded":"2024-08-18T23:42:07.261928+0000","last_fullsized":"2024-08-18T23:42:07.261928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:13.980927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:07.262089+0000","last_change":"2024-08-18T23:41:30.125201+0000","last_active":"2024-08-18T23:42:07.262089+0000","last_peered":"2024-08-18T23:42:07.262089+0000","last_clean":"2024-08-18T23:42:07.262089+0000","last_became_active":"2024-08-18T23:41:30.124894+0000","last_became_peered":"2024-08-18T23:41:30.124894+0000","last_unstale":"2024-08-18T23:42:07.262089+0000","last_undegraded":"2024-08-18T23:42:07.262089+0000","last_fullsized":"2024-08-18T23:42:07.262089+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:30:17.721829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437803+0000","last_change":"2024-08-18T23:41:32.145452+0000","last_active":"2024-08-18T23:41:42.437803+0000","last_peered":"2024-08-18T23:41:42.437803+0000","last_clean":"2024-08-18T23:41:42.437803+0000","last_became_active":"2024-08-18T23:41:32.145151+0000","last_became_peered":"2024-08-18T23:41:32.145151+0000","last_unstale":"2024-08-18T23:41:42.437803+0000","last_undegraded":"2024-08-18T23:41:42.437803+0000","last_fullsized":"2024-08-18T23:41:42.437803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:08:08.713154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438790+0000","last_change":"2024-08-18T23:41:39.193334+0000","last_active":"2024-08-18T23:41:42.438790+0000","last_peered":"2024-08-18T23:41:42.438790+0000","last_clean":"2024-08-18T23:41:42.438790+0000","last_became_active":"2024-08-18T23:41:39.192924+0000","last_became_peered":"2024-08-18T23:41:39.192924+0000","last_unstale":"2024-08-18T23:41:42.438790+0000","last_undegraded":"2024-08-18T23:41:42.438790+0000","last_fullsized":"2024-08-18T23:41:42.438790+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:07.016300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:20.291145+0000","last_change":"2024-08-18T23:41:30.125217+0000","last_active":"2024-08-18T23:42:20.291145+0000","last_peered":"2024-08-18T23:42:20.291145+0000","last_clean":"2024-08-18T23:42:20.291145+0000","last_became_active":"2024-08-18T23:41:30.124889+0000","last_became_peered":"2024-08-18T23:41:30.124889+0000","last_unstale":"2024-08-18T23:42:20.291145+0000","last_undegraded":"2024-08-18T23:42:20.291145+0000","last_fullsized":"2024-08-18T23:42:20.291145+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:43.389537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323355+0000","last_change":"2024-08-18T23:41:32.142585+0000","last_active":"2024-08-18T23:42:23.323355+0000","last_peered":"2024-08-18T23:42:23.323355+0000","last_clean":"2024-08-18T23:42:23.323355+0000","last_became_active":"2024-08-18T23:41:32.142207+0000","last_became_peered":"2024-08-18T23:41:32.142207+0000","last_unstale":"2024-08-18T23:42:23.323355+0000","last_undegraded":"2024-08-18T23:42:23.323355+0000","last_fullsized":"2024-08-18T23:42:23.323355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:16:58.755444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437736+0000","last_change":"2024-08-18T23:41:39.185618+0000","last_active":"2024-08-18T23:41:42.437736+0000","last_peered":"2024-08-18T23:41:42.437736+0000","last_clean":"2024-08-18T23:41:42.437736+0000","last_became_active":"2024-08-18T23:41:39.185222+0000","last_became_peered":"2024-08-18T23:41:39.185222+0000","last_unstale":"2024-08-18T23:41:42.437736+0000","last_undegraded":"2024-08-18T23:41:42.437736+0000","last_fullsized":"2024-08-18T23:41:42.437736+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:01:27.290998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.309157+0000","last_change":"2024-08-18T23:41:32.140719+0000","last_active":"2024-08-18T23:42:22.309157+0000","last_peered":"2024-08-18T23:42:22.309157+0000","last_clean":"2024-08-18T23:42:22.309157+0000","last_became_active":"2024-08-18T23:41:32.140131+0000","last_became_peered":"2024-08-18T23:41:32.140131+0000","last_unstale":"2024-08-18T23:42:22.309157+0000","last_undegraded":"2024-08-18T23:42:22.309157+0000","last_fullsized":"2024-08-18T23:42:22.309157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:48:26.778094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438674+0000","last_change":"2024-08-18T23:41:32.138778+0000","last_active":"2024-08-18T23:41:42.438674+0000","last_peered":"2024-08-18T23:41:42.438674+0000","last_clean":"2024-08-18T23:41:42.438674+0000","last_became_active":"2024-08-18T23:41:32.138408+0000","last_became_peered":"2024-08-18T23:41:32.138408+0000","last_unstale":"2024-08-18T23:41:42.438674+0000","last_undegraded":"2024-08-18T23:41:42.438674+0000","last_fullsized":"2024-08-18T23:41:42.438674+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:29:41.621518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436698+0000","last_change":"2024-08-18T23:41:32.149522+0000","last_active":"2024-08-18T23:41:42.436698+0000","last_peered":"2024-08-18T23:41:42.436698+0000","last_clean":"2024-08-18T23:41:42.436698+0000","last_became_active":"2024-08-18T23:41:32.149182+0000","last_became_peered":"2024-08-18T23:41:32.149182+0000","last_unstale":"2024-08-18T23:41:42.436698+0000","last_undegraded":"2024-08-18T23:41:42.436698+0000","last_fullsized":"2024-08-18T23:41:42.436698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:30:16.945045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436641+0000","last_change":"2024-08-18T23:41:32.149853+0000","last_active":"2024-08-18T23:41:42.436641+0000","last_peered":"2024-08-18T23:41:42.436641+0000","last_clean":"2024-08-18T23:41:42.436641+0000","last_became_active":"2024-08-18T23:41:32.141907+0000","last_became_peered":"2024-08-18T23:41:32.141907+0000","last_unstale":"2024-08-18T23:41:42.436641+0000","last_undegraded":"2024-08-18T23:41:42.436641+0000","last_fullsized":"2024-08-18T23:41:42.436641+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:24:37.323567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481741+0000","last_change":"2024-08-18T23:41:32.136179+0000","last_active":"2024-08-18T23:41:42.481741+0000","last_peered":"2024-08-18T23:41:42.481741+0000","last_clean":"2024-08-18T23:41:42.481741+0000","last_became_active":"2024-08-18T23:41:32.135939+0000","last_became_peered":"2024-08-18T23:41:32.135939+0000","last_unstale":"2024-08-18T23:41:42.481741+0000","last_undegraded":"2024-08-18T23:41:42.481741+0000","last_fullsized":"2024-08-18T23:41:42.481741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:15.392061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.310009+0000","last_change":"2024-08-18T23:41:32.137509+0000","last_active":"2024-08-18T23:42:22.310009+0000","last_peered":"2024-08-18T23:42:22.310009+0000","last_clean":"2024-08-18T23:42:22.310009+0000","last_became_active":"2024-08-18T23:41:32.136380+0000","last_became_peered":"2024-08-18T23:41:32.136380+0000","last_unstale":"2024-08-18T23:42:22.310009+0000","last_undegraded":"2024-08-18T23:42:22.310009+0000","last_fullsized":"2024-08-18T23:42:22.310009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:49.776225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438165+0000","last_change":"2024-08-18T23:41:32.139271+0000","last_active":"2024-08-18T23:41:42.438165+0000","last_peered":"2024-08-18T23:41:42.438165+0000","last_clean":"2024-08-18T23:41:42.438165+0000","last_became_active":"2024-08-18T23:41:32.138868+0000","last_became_peered":"2024-08-18T23:41:32.138868+0000","last_unstale":"2024-08-18T23:41:42.438165+0000","last_undegraded":"2024-08-18T23:41:42.438165+0000","last_fullsized":"2024-08-18T23:41:42.438165+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:49:27.890857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436507+0000","last_change":"2024-08-18T23:41:32.144460+0000","last_active":"2024-08-18T23:41:42.436507+0000","last_peered":"2024-08-18T23:41:42.436507+0000","last_clean":"2024-08-18T23:41:42.436507+0000","last_became_active":"2024-08-18T23:41:32.144178+0000","last_became_peered":"2024-08-18T23:41:32.144178+0000","last_unstale":"2024-08-18T23:41:42.436507+0000","last_undegraded":"2024-08-18T23:41:42.436507+0000","last_fullsized":"2024-08-18T23:41:42.436507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:56.252215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437434+0000","last_change":"2024-08-18T23:41:32.147973+0000","last_active":"2024-08-18T23:41:42.437434+0000","last_peered":"2024-08-18T23:41:42.437434+0000","last_clean":"2024-08-18T23:41:42.437434+0000","last_became_active":"2024-08-18T23:41:32.147621+0000","last_became_peered":"2024-08-18T23:41:32.147621+0000","last_unstale":"2024-08-18T23:41:42.437434+0000","last_undegraded":"2024-08-18T23:41:42.437434+0000","last_fullsized":"2024-08-18T23:41:42.437434+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:14.176569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436418+0000","last_change":"2024-08-18T23:41:32.146029+0000","last_active":"2024-08-18T23:41:42.436418+0000","last_peered":"2024-08-18T23:41:42.436418+0000","last_clean":"2024-08-18T23:41:42.436418+0000","last_became_active":"2024-08-18T23:41:32.145584+0000","last_became_peered":"2024-08-18T23:41:32.145584+0000","last_unstale":"2024-08-18T23:41:42.436418+0000","last_undegraded":"2024-08-18T23:41:42.436418+0000","last_fullsized":"2024-08-18T23:41:42.436418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:25:33.884906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437194+0000","last_change":"2024-08-18T23:41:32.145950+0000","last_active":"2024-08-18T23:41:42.437194+0000","last_peered":"2024-08-18T23:41:42.437194+0000","last_clean":"2024-08-18T23:41:42.437194+0000","last_became_active":"2024-08-18T23:41:32.145618+0000","last_became_peered":"2024-08-18T23:41:32.145618+0000","last_unstale":"2024-08-18T23:41:42.437194+0000","last_undegraded":"2024-08-18T23:41:42.437194+0000","last_fullsized":"2024-08-18T23:41:42.437194+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:05:58.889240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481678+0000","last_change":"2024-08-18T23:41:32.130684+0000","last_active":"2024-08-18T23:41:42.481678+0000","last_peered":"2024-08-18T23:41:42.481678+0000","last_clean":"2024-08-18T23:41:42.481678+0000","last_became_active":"2024-08-18T23:41:32.130369+0000","last_became_peered":"2024-08-18T23:41:32.130369+0000","last_unstale":"2024-08-18T23:41:42.481678+0000","last_undegraded":"2024-08-18T23:41:42.481678+0000","last_fullsized":"2024-08-18T23:41:42.481678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:22.537018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.310146+0000","last_change":"2024-08-18T23:41:32.137291+0000","last_active":"2024-08-18T23:42:22.310146+0000","last_peered":"2024-08-18T23:42:22.310146+0000","last_clean":"2024-08-18T23:42:22.310146+0000","last_became_active":"2024-08-18T23:41:32.136977+0000","last_became_peered":"2024-08-18T23:41:32.136977+0000","last_unstale":"2024-08-18T23:42:22.310146+0000","last_undegraded":"2024-08-18T23:42:22.310146+0000","last_fullsized":"2024-08-18T23:42:22.310146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:18:45.544760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437895+0000","last_change":"2024-08-18T23:41:32.144434+0000","last_active":"2024-08-18T23:41:42.437895+0000","last_peered":"2024-08-18T23:41:42.437895+0000","last_clean":"2024-08-18T23:41:42.437895+0000","last_became_active":"2024-08-18T23:41:32.144119+0000","last_became_peered":"2024-08-18T23:41:32.144119+0000","last_unstale":"2024-08-18T23:41:42.437895+0000","last_undegraded":"2024-08-18T23:41:42.437895+0000","last_fullsized":"2024-08-18T23:41:42.437895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:50:54.528630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.184493+0000","last_change":"2024-08-18T23:41:32.135034+0000","last_active":"2024-08-18T23:41:42.184493+0000","last_peered":"2024-08-18T23:41:42.184493+0000","last_clean":"2024-08-18T23:41:42.184493+0000","last_became_active":"2024-08-18T23:41:32.134700+0000","last_became_peered":"2024-08-18T23:41:32.134700+0000","last_unstale":"2024-08-18T23:41:42.184493+0000","last_undegraded":"2024-08-18T23:41:42.184493+0000","last_fullsized":"2024-08-18T23:41:42.184493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:06:31.263114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.310319+0000","last_change":"2024-08-18T23:41:32.141191+0000","last_active":"2024-08-18T23:42:22.310319+0000","last_peered":"2024-08-18T23:42:22.310319+0000","last_clean":"2024-08-18T23:42:22.310319+0000","last_became_active":"2024-08-18T23:41:32.140901+0000","last_became_peered":"2024-08-18T23:41:32.140901+0000","last_unstale":"2024-08-18T23:42:22.310319+0000","last_undegraded":"2024-08-18T23:42:22.310319+0000","last_fullsized":"2024-08-18T23:42:22.310319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:24.849985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437507+0000","last_change":"2024-08-18T23:41:32.142543+0000","last_active":"2024-08-18T23:41:42.437507+0000","last_peered":"2024-08-18T23:41:42.437507+0000","last_clean":"2024-08-18T23:41:42.437507+0000","last_became_active":"2024-08-18T23:41:32.142084+0000","last_became_peered":"2024-08-18T23:41:32.142084+0000","last_unstale":"2024-08-18T23:41:42.437507+0000","last_undegraded":"2024-08-18T23:41:42.437507+0000","last_fullsized":"2024-08-18T23:41:42.437507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:54:18.200016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436457+0000","last_change":"2024-08-18T23:41:32.148968+0000","last_active":"2024-08-18T23:41:42.436457+0000","last_peered":"2024-08-18T23:41:42.436457+0000","last_clean":"2024-08-18T23:41:42.436457+0000","last_became_active":"2024-08-18T23:41:32.148625+0000","last_became_peered":"2024-08-18T23:41:32.148625+0000","last_unstale":"2024-08-18T23:41:42.436457+0000","last_undegraded":"2024-08-18T23:41:42.436457+0000","last_fullsized":"2024-08-18T23:41:42.436457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:11:40.172139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436410+0000","last_change":"2024-08-18T23:41:32.133478+0000","last_active":"2024-08-18T23:41:42.436410+0000","last_peered":"2024-08-18T23:41:42.436410+0000","last_clean":"2024-08-18T23:41:42.436410+0000","last_became_active":"2024-08-18T23:41:32.133149+0000","last_became_peered":"2024-08-18T23:41:32.133149+0000","last_unstale":"2024-08-18T23:41:42.436410+0000","last_undegraded":"2024-08-18T23:41:42.436410+0000","last_fullsized":"2024-08-18T23:41:42.436410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:57:53.713309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.481649+0000","last_change":"2024-08-18T23:41:32.138206+0000","last_active":"2024-08-18T23:41:42.481649+0000","last_peered":"2024-08-18T23:41:42.481649+0000","last_clean":"2024-08-18T23:41:42.481649+0000","last_became_active":"2024-08-18T23:41:32.137903+0000","last_became_peered":"2024-08-18T23:41:32.137903+0000","last_unstale":"2024-08-18T23:41:42.481649+0000","last_undegraded":"2024-08-18T23:41:42.481649+0000","last_fullsized":"2024-08-18T23:41:42.481649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:18:48.600658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437946+0000","last_change":"2024-08-18T23:41:32.136303+0000","last_active":"2024-08-18T23:41:42.437946+0000","last_peered":"2024-08-18T23:41:42.437946+0000","last_clean":"2024-08-18T23:41:42.437946+0000","last_became_active":"2024-08-18T23:41:32.134628+0000","last_became_peered":"2024-08-18T23:41:32.134628+0000","last_unstale":"2024-08-18T23:41:42.437946+0000","last_undegraded":"2024-08-18T23:41:42.437946+0000","last_fullsized":"2024-08-18T23:41:42.437946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:28:51.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438368+0000","last_change":"2024-08-18T23:41:32.137512+0000","last_active":"2024-08-18T23:41:42.438368+0000","last_peered":"2024-08-18T23:41:42.438368+0000","last_clean":"2024-08-18T23:41:42.438368+0000","last_became_active":"2024-08-18T23:41:32.137150+0000","last_became_peered":"2024-08-18T23:41:32.137150+0000","last_unstale":"2024-08-18T23:41:42.438368+0000","last_undegraded":"2024-08-18T23:41:42.438368+0000","last_fullsized":"2024-08-18T23:41:42.438368+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:44:08.020780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.182271+0000","last_change":"2024-08-18T23:41:32.139229+0000","last_active":"2024-08-18T23:41:42.182271+0000","last_peered":"2024-08-18T23:41:42.182271+0000","last_clean":"2024-08-18T23:41:42.182271+0000","last_became_active":"2024-08-18T23:41:32.138864+0000","last_became_peered":"2024-08-18T23:41:32.138864+0000","last_unstale":"2024-08-18T23:41:42.182271+0000","last_undegraded":"2024-08-18T23:41:42.182271+0000","last_fullsized":"2024-08-18T23:41:42.182271+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:54:21.701750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437965+0000","last_change":"2024-08-18T23:41:32.148727+0000","last_active":"2024-08-18T23:41:42.437965+0000","last_peered":"2024-08-18T23:41:42.437965+0000","last_clean":"2024-08-18T23:41:42.437965+0000","last_became_active":"2024-08-18T23:41:32.146828+0000","last_became_peered":"2024-08-18T23:41:32.146828+0000","last_unstale":"2024-08-18T23:41:42.437965+0000","last_undegraded":"2024-08-18T23:41:42.437965+0000","last_fullsized":"2024-08-18T23:41:42.437965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:51:49.175150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436255+0000","last_change":"2024-08-18T23:41:32.144796+0000","last_active":"2024-08-18T23:41:42.436255+0000","last_peered":"2024-08-18T23:41:42.436255+0000","last_clean":"2024-08-18T23:41:42.436255+0000","last_became_active":"2024-08-18T23:41:32.141931+0000","last_became_peered":"2024-08-18T23:41:32.141931+0000","last_unstale":"2024-08-18T23:41:42.436255+0000","last_undegraded":"2024-08-18T23:41:42.436255+0000","last_fullsized":"2024-08-18T23:41:42.436255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:48:08.428845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438077+0000","last_change":"2024-08-18T23:41:32.135265+0000","last_active":"2024-08-18T23:41:42.438077+0000","last_peered":"2024-08-18T23:41:42.438077+0000","last_clean":"2024-08-18T23:41:42.438077+0000","last_became_active":"2024-08-18T23:41:32.134952+0000","last_became_peered":"2024-08-18T23:41:32.134952+0000","last_unstale":"2024-08-18T23:41:42.438077+0000","last_undegraded":"2024-08-18T23:41:42.438077+0000","last_fullsized":"2024-08-18T23:41:42.438077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:35:14.099536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.181837+0000","last_change":"2024-08-18T23:41:32.135941+0000","last_active":"2024-08-18T23:41:42.181837+0000","last_peered":"2024-08-18T23:41:42.181837+0000","last_clean":"2024-08-18T23:41:42.181837+0000","last_became_active":"2024-08-18T23:41:32.135623+0000","last_became_peered":"2024-08-18T23:41:32.135623+0000","last_unstale":"2024-08-18T23:41:42.181837+0000","last_undegraded":"2024-08-18T23:41:42.181837+0000","last_fullsized":"2024-08-18T23:41:42.181837+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:43:36.504738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324078+0000","last_change":"2024-08-18T23:41:32.141643+0000","last_active":"2024-08-18T23:42:23.324078+0000","last_peered":"2024-08-18T23:42:23.324078+0000","last_clean":"2024-08-18T23:42:23.324078+0000","last_became_active":"2024-08-18T23:41:32.141301+0000","last_became_peered":"2024-08-18T23:41:32.141301+0000","last_unstale":"2024-08-18T23:42:23.324078+0000","last_undegraded":"2024-08-18T23:42:23.324078+0000","last_fullsized":"2024-08-18T23:42:23.324078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:23:53.100740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437849+0000","last_change":"2024-08-18T23:41:32.146730+0000","last_active":"2024-08-18T23:41:42.437849+0000","last_peered":"2024-08-18T23:41:42.437849+0000","last_clean":"2024-08-18T23:41:42.437849+0000","last_became_active":"2024-08-18T23:41:32.146252+0000","last_became_peered":"2024-08-18T23:41:32.146252+0000","last_unstale":"2024-08-18T23:41:42.437849+0000","last_undegraded":"2024-08-18T23:41:42.437849+0000","last_fullsized":"2024-08-18T23:41:42.437849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:41.481761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436851+0000","last_change":"2024-08-18T23:41:32.149569+0000","last_active":"2024-08-18T23:41:42.436851+0000","last_peered":"2024-08-18T23:41:42.436851+0000","last_clean":"2024-08-18T23:41:42.436851+0000","last_became_active":"2024-08-18T23:41:32.149277+0000","last_became_peered":"2024-08-18T23:41:32.149277+0000","last_unstale":"2024-08-18T23:41:42.436851+0000","last_undegraded":"2024-08-18T23:41:42.436851+0000","last_fullsized":"2024-08-18T23:41:42.436851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:34:00.547909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436812+0000","last_change":"2024-08-18T23:41:32.137889+0000","last_active":"2024-08-18T23:41:42.436812+0000","last_peered":"2024-08-18T23:41:42.436812+0000","last_clean":"2024-08-18T23:41:42.436812+0000","last_became_active":"2024-08-18T23:41:32.137578+0000","last_became_peered":"2024-08-18T23:41:32.137578+0000","last_unstale":"2024-08-18T23:41:42.436812+0000","last_undegraded":"2024-08-18T23:41:42.436812+0000","last_fullsized":"2024-08-18T23:41:42.436812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:18:09.987838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.310210+0000","last_change":"2024-08-18T23:41:32.142084+0000","last_active":"2024-08-18T23:42:22.310210+0000","last_peered":"2024-08-18T23:42:22.310210+0000","last_clean":"2024-08-18T23:42:22.310210+0000","last_became_active":"2024-08-18T23:41:32.141670+0000","last_became_peered":"2024-08-18T23:41:32.141670+0000","last_unstale":"2024-08-18T23:42:22.310210+0000","last_undegraded":"2024-08-18T23:42:22.310210+0000","last_fullsized":"2024-08-18T23:42:22.310210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:03.990351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":36864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53482,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":99,"num_read_kb":106,"num_write":157,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":160446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514519,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":486092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776155,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1662976,"data_stored":1122842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939680,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":490424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168613,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":495830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397544,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36896,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708256,"statfs":{"total":95995035648,"available":95957254144,"internally_reserved":0,"allocated":1597440,"data_stored":1082546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561069,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":481504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790002,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1605632,"data_stored":1077744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953526,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":486544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":481698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":532246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":530506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":482892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":49092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":50832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":51060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T23:42:26.372 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-18T23:42:26.622 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:27.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.093+0000 7f2a31651640 1 -- 172.21.15.78:0/1849742002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a2c102250 msgr2=0x7f2a2c10e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:27.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.093+0000 7f2a31651640 1 --2- 172.21.15.78:0/1849742002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a2c102250 0x7f2a2c10e780 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009cd0 tx=0x7f2a2402a8e0 comp rx=0 tx=0).stop 2024-08-18T23:42:27.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.093+0000 7f2a31651640 1 -- 172.21.15.78:0/1849742002 shutdown_connections 2024-08-18T23:42:27.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.093+0000 7f2a31651640 1 --2- 172.21.15.78:0/1849742002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a2c102250 0x7f2a2c10e780 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:27.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.093+0000 7f2a31651640 1 --2- 172.21.15.78:0/1849742002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c1018b0 0x7f2a2c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:27.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.093+0000 7f2a31651640 1 --2- 172.21.15.78:0/1849742002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c1078b0 0x7f2a2c107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:27.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.093+0000 7f2a31651640 1 -- 172.21.15.78:0/1849742002 >> 172.21.15.78:0/1849742002 conn(0x7f2a2c0fd5f0 msgr2=0x7f2a2c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:27.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.093+0000 7f2a31651640 1 -- 172.21.15.78:0/1849742002 shutdown_connections 2024-08-18T23:42:27.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.093+0000 7f2a31651640 1 -- 172.21.15.78:0/1849742002 wait complete. 2024-08-18T23:42:27.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.094+0000 7f2a31651640 1 Processor -- start 2024-08-18T23:42:27.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.094+0000 7f2a31651640 1 -- start start 2024-08-18T23:42:27.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.094+0000 7f2a31651640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c1018b0 0x7f2a2c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:27.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.094+0000 7f2a31651640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c102250 0x7f2a2c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:27.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.094+0000 7f2a31651640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a2c1078b0 0x7f2a2c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:27.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.094+0000 7f2a31651640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c111cd0 con 0x7f2a2c1078b0 2024-08-18T23:42:27.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.094+0000 7f2a31651640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c111b50 con 0x7f2a2c102250 2024-08-18T23:42:27.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.094+0000 7f2a31651640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c111e50 con 0x7f2a2c1018b0 2024-08-18T23:42:27.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.094+0000 7f2a2b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a2c1078b0 0x7f2a2c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:27.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.094+0000 7f2a2b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a2c1078b0 0x7f2a2c19c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32924/0 (socket says 172.21.15.78:32924) 2024-08-18T23:42:27.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.094+0000 7f2a2b7fe640 1 -- 172.21.15.78:0/1978435030 learned_addr learned my addr 172.21.15.78:0/1978435030 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:27.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.095+0000 7f2a2b7fe640 1 -- 172.21.15.78:0/1978435030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c1018b0 msgr2=0x7f2a2c198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:27.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.095+0000 7f2a2affd640 1 --2- 172.21.15.78:0/1978435030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c1018b0 0x7f2a2c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:27.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.095+0000 7f2a23fff640 1 --2- 172.21.15.78:0/1978435030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c102250 0x7f2a2c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:27.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.095+0000 7f2a2b7fe640 1 --2- 172.21.15.78:0/1978435030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c1018b0 0x7f2a2c198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:27.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.095+0000 7f2a2b7fe640 1 -- 172.21.15.78:0/1978435030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c102250 msgr2=0x7f2a2c198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:27.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.095+0000 7f2a2b7fe640 1 --2- 172.21.15.78:0/1978435030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c102250 0x7f2a2c198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:27.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.095+0000 7f2a2b7fe640 1 -- 172.21.15.78:0/1978435030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2c19d0c0 con 0x7f2a2c1078b0 2024-08-18T23:42:27.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.095+0000 7f2a2b7fe640 1 --2- 172.21.15.78:0/1978435030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a2c1078b0 0x7f2a2c19c9e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2a24006010 tx=0x7f2a24003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:27.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.095+0000 7f2a28ff9640 1 -- 172.21.15.78:0/1978435030 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24004450 con 0x7f2a2c1078b0 2024-08-18T23:42:27.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.095+0000 7f2a31651640 1 -- 172.21.15.78:0/1978435030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a2c19d350 con 0x7f2a2c1078b0 2024-08-18T23:42:27.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.095+0000 7f2a28ff9640 1 -- 172.21.15.78:0/1978435030 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a24003e10 con 0x7f2a2c1078b0 2024-08-18T23:42:27.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.095+0000 7f2a28ff9640 1 -- 172.21.15.78:0/1978435030 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a240466e0 con 0x7f2a2c1078b0 2024-08-18T23:42:27.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.095+0000 7f2a31651640 1 -- 172.21.15.78:0/1978435030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a2c1a4c90 con 0x7f2a2c1078b0 2024-08-18T23:42:27.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.096+0000 7f2a31651640 1 -- 172.21.15.78:0/1978435030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a2c101d10 con 0x7f2a2c1078b0 2024-08-18T23:42:27.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.097+0000 7f2a28ff9640 1 -- 172.21.15.78:0/1978435030 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2a240305a0 con 0x7f2a2c1078b0 2024-08-18T23:42:27.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.098+0000 7f2a28ff9640 1 --2- 172.21.15.78:0/1978435030 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2a180a2c10 0x7f2a180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:27.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.098+0000 7f2a28ff9640 1 -- 172.21.15.78:0/1978435030 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f2a2402e030 con 0x7f2a2c1078b0 2024-08-18T23:42:27.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.098+0000 7f2a2affd640 1 --2- 172.21.15.78:0/1978435030 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2a180a2c10 0x7f2a180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:27.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.098+0000 7f2a2affd640 1 --2- 172.21.15.78:0/1978435030 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2a180a2c10 0x7f2a180a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00ba40 tx=0x7f2a1c005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:27.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.102+0000 7f2a28ff9640 1 -- 172.21.15.78:0/1978435030 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2a2404b060 con 0x7f2a2c1078b0 2024-08-18T23:42:27.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.271+0000 7f2a31651640 1 -- 172.21.15.78:0/1978435030 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2a2c0fe3f0 con 0x7f2a180a2c10 2024-08-18T23:42:27.273 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:27 smithi078 ceph-mon[18440]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:42:27.273 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:27 smithi078 ceph-mon[18440]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.9 KiB/s wr, 9 op/s; 72 B/s, 0 objects/s recovering 2024-08-18T23:42:27.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.279+0000 7f2a28ff9640 1 -- 172.21.15.78:0/1978435030 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340319 (secure 0 0 0) 0x7f2a2c0fe3f0 con 0x7f2a180a2c10 2024-08-18T23:42:27.286 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:42:27.288 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-08-18T23:42:27.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.282+0000 7f2a31651640 1 -- 172.21.15.78:0/1978435030 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2a180a2c10 msgr2=0x7f2a180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:27.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.282+0000 7f2a31651640 1 --2- 172.21.15.78:0/1978435030 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2a180a2c10 0x7f2a180a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00ba40 tx=0x7f2a1c005cc0 comp rx=0 tx=0).stop 2024-08-18T23:42:27.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.282+0000 7f2a31651640 1 -- 172.21.15.78:0/1978435030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a2c1078b0 msgr2=0x7f2a2c19c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:27.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.282+0000 7f2a31651640 1 --2- 172.21.15.78:0/1978435030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a2c1078b0 0x7f2a2c19c9e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2a24006010 tx=0x7f2a24003c20 comp rx=0 tx=0).stop 2024-08-18T23:42:27.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.282+0000 7f2a31651640 1 -- 172.21.15.78:0/1978435030 shutdown_connections 2024-08-18T23:42:27.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.282+0000 7f2a31651640 1 --2- 172.21.15.78:0/1978435030 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2a180a2c10 0x7f2a180a50d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:27.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.282+0000 7f2a31651640 1 --2- 172.21.15.78:0/1978435030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a2c1078b0 0x7f2a2c19c9e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:27.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.282+0000 7f2a31651640 1 --2- 172.21.15.78:0/1978435030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a2c102250 0x7f2a2c198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:27.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.282+0000 7f2a31651640 1 --2- 172.21.15.78:0/1978435030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a2c1018b0 0x7f2a2c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:27.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.282+0000 7f2a31651640 1 -- 172.21.15.78:0/1978435030 >> 172.21.15.78:0/1978435030 conn(0x7f2a2c0fd5f0 msgr2=0x7f2a2c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:27.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.282+0000 7f2a31651640 1 -- 172.21.15.78:0/1978435030 shutdown_connections 2024-08-18T23:42:27.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:27.282+0000 7f2a31651640 1 -- 172.21.15.78:0/1978435030 wait complete. 2024-08-18T23:42:27.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:27 smithi161 ceph-mon[22027]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:42:27.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:27 smithi161 ceph-mon[22027]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.9 KiB/s wr, 9 op/s; 72 B/s, 0 objects/s recovering 2024-08-18T23:42:27.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:27 smithi123 ceph-mon[21808]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:42:27.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:27 smithi123 ceph-mon[21808]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.9 KiB/s wr, 9 op/s; 72 B/s, 0 objects/s recovering 2024-08-18T23:42:28.017 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-08-18T23:42:26.121290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651735,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":250,"num_read_kb":237,"num_write":343,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387968,"kb_used_data":13812,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553856,"statfs":{"total":1151940427776,"available":1151543148544,"internally_reserved":0,"allocated":14143488,"data_stored":7861362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32286,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":57,"num_read_kb":62,"num_write":18,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002038"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:07.263169+0000","last_change":"2024-08-18T23:41:30.120580+0000","last_active":"2024-08-18T23:42:07.263169+0000","last_peered":"2024-08-18T23:42:07.263169+0000","last_clean":"2024-08-18T23:42:07.263169+0000","last_became_active":"2024-08-18T23:41:30.120266+0000","last_became_peered":"2024-08-18T23:41:30.120266+0000","last_unstale":"2024-08-18T23:42:07.263169+0000","last_undegraded":"2024-08-18T23:42:07.263169+0000","last_fullsized":"2024-08-18T23:42:07.263169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:06:40.997070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436535+0000","last_change":"2024-08-18T23:41:32.143232+0000","last_active":"2024-08-18T23:41:42.436535+0000","last_peered":"2024-08-18T23:41:42.436535+0000","last_clean":"2024-08-18T23:41:42.436535+0000","last_became_active":"2024-08-18T23:41:32.142916+0000","last_became_peered":"2024-08-18T23:41:32.142916+0000","last_unstale":"2024-08-18T23:41:42.436535+0000","last_undegraded":"2024-08-18T23:41:42.436535+0000","last_fullsized":"2024-08-18T23:41:42.436535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:50:54.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438428+0000","last_change":"2024-08-18T23:41:39.188772+0000","last_active":"2024-08-18T23:41:42.438428+0000","last_peered":"2024-08-18T23:41:42.438428+0000","last_clean":"2024-08-18T23:41:42.438428+0000","last_became_active":"2024-08-18T23:41:39.188362+0000","last_became_peered":"2024-08-18T23:41:39.188362+0000","last_unstale":"2024-08-18T23:41:42.438428+0000","last_undegraded":"2024-08-18T23:41:42.438428+0000","last_fullsized":"2024-08-18T23:41:42.438428+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:47:50.255543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436883+0000","last_change":"2024-08-18T23:41:30.137179+0000","last_active":"2024-08-18T23:41:42.436883+0000","last_peered":"2024-08-18T23:41:42.436883+0000","last_clean":"2024-08-18T23:41:42.436883+0000","last_became_active":"2024-08-18T23:41:30.136618+0000","last_became_peered":"2024-08-18T23:41:30.136618+0000","last_unstale":"2024-08-18T23:41:42.436883+0000","last_undegraded":"2024-08-18T23:41:42.436883+0000","last_fullsized":"2024-08-18T23:41:42.436883+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:59.826364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316847+0000","last_change":"2024-08-18T23:41:32.142075+0000","last_active":"2024-08-18T23:42:23.316847+0000","last_peered":"2024-08-18T23:42:23.316847+0000","last_clean":"2024-08-18T23:42:23.316847+0000","last_became_active":"2024-08-18T23:41:32.141709+0000","last_became_peered":"2024-08-18T23:41:32.141709+0000","last_unstale":"2024-08-18T23:42:23.316847+0000","last_undegraded":"2024-08-18T23:42:23.316847+0000","last_fullsized":"2024-08-18T23:42:23.316847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:08.324627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317948+0000","last_change":"2024-08-18T23:41:39.184575+0000","last_active":"2024-08-18T23:42:23.317948+0000","last_peered":"2024-08-18T23:42:23.317948+0000","last_clean":"2024-08-18T23:42:23.317948+0000","last_became_active":"2024-08-18T23:41:39.184169+0000","last_became_peered":"2024-08-18T23:41:39.184169+0000","last_unstale":"2024-08-18T23:42:23.317948+0000","last_undegraded":"2024-08-18T23:42:23.317948+0000","last_fullsized":"2024-08-18T23:42:23.317948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:32.218172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324756+0000","last_change":"2024-08-18T23:41:30.121638+0000","last_active":"2024-08-18T23:42:23.324756+0000","last_peered":"2024-08-18T23:42:23.324756+0000","last_clean":"2024-08-18T23:42:23.324756+0000","last_became_active":"2024-08-18T23:41:30.121290+0000","last_became_peered":"2024-08-18T23:41:30.121290+0000","last_unstale":"2024-08-18T23:42:23.324756+0000","last_undegraded":"2024-08-18T23:42:23.324756+0000","last_fullsized":"2024-08-18T23:42:23.324756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:43:55.638780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315534+0000","last_change":"2024-08-18T23:41:32.142177+0000","last_active":"2024-08-18T23:42:23.315534+0000","last_peered":"2024-08-18T23:42:23.315534+0000","last_clean":"2024-08-18T23:42:23.315534+0000","last_became_active":"2024-08-18T23:41:32.141731+0000","last_became_peered":"2024-08-18T23:41:32.141731+0000","last_unstale":"2024-08-18T23:42:23.315534+0000","last_undegraded":"2024-08-18T23:42:23.315534+0000","last_fullsized":"2024-08-18T23:42:23.315534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:36.283299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318344+0000","last_change":"2024-08-18T23:41:39.182931+0000","last_active":"2024-08-18T23:42:23.318344+0000","last_peered":"2024-08-18T23:42:23.318344+0000","last_clean":"2024-08-18T23:42:23.318344+0000","last_became_active":"2024-08-18T23:41:39.182008+0000","last_became_peered":"2024-08-18T23:41:39.182008+0000","last_unstale":"2024-08-18T23:42:23.318344+0000","last_undegraded":"2024-08-18T23:42:23.318344+0000","last_fullsized":"2024-08-18T23:42:23.318344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:58:38.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":2,"reported_epoch":89,"state":"peering","last_fresh":"2024-08-18T23:41:33.156065+0000","last_change":"2024-08-18T23:42:22.476106+0000","last_active":"2024-08-18T23:42:22.476106+0000","last_peered":"2024-08-18T23:41:33.156065+0000","last_clean":"2024-08-18T23:41:33.156065+0000","last_became_active":"2024-08-18T23:41:30.119986+0000","last_became_peered":"2024-08-18T23:41:30.119986+0000","last_unstale":"2024-08-18T23:41:33.156065+0000","last_undegraded":"2024-08-18T23:41:33.156065+0000","last_fullsized":"2024-08-18T23:41:33.156065+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438033+0000","last_change":"2024-08-18T23:41:32.148662+0000","last_active":"2024-08-18T23:41:42.438033+0000","last_peered":"2024-08-18T23:41:42.438033+0000","last_clean":"2024-08-18T23:41:42.438033+0000","last_became_active":"2024-08-18T23:41:32.148318+0000","last_became_peered":"2024-08-18T23:41:32.148318+0000","last_unstale":"2024-08-18T23:41:42.438033+0000","last_undegraded":"2024-08-18T23:41:42.438033+0000","last_fullsized":"2024-08-18T23:41:42.438033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:58.668669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318492+0000","last_change":"2024-08-18T23:41:39.181114+0000","last_active":"2024-08-18T23:42:23.318492+0000","last_peered":"2024-08-18T23:42:23.318492+0000","last_clean":"2024-08-18T23:42:23.318492+0000","last_became_active":"2024-08-18T23:41:39.180599+0000","last_became_peered":"2024-08-18T23:41:39.180599+0000","last_unstale":"2024-08-18T23:42:23.318492+0000","last_undegraded":"2024-08-18T23:42:23.318492+0000","last_fullsized":"2024-08-18T23:42:23.318492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:36:08.749143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437425+0000","last_change":"2024-08-18T23:41:32.144326+0000","last_active":"2024-08-18T23:41:42.437425+0000","last_peered":"2024-08-18T23:41:42.437425+0000","last_clean":"2024-08-18T23:41:42.437425+0000","last_became_active":"2024-08-18T23:41:32.144004+0000","last_became_peered":"2024-08-18T23:41:32.144004+0000","last_unstale":"2024-08-18T23:41:42.437425+0000","last_undegraded":"2024-08-18T23:41:42.437425+0000","last_fullsized":"2024-08-18T23:41:42.437425+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:10.918624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324188+0000","last_change":"2024-08-18T23:41:30.125160+0000","last_active":"2024-08-18T23:42:23.324188+0000","last_peered":"2024-08-18T23:42:23.324188+0000","last_clean":"2024-08-18T23:42:23.324188+0000","last_became_active":"2024-08-18T23:41:30.124685+0000","last_became_peered":"2024-08-18T23:41:30.124685+0000","last_unstale":"2024-08-18T23:42:23.324188+0000","last_undegraded":"2024-08-18T23:42:23.324188+0000","last_fullsized":"2024-08-18T23:42:23.324188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:52:48.997902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318342+0000","last_change":"2024-08-18T23:41:39.681481+0000","last_active":"2024-08-18T23:42:23.318342+0000","last_peered":"2024-08-18T23:42:23.318342+0000","last_clean":"2024-08-18T23:42:23.318342+0000","last_became_active":"2024-08-18T23:41:39.680204+0000","last_became_peered":"2024-08-18T23:41:39.680204+0000","last_unstale":"2024-08-18T23:42:23.318342+0000","last_undegraded":"2024-08-18T23:42:23.318342+0000","last_fullsized":"2024-08-18T23:42:23.318342+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:00:05.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436405+0000","last_change":"2024-08-18T23:41:32.151335+0000","last_active":"2024-08-18T23:41:42.436405+0000","last_peered":"2024-08-18T23:41:42.436405+0000","last_clean":"2024-08-18T23:41:42.436405+0000","last_became_active":"2024-08-18T23:41:32.151040+0000","last_became_peered":"2024-08-18T23:41:32.151040+0000","last_unstale":"2024-08-18T23:41:42.436405+0000","last_undegraded":"2024-08-18T23:41:42.436405+0000","last_fullsized":"2024-08-18T23:41:42.436405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:16.108555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436860+0000","last_change":"2024-08-18T23:41:30.122301+0000","last_active":"2024-08-18T23:41:42.436860+0000","last_peered":"2024-08-18T23:41:42.436860+0000","last_clean":"2024-08-18T23:41:42.436860+0000","last_became_active":"2024-08-18T23:41:30.121117+0000","last_became_peered":"2024-08-18T23:41:30.121117+0000","last_unstale":"2024-08-18T23:41:42.436860+0000","last_undegraded":"2024-08-18T23:41:42.436860+0000","last_fullsized":"2024-08-18T23:41:42.436860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:40:12.678803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318961+0000","last_change":"2024-08-18T23:41:39.182395+0000","last_active":"2024-08-18T23:42:23.318961+0000","last_peered":"2024-08-18T23:42:23.318961+0000","last_clean":"2024-08-18T23:42:23.318961+0000","last_became_active":"2024-08-18T23:41:39.181979+0000","last_became_peered":"2024-08-18T23:41:39.181979+0000","last_unstale":"2024-08-18T23:42:23.318961+0000","last_undegraded":"2024-08-18T23:42:23.318961+0000","last_fullsized":"2024-08-18T23:42:23.318961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:04.674503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.310565+0000","last_change":"2024-08-18T23:41:32.140464+0000","last_active":"2024-08-18T23:42:22.310565+0000","last_peered":"2024-08-18T23:42:22.310565+0000","last_clean":"2024-08-18T23:42:22.310565+0000","last_became_active":"2024-08-18T23:41:32.139153+0000","last_became_peered":"2024-08-18T23:41:32.139153+0000","last_unstale":"2024-08-18T23:42:22.310565+0000","last_undegraded":"2024-08-18T23:42:22.310565+0000","last_fullsized":"2024-08-18T23:42:22.310565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:27.387680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:06.256815+0000","last_change":"2024-08-18T23:41:30.122787+0000","last_active":"2024-08-18T23:42:06.256815+0000","last_peered":"2024-08-18T23:42:06.256815+0000","last_clean":"2024-08-18T23:42:06.256815+0000","last_became_active":"2024-08-18T23:41:30.122462+0000","last_became_peered":"2024-08-18T23:41:30.122462+0000","last_unstale":"2024-08-18T23:42:06.256815+0000","last_undegraded":"2024-08-18T23:42:06.256815+0000","last_fullsized":"2024-08-18T23:42:06.256815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:55.568831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.184346+0000","last_change":"2024-08-18T23:41:39.190511+0000","last_active":"2024-08-18T23:41:42.184346+0000","last_peered":"2024-08-18T23:41:42.184346+0000","last_clean":"2024-08-18T23:41:42.184346+0000","last_became_active":"2024-08-18T23:41:39.190154+0000","last_became_peered":"2024-08-18T23:41:39.190154+0000","last_unstale":"2024-08-18T23:41:42.184346+0000","last_undegraded":"2024-08-18T23:41:42.184346+0000","last_fullsized":"2024-08-18T23:41:42.184346+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:33:34.514827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323413+0000","last_change":"2024-08-18T23:41:32.143867+0000","last_active":"2024-08-18T23:42:23.323413+0000","last_peered":"2024-08-18T23:42:23.323413+0000","last_clean":"2024-08-18T23:42:23.323413+0000","last_became_active":"2024-08-18T23:41:32.143527+0000","last_became_peered":"2024-08-18T23:41:32.143527+0000","last_unstale":"2024-08-18T23:42:23.323413+0000","last_undegraded":"2024-08-18T23:42:23.323413+0000","last_fullsized":"2024-08-18T23:42:23.323413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:40:25.902133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.325102+0000","last_change":"2024-08-18T23:42:23.325102+0000","last_active":"2024-08-18T23:42:23.325102+0000","last_peered":"2024-08-18T23:42:23.325102+0000","last_clean":"2024-08-18T23:42:23.325102+0000","last_became_active":"2024-08-18T23:42:23.321583+0000","last_became_peered":"2024-08-18T23:42:23.321583+0000","last_unstale":"2024-08-18T23:42:23.325102+0000","last_undegraded":"2024-08-18T23:42:23.325102+0000","last_fullsized":"2024-08-18T23:42:23.325102+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:57:42.546686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.559756+0000","last_change":"2024-08-18T23:42:23.559756+0000","last_active":"2024-08-18T23:42:23.559756+0000","last_peered":"2024-08-18T23:42:23.559756+0000","last_clean":"2024-08-18T23:42:23.559756+0000","last_became_active":"2024-08-18T23:42:23.559242+0000","last_became_peered":"2024-08-18T23:42:23.559242+0000","last_unstale":"2024-08-18T23:42:23.559756+0000","last_undegraded":"2024-08-18T23:42:23.559756+0000","last_fullsized":"2024-08-18T23:42:23.559756+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:25:40.790313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316766+0000","last_change":"2024-08-18T23:41:32.139025+0000","last_active":"2024-08-18T23:42:23.316766+0000","last_peered":"2024-08-18T23:42:23.316766+0000","last_clean":"2024-08-18T23:42:23.316766+0000","last_became_active":"2024-08-18T23:41:32.138733+0000","last_became_peered":"2024-08-18T23:41:32.138733+0000","last_unstale":"2024-08-18T23:42:23.316766+0000","last_undegraded":"2024-08-18T23:42:23.316766+0000","last_fullsized":"2024-08-18T23:42:23.316766+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:01:39.826391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319270+0000","last_change":"2024-08-18T23:41:30.477359+0000","last_active":"2024-08-18T23:42:23.319270+0000","last_peered":"2024-08-18T23:42:23.319270+0000","last_clean":"2024-08-18T23:42:23.319270+0000","last_became_active":"2024-08-18T23:41:30.477011+0000","last_became_peered":"2024-08-18T23:41:30.477011+0000","last_unstale":"2024-08-18T23:42:23.319270+0000","last_undegraded":"2024-08-18T23:42:23.319270+0000","last_fullsized":"2024-08-18T23:42:23.319270+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:53:10.052859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438585+0000","last_change":"2024-08-18T23:41:39.182140+0000","last_active":"2024-08-18T23:41:42.438585+0000","last_peered":"2024-08-18T23:41:42.438585+0000","last_clean":"2024-08-18T23:41:42.438585+0000","last_became_active":"2024-08-18T23:41:39.181667+0000","last_became_peered":"2024-08-18T23:41:39.181667+0000","last_unstale":"2024-08-18T23:41:42.438585+0000","last_undegraded":"2024-08-18T23:41:42.438585+0000","last_fullsized":"2024-08-18T23:41:42.438585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:02:18.128028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436969+0000","last_change":"2024-08-18T23:41:32.137064+0000","last_active":"2024-08-18T23:41:42.436969+0000","last_peered":"2024-08-18T23:41:42.436969+0000","last_clean":"2024-08-18T23:41:42.436969+0000","last_became_active":"2024-08-18T23:41:32.136738+0000","last_became_peered":"2024-08-18T23:41:32.136738+0000","last_unstale":"2024-08-18T23:41:42.436969+0000","last_undegraded":"2024-08-18T23:41:42.436969+0000","last_fullsized":"2024-08-18T23:41:42.436969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:15:11.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'9","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:19.153483+0000","last_change":"2024-08-18T23:41:30.120042+0000","last_active":"2024-08-18T23:42:19.153483+0000","last_peered":"2024-08-18T23:42:19.153483+0000","last_clean":"2024-08-18T23:42:19.153483+0000","last_became_active":"2024-08-18T23:41:30.119719+0000","last_became_peered":"2024-08-18T23:41:30.119719+0000","last_unstale":"2024-08-18T23:42:19.153483+0000","last_undegraded":"2024-08-18T23:42:19.153483+0000","last_fullsized":"2024-08-18T23:42:19.153483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:35:44.466330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436182+0000","last_change":"2024-08-18T23:41:39.680405+0000","last_active":"2024-08-18T23:41:42.436182+0000","last_peered":"2024-08-18T23:41:42.436182+0000","last_clean":"2024-08-18T23:41:42.436182+0000","last_became_active":"2024-08-18T23:41:39.679940+0000","last_became_peered":"2024-08-18T23:41:39.679940+0000","last_unstale":"2024-08-18T23:41:42.436182+0000","last_undegraded":"2024-08-18T23:41:42.436182+0000","last_fullsized":"2024-08-18T23:41:42.436182+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:15:03.942116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314555+0000","last_change":"2024-08-18T23:41:32.139688+0000","last_active":"2024-08-18T23:42:23.314555+0000","last_peered":"2024-08-18T23:42:23.314555+0000","last_clean":"2024-08-18T23:42:23.314555+0000","last_became_active":"2024-08-18T23:41:32.139397+0000","last_became_peered":"2024-08-18T23:41:32.139397+0000","last_unstale":"2024-08-18T23:42:23.314555+0000","last_undegraded":"2024-08-18T23:42:23.314555+0000","last_fullsized":"2024-08-18T23:42:23.314555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:03:57.017488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324876+0000","last_change":"2024-08-18T23:41:30.137578+0000","last_active":"2024-08-18T23:42:23.324876+0000","last_peered":"2024-08-18T23:42:23.324876+0000","last_clean":"2024-08-18T23:42:23.324876+0000","last_became_active":"2024-08-18T23:41:30.137246+0000","last_became_peered":"2024-08-18T23:41:30.137246+0000","last_unstale":"2024-08-18T23:42:23.324876+0000","last_undegraded":"2024-08-18T23:42:23.324876+0000","last_fullsized":"2024-08-18T23:42:23.324876+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:48:54.839432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.309910+0000","last_change":"2024-08-18T23:41:39.186360+0000","last_active":"2024-08-18T23:42:22.309910+0000","last_peered":"2024-08-18T23:42:22.309910+0000","last_clean":"2024-08-18T23:42:22.309910+0000","last_became_active":"2024-08-18T23:41:39.185808+0000","last_became_peered":"2024-08-18T23:41:39.185808+0000","last_unstale":"2024-08-18T23:42:22.309910+0000","last_undegraded":"2024-08-18T23:42:22.309910+0000","last_fullsized":"2024-08-18T23:42:22.309910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:53.891061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558281+0000","last_change":"2024-08-18T23:41:32.146070+0000","last_active":"2024-08-18T23:42:23.558281+0000","last_peered":"2024-08-18T23:42:23.558281+0000","last_clean":"2024-08-18T23:42:23.558281+0000","last_became_active":"2024-08-18T23:41:32.145640+0000","last_became_peered":"2024-08-18T23:41:32.145640+0000","last_unstale":"2024-08-18T23:42:23.558281+0000","last_undegraded":"2024-08-18T23:42:23.558281+0000","last_fullsized":"2024-08-18T23:42:23.558281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:56:21.368762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.184447+0000","last_change":"2024-08-18T23:41:30.138565+0000","last_active":"2024-08-18T23:41:42.184447+0000","last_peered":"2024-08-18T23:41:42.184447+0000","last_clean":"2024-08-18T23:41:42.184447+0000","last_became_active":"2024-08-18T23:41:30.138210+0000","last_became_peered":"2024-08-18T23:41:30.138210+0000","last_unstale":"2024-08-18T23:41:42.184447+0000","last_undegraded":"2024-08-18T23:41:42.184447+0000","last_fullsized":"2024-08-18T23:41:42.184447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:42:14.928884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323149+0000","last_change":"2024-08-18T23:41:39.680656+0000","last_active":"2024-08-18T23:42:23.323149+0000","last_peered":"2024-08-18T23:42:23.323149+0000","last_clean":"2024-08-18T23:42:23.323149+0000","last_became_active":"2024-08-18T23:41:39.680203+0000","last_became_peered":"2024-08-18T23:41:39.680203+0000","last_unstale":"2024-08-18T23:42:23.323149+0000","last_undegraded":"2024-08-18T23:42:23.323149+0000","last_fullsized":"2024-08-18T23:42:23.323149+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:55:43.138869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558246+0000","last_change":"2024-08-18T23:41:32.130222+0000","last_active":"2024-08-18T23:42:23.558246+0000","last_peered":"2024-08-18T23:42:23.558246+0000","last_clean":"2024-08-18T23:42:23.558246+0000","last_became_active":"2024-08-18T23:41:32.129844+0000","last_became_peered":"2024-08-18T23:41:32.129844+0000","last_unstale":"2024-08-18T23:42:23.558246+0000","last_undegraded":"2024-08-18T23:42:23.558246+0000","last_fullsized":"2024-08-18T23:42:23.558246+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:39:21.106343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437643+0000","last_change":"2024-08-18T23:41:30.123142+0000","last_active":"2024-08-18T23:41:42.437643+0000","last_peered":"2024-08-18T23:41:42.437643+0000","last_clean":"2024-08-18T23:41:42.437643+0000","last_became_active":"2024-08-18T23:41:30.121645+0000","last_became_peered":"2024-08-18T23:41:30.121645+0000","last_unstale":"2024-08-18T23:41:42.437643+0000","last_undegraded":"2024-08-18T23:41:42.437643+0000","last_fullsized":"2024-08-18T23:41:42.437643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:07:24.912259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314454+0000","last_change":"2024-08-18T23:41:39.186033+0000","last_active":"2024-08-18T23:42:23.314454+0000","last_peered":"2024-08-18T23:42:23.314454+0000","last_clean":"2024-08-18T23:42:23.314454+0000","last_became_active":"2024-08-18T23:41:39.185451+0000","last_became_peered":"2024-08-18T23:41:39.185451+0000","last_unstale":"2024-08-18T23:42:23.314454+0000","last_undegraded":"2024-08-18T23:42:23.314454+0000","last_fullsized":"2024-08-18T23:42:23.314454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:52:13.556760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437517+0000","last_change":"2024-08-18T23:41:32.141065+0000","last_active":"2024-08-18T23:41:42.437517+0000","last_peered":"2024-08-18T23:41:42.437517+0000","last_clean":"2024-08-18T23:41:42.437517+0000","last_became_active":"2024-08-18T23:41:32.140732+0000","last_became_peered":"2024-08-18T23:41:32.140732+0000","last_unstale":"2024-08-18T23:41:42.437517+0000","last_undegraded":"2024-08-18T23:41:42.437517+0000","last_fullsized":"2024-08-18T23:41:42.437517+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:08:32.287835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314172+0000","last_change":"2024-08-18T23:41:30.136838+0000","last_active":"2024-08-18T23:42:23.314172+0000","last_peered":"2024-08-18T23:42:23.314172+0000","last_clean":"2024-08-18T23:42:23.314172+0000","last_became_active":"2024-08-18T23:41:30.136523+0000","last_became_peered":"2024-08-18T23:41:30.136523+0000","last_unstale":"2024-08-18T23:42:23.314172+0000","last_undegraded":"2024-08-18T23:42:23.314172+0000","last_fullsized":"2024-08-18T23:42:23.314172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:56:17.750535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314559+0000","last_change":"2024-08-18T23:41:39.680314+0000","last_active":"2024-08-18T23:42:23.314559+0000","last_peered":"2024-08-18T23:42:23.314559+0000","last_clean":"2024-08-18T23:42:23.314559+0000","last_became_active":"2024-08-18T23:41:39.679920+0000","last_became_peered":"2024-08-18T23:41:39.679920+0000","last_unstale":"2024-08-18T23:42:23.314559+0000","last_undegraded":"2024-08-18T23:42:23.314559+0000","last_fullsized":"2024-08-18T23:42:23.314559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:26:29.866081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314066+0000","last_change":"2024-08-18T23:41:32.139752+0000","last_active":"2024-08-18T23:42:23.314066+0000","last_peered":"2024-08-18T23:42:23.314066+0000","last_clean":"2024-08-18T23:42:23.314066+0000","last_became_active":"2024-08-18T23:41:32.139451+0000","last_became_peered":"2024-08-18T23:41:32.139451+0000","last_unstale":"2024-08-18T23:42:23.314066+0000","last_undegraded":"2024-08-18T23:42:23.314066+0000","last_fullsized":"2024-08-18T23:42:23.314066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:15:13.390529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:07.263221+0000","last_change":"2024-08-18T23:41:30.126513+0000","last_active":"2024-08-18T23:42:07.263221+0000","last_peered":"2024-08-18T23:42:07.263221+0000","last_clean":"2024-08-18T23:42:07.263221+0000","last_became_active":"2024-08-18T23:41:30.126046+0000","last_became_peered":"2024-08-18T23:41:30.126046+0000","last_unstale":"2024-08-18T23:42:07.263221+0000","last_undegraded":"2024-08-18T23:42:07.263221+0000","last_fullsized":"2024-08-18T23:42:07.263221+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:49:16.565565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438321+0000","last_change":"2024-08-18T23:41:39.681006+0000","last_active":"2024-08-18T23:41:42.438321+0000","last_peered":"2024-08-18T23:41:42.438321+0000","last_clean":"2024-08-18T23:41:42.438321+0000","last_became_active":"2024-08-18T23:41:39.680409+0000","last_became_peered":"2024-08-18T23:41:39.680409+0000","last_unstale":"2024-08-18T23:41:42.438321+0000","last_undegraded":"2024-08-18T23:41:42.438321+0000","last_fullsized":"2024-08-18T23:41:42.438321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:47.778866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436780+0000","last_change":"2024-08-18T23:41:32.147545+0000","last_active":"2024-08-18T23:41:42.436780+0000","last_peered":"2024-08-18T23:41:42.436780+0000","last_clean":"2024-08-18T23:41:42.436780+0000","last_became_active":"2024-08-18T23:41:32.147238+0000","last_became_peered":"2024-08-18T23:41:32.147238+0000","last_unstale":"2024-08-18T23:41:42.436780+0000","last_undegraded":"2024-08-18T23:41:42.436780+0000","last_fullsized":"2024-08-18T23:41:42.436780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:12:13.589535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319674+0000","last_change":"2024-08-18T23:41:30.120650+0000","last_active":"2024-08-18T23:42:23.319674+0000","last_peered":"2024-08-18T23:42:23.319674+0000","last_clean":"2024-08-18T23:42:23.319674+0000","last_became_active":"2024-08-18T23:41:30.120196+0000","last_became_peered":"2024-08-18T23:41:30.120196+0000","last_unstale":"2024-08-18T23:42:23.319674+0000","last_undegraded":"2024-08-18T23:42:23.319674+0000","last_fullsized":"2024-08-18T23:42:23.319674+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:40:23.370088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437295+0000","last_change":"2024-08-18T23:40:22.013519+0000","last_active":"2024-08-18T23:41:42.437295+0000","last_peered":"2024-08-18T23:41:42.437295+0000","last_clean":"2024-08-18T23:41:42.437295+0000","last_became_active":"2024-08-18T23:40:22.012986+0000","last_became_peered":"2024-08-18T23:40:22.012986+0000","last_unstale":"2024-08-18T23:41:42.437295+0000","last_undegraded":"2024-08-18T23:41:42.437295+0000","last_fullsized":"2024-08-18T23:41:42.437295+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_clean_scrub_stamp":"2024-08-18T23:37:22.388424+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:19.874356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438358+0000","last_change":"2024-08-18T23:41:39.188400+0000","last_active":"2024-08-18T23:41:42.438358+0000","last_peered":"2024-08-18T23:41:42.438358+0000","last_clean":"2024-08-18T23:41:42.438358+0000","last_became_active":"2024-08-18T23:41:39.188023+0000","last_became_peered":"2024-08-18T23:41:39.188023+0000","last_unstale":"2024-08-18T23:41:42.438358+0000","last_undegraded":"2024-08-18T23:41:42.438358+0000","last_fullsized":"2024-08-18T23:41:42.438358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:07:33.845809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438418+0000","last_change":"2024-08-18T23:41:32.139907+0000","last_active":"2024-08-18T23:41:42.438418+0000","last_peered":"2024-08-18T23:41:42.438418+0000","last_clean":"2024-08-18T23:41:42.438418+0000","last_became_active":"2024-08-18T23:41:32.137956+0000","last_became_peered":"2024-08-18T23:41:32.137956+0000","last_unstale":"2024-08-18T23:41:42.438418+0000","last_undegraded":"2024-08-18T23:41:42.438418+0000","last_fullsized":"2024-08-18T23:41:42.438418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:10:13.223328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.330746+0000","last_change":"2024-08-18T23:42:23.330062+0000","last_active":"2024-08-18T23:42:23.330746+0000","last_peered":"2024-08-18T23:42:23.330746+0000","last_clean":"2024-08-18T23:42:23.330746+0000","last_became_active":"2024-08-18T23:42:23.323196+0000","last_became_peered":"2024-08-18T23:42:23.323196+0000","last_unstale":"2024-08-18T23:42:23.330746+0000","last_undegraded":"2024-08-18T23:42:23.330746+0000","last_fullsized":"2024-08-18T23:42:23.330746+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:53:29.059542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315093+0000","last_change":"2024-08-18T23:41:39.185917+0000","last_active":"2024-08-18T23:42:23.315093+0000","last_peered":"2024-08-18T23:42:23.315093+0000","last_clean":"2024-08-18T23:42:23.315093+0000","last_became_active":"2024-08-18T23:41:39.185427+0000","last_became_peered":"2024-08-18T23:41:39.185427+0000","last_unstale":"2024-08-18T23:42:23.315093+0000","last_undegraded":"2024-08-18T23:42:23.315093+0000","last_fullsized":"2024-08-18T23:42:23.315093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:52:13.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437928+0000","last_change":"2024-08-18T23:41:32.147530+0000","last_active":"2024-08-18T23:41:42.437928+0000","last_peered":"2024-08-18T23:41:42.437928+0000","last_clean":"2024-08-18T23:41:42.437928+0000","last_became_active":"2024-08-18T23:41:32.147216+0000","last_became_peered":"2024-08-18T23:41:32.147216+0000","last_unstale":"2024-08-18T23:41:42.437928+0000","last_undegraded":"2024-08-18T23:41:42.437928+0000","last_fullsized":"2024-08-18T23:41:42.437928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:04:06.421490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438376+0000","last_change":"2024-08-18T23:41:30.121307+0000","last_active":"2024-08-18T23:41:42.438376+0000","last_peered":"2024-08-18T23:41:42.438376+0000","last_clean":"2024-08-18T23:41:42.438376+0000","last_became_active":"2024-08-18T23:41:30.120932+0000","last_became_peered":"2024-08-18T23:41:30.120932+0000","last_unstale":"2024-08-18T23:41:42.438376+0000","last_undegraded":"2024-08-18T23:41:42.438376+0000","last_fullsized":"2024-08-18T23:41:42.438376+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:06:07.742880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316371+0000","last_change":"2024-08-18T23:41:39.184647+0000","last_active":"2024-08-18T23:42:23.316371+0000","last_peered":"2024-08-18T23:42:23.316371+0000","last_clean":"2024-08-18T23:42:23.316371+0000","last_became_active":"2024-08-18T23:41:39.184251+0000","last_became_peered":"2024-08-18T23:41:39.184251+0000","last_unstale":"2024-08-18T23:42:23.316371+0000","last_undegraded":"2024-08-18T23:42:23.316371+0000","last_fullsized":"2024-08-18T23:42:23.316371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:55.200801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437485+0000","last_change":"2024-08-18T23:41:32.137007+0000","last_active":"2024-08-18T23:41:42.437485+0000","last_peered":"2024-08-18T23:41:42.437485+0000","last_clean":"2024-08-18T23:41:42.437485+0000","last_became_active":"2024-08-18T23:41:32.136684+0000","last_became_peered":"2024-08-18T23:41:32.136684+0000","last_unstale":"2024-08-18T23:41:42.437485+0000","last_undegraded":"2024-08-18T23:41:42.437485+0000","last_fullsized":"2024-08-18T23:41:42.437485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:21:26.584303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314514+0000","last_change":"2024-08-18T23:41:30.121183+0000","last_active":"2024-08-18T23:42:23.314514+0000","last_peered":"2024-08-18T23:42:23.314514+0000","last_clean":"2024-08-18T23:42:23.314514+0000","last_became_active":"2024-08-18T23:41:30.120847+0000","last_became_peered":"2024-08-18T23:41:30.120847+0000","last_unstale":"2024-08-18T23:42:23.314514+0000","last_undegraded":"2024-08-18T23:42:23.314514+0000","last_fullsized":"2024-08-18T23:42:23.314514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:51.025821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437287+0000","last_change":"2024-08-18T23:41:39.183375+0000","last_active":"2024-08-18T23:41:42.437287+0000","last_peered":"2024-08-18T23:41:42.437287+0000","last_clean":"2024-08-18T23:41:42.437287+0000","last_became_active":"2024-08-18T23:41:39.182339+0000","last_became_peered":"2024-08-18T23:41:39.182339+0000","last_unstale":"2024-08-18T23:41:42.437287+0000","last_undegraded":"2024-08-18T23:41:42.437287+0000","last_fullsized":"2024-08-18T23:41:42.437287+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:26.119056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.313184+0000","last_change":"2024-08-18T23:41:32.140748+0000","last_active":"2024-08-18T23:42:22.313184+0000","last_peered":"2024-08-18T23:42:22.313184+0000","last_clean":"2024-08-18T23:42:22.313184+0000","last_became_active":"2024-08-18T23:41:32.140306+0000","last_became_peered":"2024-08-18T23:41:32.140306+0000","last_unstale":"2024-08-18T23:42:22.313184+0000","last_undegraded":"2024-08-18T23:42:22.313184+0000","last_fullsized":"2024-08-18T23:42:22.313184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:23:02.371529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324632+0000","last_change":"2024-08-18T23:41:30.121617+0000","last_active":"2024-08-18T23:42:23.324632+0000","last_peered":"2024-08-18T23:42:23.324632+0000","last_clean":"2024-08-18T23:42:23.324632+0000","last_became_active":"2024-08-18T23:41:30.121257+0000","last_became_peered":"2024-08-18T23:41:30.121257+0000","last_unstale":"2024-08-18T23:42:23.324632+0000","last_undegraded":"2024-08-18T23:42:23.324632+0000","last_fullsized":"2024-08-18T23:42:23.324632+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:23:42.091553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438472+0000","last_change":"2024-08-18T23:41:39.183833+0000","last_active":"2024-08-18T23:41:42.438472+0000","last_peered":"2024-08-18T23:41:42.438472+0000","last_clean":"2024-08-18T23:41:42.438472+0000","last_became_active":"2024-08-18T23:41:39.183176+0000","last_became_peered":"2024-08-18T23:41:39.183176+0000","last_unstale":"2024-08-18T23:41:42.438472+0000","last_undegraded":"2024-08-18T23:41:42.438472+0000","last_fullsized":"2024-08-18T23:41:42.438472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:36:09.827378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438671+0000","last_change":"2024-08-18T23:41:32.147507+0000","last_active":"2024-08-18T23:41:42.438671+0000","last_peered":"2024-08-18T23:41:42.438671+0000","last_clean":"2024-08-18T23:41:42.438671+0000","last_became_active":"2024-08-18T23:41:32.146042+0000","last_became_peered":"2024-08-18T23:41:32.146042+0000","last_unstale":"2024-08-18T23:41:42.438671+0000","last_undegraded":"2024-08-18T23:41:42.438671+0000","last_fullsized":"2024-08-18T23:41:42.438671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:48.941442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323009+0000","last_change":"2024-08-18T23:41:30.137755+0000","last_active":"2024-08-18T23:42:23.323009+0000","last_peered":"2024-08-18T23:42:23.323009+0000","last_clean":"2024-08-18T23:42:23.323009+0000","last_became_active":"2024-08-18T23:41:30.137194+0000","last_became_peered":"2024-08-18T23:41:30.137194+0000","last_unstale":"2024-08-18T23:42:23.323009+0000","last_undegraded":"2024-08-18T23:42:23.323009+0000","last_fullsized":"2024-08-18T23:42:23.323009+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:15:49.681405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314506+0000","last_change":"2024-08-18T23:41:39.183986+0000","last_active":"2024-08-18T23:42:23.314506+0000","last_peered":"2024-08-18T23:42:23.314506+0000","last_clean":"2024-08-18T23:42:23.314506+0000","last_became_active":"2024-08-18T23:41:39.183384+0000","last_became_peered":"2024-08-18T23:41:39.183384+0000","last_unstale":"2024-08-18T23:42:23.314506+0000","last_undegraded":"2024-08-18T23:42:23.314506+0000","last_fullsized":"2024-08-18T23:42:23.314506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:04:28.920804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436902+0000","last_change":"2024-08-18T23:41:32.134729+0000","last_active":"2024-08-18T23:41:42.436902+0000","last_peered":"2024-08-18T23:41:42.436902+0000","last_clean":"2024-08-18T23:41:42.436902+0000","last_became_active":"2024-08-18T23:41:32.134366+0000","last_became_peered":"2024-08-18T23:41:32.134366+0000","last_unstale":"2024-08-18T23:41:42.436902+0000","last_undegraded":"2024-08-18T23:41:42.436902+0000","last_fullsized":"2024-08-18T23:41:42.436902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:38.610561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318624+0000","last_change":"2024-08-18T23:41:30.126443+0000","last_active":"2024-08-18T23:42:23.318624+0000","last_peered":"2024-08-18T23:42:23.318624+0000","last_clean":"2024-08-18T23:42:23.318624+0000","last_became_active":"2024-08-18T23:41:30.124264+0000","last_became_peered":"2024-08-18T23:41:30.124264+0000","last_unstale":"2024-08-18T23:42:23.318624+0000","last_undegraded":"2024-08-18T23:42:23.318624+0000","last_fullsized":"2024-08-18T23:42:23.318624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:43.727114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.184435+0000","last_change":"2024-08-18T23:41:39.192164+0000","last_active":"2024-08-18T23:41:42.184435+0000","last_peered":"2024-08-18T23:41:42.184435+0000","last_clean":"2024-08-18T23:41:42.184435+0000","last_became_active":"2024-08-18T23:41:39.191833+0000","last_became_peered":"2024-08-18T23:41:39.191833+0000","last_unstale":"2024-08-18T23:41:42.184435+0000","last_undegraded":"2024-08-18T23:41:42.184435+0000","last_fullsized":"2024-08-18T23:41:42.184435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:33:50.243930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437783+0000","last_change":"2024-08-18T23:41:32.137384+0000","last_active":"2024-08-18T23:41:42.437783+0000","last_peered":"2024-08-18T23:41:42.437783+0000","last_clean":"2024-08-18T23:41:42.437783+0000","last_became_active":"2024-08-18T23:41:32.137051+0000","last_became_peered":"2024-08-18T23:41:32.137051+0000","last_unstale":"2024-08-18T23:41:42.437783+0000","last_undegraded":"2024-08-18T23:41:42.437783+0000","last_fullsized":"2024-08-18T23:41:42.437783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:22:04.367548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:06.259611+0000","last_change":"2024-08-18T23:41:30.124071+0000","last_active":"2024-08-18T23:42:06.259611+0000","last_peered":"2024-08-18T23:42:06.259611+0000","last_clean":"2024-08-18T23:42:06.259611+0000","last_became_active":"2024-08-18T23:41:30.123118+0000","last_became_peered":"2024-08-18T23:41:30.123118+0000","last_unstale":"2024-08-18T23:42:06.259611+0000","last_undegraded":"2024-08-18T23:42:06.259611+0000","last_fullsized":"2024-08-18T23:42:06.259611+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:21:39.015187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.310039+0000","last_change":"2024-08-18T23:41:39.184038+0000","last_active":"2024-08-18T23:42:22.310039+0000","last_peered":"2024-08-18T23:42:22.310039+0000","last_clean":"2024-08-18T23:42:22.310039+0000","last_became_active":"2024-08-18T23:41:39.183611+0000","last_became_peered":"2024-08-18T23:41:39.183611+0000","last_unstale":"2024-08-18T23:42:22.310039+0000","last_undegraded":"2024-08-18T23:42:22.310039+0000","last_fullsized":"2024-08-18T23:42:22.310039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:32:26.029151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314264+0000","last_change":"2024-08-18T23:41:32.143631+0000","last_active":"2024-08-18T23:42:23.314264+0000","last_peered":"2024-08-18T23:42:23.314264+0000","last_clean":"2024-08-18T23:42:23.314264+0000","last_became_active":"2024-08-18T23:41:32.143350+0000","last_became_peered":"2024-08-18T23:41:32.143350+0000","last_unstale":"2024-08-18T23:42:23.314264+0000","last_undegraded":"2024-08-18T23:42:23.314264+0000","last_fullsized":"2024-08-18T23:42:23.314264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:41:23.713467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:07.261492+0000","last_change":"2024-08-18T23:41:30.123383+0000","last_active":"2024-08-18T23:42:07.261492+0000","last_peered":"2024-08-18T23:42:07.261492+0000","last_clean":"2024-08-18T23:42:07.261492+0000","last_became_active":"2024-08-18T23:41:30.123037+0000","last_became_peered":"2024-08-18T23:41:30.123037+0000","last_unstale":"2024-08-18T23:42:07.261492+0000","last_undegraded":"2024-08-18T23:42:07.261492+0000","last_fullsized":"2024-08-18T23:42:07.261492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:27.702478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437947+0000","last_change":"2024-08-18T23:41:39.184302+0000","last_active":"2024-08-18T23:41:42.437947+0000","last_peered":"2024-08-18T23:41:42.437947+0000","last_clean":"2024-08-18T23:41:42.437947+0000","last_became_active":"2024-08-18T23:41:39.183172+0000","last_became_peered":"2024-08-18T23:41:39.183172+0000","last_unstale":"2024-08-18T23:41:42.437947+0000","last_undegraded":"2024-08-18T23:41:42.437947+0000","last_fullsized":"2024-08-18T23:41:42.437947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:23:26.655865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324579+0000","last_change":"2024-08-18T23:41:32.137426+0000","last_active":"2024-08-18T23:42:23.324579+0000","last_peered":"2024-08-18T23:42:23.324579+0000","last_clean":"2024-08-18T23:42:23.324579+0000","last_became_active":"2024-08-18T23:41:32.137106+0000","last_became_peered":"2024-08-18T23:41:32.137106+0000","last_unstale":"2024-08-18T23:42:23.324579+0000","last_undegraded":"2024-08-18T23:42:23.324579+0000","last_fullsized":"2024-08-18T23:42:23.324579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:34:51.889663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'29","reported_seq":104,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:24.297368+0000","last_change":"2024-08-18T23:41:30.124320+0000","last_active":"2024-08-18T23:42:24.297368+0000","last_peered":"2024-08-18T23:42:24.297368+0000","last_clean":"2024-08-18T23:42:24.297368+0000","last_became_active":"2024-08-18T23:41:30.123979+0000","last_became_peered":"2024-08-18T23:41:30.123979+0000","last_unstale":"2024-08-18T23:42:24.297368+0000","last_undegraded":"2024-08-18T23:42:24.297368+0000","last_fullsized":"2024-08-18T23:42:24.297368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:31:03.689720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56743,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":76,"num_write":30,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.184049+0000","last_change":"2024-08-18T23:41:39.190481+0000","last_active":"2024-08-18T23:41:42.184049+0000","last_peered":"2024-08-18T23:41:42.184049+0000","last_clean":"2024-08-18T23:41:42.184049+0000","last_became_active":"2024-08-18T23:41:39.190076+0000","last_became_peered":"2024-08-18T23:41:39.190076+0000","last_unstale":"2024-08-18T23:41:42.184049+0000","last_undegraded":"2024-08-18T23:41:42.184049+0000","last_fullsized":"2024-08-18T23:41:42.184049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:47:39.313142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315607+0000","last_change":"2024-08-18T23:41:32.145422+0000","last_active":"2024-08-18T23:42:23.315607+0000","last_peered":"2024-08-18T23:42:23.315607+0000","last_clean":"2024-08-18T23:42:23.315607+0000","last_became_active":"2024-08-18T23:41:32.145047+0000","last_became_peered":"2024-08-18T23:41:32.145047+0000","last_unstale":"2024-08-18T23:42:23.315607+0000","last_undegraded":"2024-08-18T23:42:23.315607+0000","last_fullsized":"2024-08-18T23:42:23.315607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:31.905248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:06.257992+0000","last_change":"2024-08-18T23:41:30.137364+0000","last_active":"2024-08-18T23:42:06.257992+0000","last_peered":"2024-08-18T23:42:06.257992+0000","last_clean":"2024-08-18T23:42:06.257992+0000","last_became_active":"2024-08-18T23:41:30.137004+0000","last_became_peered":"2024-08-18T23:41:30.137004+0000","last_unstale":"2024-08-18T23:42:06.257992+0000","last_undegraded":"2024-08-18T23:42:06.257992+0000","last_fullsized":"2024-08-18T23:42:06.257992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:20:56.392149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.326359+0000","last_change":"2024-08-18T23:42:23.326359+0000","last_active":"2024-08-18T23:42:23.326359+0000","last_peered":"2024-08-18T23:42:23.326359+0000","last_clean":"2024-08-18T23:42:23.326359+0000","last_became_active":"2024-08-18T23:42:23.325918+0000","last_became_peered":"2024-08-18T23:42:23.325918+0000","last_unstale":"2024-08-18T23:42:23.326359+0000","last_undegraded":"2024-08-18T23:42:23.326359+0000","last_fullsized":"2024-08-18T23:42:23.326359+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:01:15.476268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.325691+0000","last_change":"2024-08-18T23:42:23.325691+0000","last_active":"2024-08-18T23:42:23.325691+0000","last_peered":"2024-08-18T23:42:23.325691+0000","last_clean":"2024-08-18T23:42:23.325691+0000","last_became_active":"2024-08-18T23:42:23.325467+0000","last_became_peered":"2024-08-18T23:42:23.325467+0000","last_unstale":"2024-08-18T23:42:23.325691+0000","last_undegraded":"2024-08-18T23:42:23.325691+0000","last_fullsized":"2024-08-18T23:42:23.325691+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:10:26.288880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:06.259464+0000","last_change":"2024-08-18T23:41:30.124430+0000","last_active":"2024-08-18T23:42:06.259464+0000","last_peered":"2024-08-18T23:42:06.259464+0000","last_clean":"2024-08-18T23:42:06.259464+0000","last_became_active":"2024-08-18T23:41:30.123626+0000","last_became_peered":"2024-08-18T23:41:30.123626+0000","last_unstale":"2024-08-18T23:42:06.259464+0000","last_undegraded":"2024-08-18T23:42:06.259464+0000","last_fullsized":"2024-08-18T23:42:06.259464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:26:18.426010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.183907+0000","last_change":"2024-08-18T23:41:39.192765+0000","last_active":"2024-08-18T23:41:42.183907+0000","last_peered":"2024-08-18T23:41:42.183907+0000","last_clean":"2024-08-18T23:41:42.183907+0000","last_became_active":"2024-08-18T23:41:39.192250+0000","last_became_peered":"2024-08-18T23:41:39.192250+0000","last_unstale":"2024-08-18T23:41:42.183907+0000","last_undegraded":"2024-08-18T23:41:42.183907+0000","last_fullsized":"2024-08-18T23:41:42.183907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:51:16.794404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316158+0000","last_change":"2024-08-18T23:41:32.141966+0000","last_active":"2024-08-18T23:42:23.316158+0000","last_peered":"2024-08-18T23:42:23.316158+0000","last_clean":"2024-08-18T23:42:23.316158+0000","last_became_active":"2024-08-18T23:41:32.141561+0000","last_became_peered":"2024-08-18T23:41:32.141561+0000","last_unstale":"2024-08-18T23:42:23.316158+0000","last_undegraded":"2024-08-18T23:42:23.316158+0000","last_fullsized":"2024-08-18T23:42:23.316158+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:20.696089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558753+0000","last_change":"2024-08-18T23:41:30.121598+0000","last_active":"2024-08-18T23:42:23.558753+0000","last_peered":"2024-08-18T23:42:23.558753+0000","last_clean":"2024-08-18T23:42:23.558753+0000","last_became_active":"2024-08-18T23:41:30.120533+0000","last_became_peered":"2024-08-18T23:41:30.120533+0000","last_unstale":"2024-08-18T23:42:23.558753+0000","last_undegraded":"2024-08-18T23:42:23.558753+0000","last_fullsized":"2024-08-18T23:42:23.558753+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:06:37.458272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436352+0000","last_change":"2024-08-18T23:41:39.178877+0000","last_active":"2024-08-18T23:41:42.436352+0000","last_peered":"2024-08-18T23:41:42.436352+0000","last_clean":"2024-08-18T23:41:42.436352+0000","last_became_active":"2024-08-18T23:41:39.178381+0000","last_became_peered":"2024-08-18T23:41:39.178381+0000","last_unstale":"2024-08-18T23:41:42.436352+0000","last_undegraded":"2024-08-18T23:41:42.436352+0000","last_fullsized":"2024-08-18T23:41:42.436352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:27.275893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316665+0000","last_change":"2024-08-18T23:41:39.188381+0000","last_active":"2024-08-18T23:42:23.316665+0000","last_peered":"2024-08-18T23:42:23.316665+0000","last_clean":"2024-08-18T23:42:23.316665+0000","last_became_active":"2024-08-18T23:41:39.187983+0000","last_became_peered":"2024-08-18T23:41:39.187983+0000","last_unstale":"2024-08-18T23:42:23.316665+0000","last_undegraded":"2024-08-18T23:42:23.316665+0000","last_fullsized":"2024-08-18T23:42:23.316665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:04:45.460576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.183985+0000","last_change":"2024-08-18T23:41:32.144637+0000","last_active":"2024-08-18T23:41:42.183985+0000","last_peered":"2024-08-18T23:41:42.183985+0000","last_clean":"2024-08-18T23:41:42.183985+0000","last_became_active":"2024-08-18T23:41:32.144329+0000","last_became_peered":"2024-08-18T23:41:32.144329+0000","last_unstale":"2024-08-18T23:41:42.183985+0000","last_undegraded":"2024-08-18T23:41:42.183985+0000","last_fullsized":"2024-08-18T23:41:42.183985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:29:39.791264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437701+0000","last_change":"2024-08-18T23:41:30.122325+0000","last_active":"2024-08-18T23:41:42.437701+0000","last_peered":"2024-08-18T23:41:42.437701+0000","last_clean":"2024-08-18T23:41:42.437701+0000","last_became_active":"2024-08-18T23:41:30.121480+0000","last_became_peered":"2024-08-18T23:41:30.121480+0000","last_unstale":"2024-08-18T23:41:42.437701+0000","last_undegraded":"2024-08-18T23:41:42.437701+0000","last_fullsized":"2024-08-18T23:41:42.437701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:32:12.810650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"peering","last_fresh":"2024-08-18T23:41:42.471784+0000","last_change":"2024-08-18T23:42:22.309631+0000","last_active":"2024-08-18T23:42:22.309631+0000","last_peered":"2024-08-18T23:41:42.471784+0000","last_clean":"2024-08-18T23:41:42.471784+0000","last_became_active":"2024-08-18T23:41:39.185811+0000","last_became_peered":"2024-08-18T23:41:39.185811+0000","last_unstale":"2024-08-18T23:41:42.471784+0000","last_undegraded":"2024-08-18T23:41:42.471784+0000","last_fullsized":"2024-08-18T23:41:42.471784+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4,9,10],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436290+0000","last_change":"2024-08-18T23:41:32.145545+0000","last_active":"2024-08-18T23:41:42.436290+0000","last_peered":"2024-08-18T23:41:42.436290+0000","last_clean":"2024-08-18T23:41:42.436290+0000","last_became_active":"2024-08-18T23:41:32.145183+0000","last_became_peered":"2024-08-18T23:41:32.145183+0000","last_unstale":"2024-08-18T23:41:42.436290+0000","last_undegraded":"2024-08-18T23:41:42.436290+0000","last_fullsized":"2024-08-18T23:41:42.436290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:03:04.111658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314074+0000","last_change":"2024-08-18T23:41:30.118776+0000","last_active":"2024-08-18T23:42:23.314074+0000","last_peered":"2024-08-18T23:42:23.314074+0000","last_clean":"2024-08-18T23:42:23.314074+0000","last_became_active":"2024-08-18T23:41:30.118436+0000","last_became_peered":"2024-08-18T23:41:30.118436+0000","last_unstale":"2024-08-18T23:42:23.314074+0000","last_undegraded":"2024-08-18T23:42:23.314074+0000","last_fullsized":"2024-08-18T23:42:23.314074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:13.980927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558143+0000","last_change":"2024-08-18T23:41:30.125201+0000","last_active":"2024-08-18T23:42:23.558143+0000","last_peered":"2024-08-18T23:42:23.558143+0000","last_clean":"2024-08-18T23:42:23.558143+0000","last_became_active":"2024-08-18T23:41:30.124894+0000","last_became_peered":"2024-08-18T23:41:30.124894+0000","last_unstale":"2024-08-18T23:42:23.558143+0000","last_undegraded":"2024-08-18T23:42:23.558143+0000","last_fullsized":"2024-08-18T23:42:23.558143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:30:17.721829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437803+0000","last_change":"2024-08-18T23:41:32.145452+0000","last_active":"2024-08-18T23:41:42.437803+0000","last_peered":"2024-08-18T23:41:42.437803+0000","last_clean":"2024-08-18T23:41:42.437803+0000","last_became_active":"2024-08-18T23:41:32.145151+0000","last_became_peered":"2024-08-18T23:41:32.145151+0000","last_unstale":"2024-08-18T23:41:42.437803+0000","last_undegraded":"2024-08-18T23:41:42.437803+0000","last_fullsized":"2024-08-18T23:41:42.437803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:08:08.713154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438790+0000","last_change":"2024-08-18T23:41:39.193334+0000","last_active":"2024-08-18T23:41:42.438790+0000","last_peered":"2024-08-18T23:41:42.438790+0000","last_clean":"2024-08-18T23:41:42.438790+0000","last_became_active":"2024-08-18T23:41:39.192924+0000","last_became_peered":"2024-08-18T23:41:39.192924+0000","last_unstale":"2024-08-18T23:41:42.438790+0000","last_undegraded":"2024-08-18T23:41:42.438790+0000","last_fullsized":"2024-08-18T23:41:42.438790+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:07.016300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:42:20.291145+0000","last_change":"2024-08-18T23:41:30.125217+0000","last_active":"2024-08-18T23:42:20.291145+0000","last_peered":"2024-08-18T23:42:20.291145+0000","last_clean":"2024-08-18T23:42:20.291145+0000","last_became_active":"2024-08-18T23:41:30.124889+0000","last_became_peered":"2024-08-18T23:41:30.124889+0000","last_unstale":"2024-08-18T23:42:20.291145+0000","last_undegraded":"2024-08-18T23:42:20.291145+0000","last_fullsized":"2024-08-18T23:42:20.291145+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:43.389537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323355+0000","last_change":"2024-08-18T23:41:32.142585+0000","last_active":"2024-08-18T23:42:23.323355+0000","last_peered":"2024-08-18T23:42:23.323355+0000","last_clean":"2024-08-18T23:42:23.323355+0000","last_became_active":"2024-08-18T23:41:32.142207+0000","last_became_peered":"2024-08-18T23:41:32.142207+0000","last_unstale":"2024-08-18T23:42:23.323355+0000","last_undegraded":"2024-08-18T23:42:23.323355+0000","last_fullsized":"2024-08-18T23:42:23.323355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:16:58.755444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437736+0000","last_change":"2024-08-18T23:41:39.185618+0000","last_active":"2024-08-18T23:41:42.437736+0000","last_peered":"2024-08-18T23:41:42.437736+0000","last_clean":"2024-08-18T23:41:42.437736+0000","last_became_active":"2024-08-18T23:41:39.185222+0000","last_became_peered":"2024-08-18T23:41:39.185222+0000","last_unstale":"2024-08-18T23:41:42.437736+0000","last_undegraded":"2024-08-18T23:41:42.437736+0000","last_fullsized":"2024-08-18T23:41:42.437736+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:01:27.290998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.309157+0000","last_change":"2024-08-18T23:41:32.140719+0000","last_active":"2024-08-18T23:42:22.309157+0000","last_peered":"2024-08-18T23:42:22.309157+0000","last_clean":"2024-08-18T23:42:22.309157+0000","last_became_active":"2024-08-18T23:41:32.140131+0000","last_became_peered":"2024-08-18T23:41:32.140131+0000","last_unstale":"2024-08-18T23:42:22.309157+0000","last_undegraded":"2024-08-18T23:42:22.309157+0000","last_fullsized":"2024-08-18T23:42:22.309157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:48:26.778094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438674+0000","last_change":"2024-08-18T23:41:32.138778+0000","last_active":"2024-08-18T23:41:42.438674+0000","last_peered":"2024-08-18T23:41:42.438674+0000","last_clean":"2024-08-18T23:41:42.438674+0000","last_became_active":"2024-08-18T23:41:32.138408+0000","last_became_peered":"2024-08-18T23:41:32.138408+0000","last_unstale":"2024-08-18T23:41:42.438674+0000","last_undegraded":"2024-08-18T23:41:42.438674+0000","last_fullsized":"2024-08-18T23:41:42.438674+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:29:41.621518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436698+0000","last_change":"2024-08-18T23:41:32.149522+0000","last_active":"2024-08-18T23:41:42.436698+0000","last_peered":"2024-08-18T23:41:42.436698+0000","last_clean":"2024-08-18T23:41:42.436698+0000","last_became_active":"2024-08-18T23:41:32.149182+0000","last_became_peered":"2024-08-18T23:41:32.149182+0000","last_unstale":"2024-08-18T23:41:42.436698+0000","last_undegraded":"2024-08-18T23:41:42.436698+0000","last_fullsized":"2024-08-18T23:41:42.436698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:30:16.945045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316398+0000","last_change":"2024-08-18T23:41:32.149853+0000","last_active":"2024-08-18T23:42:23.316398+0000","last_peered":"2024-08-18T23:42:23.316398+0000","last_clean":"2024-08-18T23:42:23.316398+0000","last_became_active":"2024-08-18T23:41:32.141907+0000","last_became_peered":"2024-08-18T23:41:32.141907+0000","last_unstale":"2024-08-18T23:42:23.316398+0000","last_undegraded":"2024-08-18T23:42:23.316398+0000","last_fullsized":"2024-08-18T23:42:23.316398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:24:37.323567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314554+0000","last_change":"2024-08-18T23:41:32.136179+0000","last_active":"2024-08-18T23:42:23.314554+0000","last_peered":"2024-08-18T23:42:23.314554+0000","last_clean":"2024-08-18T23:42:23.314554+0000","last_became_active":"2024-08-18T23:41:32.135939+0000","last_became_peered":"2024-08-18T23:41:32.135939+0000","last_unstale":"2024-08-18T23:42:23.314554+0000","last_undegraded":"2024-08-18T23:42:23.314554+0000","last_fullsized":"2024-08-18T23:42:23.314554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:15.392061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.310009+0000","last_change":"2024-08-18T23:41:32.137509+0000","last_active":"2024-08-18T23:42:22.310009+0000","last_peered":"2024-08-18T23:42:22.310009+0000","last_clean":"2024-08-18T23:42:22.310009+0000","last_became_active":"2024-08-18T23:41:32.136380+0000","last_became_peered":"2024-08-18T23:41:32.136380+0000","last_unstale":"2024-08-18T23:42:22.310009+0000","last_undegraded":"2024-08-18T23:42:22.310009+0000","last_fullsized":"2024-08-18T23:42:22.310009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:49.776225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438165+0000","last_change":"2024-08-18T23:41:32.139271+0000","last_active":"2024-08-18T23:41:42.438165+0000","last_peered":"2024-08-18T23:41:42.438165+0000","last_clean":"2024-08-18T23:41:42.438165+0000","last_became_active":"2024-08-18T23:41:32.138868+0000","last_became_peered":"2024-08-18T23:41:32.138868+0000","last_unstale":"2024-08-18T23:41:42.438165+0000","last_undegraded":"2024-08-18T23:41:42.438165+0000","last_fullsized":"2024-08-18T23:41:42.438165+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:49:27.890857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436507+0000","last_change":"2024-08-18T23:41:32.144460+0000","last_active":"2024-08-18T23:41:42.436507+0000","last_peered":"2024-08-18T23:41:42.436507+0000","last_clean":"2024-08-18T23:41:42.436507+0000","last_became_active":"2024-08-18T23:41:32.144178+0000","last_became_peered":"2024-08-18T23:41:32.144178+0000","last_unstale":"2024-08-18T23:41:42.436507+0000","last_undegraded":"2024-08-18T23:41:42.436507+0000","last_fullsized":"2024-08-18T23:41:42.436507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:56.252215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437434+0000","last_change":"2024-08-18T23:41:32.147973+0000","last_active":"2024-08-18T23:41:42.437434+0000","last_peered":"2024-08-18T23:41:42.437434+0000","last_clean":"2024-08-18T23:41:42.437434+0000","last_became_active":"2024-08-18T23:41:32.147621+0000","last_became_peered":"2024-08-18T23:41:32.147621+0000","last_unstale":"2024-08-18T23:41:42.437434+0000","last_undegraded":"2024-08-18T23:41:42.437434+0000","last_fullsized":"2024-08-18T23:41:42.437434+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:14.176569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436418+0000","last_change":"2024-08-18T23:41:32.146029+0000","last_active":"2024-08-18T23:41:42.436418+0000","last_peered":"2024-08-18T23:41:42.436418+0000","last_clean":"2024-08-18T23:41:42.436418+0000","last_became_active":"2024-08-18T23:41:32.145584+0000","last_became_peered":"2024-08-18T23:41:32.145584+0000","last_unstale":"2024-08-18T23:41:42.436418+0000","last_undegraded":"2024-08-18T23:41:42.436418+0000","last_fullsized":"2024-08-18T23:41:42.436418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:25:33.884906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315440+0000","last_change":"2024-08-18T23:41:32.145950+0000","last_active":"2024-08-18T23:42:23.315440+0000","last_peered":"2024-08-18T23:42:23.315440+0000","last_clean":"2024-08-18T23:42:23.315440+0000","last_became_active":"2024-08-18T23:41:32.145618+0000","last_became_peered":"2024-08-18T23:41:32.145618+0000","last_unstale":"2024-08-18T23:42:23.315440+0000","last_undegraded":"2024-08-18T23:42:23.315440+0000","last_fullsized":"2024-08-18T23:42:23.315440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:05:58.889240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314395+0000","last_change":"2024-08-18T23:41:32.130684+0000","last_active":"2024-08-18T23:42:23.314395+0000","last_peered":"2024-08-18T23:42:23.314395+0000","last_clean":"2024-08-18T23:42:23.314395+0000","last_became_active":"2024-08-18T23:41:32.130369+0000","last_became_peered":"2024-08-18T23:41:32.130369+0000","last_unstale":"2024-08-18T23:42:23.314395+0000","last_undegraded":"2024-08-18T23:42:23.314395+0000","last_fullsized":"2024-08-18T23:42:23.314395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:22.537018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.310146+0000","last_change":"2024-08-18T23:41:32.137291+0000","last_active":"2024-08-18T23:42:22.310146+0000","last_peered":"2024-08-18T23:42:22.310146+0000","last_clean":"2024-08-18T23:42:22.310146+0000","last_became_active":"2024-08-18T23:41:32.136977+0000","last_became_peered":"2024-08-18T23:41:32.136977+0000","last_unstale":"2024-08-18T23:42:22.310146+0000","last_undegraded":"2024-08-18T23:42:22.310146+0000","last_fullsized":"2024-08-18T23:42:22.310146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:18:45.544760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.437895+0000","last_change":"2024-08-18T23:41:32.144434+0000","last_active":"2024-08-18T23:41:42.437895+0000","last_peered":"2024-08-18T23:41:42.437895+0000","last_clean":"2024-08-18T23:41:42.437895+0000","last_became_active":"2024-08-18T23:41:32.144119+0000","last_became_peered":"2024-08-18T23:41:32.144119+0000","last_unstale":"2024-08-18T23:41:42.437895+0000","last_undegraded":"2024-08-18T23:41:42.437895+0000","last_fullsized":"2024-08-18T23:41:42.437895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:50:54.528630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.184493+0000","last_change":"2024-08-18T23:41:32.135034+0000","last_active":"2024-08-18T23:41:42.184493+0000","last_peered":"2024-08-18T23:41:42.184493+0000","last_clean":"2024-08-18T23:41:42.184493+0000","last_became_active":"2024-08-18T23:41:32.134700+0000","last_became_peered":"2024-08-18T23:41:32.134700+0000","last_unstale":"2024-08-18T23:41:42.184493+0000","last_undegraded":"2024-08-18T23:41:42.184493+0000","last_fullsized":"2024-08-18T23:41:42.184493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:06:31.263114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.310319+0000","last_change":"2024-08-18T23:41:32.141191+0000","last_active":"2024-08-18T23:42:22.310319+0000","last_peered":"2024-08-18T23:42:22.310319+0000","last_clean":"2024-08-18T23:42:22.310319+0000","last_became_active":"2024-08-18T23:41:32.140901+0000","last_became_peered":"2024-08-18T23:41:32.140901+0000","last_unstale":"2024-08-18T23:42:22.310319+0000","last_undegraded":"2024-08-18T23:42:22.310319+0000","last_fullsized":"2024-08-18T23:42:22.310319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:24.849985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314798+0000","last_change":"2024-08-18T23:41:32.142543+0000","last_active":"2024-08-18T23:42:23.314798+0000","last_peered":"2024-08-18T23:42:23.314798+0000","last_clean":"2024-08-18T23:42:23.314798+0000","last_became_active":"2024-08-18T23:41:32.142084+0000","last_became_peered":"2024-08-18T23:41:32.142084+0000","last_unstale":"2024-08-18T23:42:23.314798+0000","last_undegraded":"2024-08-18T23:42:23.314798+0000","last_fullsized":"2024-08-18T23:42:23.314798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:54:18.200016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316269+0000","last_change":"2024-08-18T23:41:32.148968+0000","last_active":"2024-08-18T23:42:23.316269+0000","last_peered":"2024-08-18T23:42:23.316269+0000","last_clean":"2024-08-18T23:42:23.316269+0000","last_became_active":"2024-08-18T23:41:32.148625+0000","last_became_peered":"2024-08-18T23:41:32.148625+0000","last_unstale":"2024-08-18T23:42:23.316269+0000","last_undegraded":"2024-08-18T23:42:23.316269+0000","last_fullsized":"2024-08-18T23:42:23.316269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:11:40.172139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316345+0000","last_change":"2024-08-18T23:41:32.133478+0000","last_active":"2024-08-18T23:42:23.316345+0000","last_peered":"2024-08-18T23:42:23.316345+0000","last_clean":"2024-08-18T23:42:23.316345+0000","last_became_active":"2024-08-18T23:41:32.133149+0000","last_became_peered":"2024-08-18T23:41:32.133149+0000","last_unstale":"2024-08-18T23:42:23.316345+0000","last_undegraded":"2024-08-18T23:42:23.316345+0000","last_fullsized":"2024-08-18T23:42:23.316345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:57:53.713309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314293+0000","last_change":"2024-08-18T23:41:32.138206+0000","last_active":"2024-08-18T23:42:23.314293+0000","last_peered":"2024-08-18T23:42:23.314293+0000","last_clean":"2024-08-18T23:42:23.314293+0000","last_became_active":"2024-08-18T23:41:32.137903+0000","last_became_peered":"2024-08-18T23:41:32.137903+0000","last_unstale":"2024-08-18T23:42:23.314293+0000","last_undegraded":"2024-08-18T23:42:23.314293+0000","last_fullsized":"2024-08-18T23:42:23.314293+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:18:48.600658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319090+0000","last_change":"2024-08-18T23:41:32.136303+0000","last_active":"2024-08-18T23:42:23.319090+0000","last_peered":"2024-08-18T23:42:23.319090+0000","last_clean":"2024-08-18T23:42:23.319090+0000","last_became_active":"2024-08-18T23:41:32.134628+0000","last_became_peered":"2024-08-18T23:41:32.134628+0000","last_unstale":"2024-08-18T23:42:23.319090+0000","last_undegraded":"2024-08-18T23:42:23.319090+0000","last_fullsized":"2024-08-18T23:42:23.319090+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:28:51.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438368+0000","last_change":"2024-08-18T23:41:32.137512+0000","last_active":"2024-08-18T23:41:42.438368+0000","last_peered":"2024-08-18T23:41:42.438368+0000","last_clean":"2024-08-18T23:41:42.438368+0000","last_became_active":"2024-08-18T23:41:32.137150+0000","last_became_peered":"2024-08-18T23:41:32.137150+0000","last_unstale":"2024-08-18T23:41:42.438368+0000","last_undegraded":"2024-08-18T23:41:42.438368+0000","last_fullsized":"2024-08-18T23:41:42.438368+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:44:08.020780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558680+0000","last_change":"2024-08-18T23:41:32.139229+0000","last_active":"2024-08-18T23:42:23.558680+0000","last_peered":"2024-08-18T23:42:23.558680+0000","last_clean":"2024-08-18T23:42:23.558680+0000","last_became_active":"2024-08-18T23:41:32.138864+0000","last_became_peered":"2024-08-18T23:41:32.138864+0000","last_unstale":"2024-08-18T23:42:23.558680+0000","last_undegraded":"2024-08-18T23:42:23.558680+0000","last_fullsized":"2024-08-18T23:42:23.558680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:54:21.701750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316215+0000","last_change":"2024-08-18T23:41:32.148727+0000","last_active":"2024-08-18T23:42:23.316215+0000","last_peered":"2024-08-18T23:42:23.316215+0000","last_clean":"2024-08-18T23:42:23.316215+0000","last_became_active":"2024-08-18T23:41:32.146828+0000","last_became_peered":"2024-08-18T23:41:32.146828+0000","last_unstale":"2024-08-18T23:42:23.316215+0000","last_undegraded":"2024-08-18T23:42:23.316215+0000","last_fullsized":"2024-08-18T23:42:23.316215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:51:49.175150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.436255+0000","last_change":"2024-08-18T23:41:32.144796+0000","last_active":"2024-08-18T23:41:42.436255+0000","last_peered":"2024-08-18T23:41:42.436255+0000","last_clean":"2024-08-18T23:41:42.436255+0000","last_became_active":"2024-08-18T23:41:32.141931+0000","last_became_peered":"2024-08-18T23:41:32.141931+0000","last_unstale":"2024-08-18T23:41:42.436255+0000","last_undegraded":"2024-08-18T23:41:42.436255+0000","last_fullsized":"2024-08-18T23:41:42.436255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:48:08.428845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-18T23:41:42.438077+0000","last_change":"2024-08-18T23:41:32.135265+0000","last_active":"2024-08-18T23:41:42.438077+0000","last_peered":"2024-08-18T23:41:42.438077+0000","last_clean":"2024-08-18T23:41:42.438077+0000","last_became_active":"2024-08-18T23:41:32.134952+0000","last_became_peered":"2024-08-18T23:41:32.134952+0000","last_unstale":"2024-08-18T23:41:42.438077+0000","last_undegraded":"2024-08-18T23:41:42.438077+0000","last_fullsized":"2024-08-18T23:41:42.438077+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:35:14.099536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558127+0000","last_change":"2024-08-18T23:41:32.135941+0000","last_active":"2024-08-18T23:42:23.558127+0000","last_peered":"2024-08-18T23:42:23.558127+0000","last_clean":"2024-08-18T23:42:23.558127+0000","last_became_active":"2024-08-18T23:41:32.135623+0000","last_became_peered":"2024-08-18T23:41:32.135623+0000","last_unstale":"2024-08-18T23:42:23.558127+0000","last_undegraded":"2024-08-18T23:42:23.558127+0000","last_fullsized":"2024-08-18T23:42:23.558127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:43:36.504738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324078+0000","last_change":"2024-08-18T23:41:32.141643+0000","last_active":"2024-08-18T23:42:23.324078+0000","last_peered":"2024-08-18T23:42:23.324078+0000","last_clean":"2024-08-18T23:42:23.324078+0000","last_became_active":"2024-08-18T23:41:32.141301+0000","last_became_peered":"2024-08-18T23:41:32.141301+0000","last_unstale":"2024-08-18T23:42:23.324078+0000","last_undegraded":"2024-08-18T23:42:23.324078+0000","last_fullsized":"2024-08-18T23:42:23.324078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:23:53.100740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318979+0000","last_change":"2024-08-18T23:41:32.146730+0000","last_active":"2024-08-18T23:42:23.318979+0000","last_peered":"2024-08-18T23:42:23.318979+0000","last_clean":"2024-08-18T23:42:23.318979+0000","last_became_active":"2024-08-18T23:41:32.146252+0000","last_became_peered":"2024-08-18T23:41:32.146252+0000","last_unstale":"2024-08-18T23:42:23.318979+0000","last_undegraded":"2024-08-18T23:42:23.318979+0000","last_fullsized":"2024-08-18T23:42:23.318979+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:41.481761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316570+0000","last_change":"2024-08-18T23:41:32.149569+0000","last_active":"2024-08-18T23:42:23.316570+0000","last_peered":"2024-08-18T23:42:23.316570+0000","last_clean":"2024-08-18T23:42:23.316570+0000","last_became_active":"2024-08-18T23:41:32.149277+0000","last_became_peered":"2024-08-18T23:41:32.149277+0000","last_unstale":"2024-08-18T23:42:23.316570+0000","last_undegraded":"2024-08-18T23:42:23.316570+0000","last_fullsized":"2024-08-18T23:42:23.316570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:34:00.547909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316678+0000","last_change":"2024-08-18T23:41:32.137889+0000","last_active":"2024-08-18T23:42:23.316678+0000","last_peered":"2024-08-18T23:42:23.316678+0000","last_clean":"2024-08-18T23:42:23.316678+0000","last_became_active":"2024-08-18T23:41:32.137578+0000","last_became_peered":"2024-08-18T23:41:32.137578+0000","last_unstale":"2024-08-18T23:42:23.316678+0000","last_undegraded":"2024-08-18T23:42:23.316678+0000","last_fullsized":"2024-08-18T23:42:23.316678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:18:09.987838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-18T23:42:22.310210+0000","last_change":"2024-08-18T23:41:32.142084+0000","last_active":"2024-08-18T23:42:22.310210+0000","last_peered":"2024-08-18T23:42:22.310210+0000","last_clean":"2024-08-18T23:42:22.310210+0000","last_became_active":"2024-08-18T23:41:32.141670+0000","last_became_peered":"2024-08-18T23:41:32.141670+0000","last_unstale":"2024-08-18T23:42:22.310210+0000","last_undegraded":"2024-08-18T23:42:22.310210+0000","last_fullsized":"2024-08-18T23:42:22.310210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:03.990351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":36864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61133,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":124,"num_read_kb":128,"num_write":160,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":176330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514519,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":486092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776156,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28688,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716464,"statfs":{"total":95995035648,"available":95965659136,"internally_reserved":0,"allocated":1703936,"data_stored":1149415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939680,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":490424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168613,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":495830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397544,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36896,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708256,"statfs":{"total":95995035648,"available":95957254144,"internally_reserved":0,"allocated":1597440,"data_stored":1082546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561070,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":500426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790002,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1605632,"data_stored":1077744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953527,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":506048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":481698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":532246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":557079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":501814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":50832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T23:42:28.022 INFO:tasks.cephadm.ceph_manager.ceph:PG 2.1c is not active+clean 2024-08-18T23:42:28.022 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '2.1c', 'version': "82'2", 'reported_seq': 2, 'reported_epoch': 89, 'state': 'peering', 'last_fresh': '2024-08-18T23:41:33.156065+0000', 'last_change': '2024-08-18T23:42:22.476106+0000', 'last_active': '2024-08-18T23:42:22.476106+0000', 'last_peered': '2024-08-18T23:41:33.156065+0000', 'last_clean': '2024-08-18T23:41:33.156065+0000', 'last_became_active': '2024-08-18T23:41:30.119986+0000', 'last_became_peered': '2024-08-18T23:41:30.119986+0000', 'last_unstale': '2024-08-18T23:41:33.156065+0000', 'last_undegraded': '2024-08-18T23:41:33.156065+0000', 'last_fullsized': '2024-08-18T23:41:33.156065+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 79, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-18T23:41:29.102309+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-18T23:41:29.102309+0000', 'last_clean_scrub_stamp': '2024-08-18T23:41:29.102309+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 3, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 1, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [11, 1, 5], 'acting': [11, 1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 11, 'acting_primary': 11, 'purged_snaps': []} 2024-08-18T23:42:28.022 INFO:tasks.cephadm.ceph_manager.ceph:PG 4.1f is not active+clean 2024-08-18T23:42:28.022 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 18, 'reported_epoch': 88, 'state': 'peering', 'last_fresh': '2024-08-18T23:41:42.471784+0000', 'last_change': '2024-08-18T23:42:22.309631+0000', 'last_active': '2024-08-18T23:42:22.309631+0000', 'last_peered': '2024-08-18T23:41:42.471784+0000', 'last_clean': '2024-08-18T23:41:42.471784+0000', 'last_became_active': '2024-08-18T23:41:39.185811+0000', 'last_became_peered': '2024-08-18T23:41:39.185811+0000', 'last_unstale': '2024-08-18T23:41:42.471784+0000', 'last_undegraded': '2024-08-18T23:41:42.471784+0000', 'last_fullsized': '2024-08-18T23:41:42.471784+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 86, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-18T23:41:38.163790+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-18T23:41:38.163790+0000', 'last_clean_scrub_stamp': '2024-08-18T23:41:38.163790+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [11, 4, 10, 0], 'acting': [11, 4, 10, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 4, 9, 10], 'up_primary': 11, 'acting_primary': 11, 'purged_snaps': []} 2024-08-18T23:42:28.023 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph status --format=json 2024-08-18T23:42:28.279 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:28.401 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:28 smithi078 ceph-mon[18440]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:42:28.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:28 smithi161 ceph-mon[22027]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:42:28.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:28 smithi123 ceph-mon[21808]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:42:28.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.751+0000 7f050e06b640 1 -- 172.21.15.78:0/2228977069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05081022a0 msgr2=0x7f05081026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:28.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.751+0000 7f050e06b640 1 --2- 172.21.15.78:0/2228977069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05081022a0 0x7f05081026e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009900 tx=0x7f04f002a480 comp rx=0 tx=0).stop 2024-08-18T23:42:28.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.751+0000 7f050e06b640 1 -- 172.21.15.78:0/2228977069 shutdown_connections 2024-08-18T23:42:28.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.751+0000 7f050e06b640 1 --2- 172.21.15.78:0/2228977069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0508102c20 0x7f050810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:28.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.751+0000 7f050e06b640 1 --2- 172.21.15.78:0/2228977069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05081022a0 0x7f05081026e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:28.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.751+0000 7f050e06b640 1 --2- 172.21.15.78:0/2228977069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0508100b50 0x7f0508100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:28.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.751+0000 7f050e06b640 1 -- 172.21.15.78:0/2228977069 >> 172.21.15.78:0/2228977069 conn(0x7f05080fc500 msgr2=0x7f05080fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:28.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.751+0000 7f050e06b640 1 -- 172.21.15.78:0/2228977069 shutdown_connections 2024-08-18T23:42:28.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.751+0000 7f050e06b640 1 -- 172.21.15.78:0/2228977069 wait complete. 2024-08-18T23:42:28.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.752+0000 7f050e06b640 1 Processor -- start 2024-08-18T23:42:28.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.752+0000 7f050e06b640 1 -- start start 2024-08-18T23:42:28.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.752+0000 7f050e06b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0508100b50 0x7f050819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:28.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.752+0000 7f050e06b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05081022a0 0x7f050819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:28.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.752+0000 7f050e06b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0508102c20 0x7f050819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:28.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.752+0000 7f050e06b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0508110d20 con 0x7f05081022a0 2024-08-18T23:42:28.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.752+0000 7f050e06b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0508110ba0 con 0x7f0508102c20 2024-08-18T23:42:28.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.752+0000 7f050e06b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0508110ea0 con 0x7f0508100b50 2024-08-18T23:42:28.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.752+0000 7f04fffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05081022a0 0x7f050819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:28.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.752+0000 7f04fffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05081022a0 0x7f050819a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32942/0 (socket says 172.21.15.78:32942) 2024-08-18T23:42:28.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.752+0000 7f04fffff640 1 -- 172.21.15.78:0/2357611831 learned_addr learned my addr 172.21.15.78:0/2357611831 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:28.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.753+0000 7f05077fe640 1 --2- 172.21.15.78:0/2357611831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0508100b50 0x7f050819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:28.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.753+0000 7f0507fff640 1 --2- 172.21.15.78:0/2357611831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0508102c20 0x7f050819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:28.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.753+0000 7f04fffff640 1 -- 172.21.15.78:0/2357611831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0508100b50 msgr2=0x7f050819a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:28.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.753+0000 7f04fffff640 1 --2- 172.21.15.78:0/2357611831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0508100b50 0x7f050819a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:28.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.753+0000 7f04fffff640 1 -- 172.21.15.78:0/2357611831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0508102c20 msgr2=0x7f050819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:28.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.753+0000 7f04fffff640 1 --2- 172.21.15.78:0/2357611831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0508102c20 0x7f050819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:28.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.753+0000 7f04fffff640 1 -- 172.21.15.78:0/2357611831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050819f3b0 con 0x7f05081022a0 2024-08-18T23:42:28.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.753+0000 7f0507fff640 1 --2- 172.21.15.78:0/2357611831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0508102c20 0x7f050819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:42:28.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.753+0000 7f04fffff640 1 --2- 172.21.15.78:0/2357611831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05081022a0 0x7f050819a940 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f04f0006010 tx=0x7f04f0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:28.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.753+0000 7f05057fa640 1 -- 172.21.15.78:0/2357611831 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f003f070 con 0x7f05081022a0 2024-08-18T23:42:28.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.753+0000 7f050e06b640 1 -- 172.21.15.78:0/2357611831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f050819f640 con 0x7f05081022a0 2024-08-18T23:42:28.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.753+0000 7f05057fa640 1 -- 172.21.15.78:0/2357611831 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04f0003e30 con 0x7f05081022a0 2024-08-18T23:42:28.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.753+0000 7f05057fa640 1 -- 172.21.15.78:0/2357611831 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0035030 con 0x7f05081022a0 2024-08-18T23:42:28.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.753+0000 7f050e06b640 1 -- 172.21.15.78:0/2357611831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05081a6f80 con 0x7f05081022a0 2024-08-18T23:42:28.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.754+0000 7f050e06b640 1 -- 172.21.15.78:0/2357611831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0508100f30 con 0x7f05081022a0 2024-08-18T23:42:28.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.755+0000 7f05057fa640 1 -- 172.21.15.78:0/2357611831 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f04f0042070 con 0x7f05081022a0 2024-08-18T23:42:28.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.756+0000 7f05057fa640 1 --2- 172.21.15.78:0/2357611831 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f04f40a2c10 0x7f04f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:28.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.756+0000 7f05077fe640 1 --2- 172.21.15.78:0/2357611831 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f04f40a2c10 0x7f04f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:28.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.756+0000 7f05057fa640 1 -- 172.21.15.78:0/2357611831 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f04f00f04a0 con 0x7f05081022a0 2024-08-18T23:42:28.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.756+0000 7f05077fe640 1 --2- 172.21.15.78:0/2357611831 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f04f40a2c10 0x7f04f40a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f04f80058c0 tx=0x7f04f800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:28.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.759+0000 7f05057fa640 1 -- 172.21.15.78:0/2357611831 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f04f00bcf40 con 0x7f05081022a0 2024-08-18T23:42:28.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.991+0000 7f050e06b640 1 -- 172.21.15.78:0/2357611831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f050819b760 con 0x7f05081022a0 2024-08-18T23:42:28.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.991+0000 7f05057fa640 1 -- 172.21.15.78:0/2357611831 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2508 (secure 0 0 0) 0x7f050819b760 con 0x7f05081022a0 2024-08-18T23:42:28.992 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:42:28.992 INFO:teuthology.orchestra.run.smithi078.stdout:{"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":379,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":90,"num_osds":12,"num_up_osds":12,"osd_up_since":1724024442,"num_in_osds":12,"osd_in_since":1724024422,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129}],"num_pgs":129,"num_pools":4,"num_objects":64,"data_bytes":651735,"bytes_used":397512704,"bytes_avail":1151542915072,"bytes_total":1151940427776,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":87,"recovering_keys_per_sec":0,"num_objects_recovered":5,"num_bytes_recovered":699,"num_keys_recovered":0,"read_bytes_sec":2815,"write_bytes_sec":1407,"read_op_per_sec":4,"write_op_per_sec":1},"fsmap":{"epoch":24,"btime":"2024-08-18T23:42:10:259464+0000","id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"f","status":"up:standby-replay","gid":24397},{"filesystem_id":1,"rank":1,"name":"c","status":"up:standby-replay","gid":24403},{"filesystem_id":1,"rank":0,"name":"e","status":"up:standby-replay","gid":24407},{"filesystem_id":1,"rank":2,"name":"i","status":"up:active","gid":24427},{"filesystem_id":1,"rank":1,"name":"h","status":"up:active","gid":24428},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24452}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.78:8443/"}},"servicemap":{"epoch":27,"modified":"2024-08-18T23:42:22.120591+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-08-18T23:42:28.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.993+0000 7f050e06b640 1 -- 172.21.15.78:0/2357611831 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f04f40a2c10 msgr2=0x7f04f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:28.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.993+0000 7f050e06b640 1 --2- 172.21.15.78:0/2357611831 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f04f40a2c10 0x7f04f40a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f04f80058c0 tx=0x7f04f800a2d0 comp rx=0 tx=0).stop 2024-08-18T23:42:28.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.993+0000 7f050e06b640 1 -- 172.21.15.78:0/2357611831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05081022a0 msgr2=0x7f050819a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:28.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.993+0000 7f050e06b640 1 --2- 172.21.15.78:0/2357611831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05081022a0 0x7f050819a940 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f04f0006010 tx=0x7f04f0003c10 comp rx=0 tx=0).stop 2024-08-18T23:42:28.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.993+0000 7f050e06b640 1 -- 172.21.15.78:0/2357611831 shutdown_connections 2024-08-18T23:42:28.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.993+0000 7f050e06b640 1 --2- 172.21.15.78:0/2357611831 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f04f40a2c10 0x7f04f40a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:28.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.993+0000 7f050e06b640 1 --2- 172.21.15.78:0/2357611831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0508102c20 0x7f050819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:28.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.993+0000 7f050e06b640 1 --2- 172.21.15.78:0/2357611831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05081022a0 0x7f050819a940 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:28.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.993+0000 7f050e06b640 1 --2- 172.21.15.78:0/2357611831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0508100b50 0x7f050819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:28.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.993+0000 7f050e06b640 1 -- 172.21.15.78:0/2357611831 >> 172.21.15.78:0/2357611831 conn(0x7f05080fc500 msgr2=0x7f050806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:28.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.993+0000 7f050e06b640 1 -- 172.21.15.78:0/2357611831 shutdown_connections 2024-08-18T23:42:28.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:28.993+0000 7f050e06b640 1 -- 172.21.15.78:0/2357611831 wait complete. 2024-08-18T23:42:29.320 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:29 smithi078 ceph-mon[18440]: pgmap v293: 129 pgs: 129 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 87 B/s, 0 objects/s recovering 2024-08-18T23:42:29.320 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:29 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2357611831' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-08-18T23:42:29.320 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:29 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:42:29.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:29 smithi123 ceph-mon[21808]: pgmap v293: 129 pgs: 129 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 87 B/s, 0 objects/s recovering 2024-08-18T23:42:29.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:29 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2357611831' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-08-18T23:42:29.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:29 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:42:29.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:29 smithi161 ceph-mon[22027]: pgmap v293: 129 pgs: 129 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 87 B/s, 0 objects/s recovering 2024-08-18T23:42:29.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:29 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2357611831' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-08-18T23:42:29.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:29 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:42:29.793 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-08-18T23:42:29.793 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-18T23:42:30.682 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:31.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.174+0000 7f44bbe88640 1 -- 172.21.15.78:0/321904327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4102260 msgr2=0x7f44b410e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:31.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.174+0000 7f44bbe88640 1 --2- 172.21.15.78:0/321904327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4102260 0x7f44b410e790 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009930 tx=0x7f44ac02a420 comp rx=0 tx=0).stop 2024-08-18T23:42:31.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.174+0000 7f44bbe88640 1 -- 172.21.15.78:0/321904327 shutdown_connections 2024-08-18T23:42:31.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.174+0000 7f44bbe88640 1 --2- 172.21.15.78:0/321904327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4102260 0x7f44b410e790 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:31.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.174+0000 7f44bbe88640 1 --2- 172.21.15.78:0/321904327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b41018c0 0x7f44b4101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:31.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.174+0000 7f44bbe88640 1 --2- 172.21.15.78:0/321904327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b41078c0 0x7f44b4107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:31.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.174+0000 7f44bbe88640 1 -- 172.21.15.78:0/321904327 >> 172.21.15.78:0/321904327 conn(0x7f44b40fd5e0 msgr2=0x7f44b40ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:31.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.174+0000 7f44bbe88640 1 -- 172.21.15.78:0/321904327 shutdown_connections 2024-08-18T23:42:31.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.174+0000 7f44bbe88640 1 -- 172.21.15.78:0/321904327 wait complete. 2024-08-18T23:42:31.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.174+0000 7f44bbe88640 1 Processor -- start 2024-08-18T23:42:31.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.174+0000 7f44bbe88640 1 -- start start 2024-08-18T23:42:31.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44bbe88640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b41018c0 0x7f44b4198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:31.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44bbe88640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4102260 0x7f44b4198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:31.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44bbe88640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b41078c0 0x7f44b419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:31.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44bbe88640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f44b4111cc0 con 0x7f44b4102260 2024-08-18T23:42:31.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44bbe88640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f44b4111b40 con 0x7f44b41018c0 2024-08-18T23:42:31.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44bbe88640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f44b4111e40 con 0x7f44b41078c0 2024-08-18T23:42:31.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44b9423640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4102260 0x7f44b4198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:31.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44b9423640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4102260 0x7f44b4198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32952/0 (socket says 172.21.15.78:32952) 2024-08-18T23:42:31.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44b9423640 1 -- 172.21.15.78:0/349518241 learned_addr learned my addr 172.21.15.78:0/349518241 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:31.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44b9c24640 1 --2- 172.21.15.78:0/349518241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b41018c0 0x7f44b4198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:31.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44ba425640 1 --2- 172.21.15.78:0/349518241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b41078c0 0x7f44b419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:31.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44b9423640 1 -- 172.21.15.78:0/349518241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b41078c0 msgr2=0x7f44b419ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:31.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44b9423640 1 --2- 172.21.15.78:0/349518241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b41078c0 0x7f44b419ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:31.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44b9423640 1 -- 172.21.15.78:0/349518241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b41018c0 msgr2=0x7f44b4198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:31.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44b9423640 1 --2- 172.21.15.78:0/349518241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b41018c0 0x7f44b4198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:31.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44b9423640 1 -- 172.21.15.78:0/349518241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b419d0f0 con 0x7f44b4102260 2024-08-18T23:42:31.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44b9c24640 1 --2- 172.21.15.78:0/349518241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b41018c0 0x7f44b4198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:42:31.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44b9423640 1 --2- 172.21.15.78:0/349518241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4102260 0x7f44b4198680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f44a800d950 tx=0x7f44a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:31.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44b2ffd640 1 -- 172.21.15.78:0/349518241 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8017030 con 0x7f44b4102260 2024-08-18T23:42:31.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44bbe88640 1 -- 172.21.15.78:0/349518241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b419d3e0 con 0x7f44b4102260 2024-08-18T23:42:31.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44b2ffd640 1 -- 172.21.15.78:0/349518241 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a8004020 con 0x7f44b4102260 2024-08-18T23:42:31.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44b2ffd640 1 -- 172.21.15.78:0/349518241 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a80047a0 con 0x7f44b4102260 2024-08-18T23:42:31.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.175+0000 7f44bbe88640 1 -- 172.21.15.78:0/349518241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b41a4cc0 con 0x7f44b4102260 2024-08-18T23:42:31.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.176+0000 7f44bbe88640 1 -- 172.21.15.78:0/349518241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b4101d20 con 0x7f44b4102260 2024-08-18T23:42:31.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.177+0000 7f44b2ffd640 1 -- 172.21.15.78:0/349518241 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f44a80041d0 con 0x7f44b4102260 2024-08-18T23:42:31.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.178+0000 7f44b2ffd640 1 --2- 172.21.15.78:0/349518241 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f449c0a2c10 0x7f449c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:31.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.178+0000 7f44b9c24640 1 --2- 172.21.15.78:0/349518241 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f449c0a2c10 0x7f449c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:31.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.178+0000 7f44b2ffd640 1 -- 172.21.15.78:0/349518241 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f44a80cefd0 con 0x7f44b4102260 2024-08-18T23:42:31.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.179+0000 7f44b9c24640 1 --2- 172.21.15.78:0/349518241 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f449c0a2c10 0x7f449c0a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f44a4005fd0 tx=0x7f44a40093c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:31.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.183+0000 7f44b2ffd640 1 -- 172.21.15.78:0/349518241 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f44a809ba70 con 0x7f44b4102260 2024-08-18T23:42:31.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.383+0000 7f44bbe88640 1 -- 172.21.15.78:0/349518241 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f44b4062360 con 0x7f449c0a2c10 2024-08-18T23:42:31.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:31 smithi078 ceph-mon[18440]: pgmap v294: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 9 op/s; 87 B/s, 0 objects/s recovering 2024-08-18T23:42:31.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:31 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:31.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:31 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:31.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:31 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:31.384 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:31 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:31.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.391+0000 7f44b2ffd640 1 -- 172.21.15.78:0/349518241 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340393 (secure 0 0 0) 0x7f44b4062360 con 0x7f449c0a2c10 2024-08-18T23:42:31.395 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:42:31.397 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-08-18T23:42:31.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.395+0000 7f44bbe88640 1 -- 172.21.15.78:0/349518241 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f449c0a2c10 msgr2=0x7f449c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:31.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.395+0000 7f44bbe88640 1 --2- 172.21.15.78:0/349518241 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f449c0a2c10 0x7f449c0a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f44a4005fd0 tx=0x7f44a40093c0 comp rx=0 tx=0).stop 2024-08-18T23:42:31.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.395+0000 7f44bbe88640 1 -- 172.21.15.78:0/349518241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4102260 msgr2=0x7f44b4198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:31.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.395+0000 7f44bbe88640 1 --2- 172.21.15.78:0/349518241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4102260 0x7f44b4198680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f44a800d950 tx=0x7f44a800de10 comp rx=0 tx=0).stop 2024-08-18T23:42:31.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.395+0000 7f44bbe88640 1 -- 172.21.15.78:0/349518241 shutdown_connections 2024-08-18T23:42:31.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.395+0000 7f44bbe88640 1 --2- 172.21.15.78:0/349518241 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f449c0a2c10 0x7f449c0a50d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:31.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.395+0000 7f44bbe88640 1 --2- 172.21.15.78:0/349518241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44b41078c0 0x7f44b419ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:31.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.395+0000 7f44bbe88640 1 --2- 172.21.15.78:0/349518241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4102260 0x7f44b4198680 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:31.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.395+0000 7f44bbe88640 1 --2- 172.21.15.78:0/349518241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b41018c0 0x7f44b4198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:31.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.395+0000 7f44bbe88640 1 -- 172.21.15.78:0/349518241 >> 172.21.15.78:0/349518241 conn(0x7f44b40fd5e0 msgr2=0x7f44b40fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:31.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.395+0000 7f44bbe88640 1 -- 172.21.15.78:0/349518241 shutdown_connections 2024-08-18T23:42:31.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:31.395+0000 7f44bbe88640 1 -- 172.21.15.78:0/349518241 wait complete. 2024-08-18T23:42:31.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:31 smithi123 ceph-mon[21808]: pgmap v294: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 9 op/s; 87 B/s, 0 objects/s recovering 2024-08-18T23:42:31.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:31 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:31.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:31 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:31.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:31 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:31.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:31 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:31 smithi161 ceph-mon[22027]: pgmap v294: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 9 op/s; 87 B/s, 0 objects/s recovering 2024-08-18T23:42:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:31 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:31 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:31 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:31 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:32.259 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-08-18T23:42:30.122563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656000,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":281,"num_read_kb":257,"num_write":349,"num_write_kb":3055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":423,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388216,"kb_used_data":14028,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553608,"statfs":{"total":1151940427776,"available":1151542894592,"internally_reserved":0,"allocated":14364672,"data_stored":7995568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11916,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56,"num_read_kb":42,"num_write":17,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002164"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319578+0000","last_change":"2024-08-18T23:41:30.120580+0000","last_active":"2024-08-18T23:42:23.319578+0000","last_peered":"2024-08-18T23:42:23.319578+0000","last_clean":"2024-08-18T23:42:23.319578+0000","last_became_active":"2024-08-18T23:41:30.120266+0000","last_became_peered":"2024-08-18T23:41:30.120266+0000","last_unstale":"2024-08-18T23:42:23.319578+0000","last_undegraded":"2024-08-18T23:42:23.319578+0000","last_fullsized":"2024-08-18T23:42:23.319578+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:06:40.997070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557745+0000","last_change":"2024-08-18T23:41:32.143232+0000","last_active":"2024-08-18T23:42:23.557745+0000","last_peered":"2024-08-18T23:42:23.557745+0000","last_clean":"2024-08-18T23:42:23.557745+0000","last_became_active":"2024-08-18T23:41:32.142916+0000","last_became_peered":"2024-08-18T23:41:32.142916+0000","last_unstale":"2024-08-18T23:42:23.557745+0000","last_undegraded":"2024-08-18T23:42:23.557745+0000","last_fullsized":"2024-08-18T23:42:23.557745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:50:54.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320593+0000","last_change":"2024-08-18T23:41:39.188772+0000","last_active":"2024-08-18T23:42:23.320593+0000","last_peered":"2024-08-18T23:42:23.320593+0000","last_clean":"2024-08-18T23:42:23.320593+0000","last_became_active":"2024-08-18T23:41:39.188362+0000","last_became_peered":"2024-08-18T23:41:39.188362+0000","last_unstale":"2024-08-18T23:42:23.320593+0000","last_undegraded":"2024-08-18T23:42:23.320593+0000","last_fullsized":"2024-08-18T23:42:23.320593+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:47:50.255543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557888+0000","last_change":"2024-08-18T23:41:30.137179+0000","last_active":"2024-08-18T23:42:23.557888+0000","last_peered":"2024-08-18T23:42:23.557888+0000","last_clean":"2024-08-18T23:42:23.557888+0000","last_became_active":"2024-08-18T23:41:30.136618+0000","last_became_peered":"2024-08-18T23:41:30.136618+0000","last_unstale":"2024-08-18T23:42:23.557888+0000","last_undegraded":"2024-08-18T23:42:23.557888+0000","last_fullsized":"2024-08-18T23:42:23.557888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:59.826364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316847+0000","last_change":"2024-08-18T23:41:32.142075+0000","last_active":"2024-08-18T23:42:23.316847+0000","last_peered":"2024-08-18T23:42:23.316847+0000","last_clean":"2024-08-18T23:42:23.316847+0000","last_became_active":"2024-08-18T23:41:32.141709+0000","last_became_peered":"2024-08-18T23:41:32.141709+0000","last_unstale":"2024-08-18T23:42:23.316847+0000","last_undegraded":"2024-08-18T23:42:23.316847+0000","last_fullsized":"2024-08-18T23:42:23.316847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:08.324627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317948+0000","last_change":"2024-08-18T23:41:39.184575+0000","last_active":"2024-08-18T23:42:23.317948+0000","last_peered":"2024-08-18T23:42:23.317948+0000","last_clean":"2024-08-18T23:42:23.317948+0000","last_became_active":"2024-08-18T23:41:39.184169+0000","last_became_peered":"2024-08-18T23:41:39.184169+0000","last_unstale":"2024-08-18T23:42:23.317948+0000","last_undegraded":"2024-08-18T23:42:23.317948+0000","last_fullsized":"2024-08-18T23:42:23.317948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:32.218172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324756+0000","last_change":"2024-08-18T23:41:30.121638+0000","last_active":"2024-08-18T23:42:23.324756+0000","last_peered":"2024-08-18T23:42:23.324756+0000","last_clean":"2024-08-18T23:42:23.324756+0000","last_became_active":"2024-08-18T23:41:30.121290+0000","last_became_peered":"2024-08-18T23:41:30.121290+0000","last_unstale":"2024-08-18T23:42:23.324756+0000","last_undegraded":"2024-08-18T23:42:23.324756+0000","last_fullsized":"2024-08-18T23:42:23.324756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:43:55.638780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315534+0000","last_change":"2024-08-18T23:41:32.142177+0000","last_active":"2024-08-18T23:42:23.315534+0000","last_peered":"2024-08-18T23:42:23.315534+0000","last_clean":"2024-08-18T23:42:23.315534+0000","last_became_active":"2024-08-18T23:41:32.141731+0000","last_became_peered":"2024-08-18T23:41:32.141731+0000","last_unstale":"2024-08-18T23:42:23.315534+0000","last_undegraded":"2024-08-18T23:42:23.315534+0000","last_fullsized":"2024-08-18T23:42:23.315534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:36.283299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318344+0000","last_change":"2024-08-18T23:41:39.182931+0000","last_active":"2024-08-18T23:42:23.318344+0000","last_peered":"2024-08-18T23:42:23.318344+0000","last_clean":"2024-08-18T23:42:23.318344+0000","last_became_active":"2024-08-18T23:41:39.182008+0000","last_became_peered":"2024-08-18T23:41:39.182008+0000","last_unstale":"2024-08-18T23:42:23.318344+0000","last_undegraded":"2024-08-18T23:42:23.318344+0000","last_fullsized":"2024-08-18T23:42:23.318344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:58:38.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.329762+0000","last_change":"2024-08-18T23:42:23.329762+0000","last_active":"2024-08-18T23:42:23.329762+0000","last_peered":"2024-08-18T23:42:23.329762+0000","last_clean":"2024-08-18T23:42:23.329762+0000","last_became_active":"2024-08-18T23:42:23.325753+0000","last_became_peered":"2024-08-18T23:42:23.325753+0000","last_unstale":"2024-08-18T23:42:23.329762+0000","last_undegraded":"2024-08-18T23:42:23.329762+0000","last_fullsized":"2024-08-18T23:42:23.329762+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:19:43.062492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323256+0000","last_change":"2024-08-18T23:41:32.148662+0000","last_active":"2024-08-18T23:42:23.323256+0000","last_peered":"2024-08-18T23:42:23.323256+0000","last_clean":"2024-08-18T23:42:23.323256+0000","last_became_active":"2024-08-18T23:41:32.148318+0000","last_became_peered":"2024-08-18T23:41:32.148318+0000","last_unstale":"2024-08-18T23:42:23.323256+0000","last_undegraded":"2024-08-18T23:42:23.323256+0000","last_fullsized":"2024-08-18T23:42:23.323256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:58.668669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318492+0000","last_change":"2024-08-18T23:41:39.181114+0000","last_active":"2024-08-18T23:42:23.318492+0000","last_peered":"2024-08-18T23:42:23.318492+0000","last_clean":"2024-08-18T23:42:23.318492+0000","last_became_active":"2024-08-18T23:41:39.180599+0000","last_became_peered":"2024-08-18T23:41:39.180599+0000","last_unstale":"2024-08-18T23:42:23.318492+0000","last_undegraded":"2024-08-18T23:42:23.318492+0000","last_fullsized":"2024-08-18T23:42:23.318492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:36:08.749143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316660+0000","last_change":"2024-08-18T23:41:32.144326+0000","last_active":"2024-08-18T23:42:23.316660+0000","last_peered":"2024-08-18T23:42:23.316660+0000","last_clean":"2024-08-18T23:42:23.316660+0000","last_became_active":"2024-08-18T23:41:32.144004+0000","last_became_peered":"2024-08-18T23:41:32.144004+0000","last_unstale":"2024-08-18T23:42:23.316660+0000","last_undegraded":"2024-08-18T23:42:23.316660+0000","last_fullsized":"2024-08-18T23:42:23.316660+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:10.918624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324188+0000","last_change":"2024-08-18T23:41:30.125160+0000","last_active":"2024-08-18T23:42:23.324188+0000","last_peered":"2024-08-18T23:42:23.324188+0000","last_clean":"2024-08-18T23:42:23.324188+0000","last_became_active":"2024-08-18T23:41:30.124685+0000","last_became_peered":"2024-08-18T23:41:30.124685+0000","last_unstale":"2024-08-18T23:42:23.324188+0000","last_undegraded":"2024-08-18T23:42:23.324188+0000","last_fullsized":"2024-08-18T23:42:23.324188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:52:48.997902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318342+0000","last_change":"2024-08-18T23:41:39.681481+0000","last_active":"2024-08-18T23:42:23.318342+0000","last_peered":"2024-08-18T23:42:23.318342+0000","last_clean":"2024-08-18T23:42:23.318342+0000","last_became_active":"2024-08-18T23:41:39.680204+0000","last_became_peered":"2024-08-18T23:41:39.680204+0000","last_unstale":"2024-08-18T23:42:23.318342+0000","last_undegraded":"2024-08-18T23:42:23.318342+0000","last_fullsized":"2024-08-18T23:42:23.318342+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:00:05.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557591+0000","last_change":"2024-08-18T23:41:32.151335+0000","last_active":"2024-08-18T23:42:23.557591+0000","last_peered":"2024-08-18T23:42:23.557591+0000","last_clean":"2024-08-18T23:42:23.557591+0000","last_became_active":"2024-08-18T23:41:32.151040+0000","last_became_peered":"2024-08-18T23:41:32.151040+0000","last_unstale":"2024-08-18T23:42:23.557591+0000","last_undegraded":"2024-08-18T23:42:23.557591+0000","last_fullsized":"2024-08-18T23:42:23.557591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:16.108555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.321637+0000","last_change":"2024-08-18T23:41:30.122301+0000","last_active":"2024-08-18T23:42:23.321637+0000","last_peered":"2024-08-18T23:42:23.321637+0000","last_clean":"2024-08-18T23:42:23.321637+0000","last_became_active":"2024-08-18T23:41:30.121117+0000","last_became_peered":"2024-08-18T23:41:30.121117+0000","last_unstale":"2024-08-18T23:42:23.321637+0000","last_undegraded":"2024-08-18T23:42:23.321637+0000","last_fullsized":"2024-08-18T23:42:23.321637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:40:12.678803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318961+0000","last_change":"2024-08-18T23:41:39.182395+0000","last_active":"2024-08-18T23:42:23.318961+0000","last_peered":"2024-08-18T23:42:23.318961+0000","last_clean":"2024-08-18T23:42:23.318961+0000","last_became_active":"2024-08-18T23:41:39.181979+0000","last_became_peered":"2024-08-18T23:41:39.181979+0000","last_unstale":"2024-08-18T23:42:23.318961+0000","last_undegraded":"2024-08-18T23:42:23.318961+0000","last_fullsized":"2024-08-18T23:42:23.318961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:04.674503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318695+0000","last_change":"2024-08-18T23:41:32.140464+0000","last_active":"2024-08-18T23:42:23.318695+0000","last_peered":"2024-08-18T23:42:23.318695+0000","last_clean":"2024-08-18T23:42:23.318695+0000","last_became_active":"2024-08-18T23:41:32.139153+0000","last_became_peered":"2024-08-18T23:41:32.139153+0000","last_unstale":"2024-08-18T23:42:23.318695+0000","last_undegraded":"2024-08-18T23:42:23.318695+0000","last_fullsized":"2024-08-18T23:42:23.318695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:27.387680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557664+0000","last_change":"2024-08-18T23:41:30.122787+0000","last_active":"2024-08-18T23:42:23.557664+0000","last_peered":"2024-08-18T23:42:23.557664+0000","last_clean":"2024-08-18T23:42:23.557664+0000","last_became_active":"2024-08-18T23:41:30.122462+0000","last_became_peered":"2024-08-18T23:41:30.122462+0000","last_unstale":"2024-08-18T23:42:23.557664+0000","last_undegraded":"2024-08-18T23:42:23.557664+0000","last_fullsized":"2024-08-18T23:42:23.557664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:55.568831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319322+0000","last_change":"2024-08-18T23:41:39.190511+0000","last_active":"2024-08-18T23:42:23.319322+0000","last_peered":"2024-08-18T23:42:23.319322+0000","last_clean":"2024-08-18T23:42:23.319322+0000","last_became_active":"2024-08-18T23:41:39.190154+0000","last_became_peered":"2024-08-18T23:41:39.190154+0000","last_unstale":"2024-08-18T23:42:23.319322+0000","last_undegraded":"2024-08-18T23:42:23.319322+0000","last_fullsized":"2024-08-18T23:42:23.319322+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:33:34.514827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323413+0000","last_change":"2024-08-18T23:41:32.143867+0000","last_active":"2024-08-18T23:42:23.323413+0000","last_peered":"2024-08-18T23:42:23.323413+0000","last_clean":"2024-08-18T23:42:23.323413+0000","last_became_active":"2024-08-18T23:41:32.143527+0000","last_became_peered":"2024-08-18T23:41:32.143527+0000","last_unstale":"2024-08-18T23:42:23.323413+0000","last_undegraded":"2024-08-18T23:42:23.323413+0000","last_fullsized":"2024-08-18T23:42:23.323413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:40:25.902133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.325102+0000","last_change":"2024-08-18T23:42:23.325102+0000","last_active":"2024-08-18T23:42:23.325102+0000","last_peered":"2024-08-18T23:42:23.325102+0000","last_clean":"2024-08-18T23:42:23.325102+0000","last_became_active":"2024-08-18T23:42:23.321583+0000","last_became_peered":"2024-08-18T23:42:23.321583+0000","last_unstale":"2024-08-18T23:42:23.325102+0000","last_undegraded":"2024-08-18T23:42:23.325102+0000","last_fullsized":"2024-08-18T23:42:23.325102+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:57:42.546686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.559756+0000","last_change":"2024-08-18T23:42:23.559756+0000","last_active":"2024-08-18T23:42:23.559756+0000","last_peered":"2024-08-18T23:42:23.559756+0000","last_clean":"2024-08-18T23:42:23.559756+0000","last_became_active":"2024-08-18T23:42:23.559242+0000","last_became_peered":"2024-08-18T23:42:23.559242+0000","last_unstale":"2024-08-18T23:42:23.559756+0000","last_undegraded":"2024-08-18T23:42:23.559756+0000","last_fullsized":"2024-08-18T23:42:23.559756+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:25:40.790313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316766+0000","last_change":"2024-08-18T23:41:32.139025+0000","last_active":"2024-08-18T23:42:23.316766+0000","last_peered":"2024-08-18T23:42:23.316766+0000","last_clean":"2024-08-18T23:42:23.316766+0000","last_became_active":"2024-08-18T23:41:32.138733+0000","last_became_peered":"2024-08-18T23:41:32.138733+0000","last_unstale":"2024-08-18T23:42:23.316766+0000","last_undegraded":"2024-08-18T23:42:23.316766+0000","last_fullsized":"2024-08-18T23:42:23.316766+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:01:39.826391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319270+0000","last_change":"2024-08-18T23:41:30.477359+0000","last_active":"2024-08-18T23:42:23.319270+0000","last_peered":"2024-08-18T23:42:23.319270+0000","last_clean":"2024-08-18T23:42:23.319270+0000","last_became_active":"2024-08-18T23:41:30.477011+0000","last_became_peered":"2024-08-18T23:41:30.477011+0000","last_unstale":"2024-08-18T23:42:23.319270+0000","last_undegraded":"2024-08-18T23:42:23.319270+0000","last_fullsized":"2024-08-18T23:42:23.319270+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:53:10.052859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320716+0000","last_change":"2024-08-18T23:41:39.182140+0000","last_active":"2024-08-18T23:42:23.320716+0000","last_peered":"2024-08-18T23:42:23.320716+0000","last_clean":"2024-08-18T23:42:23.320716+0000","last_became_active":"2024-08-18T23:41:39.181667+0000","last_became_peered":"2024-08-18T23:41:39.181667+0000","last_unstale":"2024-08-18T23:42:23.320716+0000","last_undegraded":"2024-08-18T23:42:23.320716+0000","last_fullsized":"2024-08-18T23:42:23.320716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:02:18.128028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557897+0000","last_change":"2024-08-18T23:41:32.137064+0000","last_active":"2024-08-18T23:42:23.557897+0000","last_peered":"2024-08-18T23:42:23.557897+0000","last_clean":"2024-08-18T23:42:23.557897+0000","last_became_active":"2024-08-18T23:41:32.136738+0000","last_became_peered":"2024-08-18T23:41:32.136738+0000","last_unstale":"2024-08-18T23:42:23.557897+0000","last_undegraded":"2024-08-18T23:42:23.557897+0000","last_fullsized":"2024-08-18T23:42:23.557897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:15:11.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319512+0000","last_change":"2024-08-18T23:41:30.120042+0000","last_active":"2024-08-18T23:42:23.319512+0000","last_peered":"2024-08-18T23:42:23.319512+0000","last_clean":"2024-08-18T23:42:23.319512+0000","last_became_active":"2024-08-18T23:41:30.119719+0000","last_became_peered":"2024-08-18T23:41:30.119719+0000","last_unstale":"2024-08-18T23:42:23.319512+0000","last_undegraded":"2024-08-18T23:42:23.319512+0000","last_fullsized":"2024-08-18T23:42:23.319512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:35:44.466330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557353+0000","last_change":"2024-08-18T23:41:39.680405+0000","last_active":"2024-08-18T23:42:23.557353+0000","last_peered":"2024-08-18T23:42:23.557353+0000","last_clean":"2024-08-18T23:42:23.557353+0000","last_became_active":"2024-08-18T23:41:39.679940+0000","last_became_peered":"2024-08-18T23:41:39.679940+0000","last_unstale":"2024-08-18T23:42:23.557353+0000","last_undegraded":"2024-08-18T23:42:23.557353+0000","last_fullsized":"2024-08-18T23:42:23.557353+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:15:03.942116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314555+0000","last_change":"2024-08-18T23:41:32.139688+0000","last_active":"2024-08-18T23:42:23.314555+0000","last_peered":"2024-08-18T23:42:23.314555+0000","last_clean":"2024-08-18T23:42:23.314555+0000","last_became_active":"2024-08-18T23:41:32.139397+0000","last_became_peered":"2024-08-18T23:41:32.139397+0000","last_unstale":"2024-08-18T23:42:23.314555+0000","last_undegraded":"2024-08-18T23:42:23.314555+0000","last_fullsized":"2024-08-18T23:42:23.314555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:03:57.017488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324876+0000","last_change":"2024-08-18T23:41:30.137578+0000","last_active":"2024-08-18T23:42:23.324876+0000","last_peered":"2024-08-18T23:42:23.324876+0000","last_clean":"2024-08-18T23:42:23.324876+0000","last_became_active":"2024-08-18T23:41:30.137246+0000","last_became_peered":"2024-08-18T23:41:30.137246+0000","last_unstale":"2024-08-18T23:42:23.324876+0000","last_undegraded":"2024-08-18T23:42:23.324876+0000","last_fullsized":"2024-08-18T23:42:23.324876+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:48:54.839432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318001+0000","last_change":"2024-08-18T23:41:39.186360+0000","last_active":"2024-08-18T23:42:23.318001+0000","last_peered":"2024-08-18T23:42:23.318001+0000","last_clean":"2024-08-18T23:42:23.318001+0000","last_became_active":"2024-08-18T23:41:39.185808+0000","last_became_peered":"2024-08-18T23:41:39.185808+0000","last_unstale":"2024-08-18T23:42:23.318001+0000","last_undegraded":"2024-08-18T23:42:23.318001+0000","last_fullsized":"2024-08-18T23:42:23.318001+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:53.891061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558281+0000","last_change":"2024-08-18T23:41:32.146070+0000","last_active":"2024-08-18T23:42:23.558281+0000","last_peered":"2024-08-18T23:42:23.558281+0000","last_clean":"2024-08-18T23:42:23.558281+0000","last_became_active":"2024-08-18T23:41:32.145640+0000","last_became_peered":"2024-08-18T23:41:32.145640+0000","last_unstale":"2024-08-18T23:42:23.558281+0000","last_undegraded":"2024-08-18T23:42:23.558281+0000","last_fullsized":"2024-08-18T23:42:23.558281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:56:21.368762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319576+0000","last_change":"2024-08-18T23:41:30.138565+0000","last_active":"2024-08-18T23:42:23.319576+0000","last_peered":"2024-08-18T23:42:23.319576+0000","last_clean":"2024-08-18T23:42:23.319576+0000","last_became_active":"2024-08-18T23:41:30.138210+0000","last_became_peered":"2024-08-18T23:41:30.138210+0000","last_unstale":"2024-08-18T23:42:23.319576+0000","last_undegraded":"2024-08-18T23:42:23.319576+0000","last_fullsized":"2024-08-18T23:42:23.319576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:42:14.928884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323149+0000","last_change":"2024-08-18T23:41:39.680656+0000","last_active":"2024-08-18T23:42:23.323149+0000","last_peered":"2024-08-18T23:42:23.323149+0000","last_clean":"2024-08-18T23:42:23.323149+0000","last_became_active":"2024-08-18T23:41:39.680203+0000","last_became_peered":"2024-08-18T23:41:39.680203+0000","last_unstale":"2024-08-18T23:42:23.323149+0000","last_undegraded":"2024-08-18T23:42:23.323149+0000","last_fullsized":"2024-08-18T23:42:23.323149+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:55:43.138869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558246+0000","last_change":"2024-08-18T23:41:32.130222+0000","last_active":"2024-08-18T23:42:23.558246+0000","last_peered":"2024-08-18T23:42:23.558246+0000","last_clean":"2024-08-18T23:42:23.558246+0000","last_became_active":"2024-08-18T23:41:32.129844+0000","last_became_peered":"2024-08-18T23:41:32.129844+0000","last_unstale":"2024-08-18T23:42:23.558246+0000","last_undegraded":"2024-08-18T23:42:23.558246+0000","last_fullsized":"2024-08-18T23:42:23.558246+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:39:21.106343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316823+0000","last_change":"2024-08-18T23:41:30.123142+0000","last_active":"2024-08-18T23:42:23.316823+0000","last_peered":"2024-08-18T23:42:23.316823+0000","last_clean":"2024-08-18T23:42:23.316823+0000","last_became_active":"2024-08-18T23:41:30.121645+0000","last_became_peered":"2024-08-18T23:41:30.121645+0000","last_unstale":"2024-08-18T23:42:23.316823+0000","last_undegraded":"2024-08-18T23:42:23.316823+0000","last_fullsized":"2024-08-18T23:42:23.316823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:07:24.912259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314454+0000","last_change":"2024-08-18T23:41:39.186033+0000","last_active":"2024-08-18T23:42:23.314454+0000","last_peered":"2024-08-18T23:42:23.314454+0000","last_clean":"2024-08-18T23:42:23.314454+0000","last_became_active":"2024-08-18T23:41:39.185451+0000","last_became_peered":"2024-08-18T23:41:39.185451+0000","last_unstale":"2024-08-18T23:42:23.314454+0000","last_undegraded":"2024-08-18T23:42:23.314454+0000","last_fullsized":"2024-08-18T23:42:23.314454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:52:13.556760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320803+0000","last_change":"2024-08-18T23:41:32.141065+0000","last_active":"2024-08-18T23:42:23.320803+0000","last_peered":"2024-08-18T23:42:23.320803+0000","last_clean":"2024-08-18T23:42:23.320803+0000","last_became_active":"2024-08-18T23:41:32.140732+0000","last_became_peered":"2024-08-18T23:41:32.140732+0000","last_unstale":"2024-08-18T23:42:23.320803+0000","last_undegraded":"2024-08-18T23:42:23.320803+0000","last_fullsized":"2024-08-18T23:42:23.320803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:08:32.287835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:27.300919+0000","last_change":"2024-08-18T23:41:30.136838+0000","last_active":"2024-08-18T23:42:27.300919+0000","last_peered":"2024-08-18T23:42:27.300919+0000","last_clean":"2024-08-18T23:42:27.300919+0000","last_became_active":"2024-08-18T23:41:30.136523+0000","last_became_peered":"2024-08-18T23:41:30.136523+0000","last_unstale":"2024-08-18T23:42:27.300919+0000","last_undegraded":"2024-08-18T23:42:27.300919+0000","last_fullsized":"2024-08-18T23:42:27.300919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:56:17.750535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314559+0000","last_change":"2024-08-18T23:41:39.680314+0000","last_active":"2024-08-18T23:42:23.314559+0000","last_peered":"2024-08-18T23:42:23.314559+0000","last_clean":"2024-08-18T23:42:23.314559+0000","last_became_active":"2024-08-18T23:41:39.679920+0000","last_became_peered":"2024-08-18T23:41:39.679920+0000","last_unstale":"2024-08-18T23:42:23.314559+0000","last_undegraded":"2024-08-18T23:42:23.314559+0000","last_fullsized":"2024-08-18T23:42:23.314559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:26:29.866081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314066+0000","last_change":"2024-08-18T23:41:32.139752+0000","last_active":"2024-08-18T23:42:23.314066+0000","last_peered":"2024-08-18T23:42:23.314066+0000","last_clean":"2024-08-18T23:42:23.314066+0000","last_became_active":"2024-08-18T23:41:32.139451+0000","last_became_peered":"2024-08-18T23:41:32.139451+0000","last_unstale":"2024-08-18T23:42:23.314066+0000","last_undegraded":"2024-08-18T23:42:23.314066+0000","last_fullsized":"2024-08-18T23:42:23.314066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:15:13.390529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316737+0000","last_change":"2024-08-18T23:41:30.126513+0000","last_active":"2024-08-18T23:42:23.316737+0000","last_peered":"2024-08-18T23:42:23.316737+0000","last_clean":"2024-08-18T23:42:23.316737+0000","last_became_active":"2024-08-18T23:41:30.126046+0000","last_became_peered":"2024-08-18T23:41:30.126046+0000","last_unstale":"2024-08-18T23:42:23.316737+0000","last_undegraded":"2024-08-18T23:42:23.316737+0000","last_fullsized":"2024-08-18T23:42:23.316737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:49:16.565565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323372+0000","last_change":"2024-08-18T23:41:39.681006+0000","last_active":"2024-08-18T23:42:23.323372+0000","last_peered":"2024-08-18T23:42:23.323372+0000","last_clean":"2024-08-18T23:42:23.323372+0000","last_became_active":"2024-08-18T23:41:39.680409+0000","last_became_peered":"2024-08-18T23:41:39.680409+0000","last_unstale":"2024-08-18T23:42:23.323372+0000","last_undegraded":"2024-08-18T23:42:23.323372+0000","last_fullsized":"2024-08-18T23:42:23.323372+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:47.778866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557596+0000","last_change":"2024-08-18T23:41:32.147545+0000","last_active":"2024-08-18T23:42:23.557596+0000","last_peered":"2024-08-18T23:42:23.557596+0000","last_clean":"2024-08-18T23:42:23.557596+0000","last_became_active":"2024-08-18T23:41:32.147238+0000","last_became_peered":"2024-08-18T23:41:32.147238+0000","last_unstale":"2024-08-18T23:42:23.557596+0000","last_undegraded":"2024-08-18T23:42:23.557596+0000","last_fullsized":"2024-08-18T23:42:23.557596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:12:13.589535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319674+0000","last_change":"2024-08-18T23:41:30.120650+0000","last_active":"2024-08-18T23:42:23.319674+0000","last_peered":"2024-08-18T23:42:23.319674+0000","last_clean":"2024-08-18T23:42:23.319674+0000","last_became_active":"2024-08-18T23:41:30.120196+0000","last_became_peered":"2024-08-18T23:41:30.120196+0000","last_unstale":"2024-08-18T23:42:23.319674+0000","last_undegraded":"2024-08-18T23:42:23.319674+0000","last_fullsized":"2024-08-18T23:42:23.319674+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:40:23.370088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317120+0000","last_change":"2024-08-18T23:40:22.013519+0000","last_active":"2024-08-18T23:42:23.317120+0000","last_peered":"2024-08-18T23:42:23.317120+0000","last_clean":"2024-08-18T23:42:23.317120+0000","last_became_active":"2024-08-18T23:40:22.012986+0000","last_became_peered":"2024-08-18T23:40:22.012986+0000","last_unstale":"2024-08-18T23:42:23.317120+0000","last_undegraded":"2024-08-18T23:42:23.317120+0000","last_fullsized":"2024-08-18T23:42:23.317120+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_clean_scrub_stamp":"2024-08-18T23:37:22.388424+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:19.874356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.332800+0000","last_change":"2024-08-18T23:42:23.332800+0000","last_active":"2024-08-18T23:42:23.332800+0000","last_peered":"2024-08-18T23:42:23.332800+0000","last_clean":"2024-08-18T23:42:23.332800+0000","last_became_active":"2024-08-18T23:42:23.326441+0000","last_became_peered":"2024-08-18T23:42:23.326441+0000","last_unstale":"2024-08-18T23:42:23.332800+0000","last_undegraded":"2024-08-18T23:42:23.332800+0000","last_fullsized":"2024-08-18T23:42:23.332800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:13:58.243518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317150+0000","last_change":"2024-08-18T23:41:32.139907+0000","last_active":"2024-08-18T23:42:23.317150+0000","last_peered":"2024-08-18T23:42:23.317150+0000","last_clean":"2024-08-18T23:42:23.317150+0000","last_became_active":"2024-08-18T23:41:32.137956+0000","last_became_peered":"2024-08-18T23:41:32.137956+0000","last_unstale":"2024-08-18T23:42:23.317150+0000","last_undegraded":"2024-08-18T23:42:23.317150+0000","last_fullsized":"2024-08-18T23:42:23.317150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:10:13.223328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.330746+0000","last_change":"2024-08-18T23:42:23.330062+0000","last_active":"2024-08-18T23:42:23.330746+0000","last_peered":"2024-08-18T23:42:23.330746+0000","last_clean":"2024-08-18T23:42:23.330746+0000","last_became_active":"2024-08-18T23:42:23.323196+0000","last_became_peered":"2024-08-18T23:42:23.323196+0000","last_unstale":"2024-08-18T23:42:23.330746+0000","last_undegraded":"2024-08-18T23:42:23.330746+0000","last_fullsized":"2024-08-18T23:42:23.330746+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:53:29.059542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315093+0000","last_change":"2024-08-18T23:41:39.185917+0000","last_active":"2024-08-18T23:42:23.315093+0000","last_peered":"2024-08-18T23:42:23.315093+0000","last_clean":"2024-08-18T23:42:23.315093+0000","last_became_active":"2024-08-18T23:41:39.185427+0000","last_became_peered":"2024-08-18T23:41:39.185427+0000","last_unstale":"2024-08-18T23:42:23.315093+0000","last_undegraded":"2024-08-18T23:42:23.315093+0000","last_fullsized":"2024-08-18T23:42:23.315093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:52:13.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317336+0000","last_change":"2024-08-18T23:41:32.147530+0000","last_active":"2024-08-18T23:42:23.317336+0000","last_peered":"2024-08-18T23:42:23.317336+0000","last_clean":"2024-08-18T23:42:23.317336+0000","last_became_active":"2024-08-18T23:41:32.147216+0000","last_became_peered":"2024-08-18T23:41:32.147216+0000","last_unstale":"2024-08-18T23:42:23.317336+0000","last_undegraded":"2024-08-18T23:42:23.317336+0000","last_fullsized":"2024-08-18T23:42:23.317336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:04:06.421490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323538+0000","last_change":"2024-08-18T23:41:30.121307+0000","last_active":"2024-08-18T23:42:23.323538+0000","last_peered":"2024-08-18T23:42:23.323538+0000","last_clean":"2024-08-18T23:42:23.323538+0000","last_became_active":"2024-08-18T23:41:30.120932+0000","last_became_peered":"2024-08-18T23:41:30.120932+0000","last_unstale":"2024-08-18T23:42:23.323538+0000","last_undegraded":"2024-08-18T23:42:23.323538+0000","last_fullsized":"2024-08-18T23:42:23.323538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:06:07.742880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316371+0000","last_change":"2024-08-18T23:41:39.184647+0000","last_active":"2024-08-18T23:42:23.316371+0000","last_peered":"2024-08-18T23:42:23.316371+0000","last_clean":"2024-08-18T23:42:23.316371+0000","last_became_active":"2024-08-18T23:41:39.184251+0000","last_became_peered":"2024-08-18T23:41:39.184251+0000","last_unstale":"2024-08-18T23:42:23.316371+0000","last_undegraded":"2024-08-18T23:42:23.316371+0000","last_fullsized":"2024-08-18T23:42:23.316371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:55.200801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317327+0000","last_change":"2024-08-18T23:41:32.137007+0000","last_active":"2024-08-18T23:42:23.317327+0000","last_peered":"2024-08-18T23:42:23.317327+0000","last_clean":"2024-08-18T23:42:23.317327+0000","last_became_active":"2024-08-18T23:41:32.136684+0000","last_became_peered":"2024-08-18T23:41:32.136684+0000","last_unstale":"2024-08-18T23:42:23.317327+0000","last_undegraded":"2024-08-18T23:42:23.317327+0000","last_fullsized":"2024-08-18T23:42:23.317327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:21:26.584303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314514+0000","last_change":"2024-08-18T23:41:30.121183+0000","last_active":"2024-08-18T23:42:23.314514+0000","last_peered":"2024-08-18T23:42:23.314514+0000","last_clean":"2024-08-18T23:42:23.314514+0000","last_became_active":"2024-08-18T23:41:30.120847+0000","last_became_peered":"2024-08-18T23:41:30.120847+0000","last_unstale":"2024-08-18T23:42:23.314514+0000","last_undegraded":"2024-08-18T23:42:23.314514+0000","last_fullsized":"2024-08-18T23:42:23.314514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:51.025821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320528+0000","last_change":"2024-08-18T23:41:39.183375+0000","last_active":"2024-08-18T23:42:23.320528+0000","last_peered":"2024-08-18T23:42:23.320528+0000","last_clean":"2024-08-18T23:42:23.320528+0000","last_became_active":"2024-08-18T23:41:39.182339+0000","last_became_peered":"2024-08-18T23:41:39.182339+0000","last_unstale":"2024-08-18T23:42:23.320528+0000","last_undegraded":"2024-08-18T23:42:23.320528+0000","last_fullsized":"2024-08-18T23:42:23.320528+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:26.119056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318323+0000","last_change":"2024-08-18T23:41:32.140748+0000","last_active":"2024-08-18T23:42:23.318323+0000","last_peered":"2024-08-18T23:42:23.318323+0000","last_clean":"2024-08-18T23:42:23.318323+0000","last_became_active":"2024-08-18T23:41:32.140306+0000","last_became_peered":"2024-08-18T23:41:32.140306+0000","last_unstale":"2024-08-18T23:42:23.318323+0000","last_undegraded":"2024-08-18T23:42:23.318323+0000","last_fullsized":"2024-08-18T23:42:23.318323+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:23:02.371529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324632+0000","last_change":"2024-08-18T23:41:30.121617+0000","last_active":"2024-08-18T23:42:23.324632+0000","last_peered":"2024-08-18T23:42:23.324632+0000","last_clean":"2024-08-18T23:42:23.324632+0000","last_became_active":"2024-08-18T23:41:30.121257+0000","last_became_peered":"2024-08-18T23:41:30.121257+0000","last_unstale":"2024-08-18T23:42:23.324632+0000","last_undegraded":"2024-08-18T23:42:23.324632+0000","last_fullsized":"2024-08-18T23:42:23.324632+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:23:42.091553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320637+0000","last_change":"2024-08-18T23:41:39.183833+0000","last_active":"2024-08-18T23:42:23.320637+0000","last_peered":"2024-08-18T23:42:23.320637+0000","last_clean":"2024-08-18T23:42:23.320637+0000","last_became_active":"2024-08-18T23:41:39.183176+0000","last_became_peered":"2024-08-18T23:41:39.183176+0000","last_unstale":"2024-08-18T23:42:23.320637+0000","last_undegraded":"2024-08-18T23:42:23.320637+0000","last_fullsized":"2024-08-18T23:42:23.320637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:36:09.827378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322357+0000","last_change":"2024-08-18T23:41:32.147507+0000","last_active":"2024-08-18T23:42:23.322357+0000","last_peered":"2024-08-18T23:42:23.322357+0000","last_clean":"2024-08-18T23:42:23.322357+0000","last_became_active":"2024-08-18T23:41:32.146042+0000","last_became_peered":"2024-08-18T23:41:32.146042+0000","last_unstale":"2024-08-18T23:42:23.322357+0000","last_undegraded":"2024-08-18T23:42:23.322357+0000","last_fullsized":"2024-08-18T23:42:23.322357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:48.941442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323009+0000","last_change":"2024-08-18T23:41:30.137755+0000","last_active":"2024-08-18T23:42:23.323009+0000","last_peered":"2024-08-18T23:42:23.323009+0000","last_clean":"2024-08-18T23:42:23.323009+0000","last_became_active":"2024-08-18T23:41:30.137194+0000","last_became_peered":"2024-08-18T23:41:30.137194+0000","last_unstale":"2024-08-18T23:42:23.323009+0000","last_undegraded":"2024-08-18T23:42:23.323009+0000","last_fullsized":"2024-08-18T23:42:23.323009+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:15:49.681405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314506+0000","last_change":"2024-08-18T23:41:39.183986+0000","last_active":"2024-08-18T23:42:23.314506+0000","last_peered":"2024-08-18T23:42:23.314506+0000","last_clean":"2024-08-18T23:42:23.314506+0000","last_became_active":"2024-08-18T23:41:39.183384+0000","last_became_peered":"2024-08-18T23:41:39.183384+0000","last_unstale":"2024-08-18T23:42:23.314506+0000","last_undegraded":"2024-08-18T23:42:23.314506+0000","last_fullsized":"2024-08-18T23:42:23.314506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:04:28.920804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316556+0000","last_change":"2024-08-18T23:41:32.134729+0000","last_active":"2024-08-18T23:42:23.316556+0000","last_peered":"2024-08-18T23:42:23.316556+0000","last_clean":"2024-08-18T23:42:23.316556+0000","last_became_active":"2024-08-18T23:41:32.134366+0000","last_became_peered":"2024-08-18T23:41:32.134366+0000","last_unstale":"2024-08-18T23:42:23.316556+0000","last_undegraded":"2024-08-18T23:42:23.316556+0000","last_fullsized":"2024-08-18T23:42:23.316556+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:38.610561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:28.302774+0000","last_change":"2024-08-18T23:41:30.126443+0000","last_active":"2024-08-18T23:42:28.302774+0000","last_peered":"2024-08-18T23:42:28.302774+0000","last_clean":"2024-08-18T23:42:28.302774+0000","last_became_active":"2024-08-18T23:41:30.124264+0000","last_became_peered":"2024-08-18T23:41:30.124264+0000","last_unstale":"2024-08-18T23:42:28.302774+0000","last_undegraded":"2024-08-18T23:42:28.302774+0000","last_fullsized":"2024-08-18T23:42:28.302774+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:43.727114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319340+0000","last_change":"2024-08-18T23:41:39.192164+0000","last_active":"2024-08-18T23:42:23.319340+0000","last_peered":"2024-08-18T23:42:23.319340+0000","last_clean":"2024-08-18T23:42:23.319340+0000","last_became_active":"2024-08-18T23:41:39.191833+0000","last_became_peered":"2024-08-18T23:41:39.191833+0000","last_unstale":"2024-08-18T23:42:23.319340+0000","last_undegraded":"2024-08-18T23:42:23.319340+0000","last_fullsized":"2024-08-18T23:42:23.319340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:33:50.243930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320662+0000","last_change":"2024-08-18T23:42:23.320662+0000","last_active":"2024-08-18T23:42:23.320662+0000","last_peered":"2024-08-18T23:42:23.320662+0000","last_clean":"2024-08-18T23:42:23.320662+0000","last_became_active":"2024-08-18T23:42:23.320249+0000","last_became_peered":"2024-08-18T23:42:23.320249+0000","last_unstale":"2024-08-18T23:42:23.320662+0000","last_undegraded":"2024-08-18T23:42:23.320662+0000","last_fullsized":"2024-08-18T23:42:23.320662+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:03:17.121915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322332+0000","last_change":"2024-08-18T23:41:30.124071+0000","last_active":"2024-08-18T23:42:23.322332+0000","last_peered":"2024-08-18T23:42:23.322332+0000","last_clean":"2024-08-18T23:42:23.322332+0000","last_became_active":"2024-08-18T23:41:30.123118+0000","last_became_peered":"2024-08-18T23:41:30.123118+0000","last_unstale":"2024-08-18T23:42:23.322332+0000","last_undegraded":"2024-08-18T23:42:23.322332+0000","last_fullsized":"2024-08-18T23:42:23.322332+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:21:39.015187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319331+0000","last_change":"2024-08-18T23:41:39.184038+0000","last_active":"2024-08-18T23:42:23.319331+0000","last_peered":"2024-08-18T23:42:23.319331+0000","last_clean":"2024-08-18T23:42:23.319331+0000","last_became_active":"2024-08-18T23:41:39.183611+0000","last_became_peered":"2024-08-18T23:41:39.183611+0000","last_unstale":"2024-08-18T23:42:23.319331+0000","last_undegraded":"2024-08-18T23:42:23.319331+0000","last_fullsized":"2024-08-18T23:42:23.319331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:32:26.029151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314264+0000","last_change":"2024-08-18T23:41:32.143631+0000","last_active":"2024-08-18T23:42:23.314264+0000","last_peered":"2024-08-18T23:42:23.314264+0000","last_clean":"2024-08-18T23:42:23.314264+0000","last_became_active":"2024-08-18T23:41:32.143350+0000","last_became_peered":"2024-08-18T23:41:32.143350+0000","last_unstale":"2024-08-18T23:42:23.314264+0000","last_undegraded":"2024-08-18T23:42:23.314264+0000","last_fullsized":"2024-08-18T23:42:23.314264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:41:23.713467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319866+0000","last_change":"2024-08-18T23:41:30.123383+0000","last_active":"2024-08-18T23:42:23.319866+0000","last_peered":"2024-08-18T23:42:23.319866+0000","last_clean":"2024-08-18T23:42:23.319866+0000","last_became_active":"2024-08-18T23:41:30.123037+0000","last_became_peered":"2024-08-18T23:41:30.123037+0000","last_unstale":"2024-08-18T23:42:23.319866+0000","last_undegraded":"2024-08-18T23:42:23.319866+0000","last_fullsized":"2024-08-18T23:42:23.319866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:27.702478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323158+0000","last_change":"2024-08-18T23:41:39.184302+0000","last_active":"2024-08-18T23:42:23.323158+0000","last_peered":"2024-08-18T23:42:23.323158+0000","last_clean":"2024-08-18T23:42:23.323158+0000","last_became_active":"2024-08-18T23:41:39.183172+0000","last_became_peered":"2024-08-18T23:41:39.183172+0000","last_unstale":"2024-08-18T23:42:23.323158+0000","last_undegraded":"2024-08-18T23:42:23.323158+0000","last_fullsized":"2024-08-18T23:42:23.323158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:23:26.655865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324579+0000","last_change":"2024-08-18T23:41:32.137426+0000","last_active":"2024-08-18T23:42:23.324579+0000","last_peered":"2024-08-18T23:42:23.324579+0000","last_clean":"2024-08-18T23:42:23.324579+0000","last_became_active":"2024-08-18T23:41:32.137106+0000","last_became_peered":"2024-08-18T23:41:32.137106+0000","last_unstale":"2024-08-18T23:42:23.324579+0000","last_undegraded":"2024-08-18T23:42:23.324579+0000","last_fullsized":"2024-08-18T23:42:23.324579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:34:51.889663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'31","reported_seq":117,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:29.304375+0000","last_change":"2024-08-18T23:41:30.124320+0000","last_active":"2024-08-18T23:42:29.304375+0000","last_peered":"2024-08-18T23:42:29.304375+0000","last_clean":"2024-08-18T23:42:29.304375+0000","last_became_active":"2024-08-18T23:41:30.123979+0000","last_became_peered":"2024-08-18T23:41:30.123979+0000","last_unstale":"2024-08-18T23:42:29.304375+0000","last_undegraded":"2024-08-18T23:42:29.304375+0000","last_fullsized":"2024-08-18T23:42:29.304375+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:31:03.689720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61008,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":52,"num_read_kb":86,"num_write":32,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319063+0000","last_change":"2024-08-18T23:41:39.190481+0000","last_active":"2024-08-18T23:42:23.319063+0000","last_peered":"2024-08-18T23:42:23.319063+0000","last_clean":"2024-08-18T23:42:23.319063+0000","last_became_active":"2024-08-18T23:41:39.190076+0000","last_became_peered":"2024-08-18T23:41:39.190076+0000","last_unstale":"2024-08-18T23:42:23.319063+0000","last_undegraded":"2024-08-18T23:42:23.319063+0000","last_fullsized":"2024-08-18T23:42:23.319063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:47:39.313142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315607+0000","last_change":"2024-08-18T23:41:32.145422+0000","last_active":"2024-08-18T23:42:23.315607+0000","last_peered":"2024-08-18T23:42:23.315607+0000","last_clean":"2024-08-18T23:42:23.315607+0000","last_became_active":"2024-08-18T23:41:32.145047+0000","last_became_peered":"2024-08-18T23:41:32.145047+0000","last_unstale":"2024-08-18T23:42:23.315607+0000","last_undegraded":"2024-08-18T23:42:23.315607+0000","last_fullsized":"2024-08-18T23:42:23.315607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:31.905248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323427+0000","last_change":"2024-08-18T23:41:30.137364+0000","last_active":"2024-08-18T23:42:23.323427+0000","last_peered":"2024-08-18T23:42:23.323427+0000","last_clean":"2024-08-18T23:42:23.323427+0000","last_became_active":"2024-08-18T23:41:30.137004+0000","last_became_peered":"2024-08-18T23:41:30.137004+0000","last_unstale":"2024-08-18T23:42:23.323427+0000","last_undegraded":"2024-08-18T23:42:23.323427+0000","last_fullsized":"2024-08-18T23:42:23.323427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:20:56.392149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.326359+0000","last_change":"2024-08-18T23:42:23.326359+0000","last_active":"2024-08-18T23:42:23.326359+0000","last_peered":"2024-08-18T23:42:23.326359+0000","last_clean":"2024-08-18T23:42:23.326359+0000","last_became_active":"2024-08-18T23:42:23.325918+0000","last_became_peered":"2024-08-18T23:42:23.325918+0000","last_unstale":"2024-08-18T23:42:23.326359+0000","last_undegraded":"2024-08-18T23:42:23.326359+0000","last_fullsized":"2024-08-18T23:42:23.326359+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:01:15.476268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.325691+0000","last_change":"2024-08-18T23:42:23.325691+0000","last_active":"2024-08-18T23:42:23.325691+0000","last_peered":"2024-08-18T23:42:23.325691+0000","last_clean":"2024-08-18T23:42:23.325691+0000","last_became_active":"2024-08-18T23:42:23.325467+0000","last_became_peered":"2024-08-18T23:42:23.325467+0000","last_unstale":"2024-08-18T23:42:23.325691+0000","last_undegraded":"2024-08-18T23:42:23.325691+0000","last_fullsized":"2024-08-18T23:42:23.325691+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:10:26.288880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317472+0000","last_change":"2024-08-18T23:41:30.124430+0000","last_active":"2024-08-18T23:42:23.317472+0000","last_peered":"2024-08-18T23:42:23.317472+0000","last_clean":"2024-08-18T23:42:23.317472+0000","last_became_active":"2024-08-18T23:41:30.123626+0000","last_became_peered":"2024-08-18T23:41:30.123626+0000","last_unstale":"2024-08-18T23:42:23.317472+0000","last_undegraded":"2024-08-18T23:42:23.317472+0000","last_fullsized":"2024-08-18T23:42:23.317472+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:26:18.426010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318912+0000","last_change":"2024-08-18T23:41:39.192765+0000","last_active":"2024-08-18T23:42:23.318912+0000","last_peered":"2024-08-18T23:42:23.318912+0000","last_clean":"2024-08-18T23:42:23.318912+0000","last_became_active":"2024-08-18T23:41:39.192250+0000","last_became_peered":"2024-08-18T23:41:39.192250+0000","last_unstale":"2024-08-18T23:42:23.318912+0000","last_undegraded":"2024-08-18T23:42:23.318912+0000","last_fullsized":"2024-08-18T23:42:23.318912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:51:16.794404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316158+0000","last_change":"2024-08-18T23:41:32.141966+0000","last_active":"2024-08-18T23:42:23.316158+0000","last_peered":"2024-08-18T23:42:23.316158+0000","last_clean":"2024-08-18T23:42:23.316158+0000","last_became_active":"2024-08-18T23:41:32.141561+0000","last_became_peered":"2024-08-18T23:41:32.141561+0000","last_unstale":"2024-08-18T23:42:23.316158+0000","last_undegraded":"2024-08-18T23:42:23.316158+0000","last_fullsized":"2024-08-18T23:42:23.316158+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:20.696089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:28.302448+0000","last_change":"2024-08-18T23:41:30.121598+0000","last_active":"2024-08-18T23:42:28.302448+0000","last_peered":"2024-08-18T23:42:28.302448+0000","last_clean":"2024-08-18T23:42:28.302448+0000","last_became_active":"2024-08-18T23:41:30.120533+0000","last_became_peered":"2024-08-18T23:41:30.120533+0000","last_unstale":"2024-08-18T23:42:28.302448+0000","last_undegraded":"2024-08-18T23:42:28.302448+0000","last_fullsized":"2024-08-18T23:42:28.302448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:06:37.458272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557461+0000","last_change":"2024-08-18T23:41:39.178877+0000","last_active":"2024-08-18T23:42:23.557461+0000","last_peered":"2024-08-18T23:42:23.557461+0000","last_clean":"2024-08-18T23:42:23.557461+0000","last_became_active":"2024-08-18T23:41:39.178381+0000","last_became_peered":"2024-08-18T23:41:39.178381+0000","last_unstale":"2024-08-18T23:42:23.557461+0000","last_undegraded":"2024-08-18T23:42:23.557461+0000","last_fullsized":"2024-08-18T23:42:23.557461+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:27.275893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316665+0000","last_change":"2024-08-18T23:41:39.188381+0000","last_active":"2024-08-18T23:42:23.316665+0000","last_peered":"2024-08-18T23:42:23.316665+0000","last_clean":"2024-08-18T23:42:23.316665+0000","last_became_active":"2024-08-18T23:41:39.187983+0000","last_became_peered":"2024-08-18T23:41:39.187983+0000","last_unstale":"2024-08-18T23:42:23.316665+0000","last_undegraded":"2024-08-18T23:42:23.316665+0000","last_fullsized":"2024-08-18T23:42:23.316665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:04:45.460576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319101+0000","last_change":"2024-08-18T23:41:32.144637+0000","last_active":"2024-08-18T23:42:23.319101+0000","last_peered":"2024-08-18T23:42:23.319101+0000","last_clean":"2024-08-18T23:42:23.319101+0000","last_became_active":"2024-08-18T23:41:32.144329+0000","last_became_peered":"2024-08-18T23:41:32.144329+0000","last_unstale":"2024-08-18T23:42:23.319101+0000","last_undegraded":"2024-08-18T23:42:23.319101+0000","last_fullsized":"2024-08-18T23:42:23.319101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:29:39.791264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320980+0000","last_change":"2024-08-18T23:41:30.122325+0000","last_active":"2024-08-18T23:42:23.320980+0000","last_peered":"2024-08-18T23:42:23.320980+0000","last_clean":"2024-08-18T23:42:23.320980+0000","last_became_active":"2024-08-18T23:41:30.121480+0000","last_became_peered":"2024-08-18T23:41:30.121480+0000","last_unstale":"2024-08-18T23:42:23.320980+0000","last_undegraded":"2024-08-18T23:42:23.320980+0000","last_fullsized":"2024-08-18T23:42:23.320980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:32:12.810650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.560632+0000","last_change":"2024-08-18T23:42:23.560632+0000","last_active":"2024-08-18T23:42:23.560632+0000","last_peered":"2024-08-18T23:42:23.560632+0000","last_clean":"2024-08-18T23:42:23.560632+0000","last_became_active":"2024-08-18T23:42:23.559920+0000","last_became_peered":"2024-08-18T23:42:23.559920+0000","last_unstale":"2024-08-18T23:42:23.560632+0000","last_undegraded":"2024-08-18T23:42:23.560632+0000","last_fullsized":"2024-08-18T23:42:23.560632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:01:59.580843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557464+0000","last_change":"2024-08-18T23:41:32.145545+0000","last_active":"2024-08-18T23:42:23.557464+0000","last_peered":"2024-08-18T23:42:23.557464+0000","last_clean":"2024-08-18T23:42:23.557464+0000","last_became_active":"2024-08-18T23:41:32.145183+0000","last_became_peered":"2024-08-18T23:41:32.145183+0000","last_unstale":"2024-08-18T23:42:23.557464+0000","last_undegraded":"2024-08-18T23:42:23.557464+0000","last_fullsized":"2024-08-18T23:42:23.557464+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:03:04.111658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314074+0000","last_change":"2024-08-18T23:41:30.118776+0000","last_active":"2024-08-18T23:42:23.314074+0000","last_peered":"2024-08-18T23:42:23.314074+0000","last_clean":"2024-08-18T23:42:23.314074+0000","last_became_active":"2024-08-18T23:41:30.118436+0000","last_became_peered":"2024-08-18T23:41:30.118436+0000","last_unstale":"2024-08-18T23:42:23.314074+0000","last_undegraded":"2024-08-18T23:42:23.314074+0000","last_fullsized":"2024-08-18T23:42:23.314074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:13.980927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558143+0000","last_change":"2024-08-18T23:41:30.125201+0000","last_active":"2024-08-18T23:42:23.558143+0000","last_peered":"2024-08-18T23:42:23.558143+0000","last_clean":"2024-08-18T23:42:23.558143+0000","last_became_active":"2024-08-18T23:41:30.124894+0000","last_became_peered":"2024-08-18T23:41:30.124894+0000","last_unstale":"2024-08-18T23:42:23.558143+0000","last_undegraded":"2024-08-18T23:42:23.558143+0000","last_fullsized":"2024-08-18T23:42:23.558143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:30:17.721829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322555+0000","last_change":"2024-08-18T23:41:32.145452+0000","last_active":"2024-08-18T23:42:23.322555+0000","last_peered":"2024-08-18T23:42:23.322555+0000","last_clean":"2024-08-18T23:42:23.322555+0000","last_became_active":"2024-08-18T23:41:32.145151+0000","last_became_peered":"2024-08-18T23:41:32.145151+0000","last_unstale":"2024-08-18T23:42:23.322555+0000","last_undegraded":"2024-08-18T23:42:23.322555+0000","last_fullsized":"2024-08-18T23:42:23.322555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:08:08.713154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319503+0000","last_change":"2024-08-18T23:41:39.193334+0000","last_active":"2024-08-18T23:42:23.319503+0000","last_peered":"2024-08-18T23:42:23.319503+0000","last_clean":"2024-08-18T23:42:23.319503+0000","last_became_active":"2024-08-18T23:41:39.192924+0000","last_became_peered":"2024-08-18T23:41:39.192924+0000","last_unstale":"2024-08-18T23:42:23.319503+0000","last_undegraded":"2024-08-18T23:42:23.319503+0000","last_fullsized":"2024-08-18T23:42:23.319503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:07.016300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:25.298805+0000","last_change":"2024-08-18T23:41:30.125217+0000","last_active":"2024-08-18T23:42:25.298805+0000","last_peered":"2024-08-18T23:42:25.298805+0000","last_clean":"2024-08-18T23:42:25.298805+0000","last_became_active":"2024-08-18T23:41:30.124889+0000","last_became_peered":"2024-08-18T23:41:30.124889+0000","last_unstale":"2024-08-18T23:42:25.298805+0000","last_undegraded":"2024-08-18T23:42:25.298805+0000","last_fullsized":"2024-08-18T23:42:25.298805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:43.389537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323355+0000","last_change":"2024-08-18T23:41:32.142585+0000","last_active":"2024-08-18T23:42:23.323355+0000","last_peered":"2024-08-18T23:42:23.323355+0000","last_clean":"2024-08-18T23:42:23.323355+0000","last_became_active":"2024-08-18T23:41:32.142207+0000","last_became_peered":"2024-08-18T23:41:32.142207+0000","last_unstale":"2024-08-18T23:42:23.323355+0000","last_undegraded":"2024-08-18T23:42:23.323355+0000","last_fullsized":"2024-08-18T23:42:23.323355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:16:58.755444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.321952+0000","last_change":"2024-08-18T23:41:39.185618+0000","last_active":"2024-08-18T23:42:23.321952+0000","last_peered":"2024-08-18T23:42:23.321952+0000","last_clean":"2024-08-18T23:42:23.321952+0000","last_became_active":"2024-08-18T23:41:39.185222+0000","last_became_peered":"2024-08-18T23:41:39.185222+0000","last_unstale":"2024-08-18T23:42:23.321952+0000","last_undegraded":"2024-08-18T23:42:23.321952+0000","last_fullsized":"2024-08-18T23:42:23.321952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:01:27.290998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318594+0000","last_change":"2024-08-18T23:41:32.140719+0000","last_active":"2024-08-18T23:42:23.318594+0000","last_peered":"2024-08-18T23:42:23.318594+0000","last_clean":"2024-08-18T23:42:23.318594+0000","last_became_active":"2024-08-18T23:41:32.140131+0000","last_became_peered":"2024-08-18T23:41:32.140131+0000","last_unstale":"2024-08-18T23:42:23.318594+0000","last_undegraded":"2024-08-18T23:42:23.318594+0000","last_fullsized":"2024-08-18T23:42:23.318594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:48:26.778094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.326683+0000","last_change":"2024-08-18T23:42:23.326683+0000","last_active":"2024-08-18T23:42:23.326683+0000","last_peered":"2024-08-18T23:42:23.326683+0000","last_clean":"2024-08-18T23:42:23.326683+0000","last_became_active":"2024-08-18T23:42:23.326309+0000","last_became_peered":"2024-08-18T23:42:23.326309+0000","last_unstale":"2024-08-18T23:42:23.326683+0000","last_undegraded":"2024-08-18T23:42:23.326683+0000","last_fullsized":"2024-08-18T23:42:23.326683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:14.320894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.321522+0000","last_change":"2024-08-18T23:41:32.149522+0000","last_active":"2024-08-18T23:42:23.321522+0000","last_peered":"2024-08-18T23:42:23.321522+0000","last_clean":"2024-08-18T23:42:23.321522+0000","last_became_active":"2024-08-18T23:41:32.149182+0000","last_became_peered":"2024-08-18T23:41:32.149182+0000","last_unstale":"2024-08-18T23:42:23.321522+0000","last_undegraded":"2024-08-18T23:42:23.321522+0000","last_fullsized":"2024-08-18T23:42:23.321522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:30:16.945045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316398+0000","last_change":"2024-08-18T23:41:32.149853+0000","last_active":"2024-08-18T23:42:23.316398+0000","last_peered":"2024-08-18T23:42:23.316398+0000","last_clean":"2024-08-18T23:42:23.316398+0000","last_became_active":"2024-08-18T23:41:32.141907+0000","last_became_peered":"2024-08-18T23:41:32.141907+0000","last_unstale":"2024-08-18T23:42:23.316398+0000","last_undegraded":"2024-08-18T23:42:23.316398+0000","last_fullsized":"2024-08-18T23:42:23.316398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:24:37.323567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314554+0000","last_change":"2024-08-18T23:41:32.136179+0000","last_active":"2024-08-18T23:42:23.314554+0000","last_peered":"2024-08-18T23:42:23.314554+0000","last_clean":"2024-08-18T23:42:23.314554+0000","last_became_active":"2024-08-18T23:41:32.135939+0000","last_became_peered":"2024-08-18T23:41:32.135939+0000","last_unstale":"2024-08-18T23:42:23.314554+0000","last_undegraded":"2024-08-18T23:42:23.314554+0000","last_fullsized":"2024-08-18T23:42:23.314554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:15.392061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318883+0000","last_change":"2024-08-18T23:41:32.137509+0000","last_active":"2024-08-18T23:42:23.318883+0000","last_peered":"2024-08-18T23:42:23.318883+0000","last_clean":"2024-08-18T23:42:23.318883+0000","last_became_active":"2024-08-18T23:41:32.136380+0000","last_became_peered":"2024-08-18T23:41:32.136380+0000","last_unstale":"2024-08-18T23:42:23.318883+0000","last_undegraded":"2024-08-18T23:42:23.318883+0000","last_fullsized":"2024-08-18T23:42:23.318883+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:49.776225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322689+0000","last_change":"2024-08-18T23:41:32.139271+0000","last_active":"2024-08-18T23:42:23.322689+0000","last_peered":"2024-08-18T23:42:23.322689+0000","last_clean":"2024-08-18T23:42:23.322689+0000","last_became_active":"2024-08-18T23:41:32.138868+0000","last_became_peered":"2024-08-18T23:41:32.138868+0000","last_unstale":"2024-08-18T23:42:23.322689+0000","last_undegraded":"2024-08-18T23:42:23.322689+0000","last_fullsized":"2024-08-18T23:42:23.322689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:49:27.890857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557875+0000","last_change":"2024-08-18T23:41:32.144460+0000","last_active":"2024-08-18T23:42:23.557875+0000","last_peered":"2024-08-18T23:42:23.557875+0000","last_clean":"2024-08-18T23:42:23.557875+0000","last_became_active":"2024-08-18T23:41:32.144178+0000","last_became_peered":"2024-08-18T23:41:32.144178+0000","last_unstale":"2024-08-18T23:42:23.557875+0000","last_undegraded":"2024-08-18T23:42:23.557875+0000","last_fullsized":"2024-08-18T23:42:23.557875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:56.252215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320708+0000","last_change":"2024-08-18T23:41:32.147973+0000","last_active":"2024-08-18T23:42:23.320708+0000","last_peered":"2024-08-18T23:42:23.320708+0000","last_clean":"2024-08-18T23:42:23.320708+0000","last_became_active":"2024-08-18T23:41:32.147621+0000","last_became_peered":"2024-08-18T23:41:32.147621+0000","last_unstale":"2024-08-18T23:42:23.320708+0000","last_undegraded":"2024-08-18T23:42:23.320708+0000","last_fullsized":"2024-08-18T23:42:23.320708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:14.176569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557525+0000","last_change":"2024-08-18T23:41:32.146029+0000","last_active":"2024-08-18T23:42:23.557525+0000","last_peered":"2024-08-18T23:42:23.557525+0000","last_clean":"2024-08-18T23:42:23.557525+0000","last_became_active":"2024-08-18T23:41:32.145584+0000","last_became_peered":"2024-08-18T23:41:32.145584+0000","last_unstale":"2024-08-18T23:42:23.557525+0000","last_undegraded":"2024-08-18T23:42:23.557525+0000","last_fullsized":"2024-08-18T23:42:23.557525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:25:33.884906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315440+0000","last_change":"2024-08-18T23:41:32.145950+0000","last_active":"2024-08-18T23:42:23.315440+0000","last_peered":"2024-08-18T23:42:23.315440+0000","last_clean":"2024-08-18T23:42:23.315440+0000","last_became_active":"2024-08-18T23:41:32.145618+0000","last_became_peered":"2024-08-18T23:41:32.145618+0000","last_unstale":"2024-08-18T23:42:23.315440+0000","last_undegraded":"2024-08-18T23:42:23.315440+0000","last_fullsized":"2024-08-18T23:42:23.315440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:05:58.889240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314395+0000","last_change":"2024-08-18T23:41:32.130684+0000","last_active":"2024-08-18T23:42:23.314395+0000","last_peered":"2024-08-18T23:42:23.314395+0000","last_clean":"2024-08-18T23:42:23.314395+0000","last_became_active":"2024-08-18T23:41:32.130369+0000","last_became_peered":"2024-08-18T23:41:32.130369+0000","last_unstale":"2024-08-18T23:42:23.314395+0000","last_undegraded":"2024-08-18T23:42:23.314395+0000","last_fullsized":"2024-08-18T23:42:23.314395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:22.537018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320123+0000","last_change":"2024-08-18T23:41:32.137291+0000","last_active":"2024-08-18T23:42:23.320123+0000","last_peered":"2024-08-18T23:42:23.320123+0000","last_clean":"2024-08-18T23:42:23.320123+0000","last_became_active":"2024-08-18T23:41:32.136977+0000","last_became_peered":"2024-08-18T23:41:32.136977+0000","last_unstale":"2024-08-18T23:42:23.320123+0000","last_undegraded":"2024-08-18T23:42:23.320123+0000","last_fullsized":"2024-08-18T23:42:23.320123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:18:45.544760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322146+0000","last_change":"2024-08-18T23:41:32.144434+0000","last_active":"2024-08-18T23:42:23.322146+0000","last_peered":"2024-08-18T23:42:23.322146+0000","last_clean":"2024-08-18T23:42:23.322146+0000","last_became_active":"2024-08-18T23:41:32.144119+0000","last_became_peered":"2024-08-18T23:41:32.144119+0000","last_unstale":"2024-08-18T23:42:23.322146+0000","last_undegraded":"2024-08-18T23:42:23.322146+0000","last_fullsized":"2024-08-18T23:42:23.322146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:50:54.528630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319436+0000","last_change":"2024-08-18T23:41:32.135034+0000","last_active":"2024-08-18T23:42:23.319436+0000","last_peered":"2024-08-18T23:42:23.319436+0000","last_clean":"2024-08-18T23:42:23.319436+0000","last_became_active":"2024-08-18T23:41:32.134700+0000","last_became_peered":"2024-08-18T23:41:32.134700+0000","last_unstale":"2024-08-18T23:42:23.319436+0000","last_undegraded":"2024-08-18T23:42:23.319436+0000","last_fullsized":"2024-08-18T23:42:23.319436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:06:31.263114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319688+0000","last_change":"2024-08-18T23:41:32.141191+0000","last_active":"2024-08-18T23:42:23.319688+0000","last_peered":"2024-08-18T23:42:23.319688+0000","last_clean":"2024-08-18T23:42:23.319688+0000","last_became_active":"2024-08-18T23:41:32.140901+0000","last_became_peered":"2024-08-18T23:41:32.140901+0000","last_unstale":"2024-08-18T23:42:23.319688+0000","last_undegraded":"2024-08-18T23:42:23.319688+0000","last_fullsized":"2024-08-18T23:42:23.319688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:24.849985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314798+0000","last_change":"2024-08-18T23:41:32.142543+0000","last_active":"2024-08-18T23:42:23.314798+0000","last_peered":"2024-08-18T23:42:23.314798+0000","last_clean":"2024-08-18T23:42:23.314798+0000","last_became_active":"2024-08-18T23:41:32.142084+0000","last_became_peered":"2024-08-18T23:41:32.142084+0000","last_unstale":"2024-08-18T23:42:23.314798+0000","last_undegraded":"2024-08-18T23:42:23.314798+0000","last_fullsized":"2024-08-18T23:42:23.314798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:54:18.200016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316269+0000","last_change":"2024-08-18T23:41:32.148968+0000","last_active":"2024-08-18T23:42:23.316269+0000","last_peered":"2024-08-18T23:42:23.316269+0000","last_clean":"2024-08-18T23:42:23.316269+0000","last_became_active":"2024-08-18T23:41:32.148625+0000","last_became_peered":"2024-08-18T23:41:32.148625+0000","last_unstale":"2024-08-18T23:42:23.316269+0000","last_undegraded":"2024-08-18T23:42:23.316269+0000","last_fullsized":"2024-08-18T23:42:23.316269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:11:40.172139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316345+0000","last_change":"2024-08-18T23:41:32.133478+0000","last_active":"2024-08-18T23:42:23.316345+0000","last_peered":"2024-08-18T23:42:23.316345+0000","last_clean":"2024-08-18T23:42:23.316345+0000","last_became_active":"2024-08-18T23:41:32.133149+0000","last_became_peered":"2024-08-18T23:41:32.133149+0000","last_unstale":"2024-08-18T23:42:23.316345+0000","last_undegraded":"2024-08-18T23:42:23.316345+0000","last_fullsized":"2024-08-18T23:42:23.316345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:57:53.713309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314293+0000","last_change":"2024-08-18T23:41:32.138206+0000","last_active":"2024-08-18T23:42:23.314293+0000","last_peered":"2024-08-18T23:42:23.314293+0000","last_clean":"2024-08-18T23:42:23.314293+0000","last_became_active":"2024-08-18T23:41:32.137903+0000","last_became_peered":"2024-08-18T23:41:32.137903+0000","last_unstale":"2024-08-18T23:42:23.314293+0000","last_undegraded":"2024-08-18T23:42:23.314293+0000","last_fullsized":"2024-08-18T23:42:23.314293+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:18:48.600658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319090+0000","last_change":"2024-08-18T23:41:32.136303+0000","last_active":"2024-08-18T23:42:23.319090+0000","last_peered":"2024-08-18T23:42:23.319090+0000","last_clean":"2024-08-18T23:42:23.319090+0000","last_became_active":"2024-08-18T23:41:32.134628+0000","last_became_peered":"2024-08-18T23:41:32.134628+0000","last_unstale":"2024-08-18T23:42:23.319090+0000","last_undegraded":"2024-08-18T23:42:23.319090+0000","last_fullsized":"2024-08-18T23:42:23.319090+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:28:51.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317055+0000","last_change":"2024-08-18T23:41:32.137512+0000","last_active":"2024-08-18T23:42:23.317055+0000","last_peered":"2024-08-18T23:42:23.317055+0000","last_clean":"2024-08-18T23:42:23.317055+0000","last_became_active":"2024-08-18T23:41:32.137150+0000","last_became_peered":"2024-08-18T23:41:32.137150+0000","last_unstale":"2024-08-18T23:42:23.317055+0000","last_undegraded":"2024-08-18T23:42:23.317055+0000","last_fullsized":"2024-08-18T23:42:23.317055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:44:08.020780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558680+0000","last_change":"2024-08-18T23:41:32.139229+0000","last_active":"2024-08-18T23:42:23.558680+0000","last_peered":"2024-08-18T23:42:23.558680+0000","last_clean":"2024-08-18T23:42:23.558680+0000","last_became_active":"2024-08-18T23:41:32.138864+0000","last_became_peered":"2024-08-18T23:41:32.138864+0000","last_unstale":"2024-08-18T23:42:23.558680+0000","last_undegraded":"2024-08-18T23:42:23.558680+0000","last_fullsized":"2024-08-18T23:42:23.558680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:54:21.701750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316215+0000","last_change":"2024-08-18T23:41:32.148727+0000","last_active":"2024-08-18T23:42:23.316215+0000","last_peered":"2024-08-18T23:42:23.316215+0000","last_clean":"2024-08-18T23:42:23.316215+0000","last_became_active":"2024-08-18T23:41:32.146828+0000","last_became_peered":"2024-08-18T23:41:32.146828+0000","last_unstale":"2024-08-18T23:42:23.316215+0000","last_undegraded":"2024-08-18T23:42:23.316215+0000","last_fullsized":"2024-08-18T23:42:23.316215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:51:49.175150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557445+0000","last_change":"2024-08-18T23:41:32.144796+0000","last_active":"2024-08-18T23:42:23.557445+0000","last_peered":"2024-08-18T23:42:23.557445+0000","last_clean":"2024-08-18T23:42:23.557445+0000","last_became_active":"2024-08-18T23:41:32.141931+0000","last_became_peered":"2024-08-18T23:41:32.141931+0000","last_unstale":"2024-08-18T23:42:23.557445+0000","last_undegraded":"2024-08-18T23:42:23.557445+0000","last_fullsized":"2024-08-18T23:42:23.557445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:48:08.428845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316954+0000","last_change":"2024-08-18T23:41:32.135265+0000","last_active":"2024-08-18T23:42:23.316954+0000","last_peered":"2024-08-18T23:42:23.316954+0000","last_clean":"2024-08-18T23:42:23.316954+0000","last_became_active":"2024-08-18T23:41:32.134952+0000","last_became_peered":"2024-08-18T23:41:32.134952+0000","last_unstale":"2024-08-18T23:42:23.316954+0000","last_undegraded":"2024-08-18T23:42:23.316954+0000","last_fullsized":"2024-08-18T23:42:23.316954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:35:14.099536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558127+0000","last_change":"2024-08-18T23:41:32.135941+0000","last_active":"2024-08-18T23:42:23.558127+0000","last_peered":"2024-08-18T23:42:23.558127+0000","last_clean":"2024-08-18T23:42:23.558127+0000","last_became_active":"2024-08-18T23:41:32.135623+0000","last_became_peered":"2024-08-18T23:41:32.135623+0000","last_unstale":"2024-08-18T23:42:23.558127+0000","last_undegraded":"2024-08-18T23:42:23.558127+0000","last_fullsized":"2024-08-18T23:42:23.558127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:43:36.504738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324078+0000","last_change":"2024-08-18T23:41:32.141643+0000","last_active":"2024-08-18T23:42:23.324078+0000","last_peered":"2024-08-18T23:42:23.324078+0000","last_clean":"2024-08-18T23:42:23.324078+0000","last_became_active":"2024-08-18T23:41:32.141301+0000","last_became_peered":"2024-08-18T23:41:32.141301+0000","last_unstale":"2024-08-18T23:42:23.324078+0000","last_undegraded":"2024-08-18T23:42:23.324078+0000","last_fullsized":"2024-08-18T23:42:23.324078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:23:53.100740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318979+0000","last_change":"2024-08-18T23:41:32.146730+0000","last_active":"2024-08-18T23:42:23.318979+0000","last_peered":"2024-08-18T23:42:23.318979+0000","last_clean":"2024-08-18T23:42:23.318979+0000","last_became_active":"2024-08-18T23:41:32.146252+0000","last_became_peered":"2024-08-18T23:41:32.146252+0000","last_unstale":"2024-08-18T23:42:23.318979+0000","last_undegraded":"2024-08-18T23:42:23.318979+0000","last_fullsized":"2024-08-18T23:42:23.318979+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:41.481761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316570+0000","last_change":"2024-08-18T23:41:32.149569+0000","last_active":"2024-08-18T23:42:23.316570+0000","last_peered":"2024-08-18T23:42:23.316570+0000","last_clean":"2024-08-18T23:42:23.316570+0000","last_became_active":"2024-08-18T23:41:32.149277+0000","last_became_peered":"2024-08-18T23:41:32.149277+0000","last_unstale":"2024-08-18T23:42:23.316570+0000","last_undegraded":"2024-08-18T23:42:23.316570+0000","last_fullsized":"2024-08-18T23:42:23.316570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:34:00.547909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316678+0000","last_change":"2024-08-18T23:41:32.137889+0000","last_active":"2024-08-18T23:42:23.316678+0000","last_peered":"2024-08-18T23:42:23.316678+0000","last_clean":"2024-08-18T23:42:23.316678+0000","last_became_active":"2024-08-18T23:41:32.137578+0000","last_became_peered":"2024-08-18T23:41:32.137578+0000","last_unstale":"2024-08-18T23:42:23.316678+0000","last_undegraded":"2024-08-18T23:42:23.316678+0000","last_fullsized":"2024-08-18T23:42:23.316678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:18:09.987838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319506+0000","last_change":"2024-08-18T23:41:32.142084+0000","last_active":"2024-08-18T23:42:23.319506+0000","last_peered":"2024-08-18T23:42:23.319506+0000","last_clean":"2024-08-18T23:42:23.319506+0000","last_became_active":"2024-08-18T23:41:32.141670+0000","last_became_peered":"2024-08-18T23:41:32.141670+0000","last_unstale":"2024-08-18T23:42:23.319506+0000","last_undegraded":"2024-08-18T23:42:23.319506+0000","last_fullsized":"2024-08-18T23:42:23.319506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:03.990351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65398,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":155,"num_read_kb":148,"num_write":166,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":187664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514520,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":505014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776157,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28684,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716468,"statfs":{"total":95995035648,"available":95965663232,"internally_reserved":0,"allocated":1708032,"data_stored":1153680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939681,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":509346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168614,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":509266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397545,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36936,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708216,"statfs":{"total":95995035648,"available":95957213184,"internally_reserved":0,"allocated":1630208,"data_stored":1101468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561071,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":500426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790003,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1092570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953528,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":506048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":500620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":558237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":557079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":501814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":57901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T23:42:33.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:33 smithi078 ceph-mon[18440]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:42:33.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:33 smithi078 ceph-mon[18440]: pgmap v295: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s; 71 B/s, 0 objects/s recovering 2024-08-18T23:42:33.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:33.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:33.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:42:33.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:42:33.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:33.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:33 smithi123 ceph-mon[21808]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:42:33.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:33 smithi123 ceph-mon[21808]: pgmap v295: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s; 71 B/s, 0 objects/s recovering 2024-08-18T23:42:33.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:33 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:33.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:33 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:33.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:33 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:42:33.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:33 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:42:33.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:33 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:33 smithi161 ceph-mon[22027]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:42:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:33 smithi161 ceph-mon[22027]: pgmap v295: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s; 71 B/s, 0 objects/s recovering 2024-08-18T23:42:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:33 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:33 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:33 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:42:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:33 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:42:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:33 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:42:35.265 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-18T23:42:35.292 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:35 smithi078 ceph-mon[18440]: pgmap v296: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 64 B/s, 0 objects/s recovering 2024-08-18T23:42:35.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:35 smithi123 ceph-mon[21808]: pgmap v296: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 64 B/s, 0 objects/s recovering 2024-08-18T23:42:35.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:35 smithi161 ceph-mon[22027]: pgmap v296: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 64 B/s, 0 objects/s recovering 2024-08-18T23:42:35.529 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:36.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.021+0000 7f2087209640 1 -- 172.21.15.78:0/3315533292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20801022a0 msgr2=0x7f20801026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:36.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.021+0000 7f2087209640 1 --2- 172.21.15.78:0/3315533292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20801022a0 0x7f20801026e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f206c009900 tx=0x7f206c02a480 comp rx=0 tx=0).stop 2024-08-18T23:42:36.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.021+0000 7f2087209640 1 -- 172.21.15.78:0/3315533292 shutdown_connections 2024-08-18T23:42:36.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.021+0000 7f2087209640 1 --2- 172.21.15.78:0/3315533292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2080102c20 0x7f208010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:36.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.021+0000 7f2087209640 1 --2- 172.21.15.78:0/3315533292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20801022a0 0x7f20801026e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:36.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.021+0000 7f2087209640 1 --2- 172.21.15.78:0/3315533292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2080100b50 0x7f2080100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:36.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.021+0000 7f2087209640 1 -- 172.21.15.78:0/3315533292 >> 172.21.15.78:0/3315533292 conn(0x7f20800fc500 msgr2=0x7f20800fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:36.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.021+0000 7f2087209640 1 -- 172.21.15.78:0/3315533292 shutdown_connections 2024-08-18T23:42:36.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.022+0000 7f2087209640 1 -- 172.21.15.78:0/3315533292 wait complete. 2024-08-18T23:42:36.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.022+0000 7f2087209640 1 Processor -- start 2024-08-18T23:42:36.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.022+0000 7f2087209640 1 -- start start 2024-08-18T23:42:36.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.022+0000 7f2087209640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2080100b50 0x7f20800fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:36.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.022+0000 7f2087209640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20801022a0 0x7f20800fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:36.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.022+0000 7f2087209640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2080102c20 0x7f20800fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:36.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.022+0000 7f2087209640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2080110d50 con 0x7f2080100b50 2024-08-18T23:42:36.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.022+0000 7f2087209640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2080110bd0 con 0x7f20801022a0 2024-08-18T23:42:36.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.022+0000 7f2087209640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2080110ed0 con 0x7f2080102c20 2024-08-18T23:42:36.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.022+0000 7f2084fa5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2080100b50 0x7f20800fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:36.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.023+0000 7f2084fa5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2080100b50 0x7f20800fc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56624/0 (socket says 172.21.15.78:56624) 2024-08-18T23:42:36.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.023+0000 7f2084fa5640 1 -- 172.21.15.78:0/82378856 learned_addr learned my addr 172.21.15.78:0/82378856 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:36.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.023+0000 7f207ffff640 1 --2- 172.21.15.78:0/82378856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20801022a0 0x7f20800fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:36.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.023+0000 7f20857a6640 1 --2- 172.21.15.78:0/82378856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2080102c20 0x7f20800fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:36.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.023+0000 7f2084fa5640 1 -- 172.21.15.78:0/82378856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2080102c20 msgr2=0x7f20800fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:36.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.023+0000 7f2084fa5640 1 --2- 172.21.15.78:0/82378856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2080102c20 0x7f20800fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:36.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.023+0000 7f2084fa5640 1 -- 172.21.15.78:0/82378856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20801022a0 msgr2=0x7f20800fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:36.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.023+0000 7f2084fa5640 1 --2- 172.21.15.78:0/82378856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20801022a0 0x7f20800fcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:36.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.023+0000 7f2084fa5640 1 -- 172.21.15.78:0/82378856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20801a68f0 con 0x7f2080100b50 2024-08-18T23:42:36.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.023+0000 7f207ffff640 1 --2- 172.21.15.78:0/82378856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20801022a0 0x7f20800fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:42:36.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.023+0000 7f2084fa5640 1 --2- 172.21.15.78:0/82378856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2080100b50 0x7f20800fc9b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f207400b4a0 tx=0x7f207400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:36.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.023+0000 7f207dffb640 1 -- 172.21.15.78:0/82378856 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2074014070 con 0x7f2080100b50 2024-08-18T23:42:36.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.023+0000 7f2087209640 1 -- 172.21.15.78:0/82378856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20801a6be0 con 0x7f2080100b50 2024-08-18T23:42:36.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.023+0000 7f207dffb640 1 -- 172.21.15.78:0/82378856 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2074010af0 con 0x7f2080100b50 2024-08-18T23:42:36.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.023+0000 7f207dffb640 1 -- 172.21.15.78:0/82378856 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2074017d10 con 0x7f2080100b50 2024-08-18T23:42:36.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.023+0000 7f2087209640 1 -- 172.21.15.78:0/82378856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20801a7120 con 0x7f2080100b50 2024-08-18T23:42:36.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.024+0000 7f2087209640 1 -- 172.21.15.78:0/82378856 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2080100fb0 con 0x7f2080100b50 2024-08-18T23:42:36.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.026+0000 7f207dffb640 1 -- 172.21.15.78:0/82378856 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2074017070 con 0x7f2080100b50 2024-08-18T23:42:36.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.026+0000 7f207dffb640 1 --2- 172.21.15.78:0/82378856 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f20700a2c10 0x7f20700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:36.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.026+0000 7f207ffff640 1 --2- 172.21.15.78:0/82378856 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f20700a2c10 0x7f20700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:36.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.026+0000 7f207dffb640 1 -- 172.21.15.78:0/82378856 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f20740ce840 con 0x7f2080100b50 2024-08-18T23:42:36.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.027+0000 7f207ffff640 1 --2- 172.21.15.78:0/82378856 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f20700a2c10 0x7f20700a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f20800fe440 tx=0x7f206c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:36.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.030+0000 7f207dffb640 1 -- 172.21.15.78:0/82378856 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f207409b3f0 con 0x7f2080100b50 2024-08-18T23:42:36.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.200+0000 7f2087209640 1 -- 172.21.15.78:0/82378856 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2080109e40 con 0x7f20700a2c10 2024-08-18T23:42:36.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.209+0000 7f207dffb640 1 -- 172.21.15.78:0/82378856 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340371 (secure 0 0 0) 0x7f206c0043b0 con 0x7f20700a2c10 2024-08-18T23:42:36.213 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:42:36.214 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-08-18T23:42:36.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.212+0000 7f2087209640 1 -- 172.21.15.78:0/82378856 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f20700a2c10 msgr2=0x7f20700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:36.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.212+0000 7f2087209640 1 --2- 172.21.15.78:0/82378856 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f20700a2c10 0x7f20700a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f20800fe440 tx=0x7f206c005a20 comp rx=0 tx=0).stop 2024-08-18T23:42:36.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.212+0000 7f2087209640 1 -- 172.21.15.78:0/82378856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2080100b50 msgr2=0x7f20800fc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:36.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.212+0000 7f2087209640 1 --2- 172.21.15.78:0/82378856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2080100b50 0x7f20800fc9b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f207400b4a0 tx=0x7f207400b960 comp rx=0 tx=0).stop 2024-08-18T23:42:36.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.212+0000 7f2087209640 1 -- 172.21.15.78:0/82378856 shutdown_connections 2024-08-18T23:42:36.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.212+0000 7f2087209640 1 --2- 172.21.15.78:0/82378856 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f20700a2c10 0x7f20700a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:36.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.212+0000 7f2087209640 1 --2- 172.21.15.78:0/82378856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2080102c20 0x7f20800fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:36.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.212+0000 7f2087209640 1 --2- 172.21.15.78:0/82378856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20801022a0 0x7f20800fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:36.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.212+0000 7f2087209640 1 --2- 172.21.15.78:0/82378856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2080100b50 0x7f20800fc9b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:36.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.212+0000 7f2087209640 1 -- 172.21.15.78:0/82378856 >> 172.21.15.78:0/82378856 conn(0x7f20800fc500 msgr2=0x7f208006a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:36.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.212+0000 7f2087209640 1 -- 172.21.15.78:0/82378856 shutdown_connections 2024-08-18T23:42:36.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:36.212+0000 7f2087209640 1 -- 172.21.15.78:0/82378856 wait complete. 2024-08-18T23:42:37.003 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-08-18T23:42:36.124271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657988,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":312,"num_read_kb":274,"num_write":350,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":282,"ondisk_log_size":282,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388216,"kb_used_data":14044,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553608,"statfs":{"total":1151940427776,"available":1151542894592,"internally_reserved":0,"allocated":14381056,"data_stored":8008074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13904,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":59,"num_write":10,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003368"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319578+0000","last_change":"2024-08-18T23:41:30.120580+0000","last_active":"2024-08-18T23:42:23.319578+0000","last_peered":"2024-08-18T23:42:23.319578+0000","last_clean":"2024-08-18T23:42:23.319578+0000","last_became_active":"2024-08-18T23:41:30.120266+0000","last_became_peered":"2024-08-18T23:41:30.120266+0000","last_unstale":"2024-08-18T23:42:23.319578+0000","last_undegraded":"2024-08-18T23:42:23.319578+0000","last_fullsized":"2024-08-18T23:42:23.319578+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:06:40.997070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557745+0000","last_change":"2024-08-18T23:41:32.143232+0000","last_active":"2024-08-18T23:42:23.557745+0000","last_peered":"2024-08-18T23:42:23.557745+0000","last_clean":"2024-08-18T23:42:23.557745+0000","last_became_active":"2024-08-18T23:41:32.142916+0000","last_became_peered":"2024-08-18T23:41:32.142916+0000","last_unstale":"2024-08-18T23:42:23.557745+0000","last_undegraded":"2024-08-18T23:42:23.557745+0000","last_fullsized":"2024-08-18T23:42:23.557745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:50:54.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320593+0000","last_change":"2024-08-18T23:41:39.188772+0000","last_active":"2024-08-18T23:42:23.320593+0000","last_peered":"2024-08-18T23:42:23.320593+0000","last_clean":"2024-08-18T23:42:23.320593+0000","last_became_active":"2024-08-18T23:41:39.188362+0000","last_became_peered":"2024-08-18T23:41:39.188362+0000","last_unstale":"2024-08-18T23:42:23.320593+0000","last_undegraded":"2024-08-18T23:42:23.320593+0000","last_fullsized":"2024-08-18T23:42:23.320593+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:47:50.255543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557888+0000","last_change":"2024-08-18T23:41:30.137179+0000","last_active":"2024-08-18T23:42:23.557888+0000","last_peered":"2024-08-18T23:42:23.557888+0000","last_clean":"2024-08-18T23:42:23.557888+0000","last_became_active":"2024-08-18T23:41:30.136618+0000","last_became_peered":"2024-08-18T23:41:30.136618+0000","last_unstale":"2024-08-18T23:42:23.557888+0000","last_undegraded":"2024-08-18T23:42:23.557888+0000","last_fullsized":"2024-08-18T23:42:23.557888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:59.826364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316847+0000","last_change":"2024-08-18T23:41:32.142075+0000","last_active":"2024-08-18T23:42:23.316847+0000","last_peered":"2024-08-18T23:42:23.316847+0000","last_clean":"2024-08-18T23:42:23.316847+0000","last_became_active":"2024-08-18T23:41:32.141709+0000","last_became_peered":"2024-08-18T23:41:32.141709+0000","last_unstale":"2024-08-18T23:42:23.316847+0000","last_undegraded":"2024-08-18T23:42:23.316847+0000","last_fullsized":"2024-08-18T23:42:23.316847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:08.324627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317948+0000","last_change":"2024-08-18T23:41:39.184575+0000","last_active":"2024-08-18T23:42:23.317948+0000","last_peered":"2024-08-18T23:42:23.317948+0000","last_clean":"2024-08-18T23:42:23.317948+0000","last_became_active":"2024-08-18T23:41:39.184169+0000","last_became_peered":"2024-08-18T23:41:39.184169+0000","last_unstale":"2024-08-18T23:42:23.317948+0000","last_undegraded":"2024-08-18T23:42:23.317948+0000","last_fullsized":"2024-08-18T23:42:23.317948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:32.218172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324756+0000","last_change":"2024-08-18T23:41:30.121638+0000","last_active":"2024-08-18T23:42:23.324756+0000","last_peered":"2024-08-18T23:42:23.324756+0000","last_clean":"2024-08-18T23:42:23.324756+0000","last_became_active":"2024-08-18T23:41:30.121290+0000","last_became_peered":"2024-08-18T23:41:30.121290+0000","last_unstale":"2024-08-18T23:42:23.324756+0000","last_undegraded":"2024-08-18T23:42:23.324756+0000","last_fullsized":"2024-08-18T23:42:23.324756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:43:55.638780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315534+0000","last_change":"2024-08-18T23:41:32.142177+0000","last_active":"2024-08-18T23:42:23.315534+0000","last_peered":"2024-08-18T23:42:23.315534+0000","last_clean":"2024-08-18T23:42:23.315534+0000","last_became_active":"2024-08-18T23:41:32.141731+0000","last_became_peered":"2024-08-18T23:41:32.141731+0000","last_unstale":"2024-08-18T23:42:23.315534+0000","last_undegraded":"2024-08-18T23:42:23.315534+0000","last_fullsized":"2024-08-18T23:42:23.315534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:36.283299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318344+0000","last_change":"2024-08-18T23:41:39.182931+0000","last_active":"2024-08-18T23:42:23.318344+0000","last_peered":"2024-08-18T23:42:23.318344+0000","last_clean":"2024-08-18T23:42:23.318344+0000","last_became_active":"2024-08-18T23:41:39.182008+0000","last_became_peered":"2024-08-18T23:41:39.182008+0000","last_unstale":"2024-08-18T23:42:23.318344+0000","last_undegraded":"2024-08-18T23:42:23.318344+0000","last_fullsized":"2024-08-18T23:42:23.318344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:58:38.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.329762+0000","last_change":"2024-08-18T23:42:23.329762+0000","last_active":"2024-08-18T23:42:23.329762+0000","last_peered":"2024-08-18T23:42:23.329762+0000","last_clean":"2024-08-18T23:42:23.329762+0000","last_became_active":"2024-08-18T23:42:23.325753+0000","last_became_peered":"2024-08-18T23:42:23.325753+0000","last_unstale":"2024-08-18T23:42:23.329762+0000","last_undegraded":"2024-08-18T23:42:23.329762+0000","last_fullsized":"2024-08-18T23:42:23.329762+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:19:43.062492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323256+0000","last_change":"2024-08-18T23:41:32.148662+0000","last_active":"2024-08-18T23:42:23.323256+0000","last_peered":"2024-08-18T23:42:23.323256+0000","last_clean":"2024-08-18T23:42:23.323256+0000","last_became_active":"2024-08-18T23:41:32.148318+0000","last_became_peered":"2024-08-18T23:41:32.148318+0000","last_unstale":"2024-08-18T23:42:23.323256+0000","last_undegraded":"2024-08-18T23:42:23.323256+0000","last_fullsized":"2024-08-18T23:42:23.323256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:58.668669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318492+0000","last_change":"2024-08-18T23:41:39.181114+0000","last_active":"2024-08-18T23:42:23.318492+0000","last_peered":"2024-08-18T23:42:23.318492+0000","last_clean":"2024-08-18T23:42:23.318492+0000","last_became_active":"2024-08-18T23:41:39.180599+0000","last_became_peered":"2024-08-18T23:41:39.180599+0000","last_unstale":"2024-08-18T23:42:23.318492+0000","last_undegraded":"2024-08-18T23:42:23.318492+0000","last_fullsized":"2024-08-18T23:42:23.318492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:36:08.749143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316660+0000","last_change":"2024-08-18T23:41:32.144326+0000","last_active":"2024-08-18T23:42:23.316660+0000","last_peered":"2024-08-18T23:42:23.316660+0000","last_clean":"2024-08-18T23:42:23.316660+0000","last_became_active":"2024-08-18T23:41:32.144004+0000","last_became_peered":"2024-08-18T23:41:32.144004+0000","last_unstale":"2024-08-18T23:42:23.316660+0000","last_undegraded":"2024-08-18T23:42:23.316660+0000","last_fullsized":"2024-08-18T23:42:23.316660+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:10.918624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324188+0000","last_change":"2024-08-18T23:41:30.125160+0000","last_active":"2024-08-18T23:42:23.324188+0000","last_peered":"2024-08-18T23:42:23.324188+0000","last_clean":"2024-08-18T23:42:23.324188+0000","last_became_active":"2024-08-18T23:41:30.124685+0000","last_became_peered":"2024-08-18T23:41:30.124685+0000","last_unstale":"2024-08-18T23:42:23.324188+0000","last_undegraded":"2024-08-18T23:42:23.324188+0000","last_fullsized":"2024-08-18T23:42:23.324188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:52:48.997902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318342+0000","last_change":"2024-08-18T23:41:39.681481+0000","last_active":"2024-08-18T23:42:23.318342+0000","last_peered":"2024-08-18T23:42:23.318342+0000","last_clean":"2024-08-18T23:42:23.318342+0000","last_became_active":"2024-08-18T23:41:39.680204+0000","last_became_peered":"2024-08-18T23:41:39.680204+0000","last_unstale":"2024-08-18T23:42:23.318342+0000","last_undegraded":"2024-08-18T23:42:23.318342+0000","last_fullsized":"2024-08-18T23:42:23.318342+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:00:05.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557591+0000","last_change":"2024-08-18T23:41:32.151335+0000","last_active":"2024-08-18T23:42:23.557591+0000","last_peered":"2024-08-18T23:42:23.557591+0000","last_clean":"2024-08-18T23:42:23.557591+0000","last_became_active":"2024-08-18T23:41:32.151040+0000","last_became_peered":"2024-08-18T23:41:32.151040+0000","last_unstale":"2024-08-18T23:42:23.557591+0000","last_undegraded":"2024-08-18T23:42:23.557591+0000","last_fullsized":"2024-08-18T23:42:23.557591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:16.108555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.321637+0000","last_change":"2024-08-18T23:41:30.122301+0000","last_active":"2024-08-18T23:42:23.321637+0000","last_peered":"2024-08-18T23:42:23.321637+0000","last_clean":"2024-08-18T23:42:23.321637+0000","last_became_active":"2024-08-18T23:41:30.121117+0000","last_became_peered":"2024-08-18T23:41:30.121117+0000","last_unstale":"2024-08-18T23:42:23.321637+0000","last_undegraded":"2024-08-18T23:42:23.321637+0000","last_fullsized":"2024-08-18T23:42:23.321637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:40:12.678803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318961+0000","last_change":"2024-08-18T23:41:39.182395+0000","last_active":"2024-08-18T23:42:23.318961+0000","last_peered":"2024-08-18T23:42:23.318961+0000","last_clean":"2024-08-18T23:42:23.318961+0000","last_became_active":"2024-08-18T23:41:39.181979+0000","last_became_peered":"2024-08-18T23:41:39.181979+0000","last_unstale":"2024-08-18T23:42:23.318961+0000","last_undegraded":"2024-08-18T23:42:23.318961+0000","last_fullsized":"2024-08-18T23:42:23.318961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:04.674503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318695+0000","last_change":"2024-08-18T23:41:32.140464+0000","last_active":"2024-08-18T23:42:23.318695+0000","last_peered":"2024-08-18T23:42:23.318695+0000","last_clean":"2024-08-18T23:42:23.318695+0000","last_became_active":"2024-08-18T23:41:32.139153+0000","last_became_peered":"2024-08-18T23:41:32.139153+0000","last_unstale":"2024-08-18T23:42:23.318695+0000","last_undegraded":"2024-08-18T23:42:23.318695+0000","last_fullsized":"2024-08-18T23:42:23.318695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:27.387680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557664+0000","last_change":"2024-08-18T23:41:30.122787+0000","last_active":"2024-08-18T23:42:23.557664+0000","last_peered":"2024-08-18T23:42:23.557664+0000","last_clean":"2024-08-18T23:42:23.557664+0000","last_became_active":"2024-08-18T23:41:30.122462+0000","last_became_peered":"2024-08-18T23:41:30.122462+0000","last_unstale":"2024-08-18T23:42:23.557664+0000","last_undegraded":"2024-08-18T23:42:23.557664+0000","last_fullsized":"2024-08-18T23:42:23.557664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:55.568831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319322+0000","last_change":"2024-08-18T23:41:39.190511+0000","last_active":"2024-08-18T23:42:23.319322+0000","last_peered":"2024-08-18T23:42:23.319322+0000","last_clean":"2024-08-18T23:42:23.319322+0000","last_became_active":"2024-08-18T23:41:39.190154+0000","last_became_peered":"2024-08-18T23:41:39.190154+0000","last_unstale":"2024-08-18T23:42:23.319322+0000","last_undegraded":"2024-08-18T23:42:23.319322+0000","last_fullsized":"2024-08-18T23:42:23.319322+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:33:34.514827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323413+0000","last_change":"2024-08-18T23:41:32.143867+0000","last_active":"2024-08-18T23:42:23.323413+0000","last_peered":"2024-08-18T23:42:23.323413+0000","last_clean":"2024-08-18T23:42:23.323413+0000","last_became_active":"2024-08-18T23:41:32.143527+0000","last_became_peered":"2024-08-18T23:41:32.143527+0000","last_unstale":"2024-08-18T23:42:23.323413+0000","last_undegraded":"2024-08-18T23:42:23.323413+0000","last_fullsized":"2024-08-18T23:42:23.323413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:40:25.902133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.325102+0000","last_change":"2024-08-18T23:42:23.325102+0000","last_active":"2024-08-18T23:42:23.325102+0000","last_peered":"2024-08-18T23:42:23.325102+0000","last_clean":"2024-08-18T23:42:23.325102+0000","last_became_active":"2024-08-18T23:42:23.321583+0000","last_became_peered":"2024-08-18T23:42:23.321583+0000","last_unstale":"2024-08-18T23:42:23.325102+0000","last_undegraded":"2024-08-18T23:42:23.325102+0000","last_fullsized":"2024-08-18T23:42:23.325102+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:57:42.546686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.559756+0000","last_change":"2024-08-18T23:42:23.559756+0000","last_active":"2024-08-18T23:42:23.559756+0000","last_peered":"2024-08-18T23:42:23.559756+0000","last_clean":"2024-08-18T23:42:23.559756+0000","last_became_active":"2024-08-18T23:42:23.559242+0000","last_became_peered":"2024-08-18T23:42:23.559242+0000","last_unstale":"2024-08-18T23:42:23.559756+0000","last_undegraded":"2024-08-18T23:42:23.559756+0000","last_fullsized":"2024-08-18T23:42:23.559756+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:25:40.790313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316766+0000","last_change":"2024-08-18T23:41:32.139025+0000","last_active":"2024-08-18T23:42:23.316766+0000","last_peered":"2024-08-18T23:42:23.316766+0000","last_clean":"2024-08-18T23:42:23.316766+0000","last_became_active":"2024-08-18T23:41:32.138733+0000","last_became_peered":"2024-08-18T23:41:32.138733+0000","last_unstale":"2024-08-18T23:42:23.316766+0000","last_undegraded":"2024-08-18T23:42:23.316766+0000","last_fullsized":"2024-08-18T23:42:23.316766+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:01:39.826391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319270+0000","last_change":"2024-08-18T23:41:30.477359+0000","last_active":"2024-08-18T23:42:23.319270+0000","last_peered":"2024-08-18T23:42:23.319270+0000","last_clean":"2024-08-18T23:42:23.319270+0000","last_became_active":"2024-08-18T23:41:30.477011+0000","last_became_peered":"2024-08-18T23:41:30.477011+0000","last_unstale":"2024-08-18T23:42:23.319270+0000","last_undegraded":"2024-08-18T23:42:23.319270+0000","last_fullsized":"2024-08-18T23:42:23.319270+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:53:10.052859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320716+0000","last_change":"2024-08-18T23:41:39.182140+0000","last_active":"2024-08-18T23:42:23.320716+0000","last_peered":"2024-08-18T23:42:23.320716+0000","last_clean":"2024-08-18T23:42:23.320716+0000","last_became_active":"2024-08-18T23:41:39.181667+0000","last_became_peered":"2024-08-18T23:41:39.181667+0000","last_unstale":"2024-08-18T23:42:23.320716+0000","last_undegraded":"2024-08-18T23:42:23.320716+0000","last_fullsized":"2024-08-18T23:42:23.320716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:02:18.128028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557897+0000","last_change":"2024-08-18T23:41:32.137064+0000","last_active":"2024-08-18T23:42:23.557897+0000","last_peered":"2024-08-18T23:42:23.557897+0000","last_clean":"2024-08-18T23:42:23.557897+0000","last_became_active":"2024-08-18T23:41:32.136738+0000","last_became_peered":"2024-08-18T23:41:32.136738+0000","last_unstale":"2024-08-18T23:42:23.557897+0000","last_undegraded":"2024-08-18T23:42:23.557897+0000","last_fullsized":"2024-08-18T23:42:23.557897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:15:11.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319512+0000","last_change":"2024-08-18T23:41:30.120042+0000","last_active":"2024-08-18T23:42:23.319512+0000","last_peered":"2024-08-18T23:42:23.319512+0000","last_clean":"2024-08-18T23:42:23.319512+0000","last_became_active":"2024-08-18T23:41:30.119719+0000","last_became_peered":"2024-08-18T23:41:30.119719+0000","last_unstale":"2024-08-18T23:42:23.319512+0000","last_undegraded":"2024-08-18T23:42:23.319512+0000","last_fullsized":"2024-08-18T23:42:23.319512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:35:44.466330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557353+0000","last_change":"2024-08-18T23:41:39.680405+0000","last_active":"2024-08-18T23:42:23.557353+0000","last_peered":"2024-08-18T23:42:23.557353+0000","last_clean":"2024-08-18T23:42:23.557353+0000","last_became_active":"2024-08-18T23:41:39.679940+0000","last_became_peered":"2024-08-18T23:41:39.679940+0000","last_unstale":"2024-08-18T23:42:23.557353+0000","last_undegraded":"2024-08-18T23:42:23.557353+0000","last_fullsized":"2024-08-18T23:42:23.557353+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:15:03.942116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314555+0000","last_change":"2024-08-18T23:41:32.139688+0000","last_active":"2024-08-18T23:42:23.314555+0000","last_peered":"2024-08-18T23:42:23.314555+0000","last_clean":"2024-08-18T23:42:23.314555+0000","last_became_active":"2024-08-18T23:41:32.139397+0000","last_became_peered":"2024-08-18T23:41:32.139397+0000","last_unstale":"2024-08-18T23:42:23.314555+0000","last_undegraded":"2024-08-18T23:42:23.314555+0000","last_fullsized":"2024-08-18T23:42:23.314555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:03:57.017488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324876+0000","last_change":"2024-08-18T23:41:30.137578+0000","last_active":"2024-08-18T23:42:23.324876+0000","last_peered":"2024-08-18T23:42:23.324876+0000","last_clean":"2024-08-18T23:42:23.324876+0000","last_became_active":"2024-08-18T23:41:30.137246+0000","last_became_peered":"2024-08-18T23:41:30.137246+0000","last_unstale":"2024-08-18T23:42:23.324876+0000","last_undegraded":"2024-08-18T23:42:23.324876+0000","last_fullsized":"2024-08-18T23:42:23.324876+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:48:54.839432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318001+0000","last_change":"2024-08-18T23:41:39.186360+0000","last_active":"2024-08-18T23:42:23.318001+0000","last_peered":"2024-08-18T23:42:23.318001+0000","last_clean":"2024-08-18T23:42:23.318001+0000","last_became_active":"2024-08-18T23:41:39.185808+0000","last_became_peered":"2024-08-18T23:41:39.185808+0000","last_unstale":"2024-08-18T23:42:23.318001+0000","last_undegraded":"2024-08-18T23:42:23.318001+0000","last_fullsized":"2024-08-18T23:42:23.318001+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:53.891061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558281+0000","last_change":"2024-08-18T23:41:32.146070+0000","last_active":"2024-08-18T23:42:23.558281+0000","last_peered":"2024-08-18T23:42:23.558281+0000","last_clean":"2024-08-18T23:42:23.558281+0000","last_became_active":"2024-08-18T23:41:32.145640+0000","last_became_peered":"2024-08-18T23:41:32.145640+0000","last_unstale":"2024-08-18T23:42:23.558281+0000","last_undegraded":"2024-08-18T23:42:23.558281+0000","last_fullsized":"2024-08-18T23:42:23.558281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:56:21.368762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319576+0000","last_change":"2024-08-18T23:41:30.138565+0000","last_active":"2024-08-18T23:42:23.319576+0000","last_peered":"2024-08-18T23:42:23.319576+0000","last_clean":"2024-08-18T23:42:23.319576+0000","last_became_active":"2024-08-18T23:41:30.138210+0000","last_became_peered":"2024-08-18T23:41:30.138210+0000","last_unstale":"2024-08-18T23:42:23.319576+0000","last_undegraded":"2024-08-18T23:42:23.319576+0000","last_fullsized":"2024-08-18T23:42:23.319576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:42:14.928884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323149+0000","last_change":"2024-08-18T23:41:39.680656+0000","last_active":"2024-08-18T23:42:23.323149+0000","last_peered":"2024-08-18T23:42:23.323149+0000","last_clean":"2024-08-18T23:42:23.323149+0000","last_became_active":"2024-08-18T23:41:39.680203+0000","last_became_peered":"2024-08-18T23:41:39.680203+0000","last_unstale":"2024-08-18T23:42:23.323149+0000","last_undegraded":"2024-08-18T23:42:23.323149+0000","last_fullsized":"2024-08-18T23:42:23.323149+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:55:43.138869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558246+0000","last_change":"2024-08-18T23:41:32.130222+0000","last_active":"2024-08-18T23:42:23.558246+0000","last_peered":"2024-08-18T23:42:23.558246+0000","last_clean":"2024-08-18T23:42:23.558246+0000","last_became_active":"2024-08-18T23:41:32.129844+0000","last_became_peered":"2024-08-18T23:41:32.129844+0000","last_unstale":"2024-08-18T23:42:23.558246+0000","last_undegraded":"2024-08-18T23:42:23.558246+0000","last_fullsized":"2024-08-18T23:42:23.558246+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:39:21.106343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316823+0000","last_change":"2024-08-18T23:41:30.123142+0000","last_active":"2024-08-18T23:42:23.316823+0000","last_peered":"2024-08-18T23:42:23.316823+0000","last_clean":"2024-08-18T23:42:23.316823+0000","last_became_active":"2024-08-18T23:41:30.121645+0000","last_became_peered":"2024-08-18T23:41:30.121645+0000","last_unstale":"2024-08-18T23:42:23.316823+0000","last_undegraded":"2024-08-18T23:42:23.316823+0000","last_fullsized":"2024-08-18T23:42:23.316823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:07:24.912259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314454+0000","last_change":"2024-08-18T23:41:39.186033+0000","last_active":"2024-08-18T23:42:23.314454+0000","last_peered":"2024-08-18T23:42:23.314454+0000","last_clean":"2024-08-18T23:42:23.314454+0000","last_became_active":"2024-08-18T23:41:39.185451+0000","last_became_peered":"2024-08-18T23:41:39.185451+0000","last_unstale":"2024-08-18T23:42:23.314454+0000","last_undegraded":"2024-08-18T23:42:23.314454+0000","last_fullsized":"2024-08-18T23:42:23.314454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:52:13.556760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320803+0000","last_change":"2024-08-18T23:41:32.141065+0000","last_active":"2024-08-18T23:42:23.320803+0000","last_peered":"2024-08-18T23:42:23.320803+0000","last_clean":"2024-08-18T23:42:23.320803+0000","last_became_active":"2024-08-18T23:41:32.140732+0000","last_became_peered":"2024-08-18T23:41:32.140732+0000","last_unstale":"2024-08-18T23:42:23.320803+0000","last_undegraded":"2024-08-18T23:42:23.320803+0000","last_fullsized":"2024-08-18T23:42:23.320803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:08:32.287835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:32.307011+0000","last_change":"2024-08-18T23:41:30.136838+0000","last_active":"2024-08-18T23:42:32.307011+0000","last_peered":"2024-08-18T23:42:32.307011+0000","last_clean":"2024-08-18T23:42:32.307011+0000","last_became_active":"2024-08-18T23:41:30.136523+0000","last_became_peered":"2024-08-18T23:41:30.136523+0000","last_unstale":"2024-08-18T23:42:32.307011+0000","last_undegraded":"2024-08-18T23:42:32.307011+0000","last_fullsized":"2024-08-18T23:42:32.307011+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:56:17.750535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314559+0000","last_change":"2024-08-18T23:41:39.680314+0000","last_active":"2024-08-18T23:42:23.314559+0000","last_peered":"2024-08-18T23:42:23.314559+0000","last_clean":"2024-08-18T23:42:23.314559+0000","last_became_active":"2024-08-18T23:41:39.679920+0000","last_became_peered":"2024-08-18T23:41:39.679920+0000","last_unstale":"2024-08-18T23:42:23.314559+0000","last_undegraded":"2024-08-18T23:42:23.314559+0000","last_fullsized":"2024-08-18T23:42:23.314559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:26:29.866081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314066+0000","last_change":"2024-08-18T23:41:32.139752+0000","last_active":"2024-08-18T23:42:23.314066+0000","last_peered":"2024-08-18T23:42:23.314066+0000","last_clean":"2024-08-18T23:42:23.314066+0000","last_became_active":"2024-08-18T23:41:32.139451+0000","last_became_peered":"2024-08-18T23:41:32.139451+0000","last_unstale":"2024-08-18T23:42:23.314066+0000","last_undegraded":"2024-08-18T23:42:23.314066+0000","last_fullsized":"2024-08-18T23:42:23.314066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:15:13.390529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316737+0000","last_change":"2024-08-18T23:41:30.126513+0000","last_active":"2024-08-18T23:42:23.316737+0000","last_peered":"2024-08-18T23:42:23.316737+0000","last_clean":"2024-08-18T23:42:23.316737+0000","last_became_active":"2024-08-18T23:41:30.126046+0000","last_became_peered":"2024-08-18T23:41:30.126046+0000","last_unstale":"2024-08-18T23:42:23.316737+0000","last_undegraded":"2024-08-18T23:42:23.316737+0000","last_fullsized":"2024-08-18T23:42:23.316737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:49:16.565565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323372+0000","last_change":"2024-08-18T23:41:39.681006+0000","last_active":"2024-08-18T23:42:23.323372+0000","last_peered":"2024-08-18T23:42:23.323372+0000","last_clean":"2024-08-18T23:42:23.323372+0000","last_became_active":"2024-08-18T23:41:39.680409+0000","last_became_peered":"2024-08-18T23:41:39.680409+0000","last_unstale":"2024-08-18T23:42:23.323372+0000","last_undegraded":"2024-08-18T23:42:23.323372+0000","last_fullsized":"2024-08-18T23:42:23.323372+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:47.778866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557596+0000","last_change":"2024-08-18T23:41:32.147545+0000","last_active":"2024-08-18T23:42:23.557596+0000","last_peered":"2024-08-18T23:42:23.557596+0000","last_clean":"2024-08-18T23:42:23.557596+0000","last_became_active":"2024-08-18T23:41:32.147238+0000","last_became_peered":"2024-08-18T23:41:32.147238+0000","last_unstale":"2024-08-18T23:42:23.557596+0000","last_undegraded":"2024-08-18T23:42:23.557596+0000","last_fullsized":"2024-08-18T23:42:23.557596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:12:13.589535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319674+0000","last_change":"2024-08-18T23:41:30.120650+0000","last_active":"2024-08-18T23:42:23.319674+0000","last_peered":"2024-08-18T23:42:23.319674+0000","last_clean":"2024-08-18T23:42:23.319674+0000","last_became_active":"2024-08-18T23:41:30.120196+0000","last_became_peered":"2024-08-18T23:41:30.120196+0000","last_unstale":"2024-08-18T23:42:23.319674+0000","last_undegraded":"2024-08-18T23:42:23.319674+0000","last_fullsized":"2024-08-18T23:42:23.319674+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:40:23.370088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317120+0000","last_change":"2024-08-18T23:40:22.013519+0000","last_active":"2024-08-18T23:42:23.317120+0000","last_peered":"2024-08-18T23:42:23.317120+0000","last_clean":"2024-08-18T23:42:23.317120+0000","last_became_active":"2024-08-18T23:40:22.012986+0000","last_became_peered":"2024-08-18T23:40:22.012986+0000","last_unstale":"2024-08-18T23:42:23.317120+0000","last_undegraded":"2024-08-18T23:42:23.317120+0000","last_fullsized":"2024-08-18T23:42:23.317120+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_clean_scrub_stamp":"2024-08-18T23:37:22.388424+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:19.874356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.332800+0000","last_change":"2024-08-18T23:42:23.332800+0000","last_active":"2024-08-18T23:42:23.332800+0000","last_peered":"2024-08-18T23:42:23.332800+0000","last_clean":"2024-08-18T23:42:23.332800+0000","last_became_active":"2024-08-18T23:42:23.326441+0000","last_became_peered":"2024-08-18T23:42:23.326441+0000","last_unstale":"2024-08-18T23:42:23.332800+0000","last_undegraded":"2024-08-18T23:42:23.332800+0000","last_fullsized":"2024-08-18T23:42:23.332800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:13:58.243518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317150+0000","last_change":"2024-08-18T23:41:32.139907+0000","last_active":"2024-08-18T23:42:23.317150+0000","last_peered":"2024-08-18T23:42:23.317150+0000","last_clean":"2024-08-18T23:42:23.317150+0000","last_became_active":"2024-08-18T23:41:32.137956+0000","last_became_peered":"2024-08-18T23:41:32.137956+0000","last_unstale":"2024-08-18T23:42:23.317150+0000","last_undegraded":"2024-08-18T23:42:23.317150+0000","last_fullsized":"2024-08-18T23:42:23.317150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:10:13.223328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.330746+0000","last_change":"2024-08-18T23:42:23.330062+0000","last_active":"2024-08-18T23:42:23.330746+0000","last_peered":"2024-08-18T23:42:23.330746+0000","last_clean":"2024-08-18T23:42:23.330746+0000","last_became_active":"2024-08-18T23:42:23.323196+0000","last_became_peered":"2024-08-18T23:42:23.323196+0000","last_unstale":"2024-08-18T23:42:23.330746+0000","last_undegraded":"2024-08-18T23:42:23.330746+0000","last_fullsized":"2024-08-18T23:42:23.330746+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:53:29.059542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315093+0000","last_change":"2024-08-18T23:41:39.185917+0000","last_active":"2024-08-18T23:42:23.315093+0000","last_peered":"2024-08-18T23:42:23.315093+0000","last_clean":"2024-08-18T23:42:23.315093+0000","last_became_active":"2024-08-18T23:41:39.185427+0000","last_became_peered":"2024-08-18T23:41:39.185427+0000","last_unstale":"2024-08-18T23:42:23.315093+0000","last_undegraded":"2024-08-18T23:42:23.315093+0000","last_fullsized":"2024-08-18T23:42:23.315093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:52:13.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317336+0000","last_change":"2024-08-18T23:41:32.147530+0000","last_active":"2024-08-18T23:42:23.317336+0000","last_peered":"2024-08-18T23:42:23.317336+0000","last_clean":"2024-08-18T23:42:23.317336+0000","last_became_active":"2024-08-18T23:41:32.147216+0000","last_became_peered":"2024-08-18T23:41:32.147216+0000","last_unstale":"2024-08-18T23:42:23.317336+0000","last_undegraded":"2024-08-18T23:42:23.317336+0000","last_fullsized":"2024-08-18T23:42:23.317336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:04:06.421490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323538+0000","last_change":"2024-08-18T23:41:30.121307+0000","last_active":"2024-08-18T23:42:23.323538+0000","last_peered":"2024-08-18T23:42:23.323538+0000","last_clean":"2024-08-18T23:42:23.323538+0000","last_became_active":"2024-08-18T23:41:30.120932+0000","last_became_peered":"2024-08-18T23:41:30.120932+0000","last_unstale":"2024-08-18T23:42:23.323538+0000","last_undegraded":"2024-08-18T23:42:23.323538+0000","last_fullsized":"2024-08-18T23:42:23.323538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:06:07.742880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316371+0000","last_change":"2024-08-18T23:41:39.184647+0000","last_active":"2024-08-18T23:42:23.316371+0000","last_peered":"2024-08-18T23:42:23.316371+0000","last_clean":"2024-08-18T23:42:23.316371+0000","last_became_active":"2024-08-18T23:41:39.184251+0000","last_became_peered":"2024-08-18T23:41:39.184251+0000","last_unstale":"2024-08-18T23:42:23.316371+0000","last_undegraded":"2024-08-18T23:42:23.316371+0000","last_fullsized":"2024-08-18T23:42:23.316371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:55.200801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317327+0000","last_change":"2024-08-18T23:41:32.137007+0000","last_active":"2024-08-18T23:42:23.317327+0000","last_peered":"2024-08-18T23:42:23.317327+0000","last_clean":"2024-08-18T23:42:23.317327+0000","last_became_active":"2024-08-18T23:41:32.136684+0000","last_became_peered":"2024-08-18T23:41:32.136684+0000","last_unstale":"2024-08-18T23:42:23.317327+0000","last_undegraded":"2024-08-18T23:42:23.317327+0000","last_fullsized":"2024-08-18T23:42:23.317327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:21:26.584303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314514+0000","last_change":"2024-08-18T23:41:30.121183+0000","last_active":"2024-08-18T23:42:23.314514+0000","last_peered":"2024-08-18T23:42:23.314514+0000","last_clean":"2024-08-18T23:42:23.314514+0000","last_became_active":"2024-08-18T23:41:30.120847+0000","last_became_peered":"2024-08-18T23:41:30.120847+0000","last_unstale":"2024-08-18T23:42:23.314514+0000","last_undegraded":"2024-08-18T23:42:23.314514+0000","last_fullsized":"2024-08-18T23:42:23.314514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:51.025821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320528+0000","last_change":"2024-08-18T23:41:39.183375+0000","last_active":"2024-08-18T23:42:23.320528+0000","last_peered":"2024-08-18T23:42:23.320528+0000","last_clean":"2024-08-18T23:42:23.320528+0000","last_became_active":"2024-08-18T23:41:39.182339+0000","last_became_peered":"2024-08-18T23:41:39.182339+0000","last_unstale":"2024-08-18T23:42:23.320528+0000","last_undegraded":"2024-08-18T23:42:23.320528+0000","last_fullsized":"2024-08-18T23:42:23.320528+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:26.119056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318323+0000","last_change":"2024-08-18T23:41:32.140748+0000","last_active":"2024-08-18T23:42:23.318323+0000","last_peered":"2024-08-18T23:42:23.318323+0000","last_clean":"2024-08-18T23:42:23.318323+0000","last_became_active":"2024-08-18T23:41:32.140306+0000","last_became_peered":"2024-08-18T23:41:32.140306+0000","last_unstale":"2024-08-18T23:42:23.318323+0000","last_undegraded":"2024-08-18T23:42:23.318323+0000","last_fullsized":"2024-08-18T23:42:23.318323+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:23:02.371529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324632+0000","last_change":"2024-08-18T23:41:30.121617+0000","last_active":"2024-08-18T23:42:23.324632+0000","last_peered":"2024-08-18T23:42:23.324632+0000","last_clean":"2024-08-18T23:42:23.324632+0000","last_became_active":"2024-08-18T23:41:30.121257+0000","last_became_peered":"2024-08-18T23:41:30.121257+0000","last_unstale":"2024-08-18T23:42:23.324632+0000","last_undegraded":"2024-08-18T23:42:23.324632+0000","last_fullsized":"2024-08-18T23:42:23.324632+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:23:42.091553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320637+0000","last_change":"2024-08-18T23:41:39.183833+0000","last_active":"2024-08-18T23:42:23.320637+0000","last_peered":"2024-08-18T23:42:23.320637+0000","last_clean":"2024-08-18T23:42:23.320637+0000","last_became_active":"2024-08-18T23:41:39.183176+0000","last_became_peered":"2024-08-18T23:41:39.183176+0000","last_unstale":"2024-08-18T23:42:23.320637+0000","last_undegraded":"2024-08-18T23:42:23.320637+0000","last_fullsized":"2024-08-18T23:42:23.320637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:36:09.827378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322357+0000","last_change":"2024-08-18T23:41:32.147507+0000","last_active":"2024-08-18T23:42:23.322357+0000","last_peered":"2024-08-18T23:42:23.322357+0000","last_clean":"2024-08-18T23:42:23.322357+0000","last_became_active":"2024-08-18T23:41:32.146042+0000","last_became_peered":"2024-08-18T23:41:32.146042+0000","last_unstale":"2024-08-18T23:42:23.322357+0000","last_undegraded":"2024-08-18T23:42:23.322357+0000","last_fullsized":"2024-08-18T23:42:23.322357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:48.941442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323009+0000","last_change":"2024-08-18T23:41:30.137755+0000","last_active":"2024-08-18T23:42:23.323009+0000","last_peered":"2024-08-18T23:42:23.323009+0000","last_clean":"2024-08-18T23:42:23.323009+0000","last_became_active":"2024-08-18T23:41:30.137194+0000","last_became_peered":"2024-08-18T23:41:30.137194+0000","last_unstale":"2024-08-18T23:42:23.323009+0000","last_undegraded":"2024-08-18T23:42:23.323009+0000","last_fullsized":"2024-08-18T23:42:23.323009+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:15:49.681405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314506+0000","last_change":"2024-08-18T23:41:39.183986+0000","last_active":"2024-08-18T23:42:23.314506+0000","last_peered":"2024-08-18T23:42:23.314506+0000","last_clean":"2024-08-18T23:42:23.314506+0000","last_became_active":"2024-08-18T23:41:39.183384+0000","last_became_peered":"2024-08-18T23:41:39.183384+0000","last_unstale":"2024-08-18T23:42:23.314506+0000","last_undegraded":"2024-08-18T23:42:23.314506+0000","last_fullsized":"2024-08-18T23:42:23.314506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:04:28.920804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316556+0000","last_change":"2024-08-18T23:41:32.134729+0000","last_active":"2024-08-18T23:42:23.316556+0000","last_peered":"2024-08-18T23:42:23.316556+0000","last_clean":"2024-08-18T23:42:23.316556+0000","last_became_active":"2024-08-18T23:41:32.134366+0000","last_became_peered":"2024-08-18T23:41:32.134366+0000","last_unstale":"2024-08-18T23:42:23.316556+0000","last_undegraded":"2024-08-18T23:42:23.316556+0000","last_fullsized":"2024-08-18T23:42:23.316556+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:38.610561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":68,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:33.309155+0000","last_change":"2024-08-18T23:41:30.126443+0000","last_active":"2024-08-18T23:42:33.309155+0000","last_peered":"2024-08-18T23:42:33.309155+0000","last_clean":"2024-08-18T23:42:33.309155+0000","last_became_active":"2024-08-18T23:41:30.124264+0000","last_became_peered":"2024-08-18T23:41:30.124264+0000","last_unstale":"2024-08-18T23:42:33.309155+0000","last_undegraded":"2024-08-18T23:42:33.309155+0000","last_fullsized":"2024-08-18T23:42:33.309155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:43.727114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319340+0000","last_change":"2024-08-18T23:41:39.192164+0000","last_active":"2024-08-18T23:42:23.319340+0000","last_peered":"2024-08-18T23:42:23.319340+0000","last_clean":"2024-08-18T23:42:23.319340+0000","last_became_active":"2024-08-18T23:41:39.191833+0000","last_became_peered":"2024-08-18T23:41:39.191833+0000","last_unstale":"2024-08-18T23:42:23.319340+0000","last_undegraded":"2024-08-18T23:42:23.319340+0000","last_fullsized":"2024-08-18T23:42:23.319340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:33:50.243930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320662+0000","last_change":"2024-08-18T23:42:23.320662+0000","last_active":"2024-08-18T23:42:23.320662+0000","last_peered":"2024-08-18T23:42:23.320662+0000","last_clean":"2024-08-18T23:42:23.320662+0000","last_became_active":"2024-08-18T23:42:23.320249+0000","last_became_peered":"2024-08-18T23:42:23.320249+0000","last_unstale":"2024-08-18T23:42:23.320662+0000","last_undegraded":"2024-08-18T23:42:23.320662+0000","last_fullsized":"2024-08-18T23:42:23.320662+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:03:17.121915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322332+0000","last_change":"2024-08-18T23:41:30.124071+0000","last_active":"2024-08-18T23:42:23.322332+0000","last_peered":"2024-08-18T23:42:23.322332+0000","last_clean":"2024-08-18T23:42:23.322332+0000","last_became_active":"2024-08-18T23:41:30.123118+0000","last_became_peered":"2024-08-18T23:41:30.123118+0000","last_unstale":"2024-08-18T23:42:23.322332+0000","last_undegraded":"2024-08-18T23:42:23.322332+0000","last_fullsized":"2024-08-18T23:42:23.322332+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:21:39.015187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319331+0000","last_change":"2024-08-18T23:41:39.184038+0000","last_active":"2024-08-18T23:42:23.319331+0000","last_peered":"2024-08-18T23:42:23.319331+0000","last_clean":"2024-08-18T23:42:23.319331+0000","last_became_active":"2024-08-18T23:41:39.183611+0000","last_became_peered":"2024-08-18T23:41:39.183611+0000","last_unstale":"2024-08-18T23:42:23.319331+0000","last_undegraded":"2024-08-18T23:42:23.319331+0000","last_fullsized":"2024-08-18T23:42:23.319331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:32:26.029151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314264+0000","last_change":"2024-08-18T23:41:32.143631+0000","last_active":"2024-08-18T23:42:23.314264+0000","last_peered":"2024-08-18T23:42:23.314264+0000","last_clean":"2024-08-18T23:42:23.314264+0000","last_became_active":"2024-08-18T23:41:32.143350+0000","last_became_peered":"2024-08-18T23:41:32.143350+0000","last_unstale":"2024-08-18T23:42:23.314264+0000","last_undegraded":"2024-08-18T23:42:23.314264+0000","last_fullsized":"2024-08-18T23:42:23.314264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:41:23.713467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319866+0000","last_change":"2024-08-18T23:41:30.123383+0000","last_active":"2024-08-18T23:42:23.319866+0000","last_peered":"2024-08-18T23:42:23.319866+0000","last_clean":"2024-08-18T23:42:23.319866+0000","last_became_active":"2024-08-18T23:41:30.123037+0000","last_became_peered":"2024-08-18T23:41:30.123037+0000","last_unstale":"2024-08-18T23:42:23.319866+0000","last_undegraded":"2024-08-18T23:42:23.319866+0000","last_fullsized":"2024-08-18T23:42:23.319866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:27.702478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323158+0000","last_change":"2024-08-18T23:41:39.184302+0000","last_active":"2024-08-18T23:42:23.323158+0000","last_peered":"2024-08-18T23:42:23.323158+0000","last_clean":"2024-08-18T23:42:23.323158+0000","last_became_active":"2024-08-18T23:41:39.183172+0000","last_became_peered":"2024-08-18T23:41:39.183172+0000","last_unstale":"2024-08-18T23:42:23.323158+0000","last_undegraded":"2024-08-18T23:42:23.323158+0000","last_fullsized":"2024-08-18T23:42:23.323158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:23:26.655865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324579+0000","last_change":"2024-08-18T23:41:32.137426+0000","last_active":"2024-08-18T23:42:23.324579+0000","last_peered":"2024-08-18T23:42:23.324579+0000","last_clean":"2024-08-18T23:42:23.324579+0000","last_became_active":"2024-08-18T23:41:32.137106+0000","last_became_peered":"2024-08-18T23:41:32.137106+0000","last_unstale":"2024-08-18T23:42:23.324579+0000","last_undegraded":"2024-08-18T23:42:23.324579+0000","last_fullsized":"2024-08-18T23:42:23.324579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:34:51.889663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'32","reported_seq":129,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:34.310306+0000","last_change":"2024-08-18T23:41:30.124320+0000","last_active":"2024-08-18T23:42:34.310306+0000","last_peered":"2024-08-18T23:42:34.310306+0000","last_clean":"2024-08-18T23:42:34.310306+0000","last_became_active":"2024-08-18T23:41:30.123979+0000","last_became_peered":"2024-08-18T23:41:30.123979+0000","last_unstale":"2024-08-18T23:42:34.310306+0000","last_undegraded":"2024-08-18T23:42:34.310306+0000","last_fullsized":"2024-08-18T23:42:34.310306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:31:03.689720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62996,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":63,"num_read_kb":93,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319063+0000","last_change":"2024-08-18T23:41:39.190481+0000","last_active":"2024-08-18T23:42:23.319063+0000","last_peered":"2024-08-18T23:42:23.319063+0000","last_clean":"2024-08-18T23:42:23.319063+0000","last_became_active":"2024-08-18T23:41:39.190076+0000","last_became_peered":"2024-08-18T23:41:39.190076+0000","last_unstale":"2024-08-18T23:42:23.319063+0000","last_undegraded":"2024-08-18T23:42:23.319063+0000","last_fullsized":"2024-08-18T23:42:23.319063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:47:39.313142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315607+0000","last_change":"2024-08-18T23:41:32.145422+0000","last_active":"2024-08-18T23:42:23.315607+0000","last_peered":"2024-08-18T23:42:23.315607+0000","last_clean":"2024-08-18T23:42:23.315607+0000","last_became_active":"2024-08-18T23:41:32.145047+0000","last_became_peered":"2024-08-18T23:41:32.145047+0000","last_unstale":"2024-08-18T23:42:23.315607+0000","last_undegraded":"2024-08-18T23:42:23.315607+0000","last_fullsized":"2024-08-18T23:42:23.315607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:31.905248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323427+0000","last_change":"2024-08-18T23:41:30.137364+0000","last_active":"2024-08-18T23:42:23.323427+0000","last_peered":"2024-08-18T23:42:23.323427+0000","last_clean":"2024-08-18T23:42:23.323427+0000","last_became_active":"2024-08-18T23:41:30.137004+0000","last_became_peered":"2024-08-18T23:41:30.137004+0000","last_unstale":"2024-08-18T23:42:23.323427+0000","last_undegraded":"2024-08-18T23:42:23.323427+0000","last_fullsized":"2024-08-18T23:42:23.323427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:20:56.392149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.326359+0000","last_change":"2024-08-18T23:42:23.326359+0000","last_active":"2024-08-18T23:42:23.326359+0000","last_peered":"2024-08-18T23:42:23.326359+0000","last_clean":"2024-08-18T23:42:23.326359+0000","last_became_active":"2024-08-18T23:42:23.325918+0000","last_became_peered":"2024-08-18T23:42:23.325918+0000","last_unstale":"2024-08-18T23:42:23.326359+0000","last_undegraded":"2024-08-18T23:42:23.326359+0000","last_fullsized":"2024-08-18T23:42:23.326359+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:01:15.476268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.325691+0000","last_change":"2024-08-18T23:42:23.325691+0000","last_active":"2024-08-18T23:42:23.325691+0000","last_peered":"2024-08-18T23:42:23.325691+0000","last_clean":"2024-08-18T23:42:23.325691+0000","last_became_active":"2024-08-18T23:42:23.325467+0000","last_became_peered":"2024-08-18T23:42:23.325467+0000","last_unstale":"2024-08-18T23:42:23.325691+0000","last_undegraded":"2024-08-18T23:42:23.325691+0000","last_fullsized":"2024-08-18T23:42:23.325691+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:10:26.288880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317472+0000","last_change":"2024-08-18T23:41:30.124430+0000","last_active":"2024-08-18T23:42:23.317472+0000","last_peered":"2024-08-18T23:42:23.317472+0000","last_clean":"2024-08-18T23:42:23.317472+0000","last_became_active":"2024-08-18T23:41:30.123626+0000","last_became_peered":"2024-08-18T23:41:30.123626+0000","last_unstale":"2024-08-18T23:42:23.317472+0000","last_undegraded":"2024-08-18T23:42:23.317472+0000","last_fullsized":"2024-08-18T23:42:23.317472+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:26:18.426010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318912+0000","last_change":"2024-08-18T23:41:39.192765+0000","last_active":"2024-08-18T23:42:23.318912+0000","last_peered":"2024-08-18T23:42:23.318912+0000","last_clean":"2024-08-18T23:42:23.318912+0000","last_became_active":"2024-08-18T23:41:39.192250+0000","last_became_peered":"2024-08-18T23:41:39.192250+0000","last_unstale":"2024-08-18T23:42:23.318912+0000","last_undegraded":"2024-08-18T23:42:23.318912+0000","last_fullsized":"2024-08-18T23:42:23.318912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:51:16.794404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316158+0000","last_change":"2024-08-18T23:41:32.141966+0000","last_active":"2024-08-18T23:42:23.316158+0000","last_peered":"2024-08-18T23:42:23.316158+0000","last_clean":"2024-08-18T23:42:23.316158+0000","last_became_active":"2024-08-18T23:41:32.141561+0000","last_became_peered":"2024-08-18T23:41:32.141561+0000","last_unstale":"2024-08-18T23:42:23.316158+0000","last_undegraded":"2024-08-18T23:42:23.316158+0000","last_fullsized":"2024-08-18T23:42:23.316158+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:20.696089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:33.308465+0000","last_change":"2024-08-18T23:41:30.121598+0000","last_active":"2024-08-18T23:42:33.308465+0000","last_peered":"2024-08-18T23:42:33.308465+0000","last_clean":"2024-08-18T23:42:33.308465+0000","last_became_active":"2024-08-18T23:41:30.120533+0000","last_became_peered":"2024-08-18T23:41:30.120533+0000","last_unstale":"2024-08-18T23:42:33.308465+0000","last_undegraded":"2024-08-18T23:42:33.308465+0000","last_fullsized":"2024-08-18T23:42:33.308465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:06:37.458272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557461+0000","last_change":"2024-08-18T23:41:39.178877+0000","last_active":"2024-08-18T23:42:23.557461+0000","last_peered":"2024-08-18T23:42:23.557461+0000","last_clean":"2024-08-18T23:42:23.557461+0000","last_became_active":"2024-08-18T23:41:39.178381+0000","last_became_peered":"2024-08-18T23:41:39.178381+0000","last_unstale":"2024-08-18T23:42:23.557461+0000","last_undegraded":"2024-08-18T23:42:23.557461+0000","last_fullsized":"2024-08-18T23:42:23.557461+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:27.275893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316665+0000","last_change":"2024-08-18T23:41:39.188381+0000","last_active":"2024-08-18T23:42:23.316665+0000","last_peered":"2024-08-18T23:42:23.316665+0000","last_clean":"2024-08-18T23:42:23.316665+0000","last_became_active":"2024-08-18T23:41:39.187983+0000","last_became_peered":"2024-08-18T23:41:39.187983+0000","last_unstale":"2024-08-18T23:42:23.316665+0000","last_undegraded":"2024-08-18T23:42:23.316665+0000","last_fullsized":"2024-08-18T23:42:23.316665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:04:45.460576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319101+0000","last_change":"2024-08-18T23:41:32.144637+0000","last_active":"2024-08-18T23:42:23.319101+0000","last_peered":"2024-08-18T23:42:23.319101+0000","last_clean":"2024-08-18T23:42:23.319101+0000","last_became_active":"2024-08-18T23:41:32.144329+0000","last_became_peered":"2024-08-18T23:41:32.144329+0000","last_unstale":"2024-08-18T23:42:23.319101+0000","last_undegraded":"2024-08-18T23:42:23.319101+0000","last_fullsized":"2024-08-18T23:42:23.319101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:29:39.791264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320980+0000","last_change":"2024-08-18T23:41:30.122325+0000","last_active":"2024-08-18T23:42:23.320980+0000","last_peered":"2024-08-18T23:42:23.320980+0000","last_clean":"2024-08-18T23:42:23.320980+0000","last_became_active":"2024-08-18T23:41:30.121480+0000","last_became_peered":"2024-08-18T23:41:30.121480+0000","last_unstale":"2024-08-18T23:42:23.320980+0000","last_undegraded":"2024-08-18T23:42:23.320980+0000","last_fullsized":"2024-08-18T23:42:23.320980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:32:12.810650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.560632+0000","last_change":"2024-08-18T23:42:23.560632+0000","last_active":"2024-08-18T23:42:23.560632+0000","last_peered":"2024-08-18T23:42:23.560632+0000","last_clean":"2024-08-18T23:42:23.560632+0000","last_became_active":"2024-08-18T23:42:23.559920+0000","last_became_peered":"2024-08-18T23:42:23.559920+0000","last_unstale":"2024-08-18T23:42:23.560632+0000","last_undegraded":"2024-08-18T23:42:23.560632+0000","last_fullsized":"2024-08-18T23:42:23.560632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:01:59.580843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557464+0000","last_change":"2024-08-18T23:41:32.145545+0000","last_active":"2024-08-18T23:42:23.557464+0000","last_peered":"2024-08-18T23:42:23.557464+0000","last_clean":"2024-08-18T23:42:23.557464+0000","last_became_active":"2024-08-18T23:41:32.145183+0000","last_became_peered":"2024-08-18T23:41:32.145183+0000","last_unstale":"2024-08-18T23:42:23.557464+0000","last_undegraded":"2024-08-18T23:42:23.557464+0000","last_fullsized":"2024-08-18T23:42:23.557464+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:03:04.111658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314074+0000","last_change":"2024-08-18T23:41:30.118776+0000","last_active":"2024-08-18T23:42:23.314074+0000","last_peered":"2024-08-18T23:42:23.314074+0000","last_clean":"2024-08-18T23:42:23.314074+0000","last_became_active":"2024-08-18T23:41:30.118436+0000","last_became_peered":"2024-08-18T23:41:30.118436+0000","last_unstale":"2024-08-18T23:42:23.314074+0000","last_undegraded":"2024-08-18T23:42:23.314074+0000","last_fullsized":"2024-08-18T23:42:23.314074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:13.980927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558143+0000","last_change":"2024-08-18T23:41:30.125201+0000","last_active":"2024-08-18T23:42:23.558143+0000","last_peered":"2024-08-18T23:42:23.558143+0000","last_clean":"2024-08-18T23:42:23.558143+0000","last_became_active":"2024-08-18T23:41:30.124894+0000","last_became_peered":"2024-08-18T23:41:30.124894+0000","last_unstale":"2024-08-18T23:42:23.558143+0000","last_undegraded":"2024-08-18T23:42:23.558143+0000","last_fullsized":"2024-08-18T23:42:23.558143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:30:17.721829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322555+0000","last_change":"2024-08-18T23:41:32.145452+0000","last_active":"2024-08-18T23:42:23.322555+0000","last_peered":"2024-08-18T23:42:23.322555+0000","last_clean":"2024-08-18T23:42:23.322555+0000","last_became_active":"2024-08-18T23:41:32.145151+0000","last_became_peered":"2024-08-18T23:41:32.145151+0000","last_unstale":"2024-08-18T23:42:23.322555+0000","last_undegraded":"2024-08-18T23:42:23.322555+0000","last_fullsized":"2024-08-18T23:42:23.322555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:08:08.713154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319503+0000","last_change":"2024-08-18T23:41:39.193334+0000","last_active":"2024-08-18T23:42:23.319503+0000","last_peered":"2024-08-18T23:42:23.319503+0000","last_clean":"2024-08-18T23:42:23.319503+0000","last_became_active":"2024-08-18T23:41:39.192924+0000","last_became_peered":"2024-08-18T23:41:39.192924+0000","last_unstale":"2024-08-18T23:42:23.319503+0000","last_undegraded":"2024-08-18T23:42:23.319503+0000","last_fullsized":"2024-08-18T23:42:23.319503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:07.016300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":67,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:30.305213+0000","last_change":"2024-08-18T23:41:30.125217+0000","last_active":"2024-08-18T23:42:30.305213+0000","last_peered":"2024-08-18T23:42:30.305213+0000","last_clean":"2024-08-18T23:42:30.305213+0000","last_became_active":"2024-08-18T23:41:30.124889+0000","last_became_peered":"2024-08-18T23:41:30.124889+0000","last_unstale":"2024-08-18T23:42:30.305213+0000","last_undegraded":"2024-08-18T23:42:30.305213+0000","last_fullsized":"2024-08-18T23:42:30.305213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:43.389537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323355+0000","last_change":"2024-08-18T23:41:32.142585+0000","last_active":"2024-08-18T23:42:23.323355+0000","last_peered":"2024-08-18T23:42:23.323355+0000","last_clean":"2024-08-18T23:42:23.323355+0000","last_became_active":"2024-08-18T23:41:32.142207+0000","last_became_peered":"2024-08-18T23:41:32.142207+0000","last_unstale":"2024-08-18T23:42:23.323355+0000","last_undegraded":"2024-08-18T23:42:23.323355+0000","last_fullsized":"2024-08-18T23:42:23.323355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:16:58.755444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.321952+0000","last_change":"2024-08-18T23:41:39.185618+0000","last_active":"2024-08-18T23:42:23.321952+0000","last_peered":"2024-08-18T23:42:23.321952+0000","last_clean":"2024-08-18T23:42:23.321952+0000","last_became_active":"2024-08-18T23:41:39.185222+0000","last_became_peered":"2024-08-18T23:41:39.185222+0000","last_unstale":"2024-08-18T23:42:23.321952+0000","last_undegraded":"2024-08-18T23:42:23.321952+0000","last_fullsized":"2024-08-18T23:42:23.321952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:01:27.290998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318594+0000","last_change":"2024-08-18T23:41:32.140719+0000","last_active":"2024-08-18T23:42:23.318594+0000","last_peered":"2024-08-18T23:42:23.318594+0000","last_clean":"2024-08-18T23:42:23.318594+0000","last_became_active":"2024-08-18T23:41:32.140131+0000","last_became_peered":"2024-08-18T23:41:32.140131+0000","last_unstale":"2024-08-18T23:42:23.318594+0000","last_undegraded":"2024-08-18T23:42:23.318594+0000","last_fullsized":"2024-08-18T23:42:23.318594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:48:26.778094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.326683+0000","last_change":"2024-08-18T23:42:23.326683+0000","last_active":"2024-08-18T23:42:23.326683+0000","last_peered":"2024-08-18T23:42:23.326683+0000","last_clean":"2024-08-18T23:42:23.326683+0000","last_became_active":"2024-08-18T23:42:23.326309+0000","last_became_peered":"2024-08-18T23:42:23.326309+0000","last_unstale":"2024-08-18T23:42:23.326683+0000","last_undegraded":"2024-08-18T23:42:23.326683+0000","last_fullsized":"2024-08-18T23:42:23.326683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:14.320894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.321522+0000","last_change":"2024-08-18T23:41:32.149522+0000","last_active":"2024-08-18T23:42:23.321522+0000","last_peered":"2024-08-18T23:42:23.321522+0000","last_clean":"2024-08-18T23:42:23.321522+0000","last_became_active":"2024-08-18T23:41:32.149182+0000","last_became_peered":"2024-08-18T23:41:32.149182+0000","last_unstale":"2024-08-18T23:42:23.321522+0000","last_undegraded":"2024-08-18T23:42:23.321522+0000","last_fullsized":"2024-08-18T23:42:23.321522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:30:16.945045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316398+0000","last_change":"2024-08-18T23:41:32.149853+0000","last_active":"2024-08-18T23:42:23.316398+0000","last_peered":"2024-08-18T23:42:23.316398+0000","last_clean":"2024-08-18T23:42:23.316398+0000","last_became_active":"2024-08-18T23:41:32.141907+0000","last_became_peered":"2024-08-18T23:41:32.141907+0000","last_unstale":"2024-08-18T23:42:23.316398+0000","last_undegraded":"2024-08-18T23:42:23.316398+0000","last_fullsized":"2024-08-18T23:42:23.316398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:24:37.323567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314554+0000","last_change":"2024-08-18T23:41:32.136179+0000","last_active":"2024-08-18T23:42:23.314554+0000","last_peered":"2024-08-18T23:42:23.314554+0000","last_clean":"2024-08-18T23:42:23.314554+0000","last_became_active":"2024-08-18T23:41:32.135939+0000","last_became_peered":"2024-08-18T23:41:32.135939+0000","last_unstale":"2024-08-18T23:42:23.314554+0000","last_undegraded":"2024-08-18T23:42:23.314554+0000","last_fullsized":"2024-08-18T23:42:23.314554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:15.392061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318883+0000","last_change":"2024-08-18T23:41:32.137509+0000","last_active":"2024-08-18T23:42:23.318883+0000","last_peered":"2024-08-18T23:42:23.318883+0000","last_clean":"2024-08-18T23:42:23.318883+0000","last_became_active":"2024-08-18T23:41:32.136380+0000","last_became_peered":"2024-08-18T23:41:32.136380+0000","last_unstale":"2024-08-18T23:42:23.318883+0000","last_undegraded":"2024-08-18T23:42:23.318883+0000","last_fullsized":"2024-08-18T23:42:23.318883+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:49.776225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322689+0000","last_change":"2024-08-18T23:41:32.139271+0000","last_active":"2024-08-18T23:42:23.322689+0000","last_peered":"2024-08-18T23:42:23.322689+0000","last_clean":"2024-08-18T23:42:23.322689+0000","last_became_active":"2024-08-18T23:41:32.138868+0000","last_became_peered":"2024-08-18T23:41:32.138868+0000","last_unstale":"2024-08-18T23:42:23.322689+0000","last_undegraded":"2024-08-18T23:42:23.322689+0000","last_fullsized":"2024-08-18T23:42:23.322689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:49:27.890857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557875+0000","last_change":"2024-08-18T23:41:32.144460+0000","last_active":"2024-08-18T23:42:23.557875+0000","last_peered":"2024-08-18T23:42:23.557875+0000","last_clean":"2024-08-18T23:42:23.557875+0000","last_became_active":"2024-08-18T23:41:32.144178+0000","last_became_peered":"2024-08-18T23:41:32.144178+0000","last_unstale":"2024-08-18T23:42:23.557875+0000","last_undegraded":"2024-08-18T23:42:23.557875+0000","last_fullsized":"2024-08-18T23:42:23.557875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:56.252215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320708+0000","last_change":"2024-08-18T23:41:32.147973+0000","last_active":"2024-08-18T23:42:23.320708+0000","last_peered":"2024-08-18T23:42:23.320708+0000","last_clean":"2024-08-18T23:42:23.320708+0000","last_became_active":"2024-08-18T23:41:32.147621+0000","last_became_peered":"2024-08-18T23:41:32.147621+0000","last_unstale":"2024-08-18T23:42:23.320708+0000","last_undegraded":"2024-08-18T23:42:23.320708+0000","last_fullsized":"2024-08-18T23:42:23.320708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:14.176569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557525+0000","last_change":"2024-08-18T23:41:32.146029+0000","last_active":"2024-08-18T23:42:23.557525+0000","last_peered":"2024-08-18T23:42:23.557525+0000","last_clean":"2024-08-18T23:42:23.557525+0000","last_became_active":"2024-08-18T23:41:32.145584+0000","last_became_peered":"2024-08-18T23:41:32.145584+0000","last_unstale":"2024-08-18T23:42:23.557525+0000","last_undegraded":"2024-08-18T23:42:23.557525+0000","last_fullsized":"2024-08-18T23:42:23.557525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:25:33.884906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315440+0000","last_change":"2024-08-18T23:41:32.145950+0000","last_active":"2024-08-18T23:42:23.315440+0000","last_peered":"2024-08-18T23:42:23.315440+0000","last_clean":"2024-08-18T23:42:23.315440+0000","last_became_active":"2024-08-18T23:41:32.145618+0000","last_became_peered":"2024-08-18T23:41:32.145618+0000","last_unstale":"2024-08-18T23:42:23.315440+0000","last_undegraded":"2024-08-18T23:42:23.315440+0000","last_fullsized":"2024-08-18T23:42:23.315440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:05:58.889240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314395+0000","last_change":"2024-08-18T23:41:32.130684+0000","last_active":"2024-08-18T23:42:23.314395+0000","last_peered":"2024-08-18T23:42:23.314395+0000","last_clean":"2024-08-18T23:42:23.314395+0000","last_became_active":"2024-08-18T23:41:32.130369+0000","last_became_peered":"2024-08-18T23:41:32.130369+0000","last_unstale":"2024-08-18T23:42:23.314395+0000","last_undegraded":"2024-08-18T23:42:23.314395+0000","last_fullsized":"2024-08-18T23:42:23.314395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:22.537018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320123+0000","last_change":"2024-08-18T23:41:32.137291+0000","last_active":"2024-08-18T23:42:23.320123+0000","last_peered":"2024-08-18T23:42:23.320123+0000","last_clean":"2024-08-18T23:42:23.320123+0000","last_became_active":"2024-08-18T23:41:32.136977+0000","last_became_peered":"2024-08-18T23:41:32.136977+0000","last_unstale":"2024-08-18T23:42:23.320123+0000","last_undegraded":"2024-08-18T23:42:23.320123+0000","last_fullsized":"2024-08-18T23:42:23.320123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:18:45.544760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322146+0000","last_change":"2024-08-18T23:41:32.144434+0000","last_active":"2024-08-18T23:42:23.322146+0000","last_peered":"2024-08-18T23:42:23.322146+0000","last_clean":"2024-08-18T23:42:23.322146+0000","last_became_active":"2024-08-18T23:41:32.144119+0000","last_became_peered":"2024-08-18T23:41:32.144119+0000","last_unstale":"2024-08-18T23:42:23.322146+0000","last_undegraded":"2024-08-18T23:42:23.322146+0000","last_fullsized":"2024-08-18T23:42:23.322146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:50:54.528630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319436+0000","last_change":"2024-08-18T23:41:32.135034+0000","last_active":"2024-08-18T23:42:23.319436+0000","last_peered":"2024-08-18T23:42:23.319436+0000","last_clean":"2024-08-18T23:42:23.319436+0000","last_became_active":"2024-08-18T23:41:32.134700+0000","last_became_peered":"2024-08-18T23:41:32.134700+0000","last_unstale":"2024-08-18T23:42:23.319436+0000","last_undegraded":"2024-08-18T23:42:23.319436+0000","last_fullsized":"2024-08-18T23:42:23.319436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:06:31.263114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319688+0000","last_change":"2024-08-18T23:41:32.141191+0000","last_active":"2024-08-18T23:42:23.319688+0000","last_peered":"2024-08-18T23:42:23.319688+0000","last_clean":"2024-08-18T23:42:23.319688+0000","last_became_active":"2024-08-18T23:41:32.140901+0000","last_became_peered":"2024-08-18T23:41:32.140901+0000","last_unstale":"2024-08-18T23:42:23.319688+0000","last_undegraded":"2024-08-18T23:42:23.319688+0000","last_fullsized":"2024-08-18T23:42:23.319688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:24.849985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314798+0000","last_change":"2024-08-18T23:41:32.142543+0000","last_active":"2024-08-18T23:42:23.314798+0000","last_peered":"2024-08-18T23:42:23.314798+0000","last_clean":"2024-08-18T23:42:23.314798+0000","last_became_active":"2024-08-18T23:41:32.142084+0000","last_became_peered":"2024-08-18T23:41:32.142084+0000","last_unstale":"2024-08-18T23:42:23.314798+0000","last_undegraded":"2024-08-18T23:42:23.314798+0000","last_fullsized":"2024-08-18T23:42:23.314798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:54:18.200016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316269+0000","last_change":"2024-08-18T23:41:32.148968+0000","last_active":"2024-08-18T23:42:23.316269+0000","last_peered":"2024-08-18T23:42:23.316269+0000","last_clean":"2024-08-18T23:42:23.316269+0000","last_became_active":"2024-08-18T23:41:32.148625+0000","last_became_peered":"2024-08-18T23:41:32.148625+0000","last_unstale":"2024-08-18T23:42:23.316269+0000","last_undegraded":"2024-08-18T23:42:23.316269+0000","last_fullsized":"2024-08-18T23:42:23.316269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:11:40.172139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316345+0000","last_change":"2024-08-18T23:41:32.133478+0000","last_active":"2024-08-18T23:42:23.316345+0000","last_peered":"2024-08-18T23:42:23.316345+0000","last_clean":"2024-08-18T23:42:23.316345+0000","last_became_active":"2024-08-18T23:41:32.133149+0000","last_became_peered":"2024-08-18T23:41:32.133149+0000","last_unstale":"2024-08-18T23:42:23.316345+0000","last_undegraded":"2024-08-18T23:42:23.316345+0000","last_fullsized":"2024-08-18T23:42:23.316345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:57:53.713309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314293+0000","last_change":"2024-08-18T23:41:32.138206+0000","last_active":"2024-08-18T23:42:23.314293+0000","last_peered":"2024-08-18T23:42:23.314293+0000","last_clean":"2024-08-18T23:42:23.314293+0000","last_became_active":"2024-08-18T23:41:32.137903+0000","last_became_peered":"2024-08-18T23:41:32.137903+0000","last_unstale":"2024-08-18T23:42:23.314293+0000","last_undegraded":"2024-08-18T23:42:23.314293+0000","last_fullsized":"2024-08-18T23:42:23.314293+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:18:48.600658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319090+0000","last_change":"2024-08-18T23:41:32.136303+0000","last_active":"2024-08-18T23:42:23.319090+0000","last_peered":"2024-08-18T23:42:23.319090+0000","last_clean":"2024-08-18T23:42:23.319090+0000","last_became_active":"2024-08-18T23:41:32.134628+0000","last_became_peered":"2024-08-18T23:41:32.134628+0000","last_unstale":"2024-08-18T23:42:23.319090+0000","last_undegraded":"2024-08-18T23:42:23.319090+0000","last_fullsized":"2024-08-18T23:42:23.319090+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:28:51.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317055+0000","last_change":"2024-08-18T23:41:32.137512+0000","last_active":"2024-08-18T23:42:23.317055+0000","last_peered":"2024-08-18T23:42:23.317055+0000","last_clean":"2024-08-18T23:42:23.317055+0000","last_became_active":"2024-08-18T23:41:32.137150+0000","last_became_peered":"2024-08-18T23:41:32.137150+0000","last_unstale":"2024-08-18T23:42:23.317055+0000","last_undegraded":"2024-08-18T23:42:23.317055+0000","last_fullsized":"2024-08-18T23:42:23.317055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:44:08.020780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558680+0000","last_change":"2024-08-18T23:41:32.139229+0000","last_active":"2024-08-18T23:42:23.558680+0000","last_peered":"2024-08-18T23:42:23.558680+0000","last_clean":"2024-08-18T23:42:23.558680+0000","last_became_active":"2024-08-18T23:41:32.138864+0000","last_became_peered":"2024-08-18T23:41:32.138864+0000","last_unstale":"2024-08-18T23:42:23.558680+0000","last_undegraded":"2024-08-18T23:42:23.558680+0000","last_fullsized":"2024-08-18T23:42:23.558680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:54:21.701750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316215+0000","last_change":"2024-08-18T23:41:32.148727+0000","last_active":"2024-08-18T23:42:23.316215+0000","last_peered":"2024-08-18T23:42:23.316215+0000","last_clean":"2024-08-18T23:42:23.316215+0000","last_became_active":"2024-08-18T23:41:32.146828+0000","last_became_peered":"2024-08-18T23:41:32.146828+0000","last_unstale":"2024-08-18T23:42:23.316215+0000","last_undegraded":"2024-08-18T23:42:23.316215+0000","last_fullsized":"2024-08-18T23:42:23.316215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:51:49.175150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557445+0000","last_change":"2024-08-18T23:41:32.144796+0000","last_active":"2024-08-18T23:42:23.557445+0000","last_peered":"2024-08-18T23:42:23.557445+0000","last_clean":"2024-08-18T23:42:23.557445+0000","last_became_active":"2024-08-18T23:41:32.141931+0000","last_became_peered":"2024-08-18T23:41:32.141931+0000","last_unstale":"2024-08-18T23:42:23.557445+0000","last_undegraded":"2024-08-18T23:42:23.557445+0000","last_fullsized":"2024-08-18T23:42:23.557445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:48:08.428845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316954+0000","last_change":"2024-08-18T23:41:32.135265+0000","last_active":"2024-08-18T23:42:23.316954+0000","last_peered":"2024-08-18T23:42:23.316954+0000","last_clean":"2024-08-18T23:42:23.316954+0000","last_became_active":"2024-08-18T23:41:32.134952+0000","last_became_peered":"2024-08-18T23:41:32.134952+0000","last_unstale":"2024-08-18T23:42:23.316954+0000","last_undegraded":"2024-08-18T23:42:23.316954+0000","last_fullsized":"2024-08-18T23:42:23.316954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:35:14.099536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558127+0000","last_change":"2024-08-18T23:41:32.135941+0000","last_active":"2024-08-18T23:42:23.558127+0000","last_peered":"2024-08-18T23:42:23.558127+0000","last_clean":"2024-08-18T23:42:23.558127+0000","last_became_active":"2024-08-18T23:41:32.135623+0000","last_became_peered":"2024-08-18T23:41:32.135623+0000","last_unstale":"2024-08-18T23:42:23.558127+0000","last_undegraded":"2024-08-18T23:42:23.558127+0000","last_fullsized":"2024-08-18T23:42:23.558127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:43:36.504738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324078+0000","last_change":"2024-08-18T23:41:32.141643+0000","last_active":"2024-08-18T23:42:23.324078+0000","last_peered":"2024-08-18T23:42:23.324078+0000","last_clean":"2024-08-18T23:42:23.324078+0000","last_became_active":"2024-08-18T23:41:32.141301+0000","last_became_peered":"2024-08-18T23:41:32.141301+0000","last_unstale":"2024-08-18T23:42:23.324078+0000","last_undegraded":"2024-08-18T23:42:23.324078+0000","last_fullsized":"2024-08-18T23:42:23.324078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:23:53.100740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318979+0000","last_change":"2024-08-18T23:41:32.146730+0000","last_active":"2024-08-18T23:42:23.318979+0000","last_peered":"2024-08-18T23:42:23.318979+0000","last_clean":"2024-08-18T23:42:23.318979+0000","last_became_active":"2024-08-18T23:41:32.146252+0000","last_became_peered":"2024-08-18T23:41:32.146252+0000","last_unstale":"2024-08-18T23:42:23.318979+0000","last_undegraded":"2024-08-18T23:42:23.318979+0000","last_fullsized":"2024-08-18T23:42:23.318979+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:41.481761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316570+0000","last_change":"2024-08-18T23:41:32.149569+0000","last_active":"2024-08-18T23:42:23.316570+0000","last_peered":"2024-08-18T23:42:23.316570+0000","last_clean":"2024-08-18T23:42:23.316570+0000","last_became_active":"2024-08-18T23:41:32.149277+0000","last_became_peered":"2024-08-18T23:41:32.149277+0000","last_unstale":"2024-08-18T23:42:23.316570+0000","last_undegraded":"2024-08-18T23:42:23.316570+0000","last_fullsized":"2024-08-18T23:42:23.316570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:34:00.547909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316678+0000","last_change":"2024-08-18T23:41:32.137889+0000","last_active":"2024-08-18T23:42:23.316678+0000","last_peered":"2024-08-18T23:42:23.316678+0000","last_clean":"2024-08-18T23:42:23.316678+0000","last_became_active":"2024-08-18T23:41:32.137578+0000","last_became_peered":"2024-08-18T23:41:32.137578+0000","last_unstale":"2024-08-18T23:42:23.316678+0000","last_undegraded":"2024-08-18T23:42:23.316678+0000","last_fullsized":"2024-08-18T23:42:23.316678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:18:09.987838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319506+0000","last_change":"2024-08-18T23:41:32.142084+0000","last_active":"2024-08-18T23:42:23.319506+0000","last_peered":"2024-08-18T23:42:23.319506+0000","last_clean":"2024-08-18T23:42:23.319506+0000","last_became_active":"2024-08-18T23:41:32.141670+0000","last_became_peered":"2024-08-18T23:41:32.141670+0000","last_unstale":"2024-08-18T23:42:23.319506+0000","last_undegraded":"2024-08-18T23:42:23.319506+0000","last_fullsized":"2024-08-18T23:42:23.319506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:03.990351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67386,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":186,"num_read_kb":165,"num_write":167,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":200170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514521,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":505014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776158,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28688,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716464,"statfs":{"total":95995035648,"available":95965659136,"internally_reserved":0,"allocated":1712128,"data_stored":1155668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939682,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":509346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168615,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":509266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397546,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36936,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708216,"statfs":{"total":95995035648,"available":95957213184,"internally_reserved":0,"allocated":1630208,"data_stored":1101468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561072,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":500426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790004,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1092570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953529,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":506048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182460,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":500620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":562502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":563332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":501814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T23:42:37.008 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-18T23:42:37.008 INFO:tasks.cephadm:Setting up client nodes... 2024-08-18T23:42:37.008 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-08-18T23:42:37.262 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:37.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:37 smithi078 ceph-mon[18440]: pgmap v297: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 8 op/s; 58 B/s, 0 objects/s recovering 2024-08-18T23:42:37.381 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:37 smithi078 ceph-mon[18440]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:42:37.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:37 smithi123 ceph-mon[21808]: pgmap v297: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 8 op/s; 58 B/s, 0 objects/s recovering 2024-08-18T23:42:37.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:37 smithi123 ceph-mon[21808]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:42:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:37 smithi161 ceph-mon[22027]: pgmap v297: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 8 op/s; 58 B/s, 0 objects/s recovering 2024-08-18T23:42:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:37 smithi161 ceph-mon[22027]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:42:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.762+0000 7efc3713d640 1 -- 172.21.15.78:0/51377115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc30103910 msgr2=0x7efc300ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.762+0000 7efc3713d640 1 --2- 172.21.15.78:0/51377115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc30103910 0x7efc300ff9c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009900 tx=0x7efc1c02a410 comp rx=0 tx=0).stop 2024-08-18T23:42:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.763+0000 7efc3713d640 1 -- 172.21.15.78:0/51377115 shutdown_connections 2024-08-18T23:42:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.763+0000 7efc3713d640 1 --2- 172.21.15.78:0/51377115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc30100080 0x7efc30109f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.763+0000 7efc3713d640 1 --2- 172.21.15.78:0/51377115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc30103910 0x7efc300ff9c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.763+0000 7efc3713d640 1 --2- 172.21.15.78:0/51377115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc30102f40 0x7efc30103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.763+0000 7efc3713d640 1 -- 172.21.15.78:0/51377115 >> 172.21.15.78:0/51377115 conn(0x7efc300fb770 msgr2=0x7efc300fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.763+0000 7efc3713d640 1 -- 172.21.15.78:0/51377115 shutdown_connections 2024-08-18T23:42:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.763+0000 7efc3713d640 1 -- 172.21.15.78:0/51377115 wait complete. 2024-08-18T23:42:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.763+0000 7efc3713d640 1 Processor -- start 2024-08-18T23:42:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.763+0000 7efc3713d640 1 -- start start 2024-08-18T23:42:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.763+0000 7efc3713d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc30100080 0x7efc30197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.763+0000 7efc3713d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc30102f40 0x7efc30198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.763+0000 7efc3713d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc30103910 0x7efc3019f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.763+0000 7efc3713d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efc3010d6d0 con 0x7efc30103910 2024-08-18T23:42:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.763+0000 7efc3713d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc3010d550 con 0x7efc30102f40 2024-08-18T23:42:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.763+0000 7efc3713d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efc3010d850 con 0x7efc30100080 2024-08-18T23:42:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.764+0000 7efc356da640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc30103910 0x7efc3019f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.764+0000 7efc356da640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc30103910 0x7efc3019f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56646/0 (socket says 172.21.15.78:56646) 2024-08-18T23:42:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.764+0000 7efc356da640 1 -- 172.21.15.78:0/46848710 learned_addr learned my addr 172.21.15.78:0/46848710 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.764+0000 7efc2ffff640 1 --2- 172.21.15.78:0/46848710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc30102f40 0x7efc30198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.764+0000 7efc34ed9640 1 --2- 172.21.15.78:0/46848710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc30100080 0x7efc30197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.764+0000 7efc356da640 1 -- 172.21.15.78:0/46848710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc30100080 msgr2=0x7efc30197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.764+0000 7efc356da640 1 --2- 172.21.15.78:0/46848710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc30100080 0x7efc30197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.764+0000 7efc356da640 1 -- 172.21.15.78:0/46848710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc30102f40 msgr2=0x7efc30198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.764+0000 7efc356da640 1 --2- 172.21.15.78:0/46848710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc30102f40 0x7efc30198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.764+0000 7efc356da640 1 -- 172.21.15.78:0/46848710 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3019fa60 con 0x7efc30103910 2024-08-18T23:42:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.764+0000 7efc34ed9640 1 --2- 172.21.15.78:0/46848710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc30100080 0x7efc30197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:42:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.764+0000 7efc356da640 1 --2- 172.21.15.78:0/46848710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc30103910 0x7efc3019f3f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efc2800b970 tx=0x7efc2800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.764+0000 7efc2dffb640 1 -- 172.21.15.78:0/46848710 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc28019070 con 0x7efc30103910 2024-08-18T23:42:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.764+0000 7efc2dffb640 1 -- 172.21.15.78:0/46848710 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc2800cb40 con 0x7efc30103910 2024-08-18T23:42:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.764+0000 7efc3713d640 1 -- 172.21.15.78:0/46848710 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3019fd50 con 0x7efc30103910 2024-08-18T23:42:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.764+0000 7efc2dffb640 1 -- 172.21.15.78:0/46848710 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc28007650 con 0x7efc30103910 2024-08-18T23:42:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.765+0000 7efc3713d640 1 -- 172.21.15.78:0/46848710 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc301a0280 con 0x7efc30103910 2024-08-18T23:42:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.765+0000 7efc3713d640 1 -- 172.21.15.78:0/46848710 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc30108fd0 con 0x7efc30103910 2024-08-18T23:42:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.766+0000 7efc2dffb640 1 -- 172.21.15.78:0/46848710 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efc2801c070 con 0x7efc30103910 2024-08-18T23:42:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.767+0000 7efc2dffb640 1 --2- 172.21.15.78:0/46848710 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efc200a2cc0 0x7efc200a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.767+0000 7efc34ed9640 1 --2- 172.21.15.78:0/46848710 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efc200a2cc0 0x7efc200a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.767+0000 7efc2dffb640 1 -- 172.21.15.78:0/46848710 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7efc280ce920 con 0x7efc30103910 2024-08-18T23:42:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.768+0000 7efc34ed9640 1 --2- 172.21.15.78:0/46848710 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efc200a2cc0 0x7efc200a5180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efc3006a1c0 tx=0x7efc24008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:37.771+0000 7efc2dffb640 1 -- 172.21.15.78:0/46848710 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7efc2809ae80 con 0x7efc30103910 2024-08-18T23:42:38.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:38.001+0000 7efc3713d640 1 -- 172.21.15.78:0/46848710 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7efc30062b20 con 0x7efc30103910 2024-08-18T23:42:38.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:38.004+0000 7efc2dffb640 1 -- 172.21.15.78:0/46848710 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7efc2809f180 con 0x7efc30103910 2024-08-18T23:42:38.005 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-08-18T23:42:38.005 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQDuhsJm4/EvABAASTuhj30rKGuRukheWsq10A== 2024-08-18T23:42:38.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:38.006+0000 7efc3713d640 1 -- 172.21.15.78:0/46848710 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efc200a2cc0 msgr2=0x7efc200a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:38.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:38.006+0000 7efc3713d640 1 --2- 172.21.15.78:0/46848710 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efc200a2cc0 0x7efc200a5180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efc3006a1c0 tx=0x7efc24008040 comp rx=0 tx=0).stop 2024-08-18T23:42:38.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:38.006+0000 7efc3713d640 1 -- 172.21.15.78:0/46848710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc30103910 msgr2=0x7efc3019f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:38.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:38.006+0000 7efc3713d640 1 --2- 172.21.15.78:0/46848710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc30103910 0x7efc3019f3f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efc2800b970 tx=0x7efc2800be30 comp rx=0 tx=0).stop 2024-08-18T23:42:38.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:38.006+0000 7efc3713d640 1 -- 172.21.15.78:0/46848710 shutdown_connections 2024-08-18T23:42:38.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:38.006+0000 7efc3713d640 1 --2- 172.21.15.78:0/46848710 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efc200a2cc0 0x7efc200a5180 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:38.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:38.006+0000 7efc3713d640 1 --2- 172.21.15.78:0/46848710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc30103910 0x7efc3019f3f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:38.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:38.006+0000 7efc3713d640 1 --2- 172.21.15.78:0/46848710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc30102f40 0x7efc30198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:38.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:38.006+0000 7efc3713d640 1 --2- 172.21.15.78:0/46848710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc30100080 0x7efc30197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:38.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:38.006+0000 7efc3713d640 1 -- 172.21.15.78:0/46848710 >> 172.21.15.78:0/46848710 conn(0x7efc300fb770 msgr2=0x7efc301061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:38.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:38.006+0000 7efc3713d640 1 -- 172.21.15.78:0/46848710 shutdown_connections 2024-08-18T23:42:38.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:38.006+0000 7efc3713d640 1 -- 172.21.15.78:0/46848710 wait complete. 2024-08-18T23:42:38.329 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:38 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/46848710' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-18T23:42:38.329 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:38 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/46848710' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-18T23:42:38.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:38 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/46848710' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-18T23:42:38.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:38 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/46848710' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-18T23:42:38.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:38 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/46848710' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-18T23:42:38.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:38 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/46848710' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-18T23:42:38.752 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:42:38.752 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-08-18T23:42:38.752 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-18T23:42:38.790 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-18T23:42:38.790 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-08-18T23:42:38.790 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph mgr dump --format=json 2024-08-18T23:42:39.081 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:39 smithi078 ceph-mon[18440]: pgmap v298: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 6 op/s; 9 B/s, 0 objects/s recovering 2024-08-18T23:42:39.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:39 smithi123 ceph-mon[21808]: pgmap v298: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 6 op/s; 9 B/s, 0 objects/s recovering 2024-08-18T23:42:39.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.555+0000 7f8c9d5c9640 1 -- 172.21.15.78:0/2313063138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c98106c80 msgr2=0x7f8c9810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:39.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.555+0000 7f8c9d5c9640 1 --2- 172.21.15.78:0/2313063138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c98106c80 0x7f8c9810e820 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009cd0 tx=0x7f8c9002a8e0 comp rx=0 tx=0).stop 2024-08-18T23:42:39.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.555+0000 7f8c9d5c9640 1 -- 172.21.15.78:0/2313063138 shutdown_connections 2024-08-18T23:42:39.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.555+0000 7f8c9d5c9640 1 --2- 172.21.15.78:0/2313063138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c98106c80 0x7f8c9810e820 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:39.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.555+0000 7f8c9d5c9640 1 --2- 172.21.15.78:0/2313063138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c981009d0 0x7f8c98106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:39.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.555+0000 7f8c9d5c9640 1 --2- 172.21.15.78:0/2313063138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c98100020 0x7f8c98100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:39.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.555+0000 7f8c9d5c9640 1 -- 172.21.15.78:0/2313063138 >> 172.21.15.78:0/2313063138 conn(0x7f8c98076f40 msgr2=0x7f8c980fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:39.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.555+0000 7f8c9d5c9640 1 -- 172.21.15.78:0/2313063138 shutdown_connections 2024-08-18T23:42:39.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.555+0000 7f8c9d5c9640 1 -- 172.21.15.78:0/2313063138 wait complete. 2024-08-18T23:42:39.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.556+0000 7f8c9d5c9640 1 Processor -- start 2024-08-18T23:42:39.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.556+0000 7f8c9d5c9640 1 -- start start 2024-08-18T23:42:39.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.556+0000 7f8c9d5c9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c98100020 0x7f8c98198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:39.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.556+0000 7f8c9d5c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c981009d0 0x7f8c981986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:39.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.556+0000 7f8c9d5c9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c98106c80 0x7f8c9819ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:39.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.556+0000 7f8c9d5c9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c98111cd0 con 0x7f8c981009d0 2024-08-18T23:42:39.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.556+0000 7f8c9d5c9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c98111b50 con 0x7f8c98100020 2024-08-18T23:42:39.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.556+0000 7f8c9d5c9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8c98111e50 con 0x7f8c98106c80 2024-08-18T23:42:39.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.556+0000 7f8c8ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c981009d0 0x7f8c981986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:39.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.556+0000 7f8c8ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c981009d0 0x7f8c981986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56662/0 (socket says 172.21.15.78:56662) 2024-08-18T23:42:39.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.556+0000 7f8c8ffff640 1 -- 172.21.15.78:0/2993337567 learned_addr learned my addr 172.21.15.78:0/2993337567 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:39.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.556+0000 7f8c977fe640 1 --2- 172.21.15.78:0/2993337567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c98106c80 0x7f8c9819ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:39.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.557+0000 7f8c96ffd640 1 --2- 172.21.15.78:0/2993337567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c98100020 0x7f8c98198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:39.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.557+0000 7f8c8ffff640 1 -- 172.21.15.78:0/2993337567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c98106c80 msgr2=0x7f8c9819ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:39.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.557+0000 7f8c8ffff640 1 --2- 172.21.15.78:0/2993337567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c98106c80 0x7f8c9819ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:39.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.557+0000 7f8c8ffff640 1 -- 172.21.15.78:0/2993337567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c98100020 msgr2=0x7f8c98198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:39.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.557+0000 7f8c8ffff640 1 --2- 172.21.15.78:0/2993337567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c98100020 0x7f8c98198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:39.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.557+0000 7f8c8ffff640 1 -- 172.21.15.78:0/2993337567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9819d140 con 0x7f8c981009d0 2024-08-18T23:42:39.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.557+0000 7f8c8ffff640 1 --2- 172.21.15.78:0/2993337567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c981009d0 0x7f8c981986d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000d740 tx=0x7f8c8000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:39.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.557+0000 7f8c94ff9640 1 -- 172.21.15.78:0/2993337567 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80017070 con 0x7f8c981009d0 2024-08-18T23:42:39.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.557+0000 7f8c9d5c9640 1 -- 172.21.15.78:0/2993337567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c9819d430 con 0x7f8c981009d0 2024-08-18T23:42:39.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.557+0000 7f8c94ff9640 1 -- 172.21.15.78:0/2993337567 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c8000de50 con 0x7f8c981009d0 2024-08-18T23:42:39.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.557+0000 7f8c94ff9640 1 -- 172.21.15.78:0/2993337567 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80004e00 con 0x7f8c981009d0 2024-08-18T23:42:39.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.557+0000 7f8c9d5c9640 1 -- 172.21.15.78:0/2993337567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c981a4d10 con 0x7f8c981009d0 2024-08-18T23:42:39.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.558+0000 7f8c9d5c9640 1 -- 172.21.15.78:0/2993337567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c98100400 con 0x7f8c981009d0 2024-08-18T23:42:39.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.559+0000 7f8c94ff9640 1 -- 172.21.15.78:0/2993337567 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8c80004fa0 con 0x7f8c981009d0 2024-08-18T23:42:39.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.560+0000 7f8c94ff9640 1 --2- 172.21.15.78:0/2993337567 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8c840a2c10 0x7f8c840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:39.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.560+0000 7f8c96ffd640 1 --2- 172.21.15.78:0/2993337567 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8c840a2c10 0x7f8c840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:39.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.560+0000 7f8c94ff9640 1 -- 172.21.15.78:0/2993337567 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8c800cf2a0 con 0x7f8c981009d0 2024-08-18T23:42:39.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.560+0000 7f8c96ffd640 1 --2- 172.21.15.78:0/2993337567 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8c840a2c10 0x7f8c840a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8c88005fd0 tx=0x7f8c88005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:39.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.563+0000 7f8c94ff9640 1 -- 172.21.15.78:0/2993337567 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8c8009bd40 con 0x7f8c981009d0 2024-08-18T23:42:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:39 smithi161 ceph-mon[22027]: pgmap v298: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 6 op/s; 9 B/s, 0 objects/s recovering 2024-08-18T23:42:39.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.774+0000 7f8c9d5c9640 1 -- 172.21.15.78:0/2993337567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f8c981994f0 con 0x7f8c981009d0 2024-08-18T23:42:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.779+0000 7f8c94ff9640 1 -- 172.21.15.78:0/2993337567 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263201 (secure 0 0 0) 0x7f8c981994f0 con 0x7f8c981009d0 2024-08-18T23:42:39.783 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:42:39.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.781+0000 7f8c9d5c9640 1 -- 172.21.15.78:0/2993337567 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8c840a2c10 msgr2=0x7f8c840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:39.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.781+0000 7f8c9d5c9640 1 --2- 172.21.15.78:0/2993337567 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8c840a2c10 0x7f8c840a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8c88005fd0 tx=0x7f8c88005850 comp rx=0 tx=0).stop 2024-08-18T23:42:39.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.781+0000 7f8c9d5c9640 1 -- 172.21.15.78:0/2993337567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c981009d0 msgr2=0x7f8c981986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:39.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.781+0000 7f8c9d5c9640 1 --2- 172.21.15.78:0/2993337567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c981009d0 0x7f8c981986d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000d740 tx=0x7f8c8000dc00 comp rx=0 tx=0).stop 2024-08-18T23:42:39.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.781+0000 7f8c9d5c9640 1 -- 172.21.15.78:0/2993337567 shutdown_connections 2024-08-18T23:42:39.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.781+0000 7f8c9d5c9640 1 --2- 172.21.15.78:0/2993337567 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8c840a2c10 0x7f8c840a50d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:39.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.781+0000 7f8c9d5c9640 1 --2- 172.21.15.78:0/2993337567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c98106c80 0x7f8c9819ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:39.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.781+0000 7f8c9d5c9640 1 --2- 172.21.15.78:0/2993337567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c981009d0 0x7f8c981986d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:39.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.781+0000 7f8c9d5c9640 1 --2- 172.21.15.78:0/2993337567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c98100020 0x7f8c98198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:39.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.781+0000 7f8c9d5c9640 1 -- 172.21.15.78:0/2993337567 >> 172.21.15.78:0/2993337567 conn(0x7f8c98076f40 msgr2=0x7f8c98105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:39.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.782+0000 7f8c9d5c9640 1 -- 172.21.15.78:0/2993337567 shutdown_connections 2024-08-18T23:42:39.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:39.782+0000 7f8c9d5c9640 1 -- 172.21.15.78:0/2993337567 wait complete. 2024-08-18T23:42:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:40 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2993337567' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-08-18T23:42:40.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:40 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2993337567' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-08-18T23:42:40.558 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4000744943},{"type":"v1","addr":"172.21.15.78:6801","nonce":4000744943}]},"active_addr":"172.21.15.78:6801/4000744943","active_change":"2024-08-18T23:35:22.036194+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.78:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":2397561295}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":1917247417}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":2404266161}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":2712900765}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":2673796366}]}]} 2024-08-18T23:42:40.563 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-08-18T23:42:40.563 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-08-18T23:42:40.563 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-18T23:42:40.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:40 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2993337567' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-08-18T23:42:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:41.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.364+0000 7fda20ec7640 1 -- 172.21.15.78:0/3009418435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c100b50 msgr2=0x7fda1c100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:41.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.364+0000 7fda20ec7640 1 --2- 172.21.15.78:0/3009418435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c100b50 0x7fda1c100f30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009900 tx=0x7fda0c02a480 comp rx=0 tx=0).stop 2024-08-18T23:42:41.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.364+0000 7fda20ec7640 1 -- 172.21.15.78:0/3009418435 shutdown_connections 2024-08-18T23:42:41.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.364+0000 7fda20ec7640 1 --2- 172.21.15.78:0/3009418435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1c102c20 0x7fda1c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:41.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.364+0000 7fda20ec7640 1 --2- 172.21.15.78:0/3009418435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda1c1022a0 0x7fda1c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:41.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.364+0000 7fda20ec7640 1 --2- 172.21.15.78:0/3009418435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c100b50 0x7fda1c100f30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:41.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.364+0000 7fda20ec7640 1 -- 172.21.15.78:0/3009418435 >> 172.21.15.78:0/3009418435 conn(0x7fda1c0fc500 msgr2=0x7fda1c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:41.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.364+0000 7fda20ec7640 1 -- 172.21.15.78:0/3009418435 shutdown_connections 2024-08-18T23:42:41.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.364+0000 7fda20ec7640 1 -- 172.21.15.78:0/3009418435 wait complete. 2024-08-18T23:42:41.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.364+0000 7fda20ec7640 1 Processor -- start 2024-08-18T23:42:41.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.364+0000 7fda20ec7640 1 -- start start 2024-08-18T23:42:41.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda20ec7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda1c100b50 0x7fda1c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:41.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda20ec7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c1022a0 0x7fda1c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:41.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda20ec7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1c102c20 0x7fda1c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:41.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda20ec7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c110cf0 con 0x7fda1c1022a0 2024-08-18T23:42:41.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda20ec7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c110b70 con 0x7fda1c100b50 2024-08-18T23:42:41.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda20ec7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c110e70 con 0x7fda1c102c20 2024-08-18T23:42:41.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda19d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c1022a0 0x7fda1c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:41.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda19d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c1022a0 0x7fda1c198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56680/0 (socket says 172.21.15.78:56680) 2024-08-18T23:42:41.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda19d9b640 1 -- 172.21.15.78:0/4098709643 learned_addr learned my addr 172.21.15.78:0/4098709643 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:41.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda1ad9d640 1 --2- 172.21.15.78:0/4098709643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1c102c20 0x7fda1c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:41.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda1a59c640 1 --2- 172.21.15.78:0/4098709643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda1c100b50 0x7fda1c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:41.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda19d9b640 1 -- 172.21.15.78:0/4098709643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1c102c20 msgr2=0x7fda1c19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:41.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda19d9b640 1 --2- 172.21.15.78:0/4098709643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1c102c20 0x7fda1c19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:41.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda19d9b640 1 -- 172.21.15.78:0/4098709643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda1c100b50 msgr2=0x7fda1c1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:41.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda19d9b640 1 --2- 172.21.15.78:0/4098709643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda1c100b50 0x7fda1c1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:41.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda19d9b640 1 -- 172.21.15.78:0/4098709643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1c19d270 con 0x7fda1c1022a0 2024-08-18T23:42:41.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda1a59c640 1 --2- 172.21.15.78:0/4098709643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda1c100b50 0x7fda1c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:42:41.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda19d9b640 1 --2- 172.21.15.78:0/4098709643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c1022a0 0x7fda1c198800 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fda1000b4a0 tx=0x7fda1000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:41.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda0b7fe640 1 -- 172.21.15.78:0/4098709643 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda10014070 con 0x7fda1c1022a0 2024-08-18T23:42:41.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda0b7fe640 1 -- 172.21.15.78:0/4098709643 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda10010af0 con 0x7fda1c1022a0 2024-08-18T23:42:41.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda20ec7640 1 -- 172.21.15.78:0/4098709643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1c19d560 con 0x7fda1c1022a0 2024-08-18T23:42:41.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.365+0000 7fda0b7fe640 1 -- 172.21.15.78:0/4098709643 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda10017d10 con 0x7fda1c1022a0 2024-08-18T23:42:41.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.366+0000 7fda20ec7640 1 -- 172.21.15.78:0/4098709643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda1c1a4e40 con 0x7fda1c1022a0 2024-08-18T23:42:41.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.366+0000 7fda20ec7640 1 -- 172.21.15.78:0/4098709643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda1c100f30 con 0x7fda1c1022a0 2024-08-18T23:42:41.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.367+0000 7fda0b7fe640 1 -- 172.21.15.78:0/4098709643 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fda10017070 con 0x7fda1c1022a0 2024-08-18T23:42:41.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.368+0000 7fda0b7fe640 1 --2- 172.21.15.78:0/4098709643 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fda040a2c10 0x7fda040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:41.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.368+0000 7fda1a59c640 1 --2- 172.21.15.78:0/4098709643 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fda040a2c10 0x7fda040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:41.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.368+0000 7fda0b7fe640 1 -- 172.21.15.78:0/4098709643 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fda100cec30 con 0x7fda1c1022a0 2024-08-18T23:42:41.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.369+0000 7fda1a59c640 1 --2- 172.21.15.78:0/4098709643 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fda040a2c10 0x7fda040a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fda0c000c00 tx=0x7fda0c004100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:41.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.371+0000 7fda0b7fe640 1 -- 172.21.15.78:0/4098709643 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fda1009b7e0 con 0x7fda1c1022a0 2024-08-18T23:42:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:41 smithi078 ceph-mon[18440]: pgmap v299: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-18T23:42:41.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.542+0000 7fda20ec7640 1 -- 172.21.15.78:0/4098709643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda1c1a5040 con 0x7fda1c1022a0 2024-08-18T23:42:41.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.543+0000 7fda0b7fe640 1 -- 172.21.15.78:0/4098709643 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24654 (secure 0 0 0) 0x7fda1009b0e0 con 0x7fda1c1022a0 2024-08-18T23:42:41.544 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:42:41.544 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","created":"2024-08-18T23:34:33.867246+0000","modified":"2024-08-18T23:42:23.304291+0000","last_up_change":"2024-08-18T23:40:42.200381+0000","last_in_change":"2024-08-18T23:40:22.425813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T23:37:22.338211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T23:41:28.920802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T23:41:31.026094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T23:41:37.403238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4de1c78-6ebc-49a9-b9c5-85d43535fab7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6803","nonce":3639645311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6805","nonce":3639645311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6809","nonce":3639645311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6807","nonce":3639645311}]},"public_addr":"172.21.15.78:6803/3639645311","cluster_addr":"172.21.15.78:6805/3639645311","heartbeat_back_addr":"172.21.15.78:6809/3639645311","heartbeat_front_addr":"172.21.15.78:6807/3639645311","state":["exists","up"]},{"osd":1,"uuid":"4f8e0c3d-472b-4230-ad23-2b0a34861251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6801","nonce":4199446368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6803","nonce":4199446368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6807","nonce":4199446368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6805","nonce":4199446368}]},"public_addr":"172.21.15.123:6801/4199446368","cluster_addr":"172.21.15.123:6803/4199446368","heartbeat_back_addr":"172.21.15.123:6807/4199446368","heartbeat_front_addr":"172.21.15.123:6805/4199446368","state":["exists","up"]},{"osd":2,"uuid":"27ef3314-bb6f-413d-be05-e10a105dec9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6801","nonce":2582810057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6803","nonce":2582810057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6807","nonce":2582810057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6805","nonce":2582810057}]},"public_addr":"172.21.15.161:6801/2582810057","cluster_addr":"172.21.15.161:6803/2582810057","heartbeat_back_addr":"172.21.15.161:6807/2582810057","heartbeat_front_addr":"172.21.15.161:6805/2582810057","state":["exists","up"]},{"osd":3,"uuid":"e9e52662-5ef9-4d70-bdd7-2b92165822ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6811","nonce":2697347521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6813","nonce":2697347521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6817","nonce":2697347521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6815","nonce":2697347521}]},"public_addr":"172.21.15.78:6811/2697347521","cluster_addr":"172.21.15.78:6813/2697347521","heartbeat_back_addr":"172.21.15.78:6817/2697347521","heartbeat_front_addr":"172.21.15.78:6815/2697347521","state":["exists","up"]},{"osd":4,"uuid":"5ed26775-79e3-4b24-9557-69fcf5e8e824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6809","nonce":1281511569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6811","nonce":1281511569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6815","nonce":1281511569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6813","nonce":1281511569}]},"public_addr":"172.21.15.123:6809/1281511569","cluster_addr":"172.21.15.123:6811/1281511569","heartbeat_back_addr":"172.21.15.123:6815/1281511569","heartbeat_front_addr":"172.21.15.123:6813/1281511569","state":["exists","up"]},{"osd":5,"uuid":"a9793b1d-5613-48a4-938e-adb718f4fa4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6809","nonce":2751796117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6811","nonce":2751796117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6815","nonce":2751796117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6813","nonce":2751796117}]},"public_addr":"172.21.15.161:6809/2751796117","cluster_addr":"172.21.15.161:6811/2751796117","heartbeat_back_addr":"172.21.15.161:6815/2751796117","heartbeat_front_addr":"172.21.15.161:6813/2751796117","state":["exists","up"]},{"osd":6,"uuid":"5dbc5e9c-a14f-4b71-910a-1e6d4827bb32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6819","nonce":95383223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6821","nonce":95383223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6825","nonce":95383223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6823","nonce":95383223}]},"public_addr":"172.21.15.78:6819/95383223","cluster_addr":"172.21.15.78:6821/95383223","heartbeat_back_addr":"172.21.15.78:6825/95383223","heartbeat_front_addr":"172.21.15.78:6823/95383223","state":["exists","up"]},{"osd":7,"uuid":"26fb7b69-131f-42ee-b745-6fe8b4f65391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6817","nonce":3440305156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6819","nonce":3440305156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6823","nonce":3440305156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6821","nonce":3440305156}]},"public_addr":"172.21.15.123:6817/3440305156","cluster_addr":"172.21.15.123:6819/3440305156","heartbeat_back_addr":"172.21.15.123:6823/3440305156","heartbeat_front_addr":"172.21.15.123:6821/3440305156","state":["exists","up"]},{"osd":8,"uuid":"e267bc50-0c29-4d7b-b368-b1621e29b2df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6817","nonce":1485008778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6819","nonce":1485008778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6823","nonce":1485008778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6821","nonce":1485008778}]},"public_addr":"172.21.15.161:6817/1485008778","cluster_addr":"172.21.15.161:6819/1485008778","heartbeat_back_addr":"172.21.15.161:6823/1485008778","heartbeat_front_addr":"172.21.15.161:6821/1485008778","state":["exists","up"]},{"osd":9,"uuid":"b7654818-b767-4fe6-8211-346297895f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6827","nonce":2884475661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6829","nonce":2884475661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6833","nonce":2884475661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6831","nonce":2884475661}]},"public_addr":"172.21.15.78:6827/2884475661","cluster_addr":"172.21.15.78:6829/2884475661","heartbeat_back_addr":"172.21.15.78:6833/2884475661","heartbeat_front_addr":"172.21.15.78:6831/2884475661","state":["exists","up"]},{"osd":10,"uuid":"59aa32ff-510e-4b52-8170-08a34a0b6b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6825","nonce":122226286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6827","nonce":122226286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6831","nonce":122226286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6829","nonce":122226286}]},"public_addr":"172.21.15.123:6825/122226286","cluster_addr":"172.21.15.123:6827/122226286","heartbeat_back_addr":"172.21.15.123:6831/122226286","heartbeat_front_addr":"172.21.15.123:6829/122226286","state":["exists","up"]},{"osd":11,"uuid":"67bacab4-d078-4c5c-b4ff-9f6a8a00f507","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6825","nonce":2037560985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6827","nonce":2037560985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6831","nonce":2037560985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6829","nonce":2037560985}]},"public_addr":"172.21.15.161:6825/2037560985","cluster_addr":"172.21.15.161:6827/2037560985","heartbeat_back_addr":"172.21.15.161:6831/2037560985","heartbeat_front_addr":"172.21.15.161:6829/2037560985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:36.826044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:57.688421+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:18.297424+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:40.355215+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:02.167902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:23.968759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:46.254083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:08.522038+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:31.028119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:54.119807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:16.847034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:39.607886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6800/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3040499240":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6801/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:6800/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/757409165":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/2562072965":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/636184437":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/1570708294":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3137806628":"2024-08-19T23:34:59.021165+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T23:42:41.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.545+0000 7fda20ec7640 1 -- 172.21.15.78:0/4098709643 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fda040a2c10 msgr2=0x7fda040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:41.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.545+0000 7fda20ec7640 1 --2- 172.21.15.78:0/4098709643 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fda040a2c10 0x7fda040a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fda0c000c00 tx=0x7fda0c004100 comp rx=0 tx=0).stop 2024-08-18T23:42:41.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.545+0000 7fda20ec7640 1 -- 172.21.15.78:0/4098709643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c1022a0 msgr2=0x7fda1c198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:41.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.545+0000 7fda20ec7640 1 --2- 172.21.15.78:0/4098709643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c1022a0 0x7fda1c198800 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fda1000b4a0 tx=0x7fda1000b960 comp rx=0 tx=0).stop 2024-08-18T23:42:41.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.545+0000 7fda20ec7640 1 -- 172.21.15.78:0/4098709643 shutdown_connections 2024-08-18T23:42:41.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.545+0000 7fda20ec7640 1 --2- 172.21.15.78:0/4098709643 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fda040a2c10 0x7fda040a50d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:41.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.545+0000 7fda20ec7640 1 --2- 172.21.15.78:0/4098709643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda1c102c20 0x7fda1c19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:41.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.545+0000 7fda20ec7640 1 --2- 172.21.15.78:0/4098709643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda1c1022a0 0x7fda1c198800 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:41.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.545+0000 7fda20ec7640 1 --2- 172.21.15.78:0/4098709643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda1c100b50 0x7fda1c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:41.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.545+0000 7fda20ec7640 1 -- 172.21.15.78:0/4098709643 >> 172.21.15.78:0/4098709643 conn(0x7fda1c0fc500 msgr2=0x7fda1c069ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:41.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.545+0000 7fda20ec7640 1 -- 172.21.15.78:0/4098709643 shutdown_connections 2024-08-18T23:42:41.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:41.545+0000 7fda20ec7640 1 -- 172.21.15.78:0/4098709643 wait complete. 2024-08-18T23:42:41.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:41 smithi123 ceph-mon[21808]: pgmap v299: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-18T23:42:41.578 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:41 smithi161 ceph-mon[22027]: pgmap v299: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-18T23:42:42.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:42 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/4098709643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:42:42.369 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-08-18T23:42:42.369 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-18T23:42:42.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:42 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/4098709643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:42:42.620 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:42.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:42 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/4098709643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:42:43.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.081+0000 7f3e5749c640 1 -- 172.21.15.78:0/902686313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e50102c20 msgr2=0x7f3e5010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:43.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.081+0000 7f3e5749c640 1 --2- 172.21.15.78:0/902686313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e50102c20 0x7f3e5010d770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009ca0 tx=0x7f3e4802a950 comp rx=0 tx=0).stop 2024-08-18T23:42:43.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.081+0000 7f3e5749c640 1 -- 172.21.15.78:0/902686313 shutdown_connections 2024-08-18T23:42:43.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.081+0000 7f3e5749c640 1 --2- 172.21.15.78:0/902686313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e50102c20 0x7f3e5010d770 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.081+0000 7f3e5749c640 1 --2- 172.21.15.78:0/902686313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e501022a0 0x7f3e501026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.081+0000 7f3e5749c640 1 --2- 172.21.15.78:0/902686313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e50100b50 0x7f3e50100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.081+0000 7f3e5749c640 1 -- 172.21.15.78:0/902686313 >> 172.21.15.78:0/902686313 conn(0x7f3e500fc500 msgr2=0x7f3e500fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.081+0000 7f3e5749c640 1 -- 172.21.15.78:0/902686313 shutdown_connections 2024-08-18T23:42:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.081+0000 7f3e5749c640 1 -- 172.21.15.78:0/902686313 wait complete. 2024-08-18T23:42:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.082+0000 7f3e5749c640 1 Processor -- start 2024-08-18T23:42:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.082+0000 7f3e5749c640 1 -- start start 2024-08-18T23:42:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.082+0000 7f3e5749c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e50100b50 0x7f3e50198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.082+0000 7f3e5749c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e501022a0 0x7f3e501994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.082+0000 7f3e5749c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e50102c20 0x7f3e501a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:43.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.082+0000 7f3e5749c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3e50110cb0 con 0x7f3e501022a0 2024-08-18T23:42:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.082+0000 7f3e5749c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3e50110b30 con 0x7f3e50102c20 2024-08-18T23:42:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.082+0000 7f3e5749c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3e50110e30 con 0x7f3e50100b50 2024-08-18T23:42:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.082+0000 7f3e54a37640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e501022a0 0x7f3e501994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.082+0000 7f3e55238640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e50100b50 0x7f3e50198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.082+0000 7f3e54a37640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e501022a0 0x7f3e501994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56698/0 (socket says 172.21.15.78:56698) 2024-08-18T23:42:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.082+0000 7f3e54a37640 1 -- 172.21.15.78:0/2332541668 learned_addr learned my addr 172.21.15.78:0/2332541668 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.082+0000 7f3e55a39640 1 --2- 172.21.15.78:0/2332541668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e50102c20 0x7f3e501a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.082+0000 7f3e54a37640 1 -- 172.21.15.78:0/2332541668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e50100b50 msgr2=0x7f3e50198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.082+0000 7f3e54a37640 1 --2- 172.21.15.78:0/2332541668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e50100b50 0x7f3e50198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.082+0000 7f3e54a37640 1 -- 172.21.15.78:0/2332541668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e50102c20 msgr2=0x7f3e501a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:43.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.082+0000 7f3e54a37640 1 --2- 172.21.15.78:0/2332541668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e50102c20 0x7f3e501a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:43.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.082+0000 7f3e54a37640 1 -- 172.21.15.78:0/2332541668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e501a4540 con 0x7f3e501022a0 2024-08-18T23:42:43.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.082+0000 7f3e54a37640 1 --2- 172.21.15.78:0/2332541668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e501022a0 0x7f3e501994c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00c920 tx=0x7f3e4c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:43.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.083+0000 7f3e467fc640 1 -- 172.21.15.78:0/2332541668 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c007c70 con 0x7f3e501022a0 2024-08-18T23:42:43.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.083+0000 7f3e467fc640 1 -- 172.21.15.78:0/2332541668 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e4c007e10 con 0x7f3e501022a0 2024-08-18T23:42:43.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.083+0000 7f3e5749c640 1 -- 172.21.15.78:0/2332541668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e501a4710 con 0x7f3e501022a0 2024-08-18T23:42:43.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.083+0000 7f3e467fc640 1 -- 172.21.15.78:0/2332541668 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c018890 con 0x7f3e501022a0 2024-08-18T23:42:43.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.083+0000 7f3e5749c640 1 -- 172.21.15.78:0/2332541668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e501a4c20 con 0x7f3e501022a0 2024-08-18T23:42:43.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.084+0000 7f3e5749c640 1 -- 172.21.15.78:0/2332541668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e50100f30 con 0x7f3e501022a0 2024-08-18T23:42:43.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.085+0000 7f3e467fc640 1 -- 172.21.15.78:0/2332541668 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3e4c018a70 con 0x7f3e501022a0 2024-08-18T23:42:43.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.085+0000 7f3e467fc640 1 --2- 172.21.15.78:0/2332541668 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3e3c0a2c10 0x7f3e3c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:43.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.086+0000 7f3e55238640 1 --2- 172.21.15.78:0/2332541668 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3e3c0a2c10 0x7f3e3c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:43.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.086+0000 7f3e467fc640 1 -- 172.21.15.78:0/2332541668 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3e4c0cf630 con 0x7f3e501022a0 2024-08-18T23:42:43.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.086+0000 7f3e55238640 1 --2- 172.21.15.78:0/2332541668 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3e3c0a2c10 0x7f3e3c0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e400058c0 tx=0x7f3e4000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:43.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.089+0000 7f3e467fc640 1 -- 172.21.15.78:0/2332541668 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3e4c014030 con 0x7f3e501022a0 2024-08-18T23:42:43.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.259+0000 7f3e5749c640 1 -- 172.21.15.78:0/2332541668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e501a4ee0 con 0x7f3e501022a0 2024-08-18T23:42:43.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:43 smithi078 ceph-mon[18440]: pgmap v300: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-18T23:42:43.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.260+0000 7f3e467fc640 1 -- 172.21.15.78:0/2332541668 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24654 (secure 0 0 0) 0x7f3e4c09c0d0 con 0x7f3e501022a0 2024-08-18T23:42:43.262 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:42:43.262 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","created":"2024-08-18T23:34:33.867246+0000","modified":"2024-08-18T23:42:23.304291+0000","last_up_change":"2024-08-18T23:40:42.200381+0000","last_in_change":"2024-08-18T23:40:22.425813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T23:37:22.338211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T23:41:28.920802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T23:41:31.026094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T23:41:37.403238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4de1c78-6ebc-49a9-b9c5-85d43535fab7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6803","nonce":3639645311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6805","nonce":3639645311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6809","nonce":3639645311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6807","nonce":3639645311}]},"public_addr":"172.21.15.78:6803/3639645311","cluster_addr":"172.21.15.78:6805/3639645311","heartbeat_back_addr":"172.21.15.78:6809/3639645311","heartbeat_front_addr":"172.21.15.78:6807/3639645311","state":["exists","up"]},{"osd":1,"uuid":"4f8e0c3d-472b-4230-ad23-2b0a34861251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6801","nonce":4199446368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6803","nonce":4199446368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6807","nonce":4199446368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6805","nonce":4199446368}]},"public_addr":"172.21.15.123:6801/4199446368","cluster_addr":"172.21.15.123:6803/4199446368","heartbeat_back_addr":"172.21.15.123:6807/4199446368","heartbeat_front_addr":"172.21.15.123:6805/4199446368","state":["exists","up"]},{"osd":2,"uuid":"27ef3314-bb6f-413d-be05-e10a105dec9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6801","nonce":2582810057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6803","nonce":2582810057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6807","nonce":2582810057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6805","nonce":2582810057}]},"public_addr":"172.21.15.161:6801/2582810057","cluster_addr":"172.21.15.161:6803/2582810057","heartbeat_back_addr":"172.21.15.161:6807/2582810057","heartbeat_front_addr":"172.21.15.161:6805/2582810057","state":["exists","up"]},{"osd":3,"uuid":"e9e52662-5ef9-4d70-bdd7-2b92165822ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6811","nonce":2697347521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6813","nonce":2697347521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6817","nonce":2697347521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6815","nonce":2697347521}]},"public_addr":"172.21.15.78:6811/2697347521","cluster_addr":"172.21.15.78:6813/2697347521","heartbeat_back_addr":"172.21.15.78:6817/2697347521","heartbeat_front_addr":"172.21.15.78:6815/2697347521","state":["exists","up"]},{"osd":4,"uuid":"5ed26775-79e3-4b24-9557-69fcf5e8e824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6809","nonce":1281511569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6811","nonce":1281511569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6815","nonce":1281511569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6813","nonce":1281511569}]},"public_addr":"172.21.15.123:6809/1281511569","cluster_addr":"172.21.15.123:6811/1281511569","heartbeat_back_addr":"172.21.15.123:6815/1281511569","heartbeat_front_addr":"172.21.15.123:6813/1281511569","state":["exists","up"]},{"osd":5,"uuid":"a9793b1d-5613-48a4-938e-adb718f4fa4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6809","nonce":2751796117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6811","nonce":2751796117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6815","nonce":2751796117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6813","nonce":2751796117}]},"public_addr":"172.21.15.161:6809/2751796117","cluster_addr":"172.21.15.161:6811/2751796117","heartbeat_back_addr":"172.21.15.161:6815/2751796117","heartbeat_front_addr":"172.21.15.161:6813/2751796117","state":["exists","up"]},{"osd":6,"uuid":"5dbc5e9c-a14f-4b71-910a-1e6d4827bb32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6819","nonce":95383223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6821","nonce":95383223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6825","nonce":95383223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6823","nonce":95383223}]},"public_addr":"172.21.15.78:6819/95383223","cluster_addr":"172.21.15.78:6821/95383223","heartbeat_back_addr":"172.21.15.78:6825/95383223","heartbeat_front_addr":"172.21.15.78:6823/95383223","state":["exists","up"]},{"osd":7,"uuid":"26fb7b69-131f-42ee-b745-6fe8b4f65391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6817","nonce":3440305156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6819","nonce":3440305156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6823","nonce":3440305156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6821","nonce":3440305156}]},"public_addr":"172.21.15.123:6817/3440305156","cluster_addr":"172.21.15.123:6819/3440305156","heartbeat_back_addr":"172.21.15.123:6823/3440305156","heartbeat_front_addr":"172.21.15.123:6821/3440305156","state":["exists","up"]},{"osd":8,"uuid":"e267bc50-0c29-4d7b-b368-b1621e29b2df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6817","nonce":1485008778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6819","nonce":1485008778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6823","nonce":1485008778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6821","nonce":1485008778}]},"public_addr":"172.21.15.161:6817/1485008778","cluster_addr":"172.21.15.161:6819/1485008778","heartbeat_back_addr":"172.21.15.161:6823/1485008778","heartbeat_front_addr":"172.21.15.161:6821/1485008778","state":["exists","up"]},{"osd":9,"uuid":"b7654818-b767-4fe6-8211-346297895f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6827","nonce":2884475661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6829","nonce":2884475661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6833","nonce":2884475661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6831","nonce":2884475661}]},"public_addr":"172.21.15.78:6827/2884475661","cluster_addr":"172.21.15.78:6829/2884475661","heartbeat_back_addr":"172.21.15.78:6833/2884475661","heartbeat_front_addr":"172.21.15.78:6831/2884475661","state":["exists","up"]},{"osd":10,"uuid":"59aa32ff-510e-4b52-8170-08a34a0b6b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6825","nonce":122226286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6827","nonce":122226286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6831","nonce":122226286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6829","nonce":122226286}]},"public_addr":"172.21.15.123:6825/122226286","cluster_addr":"172.21.15.123:6827/122226286","heartbeat_back_addr":"172.21.15.123:6831/122226286","heartbeat_front_addr":"172.21.15.123:6829/122226286","state":["exists","up"]},{"osd":11,"uuid":"67bacab4-d078-4c5c-b4ff-9f6a8a00f507","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6825","nonce":2037560985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6827","nonce":2037560985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6831","nonce":2037560985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6829","nonce":2037560985}]},"public_addr":"172.21.15.161:6825/2037560985","cluster_addr":"172.21.15.161:6827/2037560985","heartbeat_back_addr":"172.21.15.161:6831/2037560985","heartbeat_front_addr":"172.21.15.161:6829/2037560985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:36.826044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:57.688421+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:18.297424+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:40.355215+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:02.167902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:23.968759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:46.254083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:08.522038+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:31.028119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:54.119807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:16.847034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:39.607886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6800/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3040499240":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6801/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:6800/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/757409165":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/2562072965":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/636184437":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/1570708294":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3137806628":"2024-08-19T23:34:59.021165+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T23:42:43.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.263+0000 7f3e5749c640 1 -- 172.21.15.78:0/2332541668 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3e3c0a2c10 msgr2=0x7f3e3c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:43.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.263+0000 7f3e5749c640 1 --2- 172.21.15.78:0/2332541668 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3e3c0a2c10 0x7f3e3c0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e400058c0 tx=0x7f3e4000a250 comp rx=0 tx=0).stop 2024-08-18T23:42:43.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.263+0000 7f3e5749c640 1 -- 172.21.15.78:0/2332541668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e501022a0 msgr2=0x7f3e501994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:43.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.263+0000 7f3e5749c640 1 --2- 172.21.15.78:0/2332541668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e501022a0 0x7f3e501994c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00c920 tx=0x7f3e4c00cde0 comp rx=0 tx=0).stop 2024-08-18T23:42:43.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.263+0000 7f3e5749c640 1 -- 172.21.15.78:0/2332541668 shutdown_connections 2024-08-18T23:42:43.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.263+0000 7f3e5749c640 1 --2- 172.21.15.78:0/2332541668 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3e3c0a2c10 0x7f3e3c0a50d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:43.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.263+0000 7f3e5749c640 1 --2- 172.21.15.78:0/2332541668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e50102c20 0x7f3e501a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:43.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.263+0000 7f3e5749c640 1 --2- 172.21.15.78:0/2332541668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e501022a0 0x7f3e501994c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:43.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.263+0000 7f3e5749c640 1 --2- 172.21.15.78:0/2332541668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e50100b50 0x7f3e50198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:43.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.263+0000 7f3e5749c640 1 -- 172.21.15.78:0/2332541668 >> 172.21.15.78:0/2332541668 conn(0x7f3e500fc500 msgr2=0x7f3e50106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:43.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.263+0000 7f3e5749c640 1 -- 172.21.15.78:0/2332541668 shutdown_connections 2024-08-18T23:42:43.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:43.263+0000 7f3e5749c640 1 -- 172.21.15.78:0/2332541668 wait complete. 2024-08-18T23:42:43.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:43 smithi123 ceph-mon[21808]: pgmap v300: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-18T23:42:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:43 smithi161 ceph-mon[22027]: pgmap v300: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-18T23:42:44.015 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-08-18T23:42:44.319 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-08-18T23:42:44.319 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-08-18T23:42:44.319 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-08-18T23:42:44.320 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-08-18T23:42:44.320 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-08-18T23:42:44.321 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-08-18T23:42:44.321 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-08-18T23:42:44.321 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-08-18T23:42:44.322 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-08-18T23:42:44.322 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-08-18T23:42:44.323 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-08-18T23:42:44.364 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:44 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2332541668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:42:44.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:44 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2332541668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:42:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:44 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2332541668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:42:45.009 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:45.009 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:45.014 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:45.025 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:45.070 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:45.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:45 smithi078 ceph-mon[18440]: pgmap v301: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-18T23:42:45.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:45 smithi123 ceph-mon[21808]: pgmap v301: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-18T23:42:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:45 smithi161 ceph-mon[22027]: pgmap v301: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-18T23:42:46.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.118+0000 7f2fc0667640 1 -- 172.21.15.78:0/526553110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb81018c0 msgr2=0x7f2fb8101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:46.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.118+0000 7f2fc0667640 1 --2- 172.21.15.78:0/526553110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb81018c0 0x7f2fb8101d20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009930 tx=0x7f2fa802a420 comp rx=0 tx=0).stop 2024-08-18T23:42:46.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.118+0000 7f2fc0667640 1 -- 172.21.15.78:0/526553110 shutdown_connections 2024-08-18T23:42:46.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.118+0000 7f2fc0667640 1 --2- 172.21.15.78:0/526553110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fb8102260 0x7f2fb810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.118+0000 7f2fc0667640 1 --2- 172.21.15.78:0/526553110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb81018c0 0x7f2fb8101d20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.118+0000 7f2fc0667640 1 --2- 172.21.15.78:0/526553110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fb81078c0 0x7f2fb8107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.118+0000 7f2fc0667640 1 -- 172.21.15.78:0/526553110 >> 172.21.15.78:0/526553110 conn(0x7f2fb80fd5e0 msgr2=0x7f2fb80ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:46.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.118+0000 7f2fc0667640 1 -- 172.21.15.78:0/526553110 shutdown_connections 2024-08-18T23:42:46.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.118+0000 7f2fc0667640 1 -- 172.21.15.78:0/526553110 wait complete. 2024-08-18T23:42:46.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.119+0000 7f2fc0667640 1 Processor -- start 2024-08-18T23:42:46.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.119+0000 7f2fc0667640 1 -- start start 2024-08-18T23:42:46.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.119+0000 7f2fc0667640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fb81018c0 0x7f2fb81981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:46.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.119+0000 7f2fc0667640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb8102260 0x7f2fb81986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:46.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.119+0000 7f2fc0667640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fb81078c0 0x7f2fb819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:46.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.119+0000 7f2fc0667640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb8111d10 con 0x7f2fb8102260 2024-08-18T23:42:46.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.119+0000 7f2fc0667640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb8111b90 con 0x7f2fb81018c0 2024-08-18T23:42:46.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.119+0000 7f2fc0667640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb8111e90 con 0x7f2fb81078c0 2024-08-18T23:42:46.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.119+0000 7f2fbdc02640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb8102260 0x7f2fb81986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:46.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.119+0000 7f2fbdc02640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb8102260 0x7f2fb81986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59320/0 (socket says 172.21.15.78:59320) 2024-08-18T23:42:46.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.119+0000 7f2fbdc02640 1 -- 172.21.15.78:0/3669208312 learned_addr learned my addr 172.21.15.78:0/3669208312 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:46.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.120+0000 7f2fbec04640 1 --2- 172.21.15.78:0/3669208312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fb81078c0 0x7f2fb819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:46.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.120+0000 7f2fbe403640 1 --2- 172.21.15.78:0/3669208312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fb81018c0 0x7f2fb81981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:46.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.120+0000 7f2fbdc02640 1 -- 172.21.15.78:0/3669208312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fb81078c0 msgr2=0x7f2fb819ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.120+0000 7f2fbdc02640 1 --2- 172.21.15.78:0/3669208312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fb81078c0 0x7f2fb819ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.120+0000 7f2fbdc02640 1 -- 172.21.15.78:0/3669208312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fb81018c0 msgr2=0x7f2fb81981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.120+0000 7f2fbdc02640 1 --2- 172.21.15.78:0/3669208312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fb81018c0 0x7f2fb81981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.120+0000 7f2fbdc02640 1 -- 172.21.15.78:0/3669208312 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb819d160 con 0x7f2fb8102260 2024-08-18T23:42:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.120+0000 7f2fbdc02640 1 --2- 172.21.15.78:0/3669208312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb8102260 0x7f2fb81986f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2fa802a940 tx=0x7f2fa8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.120+0000 7f2faf7fe640 1 -- 172.21.15.78:0/3669208312 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa803f070 con 0x7f2fb8102260 2024-08-18T23:42:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.120+0000 7f2faf7fe640 1 -- 172.21.15.78:0/3669208312 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa8031c90 con 0x7f2fb8102260 2024-08-18T23:42:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.120+0000 7f2fc0667640 1 -- 172.21.15.78:0/3669208312 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb819d3f0 con 0x7f2fb8102260 2024-08-18T23:42:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.120+0000 7f2faf7fe640 1 -- 172.21.15.78:0/3669208312 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8035030 con 0x7f2fb8102260 2024-08-18T23:42:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.120+0000 7f2fc0667640 1 -- 172.21.15.78:0/3669208312 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb81a4d30 con 0x7f2fb8102260 2024-08-18T23:42:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.121+0000 7f2fc0667640 1 -- 172.21.15.78:0/3669208312 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2fb80626b0 con 0x7f2fb8102260 2024-08-18T23:42:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.123+0000 7f2faf7fe640 1 -- 172.21.15.78:0/3669208312 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2fa8042070 con 0x7f2fb8102260 2024-08-18T23:42:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.123+0000 7f2faf7fe640 1 --2- 172.21.15.78:0/3669208312 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2fa40a2c10 0x7f2fa40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.123+0000 7f2faf7fe640 1 -- 172.21.15.78:0/3669208312 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f2fa80f0090 con 0x7f2fb8102260 2024-08-18T23:42:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.123+0000 7f2fbe403640 1 --2- 172.21.15.78:0/3669208312 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2fa40a2c10 0x7f2fa40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.123+0000 7f2faf7fe640 1 --2- 172.21.15.78:0/3669208312 >> [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223] conn(0x7f2fa40adde0 0x7f2fa40b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.123+0000 7f2faf7fe640 1 -- 172.21.15.78:0/3669208312 --> [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2fa8038e90 con 0x7f2fa40adde0 2024-08-18T23:42:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.123+0000 7f2faf7fe640 1 -- 172.21.15.78:0/3669208312 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f2fa80f04e0 con 0x7f2fb8102260 2024-08-18T23:42:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.124+0000 7f2fbec04640 1 --2- 172.21.15.78:0/3669208312 >> [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223] conn(0x7f2fa40adde0 0x7f2fa40b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.124+0000 7f2fbe403640 1 --2- 172.21.15.78:0/3669208312 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2fa40a2c10 0x7f2fa40a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4005fd0 tx=0x7f2fb4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.124+0000 7f2fbec04640 1 --2- 172.21.15.78:0/3669208312 >> [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223] conn(0x7f2fa40adde0 0x7f2fa40b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.125+0000 7f2faf7fe640 1 -- 172.21.15.78:0/3669208312 <== osd.6 v2:172.21.15.78:6818/95383223 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2fa8038e90 con 0x7f2fa40adde0 2024-08-18T23:42:46.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.139+0000 7f2fc0667640 1 -- 172.21.15.78:0/3669208312 --> [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2fb8062b30 con 0x7f2fa40adde0 2024-08-18T23:42:46.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.139+0000 7f2faf7fe640 1 -- 172.21.15.78:0/3669208312 <== osd.6 v2:172.21.15.78:6818/95383223 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2fb8062b30 con 0x7f2fa40adde0 2024-08-18T23:42:46.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.139+0000 7f2fc0667640 1 -- 172.21.15.78:0/3669208312 >> [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223] conn(0x7f2fa40adde0 msgr2=0x7f2fa40b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:46.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.139+0000 7f2fc0667640 1 --2- 172.21.15.78:0/3669208312 >> [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223] conn(0x7f2fa40adde0 0x7f2fa40b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.139+0000 7f2fc0667640 1 -- 172.21.15.78:0/3669208312 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2fa40a2c10 msgr2=0x7f2fa40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:46.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.139+0000 7f2fc0667640 1 --2- 172.21.15.78:0/3669208312 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2fa40a2c10 0x7f2fa40a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4005fd0 tx=0x7f2fb4005850 comp rx=0 tx=0).stop 2024-08-18T23:42:46.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.139+0000 7f2fc0667640 1 -- 172.21.15.78:0/3669208312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb8102260 msgr2=0x7f2fb81986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:46.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.139+0000 7f2fc0667640 1 --2- 172.21.15.78:0/3669208312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb8102260 0x7f2fb81986f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2fa802a940 tx=0x7f2fa8003b90 comp rx=0 tx=0).stop 2024-08-18T23:42:46.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.139+0000 7f2fbec04640 1 -- 172.21.15.78:0/3669208312 reap_dead start 2024-08-18T23:42:46.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.139+0000 7f2fc0667640 1 -- 172.21.15.78:0/3669208312 shutdown_connections 2024-08-18T23:42:46.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.139+0000 7f2fc0667640 1 -- 172.21.15.78:0/3669208312 >> 172.21.15.78:0/3669208312 conn(0x7f2fb80fd5e0 msgr2=0x7f2fb80fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:46.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.140+0000 7f2fc0667640 1 -- 172.21.15.78:0/3669208312 shutdown_connections 2024-08-18T23:42:46.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.140+0000 7f2fc0667640 1 -- 172.21.15.78:0/3669208312 wait complete. 2024-08-18T23:42:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.017+0000 7fdd03bb2640 1 -- 172.21.15.78:0/3267081456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc101150 msgr2=0x7fdcfc10d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.017+0000 7fdd03bb2640 1 --2- 172.21.15.78:0/3267081456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc101150 0x7fdcfc10d680 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4009cd0 tx=0x7fdcf402a8e0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.017+0000 7fdd03bb2640 1 -- 172.21.15.78:0/3267081456 shutdown_connections 2024-08-18T23:42:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.017+0000 7fdd03bb2640 1 --2- 172.21.15.78:0/3267081456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc101150 0x7fdcfc10d680 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.017+0000 7fdd03bb2640 1 --2- 172.21.15.78:0/3267081456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcfc1007b0 0x7fdcfc100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.017+0000 7fdd03bb2640 1 --2- 172.21.15.78:0/3267081456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcfc1067b0 0x7fdcfc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.017+0000 7fdd03bb2640 1 -- 172.21.15.78:0/3267081456 >> 172.21.15.78:0/3267081456 conn(0x7fdcfc0fc4d0 msgr2=0x7fdcfc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.017+0000 7fdd03bb2640 1 -- 172.21.15.78:0/3267081456 shutdown_connections 2024-08-18T23:42:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.017+0000 7fdd03bb2640 1 -- 172.21.15.78:0/3267081456 wait complete. 2024-08-18T23:42:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.018+0000 7fdd03bb2640 1 Processor -- start 2024-08-18T23:42:46.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.018+0000 7fdd03bb2640 1 -- start start 2024-08-18T23:42:46.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.018+0000 7fdd03bb2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcfc1007b0 0x7fdcfc198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:46.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.018+0000 7fdd03bb2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc101150 0x7fdcfc1994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:46.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.018+0000 7fdd03bb2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcfc1067b0 0x7fdcfc1a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:46.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.018+0000 7fdd03bb2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc110bb0 con 0x7fdcfc101150 2024-08-18T23:42:46.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.018+0000 7fdd03bb2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc110a30 con 0x7fdcfc1007b0 2024-08-18T23:42:46.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.018+0000 7fdd03bb2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc110d30 con 0x7fdcfc1067b0 2024-08-18T23:42:46.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.018+0000 7fdd0114d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc101150 0x7fdcfc1994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:46.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.018+0000 7fdd0114d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc101150 0x7fdcfc1994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59300/0 (socket says 172.21.15.78:59300) 2024-08-18T23:42:46.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.018+0000 7fdd0114d640 1 -- 172.21.15.78:0/2694890484 learned_addr learned my addr 172.21.15.78:0/2694890484 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:46.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.018+0000 7fdd0194e640 1 --2- 172.21.15.78:0/2694890484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcfc1007b0 0x7fdcfc198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:46.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.018+0000 7fdd0214f640 1 --2- 172.21.15.78:0/2694890484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcfc1067b0 0x7fdcfc1a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:46.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.018+0000 7fdd0114d640 1 -- 172.21.15.78:0/2694890484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcfc1067b0 msgr2=0x7fdcfc1a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:46.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.018+0000 7fdd0114d640 1 --2- 172.21.15.78:0/2694890484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdcfc1067b0 0x7fdcfc1a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.018+0000 7fdd0114d640 1 -- 172.21.15.78:0/2694890484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcfc1007b0 msgr2=0x7fdcfc198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:46.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.018+0000 7fdd0114d640 1 --2- 172.21.15.78:0/2694890484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcfc1007b0 0x7fdcfc198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.018+0000 7fdd0114d640 1 -- 172.21.15.78:0/2694890484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcfc1a4540 con 0x7fdcfc101150 2024-08-18T23:42:46.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.019+0000 7fdd0114d640 1 --2- 172.21.15.78:0/2694890484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc101150 0x7fdcfc1994c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00d950 tx=0x7fdcec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:46.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.019+0000 7fdcf2ffd640 1 -- 172.21.15.78:0/2694890484 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec017070 con 0x7fdcfc101150 2024-08-18T23:42:46.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.019+0000 7fdd03bb2640 1 -- 172.21.15.78:0/2694890484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcfc1a4710 con 0x7fdcfc101150 2024-08-18T23:42:46.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.019+0000 7fdcf2ffd640 1 -- 172.21.15.78:0/2694890484 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcec009c30 con 0x7fdcfc101150 2024-08-18T23:42:46.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.019+0000 7fdcf2ffd640 1 -- 172.21.15.78:0/2694890484 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec00f6b0 con 0x7fdcfc101150 2024-08-18T23:42:46.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.019+0000 7fdd03bb2640 1 -- 172.21.15.78:0/2694890484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcfc1a4c50 con 0x7fdcfc101150 2024-08-18T23:42:46.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.020+0000 7fdd03bb2640 1 -- 172.21.15.78:0/2694890484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdcfc0fd630 con 0x7fdcfc101150 2024-08-18T23:42:46.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.022+0000 7fdcf2ffd640 1 -- 172.21.15.78:0/2694890484 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdcec0109d0 con 0x7fdcfc101150 2024-08-18T23:42:46.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.022+0000 7fdcf2ffd640 1 --2- 172.21.15.78:0/2694890484 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdce80a2c10 0x7fdce80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:46.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.022+0000 7fdd0194e640 1 --2- 172.21.15.78:0/2694890484 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdce80a2c10 0x7fdce80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:46.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.022+0000 7fdcf2ffd640 1 -- 172.21.15.78:0/2694890484 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fdcec0cf5c0 con 0x7fdcfc101150 2024-08-18T23:42:46.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.022+0000 7fdcf2ffd640 1 --2- 172.21.15.78:0/2694890484 >> [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661] conn(0x7fdce80adde0 0x7fdce80b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:46.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.023+0000 7fdcf2ffd640 1 -- 172.21.15.78:0/2694890484 --> [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdce80b0950 con 0x7fdce80adde0 2024-08-18T23:42:46.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.023+0000 7fdcf2ffd640 1 -- 172.21.15.78:0/2694890484 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fdcec0cfa10 con 0x7fdcfc101150 2024-08-18T23:42:46.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.023+0000 7fdd0214f640 1 --2- 172.21.15.78:0/2694890484 >> [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661] conn(0x7fdce80adde0 0x7fdce80b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:46.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.023+0000 7fdd0194e640 1 --2- 172.21.15.78:0/2694890484 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdce80a2c10 0x7fdce80a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8005fd0 tx=0x7fdcf8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:46.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.023+0000 7fdd0214f640 1 --2- 172.21.15.78:0/2694890484 >> [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661] conn(0x7fdce80adde0 0x7fdce80b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:46.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.024+0000 7fdcf2ffd640 1 -- 172.21.15.78:0/2694890484 <== osd.9 v2:172.21.15.78:6826/2884475661 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdce80b0950 con 0x7fdce80adde0 2024-08-18T23:42:46.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.038+0000 7fdd03bb2640 1 -- 172.21.15.78:0/2694890484 --> [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdcfc101ef0 con 0x7fdce80adde0 2024-08-18T23:42:46.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.038+0000 7fdcf2ffd640 1 -- 172.21.15.78:0/2694890484 <== osd.9 v2:172.21.15.78:6826/2884475661 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdcfc101ef0 con 0x7fdce80adde0 2024-08-18T23:42:46.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.038+0000 7fdd03bb2640 1 -- 172.21.15.78:0/2694890484 >> [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661] conn(0x7fdce80adde0 msgr2=0x7fdce80b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:46.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.038+0000 7fdd03bb2640 1 --2- 172.21.15.78:0/2694890484 >> [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661] conn(0x7fdce80adde0 0x7fdce80b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.038+0000 7fdd03bb2640 1 -- 172.21.15.78:0/2694890484 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdce80a2c10 msgr2=0x7fdce80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:46.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.038+0000 7fdd03bb2640 1 --2- 172.21.15.78:0/2694890484 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdce80a2c10 0x7fdce80a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8005fd0 tx=0x7fdcf8005850 comp rx=0 tx=0).stop 2024-08-18T23:42:46.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.038+0000 7fdd03bb2640 1 -- 172.21.15.78:0/2694890484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc101150 msgr2=0x7fdcfc1994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:46.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.038+0000 7fdd03bb2640 1 --2- 172.21.15.78:0/2694890484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcfc101150 0x7fdcfc1994c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00d950 tx=0x7fdcec00de10 comp rx=0 tx=0).stop 2024-08-18T23:42:46.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.038+0000 7fdd0214f640 1 -- 172.21.15.78:0/2694890484 reap_dead start 2024-08-18T23:42:46.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.039+0000 7fdd03bb2640 1 -- 172.21.15.78:0/2694890484 shutdown_connections 2024-08-18T23:42:46.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.039+0000 7fdd03bb2640 1 -- 172.21.15.78:0/2694890484 >> 172.21.15.78:0/2694890484 conn(0x7fdcfc0fc4d0 msgr2=0x7fdcfc0fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:46.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.039+0000 7fdd03bb2640 1 -- 172.21.15.78:0/2694890484 shutdown_connections 2024-08-18T23:42:46.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.039+0000 7fdd03bb2640 1 -- 172.21.15.78:0/2694890484 wait complete. 2024-08-18T23:42:46.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.651+0000 7fcaa80be640 1 -- 172.21.15.78:0/2336782372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa0102b00 msgr2=0x7fcaa0102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:46.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.651+0000 7fcaa80be640 1 --2- 172.21.15.78:0/2336782372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa0102b00 0x7fcaa0102ee0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009930 tx=0x7fca9c02a440 comp rx=0 tx=0).stop 2024-08-18T23:42:46.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.651+0000 7fcaa80be640 1 -- 172.21.15.78:0/2336782372 shutdown_connections 2024-08-18T23:42:46.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.651+0000 7fcaa80be640 1 --2- 172.21.15.78:0/2336782372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcaa0108030 0x7fcaa010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.651+0000 7fcaa80be640 1 --2- 172.21.15.78:0/2336782372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaa01034b0 0x7fcaa01078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.651+0000 7fcaa80be640 1 --2- 172.21.15.78:0/2336782372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa0102b00 0x7fcaa0102ee0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.651+0000 7fcaa80be640 1 -- 172.21.15.78:0/2336782372 >> 172.21.15.78:0/2336782372 conn(0x7fcaa00fc500 msgr2=0x7fcaa00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:46.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.651+0000 7fcaa80be640 1 -- 172.21.15.78:0/2336782372 shutdown_connections 2024-08-18T23:42:46.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.651+0000 7fcaa80be640 1 -- 172.21.15.78:0/2336782372 wait complete. 2024-08-18T23:42:46.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa80be640 1 Processor -- start 2024-08-18T23:42:46.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa80be640 1 -- start start 2024-08-18T23:42:46.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa80be640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcaa0102b00 0x7fcaa0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:46.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa80be640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa01034b0 0x7fcaa0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:46.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa80be640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaa0108030 0x7fcaa019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:46.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa80be640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa0111e20 con 0x7fcaa01034b0 2024-08-18T23:42:46.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa80be640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa0111ca0 con 0x7fcaa0102b00 2024-08-18T23:42:46.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa80be640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa0111fa0 con 0x7fcaa0108030 2024-08-18T23:42:46.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa5659640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa01034b0 0x7fcaa0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:46.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa5659640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa01034b0 0x7fcaa0198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59334/0 (socket says 172.21.15.78:59334) 2024-08-18T23:42:46.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa5659640 1 -- 172.21.15.78:0/2221626104 learned_addr learned my addr 172.21.15.78:0/2221626104 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:46.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa665b640 1 --2- 172.21.15.78:0/2221626104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaa0108030 0x7fcaa019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:46.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa5e5a640 1 --2- 172.21.15.78:0/2221626104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcaa0102b00 0x7fcaa0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:46.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa5659640 1 -- 172.21.15.78:0/2221626104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaa0108030 msgr2=0x7fcaa019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:46.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa5659640 1 --2- 172.21.15.78:0/2221626104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaa0108030 0x7fcaa019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa5659640 1 -- 172.21.15.78:0/2221626104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcaa0102b00 msgr2=0x7fcaa0198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:46.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa5659640 1 --2- 172.21.15.78:0/2221626104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcaa0102b00 0x7fcaa0198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa5659640 1 -- 172.21.15.78:0/2221626104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa019d2e0 con 0x7fcaa01034b0 2024-08-18T23:42:46.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa5e5a640 1 --2- 172.21.15.78:0/2221626104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcaa0102b00 0x7fcaa0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:42:46.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.652+0000 7fcaa5659640 1 --2- 172.21.15.78:0/2221626104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa01034b0 0x7fcaa0198870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fca94006fd0 tx=0x7fca9400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:46.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.653+0000 7fca92ffd640 1 -- 172.21.15.78:0/2221626104 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94011830 con 0x7fcaa01034b0 2024-08-18T23:42:46.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.653+0000 7fca92ffd640 1 -- 172.21.15.78:0/2221626104 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca94016070 con 0x7fcaa01034b0 2024-08-18T23:42:46.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.653+0000 7fcaa80be640 1 -- 172.21.15.78:0/2221626104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa019d5d0 con 0x7fcaa01034b0 2024-08-18T23:42:46.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.653+0000 7fca92ffd640 1 -- 172.21.15.78:0/2221626104 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9400f2e0 con 0x7fcaa01034b0 2024-08-18T23:42:46.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.653+0000 7fcaa80be640 1 -- 172.21.15.78:0/2221626104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa01a4eb0 con 0x7fcaa01034b0 2024-08-18T23:42:46.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.653+0000 7fcaa80be640 1 -- 172.21.15.78:0/2221626104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcaa0062630 con 0x7fcaa01034b0 2024-08-18T23:42:46.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.655+0000 7fca92ffd640 1 -- 172.21.15.78:0/2221626104 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fca9401b460 con 0x7fcaa01034b0 2024-08-18T23:42:46.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.656+0000 7fca92ffd640 1 --2- 172.21.15.78:0/2221626104 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fca8c0a2c90 0x7fca8c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:46.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.656+0000 7fcaa5e5a640 1 --2- 172.21.15.78:0/2221626104 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fca8c0a2c90 0x7fca8c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:46.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.656+0000 7fca92ffd640 1 -- 172.21.15.78:0/2221626104 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fca940cf620 con 0x7fcaa01034b0 2024-08-18T23:42:46.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.656+0000 7fca92ffd640 1 --2- 172.21.15.78:0/2221626104 >> [v2:172.21.15.123:6800/4199446368,v1:172.21.15.123:6801/4199446368] conn(0x7fca8c0ade60 0x7fca8c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:46.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.656+0000 7fca92ffd640 1 -- 172.21.15.78:0/2221626104 --> [v2:172.21.15.123:6800/4199446368,v1:172.21.15.123:6801/4199446368] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fca8c0b09d0 con 0x7fca8c0ade60 2024-08-18T23:42:46.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.656+0000 7fca92ffd640 1 -- 172.21.15.78:0/2221626104 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fca940cf9d0 con 0x7fcaa01034b0 2024-08-18T23:42:46.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.656+0000 7fcaa5e5a640 1 --2- 172.21.15.78:0/2221626104 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fca8c0a2c90 0x7fca8c0a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0096c0 tx=0x7fca9c032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:46.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.656+0000 7fcaa665b640 1 --2- 172.21.15.78:0/2221626104 >> [v2:172.21.15.123:6800/4199446368,v1:172.21.15.123:6801/4199446368] conn(0x7fca8c0ade60 0x7fca8c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:46.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.657+0000 7fcaa665b640 1 --2- 172.21.15.78:0/2221626104 >> [v2:172.21.15.123:6800/4199446368,v1:172.21.15.123:6801/4199446368] conn(0x7fca8c0ade60 0x7fca8c0b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:46.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.658+0000 7fca92ffd640 1 -- 172.21.15.78:0/2221626104 <== osd.1 v2:172.21.15.123:6800/4199446368 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fca8c0b09d0 con 0x7fca8c0ade60 2024-08-18T23:42:46.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.671+0000 7fcaa80be640 1 -- 172.21.15.78:0/2221626104 --> [v2:172.21.15.123:6800/4199446368,v1:172.21.15.123:6801/4199446368] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcaa0062a30 con 0x7fca8c0ade60 2024-08-18T23:42:46.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.672+0000 7fca92ffd640 1 -- 172.21.15.78:0/2221626104 <== osd.1 v2:172.21.15.123:6800/4199446368 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcaa0062a30 con 0x7fca8c0ade60 2024-08-18T23:42:46.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.672+0000 7fcaa80be640 1 -- 172.21.15.78:0/2221626104 >> [v2:172.21.15.123:6800/4199446368,v1:172.21.15.123:6801/4199446368] conn(0x7fca8c0ade60 msgr2=0x7fca8c0b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:46.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.672+0000 7fcaa80be640 1 --2- 172.21.15.78:0/2221626104 >> [v2:172.21.15.123:6800/4199446368,v1:172.21.15.123:6801/4199446368] conn(0x7fca8c0ade60 0x7fca8c0b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.672+0000 7fcaa80be640 1 -- 172.21.15.78:0/2221626104 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fca8c0a2c90 msgr2=0x7fca8c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:46.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.672+0000 7fcaa80be640 1 --2- 172.21.15.78:0/2221626104 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fca8c0a2c90 0x7fca8c0a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0096c0 tx=0x7fca9c032c70 comp rx=0 tx=0).stop 2024-08-18T23:42:46.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.672+0000 7fcaa80be640 1 -- 172.21.15.78:0/2221626104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa01034b0 msgr2=0x7fcaa0198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:46.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.672+0000 7fcaa80be640 1 --2- 172.21.15.78:0/2221626104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa01034b0 0x7fcaa0198870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fca94006fd0 tx=0x7fca9400b8e0 comp rx=0 tx=0).stop 2024-08-18T23:42:46.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.672+0000 7fcaa665b640 1 -- 172.21.15.78:0/2221626104 reap_dead start 2024-08-18T23:42:46.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.672+0000 7fcaa80be640 1 -- 172.21.15.78:0/2221626104 shutdown_connections 2024-08-18T23:42:46.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.672+0000 7fcaa80be640 1 -- 172.21.15.78:0/2221626104 >> 172.21.15.78:0/2221626104 conn(0x7fcaa00fc500 msgr2=0x7fcaa0105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:46.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.672+0000 7fcaa80be640 1 -- 172.21.15.78:0/2221626104 shutdown_connections 2024-08-18T23:42:46.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:46.673+0000 7fcaa80be640 1 -- 172.21.15.78:0/2221626104 wait complete. 2024-08-18T23:42:47.319 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:47 smithi078 ceph-mon[18440]: pgmap v302: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-08-18T23:42:47.347 INFO:teuthology.orchestra.run.smithi078.stdout:55834574921 2024-08-18T23:42:47.348 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-08-18T23:42:47.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:47 smithi123 ceph-mon[21808]: pgmap v302: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-08-18T23:42:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:47 smithi161 ceph-mon[22027]: pgmap v302: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-08-18T23:42:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:47.837 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:47.837 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:48.316 INFO:teuthology.orchestra.run.smithi078.stdout:188978561075 2024-08-18T23:42:48.317 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-08-18T23:42:49.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.980+0000 7f6048735640 1 -- 172.21.15.78:0/3290914789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6040102250 msgr2=0x7f604010e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:49.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.980+0000 7f6048735640 1 --2- 172.21.15.78:0/3290914789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6040102250 0x7f604010e780 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6030009cd0 tx=0x7f603002a8e0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.980+0000 7f6048735640 1 -- 172.21.15.78:0/3290914789 shutdown_connections 2024-08-18T23:42:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.980+0000 7f6048735640 1 --2- 172.21.15.78:0/3290914789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6040102250 0x7f604010e780 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.980+0000 7f6048735640 1 --2- 172.21.15.78:0/3290914789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60401018b0 0x7f6040101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.980+0000 7f6048735640 1 --2- 172.21.15.78:0/3290914789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f60401078b0 0x7f6040107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.980+0000 7f6048735640 1 -- 172.21.15.78:0/3290914789 >> 172.21.15.78:0/3290914789 conn(0x7f60400fd5f0 msgr2=0x7f60400ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.980+0000 7f6048735640 1 -- 172.21.15.78:0/3290914789 shutdown_connections 2024-08-18T23:42:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.980+0000 7f6048735640 1 -- 172.21.15.78:0/3290914789 wait complete. 2024-08-18T23:42:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.981+0000 7f6048735640 1 Processor -- start 2024-08-18T23:42:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.981+0000 7f6048735640 1 -- start start 2024-08-18T23:42:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.981+0000 7f6048735640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60401018b0 0x7f6040198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.981+0000 7f6048735640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6040102250 0x7f6040198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.981+0000 7f6048735640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60401078b0 0x7f604019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.981+0000 7f6048735640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6040111cd0 con 0x7f60401078b0 2024-08-18T23:42:49.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.981+0000 7f6048735640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6040111b50 con 0x7f6040102250 2024-08-18T23:42:49.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.981+0000 7f6048735640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6040111e50 con 0x7f60401018b0 2024-08-18T23:42:49.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.981+0000 7f6046cd2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60401078b0 0x7f604019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:49.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.981+0000 7f6046cd2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60401078b0 0x7f604019ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59350/0 (socket says 172.21.15.78:59350) 2024-08-18T23:42:49.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.981+0000 7f6046cd2640 1 -- 172.21.15.78:0/2463348965 learned_addr learned my addr 172.21.15.78:0/2463348965 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:49.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.981+0000 7f60464d1640 1 --2- 172.21.15.78:0/2463348965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60401018b0 0x7f6040198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:49.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.981+0000 7f6045cd0640 1 --2- 172.21.15.78:0/2463348965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6040102250 0x7f6040198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:49.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.981+0000 7f6046cd2640 1 -- 172.21.15.78:0/2463348965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60401018b0 msgr2=0x7f6040198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:49.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.981+0000 7f6046cd2640 1 --2- 172.21.15.78:0/2463348965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60401018b0 0x7f6040198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.981+0000 7f6046cd2640 1 -- 172.21.15.78:0/2463348965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6040102250 msgr2=0x7f6040198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:49.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.981+0000 7f6046cd2640 1 --2- 172.21.15.78:0/2463348965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6040102250 0x7f6040198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.981+0000 7f6046cd2640 1 -- 172.21.15.78:0/2463348965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604019d0e0 con 0x7f60401078b0 2024-08-18T23:42:49.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.981+0000 7f6045cd0640 1 --2- 172.21.15.78:0/2463348965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6040102250 0x7f6040198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:42:49.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.982+0000 7f6046cd2640 1 --2- 172.21.15.78:0/2463348965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60401078b0 0x7f604019ca00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6030006010 tx=0x7f6030003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:49.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.982+0000 7f603f7fe640 1 -- 172.21.15.78:0/2463348965 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030004450 con 0x7f60401078b0 2024-08-18T23:42:49.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.982+0000 7f603f7fe640 1 -- 172.21.15.78:0/2463348965 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6030003e10 con 0x7f60401078b0 2024-08-18T23:42:49.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.982+0000 7f6048735640 1 -- 172.21.15.78:0/2463348965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604019d370 con 0x7f60401078b0 2024-08-18T23:42:49.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.982+0000 7f603f7fe640 1 -- 172.21.15.78:0/2463348965 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60300466e0 con 0x7f60401078b0 2024-08-18T23:42:49.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.982+0000 7f6048735640 1 -- 172.21.15.78:0/2463348965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60401a4cb0 con 0x7f60401078b0 2024-08-18T23:42:49.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.983+0000 7f6048735640 1 -- 172.21.15.78:0/2463348965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f60400fe740 con 0x7f60401078b0 2024-08-18T23:42:49.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.984+0000 7f603f7fe640 1 -- 172.21.15.78:0/2463348965 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f60300305a0 con 0x7f60401078b0 2024-08-18T23:42:49.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.985+0000 7f603f7fe640 1 --2- 172.21.15.78:0/2463348965 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f60280a2c10 0x7f60280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:49.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.985+0000 7f60464d1640 1 --2- 172.21.15.78:0/2463348965 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f60280a2c10 0x7f60280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:49.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.985+0000 7f603f7fe640 1 -- 172.21.15.78:0/2463348965 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f603002e030 con 0x7f60401078b0 2024-08-18T23:42:49.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.985+0000 7f603f7fe640 1 --2- 172.21.15.78:0/2463348965 >> [v2:172.21.15.161:6816/1485008778,v1:172.21.15.161:6817/1485008778] conn(0x7f60280adde0 0x7f60280b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:49.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.985+0000 7f603f7fe640 1 -- 172.21.15.78:0/2463348965 --> [v2:172.21.15.161:6816/1485008778,v1:172.21.15.161:6817/1485008778] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60300020b0 con 0x7f60280adde0 2024-08-18T23:42:49.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.985+0000 7f603f7fe640 1 -- 172.21.15.78:0/2463348965 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f60300f1430 con 0x7f60401078b0 2024-08-18T23:42:49.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.985+0000 7f6045cd0640 1 --2- 172.21.15.78:0/2463348965 >> [v2:172.21.15.161:6816/1485008778,v1:172.21.15.161:6817/1485008778] conn(0x7f60280adde0 0x7f60280b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:49.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.985+0000 7f60464d1640 1 --2- 172.21.15.78:0/2463348965 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f60280a2c10 0x7f60280a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6038009890 tx=0x7f6038008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:49.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.986+0000 7f6045cd0640 1 --2- 172.21.15.78:0/2463348965 >> [v2:172.21.15.161:6816/1485008778,v1:172.21.15.161:6817/1485008778] conn(0x7f60280adde0 0x7f60280b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:49.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:48.986+0000 7f603f7fe640 1 -- 172.21.15.78:0/2463348965 <== osd.8 v2:172.21.15.161:6816/1485008778 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60300020b0 con 0x7f60280adde0 2024-08-18T23:42:49.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.000+0000 7f6048735640 1 -- 172.21.15.78:0/2463348965 --> [v2:172.21.15.161:6816/1485008778,v1:172.21.15.161:6817/1485008778] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6040102ff0 con 0x7f60280adde0 2024-08-18T23:42:49.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.000+0000 7f603f7fe640 1 -- 172.21.15.78:0/2463348965 <== osd.8 v2:172.21.15.161:6816/1485008778 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6040102ff0 con 0x7f60280adde0 2024-08-18T23:42:49.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.001+0000 7f6048735640 1 -- 172.21.15.78:0/2463348965 >> [v2:172.21.15.161:6816/1485008778,v1:172.21.15.161:6817/1485008778] conn(0x7f60280adde0 msgr2=0x7f60280b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:49.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.001+0000 7f6048735640 1 --2- 172.21.15.78:0/2463348965 >> [v2:172.21.15.161:6816/1485008778,v1:172.21.15.161:6817/1485008778] conn(0x7f60280adde0 0x7f60280b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.001+0000 7f6048735640 1 -- 172.21.15.78:0/2463348965 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f60280a2c10 msgr2=0x7f60280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:49.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.001+0000 7f6048735640 1 --2- 172.21.15.78:0/2463348965 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f60280a2c10 0x7f60280a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6038009890 tx=0x7f6038008040 comp rx=0 tx=0).stop 2024-08-18T23:42:49.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.001+0000 7f6048735640 1 -- 172.21.15.78:0/2463348965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60401078b0 msgr2=0x7f604019ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:49.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.001+0000 7f6048735640 1 --2- 172.21.15.78:0/2463348965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60401078b0 0x7f604019ca00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6030006010 tx=0x7f6030003c20 comp rx=0 tx=0).stop 2024-08-18T23:42:49.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.001+0000 7f6046cd2640 1 -- 172.21.15.78:0/2463348965 reap_dead start 2024-08-18T23:42:49.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.001+0000 7f6048735640 1 -- 172.21.15.78:0/2463348965 shutdown_connections 2024-08-18T23:42:49.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.001+0000 7f6048735640 1 -- 172.21.15.78:0/2463348965 >> 172.21.15.78:0/2463348965 conn(0x7f60400fd5f0 msgr2=0x7f60400fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:49.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.001+0000 7f6048735640 1 -- 172.21.15.78:0/2463348965 shutdown_connections 2024-08-18T23:42:49.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.001+0000 7f6048735640 1 -- 172.21.15.78:0/2463348965 wait complete. 2024-08-18T23:42:49.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.405+0000 7ff812379640 1 -- 172.21.15.78:0/2758492590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff80c10bea0 msgr2=0x7ff80c10e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:49.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.405+0000 7ff812379640 1 --2- 172.21.15.78:0/2758492590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff80c10bea0 0x7ff80c10e340 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff804009ca0 tx=0x7ff80402a950 comp rx=0 tx=0).stop 2024-08-18T23:42:49.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.405+0000 7ff812379640 1 -- 172.21.15.78:0/2758492590 shutdown_connections 2024-08-18T23:42:49.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.405+0000 7ff812379640 1 --2- 172.21.15.78:0/2758492590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff80c10bea0 0x7ff80c10e340 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.405+0000 7ff812379640 1 --2- 172.21.15.78:0/2758492590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c103390 0x7ff80c10b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.405+0000 7ff812379640 1 --2- 172.21.15.78:0/2758492590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff80c102a70 0x7ff80c102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.405+0000 7ff812379640 1 -- 172.21.15.78:0/2758492590 >> 172.21.15.78:0/2758492590 conn(0x7ff80c0fc4d0 msgr2=0x7ff80c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.405+0000 7ff812379640 1 -- 172.21.15.78:0/2758492590 shutdown_connections 2024-08-18T23:42:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.405+0000 7ff812379640 1 -- 172.21.15.78:0/2758492590 wait complete. 2024-08-18T23:42:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.406+0000 7ff812379640 1 Processor -- start 2024-08-18T23:42:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.406+0000 7ff812379640 1 -- start start 2024-08-18T23:42:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.406+0000 7ff812379640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c102a70 0x7ff80c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.406+0000 7ff812379640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff80c103390 0x7ff80c19a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.406+0000 7ff812379640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff80c10bea0 0x7ff80c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.406+0000 7ff812379640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c111ca0 con 0x7ff80c103390 2024-08-18T23:42:49.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.406+0000 7ff812379640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c111b20 con 0x7ff80c10bea0 2024-08-18T23:42:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.406+0000 7ff812379640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c111e20 con 0x7ff80c102a70 2024-08-18T23:42:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.406+0000 7ff80b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff80c103390 0x7ff80c19a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.406+0000 7ff80b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff80c103390 0x7ff80c19a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59368/0 (socket says 172.21.15.78:59368) 2024-08-18T23:42:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.406+0000 7ff80b7fe640 1 -- 172.21.15.78:0/4140140499 learned_addr learned my addr 172.21.15.78:0/4140140499 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.407+0000 7ff80bfff640 1 --2- 172.21.15.78:0/4140140499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c102a70 0x7ff80c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.407+0000 7ff810916640 1 --2- 172.21.15.78:0/4140140499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff80c10bea0 0x7ff80c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.407+0000 7ff80b7fe640 1 -- 172.21.15.78:0/4140140499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c102a70 msgr2=0x7ff80c19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.407+0000 7ff80b7fe640 1 --2- 172.21.15.78:0/4140140499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c102a70 0x7ff80c19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.407+0000 7ff80b7fe640 1 -- 172.21.15.78:0/4140140499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff80c10bea0 msgr2=0x7ff80c19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.407+0000 7ff80b7fe640 1 --2- 172.21.15.78:0/4140140499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff80c10bea0 0x7ff80c19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.407+0000 7ff80b7fe640 1 -- 172.21.15.78:0/4140140499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80c19f240 con 0x7ff80c103390 2024-08-18T23:42:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.407+0000 7ff80bfff640 1 --2- 172.21.15.78:0/4140140499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff80c102a70 0x7ff80c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:42:49.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.407+0000 7ff80b7fe640 1 --2- 172.21.15.78:0/4140140499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff80c103390 0x7ff80c19a7d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff80000c970 tx=0x7ff80000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.407+0000 7ff8097fa640 1 -- 172.21.15.78:0/4140140499 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff800007bb0 con 0x7ff80c103390 2024-08-18T23:42:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.407+0000 7ff8097fa640 1 -- 172.21.15.78:0/4140140499 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff800007d50 con 0x7ff80c103390 2024-08-18T23:42:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.407+0000 7ff812379640 1 -- 172.21.15.78:0/4140140499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80c19f530 con 0x7ff80c103390 2024-08-18T23:42:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.407+0000 7ff8097fa640 1 -- 172.21.15.78:0/4140140499 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff800018820 con 0x7ff80c103390 2024-08-18T23:42:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.407+0000 7ff812379640 1 -- 172.21.15.78:0/4140140499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff80c1a6e10 con 0x7ff80c103390 2024-08-18T23:42:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.408+0000 7ff812379640 1 -- 172.21.15.78:0/4140140499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff80c0fea80 con 0x7ff80c103390 2024-08-18T23:42:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.409+0000 7ff8097fa640 1 -- 172.21.15.78:0/4140140499 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff8000189c0 con 0x7ff80c103390 2024-08-18T23:42:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.410+0000 7ff8097fa640 1 --2- 172.21.15.78:0/4140140499 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff7f40a2c90 0x7ff7f40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.410+0000 7ff80bfff640 1 --2- 172.21.15.78:0/4140140499 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff7f40a2c90 0x7ff7f40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.410+0000 7ff8097fa640 1 -- 172.21.15.78:0/4140140499 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff8000cf5a0 con 0x7ff80c103390 2024-08-18T23:42:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.410+0000 7ff8097fa640 1 --2- 172.21.15.78:0/4140140499 >> [v2:172.21.15.123:6824/122226286,v1:172.21.15.123:6825/122226286] conn(0x7ff7f40ade50 0x7ff7f40b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:49.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.410+0000 7ff8097fa640 1 -- 172.21.15.78:0/4140140499 --> [v2:172.21.15.123:6824/122226286,v1:172.21.15.123:6825/122226286] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff800012ec0 con 0x7ff7f40ade50 2024-08-18T23:42:49.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.410+0000 7ff8097fa640 1 -- 172.21.15.78:0/4140140499 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ff8000d6050 con 0x7ff80c103390 2024-08-18T23:42:49.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.410+0000 7ff80bfff640 1 --2- 172.21.15.78:0/4140140499 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff7f40a2c90 0x7ff7f40a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00abe0 tx=0x7ff7fc00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:49.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.410+0000 7ff810916640 1 --2- 172.21.15.78:0/4140140499 >> [v2:172.21.15.123:6824/122226286,v1:172.21.15.123:6825/122226286] conn(0x7ff7f40ade50 0x7ff7f40b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:49.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.411+0000 7ff810916640 1 --2- 172.21.15.78:0/4140140499 >> [v2:172.21.15.123:6824/122226286,v1:172.21.15.123:6825/122226286] conn(0x7ff7f40ade50 0x7ff7f40b02f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:49.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.411+0000 7ff8097fa640 1 -- 172.21.15.78:0/4140140499 <== osd.10 v2:172.21.15.123:6824/122226286 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff800012ec0 con 0x7ff7f40ade50 2024-08-18T23:42:49.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.425+0000 7ff812379640 1 -- 172.21.15.78:0/4140140499 --> [v2:172.21.15.123:6824/122226286,v1:172.21.15.123:6825/122226286] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff80c06ada0 con 0x7ff7f40ade50 2024-08-18T23:42:49.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.426+0000 7ff8097fa640 1 -- 172.21.15.78:0/4140140499 <== osd.10 v2:172.21.15.123:6824/122226286 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff80c06ada0 con 0x7ff7f40ade50 2024-08-18T23:42:49.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.426+0000 7ff812379640 1 -- 172.21.15.78:0/4140140499 >> [v2:172.21.15.123:6824/122226286,v1:172.21.15.123:6825/122226286] conn(0x7ff7f40ade50 msgr2=0x7ff7f40b02f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:49.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.426+0000 7ff812379640 1 --2- 172.21.15.78:0/4140140499 >> [v2:172.21.15.123:6824/122226286,v1:172.21.15.123:6825/122226286] conn(0x7ff7f40ade50 0x7ff7f40b02f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.426+0000 7ff812379640 1 -- 172.21.15.78:0/4140140499 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff7f40a2c90 msgr2=0x7ff7f40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:49.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.426+0000 7ff812379640 1 --2- 172.21.15.78:0/4140140499 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff7f40a2c90 0x7ff7f40a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00abe0 tx=0x7ff7fc00a5c0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.426+0000 7ff812379640 1 -- 172.21.15.78:0/4140140499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff80c103390 msgr2=0x7ff80c19a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:49.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.426+0000 7ff812379640 1 --2- 172.21.15.78:0/4140140499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff80c103390 0x7ff80c19a7d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff80000c970 tx=0x7ff80000ce30 comp rx=0 tx=0).stop 2024-08-18T23:42:49.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.426+0000 7ff810916640 1 -- 172.21.15.78:0/4140140499 reap_dead start 2024-08-18T23:42:49.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.426+0000 7ff812379640 1 -- 172.21.15.78:0/4140140499 shutdown_connections 2024-08-18T23:42:49.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.426+0000 7ff812379640 1 -- 172.21.15.78:0/4140140499 >> 172.21.15.78:0/4140140499 conn(0x7ff80c0fc4d0 msgr2=0x7ff80c0fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:49.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.426+0000 7ff812379640 1 -- 172.21.15.78:0/4140140499 shutdown_connections 2024-08-18T23:42:49.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.426+0000 7ff812379640 1 -- 172.21.15.78:0/4140140499 wait complete. 2024-08-18T23:42:49.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:49 smithi078 ceph-mon[18440]: pgmap v303: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-18T23:42:49.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:49 smithi123 ceph-mon[21808]: pgmap v303: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-18T23:42:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:49 smithi161 ceph-mon[22027]: pgmap v303: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-18T23:42:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.843+0000 7efed4da8640 1 -- 172.21.15.78:0/3756960343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed01078b0 msgr2=0x7efed0107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.843+0000 7efed4da8640 1 --2- 172.21.15.78:0/3756960343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed01078b0 0x7efed0107c90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efec0009930 tx=0x7efec002a420 comp rx=0 tx=0).stop 2024-08-18T23:42:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.843+0000 7efed4da8640 1 -- 172.21.15.78:0/3756960343 shutdown_connections 2024-08-18T23:42:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.843+0000 7efed4da8640 1 --2- 172.21.15.78:0/3756960343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efed0102250 0x7efed010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.843+0000 7efed4da8640 1 --2- 172.21.15.78:0/3756960343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efed01018b0 0x7efed0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.843+0000 7efed4da8640 1 --2- 172.21.15.78:0/3756960343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed01078b0 0x7efed0107c90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.843+0000 7efed4da8640 1 -- 172.21.15.78:0/3756960343 >> 172.21.15.78:0/3756960343 conn(0x7efed00fd5f0 msgr2=0x7efed00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.843+0000 7efed4da8640 1 -- 172.21.15.78:0/3756960343 shutdown_connections 2024-08-18T23:42:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.843+0000 7efed4da8640 1 -- 172.21.15.78:0/3756960343 wait complete. 2024-08-18T23:42:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.844+0000 7efed4da8640 1 Processor -- start 2024-08-18T23:42:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.844+0000 7efed4da8640 1 -- start start 2024-08-18T23:42:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.844+0000 7efed4da8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efed01018b0 0x7efed0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.844+0000 7efed4da8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed0102250 0x7efed0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.844+0000 7efed4da8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efed01078b0 0x7efed019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.844+0000 7efed4da8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efed0111cb0 con 0x7efed0102250 2024-08-18T23:42:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.844+0000 7efed4da8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efed0111b30 con 0x7efed01018b0 2024-08-18T23:42:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.844+0000 7efed4da8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efed0111e30 con 0x7efed01078b0 2024-08-18T23:42:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.844+0000 7efec7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed0102250 0x7efed0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.844+0000 7efec7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed0102250 0x7efed0198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59382/0 (socket says 172.21.15.78:59382) 2024-08-18T23:42:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.844+0000 7efec7fff640 1 -- 172.21.15.78:0/3888195852 learned_addr learned my addr 172.21.15.78:0/3888195852 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.844+0000 7efeced9d640 1 --2- 172.21.15.78:0/3888195852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efed01078b0 0x7efed019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.844+0000 7efece59c640 1 --2- 172.21.15.78:0/3888195852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efed01018b0 0x7efed0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.844+0000 7efec7fff640 1 -- 172.21.15.78:0/3888195852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efed01078b0 msgr2=0x7efed019cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.844+0000 7efec7fff640 1 --2- 172.21.15.78:0/3888195852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efed01078b0 0x7efed019cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.844+0000 7efec7fff640 1 -- 172.21.15.78:0/3888195852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efed01018b0 msgr2=0x7efed0198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.844+0000 7efec7fff640 1 --2- 172.21.15.78:0/3888195852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efed01018b0 0x7efed0198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.844+0000 7efec7fff640 1 -- 172.21.15.78:0/3888195852 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed019d1b0 con 0x7efed0102250 2024-08-18T23:42:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.845+0000 7efec7fff640 1 --2- 172.21.15.78:0/3888195852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed0102250 0x7efed0198740 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efeb800c970 tx=0x7efeb800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.845+0000 7efec77fe640 1 -- 172.21.15.78:0/3888195852 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8007bb0 con 0x7efed0102250 2024-08-18T23:42:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.845+0000 7efec77fe640 1 -- 172.21.15.78:0/3888195852 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeb8007d50 con 0x7efed0102250 2024-08-18T23:42:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.845+0000 7efed4da8640 1 -- 172.21.15.78:0/3888195852 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed019d4a0 con 0x7efed0102250 2024-08-18T23:42:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.845+0000 7efec77fe640 1 -- 172.21.15.78:0/3888195852 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb800f450 con 0x7efed0102250 2024-08-18T23:42:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.845+0000 7efed4da8640 1 -- 172.21.15.78:0/3888195852 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed01a4d80 con 0x7efed0102250 2024-08-18T23:42:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.846+0000 7efed4da8640 1 -- 172.21.15.78:0/3888195852 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efed00626b0 con 0x7efed0102250 2024-08-18T23:42:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.847+0000 7efec77fe640 1 -- 172.21.15.78:0/3888195852 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efeb8004fa0 con 0x7efed0102250 2024-08-18T23:42:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.848+0000 7efec77fe640 1 --2- 172.21.15.78:0/3888195852 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efebc0a2c10 0x7efebc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.848+0000 7efece59c640 1 --2- 172.21.15.78:0/3888195852 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efebc0a2c10 0x7efebc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.848+0000 7efec77fe640 1 -- 172.21.15.78:0/3888195852 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7efeb80cf780 con 0x7efed0102250 2024-08-18T23:42:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.848+0000 7efec77fe640 1 --2- 172.21.15.78:0/3888195852 >> [v2:172.21.15.161:6800/2582810057,v1:172.21.15.161:6801/2582810057] conn(0x7efebc0adde0 0x7efebc0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:49.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.848+0000 7efec77fe640 1 -- 172.21.15.78:0/3888195852 --> [v2:172.21.15.161:6800/2582810057,v1:172.21.15.161:6801/2582810057] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efebc0b0950 con 0x7efebc0adde0 2024-08-18T23:42:49.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.848+0000 7efec77fe640 1 -- 172.21.15.78:0/3888195852 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7efeb80cfbd0 con 0x7efed0102250 2024-08-18T23:42:49.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.848+0000 7efece59c640 1 --2- 172.21.15.78:0/3888195852 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efebc0a2c10 0x7efebc0a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efec00096c0 tx=0x7efec0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:49.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.848+0000 7efeced9d640 1 --2- 172.21.15.78:0/3888195852 >> [v2:172.21.15.161:6800/2582810057,v1:172.21.15.161:6801/2582810057] conn(0x7efebc0adde0 0x7efebc0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:49.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.849+0000 7efeced9d640 1 --2- 172.21.15.78:0/3888195852 >> [v2:172.21.15.161:6800/2582810057,v1:172.21.15.161:6801/2582810057] conn(0x7efebc0adde0 0x7efebc0b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:49.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.850+0000 7efec77fe640 1 -- 172.21.15.78:0/3888195852 <== osd.2 v2:172.21.15.161:6800/2582810057 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efebc0b0950 con 0x7efebc0adde0 2024-08-18T23:42:49.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.863+0000 7efed4da8640 1 -- 172.21.15.78:0/3888195852 --> [v2:172.21.15.161:6800/2582810057,v1:172.21.15.161:6801/2582810057] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efed0062b30 con 0x7efebc0adde0 2024-08-18T23:42:49.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.864+0000 7efec77fe640 1 -- 172.21.15.78:0/3888195852 <== osd.2 v2:172.21.15.161:6800/2582810057 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efed0062b30 con 0x7efebc0adde0 2024-08-18T23:42:49.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.864+0000 7efed4da8640 1 -- 172.21.15.78:0/3888195852 >> [v2:172.21.15.161:6800/2582810057,v1:172.21.15.161:6801/2582810057] conn(0x7efebc0adde0 msgr2=0x7efebc0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:49.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.864+0000 7efed4da8640 1 --2- 172.21.15.78:0/3888195852 >> [v2:172.21.15.161:6800/2582810057,v1:172.21.15.161:6801/2582810057] conn(0x7efebc0adde0 0x7efebc0b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.864+0000 7efed4da8640 1 -- 172.21.15.78:0/3888195852 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efebc0a2c10 msgr2=0x7efebc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:49.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.864+0000 7efed4da8640 1 --2- 172.21.15.78:0/3888195852 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efebc0a2c10 0x7efebc0a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efec00096c0 tx=0x7efec0000bc0 comp rx=0 tx=0).stop 2024-08-18T23:42:49.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.864+0000 7efed4da8640 1 -- 172.21.15.78:0/3888195852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed0102250 msgr2=0x7efed0198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:49.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.864+0000 7efed4da8640 1 --2- 172.21.15.78:0/3888195852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed0102250 0x7efed0198740 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efeb800c970 tx=0x7efeb800ce30 comp rx=0 tx=0).stop 2024-08-18T23:42:49.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.865+0000 7efeced9d640 1 -- 172.21.15.78:0/3888195852 reap_dead start 2024-08-18T23:42:49.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.865+0000 7efed4da8640 1 -- 172.21.15.78:0/3888195852 shutdown_connections 2024-08-18T23:42:49.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.865+0000 7efed4da8640 1 -- 172.21.15.78:0/3888195852 >> 172.21.15.78:0/3888195852 conn(0x7efed00fd5f0 msgr2=0x7efed00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:49.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.865+0000 7efed4da8640 1 -- 172.21.15.78:0/3888195852 shutdown_connections 2024-08-18T23:42:49.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:49.865+0000 7efed4da8640 1 -- 172.21.15.78:0/3888195852 wait complete. 2024-08-18T23:42:50.554 INFO:teuthology.orchestra.run.smithi078.stdout:77309411397 2024-08-18T23:42:50.554 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-08-18T23:42:51.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.190+0000 7f2839c08640 1 -- 172.21.15.78:0/3404111565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2834102a60 msgr2=0x7f2834102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:51.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.190+0000 7f2839c08640 1 --2- 172.21.15.78:0/3404111565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2834102a60 0x7f2834102e40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f282c009cd0 tx=0x7f282c02a8e0 comp rx=0 tx=0).stop 2024-08-18T23:42:51.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.190+0000 7f2839c08640 1 -- 172.21.15.78:0/3404111565 shutdown_connections 2024-08-18T23:42:51.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.190+0000 7f2839c08640 1 --2- 172.21.15.78:0/3404111565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2834107fd0 0x7f283410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.190+0000 7f2839c08640 1 --2- 172.21.15.78:0/3404111565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2834103410 0x7f28341078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.190+0000 7f2839c08640 1 --2- 172.21.15.78:0/3404111565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2834102a60 0x7f2834102e40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.190+0000 7f2839c08640 1 -- 172.21.15.78:0/3404111565 >> 172.21.15.78:0/3404111565 conn(0x7f28340fc500 msgr2=0x7f28340fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.190+0000 7f2839c08640 1 -- 172.21.15.78:0/3404111565 shutdown_connections 2024-08-18T23:42:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.190+0000 7f2839c08640 1 -- 172.21.15.78:0/3404111565 wait complete. 2024-08-18T23:42:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.191+0000 7f2839c08640 1 Processor -- start 2024-08-18T23:42:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.191+0000 7f2839c08640 1 -- start start 2024-08-18T23:42:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.191+0000 7f2839c08640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2834102a60 0x7f28341982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.191+0000 7f2839c08640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2834103410 0x7f2834198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.191+0000 7f2839c08640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2834107fd0 0x7f283419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.191+0000 7f2839c08640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2834111dd0 con 0x7f2834103410 2024-08-18T23:42:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.191+0000 7f2839c08640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2834111c50 con 0x7f2834107fd0 2024-08-18T23:42:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.191+0000 7f2839c08640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2834111f50 con 0x7f2834102a60 2024-08-18T23:42:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.191+0000 7f2832ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2834103410 0x7f2834198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.191+0000 7f2832ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2834103410 0x7f2834198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59412/0 (socket says 172.21.15.78:59412) 2024-08-18T23:42:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.191+0000 7f2832ffd640 1 -- 172.21.15.78:0/1653516550 learned_addr learned my addr 172.21.15.78:0/1653516550 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.191+0000 7f28337fe640 1 --2- 172.21.15.78:0/1653516550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2834102a60 0x7f28341982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.191+0000 7f2833fff640 1 --2- 172.21.15.78:0/1653516550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2834107fd0 0x7f283419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.191+0000 7f2832ffd640 1 -- 172.21.15.78:0/1653516550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2834102a60 msgr2=0x7f28341982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.191+0000 7f2832ffd640 1 --2- 172.21.15.78:0/1653516550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2834102a60 0x7f28341982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.191+0000 7f2832ffd640 1 -- 172.21.15.78:0/1653516550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2834107fd0 msgr2=0x7f283419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.191+0000 7f2832ffd640 1 --2- 172.21.15.78:0/1653516550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2834107fd0 0x7f283419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.191+0000 7f2832ffd640 1 -- 172.21.15.78:0/1653516550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283419d290 con 0x7f2834103410 2024-08-18T23:42:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.192+0000 7f2832ffd640 1 --2- 172.21.15.78:0/1653516550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2834103410 0x7f2834198820 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f281c00d740 tx=0x7f281c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.192+0000 7f2830ff9640 1 -- 172.21.15.78:0/1653516550 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c017070 con 0x7f2834103410 2024-08-18T23:42:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.192+0000 7f2839c08640 1 -- 172.21.15.78:0/1653516550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283419d580 con 0x7f2834103410 2024-08-18T23:42:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.192+0000 7f2839c08640 1 -- 172.21.15.78:0/1653516550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28341a4e60 con 0x7f2834103410 2024-08-18T23:42:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.192+0000 7f2830ff9640 1 -- 172.21.15.78:0/1653516550 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f281c00de50 con 0x7f2834103410 2024-08-18T23:42:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.192+0000 7f2830ff9640 1 -- 172.21.15.78:0/1653516550 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c021060 con 0x7f2834103410 2024-08-18T23:42:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.193+0000 7f2839c08640 1 -- 172.21.15.78:0/1653516550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2814001030 con 0x7f2834103410 2024-08-18T23:42:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.194+0000 7f2830ff9640 1 -- 172.21.15.78:0/1653516550 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f281c01a070 con 0x7f2834103410 2024-08-18T23:42:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.195+0000 7f2830ff9640 1 --2- 172.21.15.78:0/1653516550 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f28200a2c90 0x7f28200a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.195+0000 7f28337fe640 1 --2- 172.21.15.78:0/1653516550 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f28200a2c90 0x7f28200a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.195+0000 7f2830ff9640 1 -- 172.21.15.78:0/1653516550 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f281c0cf270 con 0x7f2834103410 2024-08-18T23:42:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.195+0000 7f2830ff9640 1 --2- 172.21.15.78:0/1653516550 >> [v2:172.21.15.123:6808/1281511569,v1:172.21.15.123:6809/1281511569] conn(0x7f28200ade60 0x7f28200b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.195+0000 7f2830ff9640 1 -- 172.21.15.78:0/1653516550 --> [v2:172.21.15.123:6808/1281511569,v1:172.21.15.123:6809/1281511569] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f281c004470 con 0x7f28200ade60 2024-08-18T23:42:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.195+0000 7f2830ff9640 1 -- 172.21.15.78:0/1653516550 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f281c0cf6c0 con 0x7f2834103410 2024-08-18T23:42:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.195+0000 7f28337fe640 1 --2- 172.21.15.78:0/1653516550 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f28200a2c90 0x7f28200a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f282c006010 tx=0x7f282c009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.195+0000 7f2833fff640 1 --2- 172.21.15.78:0/1653516550 >> [v2:172.21.15.123:6808/1281511569,v1:172.21.15.123:6809/1281511569] conn(0x7f28200ade60 0x7f28200b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.196+0000 7f2833fff640 1 --2- 172.21.15.78:0/1653516550 >> [v2:172.21.15.123:6808/1281511569,v1:172.21.15.123:6809/1281511569] conn(0x7f28200ade60 0x7f28200b0300 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.197+0000 7f2830ff9640 1 -- 172.21.15.78:0/1653516550 <== osd.4 v2:172.21.15.123:6808/1281511569 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f281c004470 con 0x7f28200ade60 2024-08-18T23:42:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.211+0000 7f2839c08640 1 -- 172.21.15.78:0/1653516550 --> [v2:172.21.15.123:6808/1281511569,v1:172.21.15.123:6809/1281511569] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2814002dd0 con 0x7f28200ade60 2024-08-18T23:42:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.211+0000 7f2830ff9640 1 -- 172.21.15.78:0/1653516550 <== osd.4 v2:172.21.15.123:6808/1281511569 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2814002dd0 con 0x7f28200ade60 2024-08-18T23:42:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.211+0000 7f2839c08640 1 -- 172.21.15.78:0/1653516550 >> [v2:172.21.15.123:6808/1281511569,v1:172.21.15.123:6809/1281511569] conn(0x7f28200ade60 msgr2=0x7f28200b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.211+0000 7f2839c08640 1 --2- 172.21.15.78:0/1653516550 >> [v2:172.21.15.123:6808/1281511569,v1:172.21.15.123:6809/1281511569] conn(0x7f28200ade60 0x7f28200b0300 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.211+0000 7f2839c08640 1 -- 172.21.15.78:0/1653516550 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f28200a2c90 msgr2=0x7f28200a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.211+0000 7f2839c08640 1 --2- 172.21.15.78:0/1653516550 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f28200a2c90 0x7f28200a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f282c006010 tx=0x7f282c009360 comp rx=0 tx=0).stop 2024-08-18T23:42:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.211+0000 7f2839c08640 1 -- 172.21.15.78:0/1653516550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2834103410 msgr2=0x7f2834198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.211+0000 7f2839c08640 1 --2- 172.21.15.78:0/1653516550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2834103410 0x7f2834198820 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f281c00d740 tx=0x7f281c00dc00 comp rx=0 tx=0).stop 2024-08-18T23:42:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.211+0000 7f2833fff640 1 -- 172.21.15.78:0/1653516550 reap_dead start 2024-08-18T23:42:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.211+0000 7f2839c08640 1 -- 172.21.15.78:0/1653516550 shutdown_connections 2024-08-18T23:42:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.211+0000 7f2839c08640 1 -- 172.21.15.78:0/1653516550 >> 172.21.15.78:0/1653516550 conn(0x7f28340fc500 msgr2=0x7f2834105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.212+0000 7f2839c08640 1 -- 172.21.15.78:0/1653516550 shutdown_connections 2024-08-18T23:42:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:51.212+0000 7f2839c08640 1 -- 172.21.15.78:0/1653516550 wait complete. 2024-08-18T23:42:51.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:51 smithi078 ceph-mon[18440]: pgmap v304: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:42:51.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:51 smithi123 ceph-mon[21808]: pgmap v304: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:42:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:51 smithi161 ceph-mon[22027]: pgmap v304: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:42:51.888 INFO:teuthology.orchestra.run.smithi078.stdout:137438953533 2024-08-18T23:42:51.889 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-08-18T23:42:52.348 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:52.348 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:52.349 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:52.359 INFO:teuthology.orchestra.run.smithi078.stdout:240518168619 2024-08-18T23:42:52.360 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-08-18T23:42:52.879 INFO:teuthology.orchestra.run.smithi078.stdout:292057776161 2024-08-18T23:42:52.879 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-08-18T23:42:53.462 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:53 smithi078 ceph-mon[18440]: pgmap v305: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:42:53.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:53 smithi123 ceph-mon[21808]: pgmap v305: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:42:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:53 smithi161 ceph-mon[22027]: pgmap v305: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:42:53.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.978+0000 7ff54a692640 1 -- 172.21.15.78:0/3988487351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff544102b00 msgr2=0x7ff544102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:53.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.978+0000 7ff54a692640 1 --2- 172.21.15.78:0/3988487351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff544102b00 0x7ff544102ee0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff534009930 tx=0x7ff53402a420 comp rx=0 tx=0).stop 2024-08-18T23:42:53.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.978+0000 7ff54a692640 1 -- 172.21.15.78:0/3988487351 shutdown_connections 2024-08-18T23:42:53.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.978+0000 7ff54a692640 1 --2- 172.21.15.78:0/3988487351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff544108030 0x7ff54410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:53.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.978+0000 7ff54a692640 1 --2- 172.21.15.78:0/3988487351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5441034b0 0x7ff5441078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:53.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.978+0000 7ff54a692640 1 --2- 172.21.15.78:0/3988487351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff544102b00 0x7ff544102ee0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:53.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.978+0000 7ff54a692640 1 -- 172.21.15.78:0/3988487351 >> 172.21.15.78:0/3988487351 conn(0x7ff5440fc500 msgr2=0x7ff5440fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:53.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.978+0000 7ff54a692640 1 -- 172.21.15.78:0/3988487351 shutdown_connections 2024-08-18T23:42:53.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.978+0000 7ff54a692640 1 -- 172.21.15.78:0/3988487351 wait complete. 2024-08-18T23:42:53.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.979+0000 7ff54a692640 1 Processor -- start 2024-08-18T23:42:53.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.979+0000 7ff54a692640 1 -- start start 2024-08-18T23:42:53.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.979+0000 7ff54a692640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff544102b00 0x7ff544198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:54.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.979+0000 7ff54a692640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5441034b0 0x7ff544198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:54.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.979+0000 7ff54a692640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff544108030 0x7ff54419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:54.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.979+0000 7ff54a692640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff544111e40 con 0x7ff544108030 2024-08-18T23:42:54.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.979+0000 7ff54a692640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff544111cc0 con 0x7ff5441034b0 2024-08-18T23:42:54.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.979+0000 7ff54a692640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff544111fc0 con 0x7ff544102b00 2024-08-18T23:42:54.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.979+0000 7ff548c2f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff544108030 0x7ff54419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:54.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.979+0000 7ff548c2f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff544108030 0x7ff54419cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59466/0 (socket says 172.21.15.78:59466) 2024-08-18T23:42:54.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.979+0000 7ff548c2f640 1 -- 172.21.15.78:0/3122746928 learned_addr learned my addr 172.21.15.78:0/3122746928 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:54.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.979+0000 7ff543fff640 1 --2- 172.21.15.78:0/3122746928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff544102b00 0x7ff544198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:54.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.979+0000 7ff5437fe640 1 --2- 172.21.15.78:0/3122746928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5441034b0 0x7ff544198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:54.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.979+0000 7ff548c2f640 1 -- 172.21.15.78:0/3122746928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff544102b00 msgr2=0x7ff544198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:54.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.979+0000 7ff548c2f640 1 --2- 172.21.15.78:0/3122746928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff544102b00 0x7ff544198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:54.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.979+0000 7ff548c2f640 1 -- 172.21.15.78:0/3122746928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5441034b0 msgr2=0x7ff544198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:54.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.979+0000 7ff548c2f640 1 --2- 172.21.15.78:0/3122746928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5441034b0 0x7ff544198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:54.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.979+0000 7ff548c2f640 1 -- 172.21.15.78:0/3122746928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54419d2f0 con 0x7ff544108030 2024-08-18T23:42:54.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.980+0000 7ff548c2f640 1 --2- 172.21.15.78:0/3122746928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff544108030 0x7ff54419cc10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00b970 tx=0x7ff53c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:54.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.980+0000 7ff5417fa640 1 -- 172.21.15.78:0/3122746928 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c019070 con 0x7ff544108030 2024-08-18T23:42:54.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.980+0000 7ff5417fa640 1 -- 172.21.15.78:0/3122746928 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff53c00cb40 con 0x7ff544108030 2024-08-18T23:42:54.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.980+0000 7ff54a692640 1 -- 172.21.15.78:0/3122746928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff54419d5e0 con 0x7ff544108030 2024-08-18T23:42:54.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.980+0000 7ff5417fa640 1 -- 172.21.15.78:0/3122746928 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c007650 con 0x7ff544108030 2024-08-18T23:42:54.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.980+0000 7ff54a692640 1 -- 172.21.15.78:0/3122746928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5441a4ec0 con 0x7ff544108030 2024-08-18T23:42:54.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.981+0000 7ff54a692640 1 -- 172.21.15.78:0/3122746928 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff544062670 con 0x7ff544108030 2024-08-18T23:42:54.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.982+0000 7ff5417fa640 1 -- 172.21.15.78:0/3122746928 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff53c01c070 con 0x7ff544108030 2024-08-18T23:42:54.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.983+0000 7ff5417fa640 1 --2- 172.21.15.78:0/3122746928 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff52c0a2c90 0x7ff52c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:54.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.983+0000 7ff543fff640 1 --2- 172.21.15.78:0/3122746928 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff52c0a2c90 0x7ff52c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:54.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.983+0000 7ff5417fa640 1 -- 172.21.15.78:0/3122746928 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff53c0cf190 con 0x7ff544108030 2024-08-18T23:42:54.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.983+0000 7ff5417fa640 1 --2- 172.21.15.78:0/3122746928 >> [v2:172.21.15.123:6816/3440305156,v1:172.21.15.123:6817/3440305156] conn(0x7ff52c0ade60 0x7ff52c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:54.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.983+0000 7ff5417fa640 1 -- 172.21.15.78:0/3122746928 --> [v2:172.21.15.123:6816/3440305156,v1:172.21.15.123:6817/3440305156] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff53c0047d0 con 0x7ff52c0ade60 2024-08-18T23:42:54.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.983+0000 7ff5417fa640 1 -- 172.21.15.78:0/3122746928 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ff53c0cf5e0 con 0x7ff544108030 2024-08-18T23:42:54.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.983+0000 7ff543fff640 1 --2- 172.21.15.78:0/3122746928 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff52c0a2c90 0x7ff52c0a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff534009900 tx=0x7ff5340058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:54.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.983+0000 7ff5437fe640 1 --2- 172.21.15.78:0/3122746928 >> [v2:172.21.15.123:6816/3440305156,v1:172.21.15.123:6817/3440305156] conn(0x7ff52c0ade60 0x7ff52c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:54.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.984+0000 7ff5437fe640 1 --2- 172.21.15.78:0/3122746928 >> [v2:172.21.15.123:6816/3440305156,v1:172.21.15.123:6817/3440305156] conn(0x7ff52c0ade60 0x7ff52c0b0300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:54.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.985+0000 7ff5417fa640 1 -- 172.21.15.78:0/3122746928 <== osd.7 v2:172.21.15.123:6816/3440305156 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff53c0047d0 con 0x7ff52c0ade60 2024-08-18T23:42:54.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.998+0000 7ff54a692640 1 -- 172.21.15.78:0/3122746928 --> [v2:172.21.15.123:6816/3440305156,v1:172.21.15.123:6817/3440305156] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff544062af0 con 0x7ff52c0ade60 2024-08-18T23:42:54.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.998+0000 7ff5417fa640 1 -- 172.21.15.78:0/3122746928 <== osd.7 v2:172.21.15.123:6816/3440305156 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff544062af0 con 0x7ff52c0ade60 2024-08-18T23:42:54.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.999+0000 7ff54a692640 1 -- 172.21.15.78:0/3122746928 >> [v2:172.21.15.123:6816/3440305156,v1:172.21.15.123:6817/3440305156] conn(0x7ff52c0ade60 msgr2=0x7ff52c0b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:54.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.999+0000 7ff54a692640 1 --2- 172.21.15.78:0/3122746928 >> [v2:172.21.15.123:6816/3440305156,v1:172.21.15.123:6817/3440305156] conn(0x7ff52c0ade60 0x7ff52c0b0300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:54.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.999+0000 7ff54a692640 1 -- 172.21.15.78:0/3122746928 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff52c0a2c90 msgr2=0x7ff52c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:54.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.999+0000 7ff54a692640 1 --2- 172.21.15.78:0/3122746928 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff52c0a2c90 0x7ff52c0a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff534009900 tx=0x7ff5340058d0 comp rx=0 tx=0).stop 2024-08-18T23:42:54.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.999+0000 7ff54a692640 1 -- 172.21.15.78:0/3122746928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff544108030 msgr2=0x7ff54419cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:54.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.999+0000 7ff54a692640 1 --2- 172.21.15.78:0/3122746928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff544108030 0x7ff54419cc10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00b970 tx=0x7ff53c00be30 comp rx=0 tx=0).stop 2024-08-18T23:42:54.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.999+0000 7ff548c2f640 1 -- 172.21.15.78:0/3122746928 reap_dead start 2024-08-18T23:42:54.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.999+0000 7ff54a692640 1 -- 172.21.15.78:0/3122746928 shutdown_connections 2024-08-18T23:42:54.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.999+0000 7ff54a692640 1 -- 172.21.15.78:0/3122746928 >> 172.21.15.78:0/3122746928 conn(0x7ff5440fc500 msgr2=0x7ff5441060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:54.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.999+0000 7ff54a692640 1 -- 172.21.15.78:0/3122746928 shutdown_connections 2024-08-18T23:42:54.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.999+0000 7ff54a692640 1 -- 172.21.15.78:0/3122746928 wait complete. 2024-08-18T23:42:54.658 INFO:teuthology.orchestra.run.smithi078.stdout:210453397551 2024-08-18T23:42:54.659 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-08-18T23:42:54.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.852+0000 7f34a7c2c640 1 -- 172.21.15.78:0/2560136088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34a0103470 msgr2=0x7f34a01078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.852+0000 7f34a7c2c640 1 --2- 172.21.15.78:0/2560136088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34a0103470 0x7f34a01078b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3490009930 tx=0x7f349002a420 comp rx=0 tx=0).stop 2024-08-18T23:42:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.852+0000 7f34a7c2c640 1 -- 172.21.15.78:0/2560136088 shutdown_connections 2024-08-18T23:42:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.852+0000 7f34a7c2c640 1 --2- 172.21.15.78:0/2560136088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a0107fc0 0x7f34a010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.852+0000 7f34a7c2c640 1 --2- 172.21.15.78:0/2560136088 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34a0103470 0x7f34a01078b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.852+0000 7f34a7c2c640 1 --2- 172.21.15.78:0/2560136088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34a0102ac0 0x7f34a0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.852+0000 7f34a7c2c640 1 -- 172.21.15.78:0/2560136088 >> 172.21.15.78:0/2560136088 conn(0x7f34a00fc500 msgr2=0x7f34a00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.852+0000 7f34a7c2c640 1 -- 172.21.15.78:0/2560136088 shutdown_connections 2024-08-18T23:42:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.852+0000 7f34a7c2c640 1 -- 172.21.15.78:0/2560136088 wait complete. 2024-08-18T23:42:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.853+0000 7f34a7c2c640 1 Processor -- start 2024-08-18T23:42:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.853+0000 7f34a7c2c640 1 -- start start 2024-08-18T23:42:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.853+0000 7f34a7c2c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a0102ac0 0x7f34a0198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.853+0000 7f34a7c2c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34a0103470 0x7f34a01987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.853+0000 7f34a7c2c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34a0107fc0 0x7f34a019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.853+0000 7f34a7c2c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0111dc0 con 0x7f34a0103470 2024-08-18T23:42:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.853+0000 7f34a7c2c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0111c40 con 0x7f34a0107fc0 2024-08-18T23:42:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.853+0000 7f34a7c2c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0111f40 con 0x7f34a0102ac0 2024-08-18T23:42:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.853+0000 7f34a51c7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34a0103470 0x7f34a01987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.853+0000 7f34a51c7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34a0103470 0x7f34a01987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59488/0 (socket says 172.21.15.78:59488) 2024-08-18T23:42:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.853+0000 7f34a51c7640 1 -- 172.21.15.78:0/2416265483 learned_addr learned my addr 172.21.15.78:0/2416265483 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.854+0000 7f34a59c8640 1 --2- 172.21.15.78:0/2416265483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a0102ac0 0x7f34a0198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.854+0000 7f34a61c9640 1 --2- 172.21.15.78:0/2416265483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34a0107fc0 0x7f34a019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.854+0000 7f34a51c7640 1 -- 172.21.15.78:0/2416265483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a0102ac0 msgr2=0x7f34a0198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.854+0000 7f34a51c7640 1 --2- 172.21.15.78:0/2416265483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a0102ac0 0x7f34a0198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.854+0000 7f34a51c7640 1 -- 172.21.15.78:0/2416265483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34a0107fc0 msgr2=0x7f34a019cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:54.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.854+0000 7f34a51c7640 1 --2- 172.21.15.78:0/2416265483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34a0107fc0 0x7f34a019cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:54.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.854+0000 7f34a51c7640 1 -- 172.21.15.78:0/2416265483 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a019d220 con 0x7f34a0103470 2024-08-18T23:42:54.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.854+0000 7f34a61c9640 1 --2- 172.21.15.78:0/2416265483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34a0107fc0 0x7f34a019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:42:54.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.854+0000 7f34a59c8640 1 --2- 172.21.15.78:0/2416265483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34a0102ac0 0x7f34a0198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:42:54.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.854+0000 7f34a51c7640 1 --2- 172.21.15.78:0/2416265483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34a0103470 0x7f34a01987b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3490009230 tx=0x7f34900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:54.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.854+0000 7f3496ffd640 1 -- 172.21.15.78:0/2416265483 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349003f030 con 0x7f34a0103470 2024-08-18T23:42:54.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.854+0000 7f3496ffd640 1 -- 172.21.15.78:0/2416265483 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3490003ce0 con 0x7f34a0103470 2024-08-18T23:42:54.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.854+0000 7f3496ffd640 1 -- 172.21.15.78:0/2416265483 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490035070 con 0x7f34a0103470 2024-08-18T23:42:54.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.854+0000 7f34a7c2c640 1 -- 172.21.15.78:0/2416265483 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a019d4b0 con 0x7f34a0103470 2024-08-18T23:42:54.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.854+0000 7f34a7c2c640 1 -- 172.21.15.78:0/2416265483 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a01a4df0 con 0x7f34a0103470 2024-08-18T23:42:54.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.855+0000 7f34a7c2c640 1 -- 172.21.15.78:0/2416265483 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f34a00626b0 con 0x7f34a0103470 2024-08-18T23:42:54.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.856+0000 7f3496ffd640 1 -- 172.21.15.78:0/2416265483 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3490042030 con 0x7f34a0103470 2024-08-18T23:42:54.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.857+0000 7f3496ffd640 1 --2- 172.21.15.78:0/2416265483 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f348c0a2c10 0x7f348c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:54.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.857+0000 7f3496ffd640 1 -- 172.21.15.78:0/2416265483 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f34900eff90 con 0x7f34a0103470 2024-08-18T23:42:54.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.857+0000 7f34a59c8640 1 --2- 172.21.15.78:0/2416265483 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f348c0a2c10 0x7f348c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:54.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.857+0000 7f3496ffd640 1 --2- 172.21.15.78:0/2416265483 >> [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521] conn(0x7f348c0adde0 0x7f348c0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:54.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.857+0000 7f3496ffd640 1 -- 172.21.15.78:0/2416265483 --> [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f349000b0b0 con 0x7f348c0adde0 2024-08-18T23:42:54.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.857+0000 7f3496ffd640 1 -- 172.21.15.78:0/2416265483 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f34900f03e0 con 0x7f34a0103470 2024-08-18T23:42:54.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.857+0000 7f34a61c9640 1 --2- 172.21.15.78:0/2416265483 >> [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521] conn(0x7f348c0adde0 0x7f348c0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:54.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.857+0000 7f34a59c8640 1 --2- 172.21.15.78:0/2416265483 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f348c0a2c10 0x7f348c0a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f349c0058c0 tx=0x7f349c005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:54.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.858+0000 7f34a61c9640 1 --2- 172.21.15.78:0/2416265483 >> [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521] conn(0x7f348c0adde0 0x7f348c0b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:54.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.858+0000 7f3496ffd640 1 -- 172.21.15.78:0/2416265483 <== osd.3 v2:172.21.15.78:6810/2697347521 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f349000b0b0 con 0x7f348c0adde0 2024-08-18T23:42:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.872+0000 7f34a7c2c640 1 -- 172.21.15.78:0/2416265483 --> [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f34a0062ab0 con 0x7f348c0adde0 2024-08-18T23:42:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.872+0000 7f3496ffd640 1 -- 172.21.15.78:0/2416265483 <== osd.3 v2:172.21.15.78:6810/2697347521 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f34a0062ab0 con 0x7f348c0adde0 2024-08-18T23:42:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.872+0000 7f34a7c2c640 1 -- 172.21.15.78:0/2416265483 >> [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521] conn(0x7f348c0adde0 msgr2=0x7f348c0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.872+0000 7f34a7c2c640 1 --2- 172.21.15.78:0/2416265483 >> [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521] conn(0x7f348c0adde0 0x7f348c0b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.872+0000 7f34a7c2c640 1 -- 172.21.15.78:0/2416265483 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f348c0a2c10 msgr2=0x7f348c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.872+0000 7f34a7c2c640 1 --2- 172.21.15.78:0/2416265483 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f348c0a2c10 0x7f348c0a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f349c0058c0 tx=0x7f349c005ec0 comp rx=0 tx=0).stop 2024-08-18T23:42:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.872+0000 7f34a7c2c640 1 -- 172.21.15.78:0/2416265483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34a0103470 msgr2=0x7f34a01987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.872+0000 7f34a7c2c640 1 --2- 172.21.15.78:0/2416265483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34a0103470 0x7f34a01987b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3490009230 tx=0x7f34900042f0 comp rx=0 tx=0).stop 2024-08-18T23:42:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.872+0000 7f34a61c9640 1 -- 172.21.15.78:0/2416265483 reap_dead start 2024-08-18T23:42:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.872+0000 7f34a7c2c640 1 -- 172.21.15.78:0/2416265483 shutdown_connections 2024-08-18T23:42:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.872+0000 7f34a7c2c640 1 -- 172.21.15.78:0/2416265483 >> 172.21.15.78:0/2416265483 conn(0x7f34a00fc500 msgr2=0x7f34a0105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.873+0000 7f34a7c2c640 1 -- 172.21.15.78:0/2416265483 shutdown_connections 2024-08-18T23:42:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:54.873+0000 7f34a7c2c640 1 -- 172.21.15.78:0/2416265483 wait complete. 2024-08-18T23:42:55.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.868+0000 7f8592249640 1 -- 172.21.15.78:0/3555035706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c101150 msgr2=0x7f858c10d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:55.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.868+0000 7f8592249640 1 --2- 172.21.15.78:0/3555035706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c101150 0x7f858c10d680 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8584009cd0 tx=0x7f858402a8e0 comp rx=0 tx=0).stop 2024-08-18T23:42:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.869+0000 7f8592249640 1 -- 172.21.15.78:0/3555035706 shutdown_connections 2024-08-18T23:42:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.869+0000 7f8592249640 1 --2- 172.21.15.78:0/3555035706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c101150 0x7f858c10d680 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.869+0000 7f8592249640 1 --2- 172.21.15.78:0/3555035706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f858c1007b0 0x7f858c100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.869+0000 7f8592249640 1 --2- 172.21.15.78:0/3555035706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f858c1067b0 0x7f858c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.869+0000 7f8592249640 1 -- 172.21.15.78:0/3555035706 >> 172.21.15.78:0/3555035706 conn(0x7f858c0fc4d0 msgr2=0x7f858c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.869+0000 7f8592249640 1 -- 172.21.15.78:0/3555035706 shutdown_connections 2024-08-18T23:42:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.869+0000 7f8592249640 1 -- 172.21.15.78:0/3555035706 wait complete. 2024-08-18T23:42:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.869+0000 7f8592249640 1 Processor -- start 2024-08-18T23:42:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.869+0000 7f8592249640 1 -- start start 2024-08-18T23:42:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f8592249640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c1007b0 0x7f858c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f8592249640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f858c101150 0x7f858c19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f8592249640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f858c1067b0 0x7f858c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f8592249640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f858c110bd0 con 0x7f858c1007b0 2024-08-18T23:42:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f8592249640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f858c110a50 con 0x7f858c1067b0 2024-08-18T23:42:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f8592249640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f858c110d50 con 0x7f858c101150 2024-08-18T23:42:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f858b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c1007b0 0x7f858c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f858b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c1007b0 0x7f858c19a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59446/0 (socket says 172.21.15.78:59446) 2024-08-18T23:42:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f858b7fe640 1 -- 172.21.15.78:0/272303093 learned_addr learned my addr 172.21.15.78:0/272303093 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f858bfff640 1 --2- 172.21.15.78:0/272303093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f858c1067b0 0x7f858c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:55.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f858affd640 1 --2- 172.21.15.78:0/272303093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f858c101150 0x7f858c19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:55.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f858b7fe640 1 -- 172.21.15.78:0/272303093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f858c101150 msgr2=0x7f858c19a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:55.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f858b7fe640 1 --2- 172.21.15.78:0/272303093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f858c101150 0x7f858c19a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:55.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f858b7fe640 1 -- 172.21.15.78:0/272303093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f858c1067b0 msgr2=0x7f858c19ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:55.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f858b7fe640 1 --2- 172.21.15.78:0/272303093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f858c1067b0 0x7f858c19ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:55.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f858b7fe640 1 -- 172.21.15.78:0/272303093 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858c19f3e0 con 0x7f858c1007b0 2024-08-18T23:42:55.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f858bfff640 1 --2- 172.21.15.78:0/272303093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f858c1067b0 0x7f858c19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:42:55.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f858affd640 1 --2- 172.21.15.78:0/272303093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f858c101150 0x7f858c19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:42:55.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f858b7fe640 1 --2- 172.21.15.78:0/272303093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c1007b0 0x7f858c19a430 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f857c00b4a0 tx=0x7f857c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:55.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f8588ff9640 1 -- 172.21.15.78:0/272303093 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c014070 con 0x7f858c1007b0 2024-08-18T23:42:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f8592249640 1 -- 172.21.15.78:0/272303093 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f858c19f6d0 con 0x7f858c1007b0 2024-08-18T23:42:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f8588ff9640 1 -- 172.21.15.78:0/272303093 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f857c010af0 con 0x7f858c1007b0 2024-08-18T23:42:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f8588ff9640 1 -- 172.21.15.78:0/272303093 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c017d10 con 0x7f858c1007b0 2024-08-18T23:42:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.870+0000 7f8592249640 1 -- 172.21.15.78:0/272303093 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f858c1a6fb0 con 0x7f858c1007b0 2024-08-18T23:42:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.871+0000 7f8592249640 1 -- 172.21.15.78:0/272303093 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f858c06aea0 con 0x7f858c1007b0 2024-08-18T23:42:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.872+0000 7f8588ff9640 1 -- 172.21.15.78:0/272303093 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f857c017070 con 0x7f858c1007b0 2024-08-18T23:42:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.873+0000 7f8588ff9640 1 --2- 172.21.15.78:0/272303093 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f85780a2c10 0x7f85780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.873+0000 7f858affd640 1 --2- 172.21.15.78:0/272303093 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f85780a2c10 0x7f85780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.873+0000 7f8588ff9640 1 -- 172.21.15.78:0/272303093 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f857c0cec40 con 0x7f858c1007b0 2024-08-18T23:42:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.873+0000 7f8588ff9640 1 --2- 172.21.15.78:0/272303093 >> [v2:172.21.15.161:6824/2037560985,v1:172.21.15.161:6825/2037560985] conn(0x7f85780adde0 0x7f85780b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.873+0000 7f8588ff9640 1 -- 172.21.15.78:0/272303093 --> [v2:172.21.15.161:6824/2037560985,v1:172.21.15.161:6825/2037560985] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f857c01ae50 con 0x7f85780adde0 2024-08-18T23:42:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.873+0000 7f8588ff9640 1 -- 172.21.15.78:0/272303093 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f857c0cf090 con 0x7f858c1007b0 2024-08-18T23:42:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.874+0000 7f858affd640 1 --2- 172.21.15.78:0/272303093 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f85780a2c10 0x7f85780a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f85740058c0 tx=0x7f8574005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.874+0000 7f858bfff640 1 --2- 172.21.15.78:0/272303093 >> [v2:172.21.15.161:6824/2037560985,v1:172.21.15.161:6825/2037560985] conn(0x7f85780adde0 0x7f85780b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.874+0000 7f858bfff640 1 --2- 172.21.15.78:0/272303093 >> [v2:172.21.15.161:6824/2037560985,v1:172.21.15.161:6825/2037560985] conn(0x7f85780adde0 0x7f85780b0280 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.875+0000 7f8588ff9640 1 -- 172.21.15.78:0/272303093 <== osd.11 v2:172.21.15.161:6824/2037560985 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f857c01ae50 con 0x7f85780adde0 2024-08-18T23:42:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.888+0000 7f8592249640 1 -- 172.21.15.78:0/272303093 --> [v2:172.21.15.161:6824/2037560985,v1:172.21.15.161:6825/2037560985] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f858c101ef0 con 0x7f85780adde0 2024-08-18T23:42:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.889+0000 7f8588ff9640 1 -- 172.21.15.78:0/272303093 <== osd.11 v2:172.21.15.161:6824/2037560985 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f858c101ef0 con 0x7f85780adde0 2024-08-18T23:42:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.889+0000 7f8592249640 1 -- 172.21.15.78:0/272303093 >> [v2:172.21.15.161:6824/2037560985,v1:172.21.15.161:6825/2037560985] conn(0x7f85780adde0 msgr2=0x7f85780b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.889+0000 7f8592249640 1 --2- 172.21.15.78:0/272303093 >> [v2:172.21.15.161:6824/2037560985,v1:172.21.15.161:6825/2037560985] conn(0x7f85780adde0 0x7f85780b0280 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.889+0000 7f8592249640 1 -- 172.21.15.78:0/272303093 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f85780a2c10 msgr2=0x7f85780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:55.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.889+0000 7f8592249640 1 --2- 172.21.15.78:0/272303093 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f85780a2c10 0x7f85780a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f85740058c0 tx=0x7f8574005850 comp rx=0 tx=0).stop 2024-08-18T23:42:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.889+0000 7f8592249640 1 -- 172.21.15.78:0/272303093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c1007b0 msgr2=0x7f858c19a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.889+0000 7f8592249640 1 --2- 172.21.15.78:0/272303093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c1007b0 0x7f858c19a430 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f857c00b4a0 tx=0x7f857c00b960 comp rx=0 tx=0).stop 2024-08-18T23:42:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.889+0000 7f858bfff640 1 -- 172.21.15.78:0/272303093 reap_dead start 2024-08-18T23:42:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.889+0000 7f8592249640 1 -- 172.21.15.78:0/272303093 shutdown_connections 2024-08-18T23:42:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.889+0000 7f8592249640 1 -- 172.21.15.78:0/272303093 >> 172.21.15.78:0/272303093 conn(0x7f858c0fc4d0 msgr2=0x7f858c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.889+0000 7f8592249640 1 -- 172.21.15.78:0/272303093 shutdown_connections 2024-08-18T23:42:55.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.890+0000 7f8592249640 1 -- 172.21.15.78:0/272303093 wait complete. 2024-08-18T23:42:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:55 smithi078 ceph-mon[18440]: pgmap v306: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:42:55.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:55 smithi123 ceph-mon[21808]: pgmap v306: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:42:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:55 smithi161 ceph-mon[22027]: pgmap v306: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:42:55.857 INFO:teuthology.orchestra.run.smithi078.stdout:326417514526 2024-08-18T23:42:55.858 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-08-18T23:42:56.393 INFO:teuthology.orchestra.run.smithi078.stdout:107374182465 2024-08-18T23:42:56.394 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-08-18T23:42:56.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.431+0000 7fd2d781a640 1 -- 172.21.15.78:0/1702196404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2d0101150 msgr2=0x7fd2d010d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:56.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.431+0000 7fd2d781a640 1 --2- 172.21.15.78:0/1702196404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2d0101150 0x7fd2d010d680 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009cd0 tx=0x7fd2c802a8e0 comp rx=0 tx=0).stop 2024-08-18T23:42:56.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.431+0000 7fd2d781a640 1 -- 172.21.15.78:0/1702196404 shutdown_connections 2024-08-18T23:42:56.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.431+0000 7fd2d781a640 1 --2- 172.21.15.78:0/1702196404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2d0101150 0x7fd2d010d680 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:56.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.431+0000 7fd2d781a640 1 --2- 172.21.15.78:0/1702196404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2d01007b0 0x7fd2d0100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:56.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.431+0000 7fd2d781a640 1 --2- 172.21.15.78:0/1702196404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2d01067b0 0x7fd2d0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:56.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.431+0000 7fd2d781a640 1 -- 172.21.15.78:0/1702196404 >> 172.21.15.78:0/1702196404 conn(0x7fd2d00fc4d0 msgr2=0x7fd2d00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:56.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.431+0000 7fd2d781a640 1 -- 172.21.15.78:0/1702196404 shutdown_connections 2024-08-18T23:42:56.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.431+0000 7fd2d781a640 1 -- 172.21.15.78:0/1702196404 wait complete. 2024-08-18T23:42:56.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.432+0000 7fd2d781a640 1 Processor -- start 2024-08-18T23:42:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.432+0000 7fd2d781a640 1 -- start start 2024-08-18T23:42:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.432+0000 7fd2d781a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2d01007b0 0x7fd2d0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.432+0000 7fd2d781a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2d0101150 0x7fd2d01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.432+0000 7fd2d781a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2d01067b0 0x7fd2d019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.432+0000 7fd2d781a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d0110bb0 con 0x7fd2d01007b0 2024-08-18T23:42:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.432+0000 7fd2d781a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d0110a30 con 0x7fd2d01067b0 2024-08-18T23:42:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.432+0000 7fd2d781a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d0110d30 con 0x7fd2d0101150 2024-08-18T23:42:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.432+0000 7fd2d55b6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2d01007b0 0x7fd2d0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.432+0000 7fd2d55b6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2d01007b0 0x7fd2d0198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59436/0 (socket says 172.21.15.78:59436) 2024-08-18T23:42:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.432+0000 7fd2d55b6640 1 -- 172.21.15.78:0/1610644982 learned_addr learned my addr 172.21.15.78:0/1610644982 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.432+0000 7fd2d5db7640 1 --2- 172.21.15.78:0/1610644982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2d01067b0 0x7fd2d019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.432+0000 7fd2d4db5640 1 --2- 172.21.15.78:0/1610644982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2d0101150 0x7fd2d01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:56.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.432+0000 7fd2d55b6640 1 -- 172.21.15.78:0/1610644982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2d0101150 msgr2=0x7fd2d01985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:56.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.432+0000 7fd2d55b6640 1 --2- 172.21.15.78:0/1610644982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2d0101150 0x7fd2d01985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:56.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.432+0000 7fd2d55b6640 1 -- 172.21.15.78:0/1610644982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2d01067b0 msgr2=0x7fd2d019c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:56.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.432+0000 7fd2d55b6640 1 --2- 172.21.15.78:0/1610644982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2d01067b0 0x7fd2d019c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:56.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.432+0000 7fd2d55b6640 1 -- 172.21.15.78:0/1610644982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d019d030 con 0x7fd2d01007b0 2024-08-18T23:42:56.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.433+0000 7fd2d55b6640 1 --2- 172.21.15.78:0/1610644982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2d01007b0 0x7fd2d0198080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00d950 tx=0x7fd2cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:56.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.433+0000 7fd2c27fc640 1 -- 172.21.15.78:0/1610644982 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc017070 con 0x7fd2d01007b0 2024-08-18T23:42:56.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.433+0000 7fd2d781a640 1 -- 172.21.15.78:0/1610644982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d019d320 con 0x7fd2d01007b0 2024-08-18T23:42:56.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.433+0000 7fd2c27fc640 1 -- 172.21.15.78:0/1610644982 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2cc009c30 con 0x7fd2d01007b0 2024-08-18T23:42:56.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.433+0000 7fd2c27fc640 1 -- 172.21.15.78:0/1610644982 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc00f6b0 con 0x7fd2d01007b0 2024-08-18T23:42:56.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.433+0000 7fd2d781a640 1 -- 172.21.15.78:0/1610644982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d01a4c00 con 0x7fd2d01007b0 2024-08-18T23:42:56.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.434+0000 7fd2d781a640 1 -- 172.21.15.78:0/1610644982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd2d00626b0 con 0x7fd2d01007b0 2024-08-18T23:42:56.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.435+0000 7fd2c27fc640 1 -- 172.21.15.78:0/1610644982 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd2cc0109d0 con 0x7fd2d01007b0 2024-08-18T23:42:56.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.435+0000 7fd2c27fc640 1 --2- 172.21.15.78:0/1610644982 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2bc0a2c10 0x7fd2bc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:56.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.436+0000 7fd2c27fc640 1 -- 172.21.15.78:0/1610644982 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd2cc0cf560 con 0x7fd2d01007b0 2024-08-18T23:42:56.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.436+0000 7fd2d4db5640 1 --2- 172.21.15.78:0/1610644982 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2bc0a2c10 0x7fd2bc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:56.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.436+0000 7fd2c27fc640 1 --2- 172.21.15.78:0/1610644982 >> [v2:172.21.15.161:6808/2751796117,v1:172.21.15.161:6809/2751796117] conn(0x7fd2bc0adde0 0x7fd2bc0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:56.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.436+0000 7fd2c27fc640 1 -- 172.21.15.78:0/1610644982 --> [v2:172.21.15.161:6808/2751796117,v1:172.21.15.161:6809/2751796117] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd2bc0b0950 con 0x7fd2bc0adde0 2024-08-18T23:42:56.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.436+0000 7fd2c27fc640 1 -- 172.21.15.78:0/1610644982 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fd2cc0cf9b0 con 0x7fd2d01007b0 2024-08-18T23:42:56.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.436+0000 7fd2d5db7640 1 --2- 172.21.15.78:0/1610644982 >> [v2:172.21.15.161:6808/2751796117,v1:172.21.15.161:6809/2751796117] conn(0x7fd2bc0adde0 0x7fd2bc0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:56.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.436+0000 7fd2d4db5640 1 --2- 172.21.15.78:0/1610644982 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2bc0a2c10 0x7fd2bc0a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4005e10 tx=0x7fd2c4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:56.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.436+0000 7fd2d5db7640 1 --2- 172.21.15.78:0/1610644982 >> [v2:172.21.15.161:6808/2751796117,v1:172.21.15.161:6809/2751796117] conn(0x7fd2bc0adde0 0x7fd2bc0b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:56.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.437+0000 7fd2c27fc640 1 -- 172.21.15.78:0/1610644982 <== osd.5 v2:172.21.15.161:6808/2751796117 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd2bc0b0950 con 0x7fd2bc0adde0 2024-08-18T23:42:56.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.451+0000 7fd2d781a640 1 -- 172.21.15.78:0/1610644982 --> [v2:172.21.15.161:6808/2751796117,v1:172.21.15.161:6809/2751796117] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd2d0062b30 con 0x7fd2bc0adde0 2024-08-18T23:42:56.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.452+0000 7fd2c27fc640 1 -- 172.21.15.78:0/1610644982 <== osd.5 v2:172.21.15.161:6808/2751796117 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd2d0062b30 con 0x7fd2bc0adde0 2024-08-18T23:42:56.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.452+0000 7fd2d781a640 1 -- 172.21.15.78:0/1610644982 >> [v2:172.21.15.161:6808/2751796117,v1:172.21.15.161:6809/2751796117] conn(0x7fd2bc0adde0 msgr2=0x7fd2bc0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:56.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.452+0000 7fd2d781a640 1 --2- 172.21.15.78:0/1610644982 >> [v2:172.21.15.161:6808/2751796117,v1:172.21.15.161:6809/2751796117] conn(0x7fd2bc0adde0 0x7fd2bc0b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:56.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.452+0000 7fd2d781a640 1 -- 172.21.15.78:0/1610644982 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2bc0a2c10 msgr2=0x7fd2bc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:56.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.452+0000 7fd2d781a640 1 --2- 172.21.15.78:0/1610644982 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2bc0a2c10 0x7fd2bc0a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4005e10 tx=0x7fd2c4005da0 comp rx=0 tx=0).stop 2024-08-18T23:42:56.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.452+0000 7fd2d781a640 1 -- 172.21.15.78:0/1610644982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2d01007b0 msgr2=0x7fd2d0198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:56.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.452+0000 7fd2d781a640 1 --2- 172.21.15.78:0/1610644982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2d01007b0 0x7fd2d0198080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00d950 tx=0x7fd2cc00de10 comp rx=0 tx=0).stop 2024-08-18T23:42:56.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.452+0000 7fd2d5db7640 1 -- 172.21.15.78:0/1610644982 reap_dead start 2024-08-18T23:42:56.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.452+0000 7fd2d781a640 1 -- 172.21.15.78:0/1610644982 shutdown_connections 2024-08-18T23:42:56.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.452+0000 7fd2d781a640 1 -- 172.21.15.78:0/1610644982 >> 172.21.15.78:0/1610644982 conn(0x7fd2d00fc4d0 msgr2=0x7fd2d006a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:56.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.452+0000 7fd2d781a640 1 -- 172.21.15.78:0/1610644982 shutdown_connections 2024-08-18T23:42:56.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:53.452+0000 7fd2d781a640 1 -- 172.21.15.78:0/1610644982 wait complete. 2024-08-18T23:42:57.156 INFO:teuthology.orchestra.run.smithi078.stdout:158913790009 2024-08-18T23:42:57.157 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-08-18T23:42:57.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:57 smithi078 ceph-mon[18440]: pgmap v307: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:42:57.529 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:57.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:57 smithi123 ceph-mon[21808]: pgmap v307: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:42:57.559 INFO:teuthology.orchestra.run.smithi078.stdout:270582939685 2024-08-18T23:42:57.559 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-08-18T23:42:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:57 smithi161 ceph-mon[22027]: pgmap v307: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:42:58.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.238+0000 7fc1ee0e7640 1 -- 172.21.15.78:0/2559812022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1e8100b50 msgr2=0x7fc1e8100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:58.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.238+0000 7fc1ee0e7640 1 --2- 172.21.15.78:0/2559812022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1e8100b50 0x7fc1e8100f30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009900 tx=0x7fc1d802a480 comp rx=0 tx=0).stop 2024-08-18T23:42:58.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.238+0000 7fc1ee0e7640 1 -- 172.21.15.78:0/2559812022 shutdown_connections 2024-08-18T23:42:58.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.238+0000 7fc1ee0e7640 1 --2- 172.21.15.78:0/2559812022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1e8102c20 0x7fc1e810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:58.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.238+0000 7fc1ee0e7640 1 --2- 172.21.15.78:0/2559812022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1e81022a0 0x7fc1e81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:58.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.238+0000 7fc1ee0e7640 1 --2- 172.21.15.78:0/2559812022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1e8100b50 0x7fc1e8100f30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:58.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.238+0000 7fc1ee0e7640 1 -- 172.21.15.78:0/2559812022 >> 172.21.15.78:0/2559812022 conn(0x7fc1e80fc500 msgr2=0x7fc1e80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:58.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.239+0000 7fc1ee0e7640 1 -- 172.21.15.78:0/2559812022 shutdown_connections 2024-08-18T23:42:58.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.239+0000 7fc1ee0e7640 1 -- 172.21.15.78:0/2559812022 wait complete. 2024-08-18T23:42:58.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.239+0000 7fc1ee0e7640 1 Processor -- start 2024-08-18T23:42:58.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.239+0000 7fc1ee0e7640 1 -- start start 2024-08-18T23:42:58.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.239+0000 7fc1ee0e7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1e8100b50 0x7fc1e81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:58.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.239+0000 7fc1ee0e7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1e81022a0 0x7fc1e81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:58.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.239+0000 7fc1ee0e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1e8102c20 0x7fc1e819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:58.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.239+0000 7fc1ee0e7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e8110d50 con 0x7fc1e8102c20 2024-08-18T23:42:58.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.239+0000 7fc1ee0e7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e8110bd0 con 0x7fc1e8100b50 2024-08-18T23:42:58.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.239+0000 7fc1ee0e7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e8110ed0 con 0x7fc1e81022a0 2024-08-18T23:42:58.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.240+0000 7fc1e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1e8102c20 0x7fc1e819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:58.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.240+0000 7fc1e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1e8102c20 0x7fc1e819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51140/0 (socket says 172.21.15.78:51140) 2024-08-18T23:42:58.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.240+0000 7fc1e7fff640 1 -- 172.21.15.78:0/848168704 learned_addr learned my addr 172.21.15.78:0/848168704 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:58.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.240+0000 7fc1e77fe640 1 --2- 172.21.15.78:0/848168704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1e8100b50 0x7fc1e81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:58.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.240+0000 7fc1deffd640 1 --2- 172.21.15.78:0/848168704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1e81022a0 0x7fc1e81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:58.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.240+0000 7fc1e7fff640 1 -- 172.21.15.78:0/848168704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1e81022a0 msgr2=0x7fc1e81988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:58.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.240+0000 7fc1e7fff640 1 --2- 172.21.15.78:0/848168704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1e81022a0 0x7fc1e81988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:58.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.240+0000 7fc1e7fff640 1 -- 172.21.15.78:0/848168704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1e8100b50 msgr2=0x7fc1e81983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:58.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.240+0000 7fc1e7fff640 1 --2- 172.21.15.78:0/848168704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1e8100b50 0x7fc1e81983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:58.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.240+0000 7fc1e7fff640 1 -- 172.21.15.78:0/848168704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e819d350 con 0x7fc1e8102c20 2024-08-18T23:42:58.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.240+0000 7fc1e7fff640 1 --2- 172.21.15.78:0/848168704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1e8102c20 0x7fc1e819cc70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000b970 tx=0x7fc1e000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:58.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.240+0000 7fc1e57fa640 1 -- 172.21.15.78:0/848168704 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0019070 con 0x7fc1e8102c20 2024-08-18T23:42:58.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.240+0000 7fc1ee0e7640 1 -- 172.21.15.78:0/848168704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e819d640 con 0x7fc1e8102c20 2024-08-18T23:42:58.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.240+0000 7fc1e57fa640 1 -- 172.21.15.78:0/848168704 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1e000cb40 con 0x7fc1e8102c20 2024-08-18T23:42:58.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.240+0000 7fc1e57fa640 1 -- 172.21.15.78:0/848168704 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0007650 con 0x7fc1e8102c20 2024-08-18T23:42:58.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.240+0000 7fc1ee0e7640 1 -- 172.21.15.78:0/848168704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e81a4f20 con 0x7fc1e8102c20 2024-08-18T23:42:58.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.241+0000 7fc1ee0e7640 1 -- 172.21.15.78:0/848168704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc1e8106f90 con 0x7fc1e8102c20 2024-08-18T23:42:58.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.242+0000 7fc1e57fa640 1 -- 172.21.15.78:0/848168704 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc1e001c070 con 0x7fc1e8102c20 2024-08-18T23:42:58.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.243+0000 7fc1e57fa640 1 --2- 172.21.15.78:0/848168704 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc1d40a2c10 0x7fc1d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:58.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.243+0000 7fc1e77fe640 1 --2- 172.21.15.78:0/848168704 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc1d40a2c10 0x7fc1d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:58.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.243+0000 7fc1e57fa640 1 -- 172.21.15.78:0/848168704 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc1e00cf1f0 con 0x7fc1e8102c20 2024-08-18T23:42:58.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.243+0000 7fc1e57fa640 1 --2- 172.21.15.78:0/848168704 >> [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311] conn(0x7fc1d40adde0 0x7fc1d40b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:58.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.243+0000 7fc1e57fa640 1 -- 172.21.15.78:0/848168704 --> [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc1e00047d0 con 0x7fc1d40adde0 2024-08-18T23:42:58.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.243+0000 7fc1e57fa640 1 -- 172.21.15.78:0/848168704 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fc1e00cf640 con 0x7fc1e8102c20 2024-08-18T23:42:58.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.243+0000 7fc1deffd640 1 --2- 172.21.15.78:0/848168704 >> [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311] conn(0x7fc1d40adde0 0x7fc1d40b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:58.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.244+0000 7fc1e77fe640 1 --2- 172.21.15.78:0/848168704 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc1d40a2c10 0x7fc1d40a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8000c00 tx=0x7fc1d80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:58.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.244+0000 7fc1deffd640 1 --2- 172.21.15.78:0/848168704 >> [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311] conn(0x7fc1d40adde0 0x7fc1d40b0280 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:58.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.244+0000 7fc1e57fa640 1 -- 172.21.15.78:0/848168704 <== osd.0 v2:172.21.15.78:6802/3639645311 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc1e00047d0 con 0x7fc1d40adde0 2024-08-18T23:42:58.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.258+0000 7fc1ee0e7640 1 -- 172.21.15.78:0/848168704 --> [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc1e8101ee0 con 0x7fc1d40adde0 2024-08-18T23:42:58.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.258+0000 7fc1e57fa640 1 -- 172.21.15.78:0/848168704 <== osd.0 v2:172.21.15.78:6802/3639645311 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc1e8101ee0 con 0x7fc1d40adde0 2024-08-18T23:42:58.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.258+0000 7fc1ee0e7640 1 -- 172.21.15.78:0/848168704 >> [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311] conn(0x7fc1d40adde0 msgr2=0x7fc1d40b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:58.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.258+0000 7fc1ee0e7640 1 --2- 172.21.15.78:0/848168704 >> [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311] conn(0x7fc1d40adde0 0x7fc1d40b0280 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:58.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.258+0000 7fc1ee0e7640 1 -- 172.21.15.78:0/848168704 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc1d40a2c10 msgr2=0x7fc1d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:58.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.258+0000 7fc1ee0e7640 1 --2- 172.21.15.78:0/848168704 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc1d40a2c10 0x7fc1d40a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8000c00 tx=0x7fc1d80058d0 comp rx=0 tx=0).stop 2024-08-18T23:42:58.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.258+0000 7fc1ee0e7640 1 -- 172.21.15.78:0/848168704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1e8102c20 msgr2=0x7fc1e819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:58.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.258+0000 7fc1ee0e7640 1 --2- 172.21.15.78:0/848168704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1e8102c20 0x7fc1e819cc70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000b970 tx=0x7fc1e000be30 comp rx=0 tx=0).stop 2024-08-18T23:42:58.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.258+0000 7fc1e7fff640 1 -- 172.21.15.78:0/848168704 reap_dead start 2024-08-18T23:42:58.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.258+0000 7fc1ee0e7640 1 -- 172.21.15.78:0/848168704 shutdown_connections 2024-08-18T23:42:58.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.258+0000 7fc1ee0e7640 1 -- 172.21.15.78:0/848168704 >> 172.21.15.78:0/848168704 conn(0x7fc1e80fc500 msgr2=0x7fc1e8069ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:58.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.258+0000 7fc1ee0e7640 1 -- 172.21.15.78:0/848168704 shutdown_connections 2024-08-18T23:42:58.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.258+0000 7fc1ee0e7640 1 -- 172.21.15.78:0/848168704 wait complete. 2024-08-18T23:42:58.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.732+0000 7f2725dfc640 1 -- 172.21.15.78:0/502358929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27201018b0 msgr2=0x7f2720101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:58.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.732+0000 7f2725dfc640 1 --2- 172.21.15.78:0/502358929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27201018b0 0x7f2720101d10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2708009930 tx=0x7f270802a420 comp rx=0 tx=0).stop 2024-08-18T23:42:58.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.732+0000 7f2725dfc640 1 -- 172.21.15.78:0/502358929 shutdown_connections 2024-08-18T23:42:58.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.732+0000 7f2725dfc640 1 --2- 172.21.15.78:0/502358929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2720102250 0x7f272010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:58.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.732+0000 7f2725dfc640 1 --2- 172.21.15.78:0/502358929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27201018b0 0x7f2720101d10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:58.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.732+0000 7f2725dfc640 1 --2- 172.21.15.78:0/502358929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27201078b0 0x7f2720107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:58.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.732+0000 7f2725dfc640 1 -- 172.21.15.78:0/502358929 >> 172.21.15.78:0/502358929 conn(0x7f27200fd5f0 msgr2=0x7f27200ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:58.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.732+0000 7f2725dfc640 1 -- 172.21.15.78:0/502358929 shutdown_connections 2024-08-18T23:42:58.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.732+0000 7f2725dfc640 1 -- 172.21.15.78:0/502358929 wait complete. 2024-08-18T23:42:58.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.733+0000 7f2725dfc640 1 Processor -- start 2024-08-18T23:42:58.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.733+0000 7f2725dfc640 1 -- start start 2024-08-18T23:42:58.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.733+0000 7f2725dfc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27201018b0 0x7f27201980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:58.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.733+0000 7f2725dfc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2720102250 0x7f27201985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:58.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.733+0000 7f2725dfc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27201078b0 0x7f272019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:58.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.733+0000 7f2725dfc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2720111cb0 con 0x7f27201018b0 2024-08-18T23:42:58.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.733+0000 7f2725dfc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2720111b30 con 0x7f2720102250 2024-08-18T23:42:58.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.733+0000 7f2725dfc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2720111e30 con 0x7f27201078b0 2024-08-18T23:42:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.733+0000 7f271f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27201018b0 0x7f27201980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.733+0000 7f271f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27201018b0 0x7f27201980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51156/0 (socket says 172.21.15.78:51156) 2024-08-18T23:42:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.733+0000 7f271f7fe640 1 -- 172.21.15.78:0/1918532212 learned_addr learned my addr 172.21.15.78:0/1918532212 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:42:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.734+0000 7f2716ffd640 1 --2- 172.21.15.78:0/1918532212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2720102250 0x7f27201985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.734+0000 7f271f7fe640 1 -- 172.21.15.78:0/1918532212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27201078b0 msgr2=0x7f272019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.734+0000 7f271f7fe640 1 --2- 172.21.15.78:0/1918532212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27201078b0 0x7f272019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.734+0000 7f271f7fe640 1 -- 172.21.15.78:0/1918532212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2720102250 msgr2=0x7f27201985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.734+0000 7f271f7fe640 1 --2- 172.21.15.78:0/1918532212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2720102250 0x7f27201985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.734+0000 7f271f7fe640 1 -- 172.21.15.78:0/1918532212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272019d060 con 0x7f27201018b0 2024-08-18T23:42:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.734+0000 7f271f7fe640 1 --2- 172.21.15.78:0/1918532212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27201018b0 0x7f27201980b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f271000c920 tx=0x7f271000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.734+0000 7f271d7fa640 1 -- 172.21.15.78:0/1918532212 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710007c70 con 0x7f27201018b0 2024-08-18T23:42:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.734+0000 7f2725dfc640 1 -- 172.21.15.78:0/1918532212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f272019d350 con 0x7f27201018b0 2024-08-18T23:42:58.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.734+0000 7f271d7fa640 1 -- 172.21.15.78:0/1918532212 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2710007e10 con 0x7f27201018b0 2024-08-18T23:42:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.734+0000 7f271d7fa640 1 -- 172.21.15.78:0/1918532212 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710018890 con 0x7f27201018b0 2024-08-18T23:42:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.734+0000 7f2725dfc640 1 -- 172.21.15.78:0/1918532212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27201a4c30 con 0x7f27201018b0 2024-08-18T23:42:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.735+0000 7f2725dfc640 1 -- 172.21.15.78:0/1918532212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2720101d10 con 0x7f27201018b0 2024-08-18T23:42:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.736+0000 7f271d7fa640 1 -- 172.21.15.78:0/1918532212 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2710018a50 con 0x7f27201018b0 2024-08-18T23:42:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.737+0000 7f271d7fa640 1 --2- 172.21.15.78:0/1918532212 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f270c0a2c10 0x7f270c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:42:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.737+0000 7f2716ffd640 1 --2- 172.21.15.78:0/1918532212 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f270c0a2c10 0x7f270c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:42:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.737+0000 7f271d7fa640 1 -- 172.21.15.78:0/1918532212 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f27100cf6b0 con 0x7f27201018b0 2024-08-18T23:42:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.737+0000 7f2716ffd640 1 --2- 172.21.15.78:0/1918532212 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f270c0a2c10 0x7f270c0a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f27080096c0 tx=0x7f2708000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:42:58.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.741+0000 7f271d7fa640 1 -- 172.21.15.78:0/1918532212 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2710014030 con 0x7f27201018b0 2024-08-18T23:42:58.763 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:42:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.925+0000 7f2725dfc640 1 -- 172.21.15.78:0/1918532212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f27201a4ef0 con 0x7f27201018b0 2024-08-18T23:42:58.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.925+0000 7f271d7fa640 1 -- 172.21.15.78:0/1918532212 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f271001f300 con 0x7f27201018b0 2024-08-18T23:42:58.928 INFO:teuthology.orchestra.run.smithi078.stdout:55834574923 2024-08-18T23:42:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.928+0000 7f2725dfc640 1 -- 172.21.15.78:0/1918532212 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f270c0a2c10 msgr2=0x7f270c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.928+0000 7f2725dfc640 1 --2- 172.21.15.78:0/1918532212 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f270c0a2c10 0x7f270c0a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f27080096c0 tx=0x7f2708000bc0 comp rx=0 tx=0).stop 2024-08-18T23:42:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.928+0000 7f2725dfc640 1 -- 172.21.15.78:0/1918532212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27201018b0 msgr2=0x7f27201980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:42:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.928+0000 7f2725dfc640 1 --2- 172.21.15.78:0/1918532212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27201018b0 0x7f27201980b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f271000c920 tx=0x7f271000cde0 comp rx=0 tx=0).stop 2024-08-18T23:42:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.928+0000 7f2725dfc640 1 -- 172.21.15.78:0/1918532212 shutdown_connections 2024-08-18T23:42:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.928+0000 7f2725dfc640 1 --2- 172.21.15.78:0/1918532212 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f270c0a2c10 0x7f270c0a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.928+0000 7f2725dfc640 1 --2- 172.21.15.78:0/1918532212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27201078b0 0x7f272019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.928+0000 7f2725dfc640 1 --2- 172.21.15.78:0/1918532212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2720102250 0x7f27201985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.928+0000 7f2725dfc640 1 --2- 172.21.15.78:0/1918532212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27201018b0 0x7f27201980b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:42:58.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.928+0000 7f2725dfc640 1 -- 172.21.15.78:0/1918532212 >> 172.21.15.78:0/1918532212 conn(0x7f27200fd5f0 msgr2=0x7f27200fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:42:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.928+0000 7f2725dfc640 1 -- 172.21.15.78:0/1918532212 shutdown_connections 2024-08-18T23:42:58.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:42:58.928+0000 7f2725dfc640 1 -- 172.21.15.78:0/1918532212 wait complete. 2024-08-18T23:42:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:59 smithi078 ceph-mon[18440]: pgmap v308: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:42:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:42:59 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1918532212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-08-18T23:42:59.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:59 smithi123 ceph-mon[21808]: pgmap v308: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:42:59.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:42:59 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1918532212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-08-18T23:42:59.654 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574921 got 55834574923 for osd.1 2024-08-18T23:42:59.654 DEBUG:teuthology.parallel:result is None 2024-08-18T23:42:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:59 smithi161 ceph-mon[22027]: pgmap v308: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:42:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:42:59 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1918532212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-08-18T23:43:00.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.159+0000 7f45defa0640 1 -- 172.21.15.78:0/443186067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45d81078b0 msgr2=0x7f45d8107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:00.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.159+0000 7f45defa0640 1 --2- 172.21.15.78:0/443186067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45d81078b0 0x7f45d8107c90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009930 tx=0x7f45cc02a420 comp rx=0 tx=0).stop 2024-08-18T23:43:00.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.159+0000 7f45defa0640 1 -- 172.21.15.78:0/443186067 shutdown_connections 2024-08-18T23:43:00.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.159+0000 7f45defa0640 1 --2- 172.21.15.78:0/443186067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45d8102250 0x7f45d810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:00.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.159+0000 7f45defa0640 1 --2- 172.21.15.78:0/443186067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45d81018b0 0x7f45d8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:00.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.159+0000 7f45defa0640 1 --2- 172.21.15.78:0/443186067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45d81078b0 0x7f45d8107c90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:00.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.159+0000 7f45defa0640 1 -- 172.21.15.78:0/443186067 >> 172.21.15.78:0/443186067 conn(0x7f45d80fd5f0 msgr2=0x7f45d80ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:00.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.159+0000 7f45defa0640 1 -- 172.21.15.78:0/443186067 shutdown_connections 2024-08-18T23:43:00.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.159+0000 7f45defa0640 1 -- 172.21.15.78:0/443186067 wait complete. 2024-08-18T23:43:00.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.160+0000 7f45defa0640 1 Processor -- start 2024-08-18T23:43:00.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.160+0000 7f45defa0640 1 -- start start 2024-08-18T23:43:00.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.160+0000 7f45defa0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45d81018b0 0x7f45d8074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:00.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.160+0000 7f45defa0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45d8102250 0x7f45d8074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:00.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.160+0000 7f45defa0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45d81078b0 0x7f45d8074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:00.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.160+0000 7f45defa0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8111cb0 con 0x7f45d8102250 2024-08-18T23:43:00.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.160+0000 7f45defa0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8111b30 con 0x7f45d81018b0 2024-08-18T23:43:00.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.160+0000 7f45defa0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8111e30 con 0x7f45d81078b0 2024-08-18T23:43:00.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.160+0000 7f45d7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45d8102250 0x7f45d8074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:00.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.160+0000 7f45d7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45d8102250 0x7f45d8074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51178/0 (socket says 172.21.15.78:51178) 2024-08-18T23:43:00.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.160+0000 7f45d7fff640 1 -- 172.21.15.78:0/2415952169 learned_addr learned my addr 172.21.15.78:0/2415952169 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:00.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.160+0000 7f45dcd3c640 1 --2- 172.21.15.78:0/2415952169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45d81018b0 0x7f45d8074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:00.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.160+0000 7f45dd53d640 1 --2- 172.21.15.78:0/2415952169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45d81078b0 0x7f45d8074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:00.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.160+0000 7f45d7fff640 1 -- 172.21.15.78:0/2415952169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45d81078b0 msgr2=0x7f45d8074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:00.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.160+0000 7f45d7fff640 1 --2- 172.21.15.78:0/2415952169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45d81078b0 0x7f45d8074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:00.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.160+0000 7f45d7fff640 1 -- 172.21.15.78:0/2415952169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45d81018b0 msgr2=0x7f45d8074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:00.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.160+0000 7f45d7fff640 1 --2- 172.21.15.78:0/2415952169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45d81018b0 0x7f45d8074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:00.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.160+0000 7f45d7fff640 1 -- 172.21.15.78:0/2415952169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d81a6670 con 0x7f45d8102250 2024-08-18T23:43:00.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.161+0000 7f45d7fff640 1 --2- 172.21.15.78:0/2415952169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45d8102250 0x7f45d8074760 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f45c400c890 tx=0x7f45c400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:00.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.161+0000 7f45d5ffb640 1 -- 172.21.15.78:0/2415952169 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4004220 con 0x7f45d8102250 2024-08-18T23:43:00.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.161+0000 7f45d5ffb640 1 -- 172.21.15.78:0/2415952169 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c40043c0 con 0x7f45d8102250 2024-08-18T23:43:00.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.161+0000 7f45d5ffb640 1 -- 172.21.15.78:0/2415952169 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c40049e0 con 0x7f45d8102250 2024-08-18T23:43:00.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.161+0000 7f45defa0640 1 -- 172.21.15.78:0/2415952169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d81a6960 con 0x7f45d8102250 2024-08-18T23:43:00.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.161+0000 7f45defa0640 1 -- 172.21.15.78:0/2415952169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d81a6e50 con 0x7f45d8102250 2024-08-18T23:43:00.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.162+0000 7f45defa0640 1 -- 172.21.15.78:0/2415952169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d8101d10 con 0x7f45d8102250 2024-08-18T23:43:00.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.163+0000 7f45d5ffb640 1 -- 172.21.15.78:0/2415952169 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f45c40079b0 con 0x7f45d8102250 2024-08-18T23:43:00.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.164+0000 7f45d5ffb640 1 --2- 172.21.15.78:0/2415952169 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f45c80a2c90 0x7f45c80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:00.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.164+0000 7f45d5ffb640 1 -- 172.21.15.78:0/2415952169 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f45c40cf550 con 0x7f45d8102250 2024-08-18T23:43:00.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.164+0000 7f45dcd3c640 1 --2- 172.21.15.78:0/2415952169 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f45c80a2c90 0x7f45c80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:00.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.164+0000 7f45dcd3c640 1 --2- 172.21.15.78:0/2415952169 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f45c80a2c90 0x7f45c80a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009900 tx=0x7f45cc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:00.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.168+0000 7f45d5ffb640 1 -- 172.21.15.78:0/2415952169 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f45c4014030 con 0x7f45d8102250 2024-08-18T23:43:00.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.344+0000 7f45defa0640 1 -- 172.21.15.78:0/2415952169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f45d8109e20 con 0x7f45d8102250 2024-08-18T23:43:00.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.345+0000 7f45d5ffb640 1 -- 172.21.15.78:0/2415952169 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f45c409bff0 con 0x7f45d8102250 2024-08-18T23:43:00.347 INFO:teuthology.orchestra.run.smithi078.stdout:188978561078 2024-08-18T23:43:00.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.347+0000 7f45defa0640 1 -- 172.21.15.78:0/2415952169 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f45c80a2c90 msgr2=0x7f45c80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:00.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.347+0000 7f45defa0640 1 --2- 172.21.15.78:0/2415952169 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f45c80a2c90 0x7f45c80a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009900 tx=0x7f45cc0058d0 comp rx=0 tx=0).stop 2024-08-18T23:43:00.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.347+0000 7f45defa0640 1 -- 172.21.15.78:0/2415952169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45d8102250 msgr2=0x7f45d8074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:00.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.347+0000 7f45defa0640 1 --2- 172.21.15.78:0/2415952169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45d8102250 0x7f45d8074760 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f45c400c890 tx=0x7f45c400cd50 comp rx=0 tx=0).stop 2024-08-18T23:43:00.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.347+0000 7f45defa0640 1 -- 172.21.15.78:0/2415952169 shutdown_connections 2024-08-18T23:43:00.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.347+0000 7f45defa0640 1 --2- 172.21.15.78:0/2415952169 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f45c80a2c90 0x7f45c80a5150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:00.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.347+0000 7f45defa0640 1 --2- 172.21.15.78:0/2415952169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45d81078b0 0x7f45d8074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:00.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.347+0000 7f45defa0640 1 --2- 172.21.15.78:0/2415952169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45d8102250 0x7f45d8074760 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:00.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.347+0000 7f45defa0640 1 --2- 172.21.15.78:0/2415952169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45d81018b0 0x7f45d8074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:00.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.347+0000 7f45defa0640 1 -- 172.21.15.78:0/2415952169 >> 172.21.15.78:0/2415952169 conn(0x7f45d80fd5f0 msgr2=0x7f45d80fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:00.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.347+0000 7f45defa0640 1 -- 172.21.15.78:0/2415952169 shutdown_connections 2024-08-18T23:43:00.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:00.347+0000 7f45defa0640 1 -- 172.21.15.78:0/2415952169 wait complete. 2024-08-18T23:43:00.703 INFO:teuthology.orchestra.run.smithi078.stdout:34359738447 2024-08-18T23:43:00.703 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-08-18T23:43:00.706 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:00.710 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:00.934 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:01.318 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:01 smithi078 ceph-mon[18440]: pgmap v309: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-08-18T23:43:01.319 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:01 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2415952169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-08-18T23:43:01.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:01 smithi123 ceph-mon[21808]: pgmap v309: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-08-18T23:43:01.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:01 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2415952169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-08-18T23:43:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:01 smithi161 ceph-mon[22027]: pgmap v309: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-08-18T23:43:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:01 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2415952169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-08-18T23:43:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.887+0000 7f3b6cd4d640 1 -- 172.21.15.78:0/1324593296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b681067b0 msgr2=0x7f3b68106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.887+0000 7f3b6cd4d640 1 --2- 172.21.15.78:0/1324593296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b681067b0 0x7f3b68106b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009930 tx=0x7f3b5802a420 comp rx=0 tx=0).stop 2024-08-18T23:43:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.888+0000 7f3b6cd4d640 1 -- 172.21.15.78:0/1324593296 shutdown_connections 2024-08-18T23:43:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.888+0000 7f3b6cd4d640 1 --2- 172.21.15.78:0/1324593296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b68101150 0x7f3b6810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.888+0000 7f3b6cd4d640 1 --2- 172.21.15.78:0/1324593296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b681007b0 0x7f3b68100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.888+0000 7f3b6cd4d640 1 --2- 172.21.15.78:0/1324593296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b681067b0 0x7f3b68106b90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.888+0000 7f3b6cd4d640 1 -- 172.21.15.78:0/1324593296 >> 172.21.15.78:0/1324593296 conn(0x7f3b680fc4d0 msgr2=0x7f3b680fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.888+0000 7f3b6cd4d640 1 -- 172.21.15.78:0/1324593296 shutdown_connections 2024-08-18T23:43:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.888+0000 7f3b6cd4d640 1 -- 172.21.15.78:0/1324593296 wait complete. 2024-08-18T23:43:02.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.888+0000 7f3b6cd4d640 1 Processor -- start 2024-08-18T23:43:02.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.889+0000 7f3b6cd4d640 1 -- start start 2024-08-18T23:43:02.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.889+0000 7f3b6cd4d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b681007b0 0x7f3b6819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:02.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.889+0000 7f3b6cd4d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b68101150 0x7f3b6819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:02.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.889+0000 7f3b6cd4d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b681067b0 0x7f3b6819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:02.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.889+0000 7f3b6cd4d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b68110bb0 con 0x7f3b681007b0 2024-08-18T23:43:02.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.889+0000 7f3b6cd4d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b68110a30 con 0x7f3b68101150 2024-08-18T23:43:02.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.889+0000 7f3b6cd4d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3b68110d30 con 0x7f3b681067b0 2024-08-18T23:43:02.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.889+0000 7f3b6659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b681007b0 0x7f3b6819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:02.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.889+0000 7f3b65d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b68101150 0x7f3b6819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:02.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.889+0000 7f3b65d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b68101150 0x7f3b6819a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:51736/0 (socket says 172.21.15.78:51736) 2024-08-18T23:43:02.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.889+0000 7f3b65d9b640 1 -- 172.21.15.78:0/1337853337 learned_addr learned my addr 172.21.15.78:0/1337853337 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:02.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.889+0000 7f3b66d9d640 1 --2- 172.21.15.78:0/1337853337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b681067b0 0x7f3b6819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:02.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.889+0000 7f3b65d9b640 1 -- 172.21.15.78:0/1337853337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b681067b0 msgr2=0x7f3b6819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:02.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.889+0000 7f3b65d9b640 1 --2- 172.21.15.78:0/1337853337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b681067b0 0x7f3b6819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.889+0000 7f3b65d9b640 1 -- 172.21.15.78:0/1337853337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b681007b0 msgr2=0x7f3b6819a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:02.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.889+0000 7f3b65d9b640 1 --2- 172.21.15.78:0/1337853337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b681007b0 0x7f3b6819a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.889+0000 7f3b65d9b640 1 -- 172.21.15.78:0/1337853337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6819f2d0 con 0x7f3b68101150 2024-08-18T23:43:02.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.890+0000 7f3b66d9d640 1 --2- 172.21.15.78:0/1337853337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b681067b0 0x7f3b6819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:43:02.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.890+0000 7f3b65d9b640 1 --2- 172.21.15.78:0/1337853337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b68101150 0x7f3b6819a860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00d950 tx=0x7f3b5c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:02.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.890+0000 7f3b577fe640 1 -- 172.21.15.78:0/1337853337 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c017070 con 0x7f3b68101150 2024-08-18T23:43:02.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.890+0000 7f3b577fe640 1 -- 172.21.15.78:0/1337853337 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b5c009c30 con 0x7f3b68101150 2024-08-18T23:43:02.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.890+0000 7f3b6cd4d640 1 -- 172.21.15.78:0/1337853337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6819f5c0 con 0x7f3b68101150 2024-08-18T23:43:02.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.890+0000 7f3b577fe640 1 -- 172.21.15.78:0/1337853337 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c00f6b0 con 0x7f3b68101150 2024-08-18T23:43:02.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.890+0000 7f3b6cd4d640 1 -- 172.21.15.78:0/1337853337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b681a6ea0 con 0x7f3b68101150 2024-08-18T23:43:02.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.891+0000 7f3b6cd4d640 1 -- 172.21.15.78:0/1337853337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b68100c10 con 0x7f3b68101150 2024-08-18T23:43:02.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.892+0000 7f3b577fe640 1 -- 172.21.15.78:0/1337853337 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3b5c0109d0 con 0x7f3b68101150 2024-08-18T23:43:02.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.893+0000 7f3b577fe640 1 --2- 172.21.15.78:0/1337853337 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3b500a2c10 0x7f3b500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:02.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.893+0000 7f3b6659c640 1 --2- 172.21.15.78:0/1337853337 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3b500a2c10 0x7f3b500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:02.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.893+0000 7f3b577fe640 1 -- 172.21.15.78:0/1337853337 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3b5c0cf1d0 con 0x7f3b68101150 2024-08-18T23:43:02.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.894+0000 7f3b6659c640 1 --2- 172.21.15.78:0/1337853337 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3b500a2c10 0x7f3b500a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009900 tx=0x7f3b580058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:02.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.897+0000 7f3b577fe640 1 -- 172.21.15.78:0/1337853337 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3b5c009de0 con 0x7f3b68101150 2024-08-18T23:43:02.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.072+0000 7f3b6cd4d640 1 -- 172.21.15.78:0/1337853337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f3b6819b680 con 0x7f3b68101150 2024-08-18T23:43:02.074 INFO:teuthology.orchestra.run.smithi078.stdout:270582939688 2024-08-18T23:43:02.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.072+0000 7f3b577fe640 1 -- 172.21.15.78:0/1337853337 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3b6819b680 con 0x7f3b68101150 2024-08-18T23:43:02.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.074+0000 7f3b6cd4d640 1 -- 172.21.15.78:0/1337853337 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3b500a2c10 msgr2=0x7f3b500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:02.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.074+0000 7f3b6cd4d640 1 --2- 172.21.15.78:0/1337853337 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3b500a2c10 0x7f3b500a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009900 tx=0x7f3b580058d0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.074+0000 7f3b6cd4d640 1 -- 172.21.15.78:0/1337853337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b68101150 msgr2=0x7f3b6819a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:02.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.074+0000 7f3b6cd4d640 1 --2- 172.21.15.78:0/1337853337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b68101150 0x7f3b6819a860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00d950 tx=0x7f3b5c00de10 comp rx=0 tx=0).stop 2024-08-18T23:43:02.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.074+0000 7f3b6cd4d640 1 -- 172.21.15.78:0/1337853337 shutdown_connections 2024-08-18T23:43:02.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.074+0000 7f3b6cd4d640 1 --2- 172.21.15.78:0/1337853337 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3b500a2c10 0x7f3b500a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.074+0000 7f3b6cd4d640 1 --2- 172.21.15.78:0/1337853337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b681067b0 0x7f3b6819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.074+0000 7f3b6cd4d640 1 --2- 172.21.15.78:0/1337853337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b68101150 0x7f3b6819a860 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.074+0000 7f3b6cd4d640 1 --2- 172.21.15.78:0/1337853337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b681007b0 0x7f3b6819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.074+0000 7f3b6cd4d640 1 -- 172.21.15.78:0/1337853337 >> 172.21.15.78:0/1337853337 conn(0x7f3b680fc4d0 msgr2=0x7f3b6806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:02.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.074+0000 7f3b6cd4d640 1 -- 172.21.15.78:0/1337853337 shutdown_connections 2024-08-18T23:43:02.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.074+0000 7f3b6cd4d640 1 -- 172.21.15.78:0/1337853337 wait complete. 2024-08-18T23:43:02.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:02 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1337853337' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-08-18T23:43:02.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:02 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/4186731916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-08-18T23:43:02.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:02 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1337853337' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-08-18T23:43:02.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:02 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/4186731916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-08-18T23:43:02.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:02 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1337853337' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-08-18T23:43:02.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:02 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/4186731916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-08-18T23:43:02.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.684+0000 7f44d98e9640 1 -- 172.21.15.78:0/3732436509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d4100b50 msgr2=0x7f44d4100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:02.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.684+0000 7f44d98e9640 1 --2- 172.21.15.78:0/3732436509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d4100b50 0x7f44d4100f30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f44c4009900 tx=0x7f44c402a480 comp rx=0 tx=0).stop 2024-08-18T23:43:02.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.684+0000 7f44d98e9640 1 -- 172.21.15.78:0/3732436509 shutdown_connections 2024-08-18T23:43:02.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.684+0000 7f44d98e9640 1 --2- 172.21.15.78:0/3732436509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44d4102c20 0x7f44d410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.684+0000 7f44d98e9640 1 --2- 172.21.15.78:0/3732436509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d41022a0 0x7f44d41026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.684+0000 7f44d98e9640 1 --2- 172.21.15.78:0/3732436509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d4100b50 0x7f44d4100f30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.684+0000 7f44d98e9640 1 -- 172.21.15.78:0/3732436509 >> 172.21.15.78:0/3732436509 conn(0x7f44d40fc500 msgr2=0x7f44d40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:02.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.684+0000 7f44d98e9640 1 -- 172.21.15.78:0/3732436509 shutdown_connections 2024-08-18T23:43:02.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.684+0000 7f44d98e9640 1 -- 172.21.15.78:0/3732436509 wait complete. 2024-08-18T23:43:02.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.685+0000 7f44d98e9640 1 Processor -- start 2024-08-18T23:43:02.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.685+0000 7f44d98e9640 1 -- start start 2024-08-18T23:43:02.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.685+0000 7f44d98e9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44d4100b50 0x7f44d419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:02.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.685+0000 7f44d98e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d41022a0 0x7f44d419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:02.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.685+0000 7f44d98e9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d4102c20 0x7f44d419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:02.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.685+0000 7f44d98e9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f44d4111df0 con 0x7f44d41022a0 2024-08-18T23:43:02.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.685+0000 7f44d98e9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f44d4111c70 con 0x7f44d4102c20 2024-08-18T23:43:02.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.685+0000 7f44d98e9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f44d4111f70 con 0x7f44d4100b50 2024-08-18T23:43:02.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.685+0000 7f44cbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d41022a0 0x7f44d419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:02.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.685+0000 7f44cbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d41022a0 0x7f44d419a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51242/0 (socket says 172.21.15.78:51242) 2024-08-18T23:43:02.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.685+0000 7f44cbfff640 1 -- 172.21.15.78:0/905847478 learned_addr learned my addr 172.21.15.78:0/905847478 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:02.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.685+0000 7f44d2ffd640 1 --2- 172.21.15.78:0/905847478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44d4100b50 0x7f44d419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:02.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.685+0000 7f44d37fe640 1 --2- 172.21.15.78:0/905847478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d4102c20 0x7f44d419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:02.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.685+0000 7f44cbfff640 1 -- 172.21.15.78:0/905847478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44d4100b50 msgr2=0x7f44d419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:02.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.685+0000 7f44cbfff640 1 --2- 172.21.15.78:0/905847478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44d4100b50 0x7f44d419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.685+0000 7f44cbfff640 1 -- 172.21.15.78:0/905847478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d4102c20 msgr2=0x7f44d419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.685+0000 7f44cbfff640 1 --2- 172.21.15.78:0/905847478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d4102c20 0x7f44d419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.685+0000 7f44cbfff640 1 -- 172.21.15.78:0/905847478 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d419f420 con 0x7f44d41022a0 2024-08-18T23:43:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.686+0000 7f44cbfff640 1 --2- 172.21.15.78:0/905847478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d41022a0 0x7f44d419a9b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00b4a0 tx=0x7f44bc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.686+0000 7f44d88e7640 1 -- 172.21.15.78:0/905847478 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc014070 con 0x7f44d41022a0 2024-08-18T23:43:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.686+0000 7f44d88e7640 1 -- 172.21.15.78:0/905847478 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44bc010af0 con 0x7f44d41022a0 2024-08-18T23:43:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.686+0000 7f44d98e9640 1 -- 172.21.15.78:0/905847478 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d419f710 con 0x7f44d41022a0 2024-08-18T23:43:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.686+0000 7f44d88e7640 1 -- 172.21.15.78:0/905847478 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc017d10 con 0x7f44d41022a0 2024-08-18T23:43:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.686+0000 7f44d98e9640 1 -- 172.21.15.78:0/905847478 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d41a6ff0 con 0x7f44d41022a0 2024-08-18T23:43:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.687+0000 7f44d98e9640 1 -- 172.21.15.78:0/905847478 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d4100f30 con 0x7f44d41022a0 2024-08-18T23:43:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.688+0000 7f44d88e7640 1 -- 172.21.15.78:0/905847478 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f44bc017070 con 0x7f44d41022a0 2024-08-18T23:43:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.689+0000 7f44d88e7640 1 --2- 172.21.15.78:0/905847478 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f44c00a2c10 0x7f44c00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.689+0000 7f44d2ffd640 1 --2- 172.21.15.78:0/905847478 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f44c00a2c10 0x7f44c00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.689+0000 7f44d88e7640 1 -- 172.21.15.78:0/905847478 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f44bc0ce790 con 0x7f44d41022a0 2024-08-18T23:43:02.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.689+0000 7f44d2ffd640 1 --2- 172.21.15.78:0/905847478 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f44c00a2c10 0x7f44c00a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f44c4000c00 tx=0x7f44c40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:02.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.692+0000 7f44d88e7640 1 -- 172.21.15.78:0/905847478 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f44bc09b340 con 0x7f44d41022a0 2024-08-18T23:43:02.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.863+0000 7f44d98e9640 1 -- 172.21.15.78:0/905847478 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f44d402d5e0 con 0x7f44d41022a0 2024-08-18T23:43:02.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.863+0000 7f44d88e7640 1 -- 172.21.15.78:0/905847478 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f44bc09ac40 con 0x7f44d41022a0 2024-08-18T23:43:02.866 INFO:teuthology.orchestra.run.smithi078.stdout:77309411400 2024-08-18T23:43:02.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.866+0000 7f44d98e9640 1 -- 172.21.15.78:0/905847478 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f44c00a2c10 msgr2=0x7f44c00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:02.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.866+0000 7f44d98e9640 1 --2- 172.21.15.78:0/905847478 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f44c00a2c10 0x7f44c00a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f44c4000c00 tx=0x7f44c40058d0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.866+0000 7f44d98e9640 1 -- 172.21.15.78:0/905847478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d41022a0 msgr2=0x7f44d419a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:02.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.866+0000 7f44d98e9640 1 --2- 172.21.15.78:0/905847478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d41022a0 0x7f44d419a9b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00b4a0 tx=0x7f44bc00b960 comp rx=0 tx=0).stop 2024-08-18T23:43:02.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.866+0000 7f44d98e9640 1 -- 172.21.15.78:0/905847478 shutdown_connections 2024-08-18T23:43:02.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.866+0000 7f44d98e9640 1 --2- 172.21.15.78:0/905847478 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f44c00a2c10 0x7f44c00a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.866+0000 7f44d98e9640 1 --2- 172.21.15.78:0/905847478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44d4102c20 0x7f44d419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.866+0000 7f44d98e9640 1 --2- 172.21.15.78:0/905847478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44d41022a0 0x7f44d419a9b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.866+0000 7f44d98e9640 1 --2- 172.21.15.78:0/905847478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44d4100b50 0x7f44d419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:02.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.866+0000 7f44d98e9640 1 -- 172.21.15.78:0/905847478 >> 172.21.15.78:0/905847478 conn(0x7f44d40fc500 msgr2=0x7f44d4105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:02.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.866+0000 7f44d98e9640 1 -- 172.21.15.78:0/905847478 shutdown_connections 2024-08-18T23:43:02.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.866+0000 7f44d98e9640 1 -- 172.21.15.78:0/905847478 wait complete. 2024-08-18T23:43:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:03 smithi078 ceph-mon[18440]: pgmap v310: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:03 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/905847478' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-08-18T23:43:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:03 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/283941245' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-08-18T23:43:03.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:03 smithi123 ceph-mon[21808]: pgmap v310: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:03.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:03 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/905847478' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-08-18T23:43:03.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:03 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/283941245' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-08-18T23:43:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:03 smithi161 ceph-mon[22027]: pgmap v310: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:03 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/905847478' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-08-18T23:43:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:03 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/283941245' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-08-18T23:43:03.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.708+0000 7fec9cb6c640 1 -- 172.21.15.78:0/673710885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec98107fd0 msgr2=0x7fec9810e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:03.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.708+0000 7fec9cb6c640 1 --2- 172.21.15.78:0/673710885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec98107fd0 0x7fec9810e970 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fec90009930 tx=0x7fec9002a440 comp rx=0 tx=0).stop 2024-08-18T23:43:03.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.709+0000 7fec9cb6c640 1 -- 172.21.15.78:0/673710885 shutdown_connections 2024-08-18T23:43:03.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.709+0000 7fec9cb6c640 1 --2- 172.21.15.78:0/673710885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec98107fd0 0x7fec9810e970 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:03.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.709+0000 7fec9cb6c640 1 --2- 172.21.15.78:0/673710885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec98103410 0x7fec981078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:03.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.709+0000 7fec9cb6c640 1 --2- 172.21.15.78:0/673710885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec98102a60 0x7fec98102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:03.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.709+0000 7fec9cb6c640 1 -- 172.21.15.78:0/673710885 >> 172.21.15.78:0/673710885 conn(0x7fec980fc500 msgr2=0x7fec980fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:03.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.709+0000 7fec9cb6c640 1 -- 172.21.15.78:0/673710885 shutdown_connections 2024-08-18T23:43:03.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.709+0000 7fec9cb6c640 1 -- 172.21.15.78:0/673710885 wait complete. 2024-08-18T23:43:03.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.709+0000 7fec9cb6c640 1 Processor -- start 2024-08-18T23:43:03.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.709+0000 7fec9cb6c640 1 -- start start 2024-08-18T23:43:03.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec9cb6c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec98102a60 0x7fec98198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:03.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec9cb6c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec98103410 0x7fec98198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:03.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec9cb6c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec98107fd0 0x7fec9819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:03.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec9cb6c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fec98111e10 con 0x7fec98102a60 2024-08-18T23:43:03.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec9cb6c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fec98111c90 con 0x7fec98107fd0 2024-08-18T23:43:03.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec9cb6c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fec98111f90 con 0x7fec98103410 2024-08-18T23:43:03.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec9659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec98102a60 0x7fec98198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:03.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec9659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec98102a60 0x7fec98198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51268/0 (socket says 172.21.15.78:51268) 2024-08-18T23:43:03.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec9659c640 1 -- 172.21.15.78:0/1655007148 learned_addr learned my addr 172.21.15.78:0/1655007148 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:03.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec96d9d640 1 --2- 172.21.15.78:0/1655007148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec98107fd0 0x7fec9819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:03.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec95d9b640 1 --2- 172.21.15.78:0/1655007148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec98103410 0x7fec98198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:03.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec9659c640 1 -- 172.21.15.78:0/1655007148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec98103410 msgr2=0x7fec98198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:03.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec9659c640 1 --2- 172.21.15.78:0/1655007148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec98103410 0x7fec98198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:03.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec9659c640 1 -- 172.21.15.78:0/1655007148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec98107fd0 msgr2=0x7fec9819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:03.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec9659c640 1 --2- 172.21.15.78:0/1655007148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec98107fd0 0x7fec9819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:03.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec9659c640 1 -- 172.21.15.78:0/1655007148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9819d300 con 0x7fec98102a60 2024-08-18T23:43:03.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec9659c640 1 --2- 172.21.15.78:0/1655007148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec98102a60 0x7fec98198350 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fec8800eab0 tx=0x7fec8800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:03.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec8f7fe640 1 -- 172.21.15.78:0/1655007148 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8800ce40 con 0x7fec98102a60 2024-08-18T23:43:03.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec8f7fe640 1 -- 172.21.15.78:0/1655007148 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec88011950 con 0x7fec98102a60 2024-08-18T23:43:03.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec9cb6c640 1 -- 172.21.15.78:0/1655007148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9819d5f0 con 0x7fec98102a60 2024-08-18T23:43:03.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.710+0000 7fec8f7fe640 1 -- 172.21.15.78:0/1655007148 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88010ae0 con 0x7fec98102a60 2024-08-18T23:43:03.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.711+0000 7fec9cb6c640 1 -- 172.21.15.78:0/1655007148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec981a4ed0 con 0x7fec98102a60 2024-08-18T23:43:03.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.712+0000 7fec9cb6c640 1 -- 172.21.15.78:0/1655007148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec98102e40 con 0x7fec98102a60 2024-08-18T23:43:03.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.713+0000 7fec8f7fe640 1 -- 172.21.15.78:0/1655007148 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fec88011b00 con 0x7fec98102a60 2024-08-18T23:43:03.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.713+0000 7fec8f7fe640 1 --2- 172.21.15.78:0/1655007148 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fec840a2c10 0x7fec840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:03.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.714+0000 7fec95d9b640 1 --2- 172.21.15.78:0/1655007148 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fec840a2c10 0x7fec840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:03.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.714+0000 7fec8f7fe640 1 -- 172.21.15.78:0/1655007148 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fec880cfc00 con 0x7fec98102a60 2024-08-18T23:43:03.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.714+0000 7fec95d9b640 1 --2- 172.21.15.78:0/1655007148 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fec840a2c10 0x7fec840a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fec980fe980 tx=0x7fec80008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:03.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.718+0000 7fec8f7fe640 1 -- 172.21.15.78:0/1655007148 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fec88016030 con 0x7fec98102a60 2024-08-18T23:43:03.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.890+0000 7fec9cb6c640 1 -- 172.21.15.78:0/1655007148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fec98109fe0 con 0x7fec98102a60 2024-08-18T23:43:03.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.891+0000 7fec8f7fe640 1 -- 172.21.15.78:0/1655007148 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fec8809c6a0 con 0x7fec98102a60 2024-08-18T23:43:03.892 INFO:teuthology.orchestra.run.smithi078.stdout:240518168621 2024-08-18T23:43:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.893+0000 7fec9cb6c640 1 -- 172.21.15.78:0/1655007148 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fec840a2c10 msgr2=0x7fec840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.893+0000 7fec9cb6c640 1 --2- 172.21.15.78:0/1655007148 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fec840a2c10 0x7fec840a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fec980fe980 tx=0x7fec80008040 comp rx=0 tx=0).stop 2024-08-18T23:43:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.893+0000 7fec9cb6c640 1 -- 172.21.15.78:0/1655007148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec98102a60 msgr2=0x7fec98198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.893+0000 7fec9cb6c640 1 --2- 172.21.15.78:0/1655007148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec98102a60 0x7fec98198350 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fec8800eab0 tx=0x7fec8800ef70 comp rx=0 tx=0).stop 2024-08-18T23:43:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.893+0000 7fec9cb6c640 1 -- 172.21.15.78:0/1655007148 shutdown_connections 2024-08-18T23:43:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.893+0000 7fec9cb6c640 1 --2- 172.21.15.78:0/1655007148 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fec840a2c10 0x7fec840a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.893+0000 7fec9cb6c640 1 --2- 172.21.15.78:0/1655007148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec98107fd0 0x7fec9819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.893+0000 7fec9cb6c640 1 --2- 172.21.15.78:0/1655007148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec98103410 0x7fec98198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:03.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.893+0000 7fec9cb6c640 1 --2- 172.21.15.78:0/1655007148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fec98102a60 0x7fec98198350 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:03.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.893+0000 7fec9cb6c640 1 -- 172.21.15.78:0/1655007148 >> 172.21.15.78:0/1655007148 conn(0x7fec980fc500 msgr2=0x7fec981060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:03.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.893+0000 7fec9cb6c640 1 -- 172.21.15.78:0/1655007148 shutdown_connections 2024-08-18T23:43:03.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.893+0000 7fec9cb6c640 1 -- 172.21.15.78:0/1655007148 wait complete. 2024-08-18T23:43:03.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.954+0000 7f8a2ff66640 1 -- 172.21.15.78:0/2024471083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a281018e0 msgr2=0x7f8a28101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:03.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.954+0000 7f8a2ff66640 1 --2- 172.21.15.78:0/2024471083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a281018e0 0x7f8a28101d40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009930 tx=0x7f8a1802a420 comp rx=0 tx=0).stop 2024-08-18T23:43:03.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.954+0000 7f8a2ff66640 1 -- 172.21.15.78:0/2024471083 shutdown_connections 2024-08-18T23:43:03.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.954+0000 7f8a2ff66640 1 --2- 172.21.15.78:0/2024471083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a28102280 0x7f8a2810e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:03.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.954+0000 7f8a2ff66640 1 --2- 172.21.15.78:0/2024471083 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a281018e0 0x7f8a28101d40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:03.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.954+0000 7f8a2ff66640 1 --2- 172.21.15.78:0/2024471083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a281078e0 0x7f8a28107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:03.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.954+0000 7f8a2ff66640 1 -- 172.21.15.78:0/2024471083 >> 172.21.15.78:0/2024471083 conn(0x7f8a280fd600 msgr2=0x7f8a280ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:03.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.954+0000 7f8a2ff66640 1 -- 172.21.15.78:0/2024471083 shutdown_connections 2024-08-18T23:43:03.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.954+0000 7f8a2ff66640 1 -- 172.21.15.78:0/2024471083 wait complete. 2024-08-18T23:43:03.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.955+0000 7f8a2ff66640 1 Processor -- start 2024-08-18T23:43:03.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.955+0000 7f8a2ff66640 1 -- start start 2024-08-18T23:43:03.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.955+0000 7f8a2ff66640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a281018e0 0x7f8a28198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:03.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.955+0000 7f8a2ff66640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a28102280 0x7f8a28198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:03.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.955+0000 7f8a2ff66640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a281078e0 0x7f8a2819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:03.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.955+0000 7f8a2ff66640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8a28111ce0 con 0x7f8a281018e0 2024-08-18T23:43:03.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.955+0000 7f8a2ff66640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a28111b60 con 0x7f8a281078e0 2024-08-18T23:43:03.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.955+0000 7f8a2ff66640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8a28111e60 con 0x7f8a28102280 2024-08-18T23:43:03.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.955+0000 7f8a2dd02640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a281018e0 0x7f8a28198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:03.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.955+0000 7f8a2dd02640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a281018e0 0x7f8a28198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51284/0 (socket says 172.21.15.78:51284) 2024-08-18T23:43:03.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.956+0000 7f8a2dd02640 1 -- 172.21.15.78:0/3458872508 learned_addr learned my addr 172.21.15.78:0/3458872508 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:03.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.956+0000 7f8a2d501640 1 --2- 172.21.15.78:0/3458872508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a28102280 0x7f8a28198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:03.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.956+0000 7f8a2e503640 1 --2- 172.21.15.78:0/3458872508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a281078e0 0x7f8a2819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:03.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.956+0000 7f8a2dd02640 1 -- 172.21.15.78:0/3458872508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a28102280 msgr2=0x7f8a28198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:03.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.956+0000 7f8a2dd02640 1 --2- 172.21.15.78:0/3458872508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a28102280 0x7f8a28198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:03.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.956+0000 7f8a2dd02640 1 -- 172.21.15.78:0/3458872508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a281078e0 msgr2=0x7f8a2819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:03.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.956+0000 7f8a2dd02640 1 --2- 172.21.15.78:0/3458872508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a281078e0 0x7f8a2819cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:03.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.956+0000 7f8a2dd02640 1 -- 172.21.15.78:0/3458872508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2819d1b0 con 0x7f8a281018e0 2024-08-18T23:43:03.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.956+0000 7f8a2dd02640 1 --2- 172.21.15.78:0/3458872508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a281018e0 0x7f8a28198200 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400d950 tx=0x7f8a2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:03.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.956+0000 7f8a1effd640 1 -- 172.21.15.78:0/3458872508 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a24017070 con 0x7f8a281018e0 2024-08-18T23:43:03.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.956+0000 7f8a1effd640 1 -- 172.21.15.78:0/3458872508 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a24009c30 con 0x7f8a281018e0 2024-08-18T23:43:03.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.956+0000 7f8a2ff66640 1 -- 172.21.15.78:0/3458872508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a2819d4a0 con 0x7f8a281018e0 2024-08-18T23:43:03.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.956+0000 7f8a1effd640 1 -- 172.21.15.78:0/3458872508 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2400f6b0 con 0x7f8a281018e0 2024-08-18T23:43:03.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.956+0000 7f8a2ff66640 1 -- 172.21.15.78:0/3458872508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a281a4d80 con 0x7f8a281018e0 2024-08-18T23:43:03.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.957+0000 7f8a2ff66640 1 -- 172.21.15.78:0/3458872508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a28101d40 con 0x7f8a281018e0 2024-08-18T23:43:03.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.958+0000 7f8a1effd640 1 -- 172.21.15.78:0/3458872508 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8a240109d0 con 0x7f8a281018e0 2024-08-18T23:43:03.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.959+0000 7f8a1effd640 1 --2- 172.21.15.78:0/3458872508 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8a140a2c90 0x7f8a140a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:03.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.959+0000 7f8a2d501640 1 --2- 172.21.15.78:0/3458872508 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8a140a2c90 0x7f8a140a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:03.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.959+0000 7f8a1effd640 1 -- 172.21.15.78:0/3458872508 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f8a240cef70 con 0x7f8a281018e0 2024-08-18T23:43:03.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.959+0000 7f8a2d501640 1 --2- 172.21.15.78:0/3458872508 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8a140a2c90 0x7f8a140a5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009900 tx=0x7f8a180058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:03.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.962+0000 7f8a1effd640 1 -- 172.21.15.78:0/3458872508 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8a24009de0 con 0x7f8a281018e0 2024-08-18T23:43:04.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:04.134+0000 7f8a2ff66640 1 -- 172.21.15.78:0/3458872508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8a28199560 con 0x7f8a281018e0 2024-08-18T23:43:04.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:04.134+0000 7f8a1effd640 1 -- 172.21.15.78:0/3458872508 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8a2409b440 con 0x7f8a281018e0 2024-08-18T23:43:04.135 INFO:teuthology.orchestra.run.smithi078.stdout:210453397553 2024-08-18T23:43:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:04.136+0000 7f8a2ff66640 1 -- 172.21.15.78:0/3458872508 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8a140a2c90 msgr2=0x7f8a140a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:04.136+0000 7f8a2ff66640 1 --2- 172.21.15.78:0/3458872508 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8a140a2c90 0x7f8a140a5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009900 tx=0x7f8a180058d0 comp rx=0 tx=0).stop 2024-08-18T23:43:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:04.136+0000 7f8a2ff66640 1 -- 172.21.15.78:0/3458872508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a281018e0 msgr2=0x7f8a28198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:04.136+0000 7f8a2ff66640 1 --2- 172.21.15.78:0/3458872508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a281018e0 0x7f8a28198200 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400d950 tx=0x7f8a2400de10 comp rx=0 tx=0).stop 2024-08-18T23:43:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:04.136+0000 7f8a2ff66640 1 -- 172.21.15.78:0/3458872508 shutdown_connections 2024-08-18T23:43:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:04.136+0000 7f8a2ff66640 1 --2- 172.21.15.78:0/3458872508 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8a140a2c90 0x7f8a140a5150 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:04.136+0000 7f8a2ff66640 1 --2- 172.21.15.78:0/3458872508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a281078e0 0x7f8a2819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:04.136+0000 7f8a2ff66640 1 --2- 172.21.15.78:0/3458872508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a28102280 0x7f8a28198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:04.136+0000 7f8a2ff66640 1 --2- 172.21.15.78:0/3458872508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a281018e0 0x7f8a28198200 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:04.136+0000 7f8a2ff66640 1 -- 172.21.15.78:0/3458872508 >> 172.21.15.78:0/3458872508 conn(0x7f8a280fd600 msgr2=0x7f8a280fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:04.137+0000 7f8a2ff66640 1 -- 172.21.15.78:0/3458872508 shutdown_connections 2024-08-18T23:43:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:04.137+0000 7f8a2ff66640 1 -- 172.21.15.78:0/3458872508 wait complete. 2024-08-18T23:43:04.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.971+0000 7f858cc85640 1 -- 172.21.15.78:0/2701358809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85881007c0 msgr2=0x7f8588100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:04.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.971+0000 7f858cc85640 1 --2- 172.21.15.78:0/2701358809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85881007c0 0x7f8588100c20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f857c009930 tx=0x7f857c02a420 comp rx=0 tx=0).stop 2024-08-18T23:43:04.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.971+0000 7f858cc85640 1 -- 172.21.15.78:0/2701358809 shutdown_connections 2024-08-18T23:43:04.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.971+0000 7f858cc85640 1 --2- 172.21.15.78:0/2701358809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8588101160 0x7f858810d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:04.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.971+0000 7f858cc85640 1 --2- 172.21.15.78:0/2701358809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85881007c0 0x7f8588100c20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:04.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.971+0000 7f858cc85640 1 --2- 172.21.15.78:0/2701358809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85881067c0 0x7f8588106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:04.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.971+0000 7f858cc85640 1 -- 172.21.15.78:0/2701358809 >> 172.21.15.78:0/2701358809 conn(0x7f85880fc500 msgr2=0x7f85880fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:04.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.971+0000 7f858cc85640 1 -- 172.21.15.78:0/2701358809 shutdown_connections 2024-08-18T23:43:04.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.971+0000 7f858cc85640 1 -- 172.21.15.78:0/2701358809 wait complete. 2024-08-18T23:43:04.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.972+0000 7f858cc85640 1 Processor -- start 2024-08-18T23:43:04.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.972+0000 7f858cc85640 1 -- start start 2024-08-18T23:43:04.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.972+0000 7f858cc85640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85881007c0 0x7f8588197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:04.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.972+0000 7f858cc85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8588101160 0x7f8588197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:04.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.972+0000 7f858cc85640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85881067c0 0x7f858819b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:04.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.972+0000 7f858cc85640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8588110bc0 con 0x7f8588101160 2024-08-18T23:43:04.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.972+0000 7f858cc85640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8588110a40 con 0x7f85881007c0 2024-08-18T23:43:04.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.972+0000 7f858cc85640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8588110d40 con 0x7f85881067c0 2024-08-18T23:43:04.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.972+0000 7f8585d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8588101160 0x7f8588197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:04.437 INFO:teuthology.orchestra.run.smithi078.stdout:326417514527 2024-08-18T23:43:04.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.972+0000 7f8585d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8588101160 0x7f8588197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51224/0 (socket says 172.21.15.78:51224) 2024-08-18T23:43:04.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.972+0000 7f8585d9b640 1 -- 172.21.15.78:0/4186731916 learned_addr learned my addr 172.21.15.78:0/4186731916 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:04.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.972+0000 7f858659c640 1 --2- 172.21.15.78:0/4186731916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85881007c0 0x7f8588197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:04.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.972+0000 7f8585d9b640 1 -- 172.21.15.78:0/4186731916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85881067c0 msgr2=0x7f858819b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:04.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.972+0000 7f8585d9b640 1 --2- 172.21.15.78:0/4186731916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85881067c0 0x7f858819b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:04.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.972+0000 7f8585d9b640 1 -- 172.21.15.78:0/4186731916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85881007c0 msgr2=0x7f8588197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:04.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.972+0000 7f8585d9b640 1 --2- 172.21.15.78:0/4186731916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85881007c0 0x7f8588197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:04.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.972+0000 7f8585d9b640 1 -- 172.21.15.78:0/4186731916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858819bff0 con 0x7f8588101160 2024-08-18T23:43:04.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.973+0000 7f8585d9b640 1 --2- 172.21.15.78:0/4186731916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8588101160 0x7f8588197580 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f857c009a50 tx=0x7f857c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:04.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.973+0000 7f85777fe640 1 -- 172.21.15.78:0/4186731916 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c03f030 con 0x7f8588101160 2024-08-18T23:43:04.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.973+0000 7f858cc85640 1 -- 172.21.15.78:0/4186731916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f858819c280 con 0x7f8588101160 2024-08-18T23:43:04.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.973+0000 7f85777fe640 1 -- 172.21.15.78:0/4186731916 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f857c003ce0 con 0x7f8588101160 2024-08-18T23:43:04.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.973+0000 7f85777fe640 1 -- 172.21.15.78:0/4186731916 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c035070 con 0x7f8588101160 2024-08-18T23:43:04.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.973+0000 7f858cc85640 1 -- 172.21.15.78:0/4186731916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85881a3bc0 con 0x7f8588101160 2024-08-18T23:43:04.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.974+0000 7f858cc85640 1 -- 172.21.15.78:0/4186731916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8588101f00 con 0x7f8588101160 2024-08-18T23:43:04.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.975+0000 7f85777fe640 1 -- 172.21.15.78:0/4186731916 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f857c042030 con 0x7f8588101160 2024-08-18T23:43:04.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.975+0000 7f85777fe640 1 --2- 172.21.15.78:0/4186731916 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f85700a2c10 0x7f85700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:04.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.976+0000 7f858659c640 1 --2- 172.21.15.78:0/4186731916 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f85700a2c10 0x7f85700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:04.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.976+0000 7f85777fe640 1 -- 172.21.15.78:0/4186731916 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f857c0efb80 con 0x7f8588101160 2024-08-18T23:43:04.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.976+0000 7f858659c640 1 --2- 172.21.15.78:0/4186731916 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f85700a2c10 0x7f85700a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f85780058c0 tx=0x7f8578005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:04.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:01.980+0000 7f85777fe640 1 -- 172.21.15.78:0/4186731916 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f857c0bc620 con 0x7f8588101160 2024-08-18T23:43:04.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.160+0000 7f858cc85640 1 -- 172.21.15.78:0/4186731916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f8588108d60 con 0x7f8588101160 2024-08-18T23:43:04.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.160+0000 7f85777fe640 1 -- 172.21.15.78:0/4186731916 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f857c0bbf20 con 0x7f8588101160 2024-08-18T23:43:04.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.162+0000 7f858cc85640 1 -- 172.21.15.78:0/4186731916 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f85700a2c10 msgr2=0x7f85700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:04.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.162+0000 7f858cc85640 1 --2- 172.21.15.78:0/4186731916 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f85700a2c10 0x7f85700a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f85780058c0 tx=0x7f8578005be0 comp rx=0 tx=0).stop 2024-08-18T23:43:04.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.162+0000 7f858cc85640 1 -- 172.21.15.78:0/4186731916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8588101160 msgr2=0x7f8588197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:04.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.162+0000 7f858cc85640 1 --2- 172.21.15.78:0/4186731916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8588101160 0x7f8588197580 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f857c009a50 tx=0x7f857c004270 comp rx=0 tx=0).stop 2024-08-18T23:43:04.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.162+0000 7f858cc85640 1 -- 172.21.15.78:0/4186731916 shutdown_connections 2024-08-18T23:43:04.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.162+0000 7f858cc85640 1 --2- 172.21.15.78:0/4186731916 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f85700a2c10 0x7f85700a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:04.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.162+0000 7f858cc85640 1 --2- 172.21.15.78:0/4186731916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85881067c0 0x7f858819b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:04.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.162+0000 7f858cc85640 1 --2- 172.21.15.78:0/4186731916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8588101160 0x7f8588197580 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:04.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.162+0000 7f858cc85640 1 --2- 172.21.15.78:0/4186731916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85881007c0 0x7f8588197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:04.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.162+0000 7f858cc85640 1 -- 172.21.15.78:0/4186731916 >> 172.21.15.78:0/4186731916 conn(0x7f85880fc500 msgr2=0x7f85880fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:04.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.162+0000 7f858cc85640 1 -- 172.21.15.78:0/4186731916 shutdown_connections 2024-08-18T23:43:04.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.162+0000 7f858cc85640 1 -- 172.21.15.78:0/4186731916 wait complete. 2024-08-18T23:43:04.440 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:04 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1655007148' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-08-18T23:43:04.440 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:04 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3458872508' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-08-18T23:43:04.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:04 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1655007148' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-08-18T23:43:04.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:04 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3458872508' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-08-18T23:43:04.644 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:04 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1655007148' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-08-18T23:43:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:04 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3458872508' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-08-18T23:43:05.115 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514526 got 326417514527 for osd.11 2024-08-18T23:43:05.115 DEBUG:teuthology.parallel:result is None 2024-08-18T23:43:05.352 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:05 smithi078 ceph-mon[18440]: pgmap v311: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-18T23:43:05.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:05 smithi123 ceph-mon[21808]: pgmap v311: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-18T23:43:05.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:05 smithi161 ceph-mon[22027]: pgmap v311: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-18T23:43:05.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.637+0000 7f86e96c6640 1 -- 172.21.15.78:0/983115279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86e4102250 msgr2=0x7f86e410e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:05.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.637+0000 7f86e96c6640 1 --2- 172.21.15.78:0/983115279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86e4102250 0x7f86e410e780 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009cd0 tx=0x7f86dc02a8e0 comp rx=0 tx=0).stop 2024-08-18T23:43:05.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.637+0000 7f86e96c6640 1 -- 172.21.15.78:0/983115279 shutdown_connections 2024-08-18T23:43:05.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.637+0000 7f86e96c6640 1 --2- 172.21.15.78:0/983115279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86e4102250 0x7f86e410e780 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:05.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.637+0000 7f86e96c6640 1 --2- 172.21.15.78:0/983115279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86e41018b0 0x7f86e4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:05.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.637+0000 7f86e96c6640 1 --2- 172.21.15.78:0/983115279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86e41078b0 0x7f86e4107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:05.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.637+0000 7f86e96c6640 1 -- 172.21.15.78:0/983115279 >> 172.21.15.78:0/983115279 conn(0x7f86e40fd5f0 msgr2=0x7f86e40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:05.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.637+0000 7f86e96c6640 1 -- 172.21.15.78:0/983115279 shutdown_connections 2024-08-18T23:43:05.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.637+0000 7f86e96c6640 1 -- 172.21.15.78:0/983115279 wait complete. 2024-08-18T23:43:05.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.638+0000 7f86e96c6640 1 Processor -- start 2024-08-18T23:43:05.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.638+0000 7f86e96c6640 1 -- start start 2024-08-18T23:43:05.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.638+0000 7f86e96c6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86e41018b0 0x7f86e4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:05.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.638+0000 7f86e96c6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86e4102250 0x7f86e41986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:05.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.638+0000 7f86e96c6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86e41078b0 0x7f86e419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:05.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.638+0000 7f86e96c6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f86e4111cb0 con 0x7f86e4102250 2024-08-18T23:43:05.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.638+0000 7f86e96c6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f86e4111b30 con 0x7f86e41018b0 2024-08-18T23:43:05.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.638+0000 7f86e96c6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f86e4111e30 con 0x7f86e41078b0 2024-08-18T23:43:05.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.639+0000 7f86e27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86e4102250 0x7f86e41986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:05.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.639+0000 7f86e27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86e4102250 0x7f86e41986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51300/0 (socket says 172.21.15.78:51300) 2024-08-18T23:43:05.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.639+0000 7f86e27fc640 1 -- 172.21.15.78:0/2884515262 learned_addr learned my addr 172.21.15.78:0/2884515262 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:05.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.639+0000 7f86e37fe640 1 --2- 172.21.15.78:0/2884515262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86e41078b0 0x7f86e419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:05.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.639+0000 7f86e2ffd640 1 --2- 172.21.15.78:0/2884515262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86e41018b0 0x7f86e4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:05.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.639+0000 7f86e27fc640 1 -- 172.21.15.78:0/2884515262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86e41078b0 msgr2=0x7f86e419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:05.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.639+0000 7f86e27fc640 1 --2- 172.21.15.78:0/2884515262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86e41078b0 0x7f86e419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:05.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.639+0000 7f86e27fc640 1 -- 172.21.15.78:0/2884515262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86e41018b0 msgr2=0x7f86e4198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:05.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.639+0000 7f86e27fc640 1 --2- 172.21.15.78:0/2884515262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86e41018b0 0x7f86e4198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:05.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.639+0000 7f86e27fc640 1 -- 172.21.15.78:0/2884515262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e419d120 con 0x7f86e4102250 2024-08-18T23:43:05.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.639+0000 7f86e27fc640 1 --2- 172.21.15.78:0/2884515262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86e4102250 0x7f86e41986b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f86d400d770 tx=0x7f86d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:05.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.639+0000 7f86dbfff640 1 -- 172.21.15.78:0/2884515262 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4017070 con 0x7f86e4102250 2024-08-18T23:43:05.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.639+0000 7f86dbfff640 1 -- 172.21.15.78:0/2884515262 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d4005500 con 0x7f86e4102250 2024-08-18T23:43:05.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.639+0000 7f86e96c6640 1 -- 172.21.15.78:0/2884515262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e419d410 con 0x7f86e4102250 2024-08-18T23:43:05.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.639+0000 7f86dbfff640 1 -- 172.21.15.78:0/2884515262 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4004e00 con 0x7f86e4102250 2024-08-18T23:43:05.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.639+0000 7f86e96c6640 1 -- 172.21.15.78:0/2884515262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e41a4cf0 con 0x7f86e4102250 2024-08-18T23:43:05.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.640+0000 7f86e96c6640 1 -- 172.21.15.78:0/2884515262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e4101d10 con 0x7f86e4102250 2024-08-18T23:43:05.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.642+0000 7f86dbfff640 1 -- 172.21.15.78:0/2884515262 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f86d4004fa0 con 0x7f86e4102250 2024-08-18T23:43:05.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.642+0000 7f86dbfff640 1 --2- 172.21.15.78:0/2884515262 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f86d00a2c10 0x7f86d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:05.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.642+0000 7f86e2ffd640 1 --2- 172.21.15.78:0/2884515262 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f86d00a2c10 0x7f86d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:05.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.642+0000 7f86dbfff640 1 -- 172.21.15.78:0/2884515262 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f86d40cf390 con 0x7f86e4102250 2024-08-18T23:43:05.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.643+0000 7f86e2ffd640 1 --2- 172.21.15.78:0/2884515262 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f86d00a2c10 0x7f86d00a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0058c0 tx=0x7f86cc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:05.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.646+0000 7f86dbfff640 1 -- 172.21.15.78:0/2884515262 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f86d409b780 con 0x7f86e4102250 2024-08-18T23:43:05.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.815+0000 7f86e96c6640 1 -- 172.21.15.78:0/2884515262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f86e41994d0 con 0x7f86e4102250 2024-08-18T23:43:05.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.816+0000 7f86dbfff640 1 -- 172.21.15.78:0/2884515262 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f86d409fa80 con 0x7f86e4102250 2024-08-18T23:43:05.817 INFO:teuthology.orchestra.run.smithi078.stdout:107374182467 2024-08-18T23:43:05.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.818+0000 7f86e96c6640 1 -- 172.21.15.78:0/2884515262 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f86d00a2c10 msgr2=0x7f86d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:05.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.818+0000 7f86e96c6640 1 --2- 172.21.15.78:0/2884515262 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f86d00a2c10 0x7f86d00a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0058c0 tx=0x7f86cc00a2d0 comp rx=0 tx=0).stop 2024-08-18T23:43:05.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.818+0000 7f86e96c6640 1 -- 172.21.15.78:0/2884515262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86e4102250 msgr2=0x7f86e41986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:05.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.818+0000 7f86e96c6640 1 --2- 172.21.15.78:0/2884515262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86e4102250 0x7f86e41986b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f86d400d770 tx=0x7f86d400dc30 comp rx=0 tx=0).stop 2024-08-18T23:43:05.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.818+0000 7f86e96c6640 1 -- 172.21.15.78:0/2884515262 shutdown_connections 2024-08-18T23:43:05.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.818+0000 7f86e96c6640 1 --2- 172.21.15.78:0/2884515262 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f86d00a2c10 0x7f86d00a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:05.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.818+0000 7f86e96c6640 1 --2- 172.21.15.78:0/2884515262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86e41078b0 0x7f86e419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:05.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.818+0000 7f86e96c6640 1 --2- 172.21.15.78:0/2884515262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86e4102250 0x7f86e41986b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:05.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.818+0000 7f86e96c6640 1 --2- 172.21.15.78:0/2884515262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86e41018b0 0x7f86e4198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:05.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.818+0000 7f86e96c6640 1 -- 172.21.15.78:0/2884515262 >> 172.21.15.78:0/2884515262 conn(0x7f86e40fd5f0 msgr2=0x7f86e40fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:05.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.818+0000 7f86e96c6640 1 -- 172.21.15.78:0/2884515262 shutdown_connections 2024-08-18T23:43:05.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:05.818+0000 7f86e96c6640 1 -- 172.21.15.78:0/2884515262 wait complete. 2024-08-18T23:43:06.366 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397551 got 210453397553 for osd.7 2024-08-18T23:43:06.367 DEBUG:teuthology.parallel:result is None 2024-08-18T23:43:06.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:06 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2884515262' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-08-18T23:43:06.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:06 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2884515262' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-08-18T23:43:06.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:06 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2884515262' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-08-18T23:43:06.921 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561075 got 188978561078 for osd.6 2024-08-18T23:43:06.921 DEBUG:teuthology.parallel:result is None 2024-08-18T23:43:07.376 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168619 got 240518168621 for osd.8 2024-08-18T23:43:07.376 DEBUG:teuthology.parallel:result is None 2024-08-18T23:43:07.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:07 smithi078 ceph-mon[18440]: pgmap v312: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-18T23:43:07.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:07 smithi123 ceph-mon[21808]: pgmap v312: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-18T23:43:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:07 smithi161 ceph-mon[22027]: pgmap v312: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-18T23:43:07.836 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182465 got 107374182467 for osd.3 2024-08-18T23:43:07.837 DEBUG:teuthology.parallel:result is None 2024-08-18T23:43:08.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.970+0000 7f991c172640 1 -- 172.21.15.78:0/1594236175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9914102260 msgr2=0x7f991410e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:08.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.970+0000 7f991c172640 1 --2- 172.21.15.78:0/1594236175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9914102260 0x7f991410e790 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f990c009930 tx=0x7f990c02a420 comp rx=0 tx=0).stop 2024-08-18T23:43:08.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.970+0000 7f991c172640 1 -- 172.21.15.78:0/1594236175 shutdown_connections 2024-08-18T23:43:08.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.970+0000 7f991c172640 1 --2- 172.21.15.78:0/1594236175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9914102260 0x7f991410e790 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:08.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.970+0000 7f991c172640 1 --2- 172.21.15.78:0/1594236175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99141018c0 0x7f9914101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:08.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.970+0000 7f991c172640 1 --2- 172.21.15.78:0/1594236175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99141078c0 0x7f9914107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:08.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.970+0000 7f991c172640 1 -- 172.21.15.78:0/1594236175 >> 172.21.15.78:0/1594236175 conn(0x7f99140fd620 msgr2=0x7f99140ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:08.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.970+0000 7f991c172640 1 -- 172.21.15.78:0/1594236175 shutdown_connections 2024-08-18T23:43:08.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.970+0000 7f991c172640 1 -- 172.21.15.78:0/1594236175 wait complete. 2024-08-18T23:43:08.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.971+0000 7f991c172640 1 Processor -- start 2024-08-18T23:43:08.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.971+0000 7f991c172640 1 -- start start 2024-08-18T23:43:08.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.971+0000 7f991c172640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99141018c0 0x7f9914074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:08.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.971+0000 7f991c172640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9914102260 0x7f9914074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:08.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.971+0000 7f991c172640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99141078c0 0x7f9914074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:08.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.971+0000 7f991c172640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9914111ce0 con 0x7f9914102260 2024-08-18T23:43:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.971+0000 7f991c172640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9914111b60 con 0x7f99141078c0 2024-08-18T23:43:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.971+0000 7f991c172640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9914111e60 con 0x7f99141018c0 2024-08-18T23:43:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.971+0000 7f991970d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9914102260 0x7f9914074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.971+0000 7f991970d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9914102260 0x7f9914074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51260/0 (socket says 172.21.15.78:51260) 2024-08-18T23:43:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.971+0000 7f991970d640 1 -- 172.21.15.78:0/283941245 learned_addr learned my addr 172.21.15.78:0/283941245 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.971+0000 7f991a70f640 1 --2- 172.21.15.78:0/283941245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99141078c0 0x7f9914074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.971+0000 7f991970d640 1 -- 172.21.15.78:0/283941245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99141018c0 msgr2=0x7f9914074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.971+0000 7f991970d640 1 --2- 172.21.15.78:0/283941245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99141018c0 0x7f9914074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.971+0000 7f991970d640 1 -- 172.21.15.78:0/283941245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99141078c0 msgr2=0x7f9914074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.971+0000 7f991970d640 1 --2- 172.21.15.78:0/283941245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99141078c0 0x7f9914074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.971+0000 7f991970d640 1 -- 172.21.15.78:0/283941245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99141a6620 con 0x7f9914102260 2024-08-18T23:43:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.971+0000 7f991970d640 1 --2- 172.21.15.78:0/283941245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9914102260 0x7f9914074760 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f990400c920 tx=0x7f990400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:08.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.972+0000 7f990affd640 1 -- 172.21.15.78:0/283941245 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904007c70 con 0x7f9914102260 2024-08-18T23:43:08.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.972+0000 7f990affd640 1 -- 172.21.15.78:0/283941245 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9904007e10 con 0x7f9914102260 2024-08-18T23:43:08.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.972+0000 7f991c172640 1 -- 172.21.15.78:0/283941245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99141a6910 con 0x7f9914102260 2024-08-18T23:43:08.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.972+0000 7f990affd640 1 -- 172.21.15.78:0/283941245 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904018890 con 0x7f9914102260 2024-08-18T23:43:08.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.972+0000 7f991c172640 1 -- 172.21.15.78:0/283941245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99141a6e50 con 0x7f9914102260 2024-08-18T23:43:08.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.973+0000 7f991c172640 1 -- 172.21.15.78:0/283941245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9914101d20 con 0x7f9914102260 2024-08-18T23:43:08.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.974+0000 7f990affd640 1 -- 172.21.15.78:0/283941245 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9904018a50 con 0x7f9914102260 2024-08-18T23:43:08.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.974+0000 7f990affd640 1 --2- 172.21.15.78:0/283941245 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f99000a2c10 0x7f99000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:08.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.975+0000 7f9919f0e640 1 --2- 172.21.15.78:0/283941245 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f99000a2c10 0x7f99000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:08.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.975+0000 7f990affd640 1 -- 172.21.15.78:0/283941245 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f99040cf9d0 con 0x7f9914102260 2024-08-18T23:43:08.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.975+0000 7f9919f0e640 1 --2- 172.21.15.78:0/283941245 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f99000a2c10 0x7f99000a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f991000ab90 tx=0x7f9910009250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:08.137 INFO:teuthology.orchestra.run.smithi078.stdout:137438953535 2024-08-18T23:43:08.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:02.978+0000 7f990affd640 1 -- 172.21.15.78:0/283941245 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9904014030 con 0x7f9914102260 2024-08-18T23:43:08.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.148+0000 7f991c172640 1 -- 172.21.15.78:0/283941245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f9914075750 con 0x7f9914102260 2024-08-18T23:43:08.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.148+0000 7f990affd640 1 -- 172.21.15.78:0/283941245 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9914075750 con 0x7f9914102260 2024-08-18T23:43:08.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.149+0000 7f991c172640 1 -- 172.21.15.78:0/283941245 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f99000a2c10 msgr2=0x7f99000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:08.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.149+0000 7f991c172640 1 --2- 172.21.15.78:0/283941245 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f99000a2c10 0x7f99000a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f991000ab90 tx=0x7f9910009250 comp rx=0 tx=0).stop 2024-08-18T23:43:08.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.149+0000 7f991c172640 1 -- 172.21.15.78:0/283941245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9914102260 msgr2=0x7f9914074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:08.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.149+0000 7f991c172640 1 --2- 172.21.15.78:0/283941245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9914102260 0x7f9914074760 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f990400c920 tx=0x7f990400cde0 comp rx=0 tx=0).stop 2024-08-18T23:43:08.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.149+0000 7f991c172640 1 -- 172.21.15.78:0/283941245 shutdown_connections 2024-08-18T23:43:08.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.149+0000 7f991c172640 1 --2- 172.21.15.78:0/283941245 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f99000a2c10 0x7f99000a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:08.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.149+0000 7f991c172640 1 --2- 172.21.15.78:0/283941245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99141078c0 0x7f9914074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:08.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.149+0000 7f991c172640 1 --2- 172.21.15.78:0/283941245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9914102260 0x7f9914074760 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:08.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.149+0000 7f991c172640 1 --2- 172.21.15.78:0/283941245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99141018c0 0x7f9914074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:08.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.149+0000 7f991c172640 1 -- 172.21.15.78:0/283941245 >> 172.21.15.78:0/283941245 conn(0x7f99140fd620 msgr2=0x7f99140fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:08.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.150+0000 7f991c172640 1 -- 172.21.15.78:0/283941245 shutdown_connections 2024-08-18T23:43:08.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:03.150+0000 7f991c172640 1 -- 172.21.15.78:0/283941245 wait complete. 2024-08-18T23:43:08.813 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953533 got 137438953535 for osd.4 2024-08-18T23:43:08.813 DEBUG:teuthology.parallel:result is None 2024-08-18T23:43:08.837 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:08.837 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:09.331 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939685 got 270582939688 for osd.9 2024-08-18T23:43:09.331 DEBUG:teuthology.parallel:result is None 2024-08-18T23:43:09.404 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:09.493 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:09 smithi078 ceph-mon[18440]: pgmap v313: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:09.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:09 smithi123 ceph-mon[21808]: pgmap v313: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:09 smithi161 ceph-mon[22027]: pgmap v313: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:09.884 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411400 for osd.2 2024-08-18T23:43:09.884 DEBUG:teuthology.parallel:result is None 2024-08-18T23:43:10.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.544+0000 7fe99953e640 1 -- 172.21.15.78:0/2263100774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9941007b0 msgr2=0x7fe994100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:10.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.544+0000 7fe99953e640 1 --2- 172.21.15.78:0/2263100774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9941007b0 0x7fe994100c10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe988009930 tx=0x7fe98802a420 comp rx=0 tx=0).stop 2024-08-18T23:43:10.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.544+0000 7fe99953e640 1 -- 172.21.15.78:0/2263100774 shutdown_connections 2024-08-18T23:43:10.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.544+0000 7fe99953e640 1 --2- 172.21.15.78:0/2263100774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe994101150 0x7fe99410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:10.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.544+0000 7fe99953e640 1 --2- 172.21.15.78:0/2263100774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9941007b0 0x7fe994100c10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.544+0000 7fe99953e640 1 --2- 172.21.15.78:0/2263100774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9941067b0 0x7fe994106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.544+0000 7fe99953e640 1 -- 172.21.15.78:0/2263100774 >> 172.21.15.78:0/2263100774 conn(0x7fe9940fc4d0 msgr2=0x7fe9940fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.544+0000 7fe99953e640 1 -- 172.21.15.78:0/2263100774 shutdown_connections 2024-08-18T23:43:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.544+0000 7fe99953e640 1 -- 172.21.15.78:0/2263100774 wait complete. 2024-08-18T23:43:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.545+0000 7fe99953e640 1 Processor -- start 2024-08-18T23:43:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.545+0000 7fe99953e640 1 -- start start 2024-08-18T23:43:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.545+0000 7fe99953e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9941007b0 0x7fe994199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.545+0000 7fe99953e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe994101150 0x7fe994199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.545+0000 7fe99953e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9941067b0 0x7fe9941a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.545+0000 7fe99953e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe994110b70 con 0x7fe9941067b0 2024-08-18T23:43:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.545+0000 7fe99953e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941109f0 con 0x7fe9941007b0 2024-08-18T23:43:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.545+0000 7fe99953e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe994110cf0 con 0x7fe994101150 2024-08-18T23:43:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.545+0000 7fe9937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9941067b0 0x7fe9941a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:10.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.546+0000 7fe9937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9941067b0 0x7fe9941a40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59156/0 (socket says 172.21.15.78:59156) 2024-08-18T23:43:10.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.546+0000 7fe9937fe640 1 -- 172.21.15.78:0/2915881987 learned_addr learned my addr 172.21.15.78:0/2915881987 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:10.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.546+0000 7fe9927fc640 1 --2- 172.21.15.78:0/2915881987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe994101150 0x7fe994199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:10.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.546+0000 7fe992ffd640 1 --2- 172.21.15.78:0/2915881987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9941007b0 0x7fe994199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:10.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.546+0000 7fe9937fe640 1 -- 172.21.15.78:0/2915881987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe994101150 msgr2=0x7fe994199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:10.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.546+0000 7fe9937fe640 1 --2- 172.21.15.78:0/2915881987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe994101150 0x7fe994199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:10.829 INFO:teuthology.orchestra.run.smithi078.stdout:292057776166 2024-08-18T23:43:10.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.546+0000 7fe9937fe640 1 -- 172.21.15.78:0/2915881987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9941007b0 msgr2=0x7fe994199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:10.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.546+0000 7fe9937fe640 1 --2- 172.21.15.78:0/2915881987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9941007b0 0x7fe994199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:10.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.546+0000 7fe9937fe640 1 -- 172.21.15.78:0/2915881987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9941a45e0 con 0x7fe9941067b0 2024-08-18T23:43:10.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.546+0000 7fe9937fe640 1 --2- 172.21.15.78:0/2915881987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9941067b0 0x7fe9941a40a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00cad0 tx=0x7fe98c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:10.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.546+0000 7fe97bfff640 1 -- 172.21.15.78:0/2915881987 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c00ae20 con 0x7fe9941067b0 2024-08-18T23:43:10.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.546+0000 7fe97bfff640 1 -- 172.21.15.78:0/2915881987 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c005020 con 0x7fe9941067b0 2024-08-18T23:43:10.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.546+0000 7fe97bfff640 1 -- 172.21.15.78:0/2915881987 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c00f6f0 con 0x7fe9941067b0 2024-08-18T23:43:10.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.546+0000 7fe99953e640 1 -- 172.21.15.78:0/2915881987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9941a47b0 con 0x7fe9941067b0 2024-08-18T23:43:10.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.546+0000 7fe99953e640 1 -- 172.21.15.78:0/2915881987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9941a4c70 con 0x7fe9941067b0 2024-08-18T23:43:10.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.547+0000 7fe99953e640 1 -- 172.21.15.78:0/2915881987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe994100c10 con 0x7fe9941067b0 2024-08-18T23:43:10.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.548+0000 7fe97bfff640 1 -- 172.21.15.78:0/2915881987 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe98c0051d0 con 0x7fe9941067b0 2024-08-18T23:43:10.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.549+0000 7fe97bfff640 1 --2- 172.21.15.78:0/2915881987 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe97c0a2c90 0x7fe97c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:10.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.549+0000 7fe992ffd640 1 --2- 172.21.15.78:0/2915881987 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe97c0a2c90 0x7fe97c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:10.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.549+0000 7fe97bfff640 1 -- 172.21.15.78:0/2915881987 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fe98c0cf490 con 0x7fe9941067b0 2024-08-18T23:43:10.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.550+0000 7fe992ffd640 1 --2- 172.21.15.78:0/2915881987 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe97c0a2c90 0x7fe97c0a5150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe9840058c0 tx=0x7fe98400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:10.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.553+0000 7fe97bfff640 1 -- 172.21.15.78:0/2915881987 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe98c014030 con 0x7fe9941067b0 2024-08-18T23:43:10.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.727+0000 7fe99953e640 1 -- 172.21.15.78:0/2915881987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fe994108d40 con 0x7fe9941067b0 2024-08-18T23:43:10.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.727+0000 7fe97bfff640 1 -- 172.21.15.78:0/2915881987 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fe98c09b8e0 con 0x7fe9941067b0 2024-08-18T23:43:10.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.729+0000 7fe99953e640 1 -- 172.21.15.78:0/2915881987 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe97c0a2c90 msgr2=0x7fe97c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:10.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.729+0000 7fe99953e640 1 --2- 172.21.15.78:0/2915881987 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe97c0a2c90 0x7fe97c0a5150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe9840058c0 tx=0x7fe98400a2d0 comp rx=0 tx=0).stop 2024-08-18T23:43:10.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.729+0000 7fe99953e640 1 -- 172.21.15.78:0/2915881987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9941067b0 msgr2=0x7fe9941a40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:10.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.729+0000 7fe99953e640 1 --2- 172.21.15.78:0/2915881987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9941067b0 0x7fe9941a40a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00cad0 tx=0x7fe98c00cf90 comp rx=0 tx=0).stop 2024-08-18T23:43:10.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.729+0000 7fe99953e640 1 -- 172.21.15.78:0/2915881987 shutdown_connections 2024-08-18T23:43:10.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.729+0000 7fe99953e640 1 --2- 172.21.15.78:0/2915881987 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe97c0a2c90 0x7fe97c0a5150 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:10.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.729+0000 7fe99953e640 1 --2- 172.21.15.78:0/2915881987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9941067b0 0x7fe9941a40a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:10.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.729+0000 7fe99953e640 1 --2- 172.21.15.78:0/2915881987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe994101150 0x7fe994199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:10.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.729+0000 7fe99953e640 1 --2- 172.21.15.78:0/2915881987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9941007b0 0x7fe994199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:10.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.729+0000 7fe99953e640 1 -- 172.21.15.78:0/2915881987 >> 172.21.15.78:0/2915881987 conn(0x7fe9940fc4d0 msgr2=0x7fe9940fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:10.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.730+0000 7fe99953e640 1 -- 172.21.15.78:0/2915881987 shutdown_connections 2024-08-18T23:43:10.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:10.730+0000 7fe99953e640 1 -- 172.21.15.78:0/2915881987 wait complete. 2024-08-18T23:43:11.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.223+0000 7fc1c0b3d640 1 -- 172.21.15.78:0/4117857257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1bc1022a0 msgr2=0x7fc1bc1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:11.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.223+0000 7fc1c0b3d640 1 --2- 172.21.15.78:0/4117857257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1bc1022a0 0x7fc1bc1026e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac009ca0 tx=0x7fc1ac02a950 comp rx=0 tx=0).stop 2024-08-18T23:43:11.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.224+0000 7fc1c0b3d640 1 -- 172.21.15.78:0/4117857257 shutdown_connections 2024-08-18T23:43:11.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.224+0000 7fc1c0b3d640 1 --2- 172.21.15.78:0/4117857257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1bc102c20 0x7fc1bc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:11.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.224+0000 7fc1c0b3d640 1 --2- 172.21.15.78:0/4117857257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1bc1022a0 0x7fc1bc1026e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:11.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.224+0000 7fc1c0b3d640 1 --2- 172.21.15.78:0/4117857257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1bc100b50 0x7fc1bc100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:11.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.224+0000 7fc1c0b3d640 1 -- 172.21.15.78:0/4117857257 >> 172.21.15.78:0/4117857257 conn(0x7fc1bc0fc500 msgr2=0x7fc1bc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:11.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.224+0000 7fc1c0b3d640 1 -- 172.21.15.78:0/4117857257 shutdown_connections 2024-08-18T23:43:11.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.224+0000 7fc1c0b3d640 1 -- 172.21.15.78:0/4117857257 wait complete. 2024-08-18T23:43:11.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.224+0000 7fc1c0b3d640 1 Processor -- start 2024-08-18T23:43:11.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.224+0000 7fc1c0b3d640 1 -- start start 2024-08-18T23:43:11.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1c0b3d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1bc100b50 0x7fc1bc1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:11.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1c0b3d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1bc1022a0 0x7fc1bc1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:11.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1c0b3d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1bc102c20 0x7fc1bc19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:11.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1c0b3d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc110cf0 con 0x7fc1bc100b50 2024-08-18T23:43:11.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1c0b3d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc110b70 con 0x7fc1bc102c20 2024-08-18T23:43:11.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1c0b3d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc110e70 con 0x7fc1bc1022a0 2024-08-18T23:43:11.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1ba59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1bc100b50 0x7fc1bc1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:11.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1ba59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1bc100b50 0x7fc1bc1982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59176/0 (socket says 172.21.15.78:59176) 2024-08-18T23:43:11.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1ba59c640 1 -- 172.21.15.78:0/2971683943 learned_addr learned my addr 172.21.15.78:0/2971683943 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:11.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1b9d9b640 1 --2- 172.21.15.78:0/2971683943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1bc1022a0 0x7fc1bc1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:11.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1bad9d640 1 --2- 172.21.15.78:0/2971683943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1bc102c20 0x7fc1bc19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:11.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1ba59c640 1 -- 172.21.15.78:0/2971683943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1bc1022a0 msgr2=0x7fc1bc1987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:11.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1ba59c640 1 --2- 172.21.15.78:0/2971683943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1bc1022a0 0x7fc1bc1987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:11.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1ba59c640 1 -- 172.21.15.78:0/2971683943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1bc102c20 msgr2=0x7fc1bc19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:11.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1ba59c640 1 --2- 172.21.15.78:0/2971683943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1bc102c20 0x7fc1bc19cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:11.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1ba59c640 1 -- 172.21.15.78:0/2971683943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1bc19d260 con 0x7fc1bc100b50 2024-08-18T23:43:11.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1bad9d640 1 --2- 172.21.15.78:0/2971683943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1bc102c20 0x7fc1bc19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:43:11.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1ba59c640 1 --2- 172.21.15.78:0/2971683943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1bc100b50 0x7fc1bc1982b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000c970 tx=0x7fc1b000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:11.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1ab7fe640 1 -- 172.21.15.78:0/2971683943 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0007bb0 con 0x7fc1bc100b50 2024-08-18T23:43:11.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1c0b3d640 1 -- 172.21.15.78:0/2971683943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1bc19d550 con 0x7fc1bc100b50 2024-08-18T23:43:11.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1ab7fe640 1 -- 172.21.15.78:0/2971683943 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1b0007d50 con 0x7fc1bc100b50 2024-08-18T23:43:11.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1ab7fe640 1 -- 172.21.15.78:0/2971683943 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b000f450 con 0x7fc1bc100b50 2024-08-18T23:43:11.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.225+0000 7fc1c0b3d640 1 -- 172.21.15.78:0/2971683943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1bc1a4e30 con 0x7fc1bc100b50 2024-08-18T23:43:11.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.226+0000 7fc1c0b3d640 1 -- 172.21.15.78:0/2971683943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1bc100f30 con 0x7fc1bc100b50 2024-08-18T23:43:11.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.227+0000 7fc1ab7fe640 1 -- 172.21.15.78:0/2971683943 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc1b0005020 con 0x7fc1bc100b50 2024-08-18T23:43:11.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.228+0000 7fc1ab7fe640 1 --2- 172.21.15.78:0/2971683943 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc1a40a2c10 0x7fc1a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:11.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.228+0000 7fc1b9d9b640 1 --2- 172.21.15.78:0/2971683943 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc1a40a2c10 0x7fc1a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:11.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.228+0000 7fc1ab7fe640 1 -- 172.21.15.78:0/2971683943 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc1b00cf330 con 0x7fc1bc100b50 2024-08-18T23:43:11.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.229+0000 7fc1b9d9b640 1 --2- 172.21.15.78:0/2971683943 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc1a40a2c10 0x7fc1a40a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac030040 tx=0x7fc1ac02fe90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:11.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.232+0000 7fc1ab7fe640 1 -- 172.21.15.78:0/2971683943 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc1b0014030 con 0x7fc1bc100b50 2024-08-18T23:43:11.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.403+0000 7fc1c0b3d640 1 -- 172.21.15.78:0/2971683943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc1bc199610 con 0x7fc1bc100b50 2024-08-18T23:43:11.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.403+0000 7fc1ab7fe640 1 -- 172.21.15.78:0/2971683943 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc1b009be70 con 0x7fc1bc100b50 2024-08-18T23:43:11.405 INFO:teuthology.orchestra.run.smithi078.stdout:158913790012 2024-08-18T23:43:11.405 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:11 smithi078 ceph-mon[18440]: pgmap v314: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:11.405 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:11 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2915881987' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-08-18T23:43:11.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.405+0000 7fc1c0b3d640 1 -- 172.21.15.78:0/2971683943 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc1a40a2c10 msgr2=0x7fc1a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:11.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.405+0000 7fc1c0b3d640 1 --2- 172.21.15.78:0/2971683943 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc1a40a2c10 0x7fc1a40a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac030040 tx=0x7fc1ac02fe90 comp rx=0 tx=0).stop 2024-08-18T23:43:11.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.405+0000 7fc1c0b3d640 1 -- 172.21.15.78:0/2971683943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1bc100b50 msgr2=0x7fc1bc1982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:11.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.405+0000 7fc1c0b3d640 1 --2- 172.21.15.78:0/2971683943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1bc100b50 0x7fc1bc1982b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000c970 tx=0x7fc1b000ce30 comp rx=0 tx=0).stop 2024-08-18T23:43:11.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.406+0000 7fc1c0b3d640 1 -- 172.21.15.78:0/2971683943 shutdown_connections 2024-08-18T23:43:11.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.406+0000 7fc1c0b3d640 1 --2- 172.21.15.78:0/2971683943 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc1a40a2c10 0x7fc1a40a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:11.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.406+0000 7fc1c0b3d640 1 --2- 172.21.15.78:0/2971683943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1bc102c20 0x7fc1bc19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:11.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.406+0000 7fc1c0b3d640 1 --2- 172.21.15.78:0/2971683943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1bc1022a0 0x7fc1bc1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:11.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.406+0000 7fc1c0b3d640 1 --2- 172.21.15.78:0/2971683943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1bc100b50 0x7fc1bc1982b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:11.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.406+0000 7fc1c0b3d640 1 -- 172.21.15.78:0/2971683943 >> 172.21.15.78:0/2971683943 conn(0x7fc1bc0fc500 msgr2=0x7fc1bc069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:11.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.406+0000 7fc1c0b3d640 1 -- 172.21.15.78:0/2971683943 shutdown_connections 2024-08-18T23:43:11.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.406+0000 7fc1c0b3d640 1 -- 172.21.15.78:0/2971683943 wait complete. 2024-08-18T23:43:11.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.508+0000 7ff314b13640 1 -- 172.21.15.78:0/967898690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3101018c0 msgr2=0x7ff310101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:11.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.508+0000 7ff314b13640 1 --2- 172.21.15.78:0/967898690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3101018c0 0x7ff310101d20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff304009930 tx=0x7ff30402a420 comp rx=0 tx=0).stop 2024-08-18T23:43:11.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.508+0000 7ff314b13640 1 -- 172.21.15.78:0/967898690 shutdown_connections 2024-08-18T23:43:11.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.508+0000 7ff314b13640 1 --2- 172.21.15.78:0/967898690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff310102260 0x7ff31010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:11.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.508+0000 7ff314b13640 1 --2- 172.21.15.78:0/967898690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3101018c0 0x7ff310101d20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:11.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.508+0000 7ff314b13640 1 --2- 172.21.15.78:0/967898690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3101078c0 0x7ff310107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:11.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.508+0000 7ff314b13640 1 -- 172.21.15.78:0/967898690 >> 172.21.15.78:0/967898690 conn(0x7ff3100fd5e0 msgr2=0x7ff3100ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:11.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.508+0000 7ff314b13640 1 -- 172.21.15.78:0/967898690 shutdown_connections 2024-08-18T23:43:11.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.508+0000 7ff314b13640 1 -- 172.21.15.78:0/967898690 wait complete. 2024-08-18T23:43:11.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.509+0000 7ff314b13640 1 Processor -- start 2024-08-18T23:43:11.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.509+0000 7ff314b13640 1 -- start start 2024-08-18T23:43:11.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.509+0000 7ff314b13640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3101018c0 0x7ff3101982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:11.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.509+0000 7ff314b13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff310102260 0x7ff310198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:11.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.509+0000 7ff314b13640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3101078c0 0x7ff31019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:11.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.509+0000 7ff314b13640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff310111d60 con 0x7ff310102260 2024-08-18T23:43:11.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.509+0000 7ff314b13640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff310111be0 con 0x7ff3101078c0 2024-08-18T23:43:11.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.509+0000 7ff314b13640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff310111ee0 con 0x7ff3101018c0 2024-08-18T23:43:11.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.509+0000 7ff30dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff310102260 0x7ff310198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:11.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.509+0000 7ff30dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff310102260 0x7ff310198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59194/0 (socket says 172.21.15.78:59194) 2024-08-18T23:43:11.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.509+0000 7ff30dd9b640 1 -- 172.21.15.78:0/2363570131 learned_addr learned my addr 172.21.15.78:0/2363570131 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:11.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.509+0000 7ff30ed9d640 1 --2- 172.21.15.78:0/2363570131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3101078c0 0x7ff31019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:11.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.509+0000 7ff30dd9b640 1 -- 172.21.15.78:0/2363570131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3101018c0 msgr2=0x7ff3101982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:11.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.509+0000 7ff30dd9b640 1 --2- 172.21.15.78:0/2363570131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3101018c0 0x7ff3101982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:11.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.509+0000 7ff30dd9b640 1 -- 172.21.15.78:0/2363570131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3101078c0 msgr2=0x7ff31019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:11.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.509+0000 7ff30dd9b640 1 --2- 172.21.15.78:0/2363570131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3101078c0 0x7ff31019cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:11.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.510+0000 7ff30dd9b640 1 -- 172.21.15.78:0/2363570131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31019d290 con 0x7ff310102260 2024-08-18T23:43:11.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.510+0000 7ff30dd9b640 1 --2- 172.21.15.78:0/2363570131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff310102260 0x7ff310198820 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff304009a50 tx=0x7ff304004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:11.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.510+0000 7ff2ff7fe640 1 -- 172.21.15.78:0/2363570131 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30403f030 con 0x7ff310102260 2024-08-18T23:43:11.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.510+0000 7ff2ff7fe640 1 -- 172.21.15.78:0/2363570131 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff304003ce0 con 0x7ff310102260 2024-08-18T23:43:11.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.510+0000 7ff314b13640 1 -- 172.21.15.78:0/2363570131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31019d520 con 0x7ff310102260 2024-08-18T23:43:11.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.510+0000 7ff2ff7fe640 1 -- 172.21.15.78:0/2363570131 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304035070 con 0x7ff310102260 2024-08-18T23:43:11.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.510+0000 7ff314b13640 1 -- 172.21.15.78:0/2363570131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3101a4e60 con 0x7ff310102260 2024-08-18T23:43:11.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.511+0000 7ff314b13640 1 -- 172.21.15.78:0/2363570131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff310101da0 con 0x7ff310102260 2024-08-18T23:43:11.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.512+0000 7ff2ff7fe640 1 -- 172.21.15.78:0/2363570131 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff304042030 con 0x7ff310102260 2024-08-18T23:43:11.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.513+0000 7ff2ff7fe640 1 --2- 172.21.15.78:0/2363570131 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff2f80a2c10 0x7ff2f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:11.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.513+0000 7ff30e59c640 1 --2- 172.21.15.78:0/2363570131 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff2f80a2c10 0x7ff2f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:11.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.513+0000 7ff2ff7fe640 1 -- 172.21.15.78:0/2363570131 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff3040ef820 con 0x7ff310102260 2024-08-18T23:43:11.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.513+0000 7ff30e59c640 1 --2- 172.21.15.78:0/2363570131 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff2f80a2c10 0x7ff2f80a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff300009710 tx=0x7ff300006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:11.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.517+0000 7ff2ff7fe640 1 -- 172.21.15.78:0/2363570131 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff3040bc2c0 con 0x7ff310102260 2024-08-18T23:43:11.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:11 smithi123 ceph-mon[21808]: pgmap v314: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:11.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:11 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2915881987' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-08-18T23:43:11.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:11 smithi161 ceph-mon[22027]: pgmap v314: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:11.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:11 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2915881987' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-08-18T23:43:11.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.693+0000 7ff314b13640 1 -- 172.21.15.78:0/2363570131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff310109e50 con 0x7ff310102260 2024-08-18T23:43:11.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.693+0000 7ff2ff7fe640 1 -- 172.21.15.78:0/2363570131 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff3040bbbc0 con 0x7ff310102260 2024-08-18T23:43:11.694 INFO:teuthology.orchestra.run.smithi078.stdout:34359738450 2024-08-18T23:43:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.695+0000 7ff314b13640 1 -- 172.21.15.78:0/2363570131 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff2f80a2c10 msgr2=0x7ff2f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.695+0000 7ff314b13640 1 --2- 172.21.15.78:0/2363570131 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff2f80a2c10 0x7ff2f80a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff300009710 tx=0x7ff300006c90 comp rx=0 tx=0).stop 2024-08-18T23:43:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.695+0000 7ff314b13640 1 -- 172.21.15.78:0/2363570131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff310102260 msgr2=0x7ff310198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.695+0000 7ff314b13640 1 --2- 172.21.15.78:0/2363570131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff310102260 0x7ff310198820 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff304009a50 tx=0x7ff304004270 comp rx=0 tx=0).stop 2024-08-18T23:43:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.695+0000 7ff314b13640 1 -- 172.21.15.78:0/2363570131 shutdown_connections 2024-08-18T23:43:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.695+0000 7ff314b13640 1 --2- 172.21.15.78:0/2363570131 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff2f80a2c10 0x7ff2f80a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.695+0000 7ff314b13640 1 --2- 172.21.15.78:0/2363570131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3101078c0 0x7ff31019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.695+0000 7ff314b13640 1 --2- 172.21.15.78:0/2363570131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff310102260 0x7ff310198820 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.695+0000 7ff314b13640 1 --2- 172.21.15.78:0/2363570131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3101018c0 0x7ff3101982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.695+0000 7ff314b13640 1 -- 172.21.15.78:0/2363570131 >> 172.21.15.78:0/2363570131 conn(0x7ff3100fd5e0 msgr2=0x7ff3100fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:11.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.695+0000 7ff314b13640 1 -- 172.21.15.78:0/2363570131 shutdown_connections 2024-08-18T23:43:11.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:11.696+0000 7ff314b13640 1 -- 172.21.15.78:0/2363570131 wait complete. 2024-08-18T23:43:12.296 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790009 got 158913790012 for osd.5 2024-08-18T23:43:12.296 DEBUG:teuthology.parallel:result is None 2024-08-18T23:43:12.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:12 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2971683943' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-08-18T23:43:12.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:12 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2363570131' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-08-18T23:43:12.672 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:12 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2971683943' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-08-18T23:43:12.672 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:12 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2363570131' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-08-18T23:43:12.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:12 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2971683943' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-08-18T23:43:12.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:12 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2363570131' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-08-18T23:43:13.035 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738447 got 34359738450 for osd.0 2024-08-18T23:43:13.036 DEBUG:teuthology.parallel:result is None 2024-08-18T23:43:13.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:13 smithi078 ceph-mon[18440]: pgmap v315: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:13.520 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776161 got 292057776166 for osd.10 2024-08-18T23:43:13.520 DEBUG:teuthology.parallel:result is None 2024-08-18T23:43:13.520 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-18T23:43:13.520 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-18T23:43:13.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:13 smithi123 ceph-mon[21808]: pgmap v315: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:13 smithi161 ceph-mon[22027]: pgmap v315: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:13.780 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.257+0000 7fb15b4b6640 1 -- 172.21.15.78:0/4109148256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1541018c0 msgr2=0x7fb154101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.257+0000 7fb15b4b6640 1 --2- 172.21.15.78:0/4109148256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1541018c0 0x7fb154101d20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb144009930 tx=0x7fb14402a420 comp rx=0 tx=0).stop 2024-08-18T23:43:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.257+0000 7fb15b4b6640 1 -- 172.21.15.78:0/4109148256 shutdown_connections 2024-08-18T23:43:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.257+0000 7fb15b4b6640 1 --2- 172.21.15.78:0/4109148256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb154102260 0x7fb15410e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.257+0000 7fb15b4b6640 1 --2- 172.21.15.78:0/4109148256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1541018c0 0x7fb154101d20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.257+0000 7fb15b4b6640 1 --2- 172.21.15.78:0/4109148256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1541078c0 0x7fb154107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.257+0000 7fb15b4b6640 1 -- 172.21.15.78:0/4109148256 >> 172.21.15.78:0/4109148256 conn(0x7fb1540fd620 msgr2=0x7fb1540ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.257+0000 7fb15b4b6640 1 -- 172.21.15.78:0/4109148256 shutdown_connections 2024-08-18T23:43:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.257+0000 7fb15b4b6640 1 -- 172.21.15.78:0/4109148256 wait complete. 2024-08-18T23:43:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.258+0000 7fb15b4b6640 1 Processor -- start 2024-08-18T23:43:14.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.258+0000 7fb15b4b6640 1 -- start start 2024-08-18T23:43:14.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.258+0000 7fb15b4b6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1541018c0 0x7fb154198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:14.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.258+0000 7fb15b4b6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb154102260 0x7fb154198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:14.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.258+0000 7fb15b4b6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1541078c0 0x7fb15419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:14.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.258+0000 7fb15b4b6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb154111d10 con 0x7fb1541018c0 2024-08-18T23:43:14.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.258+0000 7fb15b4b6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb154111b90 con 0x7fb154102260 2024-08-18T23:43:14.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.258+0000 7fb15b4b6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb154111e90 con 0x7fb1541078c0 2024-08-18T23:43:14.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.258+0000 7fb159252640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1541018c0 0x7fb154198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:14.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.258+0000 7fb159252640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1541018c0 0x7fb154198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59216/0 (socket says 172.21.15.78:59216) 2024-08-18T23:43:14.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.258+0000 7fb159252640 1 -- 172.21.15.78:0/1883764712 learned_addr learned my addr 172.21.15.78:0/1883764712 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:14.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.258+0000 7fb158a51640 1 --2- 172.21.15.78:0/1883764712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb154102260 0x7fb154198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:14.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.258+0000 7fb159a53640 1 --2- 172.21.15.78:0/1883764712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1541078c0 0x7fb15419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.259+0000 7fb159252640 1 -- 172.21.15.78:0/1883764712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1541078c0 msgr2=0x7fb15419cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.259+0000 7fb159252640 1 --2- 172.21.15.78:0/1883764712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1541078c0 0x7fb15419cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.259+0000 7fb159252640 1 -- 172.21.15.78:0/1883764712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb154102260 msgr2=0x7fb154198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.259+0000 7fb159252640 1 --2- 172.21.15.78:0/1883764712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb154102260 0x7fb154198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.259+0000 7fb159252640 1 -- 172.21.15.78:0/1883764712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb15419d1c0 con 0x7fb1541018c0 2024-08-18T23:43:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.259+0000 7fb159252640 1 --2- 172.21.15.78:0/1883764712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1541018c0 0x7fb154198210 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb15000d950 tx=0x7fb15000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.259+0000 7fb14a7fc640 1 -- 172.21.15.78:0/1883764712 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb150017070 con 0x7fb1541018c0 2024-08-18T23:43:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.259+0000 7fb14a7fc640 1 -- 172.21.15.78:0/1883764712 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb150009c30 con 0x7fb1541018c0 2024-08-18T23:43:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.259+0000 7fb15b4b6640 1 -- 172.21.15.78:0/1883764712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb15419d4b0 con 0x7fb1541018c0 2024-08-18T23:43:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.259+0000 7fb14a7fc640 1 -- 172.21.15.78:0/1883764712 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15000f6b0 con 0x7fb1541018c0 2024-08-18T23:43:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.259+0000 7fb15b4b6640 1 -- 172.21.15.78:0/1883764712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1541a4d90 con 0x7fb1541018c0 2024-08-18T23:43:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.260+0000 7fb15b4b6640 1 -- 172.21.15.78:0/1883764712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb154101d20 con 0x7fb1541018c0 2024-08-18T23:43:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.261+0000 7fb14a7fc640 1 -- 172.21.15.78:0/1883764712 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb1500109d0 con 0x7fb1541018c0 2024-08-18T23:43:14.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.262+0000 7fb14a7fc640 1 --2- 172.21.15.78:0/1883764712 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb1400a2c90 0x7fb1400a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:14.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.262+0000 7fb158a51640 1 --2- 172.21.15.78:0/1883764712 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb1400a2c90 0x7fb1400a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:14.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.262+0000 7fb14a7fc640 1 -- 172.21.15.78:0/1883764712 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fb1500cf130 con 0x7fb1541018c0 2024-08-18T23:43:14.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.262+0000 7fb158a51640 1 --2- 172.21.15.78:0/1883764712 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb1400a2c90 0x7fb1400a5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb144006010 tx=0x7fb144005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:14.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.266+0000 7fb14a7fc640 1 -- 172.21.15.78:0/1883764712 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb150009de0 con 0x7fb1541018c0 2024-08-18T23:43:14.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.430+0000 7fb15b4b6640 1 -- 172.21.15.78:0/1883764712 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb154062360 con 0x7fb1400a2c90 2024-08-18T23:43:14.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.438+0000 7fb14a7fc640 1 -- 172.21.15.78:0/1883764712 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340364 (secure 0 0 0) 0x7fb154062360 con 0x7fb1400a2c90 2024-08-18T23:43:14.443 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:43:14.445 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-08-18T23:43:14.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.441+0000 7fb15b4b6640 1 -- 172.21.15.78:0/1883764712 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb1400a2c90 msgr2=0x7fb1400a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:14.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.441+0000 7fb15b4b6640 1 --2- 172.21.15.78:0/1883764712 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb1400a2c90 0x7fb1400a5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb144006010 tx=0x7fb144005a20 comp rx=0 tx=0).stop 2024-08-18T23:43:14.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.441+0000 7fb15b4b6640 1 -- 172.21.15.78:0/1883764712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1541018c0 msgr2=0x7fb154198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:14.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.441+0000 7fb15b4b6640 1 --2- 172.21.15.78:0/1883764712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1541018c0 0x7fb154198210 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb15000d950 tx=0x7fb15000de10 comp rx=0 tx=0).stop 2024-08-18T23:43:14.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.441+0000 7fb15b4b6640 1 -- 172.21.15.78:0/1883764712 shutdown_connections 2024-08-18T23:43:14.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.441+0000 7fb15b4b6640 1 --2- 172.21.15.78:0/1883764712 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb1400a2c90 0x7fb1400a5150 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:14.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.441+0000 7fb15b4b6640 1 --2- 172.21.15.78:0/1883764712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1541078c0 0x7fb15419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:14.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.441+0000 7fb15b4b6640 1 --2- 172.21.15.78:0/1883764712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb154102260 0x7fb154198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:14.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.441+0000 7fb15b4b6640 1 --2- 172.21.15.78:0/1883764712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1541018c0 0x7fb154198210 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:14.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.441+0000 7fb15b4b6640 1 -- 172.21.15.78:0/1883764712 >> 172.21.15.78:0/1883764712 conn(0x7fb1540fd620 msgr2=0x7fb1540fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:14.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.441+0000 7fb15b4b6640 1 -- 172.21.15.78:0/1883764712 shutdown_connections 2024-08-18T23:43:14.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:14.441+0000 7fb15b4b6640 1 -- 172.21.15.78:0/1883764712 wait complete. 2024-08-18T23:43:15.209 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-08-18T23:43:14.134620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657988,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":527,"num_read_kb":379,"num_write":350,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":282,"ondisk_log_size":282,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388220,"kb_used_data":14048,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553604,"statfs":{"total":1151940427776,"available":1151542890496,"internally_reserved":0,"allocated":14385152,"data_stored":8010062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003204"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319578+0000","last_change":"2024-08-18T23:41:30.120580+0000","last_active":"2024-08-18T23:42:23.319578+0000","last_peered":"2024-08-18T23:42:23.319578+0000","last_clean":"2024-08-18T23:42:23.319578+0000","last_became_active":"2024-08-18T23:41:30.120266+0000","last_became_peered":"2024-08-18T23:41:30.120266+0000","last_unstale":"2024-08-18T23:42:23.319578+0000","last_undegraded":"2024-08-18T23:42:23.319578+0000","last_fullsized":"2024-08-18T23:42:23.319578+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:06:40.997070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557745+0000","last_change":"2024-08-18T23:41:32.143232+0000","last_active":"2024-08-18T23:42:23.557745+0000","last_peered":"2024-08-18T23:42:23.557745+0000","last_clean":"2024-08-18T23:42:23.557745+0000","last_became_active":"2024-08-18T23:41:32.142916+0000","last_became_peered":"2024-08-18T23:41:32.142916+0000","last_unstale":"2024-08-18T23:42:23.557745+0000","last_undegraded":"2024-08-18T23:42:23.557745+0000","last_fullsized":"2024-08-18T23:42:23.557745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:50:54.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320593+0000","last_change":"2024-08-18T23:41:39.188772+0000","last_active":"2024-08-18T23:42:23.320593+0000","last_peered":"2024-08-18T23:42:23.320593+0000","last_clean":"2024-08-18T23:42:23.320593+0000","last_became_active":"2024-08-18T23:41:39.188362+0000","last_became_peered":"2024-08-18T23:41:39.188362+0000","last_unstale":"2024-08-18T23:42:23.320593+0000","last_undegraded":"2024-08-18T23:42:23.320593+0000","last_fullsized":"2024-08-18T23:42:23.320593+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:47:50.255543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557888+0000","last_change":"2024-08-18T23:41:30.137179+0000","last_active":"2024-08-18T23:42:23.557888+0000","last_peered":"2024-08-18T23:42:23.557888+0000","last_clean":"2024-08-18T23:42:23.557888+0000","last_became_active":"2024-08-18T23:41:30.136618+0000","last_became_peered":"2024-08-18T23:41:30.136618+0000","last_unstale":"2024-08-18T23:42:23.557888+0000","last_undegraded":"2024-08-18T23:42:23.557888+0000","last_fullsized":"2024-08-18T23:42:23.557888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:59.826364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316847+0000","last_change":"2024-08-18T23:41:32.142075+0000","last_active":"2024-08-18T23:42:23.316847+0000","last_peered":"2024-08-18T23:42:23.316847+0000","last_clean":"2024-08-18T23:42:23.316847+0000","last_became_active":"2024-08-18T23:41:32.141709+0000","last_became_peered":"2024-08-18T23:41:32.141709+0000","last_unstale":"2024-08-18T23:42:23.316847+0000","last_undegraded":"2024-08-18T23:42:23.316847+0000","last_fullsized":"2024-08-18T23:42:23.316847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:08.324627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317948+0000","last_change":"2024-08-18T23:41:39.184575+0000","last_active":"2024-08-18T23:42:23.317948+0000","last_peered":"2024-08-18T23:42:23.317948+0000","last_clean":"2024-08-18T23:42:23.317948+0000","last_became_active":"2024-08-18T23:41:39.184169+0000","last_became_peered":"2024-08-18T23:41:39.184169+0000","last_unstale":"2024-08-18T23:42:23.317948+0000","last_undegraded":"2024-08-18T23:42:23.317948+0000","last_fullsized":"2024-08-18T23:42:23.317948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:32.218172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324756+0000","last_change":"2024-08-18T23:41:30.121638+0000","last_active":"2024-08-18T23:42:23.324756+0000","last_peered":"2024-08-18T23:42:23.324756+0000","last_clean":"2024-08-18T23:42:23.324756+0000","last_became_active":"2024-08-18T23:41:30.121290+0000","last_became_peered":"2024-08-18T23:41:30.121290+0000","last_unstale":"2024-08-18T23:42:23.324756+0000","last_undegraded":"2024-08-18T23:42:23.324756+0000","last_fullsized":"2024-08-18T23:42:23.324756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:43:55.638780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315534+0000","last_change":"2024-08-18T23:41:32.142177+0000","last_active":"2024-08-18T23:42:23.315534+0000","last_peered":"2024-08-18T23:42:23.315534+0000","last_clean":"2024-08-18T23:42:23.315534+0000","last_became_active":"2024-08-18T23:41:32.141731+0000","last_became_peered":"2024-08-18T23:41:32.141731+0000","last_unstale":"2024-08-18T23:42:23.315534+0000","last_undegraded":"2024-08-18T23:42:23.315534+0000","last_fullsized":"2024-08-18T23:42:23.315534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:36.283299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318344+0000","last_change":"2024-08-18T23:41:39.182931+0000","last_active":"2024-08-18T23:42:23.318344+0000","last_peered":"2024-08-18T23:42:23.318344+0000","last_clean":"2024-08-18T23:42:23.318344+0000","last_became_active":"2024-08-18T23:41:39.182008+0000","last_became_peered":"2024-08-18T23:41:39.182008+0000","last_unstale":"2024-08-18T23:42:23.318344+0000","last_undegraded":"2024-08-18T23:42:23.318344+0000","last_fullsized":"2024-08-18T23:42:23.318344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:58:38.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.329762+0000","last_change":"2024-08-18T23:42:23.329762+0000","last_active":"2024-08-18T23:42:23.329762+0000","last_peered":"2024-08-18T23:42:23.329762+0000","last_clean":"2024-08-18T23:42:23.329762+0000","last_became_active":"2024-08-18T23:42:23.325753+0000","last_became_peered":"2024-08-18T23:42:23.325753+0000","last_unstale":"2024-08-18T23:42:23.329762+0000","last_undegraded":"2024-08-18T23:42:23.329762+0000","last_fullsized":"2024-08-18T23:42:23.329762+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:19:43.062492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323256+0000","last_change":"2024-08-18T23:41:32.148662+0000","last_active":"2024-08-18T23:42:23.323256+0000","last_peered":"2024-08-18T23:42:23.323256+0000","last_clean":"2024-08-18T23:42:23.323256+0000","last_became_active":"2024-08-18T23:41:32.148318+0000","last_became_peered":"2024-08-18T23:41:32.148318+0000","last_unstale":"2024-08-18T23:42:23.323256+0000","last_undegraded":"2024-08-18T23:42:23.323256+0000","last_fullsized":"2024-08-18T23:42:23.323256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:58.668669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318492+0000","last_change":"2024-08-18T23:41:39.181114+0000","last_active":"2024-08-18T23:42:23.318492+0000","last_peered":"2024-08-18T23:42:23.318492+0000","last_clean":"2024-08-18T23:42:23.318492+0000","last_became_active":"2024-08-18T23:41:39.180599+0000","last_became_peered":"2024-08-18T23:41:39.180599+0000","last_unstale":"2024-08-18T23:42:23.318492+0000","last_undegraded":"2024-08-18T23:42:23.318492+0000","last_fullsized":"2024-08-18T23:42:23.318492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:36:08.749143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316660+0000","last_change":"2024-08-18T23:41:32.144326+0000","last_active":"2024-08-18T23:42:23.316660+0000","last_peered":"2024-08-18T23:42:23.316660+0000","last_clean":"2024-08-18T23:42:23.316660+0000","last_became_active":"2024-08-18T23:41:32.144004+0000","last_became_peered":"2024-08-18T23:41:32.144004+0000","last_unstale":"2024-08-18T23:42:23.316660+0000","last_undegraded":"2024-08-18T23:42:23.316660+0000","last_fullsized":"2024-08-18T23:42:23.316660+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:10.918624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324188+0000","last_change":"2024-08-18T23:41:30.125160+0000","last_active":"2024-08-18T23:42:23.324188+0000","last_peered":"2024-08-18T23:42:23.324188+0000","last_clean":"2024-08-18T23:42:23.324188+0000","last_became_active":"2024-08-18T23:41:30.124685+0000","last_became_peered":"2024-08-18T23:41:30.124685+0000","last_unstale":"2024-08-18T23:42:23.324188+0000","last_undegraded":"2024-08-18T23:42:23.324188+0000","last_fullsized":"2024-08-18T23:42:23.324188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:52:48.997902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318342+0000","last_change":"2024-08-18T23:41:39.681481+0000","last_active":"2024-08-18T23:42:23.318342+0000","last_peered":"2024-08-18T23:42:23.318342+0000","last_clean":"2024-08-18T23:42:23.318342+0000","last_became_active":"2024-08-18T23:41:39.680204+0000","last_became_peered":"2024-08-18T23:41:39.680204+0000","last_unstale":"2024-08-18T23:42:23.318342+0000","last_undegraded":"2024-08-18T23:42:23.318342+0000","last_fullsized":"2024-08-18T23:42:23.318342+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:00:05.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557591+0000","last_change":"2024-08-18T23:41:32.151335+0000","last_active":"2024-08-18T23:42:23.557591+0000","last_peered":"2024-08-18T23:42:23.557591+0000","last_clean":"2024-08-18T23:42:23.557591+0000","last_became_active":"2024-08-18T23:41:32.151040+0000","last_became_peered":"2024-08-18T23:41:32.151040+0000","last_unstale":"2024-08-18T23:42:23.557591+0000","last_undegraded":"2024-08-18T23:42:23.557591+0000","last_fullsized":"2024-08-18T23:42:23.557591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:16.108555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.321637+0000","last_change":"2024-08-18T23:41:30.122301+0000","last_active":"2024-08-18T23:42:23.321637+0000","last_peered":"2024-08-18T23:42:23.321637+0000","last_clean":"2024-08-18T23:42:23.321637+0000","last_became_active":"2024-08-18T23:41:30.121117+0000","last_became_peered":"2024-08-18T23:41:30.121117+0000","last_unstale":"2024-08-18T23:42:23.321637+0000","last_undegraded":"2024-08-18T23:42:23.321637+0000","last_fullsized":"2024-08-18T23:42:23.321637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:40:12.678803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318961+0000","last_change":"2024-08-18T23:41:39.182395+0000","last_active":"2024-08-18T23:42:23.318961+0000","last_peered":"2024-08-18T23:42:23.318961+0000","last_clean":"2024-08-18T23:42:23.318961+0000","last_became_active":"2024-08-18T23:41:39.181979+0000","last_became_peered":"2024-08-18T23:41:39.181979+0000","last_unstale":"2024-08-18T23:42:23.318961+0000","last_undegraded":"2024-08-18T23:42:23.318961+0000","last_fullsized":"2024-08-18T23:42:23.318961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:04.674503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318695+0000","last_change":"2024-08-18T23:41:32.140464+0000","last_active":"2024-08-18T23:42:23.318695+0000","last_peered":"2024-08-18T23:42:23.318695+0000","last_clean":"2024-08-18T23:42:23.318695+0000","last_became_active":"2024-08-18T23:41:32.139153+0000","last_became_peered":"2024-08-18T23:41:32.139153+0000","last_unstale":"2024-08-18T23:42:23.318695+0000","last_undegraded":"2024-08-18T23:42:23.318695+0000","last_fullsized":"2024-08-18T23:42:23.318695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:27.387680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557664+0000","last_change":"2024-08-18T23:41:30.122787+0000","last_active":"2024-08-18T23:42:23.557664+0000","last_peered":"2024-08-18T23:42:23.557664+0000","last_clean":"2024-08-18T23:42:23.557664+0000","last_became_active":"2024-08-18T23:41:30.122462+0000","last_became_peered":"2024-08-18T23:41:30.122462+0000","last_unstale":"2024-08-18T23:42:23.557664+0000","last_undegraded":"2024-08-18T23:42:23.557664+0000","last_fullsized":"2024-08-18T23:42:23.557664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:55.568831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319322+0000","last_change":"2024-08-18T23:41:39.190511+0000","last_active":"2024-08-18T23:42:23.319322+0000","last_peered":"2024-08-18T23:42:23.319322+0000","last_clean":"2024-08-18T23:42:23.319322+0000","last_became_active":"2024-08-18T23:41:39.190154+0000","last_became_peered":"2024-08-18T23:41:39.190154+0000","last_unstale":"2024-08-18T23:42:23.319322+0000","last_undegraded":"2024-08-18T23:42:23.319322+0000","last_fullsized":"2024-08-18T23:42:23.319322+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:33:34.514827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323413+0000","last_change":"2024-08-18T23:41:32.143867+0000","last_active":"2024-08-18T23:42:23.323413+0000","last_peered":"2024-08-18T23:42:23.323413+0000","last_clean":"2024-08-18T23:42:23.323413+0000","last_became_active":"2024-08-18T23:41:32.143527+0000","last_became_peered":"2024-08-18T23:41:32.143527+0000","last_unstale":"2024-08-18T23:42:23.323413+0000","last_undegraded":"2024-08-18T23:42:23.323413+0000","last_fullsized":"2024-08-18T23:42:23.323413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:40:25.902133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.325102+0000","last_change":"2024-08-18T23:42:23.325102+0000","last_active":"2024-08-18T23:42:23.325102+0000","last_peered":"2024-08-18T23:42:23.325102+0000","last_clean":"2024-08-18T23:42:23.325102+0000","last_became_active":"2024-08-18T23:42:23.321583+0000","last_became_peered":"2024-08-18T23:42:23.321583+0000","last_unstale":"2024-08-18T23:42:23.325102+0000","last_undegraded":"2024-08-18T23:42:23.325102+0000","last_fullsized":"2024-08-18T23:42:23.325102+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:57:42.546686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.559756+0000","last_change":"2024-08-18T23:42:23.559756+0000","last_active":"2024-08-18T23:42:23.559756+0000","last_peered":"2024-08-18T23:42:23.559756+0000","last_clean":"2024-08-18T23:42:23.559756+0000","last_became_active":"2024-08-18T23:42:23.559242+0000","last_became_peered":"2024-08-18T23:42:23.559242+0000","last_unstale":"2024-08-18T23:42:23.559756+0000","last_undegraded":"2024-08-18T23:42:23.559756+0000","last_fullsized":"2024-08-18T23:42:23.559756+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:25:40.790313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316766+0000","last_change":"2024-08-18T23:41:32.139025+0000","last_active":"2024-08-18T23:42:23.316766+0000","last_peered":"2024-08-18T23:42:23.316766+0000","last_clean":"2024-08-18T23:42:23.316766+0000","last_became_active":"2024-08-18T23:41:32.138733+0000","last_became_peered":"2024-08-18T23:41:32.138733+0000","last_unstale":"2024-08-18T23:42:23.316766+0000","last_undegraded":"2024-08-18T23:42:23.316766+0000","last_fullsized":"2024-08-18T23:42:23.316766+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:01:39.826391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319270+0000","last_change":"2024-08-18T23:41:30.477359+0000","last_active":"2024-08-18T23:42:23.319270+0000","last_peered":"2024-08-18T23:42:23.319270+0000","last_clean":"2024-08-18T23:42:23.319270+0000","last_became_active":"2024-08-18T23:41:30.477011+0000","last_became_peered":"2024-08-18T23:41:30.477011+0000","last_unstale":"2024-08-18T23:42:23.319270+0000","last_undegraded":"2024-08-18T23:42:23.319270+0000","last_fullsized":"2024-08-18T23:42:23.319270+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:53:10.052859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320716+0000","last_change":"2024-08-18T23:41:39.182140+0000","last_active":"2024-08-18T23:42:23.320716+0000","last_peered":"2024-08-18T23:42:23.320716+0000","last_clean":"2024-08-18T23:42:23.320716+0000","last_became_active":"2024-08-18T23:41:39.181667+0000","last_became_peered":"2024-08-18T23:41:39.181667+0000","last_unstale":"2024-08-18T23:42:23.320716+0000","last_undegraded":"2024-08-18T23:42:23.320716+0000","last_fullsized":"2024-08-18T23:42:23.320716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:02:18.128028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557897+0000","last_change":"2024-08-18T23:41:32.137064+0000","last_active":"2024-08-18T23:42:23.557897+0000","last_peered":"2024-08-18T23:42:23.557897+0000","last_clean":"2024-08-18T23:42:23.557897+0000","last_became_active":"2024-08-18T23:41:32.136738+0000","last_became_peered":"2024-08-18T23:41:32.136738+0000","last_unstale":"2024-08-18T23:42:23.557897+0000","last_undegraded":"2024-08-18T23:42:23.557897+0000","last_fullsized":"2024-08-18T23:42:23.557897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:15:11.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319512+0000","last_change":"2024-08-18T23:41:30.120042+0000","last_active":"2024-08-18T23:42:23.319512+0000","last_peered":"2024-08-18T23:42:23.319512+0000","last_clean":"2024-08-18T23:42:23.319512+0000","last_became_active":"2024-08-18T23:41:30.119719+0000","last_became_peered":"2024-08-18T23:41:30.119719+0000","last_unstale":"2024-08-18T23:42:23.319512+0000","last_undegraded":"2024-08-18T23:42:23.319512+0000","last_fullsized":"2024-08-18T23:42:23.319512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:35:44.466330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557353+0000","last_change":"2024-08-18T23:41:39.680405+0000","last_active":"2024-08-18T23:42:23.557353+0000","last_peered":"2024-08-18T23:42:23.557353+0000","last_clean":"2024-08-18T23:42:23.557353+0000","last_became_active":"2024-08-18T23:41:39.679940+0000","last_became_peered":"2024-08-18T23:41:39.679940+0000","last_unstale":"2024-08-18T23:42:23.557353+0000","last_undegraded":"2024-08-18T23:42:23.557353+0000","last_fullsized":"2024-08-18T23:42:23.557353+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:15:03.942116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314555+0000","last_change":"2024-08-18T23:41:32.139688+0000","last_active":"2024-08-18T23:42:23.314555+0000","last_peered":"2024-08-18T23:42:23.314555+0000","last_clean":"2024-08-18T23:42:23.314555+0000","last_became_active":"2024-08-18T23:41:32.139397+0000","last_became_peered":"2024-08-18T23:41:32.139397+0000","last_unstale":"2024-08-18T23:42:23.314555+0000","last_undegraded":"2024-08-18T23:42:23.314555+0000","last_fullsized":"2024-08-18T23:42:23.314555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:03:57.017488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324876+0000","last_change":"2024-08-18T23:41:30.137578+0000","last_active":"2024-08-18T23:42:23.324876+0000","last_peered":"2024-08-18T23:42:23.324876+0000","last_clean":"2024-08-18T23:42:23.324876+0000","last_became_active":"2024-08-18T23:41:30.137246+0000","last_became_peered":"2024-08-18T23:41:30.137246+0000","last_unstale":"2024-08-18T23:42:23.324876+0000","last_undegraded":"2024-08-18T23:42:23.324876+0000","last_fullsized":"2024-08-18T23:42:23.324876+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:48:54.839432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318001+0000","last_change":"2024-08-18T23:41:39.186360+0000","last_active":"2024-08-18T23:42:23.318001+0000","last_peered":"2024-08-18T23:42:23.318001+0000","last_clean":"2024-08-18T23:42:23.318001+0000","last_became_active":"2024-08-18T23:41:39.185808+0000","last_became_peered":"2024-08-18T23:41:39.185808+0000","last_unstale":"2024-08-18T23:42:23.318001+0000","last_undegraded":"2024-08-18T23:42:23.318001+0000","last_fullsized":"2024-08-18T23:42:23.318001+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:53.891061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558281+0000","last_change":"2024-08-18T23:41:32.146070+0000","last_active":"2024-08-18T23:42:23.558281+0000","last_peered":"2024-08-18T23:42:23.558281+0000","last_clean":"2024-08-18T23:42:23.558281+0000","last_became_active":"2024-08-18T23:41:32.145640+0000","last_became_peered":"2024-08-18T23:41:32.145640+0000","last_unstale":"2024-08-18T23:42:23.558281+0000","last_undegraded":"2024-08-18T23:42:23.558281+0000","last_fullsized":"2024-08-18T23:42:23.558281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:56:21.368762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319576+0000","last_change":"2024-08-18T23:41:30.138565+0000","last_active":"2024-08-18T23:42:23.319576+0000","last_peered":"2024-08-18T23:42:23.319576+0000","last_clean":"2024-08-18T23:42:23.319576+0000","last_became_active":"2024-08-18T23:41:30.138210+0000","last_became_peered":"2024-08-18T23:41:30.138210+0000","last_unstale":"2024-08-18T23:42:23.319576+0000","last_undegraded":"2024-08-18T23:42:23.319576+0000","last_fullsized":"2024-08-18T23:42:23.319576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:42:14.928884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323149+0000","last_change":"2024-08-18T23:41:39.680656+0000","last_active":"2024-08-18T23:42:23.323149+0000","last_peered":"2024-08-18T23:42:23.323149+0000","last_clean":"2024-08-18T23:42:23.323149+0000","last_became_active":"2024-08-18T23:41:39.680203+0000","last_became_peered":"2024-08-18T23:41:39.680203+0000","last_unstale":"2024-08-18T23:42:23.323149+0000","last_undegraded":"2024-08-18T23:42:23.323149+0000","last_fullsized":"2024-08-18T23:42:23.323149+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:55:43.138869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558246+0000","last_change":"2024-08-18T23:41:32.130222+0000","last_active":"2024-08-18T23:42:23.558246+0000","last_peered":"2024-08-18T23:42:23.558246+0000","last_clean":"2024-08-18T23:42:23.558246+0000","last_became_active":"2024-08-18T23:41:32.129844+0000","last_became_peered":"2024-08-18T23:41:32.129844+0000","last_unstale":"2024-08-18T23:42:23.558246+0000","last_undegraded":"2024-08-18T23:42:23.558246+0000","last_fullsized":"2024-08-18T23:42:23.558246+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:39:21.106343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316823+0000","last_change":"2024-08-18T23:41:30.123142+0000","last_active":"2024-08-18T23:42:23.316823+0000","last_peered":"2024-08-18T23:42:23.316823+0000","last_clean":"2024-08-18T23:42:23.316823+0000","last_became_active":"2024-08-18T23:41:30.121645+0000","last_became_peered":"2024-08-18T23:41:30.121645+0000","last_unstale":"2024-08-18T23:42:23.316823+0000","last_undegraded":"2024-08-18T23:42:23.316823+0000","last_fullsized":"2024-08-18T23:42:23.316823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:07:24.912259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314454+0000","last_change":"2024-08-18T23:41:39.186033+0000","last_active":"2024-08-18T23:42:23.314454+0000","last_peered":"2024-08-18T23:42:23.314454+0000","last_clean":"2024-08-18T23:42:23.314454+0000","last_became_active":"2024-08-18T23:41:39.185451+0000","last_became_peered":"2024-08-18T23:41:39.185451+0000","last_unstale":"2024-08-18T23:42:23.314454+0000","last_undegraded":"2024-08-18T23:42:23.314454+0000","last_fullsized":"2024-08-18T23:42:23.314454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:52:13.556760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320803+0000","last_change":"2024-08-18T23:41:32.141065+0000","last_active":"2024-08-18T23:42:23.320803+0000","last_peered":"2024-08-18T23:42:23.320803+0000","last_clean":"2024-08-18T23:42:23.320803+0000","last_became_active":"2024-08-18T23:41:32.140732+0000","last_became_peered":"2024-08-18T23:41:32.140732+0000","last_unstale":"2024-08-18T23:42:23.320803+0000","last_undegraded":"2024-08-18T23:42:23.320803+0000","last_fullsized":"2024-08-18T23:42:23.320803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:08:32.287835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":99,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:43:07.355259+0000","last_change":"2024-08-18T23:41:30.136838+0000","last_active":"2024-08-18T23:43:07.355259+0000","last_peered":"2024-08-18T23:43:07.355259+0000","last_clean":"2024-08-18T23:43:07.355259+0000","last_became_active":"2024-08-18T23:41:30.136523+0000","last_became_peered":"2024-08-18T23:41:30.136523+0000","last_unstale":"2024-08-18T23:43:07.355259+0000","last_undegraded":"2024-08-18T23:43:07.355259+0000","last_fullsized":"2024-08-18T23:43:07.355259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:56:17.750535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":61,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314559+0000","last_change":"2024-08-18T23:41:39.680314+0000","last_active":"2024-08-18T23:42:23.314559+0000","last_peered":"2024-08-18T23:42:23.314559+0000","last_clean":"2024-08-18T23:42:23.314559+0000","last_became_active":"2024-08-18T23:41:39.679920+0000","last_became_peered":"2024-08-18T23:41:39.679920+0000","last_unstale":"2024-08-18T23:42:23.314559+0000","last_undegraded":"2024-08-18T23:42:23.314559+0000","last_fullsized":"2024-08-18T23:42:23.314559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:26:29.866081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314066+0000","last_change":"2024-08-18T23:41:32.139752+0000","last_active":"2024-08-18T23:42:23.314066+0000","last_peered":"2024-08-18T23:42:23.314066+0000","last_clean":"2024-08-18T23:42:23.314066+0000","last_became_active":"2024-08-18T23:41:32.139451+0000","last_became_peered":"2024-08-18T23:41:32.139451+0000","last_unstale":"2024-08-18T23:42:23.314066+0000","last_undegraded":"2024-08-18T23:42:23.314066+0000","last_fullsized":"2024-08-18T23:42:23.314066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:15:13.390529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316737+0000","last_change":"2024-08-18T23:41:30.126513+0000","last_active":"2024-08-18T23:42:23.316737+0000","last_peered":"2024-08-18T23:42:23.316737+0000","last_clean":"2024-08-18T23:42:23.316737+0000","last_became_active":"2024-08-18T23:41:30.126046+0000","last_became_peered":"2024-08-18T23:41:30.126046+0000","last_unstale":"2024-08-18T23:42:23.316737+0000","last_undegraded":"2024-08-18T23:42:23.316737+0000","last_fullsized":"2024-08-18T23:42:23.316737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:49:16.565565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323372+0000","last_change":"2024-08-18T23:41:39.681006+0000","last_active":"2024-08-18T23:42:23.323372+0000","last_peered":"2024-08-18T23:42:23.323372+0000","last_clean":"2024-08-18T23:42:23.323372+0000","last_became_active":"2024-08-18T23:41:39.680409+0000","last_became_peered":"2024-08-18T23:41:39.680409+0000","last_unstale":"2024-08-18T23:42:23.323372+0000","last_undegraded":"2024-08-18T23:42:23.323372+0000","last_fullsized":"2024-08-18T23:42:23.323372+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:47.778866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557596+0000","last_change":"2024-08-18T23:41:32.147545+0000","last_active":"2024-08-18T23:42:23.557596+0000","last_peered":"2024-08-18T23:42:23.557596+0000","last_clean":"2024-08-18T23:42:23.557596+0000","last_became_active":"2024-08-18T23:41:32.147238+0000","last_became_peered":"2024-08-18T23:41:32.147238+0000","last_unstale":"2024-08-18T23:42:23.557596+0000","last_undegraded":"2024-08-18T23:42:23.557596+0000","last_fullsized":"2024-08-18T23:42:23.557596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:12:13.589535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319674+0000","last_change":"2024-08-18T23:41:30.120650+0000","last_active":"2024-08-18T23:42:23.319674+0000","last_peered":"2024-08-18T23:42:23.319674+0000","last_clean":"2024-08-18T23:42:23.319674+0000","last_became_active":"2024-08-18T23:41:30.120196+0000","last_became_peered":"2024-08-18T23:41:30.120196+0000","last_unstale":"2024-08-18T23:42:23.319674+0000","last_undegraded":"2024-08-18T23:42:23.319674+0000","last_fullsized":"2024-08-18T23:42:23.319674+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:40:23.370088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317120+0000","last_change":"2024-08-18T23:40:22.013519+0000","last_active":"2024-08-18T23:42:23.317120+0000","last_peered":"2024-08-18T23:42:23.317120+0000","last_clean":"2024-08-18T23:42:23.317120+0000","last_became_active":"2024-08-18T23:40:22.012986+0000","last_became_peered":"2024-08-18T23:40:22.012986+0000","last_unstale":"2024-08-18T23:42:23.317120+0000","last_undegraded":"2024-08-18T23:42:23.317120+0000","last_fullsized":"2024-08-18T23:42:23.317120+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_clean_scrub_stamp":"2024-08-18T23:37:22.388424+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:19.874356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.332800+0000","last_change":"2024-08-18T23:42:23.332800+0000","last_active":"2024-08-18T23:42:23.332800+0000","last_peered":"2024-08-18T23:42:23.332800+0000","last_clean":"2024-08-18T23:42:23.332800+0000","last_became_active":"2024-08-18T23:42:23.326441+0000","last_became_peered":"2024-08-18T23:42:23.326441+0000","last_unstale":"2024-08-18T23:42:23.332800+0000","last_undegraded":"2024-08-18T23:42:23.332800+0000","last_fullsized":"2024-08-18T23:42:23.332800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:13:58.243518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317150+0000","last_change":"2024-08-18T23:41:32.139907+0000","last_active":"2024-08-18T23:42:23.317150+0000","last_peered":"2024-08-18T23:42:23.317150+0000","last_clean":"2024-08-18T23:42:23.317150+0000","last_became_active":"2024-08-18T23:41:32.137956+0000","last_became_peered":"2024-08-18T23:41:32.137956+0000","last_unstale":"2024-08-18T23:42:23.317150+0000","last_undegraded":"2024-08-18T23:42:23.317150+0000","last_fullsized":"2024-08-18T23:42:23.317150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:10:13.223328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.330746+0000","last_change":"2024-08-18T23:42:23.330062+0000","last_active":"2024-08-18T23:42:23.330746+0000","last_peered":"2024-08-18T23:42:23.330746+0000","last_clean":"2024-08-18T23:42:23.330746+0000","last_became_active":"2024-08-18T23:42:23.323196+0000","last_became_peered":"2024-08-18T23:42:23.323196+0000","last_unstale":"2024-08-18T23:42:23.330746+0000","last_undegraded":"2024-08-18T23:42:23.330746+0000","last_fullsized":"2024-08-18T23:42:23.330746+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:53:29.059542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315093+0000","last_change":"2024-08-18T23:41:39.185917+0000","last_active":"2024-08-18T23:42:23.315093+0000","last_peered":"2024-08-18T23:42:23.315093+0000","last_clean":"2024-08-18T23:42:23.315093+0000","last_became_active":"2024-08-18T23:41:39.185427+0000","last_became_peered":"2024-08-18T23:41:39.185427+0000","last_unstale":"2024-08-18T23:42:23.315093+0000","last_undegraded":"2024-08-18T23:42:23.315093+0000","last_fullsized":"2024-08-18T23:42:23.315093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:52:13.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317336+0000","last_change":"2024-08-18T23:41:32.147530+0000","last_active":"2024-08-18T23:42:23.317336+0000","last_peered":"2024-08-18T23:42:23.317336+0000","last_clean":"2024-08-18T23:42:23.317336+0000","last_became_active":"2024-08-18T23:41:32.147216+0000","last_became_peered":"2024-08-18T23:41:32.147216+0000","last_unstale":"2024-08-18T23:42:23.317336+0000","last_undegraded":"2024-08-18T23:42:23.317336+0000","last_fullsized":"2024-08-18T23:42:23.317336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:04:06.421490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323538+0000","last_change":"2024-08-18T23:41:30.121307+0000","last_active":"2024-08-18T23:42:23.323538+0000","last_peered":"2024-08-18T23:42:23.323538+0000","last_clean":"2024-08-18T23:42:23.323538+0000","last_became_active":"2024-08-18T23:41:30.120932+0000","last_became_peered":"2024-08-18T23:41:30.120932+0000","last_unstale":"2024-08-18T23:42:23.323538+0000","last_undegraded":"2024-08-18T23:42:23.323538+0000","last_fullsized":"2024-08-18T23:42:23.323538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:06:07.742880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316371+0000","last_change":"2024-08-18T23:41:39.184647+0000","last_active":"2024-08-18T23:42:23.316371+0000","last_peered":"2024-08-18T23:42:23.316371+0000","last_clean":"2024-08-18T23:42:23.316371+0000","last_became_active":"2024-08-18T23:41:39.184251+0000","last_became_peered":"2024-08-18T23:41:39.184251+0000","last_unstale":"2024-08-18T23:42:23.316371+0000","last_undegraded":"2024-08-18T23:42:23.316371+0000","last_fullsized":"2024-08-18T23:42:23.316371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:55.200801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317327+0000","last_change":"2024-08-18T23:41:32.137007+0000","last_active":"2024-08-18T23:42:23.317327+0000","last_peered":"2024-08-18T23:42:23.317327+0000","last_clean":"2024-08-18T23:42:23.317327+0000","last_became_active":"2024-08-18T23:41:32.136684+0000","last_became_peered":"2024-08-18T23:41:32.136684+0000","last_unstale":"2024-08-18T23:42:23.317327+0000","last_undegraded":"2024-08-18T23:42:23.317327+0000","last_fullsized":"2024-08-18T23:42:23.317327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:21:26.584303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314514+0000","last_change":"2024-08-18T23:41:30.121183+0000","last_active":"2024-08-18T23:42:23.314514+0000","last_peered":"2024-08-18T23:42:23.314514+0000","last_clean":"2024-08-18T23:42:23.314514+0000","last_became_active":"2024-08-18T23:41:30.120847+0000","last_became_peered":"2024-08-18T23:41:30.120847+0000","last_unstale":"2024-08-18T23:42:23.314514+0000","last_undegraded":"2024-08-18T23:42:23.314514+0000","last_fullsized":"2024-08-18T23:42:23.314514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:51.025821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320528+0000","last_change":"2024-08-18T23:41:39.183375+0000","last_active":"2024-08-18T23:42:23.320528+0000","last_peered":"2024-08-18T23:42:23.320528+0000","last_clean":"2024-08-18T23:42:23.320528+0000","last_became_active":"2024-08-18T23:41:39.182339+0000","last_became_peered":"2024-08-18T23:41:39.182339+0000","last_unstale":"2024-08-18T23:42:23.320528+0000","last_undegraded":"2024-08-18T23:42:23.320528+0000","last_fullsized":"2024-08-18T23:42:23.320528+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:26.119056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318323+0000","last_change":"2024-08-18T23:41:32.140748+0000","last_active":"2024-08-18T23:42:23.318323+0000","last_peered":"2024-08-18T23:42:23.318323+0000","last_clean":"2024-08-18T23:42:23.318323+0000","last_became_active":"2024-08-18T23:41:32.140306+0000","last_became_peered":"2024-08-18T23:41:32.140306+0000","last_unstale":"2024-08-18T23:42:23.318323+0000","last_undegraded":"2024-08-18T23:42:23.318323+0000","last_fullsized":"2024-08-18T23:42:23.318323+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:23:02.371529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324632+0000","last_change":"2024-08-18T23:41:30.121617+0000","last_active":"2024-08-18T23:42:23.324632+0000","last_peered":"2024-08-18T23:42:23.324632+0000","last_clean":"2024-08-18T23:42:23.324632+0000","last_became_active":"2024-08-18T23:41:30.121257+0000","last_became_peered":"2024-08-18T23:41:30.121257+0000","last_unstale":"2024-08-18T23:42:23.324632+0000","last_undegraded":"2024-08-18T23:42:23.324632+0000","last_fullsized":"2024-08-18T23:42:23.324632+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:23:42.091553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320637+0000","last_change":"2024-08-18T23:41:39.183833+0000","last_active":"2024-08-18T23:42:23.320637+0000","last_peered":"2024-08-18T23:42:23.320637+0000","last_clean":"2024-08-18T23:42:23.320637+0000","last_became_active":"2024-08-18T23:41:39.183176+0000","last_became_peered":"2024-08-18T23:41:39.183176+0000","last_unstale":"2024-08-18T23:42:23.320637+0000","last_undegraded":"2024-08-18T23:42:23.320637+0000","last_fullsized":"2024-08-18T23:42:23.320637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:36:09.827378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322357+0000","last_change":"2024-08-18T23:41:32.147507+0000","last_active":"2024-08-18T23:42:23.322357+0000","last_peered":"2024-08-18T23:42:23.322357+0000","last_clean":"2024-08-18T23:42:23.322357+0000","last_became_active":"2024-08-18T23:41:32.146042+0000","last_became_peered":"2024-08-18T23:41:32.146042+0000","last_unstale":"2024-08-18T23:42:23.322357+0000","last_undegraded":"2024-08-18T23:42:23.322357+0000","last_fullsized":"2024-08-18T23:42:23.322357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:48.941442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323009+0000","last_change":"2024-08-18T23:41:30.137755+0000","last_active":"2024-08-18T23:42:23.323009+0000","last_peered":"2024-08-18T23:42:23.323009+0000","last_clean":"2024-08-18T23:42:23.323009+0000","last_became_active":"2024-08-18T23:41:30.137194+0000","last_became_peered":"2024-08-18T23:41:30.137194+0000","last_unstale":"2024-08-18T23:42:23.323009+0000","last_undegraded":"2024-08-18T23:42:23.323009+0000","last_fullsized":"2024-08-18T23:42:23.323009+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:15:49.681405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314506+0000","last_change":"2024-08-18T23:41:39.183986+0000","last_active":"2024-08-18T23:42:23.314506+0000","last_peered":"2024-08-18T23:42:23.314506+0000","last_clean":"2024-08-18T23:42:23.314506+0000","last_became_active":"2024-08-18T23:41:39.183384+0000","last_became_peered":"2024-08-18T23:41:39.183384+0000","last_unstale":"2024-08-18T23:42:23.314506+0000","last_undegraded":"2024-08-18T23:42:23.314506+0000","last_fullsized":"2024-08-18T23:42:23.314506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:04:28.920804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316556+0000","last_change":"2024-08-18T23:41:32.134729+0000","last_active":"2024-08-18T23:42:23.316556+0000","last_peered":"2024-08-18T23:42:23.316556+0000","last_clean":"2024-08-18T23:42:23.316556+0000","last_became_active":"2024-08-18T23:41:32.134366+0000","last_became_peered":"2024-08-18T23:41:32.134366+0000","last_unstale":"2024-08-18T23:42:23.316556+0000","last_undegraded":"2024-08-18T23:42:23.316556+0000","last_fullsized":"2024-08-18T23:42:23.316556+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:38.610561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":103,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:43:08.357212+0000","last_change":"2024-08-18T23:41:30.126443+0000","last_active":"2024-08-18T23:43:08.357212+0000","last_peered":"2024-08-18T23:43:08.357212+0000","last_clean":"2024-08-18T23:43:08.357212+0000","last_became_active":"2024-08-18T23:41:30.124264+0000","last_became_peered":"2024-08-18T23:41:30.124264+0000","last_unstale":"2024-08-18T23:43:08.357212+0000","last_undegraded":"2024-08-18T23:43:08.357212+0000","last_fullsized":"2024-08-18T23:43:08.357212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:43.727114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319340+0000","last_change":"2024-08-18T23:41:39.192164+0000","last_active":"2024-08-18T23:42:23.319340+0000","last_peered":"2024-08-18T23:42:23.319340+0000","last_clean":"2024-08-18T23:42:23.319340+0000","last_became_active":"2024-08-18T23:41:39.191833+0000","last_became_peered":"2024-08-18T23:41:39.191833+0000","last_unstale":"2024-08-18T23:42:23.319340+0000","last_undegraded":"2024-08-18T23:42:23.319340+0000","last_fullsized":"2024-08-18T23:42:23.319340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:33:50.243930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320662+0000","last_change":"2024-08-18T23:42:23.320662+0000","last_active":"2024-08-18T23:42:23.320662+0000","last_peered":"2024-08-18T23:42:23.320662+0000","last_clean":"2024-08-18T23:42:23.320662+0000","last_became_active":"2024-08-18T23:42:23.320249+0000","last_became_peered":"2024-08-18T23:42:23.320249+0000","last_unstale":"2024-08-18T23:42:23.320662+0000","last_undegraded":"2024-08-18T23:42:23.320662+0000","last_fullsized":"2024-08-18T23:42:23.320662+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:03:17.121915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322332+0000","last_change":"2024-08-18T23:41:30.124071+0000","last_active":"2024-08-18T23:42:23.322332+0000","last_peered":"2024-08-18T23:42:23.322332+0000","last_clean":"2024-08-18T23:42:23.322332+0000","last_became_active":"2024-08-18T23:41:30.123118+0000","last_became_peered":"2024-08-18T23:41:30.123118+0000","last_unstale":"2024-08-18T23:42:23.322332+0000","last_undegraded":"2024-08-18T23:42:23.322332+0000","last_fullsized":"2024-08-18T23:42:23.322332+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:21:39.015187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319331+0000","last_change":"2024-08-18T23:41:39.184038+0000","last_active":"2024-08-18T23:42:23.319331+0000","last_peered":"2024-08-18T23:42:23.319331+0000","last_clean":"2024-08-18T23:42:23.319331+0000","last_became_active":"2024-08-18T23:41:39.183611+0000","last_became_peered":"2024-08-18T23:41:39.183611+0000","last_unstale":"2024-08-18T23:42:23.319331+0000","last_undegraded":"2024-08-18T23:42:23.319331+0000","last_fullsized":"2024-08-18T23:42:23.319331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:32:26.029151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314264+0000","last_change":"2024-08-18T23:41:32.143631+0000","last_active":"2024-08-18T23:42:23.314264+0000","last_peered":"2024-08-18T23:42:23.314264+0000","last_clean":"2024-08-18T23:42:23.314264+0000","last_became_active":"2024-08-18T23:41:32.143350+0000","last_became_peered":"2024-08-18T23:41:32.143350+0000","last_unstale":"2024-08-18T23:42:23.314264+0000","last_undegraded":"2024-08-18T23:42:23.314264+0000","last_fullsized":"2024-08-18T23:42:23.314264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:41:23.713467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319866+0000","last_change":"2024-08-18T23:41:30.123383+0000","last_active":"2024-08-18T23:42:23.319866+0000","last_peered":"2024-08-18T23:42:23.319866+0000","last_clean":"2024-08-18T23:42:23.319866+0000","last_became_active":"2024-08-18T23:41:30.123037+0000","last_became_peered":"2024-08-18T23:41:30.123037+0000","last_unstale":"2024-08-18T23:42:23.319866+0000","last_undegraded":"2024-08-18T23:42:23.319866+0000","last_fullsized":"2024-08-18T23:42:23.319866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:27.702478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323158+0000","last_change":"2024-08-18T23:41:39.184302+0000","last_active":"2024-08-18T23:42:23.323158+0000","last_peered":"2024-08-18T23:42:23.323158+0000","last_clean":"2024-08-18T23:42:23.323158+0000","last_became_active":"2024-08-18T23:41:39.183172+0000","last_became_peered":"2024-08-18T23:41:39.183172+0000","last_unstale":"2024-08-18T23:42:23.323158+0000","last_undegraded":"2024-08-18T23:42:23.323158+0000","last_fullsized":"2024-08-18T23:42:23.323158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:23:26.655865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324579+0000","last_change":"2024-08-18T23:41:32.137426+0000","last_active":"2024-08-18T23:42:23.324579+0000","last_peered":"2024-08-18T23:42:23.324579+0000","last_clean":"2024-08-18T23:42:23.324579+0000","last_became_active":"2024-08-18T23:41:32.137106+0000","last_became_peered":"2024-08-18T23:41:32.137106+0000","last_unstale":"2024-08-18T23:42:23.324579+0000","last_undegraded":"2024-08-18T23:42:23.324579+0000","last_fullsized":"2024-08-18T23:42:23.324579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:34:51.889663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'32","reported_seq":199,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:43:09.357617+0000","last_change":"2024-08-18T23:41:30.124320+0000","last_active":"2024-08-18T23:43:09.357617+0000","last_peered":"2024-08-18T23:43:09.357617+0000","last_clean":"2024-08-18T23:43:09.357617+0000","last_became_active":"2024-08-18T23:41:30.123979+0000","last_became_peered":"2024-08-18T23:41:30.123979+0000","last_unstale":"2024-08-18T23:43:09.357617+0000","last_undegraded":"2024-08-18T23:43:09.357617+0000","last_fullsized":"2024-08-18T23:43:09.357617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:31:03.689720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62996,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":133,"num_read_kb":128,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319063+0000","last_change":"2024-08-18T23:41:39.190481+0000","last_active":"2024-08-18T23:42:23.319063+0000","last_peered":"2024-08-18T23:42:23.319063+0000","last_clean":"2024-08-18T23:42:23.319063+0000","last_became_active":"2024-08-18T23:41:39.190076+0000","last_became_peered":"2024-08-18T23:41:39.190076+0000","last_unstale":"2024-08-18T23:42:23.319063+0000","last_undegraded":"2024-08-18T23:42:23.319063+0000","last_fullsized":"2024-08-18T23:42:23.319063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:47:39.313142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315607+0000","last_change":"2024-08-18T23:41:32.145422+0000","last_active":"2024-08-18T23:42:23.315607+0000","last_peered":"2024-08-18T23:42:23.315607+0000","last_clean":"2024-08-18T23:42:23.315607+0000","last_became_active":"2024-08-18T23:41:32.145047+0000","last_became_peered":"2024-08-18T23:41:32.145047+0000","last_unstale":"2024-08-18T23:42:23.315607+0000","last_undegraded":"2024-08-18T23:42:23.315607+0000","last_fullsized":"2024-08-18T23:42:23.315607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:31.905248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323427+0000","last_change":"2024-08-18T23:41:30.137364+0000","last_active":"2024-08-18T23:42:23.323427+0000","last_peered":"2024-08-18T23:42:23.323427+0000","last_clean":"2024-08-18T23:42:23.323427+0000","last_became_active":"2024-08-18T23:41:30.137004+0000","last_became_peered":"2024-08-18T23:41:30.137004+0000","last_unstale":"2024-08-18T23:42:23.323427+0000","last_undegraded":"2024-08-18T23:42:23.323427+0000","last_fullsized":"2024-08-18T23:42:23.323427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:20:56.392149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.326359+0000","last_change":"2024-08-18T23:42:23.326359+0000","last_active":"2024-08-18T23:42:23.326359+0000","last_peered":"2024-08-18T23:42:23.326359+0000","last_clean":"2024-08-18T23:42:23.326359+0000","last_became_active":"2024-08-18T23:42:23.325918+0000","last_became_peered":"2024-08-18T23:42:23.325918+0000","last_unstale":"2024-08-18T23:42:23.326359+0000","last_undegraded":"2024-08-18T23:42:23.326359+0000","last_fullsized":"2024-08-18T23:42:23.326359+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:01:15.476268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.325691+0000","last_change":"2024-08-18T23:42:23.325691+0000","last_active":"2024-08-18T23:42:23.325691+0000","last_peered":"2024-08-18T23:42:23.325691+0000","last_clean":"2024-08-18T23:42:23.325691+0000","last_became_active":"2024-08-18T23:42:23.325467+0000","last_became_peered":"2024-08-18T23:42:23.325467+0000","last_unstale":"2024-08-18T23:42:23.325691+0000","last_undegraded":"2024-08-18T23:42:23.325691+0000","last_fullsized":"2024-08-18T23:42:23.325691+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:10:26.288880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317472+0000","last_change":"2024-08-18T23:41:30.124430+0000","last_active":"2024-08-18T23:42:23.317472+0000","last_peered":"2024-08-18T23:42:23.317472+0000","last_clean":"2024-08-18T23:42:23.317472+0000","last_became_active":"2024-08-18T23:41:30.123626+0000","last_became_peered":"2024-08-18T23:41:30.123626+0000","last_unstale":"2024-08-18T23:42:23.317472+0000","last_undegraded":"2024-08-18T23:42:23.317472+0000","last_fullsized":"2024-08-18T23:42:23.317472+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:26:18.426010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318912+0000","last_change":"2024-08-18T23:41:39.192765+0000","last_active":"2024-08-18T23:42:23.318912+0000","last_peered":"2024-08-18T23:42:23.318912+0000","last_clean":"2024-08-18T23:42:23.318912+0000","last_became_active":"2024-08-18T23:41:39.192250+0000","last_became_peered":"2024-08-18T23:41:39.192250+0000","last_unstale":"2024-08-18T23:42:23.318912+0000","last_undegraded":"2024-08-18T23:42:23.318912+0000","last_fullsized":"2024-08-18T23:42:23.318912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:51:16.794404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316158+0000","last_change":"2024-08-18T23:41:32.141966+0000","last_active":"2024-08-18T23:42:23.316158+0000","last_peered":"2024-08-18T23:42:23.316158+0000","last_clean":"2024-08-18T23:42:23.316158+0000","last_became_active":"2024-08-18T23:41:32.141561+0000","last_became_peered":"2024-08-18T23:41:32.141561+0000","last_unstale":"2024-08-18T23:42:23.316158+0000","last_undegraded":"2024-08-18T23:42:23.316158+0000","last_fullsized":"2024-08-18T23:42:23.316158+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:20.696089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":99,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:43:08.356388+0000","last_change":"2024-08-18T23:41:30.121598+0000","last_active":"2024-08-18T23:43:08.356388+0000","last_peered":"2024-08-18T23:43:08.356388+0000","last_clean":"2024-08-18T23:43:08.356388+0000","last_became_active":"2024-08-18T23:41:30.120533+0000","last_became_peered":"2024-08-18T23:41:30.120533+0000","last_unstale":"2024-08-18T23:43:08.356388+0000","last_undegraded":"2024-08-18T23:43:08.356388+0000","last_fullsized":"2024-08-18T23:43:08.356388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:06:37.458272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":62,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557461+0000","last_change":"2024-08-18T23:41:39.178877+0000","last_active":"2024-08-18T23:42:23.557461+0000","last_peered":"2024-08-18T23:42:23.557461+0000","last_clean":"2024-08-18T23:42:23.557461+0000","last_became_active":"2024-08-18T23:41:39.178381+0000","last_became_peered":"2024-08-18T23:41:39.178381+0000","last_unstale":"2024-08-18T23:42:23.557461+0000","last_undegraded":"2024-08-18T23:42:23.557461+0000","last_fullsized":"2024-08-18T23:42:23.557461+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:27.275893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316665+0000","last_change":"2024-08-18T23:41:39.188381+0000","last_active":"2024-08-18T23:42:23.316665+0000","last_peered":"2024-08-18T23:42:23.316665+0000","last_clean":"2024-08-18T23:42:23.316665+0000","last_became_active":"2024-08-18T23:41:39.187983+0000","last_became_peered":"2024-08-18T23:41:39.187983+0000","last_unstale":"2024-08-18T23:42:23.316665+0000","last_undegraded":"2024-08-18T23:42:23.316665+0000","last_fullsized":"2024-08-18T23:42:23.316665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:04:45.460576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319101+0000","last_change":"2024-08-18T23:41:32.144637+0000","last_active":"2024-08-18T23:42:23.319101+0000","last_peered":"2024-08-18T23:42:23.319101+0000","last_clean":"2024-08-18T23:42:23.319101+0000","last_became_active":"2024-08-18T23:41:32.144329+0000","last_became_peered":"2024-08-18T23:41:32.144329+0000","last_unstale":"2024-08-18T23:42:23.319101+0000","last_undegraded":"2024-08-18T23:42:23.319101+0000","last_fullsized":"2024-08-18T23:42:23.319101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:29:39.791264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320980+0000","last_change":"2024-08-18T23:41:30.122325+0000","last_active":"2024-08-18T23:42:23.320980+0000","last_peered":"2024-08-18T23:42:23.320980+0000","last_clean":"2024-08-18T23:42:23.320980+0000","last_became_active":"2024-08-18T23:41:30.121480+0000","last_became_peered":"2024-08-18T23:41:30.121480+0000","last_unstale":"2024-08-18T23:42:23.320980+0000","last_undegraded":"2024-08-18T23:42:23.320980+0000","last_fullsized":"2024-08-18T23:42:23.320980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:32:12.810650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.560632+0000","last_change":"2024-08-18T23:42:23.560632+0000","last_active":"2024-08-18T23:42:23.560632+0000","last_peered":"2024-08-18T23:42:23.560632+0000","last_clean":"2024-08-18T23:42:23.560632+0000","last_became_active":"2024-08-18T23:42:23.559920+0000","last_became_peered":"2024-08-18T23:42:23.559920+0000","last_unstale":"2024-08-18T23:42:23.560632+0000","last_undegraded":"2024-08-18T23:42:23.560632+0000","last_fullsized":"2024-08-18T23:42:23.560632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:01:59.580843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557464+0000","last_change":"2024-08-18T23:41:32.145545+0000","last_active":"2024-08-18T23:42:23.557464+0000","last_peered":"2024-08-18T23:42:23.557464+0000","last_clean":"2024-08-18T23:42:23.557464+0000","last_became_active":"2024-08-18T23:41:32.145183+0000","last_became_peered":"2024-08-18T23:41:32.145183+0000","last_unstale":"2024-08-18T23:42:23.557464+0000","last_undegraded":"2024-08-18T23:42:23.557464+0000","last_fullsized":"2024-08-18T23:42:23.557464+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:03:04.111658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314074+0000","last_change":"2024-08-18T23:41:30.118776+0000","last_active":"2024-08-18T23:42:23.314074+0000","last_peered":"2024-08-18T23:42:23.314074+0000","last_clean":"2024-08-18T23:42:23.314074+0000","last_became_active":"2024-08-18T23:41:30.118436+0000","last_became_peered":"2024-08-18T23:41:30.118436+0000","last_unstale":"2024-08-18T23:42:23.314074+0000","last_undegraded":"2024-08-18T23:42:23.314074+0000","last_fullsized":"2024-08-18T23:42:23.314074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:13.980927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558143+0000","last_change":"2024-08-18T23:41:30.125201+0000","last_active":"2024-08-18T23:42:23.558143+0000","last_peered":"2024-08-18T23:42:23.558143+0000","last_clean":"2024-08-18T23:42:23.558143+0000","last_became_active":"2024-08-18T23:41:30.124894+0000","last_became_peered":"2024-08-18T23:41:30.124894+0000","last_unstale":"2024-08-18T23:42:23.558143+0000","last_undegraded":"2024-08-18T23:42:23.558143+0000","last_fullsized":"2024-08-18T23:42:23.558143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:30:17.721829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322555+0000","last_change":"2024-08-18T23:41:32.145452+0000","last_active":"2024-08-18T23:42:23.322555+0000","last_peered":"2024-08-18T23:42:23.322555+0000","last_clean":"2024-08-18T23:42:23.322555+0000","last_became_active":"2024-08-18T23:41:32.145151+0000","last_became_peered":"2024-08-18T23:41:32.145151+0000","last_unstale":"2024-08-18T23:42:23.322555+0000","last_undegraded":"2024-08-18T23:42:23.322555+0000","last_fullsized":"2024-08-18T23:42:23.322555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:08:08.713154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319503+0000","last_change":"2024-08-18T23:41:39.193334+0000","last_active":"2024-08-18T23:42:23.319503+0000","last_peered":"2024-08-18T23:42:23.319503+0000","last_clean":"2024-08-18T23:42:23.319503+0000","last_became_active":"2024-08-18T23:41:39.192924+0000","last_became_peered":"2024-08-18T23:41:39.192924+0000","last_unstale":"2024-08-18T23:42:23.319503+0000","last_undegraded":"2024-08-18T23:42:23.319503+0000","last_fullsized":"2024-08-18T23:42:23.319503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:07.016300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":107,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:43:10.359920+0000","last_change":"2024-08-18T23:41:30.125217+0000","last_active":"2024-08-18T23:43:10.359920+0000","last_peered":"2024-08-18T23:43:10.359920+0000","last_clean":"2024-08-18T23:43:10.359920+0000","last_became_active":"2024-08-18T23:41:30.124889+0000","last_became_peered":"2024-08-18T23:41:30.124889+0000","last_unstale":"2024-08-18T23:43:10.359920+0000","last_undegraded":"2024-08-18T23:43:10.359920+0000","last_fullsized":"2024-08-18T23:43:10.359920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:43.389537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323355+0000","last_change":"2024-08-18T23:41:32.142585+0000","last_active":"2024-08-18T23:42:23.323355+0000","last_peered":"2024-08-18T23:42:23.323355+0000","last_clean":"2024-08-18T23:42:23.323355+0000","last_became_active":"2024-08-18T23:41:32.142207+0000","last_became_peered":"2024-08-18T23:41:32.142207+0000","last_unstale":"2024-08-18T23:42:23.323355+0000","last_undegraded":"2024-08-18T23:42:23.323355+0000","last_fullsized":"2024-08-18T23:42:23.323355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:16:58.755444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.321952+0000","last_change":"2024-08-18T23:41:39.185618+0000","last_active":"2024-08-18T23:42:23.321952+0000","last_peered":"2024-08-18T23:42:23.321952+0000","last_clean":"2024-08-18T23:42:23.321952+0000","last_became_active":"2024-08-18T23:41:39.185222+0000","last_became_peered":"2024-08-18T23:41:39.185222+0000","last_unstale":"2024-08-18T23:42:23.321952+0000","last_undegraded":"2024-08-18T23:42:23.321952+0000","last_fullsized":"2024-08-18T23:42:23.321952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:01:27.290998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318594+0000","last_change":"2024-08-18T23:41:32.140719+0000","last_active":"2024-08-18T23:42:23.318594+0000","last_peered":"2024-08-18T23:42:23.318594+0000","last_clean":"2024-08-18T23:42:23.318594+0000","last_became_active":"2024-08-18T23:41:32.140131+0000","last_became_peered":"2024-08-18T23:41:32.140131+0000","last_unstale":"2024-08-18T23:42:23.318594+0000","last_undegraded":"2024-08-18T23:42:23.318594+0000","last_fullsized":"2024-08-18T23:42:23.318594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:48:26.778094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.326683+0000","last_change":"2024-08-18T23:42:23.326683+0000","last_active":"2024-08-18T23:42:23.326683+0000","last_peered":"2024-08-18T23:42:23.326683+0000","last_clean":"2024-08-18T23:42:23.326683+0000","last_became_active":"2024-08-18T23:42:23.326309+0000","last_became_peered":"2024-08-18T23:42:23.326309+0000","last_unstale":"2024-08-18T23:42:23.326683+0000","last_undegraded":"2024-08-18T23:42:23.326683+0000","last_fullsized":"2024-08-18T23:42:23.326683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:14.320894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.321522+0000","last_change":"2024-08-18T23:41:32.149522+0000","last_active":"2024-08-18T23:42:23.321522+0000","last_peered":"2024-08-18T23:42:23.321522+0000","last_clean":"2024-08-18T23:42:23.321522+0000","last_became_active":"2024-08-18T23:41:32.149182+0000","last_became_peered":"2024-08-18T23:41:32.149182+0000","last_unstale":"2024-08-18T23:42:23.321522+0000","last_undegraded":"2024-08-18T23:42:23.321522+0000","last_fullsized":"2024-08-18T23:42:23.321522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:30:16.945045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316398+0000","last_change":"2024-08-18T23:41:32.149853+0000","last_active":"2024-08-18T23:42:23.316398+0000","last_peered":"2024-08-18T23:42:23.316398+0000","last_clean":"2024-08-18T23:42:23.316398+0000","last_became_active":"2024-08-18T23:41:32.141907+0000","last_became_peered":"2024-08-18T23:41:32.141907+0000","last_unstale":"2024-08-18T23:42:23.316398+0000","last_undegraded":"2024-08-18T23:42:23.316398+0000","last_fullsized":"2024-08-18T23:42:23.316398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:24:37.323567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314554+0000","last_change":"2024-08-18T23:41:32.136179+0000","last_active":"2024-08-18T23:42:23.314554+0000","last_peered":"2024-08-18T23:42:23.314554+0000","last_clean":"2024-08-18T23:42:23.314554+0000","last_became_active":"2024-08-18T23:41:32.135939+0000","last_became_peered":"2024-08-18T23:41:32.135939+0000","last_unstale":"2024-08-18T23:42:23.314554+0000","last_undegraded":"2024-08-18T23:42:23.314554+0000","last_fullsized":"2024-08-18T23:42:23.314554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:15.392061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318883+0000","last_change":"2024-08-18T23:41:32.137509+0000","last_active":"2024-08-18T23:42:23.318883+0000","last_peered":"2024-08-18T23:42:23.318883+0000","last_clean":"2024-08-18T23:42:23.318883+0000","last_became_active":"2024-08-18T23:41:32.136380+0000","last_became_peered":"2024-08-18T23:41:32.136380+0000","last_unstale":"2024-08-18T23:42:23.318883+0000","last_undegraded":"2024-08-18T23:42:23.318883+0000","last_fullsized":"2024-08-18T23:42:23.318883+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:49.776225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322689+0000","last_change":"2024-08-18T23:41:32.139271+0000","last_active":"2024-08-18T23:42:23.322689+0000","last_peered":"2024-08-18T23:42:23.322689+0000","last_clean":"2024-08-18T23:42:23.322689+0000","last_became_active":"2024-08-18T23:41:32.138868+0000","last_became_peered":"2024-08-18T23:41:32.138868+0000","last_unstale":"2024-08-18T23:42:23.322689+0000","last_undegraded":"2024-08-18T23:42:23.322689+0000","last_fullsized":"2024-08-18T23:42:23.322689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:49:27.890857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557875+0000","last_change":"2024-08-18T23:41:32.144460+0000","last_active":"2024-08-18T23:42:23.557875+0000","last_peered":"2024-08-18T23:42:23.557875+0000","last_clean":"2024-08-18T23:42:23.557875+0000","last_became_active":"2024-08-18T23:41:32.144178+0000","last_became_peered":"2024-08-18T23:41:32.144178+0000","last_unstale":"2024-08-18T23:42:23.557875+0000","last_undegraded":"2024-08-18T23:42:23.557875+0000","last_fullsized":"2024-08-18T23:42:23.557875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:56.252215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320708+0000","last_change":"2024-08-18T23:41:32.147973+0000","last_active":"2024-08-18T23:42:23.320708+0000","last_peered":"2024-08-18T23:42:23.320708+0000","last_clean":"2024-08-18T23:42:23.320708+0000","last_became_active":"2024-08-18T23:41:32.147621+0000","last_became_peered":"2024-08-18T23:41:32.147621+0000","last_unstale":"2024-08-18T23:42:23.320708+0000","last_undegraded":"2024-08-18T23:42:23.320708+0000","last_fullsized":"2024-08-18T23:42:23.320708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:14.176569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557525+0000","last_change":"2024-08-18T23:41:32.146029+0000","last_active":"2024-08-18T23:42:23.557525+0000","last_peered":"2024-08-18T23:42:23.557525+0000","last_clean":"2024-08-18T23:42:23.557525+0000","last_became_active":"2024-08-18T23:41:32.145584+0000","last_became_peered":"2024-08-18T23:41:32.145584+0000","last_unstale":"2024-08-18T23:42:23.557525+0000","last_undegraded":"2024-08-18T23:42:23.557525+0000","last_fullsized":"2024-08-18T23:42:23.557525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:25:33.884906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315440+0000","last_change":"2024-08-18T23:41:32.145950+0000","last_active":"2024-08-18T23:42:23.315440+0000","last_peered":"2024-08-18T23:42:23.315440+0000","last_clean":"2024-08-18T23:42:23.315440+0000","last_became_active":"2024-08-18T23:41:32.145618+0000","last_became_peered":"2024-08-18T23:41:32.145618+0000","last_unstale":"2024-08-18T23:42:23.315440+0000","last_undegraded":"2024-08-18T23:42:23.315440+0000","last_fullsized":"2024-08-18T23:42:23.315440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:05:58.889240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314395+0000","last_change":"2024-08-18T23:41:32.130684+0000","last_active":"2024-08-18T23:42:23.314395+0000","last_peered":"2024-08-18T23:42:23.314395+0000","last_clean":"2024-08-18T23:42:23.314395+0000","last_became_active":"2024-08-18T23:41:32.130369+0000","last_became_peered":"2024-08-18T23:41:32.130369+0000","last_unstale":"2024-08-18T23:42:23.314395+0000","last_undegraded":"2024-08-18T23:42:23.314395+0000","last_fullsized":"2024-08-18T23:42:23.314395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:22.537018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320123+0000","last_change":"2024-08-18T23:41:32.137291+0000","last_active":"2024-08-18T23:42:23.320123+0000","last_peered":"2024-08-18T23:42:23.320123+0000","last_clean":"2024-08-18T23:42:23.320123+0000","last_became_active":"2024-08-18T23:41:32.136977+0000","last_became_peered":"2024-08-18T23:41:32.136977+0000","last_unstale":"2024-08-18T23:42:23.320123+0000","last_undegraded":"2024-08-18T23:42:23.320123+0000","last_fullsized":"2024-08-18T23:42:23.320123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:18:45.544760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322146+0000","last_change":"2024-08-18T23:41:32.144434+0000","last_active":"2024-08-18T23:42:23.322146+0000","last_peered":"2024-08-18T23:42:23.322146+0000","last_clean":"2024-08-18T23:42:23.322146+0000","last_became_active":"2024-08-18T23:41:32.144119+0000","last_became_peered":"2024-08-18T23:41:32.144119+0000","last_unstale":"2024-08-18T23:42:23.322146+0000","last_undegraded":"2024-08-18T23:42:23.322146+0000","last_fullsized":"2024-08-18T23:42:23.322146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:50:54.528630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319436+0000","last_change":"2024-08-18T23:41:32.135034+0000","last_active":"2024-08-18T23:42:23.319436+0000","last_peered":"2024-08-18T23:42:23.319436+0000","last_clean":"2024-08-18T23:42:23.319436+0000","last_became_active":"2024-08-18T23:41:32.134700+0000","last_became_peered":"2024-08-18T23:41:32.134700+0000","last_unstale":"2024-08-18T23:42:23.319436+0000","last_undegraded":"2024-08-18T23:42:23.319436+0000","last_fullsized":"2024-08-18T23:42:23.319436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:06:31.263114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319688+0000","last_change":"2024-08-18T23:41:32.141191+0000","last_active":"2024-08-18T23:42:23.319688+0000","last_peered":"2024-08-18T23:42:23.319688+0000","last_clean":"2024-08-18T23:42:23.319688+0000","last_became_active":"2024-08-18T23:41:32.140901+0000","last_became_peered":"2024-08-18T23:41:32.140901+0000","last_unstale":"2024-08-18T23:42:23.319688+0000","last_undegraded":"2024-08-18T23:42:23.319688+0000","last_fullsized":"2024-08-18T23:42:23.319688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:24.849985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314798+0000","last_change":"2024-08-18T23:41:32.142543+0000","last_active":"2024-08-18T23:42:23.314798+0000","last_peered":"2024-08-18T23:42:23.314798+0000","last_clean":"2024-08-18T23:42:23.314798+0000","last_became_active":"2024-08-18T23:41:32.142084+0000","last_became_peered":"2024-08-18T23:41:32.142084+0000","last_unstale":"2024-08-18T23:42:23.314798+0000","last_undegraded":"2024-08-18T23:42:23.314798+0000","last_fullsized":"2024-08-18T23:42:23.314798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:54:18.200016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316269+0000","last_change":"2024-08-18T23:41:32.148968+0000","last_active":"2024-08-18T23:42:23.316269+0000","last_peered":"2024-08-18T23:42:23.316269+0000","last_clean":"2024-08-18T23:42:23.316269+0000","last_became_active":"2024-08-18T23:41:32.148625+0000","last_became_peered":"2024-08-18T23:41:32.148625+0000","last_unstale":"2024-08-18T23:42:23.316269+0000","last_undegraded":"2024-08-18T23:42:23.316269+0000","last_fullsized":"2024-08-18T23:42:23.316269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:11:40.172139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316345+0000","last_change":"2024-08-18T23:41:32.133478+0000","last_active":"2024-08-18T23:42:23.316345+0000","last_peered":"2024-08-18T23:42:23.316345+0000","last_clean":"2024-08-18T23:42:23.316345+0000","last_became_active":"2024-08-18T23:41:32.133149+0000","last_became_peered":"2024-08-18T23:41:32.133149+0000","last_unstale":"2024-08-18T23:42:23.316345+0000","last_undegraded":"2024-08-18T23:42:23.316345+0000","last_fullsized":"2024-08-18T23:42:23.316345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:57:53.713309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314293+0000","last_change":"2024-08-18T23:41:32.138206+0000","last_active":"2024-08-18T23:42:23.314293+0000","last_peered":"2024-08-18T23:42:23.314293+0000","last_clean":"2024-08-18T23:42:23.314293+0000","last_became_active":"2024-08-18T23:41:32.137903+0000","last_became_peered":"2024-08-18T23:41:32.137903+0000","last_unstale":"2024-08-18T23:42:23.314293+0000","last_undegraded":"2024-08-18T23:42:23.314293+0000","last_fullsized":"2024-08-18T23:42:23.314293+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:18:48.600658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319090+0000","last_change":"2024-08-18T23:41:32.136303+0000","last_active":"2024-08-18T23:42:23.319090+0000","last_peered":"2024-08-18T23:42:23.319090+0000","last_clean":"2024-08-18T23:42:23.319090+0000","last_became_active":"2024-08-18T23:41:32.134628+0000","last_became_peered":"2024-08-18T23:41:32.134628+0000","last_unstale":"2024-08-18T23:42:23.319090+0000","last_undegraded":"2024-08-18T23:42:23.319090+0000","last_fullsized":"2024-08-18T23:42:23.319090+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:28:51.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317055+0000","last_change":"2024-08-18T23:41:32.137512+0000","last_active":"2024-08-18T23:42:23.317055+0000","last_peered":"2024-08-18T23:42:23.317055+0000","last_clean":"2024-08-18T23:42:23.317055+0000","last_became_active":"2024-08-18T23:41:32.137150+0000","last_became_peered":"2024-08-18T23:41:32.137150+0000","last_unstale":"2024-08-18T23:42:23.317055+0000","last_undegraded":"2024-08-18T23:42:23.317055+0000","last_fullsized":"2024-08-18T23:42:23.317055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:44:08.020780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558680+0000","last_change":"2024-08-18T23:41:32.139229+0000","last_active":"2024-08-18T23:42:23.558680+0000","last_peered":"2024-08-18T23:42:23.558680+0000","last_clean":"2024-08-18T23:42:23.558680+0000","last_became_active":"2024-08-18T23:41:32.138864+0000","last_became_peered":"2024-08-18T23:41:32.138864+0000","last_unstale":"2024-08-18T23:42:23.558680+0000","last_undegraded":"2024-08-18T23:42:23.558680+0000","last_fullsized":"2024-08-18T23:42:23.558680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:54:21.701750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316215+0000","last_change":"2024-08-18T23:41:32.148727+0000","last_active":"2024-08-18T23:42:23.316215+0000","last_peered":"2024-08-18T23:42:23.316215+0000","last_clean":"2024-08-18T23:42:23.316215+0000","last_became_active":"2024-08-18T23:41:32.146828+0000","last_became_peered":"2024-08-18T23:41:32.146828+0000","last_unstale":"2024-08-18T23:42:23.316215+0000","last_undegraded":"2024-08-18T23:42:23.316215+0000","last_fullsized":"2024-08-18T23:42:23.316215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:51:49.175150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557445+0000","last_change":"2024-08-18T23:41:32.144796+0000","last_active":"2024-08-18T23:42:23.557445+0000","last_peered":"2024-08-18T23:42:23.557445+0000","last_clean":"2024-08-18T23:42:23.557445+0000","last_became_active":"2024-08-18T23:41:32.141931+0000","last_became_peered":"2024-08-18T23:41:32.141931+0000","last_unstale":"2024-08-18T23:42:23.557445+0000","last_undegraded":"2024-08-18T23:42:23.557445+0000","last_fullsized":"2024-08-18T23:42:23.557445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:48:08.428845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316954+0000","last_change":"2024-08-18T23:41:32.135265+0000","last_active":"2024-08-18T23:42:23.316954+0000","last_peered":"2024-08-18T23:42:23.316954+0000","last_clean":"2024-08-18T23:42:23.316954+0000","last_became_active":"2024-08-18T23:41:32.134952+0000","last_became_peered":"2024-08-18T23:41:32.134952+0000","last_unstale":"2024-08-18T23:42:23.316954+0000","last_undegraded":"2024-08-18T23:42:23.316954+0000","last_fullsized":"2024-08-18T23:42:23.316954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:35:14.099536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558127+0000","last_change":"2024-08-18T23:41:32.135941+0000","last_active":"2024-08-18T23:42:23.558127+0000","last_peered":"2024-08-18T23:42:23.558127+0000","last_clean":"2024-08-18T23:42:23.558127+0000","last_became_active":"2024-08-18T23:41:32.135623+0000","last_became_peered":"2024-08-18T23:41:32.135623+0000","last_unstale":"2024-08-18T23:42:23.558127+0000","last_undegraded":"2024-08-18T23:42:23.558127+0000","last_fullsized":"2024-08-18T23:42:23.558127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:43:36.504738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324078+0000","last_change":"2024-08-18T23:41:32.141643+0000","last_active":"2024-08-18T23:42:23.324078+0000","last_peered":"2024-08-18T23:42:23.324078+0000","last_clean":"2024-08-18T23:42:23.324078+0000","last_became_active":"2024-08-18T23:41:32.141301+0000","last_became_peered":"2024-08-18T23:41:32.141301+0000","last_unstale":"2024-08-18T23:42:23.324078+0000","last_undegraded":"2024-08-18T23:42:23.324078+0000","last_fullsized":"2024-08-18T23:42:23.324078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:23:53.100740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318979+0000","last_change":"2024-08-18T23:41:32.146730+0000","last_active":"2024-08-18T23:42:23.318979+0000","last_peered":"2024-08-18T23:42:23.318979+0000","last_clean":"2024-08-18T23:42:23.318979+0000","last_became_active":"2024-08-18T23:41:32.146252+0000","last_became_peered":"2024-08-18T23:41:32.146252+0000","last_unstale":"2024-08-18T23:42:23.318979+0000","last_undegraded":"2024-08-18T23:42:23.318979+0000","last_fullsized":"2024-08-18T23:42:23.318979+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:41.481761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316570+0000","last_change":"2024-08-18T23:41:32.149569+0000","last_active":"2024-08-18T23:42:23.316570+0000","last_peered":"2024-08-18T23:42:23.316570+0000","last_clean":"2024-08-18T23:42:23.316570+0000","last_became_active":"2024-08-18T23:41:32.149277+0000","last_became_peered":"2024-08-18T23:41:32.149277+0000","last_unstale":"2024-08-18T23:42:23.316570+0000","last_undegraded":"2024-08-18T23:42:23.316570+0000","last_fullsized":"2024-08-18T23:42:23.316570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:34:00.547909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316678+0000","last_change":"2024-08-18T23:41:32.137889+0000","last_active":"2024-08-18T23:42:23.316678+0000","last_peered":"2024-08-18T23:42:23.316678+0000","last_clean":"2024-08-18T23:42:23.316678+0000","last_became_active":"2024-08-18T23:41:32.137578+0000","last_became_peered":"2024-08-18T23:41:32.137578+0000","last_unstale":"2024-08-18T23:42:23.316678+0000","last_undegraded":"2024-08-18T23:42:23.316678+0000","last_fullsized":"2024-08-18T23:42:23.316678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:18:09.987838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319506+0000","last_change":"2024-08-18T23:41:32.142084+0000","last_active":"2024-08-18T23:42:23.319506+0000","last_peered":"2024-08-18T23:42:23.319506+0000","last_clean":"2024-08-18T23:42:23.319506+0000","last_became_active":"2024-08-18T23:41:32.141670+0000","last_became_peered":"2024-08-18T23:41:32.141670+0000","last_unstale":"2024-08-18T23:42:23.319506+0000","last_undegraded":"2024-08-18T23:42:23.319506+0000","last_fullsized":"2024-08-18T23:42:23.319506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:03.990351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67386,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":401,"num_read_kb":270,"num_write":167,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":202158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514530,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":505014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776166,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28688,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716464,"statfs":{"total":95995035648,"available":95965659136,"internally_reserved":0,"allocated":1712128,"data_stored":1155668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939691,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":509346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168624,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":509266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397555,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36936,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708216,"statfs":{"total":95995035648,"available":95957213184,"internally_reserved":0,"allocated":1630208,"data_stored":1101468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561080,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":500426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790013,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1092570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953537,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":506048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":500620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":564490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":563332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":501814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T23:43:15.286 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-18T23:43:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:15 smithi078 ceph-mon[18440]: pgmap v316: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-18T23:43:15.540 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:15.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:15 smithi123 ceph-mon[21808]: pgmap v316: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-18T23:43:15.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:15 smithi161 ceph-mon[22027]: pgmap v316: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-18T23:43:16.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.022+0000 7f654fd6e640 1 -- 172.21.15.78:0/3290291958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6548100b50 msgr2=0x7f6548100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:16.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.022+0000 7f654fd6e640 1 --2- 172.21.15.78:0/3290291958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6548100b50 0x7f6548100f30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6540009900 tx=0x7f654002a480 comp rx=0 tx=0).stop 2024-08-18T23:43:16.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.023+0000 7f654fd6e640 1 -- 172.21.15.78:0/3290291958 shutdown_connections 2024-08-18T23:43:16.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.023+0000 7f654fd6e640 1 --2- 172.21.15.78:0/3290291958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6548102c20 0x7f654810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:16.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.023+0000 7f654fd6e640 1 --2- 172.21.15.78:0/3290291958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65481022a0 0x7f65481026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:16.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.023+0000 7f654fd6e640 1 --2- 172.21.15.78:0/3290291958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6548100b50 0x7f6548100f30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:16.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.023+0000 7f654fd6e640 1 -- 172.21.15.78:0/3290291958 >> 172.21.15.78:0/3290291958 conn(0x7f65480fc500 msgr2=0x7f65480fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:16.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.023+0000 7f654fd6e640 1 -- 172.21.15.78:0/3290291958 shutdown_connections 2024-08-18T23:43:16.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.023+0000 7f654fd6e640 1 -- 172.21.15.78:0/3290291958 wait complete. 2024-08-18T23:43:16.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.023+0000 7f654fd6e640 1 Processor -- start 2024-08-18T23:43:16.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.023+0000 7f654fd6e640 1 -- start start 2024-08-18T23:43:16.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.023+0000 7f654fd6e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6548100b50 0x7f654819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:16.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.023+0000 7f654fd6e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65481022a0 0x7f654819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:16.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.023+0000 7f654fd6e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6548102c20 0x7f654819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:16.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.023+0000 7f654fd6e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6548110cd0 con 0x7f6548102c20 2024-08-18T23:43:16.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.023+0000 7f654fd6e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6548110b50 con 0x7f6548100b50 2024-08-18T23:43:16.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.023+0000 7f654fd6e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6548110e50 con 0x7f65481022a0 2024-08-18T23:43:16.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.023+0000 7f654e30b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6548102c20 0x7f654819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:16.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.024+0000 7f654e30b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6548102c20 0x7f654819ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51772/0 (socket says 172.21.15.78:51772) 2024-08-18T23:43:16.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.024+0000 7f654e30b640 1 -- 172.21.15.78:0/3139282285 learned_addr learned my addr 172.21.15.78:0/3139282285 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:16.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.024+0000 7f654db0a640 1 --2- 172.21.15.78:0/3139282285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6548100b50 0x7f654819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:16.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.024+0000 7f654e30b640 1 -- 172.21.15.78:0/3139282285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65481022a0 msgr2=0x7f654819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:16.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.024+0000 7f654e30b640 1 --2- 172.21.15.78:0/3139282285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65481022a0 0x7f654819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:16.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.024+0000 7f654e30b640 1 -- 172.21.15.78:0/3139282285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6548100b50 msgr2=0x7f654819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.024+0000 7f654e30b640 1 --2- 172.21.15.78:0/3139282285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6548100b50 0x7f654819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.024+0000 7f654e30b640 1 -- 172.21.15.78:0/3139282285 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654819f3f0 con 0x7f6548102c20 2024-08-18T23:43:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.024+0000 7f654db0a640 1 --2- 172.21.15.78:0/3139282285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6548100b50 0x7f654819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:43:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.024+0000 7f654e30b640 1 --2- 172.21.15.78:0/3139282285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6548102c20 0x7f654819ed10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f653800cad0 tx=0x7f653800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.024+0000 7f6546ffd640 1 -- 172.21.15.78:0/3139282285 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653800ae20 con 0x7f6548102c20 2024-08-18T23:43:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.024+0000 7f6546ffd640 1 -- 172.21.15.78:0/3139282285 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6538005020 con 0x7f6548102c20 2024-08-18T23:43:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.024+0000 7f6546ffd640 1 -- 172.21.15.78:0/3139282285 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653800f6f0 con 0x7f6548102c20 2024-08-18T23:43:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.024+0000 7f654fd6e640 1 -- 172.21.15.78:0/3139282285 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f654819f6e0 con 0x7f6548102c20 2024-08-18T23:43:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.024+0000 7f654fd6e640 1 -- 172.21.15.78:0/3139282285 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65481a6fc0 con 0x7f6548102c20 2024-08-18T23:43:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.025+0000 7f654fd6e640 1 -- 172.21.15.78:0/3139282285 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6548100f30 con 0x7f6548102c20 2024-08-18T23:43:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.026+0000 7f6546ffd640 1 -- 172.21.15.78:0/3139282285 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f65380051d0 con 0x7f6548102c20 2024-08-18T23:43:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.027+0000 7f6546ffd640 1 --2- 172.21.15.78:0/3139282285 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f65300a2c10 0x7f65300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:16.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.027+0000 7f654db0a640 1 --2- 172.21.15.78:0/3139282285 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f65300a2c10 0x7f65300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:16.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.027+0000 7f6546ffd640 1 -- 172.21.15.78:0/3139282285 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f65380cf9f0 con 0x7f6548102c20 2024-08-18T23:43:16.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.027+0000 7f654db0a640 1 --2- 172.21.15.78:0/3139282285 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f65300a2c10 0x7f65300a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6540005ab0 tx=0x7f6540005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:16.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.030+0000 7f6546ffd640 1 -- 172.21.15.78:0/3139282285 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6538014030 con 0x7f6548102c20 2024-08-18T23:43:16.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.197+0000 7f654fd6e640 1 -- 172.21.15.78:0/3139282285 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f654810cad0 con 0x7f65300a2c10 2024-08-18T23:43:16.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.205+0000 7f6546ffd640 1 -- 172.21.15.78:0/3139282285 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340366 (secure 0 0 0) 0x7f654810cad0 con 0x7f65300a2c10 2024-08-18T23:43:16.209 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:43:16.211 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-08-18T23:43:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.208+0000 7f654fd6e640 1 -- 172.21.15.78:0/3139282285 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f65300a2c10 msgr2=0x7f65300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.208+0000 7f654fd6e640 1 --2- 172.21.15.78:0/3139282285 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f65300a2c10 0x7f65300a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6540005ab0 tx=0x7f6540005a20 comp rx=0 tx=0).stop 2024-08-18T23:43:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.208+0000 7f654fd6e640 1 -- 172.21.15.78:0/3139282285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6548102c20 msgr2=0x7f654819ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.208+0000 7f654fd6e640 1 --2- 172.21.15.78:0/3139282285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6548102c20 0x7f654819ed10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f653800cad0 tx=0x7f653800cf90 comp rx=0 tx=0).stop 2024-08-18T23:43:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.208+0000 7f654fd6e640 1 -- 172.21.15.78:0/3139282285 shutdown_connections 2024-08-18T23:43:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.208+0000 7f654fd6e640 1 --2- 172.21.15.78:0/3139282285 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f65300a2c10 0x7f65300a50d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.208+0000 7f654fd6e640 1 --2- 172.21.15.78:0/3139282285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6548102c20 0x7f654819ed10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.208+0000 7f654fd6e640 1 --2- 172.21.15.78:0/3139282285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65481022a0 0x7f654819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.208+0000 7f654fd6e640 1 --2- 172.21.15.78:0/3139282285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6548100b50 0x7f654819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.208+0000 7f654fd6e640 1 -- 172.21.15.78:0/3139282285 >> 172.21.15.78:0/3139282285 conn(0x7f65480fc500 msgr2=0x7f654806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.208+0000 7f654fd6e640 1 -- 172.21.15.78:0/3139282285 shutdown_connections 2024-08-18T23:43:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:16.208+0000 7f654fd6e640 1 -- 172.21.15.78:0/3139282285 wait complete. 2024-08-18T23:43:16.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:16 smithi078 ceph-mon[18440]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:43:16.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:16 smithi123 ceph-mon[21808]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:43:16.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:16 smithi161 ceph-mon[22027]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:43:16.929 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-08-18T23:43:16.135247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657988,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":552,"num_read_kb":394,"num_write":350,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":282,"ondisk_log_size":282,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388220,"kb_used_data":14048,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553604,"statfs":{"total":1151940427776,"available":1151542890496,"internally_reserved":0,"allocated":14385152,"data_stored":8010062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003330"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319578+0000","last_change":"2024-08-18T23:41:30.120580+0000","last_active":"2024-08-18T23:42:23.319578+0000","last_peered":"2024-08-18T23:42:23.319578+0000","last_clean":"2024-08-18T23:42:23.319578+0000","last_became_active":"2024-08-18T23:41:30.120266+0000","last_became_peered":"2024-08-18T23:41:30.120266+0000","last_unstale":"2024-08-18T23:42:23.319578+0000","last_undegraded":"2024-08-18T23:42:23.319578+0000","last_fullsized":"2024-08-18T23:42:23.319578+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:06:40.997070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557745+0000","last_change":"2024-08-18T23:41:32.143232+0000","last_active":"2024-08-18T23:42:23.557745+0000","last_peered":"2024-08-18T23:42:23.557745+0000","last_clean":"2024-08-18T23:42:23.557745+0000","last_became_active":"2024-08-18T23:41:32.142916+0000","last_became_peered":"2024-08-18T23:41:32.142916+0000","last_unstale":"2024-08-18T23:42:23.557745+0000","last_undegraded":"2024-08-18T23:42:23.557745+0000","last_fullsized":"2024-08-18T23:42:23.557745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:50:54.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320593+0000","last_change":"2024-08-18T23:41:39.188772+0000","last_active":"2024-08-18T23:42:23.320593+0000","last_peered":"2024-08-18T23:42:23.320593+0000","last_clean":"2024-08-18T23:42:23.320593+0000","last_became_active":"2024-08-18T23:41:39.188362+0000","last_became_peered":"2024-08-18T23:41:39.188362+0000","last_unstale":"2024-08-18T23:42:23.320593+0000","last_undegraded":"2024-08-18T23:42:23.320593+0000","last_fullsized":"2024-08-18T23:42:23.320593+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:47:50.255543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557888+0000","last_change":"2024-08-18T23:41:30.137179+0000","last_active":"2024-08-18T23:42:23.557888+0000","last_peered":"2024-08-18T23:42:23.557888+0000","last_clean":"2024-08-18T23:42:23.557888+0000","last_became_active":"2024-08-18T23:41:30.136618+0000","last_became_peered":"2024-08-18T23:41:30.136618+0000","last_unstale":"2024-08-18T23:42:23.557888+0000","last_undegraded":"2024-08-18T23:42:23.557888+0000","last_fullsized":"2024-08-18T23:42:23.557888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:59.826364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316847+0000","last_change":"2024-08-18T23:41:32.142075+0000","last_active":"2024-08-18T23:42:23.316847+0000","last_peered":"2024-08-18T23:42:23.316847+0000","last_clean":"2024-08-18T23:42:23.316847+0000","last_became_active":"2024-08-18T23:41:32.141709+0000","last_became_peered":"2024-08-18T23:41:32.141709+0000","last_unstale":"2024-08-18T23:42:23.316847+0000","last_undegraded":"2024-08-18T23:42:23.316847+0000","last_fullsized":"2024-08-18T23:42:23.316847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:08.324627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317948+0000","last_change":"2024-08-18T23:41:39.184575+0000","last_active":"2024-08-18T23:42:23.317948+0000","last_peered":"2024-08-18T23:42:23.317948+0000","last_clean":"2024-08-18T23:42:23.317948+0000","last_became_active":"2024-08-18T23:41:39.184169+0000","last_became_peered":"2024-08-18T23:41:39.184169+0000","last_unstale":"2024-08-18T23:42:23.317948+0000","last_undegraded":"2024-08-18T23:42:23.317948+0000","last_fullsized":"2024-08-18T23:42:23.317948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:32.218172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324756+0000","last_change":"2024-08-18T23:41:30.121638+0000","last_active":"2024-08-18T23:42:23.324756+0000","last_peered":"2024-08-18T23:42:23.324756+0000","last_clean":"2024-08-18T23:42:23.324756+0000","last_became_active":"2024-08-18T23:41:30.121290+0000","last_became_peered":"2024-08-18T23:41:30.121290+0000","last_unstale":"2024-08-18T23:42:23.324756+0000","last_undegraded":"2024-08-18T23:42:23.324756+0000","last_fullsized":"2024-08-18T23:42:23.324756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:43:55.638780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315534+0000","last_change":"2024-08-18T23:41:32.142177+0000","last_active":"2024-08-18T23:42:23.315534+0000","last_peered":"2024-08-18T23:42:23.315534+0000","last_clean":"2024-08-18T23:42:23.315534+0000","last_became_active":"2024-08-18T23:41:32.141731+0000","last_became_peered":"2024-08-18T23:41:32.141731+0000","last_unstale":"2024-08-18T23:42:23.315534+0000","last_undegraded":"2024-08-18T23:42:23.315534+0000","last_fullsized":"2024-08-18T23:42:23.315534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:36.283299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318344+0000","last_change":"2024-08-18T23:41:39.182931+0000","last_active":"2024-08-18T23:42:23.318344+0000","last_peered":"2024-08-18T23:42:23.318344+0000","last_clean":"2024-08-18T23:42:23.318344+0000","last_became_active":"2024-08-18T23:41:39.182008+0000","last_became_peered":"2024-08-18T23:41:39.182008+0000","last_unstale":"2024-08-18T23:42:23.318344+0000","last_undegraded":"2024-08-18T23:42:23.318344+0000","last_fullsized":"2024-08-18T23:42:23.318344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:58:38.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.329762+0000","last_change":"2024-08-18T23:42:23.329762+0000","last_active":"2024-08-18T23:42:23.329762+0000","last_peered":"2024-08-18T23:42:23.329762+0000","last_clean":"2024-08-18T23:42:23.329762+0000","last_became_active":"2024-08-18T23:42:23.325753+0000","last_became_peered":"2024-08-18T23:42:23.325753+0000","last_unstale":"2024-08-18T23:42:23.329762+0000","last_undegraded":"2024-08-18T23:42:23.329762+0000","last_fullsized":"2024-08-18T23:42:23.329762+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:19:43.062492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323256+0000","last_change":"2024-08-18T23:41:32.148662+0000","last_active":"2024-08-18T23:42:23.323256+0000","last_peered":"2024-08-18T23:42:23.323256+0000","last_clean":"2024-08-18T23:42:23.323256+0000","last_became_active":"2024-08-18T23:41:32.148318+0000","last_became_peered":"2024-08-18T23:41:32.148318+0000","last_unstale":"2024-08-18T23:42:23.323256+0000","last_undegraded":"2024-08-18T23:42:23.323256+0000","last_fullsized":"2024-08-18T23:42:23.323256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:58.668669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318492+0000","last_change":"2024-08-18T23:41:39.181114+0000","last_active":"2024-08-18T23:42:23.318492+0000","last_peered":"2024-08-18T23:42:23.318492+0000","last_clean":"2024-08-18T23:42:23.318492+0000","last_became_active":"2024-08-18T23:41:39.180599+0000","last_became_peered":"2024-08-18T23:41:39.180599+0000","last_unstale":"2024-08-18T23:42:23.318492+0000","last_undegraded":"2024-08-18T23:42:23.318492+0000","last_fullsized":"2024-08-18T23:42:23.318492+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:36:08.749143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316660+0000","last_change":"2024-08-18T23:41:32.144326+0000","last_active":"2024-08-18T23:42:23.316660+0000","last_peered":"2024-08-18T23:42:23.316660+0000","last_clean":"2024-08-18T23:42:23.316660+0000","last_became_active":"2024-08-18T23:41:32.144004+0000","last_became_peered":"2024-08-18T23:41:32.144004+0000","last_unstale":"2024-08-18T23:42:23.316660+0000","last_undegraded":"2024-08-18T23:42:23.316660+0000","last_fullsized":"2024-08-18T23:42:23.316660+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:10.918624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324188+0000","last_change":"2024-08-18T23:41:30.125160+0000","last_active":"2024-08-18T23:42:23.324188+0000","last_peered":"2024-08-18T23:42:23.324188+0000","last_clean":"2024-08-18T23:42:23.324188+0000","last_became_active":"2024-08-18T23:41:30.124685+0000","last_became_peered":"2024-08-18T23:41:30.124685+0000","last_unstale":"2024-08-18T23:42:23.324188+0000","last_undegraded":"2024-08-18T23:42:23.324188+0000","last_fullsized":"2024-08-18T23:42:23.324188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:52:48.997902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318342+0000","last_change":"2024-08-18T23:41:39.681481+0000","last_active":"2024-08-18T23:42:23.318342+0000","last_peered":"2024-08-18T23:42:23.318342+0000","last_clean":"2024-08-18T23:42:23.318342+0000","last_became_active":"2024-08-18T23:41:39.680204+0000","last_became_peered":"2024-08-18T23:41:39.680204+0000","last_unstale":"2024-08-18T23:42:23.318342+0000","last_undegraded":"2024-08-18T23:42:23.318342+0000","last_fullsized":"2024-08-18T23:42:23.318342+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:00:05.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557591+0000","last_change":"2024-08-18T23:41:32.151335+0000","last_active":"2024-08-18T23:42:23.557591+0000","last_peered":"2024-08-18T23:42:23.557591+0000","last_clean":"2024-08-18T23:42:23.557591+0000","last_became_active":"2024-08-18T23:41:32.151040+0000","last_became_peered":"2024-08-18T23:41:32.151040+0000","last_unstale":"2024-08-18T23:42:23.557591+0000","last_undegraded":"2024-08-18T23:42:23.557591+0000","last_fullsized":"2024-08-18T23:42:23.557591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:16.108555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.321637+0000","last_change":"2024-08-18T23:41:30.122301+0000","last_active":"2024-08-18T23:42:23.321637+0000","last_peered":"2024-08-18T23:42:23.321637+0000","last_clean":"2024-08-18T23:42:23.321637+0000","last_became_active":"2024-08-18T23:41:30.121117+0000","last_became_peered":"2024-08-18T23:41:30.121117+0000","last_unstale":"2024-08-18T23:42:23.321637+0000","last_undegraded":"2024-08-18T23:42:23.321637+0000","last_fullsized":"2024-08-18T23:42:23.321637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:40:12.678803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318961+0000","last_change":"2024-08-18T23:41:39.182395+0000","last_active":"2024-08-18T23:42:23.318961+0000","last_peered":"2024-08-18T23:42:23.318961+0000","last_clean":"2024-08-18T23:42:23.318961+0000","last_became_active":"2024-08-18T23:41:39.181979+0000","last_became_peered":"2024-08-18T23:41:39.181979+0000","last_unstale":"2024-08-18T23:42:23.318961+0000","last_undegraded":"2024-08-18T23:42:23.318961+0000","last_fullsized":"2024-08-18T23:42:23.318961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:04.674503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318695+0000","last_change":"2024-08-18T23:41:32.140464+0000","last_active":"2024-08-18T23:42:23.318695+0000","last_peered":"2024-08-18T23:42:23.318695+0000","last_clean":"2024-08-18T23:42:23.318695+0000","last_became_active":"2024-08-18T23:41:32.139153+0000","last_became_peered":"2024-08-18T23:41:32.139153+0000","last_unstale":"2024-08-18T23:42:23.318695+0000","last_undegraded":"2024-08-18T23:42:23.318695+0000","last_fullsized":"2024-08-18T23:42:23.318695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:27.387680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557664+0000","last_change":"2024-08-18T23:41:30.122787+0000","last_active":"2024-08-18T23:42:23.557664+0000","last_peered":"2024-08-18T23:42:23.557664+0000","last_clean":"2024-08-18T23:42:23.557664+0000","last_became_active":"2024-08-18T23:41:30.122462+0000","last_became_peered":"2024-08-18T23:41:30.122462+0000","last_unstale":"2024-08-18T23:42:23.557664+0000","last_undegraded":"2024-08-18T23:42:23.557664+0000","last_fullsized":"2024-08-18T23:42:23.557664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:55.568831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319322+0000","last_change":"2024-08-18T23:41:39.190511+0000","last_active":"2024-08-18T23:42:23.319322+0000","last_peered":"2024-08-18T23:42:23.319322+0000","last_clean":"2024-08-18T23:42:23.319322+0000","last_became_active":"2024-08-18T23:41:39.190154+0000","last_became_peered":"2024-08-18T23:41:39.190154+0000","last_unstale":"2024-08-18T23:42:23.319322+0000","last_undegraded":"2024-08-18T23:42:23.319322+0000","last_fullsized":"2024-08-18T23:42:23.319322+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:33:34.514827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323413+0000","last_change":"2024-08-18T23:41:32.143867+0000","last_active":"2024-08-18T23:42:23.323413+0000","last_peered":"2024-08-18T23:42:23.323413+0000","last_clean":"2024-08-18T23:42:23.323413+0000","last_became_active":"2024-08-18T23:41:32.143527+0000","last_became_peered":"2024-08-18T23:41:32.143527+0000","last_unstale":"2024-08-18T23:42:23.323413+0000","last_undegraded":"2024-08-18T23:42:23.323413+0000","last_fullsized":"2024-08-18T23:42:23.323413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:40:25.902133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.325102+0000","last_change":"2024-08-18T23:42:23.325102+0000","last_active":"2024-08-18T23:42:23.325102+0000","last_peered":"2024-08-18T23:42:23.325102+0000","last_clean":"2024-08-18T23:42:23.325102+0000","last_became_active":"2024-08-18T23:42:23.321583+0000","last_became_peered":"2024-08-18T23:42:23.321583+0000","last_unstale":"2024-08-18T23:42:23.325102+0000","last_undegraded":"2024-08-18T23:42:23.325102+0000","last_fullsized":"2024-08-18T23:42:23.325102+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:57:42.546686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.559756+0000","last_change":"2024-08-18T23:42:23.559756+0000","last_active":"2024-08-18T23:42:23.559756+0000","last_peered":"2024-08-18T23:42:23.559756+0000","last_clean":"2024-08-18T23:42:23.559756+0000","last_became_active":"2024-08-18T23:42:23.559242+0000","last_became_peered":"2024-08-18T23:42:23.559242+0000","last_unstale":"2024-08-18T23:42:23.559756+0000","last_undegraded":"2024-08-18T23:42:23.559756+0000","last_fullsized":"2024-08-18T23:42:23.559756+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:25:40.790313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316766+0000","last_change":"2024-08-18T23:41:32.139025+0000","last_active":"2024-08-18T23:42:23.316766+0000","last_peered":"2024-08-18T23:42:23.316766+0000","last_clean":"2024-08-18T23:42:23.316766+0000","last_became_active":"2024-08-18T23:41:32.138733+0000","last_became_peered":"2024-08-18T23:41:32.138733+0000","last_unstale":"2024-08-18T23:42:23.316766+0000","last_undegraded":"2024-08-18T23:42:23.316766+0000","last_fullsized":"2024-08-18T23:42:23.316766+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:01:39.826391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319270+0000","last_change":"2024-08-18T23:41:30.477359+0000","last_active":"2024-08-18T23:42:23.319270+0000","last_peered":"2024-08-18T23:42:23.319270+0000","last_clean":"2024-08-18T23:42:23.319270+0000","last_became_active":"2024-08-18T23:41:30.477011+0000","last_became_peered":"2024-08-18T23:41:30.477011+0000","last_unstale":"2024-08-18T23:42:23.319270+0000","last_undegraded":"2024-08-18T23:42:23.319270+0000","last_fullsized":"2024-08-18T23:42:23.319270+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:53:10.052859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320716+0000","last_change":"2024-08-18T23:41:39.182140+0000","last_active":"2024-08-18T23:42:23.320716+0000","last_peered":"2024-08-18T23:42:23.320716+0000","last_clean":"2024-08-18T23:42:23.320716+0000","last_became_active":"2024-08-18T23:41:39.181667+0000","last_became_peered":"2024-08-18T23:41:39.181667+0000","last_unstale":"2024-08-18T23:42:23.320716+0000","last_undegraded":"2024-08-18T23:42:23.320716+0000","last_fullsized":"2024-08-18T23:42:23.320716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:02:18.128028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557897+0000","last_change":"2024-08-18T23:41:32.137064+0000","last_active":"2024-08-18T23:42:23.557897+0000","last_peered":"2024-08-18T23:42:23.557897+0000","last_clean":"2024-08-18T23:42:23.557897+0000","last_became_active":"2024-08-18T23:41:32.136738+0000","last_became_peered":"2024-08-18T23:41:32.136738+0000","last_unstale":"2024-08-18T23:42:23.557897+0000","last_undegraded":"2024-08-18T23:42:23.557897+0000","last_fullsized":"2024-08-18T23:42:23.557897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:15:11.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319512+0000","last_change":"2024-08-18T23:41:30.120042+0000","last_active":"2024-08-18T23:42:23.319512+0000","last_peered":"2024-08-18T23:42:23.319512+0000","last_clean":"2024-08-18T23:42:23.319512+0000","last_became_active":"2024-08-18T23:41:30.119719+0000","last_became_peered":"2024-08-18T23:41:30.119719+0000","last_unstale":"2024-08-18T23:42:23.319512+0000","last_undegraded":"2024-08-18T23:42:23.319512+0000","last_fullsized":"2024-08-18T23:42:23.319512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:35:44.466330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557353+0000","last_change":"2024-08-18T23:41:39.680405+0000","last_active":"2024-08-18T23:42:23.557353+0000","last_peered":"2024-08-18T23:42:23.557353+0000","last_clean":"2024-08-18T23:42:23.557353+0000","last_became_active":"2024-08-18T23:41:39.679940+0000","last_became_peered":"2024-08-18T23:41:39.679940+0000","last_unstale":"2024-08-18T23:42:23.557353+0000","last_undegraded":"2024-08-18T23:42:23.557353+0000","last_fullsized":"2024-08-18T23:42:23.557353+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:15:03.942116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314555+0000","last_change":"2024-08-18T23:41:32.139688+0000","last_active":"2024-08-18T23:42:23.314555+0000","last_peered":"2024-08-18T23:42:23.314555+0000","last_clean":"2024-08-18T23:42:23.314555+0000","last_became_active":"2024-08-18T23:41:32.139397+0000","last_became_peered":"2024-08-18T23:41:32.139397+0000","last_unstale":"2024-08-18T23:42:23.314555+0000","last_undegraded":"2024-08-18T23:42:23.314555+0000","last_fullsized":"2024-08-18T23:42:23.314555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:03:57.017488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324876+0000","last_change":"2024-08-18T23:41:30.137578+0000","last_active":"2024-08-18T23:42:23.324876+0000","last_peered":"2024-08-18T23:42:23.324876+0000","last_clean":"2024-08-18T23:42:23.324876+0000","last_became_active":"2024-08-18T23:41:30.137246+0000","last_became_peered":"2024-08-18T23:41:30.137246+0000","last_unstale":"2024-08-18T23:42:23.324876+0000","last_undegraded":"2024-08-18T23:42:23.324876+0000","last_fullsized":"2024-08-18T23:42:23.324876+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:48:54.839432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318001+0000","last_change":"2024-08-18T23:41:39.186360+0000","last_active":"2024-08-18T23:42:23.318001+0000","last_peered":"2024-08-18T23:42:23.318001+0000","last_clean":"2024-08-18T23:42:23.318001+0000","last_became_active":"2024-08-18T23:41:39.185808+0000","last_became_peered":"2024-08-18T23:41:39.185808+0000","last_unstale":"2024-08-18T23:42:23.318001+0000","last_undegraded":"2024-08-18T23:42:23.318001+0000","last_fullsized":"2024-08-18T23:42:23.318001+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:53.891061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558281+0000","last_change":"2024-08-18T23:41:32.146070+0000","last_active":"2024-08-18T23:42:23.558281+0000","last_peered":"2024-08-18T23:42:23.558281+0000","last_clean":"2024-08-18T23:42:23.558281+0000","last_became_active":"2024-08-18T23:41:32.145640+0000","last_became_peered":"2024-08-18T23:41:32.145640+0000","last_unstale":"2024-08-18T23:42:23.558281+0000","last_undegraded":"2024-08-18T23:42:23.558281+0000","last_fullsized":"2024-08-18T23:42:23.558281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:56:21.368762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319576+0000","last_change":"2024-08-18T23:41:30.138565+0000","last_active":"2024-08-18T23:42:23.319576+0000","last_peered":"2024-08-18T23:42:23.319576+0000","last_clean":"2024-08-18T23:42:23.319576+0000","last_became_active":"2024-08-18T23:41:30.138210+0000","last_became_peered":"2024-08-18T23:41:30.138210+0000","last_unstale":"2024-08-18T23:42:23.319576+0000","last_undegraded":"2024-08-18T23:42:23.319576+0000","last_fullsized":"2024-08-18T23:42:23.319576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:42:14.928884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323149+0000","last_change":"2024-08-18T23:41:39.680656+0000","last_active":"2024-08-18T23:42:23.323149+0000","last_peered":"2024-08-18T23:42:23.323149+0000","last_clean":"2024-08-18T23:42:23.323149+0000","last_became_active":"2024-08-18T23:41:39.680203+0000","last_became_peered":"2024-08-18T23:41:39.680203+0000","last_unstale":"2024-08-18T23:42:23.323149+0000","last_undegraded":"2024-08-18T23:42:23.323149+0000","last_fullsized":"2024-08-18T23:42:23.323149+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:55:43.138869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558246+0000","last_change":"2024-08-18T23:41:32.130222+0000","last_active":"2024-08-18T23:42:23.558246+0000","last_peered":"2024-08-18T23:42:23.558246+0000","last_clean":"2024-08-18T23:42:23.558246+0000","last_became_active":"2024-08-18T23:41:32.129844+0000","last_became_peered":"2024-08-18T23:41:32.129844+0000","last_unstale":"2024-08-18T23:42:23.558246+0000","last_undegraded":"2024-08-18T23:42:23.558246+0000","last_fullsized":"2024-08-18T23:42:23.558246+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:39:21.106343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316823+0000","last_change":"2024-08-18T23:41:30.123142+0000","last_active":"2024-08-18T23:42:23.316823+0000","last_peered":"2024-08-18T23:42:23.316823+0000","last_clean":"2024-08-18T23:42:23.316823+0000","last_became_active":"2024-08-18T23:41:30.121645+0000","last_became_peered":"2024-08-18T23:41:30.121645+0000","last_unstale":"2024-08-18T23:42:23.316823+0000","last_undegraded":"2024-08-18T23:42:23.316823+0000","last_fullsized":"2024-08-18T23:42:23.316823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:07:24.912259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314454+0000","last_change":"2024-08-18T23:41:39.186033+0000","last_active":"2024-08-18T23:42:23.314454+0000","last_peered":"2024-08-18T23:42:23.314454+0000","last_clean":"2024-08-18T23:42:23.314454+0000","last_became_active":"2024-08-18T23:41:39.185451+0000","last_became_peered":"2024-08-18T23:41:39.185451+0000","last_unstale":"2024-08-18T23:42:23.314454+0000","last_undegraded":"2024-08-18T23:42:23.314454+0000","last_fullsized":"2024-08-18T23:42:23.314454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:52:13.556760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320803+0000","last_change":"2024-08-18T23:41:32.141065+0000","last_active":"2024-08-18T23:42:23.320803+0000","last_peered":"2024-08-18T23:42:23.320803+0000","last_clean":"2024-08-18T23:42:23.320803+0000","last_became_active":"2024-08-18T23:41:32.140732+0000","last_became_peered":"2024-08-18T23:41:32.140732+0000","last_unstale":"2024-08-18T23:42:23.320803+0000","last_undegraded":"2024-08-18T23:42:23.320803+0000","last_fullsized":"2024-08-18T23:42:23.320803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:08:32.287835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":104,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:43:12.362090+0000","last_change":"2024-08-18T23:41:30.136838+0000","last_active":"2024-08-18T23:43:12.362090+0000","last_peered":"2024-08-18T23:43:12.362090+0000","last_clean":"2024-08-18T23:43:12.362090+0000","last_became_active":"2024-08-18T23:41:30.136523+0000","last_became_peered":"2024-08-18T23:41:30.136523+0000","last_unstale":"2024-08-18T23:43:12.362090+0000","last_undegraded":"2024-08-18T23:43:12.362090+0000","last_fullsized":"2024-08-18T23:43:12.362090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:56:17.750535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314559+0000","last_change":"2024-08-18T23:41:39.680314+0000","last_active":"2024-08-18T23:42:23.314559+0000","last_peered":"2024-08-18T23:42:23.314559+0000","last_clean":"2024-08-18T23:42:23.314559+0000","last_became_active":"2024-08-18T23:41:39.679920+0000","last_became_peered":"2024-08-18T23:41:39.679920+0000","last_unstale":"2024-08-18T23:42:23.314559+0000","last_undegraded":"2024-08-18T23:42:23.314559+0000","last_fullsized":"2024-08-18T23:42:23.314559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:26:29.866081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314066+0000","last_change":"2024-08-18T23:41:32.139752+0000","last_active":"2024-08-18T23:42:23.314066+0000","last_peered":"2024-08-18T23:42:23.314066+0000","last_clean":"2024-08-18T23:42:23.314066+0000","last_became_active":"2024-08-18T23:41:32.139451+0000","last_became_peered":"2024-08-18T23:41:32.139451+0000","last_unstale":"2024-08-18T23:42:23.314066+0000","last_undegraded":"2024-08-18T23:42:23.314066+0000","last_fullsized":"2024-08-18T23:42:23.314066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:15:13.390529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316737+0000","last_change":"2024-08-18T23:41:30.126513+0000","last_active":"2024-08-18T23:42:23.316737+0000","last_peered":"2024-08-18T23:42:23.316737+0000","last_clean":"2024-08-18T23:42:23.316737+0000","last_became_active":"2024-08-18T23:41:30.126046+0000","last_became_peered":"2024-08-18T23:41:30.126046+0000","last_unstale":"2024-08-18T23:42:23.316737+0000","last_undegraded":"2024-08-18T23:42:23.316737+0000","last_fullsized":"2024-08-18T23:42:23.316737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:49:16.565565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323372+0000","last_change":"2024-08-18T23:41:39.681006+0000","last_active":"2024-08-18T23:42:23.323372+0000","last_peered":"2024-08-18T23:42:23.323372+0000","last_clean":"2024-08-18T23:42:23.323372+0000","last_became_active":"2024-08-18T23:41:39.680409+0000","last_became_peered":"2024-08-18T23:41:39.680409+0000","last_unstale":"2024-08-18T23:42:23.323372+0000","last_undegraded":"2024-08-18T23:42:23.323372+0000","last_fullsized":"2024-08-18T23:42:23.323372+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:47.778866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557596+0000","last_change":"2024-08-18T23:41:32.147545+0000","last_active":"2024-08-18T23:42:23.557596+0000","last_peered":"2024-08-18T23:42:23.557596+0000","last_clean":"2024-08-18T23:42:23.557596+0000","last_became_active":"2024-08-18T23:41:32.147238+0000","last_became_peered":"2024-08-18T23:41:32.147238+0000","last_unstale":"2024-08-18T23:42:23.557596+0000","last_undegraded":"2024-08-18T23:42:23.557596+0000","last_fullsized":"2024-08-18T23:42:23.557596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:12:13.589535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319674+0000","last_change":"2024-08-18T23:41:30.120650+0000","last_active":"2024-08-18T23:42:23.319674+0000","last_peered":"2024-08-18T23:42:23.319674+0000","last_clean":"2024-08-18T23:42:23.319674+0000","last_became_active":"2024-08-18T23:41:30.120196+0000","last_became_peered":"2024-08-18T23:41:30.120196+0000","last_unstale":"2024-08-18T23:42:23.319674+0000","last_undegraded":"2024-08-18T23:42:23.319674+0000","last_fullsized":"2024-08-18T23:42:23.319674+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:40:23.370088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317120+0000","last_change":"2024-08-18T23:40:22.013519+0000","last_active":"2024-08-18T23:42:23.317120+0000","last_peered":"2024-08-18T23:42:23.317120+0000","last_clean":"2024-08-18T23:42:23.317120+0000","last_became_active":"2024-08-18T23:40:22.012986+0000","last_became_peered":"2024-08-18T23:40:22.012986+0000","last_unstale":"2024-08-18T23:42:23.317120+0000","last_undegraded":"2024-08-18T23:42:23.317120+0000","last_fullsized":"2024-08-18T23:42:23.317120+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_clean_scrub_stamp":"2024-08-18T23:37:22.388424+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:19.874356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.332800+0000","last_change":"2024-08-18T23:42:23.332800+0000","last_active":"2024-08-18T23:42:23.332800+0000","last_peered":"2024-08-18T23:42:23.332800+0000","last_clean":"2024-08-18T23:42:23.332800+0000","last_became_active":"2024-08-18T23:42:23.326441+0000","last_became_peered":"2024-08-18T23:42:23.326441+0000","last_unstale":"2024-08-18T23:42:23.332800+0000","last_undegraded":"2024-08-18T23:42:23.332800+0000","last_fullsized":"2024-08-18T23:42:23.332800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:13:58.243518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317150+0000","last_change":"2024-08-18T23:41:32.139907+0000","last_active":"2024-08-18T23:42:23.317150+0000","last_peered":"2024-08-18T23:42:23.317150+0000","last_clean":"2024-08-18T23:42:23.317150+0000","last_became_active":"2024-08-18T23:41:32.137956+0000","last_became_peered":"2024-08-18T23:41:32.137956+0000","last_unstale":"2024-08-18T23:42:23.317150+0000","last_undegraded":"2024-08-18T23:42:23.317150+0000","last_fullsized":"2024-08-18T23:42:23.317150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:10:13.223328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.330746+0000","last_change":"2024-08-18T23:42:23.330062+0000","last_active":"2024-08-18T23:42:23.330746+0000","last_peered":"2024-08-18T23:42:23.330746+0000","last_clean":"2024-08-18T23:42:23.330746+0000","last_became_active":"2024-08-18T23:42:23.323196+0000","last_became_peered":"2024-08-18T23:42:23.323196+0000","last_unstale":"2024-08-18T23:42:23.330746+0000","last_undegraded":"2024-08-18T23:42:23.330746+0000","last_fullsized":"2024-08-18T23:42:23.330746+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:53:29.059542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315093+0000","last_change":"2024-08-18T23:41:39.185917+0000","last_active":"2024-08-18T23:42:23.315093+0000","last_peered":"2024-08-18T23:42:23.315093+0000","last_clean":"2024-08-18T23:42:23.315093+0000","last_became_active":"2024-08-18T23:41:39.185427+0000","last_became_peered":"2024-08-18T23:41:39.185427+0000","last_unstale":"2024-08-18T23:42:23.315093+0000","last_undegraded":"2024-08-18T23:42:23.315093+0000","last_fullsized":"2024-08-18T23:42:23.315093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:52:13.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317336+0000","last_change":"2024-08-18T23:41:32.147530+0000","last_active":"2024-08-18T23:42:23.317336+0000","last_peered":"2024-08-18T23:42:23.317336+0000","last_clean":"2024-08-18T23:42:23.317336+0000","last_became_active":"2024-08-18T23:41:32.147216+0000","last_became_peered":"2024-08-18T23:41:32.147216+0000","last_unstale":"2024-08-18T23:42:23.317336+0000","last_undegraded":"2024-08-18T23:42:23.317336+0000","last_fullsized":"2024-08-18T23:42:23.317336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:04:06.421490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323538+0000","last_change":"2024-08-18T23:41:30.121307+0000","last_active":"2024-08-18T23:42:23.323538+0000","last_peered":"2024-08-18T23:42:23.323538+0000","last_clean":"2024-08-18T23:42:23.323538+0000","last_became_active":"2024-08-18T23:41:30.120932+0000","last_became_peered":"2024-08-18T23:41:30.120932+0000","last_unstale":"2024-08-18T23:42:23.323538+0000","last_undegraded":"2024-08-18T23:42:23.323538+0000","last_fullsized":"2024-08-18T23:42:23.323538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:06:07.742880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316371+0000","last_change":"2024-08-18T23:41:39.184647+0000","last_active":"2024-08-18T23:42:23.316371+0000","last_peered":"2024-08-18T23:42:23.316371+0000","last_clean":"2024-08-18T23:42:23.316371+0000","last_became_active":"2024-08-18T23:41:39.184251+0000","last_became_peered":"2024-08-18T23:41:39.184251+0000","last_unstale":"2024-08-18T23:42:23.316371+0000","last_undegraded":"2024-08-18T23:42:23.316371+0000","last_fullsized":"2024-08-18T23:42:23.316371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:55.200801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317327+0000","last_change":"2024-08-18T23:41:32.137007+0000","last_active":"2024-08-18T23:42:23.317327+0000","last_peered":"2024-08-18T23:42:23.317327+0000","last_clean":"2024-08-18T23:42:23.317327+0000","last_became_active":"2024-08-18T23:41:32.136684+0000","last_became_peered":"2024-08-18T23:41:32.136684+0000","last_unstale":"2024-08-18T23:42:23.317327+0000","last_undegraded":"2024-08-18T23:42:23.317327+0000","last_fullsized":"2024-08-18T23:42:23.317327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:21:26.584303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314514+0000","last_change":"2024-08-18T23:41:30.121183+0000","last_active":"2024-08-18T23:42:23.314514+0000","last_peered":"2024-08-18T23:42:23.314514+0000","last_clean":"2024-08-18T23:42:23.314514+0000","last_became_active":"2024-08-18T23:41:30.120847+0000","last_became_peered":"2024-08-18T23:41:30.120847+0000","last_unstale":"2024-08-18T23:42:23.314514+0000","last_undegraded":"2024-08-18T23:42:23.314514+0000","last_fullsized":"2024-08-18T23:42:23.314514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:51.025821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320528+0000","last_change":"2024-08-18T23:41:39.183375+0000","last_active":"2024-08-18T23:42:23.320528+0000","last_peered":"2024-08-18T23:42:23.320528+0000","last_clean":"2024-08-18T23:42:23.320528+0000","last_became_active":"2024-08-18T23:41:39.182339+0000","last_became_peered":"2024-08-18T23:41:39.182339+0000","last_unstale":"2024-08-18T23:42:23.320528+0000","last_undegraded":"2024-08-18T23:42:23.320528+0000","last_fullsized":"2024-08-18T23:42:23.320528+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:26.119056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318323+0000","last_change":"2024-08-18T23:41:32.140748+0000","last_active":"2024-08-18T23:42:23.318323+0000","last_peered":"2024-08-18T23:42:23.318323+0000","last_clean":"2024-08-18T23:42:23.318323+0000","last_became_active":"2024-08-18T23:41:32.140306+0000","last_became_peered":"2024-08-18T23:41:32.140306+0000","last_unstale":"2024-08-18T23:42:23.318323+0000","last_undegraded":"2024-08-18T23:42:23.318323+0000","last_fullsized":"2024-08-18T23:42:23.318323+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:23:02.371529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324632+0000","last_change":"2024-08-18T23:41:30.121617+0000","last_active":"2024-08-18T23:42:23.324632+0000","last_peered":"2024-08-18T23:42:23.324632+0000","last_clean":"2024-08-18T23:42:23.324632+0000","last_became_active":"2024-08-18T23:41:30.121257+0000","last_became_peered":"2024-08-18T23:41:30.121257+0000","last_unstale":"2024-08-18T23:42:23.324632+0000","last_undegraded":"2024-08-18T23:42:23.324632+0000","last_fullsized":"2024-08-18T23:42:23.324632+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:23:42.091553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320637+0000","last_change":"2024-08-18T23:41:39.183833+0000","last_active":"2024-08-18T23:42:23.320637+0000","last_peered":"2024-08-18T23:42:23.320637+0000","last_clean":"2024-08-18T23:42:23.320637+0000","last_became_active":"2024-08-18T23:41:39.183176+0000","last_became_peered":"2024-08-18T23:41:39.183176+0000","last_unstale":"2024-08-18T23:42:23.320637+0000","last_undegraded":"2024-08-18T23:42:23.320637+0000","last_fullsized":"2024-08-18T23:42:23.320637+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:36:09.827378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322357+0000","last_change":"2024-08-18T23:41:32.147507+0000","last_active":"2024-08-18T23:42:23.322357+0000","last_peered":"2024-08-18T23:42:23.322357+0000","last_clean":"2024-08-18T23:42:23.322357+0000","last_became_active":"2024-08-18T23:41:32.146042+0000","last_became_peered":"2024-08-18T23:41:32.146042+0000","last_unstale":"2024-08-18T23:42:23.322357+0000","last_undegraded":"2024-08-18T23:42:23.322357+0000","last_fullsized":"2024-08-18T23:42:23.322357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:48.941442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323009+0000","last_change":"2024-08-18T23:41:30.137755+0000","last_active":"2024-08-18T23:42:23.323009+0000","last_peered":"2024-08-18T23:42:23.323009+0000","last_clean":"2024-08-18T23:42:23.323009+0000","last_became_active":"2024-08-18T23:41:30.137194+0000","last_became_peered":"2024-08-18T23:41:30.137194+0000","last_unstale":"2024-08-18T23:42:23.323009+0000","last_undegraded":"2024-08-18T23:42:23.323009+0000","last_fullsized":"2024-08-18T23:42:23.323009+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:15:49.681405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314506+0000","last_change":"2024-08-18T23:41:39.183986+0000","last_active":"2024-08-18T23:42:23.314506+0000","last_peered":"2024-08-18T23:42:23.314506+0000","last_clean":"2024-08-18T23:42:23.314506+0000","last_became_active":"2024-08-18T23:41:39.183384+0000","last_became_peered":"2024-08-18T23:41:39.183384+0000","last_unstale":"2024-08-18T23:42:23.314506+0000","last_undegraded":"2024-08-18T23:42:23.314506+0000","last_fullsized":"2024-08-18T23:42:23.314506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:04:28.920804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316556+0000","last_change":"2024-08-18T23:41:32.134729+0000","last_active":"2024-08-18T23:42:23.316556+0000","last_peered":"2024-08-18T23:42:23.316556+0000","last_clean":"2024-08-18T23:42:23.316556+0000","last_became_active":"2024-08-18T23:41:32.134366+0000","last_became_peered":"2024-08-18T23:41:32.134366+0000","last_unstale":"2024-08-18T23:42:23.316556+0000","last_undegraded":"2024-08-18T23:42:23.316556+0000","last_fullsized":"2024-08-18T23:42:23.316556+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:38.610561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":108,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:43:13.364179+0000","last_change":"2024-08-18T23:41:30.126443+0000","last_active":"2024-08-18T23:43:13.364179+0000","last_peered":"2024-08-18T23:43:13.364179+0000","last_clean":"2024-08-18T23:43:13.364179+0000","last_became_active":"2024-08-18T23:41:30.124264+0000","last_became_peered":"2024-08-18T23:41:30.124264+0000","last_unstale":"2024-08-18T23:43:13.364179+0000","last_undegraded":"2024-08-18T23:43:13.364179+0000","last_fullsized":"2024-08-18T23:43:13.364179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:43.727114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319340+0000","last_change":"2024-08-18T23:41:39.192164+0000","last_active":"2024-08-18T23:42:23.319340+0000","last_peered":"2024-08-18T23:42:23.319340+0000","last_clean":"2024-08-18T23:42:23.319340+0000","last_became_active":"2024-08-18T23:41:39.191833+0000","last_became_peered":"2024-08-18T23:41:39.191833+0000","last_unstale":"2024-08-18T23:42:23.319340+0000","last_undegraded":"2024-08-18T23:42:23.319340+0000","last_fullsized":"2024-08-18T23:42:23.319340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:33:50.243930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320662+0000","last_change":"2024-08-18T23:42:23.320662+0000","last_active":"2024-08-18T23:42:23.320662+0000","last_peered":"2024-08-18T23:42:23.320662+0000","last_clean":"2024-08-18T23:42:23.320662+0000","last_became_active":"2024-08-18T23:42:23.320249+0000","last_became_peered":"2024-08-18T23:42:23.320249+0000","last_unstale":"2024-08-18T23:42:23.320662+0000","last_undegraded":"2024-08-18T23:42:23.320662+0000","last_fullsized":"2024-08-18T23:42:23.320662+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:03:17.121915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322332+0000","last_change":"2024-08-18T23:41:30.124071+0000","last_active":"2024-08-18T23:42:23.322332+0000","last_peered":"2024-08-18T23:42:23.322332+0000","last_clean":"2024-08-18T23:42:23.322332+0000","last_became_active":"2024-08-18T23:41:30.123118+0000","last_became_peered":"2024-08-18T23:41:30.123118+0000","last_unstale":"2024-08-18T23:42:23.322332+0000","last_undegraded":"2024-08-18T23:42:23.322332+0000","last_fullsized":"2024-08-18T23:42:23.322332+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:21:39.015187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319331+0000","last_change":"2024-08-18T23:41:39.184038+0000","last_active":"2024-08-18T23:42:23.319331+0000","last_peered":"2024-08-18T23:42:23.319331+0000","last_clean":"2024-08-18T23:42:23.319331+0000","last_became_active":"2024-08-18T23:41:39.183611+0000","last_became_peered":"2024-08-18T23:41:39.183611+0000","last_unstale":"2024-08-18T23:42:23.319331+0000","last_undegraded":"2024-08-18T23:42:23.319331+0000","last_fullsized":"2024-08-18T23:42:23.319331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:32:26.029151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314264+0000","last_change":"2024-08-18T23:41:32.143631+0000","last_active":"2024-08-18T23:42:23.314264+0000","last_peered":"2024-08-18T23:42:23.314264+0000","last_clean":"2024-08-18T23:42:23.314264+0000","last_became_active":"2024-08-18T23:41:32.143350+0000","last_became_peered":"2024-08-18T23:41:32.143350+0000","last_unstale":"2024-08-18T23:42:23.314264+0000","last_undegraded":"2024-08-18T23:42:23.314264+0000","last_fullsized":"2024-08-18T23:42:23.314264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:41:23.713467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319866+0000","last_change":"2024-08-18T23:41:30.123383+0000","last_active":"2024-08-18T23:42:23.319866+0000","last_peered":"2024-08-18T23:42:23.319866+0000","last_clean":"2024-08-18T23:42:23.319866+0000","last_became_active":"2024-08-18T23:41:30.123037+0000","last_became_peered":"2024-08-18T23:41:30.123037+0000","last_unstale":"2024-08-18T23:42:23.319866+0000","last_undegraded":"2024-08-18T23:42:23.319866+0000","last_fullsized":"2024-08-18T23:42:23.319866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:27.702478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323158+0000","last_change":"2024-08-18T23:41:39.184302+0000","last_active":"2024-08-18T23:42:23.323158+0000","last_peered":"2024-08-18T23:42:23.323158+0000","last_clean":"2024-08-18T23:42:23.323158+0000","last_became_active":"2024-08-18T23:41:39.183172+0000","last_became_peered":"2024-08-18T23:41:39.183172+0000","last_unstale":"2024-08-18T23:42:23.323158+0000","last_undegraded":"2024-08-18T23:42:23.323158+0000","last_fullsized":"2024-08-18T23:42:23.323158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:23:26.655865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324579+0000","last_change":"2024-08-18T23:41:32.137426+0000","last_active":"2024-08-18T23:42:23.324579+0000","last_peered":"2024-08-18T23:42:23.324579+0000","last_clean":"2024-08-18T23:42:23.324579+0000","last_became_active":"2024-08-18T23:41:32.137106+0000","last_became_peered":"2024-08-18T23:41:32.137106+0000","last_unstale":"2024-08-18T23:42:23.324579+0000","last_undegraded":"2024-08-18T23:42:23.324579+0000","last_fullsized":"2024-08-18T23:42:23.324579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:34:51.889663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'32","reported_seq":209,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:43:14.364493+0000","last_change":"2024-08-18T23:41:30.124320+0000","last_active":"2024-08-18T23:43:14.364493+0000","last_peered":"2024-08-18T23:43:14.364493+0000","last_clean":"2024-08-18T23:43:14.364493+0000","last_became_active":"2024-08-18T23:41:30.123979+0000","last_became_peered":"2024-08-18T23:41:30.123979+0000","last_unstale":"2024-08-18T23:43:14.364493+0000","last_undegraded":"2024-08-18T23:43:14.364493+0000","last_fullsized":"2024-08-18T23:43:14.364493+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:31:03.689720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62996,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":143,"num_read_kb":133,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319063+0000","last_change":"2024-08-18T23:41:39.190481+0000","last_active":"2024-08-18T23:42:23.319063+0000","last_peered":"2024-08-18T23:42:23.319063+0000","last_clean":"2024-08-18T23:42:23.319063+0000","last_became_active":"2024-08-18T23:41:39.190076+0000","last_became_peered":"2024-08-18T23:41:39.190076+0000","last_unstale":"2024-08-18T23:42:23.319063+0000","last_undegraded":"2024-08-18T23:42:23.319063+0000","last_fullsized":"2024-08-18T23:42:23.319063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:47:39.313142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315607+0000","last_change":"2024-08-18T23:41:32.145422+0000","last_active":"2024-08-18T23:42:23.315607+0000","last_peered":"2024-08-18T23:42:23.315607+0000","last_clean":"2024-08-18T23:42:23.315607+0000","last_became_active":"2024-08-18T23:41:32.145047+0000","last_became_peered":"2024-08-18T23:41:32.145047+0000","last_unstale":"2024-08-18T23:42:23.315607+0000","last_undegraded":"2024-08-18T23:42:23.315607+0000","last_fullsized":"2024-08-18T23:42:23.315607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:31.905248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323427+0000","last_change":"2024-08-18T23:41:30.137364+0000","last_active":"2024-08-18T23:42:23.323427+0000","last_peered":"2024-08-18T23:42:23.323427+0000","last_clean":"2024-08-18T23:42:23.323427+0000","last_became_active":"2024-08-18T23:41:30.137004+0000","last_became_peered":"2024-08-18T23:41:30.137004+0000","last_unstale":"2024-08-18T23:42:23.323427+0000","last_undegraded":"2024-08-18T23:42:23.323427+0000","last_fullsized":"2024-08-18T23:42:23.323427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:20:56.392149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.326359+0000","last_change":"2024-08-18T23:42:23.326359+0000","last_active":"2024-08-18T23:42:23.326359+0000","last_peered":"2024-08-18T23:42:23.326359+0000","last_clean":"2024-08-18T23:42:23.326359+0000","last_became_active":"2024-08-18T23:42:23.325918+0000","last_became_peered":"2024-08-18T23:42:23.325918+0000","last_unstale":"2024-08-18T23:42:23.326359+0000","last_undegraded":"2024-08-18T23:42:23.326359+0000","last_fullsized":"2024-08-18T23:42:23.326359+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:01:15.476268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.325691+0000","last_change":"2024-08-18T23:42:23.325691+0000","last_active":"2024-08-18T23:42:23.325691+0000","last_peered":"2024-08-18T23:42:23.325691+0000","last_clean":"2024-08-18T23:42:23.325691+0000","last_became_active":"2024-08-18T23:42:23.325467+0000","last_became_peered":"2024-08-18T23:42:23.325467+0000","last_unstale":"2024-08-18T23:42:23.325691+0000","last_undegraded":"2024-08-18T23:42:23.325691+0000","last_fullsized":"2024-08-18T23:42:23.325691+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:10:26.288880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317472+0000","last_change":"2024-08-18T23:41:30.124430+0000","last_active":"2024-08-18T23:42:23.317472+0000","last_peered":"2024-08-18T23:42:23.317472+0000","last_clean":"2024-08-18T23:42:23.317472+0000","last_became_active":"2024-08-18T23:41:30.123626+0000","last_became_peered":"2024-08-18T23:41:30.123626+0000","last_unstale":"2024-08-18T23:42:23.317472+0000","last_undegraded":"2024-08-18T23:42:23.317472+0000","last_fullsized":"2024-08-18T23:42:23.317472+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:26:18.426010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318912+0000","last_change":"2024-08-18T23:41:39.192765+0000","last_active":"2024-08-18T23:42:23.318912+0000","last_peered":"2024-08-18T23:42:23.318912+0000","last_clean":"2024-08-18T23:42:23.318912+0000","last_became_active":"2024-08-18T23:41:39.192250+0000","last_became_peered":"2024-08-18T23:41:39.192250+0000","last_unstale":"2024-08-18T23:42:23.318912+0000","last_undegraded":"2024-08-18T23:42:23.318912+0000","last_fullsized":"2024-08-18T23:42:23.318912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:51:16.794404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316158+0000","last_change":"2024-08-18T23:41:32.141966+0000","last_active":"2024-08-18T23:42:23.316158+0000","last_peered":"2024-08-18T23:42:23.316158+0000","last_clean":"2024-08-18T23:42:23.316158+0000","last_became_active":"2024-08-18T23:41:32.141561+0000","last_became_peered":"2024-08-18T23:41:32.141561+0000","last_unstale":"2024-08-18T23:42:23.316158+0000","last_undegraded":"2024-08-18T23:42:23.316158+0000","last_fullsized":"2024-08-18T23:42:23.316158+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:20.696089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":104,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:43:13.363287+0000","last_change":"2024-08-18T23:41:30.121598+0000","last_active":"2024-08-18T23:43:13.363287+0000","last_peered":"2024-08-18T23:43:13.363287+0000","last_clean":"2024-08-18T23:43:13.363287+0000","last_became_active":"2024-08-18T23:41:30.120533+0000","last_became_peered":"2024-08-18T23:41:30.120533+0000","last_unstale":"2024-08-18T23:43:13.363287+0000","last_undegraded":"2024-08-18T23:43:13.363287+0000","last_fullsized":"2024-08-18T23:43:13.363287+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:06:37.458272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557461+0000","last_change":"2024-08-18T23:41:39.178877+0000","last_active":"2024-08-18T23:42:23.557461+0000","last_peered":"2024-08-18T23:42:23.557461+0000","last_clean":"2024-08-18T23:42:23.557461+0000","last_became_active":"2024-08-18T23:41:39.178381+0000","last_became_peered":"2024-08-18T23:41:39.178381+0000","last_unstale":"2024-08-18T23:42:23.557461+0000","last_undegraded":"2024-08-18T23:42:23.557461+0000","last_fullsized":"2024-08-18T23:42:23.557461+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:27.275893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316665+0000","last_change":"2024-08-18T23:41:39.188381+0000","last_active":"2024-08-18T23:42:23.316665+0000","last_peered":"2024-08-18T23:42:23.316665+0000","last_clean":"2024-08-18T23:42:23.316665+0000","last_became_active":"2024-08-18T23:41:39.187983+0000","last_became_peered":"2024-08-18T23:41:39.187983+0000","last_unstale":"2024-08-18T23:42:23.316665+0000","last_undegraded":"2024-08-18T23:42:23.316665+0000","last_fullsized":"2024-08-18T23:42:23.316665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:04:45.460576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319101+0000","last_change":"2024-08-18T23:41:32.144637+0000","last_active":"2024-08-18T23:42:23.319101+0000","last_peered":"2024-08-18T23:42:23.319101+0000","last_clean":"2024-08-18T23:42:23.319101+0000","last_became_active":"2024-08-18T23:41:32.144329+0000","last_became_peered":"2024-08-18T23:41:32.144329+0000","last_unstale":"2024-08-18T23:42:23.319101+0000","last_undegraded":"2024-08-18T23:42:23.319101+0000","last_fullsized":"2024-08-18T23:42:23.319101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:29:39.791264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320980+0000","last_change":"2024-08-18T23:41:30.122325+0000","last_active":"2024-08-18T23:42:23.320980+0000","last_peered":"2024-08-18T23:42:23.320980+0000","last_clean":"2024-08-18T23:42:23.320980+0000","last_became_active":"2024-08-18T23:41:30.121480+0000","last_became_peered":"2024-08-18T23:41:30.121480+0000","last_unstale":"2024-08-18T23:42:23.320980+0000","last_undegraded":"2024-08-18T23:42:23.320980+0000","last_fullsized":"2024-08-18T23:42:23.320980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:32:12.810650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.560632+0000","last_change":"2024-08-18T23:42:23.560632+0000","last_active":"2024-08-18T23:42:23.560632+0000","last_peered":"2024-08-18T23:42:23.560632+0000","last_clean":"2024-08-18T23:42:23.560632+0000","last_became_active":"2024-08-18T23:42:23.559920+0000","last_became_peered":"2024-08-18T23:42:23.559920+0000","last_unstale":"2024-08-18T23:42:23.560632+0000","last_undegraded":"2024-08-18T23:42:23.560632+0000","last_fullsized":"2024-08-18T23:42:23.560632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:01:59.580843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557464+0000","last_change":"2024-08-18T23:41:32.145545+0000","last_active":"2024-08-18T23:42:23.557464+0000","last_peered":"2024-08-18T23:42:23.557464+0000","last_clean":"2024-08-18T23:42:23.557464+0000","last_became_active":"2024-08-18T23:41:32.145183+0000","last_became_peered":"2024-08-18T23:41:32.145183+0000","last_unstale":"2024-08-18T23:42:23.557464+0000","last_undegraded":"2024-08-18T23:42:23.557464+0000","last_fullsized":"2024-08-18T23:42:23.557464+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:03:04.111658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314074+0000","last_change":"2024-08-18T23:41:30.118776+0000","last_active":"2024-08-18T23:42:23.314074+0000","last_peered":"2024-08-18T23:42:23.314074+0000","last_clean":"2024-08-18T23:42:23.314074+0000","last_became_active":"2024-08-18T23:41:30.118436+0000","last_became_peered":"2024-08-18T23:41:30.118436+0000","last_unstale":"2024-08-18T23:42:23.314074+0000","last_undegraded":"2024-08-18T23:42:23.314074+0000","last_fullsized":"2024-08-18T23:42:23.314074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:13.980927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558143+0000","last_change":"2024-08-18T23:41:30.125201+0000","last_active":"2024-08-18T23:42:23.558143+0000","last_peered":"2024-08-18T23:42:23.558143+0000","last_clean":"2024-08-18T23:42:23.558143+0000","last_became_active":"2024-08-18T23:41:30.124894+0000","last_became_peered":"2024-08-18T23:41:30.124894+0000","last_unstale":"2024-08-18T23:42:23.558143+0000","last_undegraded":"2024-08-18T23:42:23.558143+0000","last_fullsized":"2024-08-18T23:42:23.558143+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:30:17.721829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322555+0000","last_change":"2024-08-18T23:41:32.145452+0000","last_active":"2024-08-18T23:42:23.322555+0000","last_peered":"2024-08-18T23:42:23.322555+0000","last_clean":"2024-08-18T23:42:23.322555+0000","last_became_active":"2024-08-18T23:41:32.145151+0000","last_became_peered":"2024-08-18T23:41:32.145151+0000","last_unstale":"2024-08-18T23:42:23.322555+0000","last_undegraded":"2024-08-18T23:42:23.322555+0000","last_fullsized":"2024-08-18T23:42:23.322555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:08:08.713154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319503+0000","last_change":"2024-08-18T23:41:39.193334+0000","last_active":"2024-08-18T23:42:23.319503+0000","last_peered":"2024-08-18T23:42:23.319503+0000","last_clean":"2024-08-18T23:42:23.319503+0000","last_became_active":"2024-08-18T23:41:39.192924+0000","last_became_peered":"2024-08-18T23:41:39.192924+0000","last_unstale":"2024-08-18T23:42:23.319503+0000","last_undegraded":"2024-08-18T23:42:23.319503+0000","last_fullsized":"2024-08-18T23:42:23.319503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:07.016300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":107,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:43:10.359920+0000","last_change":"2024-08-18T23:41:30.125217+0000","last_active":"2024-08-18T23:43:10.359920+0000","last_peered":"2024-08-18T23:43:10.359920+0000","last_clean":"2024-08-18T23:43:10.359920+0000","last_became_active":"2024-08-18T23:41:30.124889+0000","last_became_peered":"2024-08-18T23:41:30.124889+0000","last_unstale":"2024-08-18T23:43:10.359920+0000","last_undegraded":"2024-08-18T23:43:10.359920+0000","last_fullsized":"2024-08-18T23:43:10.359920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:43.389537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.323355+0000","last_change":"2024-08-18T23:41:32.142585+0000","last_active":"2024-08-18T23:42:23.323355+0000","last_peered":"2024-08-18T23:42:23.323355+0000","last_clean":"2024-08-18T23:42:23.323355+0000","last_became_active":"2024-08-18T23:41:32.142207+0000","last_became_peered":"2024-08-18T23:41:32.142207+0000","last_unstale":"2024-08-18T23:42:23.323355+0000","last_undegraded":"2024-08-18T23:42:23.323355+0000","last_fullsized":"2024-08-18T23:42:23.323355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:16:58.755444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.321952+0000","last_change":"2024-08-18T23:41:39.185618+0000","last_active":"2024-08-18T23:42:23.321952+0000","last_peered":"2024-08-18T23:42:23.321952+0000","last_clean":"2024-08-18T23:42:23.321952+0000","last_became_active":"2024-08-18T23:41:39.185222+0000","last_became_peered":"2024-08-18T23:41:39.185222+0000","last_unstale":"2024-08-18T23:42:23.321952+0000","last_undegraded":"2024-08-18T23:42:23.321952+0000","last_fullsized":"2024-08-18T23:42:23.321952+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:01:27.290998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318594+0000","last_change":"2024-08-18T23:41:32.140719+0000","last_active":"2024-08-18T23:42:23.318594+0000","last_peered":"2024-08-18T23:42:23.318594+0000","last_clean":"2024-08-18T23:42:23.318594+0000","last_became_active":"2024-08-18T23:41:32.140131+0000","last_became_peered":"2024-08-18T23:41:32.140131+0000","last_unstale":"2024-08-18T23:42:23.318594+0000","last_undegraded":"2024-08-18T23:42:23.318594+0000","last_fullsized":"2024-08-18T23:42:23.318594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:48:26.778094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.326683+0000","last_change":"2024-08-18T23:42:23.326683+0000","last_active":"2024-08-18T23:42:23.326683+0000","last_peered":"2024-08-18T23:42:23.326683+0000","last_clean":"2024-08-18T23:42:23.326683+0000","last_became_active":"2024-08-18T23:42:23.326309+0000","last_became_peered":"2024-08-18T23:42:23.326309+0000","last_unstale":"2024-08-18T23:42:23.326683+0000","last_undegraded":"2024-08-18T23:42:23.326683+0000","last_fullsized":"2024-08-18T23:42:23.326683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:14.320894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.321522+0000","last_change":"2024-08-18T23:41:32.149522+0000","last_active":"2024-08-18T23:42:23.321522+0000","last_peered":"2024-08-18T23:42:23.321522+0000","last_clean":"2024-08-18T23:42:23.321522+0000","last_became_active":"2024-08-18T23:41:32.149182+0000","last_became_peered":"2024-08-18T23:41:32.149182+0000","last_unstale":"2024-08-18T23:42:23.321522+0000","last_undegraded":"2024-08-18T23:42:23.321522+0000","last_fullsized":"2024-08-18T23:42:23.321522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:30:16.945045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316398+0000","last_change":"2024-08-18T23:41:32.149853+0000","last_active":"2024-08-18T23:42:23.316398+0000","last_peered":"2024-08-18T23:42:23.316398+0000","last_clean":"2024-08-18T23:42:23.316398+0000","last_became_active":"2024-08-18T23:41:32.141907+0000","last_became_peered":"2024-08-18T23:41:32.141907+0000","last_unstale":"2024-08-18T23:42:23.316398+0000","last_undegraded":"2024-08-18T23:42:23.316398+0000","last_fullsized":"2024-08-18T23:42:23.316398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:24:37.323567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314554+0000","last_change":"2024-08-18T23:41:32.136179+0000","last_active":"2024-08-18T23:42:23.314554+0000","last_peered":"2024-08-18T23:42:23.314554+0000","last_clean":"2024-08-18T23:42:23.314554+0000","last_became_active":"2024-08-18T23:41:32.135939+0000","last_became_peered":"2024-08-18T23:41:32.135939+0000","last_unstale":"2024-08-18T23:42:23.314554+0000","last_undegraded":"2024-08-18T23:42:23.314554+0000","last_fullsized":"2024-08-18T23:42:23.314554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:15.392061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318883+0000","last_change":"2024-08-18T23:41:32.137509+0000","last_active":"2024-08-18T23:42:23.318883+0000","last_peered":"2024-08-18T23:42:23.318883+0000","last_clean":"2024-08-18T23:42:23.318883+0000","last_became_active":"2024-08-18T23:41:32.136380+0000","last_became_peered":"2024-08-18T23:41:32.136380+0000","last_unstale":"2024-08-18T23:42:23.318883+0000","last_undegraded":"2024-08-18T23:42:23.318883+0000","last_fullsized":"2024-08-18T23:42:23.318883+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:49.776225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322689+0000","last_change":"2024-08-18T23:41:32.139271+0000","last_active":"2024-08-18T23:42:23.322689+0000","last_peered":"2024-08-18T23:42:23.322689+0000","last_clean":"2024-08-18T23:42:23.322689+0000","last_became_active":"2024-08-18T23:41:32.138868+0000","last_became_peered":"2024-08-18T23:41:32.138868+0000","last_unstale":"2024-08-18T23:42:23.322689+0000","last_undegraded":"2024-08-18T23:42:23.322689+0000","last_fullsized":"2024-08-18T23:42:23.322689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:49:27.890857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557875+0000","last_change":"2024-08-18T23:41:32.144460+0000","last_active":"2024-08-18T23:42:23.557875+0000","last_peered":"2024-08-18T23:42:23.557875+0000","last_clean":"2024-08-18T23:42:23.557875+0000","last_became_active":"2024-08-18T23:41:32.144178+0000","last_became_peered":"2024-08-18T23:41:32.144178+0000","last_unstale":"2024-08-18T23:42:23.557875+0000","last_undegraded":"2024-08-18T23:42:23.557875+0000","last_fullsized":"2024-08-18T23:42:23.557875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:56.252215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320708+0000","last_change":"2024-08-18T23:41:32.147973+0000","last_active":"2024-08-18T23:42:23.320708+0000","last_peered":"2024-08-18T23:42:23.320708+0000","last_clean":"2024-08-18T23:42:23.320708+0000","last_became_active":"2024-08-18T23:41:32.147621+0000","last_became_peered":"2024-08-18T23:41:32.147621+0000","last_unstale":"2024-08-18T23:42:23.320708+0000","last_undegraded":"2024-08-18T23:42:23.320708+0000","last_fullsized":"2024-08-18T23:42:23.320708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:14.176569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557525+0000","last_change":"2024-08-18T23:41:32.146029+0000","last_active":"2024-08-18T23:42:23.557525+0000","last_peered":"2024-08-18T23:42:23.557525+0000","last_clean":"2024-08-18T23:42:23.557525+0000","last_became_active":"2024-08-18T23:41:32.145584+0000","last_became_peered":"2024-08-18T23:41:32.145584+0000","last_unstale":"2024-08-18T23:42:23.557525+0000","last_undegraded":"2024-08-18T23:42:23.557525+0000","last_fullsized":"2024-08-18T23:42:23.557525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:25:33.884906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.315440+0000","last_change":"2024-08-18T23:41:32.145950+0000","last_active":"2024-08-18T23:42:23.315440+0000","last_peered":"2024-08-18T23:42:23.315440+0000","last_clean":"2024-08-18T23:42:23.315440+0000","last_became_active":"2024-08-18T23:41:32.145618+0000","last_became_peered":"2024-08-18T23:41:32.145618+0000","last_unstale":"2024-08-18T23:42:23.315440+0000","last_undegraded":"2024-08-18T23:42:23.315440+0000","last_fullsized":"2024-08-18T23:42:23.315440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:05:58.889240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314395+0000","last_change":"2024-08-18T23:41:32.130684+0000","last_active":"2024-08-18T23:42:23.314395+0000","last_peered":"2024-08-18T23:42:23.314395+0000","last_clean":"2024-08-18T23:42:23.314395+0000","last_became_active":"2024-08-18T23:41:32.130369+0000","last_became_peered":"2024-08-18T23:41:32.130369+0000","last_unstale":"2024-08-18T23:42:23.314395+0000","last_undegraded":"2024-08-18T23:42:23.314395+0000","last_fullsized":"2024-08-18T23:42:23.314395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:22.537018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.320123+0000","last_change":"2024-08-18T23:41:32.137291+0000","last_active":"2024-08-18T23:42:23.320123+0000","last_peered":"2024-08-18T23:42:23.320123+0000","last_clean":"2024-08-18T23:42:23.320123+0000","last_became_active":"2024-08-18T23:41:32.136977+0000","last_became_peered":"2024-08-18T23:41:32.136977+0000","last_unstale":"2024-08-18T23:42:23.320123+0000","last_undegraded":"2024-08-18T23:42:23.320123+0000","last_fullsized":"2024-08-18T23:42:23.320123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:18:45.544760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.322146+0000","last_change":"2024-08-18T23:41:32.144434+0000","last_active":"2024-08-18T23:42:23.322146+0000","last_peered":"2024-08-18T23:42:23.322146+0000","last_clean":"2024-08-18T23:42:23.322146+0000","last_became_active":"2024-08-18T23:41:32.144119+0000","last_became_peered":"2024-08-18T23:41:32.144119+0000","last_unstale":"2024-08-18T23:42:23.322146+0000","last_undegraded":"2024-08-18T23:42:23.322146+0000","last_fullsized":"2024-08-18T23:42:23.322146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:50:54.528630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319436+0000","last_change":"2024-08-18T23:41:32.135034+0000","last_active":"2024-08-18T23:42:23.319436+0000","last_peered":"2024-08-18T23:42:23.319436+0000","last_clean":"2024-08-18T23:42:23.319436+0000","last_became_active":"2024-08-18T23:41:32.134700+0000","last_became_peered":"2024-08-18T23:41:32.134700+0000","last_unstale":"2024-08-18T23:42:23.319436+0000","last_undegraded":"2024-08-18T23:42:23.319436+0000","last_fullsized":"2024-08-18T23:42:23.319436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:06:31.263114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319688+0000","last_change":"2024-08-18T23:41:32.141191+0000","last_active":"2024-08-18T23:42:23.319688+0000","last_peered":"2024-08-18T23:42:23.319688+0000","last_clean":"2024-08-18T23:42:23.319688+0000","last_became_active":"2024-08-18T23:41:32.140901+0000","last_became_peered":"2024-08-18T23:41:32.140901+0000","last_unstale":"2024-08-18T23:42:23.319688+0000","last_undegraded":"2024-08-18T23:42:23.319688+0000","last_fullsized":"2024-08-18T23:42:23.319688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:24.849985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314798+0000","last_change":"2024-08-18T23:41:32.142543+0000","last_active":"2024-08-18T23:42:23.314798+0000","last_peered":"2024-08-18T23:42:23.314798+0000","last_clean":"2024-08-18T23:42:23.314798+0000","last_became_active":"2024-08-18T23:41:32.142084+0000","last_became_peered":"2024-08-18T23:41:32.142084+0000","last_unstale":"2024-08-18T23:42:23.314798+0000","last_undegraded":"2024-08-18T23:42:23.314798+0000","last_fullsized":"2024-08-18T23:42:23.314798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:54:18.200016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316269+0000","last_change":"2024-08-18T23:41:32.148968+0000","last_active":"2024-08-18T23:42:23.316269+0000","last_peered":"2024-08-18T23:42:23.316269+0000","last_clean":"2024-08-18T23:42:23.316269+0000","last_became_active":"2024-08-18T23:41:32.148625+0000","last_became_peered":"2024-08-18T23:41:32.148625+0000","last_unstale":"2024-08-18T23:42:23.316269+0000","last_undegraded":"2024-08-18T23:42:23.316269+0000","last_fullsized":"2024-08-18T23:42:23.316269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:11:40.172139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316345+0000","last_change":"2024-08-18T23:41:32.133478+0000","last_active":"2024-08-18T23:42:23.316345+0000","last_peered":"2024-08-18T23:42:23.316345+0000","last_clean":"2024-08-18T23:42:23.316345+0000","last_became_active":"2024-08-18T23:41:32.133149+0000","last_became_peered":"2024-08-18T23:41:32.133149+0000","last_unstale":"2024-08-18T23:42:23.316345+0000","last_undegraded":"2024-08-18T23:42:23.316345+0000","last_fullsized":"2024-08-18T23:42:23.316345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:57:53.713309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.314293+0000","last_change":"2024-08-18T23:41:32.138206+0000","last_active":"2024-08-18T23:42:23.314293+0000","last_peered":"2024-08-18T23:42:23.314293+0000","last_clean":"2024-08-18T23:42:23.314293+0000","last_became_active":"2024-08-18T23:41:32.137903+0000","last_became_peered":"2024-08-18T23:41:32.137903+0000","last_unstale":"2024-08-18T23:42:23.314293+0000","last_undegraded":"2024-08-18T23:42:23.314293+0000","last_fullsized":"2024-08-18T23:42:23.314293+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:18:48.600658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319090+0000","last_change":"2024-08-18T23:41:32.136303+0000","last_active":"2024-08-18T23:42:23.319090+0000","last_peered":"2024-08-18T23:42:23.319090+0000","last_clean":"2024-08-18T23:42:23.319090+0000","last_became_active":"2024-08-18T23:41:32.134628+0000","last_became_peered":"2024-08-18T23:41:32.134628+0000","last_unstale":"2024-08-18T23:42:23.319090+0000","last_undegraded":"2024-08-18T23:42:23.319090+0000","last_fullsized":"2024-08-18T23:42:23.319090+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:28:51.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.317055+0000","last_change":"2024-08-18T23:41:32.137512+0000","last_active":"2024-08-18T23:42:23.317055+0000","last_peered":"2024-08-18T23:42:23.317055+0000","last_clean":"2024-08-18T23:42:23.317055+0000","last_became_active":"2024-08-18T23:41:32.137150+0000","last_became_peered":"2024-08-18T23:41:32.137150+0000","last_unstale":"2024-08-18T23:42:23.317055+0000","last_undegraded":"2024-08-18T23:42:23.317055+0000","last_fullsized":"2024-08-18T23:42:23.317055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:44:08.020780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558680+0000","last_change":"2024-08-18T23:41:32.139229+0000","last_active":"2024-08-18T23:42:23.558680+0000","last_peered":"2024-08-18T23:42:23.558680+0000","last_clean":"2024-08-18T23:42:23.558680+0000","last_became_active":"2024-08-18T23:41:32.138864+0000","last_became_peered":"2024-08-18T23:41:32.138864+0000","last_unstale":"2024-08-18T23:42:23.558680+0000","last_undegraded":"2024-08-18T23:42:23.558680+0000","last_fullsized":"2024-08-18T23:42:23.558680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:54:21.701750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316215+0000","last_change":"2024-08-18T23:41:32.148727+0000","last_active":"2024-08-18T23:42:23.316215+0000","last_peered":"2024-08-18T23:42:23.316215+0000","last_clean":"2024-08-18T23:42:23.316215+0000","last_became_active":"2024-08-18T23:41:32.146828+0000","last_became_peered":"2024-08-18T23:41:32.146828+0000","last_unstale":"2024-08-18T23:42:23.316215+0000","last_undegraded":"2024-08-18T23:42:23.316215+0000","last_fullsized":"2024-08-18T23:42:23.316215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:51:49.175150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.557445+0000","last_change":"2024-08-18T23:41:32.144796+0000","last_active":"2024-08-18T23:42:23.557445+0000","last_peered":"2024-08-18T23:42:23.557445+0000","last_clean":"2024-08-18T23:42:23.557445+0000","last_became_active":"2024-08-18T23:41:32.141931+0000","last_became_peered":"2024-08-18T23:41:32.141931+0000","last_unstale":"2024-08-18T23:42:23.557445+0000","last_undegraded":"2024-08-18T23:42:23.557445+0000","last_fullsized":"2024-08-18T23:42:23.557445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:48:08.428845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316954+0000","last_change":"2024-08-18T23:41:32.135265+0000","last_active":"2024-08-18T23:42:23.316954+0000","last_peered":"2024-08-18T23:42:23.316954+0000","last_clean":"2024-08-18T23:42:23.316954+0000","last_became_active":"2024-08-18T23:41:32.134952+0000","last_became_peered":"2024-08-18T23:41:32.134952+0000","last_unstale":"2024-08-18T23:42:23.316954+0000","last_undegraded":"2024-08-18T23:42:23.316954+0000","last_fullsized":"2024-08-18T23:42:23.316954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:35:14.099536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.558127+0000","last_change":"2024-08-18T23:41:32.135941+0000","last_active":"2024-08-18T23:42:23.558127+0000","last_peered":"2024-08-18T23:42:23.558127+0000","last_clean":"2024-08-18T23:42:23.558127+0000","last_became_active":"2024-08-18T23:41:32.135623+0000","last_became_peered":"2024-08-18T23:41:32.135623+0000","last_unstale":"2024-08-18T23:42:23.558127+0000","last_undegraded":"2024-08-18T23:42:23.558127+0000","last_fullsized":"2024-08-18T23:42:23.558127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:43:36.504738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.324078+0000","last_change":"2024-08-18T23:41:32.141643+0000","last_active":"2024-08-18T23:42:23.324078+0000","last_peered":"2024-08-18T23:42:23.324078+0000","last_clean":"2024-08-18T23:42:23.324078+0000","last_became_active":"2024-08-18T23:41:32.141301+0000","last_became_peered":"2024-08-18T23:41:32.141301+0000","last_unstale":"2024-08-18T23:42:23.324078+0000","last_undegraded":"2024-08-18T23:42:23.324078+0000","last_fullsized":"2024-08-18T23:42:23.324078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:23:53.100740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.318979+0000","last_change":"2024-08-18T23:41:32.146730+0000","last_active":"2024-08-18T23:42:23.318979+0000","last_peered":"2024-08-18T23:42:23.318979+0000","last_clean":"2024-08-18T23:42:23.318979+0000","last_became_active":"2024-08-18T23:41:32.146252+0000","last_became_peered":"2024-08-18T23:41:32.146252+0000","last_unstale":"2024-08-18T23:42:23.318979+0000","last_undegraded":"2024-08-18T23:42:23.318979+0000","last_fullsized":"2024-08-18T23:42:23.318979+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:41.481761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316570+0000","last_change":"2024-08-18T23:41:32.149569+0000","last_active":"2024-08-18T23:42:23.316570+0000","last_peered":"2024-08-18T23:42:23.316570+0000","last_clean":"2024-08-18T23:42:23.316570+0000","last_became_active":"2024-08-18T23:41:32.149277+0000","last_became_peered":"2024-08-18T23:41:32.149277+0000","last_unstale":"2024-08-18T23:42:23.316570+0000","last_undegraded":"2024-08-18T23:42:23.316570+0000","last_fullsized":"2024-08-18T23:42:23.316570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:34:00.547909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.316678+0000","last_change":"2024-08-18T23:41:32.137889+0000","last_active":"2024-08-18T23:42:23.316678+0000","last_peered":"2024-08-18T23:42:23.316678+0000","last_clean":"2024-08-18T23:42:23.316678+0000","last_became_active":"2024-08-18T23:41:32.137578+0000","last_became_peered":"2024-08-18T23:41:32.137578+0000","last_unstale":"2024-08-18T23:42:23.316678+0000","last_undegraded":"2024-08-18T23:42:23.316678+0000","last_fullsized":"2024-08-18T23:42:23.316678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:18:09.987838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-18T23:42:23.319506+0000","last_change":"2024-08-18T23:41:32.142084+0000","last_active":"2024-08-18T23:42:23.319506+0000","last_peered":"2024-08-18T23:42:23.319506+0000","last_clean":"2024-08-18T23:42:23.319506+0000","last_became_active":"2024-08-18T23:41:32.141670+0000","last_became_peered":"2024-08-18T23:41:32.141670+0000","last_unstale":"2024-08-18T23:42:23.319506+0000","last_undegraded":"2024-08-18T23:42:23.319506+0000","last_fullsized":"2024-08-18T23:42:23.319506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:03.990351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67386,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":426,"num_read_kb":285,"num_write":167,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":202158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514530,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":505014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776167,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28688,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716464,"statfs":{"total":95995035648,"available":95965659136,"internally_reserved":0,"allocated":1712128,"data_stored":1155668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939691,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":509346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168624,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":509266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397555,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36936,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708216,"statfs":{"total":95995035648,"available":95957213184,"internally_reserved":0,"allocated":1630208,"data_stored":1101468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561081,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":500426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790013,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1092570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953538,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":506048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":500620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":564490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":563332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":501814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T23:43:16.934 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-18T23:43:16.934 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-18T23:43:16.934 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-08-18T23:43:16.935 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph health --format=json 2024-08-18T23:43:17.191 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:17.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:17 smithi123 ceph-mon[21808]: pgmap v317: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:17.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:17 smithi123 ceph-mon[21808]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:43:17.619 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:17 smithi078 ceph-mon[18440]: pgmap v317: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:17.619 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:17 smithi078 ceph-mon[18440]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:43:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:17 smithi161 ceph-mon[22027]: pgmap v317: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:17 smithi161 ceph-mon[22027]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:43:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.697+0000 7ff48fa0c640 1 -- 172.21.15.78:0/470622380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff488100b50 msgr2=0x7ff488100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.697+0000 7ff48fa0c640 1 --2- 172.21.15.78:0/470622380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff488100b50 0x7ff488100f30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff484009900 tx=0x7ff48402a480 comp rx=0 tx=0).stop 2024-08-18T23:43:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.697+0000 7ff48fa0c640 1 -- 172.21.15.78:0/470622380 shutdown_connections 2024-08-18T23:43:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.697+0000 7ff48fa0c640 1 --2- 172.21.15.78:0/470622380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff488102c20 0x7ff48810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.697+0000 7ff48fa0c640 1 --2- 172.21.15.78:0/470622380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4881022a0 0x7ff4881026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.697+0000 7ff48fa0c640 1 --2- 172.21.15.78:0/470622380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff488100b50 0x7ff488100f30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.697+0000 7ff48fa0c640 1 -- 172.21.15.78:0/470622380 >> 172.21.15.78:0/470622380 conn(0x7ff4880fc500 msgr2=0x7ff4880fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.697+0000 7ff48fa0c640 1 -- 172.21.15.78:0/470622380 shutdown_connections 2024-08-18T23:43:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.697+0000 7ff48fa0c640 1 -- 172.21.15.78:0/470622380 wait complete. 2024-08-18T23:43:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48fa0c640 1 Processor -- start 2024-08-18T23:43:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48fa0c640 1 -- start start 2024-08-18T23:43:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48fa0c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff488100b50 0x7ff488198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48fa0c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4881022a0 0x7ff488198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48fa0c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff488102c20 0x7ff48819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48fa0c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff488110cb0 con 0x7ff488102c20 2024-08-18T23:43:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48fa0c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff488110b30 con 0x7ff4881022a0 2024-08-18T23:43:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48fa0c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff488110e30 con 0x7ff488100b50 2024-08-18T23:43:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48dfa9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff488102c20 0x7ff48819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48dfa9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff488102c20 0x7ff48819cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51792/0 (socket says 172.21.15.78:51792) 2024-08-18T23:43:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48dfa9640 1 -- 172.21.15.78:0/2730837841 learned_addr learned my addr 172.21.15.78:0/2730837841 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48d7a8640 1 --2- 172.21.15.78:0/2730837841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff488100b50 0x7ff488198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48cfa7640 1 --2- 172.21.15.78:0/2730837841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4881022a0 0x7ff488198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48dfa9640 1 -- 172.21.15.78:0/2730837841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff488100b50 msgr2=0x7ff488198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48dfa9640 1 --2- 172.21.15.78:0/2730837841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff488100b50 0x7ff488198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48dfa9640 1 -- 172.21.15.78:0/2730837841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4881022a0 msgr2=0x7ff488198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48dfa9640 1 --2- 172.21.15.78:0/2730837841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4881022a0 0x7ff488198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48dfa9640 1 -- 172.21.15.78:0/2730837841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48819d1e0 con 0x7ff488102c20 2024-08-18T23:43:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48d7a8640 1 --2- 172.21.15.78:0/2730837841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff488100b50 0x7ff488198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:43:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48cfa7640 1 --2- 172.21.15.78:0/2730837841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4881022a0 0x7ff488198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:43:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.698+0000 7ff48dfa9640 1 --2- 172.21.15.78:0/2730837841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff488102c20 0x7ff48819cb00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff48000eaa0 tx=0x7ff48000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.699+0000 7ff47a7fc640 1 -- 172.21.15.78:0/2730837841 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48000cdb0 con 0x7ff488102c20 2024-08-18T23:43:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.699+0000 7ff48fa0c640 1 -- 172.21.15.78:0/2730837841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48819d4d0 con 0x7ff488102c20 2024-08-18T23:43:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.699+0000 7ff47a7fc640 1 -- 172.21.15.78:0/2730837841 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff480005020 con 0x7ff488102c20 2024-08-18T23:43:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.699+0000 7ff47a7fc640 1 -- 172.21.15.78:0/2730837841 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4800116b0 con 0x7ff488102c20 2024-08-18T23:43:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.699+0000 7ff48fa0c640 1 -- 172.21.15.78:0/2730837841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4881a4db0 con 0x7ff488102c20 2024-08-18T23:43:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.700+0000 7ff48fa0c640 1 -- 172.21.15.78:0/2730837841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff488100f30 con 0x7ff488102c20 2024-08-18T23:43:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.701+0000 7ff47a7fc640 1 -- 172.21.15.78:0/2730837841 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff480005560 con 0x7ff488102c20 2024-08-18T23:43:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.701+0000 7ff47a7fc640 1 --2- 172.21.15.78:0/2730837841 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff4740a2c10 0x7ff4740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.701+0000 7ff48d7a8640 1 --2- 172.21.15.78:0/2730837841 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff4740a2c10 0x7ff4740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.701+0000 7ff47a7fc640 1 -- 172.21.15.78:0/2730837841 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ff4800cf860 con 0x7ff488102c20 2024-08-18T23:43:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.702+0000 7ff48d7a8640 1 --2- 172.21.15.78:0/2730837841 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff4740a2c10 0x7ff4740a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff484005ab0 tx=0x7ff484005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.706+0000 7ff47a7fc640 1 -- 172.21.15.78:0/2730837841 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff480016030 con 0x7ff488102c20 2024-08-18T23:43:17.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.908+0000 7ff48fa0c640 1 -- 172.21.15.78:0/2730837841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff4881a4f80 con 0x7ff488102c20 2024-08-18T23:43:17.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.909+0000 7ff47a7fc640 1 -- 172.21.15.78:0/2730837841 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff48009c300 con 0x7ff488102c20 2024-08-18T23:43:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:43:17.910 INFO:teuthology.orchestra.run.smithi078.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-18T23:43:17.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.911+0000 7ff48fa0c640 1 -- 172.21.15.78:0/2730837841 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff4740a2c10 msgr2=0x7ff4740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:17.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.911+0000 7ff48fa0c640 1 --2- 172.21.15.78:0/2730837841 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff4740a2c10 0x7ff4740a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff484005ab0 tx=0x7ff484005a20 comp rx=0 tx=0).stop 2024-08-18T23:43:17.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.911+0000 7ff48fa0c640 1 -- 172.21.15.78:0/2730837841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff488102c20 msgr2=0x7ff48819cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:17.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.911+0000 7ff48fa0c640 1 --2- 172.21.15.78:0/2730837841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff488102c20 0x7ff48819cb00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff48000eaa0 tx=0x7ff48000ef60 comp rx=0 tx=0).stop 2024-08-18T23:43:17.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.911+0000 7ff48fa0c640 1 -- 172.21.15.78:0/2730837841 shutdown_connections 2024-08-18T23:43:17.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.911+0000 7ff48fa0c640 1 --2- 172.21.15.78:0/2730837841 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff4740a2c10 0x7ff4740a50d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:17.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.911+0000 7ff48fa0c640 1 --2- 172.21.15.78:0/2730837841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff488102c20 0x7ff48819cb00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:17.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.911+0000 7ff48fa0c640 1 --2- 172.21.15.78:0/2730837841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4881022a0 0x7ff488198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:17.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.911+0000 7ff48fa0c640 1 --2- 172.21.15.78:0/2730837841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff488100b50 0x7ff488198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:17.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.911+0000 7ff48fa0c640 1 -- 172.21.15.78:0/2730837841 >> 172.21.15.78:0/2730837841 conn(0x7ff4880fc500 msgr2=0x7ff488105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:17.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.911+0000 7ff48fa0c640 1 -- 172.21.15.78:0/2730837841 shutdown_connections 2024-08-18T23:43:17.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:17.911+0000 7ff48fa0c640 1 -- 172.21.15.78:0/2730837841 wait complete. 2024-08-18T23:43:18.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:18 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2730837841' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-08-18T23:43:18.615 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:18 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2730837841' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-08-18T23:43:18.644 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-08-18T23:43:18.644 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:43:18.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:18 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2730837841' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-08-18T23:43:18.896 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:19.301 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:19 smithi078 ceph-mon[18440]: pgmap v318: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.379+0000 7f99767be640 1 -- 172.21.15.78:0/1791557797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9970102a70 msgr2=0x7f9970102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.379+0000 7f99767be640 1 --2- 172.21.15.78:0/1791557797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9970102a70 0x7f9970102e50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9960009900 tx=0x7f996002a480 comp rx=0 tx=0).stop 2024-08-18T23:43:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.379+0000 7f99767be640 1 -- 172.21.15.78:0/1791557797 shutdown_connections 2024-08-18T23:43:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.379+0000 7f99767be640 1 --2- 172.21.15.78:0/1791557797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f997010bea0 0x7f997010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.379+0000 7f99767be640 1 --2- 172.21.15.78:0/1791557797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9970103390 0x7f997010b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:19.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.379+0000 7f99767be640 1 --2- 172.21.15.78:0/1791557797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9970102a70 0x7f9970102e50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.379+0000 7f99767be640 1 -- 172.21.15.78:0/1791557797 >> 172.21.15.78:0/1791557797 conn(0x7f99700fc4d0 msgr2=0x7f99700fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.379+0000 7f99767be640 1 -- 172.21.15.78:0/1791557797 shutdown_connections 2024-08-18T23:43:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.379+0000 7f99767be640 1 -- 172.21.15.78:0/1791557797 wait complete. 2024-08-18T23:43:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.380+0000 7f99767be640 1 Processor -- start 2024-08-18T23:43:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.380+0000 7f99767be640 1 -- start start 2024-08-18T23:43:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.380+0000 7f99767be640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9970102a70 0x7f99701981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.380+0000 7f99767be640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9970103390 0x7f99701986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f99767be640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f997010bea0 0x7f997019ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f99767be640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9970111ce0 con 0x7f9970102a70 2024-08-18T23:43:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f99767be640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9970111b60 con 0x7f997010bea0 2024-08-18T23:43:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f99767be640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9970111e60 con 0x7f9970103390 2024-08-18T23:43:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f996ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9970102a70 0x7f99701981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:19.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f996ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9970102a70 0x7f99701981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51800/0 (socket says 172.21.15.78:51800) 2024-08-18T23:43:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f996ffff640 1 -- 172.21.15.78:0/1883677551 learned_addr learned my addr 172.21.15.78:0/1883677551 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f9974d5b640 1 --2- 172.21.15.78:0/1883677551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f997010bea0 0x7f997019ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f996f7fe640 1 --2- 172.21.15.78:0/1883677551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9970103390 0x7f99701986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f996ffff640 1 -- 172.21.15.78:0/1883677551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9970103390 msgr2=0x7f99701986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f996ffff640 1 --2- 172.21.15.78:0/1883677551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9970103390 0x7f99701986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f996ffff640 1 -- 172.21.15.78:0/1883677551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f997010bea0 msgr2=0x7f997019ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f996ffff640 1 --2- 172.21.15.78:0/1883677551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f997010bea0 0x7f997019ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f996ffff640 1 -- 172.21.15.78:0/1883677551 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997019d150 con 0x7f9970102a70 2024-08-18T23:43:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f996ffff640 1 --2- 172.21.15.78:0/1883677551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9970102a70 0x7f99701981a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9960005ab0 tx=0x7f99600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f996d7fa640 1 -- 172.21.15.78:0/1883677551 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996003f030 con 0x7f9970102a70 2024-08-18T23:43:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f996d7fa640 1 -- 172.21.15.78:0/1883677551 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9960003ce0 con 0x7f9970102a70 2024-08-18T23:43:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f99767be640 1 -- 172.21.15.78:0/1883677551 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997019d3e0 con 0x7f9970102a70 2024-08-18T23:43:19.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f996d7fa640 1 -- 172.21.15.78:0/1883677551 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960035070 con 0x7f9970102a70 2024-08-18T23:43:19.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.381+0000 7f99767be640 1 -- 172.21.15.78:0/1883677551 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99701a4d20 con 0x7f9970102a70 2024-08-18T23:43:19.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.382+0000 7f99767be640 1 -- 172.21.15.78:0/1883677551 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9970102e50 con 0x7f9970102a70 2024-08-18T23:43:19.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.383+0000 7f996d7fa640 1 -- 172.21.15.78:0/1883677551 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9960042030 con 0x7f9970102a70 2024-08-18T23:43:19.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.384+0000 7f996d7fa640 1 --2- 172.21.15.78:0/1883677551 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f99580a2c10 0x7f99580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:19.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.384+0000 7f996f7fe640 1 --2- 172.21.15.78:0/1883677551 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f99580a2c10 0x7f99580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:19.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.384+0000 7f996d7fa640 1 -- 172.21.15.78:0/1883677551 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f99600efeb0 con 0x7f9970102a70 2024-08-18T23:43:19.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.384+0000 7f996f7fe640 1 --2- 172.21.15.78:0/1883677551 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f99580a2c10 0x7f99580a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f99700fea60 tx=0x7f996400a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:19.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.388+0000 7f996d7fa640 1 -- 172.21.15.78:0/1883677551 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f99600bc950 con 0x7f9970102a70 2024-08-18T23:43:19.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:19 smithi123 ceph-mon[21808]: pgmap v318: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:19.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.589+0000 7f99767be640 1 -- 172.21.15.78:0/1883677551 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9970106f40 con 0x7f9970102a70 2024-08-18T23:43:19.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.590+0000 7f996d7fa640 1 -- 172.21.15.78:0/1883677551 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11930 (secure 0 0 0) 0x7f99600bc250 con 0x7f9970102a70 2024-08-18T23:43:19.591 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:43:19.591 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":24,"btime":"2024-08-18T23:42:10:259464+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:42:09.974847+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:43:19.591 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 24 2024-08-18T23:43:19.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.592+0000 7f99767be640 1 -- 172.21.15.78:0/1883677551 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f99580a2c10 msgr2=0x7f99580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:19.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.592+0000 7f99767be640 1 --2- 172.21.15.78:0/1883677551 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f99580a2c10 0x7f99580a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f99700fea60 tx=0x7f996400a560 comp rx=0 tx=0).stop 2024-08-18T23:43:19.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.592+0000 7f99767be640 1 -- 172.21.15.78:0/1883677551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9970102a70 msgr2=0x7f99701981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:19.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.592+0000 7f99767be640 1 --2- 172.21.15.78:0/1883677551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9970102a70 0x7f99701981a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9960005ab0 tx=0x7f99600042f0 comp rx=0 tx=0).stop 2024-08-18T23:43:19.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.592+0000 7f99767be640 1 -- 172.21.15.78:0/1883677551 shutdown_connections 2024-08-18T23:43:19.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.592+0000 7f99767be640 1 --2- 172.21.15.78:0/1883677551 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f99580a2c10 0x7f99580a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:19.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.592+0000 7f99767be640 1 --2- 172.21.15.78:0/1883677551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f997010bea0 0x7f997019ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:19.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.592+0000 7f99767be640 1 --2- 172.21.15.78:0/1883677551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9970103390 0x7f99701986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:19.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.592+0000 7f99767be640 1 --2- 172.21.15.78:0/1883677551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9970102a70 0x7f99701981a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:19.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.592+0000 7f99767be640 1 -- 172.21.15.78:0/1883677551 >> 172.21.15.78:0/1883677551 conn(0x7f99700fc4d0 msgr2=0x7f99700fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:19.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.592+0000 7f99767be640 1 -- 172.21.15.78:0/1883677551 shutdown_connections 2024-08-18T23:43:19.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:19.592+0000 7f99767be640 1 -- 172.21.15.78:0/1883677551 wait complete. 2024-08-18T23:43:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:19 smithi161 ceph-mon[22027]: pgmap v318: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:20.311 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:43:20.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:20 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1883677551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:43:20.561 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:20.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:20 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1883677551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:43:20.683 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:20 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1883677551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:43:21.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.070+0000 7f47ecb58640 1 -- 172.21.15.78:0/2924716291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47e81067b0 msgr2=0x7f47e8106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.070+0000 7f47ecb58640 1 --2- 172.21.15.78:0/2924716291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47e81067b0 0x7f47e8106b90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009930 tx=0x7f47d802a420 comp rx=0 tx=0).stop 2024-08-18T23:43:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.070+0000 7f47ecb58640 1 -- 172.21.15.78:0/2924716291 shutdown_connections 2024-08-18T23:43:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.070+0000 7f47ecb58640 1 --2- 172.21.15.78:0/2924716291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47e8101150 0x7f47e810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.070+0000 7f47ecb58640 1 --2- 172.21.15.78:0/2924716291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e81007b0 0x7f47e8100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.070+0000 7f47ecb58640 1 --2- 172.21.15.78:0/2924716291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47e81067b0 0x7f47e8106b90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.070+0000 7f47ecb58640 1 -- 172.21.15.78:0/2924716291 >> 172.21.15.78:0/2924716291 conn(0x7f47e80fc4d0 msgr2=0x7f47e80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.070+0000 7f47ecb58640 1 -- 172.21.15.78:0/2924716291 shutdown_connections 2024-08-18T23:43:21.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.070+0000 7f47ecb58640 1 -- 172.21.15.78:0/2924716291 wait complete. 2024-08-18T23:43:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.071+0000 7f47ecb58640 1 Processor -- start 2024-08-18T23:43:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.071+0000 7f47ecb58640 1 -- start start 2024-08-18T23:43:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.071+0000 7f47ecb58640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47e81007b0 0x7f47e81982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.071+0000 7f47ecb58640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e8101150 0x7f47e8198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.071+0000 7f47ecb58640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47e81067b0 0x7f47e819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.071+0000 7f47ecb58640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8110c50 con 0x7f47e81067b0 2024-08-18T23:43:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.071+0000 7f47ecb58640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8110ad0 con 0x7f47e8101150 2024-08-18T23:43:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.071+0000 7f47ecb58640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8110dd0 con 0x7f47e81007b0 2024-08-18T23:43:21.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.071+0000 7f47e6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47e81067b0 0x7f47e819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:21.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.071+0000 7f47e6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47e81067b0 0x7f47e819cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51816/0 (socket says 172.21.15.78:51816) 2024-08-18T23:43:21.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.071+0000 7f47e6d9d640 1 -- 172.21.15.78:0/3890473823 learned_addr learned my addr 172.21.15.78:0/3890473823 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:21.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.072+0000 7f47e659c640 1 --2- 172.21.15.78:0/3890473823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47e81007b0 0x7f47e81982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:21.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.072+0000 7f47e5d9b640 1 --2- 172.21.15.78:0/3890473823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e8101150 0x7f47e8198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:21.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.072+0000 7f47e6d9d640 1 -- 172.21.15.78:0/3890473823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47e81007b0 msgr2=0x7f47e81982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:21.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.072+0000 7f47e6d9d640 1 --2- 172.21.15.78:0/3890473823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47e81007b0 0x7f47e81982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:21.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.072+0000 7f47e6d9d640 1 -- 172.21.15.78:0/3890473823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e8101150 msgr2=0x7f47e8198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:21.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.072+0000 7f47e6d9d640 1 --2- 172.21.15.78:0/3890473823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e8101150 0x7f47e8198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:21.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.072+0000 7f47e6d9d640 1 -- 172.21.15.78:0/3890473823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e819d2a0 con 0x7f47e81067b0 2024-08-18T23:43:21.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.072+0000 7f47e6d9d640 1 --2- 172.21.15.78:0/3890473823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47e81067b0 0x7f47e819cbc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f47e000b970 tx=0x7f47e000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:21.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.072+0000 7f47d77fe640 1 -- 172.21.15.78:0/3890473823 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0019070 con 0x7f47e81067b0 2024-08-18T23:43:21.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.072+0000 7f47d77fe640 1 -- 172.21.15.78:0/3890473823 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e000cb40 con 0x7f47e81067b0 2024-08-18T23:43:21.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.072+0000 7f47d77fe640 1 -- 172.21.15.78:0/3890473823 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0007650 con 0x7f47e81067b0 2024-08-18T23:43:21.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.072+0000 7f47ecb58640 1 -- 172.21.15.78:0/3890473823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e819d590 con 0x7f47e81067b0 2024-08-18T23:43:21.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.072+0000 7f47ecb58640 1 -- 172.21.15.78:0/3890473823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e81a4e70 con 0x7f47e81067b0 2024-08-18T23:43:21.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.073+0000 7f47ecb58640 1 -- 172.21.15.78:0/3890473823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e8100c90 con 0x7f47e81067b0 2024-08-18T23:43:21.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.074+0000 7f47d77fe640 1 -- 172.21.15.78:0/3890473823 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f47e001c070 con 0x7f47e81067b0 2024-08-18T23:43:21.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.075+0000 7f47d77fe640 1 --2- 172.21.15.78:0/3890473823 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f47d00a2c10 0x7f47d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:21.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.075+0000 7f47e659c640 1 --2- 172.21.15.78:0/3890473823 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f47d00a2c10 0x7f47d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:21.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.075+0000 7f47d77fe640 1 -- 172.21.15.78:0/3890473823 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f47e00cee70 con 0x7f47e81067b0 2024-08-18T23:43:21.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.075+0000 7f47e659c640 1 --2- 172.21.15.78:0/3890473823 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f47d00a2c10 0x7f47d00a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009900 tx=0x7f47d80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:21.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.078+0000 7f47d77fe640 1 -- 172.21.15.78:0/3890473823 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f47e009b3d0 con 0x7f47e81067b0 2024-08-18T23:43:21.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.281+0000 7f47ecb58640 1 -- 172.21.15.78:0/3890473823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47e8199650 con 0x7f47e81067b0 2024-08-18T23:43:21.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.281+0000 7f47d77fe640 1 -- 172.21.15.78:0/3890473823 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11930 (secure 0 0 0) 0x7f47e009f6d0 con 0x7f47e81067b0 2024-08-18T23:43:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:43:21.283 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":24,"btime":"2024-08-18T23:42:10:259464+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:42:09.974847+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:43:21.283 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 24 2024-08-18T23:43:21.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.283+0000 7f47ecb58640 1 -- 172.21.15.78:0/3890473823 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f47d00a2c10 msgr2=0x7f47d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:21.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.283+0000 7f47ecb58640 1 --2- 172.21.15.78:0/3890473823 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f47d00a2c10 0x7f47d00a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009900 tx=0x7f47d80058d0 comp rx=0 tx=0).stop 2024-08-18T23:43:21.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.283+0000 7f47ecb58640 1 -- 172.21.15.78:0/3890473823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47e81067b0 msgr2=0x7f47e819cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:21.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.283+0000 7f47ecb58640 1 --2- 172.21.15.78:0/3890473823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47e81067b0 0x7f47e819cbc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f47e000b970 tx=0x7f47e000be30 comp rx=0 tx=0).stop 2024-08-18T23:43:21.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.283+0000 7f47ecb58640 1 -- 172.21.15.78:0/3890473823 shutdown_connections 2024-08-18T23:43:21.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.284+0000 7f47ecb58640 1 --2- 172.21.15.78:0/3890473823 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f47d00a2c10 0x7f47d00a50d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:21.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.284+0000 7f47ecb58640 1 --2- 172.21.15.78:0/3890473823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47e81067b0 0x7f47e819cbc0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:21.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.284+0000 7f47ecb58640 1 --2- 172.21.15.78:0/3890473823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47e8101150 0x7f47e8198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:21.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.284+0000 7f47ecb58640 1 --2- 172.21.15.78:0/3890473823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47e81007b0 0x7f47e81982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:21.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.284+0000 7f47ecb58640 1 -- 172.21.15.78:0/3890473823 >> 172.21.15.78:0/3890473823 conn(0x7f47e80fc4d0 msgr2=0x7f47e8104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:21.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.284+0000 7f47ecb58640 1 -- 172.21.15.78:0/3890473823 shutdown_connections 2024-08-18T23:43:21.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:21.284+0000 7f47ecb58640 1 -- 172.21.15.78:0/3890473823 wait complete. 2024-08-18T23:43:21.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:21 smithi123 ceph-mon[21808]: pgmap v319: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:21.672 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:21 smithi078 ceph-mon[18440]: pgmap v319: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:21 smithi161 ceph-mon[22027]: pgmap v319: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:22.018 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-18T23:43:22.302 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:22.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:22 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3890473823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:43:22.578 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:22 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3890473823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:43:22.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:22 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3890473823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:43:22.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.795+0000 7f5f4ed28640 1 -- 172.21.15.78:0/2847755776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f48102260 msgr2=0x7f5f4810e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:22.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.795+0000 7f5f4ed28640 1 --2- 172.21.15.78:0/2847755776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f48102260 0x7f5f4810e790 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009cd0 tx=0x7f5f4002a8e0 comp rx=0 tx=0).stop 2024-08-18T23:43:22.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.795+0000 7f5f4ed28640 1 -- 172.21.15.78:0/2847755776 shutdown_connections 2024-08-18T23:43:22.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.795+0000 7f5f4ed28640 1 --2- 172.21.15.78:0/2847755776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f48102260 0x7f5f4810e790 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:22.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.795+0000 7f5f4ed28640 1 --2- 172.21.15.78:0/2847755776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f481018c0 0x7f5f48101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:22.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.795+0000 7f5f4ed28640 1 --2- 172.21.15.78:0/2847755776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f481078c0 0x7f5f48107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:22.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.795+0000 7f5f4ed28640 1 -- 172.21.15.78:0/2847755776 >> 172.21.15.78:0/2847755776 conn(0x7f5f480fd620 msgr2=0x7f5f480ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:22.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.795+0000 7f5f4ed28640 1 -- 172.21.15.78:0/2847755776 shutdown_connections 2024-08-18T23:43:22.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.795+0000 7f5f4ed28640 1 -- 172.21.15.78:0/2847755776 wait complete. 2024-08-18T23:43:22.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.795+0000 7f5f4ed28640 1 Processor -- start 2024-08-18T23:43:22.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.795+0000 7f5f4ed28640 1 -- start start 2024-08-18T23:43:22.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f4ed28640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f481018c0 0x7f5f481981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:22.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f4ed28640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f48102260 0x7f5f48198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:22.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f4ed28640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f481078c0 0x7f5f4819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:22.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f4ed28640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48111ce0 con 0x7f5f481018c0 2024-08-18T23:43:22.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f4ed28640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48111b60 con 0x7f5f481078c0 2024-08-18T23:43:22.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f4ed28640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48111e60 con 0x7f5f48102260 2024-08-18T23:43:22.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f4cac4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f481018c0 0x7f5f481981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:22.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f4cac4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f481018c0 0x7f5f481981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51828/0 (socket says 172.21.15.78:51828) 2024-08-18T23:43:22.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f4cac4640 1 -- 172.21.15.78:0/2364795368 learned_addr learned my addr 172.21.15.78:0/2364795368 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:22.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f4d2c5640 1 --2- 172.21.15.78:0/2364795368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f481078c0 0x7f5f4819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:22.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f47fff640 1 --2- 172.21.15.78:0/2364795368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f48102260 0x7f5f48198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:22.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f4cac4640 1 -- 172.21.15.78:0/2364795368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f48102260 msgr2=0x7f5f48198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:22.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f4cac4640 1 --2- 172.21.15.78:0/2364795368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f48102260 0x7f5f48198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:22.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f4cac4640 1 -- 172.21.15.78:0/2364795368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f481078c0 msgr2=0x7f5f4819ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:22.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f4cac4640 1 --2- 172.21.15.78:0/2364795368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f481078c0 0x7f5f4819ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:22.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f4cac4640 1 -- 172.21.15.78:0/2364795368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f4819d170 con 0x7f5f481018c0 2024-08-18T23:43:22.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f4cac4640 1 --2- 172.21.15.78:0/2364795368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f481018c0 0x7f5f481981c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00b4a0 tx=0x7f5f3c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:22.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f45ffb640 1 -- 172.21.15.78:0/2364795368 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c014070 con 0x7f5f481018c0 2024-08-18T23:43:22.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f45ffb640 1 -- 172.21.15.78:0/2364795368 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3c010af0 con 0x7f5f481018c0 2024-08-18T23:43:22.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f4ed28640 1 -- 172.21.15.78:0/2364795368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f4819d460 con 0x7f5f481018c0 2024-08-18T23:43:22.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.796+0000 7f5f45ffb640 1 -- 172.21.15.78:0/2364795368 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c017d10 con 0x7f5f481018c0 2024-08-18T23:43:22.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.797+0000 7f5f4ed28640 1 -- 172.21.15.78:0/2364795368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f481a4d40 con 0x7f5f481018c0 2024-08-18T23:43:22.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.797+0000 7f5f4ed28640 1 -- 172.21.15.78:0/2364795368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f48101d20 con 0x7f5f481018c0 2024-08-18T23:43:22.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.798+0000 7f5f45ffb640 1 -- 172.21.15.78:0/2364795368 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5f3c017070 con 0x7f5f481018c0 2024-08-18T23:43:22.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.799+0000 7f5f45ffb640 1 --2- 172.21.15.78:0/2364795368 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f5f380a2b80 0x7f5f380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:22.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.799+0000 7f5f45ffb640 1 -- 172.21.15.78:0/2364795368 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f5f3c0ce770 con 0x7f5f481018c0 2024-08-18T23:43:22.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.799+0000 7f5f47fff640 1 --2- 172.21.15.78:0/2364795368 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f5f380a2b80 0x7f5f380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:22.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.800+0000 7f5f47fff640 1 --2- 172.21.15.78:0/2364795368 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f5f380a2b80 0x7f5f380a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5f34005fd0 tx=0x7f5f34005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:22.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.803+0000 7f5f45ffb640 1 -- 172.21.15.78:0/2364795368 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5f3c09b370 con 0x7f5f481018c0 2024-08-18T23:43:22.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.970+0000 7f5f4ed28640 1 -- 172.21.15.78:0/2364795368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f48109eb0 con 0x7f5f481018c0 2024-08-18T23:43:22.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.971+0000 7f5f45ffb640 1 -- 172.21.15.78:0/2364795368 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24654 (secure 0 0 0) 0x7f5f3c09ac70 con 0x7f5f481018c0 2024-08-18T23:43:22.973 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:43:22.973 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","created":"2024-08-18T23:34:33.867246+0000","modified":"2024-08-18T23:42:23.304291+0000","last_up_change":"2024-08-18T23:40:42.200381+0000","last_in_change":"2024-08-18T23:40:22.425813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T23:37:22.338211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T23:41:28.920802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T23:41:31.026094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T23:41:37.403238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4de1c78-6ebc-49a9-b9c5-85d43535fab7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6803","nonce":3639645311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6805","nonce":3639645311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6809","nonce":3639645311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6807","nonce":3639645311}]},"public_addr":"172.21.15.78:6803/3639645311","cluster_addr":"172.21.15.78:6805/3639645311","heartbeat_back_addr":"172.21.15.78:6809/3639645311","heartbeat_front_addr":"172.21.15.78:6807/3639645311","state":["exists","up"]},{"osd":1,"uuid":"4f8e0c3d-472b-4230-ad23-2b0a34861251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6801","nonce":4199446368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6803","nonce":4199446368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6807","nonce":4199446368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6805","nonce":4199446368}]},"public_addr":"172.21.15.123:6801/4199446368","cluster_addr":"172.21.15.123:6803/4199446368","heartbeat_back_addr":"172.21.15.123:6807/4199446368","heartbeat_front_addr":"172.21.15.123:6805/4199446368","state":["exists","up"]},{"osd":2,"uuid":"27ef3314-bb6f-413d-be05-e10a105dec9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6801","nonce":2582810057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6803","nonce":2582810057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6807","nonce":2582810057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6805","nonce":2582810057}]},"public_addr":"172.21.15.161:6801/2582810057","cluster_addr":"172.21.15.161:6803/2582810057","heartbeat_back_addr":"172.21.15.161:6807/2582810057","heartbeat_front_addr":"172.21.15.161:6805/2582810057","state":["exists","up"]},{"osd":3,"uuid":"e9e52662-5ef9-4d70-bdd7-2b92165822ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6811","nonce":2697347521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6813","nonce":2697347521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6817","nonce":2697347521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6815","nonce":2697347521}]},"public_addr":"172.21.15.78:6811/2697347521","cluster_addr":"172.21.15.78:6813/2697347521","heartbeat_back_addr":"172.21.15.78:6817/2697347521","heartbeat_front_addr":"172.21.15.78:6815/2697347521","state":["exists","up"]},{"osd":4,"uuid":"5ed26775-79e3-4b24-9557-69fcf5e8e824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6809","nonce":1281511569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6811","nonce":1281511569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6815","nonce":1281511569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6813","nonce":1281511569}]},"public_addr":"172.21.15.123:6809/1281511569","cluster_addr":"172.21.15.123:6811/1281511569","heartbeat_back_addr":"172.21.15.123:6815/1281511569","heartbeat_front_addr":"172.21.15.123:6813/1281511569","state":["exists","up"]},{"osd":5,"uuid":"a9793b1d-5613-48a4-938e-adb718f4fa4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6809","nonce":2751796117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6811","nonce":2751796117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6815","nonce":2751796117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6813","nonce":2751796117}]},"public_addr":"172.21.15.161:6809/2751796117","cluster_addr":"172.21.15.161:6811/2751796117","heartbeat_back_addr":"172.21.15.161:6815/2751796117","heartbeat_front_addr":"172.21.15.161:6813/2751796117","state":["exists","up"]},{"osd":6,"uuid":"5dbc5e9c-a14f-4b71-910a-1e6d4827bb32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6819","nonce":95383223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6821","nonce":95383223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6825","nonce":95383223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6823","nonce":95383223}]},"public_addr":"172.21.15.78:6819/95383223","cluster_addr":"172.21.15.78:6821/95383223","heartbeat_back_addr":"172.21.15.78:6825/95383223","heartbeat_front_addr":"172.21.15.78:6823/95383223","state":["exists","up"]},{"osd":7,"uuid":"26fb7b69-131f-42ee-b745-6fe8b4f65391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6817","nonce":3440305156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6819","nonce":3440305156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6823","nonce":3440305156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6821","nonce":3440305156}]},"public_addr":"172.21.15.123:6817/3440305156","cluster_addr":"172.21.15.123:6819/3440305156","heartbeat_back_addr":"172.21.15.123:6823/3440305156","heartbeat_front_addr":"172.21.15.123:6821/3440305156","state":["exists","up"]},{"osd":8,"uuid":"e267bc50-0c29-4d7b-b368-b1621e29b2df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6817","nonce":1485008778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6819","nonce":1485008778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6823","nonce":1485008778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6821","nonce":1485008778}]},"public_addr":"172.21.15.161:6817/1485008778","cluster_addr":"172.21.15.161:6819/1485008778","heartbeat_back_addr":"172.21.15.161:6823/1485008778","heartbeat_front_addr":"172.21.15.161:6821/1485008778","state":["exists","up"]},{"osd":9,"uuid":"b7654818-b767-4fe6-8211-346297895f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6827","nonce":2884475661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6829","nonce":2884475661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6833","nonce":2884475661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6831","nonce":2884475661}]},"public_addr":"172.21.15.78:6827/2884475661","cluster_addr":"172.21.15.78:6829/2884475661","heartbeat_back_addr":"172.21.15.78:6833/2884475661","heartbeat_front_addr":"172.21.15.78:6831/2884475661","state":["exists","up"]},{"osd":10,"uuid":"59aa32ff-510e-4b52-8170-08a34a0b6b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6825","nonce":122226286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6827","nonce":122226286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6831","nonce":122226286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6829","nonce":122226286}]},"public_addr":"172.21.15.123:6825/122226286","cluster_addr":"172.21.15.123:6827/122226286","heartbeat_back_addr":"172.21.15.123:6831/122226286","heartbeat_front_addr":"172.21.15.123:6829/122226286","state":["exists","up"]},{"osd":11,"uuid":"67bacab4-d078-4c5c-b4ff-9f6a8a00f507","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6825","nonce":2037560985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6827","nonce":2037560985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6831","nonce":2037560985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6829","nonce":2037560985}]},"public_addr":"172.21.15.161:6825/2037560985","cluster_addr":"172.21.15.161:6827/2037560985","heartbeat_back_addr":"172.21.15.161:6831/2037560985","heartbeat_front_addr":"172.21.15.161:6829/2037560985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:36.826044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:57.688421+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:18.297424+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:40.355215+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:02.167902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:23.968759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:46.254083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:08.522038+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:31.028119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:54.119807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:16.847034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:39.607886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6800/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3040499240":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6801/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:6800/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/757409165":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/2562072965":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/636184437":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/1570708294":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3137806628":"2024-08-19T23:34:59.021165+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T23:43:22.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.973+0000 7f5f4ed28640 1 -- 172.21.15.78:0/2364795368 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f5f380a2b80 msgr2=0x7f5f380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:22.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.973+0000 7f5f4ed28640 1 --2- 172.21.15.78:0/2364795368 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f5f380a2b80 0x7f5f380a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5f34005fd0 tx=0x7f5f34005850 comp rx=0 tx=0).stop 2024-08-18T23:43:22.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.973+0000 7f5f4ed28640 1 -- 172.21.15.78:0/2364795368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f481018c0 msgr2=0x7f5f481981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:22.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.973+0000 7f5f4ed28640 1 --2- 172.21.15.78:0/2364795368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f481018c0 0x7f5f481981c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00b4a0 tx=0x7f5f3c00b960 comp rx=0 tx=0).stop 2024-08-18T23:43:22.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.974+0000 7f5f4ed28640 1 -- 172.21.15.78:0/2364795368 shutdown_connections 2024-08-18T23:43:22.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.974+0000 7f5f4ed28640 1 --2- 172.21.15.78:0/2364795368 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f5f380a2b80 0x7f5f380a5040 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:22.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.974+0000 7f5f4ed28640 1 --2- 172.21.15.78:0/2364795368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5f481078c0 0x7f5f4819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:22.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.974+0000 7f5f4ed28640 1 --2- 172.21.15.78:0/2364795368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f48102260 0x7f5f48198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:22.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.974+0000 7f5f4ed28640 1 --2- 172.21.15.78:0/2364795368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f481018c0 0x7f5f481981c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:22.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.974+0000 7f5f4ed28640 1 -- 172.21.15.78:0/2364795368 >> 172.21.15.78:0/2364795368 conn(0x7f5f480fd620 msgr2=0x7f5f480fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:22.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.974+0000 7f5f4ed28640 1 -- 172.21.15.78:0/2364795368 shutdown_connections 2024-08-18T23:43:22.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:22.974+0000 7f5f4ed28640 1 -- 172.21.15.78:0/2364795368 wait complete. 2024-08-18T23:43:23.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:23 smithi123 ceph-mon[21808]: pgmap v320: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:23.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:23 smithi123 ceph-mon[21808]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-08-18T23:43:23.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-08-18T23:43:23.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:23 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-08-18T23:43:23.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:23 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2364795368' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:43:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:23 smithi161 ceph-mon[22027]: pgmap v320: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:23 smithi161 ceph-mon[22027]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-08-18T23:43:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-08-18T23:43:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:23 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-08-18T23:43:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:23 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2364795368' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:43:23.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:23 smithi078 ceph-mon[18440]: pgmap v320: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:23.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:23 smithi078 ceph-mon[18440]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-08-18T23:43:23.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-08-18T23:43:23.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:23 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-08-18T23:43:23.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:23 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2364795368' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:43:23.827 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:43:24.075 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:24.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:24 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-08-18T23:43:24.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:24 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-08-18T23:43:24.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:24 smithi078 ceph-mon[18440]: osdmap e91: 12 total, 12 up, 12 in 2024-08-18T23:43:24.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:24 smithi078 ceph-mon[18440]: pgmap v322: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-18T23:43:24.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:24 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-08-18T23:43:24.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:24 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-08-18T23:43:24.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:24 smithi123 ceph-mon[21808]: osdmap e91: 12 total, 12 up, 12 in 2024-08-18T23:43:24.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:24 smithi123 ceph-mon[21808]: pgmap v322: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-18T23:43:24.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.594+0000 7efe0124a640 1 -- 172.21.15.78:0/762126719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdfc1078b0 msgr2=0x7efdfc107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.594+0000 7efe0124a640 1 --2- 172.21.15.78:0/762126719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdfc1078b0 0x7efdfc107c90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efdec009930 tx=0x7efdec02a420 comp rx=0 tx=0).stop 2024-08-18T23:43:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.594+0000 7efe0124a640 1 -- 172.21.15.78:0/762126719 shutdown_connections 2024-08-18T23:43:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.594+0000 7efe0124a640 1 --2- 172.21.15.78:0/762126719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdfc102250 0x7efdfc10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.594+0000 7efe0124a640 1 --2- 172.21.15.78:0/762126719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdfc1018b0 0x7efdfc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.594+0000 7efe0124a640 1 --2- 172.21.15.78:0/762126719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdfc1078b0 0x7efdfc107c90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.594+0000 7efe0124a640 1 -- 172.21.15.78:0/762126719 >> 172.21.15.78:0/762126719 conn(0x7efdfc0fd5f0 msgr2=0x7efdfc0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.594+0000 7efe0124a640 1 -- 172.21.15.78:0/762126719 shutdown_connections 2024-08-18T23:43:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.594+0000 7efe0124a640 1 -- 172.21.15.78:0/762126719 wait complete. 2024-08-18T23:43:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.595+0000 7efe0124a640 1 Processor -- start 2024-08-18T23:43:24.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.595+0000 7efe0124a640 1 -- start start 2024-08-18T23:43:24.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.595+0000 7efe0124a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdfc1018b0 0x7efdfc198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:24.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.595+0000 7efe0124a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdfc102250 0x7efdfc198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:24.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.595+0000 7efe0124a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdfc1078b0 0x7efdfc19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:24.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.595+0000 7efe0124a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc111cd0 con 0x7efdfc102250 2024-08-18T23:43:24.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.595+0000 7efe0124a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc111b50 con 0x7efdfc1078b0 2024-08-18T23:43:24.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.595+0000 7efe0124a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc111e50 con 0x7efdfc1018b0 2024-08-18T23:43:24.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.595+0000 7efdfa59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdfc102250 0x7efdfc198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:24.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.595+0000 7efdfa59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdfc102250 0x7efdfc198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51834/0 (socket says 172.21.15.78:51834) 2024-08-18T23:43:24.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.595+0000 7efdfa59c640 1 -- 172.21.15.78:0/3285234706 learned_addr learned my addr 172.21.15.78:0/3285234706 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:24.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.596+0000 7efdfad9d640 1 --2- 172.21.15.78:0/3285234706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdfc1018b0 0x7efdfc198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.596+0000 7efdfb59e640 1 --2- 172.21.15.78:0/3285234706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdfc1078b0 0x7efdfc19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.596+0000 7efdfa59c640 1 -- 172.21.15.78:0/3285234706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdfc1018b0 msgr2=0x7efdfc198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.596+0000 7efdfa59c640 1 --2- 172.21.15.78:0/3285234706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdfc1018b0 0x7efdfc198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.596+0000 7efdfa59c640 1 -- 172.21.15.78:0/3285234706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdfc1078b0 msgr2=0x7efdfc19cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.596+0000 7efdfa59c640 1 --2- 172.21.15.78:0/3285234706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdfc1078b0 0x7efdfc19cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.596+0000 7efdfa59c640 1 -- 172.21.15.78:0/3285234706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdfc19d1e0 con 0x7efdfc102250 2024-08-18T23:43:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.596+0000 7efdfb59e640 1 --2- 172.21.15.78:0/3285234706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdfc1078b0 0x7efdfc19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:43:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.596+0000 7efdfa59c640 1 --2- 172.21.15.78:0/3285234706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdfc102250 0x7efdfc198770 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efdf000b4a0 tx=0x7efdf000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.596+0000 7efde3fff640 1 -- 172.21.15.78:0/3285234706 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0014070 con 0x7efdfc102250 2024-08-18T23:43:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.596+0000 7efe0124a640 1 -- 172.21.15.78:0/3285234706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdfc19d4d0 con 0x7efdfc102250 2024-08-18T23:43:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.596+0000 7efde3fff640 1 -- 172.21.15.78:0/3285234706 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdf0010af0 con 0x7efdfc102250 2024-08-18T23:43:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.596+0000 7efde3fff640 1 -- 172.21.15.78:0/3285234706 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0017d10 con 0x7efdfc102250 2024-08-18T23:43:24.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.596+0000 7efe0124a640 1 -- 172.21.15.78:0/3285234706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdfc1a4db0 con 0x7efdfc102250 2024-08-18T23:43:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.597+0000 7efe0124a640 1 -- 172.21.15.78:0/3285234706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdfc101d10 con 0x7efdfc102250 2024-08-18T23:43:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.598+0000 7efde3fff640 1 -- 172.21.15.78:0/3285234706 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efdf0017070 con 0x7efdfc102250 2024-08-18T23:43:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.599+0000 7efde3fff640 1 --2- 172.21.15.78:0/3285234706 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efde40a2b80 0x7efde40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.599+0000 7efdfad9d640 1 --2- 172.21.15.78:0/3285234706 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efde40a2b80 0x7efde40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.599+0000 7efde3fff640 1 -- 172.21.15.78:0/3285234706 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7efdf00ce7a0 con 0x7efdfc102250 2024-08-18T23:43:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.599+0000 7efdfad9d640 1 --2- 172.21.15.78:0/3285234706 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efde40a2b80 0x7efde40a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efdec009900 tx=0x7efdec0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:24.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.603+0000 7efde3fff640 1 -- 172.21.15.78:0/3285234706 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7efdf009b370 con 0x7efdfc102250 2024-08-18T23:43:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:24 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-08-18T23:43:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:24 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-08-18T23:43:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:24 smithi161 ceph-mon[22027]: osdmap e91: 12 total, 12 up, 12 in 2024-08-18T23:43:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:24 smithi161 ceph-mon[22027]: pgmap v322: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-18T23:43:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.804+0000 7efe0124a640 1 -- 172.21.15.78:0/3285234706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdfc02d5e0 con 0x7efdfc102250 2024-08-18T23:43:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.804+0000 7efde3fff640 1 -- 172.21.15.78:0/3285234706 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11930 (secure 0 0 0) 0x7efdf009ac70 con 0x7efdfc102250 2024-08-18T23:43:24.806 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:43:24.806 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":24,"btime":"2024-08-18T23:42:10:259464+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:42:09.974847+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:43:24.806 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 24 2024-08-18T23:43:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.806+0000 7efe0124a640 1 -- 172.21.15.78:0/3285234706 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efde40a2b80 msgr2=0x7efde40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.806+0000 7efe0124a640 1 --2- 172.21.15.78:0/3285234706 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efde40a2b80 0x7efde40a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efdec009900 tx=0x7efdec0058d0 comp rx=0 tx=0).stop 2024-08-18T23:43:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.806+0000 7efe0124a640 1 -- 172.21.15.78:0/3285234706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdfc102250 msgr2=0x7efdfc198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.806+0000 7efe0124a640 1 --2- 172.21.15.78:0/3285234706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdfc102250 0x7efdfc198770 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efdf000b4a0 tx=0x7efdf000b960 comp rx=0 tx=0).stop 2024-08-18T23:43:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.807+0000 7efe0124a640 1 -- 172.21.15.78:0/3285234706 shutdown_connections 2024-08-18T23:43:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.807+0000 7efe0124a640 1 --2- 172.21.15.78:0/3285234706 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efde40a2b80 0x7efde40a5040 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.807+0000 7efe0124a640 1 --2- 172.21.15.78:0/3285234706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdfc1078b0 0x7efdfc19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.807+0000 7efe0124a640 1 --2- 172.21.15.78:0/3285234706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdfc102250 0x7efdfc198770 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.807+0000 7efe0124a640 1 --2- 172.21.15.78:0/3285234706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdfc1018b0 0x7efdfc198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.807+0000 7efe0124a640 1 -- 172.21.15.78:0/3285234706 >> 172.21.15.78:0/3285234706 conn(0x7efdfc0fd5f0 msgr2=0x7efdfc0fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.807+0000 7efe0124a640 1 -- 172.21.15.78:0/3285234706 shutdown_connections 2024-08-18T23:43:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:24.807+0000 7efe0124a640 1 -- 172.21.15.78:0/3285234706 wait complete. 2024-08-18T23:43:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:25 smithi078 ceph-mon[18440]: osdmap e92: 12 total, 12 up, 12 in 2024-08-18T23:43:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:25 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3285234706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:43:25.601 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-18T23:41:32.941573+0000', 'modified': '2024-08-18T23:42:09.974847+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24452, 'mds_1': 24428, 'mds_2': 24427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24397': {'gid': 24397, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/4100464644', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 4100464644}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 4100464644}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6837/2017185021', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 2017185021}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 2017185021}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6835/2459199316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 2459199316}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 2459199316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.161:6839/2209088174', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 2209088174}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 2209088174}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.123:6837/1639958554', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 1639958554}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 1639958554}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.123:6839/2656330956', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 2656330956}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 2656330956}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24452, 'qdb_cluster': [24427, 24428, 24452]} 2024-08-18T23:43:25.601 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-18T23:43:25.602 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.i status 2024-08-18T23:43:25.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:25 smithi161 ceph-mon[22027]: osdmap e92: 12 total, 12 up, 12 in 2024-08-18T23:43:25.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:25 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3285234706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:43:25.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:25 smithi123 ceph-mon[21808]: osdmap e92: 12 total, 12 up, 12 in 2024-08-18T23:43:25.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:25 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3285234706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:43:25.851 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:26.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.339+0000 7f951635d640 1 -- 172.21.15.78:0/1671151125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95101078a0 msgr2=0x7f9510107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.339+0000 7f951635d640 1 --2- 172.21.15.78:0/1671151125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95101078a0 0x7f9510107c80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9500009930 tx=0x7f950002a420 comp rx=0 tx=0).stop 2024-08-18T23:43:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.339+0000 7f951635d640 1 -- 172.21.15.78:0/1671151125 shutdown_connections 2024-08-18T23:43:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.339+0000 7f951635d640 1 --2- 172.21.15.78:0/1671151125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9510102240 0x7f951010e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.339+0000 7f951635d640 1 --2- 172.21.15.78:0/1671151125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95101018a0 0x7f9510101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.339+0000 7f951635d640 1 --2- 172.21.15.78:0/1671151125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95101078a0 0x7f9510107c80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.339+0000 7f951635d640 1 -- 172.21.15.78:0/1671151125 >> 172.21.15.78:0/1671151125 conn(0x7f95100fd5e0 msgr2=0x7f95100ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:26.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.339+0000 7f951635d640 1 -- 172.21.15.78:0/1671151125 shutdown_connections 2024-08-18T23:43:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.339+0000 7f951635d640 1 -- 172.21.15.78:0/1671151125 wait complete. 2024-08-18T23:43:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.340+0000 7f951635d640 1 Processor -- start 2024-08-18T23:43:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.340+0000 7f951635d640 1 -- start start 2024-08-18T23:43:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.340+0000 7f951635d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95101018a0 0x7f95101981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.340+0000 7f951635d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9510102240 0x7f9510198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.340+0000 7f951635d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95101078a0 0x7f951019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.340+0000 7f951635d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9510111cc0 con 0x7f95101078a0 2024-08-18T23:43:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.340+0000 7f951635d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9510111b40 con 0x7f95101018a0 2024-08-18T23:43:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.340+0000 7f951635d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9510111e40 con 0x7f9510102240 2024-08-18T23:43:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.340+0000 7f95148fa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95101078a0 0x7f951019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:26.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.340+0000 7f95148fa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95101078a0 0x7f951019ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59244/0 (socket says 172.21.15.78:59244) 2024-08-18T23:43:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.340+0000 7f95148fa640 1 -- 172.21.15.78:0/4193992610 learned_addr learned my addr 172.21.15.78:0/4193992610 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.340+0000 7f9507fff640 1 --2- 172.21.15.78:0/4193992610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9510102240 0x7f9510198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.340+0000 7f950ffff640 1 --2- 172.21.15.78:0/4193992610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95101018a0 0x7f95101981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.340+0000 7f95148fa640 1 -- 172.21.15.78:0/4193992610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9510102240 msgr2=0x7f9510198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.340+0000 7f95148fa640 1 --2- 172.21.15.78:0/4193992610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9510102240 0x7f9510198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.340+0000 7f95148fa640 1 -- 172.21.15.78:0/4193992610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95101018a0 msgr2=0x7f95101981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.340+0000 7f95148fa640 1 --2- 172.21.15.78:0/4193992610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95101018a0 0x7f95101981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.340+0000 7f95148fa640 1 -- 172.21.15.78:0/4193992610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951019d170 con 0x7f95101078a0 2024-08-18T23:43:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.340+0000 7f950ffff640 1 --2- 172.21.15.78:0/4193992610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95101018a0 0x7f95101981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:43:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.341+0000 7f95148fa640 1 --2- 172.21.15.78:0/4193992610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95101078a0 0x7f951019ca90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f950800cce0 tx=0x7f950800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.341+0000 7f950dffb640 1 -- 172.21.15.78:0/4193992610 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950800f450 con 0x7f95101078a0 2024-08-18T23:43:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.341+0000 7f950dffb640 1 -- 172.21.15.78:0/4193992610 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f950800fad0 con 0x7f95101078a0 2024-08-18T23:43:26.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.341+0000 7f951635d640 1 -- 172.21.15.78:0/4193992610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f951019d460 con 0x7f95101078a0 2024-08-18T23:43:26.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.341+0000 7f950dffb640 1 -- 172.21.15.78:0/4193992610 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508018c10 con 0x7f95101078a0 2024-08-18T23:43:26.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.341+0000 7f951635d640 1 -- 172.21.15.78:0/4193992610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95101a4d40 con 0x7f95101078a0 2024-08-18T23:43:26.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.343+0000 7f950dffb640 1 -- 172.21.15.78:0/4193992610 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f950800f5f0 con 0x7f95101078a0 2024-08-18T23:43:26.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.343+0000 7f951875b740 1 Processor -- start 2024-08-18T23:43:26.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.343+0000 7f951875b740 1 -- start start 2024-08-18T23:43:26.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.343+0000 7f951875b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56550d5d2550 0x56550d5d2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:26.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.343+0000 7f951875b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5d2e70 0x56550d5e16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:26.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.343+0000 7f951875b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56550d5e1c20 0x56550d5e4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:26.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.343+0000 7f951875b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56550d537710 con 0x56550d5d2e70 2024-08-18T23:43:26.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.343+0000 7f951875b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56550d4cf770 con 0x56550d5d2550 2024-08-18T23:43:26.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.343+0000 7f951875b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56550d5336d0 con 0x56550d5e1c20 2024-08-18T23:43:26.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.344+0000 7f950ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5d2e70 0x56550d5e16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.344+0000 7f9507fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56550d5d2550 0x56550d5d2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.344+0000 7f950ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5d2e70 0x56550d5e16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59252/0 (socket says 172.21.15.78:59252) 2024-08-18T23:43:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.344+0000 7f950ffff640 1 -- 172.21.15.78:0/1044997923 learned_addr learned my addr 172.21.15.78:0/1044997923 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.344+0000 7f9507fff640 1 --2- 172.21.15.78:0/1044997923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56550d5e1c20 0x56550d5e4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.344+0000 7f950dffb640 1 --2- 172.21.15.78:0/4193992610 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94fc0a2c00 0x7f94fc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.344+0000 7f950ffff640 1 -- 172.21.15.78:0/1044997923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56550d5e1c20 msgr2=0x56550d5e4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.344+0000 7f950ffff640 1 --2- 172.21.15.78:0/1044997923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56550d5e1c20 0x56550d5e4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.344+0000 7f950ffff640 1 -- 172.21.15.78:0/1044997923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56550d5d2550 msgr2=0x56550d5d2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.344+0000 7f950ffff640 1 --2- 172.21.15.78:0/1044997923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56550d5d2550 0x56550d5d2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.344+0000 7f950ffff640 1 -- 172.21.15.78:0/1044997923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56550d4d63e0 con 0x56550d5d2e70 2024-08-18T23:43:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.344+0000 7f95148fa640 1 --2- 172.21.15.78:0/4193992610 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94fc0a2c00 0x7f94fc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.344+0000 7f950dffb640 1 -- 172.21.15.78:0/4193992610 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f95080cfac0 con 0x7f95101078a0 2024-08-18T23:43:26.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.344+0000 7f950ffff640 1 --2- 172.21.15.78:0/1044997923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5d2e70 0x56550d5e16e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9500005ab0 tx=0x7f9500009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b837c243c9ae16 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.344+0000 7f95067fc640 1 -- 172.21.15.78:0/1044997923 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950003f070 con 0x56550d5d2e70 2024-08-18T23:43:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.344+0000 7f95067fc640 1 -- 172.21.15.78:0/1044997923 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95000317b0 con 0x56550d5d2e70 2024-08-18T23:43:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.344+0000 7f95067fc640 1 -- 172.21.15.78:0/1044997923 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500035070 con 0x56550d5d2e70 2024-08-18T23:43:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.345+0000 7f951875b740 1 -- 172.21.15.78:0/1044997923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5d2e70 msgr2=0x56550d5e16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.345+0000 7f95148fa640 1 --2- 172.21.15.78:0/4193992610 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94fc0a2c00 0x7f94fc0a50c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f95080a46d0 tx=0x7f9508066b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.345+0000 7f951875b740 1 --2- 172.21.15.78:0/1044997923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5d2e70 0x56550d5e16e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9500005ab0 tx=0x7f9500009680 comp rx=0 tx=0).stop 2024-08-18T23:43:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.345+0000 7f951875b740 1 -- 172.21.15.78:0/1044997923 shutdown_connections 2024-08-18T23:43:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.345+0000 7f951875b740 1 --2- 172.21.15.78:0/1044997923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56550d5e1c20 0x56550d5e4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.345+0000 7f951875b740 1 --2- 172.21.15.78:0/1044997923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5d2e70 0x56550d5e16e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.345+0000 7f951875b740 1 --2- 172.21.15.78:0/1044997923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56550d5d2550 0x56550d5d2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.345+0000 7f951875b740 1 -- 172.21.15.78:0/1044997923 >> 172.21.15.78:0/1044997923 conn(0x56550d5cebc0 msgr2=0x56550d5cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.345+0000 7f951875b740 1 -- 172.21.15.78:0/1044997923 shutdown_connections 2024-08-18T23:43:26.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.345+0000 7f951875b740 1 -- 172.21.15.78:0/1044997923 wait complete. 2024-08-18T23:43:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.345+0000 7f951875b740 1 -- start start 2024-08-18T23:43:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f951875b740 1 Processor -- start 2024-08-18T23:43:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f951875b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56550d5d2550 0x56550d5deb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f951875b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56550d5d2e70 0x56550d5df090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f951875b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5e1c20 0x56550d5df5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f951875b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56550d5e4730 con 0x56550d5e1c20 2024-08-18T23:43:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f951875b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56550d4f2970 con 0x56550d5d2550 2024-08-18T23:43:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f951875b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56550d5e48b0 con 0x56550d5d2e70 2024-08-18T23:43:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f951875b740 10 client.0 _ll_register_callbacks cb 0x56550d2467b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:43:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f9507fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5e1c20 0x56550d5df5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f9507fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5e1c20 0x56550d5df5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59266/0 (socket says 172.21.15.78:59266) 2024-08-18T23:43:26.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f9507fff640 1 -- 172.21.15.78:0/2222086284 learned_addr learned my addr 172.21.15.78:0/2222086284 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f9507fff640 1 --2- 172.21.15.78:0/2222086284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56550d5d2550 0x56550d5deb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f950ffff640 1 --2- 172.21.15.78:0/2222086284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56550d5d2e70 0x56550d5df090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f9507fff640 1 -- 172.21.15.78:0/2222086284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56550d5d2e70 msgr2=0x56550d5df090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f9507fff640 1 --2- 172.21.15.78:0/2222086284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56550d5d2e70 0x56550d5df090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f9507fff640 1 -- 172.21.15.78:0/2222086284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56550d5d2550 msgr2=0x56550d5deb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f9507fff640 1 --2- 172.21.15.78:0/2222086284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56550d5d2550 0x56550d5deb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f9507fff640 1 -- 172.21.15.78:0/2222086284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56550d2a92b0 con 0x56550d5e1c20 2024-08-18T23:43:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f9507fff640 1 --2- 172.21.15.78:0/2222086284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5e1c20 0x56550d5df5d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f94f8019a40 tx=0x7f94f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a70a327481d264 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.346+0000 7f94ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:43:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.347+0000 7f94ef7fe640 1 -- 172.21.15.78:0/2222086284 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8014be0 con 0x56550d5e1c20 2024-08-18T23:43:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.347+0000 7f94ef7fe640 1 -- 172.21.15.78:0/2222086284 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f94f8014d80 con 0x56550d5e1c20 2024-08-18T23:43:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.347+0000 7f951875b740 1 -- 172.21.15.78:0/2222086284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95000317b0 con 0x56550d5e1c20 2024-08-18T23:43:26.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.347+0000 7f94ef7fe640 1 -- 172.21.15.78:0/2222086284 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f94f8029050 con 0x56550d5e1c20 2024-08-18T23:43:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.347+0000 7f951875b740 10 client.15084 fetch_fsmap learned FSMap version 24 2024-08-18T23:43:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.347+0000 7f951875b740 1 -- 172.21.15.78:0/2222086284 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56550d5f0410 con 0x56550d5e1c20 2024-08-18T23:43:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.347+0000 7f94ef7fe640 1 -- 172.21.15.78:0/2222086284 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95000317b0 con 0x56550d5e1c20 2024-08-18T23:43:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.347+0000 7f94ef7fe640 1 -- 172.21.15.78:0/2222086284 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f94f8015040 con 0x56550d5e1c20 2024-08-18T23:43:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.347+0000 7f951875b740 10 client.15084 fetch_fsmap finished waiting for FSMap version 24 2024-08-18T23:43:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.347+0000 7f951875b740 10 client.15084 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-18T23:43:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.347+0000 7f951875b740 20 client.15084 populate_metadata read hostname 'smithi078' 2024-08-18T23:43:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.347+0000 7f951875b740 1 --2- 172.21.15.78:0/2222086284 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x56550d5f2a40 0x56550d5f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.347+0000 7f951875b740 4 client.15084 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:43:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.347+0000 7f951875b740 1 -- 172.21.15.78:0/2222086284 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56550d544c80 con 0x56550d5f2a40 2024-08-18T23:43:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.347+0000 7f950ffff640 1 --2- 172.21.15.78:0/2222086284 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x56550d5f2a40 0x56550d5f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.348+0000 7f950ffff640 1 --2- 172.21.15.78:0/2222086284 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x56550d5f2a40 0x56550d5f3eb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9636d15978fd3ff server_cookie=18067c92439f5425 in_seq=0 out_seq=0 2024-08-18T23:43:26.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.348+0000 7f94ef7fe640 10 client.15084 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:43:26.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.349+0000 7f94ef7fe640 1 -- 172.21.15.78:0/2222086284 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56550d544c80 con 0x56550d5f2a40 2024-08-18T23:43:26.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.349+0000 7f94ef7fe640 10 client.15084 handle_command_reply: tid=0 2024-08-18T23:43:26.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.349+0000 7f951875b740 1 client.15084 shutdown 2024-08-18T23:43:26.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.349+0000 7f951875b740 1 -- 172.21.15.78:0/2222086284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5e1c20 msgr2=0x56550d5df5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:26.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.349+0000 7f951875b740 1 --2- 172.21.15.78:0/2222086284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5e1c20 0x56550d5df5d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f94f8019a40 tx=0x7f94f80046c0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.349+0000 7f951875b740 1 -- 172.21.15.78:0/2222086284 shutdown_connections 2024-08-18T23:43:26.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.349+0000 7f951875b740 1 --2- 172.21.15.78:0/2222086284 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x56550d5f2a40 0x56550d5f3eb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.349+0000 7f951875b740 1 --2- 172.21.15.78:0/2222086284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5e1c20 0x56550d5df5d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.349+0000 7f951875b740 1 --2- 172.21.15.78:0/2222086284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56550d5d2e70 0x56550d5df090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.349+0000 7f951875b740 1 --2- 172.21.15.78:0/2222086284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56550d5d2550 0x56550d5deb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.349+0000 7f94ef7fe640 0 client.15084 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:43:26.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.349+0000 7f951875b740 1 -- 172.21.15.78:0/2222086284 >> 172.21.15.78:0/2222086284 conn(0x56550d5cebc0 msgr2=0x56550d5cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:26.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.349+0000 7f951875b740 1 -- 172.21.15.78:0/2222086284 shutdown_connections 2024-08-18T23:43:26.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.349+0000 7f951875b740 1 -- 172.21.15.78:0/2222086284 wait complete. 2024-08-18T23:43:26.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.349+0000 7f951875b740 20 client.15084 trim_cache size 0 max 16384 2024-08-18T23:43:26.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.367+0000 7f951875b740 1 Processor -- start 2024-08-18T23:43:26.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.367+0000 7f951875b740 1 -- start start 2024-08-18T23:43:26.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.367+0000 7f951875b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5d2550 0x56550d5e12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:26.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.367+0000 7f951875b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56550d5d2e70 0x56550d5f3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:26.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.367+0000 7f951875b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56550d5e1c20 0x56550d5dbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:26.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.367+0000 7f951875b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56550d5f21f0 con 0x56550d5d2550 2024-08-18T23:43:26.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.367+0000 7f951875b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56550d5f28c0 con 0x56550d5d2e70 2024-08-18T23:43:26.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.367+0000 7f951875b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56550d5f2f90 con 0x56550d5e1c20 2024-08-18T23:43:26.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.367+0000 7f9507fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5d2550 0x56550d5e12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:26.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.367+0000 7f9507fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5d2550 0x56550d5e12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59274/0 (socket says 172.21.15.78:59274) 2024-08-18T23:43:26.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.367+0000 7f9507fff640 1 -- 172.21.15.78:0/596456833 learned_addr learned my addr 172.21.15.78:0/596456833 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:26.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.367+0000 7f9507fff640 1 -- 172.21.15.78:0/596456833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56550d5e1c20 msgr2=0x56550d5dbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:26.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.367+0000 7f9507fff640 1 --2- 172.21.15.78:0/596456833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56550d5e1c20 0x56550d5dbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.367+0000 7f9507fff640 1 -- 172.21.15.78:0/596456833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56550d5d2e70 msgr2=0x56550d5f3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:26.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.367+0000 7f9507fff640 1 --2- 172.21.15.78:0/596456833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56550d5d2e70 0x56550d5f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.367+0000 7f9507fff640 1 -- 172.21.15.78:0/596456833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56550d5e48a0 con 0x56550d5d2550 2024-08-18T23:43:26.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.368+0000 7f9507fff640 1 --2- 172.21.15.78:0/596456833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5d2550 0x56550d5e12b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f94f8019a10 tx=0x7f94f800aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce144d53f573ba28 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:26.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.368+0000 7f9505ffb640 1 -- 172.21.15.78:0/596456833 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f800a390 con 0x56550d5d2550 2024-08-18T23:43:26.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.368+0000 7f9505ffb640 1 -- 172.21.15.78:0/596456833 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94f8019500 con 0x56550d5d2550 2024-08-18T23:43:26.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.368+0000 7f9505ffb640 1 -- 172.21.15.78:0/596456833 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8022410 con 0x56550d5d2550 2024-08-18T23:43:26.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.368+0000 7f951875b740 1 -- 172.21.15.78:0/596456833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5d2550 msgr2=0x56550d5e12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:26.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.368+0000 7f951875b740 1 --2- 172.21.15.78:0/596456833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5d2550 0x56550d5e12b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f94f8019a10 tx=0x7f94f800aa20 comp rx=0 tx=0).stop 2024-08-18T23:43:26.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.368+0000 7f951875b740 1 -- 172.21.15.78:0/596456833 shutdown_connections 2024-08-18T23:43:26.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.368+0000 7f951875b740 1 --2- 172.21.15.78:0/596456833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56550d5e1c20 0x56550d5dbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.368+0000 7f951875b740 1 --2- 172.21.15.78:0/596456833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56550d5d2e70 0x56550d5f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.368+0000 7f951875b740 1 --2- 172.21.15.78:0/596456833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5d2550 0x56550d5e12b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.368+0000 7f951875b740 1 -- 172.21.15.78:0/596456833 >> 172.21.15.78:0/596456833 conn(0x56550d5cebc0 msgr2=0x56550d5dd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:26.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.368+0000 7f951875b740 1 -- 172.21.15.78:0/596456833 shutdown_connections 2024-08-18T23:43:26.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.368+0000 7f951875b740 1 -- 172.21.15.78:0/596456833 wait complete. 2024-08-18T23:43:26.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.369+0000 7f951875b740 1 -- start start 2024-08-18T23:43:26.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.369+0000 7f951875b740 1 Processor -- start 2024-08-18T23:43:26.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.369+0000 7f951875b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56550d5d2550 0x56550d5f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:26.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.369+0000 7f951875b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56550d5d2e70 0x56550d5f6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:26.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.369+0000 7f951875b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5e1c20 0x56550d5f6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:26.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.369+0000 7f951875b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56550d5f3d30 con 0x56550d5e1c20 2024-08-18T23:43:26.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.369+0000 7f951875b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56550d5f3660 con 0x56550d5d2550 2024-08-18T23:43:26.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.369+0000 7f951875b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56550d5f4400 con 0x56550d5d2e70 2024-08-18T23:43:26.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.369+0000 7f951875b740 10 client.0 _ll_register_callbacks cb 0x7f94f800aa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:43:26.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.369+0000 7f9507fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5e1c20 0x56550d5f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.369+0000 7f9507fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5e1c20 0x56550d5f6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59276/0 (socket says 172.21.15.78:59276) 2024-08-18T23:43:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.369+0000 7f9507fff640 1 -- 172.21.15.78:0/2496955660 learned_addr learned my addr 172.21.15.78:0/2496955660 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.369+0000 7f9507fff640 1 --2- 172.21.15.78:0/2496955660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56550d5d2550 0x56550d5f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.370+0000 7f950ffff640 1 --2- 172.21.15.78:0/2496955660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56550d5d2e70 0x56550d5f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.370+0000 7f9507fff640 1 -- 172.21.15.78:0/2496955660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56550d5d2e70 msgr2=0x56550d5f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.370+0000 7f9507fff640 1 --2- 172.21.15.78:0/2496955660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56550d5d2e70 0x56550d5f6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.370+0000 7f9507fff640 1 -- 172.21.15.78:0/2496955660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56550d5d2550 msgr2=0x56550d5f5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.370+0000 7f9507fff640 1 --2- 172.21.15.78:0/2496955660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56550d5d2550 0x56550d5f5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.370+0000 7f9507fff640 1 -- 172.21.15.78:0/2496955660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56550d5fa6e0 con 0x56550d5e1c20 2024-08-18T23:43:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.370+0000 7f9507fff640 1 --2- 172.21.15.78:0/2496955660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5e1c20 0x56550d5f6970 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f94f8004700 tx=0x7f94f8021660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fbbfe3d7f831ab server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.370+0000 7f94ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:43:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.370+0000 7f94ef7fe640 1 -- 172.21.15.78:0/2496955660 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8014c70 con 0x56550d5e1c20 2024-08-18T23:43:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.370+0000 7f94ef7fe640 1 -- 172.21.15.78:0/2496955660 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f94f8021a40 con 0x56550d5e1c20 2024-08-18T23:43:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.370+0000 7f951875b740 1 -- 172.21.15.78:0/2496955660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94f8019500 con 0x56550d5e1c20 2024-08-18T23:43:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.370+0000 7f94ef7fe640 1 -- 172.21.15.78:0/2496955660 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f94f8021c40 con 0x56550d5e1c20 2024-08-18T23:43:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.370+0000 7f951875b740 10 client.15090 fetch_fsmap learned FSMap version 24 2024-08-18T23:43:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.370+0000 7f951875b740 1 -- 172.21.15.78:0/2496955660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56550d65d440 con 0x56550d5e1c20 2024-08-18T23:43:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.370+0000 7f94ef7fe640 1 -- 172.21.15.78:0/2496955660 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94f8019500 con 0x56550d5e1c20 2024-08-18T23:43:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.371+0000 7f94ef7fe640 1 -- 172.21.15.78:0/2496955660 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f94f8022bf0 con 0x56550d5e1c20 2024-08-18T23:43:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.371+0000 7f951875b740 10 client.15090 fetch_fsmap finished waiting for FSMap version 24 2024-08-18T23:43:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.371+0000 7f951875b740 10 client.15090 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-18T23:43:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.371+0000 7f951875b740 20 client.15090 populate_metadata read hostname 'smithi078' 2024-08-18T23:43:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.371+0000 7f951875b740 1 --2- 172.21.15.78:0/2496955660 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x56550d661d10 0x56550d660a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.371+0000 7f951875b740 4 client.15090 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-18T23:43:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.371+0000 7f951875b740 1 -- 172.21.15.78:0/2496955660 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "status"}) -- 0x56550d53f190 con 0x56550d661d10 2024-08-18T23:43:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.371+0000 7f950ffff640 1 --2- 172.21.15.78:0/2496955660 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x56550d661d10 0x56550d660a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.371+0000 7f950ffff640 1 --2- 172.21.15.78:0/2496955660 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x56550d661d10 0x56550d660a20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4644a4397015bf3 server_cookie=f86eb3b975fbb116 in_seq=0 out_seq=0 2024-08-18T23:43:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.371+0000 7f94ef7fe640 10 client.15090 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:43:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.372+0000 7f94ef7fe640 1 -- 172.21.15.78:0/2496955660 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x56550d53f190 con 0x56550d661d10 2024-08-18T23:43:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.372+0000 7f94ef7fe640 10 client.15090 handle_command_reply: tid=0 2024-08-18T23:43:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.372+0000 7f951875b740 1 client.15090 shutdown 2024-08-18T23:43:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.372+0000 7f951875b740 1 -- 172.21.15.78:0/2496955660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5e1c20 msgr2=0x56550d5f6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.372+0000 7f951875b740 1 --2- 172.21.15.78:0/2496955660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5e1c20 0x56550d5f6970 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f94f8004700 tx=0x7f94f8021660 comp rx=0 tx=0).stop 2024-08-18T23:43:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.372+0000 7f951875b740 1 -- 172.21.15.78:0/2496955660 shutdown_connections 2024-08-18T23:43:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.372+0000 7f951875b740 1 --2- 172.21.15.78:0/2496955660 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x56550d661d10 0x56550d660a20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.372+0000 7f951875b740 1 --2- 172.21.15.78:0/2496955660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56550d5e1c20 0x56550d5f6970 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.372+0000 7f951875b740 1 --2- 172.21.15.78:0/2496955660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56550d5d2e70 0x56550d5f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.372+0000 7f94ef7fe640 0 client.15090 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:43:26.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.372+0000 7f951875b740 1 --2- 172.21.15.78:0/2496955660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56550d5d2550 0x56550d5f5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.372+0000 7f951875b740 1 -- 172.21.15.78:0/2496955660 >> 172.21.15.78:0/2496955660 conn(0x56550d5cebc0 msgr2=0x56550d5ecb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:26.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.372+0000 7f951875b740 1 -- 172.21.15.78:0/2496955660 shutdown_connections 2024-08-18T23:43:26.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.372+0000 7f951875b740 1 -- 172.21.15.78:0/2496955660 wait complete. 2024-08-18T23:43:26.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.372+0000 7f951875b740 20 client.15090 trim_cache size 0 max 16384 2024-08-18T23:43:26.390 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:43:26.390 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "4f1128e6-5dba-11ef-bccf-c7b262605968", 2024-08-18T23:43:26.390 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 2, 2024-08-18T23:43:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24427, 2024-08-18T23:43:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-08-18T23:43:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-08-18T23:43:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-08-18T23:43:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 79.117966482, 2024-08-18T23:43:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 24, 2024-08-18T23:43:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 92, 2024-08-18T23:43:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 88, 2024-08-18T23:43:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 133.10781926199999 2024-08-18T23:43:26.391 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:43:26.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.372+0000 7f951635d640 1 -- 172.21.15.78:0/4193992610 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94fc0a2c00 msgr2=0x7f94fc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:26.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.372+0000 7f951635d640 1 --2- 172.21.15.78:0/4193992610 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94fc0a2c00 0x7f94fc0a50c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f95080a46d0 tx=0x7f9508066b60 comp rx=0 tx=0).stop 2024-08-18T23:43:26.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.372+0000 7f951635d640 1 -- 172.21.15.78:0/4193992610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95101078a0 msgr2=0x7f951019ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:26.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.372+0000 7f951635d640 1 --2- 172.21.15.78:0/4193992610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95101078a0 0x7f951019ca90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f950800cce0 tx=0x7f950800a620 comp rx=0 tx=0).stop 2024-08-18T23:43:26.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.373+0000 7f951635d640 1 -- 172.21.15.78:0/4193992610 shutdown_connections 2024-08-18T23:43:26.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.373+0000 7f951635d640 1 --2- 172.21.15.78:0/4193992610 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94fc0a2c00 0x7f94fc0a50c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.373+0000 7f951635d640 1 --2- 172.21.15.78:0/4193992610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95101078a0 0x7f951019ca90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.373+0000 7f951635d640 1 --2- 172.21.15.78:0/4193992610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9510102240 0x7f9510198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.373+0000 7f951635d640 1 --2- 172.21.15.78:0/4193992610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95101018a0 0x7f95101981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:26.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.373+0000 7f951635d640 1 -- 172.21.15.78:0/4193992610 >> 172.21.15.78:0/4193992610 conn(0x7f95100fd5e0 msgr2=0x7f95100fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:26.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.373+0000 7f951635d640 1 -- 172.21.15.78:0/4193992610 shutdown_connections 2024-08-18T23:43:26.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:26.373+0000 7f951635d640 1 -- 172.21.15.78:0/4193992610 wait complete. 2024-08-18T23:43:26.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:26 smithi161 ceph-mon[22027]: pgmap v324: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-08-18T23:43:26.696 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:26 smithi078 ceph-mon[18440]: pgmap v324: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-08-18T23:43:26.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:26 smithi123 ceph-mon[21808]: pgmap v324: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-08-18T23:43:27.155 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.h status 2024-08-18T23:43:27.406 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:27.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.903+0000 7f99e11ac640 1 -- 172.21.15.78:0/2728361790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99dc100b50 msgr2=0x7f99dc100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:27.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.903+0000 7f99e11ac640 1 --2- 172.21.15.78:0/2728361790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99dc100b50 0x7f99dc100f30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f99d0009900 tx=0x7f99d002a480 comp rx=0 tx=0).stop 2024-08-18T23:43:27.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.903+0000 7f99e11ac640 1 -- 172.21.15.78:0/2728361790 shutdown_connections 2024-08-18T23:43:27.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.903+0000 7f99e11ac640 1 --2- 172.21.15.78:0/2728361790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99dc102c20 0x7f99dc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.903+0000 7f99e11ac640 1 --2- 172.21.15.78:0/2728361790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99dc1022a0 0x7f99dc1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.903+0000 7f99e11ac640 1 --2- 172.21.15.78:0/2728361790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99dc100b50 0x7f99dc100f30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.903+0000 7f99e11ac640 1 -- 172.21.15.78:0/2728361790 >> 172.21.15.78:0/2728361790 conn(0x7f99dc0fc500 msgr2=0x7f99dc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:27.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.904+0000 7f99e11ac640 1 -- 172.21.15.78:0/2728361790 shutdown_connections 2024-08-18T23:43:27.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.904+0000 7f99e11ac640 1 -- 172.21.15.78:0/2728361790 wait complete. 2024-08-18T23:43:27.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.904+0000 7f99e11ac640 1 Processor -- start 2024-08-18T23:43:27.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.904+0000 7f99e11ac640 1 -- start start 2024-08-18T23:43:27.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99e11ac640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99dc100b50 0x7f99dc197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:27.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99e11ac640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99dc1022a0 0x7f99dc197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:27.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99e11ac640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99dc102c20 0x7f99dc19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:27.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99e11ac640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc110cd0 con 0x7f99dc102c20 2024-08-18T23:43:27.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99e11ac640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc110b50 con 0x7f99dc100b50 2024-08-18T23:43:27.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99e11ac640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc110e50 con 0x7f99dc1022a0 2024-08-18T23:43:27.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99db59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99dc102c20 0x7f99dc19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:27.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99db59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99dc102c20 0x7f99dc19ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59298/0 (socket says 172.21.15.78:59298) 2024-08-18T23:43:27.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99db59e640 1 -- 172.21.15.78:0/661172494 learned_addr learned my addr 172.21.15.78:0/661172494 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:27.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99dad9d640 1 --2- 172.21.15.78:0/661172494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99dc100b50 0x7f99dc197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:27.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99da59c640 1 --2- 172.21.15.78:0/661172494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99dc1022a0 0x7f99dc197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:27.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99db59e640 1 -- 172.21.15.78:0/661172494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99dc1022a0 msgr2=0x7f99dc197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:27.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99db59e640 1 --2- 172.21.15.78:0/661172494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99dc1022a0 0x7f99dc197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99db59e640 1 -- 172.21.15.78:0/661172494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99dc100b50 msgr2=0x7f99dc197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:27.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99db59e640 1 --2- 172.21.15.78:0/661172494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99dc100b50 0x7f99dc197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99db59e640 1 -- 172.21.15.78:0/661172494 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc19c100 con 0x7f99dc102c20 2024-08-18T23:43:27.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99dad9d640 1 --2- 172.21.15.78:0/661172494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99dc100b50 0x7f99dc197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:43:27.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99db59e640 1 --2- 172.21.15.78:0/661172494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99dc102c20 0x7f99dc19ba20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f99d400db90 tx=0x7f99d400dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99c3fff640 1 -- 172.21.15.78:0/661172494 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d400e980 con 0x7f99dc102c20 2024-08-18T23:43:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99e11ac640 1 -- 172.21.15.78:0/661172494 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99dc19c390 con 0x7f99dc102c20 2024-08-18T23:43:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99c3fff640 1 -- 172.21.15.78:0/661172494 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99d4009580 con 0x7f99dc102c20 2024-08-18T23:43:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99c3fff640 1 -- 172.21.15.78:0/661172494 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4016650 con 0x7f99dc102c20 2024-08-18T23:43:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.905+0000 7f99e11ac640 1 -- 172.21.15.78:0/661172494 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99dc069ca0 con 0x7f99dc102c20 2024-08-18T23:43:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.908+0000 7f99c3fff640 1 -- 172.21.15.78:0/661172494 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f99d4009730 con 0x7f99dc102c20 2024-08-18T23:43:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.908+0000 7f99e35aa740 1 Processor -- start 2024-08-18T23:43:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.908+0000 7f99e35aa740 1 -- start start 2024-08-18T23:43:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.908+0000 7f99e35aa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27a630 0x55ddcd27aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.908+0000 7f99e35aa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd27af50 0x55ddcd2852d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99e35aa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd285810 0x55ddcd287c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99e35aa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ddcd1df710 con 0x55ddcd27a630 2024-08-18T23:43:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99e35aa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ddcd177770 con 0x55ddcd285810 2024-08-18T23:43:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99e35aa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ddcd1db6d0 con 0x55ddcd27af50 2024-08-18T23:43:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99c3fff640 1 --2- 172.21.15.78:0/661172494 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f99c40a2b80 0x7f99c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99da59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27a630 0x55ddcd27aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99da59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27a630 0x55ddcd27aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59302/0 (socket says 172.21.15.78:59302) 2024-08-18T23:43:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99da59c640 1 -- 172.21.15.78:0/2020329687 learned_addr learned my addr 172.21.15.78:0/2020329687 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99dad9d640 1 --2- 172.21.15.78:0/2020329687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd27af50 0x55ddcd2852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99db59e640 1 --2- 172.21.15.78:0/661172494 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f99c40a2b80 0x7f99c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99da59c640 1 --2- 172.21.15.78:0/2020329687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd285810 0x55ddcd287c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99c3fff640 1 -- 172.21.15.78:0/661172494 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f99d40d0bd0 con 0x7f99dc102c20 2024-08-18T23:43:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99da59c640 1 -- 172.21.15.78:0/2020329687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd27af50 msgr2=0x55ddcd2852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99da59c640 1 --2- 172.21.15.78:0/2020329687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd27af50 0x55ddcd2852d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99da59c640 1 -- 172.21.15.78:0/2020329687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd285810 msgr2=0x55ddcd287c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99da59c640 1 --2- 172.21.15.78:0/2020329687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd285810 0x55ddcd287c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99da59c640 1 -- 172.21.15.78:0/2020329687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddcd17e3e0 con 0x55ddcd27a630 2024-08-18T23:43:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99da59c640 1 --2- 172.21.15.78:0/2020329687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27a630 0x55ddcd27aa10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f99cc005fd0 tx=0x7f99cc00b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c8d450fcb5358c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99db59e640 1 --2- 172.21.15.78:0/661172494 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f99c40a2b80 0x7f99c40a5040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f99d40a5780 tx=0x7f99d4067c10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.909+0000 7f99c17fa640 1 -- 172.21.15.78:0/2020329687 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc00e670 con 0x55ddcd27a630 2024-08-18T23:43:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.910+0000 7f99c17fa640 1 -- 172.21.15.78:0/2020329687 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99cc01a070 con 0x55ddcd27a630 2024-08-18T23:43:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.910+0000 7f99c17fa640 1 -- 172.21.15.78:0/2020329687 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc012a00 con 0x55ddcd27a630 2024-08-18T23:43:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.910+0000 7f99e35aa740 1 -- 172.21.15.78:0/2020329687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27a630 msgr2=0x55ddcd27aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.910+0000 7f99e35aa740 1 --2- 172.21.15.78:0/2020329687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27a630 0x55ddcd27aa10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f99cc005fd0 tx=0x7f99cc00b470 comp rx=0 tx=0).stop 2024-08-18T23:43:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.910+0000 7f99e35aa740 1 -- 172.21.15.78:0/2020329687 shutdown_connections 2024-08-18T23:43:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.910+0000 7f99e35aa740 1 --2- 172.21.15.78:0/2020329687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd285810 0x55ddcd287c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.910+0000 7f99e35aa740 1 --2- 172.21.15.78:0/2020329687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd27af50 0x55ddcd2852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.910+0000 7f99e35aa740 1 --2- 172.21.15.78:0/2020329687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27a630 0x55ddcd27aa10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.910+0000 7f99e35aa740 1 -- 172.21.15.78:0/2020329687 >> 172.21.15.78:0/2020329687 conn(0x55ddcd276bc0 msgr2=0x55ddcd276fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.910+0000 7f99e35aa740 1 -- 172.21.15.78:0/2020329687 shutdown_connections 2024-08-18T23:43:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.910+0000 7f99e35aa740 1 -- 172.21.15.78:0/2020329687 wait complete. 2024-08-18T23:43:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99e35aa740 1 -- start start 2024-08-18T23:43:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99e35aa740 1 Processor -- start 2024-08-18T23:43:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99e35aa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd27a630 0x55ddcd2826d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99e35aa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27af50 0x55ddcd282c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99e35aa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd285810 0x55ddcd283150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99e35aa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ddcd2883e0 con 0x55ddcd27af50 2024-08-18T23:43:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99e35aa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ddcd19a970 con 0x55ddcd27a630 2024-08-18T23:43:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99e35aa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ddcd288560 con 0x55ddcd285810 2024-08-18T23:43:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99e35aa740 10 client.0 _ll_register_callbacks cb 0x55ddcceee7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:43:27.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99dad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27af50 0x55ddcd282c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99dad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27af50 0x55ddcd282c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59304/0 (socket says 172.21.15.78:59304) 2024-08-18T23:43:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99dad9d640 1 -- 172.21.15.78:0/2751359739 learned_addr learned my addr 172.21.15.78:0/2751359739 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99da59c640 1 --2- 172.21.15.78:0/2751359739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd27a630 0x55ddcd2826d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99dad9d640 1 -- 172.21.15.78:0/2751359739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd285810 msgr2=0x55ddcd283150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99da59c640 1 --2- 172.21.15.78:0/2751359739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd285810 0x55ddcd283150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99dad9d640 1 --2- 172.21.15.78:0/2751359739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd285810 0x55ddcd283150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99dad9d640 1 -- 172.21.15.78:0/2751359739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd27a630 msgr2=0x55ddcd2826d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99dad9d640 1 --2- 172.21.15.78:0/2751359739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd27a630 0x55ddcd2826d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99dad9d640 1 -- 172.21.15.78:0/2751359739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddccf512b0 con 0x55ddcd27af50 2024-08-18T23:43:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99dad9d640 1 --2- 172.21.15.78:0/2751359739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27af50 0x55ddcd282c10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f99d002a9a0 tx=0x7f99d0004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973890dda881365 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.911+0000 7f99beff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:43:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.912+0000 7f99beff5640 1 -- 172.21.15.78:0/2751359739 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d0035040 con 0x55ddcd27af50 2024-08-18T23:43:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.912+0000 7f99e35aa740 1 -- 172.21.15.78:0/2751359739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99cc01a070 con 0x55ddcd27af50 2024-08-18T23:43:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.912+0000 7f99beff5640 1 -- 172.21.15.78:0/2751359739 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f99d0003b00 con 0x55ddcd27af50 2024-08-18T23:43:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.912+0000 7f99beff5640 1 -- 172.21.15.78:0/2751359739 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f99d0003d00 con 0x55ddcd27af50 2024-08-18T23:43:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.912+0000 7f99e35aa740 10 client.15102 fetch_fsmap learned FSMap version 24 2024-08-18T23:43:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.912+0000 7f99e35aa740 1 -- 172.21.15.78:0/2751359739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ddcd294010 con 0x55ddcd27af50 2024-08-18T23:43:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.912+0000 7f99beff5640 1 -- 172.21.15.78:0/2751359739 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99cc01a070 con 0x55ddcd27af50 2024-08-18T23:43:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.912+0000 7f99beff5640 1 -- 172.21.15.78:0/2751359739 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f99d003d720 con 0x55ddcd27af50 2024-08-18T23:43:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.912+0000 7f99e35aa740 10 client.15102 fetch_fsmap finished waiting for FSMap version 24 2024-08-18T23:43:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.912+0000 7f99e35aa740 10 client.15102 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-18T23:43:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.912+0000 7f99e35aa740 20 client.15102 populate_metadata read hostname 'smithi078' 2024-08-18T23:43:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.912+0000 7f99e35aa740 1 --2- 172.21.15.78:0/2751359739 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55ddcd296a70 0x55ddcd297ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.912+0000 7f99e35aa740 4 client.15102 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:43:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.912+0000 7f99e35aa740 1 -- 172.21.15.78:0/2751359739 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddcd1ecc80 con 0x55ddcd296a70 2024-08-18T23:43:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.913+0000 7f99da59c640 1 --2- 172.21.15.78:0/2751359739 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55ddcd296a70 0x55ddcd297ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.913+0000 7f99beff5640 10 client.15102 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:43:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.913+0000 7f99da59c640 1 --2- 172.21.15.78:0/2751359739 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55ddcd296a70 0x55ddcd297ee0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0acd1f0a1661725 server_cookie=702c48e4161b7936 in_seq=0 out_seq=0 2024-08-18T23:43:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.914+0000 7f99beff5640 1 -- 172.21.15.78:0/2751359739 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ddcd1ecc80 con 0x55ddcd296a70 2024-08-18T23:43:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.914+0000 7f99beff5640 10 client.15102 handle_command_reply: tid=0 2024-08-18T23:43:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.914+0000 7f99e35aa740 1 client.15102 shutdown 2024-08-18T23:43:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.914+0000 7f99e35aa740 1 -- 172.21.15.78:0/2751359739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27af50 msgr2=0x55ddcd282c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.914+0000 7f99e35aa740 1 --2- 172.21.15.78:0/2751359739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27af50 0x55ddcd282c10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f99d002a9a0 tx=0x7f99d0004580 comp rx=0 tx=0).stop 2024-08-18T23:43:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.914+0000 7f99e35aa740 1 -- 172.21.15.78:0/2751359739 shutdown_connections 2024-08-18T23:43:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.914+0000 7f99e35aa740 1 --2- 172.21.15.78:0/2751359739 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55ddcd296a70 0x55ddcd297ee0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.914+0000 7f99e35aa740 1 --2- 172.21.15.78:0/2751359739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd285810 0x55ddcd283150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.914+0000 7f99e35aa740 1 --2- 172.21.15.78:0/2751359739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27af50 0x55ddcd282c10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.914+0000 7f99e35aa740 1 --2- 172.21.15.78:0/2751359739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd27a630 0x55ddcd2826d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.914+0000 7f99e35aa740 1 -- 172.21.15.78:0/2751359739 >> 172.21.15.78:0/2751359739 conn(0x55ddcd276bc0 msgr2=0x55ddcd27c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.914+0000 7f99beff5640 0 client.15102 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:43:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.914+0000 7f99e35aa740 1 -- 172.21.15.78:0/2751359739 shutdown_connections 2024-08-18T23:43:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.914+0000 7f99e35aa740 1 -- 172.21.15.78:0/2751359739 wait complete. 2024-08-18T23:43:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.914+0000 7f99e35aa740 20 client.15102 trim_cache size 0 max 16384 2024-08-18T23:43:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.932+0000 7f99e35aa740 1 Processor -- start 2024-08-18T23:43:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.932+0000 7f99e35aa740 1 -- start start 2024-08-18T23:43:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.932+0000 7f99e35aa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd27a630 0x55ddcd297810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.932+0000 7f99e35aa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd27af50 0x55ddcd295ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.932+0000 7f99e35aa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd285810 0x55ddcd27d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.932+0000 7f99e35aa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ddcd296220 con 0x55ddcd285810 2024-08-18T23:43:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.932+0000 7f99e35aa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ddcd2968f0 con 0x55ddcd27af50 2024-08-18T23:43:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.932+0000 7f99e35aa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ddcd296fc0 con 0x55ddcd27a630 2024-08-18T23:43:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.932+0000 7f99da59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd285810 0x55ddcd27d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.932+0000 7f99da59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd285810 0x55ddcd27d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59322/0 (socket says 172.21.15.78:59322) 2024-08-18T23:43:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.932+0000 7f99da59c640 1 -- 172.21.15.78:0/1181817995 learned_addr learned my addr 172.21.15.78:0/1181817995 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.932+0000 7f99da59c640 1 --2- 172.21.15.78:0/1181817995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd27a630 0x55ddcd297810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.932+0000 7f99dad9d640 1 --2- 172.21.15.78:0/1181817995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd27af50 0x55ddcd295ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.933+0000 7f99da59c640 1 -- 172.21.15.78:0/1181817995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd27a630 msgr2=0x55ddcd297810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.933+0000 7f99da59c640 1 --2- 172.21.15.78:0/1181817995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd27a630 0x55ddcd297810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.933+0000 7f99da59c640 1 -- 172.21.15.78:0/1181817995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd27af50 msgr2=0x55ddcd295ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.933+0000 7f99da59c640 1 --2- 172.21.15.78:0/1181817995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd27af50 0x55ddcd295ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.933+0000 7f99da59c640 1 -- 172.21.15.78:0/1181817995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddcd272130 con 0x55ddcd285810 2024-08-18T23:43:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.933+0000 7f99da59c640 1 --2- 172.21.15.78:0/1181817995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd285810 0x55ddcd27d090 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f99cc012670 tx=0x7f99cc00bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b42545eb25a129c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.933+0000 7f99c0ff9640 1 -- 172.21.15.78:0/1181817995 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc02e070 con 0x55ddcd285810 2024-08-18T23:43:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.933+0000 7f99c0ff9640 1 -- 172.21.15.78:0/1181817995 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99cc01a070 con 0x55ddcd285810 2024-08-18T23:43:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.933+0000 7f99c0ff9640 1 -- 172.21.15.78:0/1181817995 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc012e60 con 0x55ddcd285810 2024-08-18T23:43:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.934+0000 7f99e35aa740 1 -- 172.21.15.78:0/1181817995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd285810 msgr2=0x55ddcd27d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.934+0000 7f99e35aa740 1 --2- 172.21.15.78:0/1181817995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd285810 0x55ddcd27d090 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f99cc012670 tx=0x7f99cc00bde0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.934+0000 7f99e35aa740 1 -- 172.21.15.78:0/1181817995 shutdown_connections 2024-08-18T23:43:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.934+0000 7f99e35aa740 1 --2- 172.21.15.78:0/1181817995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd285810 0x55ddcd27d090 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.934+0000 7f99e35aa740 1 --2- 172.21.15.78:0/1181817995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd27af50 0x55ddcd295ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.934+0000 7f99e35aa740 1 --2- 172.21.15.78:0/1181817995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd27a630 0x55ddcd297810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.934+0000 7f99e35aa740 1 -- 172.21.15.78:0/1181817995 >> 172.21.15.78:0/1181817995 conn(0x55ddcd276bc0 msgr2=0x55ddcd280fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.934+0000 7f99e35aa740 1 -- 172.21.15.78:0/1181817995 shutdown_connections 2024-08-18T23:43:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.934+0000 7f99e35aa740 1 -- 172.21.15.78:0/1181817995 wait complete. 2024-08-18T23:43:27.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99e35aa740 1 -- start start 2024-08-18T23:43:27.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99e35aa740 1 Processor -- start 2024-08-18T23:43:27.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99e35aa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27a630 0x55ddcd27ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:27.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99e35aa740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd27af50 0x55ddcd27f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:27.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99e35aa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd285810 0x55ddcd29f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:27.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99e35aa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ddcd297d60 con 0x55ddcd27a630 2024-08-18T23:43:27.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99e35aa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ddcd297690 con 0x55ddcd285810 2024-08-18T23:43:27.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99e35aa740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ddcd298430 con 0x55ddcd27af50 2024-08-18T23:43:27.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99e35aa740 10 client.0 _ll_register_callbacks cb 0x7f99d0004210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:43:27.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99da59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27a630 0x55ddcd27ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:27.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99da59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27a630 0x55ddcd27ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59336/0 (socket says 172.21.15.78:59336) 2024-08-18T23:43:27.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99da59c640 1 -- 172.21.15.78:0/2811890736 learned_addr learned my addr 172.21.15.78:0/2811890736 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:27.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99da59c640 1 --2- 172.21.15.78:0/2811890736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd285810 0x55ddcd29f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99da59c640 1 -- 172.21.15.78:0/2811890736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd27af50 msgr2=0x55ddcd27f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99dad9d640 1 --2- 172.21.15.78:0/2811890736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd27af50 0x55ddcd27f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99da59c640 1 --2- 172.21.15.78:0/2811890736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd27af50 0x55ddcd27f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99da59c640 1 -- 172.21.15.78:0/2811890736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd285810 msgr2=0x55ddcd29f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99da59c640 1 --2- 172.21.15.78:0/2811890736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd285810 0x55ddcd29f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99da59c640 1 -- 172.21.15.78:0/2811890736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddcd29f670 con 0x55ddcd27a630 2024-08-18T23:43:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.935+0000 7f99dad9d640 1 --2- 172.21.15.78:0/2811890736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd27af50 0x55ddcd27f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:43:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.936+0000 7f99da59c640 1 --2- 172.21.15.78:0/2811890736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27a630 0x55ddcd27ed20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f99cc005a70 tx=0x7f99cc014050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5beefcf95e8a3185 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.936+0000 7f99beff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:43:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.936+0000 7f99beff5640 1 -- 172.21.15.78:0/2811890736 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc02e040 con 0x55ddcd27a630 2024-08-18T23:43:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.936+0000 7f99beff5640 1 -- 172.21.15.78:0/2811890736 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f99cc0053c0 con 0x55ddcd27a630 2024-08-18T23:43:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.936+0000 7f99e35aa740 1 -- 172.21.15.78:0/2811890736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99cc01a070 con 0x55ddcd27a630 2024-08-18T23:43:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.936+0000 7f99beff5640 1 -- 172.21.15.78:0/2811890736 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f99cc0055c0 con 0x55ddcd27a630 2024-08-18T23:43:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.936+0000 7f99e35aa740 10 client.15108 fetch_fsmap learned FSMap version 24 2024-08-18T23:43:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.936+0000 7f99e35aa740 1 -- 172.21.15.78:0/2811890736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ddcd2a08e0 con 0x55ddcd27a630 2024-08-18T23:43:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.936+0000 7f99beff5640 1 -- 172.21.15.78:0/2811890736 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99cc01a070 con 0x55ddcd27a630 2024-08-18T23:43:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.936+0000 7f99beff5640 1 -- 172.21.15.78:0/2811890736 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f99cc00e6d0 con 0x55ddcd27a630 2024-08-18T23:43:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.936+0000 7f99e35aa740 10 client.15108 fetch_fsmap finished waiting for FSMap version 24 2024-08-18T23:43:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.936+0000 7f99e35aa740 10 client.15108 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-18T23:43:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.936+0000 7f99e35aa740 20 client.15108 populate_metadata read hostname 'smithi078' 2024-08-18T23:43:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.936+0000 7f99e35aa740 1 --2- 172.21.15.78:0/2811890736 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55ddcd304bf0 0x55ddcd303900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.936+0000 7f99e35aa740 4 client.15108 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-18T23:43:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.936+0000 7f99e35aa740 1 -- 172.21.15.78:0/2811890736 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "status"}) -- 0x55ddcd1e7190 con 0x55ddcd304bf0 2024-08-18T23:43:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.937+0000 7f99dad9d640 1 --2- 172.21.15.78:0/2811890736 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55ddcd304bf0 0x55ddcd303900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.937+0000 7f99dad9d640 1 --2- 172.21.15.78:0/2811890736 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55ddcd304bf0 0x55ddcd303900 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5386a20409d43d6e server_cookie=380c56d88402a0df in_seq=0 out_seq=0 2024-08-18T23:43:27.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.937+0000 7f99beff5640 10 client.15108 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:43:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.937+0000 7f99beff5640 1 -- 172.21.15.78:0/2811890736 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ddcd1e7190 con 0x55ddcd304bf0 2024-08-18T23:43:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.937+0000 7f99beff5640 10 client.15108 handle_command_reply: tid=0 2024-08-18T23:43:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.937+0000 7f99e35aa740 1 client.15108 shutdown 2024-08-18T23:43:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.937+0000 7f99e35aa740 1 -- 172.21.15.78:0/2811890736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27a630 msgr2=0x55ddcd27ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.937+0000 7f99e35aa740 1 --2- 172.21.15.78:0/2811890736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27a630 0x55ddcd27ed20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f99cc005a70 tx=0x7f99cc014050 comp rx=0 tx=0).stop 2024-08-18T23:43:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.937+0000 7f99e35aa740 1 -- 172.21.15.78:0/2811890736 shutdown_connections 2024-08-18T23:43:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.937+0000 7f99e35aa740 1 --2- 172.21.15.78:0/2811890736 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55ddcd304bf0 0x55ddcd303900 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.937+0000 7f99e35aa740 1 --2- 172.21.15.78:0/2811890736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddcd285810 0x55ddcd29f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.937+0000 7f99e35aa740 1 --2- 172.21.15.78:0/2811890736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ddcd27af50 0x55ddcd27f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.937+0000 7f99beff5640 0 client.15108 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:43:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.937+0000 7f99e35aa740 1 --2- 172.21.15.78:0/2811890736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ddcd27a630 0x55ddcd27ed20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.937+0000 7f99e35aa740 1 -- 172.21.15.78:0/2811890736 >> 172.21.15.78:0/2811890736 conn(0x55ddcd276bc0 msgr2=0x55ddcd2817c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:27.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.938+0000 7f99e35aa740 1 -- 172.21.15.78:0/2811890736 shutdown_connections 2024-08-18T23:43:27.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.938+0000 7f99e35aa740 1 -- 172.21.15.78:0/2811890736 wait complete. 2024-08-18T23:43:27.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.938+0000 7f99e35aa740 20 client.15108 trim_cache size 0 max 16384 2024-08-18T23:43:27.956 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:43:27.956 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "4f1128e6-5dba-11ef-bccf-c7b262605968", 2024-08-18T23:43:27.956 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 1, 2024-08-18T23:43:27.956 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24428, 2024-08-18T23:43:27.956 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-08-18T23:43:27.956 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-08-18T23:43:27.956 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-08-18T23:43:27.957 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 81.689208180999998, 2024-08-18T23:43:27.957 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 24, 2024-08-18T23:43:27.957 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 92, 2024-08-18T23:43:27.957 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 88, 2024-08-18T23:43:27.957 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 137.30920843999999 2024-08-18T23:43:27.957 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:43:27.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.938+0000 7f99e11ac640 1 -- 172.21.15.78:0/661172494 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f99c40a2b80 msgr2=0x7f99c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.938+0000 7f99e11ac640 1 --2- 172.21.15.78:0/661172494 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f99c40a2b80 0x7f99c40a5040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f99d40a5780 tx=0x7f99d4067c10 comp rx=0 tx=0).stop 2024-08-18T23:43:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.938+0000 7f99e11ac640 1 -- 172.21.15.78:0/661172494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99dc102c20 msgr2=0x7f99dc19ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.938+0000 7f99e11ac640 1 --2- 172.21.15.78:0/661172494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99dc102c20 0x7f99dc19ba20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f99d400db90 tx=0x7f99d400dc80 comp rx=0 tx=0).stop 2024-08-18T23:43:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.938+0000 7f99e11ac640 1 -- 172.21.15.78:0/661172494 shutdown_connections 2024-08-18T23:43:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.938+0000 7f99e11ac640 1 --2- 172.21.15.78:0/661172494 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f99c40a2b80 0x7f99c40a5040 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.938+0000 7f99e11ac640 1 --2- 172.21.15.78:0/661172494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99dc102c20 0x7f99dc19ba20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.938+0000 7f99e11ac640 1 --2- 172.21.15.78:0/661172494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99dc1022a0 0x7f99dc197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.938+0000 7f99e11ac640 1 --2- 172.21.15.78:0/661172494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99dc100b50 0x7f99dc197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.938+0000 7f99e11ac640 1 -- 172.21.15.78:0/661172494 >> 172.21.15.78:0/661172494 conn(0x7f99dc0fc500 msgr2=0x7f99dc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.938+0000 7f99e11ac640 1 -- 172.21.15.78:0/661172494 shutdown_connections 2024-08-18T23:43:27.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:27.938+0000 7f99e11ac640 1 -- 172.21.15.78:0/661172494 wait complete. 2024-08-18T23:43:28.834 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-18T23:43:29.085 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:29.460 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:29 smithi078 ceph-mon[18440]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 895 B/s wr, 5 op/s 2024-08-18T23:43:29.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:29 smithi123 ceph-mon[21808]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 895 B/s wr, 5 op/s 2024-08-18T23:43:29.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.599+0000 7f0fad763640 1 -- 172.21.15.78:0/6760167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fa8100f10 msgr2=0x7f0fa8101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:29.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.599+0000 7f0fad763640 1 --2- 172.21.15.78:0/6760167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fa8100f10 0x7f0fa8101370 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009930 tx=0x7f0f9c02a420 comp rx=0 tx=0).stop 2024-08-18T23:43:29.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.600+0000 7f0fad763640 1 -- 172.21.15.78:0/6760167 shutdown_connections 2024-08-18T23:43:29.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.600+0000 7f0fad763640 1 --2- 172.21.15.78:0/6760167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fa81018b0 0x7f0fa810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.600+0000 7f0fad763640 1 --2- 172.21.15.78:0/6760167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fa8100f10 0x7f0fa8101370 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.600+0000 7f0fad763640 1 --2- 172.21.15.78:0/6760167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fa81069f0 0x7f0fa8106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.600+0000 7f0fad763640 1 -- 172.21.15.78:0/6760167 >> 172.21.15.78:0/6760167 conn(0x7f0fa8076f40 msgr2=0x7f0fa80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:29.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.600+0000 7f0fad763640 1 -- 172.21.15.78:0/6760167 shutdown_connections 2024-08-18T23:43:29.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.600+0000 7f0fad763640 1 -- 172.21.15.78:0/6760167 wait complete. 2024-08-18T23:43:29.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.600+0000 7f0fad763640 1 Processor -- start 2024-08-18T23:43:29.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.600+0000 7f0fad763640 1 -- start start 2024-08-18T23:43:29.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.601+0000 7f0fad763640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fa8100f10 0x7f0fa8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:29.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.601+0000 7f0fad763640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fa81018b0 0x7f0fa8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:29.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.601+0000 7f0fad763640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fa81069f0 0x7f0fa819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:29.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.601+0000 7f0fad763640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa8110df0 con 0x7f0fa81069f0 2024-08-18T23:43:29.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.601+0000 7f0fad763640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa8110c70 con 0x7f0fa8100f10 2024-08-18T23:43:29.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.601+0000 7f0fad763640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa8110f70 con 0x7f0fa81018b0 2024-08-18T23:43:29.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.601+0000 7f0fa77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fa81069f0 0x7f0fa819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:29.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.601+0000 7f0fa77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fa81069f0 0x7f0fa819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59348/0 (socket says 172.21.15.78:59348) 2024-08-18T23:43:29.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.601+0000 7f0fa77fe640 1 -- 172.21.15.78:0/3655295873 learned_addr learned my addr 172.21.15.78:0/3655295873 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:29.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.601+0000 7f0fa67fc640 1 --2- 172.21.15.78:0/3655295873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fa81018b0 0x7f0fa8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:29.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.601+0000 7f0fa6ffd640 1 --2- 172.21.15.78:0/3655295873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fa8100f10 0x7f0fa8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:29.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.601+0000 7f0fa77fe640 1 -- 172.21.15.78:0/3655295873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fa81018b0 msgr2=0x7f0fa8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:29.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.601+0000 7f0fa77fe640 1 --2- 172.21.15.78:0/3655295873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fa81018b0 0x7f0fa8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.601+0000 7f0fa77fe640 1 -- 172.21.15.78:0/3655295873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fa8100f10 msgr2=0x7f0fa8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:29.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.601+0000 7f0fa77fe640 1 --2- 172.21.15.78:0/3655295873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fa8100f10 0x7f0fa8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.601+0000 7f0fa77fe640 1 -- 172.21.15.78:0/3655295873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa819d3e0 con 0x7f0fa81069f0 2024-08-18T23:43:29.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.601+0000 7f0fa77fe640 1 --2- 172.21.15.78:0/3655295873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fa81069f0 0x7f0fa819cd00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000eaa0 tx=0x7f0fa000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.601+0000 7f0f8ffff640 1 -- 172.21.15.78:0/3655295873 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa000cdb0 con 0x7f0fa81069f0 2024-08-18T23:43:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.602+0000 7f0fad763640 1 -- 172.21.15.78:0/3655295873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa819d6d0 con 0x7f0fa81069f0 2024-08-18T23:43:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.602+0000 7f0f8ffff640 1 -- 172.21.15.78:0/3655295873 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fa0005020 con 0x7f0fa81069f0 2024-08-18T23:43:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.602+0000 7f0f8ffff640 1 -- 172.21.15.78:0/3655295873 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa00116d0 con 0x7f0fa81069f0 2024-08-18T23:43:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.602+0000 7f0fad763640 1 -- 172.21.15.78:0/3655295873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa81a4fb0 con 0x7f0fa81069f0 2024-08-18T23:43:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.604+0000 7f0fafb61740 1 Processor -- start 2024-08-18T23:43:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0f8ffff640 1 -- 172.21.15.78:0/3655295873 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0fa00054d0 con 0x7f0fa81069f0 2024-08-18T23:43:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0fafb61740 1 -- start start 2024-08-18T23:43:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0fafb61740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799d6630 0x5651799d6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0fafb61740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6f50 0x5651799e12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0fafb61740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799e1810 0x5651799e3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0fafb61740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56517993b710 con 0x5651799d6f50 2024-08-18T23:43:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0fafb61740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5651798d3770 con 0x5651799e1810 2024-08-18T23:43:29.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0fafb61740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5651799376d0 con 0x5651799d6630 2024-08-18T23:43:29.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0fa6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6f50 0x5651799e12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:29.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0fa6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6f50 0x5651799e12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59358/0 (socket says 172.21.15.78:59358) 2024-08-18T23:43:29.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0fa6ffd640 1 -- 172.21.15.78:0/936196816 learned_addr learned my addr 172.21.15.78:0/936196816 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:29.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0fa67fc640 1 --2- 172.21.15.78:0/936196816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799d6630 0x5651799d6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:29.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0fa67fc640 1 --2- 172.21.15.78:0/936196816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799e1810 0x5651799e3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:29.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0fa6ffd640 1 -- 172.21.15.78:0/936196816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799d6630 msgr2=0x5651799d6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:29.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0fa6ffd640 1 --2- 172.21.15.78:0/936196816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799d6630 0x5651799d6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0fa6ffd640 1 -- 172.21.15.78:0/936196816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799e1810 msgr2=0x5651799e3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:29.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0fa6ffd640 1 --2- 172.21.15.78:0/936196816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799e1810 0x5651799e3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0fa6ffd640 1 -- 172.21.15.78:0/936196816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651798da3e0 con 0x5651799d6f50 2024-08-18T23:43:29.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0f8ffff640 1 --2- 172.21.15.78:0/3655295873 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0f900a2b80 0x7f0f900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.605+0000 7f0fa67fc640 1 --2- 172.21.15.78:0/936196816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799e1810 0x5651799e3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:43:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.606+0000 7f0fa67fc640 1 --2- 172.21.15.78:0/3655295873 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0f900a2b80 0x7f0f900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.606+0000 7f0f8ffff640 1 -- 172.21.15.78:0/3655295873 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f0fa00cf540 con 0x7f0fa81069f0 2024-08-18T23:43:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.606+0000 7f0fa6ffd640 1 --2- 172.21.15.78:0/936196816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6f50 0x5651799e12d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009c70 tx=0x7f0f98011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e9de048238e01e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.606+0000 7f0f8d7fa640 1 -- 172.21.15.78:0/936196816 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98004230 con 0x5651799d6f50 2024-08-18T23:43:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.606+0000 7f0f8d7fa640 1 -- 172.21.15.78:0/936196816 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f980043d0 con 0x5651799d6f50 2024-08-18T23:43:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.606+0000 7f0f8d7fa640 1 -- 172.21.15.78:0/936196816 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98019740 con 0x5651799d6f50 2024-08-18T23:43:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.607+0000 7f0fafb61740 1 -- 172.21.15.78:0/936196816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6f50 msgr2=0x5651799e12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.607+0000 7f0fa67fc640 1 --2- 172.21.15.78:0/3655295873 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0f900a2b80 0x7f0f900a5040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009900 tx=0x7f0f9c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.607+0000 7f0fafb61740 1 --2- 172.21.15.78:0/936196816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6f50 0x5651799e12d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009c70 tx=0x7f0f98011810 comp rx=0 tx=0).stop 2024-08-18T23:43:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.607+0000 7f0fafb61740 1 -- 172.21.15.78:0/936196816 shutdown_connections 2024-08-18T23:43:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.607+0000 7f0fafb61740 1 --2- 172.21.15.78:0/936196816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799e1810 0x5651799e3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.607+0000 7f0fafb61740 1 --2- 172.21.15.78:0/936196816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6f50 0x5651799e12d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.607+0000 7f0fafb61740 1 --2- 172.21.15.78:0/936196816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799d6630 0x5651799d6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.607+0000 7f0fafb61740 1 -- 172.21.15.78:0/936196816 >> 172.21.15.78:0/936196816 conn(0x5651799d2bc0 msgr2=0x5651799d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.607+0000 7f0fafb61740 1 -- 172.21.15.78:0/936196816 shutdown_connections 2024-08-18T23:43:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.607+0000 7f0fafb61740 1 -- 172.21.15.78:0/936196816 wait complete. 2024-08-18T23:43:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.607+0000 7f0fafb61740 1 -- start start 2024-08-18T23:43:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.607+0000 7f0fafb61740 1 Processor -- start 2024-08-18T23:43:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.607+0000 7f0fafb61740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6630 0x5651799de690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.607+0000 7f0fafb61740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799d6f50 0x5651799debd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0fafb61740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799e1810 0x5651799df110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0fafb61740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5651799e43e0 con 0x5651799d6630 2024-08-18T23:43:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0fafb61740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5651798f6970 con 0x5651799d6f50 2024-08-18T23:43:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0fafb61740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5651799e4560 con 0x5651799e1810 2024-08-18T23:43:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0fafb61740 10 client.0 _ll_register_callbacks cb 0x56517964a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:43:29.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0fa6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6630 0x5651799de690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:29.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0fa6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6630 0x5651799de690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59370/0 (socket says 172.21.15.78:59370) 2024-08-18T23:43:29.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0fa6ffd640 1 -- 172.21.15.78:0/1127060142 learned_addr learned my addr 172.21.15.78:0/1127060142 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:29.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0fa67fc640 1 --2- 172.21.15.78:0/1127060142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799d6f50 0x5651799debd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:29.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0fa77fe640 1 --2- 172.21.15.78:0/1127060142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799e1810 0x5651799df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:29.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0fa6ffd640 1 -- 172.21.15.78:0/1127060142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799e1810 msgr2=0x5651799df110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:29.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0fa6ffd640 1 --2- 172.21.15.78:0/1127060142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799e1810 0x5651799df110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0fa6ffd640 1 -- 172.21.15.78:0/1127060142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799d6f50 msgr2=0x5651799debd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:29.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0fa6ffd640 1 --2- 172.21.15.78:0/1127060142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799d6f50 0x5651799debd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0fa6ffd640 1 -- 172.21.15.78:0/1127060142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651796ad2b0 con 0x5651799d6630 2024-08-18T23:43:29.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0fa6ffd640 1 --2- 172.21.15.78:0/1127060142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6630 0x5651799de690 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0f98005bc0 tx=0x7f0f98005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c5f537dfb53856 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:29.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0f8aff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:43:29.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0f8aff5640 1 -- 172.21.15.78:0/1127060142 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98027070 con 0x5651799d6630 2024-08-18T23:43:29.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0fafb61740 1 -- 172.21.15.78:0/1127060142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f980043d0 con 0x5651799d6630 2024-08-18T23:43:29.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0f8aff5640 1 -- 172.21.15.78:0/1127060142 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f0f98003800 con 0x5651799d6630 2024-08-18T23:43:29.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.608+0000 7f0f8aff5640 1 -- 172.21.15.78:0/1127060142 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0f98033050 con 0x5651799d6630 2024-08-18T23:43:29.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.609+0000 7f0fafb61740 10 client.15120 fetch_fsmap learned FSMap version 24 2024-08-18T23:43:29.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.609+0000 7f0fafb61740 1 -- 172.21.15.78:0/1127060142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5651799eff00 con 0x5651799d6630 2024-08-18T23:43:29.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.609+0000 7f0f8aff5640 1 -- 172.21.15.78:0/1127060142 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f980043d0 con 0x5651799d6630 2024-08-18T23:43:29.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.609+0000 7f0f8aff5640 1 -- 172.21.15.78:0/1127060142 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f0f98018070 con 0x5651799d6630 2024-08-18T23:43:29.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.609+0000 7f0fafb61740 10 client.15120 fetch_fsmap finished waiting for FSMap version 24 2024-08-18T23:43:29.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.609+0000 7f0fafb61740 10 client.15120 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-18T23:43:29.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.609+0000 7f0fafb61740 20 client.15120 populate_metadata read hostname 'smithi078' 2024-08-18T23:43:29.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.609+0000 7f0fafb61740 1 --2- 172.21.15.78:0/1127060142 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5651799f2350 0x5651799f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:29.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.609+0000 7f0fafb61740 4 client.15120 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:43:29.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.609+0000 7f0fafb61740 1 -- 172.21.15.78:0/1127060142 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565179948c80 con 0x5651799f2350 2024-08-18T23:43:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.609+0000 7f0fa67fc640 1 --2- 172.21.15.78:0/1127060142 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5651799f2350 0x5651799f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.610+0000 7f0f8aff5640 10 client.15120 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:43:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.610+0000 7f0fa67fc640 1 --2- 172.21.15.78:0/1127060142 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5651799f2350 0x5651799f37c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fee87930a53af2c6 server_cookie=98b9d68f3e85dc2f in_seq=0 out_seq=0 2024-08-18T23:43:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.610+0000 7f0f8aff5640 1 -- 172.21.15.78:0/1127060142 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565179948c80 con 0x5651799f2350 2024-08-18T23:43:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.610+0000 7f0f8aff5640 10 client.15120 handle_command_reply: tid=0 2024-08-18T23:43:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.610+0000 7f0fafb61740 1 client.15120 shutdown 2024-08-18T23:43:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.611+0000 7f0fafb61740 1 -- 172.21.15.78:0/1127060142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6630 msgr2=0x5651799de690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.611+0000 7f0fafb61740 1 --2- 172.21.15.78:0/1127060142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6630 0x5651799de690 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0f98005bc0 tx=0x7f0f98005750 comp rx=0 tx=0).stop 2024-08-18T23:43:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.611+0000 7f0fafb61740 1 -- 172.21.15.78:0/1127060142 shutdown_connections 2024-08-18T23:43:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.611+0000 7f0fafb61740 1 --2- 172.21.15.78:0/1127060142 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5651799f2350 0x5651799f37c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.611+0000 7f0fafb61740 1 --2- 172.21.15.78:0/1127060142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799e1810 0x5651799df110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.611+0000 7f0fafb61740 1 --2- 172.21.15.78:0/1127060142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799d6f50 0x5651799debd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.611+0000 7f0fafb61740 1 --2- 172.21.15.78:0/1127060142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6630 0x5651799de690 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.611+0000 7f0fafb61740 1 -- 172.21.15.78:0/1127060142 >> 172.21.15.78:0/1127060142 conn(0x5651799d2bc0 msgr2=0x5651799d88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:29.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.611+0000 7f0f8aff5640 0 client.15120 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:43:29.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.611+0000 7f0fafb61740 1 -- 172.21.15.78:0/1127060142 shutdown_connections 2024-08-18T23:43:29.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.611+0000 7f0fafb61740 1 -- 172.21.15.78:0/1127060142 wait complete. 2024-08-18T23:43:29.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.611+0000 7f0fafb61740 20 client.15120 trim_cache size 0 max 16384 2024-08-18T23:43:29.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.629+0000 7f0fafb61740 1 Processor -- start 2024-08-18T23:43:29.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.629+0000 7f0fafb61740 1 -- start start 2024-08-18T23:43:29.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.629+0000 7f0fafb61740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799d6630 0x5651799f2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:29.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.629+0000 7f0fafb61740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6f50 0x5651799fba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:29.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.629+0000 7f0fafb61740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799e1810 0x5651799fbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:29.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.629+0000 7f0fafb61740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5651799f1b00 con 0x5651799d6f50 2024-08-18T23:43:29.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.629+0000 7f0fafb61740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5651799f21d0 con 0x5651799e1810 2024-08-18T23:43:29.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.629+0000 7f0fafb61740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5651799f28a0 con 0x5651799d6630 2024-08-18T23:43:29.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.629+0000 7f0fa67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6f50 0x5651799fba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.629+0000 7f0fa67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6f50 0x5651799fba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59380/0 (socket says 172.21.15.78:59380) 2024-08-18T23:43:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.629+0000 7f0fa67fc640 1 -- 172.21.15.78:0/3043910674 learned_addr learned my addr 172.21.15.78:0/3043910674 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.629+0000 7f0fa77fe640 1 --2- 172.21.15.78:0/3043910674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799e1810 0x5651799fbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.629+0000 7f0fa6ffd640 1 --2- 172.21.15.78:0/3043910674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799d6630 0x5651799f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.629+0000 7f0fa67fc640 1 -- 172.21.15.78:0/3043910674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799d6630 msgr2=0x5651799f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.629+0000 7f0fa67fc640 1 --2- 172.21.15.78:0/3043910674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799d6630 0x5651799f2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.629+0000 7f0fa67fc640 1 -- 172.21.15.78:0/3043910674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799e1810 msgr2=0x5651799fbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.630+0000 7f0fa67fc640 1 --2- 172.21.15.78:0/3043910674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799e1810 0x5651799fbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.630+0000 7f0fa67fc640 1 -- 172.21.15.78:0/3043910674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651799d29f0 con 0x5651799d6f50 2024-08-18T23:43:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.630+0000 7f0fa67fc640 1 --2- 172.21.15.78:0/3043910674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6f50 0x5651799fba70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c006010 tx=0x7f0f9c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ee9cad69277db4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.630+0000 7f0f8cff9640 1 -- 172.21.15.78:0/3043910674 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c043070 con 0x5651799d6f50 2024-08-18T23:43:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.630+0000 7f0f8cff9640 1 -- 172.21.15.78:0/3043910674 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f9c003cd0 con 0x5651799d6f50 2024-08-18T23:43:29.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.630+0000 7f0f8cff9640 1 -- 172.21.15.78:0/3043910674 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c035040 con 0x5651799d6f50 2024-08-18T23:43:29.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.631+0000 7f0fafb61740 1 -- 172.21.15.78:0/3043910674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6f50 msgr2=0x5651799fba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:29.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.631+0000 7f0fafb61740 1 --2- 172.21.15.78:0/3043910674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6f50 0x5651799fba70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c006010 tx=0x7f0f9c03f000 comp rx=0 tx=0).stop 2024-08-18T23:43:29.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.631+0000 7f0fafb61740 1 -- 172.21.15.78:0/3043910674 shutdown_connections 2024-08-18T23:43:29.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.631+0000 7f0fafb61740 1 --2- 172.21.15.78:0/3043910674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799e1810 0x5651799fbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.631+0000 7f0fafb61740 1 --2- 172.21.15.78:0/3043910674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799d6f50 0x5651799fba70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.631+0000 7f0fafb61740 1 --2- 172.21.15.78:0/3043910674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799d6630 0x5651799f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.631+0000 7f0fafb61740 1 -- 172.21.15.78:0/3043910674 >> 172.21.15.78:0/3043910674 conn(0x5651799d2bc0 msgr2=0x5651799d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:29.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.631+0000 7f0fafb61740 1 -- 172.21.15.78:0/3043910674 shutdown_connections 2024-08-18T23:43:29.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.631+0000 7f0fafb61740 1 -- 172.21.15.78:0/3043910674 wait complete. 2024-08-18T23:43:29.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.631+0000 7f0fafb61740 1 -- start start 2024-08-18T23:43:29.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.631+0000 7f0fafb61740 1 Processor -- start 2024-08-18T23:43:29.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0fafb61740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799d6630 0x5651799f7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:29.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0fafb61740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799d6f50 0x5651799f7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:29.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0fafb61740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799e1810 0x565179a5aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:29.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0fafb61740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5651799f3640 con 0x5651799e1810 2024-08-18T23:43:29.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0fafb61740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5651799f2f70 con 0x5651799d6f50 2024-08-18T23:43:29.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0fafb61740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5651799f3d10 con 0x5651799d6630 2024-08-18T23:43:29.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0fafb61740 10 client.0 _ll_register_callbacks cb 0x7f0f9c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:43:29.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0fa77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799e1810 0x565179a5aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:29.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0fa77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799e1810 0x565179a5aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59394/0 (socket says 172.21.15.78:59394) 2024-08-18T23:43:29.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0fa77fe640 1 -- 172.21.15.78:0/486600304 learned_addr learned my addr 172.21.15.78:0/486600304 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:29.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0fa67fc640 1 --2- 172.21.15.78:0/486600304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799d6f50 0x5651799f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:29.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0fa6ffd640 1 --2- 172.21.15.78:0/486600304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799d6630 0x5651799f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:29.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0fa77fe640 1 -- 172.21.15.78:0/486600304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799d6630 msgr2=0x5651799f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:29.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0fa77fe640 1 --2- 172.21.15.78:0/486600304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799d6630 0x5651799f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0fa77fe640 1 -- 172.21.15.78:0/486600304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799d6f50 msgr2=0x5651799f7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:29.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0fa77fe640 1 --2- 172.21.15.78:0/486600304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799d6f50 0x5651799f7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0fa77fe640 1 -- 172.21.15.78:0/486600304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651799ce130 con 0x5651799e1810 2024-08-18T23:43:29.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0fa77fe640 1 --2- 172.21.15.78:0/486600304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799e1810 0x565179a5aa50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0098490 tx=0x7f0fa009bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f589b1dc724be5 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:29.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0f8aff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:43:29.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.632+0000 7f0f8aff5640 1 -- 172.21.15.78:0/486600304 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa00b3020 con 0x5651799e1810 2024-08-18T23:43:29.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.633+0000 7f0f8aff5640 1 -- 172.21.15.78:0/486600304 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f0fa00cf260 con 0x5651799e1810 2024-08-18T23:43:29.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.633+0000 7f0fafb61740 1 -- 172.21.15.78:0/486600304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f9c003cd0 con 0x5651799e1810 2024-08-18T23:43:29.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.633+0000 7f0f8aff5640 1 -- 172.21.15.78:0/486600304 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0fa00ce040 con 0x5651799e1810 2024-08-18T23:43:29.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.633+0000 7f0fafb61740 10 client.15126 fetch_fsmap learned FSMap version 24 2024-08-18T23:43:29.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.633+0000 7f0fafb61740 1 -- 172.21.15.78:0/486600304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x565179a5c1e0 con 0x5651799e1810 2024-08-18T23:43:29.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.633+0000 7f0f8aff5640 1 -- 172.21.15.78:0/486600304 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f9c003cd0 con 0x5651799e1810 2024-08-18T23:43:29.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.633+0000 7f0f8aff5640 1 -- 172.21.15.78:0/486600304 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f0fa000f040 con 0x5651799e1810 2024-08-18T23:43:29.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.633+0000 7f0fafb61740 10 client.15126 fetch_fsmap finished waiting for FSMap version 24 2024-08-18T23:43:29.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.633+0000 7f0fafb61740 10 client.15126 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-18T23:43:29.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.633+0000 7f0fafb61740 20 client.15126 populate_metadata read hostname 'smithi078' 2024-08-18T23:43:29.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.633+0000 7f0fafb61740 1 --2- 172.21.15.78:0/486600304 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x565179a60e10 0x565179a5fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:29.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.633+0000 7f0fafb61740 4 client.15126 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-18T23:43:29.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.633+0000 7f0fafb61740 1 -- 172.21.15.78:0/486600304 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "status"}) -- 0x565179943190 con 0x565179a60e10 2024-08-18T23:43:29.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.634+0000 7f0fa6ffd640 1 --2- 172.21.15.78:0/486600304 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x565179a60e10 0x565179a5fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:29.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.634+0000 7f0fa6ffd640 1 --2- 172.21.15.78:0/486600304 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x565179a60e10 0x565179a5fb20 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f5a8d4f66a9882 server_cookie=82d2f76fc59e770 in_seq=0 out_seq=0 2024-08-18T23:43:29.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.634+0000 7f0f8aff5640 10 client.15126 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:43:29.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.634+0000 7f0f8aff5640 1 -- 172.21.15.78:0/486600304 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x565179943190 con 0x565179a60e10 2024-08-18T23:43:29.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.634+0000 7f0f8aff5640 10 client.15126 handle_command_reply: tid=0 2024-08-18T23:43:29.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.634+0000 7f0fafb61740 1 client.15126 shutdown 2024-08-18T23:43:29.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.634+0000 7f0fafb61740 1 -- 172.21.15.78:0/486600304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799e1810 msgr2=0x565179a5aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:29.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.634+0000 7f0fafb61740 1 --2- 172.21.15.78:0/486600304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799e1810 0x565179a5aa50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0098490 tx=0x7f0fa009bc00 comp rx=0 tx=0).stop 2024-08-18T23:43:29.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.634+0000 7f0fafb61740 1 -- 172.21.15.78:0/486600304 shutdown_connections 2024-08-18T23:43:29.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.634+0000 7f0fafb61740 1 --2- 172.21.15.78:0/486600304 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x565179a60e10 0x565179a5fb20 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.634+0000 7f0f8aff5640 0 client.15126 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:43:29.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.634+0000 7f0fafb61740 1 --2- 172.21.15.78:0/486600304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5651799e1810 0x565179a5aa50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.634+0000 7f0fafb61740 1 --2- 172.21.15.78:0/486600304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651799d6f50 0x5651799f7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.634+0000 7f0fafb61740 1 --2- 172.21.15.78:0/486600304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5651799d6630 0x5651799f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.634+0000 7f0fafb61740 1 -- 172.21.15.78:0/486600304 >> 172.21.15.78:0/486600304 conn(0x5651799d2bc0 msgr2=0x5651799ec7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:29.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.634+0000 7f0fafb61740 1 -- 172.21.15.78:0/486600304 shutdown_connections 2024-08-18T23:43:29.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.634+0000 7f0fafb61740 1 -- 172.21.15.78:0/486600304 wait complete. 2024-08-18T23:43:29.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.635+0000 7f0fafb61740 20 client.15126 trim_cache size 0 max 16384 2024-08-18T23:43:29.641 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:43:29.642 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "4f1128e6-5dba-11ef-bccf-c7b262605968", 2024-08-18T23:43:29.642 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-08-18T23:43:29.642 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24452, 2024-08-18T23:43:29.642 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-08-18T23:43:29.642 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-08-18T23:43:29.642 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-08-18T23:43:29.642 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 116.493409465, 2024-08-18T23:43:29.642 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 24, 2024-08-18T23:43:29.642 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 92, 2024-08-18T23:43:29.642 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 88, 2024-08-18T23:43:29.642 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 127.156389369 2024-08-18T23:43:29.642 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:43:29.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.635+0000 7f0fad763640 1 -- 172.21.15.78:0/3655295873 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0f900a2b80 msgr2=0x7f0f900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:29.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.635+0000 7f0fad763640 1 --2- 172.21.15.78:0/3655295873 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0f900a2b80 0x7f0f900a5040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009900 tx=0x7f0f9c0058d0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.635+0000 7f0fad763640 1 -- 172.21.15.78:0/3655295873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fa81069f0 msgr2=0x7f0fa819cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:29.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.635+0000 7f0fad763640 1 --2- 172.21.15.78:0/3655295873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fa81069f0 0x7f0fa819cd00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000eaa0 tx=0x7f0fa000ef60 comp rx=0 tx=0).stop 2024-08-18T23:43:29.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.635+0000 7f0fad763640 1 -- 172.21.15.78:0/3655295873 shutdown_connections 2024-08-18T23:43:29.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.635+0000 7f0fad763640 1 --2- 172.21.15.78:0/3655295873 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0f900a2b80 0x7f0f900a5040 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.635+0000 7f0fad763640 1 --2- 172.21.15.78:0/3655295873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fa81069f0 0x7f0fa819cd00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.635+0000 7f0fad763640 1 --2- 172.21.15.78:0/3655295873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fa81018b0 0x7f0fa8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.635+0000 7f0fad763640 1 --2- 172.21.15.78:0/3655295873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0fa8100f10 0x7f0fa8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.635+0000 7f0fad763640 1 -- 172.21.15.78:0/3655295873 >> 172.21.15.78:0/3655295873 conn(0x7f0fa8076f40 msgr2=0x7f0fa80fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.636+0000 7f0fad763640 1 -- 172.21.15.78:0/3655295873 shutdown_connections 2024-08-18T23:43:29.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:29.636+0000 7f0fad763640 1 -- 172.21.15.78:0/3655295873 wait complete. 2024-08-18T23:43:29.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:29 smithi161 ceph-mon[22027]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 895 B/s wr, 5 op/s 2024-08-18T23:43:30.360 INFO:tasks.cephadm:Setup complete, yielding 2024-08-18T23:43:30.361 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-18T23:43:30.370 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-08-18T23:43:30.370 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- bash -c 'ceph orch status' 2024-08-18T23:43:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:31.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.114+0000 7fd49d18f640 1 -- 172.21.15.78:0/3198015273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd498101930 msgr2=0x7fd498101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:31.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.114+0000 7fd49d18f640 1 --2- 172.21.15.78:0/3198015273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd498101930 0x7fd498101d30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd488009900 tx=0x7fd48802a410 comp rx=0 tx=0).stop 2024-08-18T23:43:31.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.114+0000 7fd49d18f640 1 -- 172.21.15.78:0/3198015273 shutdown_connections 2024-08-18T23:43:31.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.114+0000 7fd49d18f640 1 --2- 172.21.15.78:0/3198015273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4981034f0 0x7fd498109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:31.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.114+0000 7fd49d18f640 1 --2- 172.21.15.78:0/3198015273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd498102b30 0x7fd498102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:31.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.114+0000 7fd49d18f640 1 --2- 172.21.15.78:0/3198015273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd498101930 0x7fd498101d30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:31.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.114+0000 7fd49d18f640 1 -- 172.21.15.78:0/3198015273 >> 172.21.15.78:0/3198015273 conn(0x7fd4980fd0c0 msgr2=0x7fd4980ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:31.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.114+0000 7fd49d18f640 1 -- 172.21.15.78:0/3198015273 shutdown_connections 2024-08-18T23:43:31.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.114+0000 7fd49d18f640 1 -- 172.21.15.78:0/3198015273 wait complete. 2024-08-18T23:43:31.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.114+0000 7fd49d18f640 1 Processor -- start 2024-08-18T23:43:31.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.115+0000 7fd49d18f640 1 -- start start 2024-08-18T23:43:31.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.115+0000 7fd49d18f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd498101930 0x7fd498075cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:31.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.115+0000 7fd49d18f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd498102b30 0x7fd498074310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:31.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.115+0000 7fd49d18f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4981034f0 0x7fd498074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:31.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.115+0000 7fd49d18f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd49810d5c0 con 0x7fd4981034f0 2024-08-18T23:43:31.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.115+0000 7fd49d18f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd49810d440 con 0x7fd498101930 2024-08-18T23:43:31.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.115+0000 7fd49d18f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd49810d740 con 0x7fd498102b30 2024-08-18T23:43:31.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.115+0000 7fd49759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4981034f0 0x7fd498074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.115+0000 7fd49759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4981034f0 0x7fd498074850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59414/0 (socket says 172.21.15.78:59414) 2024-08-18T23:43:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.115+0000 7fd49759e640 1 -- 172.21.15.78:0/919431388 learned_addr learned my addr 172.21.15.78:0/919431388 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.115+0000 7fd496d9d640 1 --2- 172.21.15.78:0/919431388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd498101930 0x7fd498075cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.115+0000 7fd49759e640 1 -- 172.21.15.78:0/919431388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd498102b30 msgr2=0x7fd498074310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.115+0000 7fd49659c640 1 --2- 172.21.15.78:0/919431388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd498102b30 0x7fd498074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.115+0000 7fd49759e640 1 --2- 172.21.15.78:0/919431388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd498102b30 0x7fd498074310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.115+0000 7fd49759e640 1 -- 172.21.15.78:0/919431388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd498101930 msgr2=0x7fd498075cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.115+0000 7fd49759e640 1 --2- 172.21.15.78:0/919431388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd498101930 0x7fd498075cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.115+0000 7fd49759e640 1 -- 172.21.15.78:0/919431388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd498075110 con 0x7fd4981034f0 2024-08-18T23:43:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.115+0000 7fd496d9d640 1 --2- 172.21.15.78:0/919431388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd498101930 0x7fd498075cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:43:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.115+0000 7fd49759e640 1 --2- 172.21.15.78:0/919431388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4981034f0 0x7fd498074850 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd49000eaa0 tx=0x7fd49000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.116+0000 7fd47ffff640 1 -- 172.21.15.78:0/919431388 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49000cdb0 con 0x7fd4981034f0 2024-08-18T23:43:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.116+0000 7fd49d18f640 1 -- 172.21.15.78:0/919431388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4981a1de0 con 0x7fd4981034f0 2024-08-18T23:43:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.116+0000 7fd47ffff640 1 -- 172.21.15.78:0/919431388 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd490005020 con 0x7fd4981034f0 2024-08-18T23:43:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.116+0000 7fd47ffff640 1 -- 172.21.15.78:0/919431388 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4900116d0 con 0x7fd4981034f0 2024-08-18T23:43:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.116+0000 7fd49d18f640 1 -- 172.21.15.78:0/919431388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4981a22e0 con 0x7fd4981034f0 2024-08-18T23:43:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.117+0000 7fd49d18f640 1 -- 172.21.15.78:0/919431388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd498101d30 con 0x7fd4981034f0 2024-08-18T23:43:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.118+0000 7fd47ffff640 1 -- 172.21.15.78:0/919431388 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd4900054d0 con 0x7fd4981034f0 2024-08-18T23:43:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.118+0000 7fd47ffff640 1 --2- 172.21.15.78:0/919431388 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd4800a2b80 0x7fd4800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.118+0000 7fd496d9d640 1 --2- 172.21.15.78:0/919431388 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd4800a2b80 0x7fd4800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.118+0000 7fd47ffff640 1 -- 172.21.15.78:0/919431388 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fd4900cf6e0 con 0x7fd4981034f0 2024-08-18T23:43:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.119+0000 7fd496d9d640 1 --2- 172.21.15.78:0/919431388 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd4800a2b80 0x7fd4800a5040 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd488006010 tx=0x7fd488032cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.122+0000 7fd47ffff640 1 -- 172.21.15.78:0/919431388 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd490016030 con 0x7fd4981034f0 2024-08-18T23:43:31.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.306+0000 7fd49d18f640 1 -- 172.21.15.78:0/919431388 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fd498062360 con 0x7fd4800a2b80 2024-08-18T23:43:31.308 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:31 smithi078 ceph-mon[18440]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 8 op/s 2024-08-18T23:43:31.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.307+0000 7fd47ffff640 1 -- 172.21.15.78:0/919431388 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fd498062360 con 0x7fd4800a2b80 2024-08-18T23:43:31.308 INFO:teuthology.orchestra.run.smithi078.stdout:Backend: cephadm 2024-08-18T23:43:31.309 INFO:teuthology.orchestra.run.smithi078.stdout:Available: Yes 2024-08-18T23:43:31.309 INFO:teuthology.orchestra.run.smithi078.stdout:Paused: No 2024-08-18T23:43:31.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.310+0000 7fd49d18f640 1 -- 172.21.15.78:0/919431388 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd4800a2b80 msgr2=0x7fd4800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:31.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.310+0000 7fd49d18f640 1 --2- 172.21.15.78:0/919431388 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd4800a2b80 0x7fd4800a5040 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd488006010 tx=0x7fd488032cf0 comp rx=0 tx=0).stop 2024-08-18T23:43:31.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.310+0000 7fd49d18f640 1 -- 172.21.15.78:0/919431388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4981034f0 msgr2=0x7fd498074850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:31.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.310+0000 7fd49d18f640 1 --2- 172.21.15.78:0/919431388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4981034f0 0x7fd498074850 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd49000eaa0 tx=0x7fd49000ef60 comp rx=0 tx=0).stop 2024-08-18T23:43:31.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.310+0000 7fd49d18f640 1 -- 172.21.15.78:0/919431388 shutdown_connections 2024-08-18T23:43:31.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.310+0000 7fd49d18f640 1 --2- 172.21.15.78:0/919431388 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd4800a2b80 0x7fd4800a5040 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:31.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.310+0000 7fd49d18f640 1 --2- 172.21.15.78:0/919431388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4981034f0 0x7fd498074850 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:31.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.310+0000 7fd49d18f640 1 --2- 172.21.15.78:0/919431388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd498102b30 0x7fd498074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:31.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.310+0000 7fd49d18f640 1 --2- 172.21.15.78:0/919431388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd498101930 0x7fd498075cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:31.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.310+0000 7fd49d18f640 1 -- 172.21.15.78:0/919431388 >> 172.21.15.78:0/919431388 conn(0x7fd4980fd0c0 msgr2=0x7fd498105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:31.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.310+0000 7fd49d18f640 1 -- 172.21.15.78:0/919431388 shutdown_connections 2024-08-18T23:43:31.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:31.310+0000 7fd49d18f640 1 -- 172.21.15.78:0/919431388 wait complete. 2024-08-18T23:43:31.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:31 smithi123 ceph-mon[21808]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 8 op/s 2024-08-18T23:43:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:31 smithi161 ceph-mon[22027]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 8 op/s 2024-08-18T23:43:32.030 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- bash -c 'ceph orch ps' 2024-08-18T23:43:32.292 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.787+0000 7f29a7772640 1 -- 172.21.15.78:0/3754128889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a0100840 msgr2=0x7f29a0100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.787+0000 7f29a7772640 1 --2- 172.21.15.78:0/3754128889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a0100840 0x7f29a0100c40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f299c009900 tx=0x7f299c02a410 comp rx=0 tx=0).stop 2024-08-18T23:43:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.787+0000 7f29a7772640 1 -- 172.21.15.78:0/3754128889 shutdown_connections 2024-08-18T23:43:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.787+0000 7f29a7772640 1 --2- 172.21.15.78:0/3754128889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a0102400 0x7f29a0108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.787+0000 7f29a7772640 1 --2- 172.21.15.78:0/3754128889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29a0101a40 0x7f29a0101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.787+0000 7f29a7772640 1 --2- 172.21.15.78:0/3754128889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a0100840 0x7f29a0100c40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.787+0000 7f29a7772640 1 -- 172.21.15.78:0/3754128889 >> 172.21.15.78:0/3754128889 conn(0x7f29a00fbfd0 msgr2=0x7f29a00fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.787+0000 7f29a7772640 1 -- 172.21.15.78:0/3754128889 shutdown_connections 2024-08-18T23:43:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.787+0000 7f29a7772640 1 -- 172.21.15.78:0/3754128889 wait complete. 2024-08-18T23:43:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.787+0000 7f29a7772640 1 Processor -- start 2024-08-18T23:43:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.787+0000 7f29a7772640 1 -- start start 2024-08-18T23:43:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.787+0000 7f29a7772640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a0100840 0x7f29a006b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.787+0000 7f29a7772640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29a0101a40 0x7f29a019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.787+0000 7f29a7772640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a0102400 0x7f29a019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.787+0000 7f29a7772640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29a010c4a0 con 0x7f29a0100840 2024-08-18T23:43:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.787+0000 7f29a7772640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29a010c320 con 0x7f29a0102400 2024-08-18T23:43:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.787+0000 7f29a7772640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f29a010c620 con 0x7f29a0101a40 2024-08-18T23:43:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.788+0000 7f29a550e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a0100840 0x7f29a006b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.788+0000 7f29a550e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a0100840 0x7f29a006b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59444/0 (socket says 172.21.15.78:59444) 2024-08-18T23:43:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.788+0000 7f29a550e640 1 -- 172.21.15.78:0/3007437681 learned_addr learned my addr 172.21.15.78:0/3007437681 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.788+0000 7f29a5d0f640 1 --2- 172.21.15.78:0/3007437681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a0102400 0x7f29a019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.788+0000 7f29a4d0d640 1 --2- 172.21.15.78:0/3007437681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29a0101a40 0x7f29a019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.788+0000 7f29a550e640 1 -- 172.21.15.78:0/3007437681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29a0101a40 msgr2=0x7f29a019c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.788+0000 7f29a550e640 1 --2- 172.21.15.78:0/3007437681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29a0101a40 0x7f29a019c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.788+0000 7f29a550e640 1 -- 172.21.15.78:0/3007437681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a0102400 msgr2=0x7f29a019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.788+0000 7f29a550e640 1 --2- 172.21.15.78:0/3007437681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a0102400 0x7f29a019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.788+0000 7f29a550e640 1 -- 172.21.15.78:0/3007437681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a019f850 con 0x7f29a0100840 2024-08-18T23:43:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.788+0000 7f29a5d0f640 1 --2- 172.21.15.78:0/3007437681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a0102400 0x7f29a019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:43:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.788+0000 7f29a550e640 1 --2- 172.21.15.78:0/3007437681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a0100840 0x7f29a006b020 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f299c031610 tx=0x7f299c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.788+0000 7f29927fc640 1 -- 172.21.15.78:0/3007437681 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c03f030 con 0x7f29a0100840 2024-08-18T23:43:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.788+0000 7f29927fc640 1 -- 172.21.15.78:0/3007437681 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f299c003ce0 con 0x7f29a0100840 2024-08-18T23:43:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.788+0000 7f29a7772640 1 -- 172.21.15.78:0/3007437681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a019fa80 con 0x7f29a0100840 2024-08-18T23:43:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.788+0000 7f29927fc640 1 -- 172.21.15.78:0/3007437681 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c035070 con 0x7f29a0100840 2024-08-18T23:43:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.788+0000 7f29a7772640 1 -- 172.21.15.78:0/3007437681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a019ff60 con 0x7f29a0100840 2024-08-18T23:43:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.789+0000 7f29a7772640 1 -- 172.21.15.78:0/3007437681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a0107c10 con 0x7f29a0100840 2024-08-18T23:43:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.790+0000 7f29927fc640 1 -- 172.21.15.78:0/3007437681 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f299c042030 con 0x7f29a0100840 2024-08-18T23:43:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.791+0000 7f29927fc640 1 --2- 172.21.15.78:0/3007437681 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f298c0a2b80 0x7f298c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.791+0000 7f29927fc640 1 -- 172.21.15.78:0/3007437681 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f299c0efa50 con 0x7f29a0100840 2024-08-18T23:43:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.791+0000 7f29a4d0d640 1 --2- 172.21.15.78:0/3007437681 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f298c0a2b80 0x7f298c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.791+0000 7f29a4d0d640 1 --2- 172.21.15.78:0/3007437681 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f298c0a2b80 0x7f298c0a5040 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2994005e10 tx=0x7f2994005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.795+0000 7f29927fc640 1 -- 172.21.15.78:0/3007437681 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f299c0bc620 con 0x7f29a0100840 2024-08-18T23:43:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:32.978+0000 7f29a7772640 1 -- 172.21.15.78:0/3007437681 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f29a0062360 con 0x7f298c0a2b80 2024-08-18T23:43:33.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:33.012+0000 7f29927fc640 1 -- 172.21.15.78:0/3007437681 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f29a0062360 con 0x7f298c0a2b80 2024-08-18T23:43:33.013 INFO:teuthology.orchestra.run.smithi078.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-08-18T23:43:33.013 INFO:teuthology.orchestra.run.smithi078.stdout:mds.a smithi078 running (2m) 60s ago 2m 15.7M - 19.1.0-1382-gde09dd01 ea7a5189f30b 89486459fb85 2024-08-18T23:43:33.013 INFO:teuthology.orchestra.run.smithi078.stdout:mds.b smithi123 running (2m) 62s ago 2m 28.9M - 19.1.0-1382-gde09dd01 ea7a5189f30b b7423c35eb0b 2024-08-18T23:43:33.013 INFO:teuthology.orchestra.run.smithi078.stdout:mds.c smithi161 running (2m) 62s ago 2m 18.4M - 19.1.0-1382-gde09dd01 ea7a5189f30b 89609bf43c69 2024-08-18T23:43:33.013 INFO:teuthology.orchestra.run.smithi078.stdout:mds.d smithi078 running (2m) 60s ago 2m 15.9M - 19.1.0-1382-gde09dd01 ea7a5189f30b d984b200609e 2024-08-18T23:43:33.013 INFO:teuthology.orchestra.run.smithi078.stdout:mds.e smithi123 running (2m) 62s ago 2m 16.1M - 19.1.0-1382-gde09dd01 ea7a5189f30b a311a99fbf62 2024-08-18T23:43:33.013 INFO:teuthology.orchestra.run.smithi078.stdout:mds.f smithi161 running (2m) 62s ago 2m 15.9M - 19.1.0-1382-gde09dd01 ea7a5189f30b 4c0696181a07 2024-08-18T23:43:33.013 INFO:teuthology.orchestra.run.smithi078.stdout:mds.g smithi078 running (2m) 60s ago 2m 15.7M - 19.1.0-1382-gde09dd01 ea7a5189f30b 241c7df2c510 2024-08-18T23:43:33.014 INFO:teuthology.orchestra.run.smithi078.stdout:mds.h smithi123 running (2m) 62s ago 2m 19.5M - 19.1.0-1382-gde09dd01 ea7a5189f30b 42145008a680 2024-08-18T23:43:33.014 INFO:teuthology.orchestra.run.smithi078.stdout:mds.i smithi161 running (2m) 62s ago 2m 18.9M - 19.1.0-1382-gde09dd01 ea7a5189f30b f0e34b4ae6b6 2024-08-18T23:43:33.014 INFO:teuthology.orchestra.run.smithi078.stdout:mds.j smithi078 running (2m) 60s ago 2m 16.7M - 19.1.0-1382-gde09dd01 ea7a5189f30b 97eba7693b06 2024-08-18T23:43:33.014 INFO:teuthology.orchestra.run.smithi078.stdout:mds.k smithi123 running (2m) 62s ago 2m 14.5M - 19.1.0-1382-gde09dd01 ea7a5189f30b 8268bfa21866 2024-08-18T23:43:33.014 INFO:teuthology.orchestra.run.smithi078.stdout:mds.l smithi161 running (2m) 62s ago 2m 14.5M - 19.1.0-1382-gde09dd01 ea7a5189f30b ef763974721e 2024-08-18T23:43:33.014 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.x smithi078 *:9283,8765 running (8m) 60s ago 8m 511M - 19.1.0-1382-gde09dd01 ea7a5189f30b 77b8af6f67c4 2024-08-18T23:43:33.014 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.y smithi123 *:8443,8765 running (7m) 62s ago 7m 441M - 19.1.0-1382-gde09dd01 ea7a5189f30b 4ec69d36f5b0 2024-08-18T23:43:33.014 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.z smithi161 *:8443,8765 running (7m) 62s ago 7m 441M - 19.1.0-1382-gde09dd01 ea7a5189f30b faf5c890d840 2024-08-18T23:43:33.014 INFO:teuthology.orchestra.run.smithi078.stdout:mon.a smithi078 running (8m) 60s ago 8m 50.8M 2048M 19.1.0-1382-gde09dd01 ea7a5189f30b c87427028777 2024-08-18T23:43:33.014 INFO:teuthology.orchestra.run.smithi078.stdout:mon.b smithi123 running (7m) 62s ago 7m 40.5M 2048M 19.1.0-1382-gde09dd01 ea7a5189f30b 12b66fb5d4d3 2024-08-18T23:43:33.014 INFO:teuthology.orchestra.run.smithi078.stdout:mon.c smithi161 running (7m) 62s ago 7m 42.1M 2048M 19.1.0-1382-gde09dd01 ea7a5189f30b bbc41949c792 2024-08-18T23:43:33.014 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 smithi078 running (7m) 60s ago 7m 45.5M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 5590187c8380 2024-08-18T23:43:33.014 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 smithi123 running (6m) 62s ago 6m 53.8M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b c0d424690978 2024-08-18T23:43:33.015 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 smithi161 running (6m) 62s ago 6m 48.0M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 917133d4f3fa 2024-08-18T23:43:33.015 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 smithi078 running (5m) 60s ago 5m 53.4M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 14e82150a24a 2024-08-18T23:43:33.015 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 smithi123 running (5m) 62s ago 5m 51.3M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b ebba362e042d 2024-08-18T23:43:33.015 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 smithi161 running (5m) 62s ago 5m 44.9M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 2f3699ab96e7 2024-08-18T23:43:33.015 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 smithi078 running (4m) 60s ago 4m 50.4M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b ab330635ae8f 2024-08-18T23:43:33.015 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 smithi123 running (4m) 62s ago 4m 51.7M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 83190ee7d0f4 2024-08-18T23:43:33.015 INFO:teuthology.orchestra.run.smithi078.stdout:osd.8 smithi161 running (4m) 62s ago 4m 45.8M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b e380cf659ff7 2024-08-18T23:43:33.015 INFO:teuthology.orchestra.run.smithi078.stdout:osd.9 smithi078 running (3m) 60s ago 3m 50.4M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b f0b9042c79aa 2024-08-18T23:43:33.015 INFO:teuthology.orchestra.run.smithi078.stdout:osd.10 smithi123 running (3m) 62s ago 3m 43.2M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b d9ff70ce75a7 2024-08-18T23:43:33.015 INFO:teuthology.orchestra.run.smithi078.stdout:osd.11 smithi161 running (2m) 62s ago 2m 50.0M 4272M 19.1.0-1382-gde09dd01 ea7a5189f30b 931c3efe781b 2024-08-18T23:43:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:33.014+0000 7f29a7772640 1 -- 172.21.15.78:0/3007437681 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f298c0a2b80 msgr2=0x7f298c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:33.014+0000 7f29a7772640 1 --2- 172.21.15.78:0/3007437681 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f298c0a2b80 0x7f298c0a5040 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2994005e10 tx=0x7f2994005da0 comp rx=0 tx=0).stop 2024-08-18T23:43:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:33.014+0000 7f29a7772640 1 -- 172.21.15.78:0/3007437681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a0100840 msgr2=0x7f29a006b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:33.014+0000 7f29a7772640 1 --2- 172.21.15.78:0/3007437681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a0100840 0x7f29a006b020 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f299c031610 tx=0x7f299c0043b0 comp rx=0 tx=0).stop 2024-08-18T23:43:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:33.014+0000 7f29a7772640 1 -- 172.21.15.78:0/3007437681 shutdown_connections 2024-08-18T23:43:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:33.014+0000 7f29a7772640 1 --2- 172.21.15.78:0/3007437681 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f298c0a2b80 0x7f298c0a5040 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:33.014+0000 7f29a7772640 1 --2- 172.21.15.78:0/3007437681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a0102400 0x7f29a019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:33.014+0000 7f29a7772640 1 --2- 172.21.15.78:0/3007437681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29a0101a40 0x7f29a019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:33.014+0000 7f29a7772640 1 --2- 172.21.15.78:0/3007437681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a0100840 0x7f29a006b020 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:33.014+0000 7f29a7772640 1 -- 172.21.15.78:0/3007437681 >> 172.21.15.78:0/3007437681 conn(0x7f29a00fbfd0 msgr2=0x7f29a0104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:33.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:33.014+0000 7f29a7772640 1 -- 172.21.15.78:0/3007437681 shutdown_connections 2024-08-18T23:43:33.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:33.014+0000 7f29a7772640 1 -- 172.21.15.78:0/3007437681 wait complete. 2024-08-18T23:43:33.388 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:33 smithi078 ceph-mon[18440]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:43:33.388 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:33 smithi078 ceph-mon[18440]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 810 B/s wr, 8 op/s 2024-08-18T23:43:33.388 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:33 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:43:33.451 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:33 smithi161 ceph-mon[22027]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:43:33.451 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:33 smithi161 ceph-mon[22027]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 810 B/s wr, 8 op/s 2024-08-18T23:43:33.451 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:33 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:43:33.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:33 smithi123 ceph-mon[21808]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:43:33.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:33 smithi123 ceph-mon[21808]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 810 B/s wr, 8 op/s 2024-08-18T23:43:33.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:33 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:43:33.859 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- bash -c 'ceph orch ls' 2024-08-18T23:43:34.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:34 smithi078 ceph-mon[18440]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:43:34.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:34 smithi123 ceph-mon[21808]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:43:34.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:34 smithi161 ceph-mon[22027]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:43:34.869 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:35.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:35 smithi078 ceph-mon[18440]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-08-18T23:43:35.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:35 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:35.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:35 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:35.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:35 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:35.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:35 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:35.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.325+0000 7f3e35f83640 1 -- 172.21.15.78:0/3771004902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e30104d60 msgr2=0x7f3e30107150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:35.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.325+0000 7f3e35f83640 1 --2- 172.21.15.78:0/3771004902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e30104d60 0x7f3e30107150 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009900 tx=0x7f3e2402a410 comp rx=0 tx=0).stop 2024-08-18T23:43:35.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.325+0000 7f3e35f83640 1 -- 172.21.15.78:0/3771004902 shutdown_connections 2024-08-18T23:43:35.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.325+0000 7f3e35f83640 1 --2- 172.21.15.78:0/3771004902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e30107690 0x7f3e30109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:35.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.325+0000 7f3e35f83640 1 --2- 172.21.15.78:0/3771004902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e30104d60 0x7f3e30107150 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:35.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.325+0000 7f3e35f83640 1 --2- 172.21.15.78:0/3771004902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e30102430 0x7f3e30104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:35.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.325+0000 7f3e35f83640 1 -- 172.21.15.78:0/3771004902 >> 172.21.15.78:0/3771004902 conn(0x7f3e300fbfb0 msgr2=0x7f3e300fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:35.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.325+0000 7f3e35f83640 1 -- 172.21.15.78:0/3771004902 shutdown_connections 2024-08-18T23:43:35.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.325+0000 7f3e35f83640 1 -- 172.21.15.78:0/3771004902 wait complete. 2024-08-18T23:43:35.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.326+0000 7f3e35f83640 1 Processor -- start 2024-08-18T23:43:35.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.326+0000 7f3e35f83640 1 -- start start 2024-08-18T23:43:35.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.326+0000 7f3e35f83640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e30102430 0x7f3e30101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:35.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.326+0000 7f3e35f83640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e30104d60 0x7f3e301003e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:35.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.326+0000 7f3e35f83640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e30107690 0x7f3e30100920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:35.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.326+0000 7f3e35f83640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3010d5e0 con 0x7f3e30107690 2024-08-18T23:43:35.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.326+0000 7f3e35f83640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3010d460 con 0x7f3e30104d60 2024-08-18T23:43:35.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.326+0000 7f3e35f83640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3010d760 con 0x7f3e30102430 2024-08-18T23:43:35.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.327+0000 7f3e2ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e30107690 0x7f3e30100920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:35.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.327+0000 7f3e2ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e30107690 0x7f3e30100920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59458/0 (socket says 172.21.15.78:59458) 2024-08-18T23:43:35.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.327+0000 7f3e2ffff640 1 -- 172.21.15.78:0/3619631898 learned_addr learned my addr 172.21.15.78:0/3619631898 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:35.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.327+0000 7f3e2effd640 1 --2- 172.21.15.78:0/3619631898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e30104d60 0x7f3e301003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:35.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.327+0000 7f3e2f7fe640 1 --2- 172.21.15.78:0/3619631898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e30102430 0x7f3e30101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:35.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.327+0000 7f3e2ffff640 1 -- 172.21.15.78:0/3619631898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e30102430 msgr2=0x7f3e30101d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:35.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.327+0000 7f3e2ffff640 1 --2- 172.21.15.78:0/3619631898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e30102430 0x7f3e30101d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:35.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.327+0000 7f3e2ffff640 1 -- 172.21.15.78:0/3619631898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e30104d60 msgr2=0x7f3e301003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:35.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.327+0000 7f3e2ffff640 1 --2- 172.21.15.78:0/3619631898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e30104d60 0x7f3e301003e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:35.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.327+0000 7f3e2ffff640 1 -- 172.21.15.78:0/3619631898 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e301011e0 con 0x7f3e30107690 2024-08-18T23:43:35.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.327+0000 7f3e2ffff640 1 --2- 172.21.15.78:0/3619631898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e30107690 0x7f3e30100920 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800b970 tx=0x7f3e2800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:35.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.327+0000 7f3e2cff9640 1 -- 172.21.15.78:0/3619631898 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28019070 con 0x7f3e30107690 2024-08-18T23:43:35.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.327+0000 7f3e35f83640 1 -- 172.21.15.78:0/3619631898 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e301a1e20 con 0x7f3e30107690 2024-08-18T23:43:35.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.327+0000 7f3e2cff9640 1 -- 172.21.15.78:0/3619631898 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e2800cb40 con 0x7f3e30107690 2024-08-18T23:43:35.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.327+0000 7f3e2cff9640 1 -- 172.21.15.78:0/3619631898 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28007650 con 0x7f3e30107690 2024-08-18T23:43:35.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.327+0000 7f3e35f83640 1 -- 172.21.15.78:0/3619631898 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e301a2350 con 0x7f3e30107690 2024-08-18T23:43:35.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.329+0000 7f3e35f83640 1 -- 172.21.15.78:0/3619631898 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e300fdaa0 con 0x7f3e30107690 2024-08-18T23:43:35.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.330+0000 7f3e2cff9640 1 -- 172.21.15.78:0/3619631898 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3e2801c070 con 0x7f3e30107690 2024-08-18T23:43:35.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.330+0000 7f3e2cff9640 1 --2- 172.21.15.78:0/3619631898 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3e180a2b80 0x7f3e180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:35.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.330+0000 7f3e2f7fe640 1 --2- 172.21.15.78:0/3619631898 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3e180a2b80 0x7f3e180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:35.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.331+0000 7f3e2cff9640 1 -- 172.21.15.78:0/3619631898 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f3e280ce960 con 0x7f3e30107690 2024-08-18T23:43:35.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.331+0000 7f3e2f7fe640 1 --2- 172.21.15.78:0/3619631898 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3e180a2b80 0x7f3e180a5040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3e200058c0 tx=0x7f3e2000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:35.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.335+0000 7f3e2cff9640 1 -- 172.21.15.78:0/3619631898 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3e2809aee0 con 0x7f3e30107690 2024-08-18T23:43:35.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.527+0000 7f3e35f83640 1 -- 172.21.15.78:0/3619631898 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f3e30062360 con 0x7f3e180a2b80 2024-08-18T23:43:35.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.533+0000 7f3e2cff9640 1 -- 172.21.15.78:0/3619631898 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f3e30062360 con 0x7f3e180a2b80 2024-08-18T23:43:35.534 INFO:teuthology.orchestra.run.smithi078.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-08-18T23:43:35.534 INFO:teuthology.orchestra.run.smithi078.stdout:mds.all 12/12 63s ago 2m smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-08-18T23:43:35.534 INFO:teuthology.orchestra.run.smithi078.stdout:mgr 3/3 63s ago 7m smithi078=x;smithi123=y;smithi161=z;count:3 2024-08-18T23:43:35.534 INFO:teuthology.orchestra.run.smithi078.stdout:mon 3/3 63s ago 7m smithi078:172.21.15.78=a;smithi123:172.21.15.123=b;smithi161:172.21.15.161=c;count:3 2024-08-18T23:43:35.534 INFO:teuthology.orchestra.run.smithi078.stdout:osd 12 63s ago - 2024-08-18T23:43:35.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.535+0000 7f3e35f83640 1 -- 172.21.15.78:0/3619631898 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3e180a2b80 msgr2=0x7f3e180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:35.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.535+0000 7f3e35f83640 1 --2- 172.21.15.78:0/3619631898 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3e180a2b80 0x7f3e180a5040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3e200058c0 tx=0x7f3e2000a2d0 comp rx=0 tx=0).stop 2024-08-18T23:43:35.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.535+0000 7f3e35f83640 1 -- 172.21.15.78:0/3619631898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e30107690 msgr2=0x7f3e30100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:35.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.535+0000 7f3e35f83640 1 --2- 172.21.15.78:0/3619631898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e30107690 0x7f3e30100920 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800b970 tx=0x7f3e2800be30 comp rx=0 tx=0).stop 2024-08-18T23:43:35.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.535+0000 7f3e35f83640 1 -- 172.21.15.78:0/3619631898 shutdown_connections 2024-08-18T23:43:35.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.535+0000 7f3e35f83640 1 --2- 172.21.15.78:0/3619631898 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3e180a2b80 0x7f3e180a5040 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:35.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.535+0000 7f3e35f83640 1 --2- 172.21.15.78:0/3619631898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e30107690 0x7f3e30100920 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:35.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.535+0000 7f3e35f83640 1 --2- 172.21.15.78:0/3619631898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e30104d60 0x7f3e301003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:35.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.535+0000 7f3e35f83640 1 --2- 172.21.15.78:0/3619631898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e30102430 0x7f3e30101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:35.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.535+0000 7f3e35f83640 1 -- 172.21.15.78:0/3619631898 >> 172.21.15.78:0/3619631898 conn(0x7f3e300fbfb0 msgr2=0x7f3e30102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:35.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.535+0000 7f3e35f83640 1 -- 172.21.15.78:0/3619631898 shutdown_connections 2024-08-18T23:43:35.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:35.535+0000 7f3e35f83640 1 -- 172.21.15.78:0/3619631898 wait complete. 2024-08-18T23:43:35.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:35 smithi123 ceph-mon[21808]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-08-18T23:43:35.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:35 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:35.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:35 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:35.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:35 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:35.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:35 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:35.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:35 smithi161 ceph-mon[22027]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-08-18T23:43:35.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:35 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:35.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:35 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:35.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:35 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:35.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:35 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:36.330 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- bash -c 'ceph orch host ls' 2024-08-18T23:43:36.594 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:37.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.090+0000 7f4dda4b5640 1 -- 172.21.15.78:0/287381263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd41023d0 msgr2=0x7f4dd41047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:37.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.090+0000 7f4dda4b5640 1 --2- 172.21.15.78:0/287381263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd41023d0 0x7f4dd41047c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009900 tx=0x7f4dc402a410 comp rx=0 tx=0).stop 2024-08-18T23:43:37.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.090+0000 7f4dda4b5640 1 -- 172.21.15.78:0/287381263 shutdown_connections 2024-08-18T23:43:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.090+0000 7f4dda4b5640 1 --2- 172.21.15.78:0/287381263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4dd4107630 0x7f4dd4109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.090+0000 7f4dda4b5640 1 --2- 172.21.15.78:0/287381263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dd4104d00 0x7f4dd41070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.090+0000 7f4dda4b5640 1 --2- 172.21.15.78:0/287381263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd41023d0 0x7f4dd41047c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.090+0000 7f4dda4b5640 1 -- 172.21.15.78:0/287381263 >> 172.21.15.78:0/287381263 conn(0x7f4dd40fbfd0 msgr2=0x7f4dd40fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.090+0000 7f4dda4b5640 1 -- 172.21.15.78:0/287381263 shutdown_connections 2024-08-18T23:43:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.090+0000 7f4dda4b5640 1 -- 172.21.15.78:0/287381263 wait complete. 2024-08-18T23:43:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dda4b5640 1 Processor -- start 2024-08-18T23:43:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dda4b5640 1 -- start start 2024-08-18T23:43:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dda4b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd41023d0 0x7f4dd4197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dda4b5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dd4104d00 0x7f4dd4198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dda4b5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4dd4107630 0x7f4dd419f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dda4b5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd410d650 con 0x7f4dd41023d0 2024-08-18T23:43:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dda4b5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd410d4d0 con 0x7f4dd4104d00 2024-08-18T23:43:37.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dda4b5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd410d7d0 con 0x7f4dd4107630 2024-08-18T23:43:37.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dd3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd41023d0 0x7f4dd4197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:37.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dd3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd41023d0 0x7f4dd4197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34356/0 (socket says 172.21.15.78:34356) 2024-08-18T23:43:37.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dd3fff640 1 -- 172.21.15.78:0/525827848 learned_addr learned my addr 172.21.15.78:0/525827848 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:37.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dd37fe640 1 --2- 172.21.15.78:0/525827848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dd4104d00 0x7f4dd4198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:37.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dd8a52640 1 --2- 172.21.15.78:0/525827848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4dd4107630 0x7f4dd419f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:37.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dd3fff640 1 -- 172.21.15.78:0/525827848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4dd4107630 msgr2=0x7f4dd419f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:37.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dd3fff640 1 --2- 172.21.15.78:0/525827848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4dd4107630 0x7f4dd419f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:37.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dd3fff640 1 -- 172.21.15.78:0/525827848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dd4104d00 msgr2=0x7f4dd4198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:37.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dd3fff640 1 --2- 172.21.15.78:0/525827848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dd4104d00 0x7f4dd4198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:37.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dd3fff640 1 -- 172.21.15.78:0/525827848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd419f920 con 0x7f4dd41023d0 2024-08-18T23:43:37.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dd37fe640 1 --2- 172.21.15.78:0/525827848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dd4104d00 0x7f4dd4198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:43:37.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.091+0000 7f4dd8a52640 1 --2- 172.21.15.78:0/525827848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4dd4107630 0x7f4dd419f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:43:37.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.092+0000 7f4dd3fff640 1 --2- 172.21.15.78:0/525827848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd41023d0 0x7f4dd4197d00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009230 tx=0x7f4dc4004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:37.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.092+0000 7f4dd17fa640 1 -- 172.21.15.78:0/525827848 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc403f030 con 0x7f4dd41023d0 2024-08-18T23:43:37.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.092+0000 7f4dda4b5640 1 -- 172.21.15.78:0/525827848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd419fbb0 con 0x7f4dd41023d0 2024-08-18T23:43:37.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.092+0000 7f4dd17fa640 1 -- 172.21.15.78:0/525827848 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dc4003ce0 con 0x7f4dd41023d0 2024-08-18T23:43:37.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.092+0000 7f4dd17fa640 1 -- 172.21.15.78:0/525827848 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4035070 con 0x7f4dd41023d0 2024-08-18T23:43:37.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.092+0000 7f4dda4b5640 1 -- 172.21.15.78:0/525827848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd41a00e0 con 0x7f4dd41023d0 2024-08-18T23:43:37.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.093+0000 7f4dda4b5640 1 -- 172.21.15.78:0/525827848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dd40fdac0 con 0x7f4dd41023d0 2024-08-18T23:43:37.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.094+0000 7f4dd17fa640 1 -- 172.21.15.78:0/525827848 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4dc4042030 con 0x7f4dd41023d0 2024-08-18T23:43:37.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.095+0000 7f4dd17fa640 1 --2- 172.21.15.78:0/525827848 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4dbc0a2b80 0x7f4dbc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:37.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.095+0000 7f4dd37fe640 1 --2- 172.21.15.78:0/525827848 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4dbc0a2b80 0x7f4dbc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:37.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.095+0000 7f4dd17fa640 1 -- 172.21.15.78:0/525827848 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f4dc40efed0 con 0x7f4dd41023d0 2024-08-18T23:43:37.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.096+0000 7f4dd37fe640 1 --2- 172.21.15.78:0/525827848 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4dbc0a2b80 0x7f4dbc0a5040 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80058c0 tx=0x7f4dc8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:37.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.098+0000 7f4dd17fa640 1 -- 172.21.15.78:0/525827848 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4dc40bcbc0 con 0x7f4dd41023d0 2024-08-18T23:43:37.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.276+0000 7f4dda4b5640 1 -- 172.21.15.78:0/525827848 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f4dd4062360 con 0x7f4dbc0a2b80 2024-08-18T23:43:37.277 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:37 smithi078 ceph-mon[18440]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:43:37.278 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:37 smithi078 ceph-mon[18440]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 518 B/s wr, 7 op/s 2024-08-18T23:43:37.278 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:37 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:37.278 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:37 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:37.278 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:37 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:43:37.278 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:37 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:43:37.278 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:37 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:37.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.277+0000 7f4dd17fa640 1 -- 172.21.15.78:0/525827848 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f4dd4062360 con 0x7f4dbc0a2b80 2024-08-18T23:43:37.279 INFO:teuthology.orchestra.run.smithi078.stdout:HOST ADDR LABELS STATUS 2024-08-18T23:43:37.279 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 172.21.15.78 2024-08-18T23:43:37.279 INFO:teuthology.orchestra.run.smithi078.stdout:smithi123 172.21.15.123 2024-08-18T23:43:37.279 INFO:teuthology.orchestra.run.smithi078.stdout:smithi161 172.21.15.161 2024-08-18T23:43:37.279 INFO:teuthology.orchestra.run.smithi078.stdout:3 hosts in cluster 2024-08-18T23:43:37.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.279+0000 7f4dda4b5640 1 -- 172.21.15.78:0/525827848 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4dbc0a2b80 msgr2=0x7f4dbc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:37.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.279+0000 7f4dda4b5640 1 --2- 172.21.15.78:0/525827848 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4dbc0a2b80 0x7f4dbc0a5040 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80058c0 tx=0x7f4dc8005850 comp rx=0 tx=0).stop 2024-08-18T23:43:37.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.279+0000 7f4dda4b5640 1 -- 172.21.15.78:0/525827848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd41023d0 msgr2=0x7f4dd4197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:37.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.279+0000 7f4dda4b5640 1 --2- 172.21.15.78:0/525827848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd41023d0 0x7f4dd4197d00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009230 tx=0x7f4dc4004360 comp rx=0 tx=0).stop 2024-08-18T23:43:37.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.280+0000 7f4dda4b5640 1 -- 172.21.15.78:0/525827848 shutdown_connections 2024-08-18T23:43:37.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.280+0000 7f4dda4b5640 1 --2- 172.21.15.78:0/525827848 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4dbc0a2b80 0x7f4dbc0a5040 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:37.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.280+0000 7f4dda4b5640 1 --2- 172.21.15.78:0/525827848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4dd4107630 0x7f4dd419f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:37.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.280+0000 7f4dda4b5640 1 --2- 172.21.15.78:0/525827848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dd4104d00 0x7f4dd4198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:37.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.280+0000 7f4dda4b5640 1 --2- 172.21.15.78:0/525827848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dd41023d0 0x7f4dd4197d00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:37.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.280+0000 7f4dda4b5640 1 -- 172.21.15.78:0/525827848 >> 172.21.15.78:0/525827848 conn(0x7f4dd40fbfd0 msgr2=0x7f4dd4102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:37.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.280+0000 7f4dda4b5640 1 -- 172.21.15.78:0/525827848 shutdown_connections 2024-08-18T23:43:37.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:37.280+0000 7f4dda4b5640 1 -- 172.21.15.78:0/525827848 wait complete. 2024-08-18T23:43:37.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:37 smithi123 ceph-mon[21808]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:43:37.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:37 smithi123 ceph-mon[21808]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 518 B/s wr, 7 op/s 2024-08-18T23:43:37.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:37 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:37.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:37 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:37.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:37 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:43:37.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:37 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:43:37.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:37 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:37 smithi161 ceph-mon[22027]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:43:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:37 smithi161 ceph-mon[22027]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 518 B/s wr, 7 op/s 2024-08-18T23:43:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:37 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:37 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:37 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:43:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:37 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:43:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:37 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:43:38.208 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- bash -c 'ceph orch device ls' 2024-08-18T23:43:38.462 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:38 smithi078 ceph-mon[18440]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:43:38.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:38 smithi123 ceph-mon[21808]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:43:38.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:38 smithi161 ceph-mon[22027]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:43:38.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.943+0000 7fe5e6117640 1 -- 172.21.15.78:0/1046118462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00fffc0 msgr2=0x7fe5e01003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:38.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.943+0000 7fe5e6117640 1 --2- 172.21.15.78:0/1046118462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00fffc0 0x7fe5e01003c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009900 tx=0x7fe5d002a410 comp rx=0 tx=0).stop 2024-08-18T23:43:38.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.943+0000 7fe5e6117640 1 -- 172.21.15.78:0/1046118462 shutdown_connections 2024-08-18T23:43:38.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.943+0000 7fe5e6117640 1 --2- 172.21.15.78:0/1046118462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5e0101b80 0x7fe5e0108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:38.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.943+0000 7fe5e6117640 1 --2- 172.21.15.78:0/1046118462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e01011c0 0x7fe5e0101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:38.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.943+0000 7fe5e6117640 1 --2- 172.21.15.78:0/1046118462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00fffc0 0x7fe5e01003c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:38.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.943+0000 7fe5e6117640 1 -- 172.21.15.78:0/1046118462 >> 172.21.15.78:0/1046118462 conn(0x7fe5e00fb770 msgr2=0x7fe5e00fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:38.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.943+0000 7fe5e6117640 1 -- 172.21.15.78:0/1046118462 shutdown_connections 2024-08-18T23:43:38.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.943+0000 7fe5e6117640 1 -- 172.21.15.78:0/1046118462 wait complete. 2024-08-18T23:43:38.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.944+0000 7fe5e6117640 1 Processor -- start 2024-08-18T23:43:38.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.944+0000 7fe5e6117640 1 -- start start 2024-08-18T23:43:38.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.945+0000 7fe5e6117640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00fffc0 0x7fe5e0197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:38.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.945+0000 7fe5e6117640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5e01011c0 0x7fe5e01981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:38.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.945+0000 7fe5e6117640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e0101b80 0x7fe5e019f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:38.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.945+0000 7fe5e6117640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e010bc60 con 0x7fe5e00fffc0 2024-08-18T23:43:38.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.945+0000 7fe5e6117640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e010bae0 con 0x7fe5e0101b80 2024-08-18T23:43:38.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.945+0000 7fe5e6117640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e010bde0 con 0x7fe5e01011c0 2024-08-18T23:43:38.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.945+0000 7fe5df7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00fffc0 0x7fe5e0197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:38.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.945+0000 7fe5df7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00fffc0 0x7fe5e0197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34374/0 (socket says 172.21.15.78:34374) 2024-08-18T23:43:38.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.945+0000 7fe5df7fe640 1 -- 172.21.15.78:0/2720575436 learned_addr learned my addr 172.21.15.78:0/2720575436 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:38.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.945+0000 7fe5deffd640 1 --2- 172.21.15.78:0/2720575436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5e01011c0 0x7fe5e01981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:38.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.945+0000 7fe5dffff640 1 --2- 172.21.15.78:0/2720575436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e0101b80 0x7fe5e019f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:38.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.945+0000 7fe5df7fe640 1 -- 172.21.15.78:0/2720575436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5e01011c0 msgr2=0x7fe5e01981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:38.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.945+0000 7fe5df7fe640 1 --2- 172.21.15.78:0/2720575436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5e01011c0 0x7fe5e01981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:38.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.945+0000 7fe5df7fe640 1 -- 172.21.15.78:0/2720575436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e0101b80 msgr2=0x7fe5e019f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:38.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.945+0000 7fe5df7fe640 1 --2- 172.21.15.78:0/2720575436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e0101b80 0x7fe5e019f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:38.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.945+0000 7fe5df7fe640 1 -- 172.21.15.78:0/2720575436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e019f880 con 0x7fe5e00fffc0 2024-08-18T23:43:38.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.945+0000 7fe5dffff640 1 --2- 172.21.15.78:0/2720575436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e0101b80 0x7fe5e019f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:43:38.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.945+0000 7fe5df7fe640 1 --2- 172.21.15.78:0/2720575436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00fffc0 0x7fe5e0197c60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0005ab0 tx=0x7fe5d0004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:38.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.945+0000 7fe5dcff9640 1 -- 172.21.15.78:0/2720575436 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d003f030 con 0x7fe5e00fffc0 2024-08-18T23:43:38.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.946+0000 7fe5e6117640 1 -- 172.21.15.78:0/2720575436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e019fb10 con 0x7fe5e00fffc0 2024-08-18T23:43:38.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.946+0000 7fe5dcff9640 1 -- 172.21.15.78:0/2720575436 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d0003ce0 con 0x7fe5e00fffc0 2024-08-18T23:43:38.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.946+0000 7fe5e6117640 1 -- 172.21.15.78:0/2720575436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e01a0040 con 0x7fe5e00fffc0 2024-08-18T23:43:38.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.946+0000 7fe5dcff9640 1 -- 172.21.15.78:0/2720575436 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0049060 con 0x7fe5e00fffc0 2024-08-18T23:43:38.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.946+0000 7fe5e6117640 1 -- 172.21.15.78:0/2720575436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e01a0270 con 0x7fe5e00fffc0 2024-08-18T23:43:38.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.948+0000 7fe5dcff9640 1 -- 172.21.15.78:0/2720575436 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe5d0042030 con 0x7fe5e00fffc0 2024-08-18T23:43:38.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.948+0000 7fe5dcff9640 1 --2- 172.21.15.78:0/2720575436 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe5c80a2b80 0x7fe5c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:38.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.948+0000 7fe5deffd640 1 --2- 172.21.15.78:0/2720575436 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe5c80a2b80 0x7fe5c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:38.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.948+0000 7fe5dcff9640 1 -- 172.21.15.78:0/2720575436 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fe5d00efd50 con 0x7fe5e00fffc0 2024-08-18T23:43:38.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.949+0000 7fe5deffd640 1 --2- 172.21.15.78:0/2720575436 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe5c80a2b80 0x7fe5c80a5040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400abe0 tx=0x7fe5d400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:38.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:38.952+0000 7fe5dcff9640 1 -- 172.21.15.78:0/2720575436 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe5d00f3030 con 0x7fe5e00fffc0 2024-08-18T23:43:39.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:39.126+0000 7fe5e6117640 1 -- 172.21.15.78:0/2720575436 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe5e0107610 con 0x7fe5c80a2b80 2024-08-18T23:43:39.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:39.128+0000 7fe5dcff9640 1 -- 172.21.15.78:0/2720575436 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe5e0107610 con 0x7fe5c80a2b80 2024-08-18T23:43:39.129 INFO:teuthology.orchestra.run.smithi078.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-08-18T23:43:39.129 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QR400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-08-18T23:43:39.129 INFO:teuthology.orchestra.run.smithi078.stdout:smithi123 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400YK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-08-18T23:43:39.130 INFO:teuthology.orchestra.run.smithi078.stdout:smithi161 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001X400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-08-18T23:43:39.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:39.130+0000 7fe5e6117640 1 -- 172.21.15.78:0/2720575436 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe5c80a2b80 msgr2=0x7fe5c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:39.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:39.130+0000 7fe5e6117640 1 --2- 172.21.15.78:0/2720575436 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe5c80a2b80 0x7fe5c80a5040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400abe0 tx=0x7fe5d400a2d0 comp rx=0 tx=0).stop 2024-08-18T23:43:39.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:39.130+0000 7fe5e6117640 1 -- 172.21.15.78:0/2720575436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00fffc0 msgr2=0x7fe5e0197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:39.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:39.130+0000 7fe5e6117640 1 --2- 172.21.15.78:0/2720575436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00fffc0 0x7fe5e0197c60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0005ab0 tx=0x7fe5d0004360 comp rx=0 tx=0).stop 2024-08-18T23:43:39.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:39.130+0000 7fe5e6117640 1 -- 172.21.15.78:0/2720575436 shutdown_connections 2024-08-18T23:43:39.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:39.130+0000 7fe5e6117640 1 --2- 172.21.15.78:0/2720575436 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe5c80a2b80 0x7fe5c80a5040 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:39.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:39.130+0000 7fe5e6117640 1 --2- 172.21.15.78:0/2720575436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e0101b80 0x7fe5e019f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:39.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:39.130+0000 7fe5e6117640 1 --2- 172.21.15.78:0/2720575436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5e01011c0 0x7fe5e01981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:39.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:39.130+0000 7fe5e6117640 1 --2- 172.21.15.78:0/2720575436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00fffc0 0x7fe5e0197c60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:39.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:39.130+0000 7fe5e6117640 1 -- 172.21.15.78:0/2720575436 >> 172.21.15.78:0/2720575436 conn(0x7fe5e00fb770 msgr2=0x7fe5e00fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:39.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:39.131+0000 7fe5e6117640 1 -- 172.21.15.78:0/2720575436 shutdown_connections 2024-08-18T23:43:39.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:39.131+0000 7fe5e6117640 1 -- 172.21.15.78:0/2720575436 wait complete. 2024-08-18T23:43:39.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:39 smithi078 ceph-mon[18440]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-18T23:43:39.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:39 smithi123 ceph-mon[21808]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-18T23:43:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:39 smithi161 ceph-mon[22027]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-18T23:43:39.977 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-18T23:43:39.985 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-08-18T23:43:39.985 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- bash -c 'ceph fs dump' 2024-08-18T23:43:40.240 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:40.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:40 smithi078 ceph-mon[18440]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:43:40.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:40 smithi123 ceph-mon[21808]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:43:40.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:40 smithi161 ceph-mon[22027]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:43:40.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.709+0000 7fa102070640 1 -- 172.21.15.78:0/4232574498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0fc0ffc00 msgr2=0x7fa0fc100000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.709+0000 7fa102070640 1 --2- 172.21.15.78:0/4232574498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0fc0ffc00 0x7fa0fc100000 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009900 tx=0x7fa0ec02a410 comp rx=0 tx=0).stop 2024-08-18T23:43:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.709+0000 7fa102070640 1 -- 172.21.15.78:0/4232574498 shutdown_connections 2024-08-18T23:43:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.709+0000 7fa102070640 1 --2- 172.21.15.78:0/4232574498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0fc101330 0x7fa0fc105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.709+0000 7fa102070640 1 --2- 172.21.15.78:0/4232574498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0fc100970 0x7fa0fc100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.709+0000 7fa102070640 1 --2- 172.21.15.78:0/4232574498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0fc0ffc00 0x7fa0fc100000 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.709+0000 7fa102070640 1 -- 172.21.15.78:0/4232574498 >> 172.21.15.78:0/4232574498 conn(0x7fa0fc0fb770 msgr2=0x7fa0fc0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.709+0000 7fa102070640 1 -- 172.21.15.78:0/4232574498 shutdown_connections 2024-08-18T23:43:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.709+0000 7fa102070640 1 -- 172.21.15.78:0/4232574498 wait complete. 2024-08-18T23:43:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.710+0000 7fa102070640 1 Processor -- start 2024-08-18T23:43:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.710+0000 7fa102070640 1 -- start start 2024-08-18T23:43:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.710+0000 7fa102070640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0fc0ffc00 0x7fa0fc19e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.710+0000 7fa102070640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0fc100970 0x7fa0fc19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.710+0000 7fa102070640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0fc101330 0x7fa0fc1a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.710+0000 7fa102070640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc113c40 con 0x7fa0fc100970 2024-08-18T23:43:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.710+0000 7fa102070640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc113ac0 con 0x7fa0fc101330 2024-08-18T23:43:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.710+0000 7fa102070640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc113dc0 con 0x7fa0fc0ffc00 2024-08-18T23:43:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.710+0000 7fa0faffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0fc100970 0x7fa0fc19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.711+0000 7fa0faffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0fc100970 0x7fa0fc19e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34386/0 (socket says 172.21.15.78:34386) 2024-08-18T23:43:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.711+0000 7fa0faffd640 1 -- 172.21.15.78:0/2070415785 learned_addr learned my addr 172.21.15.78:0/2070415785 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.711+0000 7fa0fbfff640 1 --2- 172.21.15.78:0/2070415785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0fc101330 0x7fa0fc1a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.711+0000 7fa0fb7fe640 1 --2- 172.21.15.78:0/2070415785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0fc0ffc00 0x7fa0fc19e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.711+0000 7fa0faffd640 1 -- 172.21.15.78:0/2070415785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0fc0ffc00 msgr2=0x7fa0fc19e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.711+0000 7fa0faffd640 1 --2- 172.21.15.78:0/2070415785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0fc0ffc00 0x7fa0fc19e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:40.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.711+0000 7fa0faffd640 1 -- 172.21.15.78:0/2070415785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0fc101330 msgr2=0x7fa0fc1a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:40.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.711+0000 7fa0faffd640 1 --2- 172.21.15.78:0/2070415785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0fc101330 0x7fa0fc1a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:40.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.711+0000 7fa0faffd640 1 -- 172.21.15.78:0/2070415785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc1a5f80 con 0x7fa0fc100970 2024-08-18T23:43:40.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.711+0000 7fa0faffd640 1 --2- 172.21.15.78:0/2070415785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0fc100970 0x7fa0fc19e890 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400b450 tx=0x7fa0e400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:40.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.711+0000 7fa0f8ff9640 1 -- 172.21.15.78:0/2070415785 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4015020 con 0x7fa0fc100970 2024-08-18T23:43:40.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.711+0000 7fa0f8ff9640 1 -- 172.21.15.78:0/2070415785 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e4011d70 con 0x7fa0fc100970 2024-08-18T23:43:40.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.711+0000 7fa102070640 1 -- 172.21.15.78:0/2070415785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0fc1a6270 con 0x7fa0fc100970 2024-08-18T23:43:40.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.711+0000 7fa0f8ff9640 1 -- 172.21.15.78:0/2070415785 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e400f340 con 0x7fa0fc100970 2024-08-18T23:43:40.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.711+0000 7fa102070640 1 -- 172.21.15.78:0/2070415785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0fc1a6830 con 0x7fa0fc100970 2024-08-18T23:43:40.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.712+0000 7fa102070640 1 -- 172.21.15.78:0/2070415785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0fc1a6af0 con 0x7fa0fc100970 2024-08-18T23:43:40.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.713+0000 7fa0f8ff9640 1 -- 172.21.15.78:0/2070415785 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa0e4018020 con 0x7fa0fc100970 2024-08-18T23:43:40.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.714+0000 7fa0f8ff9640 1 --2- 172.21.15.78:0/2070415785 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa0e80a2bb0 0x7fa0e80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:40.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.714+0000 7fa0fb7fe640 1 --2- 172.21.15.78:0/2070415785 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa0e80a2bb0 0x7fa0e80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:40.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.714+0000 7fa0f8ff9640 1 -- 172.21.15.78:0/2070415785 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa0e40cf360 con 0x7fa0fc100970 2024-08-18T23:43:40.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.714+0000 7fa0fb7fe640 1 --2- 172.21.15.78:0/2070415785 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa0e80a2bb0 0x7fa0e80a5070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec031610 tx=0x7fa0ec0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:40.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.717+0000 7fa0f8ff9640 1 -- 172.21.15.78:0/2070415785 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa0e401f020 con 0x7fa0fc100970 2024-08-18T23:43:40.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.916+0000 7fa102070640 1 -- 172.21.15.78:0/2070415785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fa0fc1a6cc0 con 0x7fa0fc100970 2024-08-18T23:43:40.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.917+0000 7fa0f8ff9640 1 -- 172.21.15.78:0/2070415785 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3100 (secure 0 0 0) 0x7fa0e400d040 con 0x7fa0fc100970 2024-08-18T23:43:40.918 INFO:teuthology.orchestra.run.smithi078.stdout:e24 2024-08-18T23:43:40.918 INFO:teuthology.orchestra.run.smithi078.stdout:btime 2024-08-18T23:42:10:259464+0000 2024-08-18T23:43:40.918 INFO:teuthology.orchestra.run.smithi078.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-08-18T23:43:40.918 INFO:teuthology.orchestra.run.smithi078.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-18T23:43:40.918 INFO:teuthology.orchestra.run.smithi078.stdout:legacy client fscid: 1 2024-08-18T23:43:40.918 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:43:40.919 INFO:teuthology.orchestra.run.smithi078.stdout:Filesystem 'cephfs' (1) 2024-08-18T23:43:40.919 INFO:teuthology.orchestra.run.smithi078.stdout:fs_name cephfs 2024-08-18T23:43:40.919 INFO:teuthology.orchestra.run.smithi078.stdout:epoch 24 2024-08-18T23:43:40.919 INFO:teuthology.orchestra.run.smithi078.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-08-18T23:43:40.919 INFO:teuthology.orchestra.run.smithi078.stdout:created 2024-08-18T23:41:32.941573+0000 2024-08-18T23:43:40.919 INFO:teuthology.orchestra.run.smithi078.stdout:modified 2024-08-18T23:42:09.974847+0000 2024-08-18T23:43:40.919 INFO:teuthology.orchestra.run.smithi078.stdout:tableserver 0 2024-08-18T23:43:40.919 INFO:teuthology.orchestra.run.smithi078.stdout:root 0 2024-08-18T23:43:40.919 INFO:teuthology.orchestra.run.smithi078.stdout:session_timeout 300 2024-08-18T23:43:40.919 INFO:teuthology.orchestra.run.smithi078.stdout:session_autoclose 300 2024-08-18T23:43:40.919 INFO:teuthology.orchestra.run.smithi078.stdout:max_file_size 1099511627776 2024-08-18T23:43:40.919 INFO:teuthology.orchestra.run.smithi078.stdout:max_xattr_size 65536 2024-08-18T23:43:40.920 INFO:teuthology.orchestra.run.smithi078.stdout:required_client_features {} 2024-08-18T23:43:40.920 INFO:teuthology.orchestra.run.smithi078.stdout:last_failure 0 2024-08-18T23:43:40.920 INFO:teuthology.orchestra.run.smithi078.stdout:last_failure_osd_epoch 0 2024-08-18T23:43:40.920 INFO:teuthology.orchestra.run.smithi078.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-18T23:43:40.920 INFO:teuthology.orchestra.run.smithi078.stdout:max_mds 3 2024-08-18T23:43:40.920 INFO:teuthology.orchestra.run.smithi078.stdout:in 0,1,2 2024-08-18T23:43:40.920 INFO:teuthology.orchestra.run.smithi078.stdout:up {0=24452,1=24428,2=24427} 2024-08-18T23:43:40.920 INFO:teuthology.orchestra.run.smithi078.stdout:failed 2024-08-18T23:43:40.920 INFO:teuthology.orchestra.run.smithi078.stdout:damaged 2024-08-18T23:43:40.920 INFO:teuthology.orchestra.run.smithi078.stdout:stopped 2024-08-18T23:43:40.920 INFO:teuthology.orchestra.run.smithi078.stdout:data_pools [3,4] 2024-08-18T23:43:40.920 INFO:teuthology.orchestra.run.smithi078.stdout:metadata_pool 2 2024-08-18T23:43:40.920 INFO:teuthology.orchestra.run.smithi078.stdout:inline_data disabled 2024-08-18T23:43:40.921 INFO:teuthology.orchestra.run.smithi078.stdout:balancer 2024-08-18T23:43:40.921 INFO:teuthology.orchestra.run.smithi078.stdout:bal_rank_mask -1 2024-08-18T23:43:40.921 INFO:teuthology.orchestra.run.smithi078.stdout:standby_count_wanted 0 2024-08-18T23:43:40.921 INFO:teuthology.orchestra.run.smithi078.stdout:qdb_cluster leader: 24452 members: 24427,24428,24452 2024-08-18T23:43:40.921 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b{0:24452} state up:active seq 4 addr [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] compat {c=[1],r=[1],i=[1fff]}] 2024-08-18T23:43:40.921 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.e{0:24407} state up:standby-replay seq 1 addr [v2:172.21.15.123:6834/2459199316,v1:172.21.15.123:6835/2459199316] compat {c=[1],r=[1],i=[1fff]}] 2024-08-18T23:43:40.921 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.h{1:24428} state up:active seq 15 addr [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] compat {c=[1],r=[1],i=[1fff]}] 2024-08-18T23:43:40.921 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c{1:24403} state up:standby-replay seq 1 addr [v2:172.21.15.161:6836/2017185021,v1:172.21.15.161:6837/2017185021] compat {c=[1],r=[1],i=[1fff]}] 2024-08-18T23:43:40.921 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.i{2:24427} state up:active seq 15 addr [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] compat {c=[1],r=[1],i=[1fff]}] 2024-08-18T23:43:40.921 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.f{2:24397} state up:standby-replay seq 1 addr [v2:172.21.15.161:6834/4100464644,v1:172.21.15.161:6835/4100464644] compat {c=[1],r=[1],i=[1fff]}] 2024-08-18T23:43:40.921 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:43:40.921 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:43:40.921 INFO:teuthology.orchestra.run.smithi078.stdout:Standby daemons: 2024-08-18T23:43:40.922 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:43:40.922 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.j{-1:14535} state up:standby seq 1 addr [v2:172.21.15.78:6834/3927701910,v1:172.21.15.78:6835/3927701910] compat {c=[1],r=[1],i=[1fff]}] 2024-08-18T23:43:40.922 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d{-1:14547} state up:standby seq 1 addr [v2:172.21.15.78:6836/1656541536,v1:172.21.15.78:6837/1656541536] compat {c=[1],r=[1],i=[1fff]}] 2024-08-18T23:43:40.922 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.g{-1:14595} state up:standby seq 1 addr [v2:172.21.15.78:6838/2648770058,v1:172.21.15.78:6839/2648770058] compat {c=[1],r=[1],i=[1fff]}] 2024-08-18T23:43:40.922 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a{-1:14607} state up:standby seq 1 addr [v2:172.21.15.78:6840/2010800750,v1:172.21.15.78:6841/2010800750] compat {c=[1],r=[1],i=[1fff]}] 2024-08-18T23:43:40.922 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.l{-1:24370} state up:standby seq 1 addr [v2:172.21.15.161:6832/3196205121,v1:172.21.15.161:6833/3196205121] compat {c=[1],r=[1],i=[1fff]}] 2024-08-18T23:43:40.922 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.k{-1:24386} state up:standby seq 1 addr [v2:172.21.15.123:6832/3595058071,v1:172.21.15.123:6833/3595058071] compat {c=[1],r=[1],i=[1fff]}] 2024-08-18T23:43:40.925 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 24 2024-08-18T23:43:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.919+0000 7fa102070640 1 -- 172.21.15.78:0/2070415785 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa0e80a2bb0 msgr2=0x7fa0e80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.919+0000 7fa102070640 1 --2- 172.21.15.78:0/2070415785 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa0e80a2bb0 0x7fa0e80a5070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec031610 tx=0x7fa0ec0314a0 comp rx=0 tx=0).stop 2024-08-18T23:43:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.919+0000 7fa102070640 1 -- 172.21.15.78:0/2070415785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0fc100970 msgr2=0x7fa0fc19e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.919+0000 7fa102070640 1 --2- 172.21.15.78:0/2070415785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0fc100970 0x7fa0fc19e890 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400b450 tx=0x7fa0e400b910 comp rx=0 tx=0).stop 2024-08-18T23:43:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.919+0000 7fa102070640 1 -- 172.21.15.78:0/2070415785 shutdown_connections 2024-08-18T23:43:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.919+0000 7fa102070640 1 --2- 172.21.15.78:0/2070415785 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa0e80a2bb0 0x7fa0e80a5070 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.919+0000 7fa102070640 1 --2- 172.21.15.78:0/2070415785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0fc101330 0x7fa0fc1a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.919+0000 7fa102070640 1 --2- 172.21.15.78:0/2070415785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0fc100970 0x7fa0fc19e890 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.919+0000 7fa102070640 1 --2- 172.21.15.78:0/2070415785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0fc0ffc00 0x7fa0fc19e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.919+0000 7fa102070640 1 -- 172.21.15.78:0/2070415785 >> 172.21.15.78:0/2070415785 conn(0x7fa0fc0fb770 msgr2=0x7fa0fc0fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:40.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.919+0000 7fa102070640 1 -- 172.21.15.78:0/2070415785 shutdown_connections 2024-08-18T23:43:40.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:40.919+0000 7fa102070640 1 -- 172.21.15.78:0/2070415785 wait complete. 2024-08-18T23:43:41.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:41 smithi123 ceph-mon[21808]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:41.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:41 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2070415785' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-08-18T23:43:41.578 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:41 smithi161 ceph-mon[22027]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:41.579 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:41 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2070415785' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-08-18T23:43:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:41 smithi078 ceph-mon[18440]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:41.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:41 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2070415785' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-08-18T23:43:41.752 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- bash -c 'ceph osd dump' 2024-08-18T23:43:42.015 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:42.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.503+0000 7fe6a2d9e640 1 -- 172.21.15.78:0/19770567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c107690 msgr2=0x7fe69c109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:42.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.503+0000 7fe6a2d9e640 1 --2- 172.21.15.78:0/19770567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c107690 0x7fe69c109a80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe694009ca0 tx=0x7fe69402a8c0 comp rx=0 tx=0).stop 2024-08-18T23:43:42.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.503+0000 7fe6a2d9e640 1 -- 172.21.15.78:0/19770567 shutdown_connections 2024-08-18T23:43:42.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.503+0000 7fe6a2d9e640 1 --2- 172.21.15.78:0/19770567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c107690 0x7fe69c109a80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:42.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.503+0000 7fe6a2d9e640 1 --2- 172.21.15.78:0/19770567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe69c104d60 0x7fe69c107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:42.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.503+0000 7fe6a2d9e640 1 --2- 172.21.15.78:0/19770567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe69c102430 0x7fe69c104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:42.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.503+0000 7fe6a2d9e640 1 -- 172.21.15.78:0/19770567 >> 172.21.15.78:0/19770567 conn(0x7fe69c0fbfd0 msgr2=0x7fe69c0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:42.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.503+0000 7fe6a2d9e640 1 -- 172.21.15.78:0/19770567 shutdown_connections 2024-08-18T23:43:42.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.503+0000 7fe6a2d9e640 1 -- 172.21.15.78:0/19770567 wait complete. 2024-08-18T23:43:42.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.504+0000 7fe6a2d9e640 1 Processor -- start 2024-08-18T23:43:42.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.504+0000 7fe6a2d9e640 1 -- start start 2024-08-18T23:43:42.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.504+0000 7fe6a2d9e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe69c102430 0x7fe69c101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:42.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.504+0000 7fe6a2d9e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe69c104d60 0x7fe69c1003e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:42.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.504+0000 7fe6a2d9e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c107690 0x7fe69c100920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:42.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.504+0000 7fe6a2d9e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c10d640 con 0x7fe69c107690 2024-08-18T23:43:42.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.504+0000 7fe6a2d9e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c10d4c0 con 0x7fe69c104d60 2024-08-18T23:43:42.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.504+0000 7fe6a2d9e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c10d7c0 con 0x7fe69c102430 2024-08-18T23:43:42.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.504+0000 7fe6a133b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c107690 0x7fe69c100920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:42.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.504+0000 7fe6a133b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c107690 0x7fe69c100920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34408/0 (socket says 172.21.15.78:34408) 2024-08-18T23:43:42.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.504+0000 7fe6a133b640 1 -- 172.21.15.78:0/2393116183 learned_addr learned my addr 172.21.15.78:0/2393116183 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:42.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.504+0000 7fe6a0b3a640 1 --2- 172.21.15.78:0/2393116183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe69c102430 0x7fe69c101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:42.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.504+0000 7fe69bfff640 1 --2- 172.21.15.78:0/2393116183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe69c104d60 0x7fe69c1003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:42.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.504+0000 7fe6a133b640 1 -- 172.21.15.78:0/2393116183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe69c102430 msgr2=0x7fe69c101d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:42.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.504+0000 7fe6a133b640 1 --2- 172.21.15.78:0/2393116183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe69c102430 0x7fe69c101d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:42.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.504+0000 7fe6a133b640 1 -- 172.21.15.78:0/2393116183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe69c104d60 msgr2=0x7fe69c1003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:42.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.504+0000 7fe6a133b640 1 --2- 172.21.15.78:0/2393116183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe69c104d60 0x7fe69c1003e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:42.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.504+0000 7fe6a133b640 1 -- 172.21.15.78:0/2393116183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69c1011e0 con 0x7fe69c107690 2024-08-18T23:43:42.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.505+0000 7fe6a133b640 1 --2- 172.21.15.78:0/2393116183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c107690 0x7fe69c100920 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe694005e30 tx=0x7fe694031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:42.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.505+0000 7fe699ffb640 1 -- 172.21.15.78:0/2393116183 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe694003b70 con 0x7fe69c107690 2024-08-18T23:43:42.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.505+0000 7fe6a2d9e640 1 -- 172.21.15.78:0/2393116183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69c1a1da0 con 0x7fe69c107690 2024-08-18T23:43:42.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.505+0000 7fe699ffb640 1 -- 172.21.15.78:0/2393116183 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe694003d10 con 0x7fe69c107690 2024-08-18T23:43:42.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.505+0000 7fe699ffb640 1 -- 172.21.15.78:0/2393116183 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69402fd30 con 0x7fe69c107690 2024-08-18T23:43:42.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.505+0000 7fe6a2d9e640 1 -- 172.21.15.78:0/2393116183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe69c1a22d0 con 0x7fe69c107690 2024-08-18T23:43:42.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.506+0000 7fe6a2d9e640 1 -- 172.21.15.78:0/2393116183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe69c0fdac0 con 0x7fe69c107690 2024-08-18T23:43:42.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.508+0000 7fe699ffb640 1 -- 172.21.15.78:0/2393116183 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe694004050 con 0x7fe69c107690 2024-08-18T23:43:42.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.508+0000 7fe699ffb640 1 --2- 172.21.15.78:0/2393116183 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe68c0a2b80 0x7fe68c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:42.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.509+0000 7fe6a0b3a640 1 --2- 172.21.15.78:0/2393116183 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe68c0a2b80 0x7fe68c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:42.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.509+0000 7fe699ffb640 1 -- 172.21.15.78:0/2393116183 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fe6940f07d0 con 0x7fe69c107690 2024-08-18T23:43:42.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.509+0000 7fe6a0b3a640 1 --2- 172.21.15.78:0/2393116183 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe68c0a2b80 0x7fe68c0a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe69000bbe0 tx=0x7fe69000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:42.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.512+0000 7fe699ffb640 1 -- 172.21.15.78:0/2393116183 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe6940bd3a0 con 0x7fe69c107690 2024-08-18T23:43:42.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.680+0000 7fe6a2d9e640 1 -- 172.21.15.78:0/2393116183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fe69c02d5e0 con 0x7fe69c107690 2024-08-18T23:43:42.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.681+0000 7fe699ffb640 1 -- 172.21.15.78:0/2393116183 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) ==== 56+0+5652 (secure 0 0 0) 0x7fe6940bcca0 con 0x7fe69c107690 2024-08-18T23:43:42.682 INFO:teuthology.orchestra.run.smithi078.stdout:epoch 92 2024-08-18T23:43:42.683 INFO:teuthology.orchestra.run.smithi078.stdout:fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:43:42.683 INFO:teuthology.orchestra.run.smithi078.stdout:created 2024-08-18T23:34:33.867246+0000 2024-08-18T23:43:42.683 INFO:teuthology.orchestra.run.smithi078.stdout:modified 2024-08-18T23:43:24.291494+0000 2024-08-18T23:43:42.683 INFO:teuthology.orchestra.run.smithi078.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-08-18T23:43:42.683 INFO:teuthology.orchestra.run.smithi078.stdout:crush_version 27 2024-08-18T23:43:42.683 INFO:teuthology.orchestra.run.smithi078.stdout:full_ratio 0.95 2024-08-18T23:43:42.683 INFO:teuthology.orchestra.run.smithi078.stdout:backfillfull_ratio 0.9 2024-08-18T23:43:42.683 INFO:teuthology.orchestra.run.smithi078.stdout:nearfull_ratio 0.85 2024-08-18T23:43:42.683 INFO:teuthology.orchestra.run.smithi078.stdout:require_min_compat_client luminous 2024-08-18T23:43:42.683 INFO:teuthology.orchestra.run.smithi078.stdout:min_compat_client luminous 2024-08-18T23:43:42.683 INFO:teuthology.orchestra.run.smithi078.stdout:require_osd_release squid 2024-08-18T23:43:42.683 INFO:teuthology.orchestra.run.smithi078.stdout:stretch_mode_enabled false 2024-08-18T23:43:42.684 INFO:teuthology.orchestra.run.smithi078.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-08-18T23:43:42.684 INFO:teuthology.orchestra.run.smithi078.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-08-18T23:43:42.684 INFO:teuthology.orchestra.run.smithi078.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-08-18T23:43:42.684 INFO:teuthology.orchestra.run.smithi078.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-08-18T23:43:42.684 INFO:teuthology.orchestra.run.smithi078.stdout:max_osd 12 2024-08-18T23:43:42.684 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 up in weight 1 up_from 8 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6802/3639645311,v1:172.21.15.78:6803/3639645311] [v2:172.21.15.78:6804/3639645311,v1:172.21.15.78:6805/3639645311] exists,up f4de1c78-6ebc-49a9-b9c5-85d43535fab7 2024-08-18T23:43:42.684 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6800/4199446368,v1:172.21.15.123:6801/4199446368] [v2:172.21.15.123:6802/4199446368,v1:172.21.15.123:6803/4199446368] exists,up 4f8e0c3d-472b-4230-ad23-2b0a34861251 2024-08-18T23:43:42.684 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6800/2582810057,v1:172.21.15.161:6801/2582810057] [v2:172.21.15.161:6802/2582810057,v1:172.21.15.161:6803/2582810057] exists,up 27ef3314-bb6f-413d-be05-e10a105dec9f 2024-08-18T23:43:42.684 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 up in weight 1 up_from 25 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6810/2697347521,v1:172.21.15.78:6811/2697347521] [v2:172.21.15.78:6812/2697347521,v1:172.21.15.78:6813/2697347521] exists,up e9e52662-5ef9-4d70-bdd7-2b92165822ee 2024-08-18T23:43:42.684 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 up in weight 1 up_from 32 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6808/1281511569,v1:172.21.15.123:6809/1281511569] [v2:172.21.15.123:6810/1281511569,v1:172.21.15.123:6811/1281511569] exists,up 5ed26775-79e3-4b24-9557-69fcf5e8e824 2024-08-18T23:43:42.684 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 up in weight 1 up_from 37 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6808/2751796117,v1:172.21.15.161:6809/2751796117] [v2:172.21.15.161:6810/2751796117,v1:172.21.15.161:6811/2751796117] exists,up a9793b1d-5613-48a4-938e-adb718f4fa4e 2024-08-18T23:43:42.684 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 up in weight 1 up_from 44 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6818/95383223,v1:172.21.15.78:6819/95383223] [v2:172.21.15.78:6820/95383223,v1:172.21.15.78:6821/95383223] exists,up 5dbc5e9c-a14f-4b71-910a-1e6d4827bb32 2024-08-18T23:43:42.685 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 up in weight 1 up_from 49 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6816/3440305156,v1:172.21.15.123:6817/3440305156] [v2:172.21.15.123:6818/3440305156,v1:172.21.15.123:6819/3440305156] exists,up 26fb7b69-131f-42ee-b745-6fe8b4f65391 2024-08-18T23:43:42.685 INFO:teuthology.orchestra.run.smithi078.stdout:osd.8 up in weight 1 up_from 56 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6816/1485008778,v1:172.21.15.161:6817/1485008778] [v2:172.21.15.161:6818/1485008778,v1:172.21.15.161:6819/1485008778] exists,up e267bc50-0c29-4d7b-b368-b1621e29b2df 2024-08-18T23:43:42.685 INFO:teuthology.orchestra.run.smithi078.stdout:osd.9 up in weight 1 up_from 63 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6826/2884475661,v1:172.21.15.78:6827/2884475661] [v2:172.21.15.78:6828/2884475661,v1:172.21.15.78:6829/2884475661] exists,up b7654818-b767-4fe6-8211-346297895f5e 2024-08-18T23:43:42.685 INFO:teuthology.orchestra.run.smithi078.stdout:osd.10 up in weight 1 up_from 68 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6824/122226286,v1:172.21.15.123:6825/122226286] [v2:172.21.15.123:6826/122226286,v1:172.21.15.123:6827/122226286] exists,up 59aa32ff-510e-4b52-8170-08a34a0b6b57 2024-08-18T23:43:42.685 INFO:teuthology.orchestra.run.smithi078.stdout:osd.11 up in weight 1 up_from 76 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6824/2037560985,v1:172.21.15.161:6825/2037560985] [v2:172.21.15.161:6826/2037560985,v1:172.21.15.161:6827/2037560985] exists,up 67bacab4-d078-4c5c-b4ff-9f6a8a00f507 2024-08-18T23:43:42.685 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.8 [2,11] 2024-08-18T23:43:42.685 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.c [2,4] 2024-08-18T23:43:42.685 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.1c [8,11] 2024-08-18T23:43:42.685 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.c [3,9] 2024-08-18T23:43:42.685 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.13 [4,9] 2024-08-18T23:43:42.685 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.17 [4,6] 2024-08-18T23:43:42.685 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.21 [4,8] 2024-08-18T23:43:42.685 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.33 [5,9] 2024-08-18T23:43:42.685 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 4.5 [5,8] 2024-08-18T23:43:42.686 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 4.e [9,11] 2024-08-18T23:43:42.686 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 4.13 [9,8] 2024-08-18T23:43:42.686 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 4.1f [9,10] 2024-08-18T23:43:42.686 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6801/311758909 expires 2024-08-19T23:35:22.036055+0000 2024-08-18T23:43:42.686 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6800/311758909 expires 2024-08-19T23:35:22.036055+0000 2024-08-18T23:43:42.686 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/3040499240 expires 2024-08-19T23:35:22.036055+0000 2024-08-18T23:43:42.686 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6801/2301978296 expires 2024-08-19T23:34:59.021165+0000 2024-08-18T23:43:42.686 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6800/2301978296 expires 2024-08-19T23:34:59.021165+0000 2024-08-18T23:43:42.686 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/757409165 expires 2024-08-19T23:34:59.021165+0000 2024-08-18T23:43:42.686 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/2562072965 expires 2024-08-19T23:35:22.036055+0000 2024-08-18T23:43:42.686 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/636184437 expires 2024-08-19T23:34:59.021165+0000 2024-08-18T23:43:42.686 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/1570708294 expires 2024-08-19T23:35:22.036055+0000 2024-08-18T23:43:42.686 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/3137806628 expires 2024-08-19T23:34:59.021165+0000 2024-08-18T23:43:42.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.683+0000 7fe6a2d9e640 1 -- 172.21.15.78:0/2393116183 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe68c0a2b80 msgr2=0x7fe68c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:42.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.683+0000 7fe6a2d9e640 1 --2- 172.21.15.78:0/2393116183 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe68c0a2b80 0x7fe68c0a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe69000bbe0 tx=0x7fe69000b390 comp rx=0 tx=0).stop 2024-08-18T23:43:42.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.683+0000 7fe6a2d9e640 1 -- 172.21.15.78:0/2393116183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c107690 msgr2=0x7fe69c100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:42.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.683+0000 7fe6a2d9e640 1 --2- 172.21.15.78:0/2393116183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c107690 0x7fe69c100920 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe694005e30 tx=0x7fe694031ee0 comp rx=0 tx=0).stop 2024-08-18T23:43:42.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.683+0000 7fe6a2d9e640 1 -- 172.21.15.78:0/2393116183 shutdown_connections 2024-08-18T23:43:42.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.683+0000 7fe6a2d9e640 1 --2- 172.21.15.78:0/2393116183 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe68c0a2b80 0x7fe68c0a5040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:42.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.683+0000 7fe6a2d9e640 1 --2- 172.21.15.78:0/2393116183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe69c107690 0x7fe69c100920 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:42.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.683+0000 7fe6a2d9e640 1 --2- 172.21.15.78:0/2393116183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe69c104d60 0x7fe69c1003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:42.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.683+0000 7fe6a2d9e640 1 --2- 172.21.15.78:0/2393116183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe69c102430 0x7fe69c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:42.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.683+0000 7fe6a2d9e640 1 -- 172.21.15.78:0/2393116183 >> 172.21.15.78:0/2393116183 conn(0x7fe69c0fbfd0 msgr2=0x7fe69c102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:42.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.684+0000 7fe6a2d9e640 1 -- 172.21.15.78:0/2393116183 shutdown_connections 2024-08-18T23:43:42.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:42.684+0000 7fe6a2d9e640 1 -- 172.21.15.78:0/2393116183 wait complete. 2024-08-18T23:43:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:43 smithi078 ceph-mon[18440]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:43.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:43 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2393116183' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-08-18T23:43:43.497 INFO:teuthology.run_tasks:Running task fs.ready... 2024-08-18T23:43:43.508 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:43:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:43 smithi123 ceph-mon[21808]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:43 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2393116183' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-08-18T23:43:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:43 smithi161 ceph-mon[22027]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:43 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2393116183' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-08-18T23:43:43.763 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:44.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.268+0000 7f7de568a640 1 -- 172.21.15.78:0/3579150979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de0100020 msgr2=0x7f7de0100400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.268+0000 7f7de568a640 1 --2- 172.21.15.78:0/3579150979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de0100020 0x7f7de0100400 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8009cd0 tx=0x7f7dd802a8e0 comp rx=0 tx=0).stop 2024-08-18T23:43:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.268+0000 7f7de568a640 1 -- 172.21.15.78:0/3579150979 shutdown_connections 2024-08-18T23:43:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.268+0000 7f7de568a640 1 --2- 172.21.15.78:0/3579150979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7de0106c80 0x7f7de010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.268+0000 7f7de568a640 1 --2- 172.21.15.78:0/3579150979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7de01009d0 0x7f7de0106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.268+0000 7f7de568a640 1 --2- 172.21.15.78:0/3579150979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de0100020 0x7f7de0100400 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.268+0000 7f7de568a640 1 -- 172.21.15.78:0/3579150979 >> 172.21.15.78:0/3579150979 conn(0x7f7de0076f40 msgr2=0x7f7de00fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.269+0000 7f7de568a640 1 -- 172.21.15.78:0/3579150979 shutdown_connections 2024-08-18T23:43:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.269+0000 7f7de568a640 1 -- 172.21.15.78:0/3579150979 wait complete. 2024-08-18T23:43:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.269+0000 7f7de568a640 1 Processor -- start 2024-08-18T23:43:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.269+0000 7f7de568a640 1 -- start start 2024-08-18T23:43:44.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.269+0000 7f7de568a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7de0100020 0x7f7de0198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7de568a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de01009d0 0x7f7de0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7de568a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7de0106c80 0x7f7de019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7de568a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7de0111cd0 con 0x7f7de01009d0 2024-08-18T23:43:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7de568a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7de0111b50 con 0x7f7de0106c80 2024-08-18T23:43:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7de568a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7de0111e50 con 0x7f7de0100020 2024-08-18T23:43:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7dde7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de01009d0 0x7f7de0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7dde7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de01009d0 0x7f7de0198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34434/0 (socket says 172.21.15.78:34434) 2024-08-18T23:43:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7dde7fc640 1 -- 172.21.15.78:0/1298582803 learned_addr learned my addr 172.21.15.78:0/1298582803 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7ddf7fe640 1 --2- 172.21.15.78:0/1298582803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7de0106c80 0x7f7de019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7ddeffd640 1 --2- 172.21.15.78:0/1298582803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7de0100020 0x7f7de0198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7dde7fc640 1 -- 172.21.15.78:0/1298582803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7de0100020 msgr2=0x7f7de0198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7dde7fc640 1 --2- 172.21.15.78:0/1298582803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7de0100020 0x7f7de0198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:44.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7dde7fc640 1 -- 172.21.15.78:0/1298582803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7de0106c80 msgr2=0x7f7de019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:44.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7dde7fc640 1 --2- 172.21.15.78:0/1298582803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7de0106c80 0x7f7de019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:44.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7dde7fc640 1 -- 172.21.15.78:0/1298582803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de019d1c0 con 0x7f7de01009d0 2024-08-18T23:43:44.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7ddeffd640 1 --2- 172.21.15.78:0/1298582803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7de0100020 0x7f7de0198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:43:44.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7ddf7fe640 1 --2- 172.21.15.78:0/1298582803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7de0106c80 0x7f7de019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:43:44.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7dde7fc640 1 --2- 172.21.15.78:0/1298582803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de01009d0 0x7f7de0198750 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800c970 tx=0x7f7dc800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:44.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7dd7fff640 1 -- 172.21.15.78:0/1298582803 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8007bb0 con 0x7f7de01009d0 2024-08-18T23:43:44.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7de568a640 1 -- 172.21.15.78:0/1298582803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de019d4b0 con 0x7f7de01009d0 2024-08-18T23:43:44.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7dd7fff640 1 -- 172.21.15.78:0/1298582803 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dc8007d50 con 0x7f7de01009d0 2024-08-18T23:43:44.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7dd7fff640 1 -- 172.21.15.78:0/1298582803 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc800f450 con 0x7f7de01009d0 2024-08-18T23:43:44.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.270+0000 7f7de568a640 1 -- 172.21.15.78:0/1298582803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de01a4d90 con 0x7f7de01009d0 2024-08-18T23:43:44.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.271+0000 7f7de568a640 1 -- 172.21.15.78:0/1298582803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de0100400 con 0x7f7de01009d0 2024-08-18T23:43:44.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.272+0000 7f7dd7fff640 1 -- 172.21.15.78:0/1298582803 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7dc80046c0 con 0x7f7de01009d0 2024-08-18T23:43:44.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.273+0000 7f7dd7fff640 1 --2- 172.21.15.78:0/1298582803 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7dcc0a2b80 0x7f7dcc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:44.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.273+0000 7f7ddeffd640 1 --2- 172.21.15.78:0/1298582803 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7dcc0a2b80 0x7f7dcc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:44.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.273+0000 7f7dd7fff640 1 -- 172.21.15.78:0/1298582803 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f7dc80cf4d0 con 0x7f7de01009d0 2024-08-18T23:43:44.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.273+0000 7f7ddeffd640 1 --2- 172.21.15.78:0/1298582803 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7dcc0a2b80 0x7f7dcc0a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8000f80 tx=0x7f7dd8031430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:44.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.277+0000 7f7dd7fff640 1 -- 172.21.15.78:0/1298582803 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7dc8014030 con 0x7f7de01009d0 2024-08-18T23:43:44.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.477+0000 7f7de568a640 1 -- 172.21.15.78:0/1298582803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7de002d5e0 con 0x7f7de01009d0 2024-08-18T23:43:44.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.477+0000 7f7dd7fff640 1 -- 172.21.15.78:0/1298582803 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11930 (secure 0 0 0) 0x7f7dc809c0a0 con 0x7f7de01009d0 2024-08-18T23:43:44.479 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:43:44.479 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":24,"btime":"2024-08-18T23:42:10:259464+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:42:09.974847+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:43:44.479 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 24 2024-08-18T23:43:44.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.479+0000 7f7de568a640 1 -- 172.21.15.78:0/1298582803 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7dcc0a2b80 msgr2=0x7f7dcc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:44.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.479+0000 7f7de568a640 1 --2- 172.21.15.78:0/1298582803 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7dcc0a2b80 0x7f7dcc0a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8000f80 tx=0x7f7dd8031430 comp rx=0 tx=0).stop 2024-08-18T23:43:44.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.480+0000 7f7de568a640 1 -- 172.21.15.78:0/1298582803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de01009d0 msgr2=0x7f7de0198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:44.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.480+0000 7f7de568a640 1 --2- 172.21.15.78:0/1298582803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de01009d0 0x7f7de0198750 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800c970 tx=0x7f7dc800ce30 comp rx=0 tx=0).stop 2024-08-18T23:43:44.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.480+0000 7f7de568a640 1 -- 172.21.15.78:0/1298582803 shutdown_connections 2024-08-18T23:43:44.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.480+0000 7f7de568a640 1 --2- 172.21.15.78:0/1298582803 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7dcc0a2b80 0x7f7dcc0a5040 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:44.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.480+0000 7f7de568a640 1 --2- 172.21.15.78:0/1298582803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7de0106c80 0x7f7de019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:44.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.480+0000 7f7de568a640 1 --2- 172.21.15.78:0/1298582803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de01009d0 0x7f7de0198750 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:44.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.480+0000 7f7de568a640 1 --2- 172.21.15.78:0/1298582803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7de0100020 0x7f7de0198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:44.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.480+0000 7f7de568a640 1 -- 172.21.15.78:0/1298582803 >> 172.21.15.78:0/1298582803 conn(0x7f7de0076f40 msgr2=0x7f7de0104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:44.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.480+0000 7f7de568a640 1 -- 172.21.15.78:0/1298582803 shutdown_connections 2024-08-18T23:43:44.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:44.480+0000 7f7de568a640 1 -- 172.21.15.78:0/1298582803 wait complete. 2024-08-18T23:43:45.237 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:43:45.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:45 smithi078 ceph-mon[18440]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-18T23:43:45.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:45 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1298582803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:43:45.491 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:45.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:45 smithi123 ceph-mon[21808]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-18T23:43:45.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:45 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1298582803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:43:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:45 smithi161 ceph-mon[22027]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-18T23:43:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:45 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1298582803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:43:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.085+0000 7fb1b872b640 1 -- 172.21.15.78:0/1211380548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1b0101150 msgr2=0x7fb1b010d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.085+0000 7fb1b872b640 1 --2- 172.21.15.78:0/1211380548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1b0101150 0x7fb1b010d680 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009cd0 tx=0x7fb1a802a8e0 comp rx=0 tx=0).stop 2024-08-18T23:43:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.085+0000 7fb1b872b640 1 -- 172.21.15.78:0/1211380548 shutdown_connections 2024-08-18T23:43:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.085+0000 7fb1b872b640 1 --2- 172.21.15.78:0/1211380548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1b0101150 0x7fb1b010d680 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:46.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.085+0000 7fb1b872b640 1 --2- 172.21.15.78:0/1211380548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1b01007b0 0x7fb1b0100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:46.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.085+0000 7fb1b872b640 1 --2- 172.21.15.78:0/1211380548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1b01067b0 0x7fb1b0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:46.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.085+0000 7fb1b872b640 1 -- 172.21.15.78:0/1211380548 >> 172.21.15.78:0/1211380548 conn(0x7fb1b00fc4d0 msgr2=0x7fb1b00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:46.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.085+0000 7fb1b872b640 1 -- 172.21.15.78:0/1211380548 shutdown_connections 2024-08-18T23:43:46.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.085+0000 7fb1b872b640 1 -- 172.21.15.78:0/1211380548 wait complete. 2024-08-18T23:43:46.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.086+0000 7fb1b872b640 1 Processor -- start 2024-08-18T23:43:46.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.086+0000 7fb1b872b640 1 -- start start 2024-08-18T23:43:46.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.086+0000 7fb1b872b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1b01007b0 0x7fb1b00fc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:46.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.086+0000 7fb1b872b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1b0101150 0x7fb1b00fcf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.086+0000 7fb1b872b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1b01067b0 0x7fb1b00fd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.086+0000 7fb1b872b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b0110bd0 con 0x7fb1b0101150 2024-08-18T23:43:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.086+0000 7fb1b872b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b0110a50 con 0x7fb1b01007b0 2024-08-18T23:43:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.086+0000 7fb1b872b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b0110d50 con 0x7fb1b01067b0 2024-08-18T23:43:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.086+0000 7fb1b5cc6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1b0101150 0x7fb1b00fcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.086+0000 7fb1b5cc6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1b0101150 0x7fb1b00fcf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50376/0 (socket says 172.21.15.78:50376) 2024-08-18T23:43:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.086+0000 7fb1b5cc6640 1 -- 172.21.15.78:0/3637539674 learned_addr learned my addr 172.21.15.78:0/3637539674 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.086+0000 7fb1b5cc6640 1 -- 172.21.15.78:0/3637539674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1b01067b0 msgr2=0x7fb1b00fd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.086+0000 7fb1b64c7640 1 --2- 172.21.15.78:0/3637539674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1b01007b0 0x7fb1b00fc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.086+0000 7fb1b6cc8640 1 --2- 172.21.15.78:0/3637539674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1b01067b0 0x7fb1b00fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.086+0000 7fb1b5cc6640 1 --2- 172.21.15.78:0/3637539674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1b01067b0 0x7fb1b00fd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:46.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.086+0000 7fb1b5cc6640 1 -- 172.21.15.78:0/3637539674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1b01007b0 msgr2=0x7fb1b00fc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:46.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.086+0000 7fb1b5cc6640 1 --2- 172.21.15.78:0/3637539674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1b01007b0 0x7fb1b00fc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:46.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.086+0000 7fb1b5cc6640 1 -- 172.21.15.78:0/3637539674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b01a65c0 con 0x7fb1b0101150 2024-08-18T23:43:46.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.086+0000 7fb1b5cc6640 1 --2- 172.21.15.78:0/3637539674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1b0101150 0x7fb1b00fcf10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400b4a0 tx=0x7fb1a400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:46.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.087+0000 7fb1a37fe640 1 -- 172.21.15.78:0/3637539674 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4014070 con 0x7fb1b0101150 2024-08-18T23:43:46.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.087+0000 7fb1a37fe640 1 -- 172.21.15.78:0/3637539674 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a4010af0 con 0x7fb1b0101150 2024-08-18T23:43:46.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.087+0000 7fb1b872b640 1 -- 172.21.15.78:0/3637539674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b01a68b0 con 0x7fb1b0101150 2024-08-18T23:43:46.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.087+0000 7fb1a37fe640 1 -- 172.21.15.78:0/3637539674 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4017d10 con 0x7fb1b0101150 2024-08-18T23:43:46.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.087+0000 7fb1b872b640 1 -- 172.21.15.78:0/3637539674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b01a6e40 con 0x7fb1b0101150 2024-08-18T23:43:46.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.088+0000 7fb1b872b640 1 -- 172.21.15.78:0/3637539674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b0100c10 con 0x7fb1b0101150 2024-08-18T23:43:46.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.089+0000 7fb1a37fe640 1 -- 172.21.15.78:0/3637539674 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb1a4017070 con 0x7fb1b0101150 2024-08-18T23:43:46.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.090+0000 7fb1a37fe640 1 --2- 172.21.15.78:0/3637539674 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb19c0a2c00 0x7fb19c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:46.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.090+0000 7fb1b64c7640 1 --2- 172.21.15.78:0/3637539674 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb19c0a2c00 0x7fb19c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:46.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.090+0000 7fb1a37fe640 1 -- 172.21.15.78:0/3637539674 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fb1a40ceb90 con 0x7fb1b0101150 2024-08-18T23:43:46.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.090+0000 7fb1b64c7640 1 --2- 172.21.15.78:0/3637539674 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb19c0a2c00 0x7fb19c0a50c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00ba40 tx=0x7fb1ac005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:46.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.093+0000 7fb1a37fe640 1 -- 172.21.15.78:0/3637539674 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb1a409b760 con 0x7fb1b0101150 2024-08-18T23:43:46.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.300+0000 7fb1b872b640 1 -- 172.21.15.78:0/3637539674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1b002d5e0 con 0x7fb1b0101150 2024-08-18T23:43:46.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.300+0000 7fb1a37fe640 1 -- 172.21.15.78:0/3637539674 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11930 (secure 0 0 0) 0x7fb1a409b060 con 0x7fb1b0101150 2024-08-18T23:43:46.302 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:43:46.302 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":24,"btime":"2024-08-18T23:42:10:259464+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:42:09.974847+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:43:46.302 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 24 2024-08-18T23:43:46.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.302+0000 7fb1b872b640 1 -- 172.21.15.78:0/3637539674 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb19c0a2c00 msgr2=0x7fb19c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:46.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.302+0000 7fb1b872b640 1 --2- 172.21.15.78:0/3637539674 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb19c0a2c00 0x7fb19c0a50c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00ba40 tx=0x7fb1ac005cc0 comp rx=0 tx=0).stop 2024-08-18T23:43:46.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.302+0000 7fb1b872b640 1 -- 172.21.15.78:0/3637539674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1b0101150 msgr2=0x7fb1b00fcf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:46.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.302+0000 7fb1b872b640 1 --2- 172.21.15.78:0/3637539674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1b0101150 0x7fb1b00fcf10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400b4a0 tx=0x7fb1a400b960 comp rx=0 tx=0).stop 2024-08-18T23:43:46.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.302+0000 7fb1b872b640 1 -- 172.21.15.78:0/3637539674 shutdown_connections 2024-08-18T23:43:46.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.302+0000 7fb1b872b640 1 --2- 172.21.15.78:0/3637539674 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb19c0a2c00 0x7fb19c0a50c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:46.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.302+0000 7fb1b872b640 1 --2- 172.21.15.78:0/3637539674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1b01067b0 0x7fb1b00fd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:46.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.302+0000 7fb1b872b640 1 --2- 172.21.15.78:0/3637539674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1b0101150 0x7fb1b00fcf10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:46.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.302+0000 7fb1b872b640 1 --2- 172.21.15.78:0/3637539674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1b01007b0 0x7fb1b00fc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:46.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.302+0000 7fb1b872b640 1 -- 172.21.15.78:0/3637539674 >> 172.21.15.78:0/3637539674 conn(0x7fb1b00fc4d0 msgr2=0x7fb1b006a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:46.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.302+0000 7fb1b872b640 1 -- 172.21.15.78:0/3637539674 shutdown_connections 2024-08-18T23:43:46.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:46.302+0000 7fb1b872b640 1 -- 172.21.15.78:0/3637539674 wait complete. 2024-08-18T23:43:47.108 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-18T23:43:47.365 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:47.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:47 smithi078 ceph-mon[18440]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:47.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:47 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3637539674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:43:47.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:47 smithi123 ceph-mon[21808]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:47.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:47 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3637539674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:43:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:47 smithi161 ceph-mon[22027]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:47 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3637539674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:43:47.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.937+0000 7fcde6e88640 1 -- 172.21.15.78:0/2577503039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde0102250 msgr2=0x7fcde010e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:47.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.937+0000 7fcde6e88640 1 --2- 172.21.15.78:0/2577503039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde0102250 0x7fcde010e780 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009930 tx=0x7fcdd802a420 comp rx=0 tx=0).stop 2024-08-18T23:43:47.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.937+0000 7fcde6e88640 1 -- 172.21.15.78:0/2577503039 shutdown_connections 2024-08-18T23:43:47.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.937+0000 7fcde6e88640 1 --2- 172.21.15.78:0/2577503039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde0102250 0x7fcde010e780 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:47.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.937+0000 7fcde6e88640 1 --2- 172.21.15.78:0/2577503039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcde01018b0 0x7fcde0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:47.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.937+0000 7fcde6e88640 1 --2- 172.21.15.78:0/2577503039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcde01078b0 0x7fcde0107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:47.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.937+0000 7fcde6e88640 1 -- 172.21.15.78:0/2577503039 >> 172.21.15.78:0/2577503039 conn(0x7fcde00fd5f0 msgr2=0x7fcde00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:47.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.937+0000 7fcde6e88640 1 -- 172.21.15.78:0/2577503039 shutdown_connections 2024-08-18T23:43:47.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.937+0000 7fcde6e88640 1 -- 172.21.15.78:0/2577503039 wait complete. 2024-08-18T23:43:47.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.938+0000 7fcde6e88640 1 Processor -- start 2024-08-18T23:43:47.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.938+0000 7fcde6e88640 1 -- start start 2024-08-18T23:43:47.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.938+0000 7fcde6e88640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcde01018b0 0x7fcde0198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:47.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.938+0000 7fcde6e88640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde0102250 0x7fcde01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:47.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.938+0000 7fcde6e88640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcde01078b0 0x7fcde019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:47.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.938+0000 7fcde6e88640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0111cb0 con 0x7fcde0102250 2024-08-18T23:43:47.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.938+0000 7fcde6e88640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0111b30 con 0x7fcde01018b0 2024-08-18T23:43:47.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.938+0000 7fcde6e88640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0111e30 con 0x7fcde01078b0 2024-08-18T23:43:47.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.938+0000 7fcddffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde0102250 0x7fcde01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.938+0000 7fcddffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde0102250 0x7fcde01986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50404/0 (socket says 172.21.15.78:50404) 2024-08-18T23:43:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.938+0000 7fcddffff640 1 -- 172.21.15.78:0/3669277240 learned_addr learned my addr 172.21.15.78:0/3669277240 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.938+0000 7fcde4c24640 1 --2- 172.21.15.78:0/3669277240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcde01018b0 0x7fcde0198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.939+0000 7fcde5425640 1 --2- 172.21.15.78:0/3669277240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcde01078b0 0x7fcde019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.939+0000 7fcddffff640 1 -- 172.21.15.78:0/3669277240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcde01078b0 msgr2=0x7fcde019ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.939+0000 7fcddffff640 1 --2- 172.21.15.78:0/3669277240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcde01078b0 0x7fcde019ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.939+0000 7fcddffff640 1 -- 172.21.15.78:0/3669277240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcde01018b0 msgr2=0x7fcde0198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.939+0000 7fcddffff640 1 --2- 172.21.15.78:0/3669277240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcde01018b0 0x7fcde0198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.939+0000 7fcddffff640 1 -- 172.21.15.78:0/3669277240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde019d140 con 0x7fcde0102250 2024-08-18T23:43:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.939+0000 7fcde5425640 1 --2- 172.21.15.78:0/3669277240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcde01078b0 0x7fcde019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:43:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.939+0000 7fcddffff640 1 --2- 172.21.15.78:0/3669277240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde0102250 0x7fcde01986d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000cce0 tx=0x7fcdd000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.939+0000 7fcdddffb640 1 -- 172.21.15.78:0/3669277240 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0004b20 con 0x7fcde0102250 2024-08-18T23:43:47.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.939+0000 7fcdddffb640 1 -- 172.21.15.78:0/3669277240 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdd0004cc0 con 0x7fcde0102250 2024-08-18T23:43:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.939+0000 7fcde6e88640 1 -- 172.21.15.78:0/3669277240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde019d430 con 0x7fcde0102250 2024-08-18T23:43:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.939+0000 7fcdddffb640 1 -- 172.21.15.78:0/3669277240 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd000f8c0 con 0x7fcde0102250 2024-08-18T23:43:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.939+0000 7fcde6e88640 1 -- 172.21.15.78:0/3669277240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde01a4d10 con 0x7fcde0102250 2024-08-18T23:43:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.940+0000 7fcde6e88640 1 -- 172.21.15.78:0/3669277240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde0101d10 con 0x7fcde0102250 2024-08-18T23:43:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.941+0000 7fcdddffb640 1 -- 172.21.15.78:0/3669277240 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcdd000fa60 con 0x7fcde0102250 2024-08-18T23:43:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.942+0000 7fcdddffb640 1 --2- 172.21.15.78:0/3669277240 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcdcc0a2c00 0x7fcdcc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.942+0000 7fcde4c24640 1 --2- 172.21.15.78:0/3669277240 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcdcc0a2c00 0x7fcdcc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.942+0000 7fcdddffb640 1 -- 172.21.15.78:0/3669277240 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fcdd00cf360 con 0x7fcde0102250 2024-08-18T23:43:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.942+0000 7fcde4c24640 1 --2- 172.21.15.78:0/3669277240 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcdcc0a2c00 0x7fcdcc0a50c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcdc800bb60 tx=0x7fcdc800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:47.946+0000 7fcdddffb640 1 -- 172.21.15.78:0/3669277240 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fcdd0014030 con 0x7fcde0102250 2024-08-18T23:43:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:48.113+0000 7fcde6e88640 1 -- 172.21.15.78:0/3669277240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcde002d5e0 con 0x7fcde0102250 2024-08-18T23:43:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:48.115+0000 7fcdddffb640 1 -- 172.21.15.78:0/3669277240 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24747 (secure 0 0 0) 0x7fcdd009bfd0 con 0x7fcde0102250 2024-08-18T23:43:48.116 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:43:48.117 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":92,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","created":"2024-08-18T23:34:33.867246+0000","modified":"2024-08-18T23:43:24.291494+0000","last_up_change":"2024-08-18T23:40:42.200381+0000","last_in_change":"2024-08-18T23:40:22.425813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T23:37:22.338211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T23:41:28.920802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T23:41:31.026094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T23:41:37.403238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4de1c78-6ebc-49a9-b9c5-85d43535fab7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6803","nonce":3639645311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6805","nonce":3639645311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6809","nonce":3639645311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6807","nonce":3639645311}]},"public_addr":"172.21.15.78:6803/3639645311","cluster_addr":"172.21.15.78:6805/3639645311","heartbeat_back_addr":"172.21.15.78:6809/3639645311","heartbeat_front_addr":"172.21.15.78:6807/3639645311","state":["exists","up"]},{"osd":1,"uuid":"4f8e0c3d-472b-4230-ad23-2b0a34861251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6801","nonce":4199446368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6803","nonce":4199446368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6807","nonce":4199446368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6805","nonce":4199446368}]},"public_addr":"172.21.15.123:6801/4199446368","cluster_addr":"172.21.15.123:6803/4199446368","heartbeat_back_addr":"172.21.15.123:6807/4199446368","heartbeat_front_addr":"172.21.15.123:6805/4199446368","state":["exists","up"]},{"osd":2,"uuid":"27ef3314-bb6f-413d-be05-e10a105dec9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6801","nonce":2582810057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6803","nonce":2582810057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6807","nonce":2582810057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6805","nonce":2582810057}]},"public_addr":"172.21.15.161:6801/2582810057","cluster_addr":"172.21.15.161:6803/2582810057","heartbeat_back_addr":"172.21.15.161:6807/2582810057","heartbeat_front_addr":"172.21.15.161:6805/2582810057","state":["exists","up"]},{"osd":3,"uuid":"e9e52662-5ef9-4d70-bdd7-2b92165822ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6811","nonce":2697347521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6813","nonce":2697347521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6817","nonce":2697347521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6815","nonce":2697347521}]},"public_addr":"172.21.15.78:6811/2697347521","cluster_addr":"172.21.15.78:6813/2697347521","heartbeat_back_addr":"172.21.15.78:6817/2697347521","heartbeat_front_addr":"172.21.15.78:6815/2697347521","state":["exists","up"]},{"osd":4,"uuid":"5ed26775-79e3-4b24-9557-69fcf5e8e824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6809","nonce":1281511569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6811","nonce":1281511569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6815","nonce":1281511569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6813","nonce":1281511569}]},"public_addr":"172.21.15.123:6809/1281511569","cluster_addr":"172.21.15.123:6811/1281511569","heartbeat_back_addr":"172.21.15.123:6815/1281511569","heartbeat_front_addr":"172.21.15.123:6813/1281511569","state":["exists","up"]},{"osd":5,"uuid":"a9793b1d-5613-48a4-938e-adb718f4fa4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6809","nonce":2751796117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6811","nonce":2751796117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6815","nonce":2751796117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6813","nonce":2751796117}]},"public_addr":"172.21.15.161:6809/2751796117","cluster_addr":"172.21.15.161:6811/2751796117","heartbeat_back_addr":"172.21.15.161:6815/2751796117","heartbeat_front_addr":"172.21.15.161:6813/2751796117","state":["exists","up"]},{"osd":6,"uuid":"5dbc5e9c-a14f-4b71-910a-1e6d4827bb32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6819","nonce":95383223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6821","nonce":95383223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6825","nonce":95383223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6823","nonce":95383223}]},"public_addr":"172.21.15.78:6819/95383223","cluster_addr":"172.21.15.78:6821/95383223","heartbeat_back_addr":"172.21.15.78:6825/95383223","heartbeat_front_addr":"172.21.15.78:6823/95383223","state":["exists","up"]},{"osd":7,"uuid":"26fb7b69-131f-42ee-b745-6fe8b4f65391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6817","nonce":3440305156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6819","nonce":3440305156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6823","nonce":3440305156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6821","nonce":3440305156}]},"public_addr":"172.21.15.123:6817/3440305156","cluster_addr":"172.21.15.123:6819/3440305156","heartbeat_back_addr":"172.21.15.123:6823/3440305156","heartbeat_front_addr":"172.21.15.123:6821/3440305156","state":["exists","up"]},{"osd":8,"uuid":"e267bc50-0c29-4d7b-b368-b1621e29b2df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6817","nonce":1485008778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6819","nonce":1485008778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6823","nonce":1485008778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6821","nonce":1485008778}]},"public_addr":"172.21.15.161:6817/1485008778","cluster_addr":"172.21.15.161:6819/1485008778","heartbeat_back_addr":"172.21.15.161:6823/1485008778","heartbeat_front_addr":"172.21.15.161:6821/1485008778","state":["exists","up"]},{"osd":9,"uuid":"b7654818-b767-4fe6-8211-346297895f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6827","nonce":2884475661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6829","nonce":2884475661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6833","nonce":2884475661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6831","nonce":2884475661}]},"public_addr":"172.21.15.78:6827/2884475661","cluster_addr":"172.21.15.78:6829/2884475661","heartbeat_back_addr":"172.21.15.78:6833/2884475661","heartbeat_front_addr":"172.21.15.78:6831/2884475661","state":["exists","up"]},{"osd":10,"uuid":"59aa32ff-510e-4b52-8170-08a34a0b6b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6825","nonce":122226286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6827","nonce":122226286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6831","nonce":122226286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6829","nonce":122226286}]},"public_addr":"172.21.15.123:6825/122226286","cluster_addr":"172.21.15.123:6827/122226286","heartbeat_back_addr":"172.21.15.123:6831/122226286","heartbeat_front_addr":"172.21.15.123:6829/122226286","state":["exists","up"]},{"osd":11,"uuid":"67bacab4-d078-4c5c-b4ff-9f6a8a00f507","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6825","nonce":2037560985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6827","nonce":2037560985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6831","nonce":2037560985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6829","nonce":2037560985}]},"public_addr":"172.21.15.161:6825/2037560985","cluster_addr":"172.21.15.161:6827/2037560985","heartbeat_back_addr":"172.21.15.161:6831/2037560985","heartbeat_front_addr":"172.21.15.161:6829/2037560985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:36.826044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:57.688421+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:18.297424+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:40.355215+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:02.167902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:23.968759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:46.254083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:08.522038+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:31.028119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:54.119807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:16.847034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:39.607886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6800/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3040499240":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6801/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:6800/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/757409165":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/2562072965":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/636184437":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/1570708294":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3137806628":"2024-08-19T23:34:59.021165+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T23:43:48.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:48.117+0000 7fcde6e88640 1 -- 172.21.15.78:0/3669277240 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcdcc0a2c00 msgr2=0x7fcdcc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:48.117+0000 7fcde6e88640 1 --2- 172.21.15.78:0/3669277240 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcdcc0a2c00 0x7fcdcc0a50c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcdc800bb60 tx=0x7fcdc800b390 comp rx=0 tx=0).stop 2024-08-18T23:43:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:48.117+0000 7fcde6e88640 1 -- 172.21.15.78:0/3669277240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde0102250 msgr2=0x7fcde01986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:48.117+0000 7fcde6e88640 1 --2- 172.21.15.78:0/3669277240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde0102250 0x7fcde01986d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000cce0 tx=0x7fcdd000a620 comp rx=0 tx=0).stop 2024-08-18T23:43:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:48.117+0000 7fcde6e88640 1 -- 172.21.15.78:0/3669277240 shutdown_connections 2024-08-18T23:43:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:48.117+0000 7fcde6e88640 1 --2- 172.21.15.78:0/3669277240 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcdcc0a2c00 0x7fcdcc0a50c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:48.117+0000 7fcde6e88640 1 --2- 172.21.15.78:0/3669277240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcde01078b0 0x7fcde019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:48.117+0000 7fcde6e88640 1 --2- 172.21.15.78:0/3669277240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde0102250 0x7fcde01986d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:48.117+0000 7fcde6e88640 1 --2- 172.21.15.78:0/3669277240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcde01018b0 0x7fcde0198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:48.117+0000 7fcde6e88640 1 -- 172.21.15.78:0/3669277240 >> 172.21.15.78:0/3669277240 conn(0x7fcde00fd5f0 msgr2=0x7fcde00fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:48.117+0000 7fcde6e88640 1 -- 172.21.15.78:0/3669277240 shutdown_connections 2024-08-18T23:43:48.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:48.117+0000 7fcde6e88640 1 -- 172.21.15.78:0/3669277240 wait complete. 2024-08-18T23:43:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:48 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3669277240' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:43:48.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:48 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3669277240' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:43:48.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:48 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3669277240' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:43:48.959 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-18T23:41:32.941573+0000', 'modified': '2024-08-18T23:42:09.974847+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24452, 'mds_1': 24428, 'mds_2': 24427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24397': {'gid': 24397, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/4100464644', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 4100464644}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 4100464644}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6837/2017185021', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 2017185021}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 2017185021}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6835/2459199316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 2459199316}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 2459199316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.161:6839/2209088174', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 2209088174}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 2209088174}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.123:6837/1639958554', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 1639958554}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 1639958554}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.123:6839/2656330956', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 2656330956}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 2656330956}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24452, 'qdb_cluster': [24427, 24428, 24452]} 2024-08-18T23:43:48.959 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-18T23:43:48.960 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.i status 2024-08-18T23:43:49.213 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:49.488 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:49 smithi078 ceph-mon[18440]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:49.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:49 smithi123 ceph-mon[21808]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:49 smithi161 ceph-mon[22027]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:49.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.704+0000 7f9da0f94640 1 -- 172.21.15.78:0/2269984471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d9c100b50 msgr2=0x7f9d9c100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:49.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.704+0000 7f9da0f94640 1 --2- 172.21.15.78:0/2269984471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d9c100b50 0x7f9d9c100f30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009ca0 tx=0x7f9d9002a950 comp rx=0 tx=0).stop 2024-08-18T23:43:49.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.704+0000 7f9da0f94640 1 -- 172.21.15.78:0/2269984471 shutdown_connections 2024-08-18T23:43:49.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.704+0000 7f9da0f94640 1 --2- 172.21.15.78:0/2269984471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d9c102c20 0x7f9d9c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.704+0000 7f9da0f94640 1 --2- 172.21.15.78:0/2269984471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d9c1022a0 0x7f9d9c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.704+0000 7f9da0f94640 1 --2- 172.21.15.78:0/2269984471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d9c100b50 0x7f9d9c100f30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.704+0000 7f9da0f94640 1 -- 172.21.15.78:0/2269984471 >> 172.21.15.78:0/2269984471 conn(0x7f9d9c0fc500 msgr2=0x7f9d9c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:49.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.704+0000 7f9da0f94640 1 -- 172.21.15.78:0/2269984471 shutdown_connections 2024-08-18T23:43:49.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.704+0000 7f9da0f94640 1 -- 172.21.15.78:0/2269984471 wait complete. 2024-08-18T23:43:49.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.705+0000 7f9da0f94640 1 Processor -- start 2024-08-18T23:43:49.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.705+0000 7f9da0f94640 1 -- start start 2024-08-18T23:43:49.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.705+0000 7f9da0f94640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d9c100b50 0x7f9d9c197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:49.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.705+0000 7f9da0f94640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d9c1022a0 0x7f9d9c197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:49.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.705+0000 7f9da0f94640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d9c102c20 0x7f9d9c19b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:49.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.705+0000 7f9da0f94640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c110cb0 con 0x7f9d9c102c20 2024-08-18T23:43:49.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.705+0000 7f9da0f94640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c110b30 con 0x7f9d9c100b50 2024-08-18T23:43:49.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.705+0000 7f9da0f94640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c110e30 con 0x7f9d9c1022a0 2024-08-18T23:43:49.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.705+0000 7f9d9ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d9c102c20 0x7f9d9c19b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:49.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.705+0000 7f9d9ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d9c102c20 0x7f9d9c19b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50422/0 (socket says 172.21.15.78:50422) 2024-08-18T23:43:49.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.705+0000 7f9d9ad9d640 1 -- 172.21.15.78:0/2740228904 learned_addr learned my addr 172.21.15.78:0/2740228904 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:49.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.705+0000 7f9d9a59c640 1 --2- 172.21.15.78:0/2740228904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d9c100b50 0x7f9d9c197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:49.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.705+0000 7f9d99d9b640 1 --2- 172.21.15.78:0/2740228904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d9c1022a0 0x7f9d9c197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:49.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.706+0000 7f9d9ad9d640 1 -- 172.21.15.78:0/2740228904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d9c1022a0 msgr2=0x7f9d9c197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:49.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.706+0000 7f9d9ad9d640 1 --2- 172.21.15.78:0/2740228904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d9c1022a0 0x7f9d9c197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.706+0000 7f9d9ad9d640 1 -- 172.21.15.78:0/2740228904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d9c100b50 msgr2=0x7f9d9c197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:49.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.706+0000 7f9d9ad9d640 1 --2- 172.21.15.78:0/2740228904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d9c100b50 0x7f9d9c197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.706+0000 7f9d9ad9d640 1 -- 172.21.15.78:0/2740228904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9c19c0c0 con 0x7f9d9c102c20 2024-08-18T23:43:49.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.706+0000 7f9d9ad9d640 1 --2- 172.21.15.78:0/2740228904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d9c102c20 0x7f9d9c19b9e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400c970 tx=0x7f9d9400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:49.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.706+0000 7f9d8b7fe640 1 -- 172.21.15.78:0/2740228904 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94007bb0 con 0x7f9d9c102c20 2024-08-18T23:43:49.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.706+0000 7f9da0f94640 1 -- 172.21.15.78:0/2740228904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9c19c3b0 con 0x7f9d9c102c20 2024-08-18T23:43:49.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.706+0000 7f9d8b7fe640 1 -- 172.21.15.78:0/2740228904 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d94007d50 con 0x7f9d9c102c20 2024-08-18T23:43:49.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.706+0000 7f9d8b7fe640 1 -- 172.21.15.78:0/2740228904 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9400f450 con 0x7f9d9c102c20 2024-08-18T23:43:49.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.706+0000 7f9da0f94640 1 -- 172.21.15.78:0/2740228904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d9c1a3c90 con 0x7f9d9c102c20 2024-08-18T23:43:49.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.708+0000 7f9d8b7fe640 1 -- 172.21.15.78:0/2740228904 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9d94004f60 con 0x7f9d9c102c20 2024-08-18T23:43:49.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.709+0000 7f9da3392740 1 Processor -- start 2024-08-18T23:43:49.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.709+0000 7f9da3392740 1 -- start start 2024-08-18T23:43:49.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.709+0000 7f9da3392740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574f530 0x5583d574f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:49.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.709+0000 7f9da3392740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5583d574fe50 0x5583d575c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:49.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.709+0000 7f9da3392740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 0x5583d575ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:49.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.709+0000 7f9da3392740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5583d56b4710 con 0x5583d575ca20 2024-08-18T23:43:49.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.709+0000 7f9da3392740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5583d564c770 con 0x5583d574f530 2024-08-18T23:43:49.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.709+0000 7f9da3392740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5583d56b06d0 con 0x5583d574fe50 2024-08-18T23:43:49.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.709+0000 7f9d99d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 0x5583d575ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:49.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.709+0000 7f9d99d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 0x5583d575ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50430/0 (socket says 172.21.15.78:50430) 2024-08-18T23:43:49.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.709+0000 7f9d99d9b640 1 -- 172.21.15.78:0/3204241512 learned_addr learned my addr 172.21.15.78:0/3204241512 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:49.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.709+0000 7f9d8b7fe640 1 --2- 172.21.15.78:0/2740228904 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9d840a2b80 0x7f9d840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:49.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.710+0000 7f9d99d9b640 1 --2- 172.21.15.78:0/3204241512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574f530 0x5583d574f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.710+0000 7f9d9a59c640 1 --2- 172.21.15.78:0/3204241512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5583d574fe50 0x5583d575c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.710+0000 7f9d9ad9d640 1 --2- 172.21.15.78:0/2740228904 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9d840a2b80 0x7f9d840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.710+0000 7f9d99d9b640 1 -- 172.21.15.78:0/3204241512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5583d574fe50 msgr2=0x5583d575c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.710+0000 7f9d99d9b640 1 --2- 172.21.15.78:0/3204241512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5583d574fe50 0x5583d575c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.710+0000 7f9d99d9b640 1 -- 172.21.15.78:0/3204241512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574f530 msgr2=0x5583d574f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:49.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.710+0000 7f9d99d9b640 1 --2- 172.21.15.78:0/3204241512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574f530 0x5583d574f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.710+0000 7f9d99d9b640 1 -- 172.21.15.78:0/3204241512 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583d56533e0 con 0x5583d575ca20 2024-08-18T23:43:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.710+0000 7f9d8b7fe640 1 -- 172.21.15.78:0/2740228904 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f9d940cf090 con 0x7f9d9c102c20 2024-08-18T23:43:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.710+0000 7f9d99d9b640 1 --2- 172.21.15.78:0/3204241512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 0x5583d575ee10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0125c0 tx=0x7f9d8c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273d82efd1ccec1 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.710+0000 7f9d88ff9640 1 -- 172.21.15.78:0/3204241512 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c0106b0 con 0x5583d575ca20 2024-08-18T23:43:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.710+0000 7f9d88ff9640 1 -- 172.21.15.78:0/3204241512 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8c010d30 con 0x5583d575ca20 2024-08-18T23:43:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.710+0000 7f9d88ff9640 1 -- 172.21.15.78:0/3204241512 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c01dd90 con 0x5583d575ca20 2024-08-18T23:43:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.710+0000 7f9d9ad9d640 1 --2- 172.21.15.78:0/2740228904 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9d840a2b80 0x7f9d840a5040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d9409b5a0 tx=0x7f9d9406d620 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.711+0000 7f9da3392740 1 -- 172.21.15.78:0/3204241512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 msgr2=0x5583d575ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:49.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.711+0000 7f9da3392740 1 --2- 172.21.15.78:0/3204241512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 0x5583d575ee10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0125c0 tx=0x7f9d8c00b870 comp rx=0 tx=0).stop 2024-08-18T23:43:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.711+0000 7f9da3392740 1 -- 172.21.15.78:0/3204241512 shutdown_connections 2024-08-18T23:43:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.711+0000 7f9da3392740 1 --2- 172.21.15.78:0/3204241512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 0x5583d575ee10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.711+0000 7f9da3392740 1 --2- 172.21.15.78:0/3204241512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5583d574fe50 0x5583d575c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.711+0000 7f9da3392740 1 --2- 172.21.15.78:0/3204241512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574f530 0x5583d574f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.711+0000 7f9da3392740 1 -- 172.21.15.78:0/3204241512 >> 172.21.15.78:0/3204241512 conn(0x5583d574bbc0 msgr2=0x5583d574bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.711+0000 7f9da3392740 1 -- 172.21.15.78:0/3204241512 shutdown_connections 2024-08-18T23:43:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.711+0000 7f9da3392740 1 -- 172.21.15.78:0/3204241512 wait complete. 2024-08-18T23:43:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.711+0000 7f9da3392740 1 -- start start 2024-08-18T23:43:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.711+0000 7f9da3392740 1 Processor -- start 2024-08-18T23:43:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.712+0000 7f9da3392740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5583d574f530 0x5583d57599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.712+0000 7f9da3392740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574fe50 0x5583d5759f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.712+0000 7f9da3392740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 0x5583d575a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:49.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.712+0000 7f9da3392740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5583d575f650 con 0x5583d575ca20 2024-08-18T23:43:49.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.712+0000 7f9da3392740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5583d566f970 con 0x5583d574fe50 2024-08-18T23:43:49.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.712+0000 7f9da3392740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5583d575f7d0 con 0x5583d574f530 2024-08-18T23:43:49.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.712+0000 7f9da3392740 10 client.0 _ll_register_callbacks cb 0x5583d53c37b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:43:49.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.712+0000 7f9d99d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 0x5583d575a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:49.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.712+0000 7f9d99d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 0x5583d575a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50446/0 (socket says 172.21.15.78:50446) 2024-08-18T23:43:49.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.712+0000 7f9d99d9b640 1 -- 172.21.15.78:0/1618777230 learned_addr learned my addr 172.21.15.78:0/1618777230 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:49.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.712+0000 7f9d99d9b640 1 --2- 172.21.15.78:0/1618777230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5583d574f530 0x5583d57599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:49.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.712+0000 7f9d9a59c640 1 --2- 172.21.15.78:0/1618777230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574fe50 0x5583d5759f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:49.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.712+0000 7f9d99d9b640 1 -- 172.21.15.78:0/1618777230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5583d574f530 msgr2=0x5583d57599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:49.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.712+0000 7f9d99d9b640 1 --2- 172.21.15.78:0/1618777230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5583d574f530 0x5583d57599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.712+0000 7f9d99d9b640 1 -- 172.21.15.78:0/1618777230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574fe50 msgr2=0x5583d5759f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:49.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.712+0000 7f9d99d9b640 1 --2- 172.21.15.78:0/1618777230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574fe50 0x5583d5759f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.712+0000 7f9d99d9b640 1 -- 172.21.15.78:0/1618777230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d54262b0 con 0x5583d575ca20 2024-08-18T23:43:49.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.712+0000 7f9d99d9b640 1 --2- 172.21.15.78:0/1618777230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 0x5583d575a460 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c001eb0 tx=0x7f9d8c01d850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9aa57c6b62aae1a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:49.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.712+0000 7f9d81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:43:49.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.713+0000 7f9d81ffb640 1 -- 172.21.15.78:0/1618777230 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c031060 con 0x5583d575ca20 2024-08-18T23:43:49.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.713+0000 7f9d81ffb640 1 -- 172.21.15.78:0/1618777230 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f9d8c01dd80 con 0x5583d575ca20 2024-08-18T23:43:49.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.713+0000 7f9da3392740 1 -- 172.21.15.78:0/1618777230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d8c010d30 con 0x5583d575ca20 2024-08-18T23:43:49.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.713+0000 7f9d81ffb640 1 -- 172.21.15.78:0/1618777230 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9d8c023040 con 0x5583d575ca20 2024-08-18T23:43:49.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.713+0000 7f9da3392740 10 client.15198 fetch_fsmap learned FSMap version 24 2024-08-18T23:43:49.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.713+0000 7f9da3392740 1 -- 172.21.15.78:0/1618777230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5583d576b370 con 0x5583d575ca20 2024-08-18T23:43:49.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.713+0000 7f9d81ffb640 1 -- 172.21.15.78:0/1618777230 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8c010d30 con 0x5583d575ca20 2024-08-18T23:43:49.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.713+0000 7f9d81ffb640 1 -- 172.21.15.78:0/1618777230 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f9d8c029030 con 0x5583d575ca20 2024-08-18T23:43:49.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.713+0000 7f9da3392740 10 client.15198 fetch_fsmap finished waiting for FSMap version 24 2024-08-18T23:43:49.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.713+0000 7f9da3392740 10 client.15198 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-18T23:43:49.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.713+0000 7f9da3392740 20 client.15198 populate_metadata read hostname 'smithi078' 2024-08-18T23:43:49.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.713+0000 7f9da3392740 1 --2- 172.21.15.78:0/1618777230 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5583d576d690 0x5583d576eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:49.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.713+0000 7f9da3392740 4 client.15198 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:43:49.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.713+0000 7f9da3392740 1 -- 172.21.15.78:0/1618777230 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583d56c1c80 con 0x5583d576d690 2024-08-18T23:43:49.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.714+0000 7f9d9a59c640 1 --2- 172.21.15.78:0/1618777230 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5583d576d690 0x5583d576eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:49.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.714+0000 7f9d9a59c640 1 --2- 172.21.15.78:0/1618777230 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5583d576d690 0x5583d576eb00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1e7ff2f24a79443 server_cookie=fdb3575c1b92807 in_seq=0 out_seq=0 2024-08-18T23:43:49.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.714+0000 7f9d81ffb640 10 client.15198 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:43:49.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.715+0000 7f9d81ffb640 1 -- 172.21.15.78:0/1618777230 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5583d56c1c80 con 0x5583d576d690 2024-08-18T23:43:49.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.715+0000 7f9d81ffb640 10 client.15198 handle_command_reply: tid=0 2024-08-18T23:43:49.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.715+0000 7f9da3392740 1 client.15198 shutdown 2024-08-18T23:43:49.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.715+0000 7f9da3392740 1 -- 172.21.15.78:0/1618777230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 msgr2=0x5583d575a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:49.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.715+0000 7f9da3392740 1 --2- 172.21.15.78:0/1618777230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 0x5583d575a460 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c001eb0 tx=0x7f9d8c01d850 comp rx=0 tx=0).stop 2024-08-18T23:43:49.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.715+0000 7f9da3392740 1 -- 172.21.15.78:0/1618777230 shutdown_connections 2024-08-18T23:43:49.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.715+0000 7f9da3392740 1 --2- 172.21.15.78:0/1618777230 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5583d576d690 0x5583d576eb00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.715+0000 7f9da3392740 1 --2- 172.21.15.78:0/1618777230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 0x5583d575a460 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.715+0000 7f9da3392740 1 --2- 172.21.15.78:0/1618777230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574fe50 0x5583d5759f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.715+0000 7f9da3392740 1 --2- 172.21.15.78:0/1618777230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5583d574f530 0x5583d57599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.715+0000 7f9d81ffb640 0 client.15198 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:43:49.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.715+0000 7f9da3392740 1 -- 172.21.15.78:0/1618777230 >> 172.21.15.78:0/1618777230 conn(0x5583d574bbc0 msgr2=0x5583d5751920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:49.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.715+0000 7f9da3392740 1 -- 172.21.15.78:0/1618777230 shutdown_connections 2024-08-18T23:43:49.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.715+0000 7f9da3392740 1 -- 172.21.15.78:0/1618777230 wait complete. 2024-08-18T23:43:49.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.715+0000 7f9da3392740 20 client.15198 trim_cache size 0 max 16384 2024-08-18T23:43:49.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.733+0000 7f9da3392740 1 Processor -- start 2024-08-18T23:43:49.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.733+0000 7f9da3392740 1 -- start start 2024-08-18T23:43:49.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.733+0000 7f9da3392740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d574f530 0x5583d576eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:49.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.733+0000 7f9da3392740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574fe50 0x5583d57592c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:49.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.733+0000 7f9da3392740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5583d575ca20 0x5583d5751770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:49.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.733+0000 7f9da3392740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5583d576ce40 con 0x5583d574f530 2024-08-18T23:43:49.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.733+0000 7f9da3392740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5583d576d510 con 0x5583d574fe50 2024-08-18T23:43:49.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.733+0000 7f9da3392740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5583d576dbe0 con 0x5583d575ca20 2024-08-18T23:43:49.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.733+0000 7f9d99d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d574f530 0x5583d576eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:49.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.733+0000 7f9d99d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d574f530 0x5583d576eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50454/0 (socket says 172.21.15.78:50454) 2024-08-18T23:43:49.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.733+0000 7f9d99d9b640 1 -- 172.21.15.78:0/1841808470 learned_addr learned my addr 172.21.15.78:0/1841808470 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:49.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.733+0000 7f9d9a59c640 1 --2- 172.21.15.78:0/1841808470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574fe50 0x5583d57592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:49.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.733+0000 7f9d99d9b640 1 -- 172.21.15.78:0/1841808470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5583d575ca20 msgr2=0x5583d5751770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:49.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.733+0000 7f9d99d9b640 1 --2- 172.21.15.78:0/1841808470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5583d575ca20 0x5583d5751770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.733+0000 7f9d99d9b640 1 -- 172.21.15.78:0/1841808470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574fe50 msgr2=0x5583d57592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:49.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.733+0000 7f9d99d9b640 1 --2- 172.21.15.78:0/1841808470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574fe50 0x5583d57592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.733+0000 7f9d99d9b640 1 -- 172.21.15.78:0/1841808470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583d5747130 con 0x5583d574f530 2024-08-18T23:43:49.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.733+0000 7f9d9a59c640 1 --2- 172.21.15.78:0/1841808470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574fe50 0x5583d57592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:43:49.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.733+0000 7f9d99d9b640 1 --2- 172.21.15.78:0/1841808470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d574f530 0x5583d576eb00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c01c3b0 tx=0x7f9d8c004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5bc605e6dc6fe6c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:49.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.734+0000 7f9d83fff640 1 -- 172.21.15.78:0/1841808470 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c031040 con 0x5583d574f530 2024-08-18T23:43:49.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.734+0000 7f9d83fff640 1 -- 172.21.15.78:0/1841808470 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8c010ce0 con 0x5583d574f530 2024-08-18T23:43:49.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.734+0000 7f9d83fff640 1 -- 172.21.15.78:0/1841808470 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c01e440 con 0x5583d574f530 2024-08-18T23:43:49.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.734+0000 7f9da3392740 1 -- 172.21.15.78:0/1841808470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d574f530 msgr2=0x5583d576eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:49.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.734+0000 7f9da3392740 1 --2- 172.21.15.78:0/1841808470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d574f530 0x5583d576eb00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c01c3b0 tx=0x7f9d8c004680 comp rx=0 tx=0).stop 2024-08-18T23:43:49.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.734+0000 7f9da3392740 1 -- 172.21.15.78:0/1841808470 shutdown_connections 2024-08-18T23:43:49.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.734+0000 7f9da3392740 1 --2- 172.21.15.78:0/1841808470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5583d575ca20 0x5583d5751770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.734+0000 7f9da3392740 1 --2- 172.21.15.78:0/1841808470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574fe50 0x5583d57592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.734+0000 7f9da3392740 1 --2- 172.21.15.78:0/1841808470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d574f530 0x5583d576eb00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.734+0000 7f9da3392740 1 -- 172.21.15.78:0/1841808470 >> 172.21.15.78:0/1841808470 conn(0x5583d574bbc0 msgr2=0x5583d5751060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:49.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.734+0000 7f9da3392740 1 -- 172.21.15.78:0/1841808470 shutdown_connections 2024-08-18T23:43:49.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.734+0000 7f9da3392740 1 -- 172.21.15.78:0/1841808470 wait complete. 2024-08-18T23:43:49.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.735+0000 7f9da3392740 1 -- start start 2024-08-18T23:43:49.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.735+0000 7f9da3392740 1 Processor -- start 2024-08-18T23:43:49.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.735+0000 7f9da3392740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574f530 0x5583d5777480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:49.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.735+0000 7f9da3392740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5583d574fe50 0x5583d57566f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:49.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.735+0000 7f9da3392740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 0x5583d5756c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:49.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.735+0000 7f9da3392740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5583d576e980 con 0x5583d575ca20 2024-08-18T23:43:49.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.735+0000 7f9da3392740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5583d576e2b0 con 0x5583d574f530 2024-08-18T23:43:49.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.735+0000 7f9da3392740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5583d576f050 con 0x5583d574fe50 2024-08-18T23:43:49.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.735+0000 7f9da3392740 10 client.0 _ll_register_callbacks cb 0x7f9d8c0123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:43:49.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.735+0000 7f9d99d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 0x5583d5756c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:49.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.735+0000 7f9d99d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 0x5583d5756c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50470/0 (socket says 172.21.15.78:50470) 2024-08-18T23:43:49.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.735+0000 7f9d99d9b640 1 -- 172.21.15.78:0/2280222532 learned_addr learned my addr 172.21.15.78:0/2280222532 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:49.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.735+0000 7f9d99d9b640 1 --2- 172.21.15.78:0/2280222532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574f530 0x5583d5777480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:49.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.735+0000 7f9d99d9b640 1 -- 172.21.15.78:0/2280222532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5583d574fe50 msgr2=0x5583d57566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:49.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.735+0000 7f9d99d9b640 1 --2- 172.21.15.78:0/2280222532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5583d574fe50 0x5583d57566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.735+0000 7f9d99d9b640 1 -- 172.21.15.78:0/2280222532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574f530 msgr2=0x5583d5777480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:49.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.735+0000 7f9d99d9b640 1 --2- 172.21.15.78:0/2280222532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574f530 0x5583d5777480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.735+0000 7f9d99d9b640 1 -- 172.21.15.78:0/2280222532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d5757440 con 0x5583d575ca20 2024-08-18T23:43:49.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.736+0000 7f9d99d9b640 1 --2- 172.21.15.78:0/2280222532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 0x5583d5756c30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c010cb0 tx=0x7f9d8c01dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5591e78465de3aa server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:49.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.736+0000 7f9d81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:43:49.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.736+0000 7f9d81ffb640 1 -- 172.21.15.78:0/2280222532 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c031030 con 0x5583d575ca20 2024-08-18T23:43:49.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.736+0000 7f9d81ffb640 1 -- 172.21.15.78:0/2280222532 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f9d8c0104b0 con 0x5583d575ca20 2024-08-18T23:43:49.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.736+0000 7f9da3392740 1 -- 172.21.15.78:0/2280222532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d8c010ce0 con 0x5583d575ca20 2024-08-18T23:43:49.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.736+0000 7f9d81ffb640 1 -- 172.21.15.78:0/2280222532 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9d8c0106b0 con 0x5583d575ca20 2024-08-18T23:43:49.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.736+0000 7f9da3392740 10 client.15204 fetch_fsmap learned FSMap version 24 2024-08-18T23:43:49.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.736+0000 7f9da3392740 1 -- 172.21.15.78:0/2280222532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5583d57d7da0 con 0x5583d575ca20 2024-08-18T23:43:49.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.736+0000 7f9d81ffb640 1 -- 172.21.15.78:0/2280222532 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8c010ce0 con 0x5583d575ca20 2024-08-18T23:43:49.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.736+0000 7f9d81ffb640 1 -- 172.21.15.78:0/2280222532 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f9d8c029050 con 0x5583d575ca20 2024-08-18T23:43:49.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.736+0000 7f9da3392740 10 client.15204 fetch_fsmap finished waiting for FSMap version 24 2024-08-18T23:43:49.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.736+0000 7f9da3392740 10 client.15204 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-18T23:43:49.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.736+0000 7f9da3392740 20 client.15204 populate_metadata read hostname 'smithi078' 2024-08-18T23:43:49.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.736+0000 7f9da3392740 1 --2- 172.21.15.78:0/2280222532 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5583d57dc4d0 0x5583d57db1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:49.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.736+0000 7f9da3392740 4 client.15204 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-18T23:43:49.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.736+0000 7f9da3392740 1 -- 172.21.15.78:0/2280222532 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "status"}) -- 0x5583d56bc190 con 0x5583d57dc4d0 2024-08-18T23:43:49.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.737+0000 7f9d9a59c640 1 --2- 172.21.15.78:0/2280222532 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5583d57dc4d0 0x5583d57db1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:49.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.737+0000 7f9d9a59c640 1 --2- 172.21.15.78:0/2280222532 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5583d57dc4d0 0x5583d57db1e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c0ffdf88395c8bc5 server_cookie=ec6c9853e95ac8ee in_seq=0 out_seq=0 2024-08-18T23:43:49.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.737+0000 7f9d81ffb640 10 client.15204 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:43:49.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.737+0000 7f9d81ffb640 1 -- 172.21.15.78:0/2280222532 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5583d56bc190 con 0x5583d57dc4d0 2024-08-18T23:43:49.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.737+0000 7f9d81ffb640 10 client.15204 handle_command_reply: tid=0 2024-08-18T23:43:49.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.737+0000 7f9da3392740 1 client.15204 shutdown 2024-08-18T23:43:49.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.737+0000 7f9da3392740 1 -- 172.21.15.78:0/2280222532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 msgr2=0x5583d5756c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:49.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.737+0000 7f9da3392740 1 --2- 172.21.15.78:0/2280222532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 0x5583d5756c30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c010cb0 tx=0x7f9d8c01dc50 comp rx=0 tx=0).stop 2024-08-18T23:43:49.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.737+0000 7f9da3392740 1 -- 172.21.15.78:0/2280222532 shutdown_connections 2024-08-18T23:43:49.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.737+0000 7f9da3392740 1 --2- 172.21.15.78:0/2280222532 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5583d57dc4d0 0x5583d57db1e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.737+0000 7f9da3392740 1 --2- 172.21.15.78:0/2280222532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5583d575ca20 0x5583d5756c30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.737+0000 7f9da3392740 1 --2- 172.21.15.78:0/2280222532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5583d574fe50 0x5583d57566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.737+0000 7f9da3392740 1 --2- 172.21.15.78:0/2280222532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5583d574f530 0x5583d5777480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.737+0000 7f9da3392740 1 -- 172.21.15.78:0/2280222532 >> 172.21.15.78:0/2280222532 conn(0x5583d574bbc0 msgr2=0x5583d5751880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:49.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.737+0000 7f9d81ffb640 0 client.15204 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:43:49.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.738+0000 7f9da3392740 1 -- 172.21.15.78:0/2280222532 shutdown_connections 2024-08-18T23:43:49.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.738+0000 7f9da3392740 1 -- 172.21.15.78:0/2280222532 wait complete. 2024-08-18T23:43:49.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.738+0000 7f9da3392740 20 client.15204 trim_cache size 0 max 16384 2024-08-18T23:43:49.750 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:43:49.750 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "4f1128e6-5dba-11ef-bccf-c7b262605968", 2024-08-18T23:43:49.750 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 2, 2024-08-18T23:43:49.750 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24427, 2024-08-18T23:43:49.750 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-08-18T23:43:49.750 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-08-18T23:43:49.750 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-08-18T23:43:49.750 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 102.48368320599999, 2024-08-18T23:43:49.750 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 24, 2024-08-18T23:43:49.750 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 92, 2024-08-18T23:43:49.750 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 88, 2024-08-18T23:43:49.751 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 156.473535822 2024-08-18T23:43:49.751 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:43:49.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.738+0000 7f9da0f94640 1 -- 172.21.15.78:0/2740228904 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9d840a2b80 msgr2=0x7f9d840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:49.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.738+0000 7f9da0f94640 1 --2- 172.21.15.78:0/2740228904 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9d840a2b80 0x7f9d840a5040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d9409b5a0 tx=0x7f9d9406d620 comp rx=0 tx=0).stop 2024-08-18T23:43:49.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.738+0000 7f9da0f94640 1 -- 172.21.15.78:0/2740228904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d9c102c20 msgr2=0x7f9d9c19b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:49.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.738+0000 7f9da0f94640 1 --2- 172.21.15.78:0/2740228904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d9c102c20 0x7f9d9c19b9e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400c970 tx=0x7f9d9400ce30 comp rx=0 tx=0).stop 2024-08-18T23:43:49.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.738+0000 7f9da0f94640 1 -- 172.21.15.78:0/2740228904 shutdown_connections 2024-08-18T23:43:49.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.738+0000 7f9da0f94640 1 --2- 172.21.15.78:0/2740228904 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9d840a2b80 0x7f9d840a5040 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.738+0000 7f9da0f94640 1 --2- 172.21.15.78:0/2740228904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d9c102c20 0x7f9d9c19b9e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.738+0000 7f9da0f94640 1 --2- 172.21.15.78:0/2740228904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d9c1022a0 0x7f9d9c197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.738+0000 7f9da0f94640 1 --2- 172.21.15.78:0/2740228904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d9c100b50 0x7f9d9c197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:49.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.738+0000 7f9da0f94640 1 -- 172.21.15.78:0/2740228904 >> 172.21.15.78:0/2740228904 conn(0x7f9d9c0fc500 msgr2=0x7f9d9c0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:49.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.738+0000 7f9da0f94640 1 -- 172.21.15.78:0/2740228904 shutdown_connections 2024-08-18T23:43:49.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:49.738+0000 7f9da0f94640 1 -- 172.21.15.78:0/2740228904 wait complete. 2024-08-18T23:43:50.463 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.h status 2024-08-18T23:43:50.711 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:51.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.202+0000 7f36f976a640 1 -- 172.21.15.78:0/1316847055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f41067b0 msgr2=0x7f36f4106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:51.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.202+0000 7f36f976a640 1 --2- 172.21.15.78:0/1316847055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f41067b0 0x7f36f4106b90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009930 tx=0x7f36e402a420 comp rx=0 tx=0).stop 2024-08-18T23:43:51.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.202+0000 7f36f976a640 1 -- 172.21.15.78:0/1316847055 shutdown_connections 2024-08-18T23:43:51.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.202+0000 7f36f976a640 1 --2- 172.21.15.78:0/1316847055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36f4101150 0x7f36f410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.202+0000 7f36f976a640 1 --2- 172.21.15.78:0/1316847055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36f41007b0 0x7f36f4100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.202+0000 7f36f976a640 1 --2- 172.21.15.78:0/1316847055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f41067b0 0x7f36f4106b90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.202+0000 7f36f976a640 1 -- 172.21.15.78:0/1316847055 >> 172.21.15.78:0/1316847055 conn(0x7f36f40fc4d0 msgr2=0x7f36f40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:51.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.202+0000 7f36f976a640 1 -- 172.21.15.78:0/1316847055 shutdown_connections 2024-08-18T23:43:51.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.202+0000 7f36f976a640 1 -- 172.21.15.78:0/1316847055 wait complete. 2024-08-18T23:43:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.203+0000 7f36f976a640 1 Processor -- start 2024-08-18T23:43:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.203+0000 7f36f976a640 1 -- start start 2024-08-18T23:43:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.203+0000 7f36f976a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36f41007b0 0x7f36f4196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.203+0000 7f36f976a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f4101150 0x7f36f4197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.203+0000 7f36f976a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36f41067b0 0x7f36f419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.203+0000 7f36f976a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f36f4110bb0 con 0x7f36f4101150 2024-08-18T23:43:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.203+0000 7f36f976a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36f4110a30 con 0x7f36f41007b0 2024-08-18T23:43:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.203+0000 7f36f976a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f36f4110d30 con 0x7f36f41067b0 2024-08-18T23:43:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.203+0000 7f36ebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f4101150 0x7f36f4197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.203+0000 7f36ebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f4101150 0x7f36f4197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50496/0 (socket says 172.21.15.78:50496) 2024-08-18T23:43:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.203+0000 7f36ebfff640 1 -- 172.21.15.78:0/1639915145 learned_addr learned my addr 172.21.15.78:0/1639915145 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:51.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.203+0000 7f36f2ffd640 1 --2- 172.21.15.78:0/1639915145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36f41007b0 0x7f36f4196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.203+0000 7f36f37fe640 1 --2- 172.21.15.78:0/1639915145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36f41067b0 0x7f36f419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.204+0000 7f36ebfff640 1 -- 172.21.15.78:0/1639915145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36f41067b0 msgr2=0x7f36f419b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.204+0000 7f36ebfff640 1 --2- 172.21.15.78:0/1639915145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36f41067b0 0x7f36f419b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.204+0000 7f36ebfff640 1 -- 172.21.15.78:0/1639915145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36f41007b0 msgr2=0x7f36f4196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.204+0000 7f36ebfff640 1 --2- 172.21.15.78:0/1639915145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36f41007b0 0x7f36f4196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.204+0000 7f36ebfff640 1 -- 172.21.15.78:0/1639915145 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f419bfa0 con 0x7f36f4101150 2024-08-18T23:43:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.204+0000 7f36ebfff640 1 --2- 172.21.15.78:0/1639915145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f4101150 0x7f36f4197530 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00d950 tx=0x7f36dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:51.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.204+0000 7f36f0ff9640 1 -- 172.21.15.78:0/1639915145 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc017070 con 0x7f36f4101150 2024-08-18T23:43:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.204+0000 7f36f976a640 1 -- 172.21.15.78:0/1639915145 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f419c290 con 0x7f36f4101150 2024-08-18T23:43:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.204+0000 7f36f0ff9640 1 -- 172.21.15.78:0/1639915145 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36dc009c30 con 0x7f36f4101150 2024-08-18T23:43:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.204+0000 7f36f0ff9640 1 -- 172.21.15.78:0/1639915145 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc00f6b0 con 0x7f36f4101150 2024-08-18T23:43:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.204+0000 7f36f976a640 1 -- 172.21.15.78:0/1639915145 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f41a3b70 con 0x7f36f4101150 2024-08-18T23:43:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.207+0000 7f36f0ff9640 1 -- 172.21.15.78:0/1639915145 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f36dc0109d0 con 0x7f36f4101150 2024-08-18T23:43:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.207+0000 7f36fbb68740 1 Processor -- start 2024-08-18T23:43:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.207+0000 7f36fbb68740 1 -- start start 2024-08-18T23:43:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.207+0000 7f36fbb68740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6470 0x5649109b6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36fbb68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6d90 0x5649109c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36fbb68740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109c3a20 0x5649109c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36fbb68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56491091b710 con 0x5649109b6d90 2024-08-18T23:43:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36fbb68740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5649108b3770 con 0x5649109b6470 2024-08-18T23:43:51.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36fbb68740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5649109176d0 con 0x5649109c3a20 2024-08-18T23:43:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6d90 0x5649109c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6d90 0x5649109c34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50498/0 (socket says 172.21.15.78:50498) 2024-08-18T23:43:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36f2ffd640 1 -- 172.21.15.78:0/2058880669 learned_addr learned my addr 172.21.15.78:0/2058880669 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36f37fe640 1 --2- 172.21.15.78:0/2058880669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6470 0x5649109b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36ebfff640 1 --2- 172.21.15.78:0/2058880669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109c3a20 0x5649109c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36f2ffd640 1 -- 172.21.15.78:0/2058880669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109c3a20 msgr2=0x5649109c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36f2ffd640 1 --2- 172.21.15.78:0/2058880669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109c3a20 0x5649109c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36f2ffd640 1 -- 172.21.15.78:0/2058880669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6470 msgr2=0x5649109b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36f2ffd640 1 --2- 172.21.15.78:0/2058880669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6470 0x5649109b6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36f2ffd640 1 -- 172.21.15.78:0/2058880669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649108ba3e0 con 0x5649109b6d90 2024-08-18T23:43:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36ebfff640 1 --2- 172.21.15.78:0/2058880669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109c3a20 0x5649109c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:43:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36f0ff9640 1 --2- 172.21.15.78:0/1639915145 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f36e00a2b80 0x7f36e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36f2ffd640 1 --2- 172.21.15.78:0/2058880669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6d90 0x5649109c34e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009930 tx=0x7f36e4031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859715a1edc07b67 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36f37fe640 1 --2- 172.21.15.78:0/1639915145 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f36e00a2b80 0x7f36e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36f0ff9640 1 -- 172.21.15.78:0/1639915145 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f36dc0cef90 con 0x7f36f4101150 2024-08-18T23:43:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.208+0000 7f36e97fa640 1 -- 172.21.15.78:0/2058880669 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e4040070 con 0x5649109b6d90 2024-08-18T23:43:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.209+0000 7f36e97fa640 1 -- 172.21.15.78:0/2058880669 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36e4005020 con 0x5649109b6d90 2024-08-18T23:43:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.209+0000 7f36e97fa640 1 -- 172.21.15.78:0/2058880669 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e4035040 con 0x5649109b6d90 2024-08-18T23:43:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.209+0000 7f36fbb68740 1 -- 172.21.15.78:0/2058880669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6d90 msgr2=0x5649109c34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.209+0000 7f36f37fe640 1 --2- 172.21.15.78:0/1639915145 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f36e00a2b80 0x7f36e00a5040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f36ec007f00 tx=0x7f36ec007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.209+0000 7f36fbb68740 1 --2- 172.21.15.78:0/2058880669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6d90 0x5649109c34e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009930 tx=0x7f36e4031630 comp rx=0 tx=0).stop 2024-08-18T23:43:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.209+0000 7f36fbb68740 1 -- 172.21.15.78:0/2058880669 shutdown_connections 2024-08-18T23:43:51.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.209+0000 7f36fbb68740 1 --2- 172.21.15.78:0/2058880669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109c3a20 0x5649109c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.209+0000 7f36fbb68740 1 --2- 172.21.15.78:0/2058880669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6d90 0x5649109c34e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.209+0000 7f36fbb68740 1 --2- 172.21.15.78:0/2058880669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6470 0x5649109b6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.209+0000 7f36fbb68740 1 -- 172.21.15.78:0/2058880669 >> 172.21.15.78:0/2058880669 conn(0x5649109b2bc0 msgr2=0x5649109b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.209+0000 7f36fbb68740 1 -- 172.21.15.78:0/2058880669 shutdown_connections 2024-08-18T23:43:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.209+0000 7f36fbb68740 1 -- 172.21.15.78:0/2058880669 wait complete. 2024-08-18T23:43:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36fbb68740 1 -- start start 2024-08-18T23:43:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36fbb68740 1 Processor -- start 2024-08-18T23:43:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36fbb68740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109b6470 0x5649109c08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:51.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36fbb68740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6d90 0x5649109c0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:51.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36fbb68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109c3a20 0x5649109c1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:51.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36fbb68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649109c6590 con 0x5649109c3a20 2024-08-18T23:43:51.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36fbb68740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5649108d6970 con 0x5649109b6d90 2024-08-18T23:43:51.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36fbb68740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5649109c6710 con 0x5649109b6470 2024-08-18T23:43:51.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36fbb68740 10 client.0 _ll_register_callbacks cb 0x56491062a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:43:51.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36f37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109c3a20 0x5649109c1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:51.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36f37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109c3a20 0x5649109c1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50500/0 (socket says 172.21.15.78:50500) 2024-08-18T23:43:51.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36f37fe640 1 -- 172.21.15.78:0/3757366070 learned_addr learned my addr 172.21.15.78:0/3757366070 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:51.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36ebfff640 1 --2- 172.21.15.78:0/3757366070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6d90 0x5649109c0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:51.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36f2ffd640 1 --2- 172.21.15.78:0/3757366070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109b6470 0x5649109c08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:51.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36f37fe640 1 -- 172.21.15.78:0/3757366070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109b6470 msgr2=0x5649109c08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:51.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36f37fe640 1 --2- 172.21.15.78:0/3757366070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109b6470 0x5649109c08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36f37fe640 1 -- 172.21.15.78:0/3757366070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6d90 msgr2=0x5649109c0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:51.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36f37fe640 1 --2- 172.21.15.78:0/3757366070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6d90 0x5649109c0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36f37fe640 1 -- 172.21.15.78:0/3757366070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56491068d2b0 con 0x5649109c3a20 2024-08-18T23:43:51.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.210+0000 7f36f2ffd640 1 --2- 172.21.15.78:0/3757366070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109b6470 0x5649109c08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:43:51.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.211+0000 7f36f37fe640 1 --2- 172.21.15.78:0/3757366070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109c3a20 0x5649109c1360 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f36ec01ebf0 tx=0x7f36ec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3cdfcffa2177f3 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:51.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.211+0000 7f36da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:43:51.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.211+0000 7f36da7fc640 1 -- 172.21.15.78:0/3757366070 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec00fcb0 con 0x5649109c3a20 2024-08-18T23:43:51.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.211+0000 7f36da7fc640 1 -- 172.21.15.78:0/3757366070 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f36ec016030 con 0x5649109c3a20 2024-08-18T23:43:51.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.211+0000 7f36fbb68740 1 -- 172.21.15.78:0/3757366070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36e4005020 con 0x5649109c3a20 2024-08-18T23:43:51.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.211+0000 7f36da7fc640 1 -- 172.21.15.78:0/3757366070 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f36ec031050 con 0x5649109c3a20 2024-08-18T23:43:51.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.211+0000 7f36fbb68740 10 client.15216 fetch_fsmap learned FSMap version 24 2024-08-18T23:43:51.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.211+0000 7f36fbb68740 1 -- 172.21.15.78:0/3757366070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5649109d21d0 con 0x5649109c3a20 2024-08-18T23:43:51.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.211+0000 7f36da7fc640 1 -- 172.21.15.78:0/3757366070 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36e4005020 con 0x5649109c3a20 2024-08-18T23:43:51.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.211+0000 7f36da7fc640 1 -- 172.21.15.78:0/3757366070 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f36ec026400 con 0x5649109c3a20 2024-08-18T23:43:51.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.211+0000 7f36fbb68740 10 client.15216 fetch_fsmap finished waiting for FSMap version 24 2024-08-18T23:43:51.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.211+0000 7f36fbb68740 10 client.15216 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-18T23:43:51.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.211+0000 7f36fbb68740 20 client.15216 populate_metadata read hostname 'smithi078' 2024-08-18T23:43:51.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.211+0000 7f36fbb68740 1 --2- 172.21.15.78:0/3757366070 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x5649109d48a0 0x5649109d5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:51.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.211+0000 7f36fbb68740 4 client.15216 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:43:51.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.211+0000 7f36fbb68740 1 -- 172.21.15.78:0/3757366070 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564910928c80 con 0x5649109d48a0 2024-08-18T23:43:51.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.212+0000 7f36f2ffd640 1 --2- 172.21.15.78:0/3757366070 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x5649109d48a0 0x5649109d5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:51.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.212+0000 7f36da7fc640 10 client.15216 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:43:51.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.212+0000 7f36f2ffd640 1 --2- 172.21.15.78:0/3757366070 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x5649109d48a0 0x5649109d5d10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ddc0deb5cbc3aea server_cookie=1ca3eed3aa99e450 in_seq=0 out_seq=0 2024-08-18T23:43:51.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.213+0000 7f36da7fc640 1 -- 172.21.15.78:0/3757366070 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564910928c80 con 0x5649109d48a0 2024-08-18T23:43:51.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.213+0000 7f36da7fc640 10 client.15216 handle_command_reply: tid=0 2024-08-18T23:43:51.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.213+0000 7f36fbb68740 1 client.15216 shutdown 2024-08-18T23:43:51.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.213+0000 7f36fbb68740 1 -- 172.21.15.78:0/3757366070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109c3a20 msgr2=0x5649109c1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:51.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.213+0000 7f36fbb68740 1 --2- 172.21.15.78:0/3757366070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109c3a20 0x5649109c1360 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f36ec01ebf0 tx=0x7f36ec004ab0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.213+0000 7f36fbb68740 1 -- 172.21.15.78:0/3757366070 shutdown_connections 2024-08-18T23:43:51.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.213+0000 7f36fbb68740 1 --2- 172.21.15.78:0/3757366070 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x5649109d48a0 0x5649109d5d10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.213+0000 7f36fbb68740 1 --2- 172.21.15.78:0/3757366070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109c3a20 0x5649109c1360 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.213+0000 7f36fbb68740 1 --2- 172.21.15.78:0/3757366070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6d90 0x5649109c0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.213+0000 7f36da7fc640 0 client.15216 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:43:51.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.213+0000 7f36fbb68740 1 --2- 172.21.15.78:0/3757366070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109b6470 0x5649109c08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.213+0000 7f36fbb68740 1 -- 172.21.15.78:0/3757366070 >> 172.21.15.78:0/3757366070 conn(0x5649109b2bc0 msgr2=0x5649109b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:51.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.213+0000 7f36fbb68740 1 -- 172.21.15.78:0/3757366070 shutdown_connections 2024-08-18T23:43:51.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.213+0000 7f36fbb68740 1 -- 172.21.15.78:0/3757366070 wait complete. 2024-08-18T23:43:51.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.213+0000 7f36fbb68740 20 client.15216 trim_cache size 0 max 16384 2024-08-18T23:43:51.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.231+0000 7f36fbb68740 1 Processor -- start 2024-08-18T23:43:51.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.231+0000 7f36fbb68740 1 -- start start 2024-08-18T23:43:51.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.231+0000 7f36fbb68740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6470 0x5649109c3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:51.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.231+0000 7f36fbb68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6d90 0x5649109c01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:51.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.231+0000 7f36fbb68740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109c3a20 0x5649109bf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:51.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.231+0000 7f36fbb68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649109d4050 con 0x5649109b6d90 2024-08-18T23:43:51.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.231+0000 7f36fbb68740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5649109d4720 con 0x5649109b6470 2024-08-18T23:43:51.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.231+0000 7f36fbb68740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5649109d4df0 con 0x5649109c3a20 2024-08-18T23:43:51.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.231+0000 7f36ebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6d90 0x5649109c01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:51.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.231+0000 7f36ebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6d90 0x5649109c01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50512/0 (socket says 172.21.15.78:50512) 2024-08-18T23:43:51.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.231+0000 7f36ebfff640 1 -- 172.21.15.78:0/147340304 learned_addr learned my addr 172.21.15.78:0/147340304 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:51.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.232+0000 7f36f37fe640 1 --2- 172.21.15.78:0/147340304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109c3a20 0x5649109bf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:51.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.232+0000 7f36f2ffd640 1 --2- 172.21.15.78:0/147340304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6470 0x5649109c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:51.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.232+0000 7f36ebfff640 1 -- 172.21.15.78:0/147340304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109c3a20 msgr2=0x5649109bf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:51.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.232+0000 7f36ebfff640 1 --2- 172.21.15.78:0/147340304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109c3a20 0x5649109bf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.232+0000 7f36ebfff640 1 -- 172.21.15.78:0/147340304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6470 msgr2=0x5649109c3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.232+0000 7f36ebfff640 1 --2- 172.21.15.78:0/147340304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6470 0x5649109c3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.232+0000 7f36ebfff640 1 -- 172.21.15.78:0/147340304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649109c1c60 con 0x5649109b6d90 2024-08-18T23:43:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.232+0000 7f36ebfff640 1 --2- 172.21.15.78:0/147340304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6d90 0x5649109c01a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f36dc09b4a0 tx=0x7f36dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a517cc1be655c8fb server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.232+0000 7f36e8ff9640 1 -- 172.21.15.78:0/147340304 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc020050 con 0x5649109b6d90 2024-08-18T23:43:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.232+0000 7f36e8ff9640 1 -- 172.21.15.78:0/147340304 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36dc09f810 con 0x5649109b6d90 2024-08-18T23:43:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.232+0000 7f36e8ff9640 1 -- 172.21.15.78:0/147340304 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc0ccbc0 con 0x5649109b6d90 2024-08-18T23:43:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.233+0000 7f36fbb68740 1 -- 172.21.15.78:0/147340304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6d90 msgr2=0x5649109c01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.233+0000 7f36fbb68740 1 --2- 172.21.15.78:0/147340304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6d90 0x5649109c01a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f36dc09b4a0 tx=0x7f36dc005880 comp rx=0 tx=0).stop 2024-08-18T23:43:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.233+0000 7f36fbb68740 1 -- 172.21.15.78:0/147340304 shutdown_connections 2024-08-18T23:43:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.233+0000 7f36fbb68740 1 --2- 172.21.15.78:0/147340304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109c3a20 0x5649109bf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.233+0000 7f36fbb68740 1 --2- 172.21.15.78:0/147340304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6d90 0x5649109c01a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.233+0000 7f36fbb68740 1 --2- 172.21.15.78:0/147340304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6470 0x5649109c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.233+0000 7f36fbb68740 1 -- 172.21.15.78:0/147340304 >> 172.21.15.78:0/147340304 conn(0x5649109b2bc0 msgr2=0x5649109bf1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.233+0000 7f36fbb68740 1 -- 172.21.15.78:0/147340304 shutdown_connections 2024-08-18T23:43:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.233+0000 7f36fbb68740 1 -- 172.21.15.78:0/147340304 wait complete. 2024-08-18T23:43:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.233+0000 7f36fbb68740 1 -- start start 2024-08-18T23:43:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.233+0000 7f36fbb68740 1 Processor -- start 2024-08-18T23:43:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.233+0000 7f36fbb68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6470 0x5649109de790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36fbb68740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6d90 0x5649109d7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36fbb68740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109c3a20 0x5649109d80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:51.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36fbb68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649109d5b90 con 0x5649109b6470 2024-08-18T23:43:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36fbb68740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5649109d54c0 con 0x5649109b6d90 2024-08-18T23:43:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36fbb68740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5649109d6260 con 0x5649109c3a20 2024-08-18T23:43:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6470 0x5649109de790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36fbb68740 10 client.0 _ll_register_callbacks cb 0x7f36ec01e8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:43:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6470 0x5649109de790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50528/0 (socket says 172.21.15.78:50528) 2024-08-18T23:43:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36f2ffd640 1 -- 172.21.15.78:0/210899078 learned_addr learned my addr 172.21.15.78:0/210899078 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36ebfff640 1 --2- 172.21.15.78:0/210899078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6d90 0x5649109d7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36f37fe640 1 --2- 172.21.15.78:0/210899078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109c3a20 0x5649109d80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36f2ffd640 1 -- 172.21.15.78:0/210899078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109c3a20 msgr2=0x5649109d80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:51.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36f2ffd640 1 --2- 172.21.15.78:0/210899078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109c3a20 0x5649109d80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36f2ffd640 1 -- 172.21.15.78:0/210899078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6d90 msgr2=0x5649109d7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36f2ffd640 1 --2- 172.21.15.78:0/210899078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6d90 0x5649109d7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36f2ffd640 1 -- 172.21.15.78:0/210899078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649109dc2b0 con 0x5649109b6470 2024-08-18T23:43:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36f2ffd640 1 --2- 172.21.15.78:0/210899078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6470 0x5649109de790 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009900 tx=0x7f36e40315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438389c55386edcb server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:43:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36da7fc640 1 -- 172.21.15.78:0/210899078 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e4035040 con 0x5649109b6470 2024-08-18T23:43:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36da7fc640 1 -- 172.21.15.78:0/210899078 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f36e403c640 con 0x5649109b6470 2024-08-18T23:43:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36fbb68740 1 -- 172.21.15.78:0/210899078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36dc09f810 con 0x5649109b6470 2024-08-18T23:43:51.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36da7fc640 1 -- 172.21.15.78:0/210899078 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f36e405e050 con 0x5649109b6470 2024-08-18T23:43:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36fbb68740 10 client.15222 fetch_fsmap learned FSMap version 24 2024-08-18T23:43:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.234+0000 7f36fbb68740 1 -- 172.21.15.78:0/210899078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564910a3f0a0 con 0x5649109b6470 2024-08-18T23:43:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.235+0000 7f36da7fc640 1 -- 172.21.15.78:0/210899078 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36dc09f810 con 0x5649109b6470 2024-08-18T23:43:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.235+0000 7f36da7fc640 1 -- 172.21.15.78:0/210899078 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f36e404f970 con 0x5649109b6470 2024-08-18T23:43:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.235+0000 7f36fbb68740 10 client.15222 fetch_fsmap finished waiting for FSMap version 24 2024-08-18T23:43:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.235+0000 7f36fbb68740 10 client.15222 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-18T23:43:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.235+0000 7f36fbb68740 20 client.15222 populate_metadata read hostname 'smithi078' 2024-08-18T23:43:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.235+0000 7f36fbb68740 1 --2- 172.21.15.78:0/210899078 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x564910a438a0 0x564910a425b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.235+0000 7f36fbb68740 4 client.15222 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-18T23:43:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.235+0000 7f36fbb68740 1 -- 172.21.15.78:0/210899078 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "status"}) -- 0x564910923190 con 0x564910a438a0 2024-08-18T23:43:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.235+0000 7f36ebfff640 1 --2- 172.21.15.78:0/210899078 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x564910a438a0 0x564910a425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.236+0000 7f36ebfff640 1 --2- 172.21.15.78:0/210899078 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x564910a438a0 0x564910a425b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a568f4548581263 server_cookie=2af7c44b3511ce0d in_seq=0 out_seq=0 2024-08-18T23:43:51.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.236+0000 7f36da7fc640 10 client.15222 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:43:51.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.236+0000 7f36da7fc640 1 -- 172.21.15.78:0/210899078 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564910923190 con 0x564910a438a0 2024-08-18T23:43:51.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.236+0000 7f36da7fc640 10 client.15222 handle_command_reply: tid=0 2024-08-18T23:43:51.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.236+0000 7f36fbb68740 1 client.15222 shutdown 2024-08-18T23:43:51.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.236+0000 7f36fbb68740 1 -- 172.21.15.78:0/210899078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6470 msgr2=0x5649109de790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:51.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.236+0000 7f36fbb68740 1 --2- 172.21.15.78:0/210899078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6470 0x5649109de790 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009900 tx=0x7f36e40315b0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.236+0000 7f36fbb68740 1 -- 172.21.15.78:0/210899078 shutdown_connections 2024-08-18T23:43:51.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.236+0000 7f36fbb68740 1 --2- 172.21.15.78:0/210899078 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x564910a438a0 0x564910a425b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.236+0000 7f36fbb68740 1 --2- 172.21.15.78:0/210899078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649109c3a20 0x5649109d80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.236+0000 7f36fbb68740 1 --2- 172.21.15.78:0/210899078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649109b6d90 0x5649109d7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.236+0000 7f36fbb68740 1 --2- 172.21.15.78:0/210899078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649109b6470 0x5649109de790 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.236+0000 7f36fbb68740 1 -- 172.21.15.78:0/210899078 >> 172.21.15.78:0/210899078 conn(0x5649109b2bc0 msgr2=0x5649109bf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:51.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.236+0000 7f36da7fc640 0 client.15222 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:43:51.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.236+0000 7f36fbb68740 1 -- 172.21.15.78:0/210899078 shutdown_connections 2024-08-18T23:43:51.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.236+0000 7f36fbb68740 1 -- 172.21.15.78:0/210899078 wait complete. 2024-08-18T23:43:51.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.236+0000 7f36fbb68740 20 client.15222 trim_cache size 0 max 16384 2024-08-18T23:43:51.250 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:43:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "4f1128e6-5dba-11ef-bccf-c7b262605968", 2024-08-18T23:43:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 1, 2024-08-18T23:43:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24428, 2024-08-18T23:43:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-08-18T23:43:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-08-18T23:43:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-08-18T23:43:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 104.987685332, 2024-08-18T23:43:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 24, 2024-08-18T23:43:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 92, 2024-08-18T23:43:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 88, 2024-08-18T23:43:51.252 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 160.60768483000001 2024-08-18T23:43:51.252 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:43:51.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.237+0000 7f36f976a640 1 -- 172.21.15.78:0/1639915145 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f36e00a2b80 msgr2=0x7f36e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:51.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.237+0000 7f36f976a640 1 --2- 172.21.15.78:0/1639915145 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f36e00a2b80 0x7f36e00a5040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f36ec007f00 tx=0x7f36ec007520 comp rx=0 tx=0).stop 2024-08-18T23:43:51.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.237+0000 7f36f976a640 1 -- 172.21.15.78:0/1639915145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f4101150 msgr2=0x7f36f4197530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:51.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.237+0000 7f36f976a640 1 --2- 172.21.15.78:0/1639915145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f4101150 0x7f36f4197530 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00d950 tx=0x7f36dc00de10 comp rx=0 tx=0).stop 2024-08-18T23:43:51.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.237+0000 7f36f976a640 1 -- 172.21.15.78:0/1639915145 shutdown_connections 2024-08-18T23:43:51.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.237+0000 7f36f976a640 1 --2- 172.21.15.78:0/1639915145 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f36e00a2b80 0x7f36e00a5040 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.237+0000 7f36f976a640 1 --2- 172.21.15.78:0/1639915145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36f41067b0 0x7f36f419b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.237+0000 7f36f976a640 1 --2- 172.21.15.78:0/1639915145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f4101150 0x7f36f4197530 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.237+0000 7f36f976a640 1 --2- 172.21.15.78:0/1639915145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36f41007b0 0x7f36f4196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:51.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.237+0000 7f36f976a640 1 -- 172.21.15.78:0/1639915145 >> 172.21.15.78:0/1639915145 conn(0x7f36f40fc4d0 msgr2=0x7f36f40fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:51.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.237+0000 7f36f976a640 1 -- 172.21.15.78:0/1639915145 shutdown_connections 2024-08-18T23:43:51.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:51.237+0000 7f36f976a640 1 -- 172.21.15.78:0/1639915145 wait complete. 2024-08-18T23:43:51.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:51 smithi078 ceph-mon[18440]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:51.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:51 smithi123 ceph-mon[21808]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:51 smithi161 ceph-mon[22027]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:51.938 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-18T23:43:52.184 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:52.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.720+0000 7f689a957640 1 -- 172.21.15.78:0/2344052289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f689410bea0 msgr2=0x7f689410e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:52.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.720+0000 7f689a957640 1 --2- 172.21.15.78:0/2344052289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f689410bea0 0x7f689410e340 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f688c009ca0 tx=0x7f688c02a950 comp rx=0 tx=0).stop 2024-08-18T23:43:52.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.720+0000 7f689a957640 1 -- 172.21.15.78:0/2344052289 shutdown_connections 2024-08-18T23:43:52.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.720+0000 7f689a957640 1 --2- 172.21.15.78:0/2344052289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f689410bea0 0x7f689410e340 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.720+0000 7f689a957640 1 --2- 172.21.15.78:0/2344052289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6894103390 0x7f689410b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.720+0000 7f689a957640 1 --2- 172.21.15.78:0/2344052289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6894102a70 0x7f6894102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.720+0000 7f689a957640 1 -- 172.21.15.78:0/2344052289 >> 172.21.15.78:0/2344052289 conn(0x7f68940fc4d0 msgr2=0x7f68940fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:52.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.720+0000 7f689a957640 1 -- 172.21.15.78:0/2344052289 shutdown_connections 2024-08-18T23:43:52.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.720+0000 7f689a957640 1 -- 172.21.15.78:0/2344052289 wait complete. 2024-08-18T23:43:52.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.721+0000 7f689a957640 1 Processor -- start 2024-08-18T23:43:52.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.721+0000 7f689a957640 1 -- start start 2024-08-18T23:43:52.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.721+0000 7f689a957640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6894102a70 0x7f6894198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:52.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.721+0000 7f689a957640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6894103390 0x7f6894198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:52.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.721+0000 7f689a957640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f689410bea0 0x7f689419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:52.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.721+0000 7f689a957640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6894111ca0 con 0x7f6894102a70 2024-08-18T23:43:52.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.721+0000 7f689a957640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6894111b20 con 0x7f6894103390 2024-08-18T23:43:52.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.721+0000 7f689a957640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6894111e20 con 0x7f689410bea0 2024-08-18T23:43:52.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.721+0000 7f6893fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6894102a70 0x7f6894198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:52.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.721+0000 7f6893fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6894102a70 0x7f6894198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50550/0 (socket says 172.21.15.78:50550) 2024-08-18T23:43:52.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.721+0000 7f6893fff640 1 -- 172.21.15.78:0/2498808274 learned_addr learned my addr 172.21.15.78:0/2498808274 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:52.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.722+0000 7f68937fe640 1 --2- 172.21.15.78:0/2498808274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6894103390 0x7f6894198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:52.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.722+0000 7f6898ef4640 1 --2- 172.21.15.78:0/2498808274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f689410bea0 0x7f689419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:52.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.722+0000 7f6893fff640 1 -- 172.21.15.78:0/2498808274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f689410bea0 msgr2=0x7f689419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:52.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.722+0000 7f6893fff640 1 --2- 172.21.15.78:0/2498808274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f689410bea0 0x7f689419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.722+0000 7f6893fff640 1 -- 172.21.15.78:0/2498808274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6894103390 msgr2=0x7f6894198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:52.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.722+0000 7f6893fff640 1 --2- 172.21.15.78:0/2498808274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6894103390 0x7f6894198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.722+0000 7f6893fff640 1 -- 172.21.15.78:0/2498808274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689419d1d0 con 0x7f6894102a70 2024-08-18T23:43:52.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.722+0000 7f6898ef4640 1 --2- 172.21.15.78:0/2498808274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f689410bea0 0x7f689419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:43:52.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.722+0000 7f6893fff640 1 --2- 172.21.15.78:0/2498808274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6894102a70 0x7f6894198220 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f688400d770 tx=0x7f688400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:52.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.722+0000 7f68917fa640 1 -- 172.21.15.78:0/2498808274 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884017070 con 0x7f6894102a70 2024-08-18T23:43:52.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.722+0000 7f689a957640 1 -- 172.21.15.78:0/2498808274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689419d4c0 con 0x7f6894102a70 2024-08-18T23:43:52.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.722+0000 7f68917fa640 1 -- 172.21.15.78:0/2498808274 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6884005500 con 0x7f6894102a70 2024-08-18T23:43:52.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.722+0000 7f689a957640 1 -- 172.21.15.78:0/2498808274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68941a4da0 con 0x7f6894102a70 2024-08-18T23:43:52.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.722+0000 7f68917fa640 1 -- 172.21.15.78:0/2498808274 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884004e00 con 0x7f6894102a70 2024-08-18T23:43:52.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.724+0000 7f68917fa640 1 -- 172.21.15.78:0/2498808274 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6884004fa0 con 0x7f6894102a70 2024-08-18T23:43:52.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.724+0000 7f689cd55740 1 Processor -- start 2024-08-18T23:43:52.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f689cd55740 1 -- start start 2024-08-18T23:43:52.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f689cd55740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafd630 0x56277aafda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:52.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f689cd55740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56277aafdf50 0x56277ab082d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:52.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f689cd55740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 0x56277ab0ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:52.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f689cd55740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56277aa62710 con 0x56277ab08810 2024-08-18T23:43:52.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f689cd55740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56277a9fa770 con 0x56277aafd630 2024-08-18T23:43:52.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f689cd55740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56277aa5e6d0 con 0x56277aafdf50 2024-08-18T23:43:52.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f68917fa640 1 --2- 172.21.15.78:0/2498808274 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f687c0a2b80 0x7f687c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:52.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f68937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 0x56277ab0ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:52.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f68937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 0x56277ab0ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50554/0 (socket says 172.21.15.78:50554) 2024-08-18T23:43:52.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f68937fe640 1 -- 172.21.15.78:0/2489429313 learned_addr learned my addr 172.21.15.78:0/2489429313 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:52.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f6898ef4640 1 --2- 172.21.15.78:0/2498808274 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f687c0a2b80 0x7f687c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:52.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f68917fa640 1 -- 172.21.15.78:0/2498808274 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f68840cee60 con 0x7f6894102a70 2024-08-18T23:43:52.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f6898ef4640 1 --2- 172.21.15.78:0/2489429313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafd630 0x56277aafda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:52.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f6893fff640 1 --2- 172.21.15.78:0/2489429313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56277aafdf50 0x56277ab082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:52.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f68937fe640 1 -- 172.21.15.78:0/2489429313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56277aafdf50 msgr2=0x56277ab082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:52.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f68937fe640 1 --2- 172.21.15.78:0/2489429313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56277aafdf50 0x56277ab082d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f68937fe640 1 -- 172.21.15.78:0/2489429313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafd630 msgr2=0x56277aafda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:52.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f68937fe640 1 --2- 172.21.15.78:0/2489429313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafd630 0x56277aafda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f68937fe640 1 -- 172.21.15.78:0/2489429313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56277aa013e0 con 0x56277ab08810 2024-08-18T23:43:52.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.725+0000 7f68937fe640 1 --2- 172.21.15.78:0/2489429313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 0x56277ab0ac00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f688800bc20 tx=0x7f68880107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d8608077efb4b4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:52.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.726+0000 7f6898ef4640 1 --2- 172.21.15.78:0/2498808274 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f687c0a2b80 0x7f687c0a5040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f688c005e30 tx=0x7f688c005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:52.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.726+0000 7f687a7fc640 1 -- 172.21.15.78:0/2489429313 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688801c480 con 0x56277ab08810 2024-08-18T23:43:52.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.726+0000 7f687a7fc640 1 -- 172.21.15.78:0/2489429313 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f688801cb00 con 0x56277ab08810 2024-08-18T23:43:52.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.726+0000 7f687a7fc640 1 -- 172.21.15.78:0/2489429313 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888011ba0 con 0x56277ab08810 2024-08-18T23:43:52.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.726+0000 7f689cd55740 1 -- 172.21.15.78:0/2489429313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 msgr2=0x56277ab0ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:52.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.726+0000 7f689cd55740 1 --2- 172.21.15.78:0/2489429313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 0x56277ab0ac00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f688800bc20 tx=0x7f68880107a0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.726+0000 7f689cd55740 1 -- 172.21.15.78:0/2489429313 shutdown_connections 2024-08-18T23:43:52.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.726+0000 7f689cd55740 1 --2- 172.21.15.78:0/2489429313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 0x56277ab0ac00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.726+0000 7f689cd55740 1 --2- 172.21.15.78:0/2489429313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56277aafdf50 0x56277ab082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.726+0000 7f689cd55740 1 --2- 172.21.15.78:0/2489429313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafd630 0x56277aafda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.726+0000 7f689cd55740 1 -- 172.21.15.78:0/2489429313 >> 172.21.15.78:0/2489429313 conn(0x56277aaf9bc0 msgr2=0x56277aaf9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:52.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.726+0000 7f689cd55740 1 -- 172.21.15.78:0/2489429313 shutdown_connections 2024-08-18T23:43:52.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.726+0000 7f689cd55740 1 -- 172.21.15.78:0/2489429313 wait complete. 2024-08-18T23:43:52.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.727+0000 7f689cd55740 1 -- start start 2024-08-18T23:43:52.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.727+0000 7f689cd55740 1 Processor -- start 2024-08-18T23:43:52.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.727+0000 7f689cd55740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56277aafd630 0x56277ab05690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:52.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.727+0000 7f689cd55740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafdf50 0x56277ab05bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:52.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.727+0000 7f689cd55740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 0x56277ab06110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:52.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.727+0000 7f689cd55740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56277ab0b3e0 con 0x56277ab08810 2024-08-18T23:43:52.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.727+0000 7f689cd55740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56277aa1d970 con 0x56277aafdf50 2024-08-18T23:43:52.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.727+0000 7f689cd55740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56277ab0b560 con 0x56277aafd630 2024-08-18T23:43:52.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.727+0000 7f689cd55740 10 client.0 _ll_register_callbacks cb 0x56277a7717b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:43:52.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.727+0000 7f6898ef4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 0x56277ab06110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:52.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.727+0000 7f6898ef4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 0x56277ab06110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50562/0 (socket says 172.21.15.78:50562) 2024-08-18T23:43:52.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.727+0000 7f6898ef4640 1 -- 172.21.15.78:0/1186808372 learned_addr learned my addr 172.21.15.78:0/1186808372 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:52.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.727+0000 7f6893fff640 1 --2- 172.21.15.78:0/1186808372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56277aafd630 0x56277ab05690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:52.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.728+0000 7f68937fe640 1 --2- 172.21.15.78:0/1186808372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafdf50 0x56277ab05bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:52.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.728+0000 7f6898ef4640 1 -- 172.21.15.78:0/1186808372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56277aafd630 msgr2=0x56277ab05690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:52.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.728+0000 7f6898ef4640 1 --2- 172.21.15.78:0/1186808372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56277aafd630 0x56277ab05690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.728+0000 7f6898ef4640 1 -- 172.21.15.78:0/1186808372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafdf50 msgr2=0x56277ab05bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:52.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.728+0000 7f6898ef4640 1 --2- 172.21.15.78:0/1186808372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafdf50 0x56277ab05bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.728+0000 7f6898ef4640 1 -- 172.21.15.78:0/1186808372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56277a7d42b0 con 0x56277ab08810 2024-08-18T23:43:52.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.728+0000 7f68937fe640 1 --2- 172.21.15.78:0/1186808372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafdf50 0x56277ab05bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:43:52.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.728+0000 7f6898ef4640 1 --2- 172.21.15.78:0/1186808372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 0x56277ab06110 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f688c003c20 tx=0x7f688c02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c034387c4b6979 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:52.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.728+0000 7f68737fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:43:52.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.728+0000 7f68737fe640 1 -- 172.21.15.78:0/1186808372 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c04e030 con 0x56277ab08810 2024-08-18T23:43:52.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.728+0000 7f68737fe640 1 -- 172.21.15.78:0/1186808372 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f688c044070 con 0x56277ab08810 2024-08-18T23:43:52.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.728+0000 7f689cd55740 1 -- 172.21.15.78:0/1186808372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f688801cb00 con 0x56277ab08810 2024-08-18T23:43:52.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.728+0000 7f68737fe640 1 -- 172.21.15.78:0/1186808372 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f688c034040 con 0x56277ab08810 2024-08-18T23:43:52.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.728+0000 7f689cd55740 10 client.15234 fetch_fsmap learned FSMap version 24 2024-08-18T23:43:52.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.728+0000 7f689cd55740 1 -- 172.21.15.78:0/1186808372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56277ab16f90 con 0x56277ab08810 2024-08-18T23:43:52.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.728+0000 7f68737fe640 1 -- 172.21.15.78:0/1186808372 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f688801cb00 con 0x56277ab08810 2024-08-18T23:43:52.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.728+0000 7f68737fe640 1 -- 172.21.15.78:0/1186808372 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f688c047050 con 0x56277ab08810 2024-08-18T23:43:52.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.729+0000 7f689cd55740 10 client.15234 fetch_fsmap finished waiting for FSMap version 24 2024-08-18T23:43:52.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.729+0000 7f689cd55740 10 client.15234 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-18T23:43:52.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.729+0000 7f689cd55740 20 client.15234 populate_metadata read hostname 'smithi078' 2024-08-18T23:43:52.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.729+0000 7f689cd55740 1 --2- 172.21.15.78:0/1186808372 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56277ab196b0 0x56277ab1ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:52.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.729+0000 7f689cd55740 4 client.15234 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:43:52.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.729+0000 7f689cd55740 1 -- 172.21.15.78:0/1186808372 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56277aa6fc80 con 0x56277ab196b0 2024-08-18T23:43:52.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.729+0000 7f6893fff640 1 --2- 172.21.15.78:0/1186808372 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56277ab196b0 0x56277ab1ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:52.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.729+0000 7f6893fff640 1 --2- 172.21.15.78:0/1186808372 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56277ab196b0 0x56277ab1ab20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a8b464a913b1ff9 server_cookie=4ecba586fc934572 in_seq=0 out_seq=0 2024-08-18T23:43:52.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.729+0000 7f68737fe640 10 client.15234 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:43:52.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.730+0000 7f68737fe640 1 -- 172.21.15.78:0/1186808372 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56277aa6fc80 con 0x56277ab196b0 2024-08-18T23:43:52.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.730+0000 7f68737fe640 10 client.15234 handle_command_reply: tid=0 2024-08-18T23:43:52.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.730+0000 7f689cd55740 1 client.15234 shutdown 2024-08-18T23:43:52.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.730+0000 7f689cd55740 1 -- 172.21.15.78:0/1186808372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 msgr2=0x56277ab06110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:52.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.730+0000 7f689cd55740 1 --2- 172.21.15.78:0/1186808372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 0x56277ab06110 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f688c003c20 tx=0x7f688c02a950 comp rx=0 tx=0).stop 2024-08-18T23:43:52.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.730+0000 7f689cd55740 1 -- 172.21.15.78:0/1186808372 shutdown_connections 2024-08-18T23:43:52.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.730+0000 7f689cd55740 1 --2- 172.21.15.78:0/1186808372 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56277ab196b0 0x56277ab1ab20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.730+0000 7f689cd55740 1 --2- 172.21.15.78:0/1186808372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 0x56277ab06110 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.730+0000 7f68737fe640 0 client.15234 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:43:52.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.730+0000 7f689cd55740 1 --2- 172.21.15.78:0/1186808372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafdf50 0x56277ab05bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.730+0000 7f689cd55740 1 --2- 172.21.15.78:0/1186808372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56277aafd630 0x56277ab05690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.730+0000 7f689cd55740 1 -- 172.21.15.78:0/1186808372 >> 172.21.15.78:0/1186808372 conn(0x56277aaf9bc0 msgr2=0x56277aaff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:52.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.731+0000 7f689cd55740 1 -- 172.21.15.78:0/1186808372 shutdown_connections 2024-08-18T23:43:52.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.731+0000 7f689cd55740 1 -- 172.21.15.78:0/1186808372 wait complete. 2024-08-18T23:43:52.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.731+0000 7f689cd55740 20 client.15234 trim_cache size 0 max 16384 2024-08-18T23:43:52.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.748+0000 7f689cd55740 1 Processor -- start 2024-08-18T23:43:52.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.748+0000 7f689cd55740 1 -- start start 2024-08-18T23:43:52.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.748+0000 7f689cd55740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafd630 0x56277ab1a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:52.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.748+0000 7f689cd55740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277aafdf50 0x56277ab04f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:52.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.748+0000 7f689cd55740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56277ab08810 0x56277ab074f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:52.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.748+0000 7f689cd55740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56277ab18e60 con 0x56277aafdf50 2024-08-18T23:43:52.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.748+0000 7f689cd55740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56277ab19530 con 0x56277aafd630 2024-08-18T23:43:52.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.748+0000 7f689cd55740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56277ab19c00 con 0x56277ab08810 2024-08-18T23:43:52.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.749+0000 7f68937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277aafdf50 0x56277ab04f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:52.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.749+0000 7f68937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277aafdf50 0x56277ab04f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50568/0 (socket says 172.21.15.78:50568) 2024-08-18T23:43:52.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.749+0000 7f68937fe640 1 -- 172.21.15.78:0/492488283 learned_addr learned my addr 172.21.15.78:0/492488283 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:52.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.749+0000 7f68937fe640 1 -- 172.21.15.78:0/492488283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56277ab08810 msgr2=0x56277ab074f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:52.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.749+0000 7f68937fe640 1 --2- 172.21.15.78:0/492488283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56277ab08810 0x56277ab074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.749+0000 7f68937fe640 1 -- 172.21.15.78:0/492488283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafd630 msgr2=0x56277ab1a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:52.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.749+0000 7f68937fe640 1 --2- 172.21.15.78:0/492488283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafd630 0x56277ab1a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.749+0000 7f68937fe640 1 -- 172.21.15.78:0/492488283 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56277aaf99f0 con 0x56277aafdf50 2024-08-18T23:43:52.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.749+0000 7f68937fe640 1 --2- 172.21.15.78:0/492488283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277aafdf50 0x56277ab04f70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x56277aafaf40 tx=0x7f6888005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c869d36415cc494 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:52.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.749+0000 7f6879ffb640 1 -- 172.21.15.78:0/492488283 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888005240 con 0x56277aafdf50 2024-08-18T23:43:52.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.749+0000 7f6879ffb640 1 -- 172.21.15.78:0/492488283 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f688802b070 con 0x56277aafdf50 2024-08-18T23:43:52.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.749+0000 7f6879ffb640 1 -- 172.21.15.78:0/492488283 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888004a80 con 0x56277aafdf50 2024-08-18T23:43:52.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.750+0000 7f689cd55740 1 -- 172.21.15.78:0/492488283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277aafdf50 msgr2=0x56277ab04f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:52.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.750+0000 7f689cd55740 1 --2- 172.21.15.78:0/492488283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277aafdf50 0x56277ab04f70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x56277aafaf40 tx=0x7f6888005900 comp rx=0 tx=0).stop 2024-08-18T23:43:52.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.750+0000 7f689cd55740 1 -- 172.21.15.78:0/492488283 shutdown_connections 2024-08-18T23:43:52.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.750+0000 7f689cd55740 1 --2- 172.21.15.78:0/492488283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56277ab08810 0x56277ab074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.750+0000 7f689cd55740 1 --2- 172.21.15.78:0/492488283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277aafdf50 0x56277ab04f70 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.750+0000 7f689cd55740 1 --2- 172.21.15.78:0/492488283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafd630 0x56277ab1a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.750+0000 7f689cd55740 1 -- 172.21.15.78:0/492488283 >> 172.21.15.78:0/492488283 conn(0x56277aaf9bc0 msgr2=0x56277ab06de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:52.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.750+0000 7f689cd55740 1 -- 172.21.15.78:0/492488283 shutdown_connections 2024-08-18T23:43:52.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.750+0000 7f689cd55740 1 -- 172.21.15.78:0/492488283 wait complete. 2024-08-18T23:43:52.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.750+0000 7f689cd55740 1 -- start start 2024-08-18T23:43:52.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.750+0000 7f689cd55740 1 Processor -- start 2024-08-18T23:43:52.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f689cd55740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafd630 0x56277ab09a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:52.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f689cd55740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56277aafdf50 0x56277ab09fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:52.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f689cd55740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 0x56277ab0a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:52.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f689cd55740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56277ab1a9a0 con 0x56277ab08810 2024-08-18T23:43:52.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f689cd55740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56277ab1a2d0 con 0x56277aafd630 2024-08-18T23:43:52.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f689cd55740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56277ab1b070 con 0x56277aafdf50 2024-08-18T23:43:52.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f689cd55740 10 client.0 _ll_register_callbacks cb 0x7f688c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:43:52.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f6898ef4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 0x56277ab0a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:52.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f6898ef4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 0x56277ab0a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50570/0 (socket says 172.21.15.78:50570) 2024-08-18T23:43:52.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f6898ef4640 1 -- 172.21.15.78:0/1998500934 learned_addr learned my addr 172.21.15.78:0/1998500934 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:52.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f6893fff640 1 --2- 172.21.15.78:0/1998500934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafd630 0x56277ab09a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:52.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f68937fe640 1 --2- 172.21.15.78:0/1998500934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56277aafdf50 0x56277ab09fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:52.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f6898ef4640 1 -- 172.21.15.78:0/1998500934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56277aafdf50 msgr2=0x56277ab09fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:52.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f6898ef4640 1 --2- 172.21.15.78:0/1998500934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56277aafdf50 0x56277ab09fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f6898ef4640 1 -- 172.21.15.78:0/1998500934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafd630 msgr2=0x56277ab09a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:43:52.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f6898ef4640 1 --2- 172.21.15.78:0/1998500934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafd630 0x56277ab09a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f6898ef4640 1 -- 172.21.15.78:0/1998500934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56277aaf5130 con 0x56277ab08810 2024-08-18T23:43:52.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f6898ef4640 1 --2- 172.21.15.78:0/1998500934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 0x56277ab0a510 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f688c02ae70 tx=0x7f688c004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0375f7fe61c5ea server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:52.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f68737fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:43:52.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f68737fe640 1 -- 172.21.15.78:0/1998500934 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c04e070 con 0x56277ab08810 2024-08-18T23:43:52.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f68737fe640 1 -- 172.21.15.78:0/1998500934 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f688c04a690 con 0x56277ab08810 2024-08-18T23:43:52.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.751+0000 7f689cd55740 1 -- 172.21.15.78:0/1998500934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f688802b070 con 0x56277ab08810 2024-08-18T23:43:52.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.752+0000 7f68737fe640 1 -- 172.21.15.78:0/1998500934 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f688c066050 con 0x56277ab08810 2024-08-18T23:43:52.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.752+0000 7f689cd55740 10 client.15240 fetch_fsmap learned FSMap version 24 2024-08-18T23:43:52.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.752+0000 7f689cd55740 1 -- 172.21.15.78:0/1998500934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56277ab83f40 con 0x56277ab08810 2024-08-18T23:43:52.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.752+0000 7f68737fe640 1 -- 172.21.15.78:0/1998500934 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f688802b070 con 0x56277ab08810 2024-08-18T23:43:52.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.752+0000 7f68737fe640 1 -- 172.21.15.78:0/1998500934 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f688c047040 con 0x56277ab08810 2024-08-18T23:43:52.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.752+0000 7f689cd55740 10 client.15240 fetch_fsmap finished waiting for FSMap version 24 2024-08-18T23:43:52.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.752+0000 7f689cd55740 10 client.15240 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-18T23:43:52.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.752+0000 7f689cd55740 20 client.15240 populate_metadata read hostname 'smithi078' 2024-08-18T23:43:52.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.752+0000 7f689cd55740 1 --2- 172.21.15.78:0/1998500934 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56277ab88ab0 0x56277ab877c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:52.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.752+0000 7f689cd55740 4 client.15240 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-18T23:43:52.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.752+0000 7f689cd55740 1 -- 172.21.15.78:0/1998500934 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "status"}) -- 0x56277aa6a190 con 0x56277ab88ab0 2024-08-18T23:43:52.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.752+0000 7f6893fff640 1 --2- 172.21.15.78:0/1998500934 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56277ab88ab0 0x56277ab877c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:52.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.753+0000 7f68737fe640 10 client.15240 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:43:52.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.753+0000 7f6893fff640 1 --2- 172.21.15.78:0/1998500934 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56277ab88ab0 0x56277ab877c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c0f49afaf24f69 server_cookie=e502ad5a917f28c in_seq=0 out_seq=0 2024-08-18T23:43:52.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.753+0000 7f68737fe640 1 -- 172.21.15.78:0/1998500934 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56277aa6a190 con 0x56277ab88ab0 2024-08-18T23:43:52.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.753+0000 7f68737fe640 10 client.15240 handle_command_reply: tid=0 2024-08-18T23:43:52.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.753+0000 7f689cd55740 1 client.15240 shutdown 2024-08-18T23:43:52.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.753+0000 7f689cd55740 1 -- 172.21.15.78:0/1998500934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 msgr2=0x56277ab0a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:52.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.753+0000 7f689cd55740 1 --2- 172.21.15.78:0/1998500934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 0x56277ab0a510 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f688c02ae70 tx=0x7f688c004640 comp rx=0 tx=0).stop 2024-08-18T23:43:52.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.753+0000 7f689cd55740 1 -- 172.21.15.78:0/1998500934 shutdown_connections 2024-08-18T23:43:52.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.753+0000 7f689cd55740 1 --2- 172.21.15.78:0/1998500934 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56277ab88ab0 0x56277ab877c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.753+0000 7f689cd55740 1 --2- 172.21.15.78:0/1998500934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56277ab08810 0x56277ab0a510 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.753+0000 7f689cd55740 1 --2- 172.21.15.78:0/1998500934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56277aafdf50 0x56277ab09fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.753+0000 7f689cd55740 1 --2- 172.21.15.78:0/1998500934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56277aafd630 0x56277ab09a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.753+0000 7f689cd55740 1 -- 172.21.15.78:0/1998500934 >> 172.21.15.78:0/1998500934 conn(0x56277aaf9bc0 msgr2=0x56277aaff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:52.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.753+0000 7f68737fe640 0 client.15240 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:43:52.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.753+0000 7f689cd55740 1 -- 172.21.15.78:0/1998500934 shutdown_connections 2024-08-18T23:43:52.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.753+0000 7f689cd55740 1 -- 172.21.15.78:0/1998500934 wait complete. 2024-08-18T23:43:52.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.753+0000 7f689cd55740 20 client.15240 trim_cache size 0 max 16384 2024-08-18T23:43:52.760 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:43:52.761 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "4f1128e6-5dba-11ef-bccf-c7b262605968", 2024-08-18T23:43:52.761 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-08-18T23:43:52.761 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24452, 2024-08-18T23:43:52.761 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-08-18T23:43:52.761 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-08-18T23:43:52.761 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-08-18T23:43:52.761 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 139.61226060499999, 2024-08-18T23:43:52.761 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 24, 2024-08-18T23:43:52.761 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 92, 2024-08-18T23:43:52.761 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 88, 2024-08-18T23:43:52.761 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 150.27524028799999 2024-08-18T23:43:52.761 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:43:52.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.754+0000 7f689a957640 1 -- 172.21.15.78:0/2498808274 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f687c0a2b80 msgr2=0x7f687c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:52.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.754+0000 7f689a957640 1 --2- 172.21.15.78:0/2498808274 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f687c0a2b80 0x7f687c0a5040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f688c005e30 tx=0x7f688c005d80 comp rx=0 tx=0).stop 2024-08-18T23:43:52.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.754+0000 7f689a957640 1 -- 172.21.15.78:0/2498808274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6894102a70 msgr2=0x7f6894198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:52.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.754+0000 7f689a957640 1 --2- 172.21.15.78:0/2498808274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6894102a70 0x7f6894198220 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f688400d770 tx=0x7f688400dc30 comp rx=0 tx=0).stop 2024-08-18T23:43:52.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.754+0000 7f689a957640 1 -- 172.21.15.78:0/2498808274 shutdown_connections 2024-08-18T23:43:52.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.754+0000 7f689a957640 1 --2- 172.21.15.78:0/2498808274 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f687c0a2b80 0x7f687c0a5040 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.754+0000 7f689a957640 1 --2- 172.21.15.78:0/2498808274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f689410bea0 0x7f689419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.754+0000 7f689a957640 1 --2- 172.21.15.78:0/2498808274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6894103390 0x7f6894198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.754+0000 7f689a957640 1 --2- 172.21.15.78:0/2498808274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6894102a70 0x7f6894198220 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:52.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.754+0000 7f689a957640 1 -- 172.21.15.78:0/2498808274 >> 172.21.15.78:0/2498808274 conn(0x7f68940fc4d0 msgr2=0x7f68940fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:52.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.754+0000 7f689a957640 1 -- 172.21.15.78:0/2498808274 shutdown_connections 2024-08-18T23:43:52.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:52.754+0000 7f689a957640 1 -- 172.21.15.78:0/2498808274 wait complete. 2024-08-18T23:43:53.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:53 smithi078 ceph-mon[18440]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:53.439 INFO:teuthology.run_tasks:Running task sequential... 2024-08-18T23:43:53.448 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-08-18T23:43:53.448 INFO:teuthology.task.sequential:In sequential, running task print... 2024-08-18T23:43:53.448 INFO:teuthology.task.print:Enabling mgr modules 2024-08-18T23:43:53.448 INFO:teuthology.run_tasks:Running task kclient... 2024-08-18T23:43:53.474 INFO:tasks.kclient:Mounting kernel clients... 2024-08-18T23:43:53.475 INFO:tasks.kclient:config is {'client.0': {}} 2024-08-18T23:43:53.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:43:53.475 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-08-18T23:43:53.491 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-08-18T23:43:53.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:43:53.492 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-18T23:43:53.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:53 smithi123 ceph-mon[21808]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:53.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:43:53.559 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link delete ceph-brx 2024-08-18T23:43:53.660 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-08-18T23:43:53.661 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-08-18T23:43:53.661 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-18T23:43:53.661 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs ls 2024-08-18T23:43:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:53 smithi161 ceph-mon[22027]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:53.913 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:43:54.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.420+0000 7f081cba2640 1 -- 172.21.15.78:0/378421964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08181007b0 msgr2=0x7f0818100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:54.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.420+0000 7f081cba2640 1 --2- 172.21.15.78:0/378421964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08181007b0 0x7f0818100c10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f080c009930 tx=0x7f080c02a420 comp rx=0 tx=0).stop 2024-08-18T23:43:54.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.420+0000 7f081cba2640 1 -- 172.21.15.78:0/378421964 shutdown_connections 2024-08-18T23:43:54.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.420+0000 7f081cba2640 1 --2- 172.21.15.78:0/378421964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0818101150 0x7f081810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:54.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.420+0000 7f081cba2640 1 --2- 172.21.15.78:0/378421964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08181007b0 0x7f0818100c10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:54.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.420+0000 7f081cba2640 1 --2- 172.21.15.78:0/378421964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08181067b0 0x7f0818106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:54.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.420+0000 7f081cba2640 1 -- 172.21.15.78:0/378421964 >> 172.21.15.78:0/378421964 conn(0x7f08180fc4d0 msgr2=0x7f08180fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:54.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.420+0000 7f081cba2640 1 -- 172.21.15.78:0/378421964 shutdown_connections 2024-08-18T23:43:54.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.420+0000 7f081cba2640 1 -- 172.21.15.78:0/378421964 wait complete. 2024-08-18T23:43:54.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.421+0000 7f081cba2640 1 Processor -- start 2024-08-18T23:43:54.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.421+0000 7f081cba2640 1 -- start start 2024-08-18T23:43:54.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.421+0000 7f081cba2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08181007b0 0x7f08181970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:54.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.421+0000 7f081cba2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0818101150 0x7f0818197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:54.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.421+0000 7f081cba2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08181067b0 0x7f081819b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:54.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.421+0000 7f081cba2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0818110bd0 con 0x7f08181067b0 2024-08-18T23:43:54.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.421+0000 7f081cba2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0818110a50 con 0x7f0818101150 2024-08-18T23:43:54.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.421+0000 7f081cba2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0818110d50 con 0x7f08181007b0 2024-08-18T23:43:54.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.421+0000 7f0816d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08181067b0 0x7f081819b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:54.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.421+0000 7f0816d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08181067b0 0x7f081819b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50590/0 (socket says 172.21.15.78:50590) 2024-08-18T23:43:54.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.421+0000 7f0816d9d640 1 -- 172.21.15.78:0/3169530695 learned_addr learned my addr 172.21.15.78:0/3169530695 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:54.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.421+0000 7f0815d9b640 1 --2- 172.21.15.78:0/3169530695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0818101150 0x7f0818197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:54.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.421+0000 7f081659c640 1 --2- 172.21.15.78:0/3169530695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08181007b0 0x7f08181970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:54.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.421+0000 7f0816d9d640 1 -- 172.21.15.78:0/3169530695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08181007b0 msgr2=0x7f08181970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:54.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.421+0000 7f0816d9d640 1 --2- 172.21.15.78:0/3169530695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08181007b0 0x7f08181970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:54.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.421+0000 7f0816d9d640 1 -- 172.21.15.78:0/3169530695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0818101150 msgr2=0x7f0818197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.421+0000 7f0816d9d640 1 --2- 172.21.15.78:0/3169530695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0818101150 0x7f0818197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.421+0000 7f0816d9d640 1 -- 172.21.15.78:0/3169530695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081819c0a0 con 0x7f08181067b0 2024-08-18T23:43:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.421+0000 7f081659c640 1 --2- 172.21.15.78:0/3169530695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08181007b0 0x7f08181970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:43:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.422+0000 7f0816d9d640 1 --2- 172.21.15.78:0/3169530695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08181067b0 0x7f081819b9c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f081000b970 tx=0x7f081000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.422+0000 7f08077fe640 1 -- 172.21.15.78:0/3169530695 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810019070 con 0x7f08181067b0 2024-08-18T23:43:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.422+0000 7f081cba2640 1 -- 172.21.15.78:0/3169530695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081819c390 con 0x7f08181067b0 2024-08-18T23:43:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.422+0000 7f08077fe640 1 -- 172.21.15.78:0/3169530695 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f081000cb40 con 0x7f08181067b0 2024-08-18T23:43:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.422+0000 7f08077fe640 1 -- 172.21.15.78:0/3169530695 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810007610 con 0x7f08181067b0 2024-08-18T23:43:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.422+0000 7f081cba2640 1 -- 172.21.15.78:0/3169530695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08181a3c70 con 0x7f08181067b0 2024-08-18T23:43:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.423+0000 7f081cba2640 1 -- 172.21.15.78:0/3169530695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0818100c10 con 0x7f08181067b0 2024-08-18T23:43:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.424+0000 7f08077fe640 1 -- 172.21.15.78:0/3169530695 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f081001c070 con 0x7f08181067b0 2024-08-18T23:43:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.424+0000 7f08077fe640 1 --2- 172.21.15.78:0/3169530695 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f08000a2b80 0x7f08000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:54.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.425+0000 7f081659c640 1 --2- 172.21.15.78:0/3169530695 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f08000a2b80 0x7f08000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.425+0000 7f08077fe640 1 -- 172.21.15.78:0/3169530695 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f08100ce920 con 0x7f08181067b0 2024-08-18T23:43:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.425+0000 7f081659c640 1 --2- 172.21.15.78:0/3169530695 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f08000a2b80 0x7f08000a5040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f08080058c0 tx=0x7f0808005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.429+0000 7f08077fe640 1 -- 172.21.15.78:0/3169530695 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f081009aea0 con 0x7f08181067b0 2024-08-18T23:43:54.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.630+0000 7f081cba2640 1 -- 172.21.15.78:0/3169530695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0818108da0 con 0x7f08181067b0 2024-08-18T23:43:54.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.630+0000 7f08077fe640 1 -- 172.21.15.78:0/3169530695 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7f081009f1a0 con 0x7f08181067b0 2024-08-18T23:43:54.631 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-08-18T23:43:54.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.632+0000 7f081cba2640 1 -- 172.21.15.78:0/3169530695 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f08000a2b80 msgr2=0x7f08000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:54.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.632+0000 7f081cba2640 1 --2- 172.21.15.78:0/3169530695 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f08000a2b80 0x7f08000a5040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f08080058c0 tx=0x7f0808005d30 comp rx=0 tx=0).stop 2024-08-18T23:43:54.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.632+0000 7f081cba2640 1 -- 172.21.15.78:0/3169530695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08181067b0 msgr2=0x7f081819b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:54.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.632+0000 7f081cba2640 1 --2- 172.21.15.78:0/3169530695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08181067b0 0x7f081819b9c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f081000b970 tx=0x7f081000be30 comp rx=0 tx=0).stop 2024-08-18T23:43:54.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.632+0000 7f081cba2640 1 -- 172.21.15.78:0/3169530695 shutdown_connections 2024-08-18T23:43:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.632+0000 7f081cba2640 1 --2- 172.21.15.78:0/3169530695 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f08000a2b80 0x7f08000a5040 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.632+0000 7f081cba2640 1 --2- 172.21.15.78:0/3169530695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08181067b0 0x7f081819b9c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.632+0000 7f081cba2640 1 --2- 172.21.15.78:0/3169530695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0818101150 0x7f0818197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.632+0000 7f081cba2640 1 --2- 172.21.15.78:0/3169530695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08181007b0 0x7f08181970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.632+0000 7f081cba2640 1 -- 172.21.15.78:0/3169530695 >> 172.21.15.78:0/3169530695 conn(0x7f08180fc4d0 msgr2=0x7f08180fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.632+0000 7f081cba2640 1 -- 172.21.15.78:0/3169530695 shutdown_connections 2024-08-18T23:43:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:54.632+0000 7f081cba2640 1 -- 172.21.15.78:0/3169530695 wait complete. 2024-08-18T23:43:55.338 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:55 smithi078 ceph-mon[18440]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-18T23:43:55.339 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:55 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3169530695' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-18T23:43:55.340 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-18T23:43:55.340 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-18T23:43:55.340 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-08-18T23:43:55.340 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-18T23:43:55.340 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-18T23:43:55.340 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-18T23:43:55.340 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-18T23:43:55.340 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-18T23:43:55.340 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-18T23:43:55.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:43:55.340 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-08-18T23:43:55.357 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-18T23:43:55.357 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-18T23:43:55.357 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-08-18T23:43:55.357 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-08-18T23:43:55.357 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-08-18T23:43:55.357 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-08-18T23:43:55.358 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-18T23:43:55.358 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-08-18T23:43:55.358 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-08-18T23:43:55.358 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-18T23:43:55.358 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-08-18T23:43:55.358 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-08-18T23:43:55.358 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-18T23:43:55.358 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-08-18T23:43:55.358 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-18T23:43:55.358 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-08-18T23:43:55.358 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-08-18T23:43:55.358 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42304sec preferred_lft 42304sec 2024-08-18T23:43:55.358 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link noprefixroute 2024-08-18T23:43:55.359 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-08-18T23:43:55.359 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-18T23:43:55.359 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-18T23:43:55.359 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-08-18T23:43:55.359 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add name ceph-brx type bridge 2024-08-18T23:43:55.359 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr flush dev ceph-brx 2024-08-18T23:43:55.359 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set ceph-brx up 2024-08-18T23:43:55.359 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-18T23:43:55.359 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-08-18T23:43:55.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-18T23:43:55.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-18T23:43:55.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:43:55.498 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-18T23:43:55.521 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-08-18T23:43:55.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:43:55.523 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-08-18T23:43:55.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:55 smithi123 ceph-mon[21808]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-18T23:43:55.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:55 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3169530695' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-18T23:43:55.579 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-08-18T23:43:55.579 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-18T23:43:55.580 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-18T23:43:55.580 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-08-18T23:43:55.580 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-18T23:43:55.580 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-18T23:43:55.580 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-08-18T23:43:55.580 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-18T23:43:55.580 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-18T23:43:55.580 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-18T23:43:55.580 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-08-18T23:43:55.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-18T23:43:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:55 smithi161 ceph-mon[22027]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-18T23:43:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:55 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3169530695' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-18T23:43:55.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-18T23:43:55.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:43:55.707 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-08-18T23:43:55.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:43:55.723 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list-id 2024-08-18T23:43:55.778 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-18T23:43:55.778 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-08-18T23:43:55.778 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-18T23:43:55.778 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-08-18T23:43:55.778 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-08-18T23:43:55.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-18T23:43:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-18T23:43:55.895 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-08-18T23:43:55.896 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-18T23:43:55.896 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-08-18T23:43:55.896 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-08-18T23:43:55.896 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-18T23:43:55.896 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-08-18T23:43:55.896 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-08-18T23:43:55.896 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-08-18T23:43:55.896 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-08-18T23:43:55.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-18T23:43:56.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-18T23:43:56.009 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-18T23:43:56.010 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-08-18T23:43:56.010 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-08-18T23:43:56.010 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set dev brx.0 master ceph-brx 2024-08-18T23:43:56.010 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-08-18T23:43:56.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-18T23:43:56.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-18T23:43:56.085 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-18T23:43:56.085 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-18T23:43:56.100 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-18T23:43:56.100 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-18T23:43:56.101 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-18T23:43:56.157 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-08-18T23:43:56.157 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-08-18T23:43:56.157 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/5f1302ca-91a6-4860-a5ba-eb68e67b3c86 2024-08-18T23:43:56.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:43:56.157 DEBUG:teuthology.orchestra.run.smithi078:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/5f1302ca-91a6-4860-a5ba-eb68e67b3c86 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-08-18T23:43:56.240 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-08-18T23:43:56.240 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-08-18T23:43:56.240 INFO:teuthology.orchestra.run.smithi078.stdout:invalid new device string format 2024-08-18T23:43:56.383 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-08-18T23:43:56.383 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-08-18T23:43:56.383 INFO:teuthology.orchestra.run.smithi078.stdout:invalid new device string format 2024-08-18T23:43:56.383 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.123:3300,172.21.15.161:3300" 2024-08-18T23:43:56.384 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.78:3300,172.21.15.123:3300,172.21.15.161:3300:/volumes/qa/sv_1/5f1302ca-91a6-4860-a5ba-eb68e67b3c86 2024-08-18T23:43:56.384 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=4f1128e6-5dba-11ef-bccf-c7b262605968" will pass to kernel 2024-08-18T23:43:56.385 INFO:teuthology.orchestra.run.smithi078.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-08-18T23:43:56.385 INFO:teuthology.orchestra.run.smithi078.stdout: You just mounted a file system that supports labels which does not 2024-08-18T23:43:56.385 INFO:teuthology.orchestra.run.smithi078.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-08-18T23:43:56.385 INFO:teuthology.orchestra.run.smithi078.stdout: applications will generate AVC messages and not be allowed access to 2024-08-18T23:43:56.385 INFO:teuthology.orchestra.run.smithi078.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-08-18T23:43:56.386 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-08-18T23:43:56.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:43:56.387 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-18T23:43:56.455 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-18T23:43:56.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-18T23:43:56.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-18T23:43:56.552 INFO:teuthology.orchestra.run.smithi078.stdout:client24760 2024-08-18T23:43:56.552 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:43:56.552 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-18T23:43:56.569 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-18T23:43:56.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-18T23:43:56.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-18T23:43:56.665 INFO:teuthology.orchestra.run.smithi078.stdout:client24760 2024-08-18T23:43:56.665 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-18T23:43:56.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-18T23:43:56.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-18T23:43:56.720 INFO:teuthology.orchestra.run.smithi078.stdout:client24760 2024-08-18T23:43:56.721 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968.client24760/status') 2024-08-18T23:43:56.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-18T23:43:56.779 INFO:teuthology.orchestra.run.smithi078.stdout:instance: client.24760 (3)172.21.15.78:0/2736733789 2024-08-18T23:43:56.779 INFO:teuthology.orchestra.run.smithi078.stdout:blocklisted: false 2024-08-18T23:43:56.779 INFO:teuthology.orchestra.run.smithi078.stderr:0+1 records in 2024-08-18T23:43:56.780 INFO:teuthology.orchestra.run.smithi078.stderr:0+1 records out 2024-08-18T23:43:56.780 INFO:teuthology.orchestra.run.smithi078.stderr:71 bytes copied, 3.8325e-05 s, 1.9 MB/s 2024-08-18T23:43:56.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-18T23:43:56.787 INFO:teuthology.run_tasks:Running task exec... 2024-08-18T23:43:56.798 INFO:teuthology.task.exec:Executing custom commands... 2024-08-18T23:43:56.798 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-08-18T23:43:56.798 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-08-18T23:43:56.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.898+0000 7f1a5cbcf640 1 -- 172.21.15.78:0/571420118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a58102320 msgr2=0x7f1a58108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:56.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.898+0000 7f1a5cbcf640 1 --2- 172.21.15.78:0/571420118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a58102320 0x7f1a58108bb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1a48009900 tx=0x7f1a4802a410 comp rx=0 tx=0).stop 2024-08-18T23:43:56.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.898+0000 7f1a5cbcf640 1 -- 172.21.15.78:0/571420118 shutdown_connections 2024-08-18T23:43:56.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.898+0000 7f1a5cbcf640 1 --2- 172.21.15.78:0/571420118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a58102320 0x7f1a58108bb0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:56.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.898+0000 7f1a5cbcf640 1 --2- 172.21.15.78:0/571420118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a58101960 0x7f1a58101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:56.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.898+0000 7f1a5cbcf640 1 --2- 172.21.15.78:0/571420118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a58100770 0x7f1a58100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:56.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.898+0000 7f1a5cbcf640 1 -- 172.21.15.78:0/571420118 >> 172.21.15.78:0/571420118 conn(0x7f1a580fbf10 msgr2=0x7f1a580fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:56.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.898+0000 7f1a5cbcf640 1 -- 172.21.15.78:0/571420118 shutdown_connections 2024-08-18T23:43:56.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.898+0000 7f1a5cbcf640 1 -- 172.21.15.78:0/571420118 wait complete. 2024-08-18T23:43:56.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.899+0000 7f1a5cbcf640 1 Processor -- start 2024-08-18T23:43:56.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.899+0000 7f1a5cbcf640 1 -- start start 2024-08-18T23:43:56.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.899+0000 7f1a5cbcf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a58100770 0x7f1a58197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:56.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.899+0000 7f1a5cbcf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a58101960 0x7f1a58197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:56.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.899+0000 7f1a5cbcf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a58102320 0x7f1a5819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:56.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.899+0000 7f1a5cbcf640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5810c3d0 con 0x7f1a58100770 2024-08-18T23:43:56.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.899+0000 7f1a5cbcf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5810c250 con 0x7f1a58102320 2024-08-18T23:43:56.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.899+0000 7f1a5cbcf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5810c550 con 0x7f1a58101960 2024-08-18T23:43:56.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.899+0000 7f1a5659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a58100770 0x7f1a58197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:56.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.899+0000 7f1a5659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a58100770 0x7f1a58197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51104/0 (socket says 172.21.15.78:51104) 2024-08-18T23:43:56.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.899+0000 7f1a5659c640 1 -- 172.21.15.78:0/450658956 learned_addr learned my addr 172.21.15.78:0/450658956 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:56.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.899+0000 7f1a55d9b640 1 --2- 172.21.15.78:0/450658956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a58101960 0x7f1a58197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:56.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.899+0000 7f1a5659c640 1 -- 172.21.15.78:0/450658956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a58101960 msgr2=0x7f1a58197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:56.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.899+0000 7f1a5659c640 1 --2- 172.21.15.78:0/450658956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a58101960 0x7f1a58197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:56.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.899+0000 7f1a5659c640 1 -- 172.21.15.78:0/450658956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a58102320 msgr2=0x7f1a5819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:56.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.899+0000 7f1a5659c640 1 --2- 172.21.15.78:0/450658956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a58102320 0x7f1a5819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:56.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.899+0000 7f1a5659c640 1 -- 172.21.15.78:0/450658956 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5819f640 con 0x7f1a58100770 2024-08-18T23:43:56.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.899+0000 7f1a55d9b640 1 --2- 172.21.15.78:0/450658956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a58101960 0x7f1a58197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:43:56.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.900+0000 7f1a5659c640 1 --2- 172.21.15.78:0/450658956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a58100770 0x7f1a58197a20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000ccb0 tx=0x7f1a5000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:56.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.900+0000 7f1a4f7fe640 1 -- 172.21.15.78:0/450658956 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50003b70 con 0x7f1a58100770 2024-08-18T23:43:56.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.900+0000 7f1a4f7fe640 1 -- 172.21.15.78:0/450658956 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a5000ae50 con 0x7f1a58100770 2024-08-18T23:43:56.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.900+0000 7f1a5cbcf640 1 -- 172.21.15.78:0/450658956 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a5819f930 con 0x7f1a58100770 2024-08-18T23:43:56.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.900+0000 7f1a4f7fe640 1 -- 172.21.15.78:0/450658956 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5000f810 con 0x7f1a58100770 2024-08-18T23:43:56.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.900+0000 7f1a5cbcf640 1 -- 172.21.15.78:0/450658956 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a5819fe80 con 0x7f1a58100770 2024-08-18T23:43:56.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.901+0000 7f1a5cbcf640 1 -- 172.21.15.78:0/450658956 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a581a0140 con 0x7f1a58100770 2024-08-18T23:43:56.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.902+0000 7f1a4f7fe640 1 -- 172.21.15.78:0/450658956 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1a5000f9b0 con 0x7f1a58100770 2024-08-18T23:43:56.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.903+0000 7f1a4f7fe640 1 --2- 172.21.15.78:0/450658956 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1a440a2c30 0x7f1a440a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:56.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.903+0000 7f1a55d9b640 1 --2- 172.21.15.78:0/450658956 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1a440a2c30 0x7f1a440a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:56.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.903+0000 7f1a4f7fe640 1 -- 172.21.15.78:0/450658956 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f1a500cf9a0 con 0x7f1a58100770 2024-08-18T23:43:56.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.903+0000 7f1a55d9b640 1 --2- 172.21.15.78:0/450658956 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1a440a2c30 0x7f1a440a50f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1a400058c0 tx=0x7f1a40005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:56.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:56.906+0000 7f1a4f7fe640 1 -- 172.21.15.78:0/450658956 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1a50014070 con 0x7f1a58100770 2024-08-18T23:43:57.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.112+0000 7f1a5cbcf640 1 -- 172.21.15.78:0/450658956 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f1a581a0310 con 0x7f1a58100770 2024-08-18T23:43:57.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.246+0000 7f1a4f7fe640 1 -- 172.21.15.78:0/450658956 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f1a5009c570 con 0x7f1a58100770 2024-08-18T23:43:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.248+0000 7f1a5cbcf640 1 -- 172.21.15.78:0/450658956 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1a440a2c30 msgr2=0x7f1a440a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.248+0000 7f1a5cbcf640 1 --2- 172.21.15.78:0/450658956 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1a440a2c30 0x7f1a440a50f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1a400058c0 tx=0x7f1a40005850 comp rx=0 tx=0).stop 2024-08-18T23:43:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.248+0000 7f1a5cbcf640 1 -- 172.21.15.78:0/450658956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a58100770 msgr2=0x7f1a58197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.248+0000 7f1a5cbcf640 1 --2- 172.21.15.78:0/450658956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a58100770 0x7f1a58197a20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000ccb0 tx=0x7f1a5000a620 comp rx=0 tx=0).stop 2024-08-18T23:43:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.248+0000 7f1a5cbcf640 1 -- 172.21.15.78:0/450658956 shutdown_connections 2024-08-18T23:43:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.248+0000 7f1a5cbcf640 1 --2- 172.21.15.78:0/450658956 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1a440a2c30 0x7f1a440a50f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.248+0000 7f1a5cbcf640 1 --2- 172.21.15.78:0/450658956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a58102320 0x7f1a5819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:57.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.248+0000 7f1a5cbcf640 1 --2- 172.21.15.78:0/450658956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a58101960 0x7f1a58197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.248+0000 7f1a5cbcf640 1 --2- 172.21.15.78:0/450658956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a58100770 0x7f1a58197a20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.248+0000 7f1a5cbcf640 1 -- 172.21.15.78:0/450658956 >> 172.21.15.78:0/450658956 conn(0x7f1a580fbf10 msgr2=0x7f1a580fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.248+0000 7f1a5cbcf640 1 -- 172.21.15.78:0/450658956 shutdown_connections 2024-08-18T23:43:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.248+0000 7f1a5cbcf640 1 -- 172.21.15.78:0/450658956 wait complete. 2024-08-18T23:43:57.261 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-08-18T23:43:57.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.371+0000 7ffaab588640 1 -- 172.21.15.78:0/1814032818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffaa4102a50 msgr2=0x7ffaa4104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:57.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.371+0000 7ffaab588640 1 --2- 172.21.15.78:0/1814032818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffaa4102a50 0x7ffaa4104ec0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ffa98009900 tx=0x7ffa9802a410 comp rx=0 tx=0).stop 2024-08-18T23:43:57.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.371+0000 7ffaab588640 1 -- 172.21.15.78:0/1814032818 shutdown_connections 2024-08-18T23:43:57.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.371+0000 7ffaab588640 1 --2- 172.21.15.78:0/1814032818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffaa4105400 0x7ffaa41058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:57.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.371+0000 7ffaab588640 1 --2- 172.21.15.78:0/1814032818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffaa4102a50 0x7ffaa4104ec0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:57.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.371+0000 7ffaab588640 1 --2- 172.21.15.78:0/1814032818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffaa4100100 0x7ffaa4102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:57.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.371+0000 7ffaab588640 1 -- 172.21.15.78:0/1814032818 >> 172.21.15.78:0/1814032818 conn(0x7ffaa40fbf10 msgr2=0x7ffaa40fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:57.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.371+0000 7ffaab588640 1 -- 172.21.15.78:0/1814032818 shutdown_connections 2024-08-18T23:43:57.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.372+0000 7ffaab588640 1 -- 172.21.15.78:0/1814032818 wait complete. 2024-08-18T23:43:57.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.372+0000 7ffaab588640 1 Processor -- start 2024-08-18T23:43:57.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.372+0000 7ffaab588640 1 -- start start 2024-08-18T23:43:57.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.372+0000 7ffaab588640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffaa4100100 0x7ffaa419e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:57.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.372+0000 7ffaab588640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffaa4102a50 0x7ffaa419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:57.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.372+0000 7ffaab588640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffaa4105400 0x7ffaa41a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:57.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.372+0000 7ffaab588640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa4113af0 con 0x7ffaa4102a50 2024-08-18T23:43:57.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.372+0000 7ffaab588640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa4113970 con 0x7ffaa4100100 2024-08-18T23:43:57.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.372+0000 7ffaab588640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa4113c70 con 0x7ffaa4105400 2024-08-18T23:43:57.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.372+0000 7ffaa8b23640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffaa4102a50 0x7ffaa419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:57.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.372+0000 7ffaa8b23640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffaa4102a50 0x7ffaa419e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51122/0 (socket says 172.21.15.78:51122) 2024-08-18T23:43:57.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.372+0000 7ffaa8b23640 1 -- 172.21.15.78:0/3496139499 learned_addr learned my addr 172.21.15.78:0/3496139499 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:57.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.372+0000 7ffaa9324640 1 --2- 172.21.15.78:0/3496139499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffaa4100100 0x7ffaa419e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:57.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.373+0000 7ffaa9b25640 1 --2- 172.21.15.78:0/3496139499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffaa4105400 0x7ffaa41a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:57.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.373+0000 7ffaa8b23640 1 -- 172.21.15.78:0/3496139499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffaa4105400 msgr2=0x7ffaa41a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:57.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.373+0000 7ffaa8b23640 1 --2- 172.21.15.78:0/3496139499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffaa4105400 0x7ffaa41a5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:57.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.373+0000 7ffaa8b23640 1 -- 172.21.15.78:0/3496139499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffaa4100100 msgr2=0x7ffaa419e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:57.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.373+0000 7ffaa8b23640 1 --2- 172.21.15.78:0/3496139499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffaa4100100 0x7ffaa419e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:57.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.373+0000 7ffaa8b23640 1 -- 172.21.15.78:0/3496139499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa41a5cd0 con 0x7ffaa4102a50 2024-08-18T23:43:57.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.373+0000 7ffaa8b23640 1 --2- 172.21.15.78:0/3496139499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffaa4102a50 0x7ffaa419e5f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ffa98009230 tx=0x7ffa98004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:57.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.373+0000 7ffaa27fc640 1 -- 172.21.15.78:0/3496139499 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9803f030 con 0x7ffaa4102a50 2024-08-18T23:43:57.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.373+0000 7ffaa27fc640 1 -- 172.21.15.78:0/3496139499 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa98003ce0 con 0x7ffaa4102a50 2024-08-18T23:43:57.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.373+0000 7ffaab588640 1 -- 172.21.15.78:0/3496139499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa41a5f60 con 0x7ffaa4102a50 2024-08-18T23:43:57.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.373+0000 7ffaa27fc640 1 -- 172.21.15.78:0/3496139499 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98035070 con 0x7ffaa4102a50 2024-08-18T23:43:57.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.373+0000 7ffaab588640 1 -- 172.21.15.78:0/3496139499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa41a63d0 con 0x7ffaa4102a50 2024-08-18T23:43:57.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.374+0000 7ffaab588640 1 -- 172.21.15.78:0/3496139499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa40fda00 con 0x7ffaa4102a50 2024-08-18T23:43:57.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.375+0000 7ffaa27fc640 1 -- 172.21.15.78:0/3496139499 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffa98042030 con 0x7ffaa4102a50 2024-08-18T23:43:57.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.376+0000 7ffaa27fc640 1 --2- 172.21.15.78:0/3496139499 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ffa8c0a2cb0 0x7ffa8c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:57.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.376+0000 7ffaa9324640 1 --2- 172.21.15.78:0/3496139499 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ffa8c0a2cb0 0x7ffa8c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:57.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.376+0000 7ffaa27fc640 1 -- 172.21.15.78:0/3496139499 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7ffa980efd20 con 0x7ffaa4102a50 2024-08-18T23:43:57.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.376+0000 7ffaa9324640 1 --2- 172.21.15.78:0/3496139499 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ffa8c0a2cb0 0x7ffa8c0a5170 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400bbe0 tx=0x7ffa9400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:57.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.380+0000 7ffaa27fc640 1 -- 172.21.15.78:0/3496139499 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ffa980bc8f0 con 0x7ffaa4102a50 2024-08-18T23:43:57.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:57 smithi123 ceph-mon[21808]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:57.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:57 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/450658956' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-08-18T23:43:57.585 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:57 smithi078 ceph-mon[18440]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:57.585 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:57 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/450658956' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-08-18T23:43:57.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.595+0000 7ffaab588640 1 -- 172.21.15.78:0/3496139499 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7ffaa4062400 con 0x7ffa8c0a2cb0 2024-08-18T23:43:57.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.611+0000 7ffaa27fc640 1 -- 172.21.15.78:0/3496139499 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7ffaa4062400 con 0x7ffa8c0a2cb0 2024-08-18T23:43:57.612 INFO:teuthology.orchestra.run.smithi078.stdout:{} 2024-08-18T23:43:57.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.613+0000 7ffaab588640 1 -- 172.21.15.78:0/3496139499 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ffa8c0a2cb0 msgr2=0x7ffa8c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:57.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.613+0000 7ffaab588640 1 --2- 172.21.15.78:0/3496139499 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ffa8c0a2cb0 0x7ffa8c0a5170 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400bbe0 tx=0x7ffa9400b390 comp rx=0 tx=0).stop 2024-08-18T23:43:57.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.613+0000 7ffaab588640 1 -- 172.21.15.78:0/3496139499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffaa4102a50 msgr2=0x7ffaa419e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:57.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.613+0000 7ffaab588640 1 --2- 172.21.15.78:0/3496139499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffaa4102a50 0x7ffaa419e5f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ffa98009230 tx=0x7ffa98004360 comp rx=0 tx=0).stop 2024-08-18T23:43:57.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.614+0000 7ffaab588640 1 -- 172.21.15.78:0/3496139499 shutdown_connections 2024-08-18T23:43:57.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.614+0000 7ffaab588640 1 --2- 172.21.15.78:0/3496139499 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ffa8c0a2cb0 0x7ffa8c0a5170 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:57.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.614+0000 7ffaab588640 1 --2- 172.21.15.78:0/3496139499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffaa4105400 0x7ffaa41a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:57.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.614+0000 7ffaab588640 1 --2- 172.21.15.78:0/3496139499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffaa4102a50 0x7ffaa419e5f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:57.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.614+0000 7ffaab588640 1 --2- 172.21.15.78:0/3496139499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffaa4100100 0x7ffaa419e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:57.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.614+0000 7ffaab588640 1 -- 172.21.15.78:0/3496139499 >> 172.21.15.78:0/3496139499 conn(0x7ffaa40fbf10 msgr2=0x7ffaa4100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:57.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.614+0000 7ffaab588640 1 -- 172.21.15.78:0/3496139499 shutdown_connections 2024-08-18T23:43:57.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:57.614+0000 7ffaab588640 1 -- 172.21.15.78:0/3496139499 wait complete. 2024-08-18T23:43:57.626 INFO:teuthology.run_tasks:Running task check-counter... 2024-08-18T23:43:58.056 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-08-18T23:43:58.326 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T23:43:58.328 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:57 smithi161 ceph-mon[22027]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:43:58.328 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:57 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/450658956' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-08-18T23:43:58.328 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:43:57 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:43:57.599+0000 7f3a6e368640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-18T23:43:58.328 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:43:57 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:43:57.599+0000 7f3a6e368640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-18T23:43:58.328 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:43:57 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:43:57.599+0000 7f3a6e368640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-18T23:43:58.328 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:43:57 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:43:57.599+0000 7f3a6e368640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-18T23:43:58.328 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:43:57 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:43:57.599+0000 7f3a6e368640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-18T23:43:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.429+0000 7fab1e817640 1 -- 172.21.15.78:0/728154257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab18101950 msgr2=0x7fab18101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.429+0000 7fab1e817640 1 --2- 172.21.15.78:0/728154257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab18101950 0x7fab18101dd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009900 tx=0x7fab0c02a410 comp rx=0 tx=0).stop 2024-08-18T23:43:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.429+0000 7fab1e817640 1 -- 172.21.15.78:0/728154257 shutdown_connections 2024-08-18T23:43:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.429+0000 7fab1e817640 1 --2- 172.21.15.78:0/728154257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab18102310 0x7fab18108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.429+0000 7fab1e817640 1 --2- 172.21.15.78:0/728154257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab18101950 0x7fab18101dd0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.429+0000 7fab1e817640 1 --2- 172.21.15.78:0/728154257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab18100760 0x7fab18100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.429+0000 7fab1e817640 1 -- 172.21.15.78:0/728154257 >> 172.21.15.78:0/728154257 conn(0x7fab180fbee0 msgr2=0x7fab180fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.429+0000 7fab1e817640 1 -- 172.21.15.78:0/728154257 shutdown_connections 2024-08-18T23:43:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.429+0000 7fab1e817640 1 -- 172.21.15.78:0/728154257 wait complete. 2024-08-18T23:43:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.430+0000 7fab1e817640 1 Processor -- start 2024-08-18T23:43:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.430+0000 7fab1e817640 1 -- start start 2024-08-18T23:43:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.430+0000 7fab1e817640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab18100760 0x7fab18196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.430+0000 7fab1e817640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab18101950 0x7fab18197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.430+0000 7fab1e817640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab18102310 0x7fab1819e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.430+0000 7fab1e817640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fab1810c3e0 con 0x7fab18100760 2024-08-18T23:43:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.430+0000 7fab1e817640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fab1810c260 con 0x7fab18101950 2024-08-18T23:43:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.430+0000 7fab1e817640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fab1810c560 con 0x7fab18102310 2024-08-18T23:43:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.430+0000 7fab17fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab18100760 0x7fab18196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.430+0000 7fab17fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab18100760 0x7fab18196ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51142/0 (socket says 172.21.15.78:51142) 2024-08-18T23:43:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.430+0000 7fab17fff640 1 -- 172.21.15.78:0/456479237 learned_addr learned my addr 172.21.15.78:0/456479237 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.430+0000 7fab177fe640 1 --2- 172.21.15.78:0/456479237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab18101950 0x7fab18197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.430+0000 7fab1cdb4640 1 --2- 172.21.15.78:0/456479237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab18102310 0x7fab1819e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.430+0000 7fab17fff640 1 -- 172.21.15.78:0/456479237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab18102310 msgr2=0x7fab1819e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.430+0000 7fab17fff640 1 --2- 172.21.15.78:0/456479237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab18102310 0x7fab1819e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.430+0000 7fab17fff640 1 -- 172.21.15.78:0/456479237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab18101950 msgr2=0x7fab18197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.430+0000 7fab17fff640 1 --2- 172.21.15.78:0/456479237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab18101950 0x7fab18197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.430+0000 7fab17fff640 1 -- 172.21.15.78:0/456479237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1819e6f0 con 0x7fab18100760 2024-08-18T23:43:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.431+0000 7fab17fff640 1 --2- 172.21.15.78:0/456479237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab18100760 0x7fab18196ad0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fab0800d950 tx=0x7fab0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.431+0000 7fab157fa640 1 -- 172.21.15.78:0/456479237 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab08017020 con 0x7fab18100760 2024-08-18T23:43:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.431+0000 7fab157fa640 1 -- 172.21.15.78:0/456479237 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab08009c30 con 0x7fab18100760 2024-08-18T23:43:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.431+0000 7fab157fa640 1 -- 172.21.15.78:0/456479237 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0800f740 con 0x7fab18100760 2024-08-18T23:43:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.431+0000 7fab1e817640 1 -- 172.21.15.78:0/456479237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab1819e9e0 con 0x7fab18100760 2024-08-18T23:43:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.431+0000 7fab1e817640 1 -- 172.21.15.78:0/456479237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1819ef00 con 0x7fab18100760 2024-08-18T23:43:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.432+0000 7fab1e817640 1 -- 172.21.15.78:0/456479237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab18107ba0 con 0x7fab18100760 2024-08-18T23:43:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.433+0000 7fab157fa640 1 -- 172.21.15.78:0/456479237 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fab0801a020 con 0x7fab18100760 2024-08-18T23:43:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.434+0000 7fab157fa640 1 --2- 172.21.15.78:0/456479237 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fab000a2c20 0x7fab000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.434+0000 7fab177fe640 1 --2- 172.21.15.78:0/456479237 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fab000a2c20 0x7fab000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.434+0000 7fab157fa640 1 -- 172.21.15.78:0/456479237 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fab080cf200 con 0x7fab18100760 2024-08-18T23:43:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.434+0000 7fab177fe640 1 --2- 172.21.15.78:0/456479237 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fab000a2c20 0x7fab000a50e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fab0c000c00 tx=0x7fab0c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:58.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.437+0000 7fab157fa640 1 -- 172.21.15.78:0/456479237 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fab08021070 con 0x7fab18100760 2024-08-18T23:43:58.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:58 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/450658956' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-08-18T23:43:58.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:58 smithi123 ceph-mon[21808]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:43:58.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.605+0000 7fab1e817640 1 -- 172.21.15.78:0/456479237 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab1802d5e0 con 0x7fab18100760 2024-08-18T23:43:58.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.606+0000 7fab157fa640 1 -- 172.21.15.78:0/456479237 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24747 (secure 0 0 0) 0x7fab1802d5e0 con 0x7fab18100760 2024-08-18T23:43:58.608 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:43:58.608 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":92,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","created":"2024-08-18T23:34:33.867246+0000","modified":"2024-08-18T23:43:24.291494+0000","last_up_change":"2024-08-18T23:40:42.200381+0000","last_in_change":"2024-08-18T23:40:22.425813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T23:37:22.338211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T23:41:28.920802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T23:41:31.026094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T23:41:37.403238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4de1c78-6ebc-49a9-b9c5-85d43535fab7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6803","nonce":3639645311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6805","nonce":3639645311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6809","nonce":3639645311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6807","nonce":3639645311}]},"public_addr":"172.21.15.78:6803/3639645311","cluster_addr":"172.21.15.78:6805/3639645311","heartbeat_back_addr":"172.21.15.78:6809/3639645311","heartbeat_front_addr":"172.21.15.78:6807/3639645311","state":["exists","up"]},{"osd":1,"uuid":"4f8e0c3d-472b-4230-ad23-2b0a34861251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6801","nonce":4199446368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6803","nonce":4199446368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6807","nonce":4199446368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6805","nonce":4199446368}]},"public_addr":"172.21.15.123:6801/4199446368","cluster_addr":"172.21.15.123:6803/4199446368","heartbeat_back_addr":"172.21.15.123:6807/4199446368","heartbeat_front_addr":"172.21.15.123:6805/4199446368","state":["exists","up"]},{"osd":2,"uuid":"27ef3314-bb6f-413d-be05-e10a105dec9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6801","nonce":2582810057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6803","nonce":2582810057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6807","nonce":2582810057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6805","nonce":2582810057}]},"public_addr":"172.21.15.161:6801/2582810057","cluster_addr":"172.21.15.161:6803/2582810057","heartbeat_back_addr":"172.21.15.161:6807/2582810057","heartbeat_front_addr":"172.21.15.161:6805/2582810057","state":["exists","up"]},{"osd":3,"uuid":"e9e52662-5ef9-4d70-bdd7-2b92165822ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6811","nonce":2697347521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6813","nonce":2697347521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6817","nonce":2697347521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6815","nonce":2697347521}]},"public_addr":"172.21.15.78:6811/2697347521","cluster_addr":"172.21.15.78:6813/2697347521","heartbeat_back_addr":"172.21.15.78:6817/2697347521","heartbeat_front_addr":"172.21.15.78:6815/2697347521","state":["exists","up"]},{"osd":4,"uuid":"5ed26775-79e3-4b24-9557-69fcf5e8e824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6809","nonce":1281511569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6811","nonce":1281511569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6815","nonce":1281511569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6813","nonce":1281511569}]},"public_addr":"172.21.15.123:6809/1281511569","cluster_addr":"172.21.15.123:6811/1281511569","heartbeat_back_addr":"172.21.15.123:6815/1281511569","heartbeat_front_addr":"172.21.15.123:6813/1281511569","state":["exists","up"]},{"osd":5,"uuid":"a9793b1d-5613-48a4-938e-adb718f4fa4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6809","nonce":2751796117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6811","nonce":2751796117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6815","nonce":2751796117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6813","nonce":2751796117}]},"public_addr":"172.21.15.161:6809/2751796117","cluster_addr":"172.21.15.161:6811/2751796117","heartbeat_back_addr":"172.21.15.161:6815/2751796117","heartbeat_front_addr":"172.21.15.161:6813/2751796117","state":["exists","up"]},{"osd":6,"uuid":"5dbc5e9c-a14f-4b71-910a-1e6d4827bb32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6819","nonce":95383223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6821","nonce":95383223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6825","nonce":95383223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6823","nonce":95383223}]},"public_addr":"172.21.15.78:6819/95383223","cluster_addr":"172.21.15.78:6821/95383223","heartbeat_back_addr":"172.21.15.78:6825/95383223","heartbeat_front_addr":"172.21.15.78:6823/95383223","state":["exists","up"]},{"osd":7,"uuid":"26fb7b69-131f-42ee-b745-6fe8b4f65391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6817","nonce":3440305156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6819","nonce":3440305156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6823","nonce":3440305156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6821","nonce":3440305156}]},"public_addr":"172.21.15.123:6817/3440305156","cluster_addr":"172.21.15.123:6819/3440305156","heartbeat_back_addr":"172.21.15.123:6823/3440305156","heartbeat_front_addr":"172.21.15.123:6821/3440305156","state":["exists","up"]},{"osd":8,"uuid":"e267bc50-0c29-4d7b-b368-b1621e29b2df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6817","nonce":1485008778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6819","nonce":1485008778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6823","nonce":1485008778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6821","nonce":1485008778}]},"public_addr":"172.21.15.161:6817/1485008778","cluster_addr":"172.21.15.161:6819/1485008778","heartbeat_back_addr":"172.21.15.161:6823/1485008778","heartbeat_front_addr":"172.21.15.161:6821/1485008778","state":["exists","up"]},{"osd":9,"uuid":"b7654818-b767-4fe6-8211-346297895f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6827","nonce":2884475661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6829","nonce":2884475661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6833","nonce":2884475661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6831","nonce":2884475661}]},"public_addr":"172.21.15.78:6827/2884475661","cluster_addr":"172.21.15.78:6829/2884475661","heartbeat_back_addr":"172.21.15.78:6833/2884475661","heartbeat_front_addr":"172.21.15.78:6831/2884475661","state":["exists","up"]},{"osd":10,"uuid":"59aa32ff-510e-4b52-8170-08a34a0b6b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6825","nonce":122226286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6827","nonce":122226286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6831","nonce":122226286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6829","nonce":122226286}]},"public_addr":"172.21.15.123:6825/122226286","cluster_addr":"172.21.15.123:6827/122226286","heartbeat_back_addr":"172.21.15.123:6831/122226286","heartbeat_front_addr":"172.21.15.123:6829/122226286","state":["exists","up"]},{"osd":11,"uuid":"67bacab4-d078-4c5c-b4ff-9f6a8a00f507","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6825","nonce":2037560985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6827","nonce":2037560985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6831","nonce":2037560985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6829","nonce":2037560985}]},"public_addr":"172.21.15.161:6825/2037560985","cluster_addr":"172.21.15.161:6827/2037560985","heartbeat_back_addr":"172.21.15.161:6831/2037560985","heartbeat_front_addr":"172.21.15.161:6829/2037560985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:36.826044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:57.688421+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:18.297424+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:40.355215+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:02.167902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:23.968759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:46.254083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:08.522038+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:31.028119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:54.119807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:16.847034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:39.607886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6800/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3040499240":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6801/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:6800/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/757409165":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/2562072965":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/636184437":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/1570708294":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3137806628":"2024-08-19T23:34:59.021165+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T23:43:58.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.608+0000 7fab1e817640 1 -- 172.21.15.78:0/456479237 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fab000a2c20 msgr2=0x7fab000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:58.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.608+0000 7fab1e817640 1 --2- 172.21.15.78:0/456479237 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fab000a2c20 0x7fab000a50e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fab0c000c00 tx=0x7fab0c0315b0 comp rx=0 tx=0).stop 2024-08-18T23:43:58.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.608+0000 7fab1e817640 1 -- 172.21.15.78:0/456479237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab18100760 msgr2=0x7fab18196ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:58.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.608+0000 7fab1e817640 1 --2- 172.21.15.78:0/456479237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab18100760 0x7fab18196ad0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fab0800d950 tx=0x7fab0800de10 comp rx=0 tx=0).stop 2024-08-18T23:43:58.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.609+0000 7fab1e817640 1 -- 172.21.15.78:0/456479237 shutdown_connections 2024-08-18T23:43:58.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.609+0000 7fab1e817640 1 --2- 172.21.15.78:0/456479237 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fab000a2c20 0x7fab000a50e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:58.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.609+0000 7fab1e817640 1 --2- 172.21.15.78:0/456479237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab18102310 0x7fab1819e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:58.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.609+0000 7fab1e817640 1 --2- 172.21.15.78:0/456479237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab18101950 0x7fab18197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:58.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.609+0000 7fab1e817640 1 --2- 172.21.15.78:0/456479237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab18100760 0x7fab18196ad0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:58.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.609+0000 7fab1e817640 1 -- 172.21.15.78:0/456479237 >> 172.21.15.78:0/456479237 conn(0x7fab180fbee0 msgr2=0x7fab18104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:58.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.609+0000 7fab1e817640 1 -- 172.21.15.78:0/456479237 shutdown_connections 2024-08-18T23:43:58.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.609+0000 7fab1e817640 1 -- 172.21.15.78:0/456479237 wait complete. 2024-08-18T23:43:58.620 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:58 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/450658956' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-08-18T23:43:58.620 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:58 smithi078 ceph-mon[18440]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:43:58.621 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-18T23:37:22.338211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-18T23:41:28.920802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-18T23:41:31.026094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-08-18T23:41:37.403238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-18T23:43:58.622 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-18T23:43:58.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:58 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/450658956' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-08-18T23:43:58.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:58 smithi161 ceph-mon[22027]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:43:58.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.725+0000 7f93c3b3d640 1 -- 172.21.15.78:0/142964266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93bc101990 msgr2=0x7f93bc101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:58.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.725+0000 7f93c3b3d640 1 --2- 172.21.15.78:0/142964266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93bc101990 0x7f93bc101e10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009ca0 tx=0x7f93b002a950 comp rx=0 tx=0).stop 2024-08-18T23:43:58.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.725+0000 7f93c3b3d640 1 -- 172.21.15.78:0/142964266 shutdown_connections 2024-08-18T23:43:58.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.725+0000 7f93c3b3d640 1 --2- 172.21.15.78:0/142964266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93bc102350 0x7f93bc108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:58.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.725+0000 7f93c3b3d640 1 --2- 172.21.15.78:0/142964266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93bc101990 0x7f93bc101e10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:58.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.725+0000 7f93c3b3d640 1 --2- 172.21.15.78:0/142964266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93bc100c80 0x7f93bc101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:58.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.725+0000 7f93c3b3d640 1 -- 172.21.15.78:0/142964266 >> 172.21.15.78:0/142964266 conn(0x7f93bc0fb680 msgr2=0x7f93bc0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:58.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.725+0000 7f93c3b3d640 1 -- 172.21.15.78:0/142964266 shutdown_connections 2024-08-18T23:43:58.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.725+0000 7f93c3b3d640 1 -- 172.21.15.78:0/142964266 wait complete. 2024-08-18T23:43:58.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.725+0000 7f93c3b3d640 1 Processor -- start 2024-08-18T23:43:58.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.725+0000 7f93c3b3d640 1 -- start start 2024-08-18T23:43:58.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.725+0000 7f93c3b3d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93bc100c80 0x7f93bc100340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:58.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.725+0000 7f93c3b3d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93bc101990 0x7f93bc19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:58.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.725+0000 7f93c3b3d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93bc102350 0x7f93bc19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:58.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.725+0000 7f93c3b3d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc10c410 con 0x7f93bc100c80 2024-08-18T23:43:58.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.725+0000 7f93c3b3d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc10c290 con 0x7f93bc101990 2024-08-18T23:43:58.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.725+0000 7f93c3b3d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc10c590 con 0x7f93bc102350 2024-08-18T23:43:58.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.726+0000 7f93c18d9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93bc100c80 0x7f93bc100340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:58.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.726+0000 7f93c18d9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93bc100c80 0x7f93bc100340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51158/0 (socket says 172.21.15.78:51158) 2024-08-18T23:43:58.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.726+0000 7f93c18d9640 1 -- 172.21.15.78:0/1260008527 learned_addr learned my addr 172.21.15.78:0/1260008527 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:58.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.726+0000 7f93c10d8640 1 --2- 172.21.15.78:0/1260008527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93bc101990 0x7f93bc19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:58.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.726+0000 7f93c20da640 1 --2- 172.21.15.78:0/1260008527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93bc102350 0x7f93bc19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:58.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.726+0000 7f93c18d9640 1 -- 172.21.15.78:0/1260008527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93bc102350 msgr2=0x7f93bc19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:58.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.726+0000 7f93c18d9640 1 --2- 172.21.15.78:0/1260008527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93bc102350 0x7f93bc19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:58.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.726+0000 7f93c18d9640 1 -- 172.21.15.78:0/1260008527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93bc101990 msgr2=0x7f93bc19c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:58.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.726+0000 7f93c18d9640 1 --2- 172.21.15.78:0/1260008527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93bc101990 0x7f93bc19c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:58.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.726+0000 7f93c18d9640 1 -- 172.21.15.78:0/1260008527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93bc19f7c0 con 0x7f93bc100c80 2024-08-18T23:43:58.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.726+0000 7f93c10d8640 1 --2- 172.21.15.78:0/1260008527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93bc101990 0x7f93bc19c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:43:58.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.726+0000 7f93c18d9640 1 --2- 172.21.15.78:0/1260008527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93bc100c80 0x7f93bc100340 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f93b400c970 tx=0x7f93b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:58.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.726+0000 7f93baffd640 1 -- 172.21.15.78:0/1260008527 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4007c10 con 0x7f93bc100c80 2024-08-18T23:43:58.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.726+0000 7f93baffd640 1 -- 172.21.15.78:0/1260008527 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b4007db0 con 0x7f93bc100c80 2024-08-18T23:43:58.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.726+0000 7f93baffd640 1 -- 172.21.15.78:0/1260008527 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b400f940 con 0x7f93bc100c80 2024-08-18T23:43:58.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.726+0000 7f93c3b3d640 1 -- 172.21.15.78:0/1260008527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93bc19fa50 con 0x7f93bc100c80 2024-08-18T23:43:58.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.726+0000 7f93c3b3d640 1 -- 172.21.15.78:0/1260008527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93bc19ffa0 con 0x7f93bc100c80 2024-08-18T23:43:58.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.727+0000 7f93c3b3d640 1 -- 172.21.15.78:0/1260008527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93bc107b40 con 0x7f93bc100c80 2024-08-18T23:43:58.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.728+0000 7f93baffd640 1 -- 172.21.15.78:0/1260008527 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f93b400fae0 con 0x7f93bc100c80 2024-08-18T23:43:58.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.729+0000 7f93baffd640 1 --2- 172.21.15.78:0/1260008527 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f93a40a2c20 0x7f93a40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:58.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.729+0000 7f93c10d8640 1 --2- 172.21.15.78:0/1260008527 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f93a40a2c20 0x7f93a40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:58.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.729+0000 7f93baffd640 1 -- 172.21.15.78:0/1260008527 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f93b40cf9e0 con 0x7f93bc100c80 2024-08-18T23:43:58.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.730+0000 7f93c10d8640 1 --2- 172.21.15.78:0/1260008527 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f93a40a2c20 0x7f93a40a50e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f93bc1000b0 tx=0x7f93b002f940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:58.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.733+0000 7f93baffd640 1 -- 172.21.15.78:0/1260008527 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f93b409c4e0 con 0x7f93bc100c80 2024-08-18T23:43:58.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.899+0000 7f93c3b3d640 1 -- 172.21.15.78:0/1260008527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f93bc02d850 con 0x7f93bc100c80 2024-08-18T23:43:58.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.900+0000 7f93baffd640 1 -- 172.21.15.78:0/1260008527 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f93b409bde0 con 0x7f93bc100c80 2024-08-18T23:43:58.901 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-08-18T23:43:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.901+0000 7f93c3b3d640 1 -- 172.21.15.78:0/1260008527 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f93a40a2c20 msgr2=0x7f93a40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.901+0000 7f93c3b3d640 1 --2- 172.21.15.78:0/1260008527 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f93a40a2c20 0x7f93a40a50e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f93bc1000b0 tx=0x7f93b002f940 comp rx=0 tx=0).stop 2024-08-18T23:43:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.901+0000 7f93c3b3d640 1 -- 172.21.15.78:0/1260008527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93bc100c80 msgr2=0x7f93bc100340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.901+0000 7f93c3b3d640 1 --2- 172.21.15.78:0/1260008527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93bc100c80 0x7f93bc100340 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f93b400c970 tx=0x7f93b400ce30 comp rx=0 tx=0).stop 2024-08-18T23:43:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.902+0000 7f93c3b3d640 1 -- 172.21.15.78:0/1260008527 shutdown_connections 2024-08-18T23:43:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.902+0000 7f93c3b3d640 1 --2- 172.21.15.78:0/1260008527 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f93a40a2c20 0x7f93a40a50e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.902+0000 7f93c3b3d640 1 --2- 172.21.15.78:0/1260008527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93bc102350 0x7f93bc19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.902+0000 7f93c3b3d640 1 --2- 172.21.15.78:0/1260008527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93bc101990 0x7f93bc19c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.902+0000 7f93c3b3d640 1 --2- 172.21.15.78:0/1260008527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93bc100c80 0x7f93bc100340 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:58.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.902+0000 7f93c3b3d640 1 -- 172.21.15.78:0/1260008527 >> 172.21.15.78:0/1260008527 conn(0x7f93bc0fb680 msgr2=0x7f93bc104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.902+0000 7f93c3b3d640 1 -- 172.21.15.78:0/1260008527 shutdown_connections 2024-08-18T23:43:58.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:58.902+0000 7f93c3b3d640 1 -- 172.21.15.78:0/1260008527 wait complete. 2024-08-18T23:43:58.914 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-18T23:43:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.015+0000 7fb4f6455640 1 -- 172.21.15.78:0/369940682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f01033d0 msgr2=0x7fb4f0109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.015+0000 7fb4f6455640 1 --2- 172.21.15.78:0/369940682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f01033d0 0x7fb4f0109c60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009ca0 tx=0x7fb4e802a8c0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.015+0000 7fb4f6455640 1 -- 172.21.15.78:0/369940682 shutdown_connections 2024-08-18T23:43:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.015+0000 7fb4f6455640 1 --2- 172.21.15.78:0/369940682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f01033d0 0x7fb4f0109c60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.015+0000 7fb4f6455640 1 --2- 172.21.15.78:0/369940682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f0102a10 0x7fb4f0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.015+0000 7fb4f6455640 1 --2- 172.21.15.78:0/369940682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4f0101820 0x7fb4f0101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.015+0000 7fb4f6455640 1 -- 172.21.15.78:0/369940682 >> 172.21.15.78:0/369940682 conn(0x7fb4f00fcfe0 msgr2=0x7fb4f00ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.015+0000 7fb4f6455640 1 -- 172.21.15.78:0/369940682 shutdown_connections 2024-08-18T23:43:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.015+0000 7fb4f6455640 1 -- 172.21.15.78:0/369940682 wait complete. 2024-08-18T23:43:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.016+0000 7fb4f6455640 1 Processor -- start 2024-08-18T23:43:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.016+0000 7fb4f6455640 1 -- start start 2024-08-18T23:43:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.016+0000 7fb4f6455640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f0101820 0x7fb4f0197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.016+0000 7fb4f6455640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f0102a10 0x7fb4f0198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.016+0000 7fb4f6455640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4f01033d0 0x7fb4f019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.016+0000 7fb4f6455640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f010d500 con 0x7fb4f0102a10 2024-08-18T23:43:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.016+0000 7fb4f6455640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f010d380 con 0x7fb4f01033d0 2024-08-18T23:43:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.016+0000 7fb4f6455640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f010d680 con 0x7fb4f0101820 2024-08-18T23:43:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.016+0000 7fb4ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f0102a10 0x7fb4f0198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.016+0000 7fb4ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f0102a10 0x7fb4f0198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51168/0 (socket says 172.21.15.78:51168) 2024-08-18T23:43:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.017+0000 7fb4ef7fe640 1 -- 172.21.15.78:0/1416495482 learned_addr learned my addr 172.21.15.78:0/1416495482 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.017+0000 7fb4f49f2640 1 --2- 172.21.15.78:0/1416495482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4f01033d0 0x7fb4f019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.017+0000 7fb4effff640 1 --2- 172.21.15.78:0/1416495482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f0101820 0x7fb4f0197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.017+0000 7fb4ef7fe640 1 -- 172.21.15.78:0/1416495482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f0101820 msgr2=0x7fb4f0197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.017+0000 7fb4ef7fe640 1 --2- 172.21.15.78:0/1416495482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f0101820 0x7fb4f0197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.017+0000 7fb4ef7fe640 1 -- 172.21.15.78:0/1416495482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4f01033d0 msgr2=0x7fb4f019f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.017+0000 7fb4ef7fe640 1 --2- 172.21.15.78:0/1416495482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4f01033d0 0x7fb4f019f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.017+0000 7fb4ef7fe640 1 -- 172.21.15.78:0/1416495482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f019f6e0 con 0x7fb4f0102a10 2024-08-18T23:43:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.017+0000 7fb4effff640 1 --2- 172.21.15.78:0/1416495482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f0101820 0x7fb4f0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:43:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.017+0000 7fb4f49f2640 1 --2- 172.21.15.78:0/1416495482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4f01033d0 0x7fb4f019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:43:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.017+0000 7fb4ef7fe640 1 --2- 172.21.15.78:0/1416495482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f0102a10 0x7fb4f0198000 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400b4a0 tx=0x7fb4e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.017+0000 7fb4ed7fa640 1 -- 172.21.15.78:0/1416495482 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4014070 con 0x7fb4f0102a10 2024-08-18T23:43:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.017+0000 7fb4ed7fa640 1 -- 172.21.15.78:0/1416495482 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4e4010af0 con 0x7fb4f0102a10 2024-08-18T23:43:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.017+0000 7fb4f6455640 1 -- 172.21.15.78:0/1416495482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f019f9d0 con 0x7fb4f0102a10 2024-08-18T23:43:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.017+0000 7fb4ed7fa640 1 -- 172.21.15.78:0/1416495482 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4017d10 con 0x7fb4f0102a10 2024-08-18T23:43:59.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.017+0000 7fb4f6455640 1 -- 172.21.15.78:0/1416495482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f019fff0 con 0x7fb4f0102a10 2024-08-18T23:43:59.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.018+0000 7fb4f6455640 1 -- 172.21.15.78:0/1416495482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4f0108b40 con 0x7fb4f0102a10 2024-08-18T23:43:59.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.019+0000 7fb4ed7fa640 1 -- 172.21.15.78:0/1416495482 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fb4e4017070 con 0x7fb4f0102a10 2024-08-18T23:43:59.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.020+0000 7fb4ed7fa640 1 --2- 172.21.15.78:0/1416495482 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4d80a2ca0 0x7fb4d80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:59.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.020+0000 7fb4effff640 1 --2- 172.21.15.78:0/1416495482 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4d80a2ca0 0x7fb4d80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:59.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.020+0000 7fb4ed7fa640 1 -- 172.21.15.78:0/1416495482 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fb4e40ce830 con 0x7fb4f0102a10 2024-08-18T23:43:59.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.020+0000 7fb4effff640 1 --2- 172.21.15.78:0/1416495482 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4d80a2ca0 0x7fb4d80a5160 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0102870 tx=0x7fb4e0005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:59.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.024+0000 7fb4ed7fa640 1 -- 172.21.15.78:0/1416495482 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb4e409b3b0 con 0x7fb4f0102a10 2024-08-18T23:43:59.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.190+0000 7fb4f6455640 1 -- 172.21.15.78:0/1416495482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb4f002d870 con 0x7fb4f0102a10 2024-08-18T23:43:59.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.190+0000 7fb4ed7fa640 1 -- 172.21.15.78:0/1416495482 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fb4e409acb0 con 0x7fb4f0102a10 2024-08-18T23:43:59.193 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-08-18T23:43:59.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.192+0000 7fb4f6455640 1 -- 172.21.15.78:0/1416495482 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4d80a2ca0 msgr2=0x7fb4d80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:59.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.192+0000 7fb4f6455640 1 --2- 172.21.15.78:0/1416495482 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4d80a2ca0 0x7fb4d80a5160 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0102870 tx=0x7fb4e0005db0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.192+0000 7fb4f6455640 1 -- 172.21.15.78:0/1416495482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f0102a10 msgr2=0x7fb4f0198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:59.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.192+0000 7fb4f6455640 1 --2- 172.21.15.78:0/1416495482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f0102a10 0x7fb4f0198000 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400b4a0 tx=0x7fb4e400b960 comp rx=0 tx=0).stop 2024-08-18T23:43:59.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.192+0000 7fb4f6455640 1 -- 172.21.15.78:0/1416495482 shutdown_connections 2024-08-18T23:43:59.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.192+0000 7fb4f6455640 1 --2- 172.21.15.78:0/1416495482 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb4d80a2ca0 0x7fb4d80a5160 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.192+0000 7fb4f6455640 1 --2- 172.21.15.78:0/1416495482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4f01033d0 0x7fb4f019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.192+0000 7fb4f6455640 1 --2- 172.21.15.78:0/1416495482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4f0102a10 0x7fb4f0198000 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.192+0000 7fb4f6455640 1 --2- 172.21.15.78:0/1416495482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4f0101820 0x7fb4f0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.192+0000 7fb4f6455640 1 -- 172.21.15.78:0/1416495482 >> 172.21.15.78:0/1416495482 conn(0x7fb4f00fcfe0 msgr2=0x7fb4f0105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:59.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.193+0000 7fb4f6455640 1 -- 172.21.15.78:0/1416495482 shutdown_connections 2024-08-18T23:43:59.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.193+0000 7fb4f6455640 1 -- 172.21.15.78:0/1416495482 wait complete. 2024-08-18T23:43:59.206 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-18T23:43:59.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.310+0000 7f0798193640 1 -- 172.21.15.78:0/2364589968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0790102330 msgr2=0x7f0790104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:59.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.310+0000 7f0798193640 1 --2- 172.21.15.78:0/2364589968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0790102330 0x7f0790104720 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f078c009900 tx=0x7f078c02a410 comp rx=0 tx=0).stop 2024-08-18T23:43:59.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.310+0000 7f0798193640 1 -- 172.21.15.78:0/2364589968 shutdown_connections 2024-08-18T23:43:59.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.310+0000 7f0798193640 1 --2- 172.21.15.78:0/2364589968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0790107590 0x7f0790109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.310+0000 7f0798193640 1 --2- 172.21.15.78:0/2364589968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0790104c60 0x7f0790107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.310+0000 7f0798193640 1 --2- 172.21.15.78:0/2364589968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0790102330 0x7f0790104720 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.310+0000 7f0798193640 1 -- 172.21.15.78:0/2364589968 >> 172.21.15.78:0/2364589968 conn(0x7f07900fbe70 msgr2=0x7f07900fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.310+0000 7f0798193640 1 -- 172.21.15.78:0/2364589968 shutdown_connections 2024-08-18T23:43:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.310+0000 7f0798193640 1 -- 172.21.15.78:0/2364589968 wait complete. 2024-08-18T23:43:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.311+0000 7f0798193640 1 Processor -- start 2024-08-18T23:43:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.311+0000 7f0798193640 1 -- start start 2024-08-18T23:43:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.311+0000 7f0798193640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0790102330 0x7f0790101ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.311+0000 7f0798193640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0790104c60 0x7f07901002e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.311+0000 7f0798193640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0790107590 0x7f0790100820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.311+0000 7f0798193640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f079010d630 con 0x7f0790104c60 2024-08-18T23:43:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.311+0000 7f0798193640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f079010d4b0 con 0x7f0790102330 2024-08-18T23:43:59.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.311+0000 7f0798193640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f079010d7b0 con 0x7f0790107590 2024-08-18T23:43:59.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.311+0000 7f079572e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0790104c60 0x7f07901002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:59.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.311+0000 7f079572e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0790104c60 0x7f07901002e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51198/0 (socket says 172.21.15.78:51198) 2024-08-18T23:43:59.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.311+0000 7f079572e640 1 -- 172.21.15.78:0/3017532999 learned_addr learned my addr 172.21.15.78:0/3017532999 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:59.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.311+0000 7f0796730640 1 --2- 172.21.15.78:0/3017532999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0790107590 0x7f0790100820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:59.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.311+0000 7f0795f2f640 1 --2- 172.21.15.78:0/3017532999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0790102330 0x7f0790101ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:59.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.311+0000 7f079572e640 1 -- 172.21.15.78:0/3017532999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0790107590 msgr2=0x7f0790100820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:59.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.311+0000 7f079572e640 1 --2- 172.21.15.78:0/3017532999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0790107590 0x7f0790100820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.311+0000 7f079572e640 1 -- 172.21.15.78:0/3017532999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0790102330 msgr2=0x7f0790101ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:59.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.311+0000 7f079572e640 1 --2- 172.21.15.78:0/3017532999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0790102330 0x7f0790101ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.311+0000 7f079572e640 1 -- 172.21.15.78:0/3017532999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07901010b0 con 0x7f0790104c60 2024-08-18T23:43:59.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.311+0000 7f079572e640 1 --2- 172.21.15.78:0/3017532999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0790104c60 0x7f07901002e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f078000b450 tx=0x7f078000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:59.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.312+0000 7f0786ffd640 1 -- 172.21.15.78:0/3017532999 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0780015020 con 0x7f0790104c60 2024-08-18T23:43:59.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.312+0000 7f0786ffd640 1 -- 172.21.15.78:0/3017532999 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0780011d70 con 0x7f0790104c60 2024-08-18T23:43:59.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.312+0000 7f0798193640 1 -- 172.21.15.78:0/3017532999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07901a1e70 con 0x7f0790104c60 2024-08-18T23:43:59.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.312+0000 7f0786ffd640 1 -- 172.21.15.78:0/3017532999 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078000f340 con 0x7f0790104c60 2024-08-18T23:43:59.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.312+0000 7f0798193640 1 -- 172.21.15.78:0/3017532999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07901a2350 con 0x7f0790104c60 2024-08-18T23:43:59.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.313+0000 7f0798193640 1 -- 172.21.15.78:0/3017532999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07900fd960 con 0x7f0790104c60 2024-08-18T23:43:59.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.314+0000 7f0786ffd640 1 -- 172.21.15.78:0/3017532999 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f0780018020 con 0x7f0790104c60 2024-08-18T23:43:59.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.315+0000 7f0786ffd640 1 --2- 172.21.15.78:0/3017532999 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f077c0a2c20 0x7f077c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:59.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.315+0000 7f0795f2f640 1 --2- 172.21.15.78:0/3017532999 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f077c0a2c20 0x7f077c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:59.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.315+0000 7f0786ffd640 1 -- 172.21.15.78:0/3017532999 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f07800cef90 con 0x7f0790104c60 2024-08-18T23:43:59.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.315+0000 7f0795f2f640 1 --2- 172.21.15.78:0/3017532999 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f077c0a2c20 0x7f077c0a50e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f078c006010 tx=0x7f078c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:59.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.319+0000 7f0786ffd640 1 -- 172.21.15.78:0/3017532999 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f078001f020 con 0x7f0790104c60 2024-08-18T23:43:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:59 smithi078 ceph-mon[18440]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:43:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:59 smithi078 ceph-mon[18440]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:59.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:59 smithi078 ceph-mon[18440]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-08-18T23:43:59.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:59 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/456479237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:43:59.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:59 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1260008527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-18T23:43:59.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:43:59 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1416495482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-18T23:43:59.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.530+0000 7f0798193640 1 -- 172.21.15.78:0/3017532999 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f079002d7e0 con 0x7f0790104c60 2024-08-18T23:43:59.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.530+0000 7f0786ffd640 1 -- 172.21.15.78:0/3017532999 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f078000d040 con 0x7f0790104c60 2024-08-18T23:43:59.532 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-08-18T23:43:59.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.533+0000 7f0798193640 1 -- 172.21.15.78:0/3017532999 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f077c0a2c20 msgr2=0x7f077c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:59.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.533+0000 7f0798193640 1 --2- 172.21.15.78:0/3017532999 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f077c0a2c20 0x7f077c0a50e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f078c006010 tx=0x7f078c031400 comp rx=0 tx=0).stop 2024-08-18T23:43:59.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.533+0000 7f0798193640 1 -- 172.21.15.78:0/3017532999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0790104c60 msgr2=0x7f07901002e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:59.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.533+0000 7f0798193640 1 --2- 172.21.15.78:0/3017532999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0790104c60 0x7f07901002e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f078000b450 tx=0x7f078000b910 comp rx=0 tx=0).stop 2024-08-18T23:43:59.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.533+0000 7f0798193640 1 -- 172.21.15.78:0/3017532999 shutdown_connections 2024-08-18T23:43:59.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.533+0000 7f0798193640 1 --2- 172.21.15.78:0/3017532999 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f077c0a2c20 0x7f077c0a50e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.533+0000 7f0798193640 1 --2- 172.21.15.78:0/3017532999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0790107590 0x7f0790100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.533+0000 7f0798193640 1 --2- 172.21.15.78:0/3017532999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0790104c60 0x7f07901002e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.533+0000 7f0798193640 1 --2- 172.21.15.78:0/3017532999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0790102330 0x7f0790101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.533+0000 7f0798193640 1 -- 172.21.15.78:0/3017532999 >> 172.21.15.78:0/3017532999 conn(0x7f07900fbe70 msgr2=0x7f0790102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:59.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.533+0000 7f0798193640 1 -- 172.21.15.78:0/3017532999 shutdown_connections 2024-08-18T23:43:59.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.533+0000 7f0798193640 1 -- 172.21.15.78:0/3017532999 wait complete. 2024-08-18T23:43:59.547 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-08-18T23:43:59.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:59 smithi123 ceph-mon[21808]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:43:59.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:59 smithi123 ceph-mon[21808]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:59.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:59 smithi123 ceph-mon[21808]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-08-18T23:43:59.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:59 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/456479237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:43:59.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:59 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1260008527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-18T23:43:59.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:43:59 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1416495482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-18T23:43:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.651+0000 7ff661f2c640 1 -- 172.21.15.78:0/905424699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff65c104c90 msgr2=0x7ff65c107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.651+0000 7ff661f2c640 1 --2- 172.21.15.78:0/905424699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff65c104c90 0x7ff65c107080 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff650009900 tx=0x7ff65002a410 comp rx=0 tx=0).stop 2024-08-18T23:43:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.651+0000 7ff661f2c640 1 -- 172.21.15.78:0/905424699 shutdown_connections 2024-08-18T23:43:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.651+0000 7ff661f2c640 1 --2- 172.21.15.78:0/905424699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff65c1075c0 0x7ff65c1099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.651+0000 7ff661f2c640 1 --2- 172.21.15.78:0/905424699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff65c104c90 0x7ff65c107080 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.651+0000 7ff661f2c640 1 --2- 172.21.15.78:0/905424699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff65c102360 0x7ff65c104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.651+0000 7ff661f2c640 1 -- 172.21.15.78:0/905424699 >> 172.21.15.78:0/905424699 conn(0x7ff65c0fbec0 msgr2=0x7ff65c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.651+0000 7ff661f2c640 1 -- 172.21.15.78:0/905424699 shutdown_connections 2024-08-18T23:43:59.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.651+0000 7ff661f2c640 1 -- 172.21.15.78:0/905424699 wait complete. 2024-08-18T23:43:59.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.652+0000 7ff661f2c640 1 Processor -- start 2024-08-18T23:43:59.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.652+0000 7ff661f2c640 1 -- start start 2024-08-18T23:43:59.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.652+0000 7ff661f2c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff65c102360 0x7ff65c197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:59.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.652+0000 7ff661f2c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff65c104c90 0x7ff65c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:59.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.652+0000 7ff661f2c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff65c1075c0 0x7ff65c19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:59.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.652+0000 7ff661f2c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c10d4d0 con 0x7ff65c102360 2024-08-18T23:43:59.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.652+0000 7ff661f2c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c10d350 con 0x7ff65c104c90 2024-08-18T23:43:59.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.652+0000 7ff661f2c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c10d650 con 0x7ff65c1075c0 2024-08-18T23:43:59.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.652+0000 7ff65b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff65c102360 0x7ff65c197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:59.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.652+0000 7ff65b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff65c102360 0x7ff65c197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51210/0 (socket says 172.21.15.78:51210) 2024-08-18T23:43:59.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.652+0000 7ff65b7fe640 1 -- 172.21.15.78:0/2229533889 learned_addr learned my addr 172.21.15.78:0/2229533889 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:43:59.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.652+0000 7ff65affd640 1 --2- 172.21.15.78:0/2229533889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff65c104c90 0x7ff65c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.652+0000 7ff65b7fe640 1 -- 172.21.15.78:0/2229533889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff65c1075c0 msgr2=0x7ff65c19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.652+0000 7ff65b7fe640 1 --2- 172.21.15.78:0/2229533889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff65c1075c0 0x7ff65c19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.652+0000 7ff65b7fe640 1 -- 172.21.15.78:0/2229533889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff65c104c90 msgr2=0x7ff65c198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.652+0000 7ff65b7fe640 1 --2- 172.21.15.78:0/2229533889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff65c104c90 0x7ff65c198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.652+0000 7ff65b7fe640 1 -- 172.21.15.78:0/2229533889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff65c19f800 con 0x7ff65c102360 2024-08-18T23:43:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.653+0000 7ff65b7fe640 1 --2- 172.21.15.78:0/2229533889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff65c102360 0x7ff65c197be0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff64c00d770 tx=0x7ff64c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.653+0000 7ff658ff9640 1 -- 172.21.15.78:0/2229533889 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c017070 con 0x7ff65c102360 2024-08-18T23:43:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.653+0000 7ff658ff9640 1 -- 172.21.15.78:0/2229533889 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff64c005500 con 0x7ff65c102360 2024-08-18T23:43:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.653+0000 7ff661f2c640 1 -- 172.21.15.78:0/2229533889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff65c19faf0 con 0x7ff65c102360 2024-08-18T23:43:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.653+0000 7ff658ff9640 1 -- 172.21.15.78:0/2229533889 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c004e00 con 0x7ff65c102360 2024-08-18T23:43:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.653+0000 7ff661f2c640 1 -- 172.21.15.78:0/2229533889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff65c19fff0 con 0x7ff65c102360 2024-08-18T23:43:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.654+0000 7ff661f2c640 1 -- 172.21.15.78:0/2229533889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff65c0fd9b0 con 0x7ff65c102360 2024-08-18T23:43:59.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.655+0000 7ff658ff9640 1 -- 172.21.15.78:0/2229533889 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7ff64c004fa0 con 0x7ff65c102360 2024-08-18T23:43:59.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.656+0000 7ff658ff9640 1 --2- 172.21.15.78:0/2229533889 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff6440a2c20 0x7ff6440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:43:59.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.656+0000 7ff65affd640 1 --2- 172.21.15.78:0/2229533889 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff6440a2c20 0x7ff6440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:43:59.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.656+0000 7ff658ff9640 1 -- 172.21.15.78:0/2229533889 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7ff64c0cf040 con 0x7ff65c102360 2024-08-18T23:43:59.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.656+0000 7ff65affd640 1 --2- 172.21.15.78:0/2229533889 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff6440a2c20 0x7ff6440a50e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff650006010 tx=0x7ff650000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:43:59.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.660+0000 7ff658ff9640 1 -- 172.21.15.78:0/2229533889 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff64c09bab0 con 0x7ff65c102360 2024-08-18T23:43:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:59 smithi161 ceph-mon[22027]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-08-18T23:43:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:59 smithi161 ceph-mon[22027]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:43:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:59 smithi161 ceph-mon[22027]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-08-18T23:43:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:59 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/456479237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:43:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:59 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1260008527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-18T23:43:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:43:59 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1416495482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-18T23:43:59.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.834+0000 7ff661f2c640 1 -- 172.21.15.78:0/2229533889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff65c02d820 con 0x7ff65c102360 2024-08-18T23:43:59.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.835+0000 7ff658ff9640 1 -- 172.21.15.78:0/2229533889 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7ff64c09b3b0 con 0x7ff65c102360 2024-08-18T23:43:59.837 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-08-18T23:43:59.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.837+0000 7ff661f2c640 1 -- 172.21.15.78:0/2229533889 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff6440a2c20 msgr2=0x7ff6440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:59.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.837+0000 7ff661f2c640 1 --2- 172.21.15.78:0/2229533889 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff6440a2c20 0x7ff6440a50e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff650006010 tx=0x7ff650000bc0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.837+0000 7ff661f2c640 1 -- 172.21.15.78:0/2229533889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff65c102360 msgr2=0x7ff65c197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:43:59.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.837+0000 7ff661f2c640 1 --2- 172.21.15.78:0/2229533889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff65c102360 0x7ff65c197be0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff64c00d770 tx=0x7ff64c00dc30 comp rx=0 tx=0).stop 2024-08-18T23:43:59.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.837+0000 7ff661f2c640 1 -- 172.21.15.78:0/2229533889 shutdown_connections 2024-08-18T23:43:59.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.837+0000 7ff661f2c640 1 --2- 172.21.15.78:0/2229533889 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff6440a2c20 0x7ff6440a50e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.837+0000 7ff661f2c640 1 --2- 172.21.15.78:0/2229533889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff65c1075c0 0x7ff65c19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.837+0000 7ff661f2c640 1 --2- 172.21.15.78:0/2229533889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff65c104c90 0x7ff65c198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.837+0000 7ff661f2c640 1 --2- 172.21.15.78:0/2229533889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff65c102360 0x7ff65c197be0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:43:59.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.837+0000 7ff661f2c640 1 -- 172.21.15.78:0/2229533889 >> 172.21.15.78:0/2229533889 conn(0x7ff65c0fbec0 msgr2=0x7ff65c102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:43:59.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.837+0000 7ff661f2c640 1 -- 172.21.15.78:0/2229533889 shutdown_connections 2024-08-18T23:43:59.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:43:59.837+0000 7ff661f2c640 1 -- 172.21.15.78:0/2229533889 wait complete. 2024-08-18T23:43:59.850 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-08-18T23:43:59.850 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:44:00.106 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:00.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:00 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3017532999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-18T23:44:00.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:00 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2229533889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-08-18T23:44:00.567 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:00 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3017532999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-18T23:44:00.567 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:00 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2229533889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-08-18T23:44:00.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.645+0000 7f7ed3fd1640 1 -- 172.21.15.78:0/4202269651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ecc1078b0 msgr2=0x7f7ecc107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:00.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.645+0000 7f7ed3fd1640 1 --2- 172.21.15.78:0/4202269651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ecc1078b0 0x7f7ecc107c90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009930 tx=0x7f7ec802a420 comp rx=0 tx=0).stop 2024-08-18T23:44:00.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.645+0000 7f7ed3fd1640 1 -- 172.21.15.78:0/4202269651 shutdown_connections 2024-08-18T23:44:00.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.645+0000 7f7ed3fd1640 1 --2- 172.21.15.78:0/4202269651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ecc102250 0x7f7ecc10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:00.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.645+0000 7f7ed3fd1640 1 --2- 172.21.15.78:0/4202269651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ecc1018b0 0x7f7ecc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:00.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.645+0000 7f7ed3fd1640 1 --2- 172.21.15.78:0/4202269651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ecc1078b0 0x7f7ecc107c90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:00.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.645+0000 7f7ed3fd1640 1 -- 172.21.15.78:0/4202269651 >> 172.21.15.78:0/4202269651 conn(0x7f7ecc0fd5f0 msgr2=0x7f7ecc0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:00.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.646+0000 7f7ed3fd1640 1 -- 172.21.15.78:0/4202269651 shutdown_connections 2024-08-18T23:44:00.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.646+0000 7f7ed3fd1640 1 -- 172.21.15.78:0/4202269651 wait complete. 2024-08-18T23:44:00.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.646+0000 7f7ed3fd1640 1 Processor -- start 2024-08-18T23:44:00.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.646+0000 7f7ed3fd1640 1 -- start start 2024-08-18T23:44:00.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.646+0000 7f7ed3fd1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ecc1018b0 0x7f7ecc198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:00.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.646+0000 7f7ed3fd1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ecc102250 0x7f7ecc1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:00.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.646+0000 7f7ed3fd1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ecc1078b0 0x7f7ecc19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:00.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.646+0000 7f7ed3fd1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc111cb0 con 0x7f7ecc102250 2024-08-18T23:44:00.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.646+0000 7f7ed3fd1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc111b30 con 0x7f7ecc1018b0 2024-08-18T23:44:00.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.646+0000 7f7ed3fd1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc111e30 con 0x7f7ecc1078b0 2024-08-18T23:44:00.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.646+0000 7f7ed156c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ecc102250 0x7f7ecc1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:00.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.646+0000 7f7ed156c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ecc102250 0x7f7ecc1986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51222/0 (socket says 172.21.15.78:51222) 2024-08-18T23:44:00.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.647+0000 7f7ed156c640 1 -- 172.21.15.78:0/3279209617 learned_addr learned my addr 172.21.15.78:0/3279209617 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:00.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.647+0000 7f7ed1d6d640 1 --2- 172.21.15.78:0/3279209617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ecc1018b0 0x7f7ecc198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:00.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.647+0000 7f7ed256e640 1 --2- 172.21.15.78:0/3279209617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ecc1078b0 0x7f7ecc19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:00.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.647+0000 7f7ed156c640 1 -- 172.21.15.78:0/3279209617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ecc1078b0 msgr2=0x7f7ecc19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:00.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.647+0000 7f7ed156c640 1 --2- 172.21.15.78:0/3279209617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ecc1078b0 0x7f7ecc19ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:00.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.647+0000 7f7ed156c640 1 -- 172.21.15.78:0/3279209617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ecc1018b0 msgr2=0x7f7ecc198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:00.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.647+0000 7f7ed156c640 1 --2- 172.21.15.78:0/3279209617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ecc1018b0 0x7f7ecc198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:00.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.647+0000 7f7ed156c640 1 -- 172.21.15.78:0/3279209617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc19d120 con 0x7f7ecc102250 2024-08-18T23:44:00.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.647+0000 7f7ed156c640 1 --2- 172.21.15.78:0/3279209617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ecc102250 0x7f7ecc1986b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00d950 tx=0x7f7ebc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:00.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.647+0000 7f7ec2ffd640 1 -- 172.21.15.78:0/3279209617 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc017070 con 0x7f7ecc102250 2024-08-18T23:44:00.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.647+0000 7f7ec2ffd640 1 -- 172.21.15.78:0/3279209617 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ebc009c30 con 0x7f7ecc102250 2024-08-18T23:44:00.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.647+0000 7f7ed3fd1640 1 -- 172.21.15.78:0/3279209617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ecc19d410 con 0x7f7ecc102250 2024-08-18T23:44:00.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.647+0000 7f7ec2ffd640 1 -- 172.21.15.78:0/3279209617 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc00f6b0 con 0x7f7ecc102250 2024-08-18T23:44:00.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.647+0000 7f7ed3fd1640 1 -- 172.21.15.78:0/3279209617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ecc1a4cf0 con 0x7f7ecc102250 2024-08-18T23:44:00.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.648+0000 7f7ed3fd1640 1 -- 172.21.15.78:0/3279209617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ecc101d10 con 0x7f7ecc102250 2024-08-18T23:44:00.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.649+0000 7f7ec2ffd640 1 -- 172.21.15.78:0/3279209617 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f7ebc0109d0 con 0x7f7ecc102250 2024-08-18T23:44:00.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.650+0000 7f7ec2ffd640 1 --2- 172.21.15.78:0/3279209617 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7eb80a2c10 0x7f7eb80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:00.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.650+0000 7f7ed1d6d640 1 --2- 172.21.15.78:0/3279209617 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7eb80a2c10 0x7f7eb80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:00.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.650+0000 7f7ec2ffd640 1 -- 172.21.15.78:0/3279209617 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f7ebc0cf1a0 con 0x7f7ecc102250 2024-08-18T23:44:00.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.650+0000 7f7ed1d6d640 1 --2- 172.21.15.78:0/3279209617 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7eb80a2c10 0x7f7eb80a50d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009900 tx=0x7f7ec80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:00.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.653+0000 7f7ec2ffd640 1 -- 172.21.15.78:0/3279209617 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7ebc009de0 con 0x7f7ecc102250 2024-08-18T23:44:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:00 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3017532999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-18T23:44:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:00 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2229533889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-08-18T23:44:00.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.852+0000 7f7ed3fd1640 1 -- 172.21.15.78:0/3279209617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ecc02d5e0 con 0x7f7ecc102250 2024-08-18T23:44:00.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.853+0000 7f7ec2ffd640 1 -- 172.21.15.78:0/3279209617 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7f7ebc09bc10 con 0x7f7ecc102250 2024-08-18T23:44:00.855 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:44:00.856 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"btime":"2024-08-18T23:43:57:245845+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:43:57.114120+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24428,24427]},"id":1}]} 2024-08-18T23:44:00.856 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 25 2024-08-18T23:44:00.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.855+0000 7f7ed3fd1640 1 -- 172.21.15.78:0/3279209617 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7eb80a2c10 msgr2=0x7f7eb80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:00.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.855+0000 7f7ed3fd1640 1 --2- 172.21.15.78:0/3279209617 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7eb80a2c10 0x7f7eb80a50d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009900 tx=0x7f7ec80058d0 comp rx=0 tx=0).stop 2024-08-18T23:44:00.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.855+0000 7f7ed3fd1640 1 -- 172.21.15.78:0/3279209617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ecc102250 msgr2=0x7f7ecc1986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:00.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.855+0000 7f7ed3fd1640 1 --2- 172.21.15.78:0/3279209617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ecc102250 0x7f7ecc1986b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00d950 tx=0x7f7ebc00de10 comp rx=0 tx=0).stop 2024-08-18T23:44:00.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.855+0000 7f7ed3fd1640 1 -- 172.21.15.78:0/3279209617 shutdown_connections 2024-08-18T23:44:00.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.855+0000 7f7ed3fd1640 1 --2- 172.21.15.78:0/3279209617 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7eb80a2c10 0x7f7eb80a50d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:00.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.855+0000 7f7ed3fd1640 1 --2- 172.21.15.78:0/3279209617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ecc1078b0 0x7f7ecc19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:00.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.855+0000 7f7ed3fd1640 1 --2- 172.21.15.78:0/3279209617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ecc102250 0x7f7ecc1986b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:00.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.855+0000 7f7ed3fd1640 1 --2- 172.21.15.78:0/3279209617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ecc1018b0 0x7f7ecc198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:00.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.856+0000 7f7ed3fd1640 1 -- 172.21.15.78:0/3279209617 >> 172.21.15.78:0/3279209617 conn(0x7f7ecc0fd5f0 msgr2=0x7f7ecc0feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:00.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.856+0000 7f7ed3fd1640 1 -- 172.21.15.78:0/3279209617 shutdown_connections 2024-08-18T23:44:00.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:00.856+0000 7f7ed3fd1640 1 -- 172.21.15.78:0/3279209617 wait complete. 2024-08-18T23:44:01.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:01 smithi123 ceph-mon[21808]: pgmap v341: 129 pgs: 129 active+clean; 648 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 8 op/s 2024-08-18T23:44:01.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:01 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3279209617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:01.575 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:01 smithi078 ceph-mon[18440]: pgmap v341: 129 pgs: 129 active+clean; 648 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 8 op/s 2024-08-18T23:44:01.575 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:01 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3279209617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:01.599 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-08-18T23:44:01.599 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-08-18T23:44:01.599 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-18T23:44:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:01 smithi161 ceph-mon[22027]: pgmap v341: 129 pgs: 129 active+clean; 648 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 8 op/s 2024-08-18T23:44:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:01 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3279209617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:01.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.701+0000 7ff2078fe640 1 -- 172.21.15.78:0/2196332196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff200101930 msgr2=0x7ff200101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:01.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.701+0000 7ff2078fe640 1 --2- 172.21.15.78:0/2196332196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff200101930 0x7ff200101db0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009900 tx=0x7ff1f402a410 comp rx=0 tx=0).stop 2024-08-18T23:44:01.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.701+0000 7ff2078fe640 1 -- 172.21.15.78:0/2196332196 shutdown_connections 2024-08-18T23:44:01.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.701+0000 7ff2078fe640 1 --2- 172.21.15.78:0/2196332196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2001022f0 0x7ff200108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:01.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.701+0000 7ff2078fe640 1 --2- 172.21.15.78:0/2196332196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff200101930 0x7ff200101db0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:01.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.701+0000 7ff2078fe640 1 --2- 172.21.15.78:0/2196332196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff200100740 0x7ff200100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:01.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.701+0000 7ff2078fe640 1 -- 172.21.15.78:0/2196332196 >> 172.21.15.78:0/2196332196 conn(0x7ff2000fbea0 msgr2=0x7ff2000fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:01.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.701+0000 7ff2078fe640 1 -- 172.21.15.78:0/2196332196 shutdown_connections 2024-08-18T23:44:01.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.701+0000 7ff2078fe640 1 -- 172.21.15.78:0/2196332196 wait complete. 2024-08-18T23:44:01.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.701+0000 7ff2078fe640 1 Processor -- start 2024-08-18T23:44:01.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff2078fe640 1 -- start start 2024-08-18T23:44:01.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff2078fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff200100740 0x7ff200069c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:01.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff2078fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff200101930 0x7ff20019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:01.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff2078fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2001022f0 0x7ff20019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:01.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff2078fe640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff20010c3a0 con 0x7ff2001022f0 2024-08-18T23:44:01.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff2078fe640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff20010c220 con 0x7ff200100740 2024-08-18T23:44:01.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff2078fe640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff20010c520 con 0x7ff200101930 2024-08-18T23:44:01.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff205e9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2001022f0 0x7ff20019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:01.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff205e9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2001022f0 0x7ff20019ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51236/0 (socket says 172.21.15.78:51236) 2024-08-18T23:44:01.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff205e9b640 1 -- 172.21.15.78:0/2875671520 learned_addr learned my addr 172.21.15.78:0/2875671520 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:01.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff20569a640 1 --2- 172.21.15.78:0/2875671520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff200100740 0x7ff200069c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:01.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff204e99640 1 --2- 172.21.15.78:0/2875671520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff200101930 0x7ff20019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:01.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff205e9b640 1 -- 172.21.15.78:0/2875671520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff200101930 msgr2=0x7ff20019c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:01.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff205e9b640 1 --2- 172.21.15.78:0/2875671520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff200101930 0x7ff20019c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:01.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff205e9b640 1 -- 172.21.15.78:0/2875671520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff200100740 msgr2=0x7ff200069c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:01.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff205e9b640 1 --2- 172.21.15.78:0/2875671520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff200100740 0x7ff200069c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:01.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff205e9b640 1 -- 172.21.15.78:0/2875671520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20019f770 con 0x7ff2001022f0 2024-08-18T23:44:01.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff20569a640 1 --2- 172.21.15.78:0/2875671520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff200100740 0x7ff200069c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:01.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff205e9b640 1 --2- 172.21.15.78:0/2875671520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2001022f0 0x7ff20019ee60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000cce0 tx=0x7ff1f000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:01.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff1fe7fc640 1 -- 172.21.15.78:0/2875671520 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f000f450 con 0x7ff2001022f0 2024-08-18T23:44:01.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff1fe7fc640 1 -- 172.21.15.78:0/2875671520 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1f000fad0 con 0x7ff2001022f0 2024-08-18T23:44:01.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff2078fe640 1 -- 172.21.15.78:0/2875671520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff20019fa00 con 0x7ff2001022f0 2024-08-18T23:44:01.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.702+0000 7ff1fe7fc640 1 -- 172.21.15.78:0/2875671520 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0018c10 con 0x7ff2001022f0 2024-08-18T23:44:01.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.703+0000 7ff2078fe640 1 -- 172.21.15.78:0/2875671520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff20019ff00 con 0x7ff2001022f0 2024-08-18T23:44:01.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.704+0000 7ff2078fe640 1 -- 172.21.15.78:0/2875671520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff200107ae0 con 0x7ff2001022f0 2024-08-18T23:44:01.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.705+0000 7ff1fe7fc640 1 -- 172.21.15.78:0/2875671520 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7ff1f000f5f0 con 0x7ff2001022f0 2024-08-18T23:44:01.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.706+0000 7ff1fe7fc640 1 --2- 172.21.15.78:0/2875671520 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff1e80a2c20 0x7ff1e80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:01.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.706+0000 7ff20569a640 1 --2- 172.21.15.78:0/2875671520 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff1e80a2c20 0x7ff1e80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:01.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.706+0000 7ff1fe7fc640 1 -- 172.21.15.78:0/2875671520 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7ff1f00cfd80 con 0x7ff2001022f0 2024-08-18T23:44:01.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.706+0000 7ff20569a640 1 --2- 172.21.15.78:0/2875671520 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff1e80a2c20 0x7ff1e80a50e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff200101790 tx=0x7ff1f800a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:01.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.709+0000 7ff1fe7fc640 1 -- 172.21.15.78:0/2875671520 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff1f0014030 con 0x7ff2001022f0 2024-08-18T23:44:01.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.873+0000 7ff2078fe640 1 -- 172.21.15.78:0/2875671520 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff200062360 con 0x7ff1e80a2c20 2024-08-18T23:44:01.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.881+0000 7ff1fe7fc640 1 -- 172.21.15.78:0/2875671520 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340486 (secure 0 0 0) 0x7ff200062360 con 0x7ff1e80a2c20 2024-08-18T23:44:01.887 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:44:01.889 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-08-18T23:44:01.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.884+0000 7ff2078fe640 1 -- 172.21.15.78:0/2875671520 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff1e80a2c20 msgr2=0x7ff1e80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:01.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.884+0000 7ff2078fe640 1 --2- 172.21.15.78:0/2875671520 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff1e80a2c20 0x7ff1e80a50e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff200101790 tx=0x7ff1f800a560 comp rx=0 tx=0).stop 2024-08-18T23:44:01.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.884+0000 7ff2078fe640 1 -- 172.21.15.78:0/2875671520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2001022f0 msgr2=0x7ff20019ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:01.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.884+0000 7ff2078fe640 1 --2- 172.21.15.78:0/2875671520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2001022f0 0x7ff20019ee60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000cce0 tx=0x7ff1f000a620 comp rx=0 tx=0).stop 2024-08-18T23:44:01.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.884+0000 7ff2078fe640 1 -- 172.21.15.78:0/2875671520 shutdown_connections 2024-08-18T23:44:01.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.884+0000 7ff2078fe640 1 --2- 172.21.15.78:0/2875671520 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff1e80a2c20 0x7ff1e80a50e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:01.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.884+0000 7ff2078fe640 1 --2- 172.21.15.78:0/2875671520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2001022f0 0x7ff20019ee60 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:01.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.884+0000 7ff2078fe640 1 --2- 172.21.15.78:0/2875671520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff200101930 0x7ff20019c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:01.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.884+0000 7ff2078fe640 1 --2- 172.21.15.78:0/2875671520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff200100740 0x7ff200069c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:01.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.884+0000 7ff2078fe640 1 -- 172.21.15.78:0/2875671520 >> 172.21.15.78:0/2875671520 conn(0x7ff2000fbea0 msgr2=0x7ff200104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:01.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.884+0000 7ff2078fe640 1 -- 172.21.15.78:0/2875671520 shutdown_connections 2024-08-18T23:44:01.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:01.884+0000 7ff2078fe640 1 -- 172.21.15.78:0/2875671520 wait complete. 2024-08-18T23:44:01.898 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-08-18T23:44:00.146816+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663839,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":828,"num_read_kb":539,"num_write":368,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":798,"ondisk_log_size":798,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392768,"kb_used_data":14436,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549056,"statfs":{"total":1151940427776,"available":1151538233344,"internally_reserved":0,"allocated":14782464,"data_stored":8242571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5320,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":52,"num_write":9,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003300"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298623+0000","last_change":"2024-08-18T23:41:30.120580+0000","last_active":"2024-08-18T23:43:24.298623+0000","last_peered":"2024-08-18T23:43:24.298623+0000","last_clean":"2024-08-18T23:43:24.298623+0000","last_became_active":"2024-08-18T23:41:30.120266+0000","last_became_peered":"2024-08-18T23:41:30.120266+0000","last_unstale":"2024-08-18T23:43:24.298623+0000","last_undegraded":"2024-08-18T23:43:24.298623+0000","last_fullsized":"2024-08-18T23:43:24.298623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:06:40.997070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312511+0000","last_change":"2024-08-18T23:41:32.143232+0000","last_active":"2024-08-18T23:43:24.312511+0000","last_peered":"2024-08-18T23:43:24.312511+0000","last_clean":"2024-08-18T23:43:24.312511+0000","last_became_active":"2024-08-18T23:41:32.142916+0000","last_became_peered":"2024-08-18T23:41:32.142916+0000","last_unstale":"2024-08-18T23:43:24.312511+0000","last_undegraded":"2024-08-18T23:43:24.312511+0000","last_fullsized":"2024-08-18T23:43:24.312511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:50:54.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312049+0000","last_change":"2024-08-18T23:41:39.188772+0000","last_active":"2024-08-18T23:43:24.312049+0000","last_peered":"2024-08-18T23:43:24.312049+0000","last_clean":"2024-08-18T23:43:24.312049+0000","last_became_active":"2024-08-18T23:41:39.188362+0000","last_became_peered":"2024-08-18T23:41:39.188362+0000","last_unstale":"2024-08-18T23:43:24.312049+0000","last_undegraded":"2024-08-18T23:43:24.312049+0000","last_fullsized":"2024-08-18T23:43:24.312049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:47:50.255543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312628+0000","last_change":"2024-08-18T23:41:30.137179+0000","last_active":"2024-08-18T23:43:24.312628+0000","last_peered":"2024-08-18T23:43:24.312628+0000","last_clean":"2024-08-18T23:43:24.312628+0000","last_became_active":"2024-08-18T23:41:30.136618+0000","last_became_peered":"2024-08-18T23:41:30.136618+0000","last_unstale":"2024-08-18T23:43:24.312628+0000","last_undegraded":"2024-08-18T23:43:24.312628+0000","last_fullsized":"2024-08-18T23:43:24.312628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:59.826364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312531+0000","last_change":"2024-08-18T23:41:32.142075+0000","last_active":"2024-08-18T23:43:24.312531+0000","last_peered":"2024-08-18T23:43:24.312531+0000","last_clean":"2024-08-18T23:43:24.312531+0000","last_became_active":"2024-08-18T23:41:32.141709+0000","last_became_peered":"2024-08-18T23:41:32.141709+0000","last_unstale":"2024-08-18T23:43:24.312531+0000","last_undegraded":"2024-08-18T23:43:24.312531+0000","last_fullsized":"2024-08-18T23:43:24.312531+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:08.324627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312147+0000","last_change":"2024-08-18T23:41:39.184575+0000","last_active":"2024-08-18T23:43:24.312147+0000","last_peered":"2024-08-18T23:43:24.312147+0000","last_clean":"2024-08-18T23:43:24.312147+0000","last_became_active":"2024-08-18T23:41:39.184169+0000","last_became_peered":"2024-08-18T23:41:39.184169+0000","last_unstale":"2024-08-18T23:43:24.312147+0000","last_undegraded":"2024-08-18T23:43:24.312147+0000","last_fullsized":"2024-08-18T23:43:24.312147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:32.218172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304076+0000","last_change":"2024-08-18T23:41:30.121638+0000","last_active":"2024-08-18T23:43:24.304076+0000","last_peered":"2024-08-18T23:43:24.304076+0000","last_clean":"2024-08-18T23:43:24.304076+0000","last_became_active":"2024-08-18T23:41:30.121290+0000","last_became_peered":"2024-08-18T23:41:30.121290+0000","last_unstale":"2024-08-18T23:43:24.304076+0000","last_undegraded":"2024-08-18T23:43:24.304076+0000","last_fullsized":"2024-08-18T23:43:24.304076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:43:55.638780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312374+0000","last_change":"2024-08-18T23:41:32.142177+0000","last_active":"2024-08-18T23:43:24.312374+0000","last_peered":"2024-08-18T23:43:24.312374+0000","last_clean":"2024-08-18T23:43:24.312374+0000","last_became_active":"2024-08-18T23:41:32.141731+0000","last_became_peered":"2024-08-18T23:41:32.141731+0000","last_unstale":"2024-08-18T23:43:24.312374+0000","last_undegraded":"2024-08-18T23:43:24.312374+0000","last_fullsized":"2024-08-18T23:43:24.312374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:36.283299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312604+0000","last_change":"2024-08-18T23:41:39.182931+0000","last_active":"2024-08-18T23:43:24.312604+0000","last_peered":"2024-08-18T23:43:24.312604+0000","last_clean":"2024-08-18T23:43:24.312604+0000","last_became_active":"2024-08-18T23:41:39.182008+0000","last_became_peered":"2024-08-18T23:41:39.182008+0000","last_unstale":"2024-08-18T23:43:24.312604+0000","last_undegraded":"2024-08-18T23:43:24.312604+0000","last_fullsized":"2024-08-18T23:43:24.312604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:58:38.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312863+0000","last_change":"2024-08-18T23:42:23.329762+0000","last_active":"2024-08-18T23:43:24.312863+0000","last_peered":"2024-08-18T23:43:24.312863+0000","last_clean":"2024-08-18T23:43:24.312863+0000","last_became_active":"2024-08-18T23:42:23.325753+0000","last_became_peered":"2024-08-18T23:42:23.325753+0000","last_unstale":"2024-08-18T23:43:24.312863+0000","last_undegraded":"2024-08-18T23:43:24.312863+0000","last_fullsized":"2024-08-18T23:43:24.312863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:19:43.062492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307257+0000","last_change":"2024-08-18T23:41:32.148662+0000","last_active":"2024-08-18T23:43:24.307257+0000","last_peered":"2024-08-18T23:43:24.307257+0000","last_clean":"2024-08-18T23:43:24.307257+0000","last_became_active":"2024-08-18T23:41:32.148318+0000","last_became_peered":"2024-08-18T23:41:32.148318+0000","last_unstale":"2024-08-18T23:43:24.307257+0000","last_undegraded":"2024-08-18T23:43:24.307257+0000","last_fullsized":"2024-08-18T23:43:24.307257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:58.668669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312238+0000","last_change":"2024-08-18T23:41:39.181114+0000","last_active":"2024-08-18T23:43:24.312238+0000","last_peered":"2024-08-18T23:43:24.312238+0000","last_clean":"2024-08-18T23:43:24.312238+0000","last_became_active":"2024-08-18T23:41:39.180599+0000","last_became_peered":"2024-08-18T23:41:39.180599+0000","last_unstale":"2024-08-18T23:43:24.312238+0000","last_undegraded":"2024-08-18T23:43:24.312238+0000","last_fullsized":"2024-08-18T23:43:24.312238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:36:08.749143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304715+0000","last_change":"2024-08-18T23:41:32.144326+0000","last_active":"2024-08-18T23:43:24.304715+0000","last_peered":"2024-08-18T23:43:24.304715+0000","last_clean":"2024-08-18T23:43:24.304715+0000","last_became_active":"2024-08-18T23:41:32.144004+0000","last_became_peered":"2024-08-18T23:41:32.144004+0000","last_unstale":"2024-08-18T23:43:24.304715+0000","last_undegraded":"2024-08-18T23:43:24.304715+0000","last_fullsized":"2024-08-18T23:43:24.304715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:10.918624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303665+0000","last_change":"2024-08-18T23:41:30.125160+0000","last_active":"2024-08-18T23:43:24.303665+0000","last_peered":"2024-08-18T23:43:24.303665+0000","last_clean":"2024-08-18T23:43:24.303665+0000","last_became_active":"2024-08-18T23:41:30.124685+0000","last_became_peered":"2024-08-18T23:41:30.124685+0000","last_unstale":"2024-08-18T23:43:24.303665+0000","last_undegraded":"2024-08-18T23:43:24.303665+0000","last_fullsized":"2024-08-18T23:43:24.303665+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:52:48.997902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313216+0000","last_change":"2024-08-18T23:41:39.681481+0000","last_active":"2024-08-18T23:43:24.313216+0000","last_peered":"2024-08-18T23:43:24.313216+0000","last_clean":"2024-08-18T23:43:24.313216+0000","last_became_active":"2024-08-18T23:41:39.680204+0000","last_became_peered":"2024-08-18T23:41:39.680204+0000","last_unstale":"2024-08-18T23:43:24.313216+0000","last_undegraded":"2024-08-18T23:43:24.313216+0000","last_fullsized":"2024-08-18T23:43:24.313216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:00:05.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312231+0000","last_change":"2024-08-18T23:41:32.151335+0000","last_active":"2024-08-18T23:43:24.312231+0000","last_peered":"2024-08-18T23:43:24.312231+0000","last_clean":"2024-08-18T23:43:24.312231+0000","last_became_active":"2024-08-18T23:41:32.151040+0000","last_became_peered":"2024-08-18T23:41:32.151040+0000","last_unstale":"2024-08-18T23:43:24.312231+0000","last_undegraded":"2024-08-18T23:43:24.312231+0000","last_fullsized":"2024-08-18T23:43:24.312231+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:16.108555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312486+0000","last_change":"2024-08-18T23:41:30.122301+0000","last_active":"2024-08-18T23:43:24.312486+0000","last_peered":"2024-08-18T23:43:24.312486+0000","last_clean":"2024-08-18T23:43:24.312486+0000","last_became_active":"2024-08-18T23:41:30.121117+0000","last_became_peered":"2024-08-18T23:41:30.121117+0000","last_unstale":"2024-08-18T23:43:24.312486+0000","last_undegraded":"2024-08-18T23:43:24.312486+0000","last_fullsized":"2024-08-18T23:43:24.312486+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:40:12.678803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312721+0000","last_change":"2024-08-18T23:41:39.182395+0000","last_active":"2024-08-18T23:43:24.312721+0000","last_peered":"2024-08-18T23:43:24.312721+0000","last_clean":"2024-08-18T23:43:24.312721+0000","last_became_active":"2024-08-18T23:41:39.181979+0000","last_became_peered":"2024-08-18T23:41:39.181979+0000","last_unstale":"2024-08-18T23:43:24.312721+0000","last_undegraded":"2024-08-18T23:43:24.312721+0000","last_fullsized":"2024-08-18T23:43:24.312721+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:04.674503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312810+0000","last_change":"2024-08-18T23:41:32.140464+0000","last_active":"2024-08-18T23:43:24.312810+0000","last_peered":"2024-08-18T23:43:24.312810+0000","last_clean":"2024-08-18T23:43:24.312810+0000","last_became_active":"2024-08-18T23:41:32.139153+0000","last_became_peered":"2024-08-18T23:41:32.139153+0000","last_unstale":"2024-08-18T23:43:24.312810+0000","last_undegraded":"2024-08-18T23:43:24.312810+0000","last_fullsized":"2024-08-18T23:43:24.312810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:27.387680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312414+0000","last_change":"2024-08-18T23:41:30.122787+0000","last_active":"2024-08-18T23:43:24.312414+0000","last_peered":"2024-08-18T23:43:24.312414+0000","last_clean":"2024-08-18T23:43:24.312414+0000","last_became_active":"2024-08-18T23:41:30.122462+0000","last_became_peered":"2024-08-18T23:41:30.122462+0000","last_unstale":"2024-08-18T23:43:24.312414+0000","last_undegraded":"2024-08-18T23:43:24.312414+0000","last_fullsized":"2024-08-18T23:43:24.312414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:55.568831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298325+0000","last_change":"2024-08-18T23:41:39.190511+0000","last_active":"2024-08-18T23:43:24.298325+0000","last_peered":"2024-08-18T23:43:24.298325+0000","last_clean":"2024-08-18T23:43:24.298325+0000","last_became_active":"2024-08-18T23:41:39.190154+0000","last_became_peered":"2024-08-18T23:41:39.190154+0000","last_unstale":"2024-08-18T23:43:24.298325+0000","last_undegraded":"2024-08-18T23:43:24.298325+0000","last_fullsized":"2024-08-18T23:43:24.298325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:33:34.514827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303456+0000","last_change":"2024-08-18T23:41:32.143867+0000","last_active":"2024-08-18T23:43:24.303456+0000","last_peered":"2024-08-18T23:43:24.303456+0000","last_clean":"2024-08-18T23:43:24.303456+0000","last_became_active":"2024-08-18T23:41:32.143527+0000","last_became_peered":"2024-08-18T23:41:32.143527+0000","last_unstale":"2024-08-18T23:43:24.303456+0000","last_undegraded":"2024-08-18T23:43:24.303456+0000","last_fullsized":"2024-08-18T23:43:24.303456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:40:25.902133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312100+0000","last_change":"2024-08-18T23:42:23.325102+0000","last_active":"2024-08-18T23:43:24.312100+0000","last_peered":"2024-08-18T23:43:24.312100+0000","last_clean":"2024-08-18T23:43:24.312100+0000","last_became_active":"2024-08-18T23:42:23.321583+0000","last_became_peered":"2024-08-18T23:42:23.321583+0000","last_unstale":"2024-08-18T23:43:24.312100+0000","last_undegraded":"2024-08-18T23:43:24.312100+0000","last_fullsized":"2024-08-18T23:43:24.312100+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:57:42.546686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312767+0000","last_change":"2024-08-18T23:42:23.559756+0000","last_active":"2024-08-18T23:43:24.312767+0000","last_peered":"2024-08-18T23:43:24.312767+0000","last_clean":"2024-08-18T23:43:24.312767+0000","last_became_active":"2024-08-18T23:42:23.559242+0000","last_became_peered":"2024-08-18T23:42:23.559242+0000","last_unstale":"2024-08-18T23:43:24.312767+0000","last_undegraded":"2024-08-18T23:43:24.312767+0000","last_fullsized":"2024-08-18T23:43:24.312767+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:25:40.790313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312043+0000","last_change":"2024-08-18T23:41:32.139025+0000","last_active":"2024-08-18T23:43:24.312043+0000","last_peered":"2024-08-18T23:43:24.312043+0000","last_clean":"2024-08-18T23:43:24.312043+0000","last_became_active":"2024-08-18T23:41:32.138733+0000","last_became_peered":"2024-08-18T23:41:32.138733+0000","last_unstale":"2024-08-18T23:43:24.312043+0000","last_undegraded":"2024-08-18T23:43:24.312043+0000","last_fullsized":"2024-08-18T23:43:24.312043+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:01:39.826391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313061+0000","last_change":"2024-08-18T23:41:30.477359+0000","last_active":"2024-08-18T23:43:24.313061+0000","last_peered":"2024-08-18T23:43:24.313061+0000","last_clean":"2024-08-18T23:43:24.313061+0000","last_became_active":"2024-08-18T23:41:30.477011+0000","last_became_peered":"2024-08-18T23:41:30.477011+0000","last_unstale":"2024-08-18T23:43:24.313061+0000","last_undegraded":"2024-08-18T23:43:24.313061+0000","last_fullsized":"2024-08-18T23:43:24.313061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:53:10.052859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312145+0000","last_change":"2024-08-18T23:41:39.182140+0000","last_active":"2024-08-18T23:43:24.312145+0000","last_peered":"2024-08-18T23:43:24.312145+0000","last_clean":"2024-08-18T23:43:24.312145+0000","last_became_active":"2024-08-18T23:41:39.181667+0000","last_became_peered":"2024-08-18T23:41:39.181667+0000","last_unstale":"2024-08-18T23:43:24.312145+0000","last_undegraded":"2024-08-18T23:43:24.312145+0000","last_fullsized":"2024-08-18T23:43:24.312145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:02:18.128028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312594+0000","last_change":"2024-08-18T23:41:32.137064+0000","last_active":"2024-08-18T23:43:24.312594+0000","last_peered":"2024-08-18T23:43:24.312594+0000","last_clean":"2024-08-18T23:43:24.312594+0000","last_became_active":"2024-08-18T23:41:32.136738+0000","last_became_peered":"2024-08-18T23:41:32.136738+0000","last_unstale":"2024-08-18T23:43:24.312594+0000","last_undegraded":"2024-08-18T23:43:24.312594+0000","last_fullsized":"2024-08-18T23:43:24.312594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:15:11.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'12","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298408+0000","last_change":"2024-08-18T23:41:30.120042+0000","last_active":"2024-08-18T23:43:24.298408+0000","last_peered":"2024-08-18T23:43:24.298408+0000","last_clean":"2024-08-18T23:43:24.298408+0000","last_became_active":"2024-08-18T23:41:30.119719+0000","last_became_peered":"2024-08-18T23:41:30.119719+0000","last_unstale":"2024-08-18T23:43:24.298408+0000","last_undegraded":"2024-08-18T23:43:24.298408+0000","last_fullsized":"2024-08-18T23:43:24.298408+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:35:44.466330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311900+0000","last_change":"2024-08-18T23:41:39.680405+0000","last_active":"2024-08-18T23:43:24.311900+0000","last_peered":"2024-08-18T23:43:24.311900+0000","last_clean":"2024-08-18T23:43:24.311900+0000","last_became_active":"2024-08-18T23:41:39.679940+0000","last_became_peered":"2024-08-18T23:41:39.679940+0000","last_unstale":"2024-08-18T23:43:24.311900+0000","last_undegraded":"2024-08-18T23:43:24.311900+0000","last_fullsized":"2024-08-18T23:43:24.311900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:15:03.942116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301271+0000","last_change":"2024-08-18T23:41:32.139688+0000","last_active":"2024-08-18T23:43:24.301271+0000","last_peered":"2024-08-18T23:43:24.301271+0000","last_clean":"2024-08-18T23:43:24.301271+0000","last_became_active":"2024-08-18T23:41:32.139397+0000","last_became_peered":"2024-08-18T23:41:32.139397+0000","last_unstale":"2024-08-18T23:43:24.301271+0000","last_undegraded":"2024-08-18T23:43:24.301271+0000","last_fullsized":"2024-08-18T23:43:24.301271+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:03:57.017488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304150+0000","last_change":"2024-08-18T23:41:30.137578+0000","last_active":"2024-08-18T23:43:24.304150+0000","last_peered":"2024-08-18T23:43:24.304150+0000","last_clean":"2024-08-18T23:43:24.304150+0000","last_became_active":"2024-08-18T23:41:30.137246+0000","last_became_peered":"2024-08-18T23:41:30.137246+0000","last_unstale":"2024-08-18T23:43:24.304150+0000","last_undegraded":"2024-08-18T23:43:24.304150+0000","last_fullsized":"2024-08-18T23:43:24.304150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:48:54.839432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312576+0000","last_change":"2024-08-18T23:41:39.186360+0000","last_active":"2024-08-18T23:43:24.312576+0000","last_peered":"2024-08-18T23:43:24.312576+0000","last_clean":"2024-08-18T23:43:24.312576+0000","last_became_active":"2024-08-18T23:41:39.185808+0000","last_became_peered":"2024-08-18T23:41:39.185808+0000","last_unstale":"2024-08-18T23:43:24.312576+0000","last_undegraded":"2024-08-18T23:43:24.312576+0000","last_fullsized":"2024-08-18T23:43:24.312576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:53.891061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300916+0000","last_change":"2024-08-18T23:41:32.146070+0000","last_active":"2024-08-18T23:43:24.300916+0000","last_peered":"2024-08-18T23:43:24.300916+0000","last_clean":"2024-08-18T23:43:24.300916+0000","last_became_active":"2024-08-18T23:41:32.145640+0000","last_became_peered":"2024-08-18T23:41:32.145640+0000","last_unstale":"2024-08-18T23:43:24.300916+0000","last_undegraded":"2024-08-18T23:43:24.300916+0000","last_fullsized":"2024-08-18T23:43:24.300916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:56:21.368762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298278+0000","last_change":"2024-08-18T23:41:30.138565+0000","last_active":"2024-08-18T23:43:24.298278+0000","last_peered":"2024-08-18T23:43:24.298278+0000","last_clean":"2024-08-18T23:43:24.298278+0000","last_became_active":"2024-08-18T23:41:30.138210+0000","last_became_peered":"2024-08-18T23:41:30.138210+0000","last_unstale":"2024-08-18T23:43:24.298278+0000","last_undegraded":"2024-08-18T23:43:24.298278+0000","last_fullsized":"2024-08-18T23:43:24.298278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:42:14.928884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303425+0000","last_change":"2024-08-18T23:41:39.680656+0000","last_active":"2024-08-18T23:43:24.303425+0000","last_peered":"2024-08-18T23:43:24.303425+0000","last_clean":"2024-08-18T23:43:24.303425+0000","last_became_active":"2024-08-18T23:41:39.680203+0000","last_became_peered":"2024-08-18T23:41:39.680203+0000","last_unstale":"2024-08-18T23:43:24.303425+0000","last_undegraded":"2024-08-18T23:43:24.303425+0000","last_fullsized":"2024-08-18T23:43:24.303425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:55:43.138869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300932+0000","last_change":"2024-08-18T23:41:32.130222+0000","last_active":"2024-08-18T23:43:24.300932+0000","last_peered":"2024-08-18T23:43:24.300932+0000","last_clean":"2024-08-18T23:43:24.300932+0000","last_became_active":"2024-08-18T23:41:32.129844+0000","last_became_peered":"2024-08-18T23:41:32.129844+0000","last_unstale":"2024-08-18T23:43:24.300932+0000","last_undegraded":"2024-08-18T23:43:24.300932+0000","last_fullsized":"2024-08-18T23:43:24.300932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:39:21.106343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304850+0000","last_change":"2024-08-18T23:41:30.123142+0000","last_active":"2024-08-18T23:43:24.304850+0000","last_peered":"2024-08-18T23:43:24.304850+0000","last_clean":"2024-08-18T23:43:24.304850+0000","last_became_active":"2024-08-18T23:41:30.121645+0000","last_became_peered":"2024-08-18T23:41:30.121645+0000","last_unstale":"2024-08-18T23:43:24.304850+0000","last_undegraded":"2024-08-18T23:43:24.304850+0000","last_fullsized":"2024-08-18T23:43:24.304850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:07:24.912259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301163+0000","last_change":"2024-08-18T23:41:39.186033+0000","last_active":"2024-08-18T23:43:24.301163+0000","last_peered":"2024-08-18T23:43:24.301163+0000","last_clean":"2024-08-18T23:43:24.301163+0000","last_became_active":"2024-08-18T23:41:39.185451+0000","last_became_peered":"2024-08-18T23:41:39.185451+0000","last_unstale":"2024-08-18T23:43:24.301163+0000","last_undegraded":"2024-08-18T23:43:24.301163+0000","last_fullsized":"2024-08-18T23:43:24.301163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:52:13.556760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.306993+0000","last_change":"2024-08-18T23:41:32.141065+0000","last_active":"2024-08-18T23:43:24.306993+0000","last_peered":"2024-08-18T23:43:24.306993+0000","last_clean":"2024-08-18T23:43:24.306993+0000","last_became_active":"2024-08-18T23:41:32.140732+0000","last_became_peered":"2024-08-18T23:41:32.140732+0000","last_unstale":"2024-08-18T23:43:24.306993+0000","last_undegraded":"2024-08-18T23:43:24.306993+0000","last_fullsized":"2024-08-18T23:43:24.306993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:08:32.287835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":156,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:57.429047+0000","last_change":"2024-08-18T23:41:30.136838+0000","last_active":"2024-08-18T23:43:57.429047+0000","last_peered":"2024-08-18T23:43:57.429047+0000","last_clean":"2024-08-18T23:43:57.429047+0000","last_became_active":"2024-08-18T23:41:30.136523+0000","last_became_peered":"2024-08-18T23:41:30.136523+0000","last_unstale":"2024-08-18T23:43:57.429047+0000","last_undegraded":"2024-08-18T23:43:57.429047+0000","last_fullsized":"2024-08-18T23:43:57.429047+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:56:17.750535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301550+0000","last_change":"2024-08-18T23:41:39.680314+0000","last_active":"2024-08-18T23:43:24.301550+0000","last_peered":"2024-08-18T23:43:24.301550+0000","last_clean":"2024-08-18T23:43:24.301550+0000","last_became_active":"2024-08-18T23:41:39.679920+0000","last_became_peered":"2024-08-18T23:41:39.679920+0000","last_unstale":"2024-08-18T23:43:24.301550+0000","last_undegraded":"2024-08-18T23:43:24.301550+0000","last_fullsized":"2024-08-18T23:43:24.301550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:26:29.866081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300888+0000","last_change":"2024-08-18T23:41:32.139752+0000","last_active":"2024-08-18T23:43:24.300888+0000","last_peered":"2024-08-18T23:43:24.300888+0000","last_clean":"2024-08-18T23:43:24.300888+0000","last_became_active":"2024-08-18T23:41:32.139451+0000","last_became_peered":"2024-08-18T23:41:32.139451+0000","last_unstale":"2024-08-18T23:43:24.300888+0000","last_undegraded":"2024-08-18T23:43:24.300888+0000","last_fullsized":"2024-08-18T23:43:24.300888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:15:13.390529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304894+0000","last_change":"2024-08-18T23:41:30.126513+0000","last_active":"2024-08-18T23:43:24.304894+0000","last_peered":"2024-08-18T23:43:24.304894+0000","last_clean":"2024-08-18T23:43:24.304894+0000","last_became_active":"2024-08-18T23:41:30.126046+0000","last_became_peered":"2024-08-18T23:41:30.126046+0000","last_unstale":"2024-08-18T23:43:24.304894+0000","last_undegraded":"2024-08-18T23:43:24.304894+0000","last_fullsized":"2024-08-18T23:43:24.304894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:49:16.565565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307280+0000","last_change":"2024-08-18T23:41:39.681006+0000","last_active":"2024-08-18T23:43:24.307280+0000","last_peered":"2024-08-18T23:43:24.307280+0000","last_clean":"2024-08-18T23:43:24.307280+0000","last_became_active":"2024-08-18T23:41:39.680409+0000","last_became_peered":"2024-08-18T23:41:39.680409+0000","last_unstale":"2024-08-18T23:43:24.307280+0000","last_undegraded":"2024-08-18T23:43:24.307280+0000","last_fullsized":"2024-08-18T23:43:24.307280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:47.778866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312261+0000","last_change":"2024-08-18T23:41:32.147545+0000","last_active":"2024-08-18T23:43:24.312261+0000","last_peered":"2024-08-18T23:43:24.312261+0000","last_clean":"2024-08-18T23:43:24.312261+0000","last_became_active":"2024-08-18T23:41:32.147238+0000","last_became_peered":"2024-08-18T23:41:32.147238+0000","last_unstale":"2024-08-18T23:43:24.312261+0000","last_undegraded":"2024-08-18T23:43:24.312261+0000","last_fullsized":"2024-08-18T23:43:24.312261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:12:13.589535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313054+0000","last_change":"2024-08-18T23:41:30.120650+0000","last_active":"2024-08-18T23:43:24.313054+0000","last_peered":"2024-08-18T23:43:24.313054+0000","last_clean":"2024-08-18T23:43:24.313054+0000","last_became_active":"2024-08-18T23:41:30.120196+0000","last_became_peered":"2024-08-18T23:41:30.120196+0000","last_unstale":"2024-08-18T23:43:24.313054+0000","last_undegraded":"2024-08-18T23:43:24.313054+0000","last_fullsized":"2024-08-18T23:43:24.313054+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:40:23.370088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304710+0000","last_change":"2024-08-18T23:40:22.013519+0000","last_active":"2024-08-18T23:43:24.304710+0000","last_peered":"2024-08-18T23:43:24.304710+0000","last_clean":"2024-08-18T23:43:24.304710+0000","last_became_active":"2024-08-18T23:40:22.012986+0000","last_became_peered":"2024-08-18T23:40:22.012986+0000","last_unstale":"2024-08-18T23:43:24.304710+0000","last_undegraded":"2024-08-18T23:43:24.304710+0000","last_fullsized":"2024-08-18T23:43:24.304710+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_clean_scrub_stamp":"2024-08-18T23:37:22.388424+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:19.874356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304893+0000","last_change":"2024-08-18T23:42:23.332800+0000","last_active":"2024-08-18T23:43:24.304893+0000","last_peered":"2024-08-18T23:43:24.304893+0000","last_clean":"2024-08-18T23:43:24.304893+0000","last_became_active":"2024-08-18T23:42:23.326441+0000","last_became_peered":"2024-08-18T23:42:23.326441+0000","last_unstale":"2024-08-18T23:43:24.304893+0000","last_undegraded":"2024-08-18T23:43:24.304893+0000","last_fullsized":"2024-08-18T23:43:24.304893+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:13:58.243518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.305065+0000","last_change":"2024-08-18T23:41:32.139907+0000","last_active":"2024-08-18T23:43:24.305065+0000","last_peered":"2024-08-18T23:43:24.305065+0000","last_clean":"2024-08-18T23:43:24.305065+0000","last_became_active":"2024-08-18T23:41:32.137956+0000","last_became_peered":"2024-08-18T23:41:32.137956+0000","last_unstale":"2024-08-18T23:43:24.305065+0000","last_undegraded":"2024-08-18T23:43:24.305065+0000","last_fullsized":"2024-08-18T23:43:24.305065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:10:13.223328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312461+0000","last_change":"2024-08-18T23:42:23.330062+0000","last_active":"2024-08-18T23:43:24.312461+0000","last_peered":"2024-08-18T23:43:24.312461+0000","last_clean":"2024-08-18T23:43:24.312461+0000","last_became_active":"2024-08-18T23:42:23.323196+0000","last_became_peered":"2024-08-18T23:42:23.323196+0000","last_unstale":"2024-08-18T23:43:24.312461+0000","last_undegraded":"2024-08-18T23:43:24.312461+0000","last_fullsized":"2024-08-18T23:43:24.312461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:53:29.059542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312185+0000","last_change":"2024-08-18T23:41:39.185917+0000","last_active":"2024-08-18T23:43:24.312185+0000","last_peered":"2024-08-18T23:43:24.312185+0000","last_clean":"2024-08-18T23:43:24.312185+0000","last_became_active":"2024-08-18T23:41:39.185427+0000","last_became_peered":"2024-08-18T23:41:39.185427+0000","last_unstale":"2024-08-18T23:43:24.312185+0000","last_undegraded":"2024-08-18T23:43:24.312185+0000","last_fullsized":"2024-08-18T23:43:24.312185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:52:13.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.308822+0000","last_change":"2024-08-18T23:43:24.308401+0000","last_active":"2024-08-18T23:43:24.308822+0000","last_peered":"2024-08-18T23:43:24.308822+0000","last_clean":"2024-08-18T23:43:24.308822+0000","last_became_active":"2024-08-18T23:43:24.308081+0000","last_became_peered":"2024-08-18T23:43:24.308081+0000","last_unstale":"2024-08-18T23:43:24.308822+0000","last_undegraded":"2024-08-18T23:43:24.308822+0000","last_fullsized":"2024-08-18T23:43:24.308822+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:28:57.712217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307528+0000","last_change":"2024-08-18T23:41:30.121307+0000","last_active":"2024-08-18T23:43:24.307528+0000","last_peered":"2024-08-18T23:43:24.307528+0000","last_clean":"2024-08-18T23:43:24.307528+0000","last_became_active":"2024-08-18T23:41:30.120932+0000","last_became_peered":"2024-08-18T23:41:30.120932+0000","last_unstale":"2024-08-18T23:43:24.307528+0000","last_undegraded":"2024-08-18T23:43:24.307528+0000","last_fullsized":"2024-08-18T23:43:24.307528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:06:07.742880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311751+0000","last_change":"2024-08-18T23:41:39.184647+0000","last_active":"2024-08-18T23:43:24.311751+0000","last_peered":"2024-08-18T23:43:24.311751+0000","last_clean":"2024-08-18T23:43:24.311751+0000","last_became_active":"2024-08-18T23:41:39.184251+0000","last_became_peered":"2024-08-18T23:41:39.184251+0000","last_unstale":"2024-08-18T23:43:24.311751+0000","last_undegraded":"2024-08-18T23:43:24.311751+0000","last_fullsized":"2024-08-18T23:43:24.311751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:55.200801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.305184+0000","last_change":"2024-08-18T23:41:32.137007+0000","last_active":"2024-08-18T23:43:24.305184+0000","last_peered":"2024-08-18T23:43:24.305184+0000","last_clean":"2024-08-18T23:43:24.305184+0000","last_became_active":"2024-08-18T23:41:32.136684+0000","last_became_peered":"2024-08-18T23:41:32.136684+0000","last_unstale":"2024-08-18T23:43:24.305184+0000","last_undegraded":"2024-08-18T23:43:24.305184+0000","last_fullsized":"2024-08-18T23:43:24.305184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:21:26.584303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301471+0000","last_change":"2024-08-18T23:41:30.121183+0000","last_active":"2024-08-18T23:43:24.301471+0000","last_peered":"2024-08-18T23:43:24.301471+0000","last_clean":"2024-08-18T23:43:24.301471+0000","last_became_active":"2024-08-18T23:41:30.120847+0000","last_became_peered":"2024-08-18T23:41:30.120847+0000","last_unstale":"2024-08-18T23:43:24.301471+0000","last_undegraded":"2024-08-18T23:43:24.301471+0000","last_fullsized":"2024-08-18T23:43:24.301471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:51.025821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.306833+0000","last_change":"2024-08-18T23:41:39.183375+0000","last_active":"2024-08-18T23:43:24.306833+0000","last_peered":"2024-08-18T23:43:24.306833+0000","last_clean":"2024-08-18T23:43:24.306833+0000","last_became_active":"2024-08-18T23:41:39.182339+0000","last_became_peered":"2024-08-18T23:41:39.182339+0000","last_unstale":"2024-08-18T23:43:24.306833+0000","last_undegraded":"2024-08-18T23:43:24.306833+0000","last_fullsized":"2024-08-18T23:43:24.306833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:26.119056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313146+0000","last_change":"2024-08-18T23:41:32.140748+0000","last_active":"2024-08-18T23:43:24.313146+0000","last_peered":"2024-08-18T23:43:24.313146+0000","last_clean":"2024-08-18T23:43:24.313146+0000","last_became_active":"2024-08-18T23:41:32.140306+0000","last_became_peered":"2024-08-18T23:41:32.140306+0000","last_unstale":"2024-08-18T23:43:24.313146+0000","last_undegraded":"2024-08-18T23:43:24.313146+0000","last_fullsized":"2024-08-18T23:43:24.313146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:23:02.371529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:48.359280+0000","last_change":"2024-08-18T23:41:30.121617+0000","last_active":"2024-08-18T23:43:48.359280+0000","last_peered":"2024-08-18T23:43:48.359280+0000","last_clean":"2024-08-18T23:43:48.359280+0000","last_became_active":"2024-08-18T23:41:30.121257+0000","last_became_peered":"2024-08-18T23:41:30.121257+0000","last_unstale":"2024-08-18T23:43:48.359280+0000","last_undegraded":"2024-08-18T23:43:48.359280+0000","last_fullsized":"2024-08-18T23:43:48.359280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:23:42.091553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307008+0000","last_change":"2024-08-18T23:41:39.183833+0000","last_active":"2024-08-18T23:43:24.307008+0000","last_peered":"2024-08-18T23:43:24.307008+0000","last_clean":"2024-08-18T23:43:24.307008+0000","last_became_active":"2024-08-18T23:41:39.183176+0000","last_became_peered":"2024-08-18T23:41:39.183176+0000","last_unstale":"2024-08-18T23:43:24.307008+0000","last_undegraded":"2024-08-18T23:43:24.307008+0000","last_fullsized":"2024-08-18T23:43:24.307008+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:36:09.827378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307216+0000","last_change":"2024-08-18T23:41:32.147507+0000","last_active":"2024-08-18T23:43:24.307216+0000","last_peered":"2024-08-18T23:43:24.307216+0000","last_clean":"2024-08-18T23:43:24.307216+0000","last_became_active":"2024-08-18T23:41:32.146042+0000","last_became_peered":"2024-08-18T23:41:32.146042+0000","last_unstale":"2024-08-18T23:43:24.307216+0000","last_undegraded":"2024-08-18T23:43:24.307216+0000","last_fullsized":"2024-08-18T23:43:24.307216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:48.941442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303448+0000","last_change":"2024-08-18T23:41:30.137755+0000","last_active":"2024-08-18T23:43:24.303448+0000","last_peered":"2024-08-18T23:43:24.303448+0000","last_clean":"2024-08-18T23:43:24.303448+0000","last_became_active":"2024-08-18T23:41:30.137194+0000","last_became_peered":"2024-08-18T23:41:30.137194+0000","last_unstale":"2024-08-18T23:43:24.303448+0000","last_undegraded":"2024-08-18T23:43:24.303448+0000","last_fullsized":"2024-08-18T23:43:24.303448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:15:49.681405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301312+0000","last_change":"2024-08-18T23:41:39.183986+0000","last_active":"2024-08-18T23:43:24.301312+0000","last_peered":"2024-08-18T23:43:24.301312+0000","last_clean":"2024-08-18T23:43:24.301312+0000","last_became_active":"2024-08-18T23:41:39.183384+0000","last_became_peered":"2024-08-18T23:41:39.183384+0000","last_unstale":"2024-08-18T23:43:24.301312+0000","last_undegraded":"2024-08-18T23:43:24.301312+0000","last_fullsized":"2024-08-18T23:43:24.301312+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:04:28.920804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304496+0000","last_change":"2024-08-18T23:41:32.134729+0000","last_active":"2024-08-18T23:43:24.304496+0000","last_peered":"2024-08-18T23:43:24.304496+0000","last_clean":"2024-08-18T23:43:24.304496+0000","last_became_active":"2024-08-18T23:41:32.134366+0000","last_became_peered":"2024-08-18T23:41:32.134366+0000","last_unstale":"2024-08-18T23:43:24.304496+0000","last_undegraded":"2024-08-18T23:43:24.304496+0000","last_fullsized":"2024-08-18T23:43:24.304496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:38.610561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'6","reported_seq":162,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:58.430805+0000","last_change":"2024-08-18T23:41:30.126443+0000","last_active":"2024-08-18T23:43:58.430805+0000","last_peered":"2024-08-18T23:43:58.430805+0000","last_clean":"2024-08-18T23:43:58.430805+0000","last_became_active":"2024-08-18T23:41:30.124264+0000","last_became_peered":"2024-08-18T23:41:30.124264+0000","last_unstale":"2024-08-18T23:43:58.430805+0000","last_undegraded":"2024-08-18T23:43:58.430805+0000","last_fullsized":"2024-08-18T23:43:58.430805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:43.727114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1427,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298489+0000","last_change":"2024-08-18T23:41:39.192164+0000","last_active":"2024-08-18T23:43:24.298489+0000","last_peered":"2024-08-18T23:43:24.298489+0000","last_clean":"2024-08-18T23:43:24.298489+0000","last_became_active":"2024-08-18T23:41:39.191833+0000","last_became_peered":"2024-08-18T23:41:39.191833+0000","last_unstale":"2024-08-18T23:43:24.298489+0000","last_undegraded":"2024-08-18T23:43:24.298489+0000","last_fullsized":"2024-08-18T23:43:24.298489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:33:50.243930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304927+0000","last_change":"2024-08-18T23:42:23.320662+0000","last_active":"2024-08-18T23:43:24.304927+0000","last_peered":"2024-08-18T23:43:24.304927+0000","last_clean":"2024-08-18T23:43:24.304927+0000","last_became_active":"2024-08-18T23:42:23.320249+0000","last_became_peered":"2024-08-18T23:42:23.320249+0000","last_unstale":"2024-08-18T23:43:24.304927+0000","last_undegraded":"2024-08-18T23:43:24.304927+0000","last_fullsized":"2024-08-18T23:43:24.304927+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:03:17.121915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307658+0000","last_change":"2024-08-18T23:41:30.124071+0000","last_active":"2024-08-18T23:43:24.307658+0000","last_peered":"2024-08-18T23:43:24.307658+0000","last_clean":"2024-08-18T23:43:24.307658+0000","last_became_active":"2024-08-18T23:41:30.123118+0000","last_became_peered":"2024-08-18T23:41:30.123118+0000","last_unstale":"2024-08-18T23:43:24.307658+0000","last_undegraded":"2024-08-18T23:43:24.307658+0000","last_fullsized":"2024-08-18T23:43:24.307658+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:21:39.015187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312907+0000","last_change":"2024-08-18T23:41:39.184038+0000","last_active":"2024-08-18T23:43:24.312907+0000","last_peered":"2024-08-18T23:43:24.312907+0000","last_clean":"2024-08-18T23:43:24.312907+0000","last_became_active":"2024-08-18T23:41:39.183611+0000","last_became_peered":"2024-08-18T23:41:39.183611+0000","last_unstale":"2024-08-18T23:43:24.312907+0000","last_undegraded":"2024-08-18T23:43:24.312907+0000","last_fullsized":"2024-08-18T23:43:24.312907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:32:26.029151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300998+0000","last_change":"2024-08-18T23:41:32.143631+0000","last_active":"2024-08-18T23:43:24.300998+0000","last_peered":"2024-08-18T23:43:24.300998+0000","last_clean":"2024-08-18T23:43:24.300998+0000","last_became_active":"2024-08-18T23:41:32.143350+0000","last_became_peered":"2024-08-18T23:41:32.143350+0000","last_unstale":"2024-08-18T23:43:24.300998+0000","last_undegraded":"2024-08-18T23:43:24.300998+0000","last_fullsized":"2024-08-18T23:43:24.300998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:41:23.713467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312424+0000","last_change":"2024-08-18T23:41:30.123383+0000","last_active":"2024-08-18T23:43:24.312424+0000","last_peered":"2024-08-18T23:43:24.312424+0000","last_clean":"2024-08-18T23:43:24.312424+0000","last_became_active":"2024-08-18T23:41:30.123037+0000","last_became_peered":"2024-08-18T23:41:30.123037+0000","last_unstale":"2024-08-18T23:43:24.312424+0000","last_undegraded":"2024-08-18T23:43:24.312424+0000","last_fullsized":"2024-08-18T23:43:24.312424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:27.702478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307094+0000","last_change":"2024-08-18T23:41:39.184302+0000","last_active":"2024-08-18T23:43:24.307094+0000","last_peered":"2024-08-18T23:43:24.307094+0000","last_clean":"2024-08-18T23:43:24.307094+0000","last_became_active":"2024-08-18T23:41:39.183172+0000","last_became_peered":"2024-08-18T23:41:39.183172+0000","last_unstale":"2024-08-18T23:43:24.307094+0000","last_undegraded":"2024-08-18T23:43:24.307094+0000","last_fullsized":"2024-08-18T23:43:24.307094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:23:26.655865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304016+0000","last_change":"2024-08-18T23:41:32.137426+0000","last_active":"2024-08-18T23:43:24.304016+0000","last_peered":"2024-08-18T23:43:24.304016+0000","last_clean":"2024-08-18T23:43:24.304016+0000","last_became_active":"2024-08-18T23:41:32.137106+0000","last_became_peered":"2024-08-18T23:41:32.137106+0000","last_unstale":"2024-08-18T23:43:24.304016+0000","last_undegraded":"2024-08-18T23:43:24.304016+0000","last_fullsized":"2024-08-18T23:43:24.304016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:34:51.889663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'39","reported_seq":314,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:59.432216+0000","last_change":"2024-08-18T23:41:30.124320+0000","last_active":"2024-08-18T23:43:59.432216+0000","last_peered":"2024-08-18T23:43:59.432216+0000","last_clean":"2024-08-18T23:43:59.432216+0000","last_became_active":"2024-08-18T23:41:30.123979+0000","last_became_peered":"2024-08-18T23:41:30.123979+0000","last_unstale":"2024-08-18T23:43:59.432216+0000","last_undegraded":"2024-08-18T23:43:59.432216+0000","last_fullsized":"2024-08-18T23:43:59.432216+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:31:03.689720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68257,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":236,"num_read_kb":185,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.297979+0000","last_change":"2024-08-18T23:41:39.190481+0000","last_active":"2024-08-18T23:43:24.297979+0000","last_peered":"2024-08-18T23:43:24.297979+0000","last_clean":"2024-08-18T23:43:24.297979+0000","last_became_active":"2024-08-18T23:41:39.190076+0000","last_became_peered":"2024-08-18T23:41:39.190076+0000","last_unstale":"2024-08-18T23:43:24.297979+0000","last_undegraded":"2024-08-18T23:43:24.297979+0000","last_fullsized":"2024-08-18T23:43:24.297979+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:47:39.313142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312507+0000","last_change":"2024-08-18T23:41:32.145422+0000","last_active":"2024-08-18T23:43:24.312507+0000","last_peered":"2024-08-18T23:43:24.312507+0000","last_clean":"2024-08-18T23:43:24.312507+0000","last_became_active":"2024-08-18T23:41:32.145047+0000","last_became_peered":"2024-08-18T23:41:32.145047+0000","last_unstale":"2024-08-18T23:43:24.312507+0000","last_undegraded":"2024-08-18T23:43:24.312507+0000","last_fullsized":"2024-08-18T23:43:24.312507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:31.905248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307467+0000","last_change":"2024-08-18T23:41:30.137364+0000","last_active":"2024-08-18T23:43:24.307467+0000","last_peered":"2024-08-18T23:43:24.307467+0000","last_clean":"2024-08-18T23:43:24.307467+0000","last_became_active":"2024-08-18T23:41:30.137004+0000","last_became_peered":"2024-08-18T23:41:30.137004+0000","last_unstale":"2024-08-18T23:43:24.307467+0000","last_undegraded":"2024-08-18T23:43:24.307467+0000","last_fullsized":"2024-08-18T23:43:24.307467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:20:56.392149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312779+0000","last_change":"2024-08-18T23:42:23.326359+0000","last_active":"2024-08-18T23:43:24.312779+0000","last_peered":"2024-08-18T23:43:24.312779+0000","last_clean":"2024-08-18T23:43:24.312779+0000","last_became_active":"2024-08-18T23:42:23.325918+0000","last_became_peered":"2024-08-18T23:42:23.325918+0000","last_unstale":"2024-08-18T23:43:24.312779+0000","last_undegraded":"2024-08-18T23:43:24.312779+0000","last_fullsized":"2024-08-18T23:43:24.312779+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:01:15.476268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312633+0000","last_change":"2024-08-18T23:42:23.325691+0000","last_active":"2024-08-18T23:43:24.312633+0000","last_peered":"2024-08-18T23:43:24.312633+0000","last_clean":"2024-08-18T23:43:24.312633+0000","last_became_active":"2024-08-18T23:42:23.325467+0000","last_became_peered":"2024-08-18T23:42:23.325467+0000","last_unstale":"2024-08-18T23:43:24.312633+0000","last_undegraded":"2024-08-18T23:43:24.312633+0000","last_fullsized":"2024-08-18T23:43:24.312633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:10:26.288880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.305106+0000","last_change":"2024-08-18T23:41:30.124430+0000","last_active":"2024-08-18T23:43:24.305106+0000","last_peered":"2024-08-18T23:43:24.305106+0000","last_clean":"2024-08-18T23:43:24.305106+0000","last_became_active":"2024-08-18T23:41:30.123626+0000","last_became_peered":"2024-08-18T23:41:30.123626+0000","last_unstale":"2024-08-18T23:43:24.305106+0000","last_undegraded":"2024-08-18T23:43:24.305106+0000","last_fullsized":"2024-08-18T23:43:24.305106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:26:18.426010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.297793+0000","last_change":"2024-08-18T23:41:39.192765+0000","last_active":"2024-08-18T23:43:24.297793+0000","last_peered":"2024-08-18T23:43:24.297793+0000","last_clean":"2024-08-18T23:43:24.297793+0000","last_became_active":"2024-08-18T23:41:39.192250+0000","last_became_peered":"2024-08-18T23:41:39.192250+0000","last_unstale":"2024-08-18T23:43:24.297793+0000","last_undegraded":"2024-08-18T23:43:24.297793+0000","last_fullsized":"2024-08-18T23:43:24.297793+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:51:16.794404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312691+0000","last_change":"2024-08-18T23:41:32.141966+0000","last_active":"2024-08-18T23:43:24.312691+0000","last_peered":"2024-08-18T23:43:24.312691+0000","last_clean":"2024-08-18T23:43:24.312691+0000","last_became_active":"2024-08-18T23:41:32.141561+0000","last_became_peered":"2024-08-18T23:41:32.141561+0000","last_unstale":"2024-08-18T23:43:24.312691+0000","last_undegraded":"2024-08-18T23:43:24.312691+0000","last_fullsized":"2024-08-18T23:43:24.312691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:20.696089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":156,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:58.429998+0000","last_change":"2024-08-18T23:41:30.121598+0000","last_active":"2024-08-18T23:43:58.429998+0000","last_peered":"2024-08-18T23:43:58.429998+0000","last_clean":"2024-08-18T23:43:58.429998+0000","last_became_active":"2024-08-18T23:41:30.120533+0000","last_became_peered":"2024-08-18T23:41:30.120533+0000","last_unstale":"2024-08-18T23:43:58.429998+0000","last_undegraded":"2024-08-18T23:43:58.429998+0000","last_fullsized":"2024-08-18T23:43:58.429998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:06:37.458272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312220+0000","last_change":"2024-08-18T23:41:39.178877+0000","last_active":"2024-08-18T23:43:24.312220+0000","last_peered":"2024-08-18T23:43:24.312220+0000","last_clean":"2024-08-18T23:43:24.312220+0000","last_became_active":"2024-08-18T23:41:39.178381+0000","last_became_peered":"2024-08-18T23:41:39.178381+0000","last_unstale":"2024-08-18T23:43:24.312220+0000","last_undegraded":"2024-08-18T23:43:24.312220+0000","last_fullsized":"2024-08-18T23:43:24.312220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:27.275893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311702+0000","last_change":"2024-08-18T23:41:39.188381+0000","last_active":"2024-08-18T23:43:24.311702+0000","last_peered":"2024-08-18T23:43:24.311702+0000","last_clean":"2024-08-18T23:43:24.311702+0000","last_became_active":"2024-08-18T23:41:39.187983+0000","last_became_peered":"2024-08-18T23:41:39.187983+0000","last_unstale":"2024-08-18T23:43:24.311702+0000","last_undegraded":"2024-08-18T23:43:24.311702+0000","last_fullsized":"2024-08-18T23:43:24.311702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:04:45.460576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.297935+0000","last_change":"2024-08-18T23:41:32.144637+0000","last_active":"2024-08-18T23:43:24.297935+0000","last_peered":"2024-08-18T23:43:24.297935+0000","last_clean":"2024-08-18T23:43:24.297935+0000","last_became_active":"2024-08-18T23:41:32.144329+0000","last_became_peered":"2024-08-18T23:41:32.144329+0000","last_unstale":"2024-08-18T23:43:24.297935+0000","last_undegraded":"2024-08-18T23:43:24.297935+0000","last_fullsized":"2024-08-18T23:43:24.297935+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:29:39.791264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307199+0000","last_change":"2024-08-18T23:41:30.122325+0000","last_active":"2024-08-18T23:43:24.307199+0000","last_peered":"2024-08-18T23:43:24.307199+0000","last_clean":"2024-08-18T23:43:24.307199+0000","last_became_active":"2024-08-18T23:41:30.121480+0000","last_became_peered":"2024-08-18T23:41:30.121480+0000","last_unstale":"2024-08-18T23:43:24.307199+0000","last_undegraded":"2024-08-18T23:43:24.307199+0000","last_fullsized":"2024-08-18T23:43:24.307199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:32:12.810650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312772+0000","last_change":"2024-08-18T23:42:23.560632+0000","last_active":"2024-08-18T23:43:24.312772+0000","last_peered":"2024-08-18T23:43:24.312772+0000","last_clean":"2024-08-18T23:43:24.312772+0000","last_became_active":"2024-08-18T23:42:23.559920+0000","last_became_peered":"2024-08-18T23:42:23.559920+0000","last_unstale":"2024-08-18T23:43:24.312772+0000","last_undegraded":"2024-08-18T23:43:24.312772+0000","last_fullsized":"2024-08-18T23:43:24.312772+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:01:59.580843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312091+0000","last_change":"2024-08-18T23:41:32.145545+0000","last_active":"2024-08-18T23:43:24.312091+0000","last_peered":"2024-08-18T23:43:24.312091+0000","last_clean":"2024-08-18T23:43:24.312091+0000","last_became_active":"2024-08-18T23:41:32.145183+0000","last_became_peered":"2024-08-18T23:41:32.145183+0000","last_unstale":"2024-08-18T23:43:24.312091+0000","last_undegraded":"2024-08-18T23:43:24.312091+0000","last_fullsized":"2024-08-18T23:43:24.312091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:03:04.111658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300895+0000","last_change":"2024-08-18T23:41:30.118776+0000","last_active":"2024-08-18T23:43:24.300895+0000","last_peered":"2024-08-18T23:43:24.300895+0000","last_clean":"2024-08-18T23:43:24.300895+0000","last_became_active":"2024-08-18T23:41:30.118436+0000","last_became_peered":"2024-08-18T23:41:30.118436+0000","last_unstale":"2024-08-18T23:43:24.300895+0000","last_undegraded":"2024-08-18T23:43:24.300895+0000","last_fullsized":"2024-08-18T23:43:24.300895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:13.980927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300932+0000","last_change":"2024-08-18T23:41:30.125201+0000","last_active":"2024-08-18T23:43:24.300932+0000","last_peered":"2024-08-18T23:43:24.300932+0000","last_clean":"2024-08-18T23:43:24.300932+0000","last_became_active":"2024-08-18T23:41:30.124894+0000","last_became_peered":"2024-08-18T23:41:30.124894+0000","last_unstale":"2024-08-18T23:43:24.300932+0000","last_undegraded":"2024-08-18T23:43:24.300932+0000","last_fullsized":"2024-08-18T23:43:24.300932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:30:17.721829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307745+0000","last_change":"2024-08-18T23:41:32.145452+0000","last_active":"2024-08-18T23:43:24.307745+0000","last_peered":"2024-08-18T23:43:24.307745+0000","last_clean":"2024-08-18T23:43:24.307745+0000","last_became_active":"2024-08-18T23:41:32.145151+0000","last_became_peered":"2024-08-18T23:41:32.145151+0000","last_unstale":"2024-08-18T23:43:24.307745+0000","last_undegraded":"2024-08-18T23:43:24.307745+0000","last_fullsized":"2024-08-18T23:43:24.307745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:08:08.713154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312361+0000","last_change":"2024-08-18T23:41:39.193334+0000","last_active":"2024-08-18T23:43:24.312361+0000","last_peered":"2024-08-18T23:43:24.312361+0000","last_clean":"2024-08-18T23:43:24.312361+0000","last_became_active":"2024-08-18T23:41:39.192924+0000","last_became_peered":"2024-08-18T23:41:39.192924+0000","last_unstale":"2024-08-18T23:43:24.312361+0000","last_undegraded":"2024-08-18T23:43:24.312361+0000","last_fullsized":"2024-08-18T23:43:24.312361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:07.016300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":158,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:55.426672+0000","last_change":"2024-08-18T23:41:30.125217+0000","last_active":"2024-08-18T23:43:55.426672+0000","last_peered":"2024-08-18T23:43:55.426672+0000","last_clean":"2024-08-18T23:43:55.426672+0000","last_became_active":"2024-08-18T23:41:30.124889+0000","last_became_peered":"2024-08-18T23:41:30.124889+0000","last_unstale":"2024-08-18T23:43:55.426672+0000","last_undegraded":"2024-08-18T23:43:55.426672+0000","last_fullsized":"2024-08-18T23:43:55.426672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:43.389537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303483+0000","last_change":"2024-08-18T23:41:32.142585+0000","last_active":"2024-08-18T23:43:24.303483+0000","last_peered":"2024-08-18T23:43:24.303483+0000","last_clean":"2024-08-18T23:43:24.303483+0000","last_became_active":"2024-08-18T23:41:32.142207+0000","last_became_peered":"2024-08-18T23:41:32.142207+0000","last_unstale":"2024-08-18T23:43:24.303483+0000","last_undegraded":"2024-08-18T23:43:24.303483+0000","last_fullsized":"2024-08-18T23:43:24.303483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:16:58.755444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312344+0000","last_change":"2024-08-18T23:41:39.185618+0000","last_active":"2024-08-18T23:43:24.312344+0000","last_peered":"2024-08-18T23:43:24.312344+0000","last_clean":"2024-08-18T23:43:24.312344+0000","last_became_active":"2024-08-18T23:41:39.185222+0000","last_became_peered":"2024-08-18T23:41:39.185222+0000","last_unstale":"2024-08-18T23:43:24.312344+0000","last_undegraded":"2024-08-18T23:43:24.312344+0000","last_fullsized":"2024-08-18T23:43:24.312344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:01:27.290998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312646+0000","last_change":"2024-08-18T23:41:32.140719+0000","last_active":"2024-08-18T23:43:24.312646+0000","last_peered":"2024-08-18T23:43:24.312646+0000","last_clean":"2024-08-18T23:43:24.312646+0000","last_became_active":"2024-08-18T23:41:32.140131+0000","last_became_peered":"2024-08-18T23:41:32.140131+0000","last_unstale":"2024-08-18T23:43:24.312646+0000","last_undegraded":"2024-08-18T23:43:24.312646+0000","last_fullsized":"2024-08-18T23:43:24.312646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:48:26.778094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312375+0000","last_change":"2024-08-18T23:42:23.326683+0000","last_active":"2024-08-18T23:43:24.312375+0000","last_peered":"2024-08-18T23:43:24.312375+0000","last_clean":"2024-08-18T23:43:24.312375+0000","last_became_active":"2024-08-18T23:42:23.326309+0000","last_became_peered":"2024-08-18T23:42:23.326309+0000","last_unstale":"2024-08-18T23:43:24.312375+0000","last_undegraded":"2024-08-18T23:43:24.312375+0000","last_fullsized":"2024-08-18T23:43:24.312375+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:14.320894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312340+0000","last_change":"2024-08-18T23:41:32.149522+0000","last_active":"2024-08-18T23:43:24.312340+0000","last_peered":"2024-08-18T23:43:24.312340+0000","last_clean":"2024-08-18T23:43:24.312340+0000","last_became_active":"2024-08-18T23:41:32.149182+0000","last_became_peered":"2024-08-18T23:41:32.149182+0000","last_unstale":"2024-08-18T23:43:24.312340+0000","last_undegraded":"2024-08-18T23:43:24.312340+0000","last_fullsized":"2024-08-18T23:43:24.312340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:30:16.945045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311765+0000","last_change":"2024-08-18T23:41:32.149853+0000","last_active":"2024-08-18T23:43:24.311765+0000","last_peered":"2024-08-18T23:43:24.311765+0000","last_clean":"2024-08-18T23:43:24.311765+0000","last_became_active":"2024-08-18T23:41:32.141907+0000","last_became_peered":"2024-08-18T23:41:32.141907+0000","last_unstale":"2024-08-18T23:43:24.311765+0000","last_undegraded":"2024-08-18T23:43:24.311765+0000","last_fullsized":"2024-08-18T23:43:24.311765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:24:37.323567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301279+0000","last_change":"2024-08-18T23:41:32.136179+0000","last_active":"2024-08-18T23:43:24.301279+0000","last_peered":"2024-08-18T23:43:24.301279+0000","last_clean":"2024-08-18T23:43:24.301279+0000","last_became_active":"2024-08-18T23:41:32.135939+0000","last_became_peered":"2024-08-18T23:41:32.135939+0000","last_unstale":"2024-08-18T23:43:24.301279+0000","last_undegraded":"2024-08-18T23:43:24.301279+0000","last_fullsized":"2024-08-18T23:43:24.301279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:15.392061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313279+0000","last_change":"2024-08-18T23:41:32.137509+0000","last_active":"2024-08-18T23:43:24.313279+0000","last_peered":"2024-08-18T23:43:24.313279+0000","last_clean":"2024-08-18T23:43:24.313279+0000","last_became_active":"2024-08-18T23:41:32.136380+0000","last_became_peered":"2024-08-18T23:41:32.136380+0000","last_unstale":"2024-08-18T23:43:24.313279+0000","last_undegraded":"2024-08-18T23:43:24.313279+0000","last_fullsized":"2024-08-18T23:43:24.313279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:49.776225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312602+0000","last_change":"2024-08-18T23:41:32.139271+0000","last_active":"2024-08-18T23:43:24.312602+0000","last_peered":"2024-08-18T23:43:24.312602+0000","last_clean":"2024-08-18T23:43:24.312602+0000","last_became_active":"2024-08-18T23:41:32.138868+0000","last_became_peered":"2024-08-18T23:41:32.138868+0000","last_unstale":"2024-08-18T23:43:24.312602+0000","last_undegraded":"2024-08-18T23:43:24.312602+0000","last_fullsized":"2024-08-18T23:43:24.312602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:49:27.890857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312543+0000","last_change":"2024-08-18T23:41:32.144460+0000","last_active":"2024-08-18T23:43:24.312543+0000","last_peered":"2024-08-18T23:43:24.312543+0000","last_clean":"2024-08-18T23:43:24.312543+0000","last_became_active":"2024-08-18T23:41:32.144178+0000","last_became_peered":"2024-08-18T23:41:32.144178+0000","last_unstale":"2024-08-18T23:43:24.312543+0000","last_undegraded":"2024-08-18T23:43:24.312543+0000","last_fullsized":"2024-08-18T23:43:24.312543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:56.252215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.306952+0000","last_change":"2024-08-18T23:41:32.147973+0000","last_active":"2024-08-18T23:43:24.306952+0000","last_peered":"2024-08-18T23:43:24.306952+0000","last_clean":"2024-08-18T23:43:24.306952+0000","last_became_active":"2024-08-18T23:41:32.147621+0000","last_became_peered":"2024-08-18T23:41:32.147621+0000","last_unstale":"2024-08-18T23:43:24.306952+0000","last_undegraded":"2024-08-18T23:43:24.306952+0000","last_fullsized":"2024-08-18T23:43:24.306952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:14.176569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312329+0000","last_change":"2024-08-18T23:41:32.146029+0000","last_active":"2024-08-18T23:43:24.312329+0000","last_peered":"2024-08-18T23:43:24.312329+0000","last_clean":"2024-08-18T23:43:24.312329+0000","last_became_active":"2024-08-18T23:41:32.145584+0000","last_became_peered":"2024-08-18T23:41:32.145584+0000","last_unstale":"2024-08-18T23:43:24.312329+0000","last_undegraded":"2024-08-18T23:43:24.312329+0000","last_fullsized":"2024-08-18T23:43:24.312329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:25:33.884906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312304+0000","last_change":"2024-08-18T23:41:32.145950+0000","last_active":"2024-08-18T23:43:24.312304+0000","last_peered":"2024-08-18T23:43:24.312304+0000","last_clean":"2024-08-18T23:43:24.312304+0000","last_became_active":"2024-08-18T23:41:32.145618+0000","last_became_peered":"2024-08-18T23:41:32.145618+0000","last_unstale":"2024-08-18T23:43:24.312304+0000","last_undegraded":"2024-08-18T23:43:24.312304+0000","last_fullsized":"2024-08-18T23:43:24.312304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:05:58.889240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301154+0000","last_change":"2024-08-18T23:41:32.130684+0000","last_active":"2024-08-18T23:43:24.301154+0000","last_peered":"2024-08-18T23:43:24.301154+0000","last_clean":"2024-08-18T23:43:24.301154+0000","last_became_active":"2024-08-18T23:41:32.130369+0000","last_became_peered":"2024-08-18T23:41:32.130369+0000","last_unstale":"2024-08-18T23:43:24.301154+0000","last_undegraded":"2024-08-18T23:43:24.301154+0000","last_fullsized":"2024-08-18T23:43:24.301154+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:22.537018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313085+0000","last_change":"2024-08-18T23:41:32.137291+0000","last_active":"2024-08-18T23:43:24.313085+0000","last_peered":"2024-08-18T23:43:24.313085+0000","last_clean":"2024-08-18T23:43:24.313085+0000","last_became_active":"2024-08-18T23:41:32.136977+0000","last_became_peered":"2024-08-18T23:41:32.136977+0000","last_unstale":"2024-08-18T23:43:24.313085+0000","last_undegraded":"2024-08-18T23:43:24.313085+0000","last_fullsized":"2024-08-18T23:43:24.313085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:18:45.544760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312711+0000","last_change":"2024-08-18T23:41:32.144434+0000","last_active":"2024-08-18T23:43:24.312711+0000","last_peered":"2024-08-18T23:43:24.312711+0000","last_clean":"2024-08-18T23:43:24.312711+0000","last_became_active":"2024-08-18T23:41:32.144119+0000","last_became_peered":"2024-08-18T23:41:32.144119+0000","last_unstale":"2024-08-18T23:43:24.312711+0000","last_undegraded":"2024-08-18T23:43:24.312711+0000","last_fullsized":"2024-08-18T23:43:24.312711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:50:54.528630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298360+0000","last_change":"2024-08-18T23:41:32.135034+0000","last_active":"2024-08-18T23:43:24.298360+0000","last_peered":"2024-08-18T23:43:24.298360+0000","last_clean":"2024-08-18T23:43:24.298360+0000","last_became_active":"2024-08-18T23:41:32.134700+0000","last_became_peered":"2024-08-18T23:41:32.134700+0000","last_unstale":"2024-08-18T23:43:24.298360+0000","last_undegraded":"2024-08-18T23:43:24.298360+0000","last_fullsized":"2024-08-18T23:43:24.298360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:06:31.263114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313052+0000","last_change":"2024-08-18T23:41:32.141191+0000","last_active":"2024-08-18T23:43:24.313052+0000","last_peered":"2024-08-18T23:43:24.313052+0000","last_clean":"2024-08-18T23:43:24.313052+0000","last_became_active":"2024-08-18T23:41:32.140901+0000","last_became_peered":"2024-08-18T23:41:32.140901+0000","last_unstale":"2024-08-18T23:43:24.313052+0000","last_undegraded":"2024-08-18T23:43:24.313052+0000","last_fullsized":"2024-08-18T23:43:24.313052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:24.849985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311944+0000","last_change":"2024-08-18T23:41:32.142543+0000","last_active":"2024-08-18T23:43:24.311944+0000","last_peered":"2024-08-18T23:43:24.311944+0000","last_clean":"2024-08-18T23:43:24.311944+0000","last_became_active":"2024-08-18T23:41:32.142084+0000","last_became_peered":"2024-08-18T23:41:32.142084+0000","last_unstale":"2024-08-18T23:43:24.311944+0000","last_undegraded":"2024-08-18T23:43:24.311944+0000","last_fullsized":"2024-08-18T23:43:24.311944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:54:18.200016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311562+0000","last_change":"2024-08-18T23:41:32.148968+0000","last_active":"2024-08-18T23:43:24.311562+0000","last_peered":"2024-08-18T23:43:24.311562+0000","last_clean":"2024-08-18T23:43:24.311562+0000","last_became_active":"2024-08-18T23:41:32.148625+0000","last_became_peered":"2024-08-18T23:41:32.148625+0000","last_unstale":"2024-08-18T23:43:24.311562+0000","last_undegraded":"2024-08-18T23:43:24.311562+0000","last_fullsized":"2024-08-18T23:43:24.311562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:11:40.172139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311624+0000","last_change":"2024-08-18T23:41:32.133478+0000","last_active":"2024-08-18T23:43:24.311624+0000","last_peered":"2024-08-18T23:43:24.311624+0000","last_clean":"2024-08-18T23:43:24.311624+0000","last_became_active":"2024-08-18T23:41:32.133149+0000","last_became_peered":"2024-08-18T23:41:32.133149+0000","last_unstale":"2024-08-18T23:43:24.311624+0000","last_undegraded":"2024-08-18T23:43:24.311624+0000","last_fullsized":"2024-08-18T23:43:24.311624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:57:53.713309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303573+0000","last_change":"2024-08-18T23:43:24.303232+0000","last_active":"2024-08-18T23:43:24.303573+0000","last_peered":"2024-08-18T23:43:24.303573+0000","last_clean":"2024-08-18T23:43:24.303573+0000","last_became_active":"2024-08-18T23:43:24.302849+0000","last_became_peered":"2024-08-18T23:43:24.302849+0000","last_unstale":"2024-08-18T23:43:24.303573+0000","last_undegraded":"2024-08-18T23:43:24.303573+0000","last_fullsized":"2024-08-18T23:43:24.303573+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:02:02.479891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313096+0000","last_change":"2024-08-18T23:41:32.136303+0000","last_active":"2024-08-18T23:43:24.313096+0000","last_peered":"2024-08-18T23:43:24.313096+0000","last_clean":"2024-08-18T23:43:24.313096+0000","last_became_active":"2024-08-18T23:41:32.134628+0000","last_became_peered":"2024-08-18T23:41:32.134628+0000","last_unstale":"2024-08-18T23:43:24.313096+0000","last_undegraded":"2024-08-18T23:43:24.313096+0000","last_fullsized":"2024-08-18T23:43:24.313096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:28:51.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304992+0000","last_change":"2024-08-18T23:41:32.137512+0000","last_active":"2024-08-18T23:43:24.304992+0000","last_peered":"2024-08-18T23:43:24.304992+0000","last_clean":"2024-08-18T23:43:24.304992+0000","last_became_active":"2024-08-18T23:41:32.137150+0000","last_became_peered":"2024-08-18T23:41:32.137150+0000","last_unstale":"2024-08-18T23:43:24.304992+0000","last_undegraded":"2024-08-18T23:43:24.304992+0000","last_fullsized":"2024-08-18T23:43:24.304992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:44:08.020780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301258+0000","last_change":"2024-08-18T23:41:32.139229+0000","last_active":"2024-08-18T23:43:24.301258+0000","last_peered":"2024-08-18T23:43:24.301258+0000","last_clean":"2024-08-18T23:43:24.301258+0000","last_became_active":"2024-08-18T23:41:32.138864+0000","last_became_peered":"2024-08-18T23:41:32.138864+0000","last_unstale":"2024-08-18T23:43:24.301258+0000","last_undegraded":"2024-08-18T23:43:24.301258+0000","last_fullsized":"2024-08-18T23:43:24.301258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:54:21.701750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312787+0000","last_change":"2024-08-18T23:41:32.148727+0000","last_active":"2024-08-18T23:43:24.312787+0000","last_peered":"2024-08-18T23:43:24.312787+0000","last_clean":"2024-08-18T23:43:24.312787+0000","last_became_active":"2024-08-18T23:41:32.146828+0000","last_became_peered":"2024-08-18T23:41:32.146828+0000","last_unstale":"2024-08-18T23:43:24.312787+0000","last_undegraded":"2024-08-18T23:43:24.312787+0000","last_fullsized":"2024-08-18T23:43:24.312787+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:51:49.175150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312029+0000","last_change":"2024-08-18T23:41:32.144796+0000","last_active":"2024-08-18T23:43:24.312029+0000","last_peered":"2024-08-18T23:43:24.312029+0000","last_clean":"2024-08-18T23:43:24.312029+0000","last_became_active":"2024-08-18T23:41:32.141931+0000","last_became_peered":"2024-08-18T23:41:32.141931+0000","last_unstale":"2024-08-18T23:43:24.312029+0000","last_undegraded":"2024-08-18T23:43:24.312029+0000","last_fullsized":"2024-08-18T23:43:24.312029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:48:08.428845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304514+0000","last_change":"2024-08-18T23:41:32.135265+0000","last_active":"2024-08-18T23:43:24.304514+0000","last_peered":"2024-08-18T23:43:24.304514+0000","last_clean":"2024-08-18T23:43:24.304514+0000","last_became_active":"2024-08-18T23:41:32.134952+0000","last_became_peered":"2024-08-18T23:41:32.134952+0000","last_unstale":"2024-08-18T23:43:24.304514+0000","last_undegraded":"2024-08-18T23:43:24.304514+0000","last_fullsized":"2024-08-18T23:43:24.304514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:35:14.099536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300727+0000","last_change":"2024-08-18T23:41:32.135941+0000","last_active":"2024-08-18T23:43:24.300727+0000","last_peered":"2024-08-18T23:43:24.300727+0000","last_clean":"2024-08-18T23:43:24.300727+0000","last_became_active":"2024-08-18T23:41:32.135623+0000","last_became_peered":"2024-08-18T23:41:32.135623+0000","last_unstale":"2024-08-18T23:43:24.300727+0000","last_undegraded":"2024-08-18T23:43:24.300727+0000","last_fullsized":"2024-08-18T23:43:24.300727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:43:36.504738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303544+0000","last_change":"2024-08-18T23:41:32.141643+0000","last_active":"2024-08-18T23:43:24.303544+0000","last_peered":"2024-08-18T23:43:24.303544+0000","last_clean":"2024-08-18T23:43:24.303544+0000","last_became_active":"2024-08-18T23:41:32.141301+0000","last_became_peered":"2024-08-18T23:41:32.141301+0000","last_unstale":"2024-08-18T23:43:24.303544+0000","last_undegraded":"2024-08-18T23:43:24.303544+0000","last_fullsized":"2024-08-18T23:43:24.303544+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:23:53.100740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312988+0000","last_change":"2024-08-18T23:41:32.146730+0000","last_active":"2024-08-18T23:43:24.312988+0000","last_peered":"2024-08-18T23:43:24.312988+0000","last_clean":"2024-08-18T23:43:24.312988+0000","last_became_active":"2024-08-18T23:41:32.146252+0000","last_became_peered":"2024-08-18T23:41:32.146252+0000","last_unstale":"2024-08-18T23:43:24.312988+0000","last_undegraded":"2024-08-18T23:43:24.312988+0000","last_fullsized":"2024-08-18T23:43:24.312988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:41.481761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311744+0000","last_change":"2024-08-18T23:41:32.149569+0000","last_active":"2024-08-18T23:43:24.311744+0000","last_peered":"2024-08-18T23:43:24.311744+0000","last_clean":"2024-08-18T23:43:24.311744+0000","last_became_active":"2024-08-18T23:41:32.149277+0000","last_became_peered":"2024-08-18T23:41:32.149277+0000","last_unstale":"2024-08-18T23:43:24.311744+0000","last_undegraded":"2024-08-18T23:43:24.311744+0000","last_fullsized":"2024-08-18T23:43:24.311744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:34:00.547909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311950+0000","last_change":"2024-08-18T23:41:32.137889+0000","last_active":"2024-08-18T23:43:24.311950+0000","last_peered":"2024-08-18T23:43:24.311950+0000","last_clean":"2024-08-18T23:43:24.311950+0000","last_became_active":"2024-08-18T23:41:32.137578+0000","last_became_peered":"2024-08-18T23:41:32.137578+0000","last_unstale":"2024-08-18T23:43:24.311950+0000","last_undegraded":"2024-08-18T23:43:24.311950+0000","last_fullsized":"2024-08-18T23:43:24.311950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:18:09.987838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312999+0000","last_change":"2024-08-18T23:41:32.142084+0000","last_active":"2024-08-18T23:43:24.312999+0000","last_peered":"2024-08-18T23:43:24.312999+0000","last_clean":"2024-08-18T23:43:24.312999+0000","last_became_active":"2024-08-18T23:41:32.141670+0000","last_became_peered":"2024-08-18T23:41:32.141670+0000","last_unstale":"2024-08-18T23:43:24.312999+0000","last_undegraded":"2024-08-18T23:43:24.312999+0000","last_fullsized":"2024-08-18T23:43:24.312999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:03.990351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73237,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":702,"num_read_kb":430,"num_write":185,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":209379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514539,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":523788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776176,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1180152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939700,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":528120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168633,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":528040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397564,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36968,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708184,"statfs":{"total":95995035648,"available":95957180416,"internally_reserved":0,"allocated":1662976,"data_stored":1120524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561090,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1048576,"data_stored":519200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790022,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1667072,"data_stored":1111485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953547,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1064960,"data_stored":525271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":519394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":583654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1118208,"data_stored":582355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":520588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":63245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T23:44:01.904 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-18T23:44:02.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.037+0000 7f5624247640 1 -- 172.21.15.78:0/669294103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f561c1022f0 msgr2=0x7f561c108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:02.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.037+0000 7f5624247640 1 --2- 172.21.15.78:0/669294103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f561c1022f0 0x7f561c108b80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f560c009ca0 tx=0x7f560c02a8c0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.037+0000 7f5624247640 1 -- 172.21.15.78:0/669294103 shutdown_connections 2024-08-18T23:44:02.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.037+0000 7f5624247640 1 --2- 172.21.15.78:0/669294103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f561c1022f0 0x7f561c108b80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.037+0000 7f5624247640 1 --2- 172.21.15.78:0/669294103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f561c101930 0x7f561c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.037+0000 7f5624247640 1 --2- 172.21.15.78:0/669294103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f561c100740 0x7f561c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.037+0000 7f5624247640 1 -- 172.21.15.78:0/669294103 >> 172.21.15.78:0/669294103 conn(0x7f561c0fbea0 msgr2=0x7f561c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:02.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.037+0000 7f5624247640 1 -- 172.21.15.78:0/669294103 shutdown_connections 2024-08-18T23:44:02.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.037+0000 7f5624247640 1 -- 172.21.15.78:0/669294103 wait complete. 2024-08-18T23:44:02.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f5624247640 1 Processor -- start 2024-08-18T23:44:02.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f5624247640 1 -- start start 2024-08-18T23:44:02.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f5624247640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f561c100740 0x7f561c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:02.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f5624247640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f561c101930 0x7f561c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:02.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f5624247640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f561c1022f0 0x7f561c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:02.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f5624247640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f561c10d4d0 con 0x7f561c101930 2024-08-18T23:44:02.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f5624247640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f561c10d350 con 0x7f561c1022f0 2024-08-18T23:44:02.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f5624247640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f561c10d650 con 0x7f561c100740 2024-08-18T23:44:02.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f56217e2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f561c101930 0x7f561c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:02.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f56217e2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f561c101930 0x7f561c198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51264/0 (socket says 172.21.15.78:51264) 2024-08-18T23:44:02.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f56217e2640 1 -- 172.21.15.78:0/13896212 learned_addr learned my addr 172.21.15.78:0/13896212 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:02.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f56227e4640 1 --2- 172.21.15.78:0/13896212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f561c1022f0 0x7f561c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:02.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f56217e2640 1 -- 172.21.15.78:0/13896212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f561c100740 msgr2=0x7f561c197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:02.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f56217e2640 1 --2- 172.21.15.78:0/13896212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f561c100740 0x7f561c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f56217e2640 1 -- 172.21.15.78:0/13896212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f561c1022f0 msgr2=0x7f561c19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:02.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f56217e2640 1 --2- 172.21.15.78:0/13896212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f561c1022f0 0x7f561c19f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f56217e2640 1 -- 172.21.15.78:0/13896212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561c19f730 con 0x7f561c101930 2024-08-18T23:44:02.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f56217e2640 1 --2- 172.21.15.78:0/13896212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f561c101930 0x7f561c198050 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f561000e9f0 tx=0x7f561000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:02.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f561affd640 1 -- 172.21.15.78:0/13896212 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561000cd80 con 0x7f561c101930 2024-08-18T23:44:02.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f561affd640 1 -- 172.21.15.78:0/13896212 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5610010910 con 0x7f561c101930 2024-08-18T23:44:02.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.038+0000 7f561affd640 1 -- 172.21.15.78:0/13896212 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610018990 con 0x7f561c101930 2024-08-18T23:44:02.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.039+0000 7f5624247640 1 -- 172.21.15.78:0/13896212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f561c19fa20 con 0x7f561c101930 2024-08-18T23:44:02.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.039+0000 7f5624247640 1 -- 172.21.15.78:0/13896212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f561c19ff70 con 0x7f561c101930 2024-08-18T23:44:02.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.040+0000 7f5624247640 1 -- 172.21.15.78:0/13896212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f561c107b30 con 0x7f561c101930 2024-08-18T23:44:02.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.041+0000 7f561affd640 1 -- 172.21.15.78:0/13896212 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f5610010ac0 con 0x7f561c101930 2024-08-18T23:44:02.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.041+0000 7f561affd640 1 --2- 172.21.15.78:0/13896212 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f56040a2c20 0x7f56040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:02.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.041+0000 7f5621fe3640 1 --2- 172.21.15.78:0/13896212 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f56040a2c20 0x7f56040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:02.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.041+0000 7f561affd640 1 -- 172.21.15.78:0/13896212 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f56100cf9d0 con 0x7f561c101930 2024-08-18T23:44:02.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.042+0000 7f5621fe3640 1 --2- 172.21.15.78:0/13896212 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f56040a2c20 0x7f56040a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5614005fd0 tx=0x7f561400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:02.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.045+0000 7f561affd640 1 -- 172.21.15.78:0/13896212 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f561009c440 con 0x7f561c101930 2024-08-18T23:44:02.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.212+0000 7f5624247640 1 -- 172.21.15.78:0/13896212 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f561c062360 con 0x7f56040a2c20 2024-08-18T23:44:02.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.220+0000 7f561affd640 1 -- 172.21.15.78:0/13896212 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340487 (secure 0 0 0) 0x7f561c062360 con 0x7f56040a2c20 2024-08-18T23:44:02.226 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:44:02.229 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-08-18T23:44:02.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.224+0000 7f5624247640 1 -- 172.21.15.78:0/13896212 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f56040a2c20 msgr2=0x7f56040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:02.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.224+0000 7f5624247640 1 --2- 172.21.15.78:0/13896212 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f56040a2c20 0x7f56040a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5614005fd0 tx=0x7f561400b410 comp rx=0 tx=0).stop 2024-08-18T23:44:02.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.224+0000 7f5624247640 1 -- 172.21.15.78:0/13896212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f561c101930 msgr2=0x7f561c198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:02.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.224+0000 7f5624247640 1 --2- 172.21.15.78:0/13896212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f561c101930 0x7f561c198050 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f561000e9f0 tx=0x7f561000eeb0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.224+0000 7f5624247640 1 -- 172.21.15.78:0/13896212 shutdown_connections 2024-08-18T23:44:02.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.224+0000 7f5624247640 1 --2- 172.21.15.78:0/13896212 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f56040a2c20 0x7f56040a50e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.224+0000 7f5624247640 1 --2- 172.21.15.78:0/13896212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f561c1022f0 0x7f561c19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.224+0000 7f5624247640 1 --2- 172.21.15.78:0/13896212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f561c101930 0x7f561c198050 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.224+0000 7f5624247640 1 --2- 172.21.15.78:0/13896212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f561c100740 0x7f561c197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.224+0000 7f5624247640 1 -- 172.21.15.78:0/13896212 >> 172.21.15.78:0/13896212 conn(0x7f561c0fbea0 msgr2=0x7f561c104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:02.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.224+0000 7f5624247640 1 -- 172.21.15.78:0/13896212 shutdown_connections 2024-08-18T23:44:02.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.224+0000 7f5624247640 1 -- 172.21.15.78:0/13896212 wait complete. 2024-08-18T23:44:02.237 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-08-18T23:44:02.147383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664229,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":834,"num_read_kb":540,"num_write":371,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":800,"ondisk_log_size":800,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392748,"kb_used_data":14444,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549076,"statfs":{"total":1151940427776,"available":1151538253824,"internally_reserved":0,"allocated":14790656,"data_stored":8254073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5710,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":38,"num_write":12,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003304"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298623+0000","last_change":"2024-08-18T23:41:30.120580+0000","last_active":"2024-08-18T23:43:24.298623+0000","last_peered":"2024-08-18T23:43:24.298623+0000","last_clean":"2024-08-18T23:43:24.298623+0000","last_became_active":"2024-08-18T23:41:30.120266+0000","last_became_peered":"2024-08-18T23:41:30.120266+0000","last_unstale":"2024-08-18T23:43:24.298623+0000","last_undegraded":"2024-08-18T23:43:24.298623+0000","last_fullsized":"2024-08-18T23:43:24.298623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:06:40.997070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312511+0000","last_change":"2024-08-18T23:41:32.143232+0000","last_active":"2024-08-18T23:43:24.312511+0000","last_peered":"2024-08-18T23:43:24.312511+0000","last_clean":"2024-08-18T23:43:24.312511+0000","last_became_active":"2024-08-18T23:41:32.142916+0000","last_became_peered":"2024-08-18T23:41:32.142916+0000","last_unstale":"2024-08-18T23:43:24.312511+0000","last_undegraded":"2024-08-18T23:43:24.312511+0000","last_fullsized":"2024-08-18T23:43:24.312511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:50:54.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312049+0000","last_change":"2024-08-18T23:41:39.188772+0000","last_active":"2024-08-18T23:43:24.312049+0000","last_peered":"2024-08-18T23:43:24.312049+0000","last_clean":"2024-08-18T23:43:24.312049+0000","last_became_active":"2024-08-18T23:41:39.188362+0000","last_became_peered":"2024-08-18T23:41:39.188362+0000","last_unstale":"2024-08-18T23:43:24.312049+0000","last_undegraded":"2024-08-18T23:43:24.312049+0000","last_fullsized":"2024-08-18T23:43:24.312049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:47:50.255543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312628+0000","last_change":"2024-08-18T23:41:30.137179+0000","last_active":"2024-08-18T23:43:24.312628+0000","last_peered":"2024-08-18T23:43:24.312628+0000","last_clean":"2024-08-18T23:43:24.312628+0000","last_became_active":"2024-08-18T23:41:30.136618+0000","last_became_peered":"2024-08-18T23:41:30.136618+0000","last_unstale":"2024-08-18T23:43:24.312628+0000","last_undegraded":"2024-08-18T23:43:24.312628+0000","last_fullsized":"2024-08-18T23:43:24.312628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:59.826364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312531+0000","last_change":"2024-08-18T23:41:32.142075+0000","last_active":"2024-08-18T23:43:24.312531+0000","last_peered":"2024-08-18T23:43:24.312531+0000","last_clean":"2024-08-18T23:43:24.312531+0000","last_became_active":"2024-08-18T23:41:32.141709+0000","last_became_peered":"2024-08-18T23:41:32.141709+0000","last_unstale":"2024-08-18T23:43:24.312531+0000","last_undegraded":"2024-08-18T23:43:24.312531+0000","last_fullsized":"2024-08-18T23:43:24.312531+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:08.324627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312147+0000","last_change":"2024-08-18T23:41:39.184575+0000","last_active":"2024-08-18T23:43:24.312147+0000","last_peered":"2024-08-18T23:43:24.312147+0000","last_clean":"2024-08-18T23:43:24.312147+0000","last_became_active":"2024-08-18T23:41:39.184169+0000","last_became_peered":"2024-08-18T23:41:39.184169+0000","last_unstale":"2024-08-18T23:43:24.312147+0000","last_undegraded":"2024-08-18T23:43:24.312147+0000","last_fullsized":"2024-08-18T23:43:24.312147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:32.218172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304076+0000","last_change":"2024-08-18T23:41:30.121638+0000","last_active":"2024-08-18T23:43:24.304076+0000","last_peered":"2024-08-18T23:43:24.304076+0000","last_clean":"2024-08-18T23:43:24.304076+0000","last_became_active":"2024-08-18T23:41:30.121290+0000","last_became_peered":"2024-08-18T23:41:30.121290+0000","last_unstale":"2024-08-18T23:43:24.304076+0000","last_undegraded":"2024-08-18T23:43:24.304076+0000","last_fullsized":"2024-08-18T23:43:24.304076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:43:55.638780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312374+0000","last_change":"2024-08-18T23:41:32.142177+0000","last_active":"2024-08-18T23:43:24.312374+0000","last_peered":"2024-08-18T23:43:24.312374+0000","last_clean":"2024-08-18T23:43:24.312374+0000","last_became_active":"2024-08-18T23:41:32.141731+0000","last_became_peered":"2024-08-18T23:41:32.141731+0000","last_unstale":"2024-08-18T23:43:24.312374+0000","last_undegraded":"2024-08-18T23:43:24.312374+0000","last_fullsized":"2024-08-18T23:43:24.312374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:36.283299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312604+0000","last_change":"2024-08-18T23:41:39.182931+0000","last_active":"2024-08-18T23:43:24.312604+0000","last_peered":"2024-08-18T23:43:24.312604+0000","last_clean":"2024-08-18T23:43:24.312604+0000","last_became_active":"2024-08-18T23:41:39.182008+0000","last_became_peered":"2024-08-18T23:41:39.182008+0000","last_unstale":"2024-08-18T23:43:24.312604+0000","last_undegraded":"2024-08-18T23:43:24.312604+0000","last_fullsized":"2024-08-18T23:43:24.312604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:58:38.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312863+0000","last_change":"2024-08-18T23:42:23.329762+0000","last_active":"2024-08-18T23:43:24.312863+0000","last_peered":"2024-08-18T23:43:24.312863+0000","last_clean":"2024-08-18T23:43:24.312863+0000","last_became_active":"2024-08-18T23:42:23.325753+0000","last_became_peered":"2024-08-18T23:42:23.325753+0000","last_unstale":"2024-08-18T23:43:24.312863+0000","last_undegraded":"2024-08-18T23:43:24.312863+0000","last_fullsized":"2024-08-18T23:43:24.312863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:19:43.062492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307318+0000","last_change":"2024-08-18T23:41:32.148662+0000","last_active":"2024-08-18T23:43:24.307318+0000","last_peered":"2024-08-18T23:43:24.307318+0000","last_clean":"2024-08-18T23:43:24.307318+0000","last_became_active":"2024-08-18T23:41:32.148318+0000","last_became_peered":"2024-08-18T23:41:32.148318+0000","last_unstale":"2024-08-18T23:43:24.307318+0000","last_undegraded":"2024-08-18T23:43:24.307318+0000","last_fullsized":"2024-08-18T23:43:24.307318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:58.668669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312238+0000","last_change":"2024-08-18T23:41:39.181114+0000","last_active":"2024-08-18T23:43:24.312238+0000","last_peered":"2024-08-18T23:43:24.312238+0000","last_clean":"2024-08-18T23:43:24.312238+0000","last_became_active":"2024-08-18T23:41:39.180599+0000","last_became_peered":"2024-08-18T23:41:39.180599+0000","last_unstale":"2024-08-18T23:43:24.312238+0000","last_undegraded":"2024-08-18T23:43:24.312238+0000","last_fullsized":"2024-08-18T23:43:24.312238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:36:08.749143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304715+0000","last_change":"2024-08-18T23:41:32.144326+0000","last_active":"2024-08-18T23:43:24.304715+0000","last_peered":"2024-08-18T23:43:24.304715+0000","last_clean":"2024-08-18T23:43:24.304715+0000","last_became_active":"2024-08-18T23:41:32.144004+0000","last_became_peered":"2024-08-18T23:41:32.144004+0000","last_unstale":"2024-08-18T23:43:24.304715+0000","last_undegraded":"2024-08-18T23:43:24.304715+0000","last_fullsized":"2024-08-18T23:43:24.304715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:10.918624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303665+0000","last_change":"2024-08-18T23:41:30.125160+0000","last_active":"2024-08-18T23:43:24.303665+0000","last_peered":"2024-08-18T23:43:24.303665+0000","last_clean":"2024-08-18T23:43:24.303665+0000","last_became_active":"2024-08-18T23:41:30.124685+0000","last_became_peered":"2024-08-18T23:41:30.124685+0000","last_unstale":"2024-08-18T23:43:24.303665+0000","last_undegraded":"2024-08-18T23:43:24.303665+0000","last_fullsized":"2024-08-18T23:43:24.303665+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:52:48.997902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313216+0000","last_change":"2024-08-18T23:41:39.681481+0000","last_active":"2024-08-18T23:43:24.313216+0000","last_peered":"2024-08-18T23:43:24.313216+0000","last_clean":"2024-08-18T23:43:24.313216+0000","last_became_active":"2024-08-18T23:41:39.680204+0000","last_became_peered":"2024-08-18T23:41:39.680204+0000","last_unstale":"2024-08-18T23:43:24.313216+0000","last_undegraded":"2024-08-18T23:43:24.313216+0000","last_fullsized":"2024-08-18T23:43:24.313216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:00:05.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312231+0000","last_change":"2024-08-18T23:41:32.151335+0000","last_active":"2024-08-18T23:43:24.312231+0000","last_peered":"2024-08-18T23:43:24.312231+0000","last_clean":"2024-08-18T23:43:24.312231+0000","last_became_active":"2024-08-18T23:41:32.151040+0000","last_became_peered":"2024-08-18T23:41:32.151040+0000","last_unstale":"2024-08-18T23:43:24.312231+0000","last_undegraded":"2024-08-18T23:43:24.312231+0000","last_fullsized":"2024-08-18T23:43:24.312231+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:16.108555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"92'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:41.164918+0000","last_change":"2024-08-18T23:41:30.122301+0000","last_active":"2024-08-18T23:43:41.164918+0000","last_peered":"2024-08-18T23:43:41.164918+0000","last_clean":"2024-08-18T23:43:41.164918+0000","last_became_active":"2024-08-18T23:41:30.121117+0000","last_became_peered":"2024-08-18T23:41:30.121117+0000","last_unstale":"2024-08-18T23:43:41.164918+0000","last_undegraded":"2024-08-18T23:43:41.164918+0000","last_fullsized":"2024-08-18T23:43:41.164918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:40:12.678803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312721+0000","last_change":"2024-08-18T23:41:39.182395+0000","last_active":"2024-08-18T23:43:24.312721+0000","last_peered":"2024-08-18T23:43:24.312721+0000","last_clean":"2024-08-18T23:43:24.312721+0000","last_became_active":"2024-08-18T23:41:39.181979+0000","last_became_peered":"2024-08-18T23:41:39.181979+0000","last_unstale":"2024-08-18T23:43:24.312721+0000","last_undegraded":"2024-08-18T23:43:24.312721+0000","last_fullsized":"2024-08-18T23:43:24.312721+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:04.674503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312810+0000","last_change":"2024-08-18T23:41:32.140464+0000","last_active":"2024-08-18T23:43:24.312810+0000","last_peered":"2024-08-18T23:43:24.312810+0000","last_clean":"2024-08-18T23:43:24.312810+0000","last_became_active":"2024-08-18T23:41:32.139153+0000","last_became_peered":"2024-08-18T23:41:32.139153+0000","last_unstale":"2024-08-18T23:43:24.312810+0000","last_undegraded":"2024-08-18T23:43:24.312810+0000","last_fullsized":"2024-08-18T23:43:24.312810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:27.387680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312414+0000","last_change":"2024-08-18T23:41:30.122787+0000","last_active":"2024-08-18T23:43:24.312414+0000","last_peered":"2024-08-18T23:43:24.312414+0000","last_clean":"2024-08-18T23:43:24.312414+0000","last_became_active":"2024-08-18T23:41:30.122462+0000","last_became_peered":"2024-08-18T23:41:30.122462+0000","last_unstale":"2024-08-18T23:43:24.312414+0000","last_undegraded":"2024-08-18T23:43:24.312414+0000","last_fullsized":"2024-08-18T23:43:24.312414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:55.568831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298325+0000","last_change":"2024-08-18T23:41:39.190511+0000","last_active":"2024-08-18T23:43:24.298325+0000","last_peered":"2024-08-18T23:43:24.298325+0000","last_clean":"2024-08-18T23:43:24.298325+0000","last_became_active":"2024-08-18T23:41:39.190154+0000","last_became_peered":"2024-08-18T23:41:39.190154+0000","last_unstale":"2024-08-18T23:43:24.298325+0000","last_undegraded":"2024-08-18T23:43:24.298325+0000","last_fullsized":"2024-08-18T23:43:24.298325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:33:34.514827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303456+0000","last_change":"2024-08-18T23:41:32.143867+0000","last_active":"2024-08-18T23:43:24.303456+0000","last_peered":"2024-08-18T23:43:24.303456+0000","last_clean":"2024-08-18T23:43:24.303456+0000","last_became_active":"2024-08-18T23:41:32.143527+0000","last_became_peered":"2024-08-18T23:41:32.143527+0000","last_unstale":"2024-08-18T23:43:24.303456+0000","last_undegraded":"2024-08-18T23:43:24.303456+0000","last_fullsized":"2024-08-18T23:43:24.303456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:40:25.902133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312100+0000","last_change":"2024-08-18T23:42:23.325102+0000","last_active":"2024-08-18T23:43:24.312100+0000","last_peered":"2024-08-18T23:43:24.312100+0000","last_clean":"2024-08-18T23:43:24.312100+0000","last_became_active":"2024-08-18T23:42:23.321583+0000","last_became_peered":"2024-08-18T23:42:23.321583+0000","last_unstale":"2024-08-18T23:43:24.312100+0000","last_undegraded":"2024-08-18T23:43:24.312100+0000","last_fullsized":"2024-08-18T23:43:24.312100+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:57:42.546686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312767+0000","last_change":"2024-08-18T23:42:23.559756+0000","last_active":"2024-08-18T23:43:24.312767+0000","last_peered":"2024-08-18T23:43:24.312767+0000","last_clean":"2024-08-18T23:43:24.312767+0000","last_became_active":"2024-08-18T23:42:23.559242+0000","last_became_peered":"2024-08-18T23:42:23.559242+0000","last_unstale":"2024-08-18T23:43:24.312767+0000","last_undegraded":"2024-08-18T23:43:24.312767+0000","last_fullsized":"2024-08-18T23:43:24.312767+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:25:40.790313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312043+0000","last_change":"2024-08-18T23:41:32.139025+0000","last_active":"2024-08-18T23:43:24.312043+0000","last_peered":"2024-08-18T23:43:24.312043+0000","last_clean":"2024-08-18T23:43:24.312043+0000","last_became_active":"2024-08-18T23:41:32.138733+0000","last_became_peered":"2024-08-18T23:41:32.138733+0000","last_unstale":"2024-08-18T23:43:24.312043+0000","last_undegraded":"2024-08-18T23:43:24.312043+0000","last_fullsized":"2024-08-18T23:43:24.312043+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:01:39.826391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313061+0000","last_change":"2024-08-18T23:41:30.477359+0000","last_active":"2024-08-18T23:43:24.313061+0000","last_peered":"2024-08-18T23:43:24.313061+0000","last_clean":"2024-08-18T23:43:24.313061+0000","last_became_active":"2024-08-18T23:41:30.477011+0000","last_became_peered":"2024-08-18T23:41:30.477011+0000","last_unstale":"2024-08-18T23:43:24.313061+0000","last_undegraded":"2024-08-18T23:43:24.313061+0000","last_fullsized":"2024-08-18T23:43:24.313061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:53:10.052859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312145+0000","last_change":"2024-08-18T23:41:39.182140+0000","last_active":"2024-08-18T23:43:24.312145+0000","last_peered":"2024-08-18T23:43:24.312145+0000","last_clean":"2024-08-18T23:43:24.312145+0000","last_became_active":"2024-08-18T23:41:39.181667+0000","last_became_peered":"2024-08-18T23:41:39.181667+0000","last_unstale":"2024-08-18T23:43:24.312145+0000","last_undegraded":"2024-08-18T23:43:24.312145+0000","last_fullsized":"2024-08-18T23:43:24.312145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:02:18.128028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312594+0000","last_change":"2024-08-18T23:41:32.137064+0000","last_active":"2024-08-18T23:43:24.312594+0000","last_peered":"2024-08-18T23:43:24.312594+0000","last_clean":"2024-08-18T23:43:24.312594+0000","last_became_active":"2024-08-18T23:41:32.136738+0000","last_became_peered":"2024-08-18T23:41:32.136738+0000","last_unstale":"2024-08-18T23:43:24.312594+0000","last_undegraded":"2024-08-18T23:43:24.312594+0000","last_fullsized":"2024-08-18T23:43:24.312594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:15:11.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'12","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298408+0000","last_change":"2024-08-18T23:41:30.120042+0000","last_active":"2024-08-18T23:43:24.298408+0000","last_peered":"2024-08-18T23:43:24.298408+0000","last_clean":"2024-08-18T23:43:24.298408+0000","last_became_active":"2024-08-18T23:41:30.119719+0000","last_became_peered":"2024-08-18T23:41:30.119719+0000","last_unstale":"2024-08-18T23:43:24.298408+0000","last_undegraded":"2024-08-18T23:43:24.298408+0000","last_fullsized":"2024-08-18T23:43:24.298408+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:35:44.466330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311900+0000","last_change":"2024-08-18T23:41:39.680405+0000","last_active":"2024-08-18T23:43:24.311900+0000","last_peered":"2024-08-18T23:43:24.311900+0000","last_clean":"2024-08-18T23:43:24.311900+0000","last_became_active":"2024-08-18T23:41:39.679940+0000","last_became_peered":"2024-08-18T23:41:39.679940+0000","last_unstale":"2024-08-18T23:43:24.311900+0000","last_undegraded":"2024-08-18T23:43:24.311900+0000","last_fullsized":"2024-08-18T23:43:24.311900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:15:03.942116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301271+0000","last_change":"2024-08-18T23:41:32.139688+0000","last_active":"2024-08-18T23:43:24.301271+0000","last_peered":"2024-08-18T23:43:24.301271+0000","last_clean":"2024-08-18T23:43:24.301271+0000","last_became_active":"2024-08-18T23:41:32.139397+0000","last_became_peered":"2024-08-18T23:41:32.139397+0000","last_unstale":"2024-08-18T23:43:24.301271+0000","last_undegraded":"2024-08-18T23:43:24.301271+0000","last_fullsized":"2024-08-18T23:43:24.301271+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:03:57.017488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304150+0000","last_change":"2024-08-18T23:41:30.137578+0000","last_active":"2024-08-18T23:43:24.304150+0000","last_peered":"2024-08-18T23:43:24.304150+0000","last_clean":"2024-08-18T23:43:24.304150+0000","last_became_active":"2024-08-18T23:41:30.137246+0000","last_became_peered":"2024-08-18T23:41:30.137246+0000","last_unstale":"2024-08-18T23:43:24.304150+0000","last_undegraded":"2024-08-18T23:43:24.304150+0000","last_fullsized":"2024-08-18T23:43:24.304150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:48:54.839432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312576+0000","last_change":"2024-08-18T23:41:39.186360+0000","last_active":"2024-08-18T23:43:24.312576+0000","last_peered":"2024-08-18T23:43:24.312576+0000","last_clean":"2024-08-18T23:43:24.312576+0000","last_became_active":"2024-08-18T23:41:39.185808+0000","last_became_peered":"2024-08-18T23:41:39.185808+0000","last_unstale":"2024-08-18T23:43:24.312576+0000","last_undegraded":"2024-08-18T23:43:24.312576+0000","last_fullsized":"2024-08-18T23:43:24.312576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:53.891061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300916+0000","last_change":"2024-08-18T23:41:32.146070+0000","last_active":"2024-08-18T23:43:24.300916+0000","last_peered":"2024-08-18T23:43:24.300916+0000","last_clean":"2024-08-18T23:43:24.300916+0000","last_became_active":"2024-08-18T23:41:32.145640+0000","last_became_peered":"2024-08-18T23:41:32.145640+0000","last_unstale":"2024-08-18T23:43:24.300916+0000","last_undegraded":"2024-08-18T23:43:24.300916+0000","last_fullsized":"2024-08-18T23:43:24.300916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:56:21.368762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298278+0000","last_change":"2024-08-18T23:41:30.138565+0000","last_active":"2024-08-18T23:43:24.298278+0000","last_peered":"2024-08-18T23:43:24.298278+0000","last_clean":"2024-08-18T23:43:24.298278+0000","last_became_active":"2024-08-18T23:41:30.138210+0000","last_became_peered":"2024-08-18T23:41:30.138210+0000","last_unstale":"2024-08-18T23:43:24.298278+0000","last_undegraded":"2024-08-18T23:43:24.298278+0000","last_fullsized":"2024-08-18T23:43:24.298278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:42:14.928884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303425+0000","last_change":"2024-08-18T23:41:39.680656+0000","last_active":"2024-08-18T23:43:24.303425+0000","last_peered":"2024-08-18T23:43:24.303425+0000","last_clean":"2024-08-18T23:43:24.303425+0000","last_became_active":"2024-08-18T23:41:39.680203+0000","last_became_peered":"2024-08-18T23:41:39.680203+0000","last_unstale":"2024-08-18T23:43:24.303425+0000","last_undegraded":"2024-08-18T23:43:24.303425+0000","last_fullsized":"2024-08-18T23:43:24.303425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:55:43.138869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300932+0000","last_change":"2024-08-18T23:41:32.130222+0000","last_active":"2024-08-18T23:43:24.300932+0000","last_peered":"2024-08-18T23:43:24.300932+0000","last_clean":"2024-08-18T23:43:24.300932+0000","last_became_active":"2024-08-18T23:41:32.129844+0000","last_became_peered":"2024-08-18T23:41:32.129844+0000","last_unstale":"2024-08-18T23:43:24.300932+0000","last_undegraded":"2024-08-18T23:43:24.300932+0000","last_fullsized":"2024-08-18T23:43:24.300932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:39:21.106343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304850+0000","last_change":"2024-08-18T23:41:30.123142+0000","last_active":"2024-08-18T23:43:24.304850+0000","last_peered":"2024-08-18T23:43:24.304850+0000","last_clean":"2024-08-18T23:43:24.304850+0000","last_became_active":"2024-08-18T23:41:30.121645+0000","last_became_peered":"2024-08-18T23:41:30.121645+0000","last_unstale":"2024-08-18T23:43:24.304850+0000","last_undegraded":"2024-08-18T23:43:24.304850+0000","last_fullsized":"2024-08-18T23:43:24.304850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:07:24.912259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301163+0000","last_change":"2024-08-18T23:41:39.186033+0000","last_active":"2024-08-18T23:43:24.301163+0000","last_peered":"2024-08-18T23:43:24.301163+0000","last_clean":"2024-08-18T23:43:24.301163+0000","last_became_active":"2024-08-18T23:41:39.185451+0000","last_became_peered":"2024-08-18T23:41:39.185451+0000","last_unstale":"2024-08-18T23:43:24.301163+0000","last_undegraded":"2024-08-18T23:43:24.301163+0000","last_fullsized":"2024-08-18T23:43:24.301163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:52:13.556760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.306993+0000","last_change":"2024-08-18T23:41:32.141065+0000","last_active":"2024-08-18T23:43:24.306993+0000","last_peered":"2024-08-18T23:43:24.306993+0000","last_clean":"2024-08-18T23:43:24.306993+0000","last_became_active":"2024-08-18T23:41:32.140732+0000","last_became_peered":"2024-08-18T23:41:32.140732+0000","last_unstale":"2024-08-18T23:43:24.306993+0000","last_undegraded":"2024-08-18T23:43:24.306993+0000","last_fullsized":"2024-08-18T23:43:24.306993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:08:32.287835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":156,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:57.429047+0000","last_change":"2024-08-18T23:41:30.136838+0000","last_active":"2024-08-18T23:43:57.429047+0000","last_peered":"2024-08-18T23:43:57.429047+0000","last_clean":"2024-08-18T23:43:57.429047+0000","last_became_active":"2024-08-18T23:41:30.136523+0000","last_became_peered":"2024-08-18T23:41:30.136523+0000","last_unstale":"2024-08-18T23:43:57.429047+0000","last_undegraded":"2024-08-18T23:43:57.429047+0000","last_fullsized":"2024-08-18T23:43:57.429047+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:56:17.750535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301550+0000","last_change":"2024-08-18T23:41:39.680314+0000","last_active":"2024-08-18T23:43:24.301550+0000","last_peered":"2024-08-18T23:43:24.301550+0000","last_clean":"2024-08-18T23:43:24.301550+0000","last_became_active":"2024-08-18T23:41:39.679920+0000","last_became_peered":"2024-08-18T23:41:39.679920+0000","last_unstale":"2024-08-18T23:43:24.301550+0000","last_undegraded":"2024-08-18T23:43:24.301550+0000","last_fullsized":"2024-08-18T23:43:24.301550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:26:29.866081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300888+0000","last_change":"2024-08-18T23:41:32.139752+0000","last_active":"2024-08-18T23:43:24.300888+0000","last_peered":"2024-08-18T23:43:24.300888+0000","last_clean":"2024-08-18T23:43:24.300888+0000","last_became_active":"2024-08-18T23:41:32.139451+0000","last_became_peered":"2024-08-18T23:41:32.139451+0000","last_unstale":"2024-08-18T23:43:24.300888+0000","last_undegraded":"2024-08-18T23:43:24.300888+0000","last_fullsized":"2024-08-18T23:43:24.300888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:15:13.390529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304894+0000","last_change":"2024-08-18T23:41:30.126513+0000","last_active":"2024-08-18T23:43:24.304894+0000","last_peered":"2024-08-18T23:43:24.304894+0000","last_clean":"2024-08-18T23:43:24.304894+0000","last_became_active":"2024-08-18T23:41:30.126046+0000","last_became_peered":"2024-08-18T23:41:30.126046+0000","last_unstale":"2024-08-18T23:43:24.304894+0000","last_undegraded":"2024-08-18T23:43:24.304894+0000","last_fullsized":"2024-08-18T23:43:24.304894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:49:16.565565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307280+0000","last_change":"2024-08-18T23:41:39.681006+0000","last_active":"2024-08-18T23:43:24.307280+0000","last_peered":"2024-08-18T23:43:24.307280+0000","last_clean":"2024-08-18T23:43:24.307280+0000","last_became_active":"2024-08-18T23:41:39.680409+0000","last_became_peered":"2024-08-18T23:41:39.680409+0000","last_unstale":"2024-08-18T23:43:24.307280+0000","last_undegraded":"2024-08-18T23:43:24.307280+0000","last_fullsized":"2024-08-18T23:43:24.307280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:47.778866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312261+0000","last_change":"2024-08-18T23:41:32.147545+0000","last_active":"2024-08-18T23:43:24.312261+0000","last_peered":"2024-08-18T23:43:24.312261+0000","last_clean":"2024-08-18T23:43:24.312261+0000","last_became_active":"2024-08-18T23:41:32.147238+0000","last_became_peered":"2024-08-18T23:41:32.147238+0000","last_unstale":"2024-08-18T23:43:24.312261+0000","last_undegraded":"2024-08-18T23:43:24.312261+0000","last_fullsized":"2024-08-18T23:43:24.312261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:12:13.589535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313054+0000","last_change":"2024-08-18T23:41:30.120650+0000","last_active":"2024-08-18T23:43:24.313054+0000","last_peered":"2024-08-18T23:43:24.313054+0000","last_clean":"2024-08-18T23:43:24.313054+0000","last_became_active":"2024-08-18T23:41:30.120196+0000","last_became_peered":"2024-08-18T23:41:30.120196+0000","last_unstale":"2024-08-18T23:43:24.313054+0000","last_undegraded":"2024-08-18T23:43:24.313054+0000","last_fullsized":"2024-08-18T23:43:24.313054+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:40:23.370088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304710+0000","last_change":"2024-08-18T23:40:22.013519+0000","last_active":"2024-08-18T23:43:24.304710+0000","last_peered":"2024-08-18T23:43:24.304710+0000","last_clean":"2024-08-18T23:43:24.304710+0000","last_became_active":"2024-08-18T23:40:22.012986+0000","last_became_peered":"2024-08-18T23:40:22.012986+0000","last_unstale":"2024-08-18T23:43:24.304710+0000","last_undegraded":"2024-08-18T23:43:24.304710+0000","last_fullsized":"2024-08-18T23:43:24.304710+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_clean_scrub_stamp":"2024-08-18T23:37:22.388424+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:19.874356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304893+0000","last_change":"2024-08-18T23:42:23.332800+0000","last_active":"2024-08-18T23:43:24.304893+0000","last_peered":"2024-08-18T23:43:24.304893+0000","last_clean":"2024-08-18T23:43:24.304893+0000","last_became_active":"2024-08-18T23:42:23.326441+0000","last_became_peered":"2024-08-18T23:42:23.326441+0000","last_unstale":"2024-08-18T23:43:24.304893+0000","last_undegraded":"2024-08-18T23:43:24.304893+0000","last_fullsized":"2024-08-18T23:43:24.304893+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:13:58.243518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.305065+0000","last_change":"2024-08-18T23:41:32.139907+0000","last_active":"2024-08-18T23:43:24.305065+0000","last_peered":"2024-08-18T23:43:24.305065+0000","last_clean":"2024-08-18T23:43:24.305065+0000","last_became_active":"2024-08-18T23:41:32.137956+0000","last_became_peered":"2024-08-18T23:41:32.137956+0000","last_unstale":"2024-08-18T23:43:24.305065+0000","last_undegraded":"2024-08-18T23:43:24.305065+0000","last_fullsized":"2024-08-18T23:43:24.305065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:10:13.223328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312461+0000","last_change":"2024-08-18T23:42:23.330062+0000","last_active":"2024-08-18T23:43:24.312461+0000","last_peered":"2024-08-18T23:43:24.312461+0000","last_clean":"2024-08-18T23:43:24.312461+0000","last_became_active":"2024-08-18T23:42:23.323196+0000","last_became_peered":"2024-08-18T23:42:23.323196+0000","last_unstale":"2024-08-18T23:43:24.312461+0000","last_undegraded":"2024-08-18T23:43:24.312461+0000","last_fullsized":"2024-08-18T23:43:24.312461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:53:29.059542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312185+0000","last_change":"2024-08-18T23:41:39.185917+0000","last_active":"2024-08-18T23:43:24.312185+0000","last_peered":"2024-08-18T23:43:24.312185+0000","last_clean":"2024-08-18T23:43:24.312185+0000","last_became_active":"2024-08-18T23:41:39.185427+0000","last_became_peered":"2024-08-18T23:41:39.185427+0000","last_unstale":"2024-08-18T23:43:24.312185+0000","last_undegraded":"2024-08-18T23:43:24.312185+0000","last_fullsized":"2024-08-18T23:43:24.312185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:52:13.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.308822+0000","last_change":"2024-08-18T23:43:24.308401+0000","last_active":"2024-08-18T23:43:24.308822+0000","last_peered":"2024-08-18T23:43:24.308822+0000","last_clean":"2024-08-18T23:43:24.308822+0000","last_became_active":"2024-08-18T23:43:24.308081+0000","last_became_peered":"2024-08-18T23:43:24.308081+0000","last_unstale":"2024-08-18T23:43:24.308822+0000","last_undegraded":"2024-08-18T23:43:24.308822+0000","last_fullsized":"2024-08-18T23:43:24.308822+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:28:57.712217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307528+0000","last_change":"2024-08-18T23:41:30.121307+0000","last_active":"2024-08-18T23:43:24.307528+0000","last_peered":"2024-08-18T23:43:24.307528+0000","last_clean":"2024-08-18T23:43:24.307528+0000","last_became_active":"2024-08-18T23:41:30.120932+0000","last_became_peered":"2024-08-18T23:41:30.120932+0000","last_unstale":"2024-08-18T23:43:24.307528+0000","last_undegraded":"2024-08-18T23:43:24.307528+0000","last_fullsized":"2024-08-18T23:43:24.307528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:06:07.742880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311751+0000","last_change":"2024-08-18T23:41:39.184647+0000","last_active":"2024-08-18T23:43:24.311751+0000","last_peered":"2024-08-18T23:43:24.311751+0000","last_clean":"2024-08-18T23:43:24.311751+0000","last_became_active":"2024-08-18T23:41:39.184251+0000","last_became_peered":"2024-08-18T23:41:39.184251+0000","last_unstale":"2024-08-18T23:43:24.311751+0000","last_undegraded":"2024-08-18T23:43:24.311751+0000","last_fullsized":"2024-08-18T23:43:24.311751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:55.200801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.305184+0000","last_change":"2024-08-18T23:41:32.137007+0000","last_active":"2024-08-18T23:43:24.305184+0000","last_peered":"2024-08-18T23:43:24.305184+0000","last_clean":"2024-08-18T23:43:24.305184+0000","last_became_active":"2024-08-18T23:41:32.136684+0000","last_became_peered":"2024-08-18T23:41:32.136684+0000","last_unstale":"2024-08-18T23:43:24.305184+0000","last_undegraded":"2024-08-18T23:43:24.305184+0000","last_fullsized":"2024-08-18T23:43:24.305184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:21:26.584303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301471+0000","last_change":"2024-08-18T23:41:30.121183+0000","last_active":"2024-08-18T23:43:24.301471+0000","last_peered":"2024-08-18T23:43:24.301471+0000","last_clean":"2024-08-18T23:43:24.301471+0000","last_became_active":"2024-08-18T23:41:30.120847+0000","last_became_peered":"2024-08-18T23:41:30.120847+0000","last_unstale":"2024-08-18T23:43:24.301471+0000","last_undegraded":"2024-08-18T23:43:24.301471+0000","last_fullsized":"2024-08-18T23:43:24.301471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:51.025821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.306833+0000","last_change":"2024-08-18T23:41:39.183375+0000","last_active":"2024-08-18T23:43:24.306833+0000","last_peered":"2024-08-18T23:43:24.306833+0000","last_clean":"2024-08-18T23:43:24.306833+0000","last_became_active":"2024-08-18T23:41:39.182339+0000","last_became_peered":"2024-08-18T23:41:39.182339+0000","last_unstale":"2024-08-18T23:43:24.306833+0000","last_undegraded":"2024-08-18T23:43:24.306833+0000","last_fullsized":"2024-08-18T23:43:24.306833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:26.119056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313146+0000","last_change":"2024-08-18T23:41:32.140748+0000","last_active":"2024-08-18T23:43:24.313146+0000","last_peered":"2024-08-18T23:43:24.313146+0000","last_clean":"2024-08-18T23:43:24.313146+0000","last_became_active":"2024-08-18T23:41:32.140306+0000","last_became_peered":"2024-08-18T23:41:32.140306+0000","last_unstale":"2024-08-18T23:43:24.313146+0000","last_undegraded":"2024-08-18T23:43:24.313146+0000","last_fullsized":"2024-08-18T23:43:24.313146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:23:02.371529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:48.359280+0000","last_change":"2024-08-18T23:41:30.121617+0000","last_active":"2024-08-18T23:43:48.359280+0000","last_peered":"2024-08-18T23:43:48.359280+0000","last_clean":"2024-08-18T23:43:48.359280+0000","last_became_active":"2024-08-18T23:41:30.121257+0000","last_became_peered":"2024-08-18T23:41:30.121257+0000","last_unstale":"2024-08-18T23:43:48.359280+0000","last_undegraded":"2024-08-18T23:43:48.359280+0000","last_fullsized":"2024-08-18T23:43:48.359280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:23:42.091553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307008+0000","last_change":"2024-08-18T23:41:39.183833+0000","last_active":"2024-08-18T23:43:24.307008+0000","last_peered":"2024-08-18T23:43:24.307008+0000","last_clean":"2024-08-18T23:43:24.307008+0000","last_became_active":"2024-08-18T23:41:39.183176+0000","last_became_peered":"2024-08-18T23:41:39.183176+0000","last_unstale":"2024-08-18T23:43:24.307008+0000","last_undegraded":"2024-08-18T23:43:24.307008+0000","last_fullsized":"2024-08-18T23:43:24.307008+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:36:09.827378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307216+0000","last_change":"2024-08-18T23:41:32.147507+0000","last_active":"2024-08-18T23:43:24.307216+0000","last_peered":"2024-08-18T23:43:24.307216+0000","last_clean":"2024-08-18T23:43:24.307216+0000","last_became_active":"2024-08-18T23:41:32.146042+0000","last_became_peered":"2024-08-18T23:41:32.146042+0000","last_unstale":"2024-08-18T23:43:24.307216+0000","last_undegraded":"2024-08-18T23:43:24.307216+0000","last_fullsized":"2024-08-18T23:43:24.307216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:48.941442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303448+0000","last_change":"2024-08-18T23:41:30.137755+0000","last_active":"2024-08-18T23:43:24.303448+0000","last_peered":"2024-08-18T23:43:24.303448+0000","last_clean":"2024-08-18T23:43:24.303448+0000","last_became_active":"2024-08-18T23:41:30.137194+0000","last_became_peered":"2024-08-18T23:41:30.137194+0000","last_unstale":"2024-08-18T23:43:24.303448+0000","last_undegraded":"2024-08-18T23:43:24.303448+0000","last_fullsized":"2024-08-18T23:43:24.303448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:15:49.681405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301312+0000","last_change":"2024-08-18T23:41:39.183986+0000","last_active":"2024-08-18T23:43:24.301312+0000","last_peered":"2024-08-18T23:43:24.301312+0000","last_clean":"2024-08-18T23:43:24.301312+0000","last_became_active":"2024-08-18T23:41:39.183384+0000","last_became_peered":"2024-08-18T23:41:39.183384+0000","last_unstale":"2024-08-18T23:43:24.301312+0000","last_undegraded":"2024-08-18T23:43:24.301312+0000","last_fullsized":"2024-08-18T23:43:24.301312+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:04:28.920804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304496+0000","last_change":"2024-08-18T23:41:32.134729+0000","last_active":"2024-08-18T23:43:24.304496+0000","last_peered":"2024-08-18T23:43:24.304496+0000","last_clean":"2024-08-18T23:43:24.304496+0000","last_became_active":"2024-08-18T23:41:32.134366+0000","last_became_peered":"2024-08-18T23:41:32.134366+0000","last_unstale":"2024-08-18T23:43:24.304496+0000","last_undegraded":"2024-08-18T23:43:24.304496+0000","last_fullsized":"2024-08-18T23:43:24.304496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:38.610561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'6","reported_seq":162,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:58.430805+0000","last_change":"2024-08-18T23:41:30.126443+0000","last_active":"2024-08-18T23:43:58.430805+0000","last_peered":"2024-08-18T23:43:58.430805+0000","last_clean":"2024-08-18T23:43:58.430805+0000","last_became_active":"2024-08-18T23:41:30.124264+0000","last_became_peered":"2024-08-18T23:41:30.124264+0000","last_unstale":"2024-08-18T23:43:58.430805+0000","last_undegraded":"2024-08-18T23:43:58.430805+0000","last_fullsized":"2024-08-18T23:43:58.430805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:43.727114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1427,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298489+0000","last_change":"2024-08-18T23:41:39.192164+0000","last_active":"2024-08-18T23:43:24.298489+0000","last_peered":"2024-08-18T23:43:24.298489+0000","last_clean":"2024-08-18T23:43:24.298489+0000","last_became_active":"2024-08-18T23:41:39.191833+0000","last_became_peered":"2024-08-18T23:41:39.191833+0000","last_unstale":"2024-08-18T23:43:24.298489+0000","last_undegraded":"2024-08-18T23:43:24.298489+0000","last_fullsized":"2024-08-18T23:43:24.298489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:33:50.243930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304927+0000","last_change":"2024-08-18T23:42:23.320662+0000","last_active":"2024-08-18T23:43:24.304927+0000","last_peered":"2024-08-18T23:43:24.304927+0000","last_clean":"2024-08-18T23:43:24.304927+0000","last_became_active":"2024-08-18T23:42:23.320249+0000","last_became_peered":"2024-08-18T23:42:23.320249+0000","last_unstale":"2024-08-18T23:43:24.304927+0000","last_undegraded":"2024-08-18T23:43:24.304927+0000","last_fullsized":"2024-08-18T23:43:24.304927+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:03:17.121915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307658+0000","last_change":"2024-08-18T23:41:30.124071+0000","last_active":"2024-08-18T23:43:24.307658+0000","last_peered":"2024-08-18T23:43:24.307658+0000","last_clean":"2024-08-18T23:43:24.307658+0000","last_became_active":"2024-08-18T23:41:30.123118+0000","last_became_peered":"2024-08-18T23:41:30.123118+0000","last_unstale":"2024-08-18T23:43:24.307658+0000","last_undegraded":"2024-08-18T23:43:24.307658+0000","last_fullsized":"2024-08-18T23:43:24.307658+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:21:39.015187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312907+0000","last_change":"2024-08-18T23:41:39.184038+0000","last_active":"2024-08-18T23:43:24.312907+0000","last_peered":"2024-08-18T23:43:24.312907+0000","last_clean":"2024-08-18T23:43:24.312907+0000","last_became_active":"2024-08-18T23:41:39.183611+0000","last_became_peered":"2024-08-18T23:41:39.183611+0000","last_unstale":"2024-08-18T23:43:24.312907+0000","last_undegraded":"2024-08-18T23:43:24.312907+0000","last_fullsized":"2024-08-18T23:43:24.312907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:32:26.029151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300998+0000","last_change":"2024-08-18T23:41:32.143631+0000","last_active":"2024-08-18T23:43:24.300998+0000","last_peered":"2024-08-18T23:43:24.300998+0000","last_clean":"2024-08-18T23:43:24.300998+0000","last_became_active":"2024-08-18T23:41:32.143350+0000","last_became_peered":"2024-08-18T23:41:32.143350+0000","last_unstale":"2024-08-18T23:43:24.300998+0000","last_undegraded":"2024-08-18T23:43:24.300998+0000","last_fullsized":"2024-08-18T23:43:24.300998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:41:23.713467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312424+0000","last_change":"2024-08-18T23:41:30.123383+0000","last_active":"2024-08-18T23:43:24.312424+0000","last_peered":"2024-08-18T23:43:24.312424+0000","last_clean":"2024-08-18T23:43:24.312424+0000","last_became_active":"2024-08-18T23:41:30.123037+0000","last_became_peered":"2024-08-18T23:41:30.123037+0000","last_unstale":"2024-08-18T23:43:24.312424+0000","last_undegraded":"2024-08-18T23:43:24.312424+0000","last_fullsized":"2024-08-18T23:43:24.312424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:27.702478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307094+0000","last_change":"2024-08-18T23:41:39.184302+0000","last_active":"2024-08-18T23:43:24.307094+0000","last_peered":"2024-08-18T23:43:24.307094+0000","last_clean":"2024-08-18T23:43:24.307094+0000","last_became_active":"2024-08-18T23:41:39.183172+0000","last_became_peered":"2024-08-18T23:41:39.183172+0000","last_unstale":"2024-08-18T23:43:24.307094+0000","last_undegraded":"2024-08-18T23:43:24.307094+0000","last_fullsized":"2024-08-18T23:43:24.307094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:23:26.655865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304016+0000","last_change":"2024-08-18T23:41:32.137426+0000","last_active":"2024-08-18T23:43:24.304016+0000","last_peered":"2024-08-18T23:43:24.304016+0000","last_clean":"2024-08-18T23:43:24.304016+0000","last_became_active":"2024-08-18T23:41:32.137106+0000","last_became_peered":"2024-08-18T23:41:32.137106+0000","last_unstale":"2024-08-18T23:43:24.304016+0000","last_undegraded":"2024-08-18T23:43:24.304016+0000","last_fullsized":"2024-08-18T23:43:24.304016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:34:51.889663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'39","reported_seq":314,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:59.432216+0000","last_change":"2024-08-18T23:41:30.124320+0000","last_active":"2024-08-18T23:43:59.432216+0000","last_peered":"2024-08-18T23:43:59.432216+0000","last_clean":"2024-08-18T23:43:59.432216+0000","last_became_active":"2024-08-18T23:41:30.123979+0000","last_became_peered":"2024-08-18T23:41:30.123979+0000","last_unstale":"2024-08-18T23:43:59.432216+0000","last_undegraded":"2024-08-18T23:43:59.432216+0000","last_fullsized":"2024-08-18T23:43:59.432216+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:31:03.689720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68257,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":236,"num_read_kb":185,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.297979+0000","last_change":"2024-08-18T23:41:39.190481+0000","last_active":"2024-08-18T23:43:24.297979+0000","last_peered":"2024-08-18T23:43:24.297979+0000","last_clean":"2024-08-18T23:43:24.297979+0000","last_became_active":"2024-08-18T23:41:39.190076+0000","last_became_peered":"2024-08-18T23:41:39.190076+0000","last_unstale":"2024-08-18T23:43:24.297979+0000","last_undegraded":"2024-08-18T23:43:24.297979+0000","last_fullsized":"2024-08-18T23:43:24.297979+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:47:39.313142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312507+0000","last_change":"2024-08-18T23:41:32.145422+0000","last_active":"2024-08-18T23:43:24.312507+0000","last_peered":"2024-08-18T23:43:24.312507+0000","last_clean":"2024-08-18T23:43:24.312507+0000","last_became_active":"2024-08-18T23:41:32.145047+0000","last_became_peered":"2024-08-18T23:41:32.145047+0000","last_unstale":"2024-08-18T23:43:24.312507+0000","last_undegraded":"2024-08-18T23:43:24.312507+0000","last_fullsized":"2024-08-18T23:43:24.312507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:31.905248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307467+0000","last_change":"2024-08-18T23:41:30.137364+0000","last_active":"2024-08-18T23:43:24.307467+0000","last_peered":"2024-08-18T23:43:24.307467+0000","last_clean":"2024-08-18T23:43:24.307467+0000","last_became_active":"2024-08-18T23:41:30.137004+0000","last_became_peered":"2024-08-18T23:41:30.137004+0000","last_unstale":"2024-08-18T23:43:24.307467+0000","last_undegraded":"2024-08-18T23:43:24.307467+0000","last_fullsized":"2024-08-18T23:43:24.307467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:20:56.392149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312779+0000","last_change":"2024-08-18T23:42:23.326359+0000","last_active":"2024-08-18T23:43:24.312779+0000","last_peered":"2024-08-18T23:43:24.312779+0000","last_clean":"2024-08-18T23:43:24.312779+0000","last_became_active":"2024-08-18T23:42:23.325918+0000","last_became_peered":"2024-08-18T23:42:23.325918+0000","last_unstale":"2024-08-18T23:43:24.312779+0000","last_undegraded":"2024-08-18T23:43:24.312779+0000","last_fullsized":"2024-08-18T23:43:24.312779+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:01:15.476268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312633+0000","last_change":"2024-08-18T23:42:23.325691+0000","last_active":"2024-08-18T23:43:24.312633+0000","last_peered":"2024-08-18T23:43:24.312633+0000","last_clean":"2024-08-18T23:43:24.312633+0000","last_became_active":"2024-08-18T23:42:23.325467+0000","last_became_peered":"2024-08-18T23:42:23.325467+0000","last_unstale":"2024-08-18T23:43:24.312633+0000","last_undegraded":"2024-08-18T23:43:24.312633+0000","last_fullsized":"2024-08-18T23:43:24.312633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:10:26.288880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.305106+0000","last_change":"2024-08-18T23:41:30.124430+0000","last_active":"2024-08-18T23:43:24.305106+0000","last_peered":"2024-08-18T23:43:24.305106+0000","last_clean":"2024-08-18T23:43:24.305106+0000","last_became_active":"2024-08-18T23:41:30.123626+0000","last_became_peered":"2024-08-18T23:41:30.123626+0000","last_unstale":"2024-08-18T23:43:24.305106+0000","last_undegraded":"2024-08-18T23:43:24.305106+0000","last_fullsized":"2024-08-18T23:43:24.305106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:26:18.426010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.297793+0000","last_change":"2024-08-18T23:41:39.192765+0000","last_active":"2024-08-18T23:43:24.297793+0000","last_peered":"2024-08-18T23:43:24.297793+0000","last_clean":"2024-08-18T23:43:24.297793+0000","last_became_active":"2024-08-18T23:41:39.192250+0000","last_became_peered":"2024-08-18T23:41:39.192250+0000","last_unstale":"2024-08-18T23:43:24.297793+0000","last_undegraded":"2024-08-18T23:43:24.297793+0000","last_fullsized":"2024-08-18T23:43:24.297793+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:51:16.794404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312750+0000","last_change":"2024-08-18T23:41:32.141966+0000","last_active":"2024-08-18T23:43:24.312750+0000","last_peered":"2024-08-18T23:43:24.312750+0000","last_clean":"2024-08-18T23:43:24.312750+0000","last_became_active":"2024-08-18T23:41:32.141561+0000","last_became_peered":"2024-08-18T23:41:32.141561+0000","last_unstale":"2024-08-18T23:43:24.312750+0000","last_undegraded":"2024-08-18T23:43:24.312750+0000","last_fullsized":"2024-08-18T23:43:24.312750+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:20.696089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":156,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:58.429998+0000","last_change":"2024-08-18T23:41:30.121598+0000","last_active":"2024-08-18T23:43:58.429998+0000","last_peered":"2024-08-18T23:43:58.429998+0000","last_clean":"2024-08-18T23:43:58.429998+0000","last_became_active":"2024-08-18T23:41:30.120533+0000","last_became_peered":"2024-08-18T23:41:30.120533+0000","last_unstale":"2024-08-18T23:43:58.429998+0000","last_undegraded":"2024-08-18T23:43:58.429998+0000","last_fullsized":"2024-08-18T23:43:58.429998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:06:37.458272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312220+0000","last_change":"2024-08-18T23:41:39.178877+0000","last_active":"2024-08-18T23:43:24.312220+0000","last_peered":"2024-08-18T23:43:24.312220+0000","last_clean":"2024-08-18T23:43:24.312220+0000","last_became_active":"2024-08-18T23:41:39.178381+0000","last_became_peered":"2024-08-18T23:41:39.178381+0000","last_unstale":"2024-08-18T23:43:24.312220+0000","last_undegraded":"2024-08-18T23:43:24.312220+0000","last_fullsized":"2024-08-18T23:43:24.312220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:27.275893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311702+0000","last_change":"2024-08-18T23:41:39.188381+0000","last_active":"2024-08-18T23:43:24.311702+0000","last_peered":"2024-08-18T23:43:24.311702+0000","last_clean":"2024-08-18T23:43:24.311702+0000","last_became_active":"2024-08-18T23:41:39.187983+0000","last_became_peered":"2024-08-18T23:41:39.187983+0000","last_unstale":"2024-08-18T23:43:24.311702+0000","last_undegraded":"2024-08-18T23:43:24.311702+0000","last_fullsized":"2024-08-18T23:43:24.311702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:04:45.460576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.297935+0000","last_change":"2024-08-18T23:41:32.144637+0000","last_active":"2024-08-18T23:43:24.297935+0000","last_peered":"2024-08-18T23:43:24.297935+0000","last_clean":"2024-08-18T23:43:24.297935+0000","last_became_active":"2024-08-18T23:41:32.144329+0000","last_became_peered":"2024-08-18T23:41:32.144329+0000","last_unstale":"2024-08-18T23:43:24.297935+0000","last_undegraded":"2024-08-18T23:43:24.297935+0000","last_fullsized":"2024-08-18T23:43:24.297935+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:29:39.791264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307199+0000","last_change":"2024-08-18T23:41:30.122325+0000","last_active":"2024-08-18T23:43:24.307199+0000","last_peered":"2024-08-18T23:43:24.307199+0000","last_clean":"2024-08-18T23:43:24.307199+0000","last_became_active":"2024-08-18T23:41:30.121480+0000","last_became_peered":"2024-08-18T23:41:30.121480+0000","last_unstale":"2024-08-18T23:43:24.307199+0000","last_undegraded":"2024-08-18T23:43:24.307199+0000","last_fullsized":"2024-08-18T23:43:24.307199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:32:12.810650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312772+0000","last_change":"2024-08-18T23:42:23.560632+0000","last_active":"2024-08-18T23:43:24.312772+0000","last_peered":"2024-08-18T23:43:24.312772+0000","last_clean":"2024-08-18T23:43:24.312772+0000","last_became_active":"2024-08-18T23:42:23.559920+0000","last_became_peered":"2024-08-18T23:42:23.559920+0000","last_unstale":"2024-08-18T23:43:24.312772+0000","last_undegraded":"2024-08-18T23:43:24.312772+0000","last_fullsized":"2024-08-18T23:43:24.312772+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:01:59.580843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312091+0000","last_change":"2024-08-18T23:41:32.145545+0000","last_active":"2024-08-18T23:43:24.312091+0000","last_peered":"2024-08-18T23:43:24.312091+0000","last_clean":"2024-08-18T23:43:24.312091+0000","last_became_active":"2024-08-18T23:41:32.145183+0000","last_became_peered":"2024-08-18T23:41:32.145183+0000","last_unstale":"2024-08-18T23:43:24.312091+0000","last_undegraded":"2024-08-18T23:43:24.312091+0000","last_fullsized":"2024-08-18T23:43:24.312091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:03:04.111658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300895+0000","last_change":"2024-08-18T23:41:30.118776+0000","last_active":"2024-08-18T23:43:24.300895+0000","last_peered":"2024-08-18T23:43:24.300895+0000","last_clean":"2024-08-18T23:43:24.300895+0000","last_became_active":"2024-08-18T23:41:30.118436+0000","last_became_peered":"2024-08-18T23:41:30.118436+0000","last_unstale":"2024-08-18T23:43:24.300895+0000","last_undegraded":"2024-08-18T23:43:24.300895+0000","last_fullsized":"2024-08-18T23:43:24.300895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:13.980927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300932+0000","last_change":"2024-08-18T23:41:30.125201+0000","last_active":"2024-08-18T23:43:24.300932+0000","last_peered":"2024-08-18T23:43:24.300932+0000","last_clean":"2024-08-18T23:43:24.300932+0000","last_became_active":"2024-08-18T23:41:30.124894+0000","last_became_peered":"2024-08-18T23:41:30.124894+0000","last_unstale":"2024-08-18T23:43:24.300932+0000","last_undegraded":"2024-08-18T23:43:24.300932+0000","last_fullsized":"2024-08-18T23:43:24.300932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:30:17.721829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307776+0000","last_change":"2024-08-18T23:41:32.145452+0000","last_active":"2024-08-18T23:43:24.307776+0000","last_peered":"2024-08-18T23:43:24.307776+0000","last_clean":"2024-08-18T23:43:24.307776+0000","last_became_active":"2024-08-18T23:41:32.145151+0000","last_became_peered":"2024-08-18T23:41:32.145151+0000","last_unstale":"2024-08-18T23:43:24.307776+0000","last_undegraded":"2024-08-18T23:43:24.307776+0000","last_fullsized":"2024-08-18T23:43:24.307776+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:08:08.713154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312361+0000","last_change":"2024-08-18T23:41:39.193334+0000","last_active":"2024-08-18T23:43:24.312361+0000","last_peered":"2024-08-18T23:43:24.312361+0000","last_clean":"2024-08-18T23:43:24.312361+0000","last_became_active":"2024-08-18T23:41:39.192924+0000","last_became_peered":"2024-08-18T23:41:39.192924+0000","last_unstale":"2024-08-18T23:43:24.312361+0000","last_undegraded":"2024-08-18T23:43:24.312361+0000","last_fullsized":"2024-08-18T23:43:24.312361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:07.016300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":166,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:44:01.435799+0000","last_change":"2024-08-18T23:41:30.125217+0000","last_active":"2024-08-18T23:44:01.435799+0000","last_peered":"2024-08-18T23:44:01.435799+0000","last_clean":"2024-08-18T23:44:01.435799+0000","last_became_active":"2024-08-18T23:41:30.124889+0000","last_became_peered":"2024-08-18T23:41:30.124889+0000","last_unstale":"2024-08-18T23:44:01.435799+0000","last_undegraded":"2024-08-18T23:44:01.435799+0000","last_fullsized":"2024-08-18T23:44:01.435799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:43.389537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303483+0000","last_change":"2024-08-18T23:41:32.142585+0000","last_active":"2024-08-18T23:43:24.303483+0000","last_peered":"2024-08-18T23:43:24.303483+0000","last_clean":"2024-08-18T23:43:24.303483+0000","last_became_active":"2024-08-18T23:41:32.142207+0000","last_became_peered":"2024-08-18T23:41:32.142207+0000","last_unstale":"2024-08-18T23:43:24.303483+0000","last_undegraded":"2024-08-18T23:43:24.303483+0000","last_fullsized":"2024-08-18T23:43:24.303483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:16:58.755444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312344+0000","last_change":"2024-08-18T23:41:39.185618+0000","last_active":"2024-08-18T23:43:24.312344+0000","last_peered":"2024-08-18T23:43:24.312344+0000","last_clean":"2024-08-18T23:43:24.312344+0000","last_became_active":"2024-08-18T23:41:39.185222+0000","last_became_peered":"2024-08-18T23:41:39.185222+0000","last_unstale":"2024-08-18T23:43:24.312344+0000","last_undegraded":"2024-08-18T23:43:24.312344+0000","last_fullsized":"2024-08-18T23:43:24.312344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:01:27.290998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312646+0000","last_change":"2024-08-18T23:41:32.140719+0000","last_active":"2024-08-18T23:43:24.312646+0000","last_peered":"2024-08-18T23:43:24.312646+0000","last_clean":"2024-08-18T23:43:24.312646+0000","last_became_active":"2024-08-18T23:41:32.140131+0000","last_became_peered":"2024-08-18T23:41:32.140131+0000","last_unstale":"2024-08-18T23:43:24.312646+0000","last_undegraded":"2024-08-18T23:43:24.312646+0000","last_fullsized":"2024-08-18T23:43:24.312646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:48:26.778094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312375+0000","last_change":"2024-08-18T23:42:23.326683+0000","last_active":"2024-08-18T23:43:24.312375+0000","last_peered":"2024-08-18T23:43:24.312375+0000","last_clean":"2024-08-18T23:43:24.312375+0000","last_became_active":"2024-08-18T23:42:23.326309+0000","last_became_peered":"2024-08-18T23:42:23.326309+0000","last_unstale":"2024-08-18T23:43:24.312375+0000","last_undegraded":"2024-08-18T23:43:24.312375+0000","last_fullsized":"2024-08-18T23:43:24.312375+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:14.320894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312340+0000","last_change":"2024-08-18T23:41:32.149522+0000","last_active":"2024-08-18T23:43:24.312340+0000","last_peered":"2024-08-18T23:43:24.312340+0000","last_clean":"2024-08-18T23:43:24.312340+0000","last_became_active":"2024-08-18T23:41:32.149182+0000","last_became_peered":"2024-08-18T23:41:32.149182+0000","last_unstale":"2024-08-18T23:43:24.312340+0000","last_undegraded":"2024-08-18T23:43:24.312340+0000","last_fullsized":"2024-08-18T23:43:24.312340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:30:16.945045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311765+0000","last_change":"2024-08-18T23:41:32.149853+0000","last_active":"2024-08-18T23:43:24.311765+0000","last_peered":"2024-08-18T23:43:24.311765+0000","last_clean":"2024-08-18T23:43:24.311765+0000","last_became_active":"2024-08-18T23:41:32.141907+0000","last_became_peered":"2024-08-18T23:41:32.141907+0000","last_unstale":"2024-08-18T23:43:24.311765+0000","last_undegraded":"2024-08-18T23:43:24.311765+0000","last_fullsized":"2024-08-18T23:43:24.311765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:24:37.323567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301279+0000","last_change":"2024-08-18T23:41:32.136179+0000","last_active":"2024-08-18T23:43:24.301279+0000","last_peered":"2024-08-18T23:43:24.301279+0000","last_clean":"2024-08-18T23:43:24.301279+0000","last_became_active":"2024-08-18T23:41:32.135939+0000","last_became_peered":"2024-08-18T23:41:32.135939+0000","last_unstale":"2024-08-18T23:43:24.301279+0000","last_undegraded":"2024-08-18T23:43:24.301279+0000","last_fullsized":"2024-08-18T23:43:24.301279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:15.392061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313279+0000","last_change":"2024-08-18T23:41:32.137509+0000","last_active":"2024-08-18T23:43:24.313279+0000","last_peered":"2024-08-18T23:43:24.313279+0000","last_clean":"2024-08-18T23:43:24.313279+0000","last_became_active":"2024-08-18T23:41:32.136380+0000","last_became_peered":"2024-08-18T23:41:32.136380+0000","last_unstale":"2024-08-18T23:43:24.313279+0000","last_undegraded":"2024-08-18T23:43:24.313279+0000","last_fullsized":"2024-08-18T23:43:24.313279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:49.776225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312602+0000","last_change":"2024-08-18T23:41:32.139271+0000","last_active":"2024-08-18T23:43:24.312602+0000","last_peered":"2024-08-18T23:43:24.312602+0000","last_clean":"2024-08-18T23:43:24.312602+0000","last_became_active":"2024-08-18T23:41:32.138868+0000","last_became_peered":"2024-08-18T23:41:32.138868+0000","last_unstale":"2024-08-18T23:43:24.312602+0000","last_undegraded":"2024-08-18T23:43:24.312602+0000","last_fullsized":"2024-08-18T23:43:24.312602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:49:27.890857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312543+0000","last_change":"2024-08-18T23:41:32.144460+0000","last_active":"2024-08-18T23:43:24.312543+0000","last_peered":"2024-08-18T23:43:24.312543+0000","last_clean":"2024-08-18T23:43:24.312543+0000","last_became_active":"2024-08-18T23:41:32.144178+0000","last_became_peered":"2024-08-18T23:41:32.144178+0000","last_unstale":"2024-08-18T23:43:24.312543+0000","last_undegraded":"2024-08-18T23:43:24.312543+0000","last_fullsized":"2024-08-18T23:43:24.312543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:56.252215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.306952+0000","last_change":"2024-08-18T23:41:32.147973+0000","last_active":"2024-08-18T23:43:24.306952+0000","last_peered":"2024-08-18T23:43:24.306952+0000","last_clean":"2024-08-18T23:43:24.306952+0000","last_became_active":"2024-08-18T23:41:32.147621+0000","last_became_peered":"2024-08-18T23:41:32.147621+0000","last_unstale":"2024-08-18T23:43:24.306952+0000","last_undegraded":"2024-08-18T23:43:24.306952+0000","last_fullsized":"2024-08-18T23:43:24.306952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:14.176569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312329+0000","last_change":"2024-08-18T23:41:32.146029+0000","last_active":"2024-08-18T23:43:24.312329+0000","last_peered":"2024-08-18T23:43:24.312329+0000","last_clean":"2024-08-18T23:43:24.312329+0000","last_became_active":"2024-08-18T23:41:32.145584+0000","last_became_peered":"2024-08-18T23:41:32.145584+0000","last_unstale":"2024-08-18T23:43:24.312329+0000","last_undegraded":"2024-08-18T23:43:24.312329+0000","last_fullsized":"2024-08-18T23:43:24.312329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:25:33.884906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312304+0000","last_change":"2024-08-18T23:41:32.145950+0000","last_active":"2024-08-18T23:43:24.312304+0000","last_peered":"2024-08-18T23:43:24.312304+0000","last_clean":"2024-08-18T23:43:24.312304+0000","last_became_active":"2024-08-18T23:41:32.145618+0000","last_became_peered":"2024-08-18T23:41:32.145618+0000","last_unstale":"2024-08-18T23:43:24.312304+0000","last_undegraded":"2024-08-18T23:43:24.312304+0000","last_fullsized":"2024-08-18T23:43:24.312304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:05:58.889240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301154+0000","last_change":"2024-08-18T23:41:32.130684+0000","last_active":"2024-08-18T23:43:24.301154+0000","last_peered":"2024-08-18T23:43:24.301154+0000","last_clean":"2024-08-18T23:43:24.301154+0000","last_became_active":"2024-08-18T23:41:32.130369+0000","last_became_peered":"2024-08-18T23:41:32.130369+0000","last_unstale":"2024-08-18T23:43:24.301154+0000","last_undegraded":"2024-08-18T23:43:24.301154+0000","last_fullsized":"2024-08-18T23:43:24.301154+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:22.537018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313085+0000","last_change":"2024-08-18T23:41:32.137291+0000","last_active":"2024-08-18T23:43:24.313085+0000","last_peered":"2024-08-18T23:43:24.313085+0000","last_clean":"2024-08-18T23:43:24.313085+0000","last_became_active":"2024-08-18T23:41:32.136977+0000","last_became_peered":"2024-08-18T23:41:32.136977+0000","last_unstale":"2024-08-18T23:43:24.313085+0000","last_undegraded":"2024-08-18T23:43:24.313085+0000","last_fullsized":"2024-08-18T23:43:24.313085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:18:45.544760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312711+0000","last_change":"2024-08-18T23:41:32.144434+0000","last_active":"2024-08-18T23:43:24.312711+0000","last_peered":"2024-08-18T23:43:24.312711+0000","last_clean":"2024-08-18T23:43:24.312711+0000","last_became_active":"2024-08-18T23:41:32.144119+0000","last_became_peered":"2024-08-18T23:41:32.144119+0000","last_unstale":"2024-08-18T23:43:24.312711+0000","last_undegraded":"2024-08-18T23:43:24.312711+0000","last_fullsized":"2024-08-18T23:43:24.312711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:50:54.528630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298360+0000","last_change":"2024-08-18T23:41:32.135034+0000","last_active":"2024-08-18T23:43:24.298360+0000","last_peered":"2024-08-18T23:43:24.298360+0000","last_clean":"2024-08-18T23:43:24.298360+0000","last_became_active":"2024-08-18T23:41:32.134700+0000","last_became_peered":"2024-08-18T23:41:32.134700+0000","last_unstale":"2024-08-18T23:43:24.298360+0000","last_undegraded":"2024-08-18T23:43:24.298360+0000","last_fullsized":"2024-08-18T23:43:24.298360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:06:31.263114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313052+0000","last_change":"2024-08-18T23:41:32.141191+0000","last_active":"2024-08-18T23:43:24.313052+0000","last_peered":"2024-08-18T23:43:24.313052+0000","last_clean":"2024-08-18T23:43:24.313052+0000","last_became_active":"2024-08-18T23:41:32.140901+0000","last_became_peered":"2024-08-18T23:41:32.140901+0000","last_unstale":"2024-08-18T23:43:24.313052+0000","last_undegraded":"2024-08-18T23:43:24.313052+0000","last_fullsized":"2024-08-18T23:43:24.313052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:24.849985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311944+0000","last_change":"2024-08-18T23:41:32.142543+0000","last_active":"2024-08-18T23:43:24.311944+0000","last_peered":"2024-08-18T23:43:24.311944+0000","last_clean":"2024-08-18T23:43:24.311944+0000","last_became_active":"2024-08-18T23:41:32.142084+0000","last_became_peered":"2024-08-18T23:41:32.142084+0000","last_unstale":"2024-08-18T23:43:24.311944+0000","last_undegraded":"2024-08-18T23:43:24.311944+0000","last_fullsized":"2024-08-18T23:43:24.311944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:54:18.200016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311562+0000","last_change":"2024-08-18T23:41:32.148968+0000","last_active":"2024-08-18T23:43:24.311562+0000","last_peered":"2024-08-18T23:43:24.311562+0000","last_clean":"2024-08-18T23:43:24.311562+0000","last_became_active":"2024-08-18T23:41:32.148625+0000","last_became_peered":"2024-08-18T23:41:32.148625+0000","last_unstale":"2024-08-18T23:43:24.311562+0000","last_undegraded":"2024-08-18T23:43:24.311562+0000","last_fullsized":"2024-08-18T23:43:24.311562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:11:40.172139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311624+0000","last_change":"2024-08-18T23:41:32.133478+0000","last_active":"2024-08-18T23:43:24.311624+0000","last_peered":"2024-08-18T23:43:24.311624+0000","last_clean":"2024-08-18T23:43:24.311624+0000","last_became_active":"2024-08-18T23:41:32.133149+0000","last_became_peered":"2024-08-18T23:41:32.133149+0000","last_unstale":"2024-08-18T23:43:24.311624+0000","last_undegraded":"2024-08-18T23:43:24.311624+0000","last_fullsized":"2024-08-18T23:43:24.311624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:57:53.713309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303573+0000","last_change":"2024-08-18T23:43:24.303232+0000","last_active":"2024-08-18T23:43:24.303573+0000","last_peered":"2024-08-18T23:43:24.303573+0000","last_clean":"2024-08-18T23:43:24.303573+0000","last_became_active":"2024-08-18T23:43:24.302849+0000","last_became_peered":"2024-08-18T23:43:24.302849+0000","last_unstale":"2024-08-18T23:43:24.303573+0000","last_undegraded":"2024-08-18T23:43:24.303573+0000","last_fullsized":"2024-08-18T23:43:24.303573+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:02:02.479891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313096+0000","last_change":"2024-08-18T23:41:32.136303+0000","last_active":"2024-08-18T23:43:24.313096+0000","last_peered":"2024-08-18T23:43:24.313096+0000","last_clean":"2024-08-18T23:43:24.313096+0000","last_became_active":"2024-08-18T23:41:32.134628+0000","last_became_peered":"2024-08-18T23:41:32.134628+0000","last_unstale":"2024-08-18T23:43:24.313096+0000","last_undegraded":"2024-08-18T23:43:24.313096+0000","last_fullsized":"2024-08-18T23:43:24.313096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:28:51.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304992+0000","last_change":"2024-08-18T23:41:32.137512+0000","last_active":"2024-08-18T23:43:24.304992+0000","last_peered":"2024-08-18T23:43:24.304992+0000","last_clean":"2024-08-18T23:43:24.304992+0000","last_became_active":"2024-08-18T23:41:32.137150+0000","last_became_peered":"2024-08-18T23:41:32.137150+0000","last_unstale":"2024-08-18T23:43:24.304992+0000","last_undegraded":"2024-08-18T23:43:24.304992+0000","last_fullsized":"2024-08-18T23:43:24.304992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:44:08.020780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301258+0000","last_change":"2024-08-18T23:41:32.139229+0000","last_active":"2024-08-18T23:43:24.301258+0000","last_peered":"2024-08-18T23:43:24.301258+0000","last_clean":"2024-08-18T23:43:24.301258+0000","last_became_active":"2024-08-18T23:41:32.138864+0000","last_became_peered":"2024-08-18T23:41:32.138864+0000","last_unstale":"2024-08-18T23:43:24.301258+0000","last_undegraded":"2024-08-18T23:43:24.301258+0000","last_fullsized":"2024-08-18T23:43:24.301258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:54:21.701750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312829+0000","last_change":"2024-08-18T23:41:32.148727+0000","last_active":"2024-08-18T23:43:24.312829+0000","last_peered":"2024-08-18T23:43:24.312829+0000","last_clean":"2024-08-18T23:43:24.312829+0000","last_became_active":"2024-08-18T23:41:32.146828+0000","last_became_peered":"2024-08-18T23:41:32.146828+0000","last_unstale":"2024-08-18T23:43:24.312829+0000","last_undegraded":"2024-08-18T23:43:24.312829+0000","last_fullsized":"2024-08-18T23:43:24.312829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:51:49.175150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312029+0000","last_change":"2024-08-18T23:41:32.144796+0000","last_active":"2024-08-18T23:43:24.312029+0000","last_peered":"2024-08-18T23:43:24.312029+0000","last_clean":"2024-08-18T23:43:24.312029+0000","last_became_active":"2024-08-18T23:41:32.141931+0000","last_became_peered":"2024-08-18T23:41:32.141931+0000","last_unstale":"2024-08-18T23:43:24.312029+0000","last_undegraded":"2024-08-18T23:43:24.312029+0000","last_fullsized":"2024-08-18T23:43:24.312029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:48:08.428845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304569+0000","last_change":"2024-08-18T23:41:32.135265+0000","last_active":"2024-08-18T23:43:24.304569+0000","last_peered":"2024-08-18T23:43:24.304569+0000","last_clean":"2024-08-18T23:43:24.304569+0000","last_became_active":"2024-08-18T23:41:32.134952+0000","last_became_peered":"2024-08-18T23:41:32.134952+0000","last_unstale":"2024-08-18T23:43:24.304569+0000","last_undegraded":"2024-08-18T23:43:24.304569+0000","last_fullsized":"2024-08-18T23:43:24.304569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:35:14.099536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300727+0000","last_change":"2024-08-18T23:41:32.135941+0000","last_active":"2024-08-18T23:43:24.300727+0000","last_peered":"2024-08-18T23:43:24.300727+0000","last_clean":"2024-08-18T23:43:24.300727+0000","last_became_active":"2024-08-18T23:41:32.135623+0000","last_became_peered":"2024-08-18T23:41:32.135623+0000","last_unstale":"2024-08-18T23:43:24.300727+0000","last_undegraded":"2024-08-18T23:43:24.300727+0000","last_fullsized":"2024-08-18T23:43:24.300727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:43:36.504738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303544+0000","last_change":"2024-08-18T23:41:32.141643+0000","last_active":"2024-08-18T23:43:24.303544+0000","last_peered":"2024-08-18T23:43:24.303544+0000","last_clean":"2024-08-18T23:43:24.303544+0000","last_became_active":"2024-08-18T23:41:32.141301+0000","last_became_peered":"2024-08-18T23:41:32.141301+0000","last_unstale":"2024-08-18T23:43:24.303544+0000","last_undegraded":"2024-08-18T23:43:24.303544+0000","last_fullsized":"2024-08-18T23:43:24.303544+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:23:53.100740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312988+0000","last_change":"2024-08-18T23:41:32.146730+0000","last_active":"2024-08-18T23:43:24.312988+0000","last_peered":"2024-08-18T23:43:24.312988+0000","last_clean":"2024-08-18T23:43:24.312988+0000","last_became_active":"2024-08-18T23:41:32.146252+0000","last_became_peered":"2024-08-18T23:41:32.146252+0000","last_unstale":"2024-08-18T23:43:24.312988+0000","last_undegraded":"2024-08-18T23:43:24.312988+0000","last_fullsized":"2024-08-18T23:43:24.312988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:41.481761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311744+0000","last_change":"2024-08-18T23:41:32.149569+0000","last_active":"2024-08-18T23:43:24.311744+0000","last_peered":"2024-08-18T23:43:24.311744+0000","last_clean":"2024-08-18T23:43:24.311744+0000","last_became_active":"2024-08-18T23:41:32.149277+0000","last_became_peered":"2024-08-18T23:41:32.149277+0000","last_unstale":"2024-08-18T23:43:24.311744+0000","last_undegraded":"2024-08-18T23:43:24.311744+0000","last_fullsized":"2024-08-18T23:43:24.311744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:34:00.547909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311950+0000","last_change":"2024-08-18T23:41:32.137889+0000","last_active":"2024-08-18T23:43:24.311950+0000","last_peered":"2024-08-18T23:43:24.311950+0000","last_clean":"2024-08-18T23:43:24.311950+0000","last_became_active":"2024-08-18T23:41:32.137578+0000","last_became_peered":"2024-08-18T23:41:32.137578+0000","last_unstale":"2024-08-18T23:43:24.311950+0000","last_undegraded":"2024-08-18T23:43:24.311950+0000","last_fullsized":"2024-08-18T23:43:24.311950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:18:09.987838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312999+0000","last_change":"2024-08-18T23:41:32.142084+0000","last_active":"2024-08-18T23:43:24.312999+0000","last_peered":"2024-08-18T23:43:24.312999+0000","last_clean":"2024-08-18T23:43:24.312999+0000","last_became_active":"2024-08-18T23:41:32.141670+0000","last_became_peered":"2024-08-18T23:41:32.141670+0000","last_unstale":"2024-08-18T23:43:24.312999+0000","last_undegraded":"2024-08-18T23:43:24.312999+0000","last_fullsized":"2024-08-18T23:43:24.312999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:03.990351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73627,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":708,"num_read_kb":431,"num_write":188,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":220881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514539,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":523788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776176,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1180152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939700,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":528120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168633,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":528040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397565,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1662976,"data_stored":1121222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561090,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1048576,"data_stored":519200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790023,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1111875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953547,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1064960,"data_stored":525271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":519394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":589056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":587367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":520588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T23:44:02.242 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-08-18T23:44:02.242 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-18T23:44:02.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.373+0000 7f0001c79640 1 -- 172.21.15.78:0/1602306357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efffc1013c0 msgr2=0x7efffc108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:02.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.373+0000 7f0001c79640 1 --2- 172.21.15.78:0/1602306357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efffc1013c0 0x7efffc108c60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7efff4009ca0 tx=0x7efff402a8c0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.373+0000 7f0001c79640 1 -- 172.21.15.78:0/1602306357 shutdown_connections 2024-08-18T23:44:02.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.373+0000 7f0001c79640 1 --2- 172.21.15.78:0/1602306357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efffc1013c0 0x7efffc108c60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.373+0000 7f0001c79640 1 --2- 172.21.15.78:0/1602306357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efffc100a00 0x7efffc100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.373+0000 7f0001c79640 1 --2- 172.21.15.78:0/1602306357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efffc101d60 0x7efffc102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.373+0000 7f0001c79640 1 -- 172.21.15.78:0/1602306357 >> 172.21.15.78:0/1602306357 conn(0x7efffc0fb640 msgr2=0x7efffc0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:02.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.373+0000 7f0001c79640 1 -- 172.21.15.78:0/1602306357 shutdown_connections 2024-08-18T23:44:02.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.373+0000 7f0001c79640 1 -- 172.21.15.78:0/1602306357 wait complete. 2024-08-18T23:44:02.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7f0001c79640 1 Processor -- start 2024-08-18T23:44:02.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7f0001c79640 1 -- start start 2024-08-18T23:44:02.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7f0001c79640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efffc100a00 0x7efffc1047b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:02.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7f0001c79640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efffc1013c0 0x7efffc102df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:02.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7f0001c79640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efffc101d60 0x7efffc103330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:02.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7f0001c79640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efffc10c3d0 con 0x7efffc100a00 2024-08-18T23:44:02.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7f0001c79640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efffc10c250 con 0x7efffc1013c0 2024-08-18T23:44:02.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7f0001c79640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efffc10c550 con 0x7efffc101d60 2024-08-18T23:44:02.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7efffb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efffc100a00 0x7efffc1047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:02.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7efffb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efffc100a00 0x7efffc1047b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51288/0 (socket says 172.21.15.78:51288) 2024-08-18T23:44:02.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7efffb7fe640 1 -- 172.21.15.78:0/1866848272 learned_addr learned my addr 172.21.15.78:0/1866848272 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:02.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7efffaffd640 1 --2- 172.21.15.78:0/1866848272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efffc1013c0 0x7efffc102df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:02.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7efffbfff640 1 --2- 172.21.15.78:0/1866848272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efffc101d60 0x7efffc103330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:02.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7efffb7fe640 1 -- 172.21.15.78:0/1866848272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efffc101d60 msgr2=0x7efffc103330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:02.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7efffb7fe640 1 --2- 172.21.15.78:0/1866848272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efffc101d60 0x7efffc103330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7efffb7fe640 1 -- 172.21.15.78:0/1866848272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efffc1013c0 msgr2=0x7efffc102df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:02.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7efffb7fe640 1 --2- 172.21.15.78:0/1866848272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efffc1013c0 0x7efffc102df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7efffb7fe640 1 -- 172.21.15.78:0/1866848272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efffc103bc0 con 0x7efffc100a00 2024-08-18T23:44:02.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7efffbfff640 1 --2- 172.21.15.78:0/1866848272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efffc101d60 0x7efffc103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:44:02.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.374+0000 7efffb7fe640 1 --2- 172.21.15.78:0/1866848272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efffc100a00 0x7efffc1047b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7effec00e9c0 tx=0x7effec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:02.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.375+0000 7efff8ff9640 1 -- 172.21.15.78:0/1866848272 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec00cd80 con 0x7efffc100a00 2024-08-18T23:44:02.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.375+0000 7efff8ff9640 1 -- 172.21.15.78:0/1866848272 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effec0035f0 con 0x7efffc100a00 2024-08-18T23:44:02.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.375+0000 7f0001c79640 1 -- 172.21.15.78:0/1866848272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efffc1a1c70 con 0x7efffc100a00 2024-08-18T23:44:02.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.375+0000 7efff8ff9640 1 -- 172.21.15.78:0/1866848272 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec010690 con 0x7efffc100a00 2024-08-18T23:44:02.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.375+0000 7f0001c79640 1 -- 172.21.15.78:0/1866848272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efffc1a21c0 con 0x7efffc100a00 2024-08-18T23:44:02.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.376+0000 7f0001c79640 1 -- 172.21.15.78:0/1866848272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efffc1a2390 con 0x7efffc100a00 2024-08-18T23:44:02.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.377+0000 7efff8ff9640 1 -- 172.21.15.78:0/1866848272 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7effec0037a0 con 0x7efffc100a00 2024-08-18T23:44:02.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.378+0000 7efff8ff9640 1 --2- 172.21.15.78:0/1866848272 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7effe40a2c20 0x7effe40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:02.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.378+0000 7efffaffd640 1 --2- 172.21.15.78:0/1866848272 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7effe40a2c20 0x7effe40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:02.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.378+0000 7efff8ff9640 1 -- 172.21.15.78:0/1866848272 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7effec0cfb30 con 0x7efffc100a00 2024-08-18T23:44:02.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.378+0000 7efffaffd640 1 --2- 172.21.15.78:0/1866848272 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7effe40a2c20 0x7effe40a50e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efff0005fd0 tx=0x7efff0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:02.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.381+0000 7efff8ff9640 1 -- 172.21.15.78:0/1866848272 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7effec09c5a0 con 0x7efffc100a00 2024-08-18T23:44:02.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.547+0000 7f0001c79640 1 -- 172.21.15.78:0/1866848272 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efffc0fd860 con 0x7effe40a2c20 2024-08-18T23:44:02.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.555+0000 7efff8ff9640 1 -- 172.21.15.78:0/1866848272 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340487 (secure 0 0 0) 0x7efffc0fd860 con 0x7effe40a2c20 2024-08-18T23:44:02.560 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:44:02.562 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-08-18T23:44:02.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.558+0000 7f0001c79640 1 -- 172.21.15.78:0/1866848272 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7effe40a2c20 msgr2=0x7effe40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:02.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.558+0000 7f0001c79640 1 --2- 172.21.15.78:0/1866848272 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7effe40a2c20 0x7effe40a50e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efff0005fd0 tx=0x7efff0005850 comp rx=0 tx=0).stop 2024-08-18T23:44:02.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.559+0000 7f0001c79640 1 -- 172.21.15.78:0/1866848272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efffc100a00 msgr2=0x7efffc1047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:02.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.559+0000 7f0001c79640 1 --2- 172.21.15.78:0/1866848272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efffc100a00 0x7efffc1047b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7effec00e9c0 tx=0x7effec00ee80 comp rx=0 tx=0).stop 2024-08-18T23:44:02.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.559+0000 7f0001c79640 1 -- 172.21.15.78:0/1866848272 shutdown_connections 2024-08-18T23:44:02.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.559+0000 7f0001c79640 1 --2- 172.21.15.78:0/1866848272 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7effe40a2c20 0x7effe40a50e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.559+0000 7f0001c79640 1 --2- 172.21.15.78:0/1866848272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efffc101d60 0x7efffc103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.559+0000 7f0001c79640 1 --2- 172.21.15.78:0/1866848272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efffc1013c0 0x7efffc102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.559+0000 7f0001c79640 1 --2- 172.21.15.78:0/1866848272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efffc100a00 0x7efffc1047b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:02.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.559+0000 7f0001c79640 1 -- 172.21.15.78:0/1866848272 >> 172.21.15.78:0/1866848272 conn(0x7efffc0fb640 msgr2=0x7efffc0fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:02.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.559+0000 7f0001c79640 1 -- 172.21.15.78:0/1866848272 shutdown_connections 2024-08-18T23:44:02.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:02.559+0000 7f0001c79640 1 -- 172.21.15.78:0/1866848272 wait complete. 2024-08-18T23:44:02.572 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-08-18T23:44:02.147383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664229,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":834,"num_read_kb":540,"num_write":371,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":800,"ondisk_log_size":800,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392748,"kb_used_data":14444,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549076,"statfs":{"total":1151940427776,"available":1151538253824,"internally_reserved":0,"allocated":14790656,"data_stored":8254073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5710,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":38,"num_write":12,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003304"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298623+0000","last_change":"2024-08-18T23:41:30.120580+0000","last_active":"2024-08-18T23:43:24.298623+0000","last_peered":"2024-08-18T23:43:24.298623+0000","last_clean":"2024-08-18T23:43:24.298623+0000","last_became_active":"2024-08-18T23:41:30.120266+0000","last_became_peered":"2024-08-18T23:41:30.120266+0000","last_unstale":"2024-08-18T23:43:24.298623+0000","last_undegraded":"2024-08-18T23:43:24.298623+0000","last_fullsized":"2024-08-18T23:43:24.298623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:06:40.997070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312511+0000","last_change":"2024-08-18T23:41:32.143232+0000","last_active":"2024-08-18T23:43:24.312511+0000","last_peered":"2024-08-18T23:43:24.312511+0000","last_clean":"2024-08-18T23:43:24.312511+0000","last_became_active":"2024-08-18T23:41:32.142916+0000","last_became_peered":"2024-08-18T23:41:32.142916+0000","last_unstale":"2024-08-18T23:43:24.312511+0000","last_undegraded":"2024-08-18T23:43:24.312511+0000","last_fullsized":"2024-08-18T23:43:24.312511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:50:54.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312049+0000","last_change":"2024-08-18T23:41:39.188772+0000","last_active":"2024-08-18T23:43:24.312049+0000","last_peered":"2024-08-18T23:43:24.312049+0000","last_clean":"2024-08-18T23:43:24.312049+0000","last_became_active":"2024-08-18T23:41:39.188362+0000","last_became_peered":"2024-08-18T23:41:39.188362+0000","last_unstale":"2024-08-18T23:43:24.312049+0000","last_undegraded":"2024-08-18T23:43:24.312049+0000","last_fullsized":"2024-08-18T23:43:24.312049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:47:50.255543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312628+0000","last_change":"2024-08-18T23:41:30.137179+0000","last_active":"2024-08-18T23:43:24.312628+0000","last_peered":"2024-08-18T23:43:24.312628+0000","last_clean":"2024-08-18T23:43:24.312628+0000","last_became_active":"2024-08-18T23:41:30.136618+0000","last_became_peered":"2024-08-18T23:41:30.136618+0000","last_unstale":"2024-08-18T23:43:24.312628+0000","last_undegraded":"2024-08-18T23:43:24.312628+0000","last_fullsized":"2024-08-18T23:43:24.312628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:59.826364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312531+0000","last_change":"2024-08-18T23:41:32.142075+0000","last_active":"2024-08-18T23:43:24.312531+0000","last_peered":"2024-08-18T23:43:24.312531+0000","last_clean":"2024-08-18T23:43:24.312531+0000","last_became_active":"2024-08-18T23:41:32.141709+0000","last_became_peered":"2024-08-18T23:41:32.141709+0000","last_unstale":"2024-08-18T23:43:24.312531+0000","last_undegraded":"2024-08-18T23:43:24.312531+0000","last_fullsized":"2024-08-18T23:43:24.312531+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:08.324627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312147+0000","last_change":"2024-08-18T23:41:39.184575+0000","last_active":"2024-08-18T23:43:24.312147+0000","last_peered":"2024-08-18T23:43:24.312147+0000","last_clean":"2024-08-18T23:43:24.312147+0000","last_became_active":"2024-08-18T23:41:39.184169+0000","last_became_peered":"2024-08-18T23:41:39.184169+0000","last_unstale":"2024-08-18T23:43:24.312147+0000","last_undegraded":"2024-08-18T23:43:24.312147+0000","last_fullsized":"2024-08-18T23:43:24.312147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:32.218172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304076+0000","last_change":"2024-08-18T23:41:30.121638+0000","last_active":"2024-08-18T23:43:24.304076+0000","last_peered":"2024-08-18T23:43:24.304076+0000","last_clean":"2024-08-18T23:43:24.304076+0000","last_became_active":"2024-08-18T23:41:30.121290+0000","last_became_peered":"2024-08-18T23:41:30.121290+0000","last_unstale":"2024-08-18T23:43:24.304076+0000","last_undegraded":"2024-08-18T23:43:24.304076+0000","last_fullsized":"2024-08-18T23:43:24.304076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:43:55.638780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312374+0000","last_change":"2024-08-18T23:41:32.142177+0000","last_active":"2024-08-18T23:43:24.312374+0000","last_peered":"2024-08-18T23:43:24.312374+0000","last_clean":"2024-08-18T23:43:24.312374+0000","last_became_active":"2024-08-18T23:41:32.141731+0000","last_became_peered":"2024-08-18T23:41:32.141731+0000","last_unstale":"2024-08-18T23:43:24.312374+0000","last_undegraded":"2024-08-18T23:43:24.312374+0000","last_fullsized":"2024-08-18T23:43:24.312374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:36.283299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312604+0000","last_change":"2024-08-18T23:41:39.182931+0000","last_active":"2024-08-18T23:43:24.312604+0000","last_peered":"2024-08-18T23:43:24.312604+0000","last_clean":"2024-08-18T23:43:24.312604+0000","last_became_active":"2024-08-18T23:41:39.182008+0000","last_became_peered":"2024-08-18T23:41:39.182008+0000","last_unstale":"2024-08-18T23:43:24.312604+0000","last_undegraded":"2024-08-18T23:43:24.312604+0000","last_fullsized":"2024-08-18T23:43:24.312604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:58:38.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312863+0000","last_change":"2024-08-18T23:42:23.329762+0000","last_active":"2024-08-18T23:43:24.312863+0000","last_peered":"2024-08-18T23:43:24.312863+0000","last_clean":"2024-08-18T23:43:24.312863+0000","last_became_active":"2024-08-18T23:42:23.325753+0000","last_became_peered":"2024-08-18T23:42:23.325753+0000","last_unstale":"2024-08-18T23:43:24.312863+0000","last_undegraded":"2024-08-18T23:43:24.312863+0000","last_fullsized":"2024-08-18T23:43:24.312863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:19:43.062492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307318+0000","last_change":"2024-08-18T23:41:32.148662+0000","last_active":"2024-08-18T23:43:24.307318+0000","last_peered":"2024-08-18T23:43:24.307318+0000","last_clean":"2024-08-18T23:43:24.307318+0000","last_became_active":"2024-08-18T23:41:32.148318+0000","last_became_peered":"2024-08-18T23:41:32.148318+0000","last_unstale":"2024-08-18T23:43:24.307318+0000","last_undegraded":"2024-08-18T23:43:24.307318+0000","last_fullsized":"2024-08-18T23:43:24.307318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:58.668669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312238+0000","last_change":"2024-08-18T23:41:39.181114+0000","last_active":"2024-08-18T23:43:24.312238+0000","last_peered":"2024-08-18T23:43:24.312238+0000","last_clean":"2024-08-18T23:43:24.312238+0000","last_became_active":"2024-08-18T23:41:39.180599+0000","last_became_peered":"2024-08-18T23:41:39.180599+0000","last_unstale":"2024-08-18T23:43:24.312238+0000","last_undegraded":"2024-08-18T23:43:24.312238+0000","last_fullsized":"2024-08-18T23:43:24.312238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:36:08.749143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304715+0000","last_change":"2024-08-18T23:41:32.144326+0000","last_active":"2024-08-18T23:43:24.304715+0000","last_peered":"2024-08-18T23:43:24.304715+0000","last_clean":"2024-08-18T23:43:24.304715+0000","last_became_active":"2024-08-18T23:41:32.144004+0000","last_became_peered":"2024-08-18T23:41:32.144004+0000","last_unstale":"2024-08-18T23:43:24.304715+0000","last_undegraded":"2024-08-18T23:43:24.304715+0000","last_fullsized":"2024-08-18T23:43:24.304715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:10.918624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303665+0000","last_change":"2024-08-18T23:41:30.125160+0000","last_active":"2024-08-18T23:43:24.303665+0000","last_peered":"2024-08-18T23:43:24.303665+0000","last_clean":"2024-08-18T23:43:24.303665+0000","last_became_active":"2024-08-18T23:41:30.124685+0000","last_became_peered":"2024-08-18T23:41:30.124685+0000","last_unstale":"2024-08-18T23:43:24.303665+0000","last_undegraded":"2024-08-18T23:43:24.303665+0000","last_fullsized":"2024-08-18T23:43:24.303665+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:52:48.997902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313216+0000","last_change":"2024-08-18T23:41:39.681481+0000","last_active":"2024-08-18T23:43:24.313216+0000","last_peered":"2024-08-18T23:43:24.313216+0000","last_clean":"2024-08-18T23:43:24.313216+0000","last_became_active":"2024-08-18T23:41:39.680204+0000","last_became_peered":"2024-08-18T23:41:39.680204+0000","last_unstale":"2024-08-18T23:43:24.313216+0000","last_undegraded":"2024-08-18T23:43:24.313216+0000","last_fullsized":"2024-08-18T23:43:24.313216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:00:05.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312231+0000","last_change":"2024-08-18T23:41:32.151335+0000","last_active":"2024-08-18T23:43:24.312231+0000","last_peered":"2024-08-18T23:43:24.312231+0000","last_clean":"2024-08-18T23:43:24.312231+0000","last_became_active":"2024-08-18T23:41:32.151040+0000","last_became_peered":"2024-08-18T23:41:32.151040+0000","last_unstale":"2024-08-18T23:43:24.312231+0000","last_undegraded":"2024-08-18T23:43:24.312231+0000","last_fullsized":"2024-08-18T23:43:24.312231+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:16.108555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"92'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:41.164918+0000","last_change":"2024-08-18T23:41:30.122301+0000","last_active":"2024-08-18T23:43:41.164918+0000","last_peered":"2024-08-18T23:43:41.164918+0000","last_clean":"2024-08-18T23:43:41.164918+0000","last_became_active":"2024-08-18T23:41:30.121117+0000","last_became_peered":"2024-08-18T23:41:30.121117+0000","last_unstale":"2024-08-18T23:43:41.164918+0000","last_undegraded":"2024-08-18T23:43:41.164918+0000","last_fullsized":"2024-08-18T23:43:41.164918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:40:12.678803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312721+0000","last_change":"2024-08-18T23:41:39.182395+0000","last_active":"2024-08-18T23:43:24.312721+0000","last_peered":"2024-08-18T23:43:24.312721+0000","last_clean":"2024-08-18T23:43:24.312721+0000","last_became_active":"2024-08-18T23:41:39.181979+0000","last_became_peered":"2024-08-18T23:41:39.181979+0000","last_unstale":"2024-08-18T23:43:24.312721+0000","last_undegraded":"2024-08-18T23:43:24.312721+0000","last_fullsized":"2024-08-18T23:43:24.312721+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:04.674503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312810+0000","last_change":"2024-08-18T23:41:32.140464+0000","last_active":"2024-08-18T23:43:24.312810+0000","last_peered":"2024-08-18T23:43:24.312810+0000","last_clean":"2024-08-18T23:43:24.312810+0000","last_became_active":"2024-08-18T23:41:32.139153+0000","last_became_peered":"2024-08-18T23:41:32.139153+0000","last_unstale":"2024-08-18T23:43:24.312810+0000","last_undegraded":"2024-08-18T23:43:24.312810+0000","last_fullsized":"2024-08-18T23:43:24.312810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:27.387680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312414+0000","last_change":"2024-08-18T23:41:30.122787+0000","last_active":"2024-08-18T23:43:24.312414+0000","last_peered":"2024-08-18T23:43:24.312414+0000","last_clean":"2024-08-18T23:43:24.312414+0000","last_became_active":"2024-08-18T23:41:30.122462+0000","last_became_peered":"2024-08-18T23:41:30.122462+0000","last_unstale":"2024-08-18T23:43:24.312414+0000","last_undegraded":"2024-08-18T23:43:24.312414+0000","last_fullsized":"2024-08-18T23:43:24.312414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:55.568831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298325+0000","last_change":"2024-08-18T23:41:39.190511+0000","last_active":"2024-08-18T23:43:24.298325+0000","last_peered":"2024-08-18T23:43:24.298325+0000","last_clean":"2024-08-18T23:43:24.298325+0000","last_became_active":"2024-08-18T23:41:39.190154+0000","last_became_peered":"2024-08-18T23:41:39.190154+0000","last_unstale":"2024-08-18T23:43:24.298325+0000","last_undegraded":"2024-08-18T23:43:24.298325+0000","last_fullsized":"2024-08-18T23:43:24.298325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:33:34.514827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303456+0000","last_change":"2024-08-18T23:41:32.143867+0000","last_active":"2024-08-18T23:43:24.303456+0000","last_peered":"2024-08-18T23:43:24.303456+0000","last_clean":"2024-08-18T23:43:24.303456+0000","last_became_active":"2024-08-18T23:41:32.143527+0000","last_became_peered":"2024-08-18T23:41:32.143527+0000","last_unstale":"2024-08-18T23:43:24.303456+0000","last_undegraded":"2024-08-18T23:43:24.303456+0000","last_fullsized":"2024-08-18T23:43:24.303456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:40:25.902133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312100+0000","last_change":"2024-08-18T23:42:23.325102+0000","last_active":"2024-08-18T23:43:24.312100+0000","last_peered":"2024-08-18T23:43:24.312100+0000","last_clean":"2024-08-18T23:43:24.312100+0000","last_became_active":"2024-08-18T23:42:23.321583+0000","last_became_peered":"2024-08-18T23:42:23.321583+0000","last_unstale":"2024-08-18T23:43:24.312100+0000","last_undegraded":"2024-08-18T23:43:24.312100+0000","last_fullsized":"2024-08-18T23:43:24.312100+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:57:42.546686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312767+0000","last_change":"2024-08-18T23:42:23.559756+0000","last_active":"2024-08-18T23:43:24.312767+0000","last_peered":"2024-08-18T23:43:24.312767+0000","last_clean":"2024-08-18T23:43:24.312767+0000","last_became_active":"2024-08-18T23:42:23.559242+0000","last_became_peered":"2024-08-18T23:42:23.559242+0000","last_unstale":"2024-08-18T23:43:24.312767+0000","last_undegraded":"2024-08-18T23:43:24.312767+0000","last_fullsized":"2024-08-18T23:43:24.312767+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:25:40.790313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312043+0000","last_change":"2024-08-18T23:41:32.139025+0000","last_active":"2024-08-18T23:43:24.312043+0000","last_peered":"2024-08-18T23:43:24.312043+0000","last_clean":"2024-08-18T23:43:24.312043+0000","last_became_active":"2024-08-18T23:41:32.138733+0000","last_became_peered":"2024-08-18T23:41:32.138733+0000","last_unstale":"2024-08-18T23:43:24.312043+0000","last_undegraded":"2024-08-18T23:43:24.312043+0000","last_fullsized":"2024-08-18T23:43:24.312043+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:01:39.826391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313061+0000","last_change":"2024-08-18T23:41:30.477359+0000","last_active":"2024-08-18T23:43:24.313061+0000","last_peered":"2024-08-18T23:43:24.313061+0000","last_clean":"2024-08-18T23:43:24.313061+0000","last_became_active":"2024-08-18T23:41:30.477011+0000","last_became_peered":"2024-08-18T23:41:30.477011+0000","last_unstale":"2024-08-18T23:43:24.313061+0000","last_undegraded":"2024-08-18T23:43:24.313061+0000","last_fullsized":"2024-08-18T23:43:24.313061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:53:10.052859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312145+0000","last_change":"2024-08-18T23:41:39.182140+0000","last_active":"2024-08-18T23:43:24.312145+0000","last_peered":"2024-08-18T23:43:24.312145+0000","last_clean":"2024-08-18T23:43:24.312145+0000","last_became_active":"2024-08-18T23:41:39.181667+0000","last_became_peered":"2024-08-18T23:41:39.181667+0000","last_unstale":"2024-08-18T23:43:24.312145+0000","last_undegraded":"2024-08-18T23:43:24.312145+0000","last_fullsized":"2024-08-18T23:43:24.312145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:02:18.128028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312594+0000","last_change":"2024-08-18T23:41:32.137064+0000","last_active":"2024-08-18T23:43:24.312594+0000","last_peered":"2024-08-18T23:43:24.312594+0000","last_clean":"2024-08-18T23:43:24.312594+0000","last_became_active":"2024-08-18T23:41:32.136738+0000","last_became_peered":"2024-08-18T23:41:32.136738+0000","last_unstale":"2024-08-18T23:43:24.312594+0000","last_undegraded":"2024-08-18T23:43:24.312594+0000","last_fullsized":"2024-08-18T23:43:24.312594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:15:11.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'12","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298408+0000","last_change":"2024-08-18T23:41:30.120042+0000","last_active":"2024-08-18T23:43:24.298408+0000","last_peered":"2024-08-18T23:43:24.298408+0000","last_clean":"2024-08-18T23:43:24.298408+0000","last_became_active":"2024-08-18T23:41:30.119719+0000","last_became_peered":"2024-08-18T23:41:30.119719+0000","last_unstale":"2024-08-18T23:43:24.298408+0000","last_undegraded":"2024-08-18T23:43:24.298408+0000","last_fullsized":"2024-08-18T23:43:24.298408+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:35:44.466330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311900+0000","last_change":"2024-08-18T23:41:39.680405+0000","last_active":"2024-08-18T23:43:24.311900+0000","last_peered":"2024-08-18T23:43:24.311900+0000","last_clean":"2024-08-18T23:43:24.311900+0000","last_became_active":"2024-08-18T23:41:39.679940+0000","last_became_peered":"2024-08-18T23:41:39.679940+0000","last_unstale":"2024-08-18T23:43:24.311900+0000","last_undegraded":"2024-08-18T23:43:24.311900+0000","last_fullsized":"2024-08-18T23:43:24.311900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:15:03.942116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301271+0000","last_change":"2024-08-18T23:41:32.139688+0000","last_active":"2024-08-18T23:43:24.301271+0000","last_peered":"2024-08-18T23:43:24.301271+0000","last_clean":"2024-08-18T23:43:24.301271+0000","last_became_active":"2024-08-18T23:41:32.139397+0000","last_became_peered":"2024-08-18T23:41:32.139397+0000","last_unstale":"2024-08-18T23:43:24.301271+0000","last_undegraded":"2024-08-18T23:43:24.301271+0000","last_fullsized":"2024-08-18T23:43:24.301271+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:03:57.017488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304150+0000","last_change":"2024-08-18T23:41:30.137578+0000","last_active":"2024-08-18T23:43:24.304150+0000","last_peered":"2024-08-18T23:43:24.304150+0000","last_clean":"2024-08-18T23:43:24.304150+0000","last_became_active":"2024-08-18T23:41:30.137246+0000","last_became_peered":"2024-08-18T23:41:30.137246+0000","last_unstale":"2024-08-18T23:43:24.304150+0000","last_undegraded":"2024-08-18T23:43:24.304150+0000","last_fullsized":"2024-08-18T23:43:24.304150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:48:54.839432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312576+0000","last_change":"2024-08-18T23:41:39.186360+0000","last_active":"2024-08-18T23:43:24.312576+0000","last_peered":"2024-08-18T23:43:24.312576+0000","last_clean":"2024-08-18T23:43:24.312576+0000","last_became_active":"2024-08-18T23:41:39.185808+0000","last_became_peered":"2024-08-18T23:41:39.185808+0000","last_unstale":"2024-08-18T23:43:24.312576+0000","last_undegraded":"2024-08-18T23:43:24.312576+0000","last_fullsized":"2024-08-18T23:43:24.312576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:53.891061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300916+0000","last_change":"2024-08-18T23:41:32.146070+0000","last_active":"2024-08-18T23:43:24.300916+0000","last_peered":"2024-08-18T23:43:24.300916+0000","last_clean":"2024-08-18T23:43:24.300916+0000","last_became_active":"2024-08-18T23:41:32.145640+0000","last_became_peered":"2024-08-18T23:41:32.145640+0000","last_unstale":"2024-08-18T23:43:24.300916+0000","last_undegraded":"2024-08-18T23:43:24.300916+0000","last_fullsized":"2024-08-18T23:43:24.300916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:56:21.368762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298278+0000","last_change":"2024-08-18T23:41:30.138565+0000","last_active":"2024-08-18T23:43:24.298278+0000","last_peered":"2024-08-18T23:43:24.298278+0000","last_clean":"2024-08-18T23:43:24.298278+0000","last_became_active":"2024-08-18T23:41:30.138210+0000","last_became_peered":"2024-08-18T23:41:30.138210+0000","last_unstale":"2024-08-18T23:43:24.298278+0000","last_undegraded":"2024-08-18T23:43:24.298278+0000","last_fullsized":"2024-08-18T23:43:24.298278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:42:14.928884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303425+0000","last_change":"2024-08-18T23:41:39.680656+0000","last_active":"2024-08-18T23:43:24.303425+0000","last_peered":"2024-08-18T23:43:24.303425+0000","last_clean":"2024-08-18T23:43:24.303425+0000","last_became_active":"2024-08-18T23:41:39.680203+0000","last_became_peered":"2024-08-18T23:41:39.680203+0000","last_unstale":"2024-08-18T23:43:24.303425+0000","last_undegraded":"2024-08-18T23:43:24.303425+0000","last_fullsized":"2024-08-18T23:43:24.303425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:55:43.138869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300932+0000","last_change":"2024-08-18T23:41:32.130222+0000","last_active":"2024-08-18T23:43:24.300932+0000","last_peered":"2024-08-18T23:43:24.300932+0000","last_clean":"2024-08-18T23:43:24.300932+0000","last_became_active":"2024-08-18T23:41:32.129844+0000","last_became_peered":"2024-08-18T23:41:32.129844+0000","last_unstale":"2024-08-18T23:43:24.300932+0000","last_undegraded":"2024-08-18T23:43:24.300932+0000","last_fullsized":"2024-08-18T23:43:24.300932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:39:21.106343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304850+0000","last_change":"2024-08-18T23:41:30.123142+0000","last_active":"2024-08-18T23:43:24.304850+0000","last_peered":"2024-08-18T23:43:24.304850+0000","last_clean":"2024-08-18T23:43:24.304850+0000","last_became_active":"2024-08-18T23:41:30.121645+0000","last_became_peered":"2024-08-18T23:41:30.121645+0000","last_unstale":"2024-08-18T23:43:24.304850+0000","last_undegraded":"2024-08-18T23:43:24.304850+0000","last_fullsized":"2024-08-18T23:43:24.304850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:07:24.912259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301163+0000","last_change":"2024-08-18T23:41:39.186033+0000","last_active":"2024-08-18T23:43:24.301163+0000","last_peered":"2024-08-18T23:43:24.301163+0000","last_clean":"2024-08-18T23:43:24.301163+0000","last_became_active":"2024-08-18T23:41:39.185451+0000","last_became_peered":"2024-08-18T23:41:39.185451+0000","last_unstale":"2024-08-18T23:43:24.301163+0000","last_undegraded":"2024-08-18T23:43:24.301163+0000","last_fullsized":"2024-08-18T23:43:24.301163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:52:13.556760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.306993+0000","last_change":"2024-08-18T23:41:32.141065+0000","last_active":"2024-08-18T23:43:24.306993+0000","last_peered":"2024-08-18T23:43:24.306993+0000","last_clean":"2024-08-18T23:43:24.306993+0000","last_became_active":"2024-08-18T23:41:32.140732+0000","last_became_peered":"2024-08-18T23:41:32.140732+0000","last_unstale":"2024-08-18T23:43:24.306993+0000","last_undegraded":"2024-08-18T23:43:24.306993+0000","last_fullsized":"2024-08-18T23:43:24.306993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:08:32.287835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":156,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:57.429047+0000","last_change":"2024-08-18T23:41:30.136838+0000","last_active":"2024-08-18T23:43:57.429047+0000","last_peered":"2024-08-18T23:43:57.429047+0000","last_clean":"2024-08-18T23:43:57.429047+0000","last_became_active":"2024-08-18T23:41:30.136523+0000","last_became_peered":"2024-08-18T23:41:30.136523+0000","last_unstale":"2024-08-18T23:43:57.429047+0000","last_undegraded":"2024-08-18T23:43:57.429047+0000","last_fullsized":"2024-08-18T23:43:57.429047+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:56:17.750535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301550+0000","last_change":"2024-08-18T23:41:39.680314+0000","last_active":"2024-08-18T23:43:24.301550+0000","last_peered":"2024-08-18T23:43:24.301550+0000","last_clean":"2024-08-18T23:43:24.301550+0000","last_became_active":"2024-08-18T23:41:39.679920+0000","last_became_peered":"2024-08-18T23:41:39.679920+0000","last_unstale":"2024-08-18T23:43:24.301550+0000","last_undegraded":"2024-08-18T23:43:24.301550+0000","last_fullsized":"2024-08-18T23:43:24.301550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:26:29.866081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300888+0000","last_change":"2024-08-18T23:41:32.139752+0000","last_active":"2024-08-18T23:43:24.300888+0000","last_peered":"2024-08-18T23:43:24.300888+0000","last_clean":"2024-08-18T23:43:24.300888+0000","last_became_active":"2024-08-18T23:41:32.139451+0000","last_became_peered":"2024-08-18T23:41:32.139451+0000","last_unstale":"2024-08-18T23:43:24.300888+0000","last_undegraded":"2024-08-18T23:43:24.300888+0000","last_fullsized":"2024-08-18T23:43:24.300888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:15:13.390529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304894+0000","last_change":"2024-08-18T23:41:30.126513+0000","last_active":"2024-08-18T23:43:24.304894+0000","last_peered":"2024-08-18T23:43:24.304894+0000","last_clean":"2024-08-18T23:43:24.304894+0000","last_became_active":"2024-08-18T23:41:30.126046+0000","last_became_peered":"2024-08-18T23:41:30.126046+0000","last_unstale":"2024-08-18T23:43:24.304894+0000","last_undegraded":"2024-08-18T23:43:24.304894+0000","last_fullsized":"2024-08-18T23:43:24.304894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:49:16.565565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307280+0000","last_change":"2024-08-18T23:41:39.681006+0000","last_active":"2024-08-18T23:43:24.307280+0000","last_peered":"2024-08-18T23:43:24.307280+0000","last_clean":"2024-08-18T23:43:24.307280+0000","last_became_active":"2024-08-18T23:41:39.680409+0000","last_became_peered":"2024-08-18T23:41:39.680409+0000","last_unstale":"2024-08-18T23:43:24.307280+0000","last_undegraded":"2024-08-18T23:43:24.307280+0000","last_fullsized":"2024-08-18T23:43:24.307280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:47.778866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312261+0000","last_change":"2024-08-18T23:41:32.147545+0000","last_active":"2024-08-18T23:43:24.312261+0000","last_peered":"2024-08-18T23:43:24.312261+0000","last_clean":"2024-08-18T23:43:24.312261+0000","last_became_active":"2024-08-18T23:41:32.147238+0000","last_became_peered":"2024-08-18T23:41:32.147238+0000","last_unstale":"2024-08-18T23:43:24.312261+0000","last_undegraded":"2024-08-18T23:43:24.312261+0000","last_fullsized":"2024-08-18T23:43:24.312261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:12:13.589535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313054+0000","last_change":"2024-08-18T23:41:30.120650+0000","last_active":"2024-08-18T23:43:24.313054+0000","last_peered":"2024-08-18T23:43:24.313054+0000","last_clean":"2024-08-18T23:43:24.313054+0000","last_became_active":"2024-08-18T23:41:30.120196+0000","last_became_peered":"2024-08-18T23:41:30.120196+0000","last_unstale":"2024-08-18T23:43:24.313054+0000","last_undegraded":"2024-08-18T23:43:24.313054+0000","last_fullsized":"2024-08-18T23:43:24.313054+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:40:23.370088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304710+0000","last_change":"2024-08-18T23:40:22.013519+0000","last_active":"2024-08-18T23:43:24.304710+0000","last_peered":"2024-08-18T23:43:24.304710+0000","last_clean":"2024-08-18T23:43:24.304710+0000","last_became_active":"2024-08-18T23:40:22.012986+0000","last_became_peered":"2024-08-18T23:40:22.012986+0000","last_unstale":"2024-08-18T23:43:24.304710+0000","last_undegraded":"2024-08-18T23:43:24.304710+0000","last_fullsized":"2024-08-18T23:43:24.304710+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_clean_scrub_stamp":"2024-08-18T23:37:22.388424+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:19.874356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304893+0000","last_change":"2024-08-18T23:42:23.332800+0000","last_active":"2024-08-18T23:43:24.304893+0000","last_peered":"2024-08-18T23:43:24.304893+0000","last_clean":"2024-08-18T23:43:24.304893+0000","last_became_active":"2024-08-18T23:42:23.326441+0000","last_became_peered":"2024-08-18T23:42:23.326441+0000","last_unstale":"2024-08-18T23:43:24.304893+0000","last_undegraded":"2024-08-18T23:43:24.304893+0000","last_fullsized":"2024-08-18T23:43:24.304893+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:13:58.243518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.305065+0000","last_change":"2024-08-18T23:41:32.139907+0000","last_active":"2024-08-18T23:43:24.305065+0000","last_peered":"2024-08-18T23:43:24.305065+0000","last_clean":"2024-08-18T23:43:24.305065+0000","last_became_active":"2024-08-18T23:41:32.137956+0000","last_became_peered":"2024-08-18T23:41:32.137956+0000","last_unstale":"2024-08-18T23:43:24.305065+0000","last_undegraded":"2024-08-18T23:43:24.305065+0000","last_fullsized":"2024-08-18T23:43:24.305065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:10:13.223328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312461+0000","last_change":"2024-08-18T23:42:23.330062+0000","last_active":"2024-08-18T23:43:24.312461+0000","last_peered":"2024-08-18T23:43:24.312461+0000","last_clean":"2024-08-18T23:43:24.312461+0000","last_became_active":"2024-08-18T23:42:23.323196+0000","last_became_peered":"2024-08-18T23:42:23.323196+0000","last_unstale":"2024-08-18T23:43:24.312461+0000","last_undegraded":"2024-08-18T23:43:24.312461+0000","last_fullsized":"2024-08-18T23:43:24.312461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:53:29.059542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312185+0000","last_change":"2024-08-18T23:41:39.185917+0000","last_active":"2024-08-18T23:43:24.312185+0000","last_peered":"2024-08-18T23:43:24.312185+0000","last_clean":"2024-08-18T23:43:24.312185+0000","last_became_active":"2024-08-18T23:41:39.185427+0000","last_became_peered":"2024-08-18T23:41:39.185427+0000","last_unstale":"2024-08-18T23:43:24.312185+0000","last_undegraded":"2024-08-18T23:43:24.312185+0000","last_fullsized":"2024-08-18T23:43:24.312185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:52:13.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.308822+0000","last_change":"2024-08-18T23:43:24.308401+0000","last_active":"2024-08-18T23:43:24.308822+0000","last_peered":"2024-08-18T23:43:24.308822+0000","last_clean":"2024-08-18T23:43:24.308822+0000","last_became_active":"2024-08-18T23:43:24.308081+0000","last_became_peered":"2024-08-18T23:43:24.308081+0000","last_unstale":"2024-08-18T23:43:24.308822+0000","last_undegraded":"2024-08-18T23:43:24.308822+0000","last_fullsized":"2024-08-18T23:43:24.308822+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:28:57.712217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307528+0000","last_change":"2024-08-18T23:41:30.121307+0000","last_active":"2024-08-18T23:43:24.307528+0000","last_peered":"2024-08-18T23:43:24.307528+0000","last_clean":"2024-08-18T23:43:24.307528+0000","last_became_active":"2024-08-18T23:41:30.120932+0000","last_became_peered":"2024-08-18T23:41:30.120932+0000","last_unstale":"2024-08-18T23:43:24.307528+0000","last_undegraded":"2024-08-18T23:43:24.307528+0000","last_fullsized":"2024-08-18T23:43:24.307528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:06:07.742880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311751+0000","last_change":"2024-08-18T23:41:39.184647+0000","last_active":"2024-08-18T23:43:24.311751+0000","last_peered":"2024-08-18T23:43:24.311751+0000","last_clean":"2024-08-18T23:43:24.311751+0000","last_became_active":"2024-08-18T23:41:39.184251+0000","last_became_peered":"2024-08-18T23:41:39.184251+0000","last_unstale":"2024-08-18T23:43:24.311751+0000","last_undegraded":"2024-08-18T23:43:24.311751+0000","last_fullsized":"2024-08-18T23:43:24.311751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:55.200801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.305184+0000","last_change":"2024-08-18T23:41:32.137007+0000","last_active":"2024-08-18T23:43:24.305184+0000","last_peered":"2024-08-18T23:43:24.305184+0000","last_clean":"2024-08-18T23:43:24.305184+0000","last_became_active":"2024-08-18T23:41:32.136684+0000","last_became_peered":"2024-08-18T23:41:32.136684+0000","last_unstale":"2024-08-18T23:43:24.305184+0000","last_undegraded":"2024-08-18T23:43:24.305184+0000","last_fullsized":"2024-08-18T23:43:24.305184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:21:26.584303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301471+0000","last_change":"2024-08-18T23:41:30.121183+0000","last_active":"2024-08-18T23:43:24.301471+0000","last_peered":"2024-08-18T23:43:24.301471+0000","last_clean":"2024-08-18T23:43:24.301471+0000","last_became_active":"2024-08-18T23:41:30.120847+0000","last_became_peered":"2024-08-18T23:41:30.120847+0000","last_unstale":"2024-08-18T23:43:24.301471+0000","last_undegraded":"2024-08-18T23:43:24.301471+0000","last_fullsized":"2024-08-18T23:43:24.301471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:51.025821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.306833+0000","last_change":"2024-08-18T23:41:39.183375+0000","last_active":"2024-08-18T23:43:24.306833+0000","last_peered":"2024-08-18T23:43:24.306833+0000","last_clean":"2024-08-18T23:43:24.306833+0000","last_became_active":"2024-08-18T23:41:39.182339+0000","last_became_peered":"2024-08-18T23:41:39.182339+0000","last_unstale":"2024-08-18T23:43:24.306833+0000","last_undegraded":"2024-08-18T23:43:24.306833+0000","last_fullsized":"2024-08-18T23:43:24.306833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:26.119056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313146+0000","last_change":"2024-08-18T23:41:32.140748+0000","last_active":"2024-08-18T23:43:24.313146+0000","last_peered":"2024-08-18T23:43:24.313146+0000","last_clean":"2024-08-18T23:43:24.313146+0000","last_became_active":"2024-08-18T23:41:32.140306+0000","last_became_peered":"2024-08-18T23:41:32.140306+0000","last_unstale":"2024-08-18T23:43:24.313146+0000","last_undegraded":"2024-08-18T23:43:24.313146+0000","last_fullsized":"2024-08-18T23:43:24.313146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:23:02.371529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:48.359280+0000","last_change":"2024-08-18T23:41:30.121617+0000","last_active":"2024-08-18T23:43:48.359280+0000","last_peered":"2024-08-18T23:43:48.359280+0000","last_clean":"2024-08-18T23:43:48.359280+0000","last_became_active":"2024-08-18T23:41:30.121257+0000","last_became_peered":"2024-08-18T23:41:30.121257+0000","last_unstale":"2024-08-18T23:43:48.359280+0000","last_undegraded":"2024-08-18T23:43:48.359280+0000","last_fullsized":"2024-08-18T23:43:48.359280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:23:42.091553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307008+0000","last_change":"2024-08-18T23:41:39.183833+0000","last_active":"2024-08-18T23:43:24.307008+0000","last_peered":"2024-08-18T23:43:24.307008+0000","last_clean":"2024-08-18T23:43:24.307008+0000","last_became_active":"2024-08-18T23:41:39.183176+0000","last_became_peered":"2024-08-18T23:41:39.183176+0000","last_unstale":"2024-08-18T23:43:24.307008+0000","last_undegraded":"2024-08-18T23:43:24.307008+0000","last_fullsized":"2024-08-18T23:43:24.307008+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:36:09.827378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307216+0000","last_change":"2024-08-18T23:41:32.147507+0000","last_active":"2024-08-18T23:43:24.307216+0000","last_peered":"2024-08-18T23:43:24.307216+0000","last_clean":"2024-08-18T23:43:24.307216+0000","last_became_active":"2024-08-18T23:41:32.146042+0000","last_became_peered":"2024-08-18T23:41:32.146042+0000","last_unstale":"2024-08-18T23:43:24.307216+0000","last_undegraded":"2024-08-18T23:43:24.307216+0000","last_fullsized":"2024-08-18T23:43:24.307216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:48.941442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303448+0000","last_change":"2024-08-18T23:41:30.137755+0000","last_active":"2024-08-18T23:43:24.303448+0000","last_peered":"2024-08-18T23:43:24.303448+0000","last_clean":"2024-08-18T23:43:24.303448+0000","last_became_active":"2024-08-18T23:41:30.137194+0000","last_became_peered":"2024-08-18T23:41:30.137194+0000","last_unstale":"2024-08-18T23:43:24.303448+0000","last_undegraded":"2024-08-18T23:43:24.303448+0000","last_fullsized":"2024-08-18T23:43:24.303448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:15:49.681405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301312+0000","last_change":"2024-08-18T23:41:39.183986+0000","last_active":"2024-08-18T23:43:24.301312+0000","last_peered":"2024-08-18T23:43:24.301312+0000","last_clean":"2024-08-18T23:43:24.301312+0000","last_became_active":"2024-08-18T23:41:39.183384+0000","last_became_peered":"2024-08-18T23:41:39.183384+0000","last_unstale":"2024-08-18T23:43:24.301312+0000","last_undegraded":"2024-08-18T23:43:24.301312+0000","last_fullsized":"2024-08-18T23:43:24.301312+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:04:28.920804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304496+0000","last_change":"2024-08-18T23:41:32.134729+0000","last_active":"2024-08-18T23:43:24.304496+0000","last_peered":"2024-08-18T23:43:24.304496+0000","last_clean":"2024-08-18T23:43:24.304496+0000","last_became_active":"2024-08-18T23:41:32.134366+0000","last_became_peered":"2024-08-18T23:41:32.134366+0000","last_unstale":"2024-08-18T23:43:24.304496+0000","last_undegraded":"2024-08-18T23:43:24.304496+0000","last_fullsized":"2024-08-18T23:43:24.304496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:38.610561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'6","reported_seq":162,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:58.430805+0000","last_change":"2024-08-18T23:41:30.126443+0000","last_active":"2024-08-18T23:43:58.430805+0000","last_peered":"2024-08-18T23:43:58.430805+0000","last_clean":"2024-08-18T23:43:58.430805+0000","last_became_active":"2024-08-18T23:41:30.124264+0000","last_became_peered":"2024-08-18T23:41:30.124264+0000","last_unstale":"2024-08-18T23:43:58.430805+0000","last_undegraded":"2024-08-18T23:43:58.430805+0000","last_fullsized":"2024-08-18T23:43:58.430805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:43.727114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1427,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298489+0000","last_change":"2024-08-18T23:41:39.192164+0000","last_active":"2024-08-18T23:43:24.298489+0000","last_peered":"2024-08-18T23:43:24.298489+0000","last_clean":"2024-08-18T23:43:24.298489+0000","last_became_active":"2024-08-18T23:41:39.191833+0000","last_became_peered":"2024-08-18T23:41:39.191833+0000","last_unstale":"2024-08-18T23:43:24.298489+0000","last_undegraded":"2024-08-18T23:43:24.298489+0000","last_fullsized":"2024-08-18T23:43:24.298489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:33:50.243930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304927+0000","last_change":"2024-08-18T23:42:23.320662+0000","last_active":"2024-08-18T23:43:24.304927+0000","last_peered":"2024-08-18T23:43:24.304927+0000","last_clean":"2024-08-18T23:43:24.304927+0000","last_became_active":"2024-08-18T23:42:23.320249+0000","last_became_peered":"2024-08-18T23:42:23.320249+0000","last_unstale":"2024-08-18T23:43:24.304927+0000","last_undegraded":"2024-08-18T23:43:24.304927+0000","last_fullsized":"2024-08-18T23:43:24.304927+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:03:17.121915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307658+0000","last_change":"2024-08-18T23:41:30.124071+0000","last_active":"2024-08-18T23:43:24.307658+0000","last_peered":"2024-08-18T23:43:24.307658+0000","last_clean":"2024-08-18T23:43:24.307658+0000","last_became_active":"2024-08-18T23:41:30.123118+0000","last_became_peered":"2024-08-18T23:41:30.123118+0000","last_unstale":"2024-08-18T23:43:24.307658+0000","last_undegraded":"2024-08-18T23:43:24.307658+0000","last_fullsized":"2024-08-18T23:43:24.307658+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:21:39.015187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312907+0000","last_change":"2024-08-18T23:41:39.184038+0000","last_active":"2024-08-18T23:43:24.312907+0000","last_peered":"2024-08-18T23:43:24.312907+0000","last_clean":"2024-08-18T23:43:24.312907+0000","last_became_active":"2024-08-18T23:41:39.183611+0000","last_became_peered":"2024-08-18T23:41:39.183611+0000","last_unstale":"2024-08-18T23:43:24.312907+0000","last_undegraded":"2024-08-18T23:43:24.312907+0000","last_fullsized":"2024-08-18T23:43:24.312907+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:32:26.029151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300998+0000","last_change":"2024-08-18T23:41:32.143631+0000","last_active":"2024-08-18T23:43:24.300998+0000","last_peered":"2024-08-18T23:43:24.300998+0000","last_clean":"2024-08-18T23:43:24.300998+0000","last_became_active":"2024-08-18T23:41:32.143350+0000","last_became_peered":"2024-08-18T23:41:32.143350+0000","last_unstale":"2024-08-18T23:43:24.300998+0000","last_undegraded":"2024-08-18T23:43:24.300998+0000","last_fullsized":"2024-08-18T23:43:24.300998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:41:23.713467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312424+0000","last_change":"2024-08-18T23:41:30.123383+0000","last_active":"2024-08-18T23:43:24.312424+0000","last_peered":"2024-08-18T23:43:24.312424+0000","last_clean":"2024-08-18T23:43:24.312424+0000","last_became_active":"2024-08-18T23:41:30.123037+0000","last_became_peered":"2024-08-18T23:41:30.123037+0000","last_unstale":"2024-08-18T23:43:24.312424+0000","last_undegraded":"2024-08-18T23:43:24.312424+0000","last_fullsized":"2024-08-18T23:43:24.312424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:27.702478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307094+0000","last_change":"2024-08-18T23:41:39.184302+0000","last_active":"2024-08-18T23:43:24.307094+0000","last_peered":"2024-08-18T23:43:24.307094+0000","last_clean":"2024-08-18T23:43:24.307094+0000","last_became_active":"2024-08-18T23:41:39.183172+0000","last_became_peered":"2024-08-18T23:41:39.183172+0000","last_unstale":"2024-08-18T23:43:24.307094+0000","last_undegraded":"2024-08-18T23:43:24.307094+0000","last_fullsized":"2024-08-18T23:43:24.307094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:23:26.655865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304016+0000","last_change":"2024-08-18T23:41:32.137426+0000","last_active":"2024-08-18T23:43:24.304016+0000","last_peered":"2024-08-18T23:43:24.304016+0000","last_clean":"2024-08-18T23:43:24.304016+0000","last_became_active":"2024-08-18T23:41:32.137106+0000","last_became_peered":"2024-08-18T23:41:32.137106+0000","last_unstale":"2024-08-18T23:43:24.304016+0000","last_undegraded":"2024-08-18T23:43:24.304016+0000","last_fullsized":"2024-08-18T23:43:24.304016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:34:51.889663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'39","reported_seq":314,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:59.432216+0000","last_change":"2024-08-18T23:41:30.124320+0000","last_active":"2024-08-18T23:43:59.432216+0000","last_peered":"2024-08-18T23:43:59.432216+0000","last_clean":"2024-08-18T23:43:59.432216+0000","last_became_active":"2024-08-18T23:41:30.123979+0000","last_became_peered":"2024-08-18T23:41:30.123979+0000","last_unstale":"2024-08-18T23:43:59.432216+0000","last_undegraded":"2024-08-18T23:43:59.432216+0000","last_fullsized":"2024-08-18T23:43:59.432216+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:31:03.689720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68257,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":236,"num_read_kb":185,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.297979+0000","last_change":"2024-08-18T23:41:39.190481+0000","last_active":"2024-08-18T23:43:24.297979+0000","last_peered":"2024-08-18T23:43:24.297979+0000","last_clean":"2024-08-18T23:43:24.297979+0000","last_became_active":"2024-08-18T23:41:39.190076+0000","last_became_peered":"2024-08-18T23:41:39.190076+0000","last_unstale":"2024-08-18T23:43:24.297979+0000","last_undegraded":"2024-08-18T23:43:24.297979+0000","last_fullsized":"2024-08-18T23:43:24.297979+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:47:39.313142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312507+0000","last_change":"2024-08-18T23:41:32.145422+0000","last_active":"2024-08-18T23:43:24.312507+0000","last_peered":"2024-08-18T23:43:24.312507+0000","last_clean":"2024-08-18T23:43:24.312507+0000","last_became_active":"2024-08-18T23:41:32.145047+0000","last_became_peered":"2024-08-18T23:41:32.145047+0000","last_unstale":"2024-08-18T23:43:24.312507+0000","last_undegraded":"2024-08-18T23:43:24.312507+0000","last_fullsized":"2024-08-18T23:43:24.312507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:31.905248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307467+0000","last_change":"2024-08-18T23:41:30.137364+0000","last_active":"2024-08-18T23:43:24.307467+0000","last_peered":"2024-08-18T23:43:24.307467+0000","last_clean":"2024-08-18T23:43:24.307467+0000","last_became_active":"2024-08-18T23:41:30.137004+0000","last_became_peered":"2024-08-18T23:41:30.137004+0000","last_unstale":"2024-08-18T23:43:24.307467+0000","last_undegraded":"2024-08-18T23:43:24.307467+0000","last_fullsized":"2024-08-18T23:43:24.307467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:20:56.392149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312779+0000","last_change":"2024-08-18T23:42:23.326359+0000","last_active":"2024-08-18T23:43:24.312779+0000","last_peered":"2024-08-18T23:43:24.312779+0000","last_clean":"2024-08-18T23:43:24.312779+0000","last_became_active":"2024-08-18T23:42:23.325918+0000","last_became_peered":"2024-08-18T23:42:23.325918+0000","last_unstale":"2024-08-18T23:43:24.312779+0000","last_undegraded":"2024-08-18T23:43:24.312779+0000","last_fullsized":"2024-08-18T23:43:24.312779+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:01:15.476268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312633+0000","last_change":"2024-08-18T23:42:23.325691+0000","last_active":"2024-08-18T23:43:24.312633+0000","last_peered":"2024-08-18T23:43:24.312633+0000","last_clean":"2024-08-18T23:43:24.312633+0000","last_became_active":"2024-08-18T23:42:23.325467+0000","last_became_peered":"2024-08-18T23:42:23.325467+0000","last_unstale":"2024-08-18T23:43:24.312633+0000","last_undegraded":"2024-08-18T23:43:24.312633+0000","last_fullsized":"2024-08-18T23:43:24.312633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:10:26.288880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.305106+0000","last_change":"2024-08-18T23:41:30.124430+0000","last_active":"2024-08-18T23:43:24.305106+0000","last_peered":"2024-08-18T23:43:24.305106+0000","last_clean":"2024-08-18T23:43:24.305106+0000","last_became_active":"2024-08-18T23:41:30.123626+0000","last_became_peered":"2024-08-18T23:41:30.123626+0000","last_unstale":"2024-08-18T23:43:24.305106+0000","last_undegraded":"2024-08-18T23:43:24.305106+0000","last_fullsized":"2024-08-18T23:43:24.305106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:26:18.426010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.297793+0000","last_change":"2024-08-18T23:41:39.192765+0000","last_active":"2024-08-18T23:43:24.297793+0000","last_peered":"2024-08-18T23:43:24.297793+0000","last_clean":"2024-08-18T23:43:24.297793+0000","last_became_active":"2024-08-18T23:41:39.192250+0000","last_became_peered":"2024-08-18T23:41:39.192250+0000","last_unstale":"2024-08-18T23:43:24.297793+0000","last_undegraded":"2024-08-18T23:43:24.297793+0000","last_fullsized":"2024-08-18T23:43:24.297793+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:51:16.794404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312750+0000","last_change":"2024-08-18T23:41:32.141966+0000","last_active":"2024-08-18T23:43:24.312750+0000","last_peered":"2024-08-18T23:43:24.312750+0000","last_clean":"2024-08-18T23:43:24.312750+0000","last_became_active":"2024-08-18T23:41:32.141561+0000","last_became_peered":"2024-08-18T23:41:32.141561+0000","last_unstale":"2024-08-18T23:43:24.312750+0000","last_undegraded":"2024-08-18T23:43:24.312750+0000","last_fullsized":"2024-08-18T23:43:24.312750+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:20.696089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":156,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:58.429998+0000","last_change":"2024-08-18T23:41:30.121598+0000","last_active":"2024-08-18T23:43:58.429998+0000","last_peered":"2024-08-18T23:43:58.429998+0000","last_clean":"2024-08-18T23:43:58.429998+0000","last_became_active":"2024-08-18T23:41:30.120533+0000","last_became_peered":"2024-08-18T23:41:30.120533+0000","last_unstale":"2024-08-18T23:43:58.429998+0000","last_undegraded":"2024-08-18T23:43:58.429998+0000","last_fullsized":"2024-08-18T23:43:58.429998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:06:37.458272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312220+0000","last_change":"2024-08-18T23:41:39.178877+0000","last_active":"2024-08-18T23:43:24.312220+0000","last_peered":"2024-08-18T23:43:24.312220+0000","last_clean":"2024-08-18T23:43:24.312220+0000","last_became_active":"2024-08-18T23:41:39.178381+0000","last_became_peered":"2024-08-18T23:41:39.178381+0000","last_unstale":"2024-08-18T23:43:24.312220+0000","last_undegraded":"2024-08-18T23:43:24.312220+0000","last_fullsized":"2024-08-18T23:43:24.312220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:27.275893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311702+0000","last_change":"2024-08-18T23:41:39.188381+0000","last_active":"2024-08-18T23:43:24.311702+0000","last_peered":"2024-08-18T23:43:24.311702+0000","last_clean":"2024-08-18T23:43:24.311702+0000","last_became_active":"2024-08-18T23:41:39.187983+0000","last_became_peered":"2024-08-18T23:41:39.187983+0000","last_unstale":"2024-08-18T23:43:24.311702+0000","last_undegraded":"2024-08-18T23:43:24.311702+0000","last_fullsized":"2024-08-18T23:43:24.311702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:04:45.460576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.297935+0000","last_change":"2024-08-18T23:41:32.144637+0000","last_active":"2024-08-18T23:43:24.297935+0000","last_peered":"2024-08-18T23:43:24.297935+0000","last_clean":"2024-08-18T23:43:24.297935+0000","last_became_active":"2024-08-18T23:41:32.144329+0000","last_became_peered":"2024-08-18T23:41:32.144329+0000","last_unstale":"2024-08-18T23:43:24.297935+0000","last_undegraded":"2024-08-18T23:43:24.297935+0000","last_fullsized":"2024-08-18T23:43:24.297935+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:29:39.791264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307199+0000","last_change":"2024-08-18T23:41:30.122325+0000","last_active":"2024-08-18T23:43:24.307199+0000","last_peered":"2024-08-18T23:43:24.307199+0000","last_clean":"2024-08-18T23:43:24.307199+0000","last_became_active":"2024-08-18T23:41:30.121480+0000","last_became_peered":"2024-08-18T23:41:30.121480+0000","last_unstale":"2024-08-18T23:43:24.307199+0000","last_undegraded":"2024-08-18T23:43:24.307199+0000","last_fullsized":"2024-08-18T23:43:24.307199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:32:12.810650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312772+0000","last_change":"2024-08-18T23:42:23.560632+0000","last_active":"2024-08-18T23:43:24.312772+0000","last_peered":"2024-08-18T23:43:24.312772+0000","last_clean":"2024-08-18T23:43:24.312772+0000","last_became_active":"2024-08-18T23:42:23.559920+0000","last_became_peered":"2024-08-18T23:42:23.559920+0000","last_unstale":"2024-08-18T23:43:24.312772+0000","last_undegraded":"2024-08-18T23:43:24.312772+0000","last_fullsized":"2024-08-18T23:43:24.312772+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:01:59.580843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312091+0000","last_change":"2024-08-18T23:41:32.145545+0000","last_active":"2024-08-18T23:43:24.312091+0000","last_peered":"2024-08-18T23:43:24.312091+0000","last_clean":"2024-08-18T23:43:24.312091+0000","last_became_active":"2024-08-18T23:41:32.145183+0000","last_became_peered":"2024-08-18T23:41:32.145183+0000","last_unstale":"2024-08-18T23:43:24.312091+0000","last_undegraded":"2024-08-18T23:43:24.312091+0000","last_fullsized":"2024-08-18T23:43:24.312091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:03:04.111658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300895+0000","last_change":"2024-08-18T23:41:30.118776+0000","last_active":"2024-08-18T23:43:24.300895+0000","last_peered":"2024-08-18T23:43:24.300895+0000","last_clean":"2024-08-18T23:43:24.300895+0000","last_became_active":"2024-08-18T23:41:30.118436+0000","last_became_peered":"2024-08-18T23:41:30.118436+0000","last_unstale":"2024-08-18T23:43:24.300895+0000","last_undegraded":"2024-08-18T23:43:24.300895+0000","last_fullsized":"2024-08-18T23:43:24.300895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:13.980927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300932+0000","last_change":"2024-08-18T23:41:30.125201+0000","last_active":"2024-08-18T23:43:24.300932+0000","last_peered":"2024-08-18T23:43:24.300932+0000","last_clean":"2024-08-18T23:43:24.300932+0000","last_became_active":"2024-08-18T23:41:30.124894+0000","last_became_peered":"2024-08-18T23:41:30.124894+0000","last_unstale":"2024-08-18T23:43:24.300932+0000","last_undegraded":"2024-08-18T23:43:24.300932+0000","last_fullsized":"2024-08-18T23:43:24.300932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:30:17.721829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307776+0000","last_change":"2024-08-18T23:41:32.145452+0000","last_active":"2024-08-18T23:43:24.307776+0000","last_peered":"2024-08-18T23:43:24.307776+0000","last_clean":"2024-08-18T23:43:24.307776+0000","last_became_active":"2024-08-18T23:41:32.145151+0000","last_became_peered":"2024-08-18T23:41:32.145151+0000","last_unstale":"2024-08-18T23:43:24.307776+0000","last_undegraded":"2024-08-18T23:43:24.307776+0000","last_fullsized":"2024-08-18T23:43:24.307776+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:08:08.713154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312361+0000","last_change":"2024-08-18T23:41:39.193334+0000","last_active":"2024-08-18T23:43:24.312361+0000","last_peered":"2024-08-18T23:43:24.312361+0000","last_clean":"2024-08-18T23:43:24.312361+0000","last_became_active":"2024-08-18T23:41:39.192924+0000","last_became_peered":"2024-08-18T23:41:39.192924+0000","last_unstale":"2024-08-18T23:43:24.312361+0000","last_undegraded":"2024-08-18T23:43:24.312361+0000","last_fullsized":"2024-08-18T23:43:24.312361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:07.016300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":166,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:44:01.435799+0000","last_change":"2024-08-18T23:41:30.125217+0000","last_active":"2024-08-18T23:44:01.435799+0000","last_peered":"2024-08-18T23:44:01.435799+0000","last_clean":"2024-08-18T23:44:01.435799+0000","last_became_active":"2024-08-18T23:41:30.124889+0000","last_became_peered":"2024-08-18T23:41:30.124889+0000","last_unstale":"2024-08-18T23:44:01.435799+0000","last_undegraded":"2024-08-18T23:44:01.435799+0000","last_fullsized":"2024-08-18T23:44:01.435799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:43.389537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303483+0000","last_change":"2024-08-18T23:41:32.142585+0000","last_active":"2024-08-18T23:43:24.303483+0000","last_peered":"2024-08-18T23:43:24.303483+0000","last_clean":"2024-08-18T23:43:24.303483+0000","last_became_active":"2024-08-18T23:41:32.142207+0000","last_became_peered":"2024-08-18T23:41:32.142207+0000","last_unstale":"2024-08-18T23:43:24.303483+0000","last_undegraded":"2024-08-18T23:43:24.303483+0000","last_fullsized":"2024-08-18T23:43:24.303483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:16:58.755444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312344+0000","last_change":"2024-08-18T23:41:39.185618+0000","last_active":"2024-08-18T23:43:24.312344+0000","last_peered":"2024-08-18T23:43:24.312344+0000","last_clean":"2024-08-18T23:43:24.312344+0000","last_became_active":"2024-08-18T23:41:39.185222+0000","last_became_peered":"2024-08-18T23:41:39.185222+0000","last_unstale":"2024-08-18T23:43:24.312344+0000","last_undegraded":"2024-08-18T23:43:24.312344+0000","last_fullsized":"2024-08-18T23:43:24.312344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:01:27.290998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312646+0000","last_change":"2024-08-18T23:41:32.140719+0000","last_active":"2024-08-18T23:43:24.312646+0000","last_peered":"2024-08-18T23:43:24.312646+0000","last_clean":"2024-08-18T23:43:24.312646+0000","last_became_active":"2024-08-18T23:41:32.140131+0000","last_became_peered":"2024-08-18T23:41:32.140131+0000","last_unstale":"2024-08-18T23:43:24.312646+0000","last_undegraded":"2024-08-18T23:43:24.312646+0000","last_fullsized":"2024-08-18T23:43:24.312646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:48:26.778094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312375+0000","last_change":"2024-08-18T23:42:23.326683+0000","last_active":"2024-08-18T23:43:24.312375+0000","last_peered":"2024-08-18T23:43:24.312375+0000","last_clean":"2024-08-18T23:43:24.312375+0000","last_became_active":"2024-08-18T23:42:23.326309+0000","last_became_peered":"2024-08-18T23:42:23.326309+0000","last_unstale":"2024-08-18T23:43:24.312375+0000","last_undegraded":"2024-08-18T23:43:24.312375+0000","last_fullsized":"2024-08-18T23:43:24.312375+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:14.320894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312340+0000","last_change":"2024-08-18T23:41:32.149522+0000","last_active":"2024-08-18T23:43:24.312340+0000","last_peered":"2024-08-18T23:43:24.312340+0000","last_clean":"2024-08-18T23:43:24.312340+0000","last_became_active":"2024-08-18T23:41:32.149182+0000","last_became_peered":"2024-08-18T23:41:32.149182+0000","last_unstale":"2024-08-18T23:43:24.312340+0000","last_undegraded":"2024-08-18T23:43:24.312340+0000","last_fullsized":"2024-08-18T23:43:24.312340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:30:16.945045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311765+0000","last_change":"2024-08-18T23:41:32.149853+0000","last_active":"2024-08-18T23:43:24.311765+0000","last_peered":"2024-08-18T23:43:24.311765+0000","last_clean":"2024-08-18T23:43:24.311765+0000","last_became_active":"2024-08-18T23:41:32.141907+0000","last_became_peered":"2024-08-18T23:41:32.141907+0000","last_unstale":"2024-08-18T23:43:24.311765+0000","last_undegraded":"2024-08-18T23:43:24.311765+0000","last_fullsized":"2024-08-18T23:43:24.311765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:24:37.323567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301279+0000","last_change":"2024-08-18T23:41:32.136179+0000","last_active":"2024-08-18T23:43:24.301279+0000","last_peered":"2024-08-18T23:43:24.301279+0000","last_clean":"2024-08-18T23:43:24.301279+0000","last_became_active":"2024-08-18T23:41:32.135939+0000","last_became_peered":"2024-08-18T23:41:32.135939+0000","last_unstale":"2024-08-18T23:43:24.301279+0000","last_undegraded":"2024-08-18T23:43:24.301279+0000","last_fullsized":"2024-08-18T23:43:24.301279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:15.392061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313279+0000","last_change":"2024-08-18T23:41:32.137509+0000","last_active":"2024-08-18T23:43:24.313279+0000","last_peered":"2024-08-18T23:43:24.313279+0000","last_clean":"2024-08-18T23:43:24.313279+0000","last_became_active":"2024-08-18T23:41:32.136380+0000","last_became_peered":"2024-08-18T23:41:32.136380+0000","last_unstale":"2024-08-18T23:43:24.313279+0000","last_undegraded":"2024-08-18T23:43:24.313279+0000","last_fullsized":"2024-08-18T23:43:24.313279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:49.776225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312602+0000","last_change":"2024-08-18T23:41:32.139271+0000","last_active":"2024-08-18T23:43:24.312602+0000","last_peered":"2024-08-18T23:43:24.312602+0000","last_clean":"2024-08-18T23:43:24.312602+0000","last_became_active":"2024-08-18T23:41:32.138868+0000","last_became_peered":"2024-08-18T23:41:32.138868+0000","last_unstale":"2024-08-18T23:43:24.312602+0000","last_undegraded":"2024-08-18T23:43:24.312602+0000","last_fullsized":"2024-08-18T23:43:24.312602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:49:27.890857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312543+0000","last_change":"2024-08-18T23:41:32.144460+0000","last_active":"2024-08-18T23:43:24.312543+0000","last_peered":"2024-08-18T23:43:24.312543+0000","last_clean":"2024-08-18T23:43:24.312543+0000","last_became_active":"2024-08-18T23:41:32.144178+0000","last_became_peered":"2024-08-18T23:41:32.144178+0000","last_unstale":"2024-08-18T23:43:24.312543+0000","last_undegraded":"2024-08-18T23:43:24.312543+0000","last_fullsized":"2024-08-18T23:43:24.312543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:56.252215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.306952+0000","last_change":"2024-08-18T23:41:32.147973+0000","last_active":"2024-08-18T23:43:24.306952+0000","last_peered":"2024-08-18T23:43:24.306952+0000","last_clean":"2024-08-18T23:43:24.306952+0000","last_became_active":"2024-08-18T23:41:32.147621+0000","last_became_peered":"2024-08-18T23:41:32.147621+0000","last_unstale":"2024-08-18T23:43:24.306952+0000","last_undegraded":"2024-08-18T23:43:24.306952+0000","last_fullsized":"2024-08-18T23:43:24.306952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:14.176569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312329+0000","last_change":"2024-08-18T23:41:32.146029+0000","last_active":"2024-08-18T23:43:24.312329+0000","last_peered":"2024-08-18T23:43:24.312329+0000","last_clean":"2024-08-18T23:43:24.312329+0000","last_became_active":"2024-08-18T23:41:32.145584+0000","last_became_peered":"2024-08-18T23:41:32.145584+0000","last_unstale":"2024-08-18T23:43:24.312329+0000","last_undegraded":"2024-08-18T23:43:24.312329+0000","last_fullsized":"2024-08-18T23:43:24.312329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:25:33.884906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312304+0000","last_change":"2024-08-18T23:41:32.145950+0000","last_active":"2024-08-18T23:43:24.312304+0000","last_peered":"2024-08-18T23:43:24.312304+0000","last_clean":"2024-08-18T23:43:24.312304+0000","last_became_active":"2024-08-18T23:41:32.145618+0000","last_became_peered":"2024-08-18T23:41:32.145618+0000","last_unstale":"2024-08-18T23:43:24.312304+0000","last_undegraded":"2024-08-18T23:43:24.312304+0000","last_fullsized":"2024-08-18T23:43:24.312304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:05:58.889240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301154+0000","last_change":"2024-08-18T23:41:32.130684+0000","last_active":"2024-08-18T23:43:24.301154+0000","last_peered":"2024-08-18T23:43:24.301154+0000","last_clean":"2024-08-18T23:43:24.301154+0000","last_became_active":"2024-08-18T23:41:32.130369+0000","last_became_peered":"2024-08-18T23:41:32.130369+0000","last_unstale":"2024-08-18T23:43:24.301154+0000","last_undegraded":"2024-08-18T23:43:24.301154+0000","last_fullsized":"2024-08-18T23:43:24.301154+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:22.537018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313085+0000","last_change":"2024-08-18T23:41:32.137291+0000","last_active":"2024-08-18T23:43:24.313085+0000","last_peered":"2024-08-18T23:43:24.313085+0000","last_clean":"2024-08-18T23:43:24.313085+0000","last_became_active":"2024-08-18T23:41:32.136977+0000","last_became_peered":"2024-08-18T23:41:32.136977+0000","last_unstale":"2024-08-18T23:43:24.313085+0000","last_undegraded":"2024-08-18T23:43:24.313085+0000","last_fullsized":"2024-08-18T23:43:24.313085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:18:45.544760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312711+0000","last_change":"2024-08-18T23:41:32.144434+0000","last_active":"2024-08-18T23:43:24.312711+0000","last_peered":"2024-08-18T23:43:24.312711+0000","last_clean":"2024-08-18T23:43:24.312711+0000","last_became_active":"2024-08-18T23:41:32.144119+0000","last_became_peered":"2024-08-18T23:41:32.144119+0000","last_unstale":"2024-08-18T23:43:24.312711+0000","last_undegraded":"2024-08-18T23:43:24.312711+0000","last_fullsized":"2024-08-18T23:43:24.312711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:50:54.528630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298360+0000","last_change":"2024-08-18T23:41:32.135034+0000","last_active":"2024-08-18T23:43:24.298360+0000","last_peered":"2024-08-18T23:43:24.298360+0000","last_clean":"2024-08-18T23:43:24.298360+0000","last_became_active":"2024-08-18T23:41:32.134700+0000","last_became_peered":"2024-08-18T23:41:32.134700+0000","last_unstale":"2024-08-18T23:43:24.298360+0000","last_undegraded":"2024-08-18T23:43:24.298360+0000","last_fullsized":"2024-08-18T23:43:24.298360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:06:31.263114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313052+0000","last_change":"2024-08-18T23:41:32.141191+0000","last_active":"2024-08-18T23:43:24.313052+0000","last_peered":"2024-08-18T23:43:24.313052+0000","last_clean":"2024-08-18T23:43:24.313052+0000","last_became_active":"2024-08-18T23:41:32.140901+0000","last_became_peered":"2024-08-18T23:41:32.140901+0000","last_unstale":"2024-08-18T23:43:24.313052+0000","last_undegraded":"2024-08-18T23:43:24.313052+0000","last_fullsized":"2024-08-18T23:43:24.313052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:24.849985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311944+0000","last_change":"2024-08-18T23:41:32.142543+0000","last_active":"2024-08-18T23:43:24.311944+0000","last_peered":"2024-08-18T23:43:24.311944+0000","last_clean":"2024-08-18T23:43:24.311944+0000","last_became_active":"2024-08-18T23:41:32.142084+0000","last_became_peered":"2024-08-18T23:41:32.142084+0000","last_unstale":"2024-08-18T23:43:24.311944+0000","last_undegraded":"2024-08-18T23:43:24.311944+0000","last_fullsized":"2024-08-18T23:43:24.311944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:54:18.200016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311562+0000","last_change":"2024-08-18T23:41:32.148968+0000","last_active":"2024-08-18T23:43:24.311562+0000","last_peered":"2024-08-18T23:43:24.311562+0000","last_clean":"2024-08-18T23:43:24.311562+0000","last_became_active":"2024-08-18T23:41:32.148625+0000","last_became_peered":"2024-08-18T23:41:32.148625+0000","last_unstale":"2024-08-18T23:43:24.311562+0000","last_undegraded":"2024-08-18T23:43:24.311562+0000","last_fullsized":"2024-08-18T23:43:24.311562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:11:40.172139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311624+0000","last_change":"2024-08-18T23:41:32.133478+0000","last_active":"2024-08-18T23:43:24.311624+0000","last_peered":"2024-08-18T23:43:24.311624+0000","last_clean":"2024-08-18T23:43:24.311624+0000","last_became_active":"2024-08-18T23:41:32.133149+0000","last_became_peered":"2024-08-18T23:41:32.133149+0000","last_unstale":"2024-08-18T23:43:24.311624+0000","last_undegraded":"2024-08-18T23:43:24.311624+0000","last_fullsized":"2024-08-18T23:43:24.311624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:57:53.713309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303573+0000","last_change":"2024-08-18T23:43:24.303232+0000","last_active":"2024-08-18T23:43:24.303573+0000","last_peered":"2024-08-18T23:43:24.303573+0000","last_clean":"2024-08-18T23:43:24.303573+0000","last_became_active":"2024-08-18T23:43:24.302849+0000","last_became_peered":"2024-08-18T23:43:24.302849+0000","last_unstale":"2024-08-18T23:43:24.303573+0000","last_undegraded":"2024-08-18T23:43:24.303573+0000","last_fullsized":"2024-08-18T23:43:24.303573+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:02:02.479891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313096+0000","last_change":"2024-08-18T23:41:32.136303+0000","last_active":"2024-08-18T23:43:24.313096+0000","last_peered":"2024-08-18T23:43:24.313096+0000","last_clean":"2024-08-18T23:43:24.313096+0000","last_became_active":"2024-08-18T23:41:32.134628+0000","last_became_peered":"2024-08-18T23:41:32.134628+0000","last_unstale":"2024-08-18T23:43:24.313096+0000","last_undegraded":"2024-08-18T23:43:24.313096+0000","last_fullsized":"2024-08-18T23:43:24.313096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:28:51.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304992+0000","last_change":"2024-08-18T23:41:32.137512+0000","last_active":"2024-08-18T23:43:24.304992+0000","last_peered":"2024-08-18T23:43:24.304992+0000","last_clean":"2024-08-18T23:43:24.304992+0000","last_became_active":"2024-08-18T23:41:32.137150+0000","last_became_peered":"2024-08-18T23:41:32.137150+0000","last_unstale":"2024-08-18T23:43:24.304992+0000","last_undegraded":"2024-08-18T23:43:24.304992+0000","last_fullsized":"2024-08-18T23:43:24.304992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:44:08.020780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301258+0000","last_change":"2024-08-18T23:41:32.139229+0000","last_active":"2024-08-18T23:43:24.301258+0000","last_peered":"2024-08-18T23:43:24.301258+0000","last_clean":"2024-08-18T23:43:24.301258+0000","last_became_active":"2024-08-18T23:41:32.138864+0000","last_became_peered":"2024-08-18T23:41:32.138864+0000","last_unstale":"2024-08-18T23:43:24.301258+0000","last_undegraded":"2024-08-18T23:43:24.301258+0000","last_fullsized":"2024-08-18T23:43:24.301258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:54:21.701750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312829+0000","last_change":"2024-08-18T23:41:32.148727+0000","last_active":"2024-08-18T23:43:24.312829+0000","last_peered":"2024-08-18T23:43:24.312829+0000","last_clean":"2024-08-18T23:43:24.312829+0000","last_became_active":"2024-08-18T23:41:32.146828+0000","last_became_peered":"2024-08-18T23:41:32.146828+0000","last_unstale":"2024-08-18T23:43:24.312829+0000","last_undegraded":"2024-08-18T23:43:24.312829+0000","last_fullsized":"2024-08-18T23:43:24.312829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:51:49.175150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312029+0000","last_change":"2024-08-18T23:41:32.144796+0000","last_active":"2024-08-18T23:43:24.312029+0000","last_peered":"2024-08-18T23:43:24.312029+0000","last_clean":"2024-08-18T23:43:24.312029+0000","last_became_active":"2024-08-18T23:41:32.141931+0000","last_became_peered":"2024-08-18T23:41:32.141931+0000","last_unstale":"2024-08-18T23:43:24.312029+0000","last_undegraded":"2024-08-18T23:43:24.312029+0000","last_fullsized":"2024-08-18T23:43:24.312029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:48:08.428845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304569+0000","last_change":"2024-08-18T23:41:32.135265+0000","last_active":"2024-08-18T23:43:24.304569+0000","last_peered":"2024-08-18T23:43:24.304569+0000","last_clean":"2024-08-18T23:43:24.304569+0000","last_became_active":"2024-08-18T23:41:32.134952+0000","last_became_peered":"2024-08-18T23:41:32.134952+0000","last_unstale":"2024-08-18T23:43:24.304569+0000","last_undegraded":"2024-08-18T23:43:24.304569+0000","last_fullsized":"2024-08-18T23:43:24.304569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:35:14.099536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300727+0000","last_change":"2024-08-18T23:41:32.135941+0000","last_active":"2024-08-18T23:43:24.300727+0000","last_peered":"2024-08-18T23:43:24.300727+0000","last_clean":"2024-08-18T23:43:24.300727+0000","last_became_active":"2024-08-18T23:41:32.135623+0000","last_became_peered":"2024-08-18T23:41:32.135623+0000","last_unstale":"2024-08-18T23:43:24.300727+0000","last_undegraded":"2024-08-18T23:43:24.300727+0000","last_fullsized":"2024-08-18T23:43:24.300727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:43:36.504738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303544+0000","last_change":"2024-08-18T23:41:32.141643+0000","last_active":"2024-08-18T23:43:24.303544+0000","last_peered":"2024-08-18T23:43:24.303544+0000","last_clean":"2024-08-18T23:43:24.303544+0000","last_became_active":"2024-08-18T23:41:32.141301+0000","last_became_peered":"2024-08-18T23:41:32.141301+0000","last_unstale":"2024-08-18T23:43:24.303544+0000","last_undegraded":"2024-08-18T23:43:24.303544+0000","last_fullsized":"2024-08-18T23:43:24.303544+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:23:53.100740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312988+0000","last_change":"2024-08-18T23:41:32.146730+0000","last_active":"2024-08-18T23:43:24.312988+0000","last_peered":"2024-08-18T23:43:24.312988+0000","last_clean":"2024-08-18T23:43:24.312988+0000","last_became_active":"2024-08-18T23:41:32.146252+0000","last_became_peered":"2024-08-18T23:41:32.146252+0000","last_unstale":"2024-08-18T23:43:24.312988+0000","last_undegraded":"2024-08-18T23:43:24.312988+0000","last_fullsized":"2024-08-18T23:43:24.312988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:41.481761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311744+0000","last_change":"2024-08-18T23:41:32.149569+0000","last_active":"2024-08-18T23:43:24.311744+0000","last_peered":"2024-08-18T23:43:24.311744+0000","last_clean":"2024-08-18T23:43:24.311744+0000","last_became_active":"2024-08-18T23:41:32.149277+0000","last_became_peered":"2024-08-18T23:41:32.149277+0000","last_unstale":"2024-08-18T23:43:24.311744+0000","last_undegraded":"2024-08-18T23:43:24.311744+0000","last_fullsized":"2024-08-18T23:43:24.311744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:34:00.547909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311950+0000","last_change":"2024-08-18T23:41:32.137889+0000","last_active":"2024-08-18T23:43:24.311950+0000","last_peered":"2024-08-18T23:43:24.311950+0000","last_clean":"2024-08-18T23:43:24.311950+0000","last_became_active":"2024-08-18T23:41:32.137578+0000","last_became_peered":"2024-08-18T23:41:32.137578+0000","last_unstale":"2024-08-18T23:43:24.311950+0000","last_undegraded":"2024-08-18T23:43:24.311950+0000","last_fullsized":"2024-08-18T23:43:24.311950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:18:09.987838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312999+0000","last_change":"2024-08-18T23:41:32.142084+0000","last_active":"2024-08-18T23:43:24.312999+0000","last_peered":"2024-08-18T23:43:24.312999+0000","last_clean":"2024-08-18T23:43:24.312999+0000","last_became_active":"2024-08-18T23:41:32.141670+0000","last_became_peered":"2024-08-18T23:41:32.141670+0000","last_unstale":"2024-08-18T23:43:24.312999+0000","last_undegraded":"2024-08-18T23:43:24.312999+0000","last_fullsized":"2024-08-18T23:43:24.312999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:03.990351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73627,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":708,"num_read_kb":431,"num_write":188,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":220881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514539,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":523788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776176,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1180152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939700,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":528120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168633,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":528040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397565,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1662976,"data_stored":1121222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561090,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1048576,"data_stored":519200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790023,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1111875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953547,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1064960,"data_stored":525271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":519394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":589056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1122304,"data_stored":587367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":520588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T23:44:02.578 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:44:02.862 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:03.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.393+0000 7f010eb83640 1 -- 172.21.15.78:0/3745172620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01081007b0 msgr2=0x7f0108100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:03.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.393+0000 7f010eb83640 1 --2- 172.21.15.78:0/3745172620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01081007b0 0x7f0108100c10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009930 tx=0x7f00f402a420 comp rx=0 tx=0).stop 2024-08-18T23:44:03.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.393+0000 7f010eb83640 1 -- 172.21.15.78:0/3745172620 shutdown_connections 2024-08-18T23:44:03.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.393+0000 7f010eb83640 1 --2- 172.21.15.78:0/3745172620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0108101150 0x7f010810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:03.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.393+0000 7f010eb83640 1 --2- 172.21.15.78:0/3745172620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01081007b0 0x7f0108100c10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:03.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.393+0000 7f010eb83640 1 --2- 172.21.15.78:0/3745172620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01081067b0 0x7f0108106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:03.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.393+0000 7f010eb83640 1 -- 172.21.15.78:0/3745172620 >> 172.21.15.78:0/3745172620 conn(0x7f01080fc4d0 msgr2=0x7f01080fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:03.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.393+0000 7f010eb83640 1 -- 172.21.15.78:0/3745172620 shutdown_connections 2024-08-18T23:44:03.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.393+0000 7f010eb83640 1 -- 172.21.15.78:0/3745172620 wait complete. 2024-08-18T23:44:03.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.393+0000 7f010eb83640 1 Processor -- start 2024-08-18T23:44:03.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.393+0000 7f010eb83640 1 -- start start 2024-08-18T23:44:03.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f010eb83640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01081007b0 0x7f01081971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:03.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f010eb83640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0108101150 0x7f0108197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:03.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f010eb83640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01081067b0 0x7f010819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:03.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f010eb83640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0108110c50 con 0x7f01081007b0 2024-08-18T23:44:03.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f010eb83640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0108110ad0 con 0x7f0108101150 2024-08-18T23:44:03.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f010eb83640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0108110dd0 con 0x7f01081067b0 2024-08-18T23:44:03.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f010c91f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01081007b0 0x7f01081971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:03.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f010c91f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01081007b0 0x7f01081971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51308/0 (socket says 172.21.15.78:51308) 2024-08-18T23:44:03.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f010c91f640 1 -- 172.21.15.78:0/1503036915 learned_addr learned my addr 172.21.15.78:0/1503036915 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:03.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f0107fff640 1 --2- 172.21.15.78:0/1503036915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0108101150 0x7f0108197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:03.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f010d120640 1 --2- 172.21.15.78:0/1503036915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01081067b0 0x7f010819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:03.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f010c91f640 1 -- 172.21.15.78:0/1503036915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01081067b0 msgr2=0x7f010819bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:03.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f010c91f640 1 --2- 172.21.15.78:0/1503036915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01081067b0 0x7f010819bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:03.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f010c91f640 1 -- 172.21.15.78:0/1503036915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0108101150 msgr2=0x7f0108197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:03.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f010c91f640 1 --2- 172.21.15.78:0/1503036915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0108101150 0x7f0108197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:03.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f010c91f640 1 -- 172.21.15.78:0/1503036915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010819c1a0 con 0x7f01081007b0 2024-08-18T23:44:03.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f010d120640 1 --2- 172.21.15.78:0/1503036915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01081067b0 0x7f010819bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:44:03.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f0107fff640 1 --2- 172.21.15.78:0/1503036915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0108101150 0x7f0108197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:03.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f010c91f640 1 --2- 172.21.15.78:0/1503036915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01081007b0 0x7f01081971f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00b4a0 tx=0x7f00fc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:03.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f0105ffb640 1 -- 172.21.15.78:0/1503036915 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc014070 con 0x7f01081007b0 2024-08-18T23:44:03.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f0105ffb640 1 -- 172.21.15.78:0/1503036915 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00fc010af0 con 0x7f01081007b0 2024-08-18T23:44:03.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f0105ffb640 1 -- 172.21.15.78:0/1503036915 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc017d10 con 0x7f01081007b0 2024-08-18T23:44:03.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.394+0000 7f010eb83640 1 -- 172.21.15.78:0/1503036915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010819c490 con 0x7f01081007b0 2024-08-18T23:44:03.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.395+0000 7f010eb83640 1 -- 172.21.15.78:0/1503036915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01081a3d70 con 0x7f01081007b0 2024-08-18T23:44:03.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.396+0000 7f010eb83640 1 -- 172.21.15.78:0/1503036915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0108100c90 con 0x7f01081007b0 2024-08-18T23:44:03.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.397+0000 7f0105ffb640 1 -- 172.21.15.78:0/1503036915 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f00fc017070 con 0x7f01081007b0 2024-08-18T23:44:03.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.397+0000 7f0105ffb640 1 --2- 172.21.15.78:0/1503036915 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f00f80a2c10 0x7f00f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:03.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.398+0000 7f0107fff640 1 --2- 172.21.15.78:0/1503036915 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f00f80a2c10 0x7f00f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:03.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.398+0000 7f0105ffb640 1 -- 172.21.15.78:0/1503036915 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f00fc0ce8d0 con 0x7f01081007b0 2024-08-18T23:44:03.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.398+0000 7f0107fff640 1 --2- 172.21.15.78:0/1503036915 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f00f80a2c10 0x7f00f80a50d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f00f4000c00 tx=0x7f00f4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:03.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.402+0000 7f0105ffb640 1 -- 172.21.15.78:0/1503036915 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f00fc09b450 con 0x7f01081007b0 2024-08-18T23:44:03.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:03 smithi123 ceph-mon[21808]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:44:03.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:03 smithi123 ceph-mon[21808]: pgmap v342: 129 pgs: 129 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-18T23:44:03.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:03 smithi123 ceph-mon[21808]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:44:03.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.604+0000 7f010eb83640 1 -- 172.21.15.78:0/1503036915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0108108da0 con 0x7f01081007b0 2024-08-18T23:44:03.606 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:03 smithi078 ceph-mon[18440]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:44:03.606 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:03 smithi078 ceph-mon[18440]: pgmap v342: 129 pgs: 129 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-18T23:44:03.606 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:03 smithi078 ceph-mon[18440]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:44:03.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.604+0000 7f0105ffb640 1 -- 172.21.15.78:0/1503036915 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7f00fc09ad50 con 0x7f01081007b0 2024-08-18T23:44:03.607 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:44:03.607 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"btime":"2024-08-18T23:43:57:245845+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:43:57.114120+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24428,24427]},"id":1}]} 2024-08-18T23:44:03.607 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 25 2024-08-18T23:44:03.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.606+0000 7f010eb83640 1 -- 172.21.15.78:0/1503036915 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f00f80a2c10 msgr2=0x7f00f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:03.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.606+0000 7f010eb83640 1 --2- 172.21.15.78:0/1503036915 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f00f80a2c10 0x7f00f80a50d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f00f4000c00 tx=0x7f00f4005a20 comp rx=0 tx=0).stop 2024-08-18T23:44:03.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.606+0000 7f010eb83640 1 -- 172.21.15.78:0/1503036915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01081007b0 msgr2=0x7f01081971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:03.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.606+0000 7f010eb83640 1 --2- 172.21.15.78:0/1503036915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01081007b0 0x7f01081971f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00b4a0 tx=0x7f00fc00b960 comp rx=0 tx=0).stop 2024-08-18T23:44:03.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.606+0000 7f010eb83640 1 -- 172.21.15.78:0/1503036915 shutdown_connections 2024-08-18T23:44:03.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.606+0000 7f010eb83640 1 --2- 172.21.15.78:0/1503036915 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f00f80a2c10 0x7f00f80a50d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:03.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.606+0000 7f010eb83640 1 --2- 172.21.15.78:0/1503036915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01081067b0 0x7f010819bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:03.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.606+0000 7f010eb83640 1 --2- 172.21.15.78:0/1503036915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0108101150 0x7f0108197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:03.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.606+0000 7f010eb83640 1 --2- 172.21.15.78:0/1503036915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01081007b0 0x7f01081971f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:03.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.606+0000 7f010eb83640 1 -- 172.21.15.78:0/1503036915 >> 172.21.15.78:0/1503036915 conn(0x7f01080fc4d0 msgr2=0x7f01080fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:03.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.607+0000 7f010eb83640 1 -- 172.21.15.78:0/1503036915 shutdown_connections 2024-08-18T23:44:03.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:03.607+0000 7f010eb83640 1 -- 172.21.15.78:0/1503036915 wait complete. 2024-08-18T23:44:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:03 smithi161 ceph-mon[22027]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:44:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:03 smithi161 ceph-mon[22027]: pgmap v342: 129 pgs: 129 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-18T23:44:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:03 smithi161 ceph-mon[22027]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:44:04.323 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-18T23:44:04.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:04 smithi123 ceph-mon[21808]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:44:04.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:04 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1503036915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:04.573 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:04 smithi161 ceph-mon[22027]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:44:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:04 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1503036915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:04.696 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:04 smithi078 ceph-mon[18440]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:44:04.696 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:04 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1503036915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:05.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.059+0000 7fa49fa5d640 1 -- 172.21.15.78:0/2029475891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4981128d0 msgr2=0x7fa498114cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:05.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.059+0000 7fa49fa5d640 1 --2- 172.21.15.78:0/2029475891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4981128d0 0x7fa498114cc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa490009ca0 tx=0x7fa49002a950 comp rx=0 tx=0).stop 2024-08-18T23:44:05.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.060+0000 7fa49fa5d640 1 -- 172.21.15.78:0/2029475891 shutdown_connections 2024-08-18T23:44:05.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.060+0000 7fa49fa5d640 1 --2- 172.21.15.78:0/2029475891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4981128d0 0x7fa498114cc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:05.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.060+0000 7fa49fa5d640 1 --2- 172.21.15.78:0/2029475891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa498074be0 0x7fa498112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:05.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.060+0000 7fa49fa5d640 1 --2- 172.21.15.78:0/2029475891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4980742c0 0x7fa4980746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:05.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.060+0000 7fa49fa5d640 1 -- 172.21.15.78:0/2029475891 >> 172.21.15.78:0/2029475891 conn(0x7fa4980fd5e0 msgr2=0x7fa4980ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:05.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.060+0000 7fa49fa5d640 1 -- 172.21.15.78:0/2029475891 shutdown_connections 2024-08-18T23:44:05.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.060+0000 7fa49fa5d640 1 -- 172.21.15.78:0/2029475891 wait complete. 2024-08-18T23:44:05.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.060+0000 7fa49fa5d640 1 Processor -- start 2024-08-18T23:44:05.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.060+0000 7fa49fa5d640 1 -- start start 2024-08-18T23:44:05.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.060+0000 7fa49fa5d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4980742c0 0x7fa49819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:05.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.060+0000 7fa49fa5d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa498074be0 0x7fa49819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:05.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.060+0000 7fa49fa5d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4981128d0 0x7fa4981a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:05.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.060+0000 7fa49fa5d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa498118290 con 0x7fa4981128d0 2024-08-18T23:44:05.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.060+0000 7fa49fa5d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa498118110 con 0x7fa4980742c0 2024-08-18T23:44:05.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.060+0000 7fa49fa5d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa498118410 con 0x7fa498074be0 2024-08-18T23:44:05.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.061+0000 7fa49dffa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4981128d0 0x7fa4981a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:05.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.061+0000 7fa49dffa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4981128d0 0x7fa4981a2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51336/0 (socket says 172.21.15.78:51336) 2024-08-18T23:44:05.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.061+0000 7fa49dffa640 1 -- 172.21.15.78:0/4284007594 learned_addr learned my addr 172.21.15.78:0/4284007594 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:05.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.061+0000 7fa49d7f9640 1 --2- 172.21.15.78:0/4284007594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4980742c0 0x7fa49819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:05.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.061+0000 7fa49cff8640 1 --2- 172.21.15.78:0/4284007594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa498074be0 0x7fa49819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:05.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.061+0000 7fa49dffa640 1 -- 172.21.15.78:0/4284007594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa498074be0 msgr2=0x7fa49819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:05.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.061+0000 7fa49dffa640 1 --2- 172.21.15.78:0/4284007594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa498074be0 0x7fa49819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:05.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.061+0000 7fa49dffa640 1 -- 172.21.15.78:0/4284007594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4980742c0 msgr2=0x7fa49819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:05.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.061+0000 7fa49dffa640 1 --2- 172.21.15.78:0/4284007594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4980742c0 0x7fa49819e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:05.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.061+0000 7fa49dffa640 1 -- 172.21.15.78:0/4284007594 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4981a36c0 con 0x7fa4981128d0 2024-08-18T23:44:05.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.061+0000 7fa49cff8640 1 --2- 172.21.15.78:0/4284007594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa498074be0 0x7fa49819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:05.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.061+0000 7fa49dffa640 1 --2- 172.21.15.78:0/4284007594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4981128d0 0x7fa4981a2fe0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa49002ae70 tx=0x7fa49002f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:05.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.061+0000 7fa48e7fc640 1 -- 172.21.15.78:0/4284007594 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490034040 con 0x7fa4981128d0 2024-08-18T23:44:05.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.061+0000 7fa49fa5d640 1 -- 172.21.15.78:0/4284007594 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4981a3950 con 0x7fa4981128d0 2024-08-18T23:44:05.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.061+0000 7fa48e7fc640 1 -- 172.21.15.78:0/4284007594 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa490030920 con 0x7fa4981128d0 2024-08-18T23:44:05.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.061+0000 7fa48e7fc640 1 -- 172.21.15.78:0/4284007594 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490004780 con 0x7fa4981128d0 2024-08-18T23:44:05.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.061+0000 7fa49fa5d640 1 -- 172.21.15.78:0/4284007594 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4981ab290 con 0x7fa4981128d0 2024-08-18T23:44:05.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.062+0000 7fa49fa5d640 1 -- 172.21.15.78:0/4284007594 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4980746a0 con 0x7fa4981128d0 2024-08-18T23:44:05.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.063+0000 7fa48e7fc640 1 -- 172.21.15.78:0/4284007594 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fa490037020 con 0x7fa4981128d0 2024-08-18T23:44:05.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.064+0000 7fa48e7fc640 1 --2- 172.21.15.78:0/4284007594 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa4840a2c10 0x7fa4840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:05.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.064+0000 7fa49d7f9640 1 --2- 172.21.15.78:0/4284007594 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa4840a2c10 0x7fa4840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:05.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.064+0000 7fa48e7fc640 1 -- 172.21.15.78:0/4284007594 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa4900f03b0 con 0x7fa4981128d0 2024-08-18T23:44:05.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.065+0000 7fa49d7f9640 1 --2- 172.21.15.78:0/4284007594 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa4840a2c10 0x7fa4840a50d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa49400abe0 tx=0x7fa49400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:05.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.067+0000 7fa48e7fc640 1 -- 172.21.15.78:0/4284007594 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa4900bced0 con 0x7fa4981128d0 2024-08-18T23:44:05.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.236+0000 7fa49fa5d640 1 -- 172.21.15.78:0/4284007594 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4981ab4f0 con 0x7fa4981128d0 2024-08-18T23:44:05.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.237+0000 7fa48e7fc640 1 -- 172.21.15.78:0/4284007594 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24747 (secure 0 0 0) 0x7fa4900bc7d0 con 0x7fa4981128d0 2024-08-18T23:44:05.240 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:44:05.240 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":92,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","created":"2024-08-18T23:34:33.867246+0000","modified":"2024-08-18T23:43:24.291494+0000","last_up_change":"2024-08-18T23:40:42.200381+0000","last_in_change":"2024-08-18T23:40:22.425813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T23:37:22.338211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T23:41:28.920802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T23:41:31.026094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T23:41:37.403238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4de1c78-6ebc-49a9-b9c5-85d43535fab7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6803","nonce":3639645311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6805","nonce":3639645311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6809","nonce":3639645311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6807","nonce":3639645311}]},"public_addr":"172.21.15.78:6803/3639645311","cluster_addr":"172.21.15.78:6805/3639645311","heartbeat_back_addr":"172.21.15.78:6809/3639645311","heartbeat_front_addr":"172.21.15.78:6807/3639645311","state":["exists","up"]},{"osd":1,"uuid":"4f8e0c3d-472b-4230-ad23-2b0a34861251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6801","nonce":4199446368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6803","nonce":4199446368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6807","nonce":4199446368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6805","nonce":4199446368}]},"public_addr":"172.21.15.123:6801/4199446368","cluster_addr":"172.21.15.123:6803/4199446368","heartbeat_back_addr":"172.21.15.123:6807/4199446368","heartbeat_front_addr":"172.21.15.123:6805/4199446368","state":["exists","up"]},{"osd":2,"uuid":"27ef3314-bb6f-413d-be05-e10a105dec9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6801","nonce":2582810057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6803","nonce":2582810057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6807","nonce":2582810057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6805","nonce":2582810057}]},"public_addr":"172.21.15.161:6801/2582810057","cluster_addr":"172.21.15.161:6803/2582810057","heartbeat_back_addr":"172.21.15.161:6807/2582810057","heartbeat_front_addr":"172.21.15.161:6805/2582810057","state":["exists","up"]},{"osd":3,"uuid":"e9e52662-5ef9-4d70-bdd7-2b92165822ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6811","nonce":2697347521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6813","nonce":2697347521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6817","nonce":2697347521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6815","nonce":2697347521}]},"public_addr":"172.21.15.78:6811/2697347521","cluster_addr":"172.21.15.78:6813/2697347521","heartbeat_back_addr":"172.21.15.78:6817/2697347521","heartbeat_front_addr":"172.21.15.78:6815/2697347521","state":["exists","up"]},{"osd":4,"uuid":"5ed26775-79e3-4b24-9557-69fcf5e8e824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6809","nonce":1281511569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6811","nonce":1281511569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6815","nonce":1281511569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6813","nonce":1281511569}]},"public_addr":"172.21.15.123:6809/1281511569","cluster_addr":"172.21.15.123:6811/1281511569","heartbeat_back_addr":"172.21.15.123:6815/1281511569","heartbeat_front_addr":"172.21.15.123:6813/1281511569","state":["exists","up"]},{"osd":5,"uuid":"a9793b1d-5613-48a4-938e-adb718f4fa4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6809","nonce":2751796117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6811","nonce":2751796117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6815","nonce":2751796117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6813","nonce":2751796117}]},"public_addr":"172.21.15.161:6809/2751796117","cluster_addr":"172.21.15.161:6811/2751796117","heartbeat_back_addr":"172.21.15.161:6815/2751796117","heartbeat_front_addr":"172.21.15.161:6813/2751796117","state":["exists","up"]},{"osd":6,"uuid":"5dbc5e9c-a14f-4b71-910a-1e6d4827bb32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6819","nonce":95383223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6821","nonce":95383223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6825","nonce":95383223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6823","nonce":95383223}]},"public_addr":"172.21.15.78:6819/95383223","cluster_addr":"172.21.15.78:6821/95383223","heartbeat_back_addr":"172.21.15.78:6825/95383223","heartbeat_front_addr":"172.21.15.78:6823/95383223","state":["exists","up"]},{"osd":7,"uuid":"26fb7b69-131f-42ee-b745-6fe8b4f65391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6817","nonce":3440305156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6819","nonce":3440305156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6823","nonce":3440305156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6821","nonce":3440305156}]},"public_addr":"172.21.15.123:6817/3440305156","cluster_addr":"172.21.15.123:6819/3440305156","heartbeat_back_addr":"172.21.15.123:6823/3440305156","heartbeat_front_addr":"172.21.15.123:6821/3440305156","state":["exists","up"]},{"osd":8,"uuid":"e267bc50-0c29-4d7b-b368-b1621e29b2df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6817","nonce":1485008778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6819","nonce":1485008778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6823","nonce":1485008778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6821","nonce":1485008778}]},"public_addr":"172.21.15.161:6817/1485008778","cluster_addr":"172.21.15.161:6819/1485008778","heartbeat_back_addr":"172.21.15.161:6823/1485008778","heartbeat_front_addr":"172.21.15.161:6821/1485008778","state":["exists","up"]},{"osd":9,"uuid":"b7654818-b767-4fe6-8211-346297895f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6827","nonce":2884475661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6829","nonce":2884475661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6833","nonce":2884475661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6831","nonce":2884475661}]},"public_addr":"172.21.15.78:6827/2884475661","cluster_addr":"172.21.15.78:6829/2884475661","heartbeat_back_addr":"172.21.15.78:6833/2884475661","heartbeat_front_addr":"172.21.15.78:6831/2884475661","state":["exists","up"]},{"osd":10,"uuid":"59aa32ff-510e-4b52-8170-08a34a0b6b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6825","nonce":122226286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6827","nonce":122226286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6831","nonce":122226286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6829","nonce":122226286}]},"public_addr":"172.21.15.123:6825/122226286","cluster_addr":"172.21.15.123:6827/122226286","heartbeat_back_addr":"172.21.15.123:6831/122226286","heartbeat_front_addr":"172.21.15.123:6829/122226286","state":["exists","up"]},{"osd":11,"uuid":"67bacab4-d078-4c5c-b4ff-9f6a8a00f507","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6825","nonce":2037560985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6827","nonce":2037560985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6831","nonce":2037560985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6829","nonce":2037560985}]},"public_addr":"172.21.15.161:6825/2037560985","cluster_addr":"172.21.15.161:6827/2037560985","heartbeat_back_addr":"172.21.15.161:6831/2037560985","heartbeat_front_addr":"172.21.15.161:6829/2037560985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:36.826044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:57.688421+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:18.297424+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:40.355215+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:02.167902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:23.968759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:46.254083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:08.522038+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:31.028119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:54.119807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:16.847034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:39.607886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6800/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3040499240":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6801/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:6800/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/757409165":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/2562072965":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/636184437":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/1570708294":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3137806628":"2024-08-19T23:34:59.021165+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T23:44:05.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.239+0000 7fa49fa5d640 1 -- 172.21.15.78:0/4284007594 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa4840a2c10 msgr2=0x7fa4840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.239+0000 7fa49fa5d640 1 --2- 172.21.15.78:0/4284007594 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa4840a2c10 0x7fa4840a50d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa49400abe0 tx=0x7fa49400a2d0 comp rx=0 tx=0).stop 2024-08-18T23:44:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.239+0000 7fa49fa5d640 1 -- 172.21.15.78:0/4284007594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4981128d0 msgr2=0x7fa4981a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.239+0000 7fa49fa5d640 1 --2- 172.21.15.78:0/4284007594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4981128d0 0x7fa4981a2fe0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa49002ae70 tx=0x7fa49002f5b0 comp rx=0 tx=0).stop 2024-08-18T23:44:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.239+0000 7fa49fa5d640 1 -- 172.21.15.78:0/4284007594 shutdown_connections 2024-08-18T23:44:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.239+0000 7fa49fa5d640 1 --2- 172.21.15.78:0/4284007594 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa4840a2c10 0x7fa4840a50d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.239+0000 7fa49fa5d640 1 --2- 172.21.15.78:0/4284007594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4981128d0 0x7fa4981a2fe0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.239+0000 7fa49fa5d640 1 --2- 172.21.15.78:0/4284007594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa498074be0 0x7fa49819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.239+0000 7fa49fa5d640 1 --2- 172.21.15.78:0/4284007594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4980742c0 0x7fa49819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.240+0000 7fa49fa5d640 1 -- 172.21.15.78:0/4284007594 >> 172.21.15.78:0/4284007594 conn(0x7fa4980fd5e0 msgr2=0x7fa4980febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.240+0000 7fa49fa5d640 1 -- 172.21.15.78:0/4284007594 shutdown_connections 2024-08-18T23:44:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:05.240+0000 7fa49fa5d640 1 -- 172.21.15.78:0/4284007594 wait complete. 2024-08-18T23:44:05.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:05 smithi078 ceph-mon[18440]: pgmap v343: 129 pgs: 129 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-08-18T23:44:05.517 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:05 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/4284007594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:44:05.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:05 smithi123 ceph-mon[21808]: pgmap v343: 129 pgs: 129 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-08-18T23:44:05.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:05 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/4284007594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:44:05.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:05 smithi161 ceph-mon[22027]: pgmap v343: 129 pgs: 129 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-08-18T23:44:05.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:05 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/4284007594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:44:05.958 DEBUG:tasks.fwd_scrub:Yielding 2024-08-18T23:44:05.958 INFO:teuthology.run_tasks:Running task quiescer... 2024-08-18T23:44:05.972 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-18T23:44:05.973 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-18T23:44:05.973 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-08-18T23:44:05.973 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-18T23:44:05.973 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-18T23:44:06.240 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:06.245 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:06.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.768+0000 7f9f0736b640 1 -- 172.21.15.78:0/1161970440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f001068b0 msgr2=0x7f9f00106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:06.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.768+0000 7f9f0736b640 1 --2- 172.21.15.78:0/1161970440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f001068b0 0x7f9f00106c90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009ca0 tx=0x7f9ef402a8c0 comp rx=0 tx=0).stop 2024-08-18T23:44:06.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.768+0000 7f9f0736b640 1 -- 172.21.15.78:0/1161970440 shutdown_connections 2024-08-18T23:44:06.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.768+0000 7f9f0736b640 1 --2- 172.21.15.78:0/1161970440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f00100780 0x7f9f0010e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:06.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.768+0000 7f9f0736b640 1 --2- 172.21.15.78:0/1161970440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f000ffde0 0x7f9f00100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:06.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.768+0000 7f9f0736b640 1 --2- 172.21.15.78:0/1161970440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f001068b0 0x7f9f00106c90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:06.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.768+0000 7f9f0736b640 1 -- 172.21.15.78:0/1161970440 >> 172.21.15.78:0/1161970440 conn(0x7f9f000fbc10 msgr2=0x7f9f000fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:06.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.768+0000 7f9f0736b640 1 -- 172.21.15.78:0/1161970440 shutdown_connections 2024-08-18T23:44:06.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.768+0000 7f9f0736b640 1 -- 172.21.15.78:0/1161970440 wait complete. 2024-08-18T23:44:06.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.768+0000 7f9f0736b640 1 Processor -- start 2024-08-18T23:44:06.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.768+0000 7f9f0736b640 1 -- start start 2024-08-18T23:44:06.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f0736b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f000ffde0 0x7f9f001981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:06.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f0736b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f00100780 0x7f9f00198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:06.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f0736b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f001068b0 0x7f9f0019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:06.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f0736b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f00111da0 con 0x7f9f000ffde0 2024-08-18T23:44:06.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f0736b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9f00111c20 con 0x7f9f00100780 2024-08-18T23:44:06.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f0736b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9f00111f20 con 0x7f9f001068b0 2024-08-18T23:44:06.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f05107640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f000ffde0 0x7f9f001981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:06.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f05107640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f000ffde0 0x7f9f001981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35734/0 (socket says 172.21.15.78:35734) 2024-08-18T23:44:06.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f05107640 1 -- 172.21.15.78:0/255185525 learned_addr learned my addr 172.21.15.78:0/255185525 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:06.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f05908640 1 --2- 172.21.15.78:0/255185525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f001068b0 0x7f9f0019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:06.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f04906640 1 --2- 172.21.15.78:0/255185525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f00100780 0x7f9f00198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:06.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f05107640 1 -- 172.21.15.78:0/255185525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f001068b0 msgr2=0x7f9f0019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:06.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f05107640 1 --2- 172.21.15.78:0/255185525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f001068b0 0x7f9f0019ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:06.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f05107640 1 -- 172.21.15.78:0/255185525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f00100780 msgr2=0x7f9f00198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:06.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f05107640 1 --2- 172.21.15.78:0/255185525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f00100780 0x7f9f00198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:06.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f05107640 1 -- 172.21.15.78:0/255185525 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0019d170 con 0x7f9f000ffde0 2024-08-18T23:44:06.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f05908640 1 --2- 172.21.15.78:0/255185525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f001068b0 0x7f9f0019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:06.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f04906640 1 --2- 172.21.15.78:0/255185525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f00100780 0x7f9f00198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:06.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f05107640 1 --2- 172.21.15.78:0/255185525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f000ffde0 0x7f9f001981c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4006010 tx=0x7f9ef4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:06.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9efe7fc640 1 -- 172.21.15.78:0/255185525 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4031650 con 0x7f9f000ffde0 2024-08-18T23:44:06.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9f0736b640 1 -- 172.21.15.78:0/255185525 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f0019d400 con 0x7f9f000ffde0 2024-08-18T23:44:06.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9efe7fc640 1 -- 172.21.15.78:0/255185525 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ef40317f0 con 0x7f9f000ffde0 2024-08-18T23:44:06.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.769+0000 7f9efe7fc640 1 -- 172.21.15.78:0/255185525 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4030bb0 con 0x7f9f000ffde0 2024-08-18T23:44:06.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.770+0000 7f9f0736b640 1 -- 172.21.15.78:0/255185525 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f001a4d40 con 0x7f9f000ffde0 2024-08-18T23:44:06.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.771+0000 7f9f0736b640 1 -- 172.21.15.78:0/255185525 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f00100240 con 0x7f9f000ffde0 2024-08-18T23:44:06.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.772+0000 7f9efe7fc640 1 -- 172.21.15.78:0/255185525 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f9ef4004690 con 0x7f9f000ffde0 2024-08-18T23:44:06.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.772+0000 7f9efe7fc640 1 --2- 172.21.15.78:0/255185525 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9ee80a2c10 0x7f9ee80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:06.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.772+0000 7f9f04906640 1 --2- 172.21.15.78:0/255185525 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9ee80a2c10 0x7f9ee80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:06.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.772+0000 7f9efe7fc640 1 -- 172.21.15.78:0/255185525 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f9ef40f0b70 con 0x7f9f000ffde0 2024-08-18T23:44:06.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.773+0000 7f9f04906640 1 --2- 172.21.15.78:0/255185525 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9ee80a2c10 0x7f9ee80a50d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00058c0 tx=0x7f9ef0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:06.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.775+0000 7f9efe7fc640 1 -- 172.21.15.78:0/255185525 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9ef40bd780 con 0x7f9f000ffde0 2024-08-18T23:44:06.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.940+0000 7f9f0736b640 1 -- 172.21.15.78:0/255185525 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9f00062360 con 0x7f9ee80a2c10 2024-08-18T23:44:06.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.949+0000 7f9efe7fc640 1 -- 172.21.15.78:0/255185525 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340487 (secure 0 0 0) 0x7f9f00062360 con 0x7f9ee80a2c10 2024-08-18T23:44:06.954 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:44:06.956 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-08-18T23:44:06.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.952+0000 7f9f0736b640 1 -- 172.21.15.78:0/255185525 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9ee80a2c10 msgr2=0x7f9ee80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:06.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.952+0000 7f9f0736b640 1 --2- 172.21.15.78:0/255185525 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9ee80a2c10 0x7f9ee80a50d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00058c0 tx=0x7f9ef0005850 comp rx=0 tx=0).stop 2024-08-18T23:44:06.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.952+0000 7f9f0736b640 1 -- 172.21.15.78:0/255185525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f000ffde0 msgr2=0x7f9f001981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:06.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.952+0000 7f9f0736b640 1 --2- 172.21.15.78:0/255185525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f000ffde0 0x7f9f001981c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4006010 tx=0x7f9ef4003f10 comp rx=0 tx=0).stop 2024-08-18T23:44:06.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.952+0000 7f9f0736b640 1 -- 172.21.15.78:0/255185525 shutdown_connections 2024-08-18T23:44:06.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.952+0000 7f9f0736b640 1 --2- 172.21.15.78:0/255185525 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9ee80a2c10 0x7f9ee80a50d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:06.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.952+0000 7f9f0736b640 1 --2- 172.21.15.78:0/255185525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f001068b0 0x7f9f0019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:06.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.952+0000 7f9f0736b640 1 --2- 172.21.15.78:0/255185525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f00100780 0x7f9f00198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:06.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.952+0000 7f9f0736b640 1 --2- 172.21.15.78:0/255185525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f000ffde0 0x7f9f001981c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:06.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.952+0000 7f9f0736b640 1 -- 172.21.15.78:0/255185525 >> 172.21.15.78:0/255185525 conn(0x7f9f000fbc10 msgr2=0x7f9f000fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:06.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.952+0000 7f9f0736b640 1 -- 172.21.15.78:0/255185525 shutdown_connections 2024-08-18T23:44:06.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:06.952+0000 7f9f0736b640 1 -- 172.21.15.78:0/255185525 wait complete. 2024-08-18T23:44:07.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:07 smithi078 ceph-mon[18440]: pgmap v344: 129 pgs: 129 active+clean; 651 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-08-18T23:44:07.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:07 smithi123 ceph-mon[21808]: pgmap v344: 129 pgs: 129 active+clean; 651 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-08-18T23:44:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:07 smithi161 ceph-mon[22027]: pgmap v344: 129 pgs: 129 active+clean; 651 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-08-18T23:44:07.853 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-08-18T23:44:06.148310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666749,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":860,"num_read_kb":558,"num_write":376,"num_write_kb":3083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":803,"ondisk_log_size":803,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392744,"kb_used_data":14452,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549080,"statfs":{"total":1151940427776,"available":1151538257920,"internally_reserved":0,"allocated":14798848,"data_stored":8259113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":56,"num_write":17,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003158"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298623+0000","last_change":"2024-08-18T23:41:30.120580+0000","last_active":"2024-08-18T23:43:24.298623+0000","last_peered":"2024-08-18T23:43:24.298623+0000","last_clean":"2024-08-18T23:43:24.298623+0000","last_became_active":"2024-08-18T23:41:30.120266+0000","last_became_peered":"2024-08-18T23:41:30.120266+0000","last_unstale":"2024-08-18T23:43:24.298623+0000","last_undegraded":"2024-08-18T23:43:24.298623+0000","last_fullsized":"2024-08-18T23:43:24.298623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:06:40.997070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312511+0000","last_change":"2024-08-18T23:41:32.143232+0000","last_active":"2024-08-18T23:43:24.312511+0000","last_peered":"2024-08-18T23:43:24.312511+0000","last_clean":"2024-08-18T23:43:24.312511+0000","last_became_active":"2024-08-18T23:41:32.142916+0000","last_became_peered":"2024-08-18T23:41:32.142916+0000","last_unstale":"2024-08-18T23:43:24.312511+0000","last_undegraded":"2024-08-18T23:43:24.312511+0000","last_fullsized":"2024-08-18T23:43:24.312511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:50:54.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312049+0000","last_change":"2024-08-18T23:41:39.188772+0000","last_active":"2024-08-18T23:43:24.312049+0000","last_peered":"2024-08-18T23:43:24.312049+0000","last_clean":"2024-08-18T23:43:24.312049+0000","last_became_active":"2024-08-18T23:41:39.188362+0000","last_became_peered":"2024-08-18T23:41:39.188362+0000","last_unstale":"2024-08-18T23:43:24.312049+0000","last_undegraded":"2024-08-18T23:43:24.312049+0000","last_fullsized":"2024-08-18T23:43:24.312049+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:47:50.255543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312628+0000","last_change":"2024-08-18T23:41:30.137179+0000","last_active":"2024-08-18T23:43:24.312628+0000","last_peered":"2024-08-18T23:43:24.312628+0000","last_clean":"2024-08-18T23:43:24.312628+0000","last_became_active":"2024-08-18T23:41:30.136618+0000","last_became_peered":"2024-08-18T23:41:30.136618+0000","last_unstale":"2024-08-18T23:43:24.312628+0000","last_undegraded":"2024-08-18T23:43:24.312628+0000","last_fullsized":"2024-08-18T23:43:24.312628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:59.826364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312531+0000","last_change":"2024-08-18T23:41:32.142075+0000","last_active":"2024-08-18T23:43:24.312531+0000","last_peered":"2024-08-18T23:43:24.312531+0000","last_clean":"2024-08-18T23:43:24.312531+0000","last_became_active":"2024-08-18T23:41:32.141709+0000","last_became_peered":"2024-08-18T23:41:32.141709+0000","last_unstale":"2024-08-18T23:43:24.312531+0000","last_undegraded":"2024-08-18T23:43:24.312531+0000","last_fullsized":"2024-08-18T23:43:24.312531+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:08.324627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312210+0000","last_change":"2024-08-18T23:41:39.184575+0000","last_active":"2024-08-18T23:43:24.312210+0000","last_peered":"2024-08-18T23:43:24.312210+0000","last_clean":"2024-08-18T23:43:24.312210+0000","last_became_active":"2024-08-18T23:41:39.184169+0000","last_became_peered":"2024-08-18T23:41:39.184169+0000","last_unstale":"2024-08-18T23:43:24.312210+0000","last_undegraded":"2024-08-18T23:43:24.312210+0000","last_fullsized":"2024-08-18T23:43:24.312210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:32.218172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304127+0000","last_change":"2024-08-18T23:41:30.121638+0000","last_active":"2024-08-18T23:43:24.304127+0000","last_peered":"2024-08-18T23:43:24.304127+0000","last_clean":"2024-08-18T23:43:24.304127+0000","last_became_active":"2024-08-18T23:41:30.121290+0000","last_became_peered":"2024-08-18T23:41:30.121290+0000","last_unstale":"2024-08-18T23:43:24.304127+0000","last_undegraded":"2024-08-18T23:43:24.304127+0000","last_fullsized":"2024-08-18T23:43:24.304127+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:43:55.638780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312431+0000","last_change":"2024-08-18T23:41:32.142177+0000","last_active":"2024-08-18T23:43:24.312431+0000","last_peered":"2024-08-18T23:43:24.312431+0000","last_clean":"2024-08-18T23:43:24.312431+0000","last_became_active":"2024-08-18T23:41:32.141731+0000","last_became_peered":"2024-08-18T23:41:32.141731+0000","last_unstale":"2024-08-18T23:43:24.312431+0000","last_undegraded":"2024-08-18T23:43:24.312431+0000","last_fullsized":"2024-08-18T23:43:24.312431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:36.283299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312604+0000","last_change":"2024-08-18T23:41:39.182931+0000","last_active":"2024-08-18T23:43:24.312604+0000","last_peered":"2024-08-18T23:43:24.312604+0000","last_clean":"2024-08-18T23:43:24.312604+0000","last_became_active":"2024-08-18T23:41:39.182008+0000","last_became_peered":"2024-08-18T23:41:39.182008+0000","last_unstale":"2024-08-18T23:43:24.312604+0000","last_undegraded":"2024-08-18T23:43:24.312604+0000","last_fullsized":"2024-08-18T23:43:24.312604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:58:38.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312863+0000","last_change":"2024-08-18T23:42:23.329762+0000","last_active":"2024-08-18T23:43:24.312863+0000","last_peered":"2024-08-18T23:43:24.312863+0000","last_clean":"2024-08-18T23:43:24.312863+0000","last_became_active":"2024-08-18T23:42:23.325753+0000","last_became_peered":"2024-08-18T23:42:23.325753+0000","last_unstale":"2024-08-18T23:43:24.312863+0000","last_undegraded":"2024-08-18T23:43:24.312863+0000","last_fullsized":"2024-08-18T23:43:24.312863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:19:43.062492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307318+0000","last_change":"2024-08-18T23:41:32.148662+0000","last_active":"2024-08-18T23:43:24.307318+0000","last_peered":"2024-08-18T23:43:24.307318+0000","last_clean":"2024-08-18T23:43:24.307318+0000","last_became_active":"2024-08-18T23:41:32.148318+0000","last_became_peered":"2024-08-18T23:41:32.148318+0000","last_unstale":"2024-08-18T23:43:24.307318+0000","last_undegraded":"2024-08-18T23:43:24.307318+0000","last_fullsized":"2024-08-18T23:43:24.307318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:58.668669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312238+0000","last_change":"2024-08-18T23:41:39.181114+0000","last_active":"2024-08-18T23:43:24.312238+0000","last_peered":"2024-08-18T23:43:24.312238+0000","last_clean":"2024-08-18T23:43:24.312238+0000","last_became_active":"2024-08-18T23:41:39.180599+0000","last_became_peered":"2024-08-18T23:41:39.180599+0000","last_unstale":"2024-08-18T23:43:24.312238+0000","last_undegraded":"2024-08-18T23:43:24.312238+0000","last_fullsized":"2024-08-18T23:43:24.312238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:36:08.749143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304715+0000","last_change":"2024-08-18T23:41:32.144326+0000","last_active":"2024-08-18T23:43:24.304715+0000","last_peered":"2024-08-18T23:43:24.304715+0000","last_clean":"2024-08-18T23:43:24.304715+0000","last_became_active":"2024-08-18T23:41:32.144004+0000","last_became_peered":"2024-08-18T23:41:32.144004+0000","last_unstale":"2024-08-18T23:43:24.304715+0000","last_undegraded":"2024-08-18T23:43:24.304715+0000","last_fullsized":"2024-08-18T23:43:24.304715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:10.918624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303665+0000","last_change":"2024-08-18T23:41:30.125160+0000","last_active":"2024-08-18T23:43:24.303665+0000","last_peered":"2024-08-18T23:43:24.303665+0000","last_clean":"2024-08-18T23:43:24.303665+0000","last_became_active":"2024-08-18T23:41:30.124685+0000","last_became_peered":"2024-08-18T23:41:30.124685+0000","last_unstale":"2024-08-18T23:43:24.303665+0000","last_undegraded":"2024-08-18T23:43:24.303665+0000","last_fullsized":"2024-08-18T23:43:24.303665+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:52:48.997902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313216+0000","last_change":"2024-08-18T23:41:39.681481+0000","last_active":"2024-08-18T23:43:24.313216+0000","last_peered":"2024-08-18T23:43:24.313216+0000","last_clean":"2024-08-18T23:43:24.313216+0000","last_became_active":"2024-08-18T23:41:39.680204+0000","last_became_peered":"2024-08-18T23:41:39.680204+0000","last_unstale":"2024-08-18T23:43:24.313216+0000","last_undegraded":"2024-08-18T23:43:24.313216+0000","last_fullsized":"2024-08-18T23:43:24.313216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:00:05.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312231+0000","last_change":"2024-08-18T23:41:32.151335+0000","last_active":"2024-08-18T23:43:24.312231+0000","last_peered":"2024-08-18T23:43:24.312231+0000","last_clean":"2024-08-18T23:43:24.312231+0000","last_became_active":"2024-08-18T23:41:32.151040+0000","last_became_peered":"2024-08-18T23:41:32.151040+0000","last_unstale":"2024-08-18T23:43:24.312231+0000","last_undegraded":"2024-08-18T23:43:24.312231+0000","last_fullsized":"2024-08-18T23:43:24.312231+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:16.108555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"92'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:41.164918+0000","last_change":"2024-08-18T23:41:30.122301+0000","last_active":"2024-08-18T23:43:41.164918+0000","last_peered":"2024-08-18T23:43:41.164918+0000","last_clean":"2024-08-18T23:43:41.164918+0000","last_became_active":"2024-08-18T23:41:30.121117+0000","last_became_peered":"2024-08-18T23:41:30.121117+0000","last_unstale":"2024-08-18T23:43:41.164918+0000","last_undegraded":"2024-08-18T23:43:41.164918+0000","last_fullsized":"2024-08-18T23:43:41.164918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:40:12.678803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312721+0000","last_change":"2024-08-18T23:41:39.182395+0000","last_active":"2024-08-18T23:43:24.312721+0000","last_peered":"2024-08-18T23:43:24.312721+0000","last_clean":"2024-08-18T23:43:24.312721+0000","last_became_active":"2024-08-18T23:41:39.181979+0000","last_became_peered":"2024-08-18T23:41:39.181979+0000","last_unstale":"2024-08-18T23:43:24.312721+0000","last_undegraded":"2024-08-18T23:43:24.312721+0000","last_fullsized":"2024-08-18T23:43:24.312721+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:04.674503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312810+0000","last_change":"2024-08-18T23:41:32.140464+0000","last_active":"2024-08-18T23:43:24.312810+0000","last_peered":"2024-08-18T23:43:24.312810+0000","last_clean":"2024-08-18T23:43:24.312810+0000","last_became_active":"2024-08-18T23:41:32.139153+0000","last_became_peered":"2024-08-18T23:41:32.139153+0000","last_unstale":"2024-08-18T23:43:24.312810+0000","last_undegraded":"2024-08-18T23:43:24.312810+0000","last_fullsized":"2024-08-18T23:43:24.312810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:27.387680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312414+0000","last_change":"2024-08-18T23:41:30.122787+0000","last_active":"2024-08-18T23:43:24.312414+0000","last_peered":"2024-08-18T23:43:24.312414+0000","last_clean":"2024-08-18T23:43:24.312414+0000","last_became_active":"2024-08-18T23:41:30.122462+0000","last_became_peered":"2024-08-18T23:41:30.122462+0000","last_unstale":"2024-08-18T23:43:24.312414+0000","last_undegraded":"2024-08-18T23:43:24.312414+0000","last_fullsized":"2024-08-18T23:43:24.312414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:55.568831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298325+0000","last_change":"2024-08-18T23:41:39.190511+0000","last_active":"2024-08-18T23:43:24.298325+0000","last_peered":"2024-08-18T23:43:24.298325+0000","last_clean":"2024-08-18T23:43:24.298325+0000","last_became_active":"2024-08-18T23:41:39.190154+0000","last_became_peered":"2024-08-18T23:41:39.190154+0000","last_unstale":"2024-08-18T23:43:24.298325+0000","last_undegraded":"2024-08-18T23:43:24.298325+0000","last_fullsized":"2024-08-18T23:43:24.298325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:33:34.514827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303456+0000","last_change":"2024-08-18T23:41:32.143867+0000","last_active":"2024-08-18T23:43:24.303456+0000","last_peered":"2024-08-18T23:43:24.303456+0000","last_clean":"2024-08-18T23:43:24.303456+0000","last_became_active":"2024-08-18T23:41:32.143527+0000","last_became_peered":"2024-08-18T23:41:32.143527+0000","last_unstale":"2024-08-18T23:43:24.303456+0000","last_undegraded":"2024-08-18T23:43:24.303456+0000","last_fullsized":"2024-08-18T23:43:24.303456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:40:25.902133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312100+0000","last_change":"2024-08-18T23:42:23.325102+0000","last_active":"2024-08-18T23:43:24.312100+0000","last_peered":"2024-08-18T23:43:24.312100+0000","last_clean":"2024-08-18T23:43:24.312100+0000","last_became_active":"2024-08-18T23:42:23.321583+0000","last_became_peered":"2024-08-18T23:42:23.321583+0000","last_unstale":"2024-08-18T23:43:24.312100+0000","last_undegraded":"2024-08-18T23:43:24.312100+0000","last_fullsized":"2024-08-18T23:43:24.312100+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:57:42.546686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312767+0000","last_change":"2024-08-18T23:42:23.559756+0000","last_active":"2024-08-18T23:43:24.312767+0000","last_peered":"2024-08-18T23:43:24.312767+0000","last_clean":"2024-08-18T23:43:24.312767+0000","last_became_active":"2024-08-18T23:42:23.559242+0000","last_became_peered":"2024-08-18T23:42:23.559242+0000","last_unstale":"2024-08-18T23:43:24.312767+0000","last_undegraded":"2024-08-18T23:43:24.312767+0000","last_fullsized":"2024-08-18T23:43:24.312767+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:25:40.790313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312043+0000","last_change":"2024-08-18T23:41:32.139025+0000","last_active":"2024-08-18T23:43:24.312043+0000","last_peered":"2024-08-18T23:43:24.312043+0000","last_clean":"2024-08-18T23:43:24.312043+0000","last_became_active":"2024-08-18T23:41:32.138733+0000","last_became_peered":"2024-08-18T23:41:32.138733+0000","last_unstale":"2024-08-18T23:43:24.312043+0000","last_undegraded":"2024-08-18T23:43:24.312043+0000","last_fullsized":"2024-08-18T23:43:24.312043+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:01:39.826391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313061+0000","last_change":"2024-08-18T23:41:30.477359+0000","last_active":"2024-08-18T23:43:24.313061+0000","last_peered":"2024-08-18T23:43:24.313061+0000","last_clean":"2024-08-18T23:43:24.313061+0000","last_became_active":"2024-08-18T23:41:30.477011+0000","last_became_peered":"2024-08-18T23:41:30.477011+0000","last_unstale":"2024-08-18T23:43:24.313061+0000","last_undegraded":"2024-08-18T23:43:24.313061+0000","last_fullsized":"2024-08-18T23:43:24.313061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:53:10.052859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312145+0000","last_change":"2024-08-18T23:41:39.182140+0000","last_active":"2024-08-18T23:43:24.312145+0000","last_peered":"2024-08-18T23:43:24.312145+0000","last_clean":"2024-08-18T23:43:24.312145+0000","last_became_active":"2024-08-18T23:41:39.181667+0000","last_became_peered":"2024-08-18T23:41:39.181667+0000","last_unstale":"2024-08-18T23:43:24.312145+0000","last_undegraded":"2024-08-18T23:43:24.312145+0000","last_fullsized":"2024-08-18T23:43:24.312145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:02:18.128028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312594+0000","last_change":"2024-08-18T23:41:32.137064+0000","last_active":"2024-08-18T23:43:24.312594+0000","last_peered":"2024-08-18T23:43:24.312594+0000","last_clean":"2024-08-18T23:43:24.312594+0000","last_became_active":"2024-08-18T23:41:32.136738+0000","last_became_peered":"2024-08-18T23:41:32.136738+0000","last_unstale":"2024-08-18T23:43:24.312594+0000","last_undegraded":"2024-08-18T23:43:24.312594+0000","last_fullsized":"2024-08-18T23:43:24.312594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:15:11.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'14","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:57.167799+0000","last_change":"2024-08-18T23:41:30.120042+0000","last_active":"2024-08-18T23:43:57.167799+0000","last_peered":"2024-08-18T23:43:57.167799+0000","last_clean":"2024-08-18T23:43:57.167799+0000","last_became_active":"2024-08-18T23:41:30.119719+0000","last_became_peered":"2024-08-18T23:41:30.119719+0000","last_unstale":"2024-08-18T23:43:57.167799+0000","last_undegraded":"2024-08-18T23:43:57.167799+0000","last_fullsized":"2024-08-18T23:43:57.167799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:35:44.466330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311900+0000","last_change":"2024-08-18T23:41:39.680405+0000","last_active":"2024-08-18T23:43:24.311900+0000","last_peered":"2024-08-18T23:43:24.311900+0000","last_clean":"2024-08-18T23:43:24.311900+0000","last_became_active":"2024-08-18T23:41:39.679940+0000","last_became_peered":"2024-08-18T23:41:39.679940+0000","last_unstale":"2024-08-18T23:43:24.311900+0000","last_undegraded":"2024-08-18T23:43:24.311900+0000","last_fullsized":"2024-08-18T23:43:24.311900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:15:03.942116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301271+0000","last_change":"2024-08-18T23:41:32.139688+0000","last_active":"2024-08-18T23:43:24.301271+0000","last_peered":"2024-08-18T23:43:24.301271+0000","last_clean":"2024-08-18T23:43:24.301271+0000","last_became_active":"2024-08-18T23:41:32.139397+0000","last_became_peered":"2024-08-18T23:41:32.139397+0000","last_unstale":"2024-08-18T23:43:24.301271+0000","last_undegraded":"2024-08-18T23:43:24.301271+0000","last_fullsized":"2024-08-18T23:43:24.301271+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:03:57.017488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304150+0000","last_change":"2024-08-18T23:41:30.137578+0000","last_active":"2024-08-18T23:43:24.304150+0000","last_peered":"2024-08-18T23:43:24.304150+0000","last_clean":"2024-08-18T23:43:24.304150+0000","last_became_active":"2024-08-18T23:41:30.137246+0000","last_became_peered":"2024-08-18T23:41:30.137246+0000","last_unstale":"2024-08-18T23:43:24.304150+0000","last_undegraded":"2024-08-18T23:43:24.304150+0000","last_fullsized":"2024-08-18T23:43:24.304150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:48:54.839432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312576+0000","last_change":"2024-08-18T23:41:39.186360+0000","last_active":"2024-08-18T23:43:24.312576+0000","last_peered":"2024-08-18T23:43:24.312576+0000","last_clean":"2024-08-18T23:43:24.312576+0000","last_became_active":"2024-08-18T23:41:39.185808+0000","last_became_peered":"2024-08-18T23:41:39.185808+0000","last_unstale":"2024-08-18T23:43:24.312576+0000","last_undegraded":"2024-08-18T23:43:24.312576+0000","last_fullsized":"2024-08-18T23:43:24.312576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:53.891061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300916+0000","last_change":"2024-08-18T23:41:32.146070+0000","last_active":"2024-08-18T23:43:24.300916+0000","last_peered":"2024-08-18T23:43:24.300916+0000","last_clean":"2024-08-18T23:43:24.300916+0000","last_became_active":"2024-08-18T23:41:32.145640+0000","last_became_peered":"2024-08-18T23:41:32.145640+0000","last_unstale":"2024-08-18T23:43:24.300916+0000","last_undegraded":"2024-08-18T23:43:24.300916+0000","last_fullsized":"2024-08-18T23:43:24.300916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:56:21.368762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298278+0000","last_change":"2024-08-18T23:41:30.138565+0000","last_active":"2024-08-18T23:43:24.298278+0000","last_peered":"2024-08-18T23:43:24.298278+0000","last_clean":"2024-08-18T23:43:24.298278+0000","last_became_active":"2024-08-18T23:41:30.138210+0000","last_became_peered":"2024-08-18T23:41:30.138210+0000","last_unstale":"2024-08-18T23:43:24.298278+0000","last_undegraded":"2024-08-18T23:43:24.298278+0000","last_fullsized":"2024-08-18T23:43:24.298278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:42:14.928884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303425+0000","last_change":"2024-08-18T23:41:39.680656+0000","last_active":"2024-08-18T23:43:24.303425+0000","last_peered":"2024-08-18T23:43:24.303425+0000","last_clean":"2024-08-18T23:43:24.303425+0000","last_became_active":"2024-08-18T23:41:39.680203+0000","last_became_peered":"2024-08-18T23:41:39.680203+0000","last_unstale":"2024-08-18T23:43:24.303425+0000","last_undegraded":"2024-08-18T23:43:24.303425+0000","last_fullsized":"2024-08-18T23:43:24.303425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:55:43.138869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300932+0000","last_change":"2024-08-18T23:41:32.130222+0000","last_active":"2024-08-18T23:43:24.300932+0000","last_peered":"2024-08-18T23:43:24.300932+0000","last_clean":"2024-08-18T23:43:24.300932+0000","last_became_active":"2024-08-18T23:41:32.129844+0000","last_became_peered":"2024-08-18T23:41:32.129844+0000","last_unstale":"2024-08-18T23:43:24.300932+0000","last_undegraded":"2024-08-18T23:43:24.300932+0000","last_fullsized":"2024-08-18T23:43:24.300932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:39:21.106343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304850+0000","last_change":"2024-08-18T23:41:30.123142+0000","last_active":"2024-08-18T23:43:24.304850+0000","last_peered":"2024-08-18T23:43:24.304850+0000","last_clean":"2024-08-18T23:43:24.304850+0000","last_became_active":"2024-08-18T23:41:30.121645+0000","last_became_peered":"2024-08-18T23:41:30.121645+0000","last_unstale":"2024-08-18T23:43:24.304850+0000","last_undegraded":"2024-08-18T23:43:24.304850+0000","last_fullsized":"2024-08-18T23:43:24.304850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:07:24.912259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301163+0000","last_change":"2024-08-18T23:41:39.186033+0000","last_active":"2024-08-18T23:43:24.301163+0000","last_peered":"2024-08-18T23:43:24.301163+0000","last_clean":"2024-08-18T23:43:24.301163+0000","last_became_active":"2024-08-18T23:41:39.185451+0000","last_became_peered":"2024-08-18T23:41:39.185451+0000","last_unstale":"2024-08-18T23:43:24.301163+0000","last_undegraded":"2024-08-18T23:43:24.301163+0000","last_fullsized":"2024-08-18T23:43:24.301163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:52:13.556760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.306993+0000","last_change":"2024-08-18T23:41:32.141065+0000","last_active":"2024-08-18T23:43:24.306993+0000","last_peered":"2024-08-18T23:43:24.306993+0000","last_clean":"2024-08-18T23:43:24.306993+0000","last_became_active":"2024-08-18T23:41:32.140732+0000","last_became_peered":"2024-08-18T23:41:32.140732+0000","last_unstale":"2024-08-18T23:43:24.306993+0000","last_undegraded":"2024-08-18T23:43:24.306993+0000","last_fullsized":"2024-08-18T23:43:24.306993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:08:32.287835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":161,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:44:02.435987+0000","last_change":"2024-08-18T23:41:30.136838+0000","last_active":"2024-08-18T23:44:02.435987+0000","last_peered":"2024-08-18T23:44:02.435987+0000","last_clean":"2024-08-18T23:44:02.435987+0000","last_became_active":"2024-08-18T23:41:30.136523+0000","last_became_peered":"2024-08-18T23:41:30.136523+0000","last_unstale":"2024-08-18T23:44:02.435987+0000","last_undegraded":"2024-08-18T23:44:02.435987+0000","last_fullsized":"2024-08-18T23:44:02.435987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:56:17.750535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301550+0000","last_change":"2024-08-18T23:41:39.680314+0000","last_active":"2024-08-18T23:43:24.301550+0000","last_peered":"2024-08-18T23:43:24.301550+0000","last_clean":"2024-08-18T23:43:24.301550+0000","last_became_active":"2024-08-18T23:41:39.679920+0000","last_became_peered":"2024-08-18T23:41:39.679920+0000","last_unstale":"2024-08-18T23:43:24.301550+0000","last_undegraded":"2024-08-18T23:43:24.301550+0000","last_fullsized":"2024-08-18T23:43:24.301550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:26:29.866081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300888+0000","last_change":"2024-08-18T23:41:32.139752+0000","last_active":"2024-08-18T23:43:24.300888+0000","last_peered":"2024-08-18T23:43:24.300888+0000","last_clean":"2024-08-18T23:43:24.300888+0000","last_became_active":"2024-08-18T23:41:32.139451+0000","last_became_peered":"2024-08-18T23:41:32.139451+0000","last_unstale":"2024-08-18T23:43:24.300888+0000","last_undegraded":"2024-08-18T23:43:24.300888+0000","last_fullsized":"2024-08-18T23:43:24.300888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:15:13.390529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304894+0000","last_change":"2024-08-18T23:41:30.126513+0000","last_active":"2024-08-18T23:43:24.304894+0000","last_peered":"2024-08-18T23:43:24.304894+0000","last_clean":"2024-08-18T23:43:24.304894+0000","last_became_active":"2024-08-18T23:41:30.126046+0000","last_became_peered":"2024-08-18T23:41:30.126046+0000","last_unstale":"2024-08-18T23:43:24.304894+0000","last_undegraded":"2024-08-18T23:43:24.304894+0000","last_fullsized":"2024-08-18T23:43:24.304894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:49:16.565565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307280+0000","last_change":"2024-08-18T23:41:39.681006+0000","last_active":"2024-08-18T23:43:24.307280+0000","last_peered":"2024-08-18T23:43:24.307280+0000","last_clean":"2024-08-18T23:43:24.307280+0000","last_became_active":"2024-08-18T23:41:39.680409+0000","last_became_peered":"2024-08-18T23:41:39.680409+0000","last_unstale":"2024-08-18T23:43:24.307280+0000","last_undegraded":"2024-08-18T23:43:24.307280+0000","last_fullsized":"2024-08-18T23:43:24.307280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:47.778866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312261+0000","last_change":"2024-08-18T23:41:32.147545+0000","last_active":"2024-08-18T23:43:24.312261+0000","last_peered":"2024-08-18T23:43:24.312261+0000","last_clean":"2024-08-18T23:43:24.312261+0000","last_became_active":"2024-08-18T23:41:32.147238+0000","last_became_peered":"2024-08-18T23:41:32.147238+0000","last_unstale":"2024-08-18T23:43:24.312261+0000","last_undegraded":"2024-08-18T23:43:24.312261+0000","last_fullsized":"2024-08-18T23:43:24.312261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:12:13.589535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313054+0000","last_change":"2024-08-18T23:41:30.120650+0000","last_active":"2024-08-18T23:43:24.313054+0000","last_peered":"2024-08-18T23:43:24.313054+0000","last_clean":"2024-08-18T23:43:24.313054+0000","last_became_active":"2024-08-18T23:41:30.120196+0000","last_became_peered":"2024-08-18T23:41:30.120196+0000","last_unstale":"2024-08-18T23:43:24.313054+0000","last_undegraded":"2024-08-18T23:43:24.313054+0000","last_fullsized":"2024-08-18T23:43:24.313054+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:40:23.370088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304710+0000","last_change":"2024-08-18T23:40:22.013519+0000","last_active":"2024-08-18T23:43:24.304710+0000","last_peered":"2024-08-18T23:43:24.304710+0000","last_clean":"2024-08-18T23:43:24.304710+0000","last_became_active":"2024-08-18T23:40:22.012986+0000","last_became_peered":"2024-08-18T23:40:22.012986+0000","last_unstale":"2024-08-18T23:43:24.304710+0000","last_undegraded":"2024-08-18T23:43:24.304710+0000","last_fullsized":"2024-08-18T23:43:24.304710+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_clean_scrub_stamp":"2024-08-18T23:37:22.388424+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:19.874356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304893+0000","last_change":"2024-08-18T23:42:23.332800+0000","last_active":"2024-08-18T23:43:24.304893+0000","last_peered":"2024-08-18T23:43:24.304893+0000","last_clean":"2024-08-18T23:43:24.304893+0000","last_became_active":"2024-08-18T23:42:23.326441+0000","last_became_peered":"2024-08-18T23:42:23.326441+0000","last_unstale":"2024-08-18T23:43:24.304893+0000","last_undegraded":"2024-08-18T23:43:24.304893+0000","last_fullsized":"2024-08-18T23:43:24.304893+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:13:58.243518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.305065+0000","last_change":"2024-08-18T23:41:32.139907+0000","last_active":"2024-08-18T23:43:24.305065+0000","last_peered":"2024-08-18T23:43:24.305065+0000","last_clean":"2024-08-18T23:43:24.305065+0000","last_became_active":"2024-08-18T23:41:32.137956+0000","last_became_peered":"2024-08-18T23:41:32.137956+0000","last_unstale":"2024-08-18T23:43:24.305065+0000","last_undegraded":"2024-08-18T23:43:24.305065+0000","last_fullsized":"2024-08-18T23:43:24.305065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:10:13.223328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312461+0000","last_change":"2024-08-18T23:42:23.330062+0000","last_active":"2024-08-18T23:43:24.312461+0000","last_peered":"2024-08-18T23:43:24.312461+0000","last_clean":"2024-08-18T23:43:24.312461+0000","last_became_active":"2024-08-18T23:42:23.323196+0000","last_became_peered":"2024-08-18T23:42:23.323196+0000","last_unstale":"2024-08-18T23:43:24.312461+0000","last_undegraded":"2024-08-18T23:43:24.312461+0000","last_fullsized":"2024-08-18T23:43:24.312461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:53:29.059542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312185+0000","last_change":"2024-08-18T23:41:39.185917+0000","last_active":"2024-08-18T23:43:24.312185+0000","last_peered":"2024-08-18T23:43:24.312185+0000","last_clean":"2024-08-18T23:43:24.312185+0000","last_became_active":"2024-08-18T23:41:39.185427+0000","last_became_peered":"2024-08-18T23:41:39.185427+0000","last_unstale":"2024-08-18T23:43:24.312185+0000","last_undegraded":"2024-08-18T23:43:24.312185+0000","last_fullsized":"2024-08-18T23:43:24.312185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:52:13.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.308822+0000","last_change":"2024-08-18T23:43:24.308401+0000","last_active":"2024-08-18T23:43:24.308822+0000","last_peered":"2024-08-18T23:43:24.308822+0000","last_clean":"2024-08-18T23:43:24.308822+0000","last_became_active":"2024-08-18T23:43:24.308081+0000","last_became_peered":"2024-08-18T23:43:24.308081+0000","last_unstale":"2024-08-18T23:43:24.308822+0000","last_undegraded":"2024-08-18T23:43:24.308822+0000","last_fullsized":"2024-08-18T23:43:24.308822+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:28:57.712217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307528+0000","last_change":"2024-08-18T23:41:30.121307+0000","last_active":"2024-08-18T23:43:24.307528+0000","last_peered":"2024-08-18T23:43:24.307528+0000","last_clean":"2024-08-18T23:43:24.307528+0000","last_became_active":"2024-08-18T23:41:30.120932+0000","last_became_peered":"2024-08-18T23:41:30.120932+0000","last_unstale":"2024-08-18T23:43:24.307528+0000","last_undegraded":"2024-08-18T23:43:24.307528+0000","last_fullsized":"2024-08-18T23:43:24.307528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:06:07.742880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311751+0000","last_change":"2024-08-18T23:41:39.184647+0000","last_active":"2024-08-18T23:43:24.311751+0000","last_peered":"2024-08-18T23:43:24.311751+0000","last_clean":"2024-08-18T23:43:24.311751+0000","last_became_active":"2024-08-18T23:41:39.184251+0000","last_became_peered":"2024-08-18T23:41:39.184251+0000","last_unstale":"2024-08-18T23:43:24.311751+0000","last_undegraded":"2024-08-18T23:43:24.311751+0000","last_fullsized":"2024-08-18T23:43:24.311751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:55.200801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.305184+0000","last_change":"2024-08-18T23:41:32.137007+0000","last_active":"2024-08-18T23:43:24.305184+0000","last_peered":"2024-08-18T23:43:24.305184+0000","last_clean":"2024-08-18T23:43:24.305184+0000","last_became_active":"2024-08-18T23:41:32.136684+0000","last_became_peered":"2024-08-18T23:41:32.136684+0000","last_unstale":"2024-08-18T23:43:24.305184+0000","last_undegraded":"2024-08-18T23:43:24.305184+0000","last_fullsized":"2024-08-18T23:43:24.305184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:21:26.584303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301471+0000","last_change":"2024-08-18T23:41:30.121183+0000","last_active":"2024-08-18T23:43:24.301471+0000","last_peered":"2024-08-18T23:43:24.301471+0000","last_clean":"2024-08-18T23:43:24.301471+0000","last_became_active":"2024-08-18T23:41:30.120847+0000","last_became_peered":"2024-08-18T23:41:30.120847+0000","last_unstale":"2024-08-18T23:43:24.301471+0000","last_undegraded":"2024-08-18T23:43:24.301471+0000","last_fullsized":"2024-08-18T23:43:24.301471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:51.025821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.306833+0000","last_change":"2024-08-18T23:41:39.183375+0000","last_active":"2024-08-18T23:43:24.306833+0000","last_peered":"2024-08-18T23:43:24.306833+0000","last_clean":"2024-08-18T23:43:24.306833+0000","last_became_active":"2024-08-18T23:41:39.182339+0000","last_became_peered":"2024-08-18T23:41:39.182339+0000","last_unstale":"2024-08-18T23:43:24.306833+0000","last_undegraded":"2024-08-18T23:43:24.306833+0000","last_fullsized":"2024-08-18T23:43:24.306833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:26.119056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313146+0000","last_change":"2024-08-18T23:41:32.140748+0000","last_active":"2024-08-18T23:43:24.313146+0000","last_peered":"2024-08-18T23:43:24.313146+0000","last_clean":"2024-08-18T23:43:24.313146+0000","last_became_active":"2024-08-18T23:41:32.140306+0000","last_became_peered":"2024-08-18T23:41:32.140306+0000","last_unstale":"2024-08-18T23:43:24.313146+0000","last_undegraded":"2024-08-18T23:43:24.313146+0000","last_fullsized":"2024-08-18T23:43:24.313146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:23:02.371529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:48.359280+0000","last_change":"2024-08-18T23:41:30.121617+0000","last_active":"2024-08-18T23:43:48.359280+0000","last_peered":"2024-08-18T23:43:48.359280+0000","last_clean":"2024-08-18T23:43:48.359280+0000","last_became_active":"2024-08-18T23:41:30.121257+0000","last_became_peered":"2024-08-18T23:41:30.121257+0000","last_unstale":"2024-08-18T23:43:48.359280+0000","last_undegraded":"2024-08-18T23:43:48.359280+0000","last_fullsized":"2024-08-18T23:43:48.359280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:23:42.091553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307008+0000","last_change":"2024-08-18T23:41:39.183833+0000","last_active":"2024-08-18T23:43:24.307008+0000","last_peered":"2024-08-18T23:43:24.307008+0000","last_clean":"2024-08-18T23:43:24.307008+0000","last_became_active":"2024-08-18T23:41:39.183176+0000","last_became_peered":"2024-08-18T23:41:39.183176+0000","last_unstale":"2024-08-18T23:43:24.307008+0000","last_undegraded":"2024-08-18T23:43:24.307008+0000","last_fullsized":"2024-08-18T23:43:24.307008+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:36:09.827378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307216+0000","last_change":"2024-08-18T23:41:32.147507+0000","last_active":"2024-08-18T23:43:24.307216+0000","last_peered":"2024-08-18T23:43:24.307216+0000","last_clean":"2024-08-18T23:43:24.307216+0000","last_became_active":"2024-08-18T23:41:32.146042+0000","last_became_peered":"2024-08-18T23:41:32.146042+0000","last_unstale":"2024-08-18T23:43:24.307216+0000","last_undegraded":"2024-08-18T23:43:24.307216+0000","last_fullsized":"2024-08-18T23:43:24.307216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:48.941442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303448+0000","last_change":"2024-08-18T23:41:30.137755+0000","last_active":"2024-08-18T23:43:24.303448+0000","last_peered":"2024-08-18T23:43:24.303448+0000","last_clean":"2024-08-18T23:43:24.303448+0000","last_became_active":"2024-08-18T23:41:30.137194+0000","last_became_peered":"2024-08-18T23:41:30.137194+0000","last_unstale":"2024-08-18T23:43:24.303448+0000","last_undegraded":"2024-08-18T23:43:24.303448+0000","last_fullsized":"2024-08-18T23:43:24.303448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:15:49.681405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301357+0000","last_change":"2024-08-18T23:41:39.183986+0000","last_active":"2024-08-18T23:43:24.301357+0000","last_peered":"2024-08-18T23:43:24.301357+0000","last_clean":"2024-08-18T23:43:24.301357+0000","last_became_active":"2024-08-18T23:41:39.183384+0000","last_became_peered":"2024-08-18T23:41:39.183384+0000","last_unstale":"2024-08-18T23:43:24.301357+0000","last_undegraded":"2024-08-18T23:43:24.301357+0000","last_fullsized":"2024-08-18T23:43:24.301357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:04:28.920804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304496+0000","last_change":"2024-08-18T23:41:32.134729+0000","last_active":"2024-08-18T23:43:24.304496+0000","last_peered":"2024-08-18T23:43:24.304496+0000","last_clean":"2024-08-18T23:43:24.304496+0000","last_became_active":"2024-08-18T23:41:32.134366+0000","last_became_peered":"2024-08-18T23:41:32.134366+0000","last_unstale":"2024-08-18T23:43:24.304496+0000","last_undegraded":"2024-08-18T23:43:24.304496+0000","last_fullsized":"2024-08-18T23:43:24.304496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:38.610561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'6","reported_seq":167,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:44:03.438139+0000","last_change":"2024-08-18T23:41:30.126443+0000","last_active":"2024-08-18T23:44:03.438139+0000","last_peered":"2024-08-18T23:44:03.438139+0000","last_clean":"2024-08-18T23:44:03.438139+0000","last_became_active":"2024-08-18T23:41:30.124264+0000","last_became_peered":"2024-08-18T23:41:30.124264+0000","last_unstale":"2024-08-18T23:44:03.438139+0000","last_undegraded":"2024-08-18T23:44:03.438139+0000","last_fullsized":"2024-08-18T23:44:03.438139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:43.727114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1427,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298489+0000","last_change":"2024-08-18T23:41:39.192164+0000","last_active":"2024-08-18T23:43:24.298489+0000","last_peered":"2024-08-18T23:43:24.298489+0000","last_clean":"2024-08-18T23:43:24.298489+0000","last_became_active":"2024-08-18T23:41:39.191833+0000","last_became_peered":"2024-08-18T23:41:39.191833+0000","last_unstale":"2024-08-18T23:43:24.298489+0000","last_undegraded":"2024-08-18T23:43:24.298489+0000","last_fullsized":"2024-08-18T23:43:24.298489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:33:50.243930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304927+0000","last_change":"2024-08-18T23:42:23.320662+0000","last_active":"2024-08-18T23:43:24.304927+0000","last_peered":"2024-08-18T23:43:24.304927+0000","last_clean":"2024-08-18T23:43:24.304927+0000","last_became_active":"2024-08-18T23:42:23.320249+0000","last_became_peered":"2024-08-18T23:42:23.320249+0000","last_unstale":"2024-08-18T23:43:24.304927+0000","last_undegraded":"2024-08-18T23:43:24.304927+0000","last_fullsized":"2024-08-18T23:43:24.304927+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:03:17.121915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307658+0000","last_change":"2024-08-18T23:41:30.124071+0000","last_active":"2024-08-18T23:43:24.307658+0000","last_peered":"2024-08-18T23:43:24.307658+0000","last_clean":"2024-08-18T23:43:24.307658+0000","last_became_active":"2024-08-18T23:41:30.123118+0000","last_became_peered":"2024-08-18T23:41:30.123118+0000","last_unstale":"2024-08-18T23:43:24.307658+0000","last_undegraded":"2024-08-18T23:43:24.307658+0000","last_fullsized":"2024-08-18T23:43:24.307658+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:21:39.015187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313136+0000","last_change":"2024-08-18T23:41:39.184038+0000","last_active":"2024-08-18T23:43:24.313136+0000","last_peered":"2024-08-18T23:43:24.313136+0000","last_clean":"2024-08-18T23:43:24.313136+0000","last_became_active":"2024-08-18T23:41:39.183611+0000","last_became_peered":"2024-08-18T23:41:39.183611+0000","last_unstale":"2024-08-18T23:43:24.313136+0000","last_undegraded":"2024-08-18T23:43:24.313136+0000","last_fullsized":"2024-08-18T23:43:24.313136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:32:26.029151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300998+0000","last_change":"2024-08-18T23:41:32.143631+0000","last_active":"2024-08-18T23:43:24.300998+0000","last_peered":"2024-08-18T23:43:24.300998+0000","last_clean":"2024-08-18T23:43:24.300998+0000","last_became_active":"2024-08-18T23:41:32.143350+0000","last_became_peered":"2024-08-18T23:41:32.143350+0000","last_unstale":"2024-08-18T23:43:24.300998+0000","last_undegraded":"2024-08-18T23:43:24.300998+0000","last_fullsized":"2024-08-18T23:43:24.300998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:41:23.713467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312424+0000","last_change":"2024-08-18T23:41:30.123383+0000","last_active":"2024-08-18T23:43:24.312424+0000","last_peered":"2024-08-18T23:43:24.312424+0000","last_clean":"2024-08-18T23:43:24.312424+0000","last_became_active":"2024-08-18T23:41:30.123037+0000","last_became_peered":"2024-08-18T23:41:30.123037+0000","last_unstale":"2024-08-18T23:43:24.312424+0000","last_undegraded":"2024-08-18T23:43:24.312424+0000","last_fullsized":"2024-08-18T23:43:24.312424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:27.702478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307094+0000","last_change":"2024-08-18T23:41:39.184302+0000","last_active":"2024-08-18T23:43:24.307094+0000","last_peered":"2024-08-18T23:43:24.307094+0000","last_clean":"2024-08-18T23:43:24.307094+0000","last_became_active":"2024-08-18T23:41:39.183172+0000","last_became_peered":"2024-08-18T23:41:39.183172+0000","last_unstale":"2024-08-18T23:43:24.307094+0000","last_undegraded":"2024-08-18T23:43:24.307094+0000","last_fullsized":"2024-08-18T23:43:24.307094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:23:26.655865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304016+0000","last_change":"2024-08-18T23:41:32.137426+0000","last_active":"2024-08-18T23:43:24.304016+0000","last_peered":"2024-08-18T23:43:24.304016+0000","last_clean":"2024-08-18T23:43:24.304016+0000","last_became_active":"2024-08-18T23:41:32.137106+0000","last_became_peered":"2024-08-18T23:41:32.137106+0000","last_unstale":"2024-08-18T23:43:24.304016+0000","last_undegraded":"2024-08-18T23:43:24.304016+0000","last_fullsized":"2024-08-18T23:43:24.304016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:34:51.889663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'40","reported_seq":326,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:44:04.439904+0000","last_change":"2024-08-18T23:41:30.124320+0000","last_active":"2024-08-18T23:44:04.439904+0000","last_peered":"2024-08-18T23:44:04.439904+0000","last_clean":"2024-08-18T23:44:04.439904+0000","last_became_active":"2024-08-18T23:41:30.123979+0000","last_became_peered":"2024-08-18T23:41:30.123979+0000","last_unstale":"2024-08-18T23:44:04.439904+0000","last_undegraded":"2024-08-18T23:44:04.439904+0000","last_fullsized":"2024-08-18T23:44:04.439904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:31:03.689720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70777,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":247,"num_read_kb":193,"num_write":41,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298022+0000","last_change":"2024-08-18T23:41:39.190481+0000","last_active":"2024-08-18T23:43:24.298022+0000","last_peered":"2024-08-18T23:43:24.298022+0000","last_clean":"2024-08-18T23:43:24.298022+0000","last_became_active":"2024-08-18T23:41:39.190076+0000","last_became_peered":"2024-08-18T23:41:39.190076+0000","last_unstale":"2024-08-18T23:43:24.298022+0000","last_undegraded":"2024-08-18T23:43:24.298022+0000","last_fullsized":"2024-08-18T23:43:24.298022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:47:39.313142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312507+0000","last_change":"2024-08-18T23:41:32.145422+0000","last_active":"2024-08-18T23:43:24.312507+0000","last_peered":"2024-08-18T23:43:24.312507+0000","last_clean":"2024-08-18T23:43:24.312507+0000","last_became_active":"2024-08-18T23:41:32.145047+0000","last_became_peered":"2024-08-18T23:41:32.145047+0000","last_unstale":"2024-08-18T23:43:24.312507+0000","last_undegraded":"2024-08-18T23:43:24.312507+0000","last_fullsized":"2024-08-18T23:43:24.312507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:31.905248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307467+0000","last_change":"2024-08-18T23:41:30.137364+0000","last_active":"2024-08-18T23:43:24.307467+0000","last_peered":"2024-08-18T23:43:24.307467+0000","last_clean":"2024-08-18T23:43:24.307467+0000","last_became_active":"2024-08-18T23:41:30.137004+0000","last_became_peered":"2024-08-18T23:41:30.137004+0000","last_unstale":"2024-08-18T23:43:24.307467+0000","last_undegraded":"2024-08-18T23:43:24.307467+0000","last_fullsized":"2024-08-18T23:43:24.307467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:20:56.392149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312779+0000","last_change":"2024-08-18T23:42:23.326359+0000","last_active":"2024-08-18T23:43:24.312779+0000","last_peered":"2024-08-18T23:43:24.312779+0000","last_clean":"2024-08-18T23:43:24.312779+0000","last_became_active":"2024-08-18T23:42:23.325918+0000","last_became_peered":"2024-08-18T23:42:23.325918+0000","last_unstale":"2024-08-18T23:43:24.312779+0000","last_undegraded":"2024-08-18T23:43:24.312779+0000","last_fullsized":"2024-08-18T23:43:24.312779+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:01:15.476268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312633+0000","last_change":"2024-08-18T23:42:23.325691+0000","last_active":"2024-08-18T23:43:24.312633+0000","last_peered":"2024-08-18T23:43:24.312633+0000","last_clean":"2024-08-18T23:43:24.312633+0000","last_became_active":"2024-08-18T23:42:23.325467+0000","last_became_peered":"2024-08-18T23:42:23.325467+0000","last_unstale":"2024-08-18T23:43:24.312633+0000","last_undegraded":"2024-08-18T23:43:24.312633+0000","last_fullsized":"2024-08-18T23:43:24.312633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:10:26.288880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.305106+0000","last_change":"2024-08-18T23:41:30.124430+0000","last_active":"2024-08-18T23:43:24.305106+0000","last_peered":"2024-08-18T23:43:24.305106+0000","last_clean":"2024-08-18T23:43:24.305106+0000","last_became_active":"2024-08-18T23:41:30.123626+0000","last_became_peered":"2024-08-18T23:41:30.123626+0000","last_unstale":"2024-08-18T23:43:24.305106+0000","last_undegraded":"2024-08-18T23:43:24.305106+0000","last_fullsized":"2024-08-18T23:43:24.305106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:26:18.426010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.297855+0000","last_change":"2024-08-18T23:41:39.192765+0000","last_active":"2024-08-18T23:43:24.297855+0000","last_peered":"2024-08-18T23:43:24.297855+0000","last_clean":"2024-08-18T23:43:24.297855+0000","last_became_active":"2024-08-18T23:41:39.192250+0000","last_became_peered":"2024-08-18T23:41:39.192250+0000","last_unstale":"2024-08-18T23:43:24.297855+0000","last_undegraded":"2024-08-18T23:43:24.297855+0000","last_fullsized":"2024-08-18T23:43:24.297855+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:51:16.794404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312750+0000","last_change":"2024-08-18T23:41:32.141966+0000","last_active":"2024-08-18T23:43:24.312750+0000","last_peered":"2024-08-18T23:43:24.312750+0000","last_clean":"2024-08-18T23:43:24.312750+0000","last_became_active":"2024-08-18T23:41:32.141561+0000","last_became_peered":"2024-08-18T23:41:32.141561+0000","last_unstale":"2024-08-18T23:43:24.312750+0000","last_undegraded":"2024-08-18T23:43:24.312750+0000","last_fullsized":"2024-08-18T23:43:24.312750+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:20.696089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":161,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:44:03.438138+0000","last_change":"2024-08-18T23:41:30.121598+0000","last_active":"2024-08-18T23:44:03.438138+0000","last_peered":"2024-08-18T23:44:03.438138+0000","last_clean":"2024-08-18T23:44:03.438138+0000","last_became_active":"2024-08-18T23:41:30.120533+0000","last_became_peered":"2024-08-18T23:41:30.120533+0000","last_unstale":"2024-08-18T23:44:03.438138+0000","last_undegraded":"2024-08-18T23:44:03.438138+0000","last_fullsized":"2024-08-18T23:44:03.438138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:06:37.458272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312273+0000","last_change":"2024-08-18T23:41:39.178877+0000","last_active":"2024-08-18T23:43:24.312273+0000","last_peered":"2024-08-18T23:43:24.312273+0000","last_clean":"2024-08-18T23:43:24.312273+0000","last_became_active":"2024-08-18T23:41:39.178381+0000","last_became_peered":"2024-08-18T23:41:39.178381+0000","last_unstale":"2024-08-18T23:43:24.312273+0000","last_undegraded":"2024-08-18T23:43:24.312273+0000","last_fullsized":"2024-08-18T23:43:24.312273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:27.275893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311702+0000","last_change":"2024-08-18T23:41:39.188381+0000","last_active":"2024-08-18T23:43:24.311702+0000","last_peered":"2024-08-18T23:43:24.311702+0000","last_clean":"2024-08-18T23:43:24.311702+0000","last_became_active":"2024-08-18T23:41:39.187983+0000","last_became_peered":"2024-08-18T23:41:39.187983+0000","last_unstale":"2024-08-18T23:43:24.311702+0000","last_undegraded":"2024-08-18T23:43:24.311702+0000","last_fullsized":"2024-08-18T23:43:24.311702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:04:45.460576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.297935+0000","last_change":"2024-08-18T23:41:32.144637+0000","last_active":"2024-08-18T23:43:24.297935+0000","last_peered":"2024-08-18T23:43:24.297935+0000","last_clean":"2024-08-18T23:43:24.297935+0000","last_became_active":"2024-08-18T23:41:32.144329+0000","last_became_peered":"2024-08-18T23:41:32.144329+0000","last_unstale":"2024-08-18T23:43:24.297935+0000","last_undegraded":"2024-08-18T23:43:24.297935+0000","last_fullsized":"2024-08-18T23:43:24.297935+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:29:39.791264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307199+0000","last_change":"2024-08-18T23:41:30.122325+0000","last_active":"2024-08-18T23:43:24.307199+0000","last_peered":"2024-08-18T23:43:24.307199+0000","last_clean":"2024-08-18T23:43:24.307199+0000","last_became_active":"2024-08-18T23:41:30.121480+0000","last_became_peered":"2024-08-18T23:41:30.121480+0000","last_unstale":"2024-08-18T23:43:24.307199+0000","last_undegraded":"2024-08-18T23:43:24.307199+0000","last_fullsized":"2024-08-18T23:43:24.307199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:32:12.810650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312772+0000","last_change":"2024-08-18T23:42:23.560632+0000","last_active":"2024-08-18T23:43:24.312772+0000","last_peered":"2024-08-18T23:43:24.312772+0000","last_clean":"2024-08-18T23:43:24.312772+0000","last_became_active":"2024-08-18T23:42:23.559920+0000","last_became_peered":"2024-08-18T23:42:23.559920+0000","last_unstale":"2024-08-18T23:43:24.312772+0000","last_undegraded":"2024-08-18T23:43:24.312772+0000","last_fullsized":"2024-08-18T23:43:24.312772+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:01:59.580843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312091+0000","last_change":"2024-08-18T23:41:32.145545+0000","last_active":"2024-08-18T23:43:24.312091+0000","last_peered":"2024-08-18T23:43:24.312091+0000","last_clean":"2024-08-18T23:43:24.312091+0000","last_became_active":"2024-08-18T23:41:32.145183+0000","last_became_peered":"2024-08-18T23:41:32.145183+0000","last_unstale":"2024-08-18T23:43:24.312091+0000","last_undegraded":"2024-08-18T23:43:24.312091+0000","last_fullsized":"2024-08-18T23:43:24.312091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:03:04.111658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300895+0000","last_change":"2024-08-18T23:41:30.118776+0000","last_active":"2024-08-18T23:43:24.300895+0000","last_peered":"2024-08-18T23:43:24.300895+0000","last_clean":"2024-08-18T23:43:24.300895+0000","last_became_active":"2024-08-18T23:41:30.118436+0000","last_became_peered":"2024-08-18T23:41:30.118436+0000","last_unstale":"2024-08-18T23:43:24.300895+0000","last_undegraded":"2024-08-18T23:43:24.300895+0000","last_fullsized":"2024-08-18T23:43:24.300895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:13.980927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300932+0000","last_change":"2024-08-18T23:41:30.125201+0000","last_active":"2024-08-18T23:43:24.300932+0000","last_peered":"2024-08-18T23:43:24.300932+0000","last_clean":"2024-08-18T23:43:24.300932+0000","last_became_active":"2024-08-18T23:41:30.124894+0000","last_became_peered":"2024-08-18T23:41:30.124894+0000","last_unstale":"2024-08-18T23:43:24.300932+0000","last_undegraded":"2024-08-18T23:43:24.300932+0000","last_fullsized":"2024-08-18T23:43:24.300932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:30:17.721829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307776+0000","last_change":"2024-08-18T23:41:32.145452+0000","last_active":"2024-08-18T23:43:24.307776+0000","last_peered":"2024-08-18T23:43:24.307776+0000","last_clean":"2024-08-18T23:43:24.307776+0000","last_became_active":"2024-08-18T23:41:32.145151+0000","last_became_peered":"2024-08-18T23:41:32.145151+0000","last_unstale":"2024-08-18T23:43:24.307776+0000","last_undegraded":"2024-08-18T23:43:24.307776+0000","last_fullsized":"2024-08-18T23:43:24.307776+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:08:08.713154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312361+0000","last_change":"2024-08-18T23:41:39.193334+0000","last_active":"2024-08-18T23:43:24.312361+0000","last_peered":"2024-08-18T23:43:24.312361+0000","last_clean":"2024-08-18T23:43:24.312361+0000","last_became_active":"2024-08-18T23:41:39.192924+0000","last_became_peered":"2024-08-18T23:41:39.192924+0000","last_unstale":"2024-08-18T23:43:24.312361+0000","last_undegraded":"2024-08-18T23:43:24.312361+0000","last_fullsized":"2024-08-18T23:43:24.312361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:07.016300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":166,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:44:01.435799+0000","last_change":"2024-08-18T23:41:30.125217+0000","last_active":"2024-08-18T23:44:01.435799+0000","last_peered":"2024-08-18T23:44:01.435799+0000","last_clean":"2024-08-18T23:44:01.435799+0000","last_became_active":"2024-08-18T23:41:30.124889+0000","last_became_peered":"2024-08-18T23:41:30.124889+0000","last_unstale":"2024-08-18T23:44:01.435799+0000","last_undegraded":"2024-08-18T23:44:01.435799+0000","last_fullsized":"2024-08-18T23:44:01.435799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:43.389537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303531+0000","last_change":"2024-08-18T23:41:32.142585+0000","last_active":"2024-08-18T23:43:24.303531+0000","last_peered":"2024-08-18T23:43:24.303531+0000","last_clean":"2024-08-18T23:43:24.303531+0000","last_became_active":"2024-08-18T23:41:32.142207+0000","last_became_peered":"2024-08-18T23:41:32.142207+0000","last_unstale":"2024-08-18T23:43:24.303531+0000","last_undegraded":"2024-08-18T23:43:24.303531+0000","last_fullsized":"2024-08-18T23:43:24.303531+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:16:58.755444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312344+0000","last_change":"2024-08-18T23:41:39.185618+0000","last_active":"2024-08-18T23:43:24.312344+0000","last_peered":"2024-08-18T23:43:24.312344+0000","last_clean":"2024-08-18T23:43:24.312344+0000","last_became_active":"2024-08-18T23:41:39.185222+0000","last_became_peered":"2024-08-18T23:41:39.185222+0000","last_unstale":"2024-08-18T23:43:24.312344+0000","last_undegraded":"2024-08-18T23:43:24.312344+0000","last_fullsized":"2024-08-18T23:43:24.312344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:01:27.290998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312646+0000","last_change":"2024-08-18T23:41:32.140719+0000","last_active":"2024-08-18T23:43:24.312646+0000","last_peered":"2024-08-18T23:43:24.312646+0000","last_clean":"2024-08-18T23:43:24.312646+0000","last_became_active":"2024-08-18T23:41:32.140131+0000","last_became_peered":"2024-08-18T23:41:32.140131+0000","last_unstale":"2024-08-18T23:43:24.312646+0000","last_undegraded":"2024-08-18T23:43:24.312646+0000","last_fullsized":"2024-08-18T23:43:24.312646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:48:26.778094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312375+0000","last_change":"2024-08-18T23:42:23.326683+0000","last_active":"2024-08-18T23:43:24.312375+0000","last_peered":"2024-08-18T23:43:24.312375+0000","last_clean":"2024-08-18T23:43:24.312375+0000","last_became_active":"2024-08-18T23:42:23.326309+0000","last_became_peered":"2024-08-18T23:42:23.326309+0000","last_unstale":"2024-08-18T23:43:24.312375+0000","last_undegraded":"2024-08-18T23:43:24.312375+0000","last_fullsized":"2024-08-18T23:43:24.312375+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:14.320894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312340+0000","last_change":"2024-08-18T23:41:32.149522+0000","last_active":"2024-08-18T23:43:24.312340+0000","last_peered":"2024-08-18T23:43:24.312340+0000","last_clean":"2024-08-18T23:43:24.312340+0000","last_became_active":"2024-08-18T23:41:32.149182+0000","last_became_peered":"2024-08-18T23:41:32.149182+0000","last_unstale":"2024-08-18T23:43:24.312340+0000","last_undegraded":"2024-08-18T23:43:24.312340+0000","last_fullsized":"2024-08-18T23:43:24.312340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:30:16.945045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311765+0000","last_change":"2024-08-18T23:41:32.149853+0000","last_active":"2024-08-18T23:43:24.311765+0000","last_peered":"2024-08-18T23:43:24.311765+0000","last_clean":"2024-08-18T23:43:24.311765+0000","last_became_active":"2024-08-18T23:41:32.141907+0000","last_became_peered":"2024-08-18T23:41:32.141907+0000","last_unstale":"2024-08-18T23:43:24.311765+0000","last_undegraded":"2024-08-18T23:43:24.311765+0000","last_fullsized":"2024-08-18T23:43:24.311765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:24:37.323567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301279+0000","last_change":"2024-08-18T23:41:32.136179+0000","last_active":"2024-08-18T23:43:24.301279+0000","last_peered":"2024-08-18T23:43:24.301279+0000","last_clean":"2024-08-18T23:43:24.301279+0000","last_became_active":"2024-08-18T23:41:32.135939+0000","last_became_peered":"2024-08-18T23:41:32.135939+0000","last_unstale":"2024-08-18T23:43:24.301279+0000","last_undegraded":"2024-08-18T23:43:24.301279+0000","last_fullsized":"2024-08-18T23:43:24.301279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:15.392061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313279+0000","last_change":"2024-08-18T23:41:32.137509+0000","last_active":"2024-08-18T23:43:24.313279+0000","last_peered":"2024-08-18T23:43:24.313279+0000","last_clean":"2024-08-18T23:43:24.313279+0000","last_became_active":"2024-08-18T23:41:32.136380+0000","last_became_peered":"2024-08-18T23:41:32.136380+0000","last_unstale":"2024-08-18T23:43:24.313279+0000","last_undegraded":"2024-08-18T23:43:24.313279+0000","last_fullsized":"2024-08-18T23:43:24.313279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:49.776225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312602+0000","last_change":"2024-08-18T23:41:32.139271+0000","last_active":"2024-08-18T23:43:24.312602+0000","last_peered":"2024-08-18T23:43:24.312602+0000","last_clean":"2024-08-18T23:43:24.312602+0000","last_became_active":"2024-08-18T23:41:32.138868+0000","last_became_peered":"2024-08-18T23:41:32.138868+0000","last_unstale":"2024-08-18T23:43:24.312602+0000","last_undegraded":"2024-08-18T23:43:24.312602+0000","last_fullsized":"2024-08-18T23:43:24.312602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:49:27.890857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312543+0000","last_change":"2024-08-18T23:41:32.144460+0000","last_active":"2024-08-18T23:43:24.312543+0000","last_peered":"2024-08-18T23:43:24.312543+0000","last_clean":"2024-08-18T23:43:24.312543+0000","last_became_active":"2024-08-18T23:41:32.144178+0000","last_became_peered":"2024-08-18T23:41:32.144178+0000","last_unstale":"2024-08-18T23:43:24.312543+0000","last_undegraded":"2024-08-18T23:43:24.312543+0000","last_fullsized":"2024-08-18T23:43:24.312543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:56.252215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.306952+0000","last_change":"2024-08-18T23:41:32.147973+0000","last_active":"2024-08-18T23:43:24.306952+0000","last_peered":"2024-08-18T23:43:24.306952+0000","last_clean":"2024-08-18T23:43:24.306952+0000","last_became_active":"2024-08-18T23:41:32.147621+0000","last_became_peered":"2024-08-18T23:41:32.147621+0000","last_unstale":"2024-08-18T23:43:24.306952+0000","last_undegraded":"2024-08-18T23:43:24.306952+0000","last_fullsized":"2024-08-18T23:43:24.306952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:14.176569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312329+0000","last_change":"2024-08-18T23:41:32.146029+0000","last_active":"2024-08-18T23:43:24.312329+0000","last_peered":"2024-08-18T23:43:24.312329+0000","last_clean":"2024-08-18T23:43:24.312329+0000","last_became_active":"2024-08-18T23:41:32.145584+0000","last_became_peered":"2024-08-18T23:41:32.145584+0000","last_unstale":"2024-08-18T23:43:24.312329+0000","last_undegraded":"2024-08-18T23:43:24.312329+0000","last_fullsized":"2024-08-18T23:43:24.312329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:25:33.884906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312304+0000","last_change":"2024-08-18T23:41:32.145950+0000","last_active":"2024-08-18T23:43:24.312304+0000","last_peered":"2024-08-18T23:43:24.312304+0000","last_clean":"2024-08-18T23:43:24.312304+0000","last_became_active":"2024-08-18T23:41:32.145618+0000","last_became_peered":"2024-08-18T23:41:32.145618+0000","last_unstale":"2024-08-18T23:43:24.312304+0000","last_undegraded":"2024-08-18T23:43:24.312304+0000","last_fullsized":"2024-08-18T23:43:24.312304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:05:58.889240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301154+0000","last_change":"2024-08-18T23:41:32.130684+0000","last_active":"2024-08-18T23:43:24.301154+0000","last_peered":"2024-08-18T23:43:24.301154+0000","last_clean":"2024-08-18T23:43:24.301154+0000","last_became_active":"2024-08-18T23:41:32.130369+0000","last_became_peered":"2024-08-18T23:41:32.130369+0000","last_unstale":"2024-08-18T23:43:24.301154+0000","last_undegraded":"2024-08-18T23:43:24.301154+0000","last_fullsized":"2024-08-18T23:43:24.301154+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:22.537018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313085+0000","last_change":"2024-08-18T23:41:32.137291+0000","last_active":"2024-08-18T23:43:24.313085+0000","last_peered":"2024-08-18T23:43:24.313085+0000","last_clean":"2024-08-18T23:43:24.313085+0000","last_became_active":"2024-08-18T23:41:32.136977+0000","last_became_peered":"2024-08-18T23:41:32.136977+0000","last_unstale":"2024-08-18T23:43:24.313085+0000","last_undegraded":"2024-08-18T23:43:24.313085+0000","last_fullsized":"2024-08-18T23:43:24.313085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:18:45.544760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312711+0000","last_change":"2024-08-18T23:41:32.144434+0000","last_active":"2024-08-18T23:43:24.312711+0000","last_peered":"2024-08-18T23:43:24.312711+0000","last_clean":"2024-08-18T23:43:24.312711+0000","last_became_active":"2024-08-18T23:41:32.144119+0000","last_became_peered":"2024-08-18T23:41:32.144119+0000","last_unstale":"2024-08-18T23:43:24.312711+0000","last_undegraded":"2024-08-18T23:43:24.312711+0000","last_fullsized":"2024-08-18T23:43:24.312711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:50:54.528630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298360+0000","last_change":"2024-08-18T23:41:32.135034+0000","last_active":"2024-08-18T23:43:24.298360+0000","last_peered":"2024-08-18T23:43:24.298360+0000","last_clean":"2024-08-18T23:43:24.298360+0000","last_became_active":"2024-08-18T23:41:32.134700+0000","last_became_peered":"2024-08-18T23:41:32.134700+0000","last_unstale":"2024-08-18T23:43:24.298360+0000","last_undegraded":"2024-08-18T23:43:24.298360+0000","last_fullsized":"2024-08-18T23:43:24.298360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:06:31.263114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313052+0000","last_change":"2024-08-18T23:41:32.141191+0000","last_active":"2024-08-18T23:43:24.313052+0000","last_peered":"2024-08-18T23:43:24.313052+0000","last_clean":"2024-08-18T23:43:24.313052+0000","last_became_active":"2024-08-18T23:41:32.140901+0000","last_became_peered":"2024-08-18T23:41:32.140901+0000","last_unstale":"2024-08-18T23:43:24.313052+0000","last_undegraded":"2024-08-18T23:43:24.313052+0000","last_fullsized":"2024-08-18T23:43:24.313052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:24.849985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311944+0000","last_change":"2024-08-18T23:41:32.142543+0000","last_active":"2024-08-18T23:43:24.311944+0000","last_peered":"2024-08-18T23:43:24.311944+0000","last_clean":"2024-08-18T23:43:24.311944+0000","last_became_active":"2024-08-18T23:41:32.142084+0000","last_became_peered":"2024-08-18T23:41:32.142084+0000","last_unstale":"2024-08-18T23:43:24.311944+0000","last_undegraded":"2024-08-18T23:43:24.311944+0000","last_fullsized":"2024-08-18T23:43:24.311944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:54:18.200016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311562+0000","last_change":"2024-08-18T23:41:32.148968+0000","last_active":"2024-08-18T23:43:24.311562+0000","last_peered":"2024-08-18T23:43:24.311562+0000","last_clean":"2024-08-18T23:43:24.311562+0000","last_became_active":"2024-08-18T23:41:32.148625+0000","last_became_peered":"2024-08-18T23:41:32.148625+0000","last_unstale":"2024-08-18T23:43:24.311562+0000","last_undegraded":"2024-08-18T23:43:24.311562+0000","last_fullsized":"2024-08-18T23:43:24.311562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:11:40.172139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311624+0000","last_change":"2024-08-18T23:41:32.133478+0000","last_active":"2024-08-18T23:43:24.311624+0000","last_peered":"2024-08-18T23:43:24.311624+0000","last_clean":"2024-08-18T23:43:24.311624+0000","last_became_active":"2024-08-18T23:41:32.133149+0000","last_became_peered":"2024-08-18T23:41:32.133149+0000","last_unstale":"2024-08-18T23:43:24.311624+0000","last_undegraded":"2024-08-18T23:43:24.311624+0000","last_fullsized":"2024-08-18T23:43:24.311624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:57:53.713309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303573+0000","last_change":"2024-08-18T23:43:24.303232+0000","last_active":"2024-08-18T23:43:24.303573+0000","last_peered":"2024-08-18T23:43:24.303573+0000","last_clean":"2024-08-18T23:43:24.303573+0000","last_became_active":"2024-08-18T23:43:24.302849+0000","last_became_peered":"2024-08-18T23:43:24.302849+0000","last_unstale":"2024-08-18T23:43:24.303573+0000","last_undegraded":"2024-08-18T23:43:24.303573+0000","last_fullsized":"2024-08-18T23:43:24.303573+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:02:02.479891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313096+0000","last_change":"2024-08-18T23:41:32.136303+0000","last_active":"2024-08-18T23:43:24.313096+0000","last_peered":"2024-08-18T23:43:24.313096+0000","last_clean":"2024-08-18T23:43:24.313096+0000","last_became_active":"2024-08-18T23:41:32.134628+0000","last_became_peered":"2024-08-18T23:41:32.134628+0000","last_unstale":"2024-08-18T23:43:24.313096+0000","last_undegraded":"2024-08-18T23:43:24.313096+0000","last_fullsized":"2024-08-18T23:43:24.313096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:28:51.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304992+0000","last_change":"2024-08-18T23:41:32.137512+0000","last_active":"2024-08-18T23:43:24.304992+0000","last_peered":"2024-08-18T23:43:24.304992+0000","last_clean":"2024-08-18T23:43:24.304992+0000","last_became_active":"2024-08-18T23:41:32.137150+0000","last_became_peered":"2024-08-18T23:41:32.137150+0000","last_unstale":"2024-08-18T23:43:24.304992+0000","last_undegraded":"2024-08-18T23:43:24.304992+0000","last_fullsized":"2024-08-18T23:43:24.304992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:44:08.020780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301258+0000","last_change":"2024-08-18T23:41:32.139229+0000","last_active":"2024-08-18T23:43:24.301258+0000","last_peered":"2024-08-18T23:43:24.301258+0000","last_clean":"2024-08-18T23:43:24.301258+0000","last_became_active":"2024-08-18T23:41:32.138864+0000","last_became_peered":"2024-08-18T23:41:32.138864+0000","last_unstale":"2024-08-18T23:43:24.301258+0000","last_undegraded":"2024-08-18T23:43:24.301258+0000","last_fullsized":"2024-08-18T23:43:24.301258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:54:21.701750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312829+0000","last_change":"2024-08-18T23:41:32.148727+0000","last_active":"2024-08-18T23:43:24.312829+0000","last_peered":"2024-08-18T23:43:24.312829+0000","last_clean":"2024-08-18T23:43:24.312829+0000","last_became_active":"2024-08-18T23:41:32.146828+0000","last_became_peered":"2024-08-18T23:41:32.146828+0000","last_unstale":"2024-08-18T23:43:24.312829+0000","last_undegraded":"2024-08-18T23:43:24.312829+0000","last_fullsized":"2024-08-18T23:43:24.312829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:51:49.175150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312068+0000","last_change":"2024-08-18T23:41:32.144796+0000","last_active":"2024-08-18T23:43:24.312068+0000","last_peered":"2024-08-18T23:43:24.312068+0000","last_clean":"2024-08-18T23:43:24.312068+0000","last_became_active":"2024-08-18T23:41:32.141931+0000","last_became_peered":"2024-08-18T23:41:32.141931+0000","last_unstale":"2024-08-18T23:43:24.312068+0000","last_undegraded":"2024-08-18T23:43:24.312068+0000","last_fullsized":"2024-08-18T23:43:24.312068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:48:08.428845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304569+0000","last_change":"2024-08-18T23:41:32.135265+0000","last_active":"2024-08-18T23:43:24.304569+0000","last_peered":"2024-08-18T23:43:24.304569+0000","last_clean":"2024-08-18T23:43:24.304569+0000","last_became_active":"2024-08-18T23:41:32.134952+0000","last_became_peered":"2024-08-18T23:41:32.134952+0000","last_unstale":"2024-08-18T23:43:24.304569+0000","last_undegraded":"2024-08-18T23:43:24.304569+0000","last_fullsized":"2024-08-18T23:43:24.304569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:35:14.099536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300791+0000","last_change":"2024-08-18T23:41:32.135941+0000","last_active":"2024-08-18T23:43:24.300791+0000","last_peered":"2024-08-18T23:43:24.300791+0000","last_clean":"2024-08-18T23:43:24.300791+0000","last_became_active":"2024-08-18T23:41:32.135623+0000","last_became_peered":"2024-08-18T23:41:32.135623+0000","last_unstale":"2024-08-18T23:43:24.300791+0000","last_undegraded":"2024-08-18T23:43:24.300791+0000","last_fullsized":"2024-08-18T23:43:24.300791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:43:36.504738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303603+0000","last_change":"2024-08-18T23:41:32.141643+0000","last_active":"2024-08-18T23:43:24.303603+0000","last_peered":"2024-08-18T23:43:24.303603+0000","last_clean":"2024-08-18T23:43:24.303603+0000","last_became_active":"2024-08-18T23:41:32.141301+0000","last_became_peered":"2024-08-18T23:41:32.141301+0000","last_unstale":"2024-08-18T23:43:24.303603+0000","last_undegraded":"2024-08-18T23:43:24.303603+0000","last_fullsized":"2024-08-18T23:43:24.303603+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:23:53.100740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312988+0000","last_change":"2024-08-18T23:41:32.146730+0000","last_active":"2024-08-18T23:43:24.312988+0000","last_peered":"2024-08-18T23:43:24.312988+0000","last_clean":"2024-08-18T23:43:24.312988+0000","last_became_active":"2024-08-18T23:41:32.146252+0000","last_became_peered":"2024-08-18T23:41:32.146252+0000","last_unstale":"2024-08-18T23:43:24.312988+0000","last_undegraded":"2024-08-18T23:43:24.312988+0000","last_fullsized":"2024-08-18T23:43:24.312988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:41.481761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311744+0000","last_change":"2024-08-18T23:41:32.149569+0000","last_active":"2024-08-18T23:43:24.311744+0000","last_peered":"2024-08-18T23:43:24.311744+0000","last_clean":"2024-08-18T23:43:24.311744+0000","last_became_active":"2024-08-18T23:41:32.149277+0000","last_became_peered":"2024-08-18T23:41:32.149277+0000","last_unstale":"2024-08-18T23:43:24.311744+0000","last_undegraded":"2024-08-18T23:43:24.311744+0000","last_fullsized":"2024-08-18T23:43:24.311744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:34:00.547909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311950+0000","last_change":"2024-08-18T23:41:32.137889+0000","last_active":"2024-08-18T23:43:24.311950+0000","last_peered":"2024-08-18T23:43:24.311950+0000","last_clean":"2024-08-18T23:43:24.311950+0000","last_became_active":"2024-08-18T23:41:32.137578+0000","last_became_peered":"2024-08-18T23:41:32.137578+0000","last_unstale":"2024-08-18T23:43:24.311950+0000","last_undegraded":"2024-08-18T23:43:24.311950+0000","last_fullsized":"2024-08-18T23:43:24.311950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:18:09.987838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312999+0000","last_change":"2024-08-18T23:41:32.142084+0000","last_active":"2024-08-18T23:43:24.312999+0000","last_peered":"2024-08-18T23:43:24.312999+0000","last_clean":"2024-08-18T23:43:24.312999+0000","last_became_active":"2024-08-18T23:41:32.141670+0000","last_became_peered":"2024-08-18T23:41:32.141670+0000","last_unstale":"2024-08-18T23:43:24.312999+0000","last_undegraded":"2024-08-18T23:43:24.312999+0000","last_fullsized":"2024-08-18T23:43:24.312999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:03.990351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76147,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":734,"num_read_kb":449,"num_write":193,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":450560,"data_stored":225921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164,"ondisk_log_size":164,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514540,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":523788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776177,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1753088,"data_stored":1182672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939701,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":528120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168634,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":528040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397565,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1662976,"data_stored":1121222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561091,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1048576,"data_stored":519200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790023,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1111875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953548,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1064960,"data_stored":525271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":519394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":589056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1126400,"data_stored":589887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":520588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":73194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T23:44:07.859 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-18T23:44:08.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.017+0000 7f134ed6c640 1 -- 172.21.15.78:0/3110841964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1348107fc0 msgr2=0x7f134810e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:08.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.017+0000 7f134ed6c640 1 --2- 172.21.15.78:0/3110841964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1348107fc0 0x7f134810e960 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1340009cd0 tx=0x7f134002a8f0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.017+0000 7f134ed6c640 1 -- 172.21.15.78:0/3110841964 shutdown_connections 2024-08-18T23:44:08.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.017+0000 7f134ed6c640 1 --2- 172.21.15.78:0/3110841964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1348107fc0 0x7f134810e960 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.017+0000 7f134ed6c640 1 --2- 172.21.15.78:0/3110841964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1348103470 0x7f13481078b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.017+0000 7f134ed6c640 1 --2- 172.21.15.78:0/3110841964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1348102ac0 0x7f1348102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.017+0000 7f134ed6c640 1 -- 172.21.15.78:0/3110841964 >> 172.21.15.78:0/3110841964 conn(0x7f13480fc500 msgr2=0x7f13480fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:08.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.017+0000 7f134ed6c640 1 -- 172.21.15.78:0/3110841964 shutdown_connections 2024-08-18T23:44:08.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.017+0000 7f134ed6c640 1 -- 172.21.15.78:0/3110841964 wait complete. 2024-08-18T23:44:08.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f134ed6c640 1 Processor -- start 2024-08-18T23:44:08.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f134ed6c640 1 -- start start 2024-08-18T23:44:08.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f134ed6c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1348102ac0 0x7f1348198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:08.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f134ed6c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1348103470 0x7f13481987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:08.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f134ed6c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1348107fc0 0x7f134819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:08.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f134ed6c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1348111dc0 con 0x7f1348103470 2024-08-18T23:44:08.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f134ed6c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1348111c40 con 0x7f1348107fc0 2024-08-18T23:44:08.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f134ed6c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1348111f40 con 0x7f1348102ac0 2024-08-18T23:44:08.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f1347fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1348103470 0x7f13481987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:08.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f1347fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1348103470 0x7f13481987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35756/0 (socket says 172.21.15.78:35756) 2024-08-18T23:44:08.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f1347fff640 1 -- 172.21.15.78:0/2516786675 learned_addr learned my addr 172.21.15.78:0/2516786675 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:08.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f134cb08640 1 --2- 172.21.15.78:0/2516786675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1348102ac0 0x7f1348198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:08.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f134d309640 1 --2- 172.21.15.78:0/2516786675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1348107fc0 0x7f134819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:08.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f1347fff640 1 -- 172.21.15.78:0/2516786675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1348102ac0 msgr2=0x7f1348198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:08.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f1347fff640 1 --2- 172.21.15.78:0/2516786675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1348102ac0 0x7f1348198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f1347fff640 1 -- 172.21.15.78:0/2516786675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1348107fc0 msgr2=0x7f134819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:08.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f1347fff640 1 --2- 172.21.15.78:0/2516786675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1348107fc0 0x7f134819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f1347fff640 1 -- 172.21.15.78:0/2516786675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134819d240 con 0x7f1348103470 2024-08-18T23:44:08.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f134d309640 1 --2- 172.21.15.78:0/2516786675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1348107fc0 0x7f134819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:08.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.018+0000 7f134cb08640 1 --2- 172.21.15.78:0/2516786675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1348102ac0 0x7f1348198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:44:08.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.019+0000 7f1347fff640 1 --2- 172.21.15.78:0/2516786675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1348103470 0x7f13481987d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f133400e9c0 tx=0x7f133400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:08.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.019+0000 7f1345ffb640 1 -- 172.21.15.78:0/2516786675 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133400cd80 con 0x7f1348103470 2024-08-18T23:44:08.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.019+0000 7f1345ffb640 1 -- 172.21.15.78:0/2516786675 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1334005500 con 0x7f1348103470 2024-08-18T23:44:08.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.019+0000 7f134ed6c640 1 -- 172.21.15.78:0/2516786675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f134819d530 con 0x7f1348103470 2024-08-18T23:44:08.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.019+0000 7f1345ffb640 1 -- 172.21.15.78:0/2516786675 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334010690 con 0x7f1348103470 2024-08-18T23:44:08.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.019+0000 7f134ed6c640 1 -- 172.21.15.78:0/2516786675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13481a4e10 con 0x7f1348103470 2024-08-18T23:44:08.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.021+0000 7f1345ffb640 1 -- 172.21.15.78:0/2516786675 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f13340046c0 con 0x7f1348103470 2024-08-18T23:44:08.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f135116a740 1 Processor -- start 2024-08-18T23:44:08.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f135116a740 1 -- start start 2024-08-18T23:44:08.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f135116a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a42745e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:08.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f135116a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 0x55b4a427eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:08.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f1345ffb640 1 --2- 172.21.15.78:0/2516786675 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f13380a2c10 0x7f13380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:08.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f135116a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 0x55b4a42817d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:08.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f135116a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b4a4237e40 con 0x55b4a4274200 2024-08-18T23:44:08.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f135116a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b4a41d97f0 con 0x55b4a4274b20 2024-08-18T23:44:08.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f135116a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b4a4180fb0 con 0x55b4a427f3e0 2024-08-18T23:44:08.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f134d309640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a42745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:08.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f134d309640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a42745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35772/0 (socket says 172.21.15.78:35772) 2024-08-18T23:44:08.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f134d309640 1 -- 172.21.15.78:0/2937999240 learned_addr learned my addr 172.21.15.78:0/2937999240 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:08.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f1345ffb640 1 -- 172.21.15.78:0/2516786675 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f13340cf940 con 0x7f1348103470 2024-08-18T23:44:08.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f1347fff640 1 --2- 172.21.15.78:0/2516786675 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f13380a2c10 0x7f13380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:08.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f134cb08640 1 --2- 172.21.15.78:0/2937999240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 0x55b4a427eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:08.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f134d309640 1 --2- 172.21.15.78:0/2937999240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 0x55b4a42817d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:08.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f134d309640 1 -- 172.21.15.78:0/2937999240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 msgr2=0x55b4a42817d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:08.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f134d309640 1 --2- 172.21.15.78:0/2937999240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 0x55b4a42817d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f134d309640 1 -- 172.21.15.78:0/2937999240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 msgr2=0x55b4a427eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:08.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f134d309640 1 --2- 172.21.15.78:0/2937999240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 0x55b4a427eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.022+0000 7f134d309640 1 -- 172.21.15.78:0/2937999240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4a3f4b6a0 con 0x55b4a4274200 2024-08-18T23:44:08.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.023+0000 7f134d309640 1 --2- 172.21.15.78:0/2937999240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a42745e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55b4a4271b10 tx=0x7f1340005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63811191cf82873 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:08.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.023+0000 7f1347fff640 1 --2- 172.21.15.78:0/2516786675 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f13380a2c10 0x7f13380a50d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f133400c6a0 tx=0x7f133400efb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:08.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.023+0000 7f1332ffd640 1 -- 172.21.15.78:0/2937999240 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340030930 con 0x55b4a4274200 2024-08-18T23:44:08.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.023+0000 7f1332ffd640 1 -- 172.21.15.78:0/2937999240 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1340046070 con 0x55b4a4274200 2024-08-18T23:44:08.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.023+0000 7f1332ffd640 1 -- 172.21.15.78:0/2937999240 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13400044d0 con 0x55b4a4274200 2024-08-18T23:44:08.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.024+0000 7f135116a740 1 -- 172.21.15.78:0/2937999240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 msgr2=0x55b4a42745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:08.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.024+0000 7f135116a740 1 --2- 172.21.15.78:0/2937999240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a42745e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55b4a4271b10 tx=0x7f1340005d80 comp rx=0 tx=0).stop 2024-08-18T23:44:08.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.024+0000 7f135116a740 1 -- 172.21.15.78:0/2937999240 shutdown_connections 2024-08-18T23:44:08.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.024+0000 7f135116a740 1 --2- 172.21.15.78:0/2937999240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 0x55b4a42817d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.024+0000 7f135116a740 1 --2- 172.21.15.78:0/2937999240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 0x55b4a427eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.024+0000 7f135116a740 1 --2- 172.21.15.78:0/2937999240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a42745e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.024+0000 7f135116a740 1 -- 172.21.15.78:0/2937999240 >> 172.21.15.78:0/2937999240 conn(0x55b4a4270790 msgr2=0x55b4a4270b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:08.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.024+0000 7f135116a740 1 -- 172.21.15.78:0/2937999240 shutdown_connections 2024-08-18T23:44:08.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.024+0000 7f135116a740 1 -- 172.21.15.78:0/2937999240 wait complete. 2024-08-18T23:44:08.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.024+0000 7f135116a740 1 -- start start 2024-08-18T23:44:08.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.024+0000 7f135116a740 1 Processor -- start 2024-08-18T23:44:08.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f135116a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a427c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:08.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f135116a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 0x55b4a427c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:08.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f135116a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 0x55b4a427cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:08.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f135116a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b4a4194cb0 con 0x55b4a4274200 2024-08-18T23:44:08.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f135116a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b4a41d57b0 con 0x55b4a4274b20 2024-08-18T23:44:08.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f135116a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b4a4281fb0 con 0x55b4a427f3e0 2024-08-18T23:44:08.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f135116a740 10 client.0 _ll_register_callbacks cb 0x55b4a3ee8ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:08.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f134d309640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a427c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:08.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f134d309640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a427c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35788/0 (socket says 172.21.15.78:35788) 2024-08-18T23:44:08.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f134d309640 1 -- 172.21.15.78:0/2179444490 learned_addr learned my addr 172.21.15.78:0/2179444490 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:08.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f134cb08640 1 --2- 172.21.15.78:0/2179444490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 0x55b4a427c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:08.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f134d309640 1 --2- 172.21.15.78:0/2179444490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 0x55b4a427cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:08.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f134d309640 1 -- 172.21.15.78:0/2179444490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 msgr2=0x55b4a427cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:08.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f134d309640 1 --2- 172.21.15.78:0/2179444490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 0x55b4a427cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f134d309640 1 -- 172.21.15.78:0/2179444490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 msgr2=0x55b4a427c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:08.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f134d309640 1 --2- 172.21.15.78:0/2179444490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 0x55b4a427c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f134d309640 1 -- 172.21.15.78:0/2179444490 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4a4193bc0 con 0x55b4a4274200 2024-08-18T23:44:08.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f134cb08640 1 --2- 172.21.15.78:0/2179444490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 0x55b4a427c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:44:08.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f134d309640 1 --2- 172.21.15.78:0/2179444490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a427c290 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1340005e30 tx=0x7f1340004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fca7f278fb66eb server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:08.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.025+0000 7f1327fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:44:08.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.026+0000 7f1327fff640 1 -- 172.21.15.78:0/2179444490 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340003db0 con 0x55b4a4274200 2024-08-18T23:44:08.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.026+0000 7f1327fff640 1 -- 172.21.15.78:0/2179444490 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f1340004250 con 0x55b4a4274200 2024-08-18T23:44:08.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.026+0000 7f135116a740 1 -- 172.21.15.78:0/2179444490 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1340046070 con 0x55b4a4274200 2024-08-18T23:44:08.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.026+0000 7f1327fff640 1 -- 172.21.15.78:0/2179444490 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f134005e050 con 0x55b4a4274200 2024-08-18T23:44:08.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.026+0000 7f135116a740 10 client.15348 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:08.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.026+0000 7f135116a740 1 -- 172.21.15.78:0/2179444490 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b4a428de40 con 0x55b4a4274200 2024-08-18T23:44:08.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.026+0000 7f1327fff640 1 -- 172.21.15.78:0/2179444490 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1340046070 con 0x55b4a4274200 2024-08-18T23:44:08.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.026+0000 7f1327fff640 1 -- 172.21.15.78:0/2179444490 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1340031990 con 0x55b4a4274200 2024-08-18T23:44:08.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.026+0000 7f135116a740 10 client.15348 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:08.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.026+0000 7f135116a740 10 client.15348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:44:08.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.026+0000 7f135116a740 20 client.15348 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:08.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.026+0000 7f135116a740 1 --2- 172.21.15.78:0/2179444490 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b4a428f190 0x55b4a4291580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:08.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.026+0000 7f135116a740 4 client.15348 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:44:08.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.026+0000 7f135116a740 1 -- 172.21.15.78:0/2179444490 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4a41e6d60 con 0x55b4a428f190 2024-08-18T23:44:08.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.027+0000 7f134cb08640 1 --2- 172.21.15.78:0/2179444490 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b4a428f190 0x55b4a4291580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:08.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.027+0000 7f134cb08640 1 --2- 172.21.15.78:0/2179444490 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b4a428f190 0x55b4a4291580 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc33fbd140e6716c server_cookie=9f4200c2cf7a7045 in_seq=0 out_seq=0 2024-08-18T23:44:08.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.027+0000 7f1327fff640 10 client.15348 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:08.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.028+0000 7f1327fff640 1 -- 172.21.15.78:0/2179444490 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4a41e6d60 con 0x55b4a428f190 2024-08-18T23:44:08.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.028+0000 7f1327fff640 10 client.15348 handle_command_reply: tid=0 2024-08-18T23:44:08.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.028+0000 7f135116a740 1 client.15348 shutdown 2024-08-18T23:44:08.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.028+0000 7f135116a740 1 -- 172.21.15.78:0/2179444490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 msgr2=0x55b4a427c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:08.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.028+0000 7f135116a740 1 --2- 172.21.15.78:0/2179444490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a427c290 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1340005e30 tx=0x7f1340004170 comp rx=0 tx=0).stop 2024-08-18T23:44:08.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.028+0000 7f135116a740 1 -- 172.21.15.78:0/2179444490 shutdown_connections 2024-08-18T23:44:08.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.028+0000 7f135116a740 1 --2- 172.21.15.78:0/2179444490 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b4a428f190 0x55b4a4291580 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.028+0000 7f1327fff640 0 client.15348 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:08.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.028+0000 7f135116a740 1 --2- 172.21.15.78:0/2179444490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 0x55b4a427cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.028+0000 7f135116a740 1 --2- 172.21.15.78:0/2179444490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 0x55b4a427c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.028+0000 7f135116a740 1 --2- 172.21.15.78:0/2179444490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a427c290 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.028+0000 7f135116a740 1 -- 172.21.15.78:0/2179444490 >> 172.21.15.78:0/2179444490 conn(0x55b4a4270790 msgr2=0x55b4a42764c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:08.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.028+0000 7f135116a740 1 -- 172.21.15.78:0/2179444490 shutdown_connections 2024-08-18T23:44:08.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.028+0000 7f135116a740 1 -- 172.21.15.78:0/2179444490 wait complete. 2024-08-18T23:44:08.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.028+0000 7f135116a740 20 client.15348 trim_cache size 0 max 16384 2024-08-18T23:44:08.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.058+0000 7f135116a740 1 Processor -- start 2024-08-18T23:44:08.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.058+0000 7f135116a740 1 -- start start 2024-08-18T23:44:08.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.058+0000 7f135116a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a42794f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:08.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.058+0000 7f135116a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 0x55b4a4279a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:08.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.058+0000 7f135116a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 0x55b4a428a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:08.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.058+0000 7f135116a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13481b12d0 con 0x55b4a4274200 2024-08-18T23:44:08.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.058+0000 7f135116a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f13481aeba0 con 0x55b4a4274b20 2024-08-18T23:44:08.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.058+0000 7f135116a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f13481ac9f0 con 0x55b4a427f3e0 2024-08-18T23:44:08.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.058+0000 7f134d309640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a42794f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:08.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.058+0000 7f134d309640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a42794f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35798/0 (socket says 172.21.15.78:35798) 2024-08-18T23:44:08.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.058+0000 7f134d309640 1 -- 172.21.15.78:0/1586045622 learned_addr learned my addr 172.21.15.78:0/1586045622 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:08.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.058+0000 7f134cb08640 1 --2- 172.21.15.78:0/1586045622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 0x55b4a4279a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:08.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.058+0000 7f134d309640 1 --2- 172.21.15.78:0/1586045622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 0x55b4a428a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:08.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.058+0000 7f134d309640 1 -- 172.21.15.78:0/1586045622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 msgr2=0x55b4a428a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:08.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.058+0000 7f134d309640 1 --2- 172.21.15.78:0/1586045622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 0x55b4a428a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.058+0000 7f134d309640 1 -- 172.21.15.78:0/1586045622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 msgr2=0x55b4a4279a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:08.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.058+0000 7f134d309640 1 --2- 172.21.15.78:0/1586045622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 0x55b4a4279a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.058+0000 7f134d309640 1 -- 172.21.15.78:0/1586045622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4a4274950 con 0x55b4a4274200 2024-08-18T23:44:08.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.059+0000 7f134d309640 1 --2- 172.21.15.78:0/1586045622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a42794f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f13400096c0 tx=0x7f13400021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc33b64afd3e74a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:08.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.059+0000 7f13327fc640 1 -- 172.21.15.78:0/1586045622 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340067070 con 0x55b4a4274200 2024-08-18T23:44:08.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.059+0000 7f13327fc640 1 -- 172.21.15.78:0/1586045622 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1340046040 con 0x55b4a4274200 2024-08-18T23:44:08.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.059+0000 7f13327fc640 1 -- 172.21.15.78:0/1586045622 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13400047a0 con 0x55b4a4274200 2024-08-18T23:44:08.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.060+0000 7f135116a740 1 -- 172.21.15.78:0/1586045622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 msgr2=0x55b4a42794f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:08.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.060+0000 7f135116a740 1 --2- 172.21.15.78:0/1586045622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a42794f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f13400096c0 tx=0x7f13400021d0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.060+0000 7f135116a740 1 -- 172.21.15.78:0/1586045622 shutdown_connections 2024-08-18T23:44:08.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.060+0000 7f135116a740 1 --2- 172.21.15.78:0/1586045622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 0x55b4a428a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.060+0000 7f135116a740 1 --2- 172.21.15.78:0/1586045622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 0x55b4a4279a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.060+0000 7f135116a740 1 --2- 172.21.15.78:0/1586045622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a42794f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.060+0000 7f135116a740 1 -- 172.21.15.78:0/1586045622 >> 172.21.15.78:0/1586045622 conn(0x55b4a4270790 msgr2=0x55b4a427ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.060+0000 7f135116a740 1 -- 172.21.15.78:0/1586045622 shutdown_connections 2024-08-18T23:44:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.060+0000 7f135116a740 1 -- 172.21.15.78:0/1586045622 wait complete. 2024-08-18T23:44:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.060+0000 7f135116a740 1 -- start start 2024-08-18T23:44:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.060+0000 7f135116a740 1 Processor -- start 2024-08-18T23:44:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.061+0000 7f135116a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a4291390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.061+0000 7f135116a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 0x55b4a42918d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:08.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.061+0000 7f135116a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 0x55b4a4291e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:08.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.061+0000 7f135116a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13481ad060 con 0x55b4a4274200 2024-08-18T23:44:08.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.061+0000 7f135116a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f13481ae4d0 con 0x55b4a4274b20 2024-08-18T23:44:08.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.061+0000 7f135116a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f13481ad730 con 0x55b4a427f3e0 2024-08-18T23:44:08.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.061+0000 7f135116a740 10 client.0 _ll_register_callbacks cb 0x7f133c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:08.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.061+0000 7f134d309640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a4291390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:08.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.061+0000 7f134d309640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a4291390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35814/0 (socket says 172.21.15.78:35814) 2024-08-18T23:44:08.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.061+0000 7f134d309640 1 -- 172.21.15.78:0/4065727823 learned_addr learned my addr 172.21.15.78:0/4065727823 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:08.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.061+0000 7f134cb08640 1 --2- 172.21.15.78:0/4065727823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 0x55b4a42918d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:08.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.061+0000 7f134d309640 1 --2- 172.21.15.78:0/4065727823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 0x55b4a4291e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:08.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.061+0000 7f134d309640 1 -- 172.21.15.78:0/4065727823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 msgr2=0x55b4a4291e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:08.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.061+0000 7f134d309640 1 --2- 172.21.15.78:0/4065727823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 0x55b4a4291e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.061+0000 7f134d309640 1 -- 172.21.15.78:0/4065727823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 msgr2=0x55b4a42918d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:08.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.061+0000 7f134d309640 1 --2- 172.21.15.78:0/4065727823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 0x55b4a42918d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.062+0000 7f134d309640 1 -- 172.21.15.78:0/4065727823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4a42f7ac0 con 0x55b4a4274200 2024-08-18T23:44:08.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.062+0000 7f134d309640 1 --2- 172.21.15.78:0/4065727823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a4291390 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1340004480 tx=0x7f13400099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f484d2a6a633e494 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:08.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.062+0000 7f1327fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:44:08.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.062+0000 7f1327fff640 1 -- 172.21.15.78:0/4065727823 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340067040 con 0x55b4a4274200 2024-08-18T23:44:08.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.062+0000 7f1327fff640 1 -- 172.21.15.78:0/4065727823 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f1340004520 con 0x55b4a4274200 2024-08-18T23:44:08.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.062+0000 7f135116a740 1 -- 172.21.15.78:0/4065727823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1340046040 con 0x55b4a4274200 2024-08-18T23:44:08.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.062+0000 7f1327fff640 1 -- 172.21.15.78:0/4065727823 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1340004720 con 0x55b4a4274200 2024-08-18T23:44:08.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.062+0000 7f135116a740 10 client.15354 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:08.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.062+0000 7f135116a740 1 -- 172.21.15.78:0/4065727823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b4a42f8db0 con 0x55b4a4274200 2024-08-18T23:44:08.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.063+0000 7f1327fff640 1 -- 172.21.15.78:0/4065727823 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1340055070 con 0x55b4a4274200 2024-08-18T23:44:08.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.063+0000 7f1327fff640 1 -- 172.21.15.78:0/4065727823 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1340031440 con 0x55b4a4274200 2024-08-18T23:44:08.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.063+0000 7f135116a740 10 client.15354 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:08.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.063+0000 7f135116a740 10 client.15354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:44:08.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.063+0000 7f135116a740 20 client.15354 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:08.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.063+0000 7f135116a740 1 --2- 172.21.15.78:0/4065727823 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b4a428f190 0x55b4a42fb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:08.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.063+0000 7f135116a740 4 client.15354 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-18T23:44:08.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.063+0000 7f135116a740 1 -- 172.21.15.78:0/4065727823 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f1340046040 con 0x55b4a428f190 2024-08-18T23:44:08.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.063+0000 7f134cb08640 1 --2- 172.21.15.78:0/4065727823 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b4a428f190 0x55b4a42fb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:08.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.064+0000 7f1327fff640 10 client.15354 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:08.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.064+0000 7f134cb08640 1 --2- 172.21.15.78:0/4065727823 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b4a428f190 0x55b4a42fb150 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3c6a932000433a3 server_cookie=db38e4357b3f9b8 in_seq=0 out_seq=0 2024-08-18T23:44:08.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.065+0000 7f1327fff640 1 -- 172.21.15.78:0/4065727823 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f1340046040 con 0x55b4a428f190 2024-08-18T23:44:08.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.065+0000 7f1327fff640 10 client.15354 handle_command_reply: tid=0 2024-08-18T23:44:08.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.065+0000 7f135116a740 1 client.15354 shutdown 2024-08-18T23:44:08.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.065+0000 7f135116a740 1 -- 172.21.15.78:0/4065727823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 msgr2=0x55b4a4291390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:08.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.065+0000 7f135116a740 1 --2- 172.21.15.78:0/4065727823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a4291390 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1340004480 tx=0x7f13400099c0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.065+0000 7f135116a740 1 -- 172.21.15.78:0/4065727823 shutdown_connections 2024-08-18T23:44:08.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.065+0000 7f135116a740 1 --2- 172.21.15.78:0/4065727823 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b4a428f190 0x55b4a42fb150 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.065+0000 7f135116a740 1 --2- 172.21.15.78:0/4065727823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b4a427f3e0 0x55b4a4291e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.065+0000 7f135116a740 1 --2- 172.21.15.78:0/4065727823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a4274b20 0x55b4a42918d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.065+0000 7f135116a740 1 --2- 172.21.15.78:0/4065727823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b4a4274200 0x55b4a4291390 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.065+0000 7f135116a740 1 -- 172.21.15.78:0/4065727823 >> 172.21.15.78:0/4065727823 conn(0x55b4a4270790 msgr2=0x55b4a426f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:08.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.065+0000 7f1327fff640 0 client.15354 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:08.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.065+0000 7f135116a740 1 -- 172.21.15.78:0/4065727823 shutdown_connections 2024-08-18T23:44:08.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.065+0000 7f135116a740 1 -- 172.21.15.78:0/4065727823 wait complete. 2024-08-18T23:44:08.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.066+0000 7f135116a740 20 client.15354 trim_cache size 0 max 16384 2024-08-18T23:44:08.080 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:44:08.080 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-08-18T23:44:08.080 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "bac246f7-3ef7-435a-a19f-aa6f2c8cf376", 2024-08-18T23:44:08.080 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-08-18T23:44:08.080 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:44:08.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.066+0000 7f134ed6c640 1 -- 172.21.15.78:0/2516786675 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f13380a2c10 msgr2=0x7f13380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:08.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.066+0000 7f134ed6c640 1 --2- 172.21.15.78:0/2516786675 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f13380a2c10 0x7f13380a50d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f133400c6a0 tx=0x7f133400efb0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.066+0000 7f134ed6c640 1 -- 172.21.15.78:0/2516786675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1348103470 msgr2=0x7f13481987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:08.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.066+0000 7f134ed6c640 1 --2- 172.21.15.78:0/2516786675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1348103470 0x7f13481987d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f133400e9c0 tx=0x7f133400ee80 comp rx=0 tx=0).stop 2024-08-18T23:44:08.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.066+0000 7f134ed6c640 1 -- 172.21.15.78:0/2516786675 shutdown_connections 2024-08-18T23:44:08.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.066+0000 7f134ed6c640 1 --2- 172.21.15.78:0/2516786675 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f13380a2c10 0x7f13380a50d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.066+0000 7f134ed6c640 1 --2- 172.21.15.78:0/2516786675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1348107fc0 0x7f134819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.066+0000 7f134ed6c640 1 --2- 172.21.15.78:0/2516786675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1348103470 0x7f13481987d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.066+0000 7f134ed6c640 1 --2- 172.21.15.78:0/2516786675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1348102ac0 0x7f1348198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:08.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.066+0000 7f134ed6c640 1 -- 172.21.15.78:0/2516786675 >> 172.21.15.78:0/2516786675 conn(0x7f13480fc500 msgr2=0x7f13481060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:08.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.067+0000 7f134ed6c640 1 -- 172.21.15.78:0/2516786675 shutdown_connections 2024-08-18T23:44:08.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:08.067+0000 7f134ed6c640 1 -- 172.21.15.78:0/2516786675 wait complete. 2024-08-18T23:44:08.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:08 smithi078 ceph-mon[18440]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:44:08.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:08 smithi123 ceph-mon[21808]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:44:08.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:08 smithi161 ceph-mon[22027]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:44:08.895 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:08.914 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:44:09.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.402+0000 7f36e7381640 1 -- 172.21.15.78:0/1875090217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36e010be80 msgr2=0x7f36e010e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:09.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.402+0000 7f36e7381640 1 --2- 172.21.15.78:0/1875090217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36e010be80 0x7f36e010e320 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009ca0 tx=0x7f36d802a950 comp rx=0 tx=0).stop 2024-08-18T23:44:09.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.402+0000 7f36e7381640 1 -- 172.21.15.78:0/1875090217 shutdown_connections 2024-08-18T23:44:09.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.402+0000 7f36e7381640 1 --2- 172.21.15.78:0/1875090217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36e010be80 0x7f36e010e320 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:09.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.402+0000 7f36e7381640 1 --2- 172.21.15.78:0/1875090217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36e0103370 0x7f36e010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:09.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.402+0000 7f36e7381640 1 --2- 172.21.15.78:0/1875090217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36e0102a50 0x7f36e0102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:09.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.402+0000 7f36e7381640 1 -- 172.21.15.78:0/1875090217 >> 172.21.15.78:0/1875090217 conn(0x7f36e00fc4b0 msgr2=0x7f36e00fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:09.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.402+0000 7f36e7381640 1 -- 172.21.15.78:0/1875090217 shutdown_connections 2024-08-18T23:44:09.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.402+0000 7f36e7381640 1 -- 172.21.15.78:0/1875090217 wait complete. 2024-08-18T23:44:09.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.402+0000 7f36e7381640 1 Processor -- start 2024-08-18T23:44:09.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.402+0000 7f36e7381640 1 -- start start 2024-08-18T23:44:09.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.402+0000 7f36e7381640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36e0102a50 0x7f36e01981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:09.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.402+0000 7f36e7381640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36e0103370 0x7f36e0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:09.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36e7381640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36e010be80 0x7f36e019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:09.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36e7381640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f36e0111ce0 con 0x7f36e010be80 2024-08-18T23:44:09.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36e7381640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36e0111b60 con 0x7f36e0103370 2024-08-18T23:44:09.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36e7381640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f36e0111e60 con 0x7f36e0102a50 2024-08-18T23:44:09.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36e591e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36e010be80 0x7f36e019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:09.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36e591e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36e010be80 0x7f36e019cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35826/0 (socket says 172.21.15.78:35826) 2024-08-18T23:44:09.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36e591e640 1 -- 172.21.15.78:0/2634876407 learned_addr learned my addr 172.21.15.78:0/2634876407 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:09.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36e511d640 1 --2- 172.21.15.78:0/2634876407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36e0102a50 0x7f36e01981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:09.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36e491c640 1 --2- 172.21.15.78:0/2634876407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36e0103370 0x7f36e0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:09.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36e591e640 1 -- 172.21.15.78:0/2634876407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36e0102a50 msgr2=0x7f36e01981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:09.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36e591e640 1 --2- 172.21.15.78:0/2634876407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36e0102a50 0x7f36e01981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:09.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36e591e640 1 -- 172.21.15.78:0/2634876407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36e0103370 msgr2=0x7f36e0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:09.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36e591e640 1 --2- 172.21.15.78:0/2634876407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36e0103370 0x7f36e0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:09.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36e591e640 1 -- 172.21.15.78:0/2634876407 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e019d190 con 0x7f36e010be80 2024-08-18T23:44:09.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36e491c640 1 --2- 172.21.15.78:0/2634876407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36e0103370 0x7f36e0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:09.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36e591e640 1 --2- 172.21.15.78:0/2634876407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36e010be80 0x7f36e019cab0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f36d802ae70 tx=0x7f36d802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:09.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36d67fc640 1 -- 172.21.15.78:0/2634876407 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8034040 con 0x7f36e010be80 2024-08-18T23:44:09.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36e7381640 1 -- 172.21.15.78:0/2634876407 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e019d420 con 0x7f36e010be80 2024-08-18T23:44:09.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36d67fc640 1 -- 172.21.15.78:0/2634876407 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d8030920 con 0x7f36e010be80 2024-08-18T23:44:09.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36d67fc640 1 -- 172.21.15.78:0/2634876407 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8045480 con 0x7f36e010be80 2024-08-18T23:44:09.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.403+0000 7f36e7381640 1 -- 172.21.15.78:0/2634876407 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e01a4d60 con 0x7f36e010be80 2024-08-18T23:44:09.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.404+0000 7f36e7381640 1 -- 172.21.15.78:0/2634876407 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e006ada0 con 0x7f36e010be80 2024-08-18T23:44:09.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.405+0000 7f36d67fc640 1 -- 172.21.15.78:0/2634876407 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f36d8037020 con 0x7f36e010be80 2024-08-18T23:44:09.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.406+0000 7f36d67fc640 1 --2- 172.21.15.78:0/2634876407 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f36cc0a2c10 0x7f36cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:09.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.406+0000 7f36e511d640 1 --2- 172.21.15.78:0/2634876407 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f36cc0a2c10 0x7f36cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:09.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.406+0000 7f36d67fc640 1 -- 172.21.15.78:0/2634876407 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f36d80f03b0 con 0x7f36e010be80 2024-08-18T23:44:09.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.407+0000 7f36e511d640 1 --2- 172.21.15.78:0/2634876407 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f36cc0a2c10 0x7f36cc0a50d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f36dc005fd0 tx=0x7f36dc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:09.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.410+0000 7f36d67fc640 1 -- 172.21.15.78:0/2634876407 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f36d80bcf30 con 0x7f36e010be80 2024-08-18T23:44:09.430 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:09 smithi078 ceph-mon[18440]: scrub queued for path: / 2024-08-18T23:44:09.430 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:09 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:44:09.430 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:09 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:44:09.430 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:09 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:44:09.430 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:09 smithi078 ceph-mon[18440]: pgmap v345: 129 pgs: 129 active+clean; 651 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-08-18T23:44:09.465 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:09.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:09 smithi123 ceph-mon[21808]: scrub queued for path: / 2024-08-18T23:44:09.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:09 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:44:09.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:09 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:44:09.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:09 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:44:09.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:09 smithi123 ceph-mon[21808]: pgmap v345: 129 pgs: 129 active+clean; 651 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-08-18T23:44:09.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.587+0000 7f36e7381640 1 -- 172.21.15.78:0/2634876407 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f36e0106f90 con 0x7f36cc0a2c10 2024-08-18T23:44:09.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.595+0000 7f36d67fc640 1 -- 172.21.15.78:0/2634876407 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340488 (secure 0 0 0) 0x7f36dc003890 con 0x7f36cc0a2c10 2024-08-18T23:44:09.596 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:44:09.601 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-08-18T23:44:09.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.598+0000 7f36e7381640 1 -- 172.21.15.78:0/2634876407 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f36cc0a2c10 msgr2=0x7f36cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:09.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.598+0000 7f36e7381640 1 --2- 172.21.15.78:0/2634876407 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f36cc0a2c10 0x7f36cc0a50d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f36dc005fd0 tx=0x7f36dc00b390 comp rx=0 tx=0).stop 2024-08-18T23:44:09.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.598+0000 7f36e7381640 1 -- 172.21.15.78:0/2634876407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36e010be80 msgr2=0x7f36e019cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:09.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.598+0000 7f36e7381640 1 --2- 172.21.15.78:0/2634876407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36e010be80 0x7f36e019cab0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f36d802ae70 tx=0x7f36d802f5b0 comp rx=0 tx=0).stop 2024-08-18T23:44:09.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.598+0000 7f36e7381640 1 -- 172.21.15.78:0/2634876407 shutdown_connections 2024-08-18T23:44:09.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.598+0000 7f36e7381640 1 --2- 172.21.15.78:0/2634876407 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f36cc0a2c10 0x7f36cc0a50d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:09.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.598+0000 7f36e7381640 1 --2- 172.21.15.78:0/2634876407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36e010be80 0x7f36e019cab0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:09.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.598+0000 7f36e7381640 1 --2- 172.21.15.78:0/2634876407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36e0103370 0x7f36e0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:09.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.598+0000 7f36e7381640 1 --2- 172.21.15.78:0/2634876407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36e0102a50 0x7f36e01981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:09.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.598+0000 7f36e7381640 1 -- 172.21.15.78:0/2634876407 >> 172.21.15.78:0/2634876407 conn(0x7f36e00fc4b0 msgr2=0x7f36e00fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:09.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.599+0000 7f36e7381640 1 -- 172.21.15.78:0/2634876407 shutdown_connections 2024-08-18T23:44:09.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:09.599+0000 7f36e7381640 1 -- 172.21.15.78:0/2634876407 wait complete. 2024-08-18T23:44:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:09 smithi161 ceph-mon[22027]: scrub queued for path: / 2024-08-18T23:44:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:09 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:44:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:09 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:44:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:09 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:44:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:09 smithi161 ceph-mon[22027]: pgmap v345: 129 pgs: 129 active+clean; 651 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-08-18T23:44:10.434 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-08-18T23:44:08.148953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666749,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":865,"num_read_kb":558,"num_write":376,"num_write_kb":3083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":803,"ondisk_log_size":803,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392748,"kb_used_data":14456,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549076,"statfs":{"total":1151940427776,"available":1151538253824,"internally_reserved":0,"allocated":14802944,"data_stored":8261633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":41,"num_write":17,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003383"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298623+0000","last_change":"2024-08-18T23:41:30.120580+0000","last_active":"2024-08-18T23:43:24.298623+0000","last_peered":"2024-08-18T23:43:24.298623+0000","last_clean":"2024-08-18T23:43:24.298623+0000","last_became_active":"2024-08-18T23:41:30.120266+0000","last_became_peered":"2024-08-18T23:41:30.120266+0000","last_unstale":"2024-08-18T23:43:24.298623+0000","last_undegraded":"2024-08-18T23:43:24.298623+0000","last_fullsized":"2024-08-18T23:43:24.298623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:06:40.997070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312511+0000","last_change":"2024-08-18T23:41:32.143232+0000","last_active":"2024-08-18T23:43:24.312511+0000","last_peered":"2024-08-18T23:43:24.312511+0000","last_clean":"2024-08-18T23:43:24.312511+0000","last_became_active":"2024-08-18T23:41:32.142916+0000","last_became_peered":"2024-08-18T23:41:32.142916+0000","last_unstale":"2024-08-18T23:43:24.312511+0000","last_undegraded":"2024-08-18T23:43:24.312511+0000","last_fullsized":"2024-08-18T23:43:24.312511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:50:54.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312106+0000","last_change":"2024-08-18T23:41:39.188772+0000","last_active":"2024-08-18T23:43:24.312106+0000","last_peered":"2024-08-18T23:43:24.312106+0000","last_clean":"2024-08-18T23:43:24.312106+0000","last_became_active":"2024-08-18T23:41:39.188362+0000","last_became_peered":"2024-08-18T23:41:39.188362+0000","last_unstale":"2024-08-18T23:43:24.312106+0000","last_undegraded":"2024-08-18T23:43:24.312106+0000","last_fullsized":"2024-08-18T23:43:24.312106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:47:50.255543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312628+0000","last_change":"2024-08-18T23:41:30.137179+0000","last_active":"2024-08-18T23:43:24.312628+0000","last_peered":"2024-08-18T23:43:24.312628+0000","last_clean":"2024-08-18T23:43:24.312628+0000","last_became_active":"2024-08-18T23:41:30.136618+0000","last_became_peered":"2024-08-18T23:41:30.136618+0000","last_unstale":"2024-08-18T23:43:24.312628+0000","last_undegraded":"2024-08-18T23:43:24.312628+0000","last_fullsized":"2024-08-18T23:43:24.312628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:59.826364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312531+0000","last_change":"2024-08-18T23:41:32.142075+0000","last_active":"2024-08-18T23:43:24.312531+0000","last_peered":"2024-08-18T23:43:24.312531+0000","last_clean":"2024-08-18T23:43:24.312531+0000","last_became_active":"2024-08-18T23:41:32.141709+0000","last_became_peered":"2024-08-18T23:41:32.141709+0000","last_unstale":"2024-08-18T23:43:24.312531+0000","last_undegraded":"2024-08-18T23:43:24.312531+0000","last_fullsized":"2024-08-18T23:43:24.312531+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:08.324627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312210+0000","last_change":"2024-08-18T23:41:39.184575+0000","last_active":"2024-08-18T23:43:24.312210+0000","last_peered":"2024-08-18T23:43:24.312210+0000","last_clean":"2024-08-18T23:43:24.312210+0000","last_became_active":"2024-08-18T23:41:39.184169+0000","last_became_peered":"2024-08-18T23:41:39.184169+0000","last_unstale":"2024-08-18T23:43:24.312210+0000","last_undegraded":"2024-08-18T23:43:24.312210+0000","last_fullsized":"2024-08-18T23:43:24.312210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:32.218172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304127+0000","last_change":"2024-08-18T23:41:30.121638+0000","last_active":"2024-08-18T23:43:24.304127+0000","last_peered":"2024-08-18T23:43:24.304127+0000","last_clean":"2024-08-18T23:43:24.304127+0000","last_became_active":"2024-08-18T23:41:30.121290+0000","last_became_peered":"2024-08-18T23:41:30.121290+0000","last_unstale":"2024-08-18T23:43:24.304127+0000","last_undegraded":"2024-08-18T23:43:24.304127+0000","last_fullsized":"2024-08-18T23:43:24.304127+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:43:55.638780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312431+0000","last_change":"2024-08-18T23:41:32.142177+0000","last_active":"2024-08-18T23:43:24.312431+0000","last_peered":"2024-08-18T23:43:24.312431+0000","last_clean":"2024-08-18T23:43:24.312431+0000","last_became_active":"2024-08-18T23:41:32.141731+0000","last_became_peered":"2024-08-18T23:41:32.141731+0000","last_unstale":"2024-08-18T23:43:24.312431+0000","last_undegraded":"2024-08-18T23:43:24.312431+0000","last_fullsized":"2024-08-18T23:43:24.312431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:36.283299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312604+0000","last_change":"2024-08-18T23:41:39.182931+0000","last_active":"2024-08-18T23:43:24.312604+0000","last_peered":"2024-08-18T23:43:24.312604+0000","last_clean":"2024-08-18T23:43:24.312604+0000","last_became_active":"2024-08-18T23:41:39.182008+0000","last_became_peered":"2024-08-18T23:41:39.182008+0000","last_unstale":"2024-08-18T23:43:24.312604+0000","last_undegraded":"2024-08-18T23:43:24.312604+0000","last_fullsized":"2024-08-18T23:43:24.312604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:58:38.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312863+0000","last_change":"2024-08-18T23:42:23.329762+0000","last_active":"2024-08-18T23:43:24.312863+0000","last_peered":"2024-08-18T23:43:24.312863+0000","last_clean":"2024-08-18T23:43:24.312863+0000","last_became_active":"2024-08-18T23:42:23.325753+0000","last_became_peered":"2024-08-18T23:42:23.325753+0000","last_unstale":"2024-08-18T23:43:24.312863+0000","last_undegraded":"2024-08-18T23:43:24.312863+0000","last_fullsized":"2024-08-18T23:43:24.312863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:19:43.062492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307318+0000","last_change":"2024-08-18T23:41:32.148662+0000","last_active":"2024-08-18T23:43:24.307318+0000","last_peered":"2024-08-18T23:43:24.307318+0000","last_clean":"2024-08-18T23:43:24.307318+0000","last_became_active":"2024-08-18T23:41:32.148318+0000","last_became_peered":"2024-08-18T23:41:32.148318+0000","last_unstale":"2024-08-18T23:43:24.307318+0000","last_undegraded":"2024-08-18T23:43:24.307318+0000","last_fullsized":"2024-08-18T23:43:24.307318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:58.668669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312238+0000","last_change":"2024-08-18T23:41:39.181114+0000","last_active":"2024-08-18T23:43:24.312238+0000","last_peered":"2024-08-18T23:43:24.312238+0000","last_clean":"2024-08-18T23:43:24.312238+0000","last_became_active":"2024-08-18T23:41:39.180599+0000","last_became_peered":"2024-08-18T23:41:39.180599+0000","last_unstale":"2024-08-18T23:43:24.312238+0000","last_undegraded":"2024-08-18T23:43:24.312238+0000","last_fullsized":"2024-08-18T23:43:24.312238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:36:08.749143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304715+0000","last_change":"2024-08-18T23:41:32.144326+0000","last_active":"2024-08-18T23:43:24.304715+0000","last_peered":"2024-08-18T23:43:24.304715+0000","last_clean":"2024-08-18T23:43:24.304715+0000","last_became_active":"2024-08-18T23:41:32.144004+0000","last_became_peered":"2024-08-18T23:41:32.144004+0000","last_unstale":"2024-08-18T23:43:24.304715+0000","last_undegraded":"2024-08-18T23:43:24.304715+0000","last_fullsized":"2024-08-18T23:43:24.304715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:10.918624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303665+0000","last_change":"2024-08-18T23:41:30.125160+0000","last_active":"2024-08-18T23:43:24.303665+0000","last_peered":"2024-08-18T23:43:24.303665+0000","last_clean":"2024-08-18T23:43:24.303665+0000","last_became_active":"2024-08-18T23:41:30.124685+0000","last_became_peered":"2024-08-18T23:41:30.124685+0000","last_unstale":"2024-08-18T23:43:24.303665+0000","last_undegraded":"2024-08-18T23:43:24.303665+0000","last_fullsized":"2024-08-18T23:43:24.303665+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:52:48.997902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313216+0000","last_change":"2024-08-18T23:41:39.681481+0000","last_active":"2024-08-18T23:43:24.313216+0000","last_peered":"2024-08-18T23:43:24.313216+0000","last_clean":"2024-08-18T23:43:24.313216+0000","last_became_active":"2024-08-18T23:41:39.680204+0000","last_became_peered":"2024-08-18T23:41:39.680204+0000","last_unstale":"2024-08-18T23:43:24.313216+0000","last_undegraded":"2024-08-18T23:43:24.313216+0000","last_fullsized":"2024-08-18T23:43:24.313216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:00:05.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312231+0000","last_change":"2024-08-18T23:41:32.151335+0000","last_active":"2024-08-18T23:43:24.312231+0000","last_peered":"2024-08-18T23:43:24.312231+0000","last_clean":"2024-08-18T23:43:24.312231+0000","last_became_active":"2024-08-18T23:41:32.151040+0000","last_became_peered":"2024-08-18T23:41:32.151040+0000","last_unstale":"2024-08-18T23:43:24.312231+0000","last_undegraded":"2024-08-18T23:43:24.312231+0000","last_fullsized":"2024-08-18T23:43:24.312231+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:16.108555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"92'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:41.164918+0000","last_change":"2024-08-18T23:41:30.122301+0000","last_active":"2024-08-18T23:43:41.164918+0000","last_peered":"2024-08-18T23:43:41.164918+0000","last_clean":"2024-08-18T23:43:41.164918+0000","last_became_active":"2024-08-18T23:41:30.121117+0000","last_became_peered":"2024-08-18T23:41:30.121117+0000","last_unstale":"2024-08-18T23:43:41.164918+0000","last_undegraded":"2024-08-18T23:43:41.164918+0000","last_fullsized":"2024-08-18T23:43:41.164918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:40:12.678803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312721+0000","last_change":"2024-08-18T23:41:39.182395+0000","last_active":"2024-08-18T23:43:24.312721+0000","last_peered":"2024-08-18T23:43:24.312721+0000","last_clean":"2024-08-18T23:43:24.312721+0000","last_became_active":"2024-08-18T23:41:39.181979+0000","last_became_peered":"2024-08-18T23:41:39.181979+0000","last_unstale":"2024-08-18T23:43:24.312721+0000","last_undegraded":"2024-08-18T23:43:24.312721+0000","last_fullsized":"2024-08-18T23:43:24.312721+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:04.674503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312810+0000","last_change":"2024-08-18T23:41:32.140464+0000","last_active":"2024-08-18T23:43:24.312810+0000","last_peered":"2024-08-18T23:43:24.312810+0000","last_clean":"2024-08-18T23:43:24.312810+0000","last_became_active":"2024-08-18T23:41:32.139153+0000","last_became_peered":"2024-08-18T23:41:32.139153+0000","last_unstale":"2024-08-18T23:43:24.312810+0000","last_undegraded":"2024-08-18T23:43:24.312810+0000","last_fullsized":"2024-08-18T23:43:24.312810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:27.387680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312414+0000","last_change":"2024-08-18T23:41:30.122787+0000","last_active":"2024-08-18T23:43:24.312414+0000","last_peered":"2024-08-18T23:43:24.312414+0000","last_clean":"2024-08-18T23:43:24.312414+0000","last_became_active":"2024-08-18T23:41:30.122462+0000","last_became_peered":"2024-08-18T23:41:30.122462+0000","last_unstale":"2024-08-18T23:43:24.312414+0000","last_undegraded":"2024-08-18T23:43:24.312414+0000","last_fullsized":"2024-08-18T23:43:24.312414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:55.568831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298325+0000","last_change":"2024-08-18T23:41:39.190511+0000","last_active":"2024-08-18T23:43:24.298325+0000","last_peered":"2024-08-18T23:43:24.298325+0000","last_clean":"2024-08-18T23:43:24.298325+0000","last_became_active":"2024-08-18T23:41:39.190154+0000","last_became_peered":"2024-08-18T23:41:39.190154+0000","last_unstale":"2024-08-18T23:43:24.298325+0000","last_undegraded":"2024-08-18T23:43:24.298325+0000","last_fullsized":"2024-08-18T23:43:24.298325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:33:34.514827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303456+0000","last_change":"2024-08-18T23:41:32.143867+0000","last_active":"2024-08-18T23:43:24.303456+0000","last_peered":"2024-08-18T23:43:24.303456+0000","last_clean":"2024-08-18T23:43:24.303456+0000","last_became_active":"2024-08-18T23:41:32.143527+0000","last_became_peered":"2024-08-18T23:41:32.143527+0000","last_unstale":"2024-08-18T23:43:24.303456+0000","last_undegraded":"2024-08-18T23:43:24.303456+0000","last_fullsized":"2024-08-18T23:43:24.303456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:40:25.902133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312100+0000","last_change":"2024-08-18T23:42:23.325102+0000","last_active":"2024-08-18T23:43:24.312100+0000","last_peered":"2024-08-18T23:43:24.312100+0000","last_clean":"2024-08-18T23:43:24.312100+0000","last_became_active":"2024-08-18T23:42:23.321583+0000","last_became_peered":"2024-08-18T23:42:23.321583+0000","last_unstale":"2024-08-18T23:43:24.312100+0000","last_undegraded":"2024-08-18T23:43:24.312100+0000","last_fullsized":"2024-08-18T23:43:24.312100+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:57:42.546686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312767+0000","last_change":"2024-08-18T23:42:23.559756+0000","last_active":"2024-08-18T23:43:24.312767+0000","last_peered":"2024-08-18T23:43:24.312767+0000","last_clean":"2024-08-18T23:43:24.312767+0000","last_became_active":"2024-08-18T23:42:23.559242+0000","last_became_peered":"2024-08-18T23:42:23.559242+0000","last_unstale":"2024-08-18T23:43:24.312767+0000","last_undegraded":"2024-08-18T23:43:24.312767+0000","last_fullsized":"2024-08-18T23:43:24.312767+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:25:40.790313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312043+0000","last_change":"2024-08-18T23:41:32.139025+0000","last_active":"2024-08-18T23:43:24.312043+0000","last_peered":"2024-08-18T23:43:24.312043+0000","last_clean":"2024-08-18T23:43:24.312043+0000","last_became_active":"2024-08-18T23:41:32.138733+0000","last_became_peered":"2024-08-18T23:41:32.138733+0000","last_unstale":"2024-08-18T23:43:24.312043+0000","last_undegraded":"2024-08-18T23:43:24.312043+0000","last_fullsized":"2024-08-18T23:43:24.312043+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:01:39.826391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313061+0000","last_change":"2024-08-18T23:41:30.477359+0000","last_active":"2024-08-18T23:43:24.313061+0000","last_peered":"2024-08-18T23:43:24.313061+0000","last_clean":"2024-08-18T23:43:24.313061+0000","last_became_active":"2024-08-18T23:41:30.477011+0000","last_became_peered":"2024-08-18T23:41:30.477011+0000","last_unstale":"2024-08-18T23:43:24.313061+0000","last_undegraded":"2024-08-18T23:43:24.313061+0000","last_fullsized":"2024-08-18T23:43:24.313061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:53:10.052859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312145+0000","last_change":"2024-08-18T23:41:39.182140+0000","last_active":"2024-08-18T23:43:24.312145+0000","last_peered":"2024-08-18T23:43:24.312145+0000","last_clean":"2024-08-18T23:43:24.312145+0000","last_became_active":"2024-08-18T23:41:39.181667+0000","last_became_peered":"2024-08-18T23:41:39.181667+0000","last_unstale":"2024-08-18T23:43:24.312145+0000","last_undegraded":"2024-08-18T23:43:24.312145+0000","last_fullsized":"2024-08-18T23:43:24.312145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:02:18.128028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312594+0000","last_change":"2024-08-18T23:41:32.137064+0000","last_active":"2024-08-18T23:43:24.312594+0000","last_peered":"2024-08-18T23:43:24.312594+0000","last_clean":"2024-08-18T23:43:24.312594+0000","last_became_active":"2024-08-18T23:41:32.136738+0000","last_became_peered":"2024-08-18T23:41:32.136738+0000","last_unstale":"2024-08-18T23:43:24.312594+0000","last_undegraded":"2024-08-18T23:43:24.312594+0000","last_fullsized":"2024-08-18T23:43:24.312594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:15:11.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'14","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:57.167799+0000","last_change":"2024-08-18T23:41:30.120042+0000","last_active":"2024-08-18T23:43:57.167799+0000","last_peered":"2024-08-18T23:43:57.167799+0000","last_clean":"2024-08-18T23:43:57.167799+0000","last_became_active":"2024-08-18T23:41:30.119719+0000","last_became_peered":"2024-08-18T23:41:30.119719+0000","last_unstale":"2024-08-18T23:43:57.167799+0000","last_undegraded":"2024-08-18T23:43:57.167799+0000","last_fullsized":"2024-08-18T23:43:57.167799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:35:44.466330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311900+0000","last_change":"2024-08-18T23:41:39.680405+0000","last_active":"2024-08-18T23:43:24.311900+0000","last_peered":"2024-08-18T23:43:24.311900+0000","last_clean":"2024-08-18T23:43:24.311900+0000","last_became_active":"2024-08-18T23:41:39.679940+0000","last_became_peered":"2024-08-18T23:41:39.679940+0000","last_unstale":"2024-08-18T23:43:24.311900+0000","last_undegraded":"2024-08-18T23:43:24.311900+0000","last_fullsized":"2024-08-18T23:43:24.311900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:15:03.942116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301271+0000","last_change":"2024-08-18T23:41:32.139688+0000","last_active":"2024-08-18T23:43:24.301271+0000","last_peered":"2024-08-18T23:43:24.301271+0000","last_clean":"2024-08-18T23:43:24.301271+0000","last_became_active":"2024-08-18T23:41:32.139397+0000","last_became_peered":"2024-08-18T23:41:32.139397+0000","last_unstale":"2024-08-18T23:43:24.301271+0000","last_undegraded":"2024-08-18T23:43:24.301271+0000","last_fullsized":"2024-08-18T23:43:24.301271+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:03:57.017488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304150+0000","last_change":"2024-08-18T23:41:30.137578+0000","last_active":"2024-08-18T23:43:24.304150+0000","last_peered":"2024-08-18T23:43:24.304150+0000","last_clean":"2024-08-18T23:43:24.304150+0000","last_became_active":"2024-08-18T23:41:30.137246+0000","last_became_peered":"2024-08-18T23:41:30.137246+0000","last_unstale":"2024-08-18T23:43:24.304150+0000","last_undegraded":"2024-08-18T23:43:24.304150+0000","last_fullsized":"2024-08-18T23:43:24.304150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:48:54.839432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312576+0000","last_change":"2024-08-18T23:41:39.186360+0000","last_active":"2024-08-18T23:43:24.312576+0000","last_peered":"2024-08-18T23:43:24.312576+0000","last_clean":"2024-08-18T23:43:24.312576+0000","last_became_active":"2024-08-18T23:41:39.185808+0000","last_became_peered":"2024-08-18T23:41:39.185808+0000","last_unstale":"2024-08-18T23:43:24.312576+0000","last_undegraded":"2024-08-18T23:43:24.312576+0000","last_fullsized":"2024-08-18T23:43:24.312576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:53.891061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300916+0000","last_change":"2024-08-18T23:41:32.146070+0000","last_active":"2024-08-18T23:43:24.300916+0000","last_peered":"2024-08-18T23:43:24.300916+0000","last_clean":"2024-08-18T23:43:24.300916+0000","last_became_active":"2024-08-18T23:41:32.145640+0000","last_became_peered":"2024-08-18T23:41:32.145640+0000","last_unstale":"2024-08-18T23:43:24.300916+0000","last_undegraded":"2024-08-18T23:43:24.300916+0000","last_fullsized":"2024-08-18T23:43:24.300916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:56:21.368762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298278+0000","last_change":"2024-08-18T23:41:30.138565+0000","last_active":"2024-08-18T23:43:24.298278+0000","last_peered":"2024-08-18T23:43:24.298278+0000","last_clean":"2024-08-18T23:43:24.298278+0000","last_became_active":"2024-08-18T23:41:30.138210+0000","last_became_peered":"2024-08-18T23:41:30.138210+0000","last_unstale":"2024-08-18T23:43:24.298278+0000","last_undegraded":"2024-08-18T23:43:24.298278+0000","last_fullsized":"2024-08-18T23:43:24.298278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:42:14.928884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303425+0000","last_change":"2024-08-18T23:41:39.680656+0000","last_active":"2024-08-18T23:43:24.303425+0000","last_peered":"2024-08-18T23:43:24.303425+0000","last_clean":"2024-08-18T23:43:24.303425+0000","last_became_active":"2024-08-18T23:41:39.680203+0000","last_became_peered":"2024-08-18T23:41:39.680203+0000","last_unstale":"2024-08-18T23:43:24.303425+0000","last_undegraded":"2024-08-18T23:43:24.303425+0000","last_fullsized":"2024-08-18T23:43:24.303425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:55:43.138869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300932+0000","last_change":"2024-08-18T23:41:32.130222+0000","last_active":"2024-08-18T23:43:24.300932+0000","last_peered":"2024-08-18T23:43:24.300932+0000","last_clean":"2024-08-18T23:43:24.300932+0000","last_became_active":"2024-08-18T23:41:32.129844+0000","last_became_peered":"2024-08-18T23:41:32.129844+0000","last_unstale":"2024-08-18T23:43:24.300932+0000","last_undegraded":"2024-08-18T23:43:24.300932+0000","last_fullsized":"2024-08-18T23:43:24.300932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:39:21.106343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304850+0000","last_change":"2024-08-18T23:41:30.123142+0000","last_active":"2024-08-18T23:43:24.304850+0000","last_peered":"2024-08-18T23:43:24.304850+0000","last_clean":"2024-08-18T23:43:24.304850+0000","last_became_active":"2024-08-18T23:41:30.121645+0000","last_became_peered":"2024-08-18T23:41:30.121645+0000","last_unstale":"2024-08-18T23:43:24.304850+0000","last_undegraded":"2024-08-18T23:43:24.304850+0000","last_fullsized":"2024-08-18T23:43:24.304850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:07:24.912259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301163+0000","last_change":"2024-08-18T23:41:39.186033+0000","last_active":"2024-08-18T23:43:24.301163+0000","last_peered":"2024-08-18T23:43:24.301163+0000","last_clean":"2024-08-18T23:43:24.301163+0000","last_became_active":"2024-08-18T23:41:39.185451+0000","last_became_peered":"2024-08-18T23:41:39.185451+0000","last_unstale":"2024-08-18T23:43:24.301163+0000","last_undegraded":"2024-08-18T23:43:24.301163+0000","last_fullsized":"2024-08-18T23:43:24.301163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:52:13.556760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.306993+0000","last_change":"2024-08-18T23:41:32.141065+0000","last_active":"2024-08-18T23:43:24.306993+0000","last_peered":"2024-08-18T23:43:24.306993+0000","last_clean":"2024-08-18T23:43:24.306993+0000","last_became_active":"2024-08-18T23:41:32.140732+0000","last_became_peered":"2024-08-18T23:41:32.140732+0000","last_unstale":"2024-08-18T23:43:24.306993+0000","last_undegraded":"2024-08-18T23:43:24.306993+0000","last_fullsized":"2024-08-18T23:43:24.306993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:08:32.287835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":161,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:44:02.435987+0000","last_change":"2024-08-18T23:41:30.136838+0000","last_active":"2024-08-18T23:44:02.435987+0000","last_peered":"2024-08-18T23:44:02.435987+0000","last_clean":"2024-08-18T23:44:02.435987+0000","last_became_active":"2024-08-18T23:41:30.136523+0000","last_became_peered":"2024-08-18T23:41:30.136523+0000","last_unstale":"2024-08-18T23:44:02.435987+0000","last_undegraded":"2024-08-18T23:44:02.435987+0000","last_fullsized":"2024-08-18T23:44:02.435987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:56:17.750535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301550+0000","last_change":"2024-08-18T23:41:39.680314+0000","last_active":"2024-08-18T23:43:24.301550+0000","last_peered":"2024-08-18T23:43:24.301550+0000","last_clean":"2024-08-18T23:43:24.301550+0000","last_became_active":"2024-08-18T23:41:39.679920+0000","last_became_peered":"2024-08-18T23:41:39.679920+0000","last_unstale":"2024-08-18T23:43:24.301550+0000","last_undegraded":"2024-08-18T23:43:24.301550+0000","last_fullsized":"2024-08-18T23:43:24.301550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:26:29.866081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300888+0000","last_change":"2024-08-18T23:41:32.139752+0000","last_active":"2024-08-18T23:43:24.300888+0000","last_peered":"2024-08-18T23:43:24.300888+0000","last_clean":"2024-08-18T23:43:24.300888+0000","last_became_active":"2024-08-18T23:41:32.139451+0000","last_became_peered":"2024-08-18T23:41:32.139451+0000","last_unstale":"2024-08-18T23:43:24.300888+0000","last_undegraded":"2024-08-18T23:43:24.300888+0000","last_fullsized":"2024-08-18T23:43:24.300888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:15:13.390529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304894+0000","last_change":"2024-08-18T23:41:30.126513+0000","last_active":"2024-08-18T23:43:24.304894+0000","last_peered":"2024-08-18T23:43:24.304894+0000","last_clean":"2024-08-18T23:43:24.304894+0000","last_became_active":"2024-08-18T23:41:30.126046+0000","last_became_peered":"2024-08-18T23:41:30.126046+0000","last_unstale":"2024-08-18T23:43:24.304894+0000","last_undegraded":"2024-08-18T23:43:24.304894+0000","last_fullsized":"2024-08-18T23:43:24.304894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:49:16.565565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307280+0000","last_change":"2024-08-18T23:41:39.681006+0000","last_active":"2024-08-18T23:43:24.307280+0000","last_peered":"2024-08-18T23:43:24.307280+0000","last_clean":"2024-08-18T23:43:24.307280+0000","last_became_active":"2024-08-18T23:41:39.680409+0000","last_became_peered":"2024-08-18T23:41:39.680409+0000","last_unstale":"2024-08-18T23:43:24.307280+0000","last_undegraded":"2024-08-18T23:43:24.307280+0000","last_fullsized":"2024-08-18T23:43:24.307280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:47.778866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312261+0000","last_change":"2024-08-18T23:41:32.147545+0000","last_active":"2024-08-18T23:43:24.312261+0000","last_peered":"2024-08-18T23:43:24.312261+0000","last_clean":"2024-08-18T23:43:24.312261+0000","last_became_active":"2024-08-18T23:41:32.147238+0000","last_became_peered":"2024-08-18T23:41:32.147238+0000","last_unstale":"2024-08-18T23:43:24.312261+0000","last_undegraded":"2024-08-18T23:43:24.312261+0000","last_fullsized":"2024-08-18T23:43:24.312261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:12:13.589535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313054+0000","last_change":"2024-08-18T23:41:30.120650+0000","last_active":"2024-08-18T23:43:24.313054+0000","last_peered":"2024-08-18T23:43:24.313054+0000","last_clean":"2024-08-18T23:43:24.313054+0000","last_became_active":"2024-08-18T23:41:30.120196+0000","last_became_peered":"2024-08-18T23:41:30.120196+0000","last_unstale":"2024-08-18T23:43:24.313054+0000","last_undegraded":"2024-08-18T23:43:24.313054+0000","last_fullsized":"2024-08-18T23:43:24.313054+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:40:23.370088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304710+0000","last_change":"2024-08-18T23:40:22.013519+0000","last_active":"2024-08-18T23:43:24.304710+0000","last_peered":"2024-08-18T23:43:24.304710+0000","last_clean":"2024-08-18T23:43:24.304710+0000","last_became_active":"2024-08-18T23:40:22.012986+0000","last_became_peered":"2024-08-18T23:40:22.012986+0000","last_unstale":"2024-08-18T23:43:24.304710+0000","last_undegraded":"2024-08-18T23:43:24.304710+0000","last_fullsized":"2024-08-18T23:43:24.304710+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_clean_scrub_stamp":"2024-08-18T23:37:22.388424+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:19.874356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304893+0000","last_change":"2024-08-18T23:42:23.332800+0000","last_active":"2024-08-18T23:43:24.304893+0000","last_peered":"2024-08-18T23:43:24.304893+0000","last_clean":"2024-08-18T23:43:24.304893+0000","last_became_active":"2024-08-18T23:42:23.326441+0000","last_became_peered":"2024-08-18T23:42:23.326441+0000","last_unstale":"2024-08-18T23:43:24.304893+0000","last_undegraded":"2024-08-18T23:43:24.304893+0000","last_fullsized":"2024-08-18T23:43:24.304893+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:13:58.243518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.305065+0000","last_change":"2024-08-18T23:41:32.139907+0000","last_active":"2024-08-18T23:43:24.305065+0000","last_peered":"2024-08-18T23:43:24.305065+0000","last_clean":"2024-08-18T23:43:24.305065+0000","last_became_active":"2024-08-18T23:41:32.137956+0000","last_became_peered":"2024-08-18T23:41:32.137956+0000","last_unstale":"2024-08-18T23:43:24.305065+0000","last_undegraded":"2024-08-18T23:43:24.305065+0000","last_fullsized":"2024-08-18T23:43:24.305065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:10:13.223328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312461+0000","last_change":"2024-08-18T23:42:23.330062+0000","last_active":"2024-08-18T23:43:24.312461+0000","last_peered":"2024-08-18T23:43:24.312461+0000","last_clean":"2024-08-18T23:43:24.312461+0000","last_became_active":"2024-08-18T23:42:23.323196+0000","last_became_peered":"2024-08-18T23:42:23.323196+0000","last_unstale":"2024-08-18T23:43:24.312461+0000","last_undegraded":"2024-08-18T23:43:24.312461+0000","last_fullsized":"2024-08-18T23:43:24.312461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:53:29.059542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312185+0000","last_change":"2024-08-18T23:41:39.185917+0000","last_active":"2024-08-18T23:43:24.312185+0000","last_peered":"2024-08-18T23:43:24.312185+0000","last_clean":"2024-08-18T23:43:24.312185+0000","last_became_active":"2024-08-18T23:41:39.185427+0000","last_became_peered":"2024-08-18T23:41:39.185427+0000","last_unstale":"2024-08-18T23:43:24.312185+0000","last_undegraded":"2024-08-18T23:43:24.312185+0000","last_fullsized":"2024-08-18T23:43:24.312185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:52:13.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.308822+0000","last_change":"2024-08-18T23:43:24.308401+0000","last_active":"2024-08-18T23:43:24.308822+0000","last_peered":"2024-08-18T23:43:24.308822+0000","last_clean":"2024-08-18T23:43:24.308822+0000","last_became_active":"2024-08-18T23:43:24.308081+0000","last_became_peered":"2024-08-18T23:43:24.308081+0000","last_unstale":"2024-08-18T23:43:24.308822+0000","last_undegraded":"2024-08-18T23:43:24.308822+0000","last_fullsized":"2024-08-18T23:43:24.308822+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:28:57.712217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307528+0000","last_change":"2024-08-18T23:41:30.121307+0000","last_active":"2024-08-18T23:43:24.307528+0000","last_peered":"2024-08-18T23:43:24.307528+0000","last_clean":"2024-08-18T23:43:24.307528+0000","last_became_active":"2024-08-18T23:41:30.120932+0000","last_became_peered":"2024-08-18T23:41:30.120932+0000","last_unstale":"2024-08-18T23:43:24.307528+0000","last_undegraded":"2024-08-18T23:43:24.307528+0000","last_fullsized":"2024-08-18T23:43:24.307528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:06:07.742880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311751+0000","last_change":"2024-08-18T23:41:39.184647+0000","last_active":"2024-08-18T23:43:24.311751+0000","last_peered":"2024-08-18T23:43:24.311751+0000","last_clean":"2024-08-18T23:43:24.311751+0000","last_became_active":"2024-08-18T23:41:39.184251+0000","last_became_peered":"2024-08-18T23:41:39.184251+0000","last_unstale":"2024-08-18T23:43:24.311751+0000","last_undegraded":"2024-08-18T23:43:24.311751+0000","last_fullsized":"2024-08-18T23:43:24.311751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:55.200801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.305184+0000","last_change":"2024-08-18T23:41:32.137007+0000","last_active":"2024-08-18T23:43:24.305184+0000","last_peered":"2024-08-18T23:43:24.305184+0000","last_clean":"2024-08-18T23:43:24.305184+0000","last_became_active":"2024-08-18T23:41:32.136684+0000","last_became_peered":"2024-08-18T23:41:32.136684+0000","last_unstale":"2024-08-18T23:43:24.305184+0000","last_undegraded":"2024-08-18T23:43:24.305184+0000","last_fullsized":"2024-08-18T23:43:24.305184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:21:26.584303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301471+0000","last_change":"2024-08-18T23:41:30.121183+0000","last_active":"2024-08-18T23:43:24.301471+0000","last_peered":"2024-08-18T23:43:24.301471+0000","last_clean":"2024-08-18T23:43:24.301471+0000","last_became_active":"2024-08-18T23:41:30.120847+0000","last_became_peered":"2024-08-18T23:41:30.120847+0000","last_unstale":"2024-08-18T23:43:24.301471+0000","last_undegraded":"2024-08-18T23:43:24.301471+0000","last_fullsized":"2024-08-18T23:43:24.301471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:51.025821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.306833+0000","last_change":"2024-08-18T23:41:39.183375+0000","last_active":"2024-08-18T23:43:24.306833+0000","last_peered":"2024-08-18T23:43:24.306833+0000","last_clean":"2024-08-18T23:43:24.306833+0000","last_became_active":"2024-08-18T23:41:39.182339+0000","last_became_peered":"2024-08-18T23:41:39.182339+0000","last_unstale":"2024-08-18T23:43:24.306833+0000","last_undegraded":"2024-08-18T23:43:24.306833+0000","last_fullsized":"2024-08-18T23:43:24.306833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:26.119056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313146+0000","last_change":"2024-08-18T23:41:32.140748+0000","last_active":"2024-08-18T23:43:24.313146+0000","last_peered":"2024-08-18T23:43:24.313146+0000","last_clean":"2024-08-18T23:43:24.313146+0000","last_became_active":"2024-08-18T23:41:32.140306+0000","last_became_peered":"2024-08-18T23:41:32.140306+0000","last_unstale":"2024-08-18T23:43:24.313146+0000","last_undegraded":"2024-08-18T23:43:24.313146+0000","last_fullsized":"2024-08-18T23:43:24.313146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:23:02.371529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:48.359280+0000","last_change":"2024-08-18T23:41:30.121617+0000","last_active":"2024-08-18T23:43:48.359280+0000","last_peered":"2024-08-18T23:43:48.359280+0000","last_clean":"2024-08-18T23:43:48.359280+0000","last_became_active":"2024-08-18T23:41:30.121257+0000","last_became_peered":"2024-08-18T23:41:30.121257+0000","last_unstale":"2024-08-18T23:43:48.359280+0000","last_undegraded":"2024-08-18T23:43:48.359280+0000","last_fullsized":"2024-08-18T23:43:48.359280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:23:42.091553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307008+0000","last_change":"2024-08-18T23:41:39.183833+0000","last_active":"2024-08-18T23:43:24.307008+0000","last_peered":"2024-08-18T23:43:24.307008+0000","last_clean":"2024-08-18T23:43:24.307008+0000","last_became_active":"2024-08-18T23:41:39.183176+0000","last_became_peered":"2024-08-18T23:41:39.183176+0000","last_unstale":"2024-08-18T23:43:24.307008+0000","last_undegraded":"2024-08-18T23:43:24.307008+0000","last_fullsized":"2024-08-18T23:43:24.307008+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:36:09.827378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307216+0000","last_change":"2024-08-18T23:41:32.147507+0000","last_active":"2024-08-18T23:43:24.307216+0000","last_peered":"2024-08-18T23:43:24.307216+0000","last_clean":"2024-08-18T23:43:24.307216+0000","last_became_active":"2024-08-18T23:41:32.146042+0000","last_became_peered":"2024-08-18T23:41:32.146042+0000","last_unstale":"2024-08-18T23:43:24.307216+0000","last_undegraded":"2024-08-18T23:43:24.307216+0000","last_fullsized":"2024-08-18T23:43:24.307216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:48.941442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303448+0000","last_change":"2024-08-18T23:41:30.137755+0000","last_active":"2024-08-18T23:43:24.303448+0000","last_peered":"2024-08-18T23:43:24.303448+0000","last_clean":"2024-08-18T23:43:24.303448+0000","last_became_active":"2024-08-18T23:41:30.137194+0000","last_became_peered":"2024-08-18T23:41:30.137194+0000","last_unstale":"2024-08-18T23:43:24.303448+0000","last_undegraded":"2024-08-18T23:43:24.303448+0000","last_fullsized":"2024-08-18T23:43:24.303448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:15:49.681405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301357+0000","last_change":"2024-08-18T23:41:39.183986+0000","last_active":"2024-08-18T23:43:24.301357+0000","last_peered":"2024-08-18T23:43:24.301357+0000","last_clean":"2024-08-18T23:43:24.301357+0000","last_became_active":"2024-08-18T23:41:39.183384+0000","last_became_peered":"2024-08-18T23:41:39.183384+0000","last_unstale":"2024-08-18T23:43:24.301357+0000","last_undegraded":"2024-08-18T23:43:24.301357+0000","last_fullsized":"2024-08-18T23:43:24.301357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:04:28.920804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304496+0000","last_change":"2024-08-18T23:41:32.134729+0000","last_active":"2024-08-18T23:43:24.304496+0000","last_peered":"2024-08-18T23:43:24.304496+0000","last_clean":"2024-08-18T23:43:24.304496+0000","last_became_active":"2024-08-18T23:41:32.134366+0000","last_became_peered":"2024-08-18T23:41:32.134366+0000","last_unstale":"2024-08-18T23:43:24.304496+0000","last_undegraded":"2024-08-18T23:43:24.304496+0000","last_fullsized":"2024-08-18T23:43:24.304496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:38.610561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'6","reported_seq":167,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:44:03.438139+0000","last_change":"2024-08-18T23:41:30.126443+0000","last_active":"2024-08-18T23:44:03.438139+0000","last_peered":"2024-08-18T23:44:03.438139+0000","last_clean":"2024-08-18T23:44:03.438139+0000","last_became_active":"2024-08-18T23:41:30.124264+0000","last_became_peered":"2024-08-18T23:41:30.124264+0000","last_unstale":"2024-08-18T23:44:03.438139+0000","last_undegraded":"2024-08-18T23:44:03.438139+0000","last_fullsized":"2024-08-18T23:44:03.438139+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:43.727114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1427,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298528+0000","last_change":"2024-08-18T23:41:39.192164+0000","last_active":"2024-08-18T23:43:24.298528+0000","last_peered":"2024-08-18T23:43:24.298528+0000","last_clean":"2024-08-18T23:43:24.298528+0000","last_became_active":"2024-08-18T23:41:39.191833+0000","last_became_peered":"2024-08-18T23:41:39.191833+0000","last_unstale":"2024-08-18T23:43:24.298528+0000","last_undegraded":"2024-08-18T23:43:24.298528+0000","last_fullsized":"2024-08-18T23:43:24.298528+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:33:50.243930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304927+0000","last_change":"2024-08-18T23:42:23.320662+0000","last_active":"2024-08-18T23:43:24.304927+0000","last_peered":"2024-08-18T23:43:24.304927+0000","last_clean":"2024-08-18T23:43:24.304927+0000","last_became_active":"2024-08-18T23:42:23.320249+0000","last_became_peered":"2024-08-18T23:42:23.320249+0000","last_unstale":"2024-08-18T23:43:24.304927+0000","last_undegraded":"2024-08-18T23:43:24.304927+0000","last_fullsized":"2024-08-18T23:43:24.304927+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:03:17.121915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307658+0000","last_change":"2024-08-18T23:41:30.124071+0000","last_active":"2024-08-18T23:43:24.307658+0000","last_peered":"2024-08-18T23:43:24.307658+0000","last_clean":"2024-08-18T23:43:24.307658+0000","last_became_active":"2024-08-18T23:41:30.123118+0000","last_became_peered":"2024-08-18T23:41:30.123118+0000","last_unstale":"2024-08-18T23:43:24.307658+0000","last_undegraded":"2024-08-18T23:43:24.307658+0000","last_fullsized":"2024-08-18T23:43:24.307658+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:21:39.015187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313136+0000","last_change":"2024-08-18T23:41:39.184038+0000","last_active":"2024-08-18T23:43:24.313136+0000","last_peered":"2024-08-18T23:43:24.313136+0000","last_clean":"2024-08-18T23:43:24.313136+0000","last_became_active":"2024-08-18T23:41:39.183611+0000","last_became_peered":"2024-08-18T23:41:39.183611+0000","last_unstale":"2024-08-18T23:43:24.313136+0000","last_undegraded":"2024-08-18T23:43:24.313136+0000","last_fullsized":"2024-08-18T23:43:24.313136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:32:26.029151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300998+0000","last_change":"2024-08-18T23:41:32.143631+0000","last_active":"2024-08-18T23:43:24.300998+0000","last_peered":"2024-08-18T23:43:24.300998+0000","last_clean":"2024-08-18T23:43:24.300998+0000","last_became_active":"2024-08-18T23:41:32.143350+0000","last_became_peered":"2024-08-18T23:41:32.143350+0000","last_unstale":"2024-08-18T23:43:24.300998+0000","last_undegraded":"2024-08-18T23:43:24.300998+0000","last_fullsized":"2024-08-18T23:43:24.300998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:41:23.713467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312424+0000","last_change":"2024-08-18T23:41:30.123383+0000","last_active":"2024-08-18T23:43:24.312424+0000","last_peered":"2024-08-18T23:43:24.312424+0000","last_clean":"2024-08-18T23:43:24.312424+0000","last_became_active":"2024-08-18T23:41:30.123037+0000","last_became_peered":"2024-08-18T23:41:30.123037+0000","last_unstale":"2024-08-18T23:43:24.312424+0000","last_undegraded":"2024-08-18T23:43:24.312424+0000","last_fullsized":"2024-08-18T23:43:24.312424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:27.702478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307148+0000","last_change":"2024-08-18T23:41:39.184302+0000","last_active":"2024-08-18T23:43:24.307148+0000","last_peered":"2024-08-18T23:43:24.307148+0000","last_clean":"2024-08-18T23:43:24.307148+0000","last_became_active":"2024-08-18T23:41:39.183172+0000","last_became_peered":"2024-08-18T23:41:39.183172+0000","last_unstale":"2024-08-18T23:43:24.307148+0000","last_undegraded":"2024-08-18T23:43:24.307148+0000","last_fullsized":"2024-08-18T23:43:24.307148+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:23:26.655865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304016+0000","last_change":"2024-08-18T23:41:32.137426+0000","last_active":"2024-08-18T23:43:24.304016+0000","last_peered":"2024-08-18T23:43:24.304016+0000","last_clean":"2024-08-18T23:43:24.304016+0000","last_became_active":"2024-08-18T23:41:32.137106+0000","last_became_peered":"2024-08-18T23:41:32.137106+0000","last_unstale":"2024-08-18T23:43:24.304016+0000","last_undegraded":"2024-08-18T23:43:24.304016+0000","last_fullsized":"2024-08-18T23:43:24.304016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:34:51.889663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'40","reported_seq":326,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:44:04.439904+0000","last_change":"2024-08-18T23:41:30.124320+0000","last_active":"2024-08-18T23:44:04.439904+0000","last_peered":"2024-08-18T23:44:04.439904+0000","last_clean":"2024-08-18T23:44:04.439904+0000","last_became_active":"2024-08-18T23:41:30.123979+0000","last_became_peered":"2024-08-18T23:41:30.123979+0000","last_unstale":"2024-08-18T23:44:04.439904+0000","last_undegraded":"2024-08-18T23:44:04.439904+0000","last_fullsized":"2024-08-18T23:44:04.439904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:31:03.689720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70777,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":247,"num_read_kb":193,"num_write":41,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298022+0000","last_change":"2024-08-18T23:41:39.190481+0000","last_active":"2024-08-18T23:43:24.298022+0000","last_peered":"2024-08-18T23:43:24.298022+0000","last_clean":"2024-08-18T23:43:24.298022+0000","last_became_active":"2024-08-18T23:41:39.190076+0000","last_became_peered":"2024-08-18T23:41:39.190076+0000","last_unstale":"2024-08-18T23:43:24.298022+0000","last_undegraded":"2024-08-18T23:43:24.298022+0000","last_fullsized":"2024-08-18T23:43:24.298022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:47:39.313142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312507+0000","last_change":"2024-08-18T23:41:32.145422+0000","last_active":"2024-08-18T23:43:24.312507+0000","last_peered":"2024-08-18T23:43:24.312507+0000","last_clean":"2024-08-18T23:43:24.312507+0000","last_became_active":"2024-08-18T23:41:32.145047+0000","last_became_peered":"2024-08-18T23:41:32.145047+0000","last_unstale":"2024-08-18T23:43:24.312507+0000","last_undegraded":"2024-08-18T23:43:24.312507+0000","last_fullsized":"2024-08-18T23:43:24.312507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:31.905248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307467+0000","last_change":"2024-08-18T23:41:30.137364+0000","last_active":"2024-08-18T23:43:24.307467+0000","last_peered":"2024-08-18T23:43:24.307467+0000","last_clean":"2024-08-18T23:43:24.307467+0000","last_became_active":"2024-08-18T23:41:30.137004+0000","last_became_peered":"2024-08-18T23:41:30.137004+0000","last_unstale":"2024-08-18T23:43:24.307467+0000","last_undegraded":"2024-08-18T23:43:24.307467+0000","last_fullsized":"2024-08-18T23:43:24.307467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:20:56.392149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312779+0000","last_change":"2024-08-18T23:42:23.326359+0000","last_active":"2024-08-18T23:43:24.312779+0000","last_peered":"2024-08-18T23:43:24.312779+0000","last_clean":"2024-08-18T23:43:24.312779+0000","last_became_active":"2024-08-18T23:42:23.325918+0000","last_became_peered":"2024-08-18T23:42:23.325918+0000","last_unstale":"2024-08-18T23:43:24.312779+0000","last_undegraded":"2024-08-18T23:43:24.312779+0000","last_fullsized":"2024-08-18T23:43:24.312779+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:01:15.476268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312633+0000","last_change":"2024-08-18T23:42:23.325691+0000","last_active":"2024-08-18T23:43:24.312633+0000","last_peered":"2024-08-18T23:43:24.312633+0000","last_clean":"2024-08-18T23:43:24.312633+0000","last_became_active":"2024-08-18T23:42:23.325467+0000","last_became_peered":"2024-08-18T23:42:23.325467+0000","last_unstale":"2024-08-18T23:43:24.312633+0000","last_undegraded":"2024-08-18T23:43:24.312633+0000","last_fullsized":"2024-08-18T23:43:24.312633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:10:26.288880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.305106+0000","last_change":"2024-08-18T23:41:30.124430+0000","last_active":"2024-08-18T23:43:24.305106+0000","last_peered":"2024-08-18T23:43:24.305106+0000","last_clean":"2024-08-18T23:43:24.305106+0000","last_became_active":"2024-08-18T23:41:30.123626+0000","last_became_peered":"2024-08-18T23:41:30.123626+0000","last_unstale":"2024-08-18T23:43:24.305106+0000","last_undegraded":"2024-08-18T23:43:24.305106+0000","last_fullsized":"2024-08-18T23:43:24.305106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:26:18.426010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.297855+0000","last_change":"2024-08-18T23:41:39.192765+0000","last_active":"2024-08-18T23:43:24.297855+0000","last_peered":"2024-08-18T23:43:24.297855+0000","last_clean":"2024-08-18T23:43:24.297855+0000","last_became_active":"2024-08-18T23:41:39.192250+0000","last_became_peered":"2024-08-18T23:41:39.192250+0000","last_unstale":"2024-08-18T23:43:24.297855+0000","last_undegraded":"2024-08-18T23:43:24.297855+0000","last_fullsized":"2024-08-18T23:43:24.297855+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:51:16.794404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312750+0000","last_change":"2024-08-18T23:41:32.141966+0000","last_active":"2024-08-18T23:43:24.312750+0000","last_peered":"2024-08-18T23:43:24.312750+0000","last_clean":"2024-08-18T23:43:24.312750+0000","last_became_active":"2024-08-18T23:41:32.141561+0000","last_became_peered":"2024-08-18T23:41:32.141561+0000","last_unstale":"2024-08-18T23:43:24.312750+0000","last_undegraded":"2024-08-18T23:43:24.312750+0000","last_fullsized":"2024-08-18T23:43:24.312750+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:20.696089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":161,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:44:03.438138+0000","last_change":"2024-08-18T23:41:30.121598+0000","last_active":"2024-08-18T23:44:03.438138+0000","last_peered":"2024-08-18T23:44:03.438138+0000","last_clean":"2024-08-18T23:44:03.438138+0000","last_became_active":"2024-08-18T23:41:30.120533+0000","last_became_peered":"2024-08-18T23:41:30.120533+0000","last_unstale":"2024-08-18T23:44:03.438138+0000","last_undegraded":"2024-08-18T23:44:03.438138+0000","last_fullsized":"2024-08-18T23:44:03.438138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:06:37.458272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312273+0000","last_change":"2024-08-18T23:41:39.178877+0000","last_active":"2024-08-18T23:43:24.312273+0000","last_peered":"2024-08-18T23:43:24.312273+0000","last_clean":"2024-08-18T23:43:24.312273+0000","last_became_active":"2024-08-18T23:41:39.178381+0000","last_became_peered":"2024-08-18T23:41:39.178381+0000","last_unstale":"2024-08-18T23:43:24.312273+0000","last_undegraded":"2024-08-18T23:43:24.312273+0000","last_fullsized":"2024-08-18T23:43:24.312273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:27.275893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311702+0000","last_change":"2024-08-18T23:41:39.188381+0000","last_active":"2024-08-18T23:43:24.311702+0000","last_peered":"2024-08-18T23:43:24.311702+0000","last_clean":"2024-08-18T23:43:24.311702+0000","last_became_active":"2024-08-18T23:41:39.187983+0000","last_became_peered":"2024-08-18T23:41:39.187983+0000","last_unstale":"2024-08-18T23:43:24.311702+0000","last_undegraded":"2024-08-18T23:43:24.311702+0000","last_fullsized":"2024-08-18T23:43:24.311702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:04:45.460576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.297935+0000","last_change":"2024-08-18T23:41:32.144637+0000","last_active":"2024-08-18T23:43:24.297935+0000","last_peered":"2024-08-18T23:43:24.297935+0000","last_clean":"2024-08-18T23:43:24.297935+0000","last_became_active":"2024-08-18T23:41:32.144329+0000","last_became_peered":"2024-08-18T23:41:32.144329+0000","last_unstale":"2024-08-18T23:43:24.297935+0000","last_undegraded":"2024-08-18T23:43:24.297935+0000","last_fullsized":"2024-08-18T23:43:24.297935+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:29:39.791264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307199+0000","last_change":"2024-08-18T23:41:30.122325+0000","last_active":"2024-08-18T23:43:24.307199+0000","last_peered":"2024-08-18T23:43:24.307199+0000","last_clean":"2024-08-18T23:43:24.307199+0000","last_became_active":"2024-08-18T23:41:30.121480+0000","last_became_peered":"2024-08-18T23:41:30.121480+0000","last_unstale":"2024-08-18T23:43:24.307199+0000","last_undegraded":"2024-08-18T23:43:24.307199+0000","last_fullsized":"2024-08-18T23:43:24.307199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:32:12.810650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312772+0000","last_change":"2024-08-18T23:42:23.560632+0000","last_active":"2024-08-18T23:43:24.312772+0000","last_peered":"2024-08-18T23:43:24.312772+0000","last_clean":"2024-08-18T23:43:24.312772+0000","last_became_active":"2024-08-18T23:42:23.559920+0000","last_became_peered":"2024-08-18T23:42:23.559920+0000","last_unstale":"2024-08-18T23:43:24.312772+0000","last_undegraded":"2024-08-18T23:43:24.312772+0000","last_fullsized":"2024-08-18T23:43:24.312772+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:01:59.580843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312091+0000","last_change":"2024-08-18T23:41:32.145545+0000","last_active":"2024-08-18T23:43:24.312091+0000","last_peered":"2024-08-18T23:43:24.312091+0000","last_clean":"2024-08-18T23:43:24.312091+0000","last_became_active":"2024-08-18T23:41:32.145183+0000","last_became_peered":"2024-08-18T23:41:32.145183+0000","last_unstale":"2024-08-18T23:43:24.312091+0000","last_undegraded":"2024-08-18T23:43:24.312091+0000","last_fullsized":"2024-08-18T23:43:24.312091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:03:04.111658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300895+0000","last_change":"2024-08-18T23:41:30.118776+0000","last_active":"2024-08-18T23:43:24.300895+0000","last_peered":"2024-08-18T23:43:24.300895+0000","last_clean":"2024-08-18T23:43:24.300895+0000","last_became_active":"2024-08-18T23:41:30.118436+0000","last_became_peered":"2024-08-18T23:41:30.118436+0000","last_unstale":"2024-08-18T23:43:24.300895+0000","last_undegraded":"2024-08-18T23:43:24.300895+0000","last_fullsized":"2024-08-18T23:43:24.300895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:13.980927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300932+0000","last_change":"2024-08-18T23:41:30.125201+0000","last_active":"2024-08-18T23:43:24.300932+0000","last_peered":"2024-08-18T23:43:24.300932+0000","last_clean":"2024-08-18T23:43:24.300932+0000","last_became_active":"2024-08-18T23:41:30.124894+0000","last_became_peered":"2024-08-18T23:41:30.124894+0000","last_unstale":"2024-08-18T23:43:24.300932+0000","last_undegraded":"2024-08-18T23:43:24.300932+0000","last_fullsized":"2024-08-18T23:43:24.300932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:30:17.721829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307776+0000","last_change":"2024-08-18T23:41:32.145452+0000","last_active":"2024-08-18T23:43:24.307776+0000","last_peered":"2024-08-18T23:43:24.307776+0000","last_clean":"2024-08-18T23:43:24.307776+0000","last_became_active":"2024-08-18T23:41:32.145151+0000","last_became_peered":"2024-08-18T23:41:32.145151+0000","last_unstale":"2024-08-18T23:43:24.307776+0000","last_undegraded":"2024-08-18T23:43:24.307776+0000","last_fullsized":"2024-08-18T23:43:24.307776+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:08:08.713154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312361+0000","last_change":"2024-08-18T23:41:39.193334+0000","last_active":"2024-08-18T23:43:24.312361+0000","last_peered":"2024-08-18T23:43:24.312361+0000","last_clean":"2024-08-18T23:43:24.312361+0000","last_became_active":"2024-08-18T23:41:39.192924+0000","last_became_peered":"2024-08-18T23:41:39.192924+0000","last_unstale":"2024-08-18T23:43:24.312361+0000","last_undegraded":"2024-08-18T23:43:24.312361+0000","last_fullsized":"2024-08-18T23:43:24.312361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:07.016300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":171,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:44:05.441977+0000","last_change":"2024-08-18T23:41:30.125217+0000","last_active":"2024-08-18T23:44:05.441977+0000","last_peered":"2024-08-18T23:44:05.441977+0000","last_clean":"2024-08-18T23:44:05.441977+0000","last_became_active":"2024-08-18T23:41:30.124889+0000","last_became_peered":"2024-08-18T23:41:30.124889+0000","last_unstale":"2024-08-18T23:44:05.441977+0000","last_undegraded":"2024-08-18T23:44:05.441977+0000","last_fullsized":"2024-08-18T23:44:05.441977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:43.389537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303531+0000","last_change":"2024-08-18T23:41:32.142585+0000","last_active":"2024-08-18T23:43:24.303531+0000","last_peered":"2024-08-18T23:43:24.303531+0000","last_clean":"2024-08-18T23:43:24.303531+0000","last_became_active":"2024-08-18T23:41:32.142207+0000","last_became_peered":"2024-08-18T23:41:32.142207+0000","last_unstale":"2024-08-18T23:43:24.303531+0000","last_undegraded":"2024-08-18T23:43:24.303531+0000","last_fullsized":"2024-08-18T23:43:24.303531+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:16:58.755444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312344+0000","last_change":"2024-08-18T23:41:39.185618+0000","last_active":"2024-08-18T23:43:24.312344+0000","last_peered":"2024-08-18T23:43:24.312344+0000","last_clean":"2024-08-18T23:43:24.312344+0000","last_became_active":"2024-08-18T23:41:39.185222+0000","last_became_peered":"2024-08-18T23:41:39.185222+0000","last_unstale":"2024-08-18T23:43:24.312344+0000","last_undegraded":"2024-08-18T23:43:24.312344+0000","last_fullsized":"2024-08-18T23:43:24.312344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:01:27.290998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312646+0000","last_change":"2024-08-18T23:41:32.140719+0000","last_active":"2024-08-18T23:43:24.312646+0000","last_peered":"2024-08-18T23:43:24.312646+0000","last_clean":"2024-08-18T23:43:24.312646+0000","last_became_active":"2024-08-18T23:41:32.140131+0000","last_became_peered":"2024-08-18T23:41:32.140131+0000","last_unstale":"2024-08-18T23:43:24.312646+0000","last_undegraded":"2024-08-18T23:43:24.312646+0000","last_fullsized":"2024-08-18T23:43:24.312646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:48:26.778094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312375+0000","last_change":"2024-08-18T23:42:23.326683+0000","last_active":"2024-08-18T23:43:24.312375+0000","last_peered":"2024-08-18T23:43:24.312375+0000","last_clean":"2024-08-18T23:43:24.312375+0000","last_became_active":"2024-08-18T23:42:23.326309+0000","last_became_peered":"2024-08-18T23:42:23.326309+0000","last_unstale":"2024-08-18T23:43:24.312375+0000","last_undegraded":"2024-08-18T23:43:24.312375+0000","last_fullsized":"2024-08-18T23:43:24.312375+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:14.320894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312340+0000","last_change":"2024-08-18T23:41:32.149522+0000","last_active":"2024-08-18T23:43:24.312340+0000","last_peered":"2024-08-18T23:43:24.312340+0000","last_clean":"2024-08-18T23:43:24.312340+0000","last_became_active":"2024-08-18T23:41:32.149182+0000","last_became_peered":"2024-08-18T23:41:32.149182+0000","last_unstale":"2024-08-18T23:43:24.312340+0000","last_undegraded":"2024-08-18T23:43:24.312340+0000","last_fullsized":"2024-08-18T23:43:24.312340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:30:16.945045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311765+0000","last_change":"2024-08-18T23:41:32.149853+0000","last_active":"2024-08-18T23:43:24.311765+0000","last_peered":"2024-08-18T23:43:24.311765+0000","last_clean":"2024-08-18T23:43:24.311765+0000","last_became_active":"2024-08-18T23:41:32.141907+0000","last_became_peered":"2024-08-18T23:41:32.141907+0000","last_unstale":"2024-08-18T23:43:24.311765+0000","last_undegraded":"2024-08-18T23:43:24.311765+0000","last_fullsized":"2024-08-18T23:43:24.311765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:24:37.323567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301279+0000","last_change":"2024-08-18T23:41:32.136179+0000","last_active":"2024-08-18T23:43:24.301279+0000","last_peered":"2024-08-18T23:43:24.301279+0000","last_clean":"2024-08-18T23:43:24.301279+0000","last_became_active":"2024-08-18T23:41:32.135939+0000","last_became_peered":"2024-08-18T23:41:32.135939+0000","last_unstale":"2024-08-18T23:43:24.301279+0000","last_undegraded":"2024-08-18T23:43:24.301279+0000","last_fullsized":"2024-08-18T23:43:24.301279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:15.392061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313279+0000","last_change":"2024-08-18T23:41:32.137509+0000","last_active":"2024-08-18T23:43:24.313279+0000","last_peered":"2024-08-18T23:43:24.313279+0000","last_clean":"2024-08-18T23:43:24.313279+0000","last_became_active":"2024-08-18T23:41:32.136380+0000","last_became_peered":"2024-08-18T23:41:32.136380+0000","last_unstale":"2024-08-18T23:43:24.313279+0000","last_undegraded":"2024-08-18T23:43:24.313279+0000","last_fullsized":"2024-08-18T23:43:24.313279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:49.776225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312602+0000","last_change":"2024-08-18T23:41:32.139271+0000","last_active":"2024-08-18T23:43:24.312602+0000","last_peered":"2024-08-18T23:43:24.312602+0000","last_clean":"2024-08-18T23:43:24.312602+0000","last_became_active":"2024-08-18T23:41:32.138868+0000","last_became_peered":"2024-08-18T23:41:32.138868+0000","last_unstale":"2024-08-18T23:43:24.312602+0000","last_undegraded":"2024-08-18T23:43:24.312602+0000","last_fullsized":"2024-08-18T23:43:24.312602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:49:27.890857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312543+0000","last_change":"2024-08-18T23:41:32.144460+0000","last_active":"2024-08-18T23:43:24.312543+0000","last_peered":"2024-08-18T23:43:24.312543+0000","last_clean":"2024-08-18T23:43:24.312543+0000","last_became_active":"2024-08-18T23:41:32.144178+0000","last_became_peered":"2024-08-18T23:41:32.144178+0000","last_unstale":"2024-08-18T23:43:24.312543+0000","last_undegraded":"2024-08-18T23:43:24.312543+0000","last_fullsized":"2024-08-18T23:43:24.312543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:56.252215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.306952+0000","last_change":"2024-08-18T23:41:32.147973+0000","last_active":"2024-08-18T23:43:24.306952+0000","last_peered":"2024-08-18T23:43:24.306952+0000","last_clean":"2024-08-18T23:43:24.306952+0000","last_became_active":"2024-08-18T23:41:32.147621+0000","last_became_peered":"2024-08-18T23:41:32.147621+0000","last_unstale":"2024-08-18T23:43:24.306952+0000","last_undegraded":"2024-08-18T23:43:24.306952+0000","last_fullsized":"2024-08-18T23:43:24.306952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:14.176569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312329+0000","last_change":"2024-08-18T23:41:32.146029+0000","last_active":"2024-08-18T23:43:24.312329+0000","last_peered":"2024-08-18T23:43:24.312329+0000","last_clean":"2024-08-18T23:43:24.312329+0000","last_became_active":"2024-08-18T23:41:32.145584+0000","last_became_peered":"2024-08-18T23:41:32.145584+0000","last_unstale":"2024-08-18T23:43:24.312329+0000","last_undegraded":"2024-08-18T23:43:24.312329+0000","last_fullsized":"2024-08-18T23:43:24.312329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:25:33.884906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312304+0000","last_change":"2024-08-18T23:41:32.145950+0000","last_active":"2024-08-18T23:43:24.312304+0000","last_peered":"2024-08-18T23:43:24.312304+0000","last_clean":"2024-08-18T23:43:24.312304+0000","last_became_active":"2024-08-18T23:41:32.145618+0000","last_became_peered":"2024-08-18T23:41:32.145618+0000","last_unstale":"2024-08-18T23:43:24.312304+0000","last_undegraded":"2024-08-18T23:43:24.312304+0000","last_fullsized":"2024-08-18T23:43:24.312304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:05:58.889240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301154+0000","last_change":"2024-08-18T23:41:32.130684+0000","last_active":"2024-08-18T23:43:24.301154+0000","last_peered":"2024-08-18T23:43:24.301154+0000","last_clean":"2024-08-18T23:43:24.301154+0000","last_became_active":"2024-08-18T23:41:32.130369+0000","last_became_peered":"2024-08-18T23:41:32.130369+0000","last_unstale":"2024-08-18T23:43:24.301154+0000","last_undegraded":"2024-08-18T23:43:24.301154+0000","last_fullsized":"2024-08-18T23:43:24.301154+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:22.537018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313085+0000","last_change":"2024-08-18T23:41:32.137291+0000","last_active":"2024-08-18T23:43:24.313085+0000","last_peered":"2024-08-18T23:43:24.313085+0000","last_clean":"2024-08-18T23:43:24.313085+0000","last_became_active":"2024-08-18T23:41:32.136977+0000","last_became_peered":"2024-08-18T23:41:32.136977+0000","last_unstale":"2024-08-18T23:43:24.313085+0000","last_undegraded":"2024-08-18T23:43:24.313085+0000","last_fullsized":"2024-08-18T23:43:24.313085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:18:45.544760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312711+0000","last_change":"2024-08-18T23:41:32.144434+0000","last_active":"2024-08-18T23:43:24.312711+0000","last_peered":"2024-08-18T23:43:24.312711+0000","last_clean":"2024-08-18T23:43:24.312711+0000","last_became_active":"2024-08-18T23:41:32.144119+0000","last_became_peered":"2024-08-18T23:41:32.144119+0000","last_unstale":"2024-08-18T23:43:24.312711+0000","last_undegraded":"2024-08-18T23:43:24.312711+0000","last_fullsized":"2024-08-18T23:43:24.312711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:50:54.528630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298360+0000","last_change":"2024-08-18T23:41:32.135034+0000","last_active":"2024-08-18T23:43:24.298360+0000","last_peered":"2024-08-18T23:43:24.298360+0000","last_clean":"2024-08-18T23:43:24.298360+0000","last_became_active":"2024-08-18T23:41:32.134700+0000","last_became_peered":"2024-08-18T23:41:32.134700+0000","last_unstale":"2024-08-18T23:43:24.298360+0000","last_undegraded":"2024-08-18T23:43:24.298360+0000","last_fullsized":"2024-08-18T23:43:24.298360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:06:31.263114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313052+0000","last_change":"2024-08-18T23:41:32.141191+0000","last_active":"2024-08-18T23:43:24.313052+0000","last_peered":"2024-08-18T23:43:24.313052+0000","last_clean":"2024-08-18T23:43:24.313052+0000","last_became_active":"2024-08-18T23:41:32.140901+0000","last_became_peered":"2024-08-18T23:41:32.140901+0000","last_unstale":"2024-08-18T23:43:24.313052+0000","last_undegraded":"2024-08-18T23:43:24.313052+0000","last_fullsized":"2024-08-18T23:43:24.313052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:24.849985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311944+0000","last_change":"2024-08-18T23:41:32.142543+0000","last_active":"2024-08-18T23:43:24.311944+0000","last_peered":"2024-08-18T23:43:24.311944+0000","last_clean":"2024-08-18T23:43:24.311944+0000","last_became_active":"2024-08-18T23:41:32.142084+0000","last_became_peered":"2024-08-18T23:41:32.142084+0000","last_unstale":"2024-08-18T23:43:24.311944+0000","last_undegraded":"2024-08-18T23:43:24.311944+0000","last_fullsized":"2024-08-18T23:43:24.311944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:54:18.200016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311562+0000","last_change":"2024-08-18T23:41:32.148968+0000","last_active":"2024-08-18T23:43:24.311562+0000","last_peered":"2024-08-18T23:43:24.311562+0000","last_clean":"2024-08-18T23:43:24.311562+0000","last_became_active":"2024-08-18T23:41:32.148625+0000","last_became_peered":"2024-08-18T23:41:32.148625+0000","last_unstale":"2024-08-18T23:43:24.311562+0000","last_undegraded":"2024-08-18T23:43:24.311562+0000","last_fullsized":"2024-08-18T23:43:24.311562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:11:40.172139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311624+0000","last_change":"2024-08-18T23:41:32.133478+0000","last_active":"2024-08-18T23:43:24.311624+0000","last_peered":"2024-08-18T23:43:24.311624+0000","last_clean":"2024-08-18T23:43:24.311624+0000","last_became_active":"2024-08-18T23:41:32.133149+0000","last_became_peered":"2024-08-18T23:41:32.133149+0000","last_unstale":"2024-08-18T23:43:24.311624+0000","last_undegraded":"2024-08-18T23:43:24.311624+0000","last_fullsized":"2024-08-18T23:43:24.311624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:57:53.713309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303573+0000","last_change":"2024-08-18T23:43:24.303232+0000","last_active":"2024-08-18T23:43:24.303573+0000","last_peered":"2024-08-18T23:43:24.303573+0000","last_clean":"2024-08-18T23:43:24.303573+0000","last_became_active":"2024-08-18T23:43:24.302849+0000","last_became_peered":"2024-08-18T23:43:24.302849+0000","last_unstale":"2024-08-18T23:43:24.303573+0000","last_undegraded":"2024-08-18T23:43:24.303573+0000","last_fullsized":"2024-08-18T23:43:24.303573+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:02:02.479891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313096+0000","last_change":"2024-08-18T23:41:32.136303+0000","last_active":"2024-08-18T23:43:24.313096+0000","last_peered":"2024-08-18T23:43:24.313096+0000","last_clean":"2024-08-18T23:43:24.313096+0000","last_became_active":"2024-08-18T23:41:32.134628+0000","last_became_peered":"2024-08-18T23:41:32.134628+0000","last_unstale":"2024-08-18T23:43:24.313096+0000","last_undegraded":"2024-08-18T23:43:24.313096+0000","last_fullsized":"2024-08-18T23:43:24.313096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:28:51.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304992+0000","last_change":"2024-08-18T23:41:32.137512+0000","last_active":"2024-08-18T23:43:24.304992+0000","last_peered":"2024-08-18T23:43:24.304992+0000","last_clean":"2024-08-18T23:43:24.304992+0000","last_became_active":"2024-08-18T23:41:32.137150+0000","last_became_peered":"2024-08-18T23:41:32.137150+0000","last_unstale":"2024-08-18T23:43:24.304992+0000","last_undegraded":"2024-08-18T23:43:24.304992+0000","last_fullsized":"2024-08-18T23:43:24.304992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:44:08.020780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301258+0000","last_change":"2024-08-18T23:41:32.139229+0000","last_active":"2024-08-18T23:43:24.301258+0000","last_peered":"2024-08-18T23:43:24.301258+0000","last_clean":"2024-08-18T23:43:24.301258+0000","last_became_active":"2024-08-18T23:41:32.138864+0000","last_became_peered":"2024-08-18T23:41:32.138864+0000","last_unstale":"2024-08-18T23:43:24.301258+0000","last_undegraded":"2024-08-18T23:43:24.301258+0000","last_fullsized":"2024-08-18T23:43:24.301258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:54:21.701750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312829+0000","last_change":"2024-08-18T23:41:32.148727+0000","last_active":"2024-08-18T23:43:24.312829+0000","last_peered":"2024-08-18T23:43:24.312829+0000","last_clean":"2024-08-18T23:43:24.312829+0000","last_became_active":"2024-08-18T23:41:32.146828+0000","last_became_peered":"2024-08-18T23:41:32.146828+0000","last_unstale":"2024-08-18T23:43:24.312829+0000","last_undegraded":"2024-08-18T23:43:24.312829+0000","last_fullsized":"2024-08-18T23:43:24.312829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:51:49.175150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312068+0000","last_change":"2024-08-18T23:41:32.144796+0000","last_active":"2024-08-18T23:43:24.312068+0000","last_peered":"2024-08-18T23:43:24.312068+0000","last_clean":"2024-08-18T23:43:24.312068+0000","last_became_active":"2024-08-18T23:41:32.141931+0000","last_became_peered":"2024-08-18T23:41:32.141931+0000","last_unstale":"2024-08-18T23:43:24.312068+0000","last_undegraded":"2024-08-18T23:43:24.312068+0000","last_fullsized":"2024-08-18T23:43:24.312068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:48:08.428845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304569+0000","last_change":"2024-08-18T23:41:32.135265+0000","last_active":"2024-08-18T23:43:24.304569+0000","last_peered":"2024-08-18T23:43:24.304569+0000","last_clean":"2024-08-18T23:43:24.304569+0000","last_became_active":"2024-08-18T23:41:32.134952+0000","last_became_peered":"2024-08-18T23:41:32.134952+0000","last_unstale":"2024-08-18T23:43:24.304569+0000","last_undegraded":"2024-08-18T23:43:24.304569+0000","last_fullsized":"2024-08-18T23:43:24.304569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:35:14.099536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300791+0000","last_change":"2024-08-18T23:41:32.135941+0000","last_active":"2024-08-18T23:43:24.300791+0000","last_peered":"2024-08-18T23:43:24.300791+0000","last_clean":"2024-08-18T23:43:24.300791+0000","last_became_active":"2024-08-18T23:41:32.135623+0000","last_became_peered":"2024-08-18T23:41:32.135623+0000","last_unstale":"2024-08-18T23:43:24.300791+0000","last_undegraded":"2024-08-18T23:43:24.300791+0000","last_fullsized":"2024-08-18T23:43:24.300791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:43:36.504738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303603+0000","last_change":"2024-08-18T23:41:32.141643+0000","last_active":"2024-08-18T23:43:24.303603+0000","last_peered":"2024-08-18T23:43:24.303603+0000","last_clean":"2024-08-18T23:43:24.303603+0000","last_became_active":"2024-08-18T23:41:32.141301+0000","last_became_peered":"2024-08-18T23:41:32.141301+0000","last_unstale":"2024-08-18T23:43:24.303603+0000","last_undegraded":"2024-08-18T23:43:24.303603+0000","last_fullsized":"2024-08-18T23:43:24.303603+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:23:53.100740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312988+0000","last_change":"2024-08-18T23:41:32.146730+0000","last_active":"2024-08-18T23:43:24.312988+0000","last_peered":"2024-08-18T23:43:24.312988+0000","last_clean":"2024-08-18T23:43:24.312988+0000","last_became_active":"2024-08-18T23:41:32.146252+0000","last_became_peered":"2024-08-18T23:41:32.146252+0000","last_unstale":"2024-08-18T23:43:24.312988+0000","last_undegraded":"2024-08-18T23:43:24.312988+0000","last_fullsized":"2024-08-18T23:43:24.312988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:41.481761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311744+0000","last_change":"2024-08-18T23:41:32.149569+0000","last_active":"2024-08-18T23:43:24.311744+0000","last_peered":"2024-08-18T23:43:24.311744+0000","last_clean":"2024-08-18T23:43:24.311744+0000","last_became_active":"2024-08-18T23:41:32.149277+0000","last_became_peered":"2024-08-18T23:41:32.149277+0000","last_unstale":"2024-08-18T23:43:24.311744+0000","last_undegraded":"2024-08-18T23:43:24.311744+0000","last_fullsized":"2024-08-18T23:43:24.311744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:34:00.547909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311950+0000","last_change":"2024-08-18T23:41:32.137889+0000","last_active":"2024-08-18T23:43:24.311950+0000","last_peered":"2024-08-18T23:43:24.311950+0000","last_clean":"2024-08-18T23:43:24.311950+0000","last_became_active":"2024-08-18T23:41:32.137578+0000","last_became_peered":"2024-08-18T23:41:32.137578+0000","last_unstale":"2024-08-18T23:43:24.311950+0000","last_undegraded":"2024-08-18T23:43:24.311950+0000","last_fullsized":"2024-08-18T23:43:24.311950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:18:09.987838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312999+0000","last_change":"2024-08-18T23:41:32.142084+0000","last_active":"2024-08-18T23:43:24.312999+0000","last_peered":"2024-08-18T23:43:24.312999+0000","last_clean":"2024-08-18T23:43:24.312999+0000","last_became_active":"2024-08-18T23:41:32.141670+0000","last_became_peered":"2024-08-18T23:41:32.141670+0000","last_unstale":"2024-08-18T23:43:24.312999+0000","last_undegraded":"2024-08-18T23:43:24.312999+0000","last_fullsized":"2024-08-18T23:43:24.312999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:03.990351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76147,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":739,"num_read_kb":449,"num_write":193,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":228441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164,"ondisk_log_size":164,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514541,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":523788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776177,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1753088,"data_stored":1182672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939702,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":528120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168634,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":528040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397566,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1662976,"data_stored":1121222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561091,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1048576,"data_stored":519200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790024,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1111875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953548,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1064960,"data_stored":525271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":519394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":591576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1126400,"data_stored":589887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":520588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":73194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T23:44:10.440 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-18T23:44:10.440 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-18T23:44:10.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.675+0000 7fb320cb8640 1 -- 172.21.15.78:0/3357775899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb31c107fc0 msgr2=0x7fb31c10e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:10.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.675+0000 7fb320cb8640 1 --2- 172.21.15.78:0/3357775899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb31c107fc0 0x7fb31c10e960 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb314009cd0 tx=0x7fb31402a8e0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.676+0000 7fb320cb8640 1 -- 172.21.15.78:0/3357775899 shutdown_connections 2024-08-18T23:44:10.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.676+0000 7fb320cb8640 1 --2- 172.21.15.78:0/3357775899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb31c107fc0 0x7fb31c10e960 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.676+0000 7fb320cb8640 1 --2- 172.21.15.78:0/3357775899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb31c103470 0x7fb31c1078b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.676+0000 7fb320cb8640 1 --2- 172.21.15.78:0/3357775899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb31c102ac0 0x7fb31c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.676+0000 7fb320cb8640 1 -- 172.21.15.78:0/3357775899 >> 172.21.15.78:0/3357775899 conn(0x7fb31c0fc500 msgr2=0x7fb31c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:10.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.676+0000 7fb320cb8640 1 -- 172.21.15.78:0/3357775899 shutdown_connections 2024-08-18T23:44:10.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.676+0000 7fb320cb8640 1 -- 172.21.15.78:0/3357775899 wait complete. 2024-08-18T23:44:10.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.677+0000 7fb320cb8640 1 Processor -- start 2024-08-18T23:44:10.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.677+0000 7fb320cb8640 1 -- start start 2024-08-18T23:44:10.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.677+0000 7fb320cb8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb31c102ac0 0x7fb31c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:10.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.677+0000 7fb320cb8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb31c103470 0x7fb31c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:10.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.677+0000 7fb320cb8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb31c107fc0 0x7fb31c19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.677+0000 7fb320cb8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c111dc0 con 0x7fb31c102ac0 2024-08-18T23:44:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.677+0000 7fb320cb8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c111c40 con 0x7fb31c107fc0 2024-08-18T23:44:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.677+0000 7fb320cb8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c111f40 con 0x7fb31c103470 2024-08-18T23:44:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.677+0000 7fb31a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb31c102ac0 0x7fb31c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.677+0000 7fb31a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb31c102ac0 0x7fb31c198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35832/0 (socket says 172.21.15.78:35832) 2024-08-18T23:44:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.677+0000 7fb31a59c640 1 -- 172.21.15.78:0/3197561101 learned_addr learned my addr 172.21.15.78:0/3197561101 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.677+0000 7fb319d9b640 1 --2- 172.21.15.78:0/3197561101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb31c103470 0x7fb31c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.677+0000 7fb31ad9d640 1 --2- 172.21.15.78:0/3197561101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb31c107fc0 0x7fb31c19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.677+0000 7fb31a59c640 1 -- 172.21.15.78:0/3197561101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb31c103470 msgr2=0x7fb31c1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.677+0000 7fb31a59c640 1 --2- 172.21.15.78:0/3197561101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb31c103470 0x7fb31c1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.677+0000 7fb31a59c640 1 -- 172.21.15.78:0/3197561101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb31c107fc0 msgr2=0x7fb31c19cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.677+0000 7fb31a59c640 1 --2- 172.21.15.78:0/3197561101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb31c107fc0 0x7fb31c19cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.677+0000 7fb31a59c640 1 -- 172.21.15.78:0/3197561101 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31c19d210 con 0x7fb31c102ac0 2024-08-18T23:44:10.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.678+0000 7fb31ad9d640 1 --2- 172.21.15.78:0/3197561101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb31c107fc0 0x7fb31c19cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:44:10.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.678+0000 7fb31a59c640 1 --2- 172.21.15.78:0/3197561101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb31c102ac0 0x7fb31c198260 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00d770 tx=0x7fb30c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:10.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.678+0000 7fb30b7fe640 1 -- 172.21.15.78:0/3197561101 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c017070 con 0x7fb31c102ac0 2024-08-18T23:44:10.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.678+0000 7fb30b7fe640 1 -- 172.21.15.78:0/3197561101 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb30c005500 con 0x7fb31c102ac0 2024-08-18T23:44:10.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.678+0000 7fb320cb8640 1 -- 172.21.15.78:0/3197561101 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31c19d500 con 0x7fb31c102ac0 2024-08-18T23:44:10.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.678+0000 7fb30b7fe640 1 -- 172.21.15.78:0/3197561101 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c004e00 con 0x7fb31c102ac0 2024-08-18T23:44:10.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.678+0000 7fb320cb8640 1 -- 172.21.15.78:0/3197561101 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb31c1a4de0 con 0x7fb31c102ac0 2024-08-18T23:44:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.681+0000 7fb30b7fe640 1 -- 172.21.15.78:0/3197561101 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fb30c004fa0 con 0x7fb31c102ac0 2024-08-18T23:44:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.681+0000 7fb3230b6740 1 Processor -- start 2024-08-18T23:44:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.681+0000 7fb3230b6740 1 -- start start 2024-08-18T23:44:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.681+0000 7fb3230b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d29210 0x560f15d295f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.681+0000 7fb3230b6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29b30 0x560f15d33eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.681+0000 7fb3230b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d343f0 0x560f15d367e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.681+0000 7fb3230b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560f15c8e980 con 0x560f15d343f0 2024-08-18T23:44:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.681+0000 7fb3230b6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560f15c26a70 con 0x560f15d29210 2024-08-18T23:44:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.681+0000 7fb3230b6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560f15c8a940 con 0x560f15d29b30 2024-08-18T23:44:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.681+0000 7fb319d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d343f0 0x560f15d367e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:10.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.681+0000 7fb31a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29b30 0x560f15d33eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:10.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.681+0000 7fb319d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d343f0 0x560f15d367e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35846/0 (socket says 172.21.15.78:35846) 2024-08-18T23:44:10.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.681+0000 7fb319d9b640 1 -- 172.21.15.78:0/2533211156 learned_addr learned my addr 172.21.15.78:0/2533211156 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:10.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.681+0000 7fb31ad9d640 1 --2- 172.21.15.78:0/2533211156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d29210 0x560f15d295f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:10.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.682+0000 7fb319d9b640 1 -- 172.21.15.78:0/2533211156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29b30 msgr2=0x560f15d33eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:10.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.682+0000 7fb319d9b640 1 --2- 172.21.15.78:0/2533211156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29b30 0x560f15d33eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.682+0000 7fb319d9b640 1 -- 172.21.15.78:0/2533211156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d29210 msgr2=0x560f15d295f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:10.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.682+0000 7fb319d9b640 1 --2- 172.21.15.78:0/2533211156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d29210 0x560f15d295f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.682+0000 7fb319d9b640 1 -- 172.21.15.78:0/2533211156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f15c896e0 con 0x560f15d343f0 2024-08-18T23:44:10.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.682+0000 7fb31ad9d640 1 --2- 172.21.15.78:0/2533211156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d29210 0x560f15d295f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:10.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.682+0000 7fb30b7fe640 1 --2- 172.21.15.78:0/3197561101 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb3040a2c90 0x7fb3040a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:10.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.682+0000 7fb31ad9d640 1 --2- 172.21.15.78:0/3197561101 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb3040a2c90 0x7fb3040a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:10.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.682+0000 7fb30b7fe640 1 -- 172.21.15.78:0/3197561101 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fb30c0cefe0 con 0x7fb31c102ac0 2024-08-18T23:44:10.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.682+0000 7fb319d9b640 1 --2- 172.21.15.78:0/2533211156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d343f0 0x560f15d367e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb31000bbe0 tx=0x7fb31000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279d73ef05d77c05 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:10.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.682+0000 7fb308ff9640 1 -- 172.21.15.78:0/2533211156 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310015070 con 0x560f15d343f0 2024-08-18T23:44:10.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.682+0000 7fb308ff9640 1 -- 172.21.15.78:0/2533211156 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb310011920 con 0x560f15d343f0 2024-08-18T23:44:10.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.682+0000 7fb308ff9640 1 -- 172.21.15.78:0/2533211156 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3100107d0 con 0x560f15d343f0 2024-08-18T23:44:10.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.683+0000 7fb3230b6740 1 -- 172.21.15.78:0/2533211156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d343f0 msgr2=0x560f15d367e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:10.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.683+0000 7fb31ad9d640 1 --2- 172.21.15.78:0/3197561101 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb3040a2c90 0x7fb3040a5150 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb314005e30 tx=0x7fb3140314f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:10.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.683+0000 7fb3230b6740 1 --2- 172.21.15.78:0/2533211156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d343f0 0x560f15d367e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb31000bbe0 tx=0x7fb31000f730 comp rx=0 tx=0).stop 2024-08-18T23:44:10.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.683+0000 7fb3230b6740 1 -- 172.21.15.78:0/2533211156 shutdown_connections 2024-08-18T23:44:10.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.683+0000 7fb3230b6740 1 --2- 172.21.15.78:0/2533211156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d343f0 0x560f15d367e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.683+0000 7fb3230b6740 1 --2- 172.21.15.78:0/2533211156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29b30 0x560f15d33eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.683+0000 7fb3230b6740 1 --2- 172.21.15.78:0/2533211156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d29210 0x560f15d295f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.683+0000 7fb3230b6740 1 -- 172.21.15.78:0/2533211156 >> 172.21.15.78:0/2533211156 conn(0x560f15d257a0 msgr2=0x560f15d25b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:10.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.683+0000 7fb3230b6740 1 -- 172.21.15.78:0/2533211156 shutdown_connections 2024-08-18T23:44:10.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.683+0000 7fb3230b6740 1 -- 172.21.15.78:0/2533211156 wait complete. 2024-08-18T23:44:10.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.684+0000 7fb3230b6740 1 -- start start 2024-08-18T23:44:10.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.684+0000 7fb3230b6740 1 Processor -- start 2024-08-18T23:44:10.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.684+0000 7fb3230b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d29210 0x560f15d31300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:10.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.684+0000 7fb3230b6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29b30 0x560f15d31840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:10.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.684+0000 7fb3230b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d343f0 0x560f15d31d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:10.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.684+0000 7fb3230b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560f15d36fc0 con 0x560f15d29210 2024-08-18T23:44:10.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.684+0000 7fb3230b6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560f15c49aa0 con 0x560f15d343f0 2024-08-18T23:44:10.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.684+0000 7fb3230b6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560f15d37140 con 0x560f15d29b30 2024-08-18T23:44:10.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.684+0000 7fb3230b6740 10 client.0 _ll_register_callbacks cb 0x560f15c38e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:10.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.684+0000 7fb31a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d29210 0x560f15d31300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:10.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.684+0000 7fb31a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d29210 0x560f15d31300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35852/0 (socket says 172.21.15.78:35852) 2024-08-18T23:44:10.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.684+0000 7fb31a59c640 1 -- 172.21.15.78:0/3018876753 learned_addr learned my addr 172.21.15.78:0/3018876753 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.684+0000 7fb319d9b640 1 --2- 172.21.15.78:0/3018876753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29b30 0x560f15d31840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.684+0000 7fb31a59c640 1 -- 172.21.15.78:0/3018876753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29b30 msgr2=0x560f15d31840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.684+0000 7fb31a59c640 1 --2- 172.21.15.78:0/3018876753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29b30 0x560f15d31840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.684+0000 7fb31a59c640 1 -- 172.21.15.78:0/3018876753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d343f0 msgr2=0x560f15d31d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.684+0000 7fb31a59c640 1 --2- 172.21.15.78:0/3018876753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d343f0 0x560f15d31d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.684+0000 7fb31a59c640 1 -- 172.21.15.78:0/3018876753 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f15c2d650 con 0x560f15d29210 2024-08-18T23:44:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.685+0000 7fb31a59c640 1 --2- 172.21.15.78:0/3018876753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d29210 0x560f15d31300 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb30c021000 tx=0x7fb30c0048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ecfcbd9dd64bee server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.685+0000 7fb301ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:44:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.685+0000 7fb301ffb640 1 -- 172.21.15.78:0/3018876753 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c020070 con 0x560f15d29210 2024-08-18T23:44:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.685+0000 7fb301ffb640 1 -- 172.21.15.78:0/3018876753 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fb30c009cc0 con 0x560f15d29210 2024-08-18T23:44:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.685+0000 7fb3230b6740 1 -- 172.21.15.78:0/3018876753 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb310011920 con 0x560f15d29210 2024-08-18T23:44:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.685+0000 7fb301ffb640 1 -- 172.21.15.78:0/3018876753 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb30c09ae90 con 0x560f15d29210 2024-08-18T23:44:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.685+0000 7fb3230b6740 10 client.15372 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.685+0000 7fb3230b6740 1 -- 172.21.15.78:0/3018876753 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560f15a003c0 con 0x560f15d29210 2024-08-18T23:44:10.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.685+0000 7fb301ffb640 1 -- 172.21.15.78:0/3018876753 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb310011920 con 0x560f15d29210 2024-08-18T23:44:10.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.685+0000 7fb301ffb640 1 -- 172.21.15.78:0/3018876753 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb30c0979c0 con 0x560f15d29210 2024-08-18T23:44:10.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.685+0000 7fb3230b6740 10 client.15372 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:10.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.685+0000 7fb3230b6740 10 client.15372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:44:10.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.685+0000 7fb3230b6740 20 client.15372 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:10.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.685+0000 7fb3230b6740 1 --2- 172.21.15.78:0/3018876753 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x560f15d43e30 0x560f15d46280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:10.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.685+0000 7fb3230b6740 4 client.15372 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:44:10.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.685+0000 7fb3230b6740 1 -- 172.21.15.78:0/3018876753 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f15c9bef0 con 0x560f15d43e30 2024-08-18T23:44:10.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.686+0000 7fb319d9b640 1 --2- 172.21.15.78:0/3018876753 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x560f15d43e30 0x560f15d46280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:10.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.686+0000 7fb301ffb640 10 client.15372 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:10.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.686+0000 7fb319d9b640 1 --2- 172.21.15.78:0/3018876753 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x560f15d43e30 0x560f15d46280 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcdb75f7a4d174b9 server_cookie=3dbf9b8b65fc1b76 in_seq=0 out_seq=0 2024-08-18T23:44:10.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.687+0000 7fb301ffb640 1 -- 172.21.15.78:0/3018876753 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f15c9bef0 con 0x560f15d43e30 2024-08-18T23:44:10.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.687+0000 7fb301ffb640 10 client.15372 handle_command_reply: tid=0 2024-08-18T23:44:10.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.687+0000 7fb3230b6740 1 client.15372 shutdown 2024-08-18T23:44:10.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.687+0000 7fb3230b6740 1 -- 172.21.15.78:0/3018876753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d29210 msgr2=0x560f15d31300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:10.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.687+0000 7fb3230b6740 1 --2- 172.21.15.78:0/3018876753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d29210 0x560f15d31300 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb30c021000 tx=0x7fb30c0048a0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.687+0000 7fb3230b6740 1 -- 172.21.15.78:0/3018876753 shutdown_connections 2024-08-18T23:44:10.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.687+0000 7fb3230b6740 1 --2- 172.21.15.78:0/3018876753 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x560f15d43e30 0x560f15d46280 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.687+0000 7fb3230b6740 1 --2- 172.21.15.78:0/3018876753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d343f0 0x560f15d31d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.687+0000 7fb3230b6740 1 --2- 172.21.15.78:0/3018876753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29b30 0x560f15d31840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.687+0000 7fb301ffb640 0 client.15372 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:10.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.687+0000 7fb3230b6740 1 --2- 172.21.15.78:0/3018876753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d29210 0x560f15d31300 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.687+0000 7fb3230b6740 1 -- 172.21.15.78:0/3018876753 >> 172.21.15.78:0/3018876753 conn(0x560f15d257a0 msgr2=0x560f15d2b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:10.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.687+0000 7fb3230b6740 1 -- 172.21.15.78:0/3018876753 shutdown_connections 2024-08-18T23:44:10.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.687+0000 7fb3230b6740 1 -- 172.21.15.78:0/3018876753 wait complete. 2024-08-18T23:44:10.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.687+0000 7fb3230b6740 20 client.15372 trim_cache size 0 max 16384 2024-08-18T23:44:10.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.705+0000 7fb3230b6740 1 Processor -- start 2024-08-18T23:44:10.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.705+0000 7fb3230b6740 1 -- start start 2024-08-18T23:44:10.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.705+0000 7fb3230b6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29210 0x560f15d25db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:10.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.705+0000 7fb3230b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d29b30 0x560f15d247f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:10.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.705+0000 7fb3230b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d343f0 0x560f15d4bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:10.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.705+0000 7fb3230b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c1b12c0 con 0x560f15d343f0 2024-08-18T23:44:10.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.705+0000 7fb3230b6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c1aeb90 con 0x560f15d29b30 2024-08-18T23:44:10.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.705+0000 7fb3230b6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c1ac9e0 con 0x560f15d29210 2024-08-18T23:44:10.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.705+0000 7fb31ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d343f0 0x560f15d4bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:10.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.705+0000 7fb31ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d343f0 0x560f15d4bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35858/0 (socket says 172.21.15.78:35858) 2024-08-18T23:44:10.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.705+0000 7fb31ad9d640 1 -- 172.21.15.78:0/2376083496 learned_addr learned my addr 172.21.15.78:0/2376083496 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:10.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.705+0000 7fb319d9b640 1 --2- 172.21.15.78:0/2376083496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d29b30 0x560f15d247f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:10.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.705+0000 7fb31a59c640 1 --2- 172.21.15.78:0/2376083496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29210 0x560f15d25db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:10.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.705+0000 7fb31ad9d640 1 -- 172.21.15.78:0/2376083496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29210 msgr2=0x560f15d25db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:10.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.705+0000 7fb31ad9d640 1 --2- 172.21.15.78:0/2376083496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29210 0x560f15d25db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.705+0000 7fb31ad9d640 1 -- 172.21.15.78:0/2376083496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d29b30 msgr2=0x560f15d247f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:10.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.705+0000 7fb31ad9d640 1 --2- 172.21.15.78:0/2376083496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d29b30 0x560f15d247f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.705+0000 7fb31ad9d640 1 -- 172.21.15.78:0/2376083496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f15d302b0 con 0x560f15d343f0 2024-08-18T23:44:10.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.706+0000 7fb31ad9d640 1 --2- 172.21.15.78:0/2376083496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d343f0 0x560f15d4bb50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb314009a00 tx=0x7fb31403d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4892c3fc1701b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:10.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.706+0000 7fb303fff640 1 -- 172.21.15.78:0/2376083496 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb314003b70 con 0x560f15d343f0 2024-08-18T23:44:10.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.706+0000 7fb303fff640 1 -- 172.21.15.78:0/2376083496 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb314036040 con 0x560f15d343f0 2024-08-18T23:44:10.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.706+0000 7fb303fff640 1 -- 172.21.15.78:0/2376083496 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3140308f0 con 0x560f15d343f0 2024-08-18T23:44:10.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.707+0000 7fb3230b6740 1 -- 172.21.15.78:0/2376083496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d343f0 msgr2=0x560f15d4bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:10.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.707+0000 7fb3230b6740 1 --2- 172.21.15.78:0/2376083496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d343f0 0x560f15d4bb50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb314009a00 tx=0x7fb31403d040 comp rx=0 tx=0).stop 2024-08-18T23:44:10.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.707+0000 7fb3230b6740 1 -- 172.21.15.78:0/2376083496 shutdown_connections 2024-08-18T23:44:10.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.707+0000 7fb3230b6740 1 --2- 172.21.15.78:0/2376083496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d343f0 0x560f15d4bb50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.707+0000 7fb3230b6740 1 --2- 172.21.15.78:0/2376083496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d29b30 0x560f15d247f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.707+0000 7fb3230b6740 1 --2- 172.21.15.78:0/2376083496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29210 0x560f15d25db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.707+0000 7fb3230b6740 1 -- 172.21.15.78:0/2376083496 >> 172.21.15.78:0/2376083496 conn(0x560f15d257a0 msgr2=0x560f15d240e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:10.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.707+0000 7fb3230b6740 1 -- 172.21.15.78:0/2376083496 shutdown_connections 2024-08-18T23:44:10.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.707+0000 7fb3230b6740 1 -- 172.21.15.78:0/2376083496 wait complete. 2024-08-18T23:44:10.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.707+0000 7fb3230b6740 1 -- start start 2024-08-18T23:44:10.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.707+0000 7fb3230b6740 1 Processor -- start 2024-08-18T23:44:10.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb3230b6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29210 0x560f15d3cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:10.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb3230b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d29b30 0x560f15d37f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:10.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb3230b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d343f0 0x560f15d38560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:10.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb3230b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c1ad050 con 0x560f15d29b30 2024-08-18T23:44:10.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb3230b6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c1ae4c0 con 0x560f15d343f0 2024-08-18T23:44:10.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb3230b6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c1ad720 con 0x560f15d29210 2024-08-18T23:44:10.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb319d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d29b30 0x560f15d37f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:10.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb319d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d29b30 0x560f15d37f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35868/0 (socket says 172.21.15.78:35868) 2024-08-18T23:44:10.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb319d9b640 1 -- 172.21.15.78:0/2198484271 learned_addr learned my addr 172.21.15.78:0/2198484271 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:10.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb3230b6740 10 client.0 _ll_register_callbacks cb 0x7fb30c0056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb31a59c640 1 --2- 172.21.15.78:0/2198484271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29210 0x560f15d3cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb31ad9d640 1 --2- 172.21.15.78:0/2198484271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d343f0 0x560f15d38560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb319d9b640 1 -- 172.21.15.78:0/2198484271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29210 msgr2=0x560f15d3cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb319d9b640 1 --2- 172.21.15.78:0/2198484271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29210 0x560f15d3cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb319d9b640 1 -- 172.21.15.78:0/2198484271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d343f0 msgr2=0x560f15d38560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb319d9b640 1 --2- 172.21.15.78:0/2198484271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d343f0 0x560f15d38560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb319d9b640 1 -- 172.21.15.78:0/2198484271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f15d38d90 con 0x560f15d29b30 2024-08-18T23:44:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb319d9b640 1 --2- 172.21.15.78:0/2198484271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d29b30 0x560f15d37f50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb31000fc30 tx=0x7fb31000bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95cd4661d85ce388 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb301ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:44:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb301ffb640 1 -- 172.21.15.78:0/2198484271 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310015040 con 0x560f15d29b30 2024-08-18T23:44:10.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb301ffb640 1 -- 172.21.15.78:0/2198484271 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fb310005450 con 0x560f15d29b30 2024-08-18T23:44:10.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.708+0000 7fb3230b6740 1 -- 172.21.15.78:0/2198484271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb314036040 con 0x560f15d29b30 2024-08-18T23:44:10.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.709+0000 7fb301ffb640 1 -- 172.21.15.78:0/2198484271 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb310004020 con 0x560f15d29b30 2024-08-18T23:44:10.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.709+0000 7fb3230b6740 10 client.15378 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:10.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.709+0000 7fb3230b6740 1 -- 172.21.15.78:0/2198484271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560f15daecf0 con 0x560f15d29b30 2024-08-18T23:44:10.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.709+0000 7fb301ffb640 1 -- 172.21.15.78:0/2198484271 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb314036040 con 0x560f15d29b30 2024-08-18T23:44:10.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.709+0000 7fb301ffb640 1 -- 172.21.15.78:0/2198484271 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb310004c60 con 0x560f15d29b30 2024-08-18T23:44:10.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.709+0000 7fb3230b6740 10 client.15378 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:10.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.709+0000 7fb3230b6740 10 client.15378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:44:10.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.709+0000 7fb3230b6740 20 client.15378 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:10.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.709+0000 7fb3230b6740 1 --2- 172.21.15.78:0/2198484271 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x560f15daf360 0x560f15db1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:10.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.709+0000 7fb3230b6740 4 client.15378 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:44:10.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.709+0000 7fb3230b6740 1 -- 172.21.15.78:0/2198484271 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560f15c96400 con 0x560f15daf360 2024-08-18T23:44:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.710+0000 7fb31a59c640 1 --2- 172.21.15.78:0/2198484271 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x560f15daf360 0x560f15db1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.710+0000 7fb31a59c640 1 --2- 172.21.15.78:0/2198484271 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x560f15daf360 0x560f15db1780 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67f7e886e9a7feea server_cookie=a405100f80b1486a in_seq=0 out_seq=0 2024-08-18T23:44:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.710+0000 7fb301ffb640 10 client.15378 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.710+0000 7fb301ffb640 1 -- 172.21.15.78:0/2198484271 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560f15c96400 con 0x560f15daf360 2024-08-18T23:44:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.710+0000 7fb301ffb640 10 client.15378 handle_command_reply: tid=0 2024-08-18T23:44:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.710+0000 7fb3230b6740 1 client.15378 shutdown 2024-08-18T23:44:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.710+0000 7fb3230b6740 1 -- 172.21.15.78:0/2198484271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d29b30 msgr2=0x560f15d37f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.710+0000 7fb3230b6740 1 --2- 172.21.15.78:0/2198484271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d29b30 0x560f15d37f50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb31000fc30 tx=0x7fb31000bcd0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.710+0000 7fb3230b6740 1 -- 172.21.15.78:0/2198484271 shutdown_connections 2024-08-18T23:44:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.710+0000 7fb3230b6740 1 --2- 172.21.15.78:0/2198484271 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x560f15daf360 0x560f15db1780 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.710+0000 7fb3230b6740 1 --2- 172.21.15.78:0/2198484271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560f15d343f0 0x560f15d38560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.710+0000 7fb3230b6740 1 --2- 172.21.15.78:0/2198484271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f15d29b30 0x560f15d37f50 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.710+0000 7fb3230b6740 1 --2- 172.21.15.78:0/2198484271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560f15d29210 0x560f15d3cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.710+0000 7fb3230b6740 1 -- 172.21.15.78:0/2198484271 >> 172.21.15.78:0/2198484271 conn(0x560f15d257a0 msgr2=0x560f15d2b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:10.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.710+0000 7fb301ffb640 0 client.15378 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:10.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.711+0000 7fb3230b6740 1 -- 172.21.15.78:0/2198484271 shutdown_connections 2024-08-18T23:44:10.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.711+0000 7fb3230b6740 1 -- 172.21.15.78:0/2198484271 wait complete. 2024-08-18T23:44:10.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.711+0000 7fb3230b6740 20 client.15378 trim_cache size 0 max 16384 2024-08-18T23:44:10.717 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:44:10.717 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-18T23:44:10.717 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-08-18T23:44:10.717 INFO:teuthology.orchestra.run.smithi078.stdout: "bac246f7-3ef7-435a-a19f-aa6f2c8cf376": { 2024-08-18T23:44:10.717 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-08-18T23:44:10.718 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "bac246f7-3ef7-435a-a19f-aa6f2c8cf376", 2024-08-18T23:44:10.718 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-08-18T23:44:10.718 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:44:10.718 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:44:10.718 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:44:10.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.711+0000 7fb320cb8640 1 -- 172.21.15.78:0/3197561101 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb3040a2c90 msgr2=0x7fb3040a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:10.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.711+0000 7fb320cb8640 1 --2- 172.21.15.78:0/3197561101 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb3040a2c90 0x7fb3040a5150 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb314005e30 tx=0x7fb3140314f0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.711+0000 7fb320cb8640 1 -- 172.21.15.78:0/3197561101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb31c102ac0 msgr2=0x7fb31c198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:10.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.711+0000 7fb320cb8640 1 --2- 172.21.15.78:0/3197561101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb31c102ac0 0x7fb31c198260 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00d770 tx=0x7fb30c00dc30 comp rx=0 tx=0).stop 2024-08-18T23:44:10.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.711+0000 7fb320cb8640 1 -- 172.21.15.78:0/3197561101 shutdown_connections 2024-08-18T23:44:10.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.711+0000 7fb320cb8640 1 --2- 172.21.15.78:0/3197561101 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb3040a2c90 0x7fb3040a5150 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.711+0000 7fb320cb8640 1 --2- 172.21.15.78:0/3197561101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb31c107fc0 0x7fb31c19cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.711+0000 7fb320cb8640 1 --2- 172.21.15.78:0/3197561101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb31c103470 0x7fb31c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.711+0000 7fb320cb8640 1 --2- 172.21.15.78:0/3197561101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb31c102ac0 0x7fb31c198260 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:10.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.711+0000 7fb320cb8640 1 -- 172.21.15.78:0/3197561101 >> 172.21.15.78:0/3197561101 conn(0x7fb31c0fc500 msgr2=0x7fb31c105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:10.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.711+0000 7fb320cb8640 1 -- 172.21.15.78:0/3197561101 shutdown_connections 2024-08-18T23:44:10.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:10.711+0000 7fb320cb8640 1 -- 172.21.15.78:0/3197561101 wait complete. 2024-08-18T23:44:11.404 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:11 smithi078 ceph-mon[18440]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:44:11.405 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:11 smithi078 ceph-mon[18440]: pgmap v346: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-08-18T23:44:11.412 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:11.443 INFO:tasks.cephfs.filesystem:scrub status for tag:bac246f7-3ef7-435a-a19f-aa6f2c8cf376 - {'path': '/', 'tag': 'bac246f7-3ef7-435a-a19f-aa6f2c8cf376', 'options': 'recursive,force'} 2024-08-18T23:44:11.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:11 smithi123 ceph-mon[21808]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:44:11.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:11 smithi123 ceph-mon[21808]: pgmap v346: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-08-18T23:44:11.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:11 smithi161 ceph-mon[22027]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:44:11.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:11 smithi161 ceph-mon[22027]: pgmap v346: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-08-18T23:44:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.877+0000 7f61717aa640 1 -- 172.21.15.78:0/1733560777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f616c1018a0 msgr2=0x7f616c101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.877+0000 7f61717aa640 1 --2- 172.21.15.78:0/1733560777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f616c1018a0 0x7f616c101d00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6154009930 tx=0x7f615402a420 comp rx=0 tx=0).stop 2024-08-18T23:44:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.877+0000 7f61717aa640 1 -- 172.21.15.78:0/1733560777 shutdown_connections 2024-08-18T23:44:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.877+0000 7f61717aa640 1 --2- 172.21.15.78:0/1733560777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f616c102240 0x7f616c10e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.877+0000 7f61717aa640 1 --2- 172.21.15.78:0/1733560777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f616c1018a0 0x7f616c101d00 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.877+0000 7f61717aa640 1 --2- 172.21.15.78:0/1733560777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f616c1078a0 0x7f616c107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:11.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.877+0000 7f61717aa640 1 -- 172.21.15.78:0/1733560777 >> 172.21.15.78:0/1733560777 conn(0x7f616c0fd5e0 msgr2=0x7f616c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:11.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.877+0000 7f61717aa640 1 -- 172.21.15.78:0/1733560777 shutdown_connections 2024-08-18T23:44:11.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.877+0000 7f61717aa640 1 -- 172.21.15.78:0/1733560777 wait complete. 2024-08-18T23:44:11.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.878+0000 7f61717aa640 1 Processor -- start 2024-08-18T23:44:11.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.878+0000 7f61717aa640 1 -- start start 2024-08-18T23:44:11.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.878+0000 7f61717aa640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f616c1018a0 0x7f616c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:11.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.878+0000 7f61717aa640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f616c102240 0x7f616c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:11.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.878+0000 7f61717aa640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f616c1078a0 0x7f616c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:11.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.878+0000 7f61717aa640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f616c111d40 con 0x7f616c102240 2024-08-18T23:44:11.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.878+0000 7f61717aa640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f616c111bc0 con 0x7f616c1078a0 2024-08-18T23:44:11.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.878+0000 7f61717aa640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f616c111ec0 con 0x7f616c1018a0 2024-08-18T23:44:11.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.878+0000 7f6163fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f616c102240 0x7f616c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:11.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.878+0000 7f6163fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f616c102240 0x7f616c1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35896/0 (socket says 172.21.15.78:35896) 2024-08-18T23:44:11.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.878+0000 7f6163fff640 1 -- 172.21.15.78:0/3638163351 learned_addr learned my addr 172.21.15.78:0/3638163351 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:11.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.879+0000 7f616affd640 1 --2- 172.21.15.78:0/3638163351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f616c1018a0 0x7f616c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:11.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.879+0000 7f616b7fe640 1 --2- 172.21.15.78:0/3638163351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f616c1078a0 0x7f616c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:11.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.879+0000 7f6163fff640 1 -- 172.21.15.78:0/3638163351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f616c1018a0 msgr2=0x7f616c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:11.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.879+0000 7f6163fff640 1 --2- 172.21.15.78:0/3638163351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f616c1018a0 0x7f616c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:11.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.879+0000 7f6163fff640 1 -- 172.21.15.78:0/3638163351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f616c1078a0 msgr2=0x7f616c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:11.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.879+0000 7f6163fff640 1 --2- 172.21.15.78:0/3638163351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f616c1078a0 0x7f616c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:11.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.879+0000 7f6163fff640 1 -- 172.21.15.78:0/3638163351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616c19d350 con 0x7f616c102240 2024-08-18T23:44:11.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.879+0000 7f6163fff640 1 --2- 172.21.15.78:0/3638163351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f616c102240 0x7f616c1988e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6154009a50 tx=0x7f6154004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:11.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.879+0000 7f6168ff9640 1 -- 172.21.15.78:0/3638163351 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615403f030 con 0x7f616c102240 2024-08-18T23:44:11.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.879+0000 7f6168ff9640 1 -- 172.21.15.78:0/3638163351 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6154003ce0 con 0x7f616c102240 2024-08-18T23:44:11.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.879+0000 7f61717aa640 1 -- 172.21.15.78:0/3638163351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f616c19d5e0 con 0x7f616c102240 2024-08-18T23:44:11.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.879+0000 7f6168ff9640 1 -- 172.21.15.78:0/3638163351 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6154035070 con 0x7f616c102240 2024-08-18T23:44:11.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.879+0000 7f61717aa640 1 -- 172.21.15.78:0/3638163351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f616c1a4f20 con 0x7f616c102240 2024-08-18T23:44:11.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.881+0000 7f61717aa640 1 -- 172.21.15.78:0/3638163351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f616c101d80 con 0x7f616c102240 2024-08-18T23:44:11.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.881+0000 7f6168ff9640 1 -- 172.21.15.78:0/3638163351 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f6154042030 con 0x7f616c102240 2024-08-18T23:44:11.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.882+0000 7f6168ff9640 1 --2- 172.21.15.78:0/3638163351 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f61580a2c10 0x7f61580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:11.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.882+0000 7f6168ff9640 1 -- 172.21.15.78:0/3638163351 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f61540ef810 con 0x7f616c102240 2024-08-18T23:44:11.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.882+0000 7f616affd640 1 --2- 172.21.15.78:0/3638163351 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f61580a2c10 0x7f61580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:11.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.883+0000 7f616affd640 1 --2- 172.21.15.78:0/3638163351 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f61580a2c10 0x7f61580a50d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f615c009890 tx=0x7f615c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:11.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:11.886+0000 7f6168ff9640 1 -- 172.21.15.78:0/3638163351 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f61540bc280 con 0x7f616c102240 2024-08-18T23:44:12.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:12.049+0000 7f61717aa640 1 -- 172.21.15.78:0/3638163351 --> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f616c062360 con 0x7f61580a2c10 2024-08-18T23:44:12.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:12.057+0000 7f6168ff9640 1 -- 172.21.15.78:0/3638163351 <== mgr.14150 v2:172.21.15.78:6800/4000744943 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340488 (secure 0 0 0) 0x7f616c062360 con 0x7f61580a2c10 2024-08-18T23:44:12.061 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:44:12.062 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-08-18T23:44:12.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:12.060+0000 7f61717aa640 1 -- 172.21.15.78:0/3638163351 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f61580a2c10 msgr2=0x7f61580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:12.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:12.060+0000 7f61717aa640 1 --2- 172.21.15.78:0/3638163351 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f61580a2c10 0x7f61580a50d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f615c009890 tx=0x7f615c008040 comp rx=0 tx=0).stop 2024-08-18T23:44:12.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:12.060+0000 7f61717aa640 1 -- 172.21.15.78:0/3638163351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f616c102240 msgr2=0x7f616c1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:12.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:12.060+0000 7f61717aa640 1 --2- 172.21.15.78:0/3638163351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f616c102240 0x7f616c1988e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6154009a50 tx=0x7f6154004270 comp rx=0 tx=0).stop 2024-08-18T23:44:12.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:12.060+0000 7f61717aa640 1 -- 172.21.15.78:0/3638163351 shutdown_connections 2024-08-18T23:44:12.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:12.060+0000 7f61717aa640 1 --2- 172.21.15.78:0/3638163351 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f61580a2c10 0x7f61580a50d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:12.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:12.060+0000 7f61717aa640 1 --2- 172.21.15.78:0/3638163351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f616c1078a0 0x7f616c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:12.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:12.060+0000 7f61717aa640 1 --2- 172.21.15.78:0/3638163351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f616c102240 0x7f616c1988e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:12.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:12.060+0000 7f61717aa640 1 --2- 172.21.15.78:0/3638163351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f616c1018a0 0x7f616c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:12.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:12.060+0000 7f61717aa640 1 -- 172.21.15.78:0/3638163351 >> 172.21.15.78:0/3638163351 conn(0x7f616c0fd5e0 msgr2=0x7f616c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:12.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:12.060+0000 7f61717aa640 1 -- 172.21.15.78:0/3638163351 shutdown_connections 2024-08-18T23:44:12.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:12.060+0000 7f61717aa640 1 -- 172.21.15.78:0/3638163351 wait complete. 2024-08-18T23:44:12.772 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-08-18T23:44:10.149469+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667887,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":892,"num_read_kb":576,"num_write":377,"num_write_kb":3085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":804,"ondisk_log_size":804,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392748,"kb_used_data":14456,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549076,"statfs":{"total":1151940427776,"available":1151538253824,"internally_reserved":0,"allocated":14802944,"data_stored":8262771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9368,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":59,"num_write":18,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003327"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298623+0000","last_change":"2024-08-18T23:41:30.120580+0000","last_active":"2024-08-18T23:43:24.298623+0000","last_peered":"2024-08-18T23:43:24.298623+0000","last_clean":"2024-08-18T23:43:24.298623+0000","last_became_active":"2024-08-18T23:41:30.120266+0000","last_became_peered":"2024-08-18T23:41:30.120266+0000","last_unstale":"2024-08-18T23:43:24.298623+0000","last_undegraded":"2024-08-18T23:43:24.298623+0000","last_fullsized":"2024-08-18T23:43:24.298623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:06:40.997070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312511+0000","last_change":"2024-08-18T23:41:32.143232+0000","last_active":"2024-08-18T23:43:24.312511+0000","last_peered":"2024-08-18T23:43:24.312511+0000","last_clean":"2024-08-18T23:43:24.312511+0000","last_became_active":"2024-08-18T23:41:32.142916+0000","last_became_peered":"2024-08-18T23:41:32.142916+0000","last_unstale":"2024-08-18T23:43:24.312511+0000","last_undegraded":"2024-08-18T23:43:24.312511+0000","last_fullsized":"2024-08-18T23:43:24.312511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:50:54.279309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312106+0000","last_change":"2024-08-18T23:41:39.188772+0000","last_active":"2024-08-18T23:43:24.312106+0000","last_peered":"2024-08-18T23:43:24.312106+0000","last_clean":"2024-08-18T23:43:24.312106+0000","last_became_active":"2024-08-18T23:41:39.188362+0000","last_became_peered":"2024-08-18T23:41:39.188362+0000","last_unstale":"2024-08-18T23:43:24.312106+0000","last_undegraded":"2024-08-18T23:43:24.312106+0000","last_fullsized":"2024-08-18T23:43:24.312106+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:47:50.255543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312628+0000","last_change":"2024-08-18T23:41:30.137179+0000","last_active":"2024-08-18T23:43:24.312628+0000","last_peered":"2024-08-18T23:43:24.312628+0000","last_clean":"2024-08-18T23:43:24.312628+0000","last_became_active":"2024-08-18T23:41:30.136618+0000","last_became_peered":"2024-08-18T23:41:30.136618+0000","last_unstale":"2024-08-18T23:43:24.312628+0000","last_undegraded":"2024-08-18T23:43:24.312628+0000","last_fullsized":"2024-08-18T23:43:24.312628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:59.826364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312531+0000","last_change":"2024-08-18T23:41:32.142075+0000","last_active":"2024-08-18T23:43:24.312531+0000","last_peered":"2024-08-18T23:43:24.312531+0000","last_clean":"2024-08-18T23:43:24.312531+0000","last_became_active":"2024-08-18T23:41:32.141709+0000","last_became_peered":"2024-08-18T23:41:32.141709+0000","last_unstale":"2024-08-18T23:43:24.312531+0000","last_undegraded":"2024-08-18T23:43:24.312531+0000","last_fullsized":"2024-08-18T23:43:24.312531+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:08.324627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312210+0000","last_change":"2024-08-18T23:41:39.184575+0000","last_active":"2024-08-18T23:43:24.312210+0000","last_peered":"2024-08-18T23:43:24.312210+0000","last_clean":"2024-08-18T23:43:24.312210+0000","last_became_active":"2024-08-18T23:41:39.184169+0000","last_became_peered":"2024-08-18T23:41:39.184169+0000","last_unstale":"2024-08-18T23:43:24.312210+0000","last_undegraded":"2024-08-18T23:43:24.312210+0000","last_fullsized":"2024-08-18T23:43:24.312210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:20:32.218172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304127+0000","last_change":"2024-08-18T23:41:30.121638+0000","last_active":"2024-08-18T23:43:24.304127+0000","last_peered":"2024-08-18T23:43:24.304127+0000","last_clean":"2024-08-18T23:43:24.304127+0000","last_became_active":"2024-08-18T23:41:30.121290+0000","last_became_peered":"2024-08-18T23:41:30.121290+0000","last_unstale":"2024-08-18T23:43:24.304127+0000","last_undegraded":"2024-08-18T23:43:24.304127+0000","last_fullsized":"2024-08-18T23:43:24.304127+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:43:55.638780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312431+0000","last_change":"2024-08-18T23:41:32.142177+0000","last_active":"2024-08-18T23:43:24.312431+0000","last_peered":"2024-08-18T23:43:24.312431+0000","last_clean":"2024-08-18T23:43:24.312431+0000","last_became_active":"2024-08-18T23:41:32.141731+0000","last_became_peered":"2024-08-18T23:41:32.141731+0000","last_unstale":"2024-08-18T23:43:24.312431+0000","last_undegraded":"2024-08-18T23:43:24.312431+0000","last_fullsized":"2024-08-18T23:43:24.312431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:34:36.283299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312653+0000","last_change":"2024-08-18T23:41:39.182931+0000","last_active":"2024-08-18T23:43:24.312653+0000","last_peered":"2024-08-18T23:43:24.312653+0000","last_clean":"2024-08-18T23:43:24.312653+0000","last_became_active":"2024-08-18T23:41:39.182008+0000","last_became_peered":"2024-08-18T23:41:39.182008+0000","last_unstale":"2024-08-18T23:43:24.312653+0000","last_undegraded":"2024-08-18T23:43:24.312653+0000","last_fullsized":"2024-08-18T23:43:24.312653+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:58:38.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312863+0000","last_change":"2024-08-18T23:42:23.329762+0000","last_active":"2024-08-18T23:43:24.312863+0000","last_peered":"2024-08-18T23:43:24.312863+0000","last_clean":"2024-08-18T23:43:24.312863+0000","last_became_active":"2024-08-18T23:42:23.325753+0000","last_became_peered":"2024-08-18T23:42:23.325753+0000","last_unstale":"2024-08-18T23:43:24.312863+0000","last_undegraded":"2024-08-18T23:43:24.312863+0000","last_fullsized":"2024-08-18T23:43:24.312863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:19:43.062492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307318+0000","last_change":"2024-08-18T23:41:32.148662+0000","last_active":"2024-08-18T23:43:24.307318+0000","last_peered":"2024-08-18T23:43:24.307318+0000","last_clean":"2024-08-18T23:43:24.307318+0000","last_became_active":"2024-08-18T23:41:32.148318+0000","last_became_peered":"2024-08-18T23:41:32.148318+0000","last_unstale":"2024-08-18T23:43:24.307318+0000","last_undegraded":"2024-08-18T23:43:24.307318+0000","last_fullsized":"2024-08-18T23:43:24.307318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:54:58.668669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312296+0000","last_change":"2024-08-18T23:41:39.181114+0000","last_active":"2024-08-18T23:43:24.312296+0000","last_peered":"2024-08-18T23:43:24.312296+0000","last_clean":"2024-08-18T23:43:24.312296+0000","last_became_active":"2024-08-18T23:41:39.180599+0000","last_became_peered":"2024-08-18T23:41:39.180599+0000","last_unstale":"2024-08-18T23:43:24.312296+0000","last_undegraded":"2024-08-18T23:43:24.312296+0000","last_fullsized":"2024-08-18T23:43:24.312296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:36:08.749143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304715+0000","last_change":"2024-08-18T23:41:32.144326+0000","last_active":"2024-08-18T23:43:24.304715+0000","last_peered":"2024-08-18T23:43:24.304715+0000","last_clean":"2024-08-18T23:43:24.304715+0000","last_became_active":"2024-08-18T23:41:32.144004+0000","last_became_peered":"2024-08-18T23:41:32.144004+0000","last_unstale":"2024-08-18T23:43:24.304715+0000","last_undegraded":"2024-08-18T23:43:24.304715+0000","last_fullsized":"2024-08-18T23:43:24.304715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:15:10.918624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303665+0000","last_change":"2024-08-18T23:41:30.125160+0000","last_active":"2024-08-18T23:43:24.303665+0000","last_peered":"2024-08-18T23:43:24.303665+0000","last_clean":"2024-08-18T23:43:24.303665+0000","last_became_active":"2024-08-18T23:41:30.124685+0000","last_became_peered":"2024-08-18T23:41:30.124685+0000","last_unstale":"2024-08-18T23:43:24.303665+0000","last_undegraded":"2024-08-18T23:43:24.303665+0000","last_fullsized":"2024-08-18T23:43:24.303665+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:52:48.997902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313216+0000","last_change":"2024-08-18T23:41:39.681481+0000","last_active":"2024-08-18T23:43:24.313216+0000","last_peered":"2024-08-18T23:43:24.313216+0000","last_clean":"2024-08-18T23:43:24.313216+0000","last_became_active":"2024-08-18T23:41:39.680204+0000","last_became_peered":"2024-08-18T23:41:39.680204+0000","last_unstale":"2024-08-18T23:43:24.313216+0000","last_undegraded":"2024-08-18T23:43:24.313216+0000","last_fullsized":"2024-08-18T23:43:24.313216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:00:05.248180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312231+0000","last_change":"2024-08-18T23:41:32.151335+0000","last_active":"2024-08-18T23:43:24.312231+0000","last_peered":"2024-08-18T23:43:24.312231+0000","last_clean":"2024-08-18T23:43:24.312231+0000","last_became_active":"2024-08-18T23:41:32.151040+0000","last_became_peered":"2024-08-18T23:41:32.151040+0000","last_unstale":"2024-08-18T23:43:24.312231+0000","last_undegraded":"2024-08-18T23:43:24.312231+0000","last_fullsized":"2024-08-18T23:43:24.312231+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:16.108555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"92'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:41.164918+0000","last_change":"2024-08-18T23:41:30.122301+0000","last_active":"2024-08-18T23:43:41.164918+0000","last_peered":"2024-08-18T23:43:41.164918+0000","last_clean":"2024-08-18T23:43:41.164918+0000","last_became_active":"2024-08-18T23:41:30.121117+0000","last_became_peered":"2024-08-18T23:41:30.121117+0000","last_unstale":"2024-08-18T23:43:41.164918+0000","last_undegraded":"2024-08-18T23:43:41.164918+0000","last_fullsized":"2024-08-18T23:43:41.164918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:40:12.678803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312721+0000","last_change":"2024-08-18T23:41:39.182395+0000","last_active":"2024-08-18T23:43:24.312721+0000","last_peered":"2024-08-18T23:43:24.312721+0000","last_clean":"2024-08-18T23:43:24.312721+0000","last_became_active":"2024-08-18T23:41:39.181979+0000","last_became_peered":"2024-08-18T23:41:39.181979+0000","last_unstale":"2024-08-18T23:43:24.312721+0000","last_undegraded":"2024-08-18T23:43:24.312721+0000","last_fullsized":"2024-08-18T23:43:24.312721+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:49:04.674503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312810+0000","last_change":"2024-08-18T23:41:32.140464+0000","last_active":"2024-08-18T23:43:24.312810+0000","last_peered":"2024-08-18T23:43:24.312810+0000","last_clean":"2024-08-18T23:43:24.312810+0000","last_became_active":"2024-08-18T23:41:32.139153+0000","last_became_peered":"2024-08-18T23:41:32.139153+0000","last_unstale":"2024-08-18T23:43:24.312810+0000","last_undegraded":"2024-08-18T23:43:24.312810+0000","last_fullsized":"2024-08-18T23:43:24.312810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:54:27.387680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312414+0000","last_change":"2024-08-18T23:41:30.122787+0000","last_active":"2024-08-18T23:43:24.312414+0000","last_peered":"2024-08-18T23:43:24.312414+0000","last_clean":"2024-08-18T23:43:24.312414+0000","last_became_active":"2024-08-18T23:41:30.122462+0000","last_became_peered":"2024-08-18T23:41:30.122462+0000","last_unstale":"2024-08-18T23:43:24.312414+0000","last_undegraded":"2024-08-18T23:43:24.312414+0000","last_fullsized":"2024-08-18T23:43:24.312414+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:55.568831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298325+0000","last_change":"2024-08-18T23:41:39.190511+0000","last_active":"2024-08-18T23:43:24.298325+0000","last_peered":"2024-08-18T23:43:24.298325+0000","last_clean":"2024-08-18T23:43:24.298325+0000","last_became_active":"2024-08-18T23:41:39.190154+0000","last_became_peered":"2024-08-18T23:41:39.190154+0000","last_unstale":"2024-08-18T23:43:24.298325+0000","last_undegraded":"2024-08-18T23:43:24.298325+0000","last_fullsized":"2024-08-18T23:43:24.298325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:33:34.514827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303456+0000","last_change":"2024-08-18T23:41:32.143867+0000","last_active":"2024-08-18T23:43:24.303456+0000","last_peered":"2024-08-18T23:43:24.303456+0000","last_clean":"2024-08-18T23:43:24.303456+0000","last_became_active":"2024-08-18T23:41:32.143527+0000","last_became_peered":"2024-08-18T23:41:32.143527+0000","last_unstale":"2024-08-18T23:43:24.303456+0000","last_undegraded":"2024-08-18T23:43:24.303456+0000","last_fullsized":"2024-08-18T23:43:24.303456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:40:25.902133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312100+0000","last_change":"2024-08-18T23:42:23.325102+0000","last_active":"2024-08-18T23:43:24.312100+0000","last_peered":"2024-08-18T23:43:24.312100+0000","last_clean":"2024-08-18T23:43:24.312100+0000","last_became_active":"2024-08-18T23:42:23.321583+0000","last_became_peered":"2024-08-18T23:42:23.321583+0000","last_unstale":"2024-08-18T23:43:24.312100+0000","last_undegraded":"2024-08-18T23:43:24.312100+0000","last_fullsized":"2024-08-18T23:43:24.312100+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:57:42.546686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312767+0000","last_change":"2024-08-18T23:42:23.559756+0000","last_active":"2024-08-18T23:43:24.312767+0000","last_peered":"2024-08-18T23:43:24.312767+0000","last_clean":"2024-08-18T23:43:24.312767+0000","last_became_active":"2024-08-18T23:42:23.559242+0000","last_became_peered":"2024-08-18T23:42:23.559242+0000","last_unstale":"2024-08-18T23:43:24.312767+0000","last_undegraded":"2024-08-18T23:43:24.312767+0000","last_fullsized":"2024-08-18T23:43:24.312767+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:25:40.790313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312043+0000","last_change":"2024-08-18T23:41:32.139025+0000","last_active":"2024-08-18T23:43:24.312043+0000","last_peered":"2024-08-18T23:43:24.312043+0000","last_clean":"2024-08-18T23:43:24.312043+0000","last_became_active":"2024-08-18T23:41:32.138733+0000","last_became_peered":"2024-08-18T23:41:32.138733+0000","last_unstale":"2024-08-18T23:43:24.312043+0000","last_undegraded":"2024-08-18T23:43:24.312043+0000","last_fullsized":"2024-08-18T23:43:24.312043+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:01:39.826391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313061+0000","last_change":"2024-08-18T23:41:30.477359+0000","last_active":"2024-08-18T23:43:24.313061+0000","last_peered":"2024-08-18T23:43:24.313061+0000","last_clean":"2024-08-18T23:43:24.313061+0000","last_became_active":"2024-08-18T23:41:30.477011+0000","last_became_peered":"2024-08-18T23:41:30.477011+0000","last_unstale":"2024-08-18T23:43:24.313061+0000","last_undegraded":"2024-08-18T23:43:24.313061+0000","last_fullsized":"2024-08-18T23:43:24.313061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:53:10.052859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312145+0000","last_change":"2024-08-18T23:41:39.182140+0000","last_active":"2024-08-18T23:43:24.312145+0000","last_peered":"2024-08-18T23:43:24.312145+0000","last_clean":"2024-08-18T23:43:24.312145+0000","last_became_active":"2024-08-18T23:41:39.181667+0000","last_became_peered":"2024-08-18T23:41:39.181667+0000","last_unstale":"2024-08-18T23:43:24.312145+0000","last_undegraded":"2024-08-18T23:43:24.312145+0000","last_fullsized":"2024-08-18T23:43:24.312145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:02:18.128028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312594+0000","last_change":"2024-08-18T23:41:32.137064+0000","last_active":"2024-08-18T23:43:24.312594+0000","last_peered":"2024-08-18T23:43:24.312594+0000","last_clean":"2024-08-18T23:43:24.312594+0000","last_became_active":"2024-08-18T23:41:32.136738+0000","last_became_peered":"2024-08-18T23:41:32.136738+0000","last_unstale":"2024-08-18T23:43:24.312594+0000","last_undegraded":"2024-08-18T23:43:24.312594+0000","last_fullsized":"2024-08-18T23:43:24.312594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:15:11.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'14","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:57.167799+0000","last_change":"2024-08-18T23:41:30.120042+0000","last_active":"2024-08-18T23:43:57.167799+0000","last_peered":"2024-08-18T23:43:57.167799+0000","last_clean":"2024-08-18T23:43:57.167799+0000","last_became_active":"2024-08-18T23:41:30.119719+0000","last_became_peered":"2024-08-18T23:41:30.119719+0000","last_unstale":"2024-08-18T23:43:57.167799+0000","last_undegraded":"2024-08-18T23:43:57.167799+0000","last_fullsized":"2024-08-18T23:43:57.167799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:35:44.466330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311900+0000","last_change":"2024-08-18T23:41:39.680405+0000","last_active":"2024-08-18T23:43:24.311900+0000","last_peered":"2024-08-18T23:43:24.311900+0000","last_clean":"2024-08-18T23:43:24.311900+0000","last_became_active":"2024-08-18T23:41:39.679940+0000","last_became_peered":"2024-08-18T23:41:39.679940+0000","last_unstale":"2024-08-18T23:43:24.311900+0000","last_undegraded":"2024-08-18T23:43:24.311900+0000","last_fullsized":"2024-08-18T23:43:24.311900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:15:03.942116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301271+0000","last_change":"2024-08-18T23:41:32.139688+0000","last_active":"2024-08-18T23:43:24.301271+0000","last_peered":"2024-08-18T23:43:24.301271+0000","last_clean":"2024-08-18T23:43:24.301271+0000","last_became_active":"2024-08-18T23:41:32.139397+0000","last_became_peered":"2024-08-18T23:41:32.139397+0000","last_unstale":"2024-08-18T23:43:24.301271+0000","last_undegraded":"2024-08-18T23:43:24.301271+0000","last_fullsized":"2024-08-18T23:43:24.301271+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:03:57.017488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304150+0000","last_change":"2024-08-18T23:41:30.137578+0000","last_active":"2024-08-18T23:43:24.304150+0000","last_peered":"2024-08-18T23:43:24.304150+0000","last_clean":"2024-08-18T23:43:24.304150+0000","last_became_active":"2024-08-18T23:41:30.137246+0000","last_became_peered":"2024-08-18T23:41:30.137246+0000","last_unstale":"2024-08-18T23:43:24.304150+0000","last_undegraded":"2024-08-18T23:43:24.304150+0000","last_fullsized":"2024-08-18T23:43:24.304150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:48:54.839432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312576+0000","last_change":"2024-08-18T23:41:39.186360+0000","last_active":"2024-08-18T23:43:24.312576+0000","last_peered":"2024-08-18T23:43:24.312576+0000","last_clean":"2024-08-18T23:43:24.312576+0000","last_became_active":"2024-08-18T23:41:39.185808+0000","last_became_peered":"2024-08-18T23:41:39.185808+0000","last_unstale":"2024-08-18T23:43:24.312576+0000","last_undegraded":"2024-08-18T23:43:24.312576+0000","last_fullsized":"2024-08-18T23:43:24.312576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:00:53.891061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300916+0000","last_change":"2024-08-18T23:41:32.146070+0000","last_active":"2024-08-18T23:43:24.300916+0000","last_peered":"2024-08-18T23:43:24.300916+0000","last_clean":"2024-08-18T23:43:24.300916+0000","last_became_active":"2024-08-18T23:41:32.145640+0000","last_became_peered":"2024-08-18T23:41:32.145640+0000","last_unstale":"2024-08-18T23:43:24.300916+0000","last_undegraded":"2024-08-18T23:43:24.300916+0000","last_fullsized":"2024-08-18T23:43:24.300916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:56:21.368762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298278+0000","last_change":"2024-08-18T23:41:30.138565+0000","last_active":"2024-08-18T23:43:24.298278+0000","last_peered":"2024-08-18T23:43:24.298278+0000","last_clean":"2024-08-18T23:43:24.298278+0000","last_became_active":"2024-08-18T23:41:30.138210+0000","last_became_peered":"2024-08-18T23:41:30.138210+0000","last_unstale":"2024-08-18T23:43:24.298278+0000","last_undegraded":"2024-08-18T23:43:24.298278+0000","last_fullsized":"2024-08-18T23:43:24.298278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:42:14.928884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303425+0000","last_change":"2024-08-18T23:41:39.680656+0000","last_active":"2024-08-18T23:43:24.303425+0000","last_peered":"2024-08-18T23:43:24.303425+0000","last_clean":"2024-08-18T23:43:24.303425+0000","last_became_active":"2024-08-18T23:41:39.680203+0000","last_became_peered":"2024-08-18T23:41:39.680203+0000","last_unstale":"2024-08-18T23:43:24.303425+0000","last_undegraded":"2024-08-18T23:43:24.303425+0000","last_fullsized":"2024-08-18T23:43:24.303425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:55:43.138869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300932+0000","last_change":"2024-08-18T23:41:32.130222+0000","last_active":"2024-08-18T23:43:24.300932+0000","last_peered":"2024-08-18T23:43:24.300932+0000","last_clean":"2024-08-18T23:43:24.300932+0000","last_became_active":"2024-08-18T23:41:32.129844+0000","last_became_peered":"2024-08-18T23:41:32.129844+0000","last_unstale":"2024-08-18T23:43:24.300932+0000","last_undegraded":"2024-08-18T23:43:24.300932+0000","last_fullsized":"2024-08-18T23:43:24.300932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:39:21.106343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304850+0000","last_change":"2024-08-18T23:41:30.123142+0000","last_active":"2024-08-18T23:43:24.304850+0000","last_peered":"2024-08-18T23:43:24.304850+0000","last_clean":"2024-08-18T23:43:24.304850+0000","last_became_active":"2024-08-18T23:41:30.121645+0000","last_became_peered":"2024-08-18T23:41:30.121645+0000","last_unstale":"2024-08-18T23:43:24.304850+0000","last_undegraded":"2024-08-18T23:43:24.304850+0000","last_fullsized":"2024-08-18T23:43:24.304850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:07:24.912259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301163+0000","last_change":"2024-08-18T23:41:39.186033+0000","last_active":"2024-08-18T23:43:24.301163+0000","last_peered":"2024-08-18T23:43:24.301163+0000","last_clean":"2024-08-18T23:43:24.301163+0000","last_became_active":"2024-08-18T23:41:39.185451+0000","last_became_peered":"2024-08-18T23:41:39.185451+0000","last_unstale":"2024-08-18T23:43:24.301163+0000","last_undegraded":"2024-08-18T23:43:24.301163+0000","last_fullsized":"2024-08-18T23:43:24.301163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:52:13.556760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.306993+0000","last_change":"2024-08-18T23:41:32.141065+0000","last_active":"2024-08-18T23:43:24.306993+0000","last_peered":"2024-08-18T23:43:24.306993+0000","last_clean":"2024-08-18T23:43:24.306993+0000","last_became_active":"2024-08-18T23:41:32.140732+0000","last_became_peered":"2024-08-18T23:41:32.140732+0000","last_unstale":"2024-08-18T23:43:24.306993+0000","last_undegraded":"2024-08-18T23:43:24.306993+0000","last_fullsized":"2024-08-18T23:43:24.306993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:08:32.287835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":166,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:44:07.443889+0000","last_change":"2024-08-18T23:41:30.136838+0000","last_active":"2024-08-18T23:44:07.443889+0000","last_peered":"2024-08-18T23:44:07.443889+0000","last_clean":"2024-08-18T23:44:07.443889+0000","last_became_active":"2024-08-18T23:41:30.136523+0000","last_became_peered":"2024-08-18T23:41:30.136523+0000","last_unstale":"2024-08-18T23:44:07.443889+0000","last_undegraded":"2024-08-18T23:44:07.443889+0000","last_fullsized":"2024-08-18T23:44:07.443889+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:56:17.750535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301550+0000","last_change":"2024-08-18T23:41:39.680314+0000","last_active":"2024-08-18T23:43:24.301550+0000","last_peered":"2024-08-18T23:43:24.301550+0000","last_clean":"2024-08-18T23:43:24.301550+0000","last_became_active":"2024-08-18T23:41:39.679920+0000","last_became_peered":"2024-08-18T23:41:39.679920+0000","last_unstale":"2024-08-18T23:43:24.301550+0000","last_undegraded":"2024-08-18T23:43:24.301550+0000","last_fullsized":"2024-08-18T23:43:24.301550+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:26:29.866081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300888+0000","last_change":"2024-08-18T23:41:32.139752+0000","last_active":"2024-08-18T23:43:24.300888+0000","last_peered":"2024-08-18T23:43:24.300888+0000","last_clean":"2024-08-18T23:43:24.300888+0000","last_became_active":"2024-08-18T23:41:32.139451+0000","last_became_peered":"2024-08-18T23:41:32.139451+0000","last_unstale":"2024-08-18T23:43:24.300888+0000","last_undegraded":"2024-08-18T23:43:24.300888+0000","last_fullsized":"2024-08-18T23:43:24.300888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:15:13.390529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304894+0000","last_change":"2024-08-18T23:41:30.126513+0000","last_active":"2024-08-18T23:43:24.304894+0000","last_peered":"2024-08-18T23:43:24.304894+0000","last_clean":"2024-08-18T23:43:24.304894+0000","last_became_active":"2024-08-18T23:41:30.126046+0000","last_became_peered":"2024-08-18T23:41:30.126046+0000","last_unstale":"2024-08-18T23:43:24.304894+0000","last_undegraded":"2024-08-18T23:43:24.304894+0000","last_fullsized":"2024-08-18T23:43:24.304894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:49:16.565565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307280+0000","last_change":"2024-08-18T23:41:39.681006+0000","last_active":"2024-08-18T23:43:24.307280+0000","last_peered":"2024-08-18T23:43:24.307280+0000","last_clean":"2024-08-18T23:43:24.307280+0000","last_became_active":"2024-08-18T23:41:39.680409+0000","last_became_peered":"2024-08-18T23:41:39.680409+0000","last_unstale":"2024-08-18T23:43:24.307280+0000","last_undegraded":"2024-08-18T23:43:24.307280+0000","last_fullsized":"2024-08-18T23:43:24.307280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:47.778866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312261+0000","last_change":"2024-08-18T23:41:32.147545+0000","last_active":"2024-08-18T23:43:24.312261+0000","last_peered":"2024-08-18T23:43:24.312261+0000","last_clean":"2024-08-18T23:43:24.312261+0000","last_became_active":"2024-08-18T23:41:32.147238+0000","last_became_peered":"2024-08-18T23:41:32.147238+0000","last_unstale":"2024-08-18T23:43:24.312261+0000","last_undegraded":"2024-08-18T23:43:24.312261+0000","last_fullsized":"2024-08-18T23:43:24.312261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:12:13.589535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313054+0000","last_change":"2024-08-18T23:41:30.120650+0000","last_active":"2024-08-18T23:43:24.313054+0000","last_peered":"2024-08-18T23:43:24.313054+0000","last_clean":"2024-08-18T23:43:24.313054+0000","last_became_active":"2024-08-18T23:41:30.120196+0000","last_became_peered":"2024-08-18T23:41:30.120196+0000","last_unstale":"2024-08-18T23:43:24.313054+0000","last_undegraded":"2024-08-18T23:43:24.313054+0000","last_fullsized":"2024-08-18T23:43:24.313054+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:40:23.370088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304710+0000","last_change":"2024-08-18T23:40:22.013519+0000","last_active":"2024-08-18T23:43:24.304710+0000","last_peered":"2024-08-18T23:43:24.304710+0000","last_clean":"2024-08-18T23:43:24.304710+0000","last_became_active":"2024-08-18T23:40:22.012986+0000","last_became_peered":"2024-08-18T23:40:22.012986+0000","last_unstale":"2024-08-18T23:43:24.304710+0000","last_undegraded":"2024-08-18T23:43:24.304710+0000","last_fullsized":"2024-08-18T23:43:24.304710+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:37:22.388424+0000","last_clean_scrub_stamp":"2024-08-18T23:37:22.388424+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:19.874356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304893+0000","last_change":"2024-08-18T23:42:23.332800+0000","last_active":"2024-08-18T23:43:24.304893+0000","last_peered":"2024-08-18T23:43:24.304893+0000","last_clean":"2024-08-18T23:43:24.304893+0000","last_became_active":"2024-08-18T23:42:23.326441+0000","last_became_peered":"2024-08-18T23:42:23.326441+0000","last_unstale":"2024-08-18T23:43:24.304893+0000","last_undegraded":"2024-08-18T23:43:24.304893+0000","last_fullsized":"2024-08-18T23:43:24.304893+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:13:58.243518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.305065+0000","last_change":"2024-08-18T23:41:32.139907+0000","last_active":"2024-08-18T23:43:24.305065+0000","last_peered":"2024-08-18T23:43:24.305065+0000","last_clean":"2024-08-18T23:43:24.305065+0000","last_became_active":"2024-08-18T23:41:32.137956+0000","last_became_peered":"2024-08-18T23:41:32.137956+0000","last_unstale":"2024-08-18T23:43:24.305065+0000","last_undegraded":"2024-08-18T23:43:24.305065+0000","last_fullsized":"2024-08-18T23:43:24.305065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:10:13.223328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312461+0000","last_change":"2024-08-18T23:42:23.330062+0000","last_active":"2024-08-18T23:43:24.312461+0000","last_peered":"2024-08-18T23:43:24.312461+0000","last_clean":"2024-08-18T23:43:24.312461+0000","last_became_active":"2024-08-18T23:42:23.323196+0000","last_became_peered":"2024-08-18T23:42:23.323196+0000","last_unstale":"2024-08-18T23:43:24.312461+0000","last_undegraded":"2024-08-18T23:43:24.312461+0000","last_fullsized":"2024-08-18T23:43:24.312461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:53:29.059542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312185+0000","last_change":"2024-08-18T23:41:39.185917+0000","last_active":"2024-08-18T23:43:24.312185+0000","last_peered":"2024-08-18T23:43:24.312185+0000","last_clean":"2024-08-18T23:43:24.312185+0000","last_became_active":"2024-08-18T23:41:39.185427+0000","last_became_peered":"2024-08-18T23:41:39.185427+0000","last_unstale":"2024-08-18T23:43:24.312185+0000","last_undegraded":"2024-08-18T23:43:24.312185+0000","last_fullsized":"2024-08-18T23:43:24.312185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:52:13.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.308822+0000","last_change":"2024-08-18T23:43:24.308401+0000","last_active":"2024-08-18T23:43:24.308822+0000","last_peered":"2024-08-18T23:43:24.308822+0000","last_clean":"2024-08-18T23:43:24.308822+0000","last_became_active":"2024-08-18T23:43:24.308081+0000","last_became_peered":"2024-08-18T23:43:24.308081+0000","last_unstale":"2024-08-18T23:43:24.308822+0000","last_undegraded":"2024-08-18T23:43:24.308822+0000","last_fullsized":"2024-08-18T23:43:24.308822+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:28:57.712217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307528+0000","last_change":"2024-08-18T23:41:30.121307+0000","last_active":"2024-08-18T23:43:24.307528+0000","last_peered":"2024-08-18T23:43:24.307528+0000","last_clean":"2024-08-18T23:43:24.307528+0000","last_became_active":"2024-08-18T23:41:30.120932+0000","last_became_peered":"2024-08-18T23:41:30.120932+0000","last_unstale":"2024-08-18T23:43:24.307528+0000","last_undegraded":"2024-08-18T23:43:24.307528+0000","last_fullsized":"2024-08-18T23:43:24.307528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:06:07.742880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311751+0000","last_change":"2024-08-18T23:41:39.184647+0000","last_active":"2024-08-18T23:43:24.311751+0000","last_peered":"2024-08-18T23:43:24.311751+0000","last_clean":"2024-08-18T23:43:24.311751+0000","last_became_active":"2024-08-18T23:41:39.184251+0000","last_became_peered":"2024-08-18T23:41:39.184251+0000","last_unstale":"2024-08-18T23:43:24.311751+0000","last_undegraded":"2024-08-18T23:43:24.311751+0000","last_fullsized":"2024-08-18T23:43:24.311751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:06:55.200801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.305184+0000","last_change":"2024-08-18T23:41:32.137007+0000","last_active":"2024-08-18T23:43:24.305184+0000","last_peered":"2024-08-18T23:43:24.305184+0000","last_clean":"2024-08-18T23:43:24.305184+0000","last_became_active":"2024-08-18T23:41:32.136684+0000","last_became_peered":"2024-08-18T23:41:32.136684+0000","last_unstale":"2024-08-18T23:43:24.305184+0000","last_undegraded":"2024-08-18T23:43:24.305184+0000","last_fullsized":"2024-08-18T23:43:24.305184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:21:26.584303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301471+0000","last_change":"2024-08-18T23:41:30.121183+0000","last_active":"2024-08-18T23:43:24.301471+0000","last_peered":"2024-08-18T23:43:24.301471+0000","last_clean":"2024-08-18T23:43:24.301471+0000","last_became_active":"2024-08-18T23:41:30.120847+0000","last_became_peered":"2024-08-18T23:41:30.120847+0000","last_unstale":"2024-08-18T23:43:24.301471+0000","last_undegraded":"2024-08-18T23:43:24.301471+0000","last_fullsized":"2024-08-18T23:43:24.301471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:51.025821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.306833+0000","last_change":"2024-08-18T23:41:39.183375+0000","last_active":"2024-08-18T23:43:24.306833+0000","last_peered":"2024-08-18T23:43:24.306833+0000","last_clean":"2024-08-18T23:43:24.306833+0000","last_became_active":"2024-08-18T23:41:39.182339+0000","last_became_peered":"2024-08-18T23:41:39.182339+0000","last_unstale":"2024-08-18T23:43:24.306833+0000","last_undegraded":"2024-08-18T23:43:24.306833+0000","last_fullsized":"2024-08-18T23:43:24.306833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:45:26.119056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313146+0000","last_change":"2024-08-18T23:41:32.140748+0000","last_active":"2024-08-18T23:43:24.313146+0000","last_peered":"2024-08-18T23:43:24.313146+0000","last_clean":"2024-08-18T23:43:24.313146+0000","last_became_active":"2024-08-18T23:41:32.140306+0000","last_became_peered":"2024-08-18T23:41:32.140306+0000","last_unstale":"2024-08-18T23:43:24.313146+0000","last_undegraded":"2024-08-18T23:43:24.313146+0000","last_fullsized":"2024-08-18T23:43:24.313146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:23:02.371529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:48.359280+0000","last_change":"2024-08-18T23:41:30.121617+0000","last_active":"2024-08-18T23:43:48.359280+0000","last_peered":"2024-08-18T23:43:48.359280+0000","last_clean":"2024-08-18T23:43:48.359280+0000","last_became_active":"2024-08-18T23:41:30.121257+0000","last_became_peered":"2024-08-18T23:41:30.121257+0000","last_unstale":"2024-08-18T23:43:48.359280+0000","last_undegraded":"2024-08-18T23:43:48.359280+0000","last_fullsized":"2024-08-18T23:43:48.359280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:23:42.091553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307008+0000","last_change":"2024-08-18T23:41:39.183833+0000","last_active":"2024-08-18T23:43:24.307008+0000","last_peered":"2024-08-18T23:43:24.307008+0000","last_clean":"2024-08-18T23:43:24.307008+0000","last_became_active":"2024-08-18T23:41:39.183176+0000","last_became_peered":"2024-08-18T23:41:39.183176+0000","last_unstale":"2024-08-18T23:43:24.307008+0000","last_undegraded":"2024-08-18T23:43:24.307008+0000","last_fullsized":"2024-08-18T23:43:24.307008+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:36:09.827378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307216+0000","last_change":"2024-08-18T23:41:32.147507+0000","last_active":"2024-08-18T23:43:24.307216+0000","last_peered":"2024-08-18T23:43:24.307216+0000","last_clean":"2024-08-18T23:43:24.307216+0000","last_became_active":"2024-08-18T23:41:32.146042+0000","last_became_peered":"2024-08-18T23:41:32.146042+0000","last_unstale":"2024-08-18T23:43:24.307216+0000","last_undegraded":"2024-08-18T23:43:24.307216+0000","last_fullsized":"2024-08-18T23:43:24.307216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:19:48.941442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303448+0000","last_change":"2024-08-18T23:41:30.137755+0000","last_active":"2024-08-18T23:43:24.303448+0000","last_peered":"2024-08-18T23:43:24.303448+0000","last_clean":"2024-08-18T23:43:24.303448+0000","last_became_active":"2024-08-18T23:41:30.137194+0000","last_became_peered":"2024-08-18T23:41:30.137194+0000","last_unstale":"2024-08-18T23:43:24.303448+0000","last_undegraded":"2024-08-18T23:43:24.303448+0000","last_fullsized":"2024-08-18T23:43:24.303448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:15:49.681405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301357+0000","last_change":"2024-08-18T23:41:39.183986+0000","last_active":"2024-08-18T23:43:24.301357+0000","last_peered":"2024-08-18T23:43:24.301357+0000","last_clean":"2024-08-18T23:43:24.301357+0000","last_became_active":"2024-08-18T23:41:39.183384+0000","last_became_peered":"2024-08-18T23:41:39.183384+0000","last_unstale":"2024-08-18T23:43:24.301357+0000","last_undegraded":"2024-08-18T23:43:24.301357+0000","last_fullsized":"2024-08-18T23:43:24.301357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:04:28.920804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304496+0000","last_change":"2024-08-18T23:41:32.134729+0000","last_active":"2024-08-18T23:43:24.304496+0000","last_peered":"2024-08-18T23:43:24.304496+0000","last_clean":"2024-08-18T23:43:24.304496+0000","last_became_active":"2024-08-18T23:41:32.134366+0000","last_became_peered":"2024-08-18T23:41:32.134366+0000","last_unstale":"2024-08-18T23:43:24.304496+0000","last_undegraded":"2024-08-18T23:43:24.304496+0000","last_fullsized":"2024-08-18T23:43:24.304496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:34:38.610561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'6","reported_seq":172,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:44:08.446040+0000","last_change":"2024-08-18T23:41:30.126443+0000","last_active":"2024-08-18T23:44:08.446040+0000","last_peered":"2024-08-18T23:44:08.446040+0000","last_clean":"2024-08-18T23:44:08.446040+0000","last_became_active":"2024-08-18T23:41:30.124264+0000","last_became_peered":"2024-08-18T23:41:30.124264+0000","last_unstale":"2024-08-18T23:44:08.446040+0000","last_undegraded":"2024-08-18T23:44:08.446040+0000","last_fullsized":"2024-08-18T23:44:08.446040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:56:43.727114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1427,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298528+0000","last_change":"2024-08-18T23:41:39.192164+0000","last_active":"2024-08-18T23:43:24.298528+0000","last_peered":"2024-08-18T23:43:24.298528+0000","last_clean":"2024-08-18T23:43:24.298528+0000","last_became_active":"2024-08-18T23:41:39.191833+0000","last_became_peered":"2024-08-18T23:41:39.191833+0000","last_unstale":"2024-08-18T23:43:24.298528+0000","last_undegraded":"2024-08-18T23:43:24.298528+0000","last_fullsized":"2024-08-18T23:43:24.298528+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:33:50.243930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304927+0000","last_change":"2024-08-18T23:42:23.320662+0000","last_active":"2024-08-18T23:43:24.304927+0000","last_peered":"2024-08-18T23:43:24.304927+0000","last_clean":"2024-08-18T23:43:24.304927+0000","last_became_active":"2024-08-18T23:42:23.320249+0000","last_became_peered":"2024-08-18T23:42:23.320249+0000","last_unstale":"2024-08-18T23:43:24.304927+0000","last_undegraded":"2024-08-18T23:43:24.304927+0000","last_fullsized":"2024-08-18T23:43:24.304927+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:03:17.121915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307658+0000","last_change":"2024-08-18T23:41:30.124071+0000","last_active":"2024-08-18T23:43:24.307658+0000","last_peered":"2024-08-18T23:43:24.307658+0000","last_clean":"2024-08-18T23:43:24.307658+0000","last_became_active":"2024-08-18T23:41:30.123118+0000","last_became_peered":"2024-08-18T23:41:30.123118+0000","last_unstale":"2024-08-18T23:43:24.307658+0000","last_undegraded":"2024-08-18T23:43:24.307658+0000","last_fullsized":"2024-08-18T23:43:24.307658+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:21:39.015187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313136+0000","last_change":"2024-08-18T23:41:39.184038+0000","last_active":"2024-08-18T23:43:24.313136+0000","last_peered":"2024-08-18T23:43:24.313136+0000","last_clean":"2024-08-18T23:43:24.313136+0000","last_became_active":"2024-08-18T23:41:39.183611+0000","last_became_peered":"2024-08-18T23:41:39.183611+0000","last_unstale":"2024-08-18T23:43:24.313136+0000","last_undegraded":"2024-08-18T23:43:24.313136+0000","last_fullsized":"2024-08-18T23:43:24.313136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:32:26.029151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300998+0000","last_change":"2024-08-18T23:41:32.143631+0000","last_active":"2024-08-18T23:43:24.300998+0000","last_peered":"2024-08-18T23:43:24.300998+0000","last_clean":"2024-08-18T23:43:24.300998+0000","last_became_active":"2024-08-18T23:41:32.143350+0000","last_became_peered":"2024-08-18T23:41:32.143350+0000","last_unstale":"2024-08-18T23:43:24.300998+0000","last_undegraded":"2024-08-18T23:43:24.300998+0000","last_fullsized":"2024-08-18T23:43:24.300998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:41:23.713467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312424+0000","last_change":"2024-08-18T23:41:30.123383+0000","last_active":"2024-08-18T23:43:24.312424+0000","last_peered":"2024-08-18T23:43:24.312424+0000","last_clean":"2024-08-18T23:43:24.312424+0000","last_became_active":"2024-08-18T23:41:30.123037+0000","last_became_peered":"2024-08-18T23:41:30.123037+0000","last_unstale":"2024-08-18T23:43:24.312424+0000","last_undegraded":"2024-08-18T23:43:24.312424+0000","last_fullsized":"2024-08-18T23:43:24.312424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:54:27.702478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307148+0000","last_change":"2024-08-18T23:41:39.184302+0000","last_active":"2024-08-18T23:43:24.307148+0000","last_peered":"2024-08-18T23:43:24.307148+0000","last_clean":"2024-08-18T23:43:24.307148+0000","last_became_active":"2024-08-18T23:41:39.183172+0000","last_became_peered":"2024-08-18T23:41:39.183172+0000","last_unstale":"2024-08-18T23:43:24.307148+0000","last_undegraded":"2024-08-18T23:43:24.307148+0000","last_fullsized":"2024-08-18T23:43:24.307148+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:23:26.655865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304016+0000","last_change":"2024-08-18T23:41:32.137426+0000","last_active":"2024-08-18T23:43:24.304016+0000","last_peered":"2024-08-18T23:43:24.304016+0000","last_clean":"2024-08-18T23:43:24.304016+0000","last_became_active":"2024-08-18T23:41:32.137106+0000","last_became_peered":"2024-08-18T23:41:32.137106+0000","last_unstale":"2024-08-18T23:43:24.304016+0000","last_undegraded":"2024-08-18T23:43:24.304016+0000","last_fullsized":"2024-08-18T23:43:24.304016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:34:51.889663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'41","reported_seq":339,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:44:09.447214+0000","last_change":"2024-08-18T23:41:30.124320+0000","last_active":"2024-08-18T23:44:09.447214+0000","last_peered":"2024-08-18T23:44:09.447214+0000","last_clean":"2024-08-18T23:44:09.447214+0000","last_became_active":"2024-08-18T23:41:30.123979+0000","last_became_peered":"2024-08-18T23:41:30.123979+0000","last_unstale":"2024-08-18T23:44:09.447214+0000","last_undegraded":"2024-08-18T23:44:09.447214+0000","last_fullsized":"2024-08-18T23:44:09.447214+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:31:03.689720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71915,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":259,"num_read_kb":201,"num_write":42,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298022+0000","last_change":"2024-08-18T23:41:39.190481+0000","last_active":"2024-08-18T23:43:24.298022+0000","last_peered":"2024-08-18T23:43:24.298022+0000","last_clean":"2024-08-18T23:43:24.298022+0000","last_became_active":"2024-08-18T23:41:39.190076+0000","last_became_peered":"2024-08-18T23:41:39.190076+0000","last_unstale":"2024-08-18T23:43:24.298022+0000","last_undegraded":"2024-08-18T23:43:24.298022+0000","last_fullsized":"2024-08-18T23:43:24.298022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:47:39.313142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312507+0000","last_change":"2024-08-18T23:41:32.145422+0000","last_active":"2024-08-18T23:43:24.312507+0000","last_peered":"2024-08-18T23:43:24.312507+0000","last_clean":"2024-08-18T23:43:24.312507+0000","last_became_active":"2024-08-18T23:41:32.145047+0000","last_became_peered":"2024-08-18T23:41:32.145047+0000","last_unstale":"2024-08-18T23:43:24.312507+0000","last_undegraded":"2024-08-18T23:43:24.312507+0000","last_fullsized":"2024-08-18T23:43:24.312507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:31.905248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307467+0000","last_change":"2024-08-18T23:41:30.137364+0000","last_active":"2024-08-18T23:43:24.307467+0000","last_peered":"2024-08-18T23:43:24.307467+0000","last_clean":"2024-08-18T23:43:24.307467+0000","last_became_active":"2024-08-18T23:41:30.137004+0000","last_became_peered":"2024-08-18T23:41:30.137004+0000","last_unstale":"2024-08-18T23:43:24.307467+0000","last_undegraded":"2024-08-18T23:43:24.307467+0000","last_fullsized":"2024-08-18T23:43:24.307467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:20:56.392149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312779+0000","last_change":"2024-08-18T23:42:23.326359+0000","last_active":"2024-08-18T23:43:24.312779+0000","last_peered":"2024-08-18T23:43:24.312779+0000","last_clean":"2024-08-18T23:43:24.312779+0000","last_became_active":"2024-08-18T23:42:23.325918+0000","last_became_peered":"2024-08-18T23:42:23.325918+0000","last_unstale":"2024-08-18T23:43:24.312779+0000","last_undegraded":"2024-08-18T23:43:24.312779+0000","last_fullsized":"2024-08-18T23:43:24.312779+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:01:15.476268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312633+0000","last_change":"2024-08-18T23:42:23.325691+0000","last_active":"2024-08-18T23:43:24.312633+0000","last_peered":"2024-08-18T23:43:24.312633+0000","last_clean":"2024-08-18T23:43:24.312633+0000","last_became_active":"2024-08-18T23:42:23.325467+0000","last_became_peered":"2024-08-18T23:42:23.325467+0000","last_unstale":"2024-08-18T23:43:24.312633+0000","last_undegraded":"2024-08-18T23:43:24.312633+0000","last_fullsized":"2024-08-18T23:43:24.312633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:10:26.288880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.305106+0000","last_change":"2024-08-18T23:41:30.124430+0000","last_active":"2024-08-18T23:43:24.305106+0000","last_peered":"2024-08-18T23:43:24.305106+0000","last_clean":"2024-08-18T23:43:24.305106+0000","last_became_active":"2024-08-18T23:41:30.123626+0000","last_became_peered":"2024-08-18T23:41:30.123626+0000","last_unstale":"2024-08-18T23:43:24.305106+0000","last_undegraded":"2024-08-18T23:43:24.305106+0000","last_fullsized":"2024-08-18T23:43:24.305106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:26:18.426010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.297855+0000","last_change":"2024-08-18T23:41:39.192765+0000","last_active":"2024-08-18T23:43:24.297855+0000","last_peered":"2024-08-18T23:43:24.297855+0000","last_clean":"2024-08-18T23:43:24.297855+0000","last_became_active":"2024-08-18T23:41:39.192250+0000","last_became_peered":"2024-08-18T23:41:39.192250+0000","last_unstale":"2024-08-18T23:43:24.297855+0000","last_undegraded":"2024-08-18T23:43:24.297855+0000","last_fullsized":"2024-08-18T23:43:24.297855+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:51:16.794404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312750+0000","last_change":"2024-08-18T23:41:32.141966+0000","last_active":"2024-08-18T23:43:24.312750+0000","last_peered":"2024-08-18T23:43:24.312750+0000","last_clean":"2024-08-18T23:43:24.312750+0000","last_became_active":"2024-08-18T23:41:32.141561+0000","last_became_peered":"2024-08-18T23:41:32.141561+0000","last_unstale":"2024-08-18T23:43:24.312750+0000","last_undegraded":"2024-08-18T23:43:24.312750+0000","last_fullsized":"2024-08-18T23:43:24.312750+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:16:20.696089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":166,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:44:08.445447+0000","last_change":"2024-08-18T23:41:30.121598+0000","last_active":"2024-08-18T23:44:08.445447+0000","last_peered":"2024-08-18T23:44:08.445447+0000","last_clean":"2024-08-18T23:44:08.445447+0000","last_became_active":"2024-08-18T23:41:30.120533+0000","last_became_peered":"2024-08-18T23:41:30.120533+0000","last_unstale":"2024-08-18T23:44:08.445447+0000","last_undegraded":"2024-08-18T23:44:08.445447+0000","last_fullsized":"2024-08-18T23:44:08.445447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:06:37.458272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312273+0000","last_change":"2024-08-18T23:41:39.178877+0000","last_active":"2024-08-18T23:43:24.312273+0000","last_peered":"2024-08-18T23:43:24.312273+0000","last_clean":"2024-08-18T23:43:24.312273+0000","last_became_active":"2024-08-18T23:41:39.178381+0000","last_became_peered":"2024-08-18T23:41:39.178381+0000","last_unstale":"2024-08-18T23:43:24.312273+0000","last_undegraded":"2024-08-18T23:43:24.312273+0000","last_fullsized":"2024-08-18T23:43:24.312273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:11:27.275893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311702+0000","last_change":"2024-08-18T23:41:39.188381+0000","last_active":"2024-08-18T23:43:24.311702+0000","last_peered":"2024-08-18T23:43:24.311702+0000","last_clean":"2024-08-18T23:43:24.311702+0000","last_became_active":"2024-08-18T23:41:39.187983+0000","last_became_peered":"2024-08-18T23:41:39.187983+0000","last_unstale":"2024-08-18T23:43:24.311702+0000","last_undegraded":"2024-08-18T23:43:24.311702+0000","last_fullsized":"2024-08-18T23:43:24.311702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:04:45.460576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.297935+0000","last_change":"2024-08-18T23:41:32.144637+0000","last_active":"2024-08-18T23:43:24.297935+0000","last_peered":"2024-08-18T23:43:24.297935+0000","last_clean":"2024-08-18T23:43:24.297935+0000","last_became_active":"2024-08-18T23:41:32.144329+0000","last_became_peered":"2024-08-18T23:41:32.144329+0000","last_unstale":"2024-08-18T23:43:24.297935+0000","last_undegraded":"2024-08-18T23:43:24.297935+0000","last_fullsized":"2024-08-18T23:43:24.297935+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T08:29:39.791264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307199+0000","last_change":"2024-08-18T23:41:30.122325+0000","last_active":"2024-08-18T23:43:24.307199+0000","last_peered":"2024-08-18T23:43:24.307199+0000","last_clean":"2024-08-18T23:43:24.307199+0000","last_became_active":"2024-08-18T23:41:30.121480+0000","last_became_peered":"2024-08-18T23:41:30.121480+0000","last_unstale":"2024-08-18T23:43:24.307199+0000","last_undegraded":"2024-08-18T23:43:24.307199+0000","last_fullsized":"2024-08-18T23:43:24.307199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:32:12.810650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312772+0000","last_change":"2024-08-18T23:42:23.560632+0000","last_active":"2024-08-18T23:43:24.312772+0000","last_peered":"2024-08-18T23:43:24.312772+0000","last_clean":"2024-08-18T23:43:24.312772+0000","last_became_active":"2024-08-18T23:42:23.559920+0000","last_became_peered":"2024-08-18T23:42:23.559920+0000","last_unstale":"2024-08-18T23:43:24.312772+0000","last_undegraded":"2024-08-18T23:43:24.312772+0000","last_fullsized":"2024-08-18T23:43:24.312772+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:01:59.580843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312091+0000","last_change":"2024-08-18T23:41:32.145545+0000","last_active":"2024-08-18T23:43:24.312091+0000","last_peered":"2024-08-18T23:43:24.312091+0000","last_clean":"2024-08-18T23:43:24.312091+0000","last_became_active":"2024-08-18T23:41:32.145183+0000","last_became_peered":"2024-08-18T23:41:32.145183+0000","last_unstale":"2024-08-18T23:43:24.312091+0000","last_undegraded":"2024-08-18T23:43:24.312091+0000","last_fullsized":"2024-08-18T23:43:24.312091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:03:04.111658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300895+0000","last_change":"2024-08-18T23:41:30.118776+0000","last_active":"2024-08-18T23:43:24.300895+0000","last_peered":"2024-08-18T23:43:24.300895+0000","last_clean":"2024-08-18T23:43:24.300895+0000","last_became_active":"2024-08-18T23:41:30.118436+0000","last_became_peered":"2024-08-18T23:41:30.118436+0000","last_unstale":"2024-08-18T23:43:24.300895+0000","last_undegraded":"2024-08-18T23:43:24.300895+0000","last_fullsized":"2024-08-18T23:43:24.300895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:55:13.980927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300932+0000","last_change":"2024-08-18T23:41:30.125201+0000","last_active":"2024-08-18T23:43:24.300932+0000","last_peered":"2024-08-18T23:43:24.300932+0000","last_clean":"2024-08-18T23:43:24.300932+0000","last_became_active":"2024-08-18T23:41:30.124894+0000","last_became_peered":"2024-08-18T23:41:30.124894+0000","last_unstale":"2024-08-18T23:43:24.300932+0000","last_undegraded":"2024-08-18T23:43:24.300932+0000","last_fullsized":"2024-08-18T23:43:24.300932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:30:17.721829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.307776+0000","last_change":"2024-08-18T23:41:32.145452+0000","last_active":"2024-08-18T23:43:24.307776+0000","last_peered":"2024-08-18T23:43:24.307776+0000","last_clean":"2024-08-18T23:43:24.307776+0000","last_became_active":"2024-08-18T23:41:32.145151+0000","last_became_peered":"2024-08-18T23:41:32.145151+0000","last_unstale":"2024-08-18T23:43:24.307776+0000","last_undegraded":"2024-08-18T23:43:24.307776+0000","last_fullsized":"2024-08-18T23:43:24.307776+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:08:08.713154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312361+0000","last_change":"2024-08-18T23:41:39.193334+0000","last_active":"2024-08-18T23:43:24.312361+0000","last_peered":"2024-08-18T23:43:24.312361+0000","last_clean":"2024-08-18T23:43:24.312361+0000","last_became_active":"2024-08-18T23:41:39.192924+0000","last_became_peered":"2024-08-18T23:41:39.192924+0000","last_unstale":"2024-08-18T23:43:24.312361+0000","last_undegraded":"2024-08-18T23:43:24.312361+0000","last_fullsized":"2024-08-18T23:43:24.312361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:07.016300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":171,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:44:05.441977+0000","last_change":"2024-08-18T23:41:30.125217+0000","last_active":"2024-08-18T23:44:05.441977+0000","last_peered":"2024-08-18T23:44:05.441977+0000","last_clean":"2024-08-18T23:44:05.441977+0000","last_became_active":"2024-08-18T23:41:30.124889+0000","last_became_peered":"2024-08-18T23:41:30.124889+0000","last_unstale":"2024-08-18T23:44:05.441977+0000","last_undegraded":"2024-08-18T23:44:05.441977+0000","last_fullsized":"2024-08-18T23:44:05.441977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:29.102309+0000","last_clean_scrub_stamp":"2024-08-18T23:41:29.102309+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:43.389537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303531+0000","last_change":"2024-08-18T23:41:32.142585+0000","last_active":"2024-08-18T23:43:24.303531+0000","last_peered":"2024-08-18T23:43:24.303531+0000","last_clean":"2024-08-18T23:43:24.303531+0000","last_became_active":"2024-08-18T23:41:32.142207+0000","last_became_peered":"2024-08-18T23:41:32.142207+0000","last_unstale":"2024-08-18T23:43:24.303531+0000","last_undegraded":"2024-08-18T23:43:24.303531+0000","last_fullsized":"2024-08-18T23:43:24.303531+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:16:58.755444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312344+0000","last_change":"2024-08-18T23:41:39.185618+0000","last_active":"2024-08-18T23:43:24.312344+0000","last_peered":"2024-08-18T23:43:24.312344+0000","last_clean":"2024-08-18T23:43:24.312344+0000","last_became_active":"2024-08-18T23:41:39.185222+0000","last_became_peered":"2024-08-18T23:41:39.185222+0000","last_unstale":"2024-08-18T23:43:24.312344+0000","last_undegraded":"2024-08-18T23:43:24.312344+0000","last_fullsized":"2024-08-18T23:43:24.312344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:38.163790+0000","last_clean_scrub_stamp":"2024-08-18T23:41:38.163790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:01:27.290998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312646+0000","last_change":"2024-08-18T23:41:32.140719+0000","last_active":"2024-08-18T23:43:24.312646+0000","last_peered":"2024-08-18T23:43:24.312646+0000","last_clean":"2024-08-18T23:43:24.312646+0000","last_became_active":"2024-08-18T23:41:32.140131+0000","last_became_peered":"2024-08-18T23:41:32.140131+0000","last_unstale":"2024-08-18T23:43:24.312646+0000","last_undegraded":"2024-08-18T23:43:24.312646+0000","last_fullsized":"2024-08-18T23:43:24.312646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:48:26.778094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312375+0000","last_change":"2024-08-18T23:42:23.326683+0000","last_active":"2024-08-18T23:43:24.312375+0000","last_peered":"2024-08-18T23:43:24.312375+0000","last_clean":"2024-08-18T23:43:24.312375+0000","last_became_active":"2024-08-18T23:42:23.326309+0000","last_became_peered":"2024-08-18T23:42:23.326309+0000","last_unstale":"2024-08-18T23:43:24.312375+0000","last_undegraded":"2024-08-18T23:43:24.312375+0000","last_fullsized":"2024-08-18T23:43:24.312375+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:38:14.320894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312340+0000","last_change":"2024-08-18T23:41:32.149522+0000","last_active":"2024-08-18T23:43:24.312340+0000","last_peered":"2024-08-18T23:43:24.312340+0000","last_clean":"2024-08-18T23:43:24.312340+0000","last_became_active":"2024-08-18T23:41:32.149182+0000","last_became_peered":"2024-08-18T23:41:32.149182+0000","last_unstale":"2024-08-18T23:43:24.312340+0000","last_undegraded":"2024-08-18T23:43:24.312340+0000","last_fullsized":"2024-08-18T23:43:24.312340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T02:30:16.945045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311765+0000","last_change":"2024-08-18T23:41:32.149853+0000","last_active":"2024-08-18T23:43:24.311765+0000","last_peered":"2024-08-18T23:43:24.311765+0000","last_clean":"2024-08-18T23:43:24.311765+0000","last_became_active":"2024-08-18T23:41:32.141907+0000","last_became_peered":"2024-08-18T23:41:32.141907+0000","last_unstale":"2024-08-18T23:43:24.311765+0000","last_undegraded":"2024-08-18T23:43:24.311765+0000","last_fullsized":"2024-08-18T23:43:24.311765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:24:37.323567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301279+0000","last_change":"2024-08-18T23:41:32.136179+0000","last_active":"2024-08-18T23:43:24.301279+0000","last_peered":"2024-08-18T23:43:24.301279+0000","last_clean":"2024-08-18T23:43:24.301279+0000","last_became_active":"2024-08-18T23:41:32.135939+0000","last_became_peered":"2024-08-18T23:41:32.135939+0000","last_unstale":"2024-08-18T23:43:24.301279+0000","last_undegraded":"2024-08-18T23:43:24.301279+0000","last_fullsized":"2024-08-18T23:43:24.301279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:14:15.392061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313279+0000","last_change":"2024-08-18T23:41:32.137509+0000","last_active":"2024-08-18T23:43:24.313279+0000","last_peered":"2024-08-18T23:43:24.313279+0000","last_clean":"2024-08-18T23:43:24.313279+0000","last_became_active":"2024-08-18T23:41:32.136380+0000","last_became_peered":"2024-08-18T23:41:32.136380+0000","last_unstale":"2024-08-18T23:43:24.313279+0000","last_undegraded":"2024-08-18T23:43:24.313279+0000","last_fullsized":"2024-08-18T23:43:24.313279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:49.776225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312602+0000","last_change":"2024-08-18T23:41:32.139271+0000","last_active":"2024-08-18T23:43:24.312602+0000","last_peered":"2024-08-18T23:43:24.312602+0000","last_clean":"2024-08-18T23:43:24.312602+0000","last_became_active":"2024-08-18T23:41:32.138868+0000","last_became_peered":"2024-08-18T23:41:32.138868+0000","last_unstale":"2024-08-18T23:43:24.312602+0000","last_undegraded":"2024-08-18T23:43:24.312602+0000","last_fullsized":"2024-08-18T23:43:24.312602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:49:27.890857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312543+0000","last_change":"2024-08-18T23:41:32.144460+0000","last_active":"2024-08-18T23:43:24.312543+0000","last_peered":"2024-08-18T23:43:24.312543+0000","last_clean":"2024-08-18T23:43:24.312543+0000","last_became_active":"2024-08-18T23:41:32.144178+0000","last_became_peered":"2024-08-18T23:41:32.144178+0000","last_unstale":"2024-08-18T23:43:24.312543+0000","last_undegraded":"2024-08-18T23:43:24.312543+0000","last_fullsized":"2024-08-18T23:43:24.312543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:02:56.252215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.306952+0000","last_change":"2024-08-18T23:41:32.147973+0000","last_active":"2024-08-18T23:43:24.306952+0000","last_peered":"2024-08-18T23:43:24.306952+0000","last_clean":"2024-08-18T23:43:24.306952+0000","last_became_active":"2024-08-18T23:41:32.147621+0000","last_became_peered":"2024-08-18T23:41:32.147621+0000","last_unstale":"2024-08-18T23:43:24.306952+0000","last_undegraded":"2024-08-18T23:43:24.306952+0000","last_fullsized":"2024-08-18T23:43:24.306952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-19T23:44:14.176569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312329+0000","last_change":"2024-08-18T23:41:32.146029+0000","last_active":"2024-08-18T23:43:24.312329+0000","last_peered":"2024-08-18T23:43:24.312329+0000","last_clean":"2024-08-18T23:43:24.312329+0000","last_became_active":"2024-08-18T23:41:32.145584+0000","last_became_peered":"2024-08-18T23:41:32.145584+0000","last_unstale":"2024-08-18T23:43:24.312329+0000","last_undegraded":"2024-08-18T23:43:24.312329+0000","last_fullsized":"2024-08-18T23:43:24.312329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:25:33.884906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312304+0000","last_change":"2024-08-18T23:41:32.145950+0000","last_active":"2024-08-18T23:43:24.312304+0000","last_peered":"2024-08-18T23:43:24.312304+0000","last_clean":"2024-08-18T23:43:24.312304+0000","last_became_active":"2024-08-18T23:41:32.145618+0000","last_became_peered":"2024-08-18T23:41:32.145618+0000","last_unstale":"2024-08-18T23:43:24.312304+0000","last_undegraded":"2024-08-18T23:43:24.312304+0000","last_fullsized":"2024-08-18T23:43:24.312304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:05:58.889240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301154+0000","last_change":"2024-08-18T23:41:32.130684+0000","last_active":"2024-08-18T23:43:24.301154+0000","last_peered":"2024-08-18T23:43:24.301154+0000","last_clean":"2024-08-18T23:43:24.301154+0000","last_became_active":"2024-08-18T23:41:32.130369+0000","last_became_peered":"2024-08-18T23:41:32.130369+0000","last_unstale":"2024-08-18T23:43:24.301154+0000","last_undegraded":"2024-08-18T23:43:24.301154+0000","last_fullsized":"2024-08-18T23:43:24.301154+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:05:22.537018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313085+0000","last_change":"2024-08-18T23:41:32.137291+0000","last_active":"2024-08-18T23:43:24.313085+0000","last_peered":"2024-08-18T23:43:24.313085+0000","last_clean":"2024-08-18T23:43:24.313085+0000","last_became_active":"2024-08-18T23:41:32.136977+0000","last_became_peered":"2024-08-18T23:41:32.136977+0000","last_unstale":"2024-08-18T23:43:24.313085+0000","last_undegraded":"2024-08-18T23:43:24.313085+0000","last_fullsized":"2024-08-18T23:43:24.313085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T03:18:45.544760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312711+0000","last_change":"2024-08-18T23:41:32.144434+0000","last_active":"2024-08-18T23:43:24.312711+0000","last_peered":"2024-08-18T23:43:24.312711+0000","last_clean":"2024-08-18T23:43:24.312711+0000","last_became_active":"2024-08-18T23:41:32.144119+0000","last_became_peered":"2024-08-18T23:41:32.144119+0000","last_unstale":"2024-08-18T23:43:24.312711+0000","last_undegraded":"2024-08-18T23:43:24.312711+0000","last_fullsized":"2024-08-18T23:43:24.312711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:50:54.528630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.298360+0000","last_change":"2024-08-18T23:41:32.135034+0000","last_active":"2024-08-18T23:43:24.298360+0000","last_peered":"2024-08-18T23:43:24.298360+0000","last_clean":"2024-08-18T23:43:24.298360+0000","last_became_active":"2024-08-18T23:41:32.134700+0000","last_became_peered":"2024-08-18T23:41:32.134700+0000","last_unstale":"2024-08-18T23:43:24.298360+0000","last_undegraded":"2024-08-18T23:43:24.298360+0000","last_fullsized":"2024-08-18T23:43:24.298360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:06:31.263114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313052+0000","last_change":"2024-08-18T23:41:32.141191+0000","last_active":"2024-08-18T23:43:24.313052+0000","last_peered":"2024-08-18T23:43:24.313052+0000","last_clean":"2024-08-18T23:43:24.313052+0000","last_became_active":"2024-08-18T23:41:32.140901+0000","last_became_peered":"2024-08-18T23:41:32.140901+0000","last_unstale":"2024-08-18T23:43:24.313052+0000","last_undegraded":"2024-08-18T23:43:24.313052+0000","last_fullsized":"2024-08-18T23:43:24.313052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:35:24.849985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311944+0000","last_change":"2024-08-18T23:41:32.142543+0000","last_active":"2024-08-18T23:43:24.311944+0000","last_peered":"2024-08-18T23:43:24.311944+0000","last_clean":"2024-08-18T23:43:24.311944+0000","last_became_active":"2024-08-18T23:41:32.142084+0000","last_became_peered":"2024-08-18T23:41:32.142084+0000","last_unstale":"2024-08-18T23:43:24.311944+0000","last_undegraded":"2024-08-18T23:43:24.311944+0000","last_fullsized":"2024-08-18T23:43:24.311944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T00:54:18.200016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311562+0000","last_change":"2024-08-18T23:41:32.148968+0000","last_active":"2024-08-18T23:43:24.311562+0000","last_peered":"2024-08-18T23:43:24.311562+0000","last_clean":"2024-08-18T23:43:24.311562+0000","last_became_active":"2024-08-18T23:41:32.148625+0000","last_became_peered":"2024-08-18T23:41:32.148625+0000","last_unstale":"2024-08-18T23:43:24.311562+0000","last_undegraded":"2024-08-18T23:43:24.311562+0000","last_fullsized":"2024-08-18T23:43:24.311562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:11:40.172139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311624+0000","last_change":"2024-08-18T23:41:32.133478+0000","last_active":"2024-08-18T23:43:24.311624+0000","last_peered":"2024-08-18T23:43:24.311624+0000","last_clean":"2024-08-18T23:43:24.311624+0000","last_became_active":"2024-08-18T23:41:32.133149+0000","last_became_peered":"2024-08-18T23:41:32.133149+0000","last_unstale":"2024-08-18T23:43:24.311624+0000","last_undegraded":"2024-08-18T23:43:24.311624+0000","last_fullsized":"2024-08-18T23:43:24.311624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:57:53.713309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303573+0000","last_change":"2024-08-18T23:43:24.303232+0000","last_active":"2024-08-18T23:43:24.303573+0000","last_peered":"2024-08-18T23:43:24.303573+0000","last_clean":"2024-08-18T23:43:24.303573+0000","last_became_active":"2024-08-18T23:43:24.302849+0000","last_became_peered":"2024-08-18T23:43:24.302849+0000","last_unstale":"2024-08-18T23:43:24.303573+0000","last_undegraded":"2024-08-18T23:43:24.303573+0000","last_fullsized":"2024-08-18T23:43:24.303573+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:02:02.479891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.313096+0000","last_change":"2024-08-18T23:41:32.136303+0000","last_active":"2024-08-18T23:43:24.313096+0000","last_peered":"2024-08-18T23:43:24.313096+0000","last_clean":"2024-08-18T23:43:24.313096+0000","last_became_active":"2024-08-18T23:41:32.134628+0000","last_became_peered":"2024-08-18T23:41:32.134628+0000","last_unstale":"2024-08-18T23:43:24.313096+0000","last_undegraded":"2024-08-18T23:43:24.313096+0000","last_fullsized":"2024-08-18T23:43:24.313096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T11:28:51.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304992+0000","last_change":"2024-08-18T23:41:32.137512+0000","last_active":"2024-08-18T23:43:24.304992+0000","last_peered":"2024-08-18T23:43:24.304992+0000","last_clean":"2024-08-18T23:43:24.304992+0000","last_became_active":"2024-08-18T23:41:32.137150+0000","last_became_peered":"2024-08-18T23:41:32.137150+0000","last_unstale":"2024-08-18T23:43:24.304992+0000","last_undegraded":"2024-08-18T23:43:24.304992+0000","last_fullsized":"2024-08-18T23:43:24.304992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:44:08.020780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.301258+0000","last_change":"2024-08-18T23:41:32.139229+0000","last_active":"2024-08-18T23:43:24.301258+0000","last_peered":"2024-08-18T23:43:24.301258+0000","last_clean":"2024-08-18T23:43:24.301258+0000","last_became_active":"2024-08-18T23:41:32.138864+0000","last_became_peered":"2024-08-18T23:41:32.138864+0000","last_unstale":"2024-08-18T23:43:24.301258+0000","last_undegraded":"2024-08-18T23:43:24.301258+0000","last_fullsized":"2024-08-18T23:43:24.301258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T10:54:21.701750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312829+0000","last_change":"2024-08-18T23:41:32.148727+0000","last_active":"2024-08-18T23:43:24.312829+0000","last_peered":"2024-08-18T23:43:24.312829+0000","last_clean":"2024-08-18T23:43:24.312829+0000","last_became_active":"2024-08-18T23:41:32.146828+0000","last_became_peered":"2024-08-18T23:41:32.146828+0000","last_unstale":"2024-08-18T23:43:24.312829+0000","last_undegraded":"2024-08-18T23:43:24.312829+0000","last_fullsized":"2024-08-18T23:43:24.312829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:51:49.175150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312068+0000","last_change":"2024-08-18T23:41:32.144796+0000","last_active":"2024-08-18T23:43:24.312068+0000","last_peered":"2024-08-18T23:43:24.312068+0000","last_clean":"2024-08-18T23:43:24.312068+0000","last_became_active":"2024-08-18T23:41:32.141931+0000","last_became_peered":"2024-08-18T23:41:32.141931+0000","last_unstale":"2024-08-18T23:43:24.312068+0000","last_undegraded":"2024-08-18T23:43:24.312068+0000","last_fullsized":"2024-08-18T23:43:24.312068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T01:48:08.428845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.304569+0000","last_change":"2024-08-18T23:41:32.135265+0000","last_active":"2024-08-18T23:43:24.304569+0000","last_peered":"2024-08-18T23:43:24.304569+0000","last_clean":"2024-08-18T23:43:24.304569+0000","last_became_active":"2024-08-18T23:41:32.134952+0000","last_became_peered":"2024-08-18T23:41:32.134952+0000","last_unstale":"2024-08-18T23:43:24.304569+0000","last_undegraded":"2024-08-18T23:43:24.304569+0000","last_fullsized":"2024-08-18T23:43:24.304569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T04:35:14.099536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.300791+0000","last_change":"2024-08-18T23:41:32.135941+0000","last_active":"2024-08-18T23:43:24.300791+0000","last_peered":"2024-08-18T23:43:24.300791+0000","last_clean":"2024-08-18T23:43:24.300791+0000","last_became_active":"2024-08-18T23:41:32.135623+0000","last_became_peered":"2024-08-18T23:41:32.135623+0000","last_unstale":"2024-08-18T23:43:24.300791+0000","last_undegraded":"2024-08-18T23:43:24.300791+0000","last_fullsized":"2024-08-18T23:43:24.300791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T09:43:36.504738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.303603+0000","last_change":"2024-08-18T23:41:32.141643+0000","last_active":"2024-08-18T23:43:24.303603+0000","last_peered":"2024-08-18T23:43:24.303603+0000","last_clean":"2024-08-18T23:43:24.303603+0000","last_became_active":"2024-08-18T23:41:32.141301+0000","last_became_peered":"2024-08-18T23:41:32.141301+0000","last_unstale":"2024-08-18T23:43:24.303603+0000","last_undegraded":"2024-08-18T23:43:24.303603+0000","last_fullsized":"2024-08-18T23:43:24.303603+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T07:23:53.100740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312988+0000","last_change":"2024-08-18T23:41:32.146730+0000","last_active":"2024-08-18T23:43:24.312988+0000","last_peered":"2024-08-18T23:43:24.312988+0000","last_clean":"2024-08-18T23:43:24.312988+0000","last_became_active":"2024-08-18T23:41:32.146252+0000","last_became_peered":"2024-08-18T23:41:32.146252+0000","last_unstale":"2024-08-18T23:43:24.312988+0000","last_undegraded":"2024-08-18T23:43:24.312988+0000","last_fullsized":"2024-08-18T23:43:24.312988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:44:41.481761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311744+0000","last_change":"2024-08-18T23:41:32.149569+0000","last_active":"2024-08-18T23:43:24.311744+0000","last_peered":"2024-08-18T23:43:24.311744+0000","last_clean":"2024-08-18T23:43:24.311744+0000","last_became_active":"2024-08-18T23:41:32.149277+0000","last_became_peered":"2024-08-18T23:41:32.149277+0000","last_unstale":"2024-08-18T23:43:24.311744+0000","last_undegraded":"2024-08-18T23:43:24.311744+0000","last_fullsized":"2024-08-18T23:43:24.311744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:34:00.547909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.311950+0000","last_change":"2024-08-18T23:41:32.137889+0000","last_active":"2024-08-18T23:43:24.311950+0000","last_peered":"2024-08-18T23:43:24.311950+0000","last_clean":"2024-08-18T23:43:24.311950+0000","last_became_active":"2024-08-18T23:41:32.137578+0000","last_became_peered":"2024-08-18T23:41:32.137578+0000","last_unstale":"2024-08-18T23:43:24.311950+0000","last_undegraded":"2024-08-18T23:43:24.311950+0000","last_fullsized":"2024-08-18T23:43:24.311950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T05:18:09.987838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-18T23:43:24.312999+0000","last_change":"2024-08-18T23:41:32.142084+0000","last_active":"2024-08-18T23:43:24.312999+0000","last_peered":"2024-08-18T23:43:24.312999+0000","last_clean":"2024-08-18T23:43:24.312999+0000","last_became_active":"2024-08-18T23:41:32.141670+0000","last_became_peered":"2024-08-18T23:41:32.141670+0000","last_unstale":"2024-08-18T23:43:24.312999+0000","last_undegraded":"2024-08-18T23:43:24.312999+0000","last_fullsized":"2024-08-18T23:43:24.312999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-18T23:41:31.114649+0000","last_clean_scrub_stamp":"2024-08-18T23:41:31.114649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-20T06:30:03.990351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77285,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":766,"num_read_kb":467,"num_write":194,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":229579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":165,"ondisk_log_size":165,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514541,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":523788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776178,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1753088,"data_stored":1183810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939702,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":528120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168635,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":528040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397566,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1662976,"data_stored":1121222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561092,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1048576,"data_stored":519200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790024,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1111875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953549,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1064960,"data_stored":525271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":519394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":591576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1126400,"data_stored":589887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":520588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":74332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-18T23:44:12.777 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:44:13.029 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:13.441 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:13 smithi078 ceph-mon[18440]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:44:13.441 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:13 smithi078 ceph-mon[18440]: pgmap v347: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 6 op/s 2024-08-18T23:44:13.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:13 smithi123 ceph-mon[21808]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:44:13.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:13 smithi123 ceph-mon[21808]: pgmap v347: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 6 op/s 2024-08-18T23:44:13.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.520+0000 7f7865477640 1 -- 172.21.15.78:0/4160198194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7860106790 msgr2=0x7f7860106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:13.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.520+0000 7f7865477640 1 --2- 172.21.15.78:0/4160198194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7860106790 0x7f7860106b70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7850009930 tx=0x7f785002a420 comp rx=0 tx=0).stop 2024-08-18T23:44:13.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.520+0000 7f7865477640 1 -- 172.21.15.78:0/4160198194 shutdown_connections 2024-08-18T23:44:13.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.520+0000 7f7865477640 1 --2- 172.21.15.78:0/4160198194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7860101130 0x7f786010d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:13.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.520+0000 7f7865477640 1 --2- 172.21.15.78:0/4160198194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7860100790 0x7f7860100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:13.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.520+0000 7f7865477640 1 --2- 172.21.15.78:0/4160198194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7860106790 0x7f7860106b70 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:13.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.520+0000 7f7865477640 1 -- 172.21.15.78:0/4160198194 >> 172.21.15.78:0/4160198194 conn(0x7f78600fc4b0 msgr2=0x7f78600fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:13.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.520+0000 7f7865477640 1 -- 172.21.15.78:0/4160198194 shutdown_connections 2024-08-18T23:44:13.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.520+0000 7f7865477640 1 -- 172.21.15.78:0/4160198194 wait complete. 2024-08-18T23:44:13.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f7865477640 1 Processor -- start 2024-08-18T23:44:13.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f7865477640 1 -- start start 2024-08-18T23:44:13.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f7865477640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7860100790 0x7f78600fcc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:13.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f7865477640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7860101130 0x7f78600fd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:13.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f7865477640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7860106790 0x7f78600fd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:13.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f7865477640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7860110c30 con 0x7f7860100790 2024-08-18T23:44:13.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f7865477640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7860110ab0 con 0x7f7860101130 2024-08-18T23:44:13.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f7865477640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7860110db0 con 0x7f7860106790 2024-08-18T23:44:13.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f785effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7860100790 0x7f78600fcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:13.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f785effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7860100790 0x7f78600fcc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35918/0 (socket says 172.21.15.78:35918) 2024-08-18T23:44:13.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f785effd640 1 -- 172.21.15.78:0/3233572178 learned_addr learned my addr 172.21.15.78:0/3233572178 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:13.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f785f7fe640 1 --2- 172.21.15.78:0/3233572178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7860106790 0x7f78600fd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:13.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f78567fc640 1 --2- 172.21.15.78:0/3233572178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7860101130 0x7f78600fd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:13.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f785effd640 1 -- 172.21.15.78:0/3233572178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7860106790 msgr2=0x7f78600fd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:13.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f785effd640 1 --2- 172.21.15.78:0/3233572178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7860106790 0x7f78600fd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:13.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f785effd640 1 -- 172.21.15.78:0/3233572178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7860101130 msgr2=0x7f78600fd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:13.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f785effd640 1 --2- 172.21.15.78:0/3233572178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7860101130 0x7f78600fd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:13.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f785effd640 1 -- 172.21.15.78:0/3233572178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78601a67e0 con 0x7f7860100790 2024-08-18T23:44:13.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f785f7fe640 1 --2- 172.21.15.78:0/3233572178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7860106790 0x7f78600fd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:13.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.521+0000 7f78567fc640 1 --2- 172.21.15.78:0/3233572178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7860101130 0x7f78600fd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:13.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.522+0000 7f785effd640 1 --2- 172.21.15.78:0/3233572178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7860100790 0x7f78600fcc50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7850009a50 tx=0x7f7850004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:13.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.522+0000 7f785cff9640 1 -- 172.21.15.78:0/3233572178 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785003f030 con 0x7f7860100790 2024-08-18T23:44:13.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.522+0000 7f785cff9640 1 -- 172.21.15.78:0/3233572178 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7850003ce0 con 0x7f7860100790 2024-08-18T23:44:13.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.522+0000 7f7865477640 1 -- 172.21.15.78:0/3233572178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78601a6a70 con 0x7f7860100790 2024-08-18T23:44:13.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.522+0000 7f785cff9640 1 -- 172.21.15.78:0/3233572178 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850035070 con 0x7f7860100790 2024-08-18T23:44:13.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.522+0000 7f7865477640 1 -- 172.21.15.78:0/3233572178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78601a6f50 con 0x7f7860100790 2024-08-18T23:44:13.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.523+0000 7f7865477640 1 -- 172.21.15.78:0/3233572178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7860100c70 con 0x7f7860100790 2024-08-18T23:44:13.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.524+0000 7f785cff9640 1 -- 172.21.15.78:0/3233572178 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f7850042030 con 0x7f7860100790 2024-08-18T23:44:13.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.525+0000 7f785cff9640 1 --2- 172.21.15.78:0/3233572178 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f784c0a2c10 0x7f784c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:13.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.525+0000 7f78567fc640 1 --2- 172.21.15.78:0/3233572178 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f784c0a2c10 0x7f784c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:13.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.525+0000 7f785cff9640 1 -- 172.21.15.78:0/3233572178 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f78500efbd0 con 0x7f7860100790 2024-08-18T23:44:13.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.526+0000 7f78567fc640 1 --2- 172.21.15.78:0/3233572178 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f784c0a2c10 0x7f784c0a50d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f78600fe6e0 tx=0x7f7848008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:13.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.529+0000 7f785cff9640 1 -- 172.21.15.78:0/3233572178 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f78500bc640 con 0x7f7860100790 2024-08-18T23:44:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:13 smithi161 ceph-mon[22027]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-18T23:44:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:13 smithi161 ceph-mon[22027]: pgmap v347: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 6 op/s 2024-08-18T23:44:13.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.730+0000 7f7865477640 1 -- 172.21.15.78:0/3233572178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f786002d5e0 con 0x7f7860100790 2024-08-18T23:44:13.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.731+0000 7f785cff9640 1 -- 172.21.15.78:0/3233572178 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7f78500bbf40 con 0x7f7860100790 2024-08-18T23:44:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:44:13.732 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"btime":"2024-08-18T23:43:57:245845+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:43:57.114120+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24428,24427]},"id":1}]} 2024-08-18T23:44:13.733 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 25 2024-08-18T23:44:13.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.733+0000 7f7865477640 1 -- 172.21.15.78:0/3233572178 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f784c0a2c10 msgr2=0x7f784c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:13.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.733+0000 7f7865477640 1 --2- 172.21.15.78:0/3233572178 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f784c0a2c10 0x7f784c0a50d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f78600fe6e0 tx=0x7f7848008040 comp rx=0 tx=0).stop 2024-08-18T23:44:13.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.733+0000 7f7865477640 1 -- 172.21.15.78:0/3233572178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7860100790 msgr2=0x7f78600fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:13.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.733+0000 7f7865477640 1 --2- 172.21.15.78:0/3233572178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7860100790 0x7f78600fcc50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7850009a50 tx=0x7f7850004270 comp rx=0 tx=0).stop 2024-08-18T23:44:13.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.733+0000 7f7865477640 1 -- 172.21.15.78:0/3233572178 shutdown_connections 2024-08-18T23:44:13.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.733+0000 7f7865477640 1 --2- 172.21.15.78:0/3233572178 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f784c0a2c10 0x7f784c0a50d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:13.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.733+0000 7f7865477640 1 --2- 172.21.15.78:0/3233572178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7860106790 0x7f78600fd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:13.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.733+0000 7f7865477640 1 --2- 172.21.15.78:0/3233572178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7860101130 0x7f78600fd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:13.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.733+0000 7f7865477640 1 --2- 172.21.15.78:0/3233572178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7860100790 0x7f78600fcc50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:13.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.733+0000 7f7865477640 1 -- 172.21.15.78:0/3233572178 >> 172.21.15.78:0/3233572178 conn(0x7f78600fc4b0 msgr2=0x7f786006a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:13.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.733+0000 7f7865477640 1 -- 172.21.15.78:0/3233572178 shutdown_connections 2024-08-18T23:44:13.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:13.733+0000 7f7865477640 1 -- 172.21.15.78:0/3233572178 wait complete. 2024-08-18T23:44:14.428 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:14 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3233572178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:14.455 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:44:14.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:14 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3233572178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:14 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3233572178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:14.703 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:15.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.179+0000 7f271e922640 1 -- 172.21.15.78:0/1061845731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2718102c20 msgr2=0x7f271810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:15.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.179+0000 7f271e922640 1 --2- 172.21.15.78:0/1061845731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2718102c20 0x7f271810d770 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2710009ca0 tx=0x7f271002a950 comp rx=0 tx=0).stop 2024-08-18T23:44:15.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.179+0000 7f271e922640 1 -- 172.21.15.78:0/1061845731 shutdown_connections 2024-08-18T23:44:15.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.179+0000 7f271e922640 1 --2- 172.21.15.78:0/1061845731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2718102c20 0x7f271810d770 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:15.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.179+0000 7f271e922640 1 --2- 172.21.15.78:0/1061845731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27181022a0 0x7f27181026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:15.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.179+0000 7f271e922640 1 --2- 172.21.15.78:0/1061845731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2718100b50 0x7f2718100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:15.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.179+0000 7f271e922640 1 -- 172.21.15.78:0/1061845731 >> 172.21.15.78:0/1061845731 conn(0x7f27180fc500 msgr2=0x7f27180fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:15.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.180+0000 7f271e922640 1 -- 172.21.15.78:0/1061845731 shutdown_connections 2024-08-18T23:44:15.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.180+0000 7f271e922640 1 -- 172.21.15.78:0/1061845731 wait complete. 2024-08-18T23:44:15.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.180+0000 7f271e922640 1 Processor -- start 2024-08-18T23:44:15.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.180+0000 7f271e922640 1 -- start start 2024-08-18T23:44:15.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.181+0000 7f271e922640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2718100b50 0x7f2718198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:15.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.181+0000 7f271e922640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27181022a0 0x7f2718199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:15.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.181+0000 7f271e922640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2718102c20 0x7f27181a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:15.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.181+0000 7f271e922640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2718110cd0 con 0x7f2718100b50 2024-08-18T23:44:15.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.181+0000 7f271e922640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2718110b50 con 0x7f27181022a0 2024-08-18T23:44:15.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.181+0000 7f271e922640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2718110e50 con 0x7f2718102c20 2024-08-18T23:44:15.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.181+0000 7f2717fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2718100b50 0x7f2718198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:15.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.181+0000 7f2717fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2718100b50 0x7f2718198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35926/0 (socket says 172.21.15.78:35926) 2024-08-18T23:44:15.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.181+0000 7f2717fff640 1 -- 172.21.15.78:0/2539324126 learned_addr learned my addr 172.21.15.78:0/2539324126 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:15.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.181+0000 7f270ffff640 1 --2- 172.21.15.78:0/2539324126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27181022a0 0x7f2718199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:15.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.181+0000 7f271cebf640 1 --2- 172.21.15.78:0/2539324126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2718102c20 0x7f27181a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:15.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.181+0000 7f2717fff640 1 -- 172.21.15.78:0/2539324126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2718102c20 msgr2=0x7f27181a4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:15.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.181+0000 7f2717fff640 1 --2- 172.21.15.78:0/2539324126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2718102c20 0x7f27181a4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:15.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.181+0000 7f2717fff640 1 -- 172.21.15.78:0/2539324126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27181022a0 msgr2=0x7f2718199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:15.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.181+0000 7f2717fff640 1 --2- 172.21.15.78:0/2539324126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27181022a0 0x7f2718199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:15.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.181+0000 7f2717fff640 1 -- 172.21.15.78:0/2539324126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27181a4590 con 0x7f2718100b50 2024-08-18T23:44:15.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.181+0000 7f2717fff640 1 --2- 172.21.15.78:0/2539324126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2718100b50 0x7f2718198fd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f270800c920 tx=0x7f270800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:15.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.182+0000 7f2715ffb640 1 -- 172.21.15.78:0/2539324126 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2708007c70 con 0x7f2718100b50 2024-08-18T23:44:15.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.182+0000 7f2715ffb640 1 -- 172.21.15.78:0/2539324126 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2708007e10 con 0x7f2718100b50 2024-08-18T23:44:15.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.182+0000 7f271e922640 1 -- 172.21.15.78:0/2539324126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27181a4760 con 0x7f2718100b50 2024-08-18T23:44:15.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.182+0000 7f2715ffb640 1 -- 172.21.15.78:0/2539324126 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2708018890 con 0x7f2718100b50 2024-08-18T23:44:15.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.182+0000 7f271e922640 1 -- 172.21.15.78:0/2539324126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27181a4ca0 con 0x7f2718100b50 2024-08-18T23:44:15.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.183+0000 7f271e922640 1 -- 172.21.15.78:0/2539324126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2718100f30 con 0x7f2718100b50 2024-08-18T23:44:15.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.184+0000 7f2715ffb640 1 -- 172.21.15.78:0/2539324126 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f2708018b30 con 0x7f2718100b50 2024-08-18T23:44:15.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.184+0000 7f2715ffb640 1 --2- 172.21.15.78:0/2539324126 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f27040a2c90 0x7f27040a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:15.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.185+0000 7f270ffff640 1 --2- 172.21.15.78:0/2539324126 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f27040a2c90 0x7f27040a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:15.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.185+0000 7f2715ffb640 1 -- 172.21.15.78:0/2539324126 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f27080cf730 con 0x7f2718100b50 2024-08-18T23:44:15.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.185+0000 7f270ffff640 1 --2- 172.21.15.78:0/2539324126 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f27040a2c90 0x7f27040a5150 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f271806a7f0 tx=0x7f2700008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:15.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.189+0000 7f2715ffb640 1 -- 172.21.15.78:0/2539324126 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2708014030 con 0x7f2718100b50 2024-08-18T23:44:15.368 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:15 smithi078 ceph-mon[18440]: pgmap v348: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-18T23:44:15.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.399+0000 7f271e922640 1 -- 172.21.15.78:0/2539324126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27181a4fb0 con 0x7f2718100b50 2024-08-18T23:44:15.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.399+0000 7f2715ffb640 1 -- 172.21.15.78:0/2539324126 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7f270800de20 con 0x7f2718100b50 2024-08-18T23:44:15.401 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:44:15.401 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"btime":"2024-08-18T23:43:57:245845+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:43:57.114120+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24428,24427]},"id":1}]} 2024-08-18T23:44:15.401 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 25 2024-08-18T23:44:15.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.402+0000 7f271e922640 1 -- 172.21.15.78:0/2539324126 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f27040a2c90 msgr2=0x7f27040a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:15.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.402+0000 7f271e922640 1 --2- 172.21.15.78:0/2539324126 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f27040a2c90 0x7f27040a5150 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f271806a7f0 tx=0x7f2700008040 comp rx=0 tx=0).stop 2024-08-18T23:44:15.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.402+0000 7f271e922640 1 -- 172.21.15.78:0/2539324126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2718100b50 msgr2=0x7f2718198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:15.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.402+0000 7f271e922640 1 --2- 172.21.15.78:0/2539324126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2718100b50 0x7f2718198fd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f270800c920 tx=0x7f270800cde0 comp rx=0 tx=0).stop 2024-08-18T23:44:15.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.402+0000 7f271e922640 1 -- 172.21.15.78:0/2539324126 shutdown_connections 2024-08-18T23:44:15.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.402+0000 7f271e922640 1 --2- 172.21.15.78:0/2539324126 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f27040a2c90 0x7f27040a5150 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:15.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.402+0000 7f271e922640 1 --2- 172.21.15.78:0/2539324126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2718102c20 0x7f27181a4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:15.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.402+0000 7f271e922640 1 --2- 172.21.15.78:0/2539324126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27181022a0 0x7f2718199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:15.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.402+0000 7f271e922640 1 --2- 172.21.15.78:0/2539324126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2718100b50 0x7f2718198fd0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:15.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.402+0000 7f271e922640 1 -- 172.21.15.78:0/2539324126 >> 172.21.15.78:0/2539324126 conn(0x7f27180fc500 msgr2=0x7f2718106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:15.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.402+0000 7f271e922640 1 -- 172.21.15.78:0/2539324126 shutdown_connections 2024-08-18T23:44:15.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:15.402+0000 7f271e922640 1 -- 172.21.15.78:0/2539324126 wait complete. 2024-08-18T23:44:15.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:15 smithi161 ceph-mon[22027]: pgmap v348: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-18T23:44:15.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:15 smithi123 ceph-mon[21808]: pgmap v348: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-18T23:44:16.065 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-18T23:44:16.329 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:16.579 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:16 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2539324126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:16.579 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:16 smithi078 ceph-mon[18440]: pgmap v349: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 7 op/s 2024-08-18T23:44:16.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:16 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2539324126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:16.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:16 smithi161 ceph-mon[22027]: pgmap v349: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 7 op/s 2024-08-18T23:44:16.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:16 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2539324126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:16.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:16 smithi123 ceph-mon[21808]: pgmap v349: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 7 op/s 2024-08-18T23:44:16.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.943+0000 7fab710e8640 1 -- 172.21.15.78:0/1346954249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab6c1022a0 msgr2=0x7fab6c1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:16.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.943+0000 7fab710e8640 1 --2- 172.21.15.78:0/1346954249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab6c1022a0 0x7fab6c1026e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fab54009900 tx=0x7fab5402a480 comp rx=0 tx=0).stop 2024-08-18T23:44:16.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.944+0000 7fab710e8640 1 -- 172.21.15.78:0/1346954249 shutdown_connections 2024-08-18T23:44:16.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.944+0000 7fab710e8640 1 --2- 172.21.15.78:0/1346954249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab6c102c20 0x7fab6c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:16.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.944+0000 7fab710e8640 1 --2- 172.21.15.78:0/1346954249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab6c1022a0 0x7fab6c1026e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:16.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.944+0000 7fab710e8640 1 --2- 172.21.15.78:0/1346954249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab6c100b50 0x7fab6c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:16.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.944+0000 7fab710e8640 1 -- 172.21.15.78:0/1346954249 >> 172.21.15.78:0/1346954249 conn(0x7fab6c0fc500 msgr2=0x7fab6c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:16.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.944+0000 7fab710e8640 1 -- 172.21.15.78:0/1346954249 shutdown_connections 2024-08-18T23:44:16.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.944+0000 7fab710e8640 1 -- 172.21.15.78:0/1346954249 wait complete. 2024-08-18T23:44:16.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.944+0000 7fab710e8640 1 Processor -- start 2024-08-18T23:44:16.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab710e8640 1 -- start start 2024-08-18T23:44:16.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab710e8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab6c100b50 0x7fab6c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:16.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab710e8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab6c1022a0 0x7fab6c19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:16.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab710e8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab6c102c20 0x7fab6c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:16.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab710e8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c110cd0 con 0x7fab6c1022a0 2024-08-18T23:44:16.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab710e8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c110b50 con 0x7fab6c102c20 2024-08-18T23:44:16.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab710e8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c110e50 con 0x7fab6c100b50 2024-08-18T23:44:16.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab6a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab6c1022a0 0x7fab6c19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:16.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab6a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab6c1022a0 0x7fab6c19a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59588/0 (socket says 172.21.15.78:59588) 2024-08-18T23:44:16.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab6a59c640 1 -- 172.21.15.78:0/1688788382 learned_addr learned my addr 172.21.15.78:0/1688788382 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:16.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab6ad9d640 1 --2- 172.21.15.78:0/1688788382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab6c100b50 0x7fab6c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:16.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab6b59e640 1 --2- 172.21.15.78:0/1688788382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab6c102c20 0x7fab6c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:16.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab6a59c640 1 -- 172.21.15.78:0/1688788382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab6c100b50 msgr2=0x7fab6c19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:16.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab6a59c640 1 --2- 172.21.15.78:0/1688788382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab6c100b50 0x7fab6c19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:16.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab6a59c640 1 -- 172.21.15.78:0/1688788382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab6c102c20 msgr2=0x7fab6c19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:16.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab6a59c640 1 --2- 172.21.15.78:0/1688788382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab6c102c20 0x7fab6c19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:16.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab6a59c640 1 -- 172.21.15.78:0/1688788382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6c19f3c0 con 0x7fab6c1022a0 2024-08-18T23:44:16.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab6b59e640 1 --2- 172.21.15.78:0/1688788382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab6c102c20 0x7fab6c19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:44:16.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab6a59c640 1 --2- 172.21.15.78:0/1688788382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab6c1022a0 0x7fab6c19a950 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fab54006010 tx=0x7fab54003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:16.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.945+0000 7fab63fff640 1 -- 172.21.15.78:0/1688788382 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5403f070 con 0x7fab6c1022a0 2024-08-18T23:44:16.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.946+0000 7fab63fff640 1 -- 172.21.15.78:0/1688788382 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab54031e30 con 0x7fab6c1022a0 2024-08-18T23:44:16.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.946+0000 7fab710e8640 1 -- 172.21.15.78:0/1688788382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab6c19f650 con 0x7fab6c1022a0 2024-08-18T23:44:16.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.946+0000 7fab63fff640 1 -- 172.21.15.78:0/1688788382 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54035030 con 0x7fab6c1022a0 2024-08-18T23:44:16.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.946+0000 7fab710e8640 1 -- 172.21.15.78:0/1688788382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab6c1a6f90 con 0x7fab6c1022a0 2024-08-18T23:44:16.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.947+0000 7fab710e8640 1 -- 172.21.15.78:0/1688788382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab6c100f30 con 0x7fab6c1022a0 2024-08-18T23:44:16.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.948+0000 7fab63fff640 1 -- 172.21.15.78:0/1688788382 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fab54042070 con 0x7fab6c1022a0 2024-08-18T23:44:16.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.948+0000 7fab63fff640 1 --2- 172.21.15.78:0/1688788382 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fab580a2c90 0x7fab580a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:16.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.949+0000 7fab6ad9d640 1 --2- 172.21.15.78:0/1688788382 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fab580a2c90 0x7fab580a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:16.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.949+0000 7fab63fff640 1 -- 172.21.15.78:0/1688788382 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fab540efee0 con 0x7fab6c1022a0 2024-08-18T23:44:16.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.949+0000 7fab6ad9d640 1 --2- 172.21.15.78:0/1688788382 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fab580a2c90 0x7fab580a5150 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00ba40 tx=0x7fab5c005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:16.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:16.953+0000 7fab63fff640 1 -- 172.21.15.78:0/1688788382 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fab540bc950 con 0x7fab6c1022a0 2024-08-18T23:44:17.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:17.118+0000 7fab710e8640 1 -- 172.21.15.78:0/1688788382 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab6c1a7160 con 0x7fab6c1022a0 2024-08-18T23:44:17.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:17.120+0000 7fab63fff640 1 -- 172.21.15.78:0/1688788382 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24747 (secure 0 0 0) 0x7fab540bc250 con 0x7fab6c1022a0 2024-08-18T23:44:17.121 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:44:17.121 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":92,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","created":"2024-08-18T23:34:33.867246+0000","modified":"2024-08-18T23:43:24.291494+0000","last_up_change":"2024-08-18T23:40:42.200381+0000","last_in_change":"2024-08-18T23:40:22.425813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T23:37:22.338211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T23:41:28.920802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T23:41:31.026094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T23:41:37.403238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4de1c78-6ebc-49a9-b9c5-85d43535fab7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6803","nonce":3639645311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6805","nonce":3639645311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6809","nonce":3639645311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6807","nonce":3639645311}]},"public_addr":"172.21.15.78:6803/3639645311","cluster_addr":"172.21.15.78:6805/3639645311","heartbeat_back_addr":"172.21.15.78:6809/3639645311","heartbeat_front_addr":"172.21.15.78:6807/3639645311","state":["exists","up"]},{"osd":1,"uuid":"4f8e0c3d-472b-4230-ad23-2b0a34861251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6801","nonce":4199446368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6803","nonce":4199446368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6807","nonce":4199446368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6805","nonce":4199446368}]},"public_addr":"172.21.15.123:6801/4199446368","cluster_addr":"172.21.15.123:6803/4199446368","heartbeat_back_addr":"172.21.15.123:6807/4199446368","heartbeat_front_addr":"172.21.15.123:6805/4199446368","state":["exists","up"]},{"osd":2,"uuid":"27ef3314-bb6f-413d-be05-e10a105dec9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6801","nonce":2582810057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6803","nonce":2582810057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6807","nonce":2582810057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6805","nonce":2582810057}]},"public_addr":"172.21.15.161:6801/2582810057","cluster_addr":"172.21.15.161:6803/2582810057","heartbeat_back_addr":"172.21.15.161:6807/2582810057","heartbeat_front_addr":"172.21.15.161:6805/2582810057","state":["exists","up"]},{"osd":3,"uuid":"e9e52662-5ef9-4d70-bdd7-2b92165822ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6811","nonce":2697347521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6813","nonce":2697347521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6817","nonce":2697347521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6815","nonce":2697347521}]},"public_addr":"172.21.15.78:6811/2697347521","cluster_addr":"172.21.15.78:6813/2697347521","heartbeat_back_addr":"172.21.15.78:6817/2697347521","heartbeat_front_addr":"172.21.15.78:6815/2697347521","state":["exists","up"]},{"osd":4,"uuid":"5ed26775-79e3-4b24-9557-69fcf5e8e824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6809","nonce":1281511569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6811","nonce":1281511569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6815","nonce":1281511569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6813","nonce":1281511569}]},"public_addr":"172.21.15.123:6809/1281511569","cluster_addr":"172.21.15.123:6811/1281511569","heartbeat_back_addr":"172.21.15.123:6815/1281511569","heartbeat_front_addr":"172.21.15.123:6813/1281511569","state":["exists","up"]},{"osd":5,"uuid":"a9793b1d-5613-48a4-938e-adb718f4fa4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6809","nonce":2751796117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6811","nonce":2751796117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6815","nonce":2751796117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6813","nonce":2751796117}]},"public_addr":"172.21.15.161:6809/2751796117","cluster_addr":"172.21.15.161:6811/2751796117","heartbeat_back_addr":"172.21.15.161:6815/2751796117","heartbeat_front_addr":"172.21.15.161:6813/2751796117","state":["exists","up"]},{"osd":6,"uuid":"5dbc5e9c-a14f-4b71-910a-1e6d4827bb32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6819","nonce":95383223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6821","nonce":95383223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6825","nonce":95383223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6823","nonce":95383223}]},"public_addr":"172.21.15.78:6819/95383223","cluster_addr":"172.21.15.78:6821/95383223","heartbeat_back_addr":"172.21.15.78:6825/95383223","heartbeat_front_addr":"172.21.15.78:6823/95383223","state":["exists","up"]},{"osd":7,"uuid":"26fb7b69-131f-42ee-b745-6fe8b4f65391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6817","nonce":3440305156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6819","nonce":3440305156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6823","nonce":3440305156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6821","nonce":3440305156}]},"public_addr":"172.21.15.123:6817/3440305156","cluster_addr":"172.21.15.123:6819/3440305156","heartbeat_back_addr":"172.21.15.123:6823/3440305156","heartbeat_front_addr":"172.21.15.123:6821/3440305156","state":["exists","up"]},{"osd":8,"uuid":"e267bc50-0c29-4d7b-b368-b1621e29b2df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6817","nonce":1485008778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6819","nonce":1485008778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6823","nonce":1485008778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6821","nonce":1485008778}]},"public_addr":"172.21.15.161:6817/1485008778","cluster_addr":"172.21.15.161:6819/1485008778","heartbeat_back_addr":"172.21.15.161:6823/1485008778","heartbeat_front_addr":"172.21.15.161:6821/1485008778","state":["exists","up"]},{"osd":9,"uuid":"b7654818-b767-4fe6-8211-346297895f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6827","nonce":2884475661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6829","nonce":2884475661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6833","nonce":2884475661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6831","nonce":2884475661}]},"public_addr":"172.21.15.78:6827/2884475661","cluster_addr":"172.21.15.78:6829/2884475661","heartbeat_back_addr":"172.21.15.78:6833/2884475661","heartbeat_front_addr":"172.21.15.78:6831/2884475661","state":["exists","up"]},{"osd":10,"uuid":"59aa32ff-510e-4b52-8170-08a34a0b6b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6825","nonce":122226286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6827","nonce":122226286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6831","nonce":122226286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6829","nonce":122226286}]},"public_addr":"172.21.15.123:6825/122226286","cluster_addr":"172.21.15.123:6827/122226286","heartbeat_back_addr":"172.21.15.123:6831/122226286","heartbeat_front_addr":"172.21.15.123:6829/122226286","state":["exists","up"]},{"osd":11,"uuid":"67bacab4-d078-4c5c-b4ff-9f6a8a00f507","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6825","nonce":2037560985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6827","nonce":2037560985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6831","nonce":2037560985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6829","nonce":2037560985}]},"public_addr":"172.21.15.161:6825/2037560985","cluster_addr":"172.21.15.161:6827/2037560985","heartbeat_back_addr":"172.21.15.161:6831/2037560985","heartbeat_front_addr":"172.21.15.161:6829/2037560985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:36.826044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:57.688421+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:18.297424+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:40.355215+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:02.167902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:23.968759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:46.254083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:08.522038+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:31.028119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:54.119807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:16.847034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:39.607886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6800/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3040499240":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6801/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:6800/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/757409165":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/2562072965":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/636184437":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/1570708294":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3137806628":"2024-08-19T23:34:59.021165+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T23:44:17.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:17.122+0000 7fab710e8640 1 -- 172.21.15.78:0/1688788382 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fab580a2c90 msgr2=0x7fab580a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:17.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:17.122+0000 7fab710e8640 1 --2- 172.21.15.78:0/1688788382 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fab580a2c90 0x7fab580a5150 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00ba40 tx=0x7fab5c005cc0 comp rx=0 tx=0).stop 2024-08-18T23:44:17.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:17.122+0000 7fab710e8640 1 -- 172.21.15.78:0/1688788382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab6c1022a0 msgr2=0x7fab6c19a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:17.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:17.122+0000 7fab710e8640 1 --2- 172.21.15.78:0/1688788382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab6c1022a0 0x7fab6c19a950 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fab54006010 tx=0x7fab54003c80 comp rx=0 tx=0).stop 2024-08-18T23:44:17.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:17.122+0000 7fab710e8640 1 -- 172.21.15.78:0/1688788382 shutdown_connections 2024-08-18T23:44:17.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:17.122+0000 7fab710e8640 1 --2- 172.21.15.78:0/1688788382 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fab580a2c90 0x7fab580a5150 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:17.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:17.122+0000 7fab710e8640 1 --2- 172.21.15.78:0/1688788382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab6c102c20 0x7fab6c19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:17.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:17.122+0000 7fab710e8640 1 --2- 172.21.15.78:0/1688788382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab6c1022a0 0x7fab6c19a950 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:17.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:17.122+0000 7fab710e8640 1 --2- 172.21.15.78:0/1688788382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab6c100b50 0x7fab6c19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:17.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:17.122+0000 7fab710e8640 1 -- 172.21.15.78:0/1688788382 >> 172.21.15.78:0/1688788382 conn(0x7fab6c0fc500 msgr2=0x7fab6c069ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:17.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:17.122+0000 7fab710e8640 1 -- 172.21.15.78:0/1688788382 shutdown_connections 2024-08-18T23:44:17.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:17.122+0000 7fab710e8640 1 -- 172.21.15.78:0/1688788382 wait complete. 2024-08-18T23:44:17.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:17 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1688788382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:44:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:17 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1688788382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:44:17.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:17 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1688788382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:44:17.856 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-08-18T23:44:18.105 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:18.531 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:18 smithi078 ceph-mon[18440]: pgmap v350: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-18T23:44:18.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:18 smithi161 ceph-mon[22027]: pgmap v350: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-18T23:44:18.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:18 smithi123 ceph-mon[21808]: pgmap v350: 129 pgs: 129 active+clean; 652 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-18T23:44:19.347 INFO:teuthology.orchestra.run.smithi078.stdout:{"run_dir":"/var/run/ceph"} 2024-08-18T23:44:19.348 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-08-18T23:44:19.348 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1664 2024-08-18T23:44:19.348 DEBUG:tasks.quiescer:Yielding 2024-08-18T23:44:19.348 INFO:teuthology.run_tasks:Running task workunit... 2024-08-18T23:44:19.367 INFO:tasks.workunit:Pulling workunits from ref 78215f5a1cf5eaf321d506a66539942e12851f34 2024-08-18T23:44:19.367 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-08-18T23:44:19.367 INFO:tasks.workunit:timeout=3h 2024-08-18T23:44:19.367 INFO:tasks.workunit:cleanup=True 2024-08-18T23:44:19.368 DEBUG:teuthology.orchestra.run.smithi078:> stat -- /home/ubuntu/cephtest/mnt.0 2024-08-18T23:44:19.368 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:44:19.384 INFO:teuthology.orchestra.run.smithi078.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-08-18T23:44:19.385 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-08-18T23:44:19.385 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2024-08-18T23:44:19.385 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-18T23:44:19.385 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-08-18T23:44:19.385 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-08-18 23:42:17.012314128 +0000 2024-08-18T23:44:19.385 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-08-18 23:42:17.012314128 +0000 2024-08-18T23:44:19.385 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-08-18 23:43:56.447040323 +0000 2024-08-18T23:44:19.385 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-08-18 23:42:17.012314128 +0000 2024-08-18T23:44:19.385 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-08-18T23:44:19.386 DEBUG:teuthology.orchestra.run.smithi078:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-08-18T23:44:19.458 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 78215f5a1cf5eaf321d506a66539942e12851f34 2024-08-18T23:44:19.514 INFO:tasks.workunit.client.0.smithi078.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-08-18T23:44:19.629 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:20.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.137+0000 7f3b3b97a640 1 -- 172.21.15.78:0/3866984848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b341018c0 msgr2=0x7f3b34101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:20.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.137+0000 7f3b3b97a640 1 --2- 172.21.15.78:0/3866984848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b341018c0 0x7f3b34101d20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009930 tx=0x7f3b2402a420 comp rx=0 tx=0).stop 2024-08-18T23:44:20.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.137+0000 7f3b3b97a640 1 -- 172.21.15.78:0/3866984848 shutdown_connections 2024-08-18T23:44:20.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.137+0000 7f3b3b97a640 1 --2- 172.21.15.78:0/3866984848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b34102260 0x7f3b3410e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:20.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.137+0000 7f3b3b97a640 1 --2- 172.21.15.78:0/3866984848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b341018c0 0x7f3b34101d20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:20.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.137+0000 7f3b3b97a640 1 --2- 172.21.15.78:0/3866984848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b341078c0 0x7f3b34107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:20.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.137+0000 7f3b3b97a640 1 -- 172.21.15.78:0/3866984848 >> 172.21.15.78:0/3866984848 conn(0x7f3b340fd5e0 msgr2=0x7f3b340ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:20.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.137+0000 7f3b3b97a640 1 -- 172.21.15.78:0/3866984848 shutdown_connections 2024-08-18T23:44:20.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.137+0000 7f3b3b97a640 1 -- 172.21.15.78:0/3866984848 wait complete. 2024-08-18T23:44:20.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.138+0000 7f3b3b97a640 1 Processor -- start 2024-08-18T23:44:20.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.138+0000 7f3b3b97a640 1 -- start start 2024-08-18T23:44:20.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.139+0000 7f3b3b97a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b341018c0 0x7f3b341981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:20.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.139+0000 7f3b3b97a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b34102260 0x7f3b341986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:20.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.139+0000 7f3b3b97a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b341078c0 0x7f3b3419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:20.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.139+0000 7f3b3b97a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b34111cc0 con 0x7f3b341078c0 2024-08-18T23:44:20.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.139+0000 7f3b3b97a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b34111b40 con 0x7f3b341018c0 2024-08-18T23:44:20.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.139+0000 7f3b3b97a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3b34111e40 con 0x7f3b34102260 2024-08-18T23:44:20.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.139+0000 7f3b39f17640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b341078c0 0x7f3b3419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:20.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.139+0000 7f3b39f17640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b341078c0 0x7f3b3419ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59608/0 (socket says 172.21.15.78:59608) 2024-08-18T23:44:20.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.139+0000 7f3b39f17640 1 -- 172.21.15.78:0/3617850770 learned_addr learned my addr 172.21.15.78:0/3617850770 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:20.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.139+0000 7f3b39716640 1 --2- 172.21.15.78:0/3617850770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b341018c0 0x7f3b341981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:20.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.139+0000 7f3b38f15640 1 --2- 172.21.15.78:0/3617850770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b34102260 0x7f3b341986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:20.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.139+0000 7f3b39f17640 1 -- 172.21.15.78:0/3617850770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b34102260 msgr2=0x7f3b341986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:20.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.139+0000 7f3b39f17640 1 --2- 172.21.15.78:0/3617850770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b34102260 0x7f3b341986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:20.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.139+0000 7f3b39f17640 1 -- 172.21.15.78:0/3617850770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b341018c0 msgr2=0x7f3b341981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:20.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.139+0000 7f3b39f17640 1 --2- 172.21.15.78:0/3617850770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b341018c0 0x7f3b341981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:20.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.139+0000 7f3b39f17640 1 -- 172.21.15.78:0/3617850770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3419d150 con 0x7f3b341078c0 2024-08-18T23:44:20.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.139+0000 7f3b39716640 1 --2- 172.21.15.78:0/3617850770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b341018c0 0x7f3b341981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:44:20.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.139+0000 7f3b39f17640 1 --2- 172.21.15.78:0/3617850770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b341078c0 0x7f3b3419ca70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00eaa0 tx=0x7f3b2c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:20.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.140+0000 7f3b2a7fc640 1 -- 172.21.15.78:0/3617850770 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c00cdb0 con 0x7f3b341078c0 2024-08-18T23:44:20.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.140+0000 7f3b2a7fc640 1 -- 172.21.15.78:0/3617850770 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b2c005020 con 0x7f3b341078c0 2024-08-18T23:44:20.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.140+0000 7f3b3b97a640 1 -- 172.21.15.78:0/3617850770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3419d440 con 0x7f3b341078c0 2024-08-18T23:44:20.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.140+0000 7f3b2a7fc640 1 -- 172.21.15.78:0/3617850770 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c0116b0 con 0x7f3b341078c0 2024-08-18T23:44:20.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.140+0000 7f3b3b97a640 1 -- 172.21.15.78:0/3617850770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b341a4d20 con 0x7f3b341078c0 2024-08-18T23:44:20.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.141+0000 7f3b3b97a640 1 -- 172.21.15.78:0/3617850770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b34101d20 con 0x7f3b341078c0 2024-08-18T23:44:20.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.142+0000 7f3b2a7fc640 1 -- 172.21.15.78:0/3617850770 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f3b2c005560 con 0x7f3b341078c0 2024-08-18T23:44:20.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.143+0000 7f3b2a7fc640 1 --2- 172.21.15.78:0/3617850770 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3b200a2c90 0x7f3b200a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:20.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.143+0000 7f3b2a7fc640 1 -- 172.21.15.78:0/3617850770 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f3b2c0cf680 con 0x7f3b341078c0 2024-08-18T23:44:20.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.143+0000 7f3b39716640 1 --2- 172.21.15.78:0/3617850770 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3b200a2c90 0x7f3b200a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:20.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.144+0000 7f3b39716640 1 --2- 172.21.15.78:0/3617850770 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3b200a2c90 0x7f3b200a5150 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009f40 tx=0x7f3b30005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:20.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.148+0000 7f3b2a7fc640 1 -- 172.21.15.78:0/3617850770 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3b2c016030 con 0x7f3b341078c0 2024-08-18T23:44:20.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.352+0000 7f3b3b97a640 1 -- 172.21.15.78:0/3617850770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b3402d5e0 con 0x7f3b341078c0 2024-08-18T23:44:20.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.352+0000 7f3b2a7fc640 1 -- 172.21.15.78:0/3617850770 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7f3b2c09baa0 con 0x7f3b341078c0 2024-08-18T23:44:20.354 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:44:20.354 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"btime":"2024-08-18T23:43:57:245845+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:43:57.114120+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24428,24427]},"id":1}]} 2024-08-18T23:44:20.354 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 25 2024-08-18T23:44:20.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.355+0000 7f3b3b97a640 1 -- 172.21.15.78:0/3617850770 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3b200a2c90 msgr2=0x7f3b200a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:20.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.355+0000 7f3b3b97a640 1 --2- 172.21.15.78:0/3617850770 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3b200a2c90 0x7f3b200a5150 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009f40 tx=0x7f3b30005f90 comp rx=0 tx=0).stop 2024-08-18T23:44:20.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.355+0000 7f3b3b97a640 1 -- 172.21.15.78:0/3617850770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b341078c0 msgr2=0x7f3b3419ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:20.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.355+0000 7f3b3b97a640 1 --2- 172.21.15.78:0/3617850770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b341078c0 0x7f3b3419ca70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00eaa0 tx=0x7f3b2c00ef60 comp rx=0 tx=0).stop 2024-08-18T23:44:20.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.355+0000 7f3b3b97a640 1 -- 172.21.15.78:0/3617850770 shutdown_connections 2024-08-18T23:44:20.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.355+0000 7f3b3b97a640 1 --2- 172.21.15.78:0/3617850770 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3b200a2c90 0x7f3b200a5150 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:20.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.355+0000 7f3b3b97a640 1 --2- 172.21.15.78:0/3617850770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b341078c0 0x7f3b3419ca70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:20.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.355+0000 7f3b3b97a640 1 --2- 172.21.15.78:0/3617850770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b34102260 0x7f3b341986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:20.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.355+0000 7f3b3b97a640 1 --2- 172.21.15.78:0/3617850770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b341018c0 0x7f3b341981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:20.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.355+0000 7f3b3b97a640 1 -- 172.21.15.78:0/3617850770 >> 172.21.15.78:0/3617850770 conn(0x7f3b340fd5e0 msgr2=0x7f3b340fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:20.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.355+0000 7f3b3b97a640 1 -- 172.21.15.78:0/3617850770 shutdown_connections 2024-08-18T23:44:20.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:20.355+0000 7f3b3b97a640 1 -- 172.21.15.78:0/3617850770 wait complete. 2024-08-18T23:44:21.065 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-18T23:41:32.941573+0000', 'modified': '2024-08-18T23:43:57.114120+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24452, 'mds_1': 24428, 'mds_2': 24427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24397': {'gid': 24397, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/4100464644', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 4100464644}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 4100464644}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6837/2017185021', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 2017185021}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 2017185021}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6835/2459199316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 2459199316}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 2459199316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.161:6839/2209088174', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 2209088174}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 2209088174}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.123:6837/1639958554', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 1639958554}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 1639958554}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.123:6839/2656330956', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 2656330956}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 2656330956}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24452, 'qdb_cluster': [24452, 24428, 24427]} 2024-08-18T23:44:21.065 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-18T23:44:21.066 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.i status 2024-08-18T23:44:21.359 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:21.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:21 smithi078 ceph-mon[18440]: pgmap v351: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 7 op/s 2024-08-18T23:44:21.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:21 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3617850770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:21.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:21 smithi123 ceph-mon[21808]: pgmap v351: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 7 op/s 2024-08-18T23:44:21.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:21 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3617850770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:21 smithi161 ceph-mon[22027]: pgmap v351: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 7 op/s 2024-08-18T23:44:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:21 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3617850770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:21.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.875+0000 7f4cfa9b5640 1 -- 172.21.15.78:0/1811285398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf41078b0 msgr2=0x7f4cf4107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:21.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.875+0000 7f4cfa9b5640 1 --2- 172.21.15.78:0/1811285398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf41078b0 0x7f4cf4107c90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009930 tx=0x7f4ce402a420 comp rx=0 tx=0).stop 2024-08-18T23:44:21.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.875+0000 7f4cfa9b5640 1 -- 172.21.15.78:0/1811285398 shutdown_connections 2024-08-18T23:44:21.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.875+0000 7f4cfa9b5640 1 --2- 172.21.15.78:0/1811285398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cf4102250 0x7f4cf410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.875+0000 7f4cfa9b5640 1 --2- 172.21.15.78:0/1811285398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cf41018b0 0x7f4cf4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.875+0000 7f4cfa9b5640 1 --2- 172.21.15.78:0/1811285398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf41078b0 0x7f4cf4107c90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.875+0000 7f4cfa9b5640 1 -- 172.21.15.78:0/1811285398 >> 172.21.15.78:0/1811285398 conn(0x7f4cf40fd5f0 msgr2=0x7f4cf40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:21.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.875+0000 7f4cfa9b5640 1 -- 172.21.15.78:0/1811285398 shutdown_connections 2024-08-18T23:44:21.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.875+0000 7f4cfa9b5640 1 -- 172.21.15.78:0/1811285398 wait complete. 2024-08-18T23:44:21.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.876+0000 7f4cfa9b5640 1 Processor -- start 2024-08-18T23:44:21.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.876+0000 7f4cfa9b5640 1 -- start start 2024-08-18T23:44:21.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.876+0000 7f4cfa9b5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cf41018b0 0x7f4cf41981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:21.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.876+0000 7f4cfa9b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf4102250 0x7f4cf41986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:21.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.876+0000 7f4cfa9b5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cf41078b0 0x7f4cf419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:21.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.876+0000 7f4cfa9b5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf4111cb0 con 0x7f4cf4102250 2024-08-18T23:44:21.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.876+0000 7f4cfa9b5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf4111b30 con 0x7f4cf41018b0 2024-08-18T23:44:21.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.876+0000 7f4cfa9b5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf4111e30 con 0x7f4cf41078b0 2024-08-18T23:44:21.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.876+0000 7f4cf37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf4102250 0x7f4cf41986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:21.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.876+0000 7f4cf37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf4102250 0x7f4cf41986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59624/0 (socket says 172.21.15.78:59624) 2024-08-18T23:44:21.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.876+0000 7f4cf37fe640 1 -- 172.21.15.78:0/3121822186 learned_addr learned my addr 172.21.15.78:0/3121822186 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:21.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.876+0000 7f4cf8f52640 1 --2- 172.21.15.78:0/3121822186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cf41078b0 0x7f4cf419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:21.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.877+0000 7f4cf3fff640 1 --2- 172.21.15.78:0/3121822186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cf41018b0 0x7f4cf41981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:21.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.877+0000 7f4cf37fe640 1 -- 172.21.15.78:0/3121822186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cf41078b0 msgr2=0x7f4cf419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:21.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.877+0000 7f4cf37fe640 1 --2- 172.21.15.78:0/3121822186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cf41078b0 0x7f4cf419ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.877+0000 7f4cf37fe640 1 -- 172.21.15.78:0/3121822186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cf41018b0 msgr2=0x7f4cf41981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:21.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.877+0000 7f4cf37fe640 1 --2- 172.21.15.78:0/3121822186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cf41018b0 0x7f4cf41981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.877+0000 7f4cf37fe640 1 -- 172.21.15.78:0/3121822186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf419d160 con 0x7f4cf4102250 2024-08-18T23:44:21.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.877+0000 7f4cf37fe640 1 --2- 172.21.15.78:0/3121822186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf4102250 0x7f4cf41986f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4ce800d950 tx=0x7f4ce800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:21.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.877+0000 7f4cf17fa640 1 -- 172.21.15.78:0/3121822186 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8017070 con 0x7f4cf4102250 2024-08-18T23:44:21.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.877+0000 7f4cfa9b5640 1 -- 172.21.15.78:0/3121822186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf419d450 con 0x7f4cf4102250 2024-08-18T23:44:21.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.877+0000 7f4cfa9b5640 1 -- 172.21.15.78:0/3121822186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf41a4d30 con 0x7f4cf4102250 2024-08-18T23:44:21.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.877+0000 7f4cf17fa640 1 -- 172.21.15.78:0/3121822186 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce8009c30 con 0x7f4cf4102250 2024-08-18T23:44:21.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.877+0000 7f4cf17fa640 1 -- 172.21.15.78:0/3121822186 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8020060 con 0x7f4cf4102250 2024-08-18T23:44:21.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.880+0000 7f4cf17fa640 1 -- 172.21.15.78:0/3121822186 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f4ce8010460 con 0x7f4cf4102250 2024-08-18T23:44:21.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.880+0000 7f4cfcdb3740 1 Processor -- start 2024-08-18T23:44:21.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.880+0000 7f4cfcdb3740 1 -- start start 2024-08-18T23:44:21.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.880+0000 7f4cfcdb3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868630 0x55fa9a868a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:21.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.880+0000 7f4cfcdb3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a868f50 0x55fa9a8732d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:21.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.880+0000 7f4cfcdb3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a873810 0x55fa9a875c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:21.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.880+0000 7f4cfcdb3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a7cd710 con 0x55fa9a868630 2024-08-18T23:44:21.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.880+0000 7f4cfcdb3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a765770 con 0x55fa9a873810 2024-08-18T23:44:21.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.880+0000 7f4cfcdb3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a7c96d0 con 0x55fa9a868f50 2024-08-18T23:44:21.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.880+0000 7f4cf8f52640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868630 0x55fa9a868a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:21.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.880+0000 7f4cf8f52640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868630 0x55fa9a868a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59634/0 (socket says 172.21.15.78:59634) 2024-08-18T23:44:21.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.880+0000 7f4cf8f52640 1 -- 172.21.15.78:0/2622576188 learned_addr learned my addr 172.21.15.78:0/2622576188 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:21.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.880+0000 7f4cf37fe640 1 --2- 172.21.15.78:0/2622576188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a873810 0x55fa9a875c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:21.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.881+0000 7f4cf17fa640 1 --2- 172.21.15.78:0/3121822186 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4cdc0a2c10 0x7f4cdc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:21.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.881+0000 7f4cf3fff640 1 --2- 172.21.15.78:0/2622576188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a868f50 0x55fa9a8732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:21.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.881+0000 7f4cf8f52640 1 -- 172.21.15.78:0/2622576188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a868f50 msgr2=0x55fa9a8732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:21.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.881+0000 7f4cf8f52640 1 --2- 172.21.15.78:0/2622576188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a868f50 0x55fa9a8732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.881+0000 7f4cf8f52640 1 -- 172.21.15.78:0/2622576188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a873810 msgr2=0x55fa9a875c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:21.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.881+0000 7f4cf8f52640 1 --2- 172.21.15.78:0/2622576188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a873810 0x55fa9a875c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.881+0000 7f4cf8f52640 1 -- 172.21.15.78:0/2622576188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa9a76c3e0 con 0x55fa9a868630 2024-08-18T23:44:21.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.881+0000 7f4cf8f52640 1 --2- 172.21.15.78:0/3121822186 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4cdc0a2c10 0x7f4cdc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:21.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.881+0000 7f4cf17fa640 1 -- 172.21.15.78:0/3121822186 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f4ce80cedf0 con 0x7f4cf4102250 2024-08-18T23:44:21.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.881+0000 7f4cf8f52640 1 --2- 172.21.15.78:0/2622576188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868630 0x55fa9a868a10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4cec008ec0 tx=0x7f4cec00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b11701cdf81c83 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:21.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.881+0000 7f4cda7fc640 1 -- 172.21.15.78:0/2622576188 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec022070 con 0x55fa9a868630 2024-08-18T23:44:21.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.881+0000 7f4cf8f52640 1 --2- 172.21.15.78:0/3121822186 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4cdc0a2c10 0x7f4cdc0a50d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4cec01f000 tx=0x7f4cec020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:21.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.881+0000 7f4cda7fc640 1 -- 172.21.15.78:0/2622576188 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cec01c470 con 0x55fa9a868630 2024-08-18T23:44:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.881+0000 7f4cda7fc640 1 -- 172.21.15.78:0/2622576188 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec01a660 con 0x55fa9a868630 2024-08-18T23:44:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.882+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/2622576188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868630 msgr2=0x55fa9a868a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.882+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/2622576188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868630 0x55fa9a868a10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4cec008ec0 tx=0x7f4cec00e720 comp rx=0 tx=0).stop 2024-08-18T23:44:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.882+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/2622576188 shutdown_connections 2024-08-18T23:44:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.882+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/2622576188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a873810 0x55fa9a875c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.882+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/2622576188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a868f50 0x55fa9a8732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.882+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/2622576188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868630 0x55fa9a868a10 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.882+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/2622576188 >> 172.21.15.78:0/2622576188 conn(0x55fa9a864bc0 msgr2=0x55fa9a864fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.882+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/2622576188 shutdown_connections 2024-08-18T23:44:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.882+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/2622576188 wait complete. 2024-08-18T23:44:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.882+0000 7f4cfcdb3740 1 -- start start 2024-08-18T23:44:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cfcdb3740 1 Processor -- start 2024-08-18T23:44:21.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cfcdb3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a868630 0x55fa9a870730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:21.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cfcdb3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a868f50 0x55fa9a870c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:21.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cfcdb3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a873810 0x55fa9a8711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:21.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cfcdb3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a8763e0 con 0x55fa9a873810 2024-08-18T23:44:21.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cfcdb3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a788970 con 0x55fa9a868f50 2024-08-18T23:44:21.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cfcdb3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a876560 con 0x55fa9a868630 2024-08-18T23:44:21.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cfcdb3740 10 client.0 _ll_register_callbacks cb 0x55fa9a4dc7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:21.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cf8f52640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a873810 0x55fa9a8711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:21.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cf8f52640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a873810 0x55fa9a8711b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59642/0 (socket says 172.21.15.78:59642) 2024-08-18T23:44:21.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cf8f52640 1 -- 172.21.15.78:0/484310784 learned_addr learned my addr 172.21.15.78:0/484310784 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:21.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cf3fff640 1 --2- 172.21.15.78:0/484310784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a868630 0x55fa9a870730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:21.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cf37fe640 1 --2- 172.21.15.78:0/484310784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a868f50 0x55fa9a870c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:21.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cf8f52640 1 -- 172.21.15.78:0/484310784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a868630 msgr2=0x55fa9a870730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:21.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cf8f52640 1 --2- 172.21.15.78:0/484310784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a868630 0x55fa9a870730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cf8f52640 1 -- 172.21.15.78:0/484310784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a868f50 msgr2=0x55fa9a870c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:21.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cf8f52640 1 --2- 172.21.15.78:0/484310784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a868f50 0x55fa9a870c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cf8f52640 1 -- 172.21.15.78:0/484310784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa9a53f2b0 con 0x55fa9a873810 2024-08-18T23:44:21.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cf8f52640 1 --2- 172.21.15.78:0/484310784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a873810 0x55fa9a8711b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4cec01cd80 tx=0x7f4cec01cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8879ff57e124c0f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:21.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:44:21.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.883+0000 7f4cd37fe640 1 -- 172.21.15.78:0/484310784 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec022040 con 0x55fa9a873810 2024-08-18T23:44:21.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.884+0000 7f4cd37fe640 1 -- 172.21.15.78:0/484310784 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f4cec011df0 con 0x55fa9a873810 2024-08-18T23:44:21.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.884+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/484310784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cec01c470 con 0x55fa9a873810 2024-08-18T23:44:21.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.884+0000 7f4cd37fe640 1 -- 172.21.15.78:0/484310784 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4cec03a050 con 0x55fa9a873810 2024-08-18T23:44:21.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.884+0000 7f4cfcdb3740 10 client.15420 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:21.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.884+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/484310784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa9a8864c0 con 0x55fa9a873810 2024-08-18T23:44:21.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.884+0000 7f4cd37fe640 1 -- 172.21.15.78:0/484310784 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cec01c470 con 0x55fa9a873810 2024-08-18T23:44:21.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.884+0000 7f4cd37fe640 1 -- 172.21.15.78:0/484310784 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4cec019040 con 0x55fa9a873810 2024-08-18T23:44:21.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.884+0000 7f4cfcdb3740 10 client.15420 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:21.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.884+0000 7f4cfcdb3740 10 client.15420 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-18T23:44:21.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.884+0000 7f4cfcdb3740 20 client.15420 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:21.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.884+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/484310784 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55fa9a8888b0 0x55fa9a889d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:21.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.884+0000 7f4cfcdb3740 4 client.15420 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:44:21.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.884+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/484310784 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa9a7dac80 con 0x55fa9a8888b0 2024-08-18T23:44:21.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.884+0000 7f4cf3fff640 1 --2- 172.21.15.78:0/484310784 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55fa9a8888b0 0x55fa9a889d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:21.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.885+0000 7f4cd37fe640 10 client.15420 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:44:21.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.885+0000 7f4cf3fff640 1 --2- 172.21.15.78:0/484310784 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55fa9a8888b0 0x55fa9a889d20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=afd1c254a9162b76 server_cookie=71ec500b5b46c2ed in_seq=0 out_seq=0 2024-08-18T23:44:21.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.886+0000 7f4cd37fe640 1 -- 172.21.15.78:0/484310784 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa9a7dac80 con 0x55fa9a8888b0 2024-08-18T23:44:21.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.886+0000 7f4cd37fe640 10 client.15420 handle_command_reply: tid=0 2024-08-18T23:44:21.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.886+0000 7f4cfcdb3740 1 client.15420 shutdown 2024-08-18T23:44:21.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.886+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/484310784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a873810 msgr2=0x55fa9a8711b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:21.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.886+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/484310784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a873810 0x55fa9a8711b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4cec01cd80 tx=0x7f4cec01cb20 comp rx=0 tx=0).stop 2024-08-18T23:44:21.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.886+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/484310784 shutdown_connections 2024-08-18T23:44:21.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.886+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/484310784 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55fa9a8888b0 0x55fa9a889d20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.886+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/484310784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a873810 0x55fa9a8711b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.886+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/484310784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a868f50 0x55fa9a870c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.886+0000 7f4cd37fe640 0 client.15420 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:44:21.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.886+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/484310784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a868630 0x55fa9a870730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.886+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/484310784 >> 172.21.15.78:0/484310784 conn(0x55fa9a864bc0 msgr2=0x55fa9a863d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:21.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.886+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/484310784 shutdown_connections 2024-08-18T23:44:21.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.886+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/484310784 wait complete. 2024-08-18T23:44:21.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.886+0000 7f4cfcdb3740 20 client.15420 trim_cache size 0 max 16384 2024-08-18T23:44:21.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.904+0000 7f4cfcdb3740 1 Processor -- start 2024-08-18T23:44:21.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.904+0000 7f4cfcdb3740 1 -- start start 2024-08-18T23:44:21.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.904+0000 7f4cfcdb3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868630 0x55fa9a889650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:21.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.904+0000 7f4cfcdb3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a868f50 0x55fa9a8833e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:21.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.904+0000 7f4cfcdb3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a873810 0x55fa9a870c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:21.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.904+0000 7f4cfcdb3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a888060 con 0x55fa9a868630 2024-08-18T23:44:21.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.904+0000 7f4cfcdb3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a888730 con 0x55fa9a868f50 2024-08-18T23:44:21.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.904+0000 7f4cfcdb3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a888e00 con 0x55fa9a873810 2024-08-18T23:44:21.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.904+0000 7f4cf3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868630 0x55fa9a889650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:21.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.904+0000 7f4cf3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868630 0x55fa9a889650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59658/0 (socket says 172.21.15.78:59658) 2024-08-18T23:44:21.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.904+0000 7f4cf3fff640 1 -- 172.21.15.78:0/3597444428 learned_addr learned my addr 172.21.15.78:0/3597444428 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:21.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.904+0000 7f4cf8f52640 1 --2- 172.21.15.78:0/3597444428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a873810 0x55fa9a870c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:21.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.905+0000 7f4cf37fe640 1 --2- 172.21.15.78:0/3597444428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a868f50 0x55fa9a8833e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:21.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.905+0000 7f4cf3fff640 1 -- 172.21.15.78:0/3597444428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a873810 msgr2=0x55fa9a870c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:21.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.905+0000 7f4cf3fff640 1 --2- 172.21.15.78:0/3597444428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a873810 0x55fa9a870c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.905+0000 7f4cf3fff640 1 -- 172.21.15.78:0/3597444428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a868f50 msgr2=0x55fa9a8833e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:21.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.905+0000 7f4cf3fff640 1 --2- 172.21.15.78:0/3597444428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a868f50 0x55fa9a8833e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.905+0000 7f4cf3fff640 1 -- 172.21.15.78:0/3597444428 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa9a868d80 con 0x55fa9a868630 2024-08-18T23:44:21.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.905+0000 7f4cf3fff640 1 --2- 172.21.15.78:0/3597444428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868630 0x55fa9a889650 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009900 tx=0x7f4ce40058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df17efdbbcc1662 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:21.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.905+0000 7f4cd9ffb640 1 -- 172.21.15.78:0/3597444428 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4035040 con 0x55fa9a868630 2024-08-18T23:44:21.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.905+0000 7f4cd9ffb640 1 -- 172.21.15.78:0/3597444428 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce4005040 con 0x55fa9a868630 2024-08-18T23:44:21.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.905+0000 7f4cd9ffb640 1 -- 172.21.15.78:0/3597444428 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4004bc0 con 0x55fa9a868630 2024-08-18T23:44:21.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.906+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/3597444428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868630 msgr2=0x55fa9a889650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:21.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.906+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/3597444428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868630 0x55fa9a889650 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009900 tx=0x7f4ce40058b0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.906+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/3597444428 shutdown_connections 2024-08-18T23:44:21.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.906+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/3597444428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a873810 0x55fa9a870c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.906+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/3597444428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a868f50 0x55fa9a8833e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.906+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/3597444428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868630 0x55fa9a889650 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.906+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/3597444428 >> 172.21.15.78:0/3597444428 conn(0x55fa9a864bc0 msgr2=0x55fa9a882cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:21.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.906+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/3597444428 shutdown_connections 2024-08-18T23:44:21.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.906+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/3597444428 wait complete. 2024-08-18T23:44:21.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.906+0000 7f4cfcdb3740 1 -- start start 2024-08-18T23:44:21.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.906+0000 7f4cfcdb3740 1 Processor -- start 2024-08-18T23:44:21.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.906+0000 7f4cfcdb3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a868630 0x55fa9a88cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:21.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.906+0000 7f4cfcdb3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868f50 0x55fa9a88d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:21.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.906+0000 7f4cfcdb3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a873810 0x55fa9a891d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:21.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.906+0000 7f4cfcdb3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a889ba0 con 0x55fa9a868f50 2024-08-18T23:44:21.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.906+0000 7f4cfcdb3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a8894d0 con 0x55fa9a868630 2024-08-18T23:44:21.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.906+0000 7f4cfcdb3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a88a270 con 0x55fa9a873810 2024-08-18T23:44:21.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cfcdb3740 10 client.0 _ll_register_callbacks cb 0x7f4cec020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:21.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cf37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868f50 0x55fa9a88d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:21.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cf37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868f50 0x55fa9a88d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59668/0 (socket says 172.21.15.78:59668) 2024-08-18T23:44:21.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cf37fe640 1 -- 172.21.15.78:0/3841680087 learned_addr learned my addr 172.21.15.78:0/3841680087 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cf3fff640 1 --2- 172.21.15.78:0/3841680087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a868630 0x55fa9a88cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cf8f52640 1 --2- 172.21.15.78:0/3841680087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a873810 0x55fa9a891d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cf37fe640 1 -- 172.21.15.78:0/3841680087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a873810 msgr2=0x55fa9a891d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cf37fe640 1 --2- 172.21.15.78:0/3841680087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a873810 0x55fa9a891d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cf37fe640 1 -- 172.21.15.78:0/3841680087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a868630 msgr2=0x55fa9a88cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cf37fe640 1 --2- 172.21.15.78:0/3841680087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a868630 0x55fa9a88cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cf37fe640 1 -- 172.21.15.78:0/3841680087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa9a88db30 con 0x55fa9a868f50 2024-08-18T23:44:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cf37fe640 1 --2- 172.21.15.78:0/3841680087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868f50 0x55fa9a88d280 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4ce809b0f0 tx=0x7f4ce800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72a9c5c40575150 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:44:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cd37fe640 1 -- 172.21.15.78:0/3841680087 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce809b160 con 0x55fa9a868f50 2024-08-18T23:44:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/3841680087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ce4005040 con 0x55fa9a868f50 2024-08-18T23:44:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cd37fe640 1 -- 172.21.15.78:0/3841680087 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f4ce80977d0 con 0x55fa9a868f50 2024-08-18T23:44:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cd37fe640 1 -- 172.21.15.78:0/3841680087 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4ce809f510 con 0x55fa9a868f50 2024-08-18T23:44:21.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cfcdb3740 10 client.15426 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:21.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/3841680087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa9a8f3620 con 0x55fa9a868f50 2024-08-18T23:44:21.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.907+0000 7f4cd37fe640 1 -- 172.21.15.78:0/3841680087 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce4005040 con 0x55fa9a868f50 2024-08-18T23:44:21.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.908+0000 7f4cd37fe640 1 -- 172.21.15.78:0/3841680087 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4ce8023060 con 0x55fa9a868f50 2024-08-18T23:44:21.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.908+0000 7f4cfcdb3740 10 client.15426 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:21.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.908+0000 7f4cfcdb3740 10 client.15426 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-18T23:44:21.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.908+0000 7f4cfcdb3740 20 client.15426 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:21.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.908+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/3841680087 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55fa9a8f81c0 0x55fa9a8f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:21.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.908+0000 7f4cfcdb3740 4 client.15426 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-18T23:44:21.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.908+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/3841680087 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "status"}) -- 0x55fa9a7d5190 con 0x55fa9a8f81c0 2024-08-18T23:44:21.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.908+0000 7f4cf3fff640 1 --2- 172.21.15.78:0/3841680087 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55fa9a8f81c0 0x55fa9a8f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:21.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.908+0000 7f4cd37fe640 10 client.15426 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:44:21.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.908+0000 7f4cf3fff640 1 --2- 172.21.15.78:0/3841680087 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55fa9a8f81c0 0x55fa9a8f6ed0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=596313a989b4cdc8 server_cookie=a72ff99560981cc0 in_seq=0 out_seq=0 2024-08-18T23:44:21.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.909+0000 7f4cd37fe640 1 -- 172.21.15.78:0/3841680087 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fa9a7d5190 con 0x55fa9a8f81c0 2024-08-18T23:44:21.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.909+0000 7f4cd37fe640 10 client.15426 handle_command_reply: tid=0 2024-08-18T23:44:21.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.909+0000 7f4cfcdb3740 1 client.15426 shutdown 2024-08-18T23:44:21.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.909+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/3841680087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868f50 msgr2=0x55fa9a88d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:21.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.909+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/3841680087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868f50 0x55fa9a88d280 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4ce809b0f0 tx=0x7f4ce800bf60 comp rx=0 tx=0).stop 2024-08-18T23:44:21.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.909+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/3841680087 shutdown_connections 2024-08-18T23:44:21.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.909+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/3841680087 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55fa9a8f81c0 0x55fa9a8f6ed0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.909+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/3841680087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fa9a873810 0x55fa9a891d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.909+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/3841680087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fa9a868f50 0x55fa9a88d280 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.909+0000 7f4cd37fe640 0 client.15426 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:44:21.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.909+0000 7f4cfcdb3740 1 --2- 172.21.15.78:0/3841680087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa9a868630 0x55fa9a88cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.909+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/3841680087 >> 172.21.15.78:0/3841680087 conn(0x55fa9a864bc0 msgr2=0x55fa9a8834f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:21.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.909+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/3841680087 shutdown_connections 2024-08-18T23:44:21.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.909+0000 7f4cfcdb3740 1 -- 172.21.15.78:0/3841680087 wait complete. 2024-08-18T23:44:21.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.909+0000 7f4cfcdb3740 20 client.15426 trim_cache size 0 max 16384 2024-08-18T23:44:21.922 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:44:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "4f1128e6-5dba-11ef-bccf-c7b262605968", 2024-08-18T23:44:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 2, 2024-08-18T23:44:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24427, 2024-08-18T23:44:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-08-18T23:44:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-08-18T23:44:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-08-18T23:44:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 134.65507433900001, 2024-08-18T23:44:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 25, 2024-08-18T23:44:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 92, 2024-08-18T23:44:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 92, 2024-08-18T23:44:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 188.64492657900001 2024-08-18T23:44:21.923 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:44:21.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.910+0000 7f4cfa9b5640 1 -- 172.21.15.78:0/3121822186 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4cdc0a2c10 msgr2=0x7f4cdc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:21.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.910+0000 7f4cfa9b5640 1 --2- 172.21.15.78:0/3121822186 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4cdc0a2c10 0x7f4cdc0a50d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4cec01f000 tx=0x7f4cec020000 comp rx=0 tx=0).stop 2024-08-18T23:44:21.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.910+0000 7f4cfa9b5640 1 -- 172.21.15.78:0/3121822186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf4102250 msgr2=0x7f4cf41986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:21.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.910+0000 7f4cfa9b5640 1 --2- 172.21.15.78:0/3121822186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf4102250 0x7f4cf41986f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4ce800d950 tx=0x7f4ce800de10 comp rx=0 tx=0).stop 2024-08-18T23:44:21.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.910+0000 7f4cfa9b5640 1 -- 172.21.15.78:0/3121822186 shutdown_connections 2024-08-18T23:44:21.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.910+0000 7f4cfa9b5640 1 --2- 172.21.15.78:0/3121822186 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4cdc0a2c10 0x7f4cdc0a50d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.910+0000 7f4cfa9b5640 1 --2- 172.21.15.78:0/3121822186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cf41078b0 0x7f4cf419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.910+0000 7f4cfa9b5640 1 --2- 172.21.15.78:0/3121822186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cf4102250 0x7f4cf41986f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.910+0000 7f4cfa9b5640 1 --2- 172.21.15.78:0/3121822186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cf41018b0 0x7f4cf41981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:21.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.910+0000 7f4cfa9b5640 1 -- 172.21.15.78:0/3121822186 >> 172.21.15.78:0/3121822186 conn(0x7f4cf40fd5f0 msgr2=0x7f4cf40fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:21.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.910+0000 7f4cfa9b5640 1 -- 172.21.15.78:0/3121822186 shutdown_connections 2024-08-18T23:44:21.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:21.910+0000 7f4cfa9b5640 1 -- 172.21.15.78:0/3121822186 wait complete. 2024-08-18T23:44:22.691 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.h status 2024-08-18T23:44:22.942 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:23.356 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:23 smithi078 ceph-mon[18440]: pgmap v352: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-18T23:44:23.356 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:23 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:44:23.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.434+0000 7f4164fda640 1 -- 172.21.15.78:0/3290441025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4160101150 msgr2=0x7f416010d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:23.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.434+0000 7f4164fda640 1 --2- 172.21.15.78:0/3290441025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4160101150 0x7f416010d680 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4158009cd0 tx=0x7f415802a8e0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.434+0000 7f4164fda640 1 -- 172.21.15.78:0/3290441025 shutdown_connections 2024-08-18T23:44:23.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.434+0000 7f4164fda640 1 --2- 172.21.15.78:0/3290441025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4160101150 0x7f416010d680 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.434+0000 7f4164fda640 1 --2- 172.21.15.78:0/3290441025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41601007b0 0x7f4160100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.434+0000 7f4164fda640 1 --2- 172.21.15.78:0/3290441025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41601067b0 0x7f4160106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.434+0000 7f4164fda640 1 -- 172.21.15.78:0/3290441025 >> 172.21.15.78:0/3290441025 conn(0x7f41600fc4d0 msgr2=0x7f41600fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:23.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.434+0000 7f4164fda640 1 -- 172.21.15.78:0/3290441025 shutdown_connections 2024-08-18T23:44:23.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.434+0000 7f4164fda640 1 -- 172.21.15.78:0/3290441025 wait complete. 2024-08-18T23:44:23.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.435+0000 7f4164fda640 1 Processor -- start 2024-08-18T23:44:23.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.435+0000 7f4164fda640 1 -- start start 2024-08-18T23:44:23.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.435+0000 7f4164fda640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41601007b0 0x7f4160196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:23.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.435+0000 7f4164fda640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4160101150 0x7f41601972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:23.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.435+0000 7f4164fda640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41601067b0 0x7f4160197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:23.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.435+0000 7f4164fda640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4160110bb0 con 0x7f41601007b0 2024-08-18T23:44:23.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.435+0000 7f4164fda640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4160110a30 con 0x7f41601067b0 2024-08-18T23:44:23.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.435+0000 7f4164fda640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4160110d30 con 0x7f4160101150 2024-08-18T23:44:23.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.435+0000 7f415e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41601007b0 0x7f4160196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:23.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.435+0000 7f415e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41601007b0 0x7f4160196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59684/0 (socket says 172.21.15.78:59684) 2024-08-18T23:44:23.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.435+0000 7f415e59c640 1 -- 172.21.15.78:0/1223299309 learned_addr learned my addr 172.21.15.78:0/1223299309 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:23.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.435+0000 7f415dd9b640 1 --2- 172.21.15.78:0/1223299309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4160101150 0x7f41601972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:23.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.435+0000 7f415ed9d640 1 --2- 172.21.15.78:0/1223299309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41601067b0 0x7f4160197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:23.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.435+0000 7f415e59c640 1 -- 172.21.15.78:0/1223299309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4160101150 msgr2=0x7f41601972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:23.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.435+0000 7f415e59c640 1 --2- 172.21.15.78:0/1223299309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4160101150 0x7f41601972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.435+0000 7f415e59c640 1 -- 172.21.15.78:0/1223299309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41601067b0 msgr2=0x7f4160197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:23.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.435+0000 7f415e59c640 1 --2- 172.21.15.78:0/1223299309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41601067b0 0x7f4160197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.435+0000 7f415e59c640 1 -- 172.21.15.78:0/1223299309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41601a6600 con 0x7f41601007b0 2024-08-18T23:44:23.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.435+0000 7f415e59c640 1 --2- 172.21.15.78:0/1223299309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41601007b0 0x7f4160196d90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f415000d770 tx=0x7f415000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:23.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.436+0000 7f41577fe640 1 -- 172.21.15.78:0/1223299309 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4150017070 con 0x7f41601007b0 2024-08-18T23:44:23.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.436+0000 7f41577fe640 1 -- 172.21.15.78:0/1223299309 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4150005500 con 0x7f41601007b0 2024-08-18T23:44:23.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.436+0000 7f4164fda640 1 -- 172.21.15.78:0/1223299309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41601a68f0 con 0x7f41601007b0 2024-08-18T23:44:23.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.436+0000 7f41577fe640 1 -- 172.21.15.78:0/1223299309 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4150004e00 con 0x7f41601007b0 2024-08-18T23:44:23.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.436+0000 7f4164fda640 1 -- 172.21.15.78:0/1223299309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41601a6de0 con 0x7f41601007b0 2024-08-18T23:44:23.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.438+0000 7f41577fe640 1 -- 172.21.15.78:0/1223299309 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f4150004fa0 con 0x7f41601007b0 2024-08-18T23:44:23.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.438+0000 7f41673d8740 1 Processor -- start 2024-08-18T23:44:23.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.438+0000 7f41673d8740 1 -- start start 2024-08-18T23:44:23.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.439+0000 7f41673d8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 0x55e73bcf3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:23.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.439+0000 7f41673d8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcf3f50 0x55e73bcfe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:23.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.439+0000 7f41673d8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcfe810 0x55e73bd00c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:23.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.439+0000 7f41673d8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e73bc58710 con 0x55e73bcf3630 2024-08-18T23:44:23.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.439+0000 7f41673d8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e73bbf0770 con 0x55e73bcfe810 2024-08-18T23:44:23.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.439+0000 7f41673d8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e73bc546d0 con 0x55e73bcf3f50 2024-08-18T23:44:23.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.439+0000 7f415ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 0x55e73bcf3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:23.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.439+0000 7f415ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 0x55e73bcf3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59690/0 (socket says 172.21.15.78:59690) 2024-08-18T23:44:23.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.439+0000 7f415ed9d640 1 -- 172.21.15.78:0/2144960818 learned_addr learned my addr 172.21.15.78:0/2144960818 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:23.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.439+0000 7f41577fe640 1 --2- 172.21.15.78:0/1223299309 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f414c0a2c10 0x7f414c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:23.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.439+0000 7f415e59c640 1 --2- 172.21.15.78:0/2144960818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcf3f50 0x55e73bcfe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:23.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.439+0000 7f415dd9b640 1 --2- 172.21.15.78:0/2144960818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcfe810 0x55e73bd00c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:23.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.439+0000 7f415ed9d640 1 --2- 172.21.15.78:0/1223299309 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f414c0a2c10 0x7f414c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:23.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.439+0000 7f41577fe640 1 -- 172.21.15.78:0/1223299309 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f41500ceed0 con 0x7f41601007b0 2024-08-18T23:44:23.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.439+0000 7f415ed9d640 1 -- 172.21.15.78:0/2144960818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcf3f50 msgr2=0x55e73bcfe2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:23.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.439+0000 7f415ed9d640 1 --2- 172.21.15.78:0/2144960818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcf3f50 0x55e73bcfe2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.439+0000 7f415ed9d640 1 -- 172.21.15.78:0/2144960818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcfe810 msgr2=0x55e73bd00c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:23.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.439+0000 7f415ed9d640 1 --2- 172.21.15.78:0/2144960818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcfe810 0x55e73bd00c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.439+0000 7f415ed9d640 1 -- 172.21.15.78:0/2144960818 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e73bbf73e0 con 0x55e73bcf3630 2024-08-18T23:44:23.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.440+0000 7f415ed9d640 1 --2- 172.21.15.78:0/2144960818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 0x55e73bcf3a10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4158009430 tx=0x7f4158031b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176a8ed5cbe52588 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:23.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.440+0000 7f4154ff9640 1 -- 172.21.15.78:0/2144960818 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158004610 con 0x55e73bcf3630 2024-08-18T23:44:23.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.440+0000 7f4154ff9640 1 -- 172.21.15.78:0/2144960818 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4158036070 con 0x55e73bcf3630 2024-08-18T23:44:23.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.440+0000 7f4154ff9640 1 -- 172.21.15.78:0/2144960818 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415804a450 con 0x55e73bcf3630 2024-08-18T23:44:23.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.440+0000 7f415ed9d640 1 --2- 172.21.15.78:0/1223299309 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f414c0a2c10 0x7f414c0a50d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4158009a60 tx=0x7f415804c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:23.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.440+0000 7f41673d8740 1 -- 172.21.15.78:0/2144960818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 msgr2=0x55e73bcf3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:23.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.440+0000 7f41673d8740 1 --2- 172.21.15.78:0/2144960818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 0x55e73bcf3a10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4158009430 tx=0x7f4158031b00 comp rx=0 tx=0).stop 2024-08-18T23:44:23.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.440+0000 7f41673d8740 1 -- 172.21.15.78:0/2144960818 shutdown_connections 2024-08-18T23:44:23.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.440+0000 7f41673d8740 1 --2- 172.21.15.78:0/2144960818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcfe810 0x55e73bd00c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.440+0000 7f41673d8740 1 --2- 172.21.15.78:0/2144960818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcf3f50 0x55e73bcfe2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.441+0000 7f41673d8740 1 --2- 172.21.15.78:0/2144960818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 0x55e73bcf3a10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.441+0000 7f41673d8740 1 -- 172.21.15.78:0/2144960818 >> 172.21.15.78:0/2144960818 conn(0x55e73bcefbc0 msgr2=0x55e73bceffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:23.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.441+0000 7f41673d8740 1 -- 172.21.15.78:0/2144960818 shutdown_connections 2024-08-18T23:44:23.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.441+0000 7f41673d8740 1 -- 172.21.15.78:0/2144960818 wait complete. 2024-08-18T23:44:23.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.441+0000 7f41673d8740 1 -- start start 2024-08-18T23:44:23.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.441+0000 7f41673d8740 1 Processor -- start 2024-08-18T23:44:23.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.441+0000 7f41673d8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcf3630 0x55e73bcfb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:23.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.441+0000 7f41673d8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3f50 0x55e73bcfbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:23.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.441+0000 7f41673d8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcfe810 0x55e73bcfc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:23.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.441+0000 7f41673d8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e73bd013e0 con 0x55e73bcf3f50 2024-08-18T23:44:23.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.441+0000 7f41673d8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e73bc13970 con 0x55e73bcfe810 2024-08-18T23:44:23.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.441+0000 7f41673d8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e73bd01560 con 0x55e73bcf3630 2024-08-18T23:44:23.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.441+0000 7f41673d8740 10 client.0 _ll_register_callbacks cb 0x55e73b9677b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:23.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.441+0000 7f415dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3f50 0x55e73bcfbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:23.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.441+0000 7f415dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3f50 0x55e73bcfbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59698/0 (socket says 172.21.15.78:59698) 2024-08-18T23:44:23.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.441+0000 7f415dd9b640 1 -- 172.21.15.78:0/2505261455 learned_addr learned my addr 172.21.15.78:0/2505261455 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:23.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.442+0000 7f415e59c640 1 --2- 172.21.15.78:0/2505261455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcf3630 0x55e73bcfb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:23.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.442+0000 7f415ed9d640 1 --2- 172.21.15.78:0/2505261455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcfe810 0x55e73bcfc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:23.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.442+0000 7f415dd9b640 1 -- 172.21.15.78:0/2505261455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcf3630 msgr2=0x55e73bcfb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:23.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.442+0000 7f415dd9b640 1 --2- 172.21.15.78:0/2505261455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcf3630 0x55e73bcfb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.442+0000 7f415dd9b640 1 -- 172.21.15.78:0/2505261455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcfe810 msgr2=0x55e73bcfc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:23.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.442+0000 7f415dd9b640 1 --2- 172.21.15.78:0/2505261455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcfe810 0x55e73bcfc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.442+0000 7f415dd9b640 1 -- 172.21.15.78:0/2505261455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e73b9ca2b0 con 0x55e73bcf3f50 2024-08-18T23:44:23.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.442+0000 7f415ed9d640 1 --2- 172.21.15.78:0/2505261455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcfe810 0x55e73bcfc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:23.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.442+0000 7f415dd9b640 1 --2- 172.21.15.78:0/2505261455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3f50 0x55e73bcfbc20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f414800bb00 tx=0x7f4148004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d721f6d491238416 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:23.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.442+0000 7f4145ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:44:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.442+0000 7f4145ffb640 1 -- 172.21.15.78:0/2505261455 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148003e80 con 0x55e73bcf3f50 2024-08-18T23:44:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.442+0000 7f4145ffb640 1 -- 172.21.15.78:0/2505261455 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f4148011440 con 0x55e73bcf3f50 2024-08-18T23:44:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.442+0000 7f41673d8740 1 -- 172.21.15.78:0/2505261455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4158036070 con 0x55e73bcf3f50 2024-08-18T23:44:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.442+0000 7f4145ffb640 1 -- 172.21.15.78:0/2505261455 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4148005020 con 0x55e73bcf3f50 2024-08-18T23:44:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.442+0000 7f41673d8740 10 client.15438 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.442+0000 7f41673d8740 1 -- 172.21.15.78:0/2505261455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e73bd0d090 con 0x55e73bcf3f50 2024-08-18T23:44:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.442+0000 7f4145ffb640 1 -- 172.21.15.78:0/2505261455 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4158036070 con 0x55e73bcf3f50 2024-08-18T23:44:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.443+0000 7f4145ffb640 1 -- 172.21.15.78:0/2505261455 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f41480116d0 con 0x55e73bcf3f50 2024-08-18T23:44:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.443+0000 7f41673d8740 10 client.15438 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.443+0000 7f41673d8740 10 client.15438 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-18T23:44:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.443+0000 7f41673d8740 20 client.15438 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.443+0000 7f41673d8740 1 --2- 172.21.15.78:0/2505261455 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55e73bd0f730 0x55e73bd10ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.443+0000 7f41673d8740 4 client.15438 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:44:23.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.443+0000 7f41673d8740 1 -- 172.21.15.78:0/2505261455 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e73bc65c80 con 0x55e73bd0f730 2024-08-18T23:44:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.443+0000 7f415e59c640 1 --2- 172.21.15.78:0/2505261455 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55e73bd0f730 0x55e73bd10ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.443+0000 7f4145ffb640 10 client.15438 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:44:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.443+0000 7f415e59c640 1 --2- 172.21.15.78:0/2505261455 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55e73bd0f730 0x55e73bd10ba0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef50c1cf0088ba73 server_cookie=78d52ccf038ea506 in_seq=0 out_seq=0 2024-08-18T23:44:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.444+0000 7f4145ffb640 1 -- 172.21.15.78:0/2505261455 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e73bc65c80 con 0x55e73bd0f730 2024-08-18T23:44:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.444+0000 7f4145ffb640 10 client.15438 handle_command_reply: tid=0 2024-08-18T23:44:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.444+0000 7f41673d8740 1 client.15438 shutdown 2024-08-18T23:44:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.444+0000 7f41673d8740 1 -- 172.21.15.78:0/2505261455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3f50 msgr2=0x55e73bcfbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.444+0000 7f41673d8740 1 --2- 172.21.15.78:0/2505261455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3f50 0x55e73bcfbc20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f414800bb00 tx=0x7f4148004500 comp rx=0 tx=0).stop 2024-08-18T23:44:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.444+0000 7f41673d8740 1 -- 172.21.15.78:0/2505261455 shutdown_connections 2024-08-18T23:44:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.444+0000 7f41673d8740 1 --2- 172.21.15.78:0/2505261455 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55e73bd0f730 0x55e73bd10ba0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.444+0000 7f41673d8740 1 --2- 172.21.15.78:0/2505261455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcfe810 0x55e73bcfc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.444+0000 7f41673d8740 1 --2- 172.21.15.78:0/2505261455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3f50 0x55e73bcfbc20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.444+0000 7f4145ffb640 0 client.15438 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:44:23.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.444+0000 7f41673d8740 1 --2- 172.21.15.78:0/2505261455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcf3630 0x55e73bcfb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.444+0000 7f41673d8740 1 -- 172.21.15.78:0/2505261455 >> 172.21.15.78:0/2505261455 conn(0x55e73bcefbc0 msgr2=0x55e73bceed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:23.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.444+0000 7f41673d8740 1 -- 172.21.15.78:0/2505261455 shutdown_connections 2024-08-18T23:44:23.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.444+0000 7f41673d8740 1 -- 172.21.15.78:0/2505261455 wait complete. 2024-08-18T23:44:23.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.444+0000 7f41673d8740 20 client.15438 trim_cache size 0 max 16384 2024-08-18T23:44:23.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.462+0000 7f41673d8740 1 Processor -- start 2024-08-18T23:44:23.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.462+0000 7f41673d8740 1 -- start start 2024-08-18T23:44:23.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.462+0000 7f41673d8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 0x55e73bd10ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:23.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.462+0000 7f41673d8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcf3f50 0x55e73bcf6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:23.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.462+0000 7f41673d8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcfe810 0x55e73bcf6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:23.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.462+0000 7f41673d8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e73bd0eee0 con 0x55e73bcf3630 2024-08-18T23:44:23.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.462+0000 7f41673d8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e73bd0f5b0 con 0x55e73bcf3f50 2024-08-18T23:44:23.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.462+0000 7f41673d8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e73bd0fc80 con 0x55e73bcfe810 2024-08-18T23:44:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.463+0000 7f415e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 0x55e73bd10ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.463+0000 7f415e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 0x55e73bd10ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59706/0 (socket says 172.21.15.78:59706) 2024-08-18T23:44:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.463+0000 7f415e59c640 1 -- 172.21.15.78:0/713006487 learned_addr learned my addr 172.21.15.78:0/713006487 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.463+0000 7f415dd9b640 1 --2- 172.21.15.78:0/713006487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcf3f50 0x55e73bcf6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.463+0000 7f415ed9d640 1 --2- 172.21.15.78:0/713006487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcfe810 0x55e73bcf6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.463+0000 7f415e59c640 1 -- 172.21.15.78:0/713006487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcfe810 msgr2=0x55e73bcf6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.463+0000 7f415e59c640 1 --2- 172.21.15.78:0/713006487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcfe810 0x55e73bcf6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.463+0000 7f415e59c640 1 -- 172.21.15.78:0/713006487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcf3f50 msgr2=0x55e73bcf6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.463+0000 7f415e59c640 1 --2- 172.21.15.78:0/713006487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcf3f50 0x55e73bcf6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.463+0000 7f415e59c640 1 -- 172.21.15.78:0/713006487 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e73bceb130 con 0x55e73bcf3630 2024-08-18T23:44:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.463+0000 7f415ed9d640 1 --2- 172.21.15.78:0/713006487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcfe810 0x55e73bcf6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:44:23.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.463+0000 7f415e59c640 1 --2- 172.21.15.78:0/713006487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 0x55e73bd10ba0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f415009af90 tx=0x7f41500d1320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3987de3da800b63c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.463+0000 7f4147fff640 1 -- 172.21.15.78:0/713006487 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4150020040 con 0x55e73bcf3630 2024-08-18T23:44:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.463+0000 7f4147fff640 1 -- 172.21.15.78:0/713006487 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f415009f590 con 0x55e73bcf3630 2024-08-18T23:44:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.463+0000 7f4147fff640 1 -- 172.21.15.78:0/713006487 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4150097900 con 0x55e73bcf3630 2024-08-18T23:44:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.464+0000 7f41673d8740 1 -- 172.21.15.78:0/713006487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 msgr2=0x55e73bd10ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.464+0000 7f41673d8740 1 --2- 172.21.15.78:0/713006487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 0x55e73bd10ba0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f415009af90 tx=0x7f41500d1320 comp rx=0 tx=0).stop 2024-08-18T23:44:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.464+0000 7f41673d8740 1 -- 172.21.15.78:0/713006487 shutdown_connections 2024-08-18T23:44:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.464+0000 7f41673d8740 1 --2- 172.21.15.78:0/713006487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcfe810 0x55e73bcf6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.464+0000 7f41673d8740 1 --2- 172.21.15.78:0/713006487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcf3f50 0x55e73bcf6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.464+0000 7f41673d8740 1 --2- 172.21.15.78:0/713006487 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 0x55e73bd10ba0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.464+0000 7f41673d8740 1 -- 172.21.15.78:0/713006487 >> 172.21.15.78:0/713006487 conn(0x55e73bcefbc0 msgr2=0x55e73bcfa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.464+0000 7f41673d8740 1 -- 172.21.15.78:0/713006487 shutdown_connections 2024-08-18T23:44:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.464+0000 7f41673d8740 1 -- 172.21.15.78:0/713006487 wait complete. 2024-08-18T23:44:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.464+0000 7f41673d8740 1 -- start start 2024-08-18T23:44:23.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.464+0000 7f41673d8740 1 Processor -- start 2024-08-18T23:44:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.464+0000 7f41673d8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 0x55e73bd13bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.464+0000 7f41673d8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcf3f50 0x55e73bd140f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.464+0000 7f41673d8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcfe810 0x55e73bd14630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.464+0000 7f41673d8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e73bd10a20 con 0x55e73bcf3630 2024-08-18T23:44:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.464+0000 7f41673d8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e73bd10350 con 0x55e73bcfe810 2024-08-18T23:44:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.464+0000 7f41673d8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e73bd110f0 con 0x55e73bcf3f50 2024-08-18T23:44:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.465+0000 7f41673d8740 10 client.0 _ll_register_callbacks cb 0x7f4148004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.465+0000 7f415e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 0x55e73bd13bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.465+0000 7f415e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 0x55e73bd13bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59720/0 (socket says 172.21.15.78:59720) 2024-08-18T23:44:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.465+0000 7f415e59c640 1 -- 172.21.15.78:0/3299256165 learned_addr learned my addr 172.21.15.78:0/3299256165 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.465+0000 7f415ed9d640 1 --2- 172.21.15.78:0/3299256165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcfe810 0x55e73bd14630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:23.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.465+0000 7f415dd9b640 1 --2- 172.21.15.78:0/3299256165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcf3f50 0x55e73bd140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.465+0000 7f415e59c640 1 -- 172.21.15.78:0/3299256165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcf3f50 msgr2=0x55e73bd140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.465+0000 7f415e59c640 1 --2- 172.21.15.78:0/3299256165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcf3f50 0x55e73bd140f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.465+0000 7f415e59c640 1 -- 172.21.15.78:0/3299256165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcfe810 msgr2=0x55e73bd14630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.465+0000 7f415e59c640 1 --2- 172.21.15.78:0/3299256165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcfe810 0x55e73bd14630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.465+0000 7f415e59c640 1 -- 172.21.15.78:0/3299256165 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e73bd18cb0 con 0x55e73bcf3630 2024-08-18T23:44:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.465+0000 7f415e59c640 1 --2- 172.21.15.78:0/3299256165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 0x55e73bd13bb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f41500976f0 tx=0x7f41500048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78dd94f4539a84 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.465+0000 7f4145ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:44:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.465+0000 7f4145ffb640 1 -- 172.21.15.78:0/3299256165 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4150084070 con 0x55e73bcf3630 2024-08-18T23:44:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.465+0000 7f4145ffb640 1 -- 172.21.15.78:0/3299256165 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f4150009cc0 con 0x55e73bcf3630 2024-08-18T23:44:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.465+0000 7f41673d8740 1 -- 172.21.15.78:0/3299256165 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f415009f590 con 0x55e73bcf3630 2024-08-18T23:44:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.465+0000 7f4145ffb640 1 -- 172.21.15.78:0/3299256165 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4150020040 con 0x55e73bcf3630 2024-08-18T23:44:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.465+0000 7f41673d8740 10 client.15444 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.465+0000 7f41673d8740 1 -- 172.21.15.78:0/3299256165 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e73bd7a150 con 0x55e73bcf3630 2024-08-18T23:44:23.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.466+0000 7f4145ffb640 1 -- 172.21.15.78:0/3299256165 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f415009f590 con 0x55e73bcf3630 2024-08-18T23:44:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.466+0000 7f4145ffb640 1 -- 172.21.15.78:0/3299256165 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f415009f190 con 0x55e73bcf3630 2024-08-18T23:44:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.466+0000 7f41673d8740 10 client.15444 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.466+0000 7f41673d8740 10 client.15444 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-18T23:44:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.466+0000 7f41673d8740 20 client.15444 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.466+0000 7f41673d8740 1 --2- 172.21.15.78:0/3299256165 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55e73bd0f730 0x55e73bd7d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.466+0000 7f41673d8740 4 client.15444 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-18T23:44:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.466+0000 7f41673d8740 1 -- 172.21.15.78:0/3299256165 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "status"}) -- 0x55e73bc60190 con 0x55e73bd0f730 2024-08-18T23:44:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.466+0000 7f415dd9b640 1 --2- 172.21.15.78:0/3299256165 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55e73bd0f730 0x55e73bd7d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.466+0000 7f415dd9b640 1 --2- 172.21.15.78:0/3299256165 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55e73bd0f730 0x55e73bd7d640 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15ad0e92527b85bc server_cookie=dbdf851defbe46b2 in_seq=0 out_seq=0 2024-08-18T23:44:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.466+0000 7f4145ffb640 10 client.15444 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:44:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.467+0000 7f4145ffb640 1 -- 172.21.15.78:0/3299256165 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e73bc60190 con 0x55e73bd0f730 2024-08-18T23:44:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.467+0000 7f4145ffb640 10 client.15444 handle_command_reply: tid=0 2024-08-18T23:44:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.467+0000 7f41673d8740 1 client.15444 shutdown 2024-08-18T23:44:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.467+0000 7f41673d8740 1 -- 172.21.15.78:0/3299256165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 msgr2=0x55e73bd13bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.467+0000 7f41673d8740 1 --2- 172.21.15.78:0/3299256165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 0x55e73bd13bb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f41500976f0 tx=0x7f41500048a0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.467+0000 7f41673d8740 1 -- 172.21.15.78:0/3299256165 shutdown_connections 2024-08-18T23:44:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.467+0000 7f41673d8740 1 --2- 172.21.15.78:0/3299256165 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55e73bd0f730 0x55e73bd7d640 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.467+0000 7f41673d8740 1 --2- 172.21.15.78:0/3299256165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e73bcfe810 0x55e73bd14630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.467+0000 7f4145ffb640 0 client.15444 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:44:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.467+0000 7f41673d8740 1 --2- 172.21.15.78:0/3299256165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e73bcf3f50 0x55e73bd140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.467+0000 7f41673d8740 1 --2- 172.21.15.78:0/3299256165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e73bcf3630 0x55e73bd13bb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.467+0000 7f41673d8740 1 -- 172.21.15.78:0/3299256165 >> 172.21.15.78:0/3299256165 conn(0x55e73bcefbc0 msgr2=0x55e73bd097b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.467+0000 7f41673d8740 1 -- 172.21.15.78:0/3299256165 shutdown_connections 2024-08-18T23:44:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.467+0000 7f41673d8740 1 -- 172.21.15.78:0/3299256165 wait complete. 2024-08-18T23:44:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.467+0000 7f41673d8740 20 client.15444 trim_cache size 0 max 16384 2024-08-18T23:44:23.480 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:44:23.480 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "4f1128e6-5dba-11ef-bccf-c7b262605968", 2024-08-18T23:44:23.480 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 1, 2024-08-18T23:44:23.480 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24428, 2024-08-18T23:44:23.480 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-08-18T23:44:23.480 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-08-18T23:44:23.480 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-08-18T23:44:23.480 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 137.21862890200001, 2024-08-18T23:44:23.480 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 25, 2024-08-18T23:44:23.480 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 92, 2024-08-18T23:44:23.480 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 92, 2024-08-18T23:44:23.481 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 192.838628906 2024-08-18T23:44:23.481 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:44:23.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.467+0000 7f4164fda640 1 -- 172.21.15.78:0/1223299309 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f414c0a2c10 msgr2=0x7f414c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:23.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.467+0000 7f4164fda640 1 --2- 172.21.15.78:0/1223299309 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f414c0a2c10 0x7f414c0a50d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4158009a60 tx=0x7f415804c000 comp rx=0 tx=0).stop 2024-08-18T23:44:23.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.467+0000 7f4164fda640 1 -- 172.21.15.78:0/1223299309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41601007b0 msgr2=0x7f4160196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:23.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.467+0000 7f4164fda640 1 --2- 172.21.15.78:0/1223299309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41601007b0 0x7f4160196d90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f415000d770 tx=0x7f415000dc30 comp rx=0 tx=0).stop 2024-08-18T23:44:23.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.468+0000 7f4164fda640 1 -- 172.21.15.78:0/1223299309 shutdown_connections 2024-08-18T23:44:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.468+0000 7f4164fda640 1 --2- 172.21.15.78:0/1223299309 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f414c0a2c10 0x7f414c0a50d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.468+0000 7f4164fda640 1 --2- 172.21.15.78:0/1223299309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41601067b0 0x7f4160197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.468+0000 7f4164fda640 1 --2- 172.21.15.78:0/1223299309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4160101150 0x7f41601972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.468+0000 7f4164fda640 1 --2- 172.21.15.78:0/1223299309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41601007b0 0x7f4160196d90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.468+0000 7f4164fda640 1 -- 172.21.15.78:0/1223299309 >> 172.21.15.78:0/1223299309 conn(0x7f41600fc4d0 msgr2=0x7f41600fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.468+0000 7f4164fda640 1 -- 172.21.15.78:0/1223299309 shutdown_connections 2024-08-18T23:44:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:23.468+0000 7f4164fda640 1 -- 172.21.15.78:0/1223299309 wait complete. 2024-08-18T23:44:23.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:23 smithi123 ceph-mon[21808]: pgmap v352: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-18T23:44:23.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:23 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:44:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:23 smithi161 ceph-mon[22027]: pgmap v352: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-18T23:44:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:23 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:44:24.206 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-18T23:44:24.456 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:25.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.951+0000 7fc743016640 1 -- 172.21.15.78:0/1819549704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc73c100b50 msgr2=0x7fc73c100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:25.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.951+0000 7fc743016640 1 --2- 172.21.15.78:0/1819549704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc73c100b50 0x7fc73c100f30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc738009900 tx=0x7fc73802a480 comp rx=0 tx=0).stop 2024-08-18T23:44:25.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.951+0000 7fc743016640 1 -- 172.21.15.78:0/1819549704 shutdown_connections 2024-08-18T23:44:25.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.951+0000 7fc743016640 1 --2- 172.21.15.78:0/1819549704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc73c102c20 0x7fc73c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.951+0000 7fc743016640 1 --2- 172.21.15.78:0/1819549704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc73c1022a0 0x7fc73c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.951+0000 7fc743016640 1 --2- 172.21.15.78:0/1819549704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc73c100b50 0x7fc73c100f30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.951+0000 7fc743016640 1 -- 172.21.15.78:0/1819549704 >> 172.21.15.78:0/1819549704 conn(0x7fc73c0fc500 msgr2=0x7fc73c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:25.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.951+0000 7fc743016640 1 -- 172.21.15.78:0/1819549704 shutdown_connections 2024-08-18T23:44:25.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.951+0000 7fc743016640 1 -- 172.21.15.78:0/1819549704 wait complete. 2024-08-18T23:44:25.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.952+0000 7fc743016640 1 Processor -- start 2024-08-18T23:44:25.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.952+0000 7fc743016640 1 -- start start 2024-08-18T23:44:25.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.952+0000 7fc743016640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc73c100b50 0x7fc73c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:25.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.952+0000 7fc743016640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc73c1022a0 0x7fc73c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:25.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.952+0000 7fc743016640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc73c102c20 0x7fc73c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:25.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.952+0000 7fc743016640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c10fbd0 con 0x7fc73c1022a0 2024-08-18T23:44:25.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.952+0000 7fc743016640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c10fa50 con 0x7fc73c100b50 2024-08-18T23:44:25.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.952+0000 7fc743016640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c10fd50 con 0x7fc73c102c20 2024-08-18T23:44:25.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.952+0000 7fc733fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc73c1022a0 0x7fc73c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:25.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.952+0000 7fc733fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc73c1022a0 0x7fc73c19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59728/0 (socket says 172.21.15.78:59728) 2024-08-18T23:44:25.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.952+0000 7fc733fff640 1 -- 172.21.15.78:0/2210058743 learned_addr learned my addr 172.21.15.78:0/2210058743 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:25.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.952+0000 7fc740db2640 1 --2- 172.21.15.78:0/2210058743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc73c100b50 0x7fc73c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:25.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.952+0000 7fc7415b3640 1 --2- 172.21.15.78:0/2210058743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc73c102c20 0x7fc73c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:25.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.953+0000 7fc733fff640 1 -- 172.21.15.78:0/2210058743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc73c102c20 msgr2=0x7fc73c19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:25.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.953+0000 7fc733fff640 1 --2- 172.21.15.78:0/2210058743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc73c102c20 0x7fc73c19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.953+0000 7fc733fff640 1 -- 172.21.15.78:0/2210058743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc73c100b50 msgr2=0x7fc73c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:25.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.953+0000 7fc733fff640 1 --2- 172.21.15.78:0/2210058743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc73c100b50 0x7fc73c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.953+0000 7fc733fff640 1 -- 172.21.15.78:0/2210058743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73c19f470 con 0x7fc73c1022a0 2024-08-18T23:44:25.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.953+0000 7fc733fff640 1 --2- 172.21.15.78:0/2210058743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc73c1022a0 0x7fc73c19aa00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00b4a0 tx=0x7fc72c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:25.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.953+0000 7fc731ffb640 1 -- 172.21.15.78:0/2210058743 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c014070 con 0x7fc73c1022a0 2024-08-18T23:44:25.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.953+0000 7fc731ffb640 1 -- 172.21.15.78:0/2210058743 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72c010af0 con 0x7fc73c1022a0 2024-08-18T23:44:25.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.953+0000 7fc743016640 1 -- 172.21.15.78:0/2210058743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc73c19f760 con 0x7fc73c1022a0 2024-08-18T23:44:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.953+0000 7fc731ffb640 1 -- 172.21.15.78:0/2210058743 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c017d10 con 0x7fc73c1022a0 2024-08-18T23:44:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.953+0000 7fc743016640 1 -- 172.21.15.78:0/2210058743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc73c1a7040 con 0x7fc73c1022a0 2024-08-18T23:44:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.955+0000 7fc731ffb640 1 -- 172.21.15.78:0/2210058743 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fc72c017070 con 0x7fc73c1022a0 2024-08-18T23:44:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.956+0000 7fc745414740 1 Processor -- start 2024-08-18T23:44:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.956+0000 7fc745414740 1 -- start start 2024-08-18T23:44:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.956+0000 7fc731ffb640 1 --2- 172.21.15.78:0/2210058743 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc7240a2c00 0x7fc7240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.956+0000 7fc745414740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72630 0x55f6dba72a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.956+0000 7fc745414740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba72f50 0x55f6dba7d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.956+0000 7fc745414740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba7d810 0x55f6dba7fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.956+0000 7fc745414740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f6db9d7710 con 0x55f6dba72630 2024-08-18T23:44:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.956+0000 7fc745414740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f6db96f770 con 0x55f6dba7d810 2024-08-18T23:44:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.956+0000 7fc745414740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f6db9d36d0 con 0x55f6dba72f50 2024-08-18T23:44:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.956+0000 7fc731ffb640 1 -- 172.21.15.78:0/2210058743 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc72c0ce770 con 0x7fc73c1022a0 2024-08-18T23:44:25.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.956+0000 7fc7415b3640 1 --2- 172.21.15.78:0/2210058743 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc7240a2c00 0x7fc7240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.956+0000 7fc740db2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72630 0x55f6dba72a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.956+0000 7fc740db2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72630 0x55f6dba72a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59744/0 (socket says 172.21.15.78:59744) 2024-08-18T23:44:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.956+0000 7fc740db2640 1 -- 172.21.15.78:0/2321426808 learned_addr learned my addr 172.21.15.78:0/2321426808 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.956+0000 7fc7415b3640 1 --2- 172.21.15.78:0/2321426808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba7d810 0x55f6dba7fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.956+0000 7fc733fff640 1 --2- 172.21.15.78:0/2321426808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba72f50 0x55f6dba7d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.957+0000 7fc740db2640 1 -- 172.21.15.78:0/2321426808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba72f50 msgr2=0x55f6dba7d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.957+0000 7fc740db2640 1 --2- 172.21.15.78:0/2321426808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba72f50 0x55f6dba7d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.957+0000 7fc740db2640 1 -- 172.21.15.78:0/2321426808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba7d810 msgr2=0x55f6dba7fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:25.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.957+0000 7fc740db2640 1 --2- 172.21.15.78:0/2321426808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba7d810 0x55f6dba7fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.957+0000 7fc740db2640 1 -- 172.21.15.78:0/2321426808 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6db9763e0 con 0x55f6dba72630 2024-08-18T23:44:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.957+0000 7fc7415b3640 1 --2- 172.21.15.78:0/2210058743 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc7240a2c00 0x7fc7240a50c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc734013750 tx=0x7fc734013b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.957+0000 7fc740db2640 1 --2- 172.21.15.78:0/2321426808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72630 0x55f6dba72a10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc738009900 tx=0x7fc738004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5426b7c1a33a7f59 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.957+0000 7fc722ffd640 1 -- 172.21.15.78:0/2321426808 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738035030 con 0x55f6dba72630 2024-08-18T23:44:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.957+0000 7fc722ffd640 1 -- 172.21.15.78:0/2321426808 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc738040070 con 0x55f6dba72630 2024-08-18T23:44:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.957+0000 7fc722ffd640 1 -- 172.21.15.78:0/2321426808 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738031b80 con 0x55f6dba72630 2024-08-18T23:44:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.958+0000 7fc745414740 1 -- 172.21.15.78:0/2321426808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72630 msgr2=0x55f6dba72a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.958+0000 7fc745414740 1 --2- 172.21.15.78:0/2321426808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72630 0x55f6dba72a10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc738009900 tx=0x7fc738004200 comp rx=0 tx=0).stop 2024-08-18T23:44:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.958+0000 7fc745414740 1 -- 172.21.15.78:0/2321426808 shutdown_connections 2024-08-18T23:44:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.958+0000 7fc745414740 1 --2- 172.21.15.78:0/2321426808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba7d810 0x55f6dba7fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.958+0000 7fc745414740 1 --2- 172.21.15.78:0/2321426808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba72f50 0x55f6dba7d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.958+0000 7fc745414740 1 --2- 172.21.15.78:0/2321426808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72630 0x55f6dba72a10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.958+0000 7fc745414740 1 -- 172.21.15.78:0/2321426808 >> 172.21.15.78:0/2321426808 conn(0x55f6dba6ebc0 msgr2=0x55f6dba6efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.958+0000 7fc745414740 1 -- 172.21.15.78:0/2321426808 shutdown_connections 2024-08-18T23:44:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.958+0000 7fc745414740 1 -- 172.21.15.78:0/2321426808 wait complete. 2024-08-18T23:44:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.958+0000 7fc745414740 1 -- start start 2024-08-18T23:44:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.958+0000 7fc745414740 1 Processor -- start 2024-08-18T23:44:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc745414740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba72630 0x55f6dba7a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc745414740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 0x55f6dba7ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc745414740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba7d810 0x55f6dba7b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc745414740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f6dba803e0 con 0x55f6dba72f50 2024-08-18T23:44:25.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc745414740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f6db992970 con 0x55f6dba7d810 2024-08-18T23:44:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc745414740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f6dba80560 con 0x55f6dba72630 2024-08-18T23:44:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc745414740 10 client.0 _ll_register_callbacks cb 0x55f6db6e67b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc733fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 0x55f6dba7ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc733fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 0x55f6dba7ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59750/0 (socket says 172.21.15.78:59750) 2024-08-18T23:44:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc733fff640 1 -- 172.21.15.78:0/4175162567 learned_addr learned my addr 172.21.15.78:0/4175162567 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc740db2640 1 --2- 172.21.15.78:0/4175162567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba72630 0x55f6dba7a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc7415b3640 1 --2- 172.21.15.78:0/4175162567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba7d810 0x55f6dba7b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc733fff640 1 -- 172.21.15.78:0/4175162567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba72630 msgr2=0x55f6dba7a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc733fff640 1 --2- 172.21.15.78:0/4175162567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba72630 0x55f6dba7a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc733fff640 1 -- 172.21.15.78:0/4175162567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba7d810 msgr2=0x55f6dba7b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc733fff640 1 --2- 172.21.15.78:0/4175162567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba7d810 0x55f6dba7b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc733fff640 1 -- 172.21.15.78:0/4175162567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6db7492b0 con 0x55f6dba72f50 2024-08-18T23:44:25.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc7415b3640 1 --2- 172.21.15.78:0/4175162567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba7d810 0x55f6dba7b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:44:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc740db2640 1 --2- 172.21.15.78:0/4175162567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba72630 0x55f6dba7a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:44:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc733fff640 1 --2- 172.21.15.78:0/4175162567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 0x55f6dba7ac90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009420 tx=0x7fc72c09aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ceff19dc600216b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc71bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:44:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc71bfff640 1 -- 172.21.15.78:0/4175162567 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c01e070 con 0x55f6dba72f50 2024-08-18T23:44:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc71bfff640 1 -- 172.21.15.78:0/4175162567 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc72c017360 con 0x55f6dba72f50 2024-08-18T23:44:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.959+0000 7fc745414740 1 -- 172.21.15.78:0/4175162567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc738040070 con 0x55f6dba72f50 2024-08-18T23:44:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.960+0000 7fc71bfff640 1 -- 172.21.15.78:0/4175162567 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc72c010ca0 con 0x55f6dba72f50 2024-08-18T23:44:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.960+0000 7fc745414740 10 client.15456 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.960+0000 7fc745414740 1 -- 172.21.15.78:0/4175162567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f6dba8c040 con 0x55f6dba72f50 2024-08-18T23:44:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.960+0000 7fc71bfff640 1 -- 172.21.15.78:0/4175162567 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc738040070 con 0x55f6dba72f50 2024-08-18T23:44:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.960+0000 7fc71bfff640 1 -- 172.21.15.78:0/4175162567 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc72c09a0f0 con 0x55f6dba72f50 2024-08-18T23:44:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.960+0000 7fc745414740 10 client.15456 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.960+0000 7fc745414740 10 client.15456 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-18T23:44:25.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.960+0000 7fc745414740 20 client.15456 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:25.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.960+0000 7fc745414740 1 --2- 172.21.15.78:0/4175162567 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f6dba8e6a0 0x55f6dba8fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:25.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.960+0000 7fc745414740 4 client.15456 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:44:25.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.960+0000 7fc745414740 1 -- 172.21.15.78:0/4175162567 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6db9e4c80 con 0x55f6dba8e6a0 2024-08-18T23:44:25.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.960+0000 7fc740db2640 1 --2- 172.21.15.78:0/4175162567 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f6dba8e6a0 0x55f6dba8fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:25.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.961+0000 7fc71bfff640 10 client.15456 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:25.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.961+0000 7fc740db2640 1 --2- 172.21.15.78:0/4175162567 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f6dba8e6a0 0x55f6dba8fb10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=849f6983e9151e17 server_cookie=38f7634f8f552136 in_seq=0 out_seq=0 2024-08-18T23:44:25.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.962+0000 7fc71bfff640 1 -- 172.21.15.78:0/4175162567 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6db9e4c80 con 0x55f6dba8e6a0 2024-08-18T23:44:25.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.962+0000 7fc71bfff640 10 client.15456 handle_command_reply: tid=0 2024-08-18T23:44:25.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.962+0000 7fc745414740 1 client.15456 shutdown 2024-08-18T23:44:25.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.962+0000 7fc745414740 1 -- 172.21.15.78:0/4175162567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 msgr2=0x55f6dba7ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:25.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.962+0000 7fc745414740 1 --2- 172.21.15.78:0/4175162567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 0x55f6dba7ac90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009420 tx=0x7fc72c09aca0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.962+0000 7fc745414740 1 -- 172.21.15.78:0/4175162567 shutdown_connections 2024-08-18T23:44:25.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.962+0000 7fc745414740 1 --2- 172.21.15.78:0/4175162567 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f6dba8e6a0 0x55f6dba8fb10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.962+0000 7fc745414740 1 --2- 172.21.15.78:0/4175162567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba7d810 0x55f6dba7b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.962+0000 7fc745414740 1 --2- 172.21.15.78:0/4175162567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 0x55f6dba7ac90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.962+0000 7fc745414740 1 --2- 172.21.15.78:0/4175162567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba72630 0x55f6dba7a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.962+0000 7fc71bfff640 0 client.15456 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.962+0000 7fc745414740 1 -- 172.21.15.78:0/4175162567 >> 172.21.15.78:0/4175162567 conn(0x55f6dba6ebc0 msgr2=0x55f6dba6dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.962+0000 7fc745414740 1 -- 172.21.15.78:0/4175162567 shutdown_connections 2024-08-18T23:44:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.962+0000 7fc745414740 1 -- 172.21.15.78:0/4175162567 wait complete. 2024-08-18T23:44:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.962+0000 7fc745414740 20 client.15456 trim_cache size 0 max 16384 2024-08-18T23:44:25.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.979+0000 7fc745414740 1 Processor -- start 2024-08-18T23:44:25.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.979+0000 7fc745414740 1 -- start start 2024-08-18T23:44:25.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.979+0000 7fc745414740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba72630 0x55f6dba7a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:25.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.979+0000 7fc745414740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 0x55f6dba75090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:25.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.979+0000 7fc745414740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba7d810 0x55f6dba779b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:25.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.979+0000 7fc745414740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f6dba8de50 con 0x55f6dba72f50 2024-08-18T23:44:25.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.979+0000 7fc745414740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f6dba8e520 con 0x55f6dba7d810 2024-08-18T23:44:25.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.979+0000 7fc745414740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f6dba8ebf0 con 0x55f6dba72630 2024-08-18T23:44:25.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.980+0000 7fc733fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 0x55f6dba75090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:25.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.980+0000 7fc733fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 0x55f6dba75090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59762/0 (socket says 172.21.15.78:59762) 2024-08-18T23:44:25.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.980+0000 7fc733fff640 1 -- 172.21.15.78:0/736626545 learned_addr learned my addr 172.21.15.78:0/736626545 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:25.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.980+0000 7fc7415b3640 1 --2- 172.21.15.78:0/736626545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba7d810 0x55f6dba779b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:25.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.980+0000 7fc740db2640 1 --2- 172.21.15.78:0/736626545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba72630 0x55f6dba7a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:25.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.980+0000 7fc733fff640 1 -- 172.21.15.78:0/736626545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba72630 msgr2=0x55f6dba7a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:25.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.980+0000 7fc733fff640 1 --2- 172.21.15.78:0/736626545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba72630 0x55f6dba7a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.980+0000 7fc733fff640 1 -- 172.21.15.78:0/736626545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba7d810 msgr2=0x55f6dba779b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:25.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.980+0000 7fc733fff640 1 --2- 172.21.15.78:0/736626545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba7d810 0x55f6dba779b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.980+0000 7fc733fff640 1 -- 172.21.15.78:0/736626545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6dba6a130 con 0x55f6dba72f50 2024-08-18T23:44:25.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.980+0000 7fc7415b3640 1 --2- 172.21.15.78:0/736626545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba7d810 0x55f6dba779b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:25.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.980+0000 7fc733fff640 1 --2- 172.21.15.78:0/736626545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 0x55f6dba75090 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc72c097670 tx=0x7fc72c0991b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c3a1bb84507039 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:25.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.980+0000 7fc7227fc640 1 -- 172.21.15.78:0/736626545 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c01e070 con 0x55f6dba72f50 2024-08-18T23:44:25.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.980+0000 7fc7227fc640 1 -- 172.21.15.78:0/736626545 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72c09a0f0 con 0x55f6dba72f50 2024-08-18T23:44:25.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.980+0000 7fc7227fc640 1 -- 172.21.15.78:0/736626545 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c099b60 con 0x55f6dba72f50 2024-08-18T23:44:25.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.981+0000 7fc745414740 1 -- 172.21.15.78:0/736626545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 msgr2=0x55f6dba75090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:25.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.981+0000 7fc745414740 1 --2- 172.21.15.78:0/736626545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 0x55f6dba75090 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc72c097670 tx=0x7fc72c0991b0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.981+0000 7fc745414740 1 -- 172.21.15.78:0/736626545 shutdown_connections 2024-08-18T23:44:25.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.981+0000 7fc745414740 1 --2- 172.21.15.78:0/736626545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba7d810 0x55f6dba779b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.981+0000 7fc745414740 1 --2- 172.21.15.78:0/736626545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 0x55f6dba75090 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.981+0000 7fc745414740 1 --2- 172.21.15.78:0/736626545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba72630 0x55f6dba7a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.981+0000 7fc745414740 1 -- 172.21.15.78:0/736626545 >> 172.21.15.78:0/736626545 conn(0x55f6dba6ebc0 msgr2=0x55f6dba79020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:25.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.981+0000 7fc745414740 1 -- 172.21.15.78:0/736626545 shutdown_connections 2024-08-18T23:44:25.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.981+0000 7fc745414740 1 -- 172.21.15.78:0/736626545 wait complete. 2024-08-18T23:44:25.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.981+0000 7fc745414740 1 -- start start 2024-08-18T23:44:25.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.981+0000 7fc745414740 1 Processor -- start 2024-08-18T23:44:25.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc745414740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba72630 0x55f6dba982b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:25.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc745414740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 0x55f6dba915e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:25.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc745414740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba7d810 0x55f6dba91b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:25.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc745414740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f6dba8f990 con 0x55f6dba72f50 2024-08-18T23:44:25.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc745414740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f6dba8f2c0 con 0x55f6dba72630 2024-08-18T23:44:25.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc745414740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f6dba90060 con 0x55f6dba7d810 2024-08-18T23:44:25.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc745414740 10 client.0 _ll_register_callbacks cb 0x7fc72c096f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:25.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc733fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 0x55f6dba915e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:25.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc733fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 0x55f6dba915e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59776/0 (socket says 172.21.15.78:59776) 2024-08-18T23:44:25.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc733fff640 1 -- 172.21.15.78:0/1055161553 learned_addr learned my addr 172.21.15.78:0/1055161553 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:25.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc740db2640 1 --2- 172.21.15.78:0/1055161553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba72630 0x55f6dba982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:25.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc7415b3640 1 --2- 172.21.15.78:0/1055161553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba7d810 0x55f6dba91b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:25.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc733fff640 1 -- 172.21.15.78:0/1055161553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba7d810 msgr2=0x55f6dba91b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:25.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc733fff640 1 --2- 172.21.15.78:0/1055161553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba7d810 0x55f6dba91b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc733fff640 1 -- 172.21.15.78:0/1055161553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba72630 msgr2=0x55f6dba982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:25.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc733fff640 1 --2- 172.21.15.78:0/1055161553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba72630 0x55f6dba982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc733fff640 1 -- 172.21.15.78:0/1055161553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6dba92110 con 0x55f6dba72f50 2024-08-18T23:44:25.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc740db2640 1 --2- 172.21.15.78:0/1055161553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba72630 0x55f6dba982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc7415b3640 1 --2- 172.21.15.78:0/1055161553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba7d810 0x55f6dba91b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:44:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc733fff640 1 --2- 172.21.15.78:0/1055161553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 0x55f6dba915e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc72c098f50 tx=0x7fc72c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6569b5d3e89b79e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc71bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:44:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc71bfff640 1 -- 172.21.15.78:0/1055161553 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c01e070 con 0x55f6dba72f50 2024-08-18T23:44:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc71bfff640 1 -- 172.21.15.78:0/1055161553 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc72c0969f0 con 0x55f6dba72f50 2024-08-18T23:44:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc745414740 1 -- 172.21.15.78:0/1055161553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc72c09a0f0 con 0x55f6dba72f50 2024-08-18T23:44:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc71bfff640 1 -- 172.21.15.78:0/1055161553 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc72c096c30 con 0x55f6dba72f50 2024-08-18T23:44:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc745414740 10 client.15462 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.982+0000 7fc745414740 1 -- 172.21.15.78:0/1055161553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f6dba96bf0 con 0x55f6dba72f50 2024-08-18T23:44:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.983+0000 7fc71bfff640 1 -- 172.21.15.78:0/1055161553 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72c09a0f0 con 0x55f6dba72f50 2024-08-18T23:44:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.983+0000 7fc71bfff640 1 -- 172.21.15.78:0/1055161553 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc72c005bc0 con 0x55f6dba72f50 2024-08-18T23:44:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.983+0000 7fc745414740 10 client.15462 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.983+0000 7fc745414740 10 client.15462 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-18T23:44:25.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.983+0000 7fc745414740 20 client.15462 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:25.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.983+0000 7fc745414740 1 --2- 172.21.15.78:0/1055161553 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f6dbafd0f0 0x55f6dbafbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:25.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.983+0000 7fc745414740 4 client.15462 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-18T23:44:25.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.983+0000 7fc745414740 1 -- 172.21.15.78:0/1055161553 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "status"}) -- 0x55f6db9df190 con 0x55f6dbafd0f0 2024-08-18T23:44:25.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.983+0000 7fc740db2640 1 --2- 172.21.15.78:0/1055161553 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f6dbafd0f0 0x55f6dbafbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:25.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.984+0000 7fc71bfff640 10 client.15462 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:25.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.984+0000 7fc740db2640 1 --2- 172.21.15.78:0/1055161553 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f6dbafd0f0 0x55f6dbafbe00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9be058d6dc8d5b6 server_cookie=1a040e551882418e in_seq=0 out_seq=0 2024-08-18T23:44:25.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.984+0000 7fc71bfff640 1 -- 172.21.15.78:0/1055161553 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f6db9df190 con 0x55f6dbafd0f0 2024-08-18T23:44:25.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.984+0000 7fc71bfff640 10 client.15462 handle_command_reply: tid=0 2024-08-18T23:44:25.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.984+0000 7fc745414740 1 client.15462 shutdown 2024-08-18T23:44:25.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.984+0000 7fc745414740 1 -- 172.21.15.78:0/1055161553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 msgr2=0x55f6dba915e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:25.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.984+0000 7fc745414740 1 --2- 172.21.15.78:0/1055161553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 0x55f6dba915e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc72c098f50 tx=0x7fc72c0047d0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.026 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:44:25.026 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "4f1128e6-5dba-11ef-bccf-c7b262605968", 2024-08-18T23:44:25.026 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-08-18T23:44:25.026 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24452, 2024-08-18T23:44:25.026 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-08-18T23:44:25.026 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-08-18T23:44:25.026 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-08-18T23:44:25.026 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 171.84320292500001, 2024-08-18T23:44:25.026 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 25, 2024-08-18T23:44:25.026 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 92, 2024-08-18T23:44:25.026 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 92, 2024-08-18T23:44:25.027 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 182.50618295300001 2024-08-18T23:44:25.027 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:44:25.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.984+0000 7fc745414740 1 -- 172.21.15.78:0/1055161553 shutdown_connections 2024-08-18T23:44:25.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.984+0000 7fc745414740 1 --2- 172.21.15.78:0/1055161553 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f6dbafd0f0 0x55f6dbafbe00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.984+0000 7fc745414740 1 --2- 172.21.15.78:0/1055161553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6dba7d810 0x55f6dba91b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.984+0000 7fc745414740 1 --2- 172.21.15.78:0/1055161553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f6dba72f50 0x55f6dba915e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.984+0000 7fc71bfff640 0 client.15462 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:25.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.984+0000 7fc745414740 1 --2- 172.21.15.78:0/1055161553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6dba72630 0x55f6dba982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.984+0000 7fc745414740 1 -- 172.21.15.78:0/1055161553 >> 172.21.15.78:0/1055161553 conn(0x55f6dba6ebc0 msgr2=0x55f6dba78fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.984+0000 7fc745414740 1 -- 172.21.15.78:0/1055161553 shutdown_connections 2024-08-18T23:44:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.984+0000 7fc745414740 1 -- 172.21.15.78:0/1055161553 wait complete. 2024-08-18T23:44:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.984+0000 7fc745414740 20 client.15462 trim_cache size 0 max 16384 2024-08-18T23:44:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.985+0000 7fc743016640 1 -- 172.21.15.78:0/2210058743 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc7240a2c00 msgr2=0x7fc7240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.985+0000 7fc743016640 1 --2- 172.21.15.78:0/2210058743 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc7240a2c00 0x7fc7240a50c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc734013750 tx=0x7fc734013b50 comp rx=0 tx=0).stop 2024-08-18T23:44:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.985+0000 7fc743016640 1 -- 172.21.15.78:0/2210058743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc73c1022a0 msgr2=0x7fc73c19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.985+0000 7fc743016640 1 --2- 172.21.15.78:0/2210058743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc73c1022a0 0x7fc73c19aa00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00b4a0 tx=0x7fc72c00b960 comp rx=0 tx=0).stop 2024-08-18T23:44:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.985+0000 7fc743016640 1 -- 172.21.15.78:0/2210058743 shutdown_connections 2024-08-18T23:44:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.985+0000 7fc743016640 1 --2- 172.21.15.78:0/2210058743 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc7240a2c00 0x7fc7240a50c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.985+0000 7fc743016640 1 --2- 172.21.15.78:0/2210058743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc73c102c20 0x7fc73c19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.985+0000 7fc743016640 1 --2- 172.21.15.78:0/2210058743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc73c1022a0 0x7fc73c19aa00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.985+0000 7fc743016640 1 --2- 172.21.15.78:0/2210058743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc73c100b50 0x7fc73c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:25.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.985+0000 7fc743016640 1 -- 172.21.15.78:0/2210058743 >> 172.21.15.78:0/2210058743 conn(0x7fc73c0fc500 msgr2=0x7fc73c105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:25.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.985+0000 7fc743016640 1 -- 172.21.15.78:0/2210058743 shutdown_connections 2024-08-18T23:44:25.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:24.985+0000 7fc743016640 1 -- 172.21.15.78:0/2210058743 wait complete. 2024-08-18T23:44:25.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:25 smithi078 ceph-mon[18440]: pgmap v353: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-08-18T23:44:25.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:25 smithi123 ceph-mon[21808]: pgmap v353: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-08-18T23:44:25.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:25 smithi161 ceph-mon[22027]: pgmap v353: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-08-18T23:44:25.714 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-08-18T23:44:27.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:27 smithi078 ceph-mon[18440]: pgmap v354: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-08-18T23:44:27.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:27 smithi123 ceph-mon[21808]: pgmap v354: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-08-18T23:44:27.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:27 smithi161 ceph-mon[22027]: pgmap v354: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-08-18T23:44:29.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:29 smithi078 ceph-mon[18440]: pgmap v355: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-08-18T23:44:29.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:29 smithi123 ceph-mon[21808]: pgmap v355: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-08-18T23:44:29.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:29 smithi161 ceph-mon[22027]: pgmap v355: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 6 op/s 2024-08-18T23:44:31.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:31 smithi123 ceph-mon[21808]: pgmap v356: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-08-18T23:44:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:31 smithi161 ceph-mon[22027]: pgmap v356: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-08-18T23:44:31.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:31 smithi078 ceph-mon[18440]: pgmap v356: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-08-18T23:44:33.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:33 smithi123 ceph-mon[21808]: pgmap v357: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-08-18T23:44:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:33 smithi161 ceph-mon[22027]: pgmap v357: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-08-18T23:44:33.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:33 smithi078 ceph-mon[18440]: pgmap v357: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-08-18T23:44:35.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:35 smithi123 ceph-mon[21808]: pgmap v358: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-18T23:44:35.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:35 smithi161 ceph-mon[22027]: pgmap v358: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-18T23:44:35.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:35 smithi078 ceph-mon[18440]: pgmap v358: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-18T23:44:37.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:37 smithi123 ceph-mon[21808]: pgmap v359: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:44:37.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:37 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:44:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:37 smithi161 ceph-mon[22027]: pgmap v359: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:44:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:37 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:44:37.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:37 smithi078 ceph-mon[18440]: pgmap v359: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:44:37.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:37 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:44:38.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:38 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:44:38.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:38 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:44:38.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:38 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:44:38.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:38 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:44:38.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:38 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:44:38.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:38 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:44:38.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:44:38.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:44:38.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:44:39.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:39 smithi123 ceph-mon[21808]: pgmap v360: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:44:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:39 smithi161 ceph-mon[22027]: pgmap v360: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:44:39.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:39 smithi078 ceph-mon[18440]: pgmap v360: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:44:40.715 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 34.7 2024-08-18T23:44:40.716 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:44:40.985 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:41.397 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:41 smithi078 ceph-mon[18440]: pgmap v361: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:44:41.444 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:44:41.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.483+0000 7fcbc3acc640 1 -- 172.21.15.78:0/3599786545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbbc1022a0 msgr2=0x7fcbbc1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:41.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.483+0000 7fcbc3acc640 1 --2- 172.21.15.78:0/3599786545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbbc1022a0 0x7fcbbc1026e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009900 tx=0x7fcbac02a480 comp rx=0 tx=0).stop 2024-08-18T23:44:41.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.483+0000 7fcbc3acc640 1 -- 172.21.15.78:0/3599786545 shutdown_connections 2024-08-18T23:44:41.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.483+0000 7fcbc3acc640 1 --2- 172.21.15.78:0/3599786545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbbc102c20 0x7fcbbc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:41.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.483+0000 7fcbc3acc640 1 --2- 172.21.15.78:0/3599786545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbbc1022a0 0x7fcbbc1026e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:41.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.483+0000 7fcbc3acc640 1 --2- 172.21.15.78:0/3599786545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbbc100b50 0x7fcbbc100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:41.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.483+0000 7fcbc3acc640 1 -- 172.21.15.78:0/3599786545 >> 172.21.15.78:0/3599786545 conn(0x7fcbbc0fc500 msgr2=0x7fcbbc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:41.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.483+0000 7fcbc3acc640 1 -- 172.21.15.78:0/3599786545 shutdown_connections 2024-08-18T23:44:41.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.483+0000 7fcbc3acc640 1 -- 172.21.15.78:0/3599786545 wait complete. 2024-08-18T23:44:41.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.484+0000 7fcbc3acc640 1 Processor -- start 2024-08-18T23:44:41.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.484+0000 7fcbc3acc640 1 -- start start 2024-08-18T23:44:41.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.484+0000 7fcbc3acc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbbc100b50 0x7fcbbc198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:41.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.484+0000 7fcbc3acc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbbc1022a0 0x7fcbbc198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:41.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.484+0000 7fcbc3acc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbbc102c20 0x7fcbbc19cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:41.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.484+0000 7fcbc3acc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc110c70 con 0x7fcbbc1022a0 2024-08-18T23:44:41.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.484+0000 7fcbc3acc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc110af0 con 0x7fcbbc100b50 2024-08-18T23:44:41.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.484+0000 7fcbc3acc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc110df0 con 0x7fcbbc102c20 2024-08-18T23:44:41.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.484+0000 7fcbc1067640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbbc1022a0 0x7fcbbc198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:41.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.484+0000 7fcbc1067640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbbc1022a0 0x7fcbbc198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57974/0 (socket says 172.21.15.78:57974) 2024-08-18T23:44:41.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.484+0000 7fcbc1067640 1 -- 172.21.15.78:0/1887279983 learned_addr learned my addr 172.21.15.78:0/1887279983 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:41.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.484+0000 7fcbc2069640 1 --2- 172.21.15.78:0/1887279983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbbc102c20 0x7fcbbc19cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:41.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.485+0000 7fcbc1868640 1 --2- 172.21.15.78:0/1887279983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbbc100b50 0x7fcbbc198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:41.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.485+0000 7fcbc1067640 1 -- 172.21.15.78:0/1887279983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbbc102c20 msgr2=0x7fcbbc19cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:41.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.485+0000 7fcbc1067640 1 --2- 172.21.15.78:0/1887279983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbbc102c20 0x7fcbbc19cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:41.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.485+0000 7fcbc1067640 1 -- 172.21.15.78:0/1887279983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbbc100b50 msgr2=0x7fcbbc198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:41.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.485+0000 7fcbc1067640 1 --2- 172.21.15.78:0/1887279983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbbc100b50 0x7fcbbc198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:41.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.485+0000 7fcbc1067640 1 -- 172.21.15.78:0/1887279983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbbc19d270 con 0x7fcbbc1022a0 2024-08-18T23:44:41.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.485+0000 7fcbc1868640 1 --2- 172.21.15.78:0/1887279983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbbc100b50 0x7fcbbc198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:41.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.485+0000 7fcbc1067640 1 --2- 172.21.15.78:0/1887279983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbbc1022a0 0x7fcbbc198a40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcbac006010 tx=0x7fcbac003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:41.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.485+0000 7fcbb2ffd640 1 -- 172.21.15.78:0/1887279983 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac03f070 con 0x7fcbbc1022a0 2024-08-18T23:44:41.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.485+0000 7fcbb2ffd640 1 -- 172.21.15.78:0/1887279983 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbac031c30 con 0x7fcbbc1022a0 2024-08-18T23:44:41.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.485+0000 7fcbc3acc640 1 -- 172.21.15.78:0/1887279983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbbc19d500 con 0x7fcbbc1022a0 2024-08-18T23:44:41.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.485+0000 7fcbc3acc640 1 -- 172.21.15.78:0/1887279983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbbc1a4db0 con 0x7fcbbc1022a0 2024-08-18T23:44:41.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.485+0000 7fcbb2ffd640 1 -- 172.21.15.78:0/1887279983 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac035030 con 0x7fcbbc1022a0 2024-08-18T23:44:41.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.487+0000 7fcbc3acc640 1 -- 172.21.15.78:0/1887279983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbbc100f30 con 0x7fcbbc1022a0 2024-08-18T23:44:41.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.488+0000 7fcbb2ffd640 1 -- 172.21.15.78:0/1887279983 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fcbac042070 con 0x7fcbbc1022a0 2024-08-18T23:44:41.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.488+0000 7fcbb2ffd640 1 --2- 172.21.15.78:0/1887279983 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcba80a2c90 0x7fcba80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:41.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.488+0000 7fcbc1868640 1 --2- 172.21.15.78:0/1887279983 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcba80a2c90 0x7fcba80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:41.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.488+0000 7fcbb2ffd640 1 -- 172.21.15.78:0/1887279983 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fcbac0efa10 con 0x7fcbbc1022a0 2024-08-18T23:44:41.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.489+0000 7fcbc1868640 1 --2- 172.21.15.78:0/1887279983 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcba80a2c90 0x7fcba80a5150 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8005d00 tx=0x7fcbb8005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:41.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.493+0000 7fcbb2ffd640 1 -- 172.21.15.78:0/1887279983 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fcbac031de0 con 0x7fcbbc1022a0 2024-08-18T23:44:41.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:41 smithi123 ceph-mon[21808]: pgmap v361: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:44:41.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:41 smithi161 ceph-mon[22027]: pgmap v361: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-18T23:44:41.735 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:41.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.754+0000 7fcbc3acc640 1 -- 172.21.15.78:0/1887279983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbbc1a4fe0 con 0x7fcbbc1022a0 2024-08-18T23:44:41.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.755+0000 7fcbb2ffd640 1 -- 172.21.15.78:0/1887279983 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7fcbac0bc480 con 0x7fcbbc1022a0 2024-08-18T23:44:41.756 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:44:41.756 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"btime":"2024-08-18T23:43:57:245845+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:43:57.114120+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24428,24427]},"id":1}]} 2024-08-18T23:44:41.756 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 25 2024-08-18T23:44:41.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.757+0000 7fcbc3acc640 1 -- 172.21.15.78:0/1887279983 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcba80a2c90 msgr2=0x7fcba80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:41.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.757+0000 7fcbc3acc640 1 --2- 172.21.15.78:0/1887279983 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcba80a2c90 0x7fcba80a5150 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8005d00 tx=0x7fcbb8005c70 comp rx=0 tx=0).stop 2024-08-18T23:44:41.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.757+0000 7fcbc3acc640 1 -- 172.21.15.78:0/1887279983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbbc1022a0 msgr2=0x7fcbbc198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:41.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.757+0000 7fcbc3acc640 1 --2- 172.21.15.78:0/1887279983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbbc1022a0 0x7fcbbc198a40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcbac006010 tx=0x7fcbac003c10 comp rx=0 tx=0).stop 2024-08-18T23:44:41.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.758+0000 7fcbc3acc640 1 -- 172.21.15.78:0/1887279983 shutdown_connections 2024-08-18T23:44:41.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.758+0000 7fcbc3acc640 1 --2- 172.21.15.78:0/1887279983 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcba80a2c90 0x7fcba80a5150 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:41.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.758+0000 7fcbc3acc640 1 --2- 172.21.15.78:0/1887279983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbbc102c20 0x7fcbbc19cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:41.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.758+0000 7fcbc3acc640 1 --2- 172.21.15.78:0/1887279983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcbbc1022a0 0x7fcbbc198a40 secure :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcbac006010 tx=0x7fcbac003c10 comp rx=0 tx=0).stop 2024-08-18T23:44:41.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.758+0000 7fcbc3acc640 1 --2- 172.21.15.78:0/1887279983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbbc100b50 0x7fcbbc198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:41.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.758+0000 7fcbc3acc640 1 -- 172.21.15.78:0/1887279983 >> 172.21.15.78:0/1887279983 conn(0x7fcbbc0fc500 msgr2=0x7fcbbc103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:41.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.758+0000 7fcbc3acc640 1 -- 172.21.15.78:0/1887279983 shutdown_connections 2024-08-18T23:44:41.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:41.758+0000 7fcbc3acc640 1 -- 172.21.15.78:0/1887279983 wait complete. 2024-08-18T23:44:42.393 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:42 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1887279983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:42.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.476+0000 7f1046518640 1 -- 172.21.15.78:0/3001832472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10401018b0 msgr2=0x7f1040101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:42.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.476+0000 7f1046518640 1 --2- 172.21.15.78:0/3001832472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10401018b0 0x7f1040101d10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1028009930 tx=0x7f102802a420 comp rx=0 tx=0).stop 2024-08-18T23:44:42.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.476+0000 7f1046518640 1 -- 172.21.15.78:0/3001832472 shutdown_connections 2024-08-18T23:44:42.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.476+0000 7f1046518640 1 --2- 172.21.15.78:0/3001832472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1040102250 0x7f104010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.476+0000 7f1046518640 1 --2- 172.21.15.78:0/3001832472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10401018b0 0x7f1040101d10 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.476+0000 7f1046518640 1 --2- 172.21.15.78:0/3001832472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10401078b0 0x7f1040107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.476+0000 7f1046518640 1 -- 172.21.15.78:0/3001832472 >> 172.21.15.78:0/3001832472 conn(0x7f10400fd5f0 msgr2=0x7f10400ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:42.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.476+0000 7f1046518640 1 -- 172.21.15.78:0/3001832472 shutdown_connections 2024-08-18T23:44:42.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.476+0000 7f1046518640 1 -- 172.21.15.78:0/3001832472 wait complete. 2024-08-18T23:44:42.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.477+0000 7f1046518640 1 Processor -- start 2024-08-18T23:44:42.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.477+0000 7f1046518640 1 -- start start 2024-08-18T23:44:42.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.477+0000 7f1046518640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10401018b0 0x7f10401981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:42.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.477+0000 7f1046518640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1040102250 0x7f1040198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:42.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.477+0000 7f1046518640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10401078b0 0x7f104019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:42.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.477+0000 7f1046518640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1040111cb0 con 0x7f10401018b0 2024-08-18T23:44:42.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.477+0000 7f1046518640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1040111b30 con 0x7f1040102250 2024-08-18T23:44:42.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.477+0000 7f1046518640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1040111e30 con 0x7f10401078b0 2024-08-18T23:44:42.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.477+0000 7f103ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10401018b0 0x7f10401981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:42.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.477+0000 7f103ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10401018b0 0x7f10401981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57992/0 (socket says 172.21.15.78:57992) 2024-08-18T23:44:42.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.477+0000 7f103ffff640 1 -- 172.21.15.78:0/4062686462 learned_addr learned my addr 172.21.15.78:0/4062686462 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:42.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.477+0000 7f1037fff640 1 --2- 172.21.15.78:0/4062686462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1040102250 0x7f1040198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:42.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.478+0000 7f1044ab5640 1 --2- 172.21.15.78:0/4062686462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10401078b0 0x7f104019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:42.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.478+0000 7f103ffff640 1 -- 172.21.15.78:0/4062686462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10401078b0 msgr2=0x7f104019cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:42.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.478+0000 7f103ffff640 1 --2- 172.21.15.78:0/4062686462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10401078b0 0x7f104019cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.478+0000 7f103ffff640 1 -- 172.21.15.78:0/4062686462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1040102250 msgr2=0x7f1040198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.478+0000 7f103ffff640 1 --2- 172.21.15.78:0/4062686462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1040102250 0x7f1040198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.478+0000 7f103ffff640 1 -- 172.21.15.78:0/4062686462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104019d1a0 con 0x7f10401018b0 2024-08-18T23:44:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.478+0000 7f103ffff640 1 --2- 172.21.15.78:0/4062686462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10401018b0 0x7f10401981f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f103000c970 tx=0x7f103000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.478+0000 7f103dffb640 1 -- 172.21.15.78:0/4062686462 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030007bb0 con 0x7f10401018b0 2024-08-18T23:44:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.478+0000 7f103dffb640 1 -- 172.21.15.78:0/4062686462 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1030007d50 con 0x7f10401018b0 2024-08-18T23:44:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.478+0000 7f1046518640 1 -- 172.21.15.78:0/4062686462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104019d490 con 0x7f10401018b0 2024-08-18T23:44:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.478+0000 7f103dffb640 1 -- 172.21.15.78:0/4062686462 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103000f450 con 0x7f10401018b0 2024-08-18T23:44:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.478+0000 7f1046518640 1 -- 172.21.15.78:0/4062686462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10401a4d70 con 0x7f10401018b0 2024-08-18T23:44:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.481+0000 7f103dffb640 1 -- 172.21.15.78:0/4062686462 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f1030005020 con 0x7f10401018b0 2024-08-18T23:44:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.481+0000 7f1048916740 1 Processor -- start 2024-08-18T23:44:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.481+0000 7f1048916740 1 -- start start 2024-08-18T23:44:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f1048916740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 0x55a19e7325f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f1048916740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 0x55a19e73ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f1048916740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e73d3f0 0x55a19e73f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f1048916740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a19e697980 con 0x55a19e732210 2024-08-18T23:44:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f1048916740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a19e62fa70 con 0x55a19e73d3f0 2024-08-18T23:44:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f1048916740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a19e693940 con 0x55a19e732b30 2024-08-18T23:44:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f1044ab5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 0x55a19e7325f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f1044ab5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 0x55a19e7325f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57998/0 (socket says 172.21.15.78:57998) 2024-08-18T23:44:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f1044ab5640 1 -- 172.21.15.78:0/2948557577 learned_addr learned my addr 172.21.15.78:0/2948557577 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f1037fff640 1 --2- 172.21.15.78:0/2948557577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e73d3f0 0x55a19e73f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f103dffb640 1 --2- 172.21.15.78:0/4062686462 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f102c0a2c90 0x7f102c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f1044ab5640 1 -- 172.21.15.78:0/2948557577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 msgr2=0x55a19e73ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f103ffff640 1 --2- 172.21.15.78:0/2948557577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 0x55a19e73ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f1044ab5640 1 --2- 172.21.15.78:0/2948557577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 0x55a19e73ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f1044ab5640 1 -- 172.21.15.78:0/2948557577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e73d3f0 msgr2=0x55a19e73f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f103dffb640 1 -- 172.21.15.78:0/4062686462 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f10300cf250 con 0x7f10401018b0 2024-08-18T23:44:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f1044ab5640 1 --2- 172.21.15.78:0/2948557577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e73d3f0 0x55a19e73f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f1044ab5640 1 -- 172.21.15.78:0/2948557577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a19e6926e0 con 0x55a19e732210 2024-08-18T23:44:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f1037fff640 1 --2- 172.21.15.78:0/2948557577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e73d3f0 0x55a19e73f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:44:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f103ffff640 1 --2- 172.21.15.78:0/2948557577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 0x55a19e73ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:44:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.482+0000 7f1044ab5640 1 --2- 172.21.15.78:0/4062686462 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f102c0a2c90 0x7f102c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.483+0000 7f1044ab5640 1 --2- 172.21.15.78:0/2948557577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 0x55a19e7325f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1038008f40 tx=0x7f103800e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4c9b5b97a76770 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.483+0000 7f10367fc640 1 -- 172.21.15.78:0/2948557577 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038022070 con 0x55a19e732210 2024-08-18T23:44:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.483+0000 7f10367fc640 1 -- 172.21.15.78:0/2948557577 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1038011b70 con 0x55a19e732210 2024-08-18T23:44:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.483+0000 7f1044ab5640 1 --2- 172.21.15.78:0/4062686462 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f102c0a2c90 0x7f102c0a5150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f103801f000 tx=0x7f1038020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.483+0000 7f10367fc640 1 -- 172.21.15.78:0/2948557577 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103801a400 con 0x55a19e732210 2024-08-18T23:44:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.484+0000 7f1048916740 1 -- 172.21.15.78:0/2948557577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 msgr2=0x55a19e7325f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.484+0000 7f1048916740 1 --2- 172.21.15.78:0/2948557577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 0x55a19e7325f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1038008f40 tx=0x7f103800e7e0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.484+0000 7f1048916740 1 -- 172.21.15.78:0/2948557577 shutdown_connections 2024-08-18T23:44:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.484+0000 7f1048916740 1 --2- 172.21.15.78:0/2948557577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e73d3f0 0x55a19e73f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.484+0000 7f1048916740 1 --2- 172.21.15.78:0/2948557577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 0x55a19e73ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.484+0000 7f1048916740 1 --2- 172.21.15.78:0/2948557577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 0x55a19e7325f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.484+0000 7f1048916740 1 -- 172.21.15.78:0/2948557577 >> 172.21.15.78:0/2948557577 conn(0x55a19e72e7a0 msgr2=0x55a19e72eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.484+0000 7f1048916740 1 -- 172.21.15.78:0/2948557577 shutdown_connections 2024-08-18T23:44:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.484+0000 7f1048916740 1 -- 172.21.15.78:0/2948557577 wait complete. 2024-08-18T23:44:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.484+0000 7f1048916740 1 -- start start 2024-08-18T23:44:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.484+0000 7f1048916740 1 Processor -- start 2024-08-18T23:44:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f1048916740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 0x55a19e73a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f1048916740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 0x55a19e73a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f1048916740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e73d3f0 0x55a19e73ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:42.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f1048916740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a19e740010 con 0x55a19e732210 2024-08-18T23:44:42.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f1048916740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a19e652aa0 con 0x55a19e73d3f0 2024-08-18T23:44:42.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f1048916740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a19e740190 con 0x55a19e732b30 2024-08-18T23:44:42.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f1048916740 10 client.0 _ll_register_callbacks cb 0x55a19e641e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:42.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f103ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 0x55a19e73a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:42.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f103ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 0x55a19e73a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58008/0 (socket says 172.21.15.78:58008) 2024-08-18T23:44:42.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f103ffff640 1 -- 172.21.15.78:0/2055722126 learned_addr learned my addr 172.21.15.78:0/2055722126 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:42.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f1037fff640 1 --2- 172.21.15.78:0/2055722126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 0x55a19e73a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:42.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f1044ab5640 1 --2- 172.21.15.78:0/2055722126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e73d3f0 0x55a19e73ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:42.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f103ffff640 1 -- 172.21.15.78:0/2055722126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 msgr2=0x55a19e73a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:42.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f103ffff640 1 --2- 172.21.15.78:0/2055722126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 0x55a19e73a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f103ffff640 1 -- 172.21.15.78:0/2055722126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e73d3f0 msgr2=0x55a19e73ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:42.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f103ffff640 1 --2- 172.21.15.78:0/2055722126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e73d3f0 0x55a19e73ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f103ffff640 1 -- 172.21.15.78:0/2055722126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a19e636650 con 0x55a19e732210 2024-08-18T23:44:42.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f1044ab5640 1 --2- 172.21.15.78:0/2055722126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e73d3f0 0x55a19e73ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:42.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f103ffff640 1 --2- 172.21.15.78:0/2055722126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 0x55a19e73a2f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1030007590 tx=0x7f103001c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da83005c3316438 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:42.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f101f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:44:42.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f101f7fe640 1 -- 172.21.15.78:0/2055722126 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103009bcc0 con 0x55a19e732210 2024-08-18T23:44:42.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.485+0000 7f101f7fe640 1 -- 172.21.15.78:0/2055722126 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f1030005310 con 0x55a19e732210 2024-08-18T23:44:42.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.486+0000 7f1048916740 1 -- 172.21.15.78:0/2055722126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1038011b70 con 0x55a19e732210 2024-08-18T23:44:42.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.486+0000 7f101f7fe640 1 -- 172.21.15.78:0/2055722126 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1030014210 con 0x55a19e732210 2024-08-18T23:44:42.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.486+0000 7f1048916740 10 client.15480 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:42.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.486+0000 7f1048916740 1 -- 172.21.15.78:0/2055722126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a19e4093c0 con 0x55a19e732210 2024-08-18T23:44:42.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.486+0000 7f101f7fe640 1 -- 172.21.15.78:0/2055722126 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1038011b70 con 0x55a19e732210 2024-08-18T23:44:42.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.486+0000 7f101f7fe640 1 -- 172.21.15.78:0/2055722126 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f10300a3da0 con 0x55a19e732210 2024-08-18T23:44:42.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.486+0000 7f1048916740 10 client.15480 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:42.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.486+0000 7f1048916740 10 client.15480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:44:42.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.486+0000 7f1048916740 20 client.15480 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:42.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.486+0000 7f1048916740 1 --2- 172.21.15.78:0/2055722126 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a19e74cdf0 0x55a19e74f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:42.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.486+0000 7f1048916740 4 client.15480 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:44:42.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.486+0000 7f1048916740 1 -- 172.21.15.78:0/2055722126 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a19e6a4ef0 con 0x55a19e74cdf0 2024-08-18T23:44:42.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.486+0000 7f1037fff640 1 --2- 172.21.15.78:0/2055722126 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a19e74cdf0 0x55a19e74f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:42.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.487+0000 7f101f7fe640 10 client.15480 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:42.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.487+0000 7f1037fff640 1 --2- 172.21.15.78:0/2055722126 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a19e74cdf0 0x55a19e74f240 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56156b3a58114d4f server_cookie=1a344e93d754cf50 in_seq=0 out_seq=0 2024-08-18T23:44:42.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.488+0000 7f101f7fe640 1 -- 172.21.15.78:0/2055722126 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a19e6a4ef0 con 0x55a19e74cdf0 2024-08-18T23:44:42.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.488+0000 7f101f7fe640 10 client.15480 handle_command_reply: tid=0 2024-08-18T23:44:42.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.488+0000 7f1048916740 1 client.15480 shutdown 2024-08-18T23:44:42.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.488+0000 7f1048916740 1 -- 172.21.15.78:0/2055722126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 msgr2=0x55a19e73a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:42.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.488+0000 7f1048916740 1 --2- 172.21.15.78:0/2055722126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 0x55a19e73a2f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1030007590 tx=0x7f103001c7c0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.488+0000 7f1048916740 1 -- 172.21.15.78:0/2055722126 shutdown_connections 2024-08-18T23:44:42.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.488+0000 7f1048916740 1 --2- 172.21.15.78:0/2055722126 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a19e74cdf0 0x55a19e74f240 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.488+0000 7f1048916740 1 --2- 172.21.15.78:0/2055722126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e73d3f0 0x55a19e73ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.488+0000 7f101f7fe640 0 client.15480 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:42.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.488+0000 7f1048916740 1 --2- 172.21.15.78:0/2055722126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 0x55a19e73a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.488+0000 7f1048916740 1 --2- 172.21.15.78:0/2055722126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 0x55a19e73a2f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.488+0000 7f1048916740 1 -- 172.21.15.78:0/2055722126 >> 172.21.15.78:0/2055722126 conn(0x55a19e72e7a0 msgr2=0x55a19e734480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:42.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.488+0000 7f1048916740 1 -- 172.21.15.78:0/2055722126 shutdown_connections 2024-08-18T23:44:42.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.488+0000 7f1048916740 1 -- 172.21.15.78:0/2055722126 wait complete. 2024-08-18T23:44:42.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.488+0000 7f1048916740 20 client.15480 trim_cache size 0 max 16384 2024-08-18T23:44:42.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.521+0000 7f1048916740 1 Processor -- start 2024-08-18T23:44:42.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.521+0000 7f1048916740 1 -- start start 2024-08-18T23:44:42.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.521+0000 7f1048916740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 0x55a19e734c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:42.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.521+0000 7f1048916740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 0x55a19e7355d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:42.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.521+0000 7f1048916740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e73d3f0 0x55a19e73c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:42.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.521+0000 7f1048916740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10401b1350 con 0x55a19e732210 2024-08-18T23:44:42.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.521+0000 7f1048916740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10401aec20 con 0x55a19e73d3f0 2024-08-18T23:44:42.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.521+0000 7f1048916740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f10401aca70 con 0x55a19e732b30 2024-08-18T23:44:42.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.521+0000 7f103ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 0x55a19e734c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:42.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.521+0000 7f103ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 0x55a19e734c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58016/0 (socket says 172.21.15.78:58016) 2024-08-18T23:44:42.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.521+0000 7f103ffff640 1 -- 172.21.15.78:0/54994609 learned_addr learned my addr 172.21.15.78:0/54994609 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:42.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.521+0000 7f1037fff640 1 --2- 172.21.15.78:0/54994609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 0x55a19e7355d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:42.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.521+0000 7f103ffff640 1 -- 172.21.15.78:0/54994609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 msgr2=0x55a19e7355d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:42.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.521+0000 7f1044ab5640 1 --2- 172.21.15.78:0/54994609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e73d3f0 0x55a19e73c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:42.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.521+0000 7f103ffff640 1 --2- 172.21.15.78:0/54994609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 0x55a19e7355d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.521+0000 7f103ffff640 1 -- 172.21.15.78:0/54994609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e73d3f0 msgr2=0x55a19e73c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:42.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.522+0000 7f103ffff640 1 --2- 172.21.15.78:0/54994609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e73d3f0 0x55a19e73c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.522+0000 7f103ffff640 1 -- 172.21.15.78:0/54994609 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a19e738840 con 0x55a19e732210 2024-08-18T23:44:42.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.522+0000 7f103ffff640 1 --2- 172.21.15.78:0/54994609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 0x55a19e734c70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f103000c940 tx=0x7f1030004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4714a7b41dfd28b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:42.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.522+0000 7f1035ffb640 1 -- 172.21.15.78:0/54994609 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103009b5c0 con 0x55a19e732210 2024-08-18T23:44:42.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.522+0000 7f1035ffb640 1 -- 172.21.15.78:0/54994609 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10300a4230 con 0x55a19e732210 2024-08-18T23:44:42.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.522+0000 7f1035ffb640 1 -- 172.21.15.78:0/54994609 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10300054d0 con 0x55a19e732210 2024-08-18T23:44:42.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.523+0000 7f1048916740 1 -- 172.21.15.78:0/54994609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 msgr2=0x55a19e734c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:42.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.523+0000 7f1048916740 1 --2- 172.21.15.78:0/54994609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 0x55a19e734c70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f103000c940 tx=0x7f1030004a20 comp rx=0 tx=0).stop 2024-08-18T23:44:42.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.523+0000 7f1048916740 1 -- 172.21.15.78:0/54994609 shutdown_connections 2024-08-18T23:44:42.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.523+0000 7f1048916740 1 --2- 172.21.15.78:0/54994609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e73d3f0 0x55a19e73c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.523+0000 7f1048916740 1 --2- 172.21.15.78:0/54994609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 0x55a19e7355d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.523+0000 7f1048916740 1 --2- 172.21.15.78:0/54994609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e732210 0x55a19e734c70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.523+0000 7f1048916740 1 -- 172.21.15.78:0/54994609 >> 172.21.15.78:0/54994609 conn(0x55a19e72e7a0 msgr2=0x55a19e73ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.523+0000 7f1048916740 1 -- 172.21.15.78:0/54994609 shutdown_connections 2024-08-18T23:44:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.523+0000 7f1048916740 1 -- 172.21.15.78:0/54994609 wait complete. 2024-08-18T23:44:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.523+0000 7f1048916740 1 -- start start 2024-08-18T23:44:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.523+0000 7f1048916740 1 Processor -- start 2024-08-18T23:44:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.524+0000 7f1048916740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e732210 0x55a19e7b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.524+0000 7f1048916740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 0x55a19e7453e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.524+0000 7f1048916740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e73d3f0 0x55a19e745920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.524+0000 7f1048916740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10401ad0e0 con 0x55a19e73d3f0 2024-08-18T23:44:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.524+0000 7f1048916740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10401ae550 con 0x55a19e732210 2024-08-18T23:44:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.524+0000 7f1048916740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f10401ad7b0 con 0x55a19e732b30 2024-08-18T23:44:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.524+0000 7f1048916740 10 client.0 _ll_register_callbacks cb 0x7f1030005690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.524+0000 7f1044ab5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e73d3f0 0x55a19e745920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.524+0000 7f1037fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 0x55a19e7453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.524+0000 7f1044ab5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e73d3f0 0x55a19e745920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58032/0 (socket says 172.21.15.78:58032) 2024-08-18T23:44:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.524+0000 7f1044ab5640 1 -- 172.21.15.78:0/3739250561 learned_addr learned my addr 172.21.15.78:0/3739250561 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.524+0000 7f103ffff640 1 --2- 172.21.15.78:0/3739250561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e732210 0x55a19e7b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.524+0000 7f1044ab5640 1 -- 172.21.15.78:0/3739250561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 msgr2=0x55a19e7453e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.524+0000 7f1044ab5640 1 --2- 172.21.15.78:0/3739250561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 0x55a19e7453e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.524+0000 7f1044ab5640 1 -- 172.21.15.78:0/3739250561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e732210 msgr2=0x55a19e7b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.524+0000 7f1044ab5640 1 --2- 172.21.15.78:0/3739250561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e732210 0x55a19e7b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.524+0000 7f1044ab5640 1 -- 172.21.15.78:0/3739250561 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a19e7b0710 con 0x55a19e73d3f0 2024-08-18T23:44:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.524+0000 7f1044ab5640 1 --2- 172.21.15.78:0/3739250561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e73d3f0 0x55a19e745920 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1038008f40 tx=0x7f103801c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42535fd15622f00b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.524+0000 7f101f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:44:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.525+0000 7f101f7fe640 1 -- 172.21.15.78:0/3739250561 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038022040 con 0x55a19e73d3f0 2024-08-18T23:44:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.525+0000 7f1048916740 1 -- 172.21.15.78:0/3739250561 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10300a4230 con 0x55a19e73d3f0 2024-08-18T23:44:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.525+0000 7f101f7fe640 1 -- 172.21.15.78:0/3739250561 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f1038004c00 con 0x55a19e73d3f0 2024-08-18T23:44:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.525+0000 7f101f7fe640 1 -- 172.21.15.78:0/3739250561 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f103801a400 con 0x55a19e73d3f0 2024-08-18T23:44:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.525+0000 7f1048916740 10 client.15486 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.525+0000 7f1048916740 1 -- 172.21.15.78:0/3739250561 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a19e7bc0c0 con 0x55a19e73d3f0 2024-08-18T23:44:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.525+0000 7f101f7fe640 1 -- 172.21.15.78:0/3739250561 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10300a4230 con 0x55a19e73d3f0 2024-08-18T23:44:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.525+0000 7f101f7fe640 1 -- 172.21.15.78:0/3739250561 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1038019040 con 0x55a19e73d3f0 2024-08-18T23:44:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.525+0000 7f1048916740 10 client.15486 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.525+0000 7f1048916740 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:44:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.525+0000 7f1048916740 20 client.15486 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.525+0000 7f1048916740 1 --2- 172.21.15.78:0/3739250561 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a19e74cdf0 0x55a19e7be6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.525+0000 7f1048916740 4 client.15486 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:44:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.525+0000 7f1048916740 1 -- 172.21.15.78:0/3739250561 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a19e69f400 con 0x55a19e74cdf0 2024-08-18T23:44:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.526+0000 7f103ffff640 1 --2- 172.21.15.78:0/3739250561 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a19e74cdf0 0x55a19e7be6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.526+0000 7f103ffff640 1 --2- 172.21.15.78:0/3739250561 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a19e74cdf0 0x55a19e7be6b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4de20b11af027e0b server_cookie=b16b6fa543360f0 in_seq=0 out_seq=0 2024-08-18T23:44:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.526+0000 7f101f7fe640 10 client.15486 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.526+0000 7f101f7fe640 1 -- 172.21.15.78:0/3739250561 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a19e69f400 con 0x55a19e74cdf0 2024-08-18T23:44:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.526+0000 7f101f7fe640 10 client.15486 handle_command_reply: tid=0 2024-08-18T23:44:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.526+0000 7f1048916740 1 client.15486 shutdown 2024-08-18T23:44:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.527+0000 7f1048916740 1 -- 172.21.15.78:0/3739250561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e73d3f0 msgr2=0x55a19e745920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.527+0000 7f1048916740 1 --2- 172.21.15.78:0/3739250561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e73d3f0 0x55a19e745920 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1038008f40 tx=0x7f103801c6a0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.527+0000 7f1048916740 1 -- 172.21.15.78:0/3739250561 shutdown_connections 2024-08-18T23:44:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.527+0000 7f1048916740 1 --2- 172.21.15.78:0/3739250561 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a19e74cdf0 0x55a19e7be6b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.527+0000 7f1048916740 1 --2- 172.21.15.78:0/3739250561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a19e73d3f0 0x55a19e745920 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.527+0000 7f1048916740 1 --2- 172.21.15.78:0/3739250561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a19e732b30 0x55a19e7453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.527+0000 7f1048916740 1 --2- 172.21.15.78:0/3739250561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a19e732210 0x55a19e7b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.527+0000 7f1048916740 1 -- 172.21.15.78:0/3739250561 >> 172.21.15.78:0/3739250561 conn(0x55a19e72e7a0 msgr2=0x55a19e73aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.527+0000 7f101f7fe640 0 client.15486 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.527+0000 7f1048916740 1 -- 172.21.15.78:0/3739250561 shutdown_connections 2024-08-18T23:44:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.527+0000 7f1048916740 1 -- 172.21.15.78:0/3739250561 wait complete. 2024-08-18T23:44:42.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.527+0000 7f1048916740 20 client.15486 trim_cache size 0 max 16384 2024-08-18T23:44:42.533 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:44:42.533 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-08-18T23:44:42.533 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-08-18T23:44:42.533 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:44:42.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.527+0000 7f1046518640 1 -- 172.21.15.78:0/4062686462 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f102c0a2c90 msgr2=0x7f102c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:42.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.527+0000 7f1046518640 1 --2- 172.21.15.78:0/4062686462 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f102c0a2c90 0x7f102c0a5150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f103801f000 tx=0x7f1038020000 comp rx=0 tx=0).stop 2024-08-18T23:44:42.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.527+0000 7f1046518640 1 -- 172.21.15.78:0/4062686462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10401018b0 msgr2=0x7f10401981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:42.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.527+0000 7f1046518640 1 --2- 172.21.15.78:0/4062686462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10401018b0 0x7f10401981f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f103000c970 tx=0x7f103000ce30 comp rx=0 tx=0).stop 2024-08-18T23:44:42.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.528+0000 7f1046518640 1 -- 172.21.15.78:0/4062686462 shutdown_connections 2024-08-18T23:44:42.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.528+0000 7f1046518640 1 --2- 172.21.15.78:0/4062686462 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f102c0a2c90 0x7f102c0a5150 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.528+0000 7f1046518640 1 --2- 172.21.15.78:0/4062686462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10401078b0 0x7f104019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.528+0000 7f1046518640 1 --2- 172.21.15.78:0/4062686462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1040102250 0x7f1040198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.528+0000 7f1046518640 1 --2- 172.21.15.78:0/4062686462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10401018b0 0x7f10401981f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:42.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.528+0000 7f1046518640 1 -- 172.21.15.78:0/4062686462 >> 172.21.15.78:0/4062686462 conn(0x7f10400fd5f0 msgr2=0x7f10400fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:42.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.528+0000 7f1046518640 1 -- 172.21.15.78:0/4062686462 shutdown_connections 2024-08-18T23:44:42.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:42.528+0000 7f1046518640 1 -- 172.21.15.78:0/4062686462 wait complete. 2024-08-18T23:44:42.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:42 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1887279983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:42.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:42 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1887279983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:43.187 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 154.7 --await-for 45.0' 2024-08-18T23:44:43.187 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 154.7 --await-for 45.0 2024-08-18T23:44:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:43 smithi078 ceph-mon[18440]: pgmap v362: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:44:43.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:43 smithi123 ceph-mon[21808]: pgmap v362: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:44:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:43 smithi161 ceph-mon[22027]: pgmap v362: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:44:43.907 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-18T23:44:43.907 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:44:43.990 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:44.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.517+0000 7f3e3c9c2640 1 -- 172.21.15.78:0/3495525052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38107fd0 msgr2=0x7f3e3810e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:44.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.517+0000 7f3e3c9c2640 1 --2- 172.21.15.78:0/3495525052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38107fd0 0x7f3e3810e970 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3e30009cd0 tx=0x7f3e3002a8e0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.518+0000 7f3e3c9c2640 1 -- 172.21.15.78:0/3495525052 shutdown_connections 2024-08-18T23:44:44.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.518+0000 7f3e3c9c2640 1 --2- 172.21.15.78:0/3495525052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38107fd0 0x7f3e3810e970 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.518+0000 7f3e3c9c2640 1 --2- 172.21.15.78:0/3495525052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e38103410 0x7f3e381078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.518+0000 7f3e3c9c2640 1 --2- 172.21.15.78:0/3495525052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e38102a60 0x7f3e38102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.518+0000 7f3e3c9c2640 1 -- 172.21.15.78:0/3495525052 >> 172.21.15.78:0/3495525052 conn(0x7f3e380fc500 msgr2=0x7f3e380fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:44.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.518+0000 7f3e3c9c2640 1 -- 172.21.15.78:0/3495525052 shutdown_connections 2024-08-18T23:44:44.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.518+0000 7f3e3c9c2640 1 -- 172.21.15.78:0/3495525052 wait complete. 2024-08-18T23:44:44.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.518+0000 7f3e3c9c2640 1 Processor -- start 2024-08-18T23:44:44.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.518+0000 7f3e3c9c2640 1 -- start start 2024-08-18T23:44:44.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e3c9c2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e38102a60 0x7f3e381006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:44.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e3c9c2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38103410 0x7f3e38100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:44.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e3c9c2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e38107fd0 0x7f3e38101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:44.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e3c9c2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3e38111dd0 con 0x7f3e38103410 2024-08-18T23:44:44.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e3c9c2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3e38111c50 con 0x7f3e38102a60 2024-08-18T23:44:44.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e3c9c2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3e38111f50 con 0x7f3e38107fd0 2024-08-18T23:44:44.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e35d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38103410 0x7f3e38100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:44.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e35d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38103410 0x7f3e38100c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58054/0 (socket says 172.21.15.78:58054) 2024-08-18T23:44:44.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e35d9b640 1 -- 172.21.15.78:0/4059719722 learned_addr learned my addr 172.21.15.78:0/4059719722 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:44.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e3659c640 1 --2- 172.21.15.78:0/4059719722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e38102a60 0x7f3e381006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:44.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e36d9d640 1 --2- 172.21.15.78:0/4059719722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e38107fd0 0x7f3e38101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:44.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e35d9b640 1 -- 172.21.15.78:0/4059719722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e38107fd0 msgr2=0x7f3e38101140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:44.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e35d9b640 1 --2- 172.21.15.78:0/4059719722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e38107fd0 0x7f3e38101140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e35d9b640 1 -- 172.21.15.78:0/4059719722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e38102a60 msgr2=0x7f3e381006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:44.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e35d9b640 1 --2- 172.21.15.78:0/4059719722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e38102a60 0x7f3e381006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e35d9b640 1 -- 172.21.15.78:0/4059719722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e381a6750 con 0x7f3e38103410 2024-08-18T23:44:44.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e35d9b640 1 --2- 172.21.15.78:0/4059719722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38103410 0x7f3e38100c00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00d750 tx=0x7f3e2c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:44.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e277fe640 1 -- 172.21.15.78:0/4059719722 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c017070 con 0x7f3e38103410 2024-08-18T23:44:44.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e3c9c2640 1 -- 172.21.15.78:0/4059719722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e381a6a40 con 0x7f3e38103410 2024-08-18T23:44:44.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e277fe640 1 -- 172.21.15.78:0/4059719722 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e2c005500 con 0x7f3e38103410 2024-08-18T23:44:44.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.519+0000 7f3e277fe640 1 -- 172.21.15.78:0/4059719722 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c004e00 con 0x7f3e38103410 2024-08-18T23:44:44.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.520+0000 7f3e3c9c2640 1 -- 172.21.15.78:0/4059719722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e381a6f30 con 0x7f3e38103410 2024-08-18T23:44:44.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.522+0000 7f3e3edc0740 1 Processor -- start 2024-08-18T23:44:44.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.522+0000 7f3e3edc0740 1 -- start start 2024-08-18T23:44:44.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.523+0000 7f3e3edc0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd64130 0x55a76bd64510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:44.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.523+0000 7f3e3edc0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd64a50 0x55a76bd6edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:44.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.523+0000 7f3e3edc0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd6f310 0x55a76bd71700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:44.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.523+0000 7f3e3edc0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a76bcc9a70 con 0x55a76bd64130 2024-08-18T23:44:44.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.523+0000 7f3e3edc0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a76bc61780 con 0x55a76bd64a50 2024-08-18T23:44:44.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.523+0000 7f3e3edc0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a76bcc5a30 con 0x55a76bd6f310 2024-08-18T23:44:44.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.523+0000 7f3e277fe640 1 -- 172.21.15.78:0/4059719722 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f3e2c004fa0 con 0x7f3e38103410 2024-08-18T23:44:44.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.523+0000 7f3e36d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd64130 0x55a76bd64510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:44.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.523+0000 7f3e36d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd64130 0x55a76bd64510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58056/0 (socket says 172.21.15.78:58056) 2024-08-18T23:44:44.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.523+0000 7f3e36d9d640 1 -- 172.21.15.78:0/3146868293 learned_addr learned my addr 172.21.15.78:0/3146868293 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:44.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.523+0000 7f3e3659c640 1 --2- 172.21.15.78:0/3146868293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd64a50 0x55a76bd6edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:44.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.523+0000 7f3e3659c640 1 -- 172.21.15.78:0/3146868293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd6f310 msgr2=0x55a76bd71700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:44.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.523+0000 7f3e3659c640 1 --2- 172.21.15.78:0/3146868293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd6f310 0x55a76bd71700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.523+0000 7f3e3659c640 1 -- 172.21.15.78:0/3146868293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd64130 msgr2=0x55a76bd64510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:44.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.523+0000 7f3e3659c640 1 --2- 172.21.15.78:0/3146868293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd64130 0x55a76bd64510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.523+0000 7f3e3659c640 1 -- 172.21.15.78:0/3146868293 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a76ba3bb90 con 0x55a76bd64a50 2024-08-18T23:44:44.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.523+0000 7f3e36d9d640 1 --2- 172.21.15.78:0/3146868293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd64130 0x55a76bd64510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:44:44.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.524+0000 7f3e3659c640 1 --2- 172.21.15.78:0/3146868293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd64a50 0x55a76bd6edd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800bbf0 tx=0x7f3e280107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd6b6fb83b3d8aea server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:44.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.524+0000 7f3e24ff9640 1 -- 172.21.15.78:0/3146868293 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28004030 con 0x55a76bd64a50 2024-08-18T23:44:44.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.524+0000 7f3e277fe640 1 --2- 172.21.15.78:0/4059719722 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3e200a2c10 0x7f3e200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:44.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.524+0000 7f3e24ff9640 1 -- 172.21.15.78:0/3146868293 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e280041d0 con 0x55a76bd64a50 2024-08-18T23:44:44.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.524+0000 7f3e24ff9640 1 -- 172.21.15.78:0/3146868293 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e280117b0 con 0x55a76bd64a50 2024-08-18T23:44:44.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.524+0000 7f3e277fe640 1 -- 172.21.15.78:0/4059719722 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f3e2c0ceeb0 con 0x7f3e38103410 2024-08-18T23:44:44.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.524+0000 7f3e36d9d640 1 --2- 172.21.15.78:0/4059719722 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3e200a2c10 0x7f3e200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:44.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.525+0000 7f3e3edc0740 1 -- 172.21.15.78:0/3146868293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd64a50 msgr2=0x55a76bd6edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:44.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.525+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/3146868293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd64a50 0x55a76bd6edd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800bbf0 tx=0x7f3e280107a0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.525+0000 7f3e36d9d640 1 --2- 172.21.15.78:0/4059719722 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3e200a2c10 0x7f3e200a50d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3e30005e30 tx=0x7f3e30005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:44.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.525+0000 7f3e3edc0740 1 -- 172.21.15.78:0/3146868293 shutdown_connections 2024-08-18T23:44:44.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.525+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/3146868293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd6f310 0x55a76bd71700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.525+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/3146868293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd64a50 0x55a76bd6edd0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.525+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/3146868293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd64130 0x55a76bd64510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.525+0000 7f3e3edc0740 1 -- 172.21.15.78:0/3146868293 >> 172.21.15.78:0/3146868293 conn(0x55a76bd606c0 msgr2=0x55a76bd60aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:44.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.525+0000 7f3e3edc0740 1 -- 172.21.15.78:0/3146868293 shutdown_connections 2024-08-18T23:44:44.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.525+0000 7f3e3edc0740 1 -- 172.21.15.78:0/3146868293 wait complete. 2024-08-18T23:44:44.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.525+0000 7f3e3edc0740 1 -- start start 2024-08-18T23:44:44.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.525+0000 7f3e3edc0740 1 Processor -- start 2024-08-18T23:44:44.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.526+0000 7f3e3edc0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd64130 0x55a76bd6c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:44.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.526+0000 7f3e3edc0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd64a50 0x55a76bd6c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:44.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.526+0000 7f3e3edc0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd6f310 0x55a76bd6cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:44.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.526+0000 7f3e3edc0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a76bd71ee0 con 0x55a76bd6f310 2024-08-18T23:44:44.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.526+0000 7f3e3edc0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a76bc84990 con 0x55a76bd64130 2024-08-18T23:44:44.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.526+0000 7f3e3edc0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a76bd72060 con 0x55a76bd64a50 2024-08-18T23:44:44.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.526+0000 7f3e3edc0740 10 client.0 _ll_register_callbacks cb 0x55a76b9d9090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:44.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.526+0000 7f3e36d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd6f310 0x55a76bd6cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:44.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.526+0000 7f3e35d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd64a50 0x55a76bd6c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:44.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.526+0000 7f3e36d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd6f310 0x55a76bd6cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58058/0 (socket says 172.21.15.78:58058) 2024-08-18T23:44:44.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.526+0000 7f3e36d9d640 1 -- 172.21.15.78:0/318604909 learned_addr learned my addr 172.21.15.78:0/318604909 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:44.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.526+0000 7f3e3659c640 1 --2- 172.21.15.78:0/318604909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd64130 0x55a76bd6c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:44.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.526+0000 7f3e35d9b640 1 -- 172.21.15.78:0/318604909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd64130 msgr2=0x55a76bd6c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:44.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.526+0000 7f3e35d9b640 1 --2- 172.21.15.78:0/318604909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd64130 0x55a76bd6c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.526+0000 7f3e35d9b640 1 -- 172.21.15.78:0/318604909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd6f310 msgr2=0x55a76bd6cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:44.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.526+0000 7f3e35d9b640 1 --2- 172.21.15.78:0/318604909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd6f310 0x55a76bd6cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.526+0000 7f3e35d9b640 1 -- 172.21.15.78:0/318604909 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a76bc836c0 con 0x55a76bd64a50 2024-08-18T23:44:44.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.526+0000 7f3e3659c640 1 --2- 172.21.15.78:0/318604909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd64130 0x55a76bd6c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:44.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.526+0000 7f3e36d9d640 1 --2- 172.21.15.78:0/318604909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd6f310 0x55a76bd6cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:44:44.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.527+0000 7f3e1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:44:44.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.527+0000 7f3e35d9b640 1 --2- 172.21.15.78:0/318604909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd64a50 0x55a76bd6c710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c0cc4d0 tx=0x7f3e2c00df20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3379ef3346a7377d server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:44.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.527+0000 7f3e1dffb640 1 -- 172.21.15.78:0/318604909 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c020070 con 0x55a76bd64a50 2024-08-18T23:44:44.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.527+0000 7f3e3edc0740 1 -- 172.21.15.78:0/318604909 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e280041d0 con 0x55a76bd64a50 2024-08-18T23:44:44.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.527+0000 7f3e1dffb640 1 -- 172.21.15.78:0/318604909 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f3e2c0cc720 con 0x55a76bd64a50 2024-08-18T23:44:44.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.527+0000 7f3e1dffb640 1 -- 172.21.15.78:0/318604909 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3e2c0b9e60 con 0x55a76bd64a50 2024-08-18T23:44:44.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.527+0000 7f3e3edc0740 10 client.24925 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:44.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.527+0000 7f3e3edc0740 1 -- 172.21.15.78:0/318604909 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a76bd7dc10 con 0x55a76bd64a50 2024-08-18T23:44:44.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.527+0000 7f3e1dffb640 1 -- 172.21.15.78:0/318604909 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e280041d0 con 0x55a76bd64a50 2024-08-18T23:44:44.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.528+0000 7f3e1dffb640 1 -- 172.21.15.78:0/318604909 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3e2c097890 con 0x55a76bd64a50 2024-08-18T23:44:44.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.528+0000 7f3e3edc0740 10 client.24925 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:44.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.528+0000 7f3e3edc0740 10 client.24925 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:44:44.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.528+0000 7f3e3edc0740 20 client.24925 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:44.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.528+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/318604909 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a76bd7f070 0x55a76bd81460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:44.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.528+0000 7f3e3edc0740 4 client.24925 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:44:44.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.528+0000 7f3e3edc0740 1 -- 172.21.15.78:0/318604909 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a76bcd71a0 con 0x55a76bd7f070 2024-08-18T23:44:44.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.528+0000 7f3e3659c640 1 --2- 172.21.15.78:0/318604909 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a76bd7f070 0x55a76bd81460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:44.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.528+0000 7f3e3659c640 1 --2- 172.21.15.78:0/318604909 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a76bd7f070 0x55a76bd81460 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f7399aabf0e54f0 server_cookie=9649b83e2a99a345 in_seq=0 out_seq=0 2024-08-18T23:44:44.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.528+0000 7f3e1dffb640 10 client.24925 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:44.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.529+0000 7f3e1dffb640 1 -- 172.21.15.78:0/318604909 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a76bcd71a0 con 0x55a76bd7f070 2024-08-18T23:44:44.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.529+0000 7f3e1dffb640 10 client.24925 handle_command_reply: tid=0 2024-08-18T23:44:44.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.529+0000 7f3e3edc0740 1 client.24925 shutdown 2024-08-18T23:44:44.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.529+0000 7f3e3edc0740 1 -- 172.21.15.78:0/318604909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd64a50 msgr2=0x55a76bd6c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:44.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.529+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/318604909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd64a50 0x55a76bd6c710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c0cc4d0 tx=0x7f3e2c00df20 comp rx=0 tx=0).stop 2024-08-18T23:44:44.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.529+0000 7f3e3edc0740 1 -- 172.21.15.78:0/318604909 shutdown_connections 2024-08-18T23:44:44.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.529+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/318604909 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a76bd7f070 0x55a76bd81460 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.529+0000 7f3e1dffb640 0 client.24925 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:44.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.529+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/318604909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd6f310 0x55a76bd6cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.529+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/318604909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd64a50 0x55a76bd6c710 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.529+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/318604909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd64130 0x55a76bd6c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.529+0000 7f3e3edc0740 1 -- 172.21.15.78:0/318604909 >> 172.21.15.78:0/318604909 conn(0x55a76bd606c0 msgr2=0x55a76bd663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:44.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.530+0000 7f3e3edc0740 1 -- 172.21.15.78:0/318604909 shutdown_connections 2024-08-18T23:44:44.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.530+0000 7f3e3edc0740 1 -- 172.21.15.78:0/318604909 wait complete. 2024-08-18T23:44:44.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.530+0000 7f3e3edc0740 20 client.24925 trim_cache size 0 max 16384 2024-08-18T23:44:44.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.549+0000 7f3e3edc0740 1 Processor -- start 2024-08-18T23:44:44.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.549+0000 7f3e3edc0740 1 -- start start 2024-08-18T23:44:44.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.549+0000 7f3e3edc0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd64130 0x55a76bd6cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:44.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.549+0000 7f3e3edc0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd64a50 0x55a76bd86cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:44.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.549+0000 7f3e3edc0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd6f310 0x55a76bd87230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:44.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.549+0000 7f3e3edc0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381b3570 con 0x55a76bd64a50 2024-08-18T23:44:44.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.549+0000 7f3e3edc0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381b0e40 con 0x55a76bd6f310 2024-08-18T23:44:44.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.549+0000 7f3e3edc0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381aec90 con 0x55a76bd64130 2024-08-18T23:44:44.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.549+0000 7f3e35d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd64a50 0x55a76bd86cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:44.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.549+0000 7f3e35d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd64a50 0x55a76bd86cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58062/0 (socket says 172.21.15.78:58062) 2024-08-18T23:44:44.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.549+0000 7f3e35d9b640 1 -- 172.21.15.78:0/2972616256 learned_addr learned my addr 172.21.15.78:0/2972616256 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:44.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.549+0000 7f3e36d9d640 1 --2- 172.21.15.78:0/2972616256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd6f310 0x55a76bd87230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:44.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.549+0000 7f3e3659c640 1 --2- 172.21.15.78:0/2972616256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd64130 0x55a76bd6cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:44.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.550+0000 7f3e35d9b640 1 -- 172.21.15.78:0/2972616256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd64130 msgr2=0x55a76bd6cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:44.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.550+0000 7f3e35d9b640 1 --2- 172.21.15.78:0/2972616256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd64130 0x55a76bd6cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.550+0000 7f3e35d9b640 1 -- 172.21.15.78:0/2972616256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd6f310 msgr2=0x55a76bd87230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:44.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.550+0000 7f3e35d9b640 1 --2- 172.21.15.78:0/2972616256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd6f310 0x55a76bd87230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.550+0000 7f3e35d9b640 1 -- 172.21.15.78:0/2972616256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a76bd64880 con 0x55a76bd64a50 2024-08-18T23:44:44.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.550+0000 7f3e35d9b640 1 --2- 172.21.15.78:0/2972616256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd64a50 0x55a76bd86cf0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c097bf0 tx=0x7f3e2c00df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1760405b02ebd4f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:44.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.550+0000 7f3e1ffff640 1 -- 172.21.15.78:0/2972616256 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c020070 con 0x55a76bd64a50 2024-08-18T23:44:44.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.550+0000 7f3e1ffff640 1 -- 172.21.15.78:0/2972616256 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e2c0a3c60 con 0x55a76bd64a50 2024-08-18T23:44:44.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.550+0000 7f3e1ffff640 1 -- 172.21.15.78:0/2972616256 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c099520 con 0x55a76bd64a50 2024-08-18T23:44:44.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.551+0000 7f3e3edc0740 1 -- 172.21.15.78:0/2972616256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd64a50 msgr2=0x55a76bd86cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:44.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.551+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/2972616256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd64a50 0x55a76bd86cf0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c097bf0 tx=0x7f3e2c00df90 comp rx=0 tx=0).stop 2024-08-18T23:44:44.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.551+0000 7f3e3edc0740 1 -- 172.21.15.78:0/2972616256 shutdown_connections 2024-08-18T23:44:44.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.551+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/2972616256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd6f310 0x55a76bd87230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.551+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/2972616256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd64a50 0x55a76bd86cf0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.551+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/2972616256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd64130 0x55a76bd6cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.551+0000 7f3e3edc0740 1 -- 172.21.15.78:0/2972616256 >> 172.21.15.78:0/2972616256 conn(0x55a76bd606c0 msgr2=0x55a76bd6ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:44.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.551+0000 7f3e3edc0740 1 -- 172.21.15.78:0/2972616256 shutdown_connections 2024-08-18T23:44:44.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.551+0000 7f3e3edc0740 1 -- 172.21.15.78:0/2972616256 wait complete. 2024-08-18T23:44:44.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.551+0000 7f3e3edc0740 1 -- start start 2024-08-18T23:44:44.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.552+0000 7f3e3edc0740 1 Processor -- start 2024-08-18T23:44:44.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.552+0000 7f3e3edc0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd64130 0x55a76bd69300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:44.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.552+0000 7f3e3edc0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd64a50 0x55a76bd69840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:44.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.552+0000 7f3e3edc0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd6f310 0x55a76bd69d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:44.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.552+0000 7f3e36d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd6f310 0x55a76bd69d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:44.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.552+0000 7f3e36d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd6f310 0x55a76bd69d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58072/0 (socket says 172.21.15.78:58072) 2024-08-18T23:44:44.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.552+0000 7f3e36d9d640 1 -- 172.21.15.78:0/2836869188 learned_addr learned my addr 172.21.15.78:0/2836869188 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:44.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.552+0000 7f3e3edc0740 1 -- 172.21.15.78:0/2836869188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381af300 con 0x55a76bd6f310 2024-08-18T23:44:44.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.552+0000 7f3e3edc0740 1 -- 172.21.15.78:0/2836869188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381b0770 con 0x55a76bd64130 2024-08-18T23:44:44.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.552+0000 7f3e3edc0740 1 -- 172.21.15.78:0/2836869188 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3e381af9d0 con 0x55a76bd64a50 2024-08-18T23:44:44.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.552+0000 7f3e3edc0740 10 client.0 _ll_register_callbacks cb 0x7f3e2c00d6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:44.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.552+0000 7f3e35d9b640 1 --2- 172.21.15.78:0/2836869188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd64a50 0x55a76bd69840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:44.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.552+0000 7f3e3659c640 1 --2- 172.21.15.78:0/2836869188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd64130 0x55a76bd69300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:44.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.552+0000 7f3e36d9d640 1 -- 172.21.15.78:0/2836869188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd64a50 msgr2=0x55a76bd69840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:44.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.552+0000 7f3e36d9d640 1 --2- 172.21.15.78:0/2836869188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd64a50 0x55a76bd69840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.552+0000 7f3e36d9d640 1 -- 172.21.15.78:0/2836869188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd64130 msgr2=0x55a76bd69300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:44.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.552+0000 7f3e36d9d640 1 --2- 172.21.15.78:0/2836869188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd64130 0x55a76bd69300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.552+0000 7f3e36d9d640 1 -- 172.21.15.78:0/2836869188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a76bde8d70 con 0x55a76bd6f310 2024-08-18T23:44:44.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.553+0000 7f3e36d9d640 1 --2- 172.21.15.78:0/2836869188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd6f310 0x55a76bd69d80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3e30003f50 tx=0x7f3e300095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a0fc36819c2676 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:44.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.553+0000 7f3e1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:44:44.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.553+0000 7f3e1dffb640 1 -- 172.21.15.78:0/2836869188 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30057020 con 0x55a76bd6f310 2024-08-18T23:44:44.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.553+0000 7f3e1dffb640 1 -- 172.21.15.78:0/2836869188 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f3e300046d0 con 0x55a76bd6f310 2024-08-18T23:44:44.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.553+0000 7f3e3edc0740 1 -- 172.21.15.78:0/2836869188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e2c0a3c60 con 0x55a76bd6f310 2024-08-18T23:44:44.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.553+0000 7f3e1dffb640 1 -- 172.21.15.78:0/2836869188 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3e30062050 con 0x55a76bd6f310 2024-08-18T23:44:44.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.553+0000 7f3e3edc0740 10 client.15504 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:44.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.553+0000 7f3e3edc0740 1 -- 172.21.15.78:0/2836869188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a76bdea010 con 0x55a76bd6f310 2024-08-18T23:44:44.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.553+0000 7f3e1dffb640 1 -- 172.21.15.78:0/2836869188 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e2c0a3c60 con 0x55a76bd6f310 2024-08-18T23:44:44.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.554+0000 7f3e1dffb640 1 -- 172.21.15.78:0/2836869188 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3e30030070 con 0x55a76bd6f310 2024-08-18T23:44:44.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.554+0000 7f3e3edc0740 10 client.15504 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:44.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.554+0000 7f3e3edc0740 10 client.15504 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:44:44.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.554+0000 7f3e3edc0740 20 client.15504 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:44.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.554+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/2836869188 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a76bdea7f0 0x55a76bdecc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:44.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.554+0000 7f3e3edc0740 4 client.15504 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.7, "await_for": 45.0}] 2024-08-18T23:44:44.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.554+0000 7f3e3edc0740 1 -- 172.21.15.78:0/2836869188 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.7, "await_for": 45.0}) -- 0x55a76bcd14f0 con 0x55a76bdea7f0 2024-08-18T23:44:44.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.554+0000 7f3e3659c640 1 --2- 172.21.15.78:0/2836869188 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a76bdea7f0 0x55a76bdecc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:44.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.554+0000 7f3e1dffb640 10 client.15504 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:44.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.554+0000 7f3e3659c640 1 --2- 172.21.15.78:0/2836869188 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a76bdea7f0 0x55a76bdecc10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b285ff36ac576d90 server_cookie=d5067a74e48cafd2 in_seq=0 out_seq=0 2024-08-18T23:44:44.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.565+0000 7f3e1dffb640 1 -- 172.21.15.78:0/2836869188 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a76bcd14f0 con 0x55a76bdea7f0 2024-08-18T23:44:44.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.565+0000 7f3e1dffb640 10 client.15504 handle_command_reply: tid=0 2024-08-18T23:44:44.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.565+0000 7f3e3edc0740 1 client.15504 shutdown 2024-08-18T23:44:44.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.565+0000 7f3e3edc0740 1 -- 172.21.15.78:0/2836869188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd6f310 msgr2=0x55a76bd69d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.565+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/2836869188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd6f310 0x55a76bd69d80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3e30003f50 tx=0x7f3e300095c0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.565+0000 7f3e3edc0740 1 -- 172.21.15.78:0/2836869188 shutdown_connections 2024-08-18T23:44:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.565+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/2836869188 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a76bdea7f0 0x55a76bdecc10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.565+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/2836869188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a76bd6f310 0x55a76bd69d80 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.565+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/2836869188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a76bd64a50 0x55a76bd69840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.565+0000 7f3e3edc0740 1 --2- 172.21.15.78:0/2836869188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a76bd64130 0x55a76bd69300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.565+0000 7f3e3edc0740 1 -- 172.21.15.78:0/2836869188 >> 172.21.15.78:0/2836869188 conn(0x55a76bd606c0 msgr2=0x55a76bd5eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.565+0000 7f3e1dffb640 0 client.15504 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.565+0000 7f3e3edc0740 1 -- 172.21.15.78:0/2836869188 shutdown_connections 2024-08-18T23:44:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.565+0000 7f3e3edc0740 1 -- 172.21.15.78:0/2836869188 wait complete. 2024-08-18T23:44:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.566+0000 7f3e3edc0740 20 client.15504 trim_cache size 0 max 16384 2024-08-18T23:44:44.568 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:44:44.568 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 25, 2024-08-18T23:44:44.568 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24452, 2024-08-18T23:44:44.568 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 2, 2024-08-18T23:44:44.568 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-08-18T23:44:44.568 INFO:teuthology.orchestra.run.smithi078.stdout: "f246fd05": { 2024-08-18T23:44:44.568 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 2, 2024-08-18T23:44:44.568 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-08-18T23:44:44.568 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:44:44.568 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-08-18T23:44:44.569 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:44:44.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:44:44.569 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-08-18T23:44:44.569 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 154.7, 2024-08-18T23:44:44.569 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-08-18T23:44:44.569 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-08-18T23:44:44.569 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-08-18T23:44:44.569 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:44:44.569 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-08-18T23:44:44.569 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:44:44.569 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:44:44.569 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:44:44.569 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:44:44.570 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:44:44.570 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:44:44.570 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:44:44.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.566+0000 7f3e3c9c2640 1 -- 172.21.15.78:0/4059719722 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3e200a2c10 msgr2=0x7f3e200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:44.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.566+0000 7f3e3c9c2640 1 --2- 172.21.15.78:0/4059719722 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3e200a2c10 0x7f3e200a50d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3e30005e30 tx=0x7f3e30005c50 comp rx=0 tx=0).stop 2024-08-18T23:44:44.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.566+0000 7f3e3c9c2640 1 -- 172.21.15.78:0/4059719722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38103410 msgr2=0x7f3e38100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:44.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.566+0000 7f3e3c9c2640 1 --2- 172.21.15.78:0/4059719722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38103410 0x7f3e38100c00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00d750 tx=0x7f3e2c00dc10 comp rx=0 tx=0).stop 2024-08-18T23:44:44.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.567+0000 7f3e3c9c2640 1 -- 172.21.15.78:0/4059719722 shutdown_connections 2024-08-18T23:44:44.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.567+0000 7f3e3c9c2640 1 --2- 172.21.15.78:0/4059719722 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3e200a2c10 0x7f3e200a50d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.567+0000 7f3e3c9c2640 1 --2- 172.21.15.78:0/4059719722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e38107fd0 0x7f3e38101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.567+0000 7f3e3c9c2640 1 --2- 172.21.15.78:0/4059719722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e38103410 0x7f3e38100c00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.567+0000 7f3e3c9c2640 1 --2- 172.21.15.78:0/4059719722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e38102a60 0x7f3e381006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:44.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.567+0000 7f3e3c9c2640 1 -- 172.21.15.78:0/4059719722 >> 172.21.15.78:0/4059719722 conn(0x7f3e380fc500 msgr2=0x7f3e38105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:44.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.567+0000 7f3e3c9c2640 1 -- 172.21.15.78:0/4059719722 shutdown_connections 2024-08-18T23:44:44.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:44.567+0000 7f3e3c9c2640 1 -- 172.21.15.78:0/4059719722 wait complete. 2024-08-18T23:44:44.580 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:45.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:45 smithi123 ceph-mon[21808]: pgmap v363: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-18T23:44:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:45 smithi161 ceph-mon[22027]: pgmap v363: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-18T23:44:45.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:45 smithi078 ceph-mon[18440]: pgmap v363: 129 pgs: 129 active+clean; 658 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-18T23:44:47.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:47 smithi123 ceph-mon[21808]: pgmap v364: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-18T23:44:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:47 smithi161 ceph-mon[22027]: pgmap v364: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-18T23:44:47.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:47 smithi078 ceph-mon[18440]: pgmap v364: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-18T23:44:49.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:49 smithi123 ceph-mon[21808]: pgmap v365: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-18T23:44:49.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:49 smithi078 ceph-mon[18440]: pgmap v365: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-18T23:44:49.671 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f246fd05', quiesce took 9.0 seconds. Will release after: 25.700000000000003 2024-08-18T23:44:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:49 smithi161 ceph-mon[22027]: pgmap v365: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-18T23:44:50.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.207+0000 7fc3e60d9640 1 -- 172.21.15.78:0/3135675347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0102a70 msgr2=0x7fc3e0102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:50.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.207+0000 7fc3e60d9640 1 --2- 172.21.15.78:0/3135675347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0102a70 0x7fc3e0102e50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009900 tx=0x7fc3c802a480 comp rx=0 tx=0).stop 2024-08-18T23:44:50.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.207+0000 7fc3e60d9640 1 -- 172.21.15.78:0/3135675347 shutdown_connections 2024-08-18T23:44:50.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.207+0000 7fc3e60d9640 1 --2- 172.21.15.78:0/3135675347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3e010bea0 0x7fc3e010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:50.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.207+0000 7fc3e60d9640 1 --2- 172.21.15.78:0/3135675347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3e0103390 0x7fc3e010b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:50.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.207+0000 7fc3e60d9640 1 --2- 172.21.15.78:0/3135675347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0102a70 0x7fc3e0102e50 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:50.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.207+0000 7fc3e60d9640 1 -- 172.21.15.78:0/3135675347 >> 172.21.15.78:0/3135675347 conn(0x7fc3e00fc4d0 msgr2=0x7fc3e00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:50.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.208+0000 7fc3e60d9640 1 -- 172.21.15.78:0/3135675347 shutdown_connections 2024-08-18T23:44:50.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.208+0000 7fc3e60d9640 1 -- 172.21.15.78:0/3135675347 wait complete. 2024-08-18T23:44:50.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.208+0000 7fc3e60d9640 1 Processor -- start 2024-08-18T23:44:50.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.209+0000 7fc3e60d9640 1 -- start start 2024-08-18T23:44:50.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.209+0000 7fc3e60d9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0102a70 0x7fc3e0198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:50.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.209+0000 7fc3e60d9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3e0103390 0x7fc3e01987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:50.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.209+0000 7fc3e60d9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3e010bea0 0x7fc3e019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:50.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.209+0000 7fc3e60d9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e0111ca0 con 0x7fc3e0102a70 2024-08-18T23:44:50.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.209+0000 7fc3e60d9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e0111b20 con 0x7fc3e010bea0 2024-08-18T23:44:50.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.209+0000 7fc3e60d9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e0111e20 con 0x7fc3e0103390 2024-08-18T23:44:50.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.209+0000 7fc3df7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0102a70 0x7fc3e0198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:50.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.209+0000 7fc3df7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0102a70 0x7fc3e0198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42984/0 (socket says 172.21.15.78:42984) 2024-08-18T23:44:50.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.209+0000 7fc3df7fe640 1 -- 172.21.15.78:0/3964322848 learned_addr learned my addr 172.21.15.78:0/3964322848 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:50.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.209+0000 7fc3dffff640 1 --2- 172.21.15.78:0/3964322848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3e010bea0 0x7fc3e019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:50.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.209+0000 7fc3deffd640 1 --2- 172.21.15.78:0/3964322848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3e0103390 0x7fc3e01987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:50.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.209+0000 7fc3df7fe640 1 -- 172.21.15.78:0/3964322848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3e0103390 msgr2=0x7fc3e01987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:50.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.209+0000 7fc3df7fe640 1 --2- 172.21.15.78:0/3964322848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3e0103390 0x7fc3e01987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:50.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.209+0000 7fc3df7fe640 1 -- 172.21.15.78:0/3964322848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3e010bea0 msgr2=0x7fc3e019cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:50.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.209+0000 7fc3df7fe640 1 --2- 172.21.15.78:0/3964322848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3e010bea0 0x7fc3e019cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:50.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.209+0000 7fc3df7fe640 1 -- 172.21.15.78:0/3964322848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e019d210 con 0x7fc3e0102a70 2024-08-18T23:44:50.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.209+0000 7fc3dffff640 1 --2- 172.21.15.78:0/3964322848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3e010bea0 0x7fc3e019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:50.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.210+0000 7fc3df7fe640 1 --2- 172.21.15.78:0/3964322848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0102a70 0x7fc3e0198260 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8006010 tx=0x7fc3c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:50.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.210+0000 7fc3dcff9640 1 -- 172.21.15.78:0/3964322848 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c803f030 con 0x7fc3e0102a70 2024-08-18T23:44:50.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.210+0000 7fc3dcff9640 1 -- 172.21.15.78:0/3964322848 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3c8003ce0 con 0x7fc3e0102a70 2024-08-18T23:44:50.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.210+0000 7fc3dcff9640 1 -- 172.21.15.78:0/3964322848 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8035070 con 0x7fc3e0102a70 2024-08-18T23:44:50.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.210+0000 7fc3e60d9640 1 -- 172.21.15.78:0/3964322848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e019d4a0 con 0x7fc3e0102a70 2024-08-18T23:44:50.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.210+0000 7fc3e60d9640 1 -- 172.21.15.78:0/3964322848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e019d930 con 0x7fc3e0102a70 2024-08-18T23:44:50.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.211+0000 7fc3e60d9640 1 -- 172.21.15.78:0/3964322848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e0102e50 con 0x7fc3e0102a70 2024-08-18T23:44:50.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.212+0000 7fc3dcff9640 1 -- 172.21.15.78:0/3964322848 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fc3c8042030 con 0x7fc3e0102a70 2024-08-18T23:44:50.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.212+0000 7fc3dcff9640 1 --2- 172.21.15.78:0/3964322848 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc3cc0a2c90 0x7fc3cc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:50.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.213+0000 7fc3deffd640 1 --2- 172.21.15.78:0/3964322848 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc3cc0a2c90 0x7fc3cc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:50.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.213+0000 7fc3dcff9640 1 -- 172.21.15.78:0/3964322848 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc3c80ef780 con 0x7fc3e0102a70 2024-08-18T23:44:50.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.213+0000 7fc3deffd640 1 --2- 172.21.15.78:0/3964322848 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc3cc0a2c90 0x7fc3cc0a5150 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc3d00058c0 tx=0x7fc3d0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:50.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.217+0000 7fc3dcff9640 1 -- 172.21.15.78:0/3964322848 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc3c80bc2b0 con 0x7fc3e0102a70 2024-08-18T23:44:50.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.467+0000 7fc3e60d9640 1 -- 172.21.15.78:0/3964322848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3e0106f10 con 0x7fc3e0102a70 2024-08-18T23:44:50.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.468+0000 7fc3dcff9640 1 -- 172.21.15.78:0/3964322848 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7fc3c80bbbb0 con 0x7fc3e0102a70 2024-08-18T23:44:50.472 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:44:50.473 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"btime":"2024-08-18T23:43:57:245845+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:43:57.114120+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24428,24427]},"id":1}]} 2024-08-18T23:44:50.474 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 25 2024-08-18T23:44:50.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.470+0000 7fc3e60d9640 1 -- 172.21.15.78:0/3964322848 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc3cc0a2c90 msgr2=0x7fc3cc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:50.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.470+0000 7fc3e60d9640 1 --2- 172.21.15.78:0/3964322848 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc3cc0a2c90 0x7fc3cc0a5150 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc3d00058c0 tx=0x7fc3d0005850 comp rx=0 tx=0).stop 2024-08-18T23:44:50.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.470+0000 7fc3e60d9640 1 -- 172.21.15.78:0/3964322848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0102a70 msgr2=0x7fc3e0198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:50.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.470+0000 7fc3e60d9640 1 --2- 172.21.15.78:0/3964322848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0102a70 0x7fc3e0198260 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8006010 tx=0x7fc3c80042f0 comp rx=0 tx=0).stop 2024-08-18T23:44:50.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.470+0000 7fc3e60d9640 1 -- 172.21.15.78:0/3964322848 shutdown_connections 2024-08-18T23:44:50.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.470+0000 7fc3e60d9640 1 --2- 172.21.15.78:0/3964322848 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc3cc0a2c90 0x7fc3cc0a5150 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:50.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.470+0000 7fc3e60d9640 1 --2- 172.21.15.78:0/3964322848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3e010bea0 0x7fc3e019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:50.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.470+0000 7fc3e60d9640 1 --2- 172.21.15.78:0/3964322848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3e0103390 0x7fc3e01987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:50.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.470+0000 7fc3e60d9640 1 --2- 172.21.15.78:0/3964322848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3e0102a70 0x7fc3e0198260 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:50.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.470+0000 7fc3e60d9640 1 -- 172.21.15.78:0/3964322848 >> 172.21.15.78:0/3964322848 conn(0x7fc3e00fc4d0 msgr2=0x7fc3e00fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:50.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.470+0000 7fc3e60d9640 1 -- 172.21.15.78:0/3964322848 shutdown_connections 2024-08-18T23:44:50.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:50.470+0000 7fc3e60d9640 1 -- 172.21.15.78:0/3964322848 wait complete. 2024-08-18T23:44:51.304 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-18T23:44:51.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:51 smithi123 ceph-mon[21808]: pgmap v366: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-18T23:44:51.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:51 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3964322848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:51.608 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:51 smithi161 ceph-mon[22027]: pgmap v366: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-18T23:44:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:51 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3964322848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:51.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:51 smithi078 ceph-mon[18440]: pgmap v366: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-18T23:44:51.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:51 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3964322848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:44:52.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.172+0000 7f2c7c03c640 1 -- 172.21.15.78:0/2497875223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c74102250 msgr2=0x7f2c7410e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:52.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.172+0000 7f2c7c03c640 1 --2- 172.21.15.78:0/2497875223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c74102250 0x7f2c7410e780 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009cd0 tx=0x7f2c6c02a8e0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.172+0000 7f2c7c03c640 1 -- 172.21.15.78:0/2497875223 shutdown_connections 2024-08-18T23:44:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.172+0000 7f2c7c03c640 1 --2- 172.21.15.78:0/2497875223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c74102250 0x7f2c7410e780 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.172+0000 7f2c7c03c640 1 --2- 172.21.15.78:0/2497875223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c741018b0 0x7f2c74101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.172+0000 7f2c7c03c640 1 --2- 172.21.15.78:0/2497875223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c741078b0 0x7f2c74107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.172+0000 7f2c7c03c640 1 -- 172.21.15.78:0/2497875223 >> 172.21.15.78:0/2497875223 conn(0x7f2c740fd5f0 msgr2=0x7f2c740ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.172+0000 7f2c7c03c640 1 -- 172.21.15.78:0/2497875223 shutdown_connections 2024-08-18T23:44:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.172+0000 7f2c7c03c640 1 -- 172.21.15.78:0/2497875223 wait complete. 2024-08-18T23:44:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.173+0000 7f2c7c03c640 1 Processor -- start 2024-08-18T23:44:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.173+0000 7f2c7c03c640 1 -- start start 2024-08-18T23:44:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.173+0000 7f2c7c03c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c741018b0 0x7f2c741980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.173+0000 7f2c7c03c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c74102250 0x7f2c741985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.173+0000 7f2c7c03c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c741078b0 0x7f2c7419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:52.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.173+0000 7f2c7c03c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c74111cb0 con 0x7f2c741018b0 2024-08-18T23:44:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.173+0000 7f2c7c03c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c74111b30 con 0x7f2c741078b0 2024-08-18T23:44:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.173+0000 7f2c7c03c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c74111e30 con 0x7f2c74102250 2024-08-18T23:44:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.173+0000 7f2c79dd8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c741018b0 0x7f2c741980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.173+0000 7f2c79dd8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c741018b0 0x7f2c741980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43008/0 (socket says 172.21.15.78:43008) 2024-08-18T23:44:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.173+0000 7f2c79dd8640 1 -- 172.21.15.78:0/786509297 learned_addr learned my addr 172.21.15.78:0/786509297 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.174+0000 7f2c795d7640 1 --2- 172.21.15.78:0/786509297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c74102250 0x7f2c741985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.174+0000 7f2c7a5d9640 1 --2- 172.21.15.78:0/786509297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c741078b0 0x7f2c7419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.174+0000 7f2c79dd8640 1 -- 172.21.15.78:0/786509297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c74102250 msgr2=0x7f2c741985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.174+0000 7f2c79dd8640 1 --2- 172.21.15.78:0/786509297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c74102250 0x7f2c741985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.174+0000 7f2c79dd8640 1 -- 172.21.15.78:0/786509297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c741078b0 msgr2=0x7f2c7419c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.174+0000 7f2c79dd8640 1 --2- 172.21.15.78:0/786509297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c741078b0 0x7f2c7419c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.174+0000 7f2c79dd8640 1 -- 172.21.15.78:0/786509297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7419d060 con 0x7f2c741018b0 2024-08-18T23:44:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.174+0000 7f2c795d7640 1 --2- 172.21.15.78:0/786509297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c74102250 0x7f2c741985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:52.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.174+0000 7f2c7a5d9640 1 --2- 172.21.15.78:0/786509297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c741078b0 0x7f2c7419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:52.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.174+0000 7f2c79dd8640 1 --2- 172.21.15.78:0/786509297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c741018b0 0x7f2c741980b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000c890 tx=0x7f2c7000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:52.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.174+0000 7f2c6affd640 1 -- 172.21.15.78:0/786509297 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70004220 con 0x7f2c741018b0 2024-08-18T23:44:52.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.174+0000 7f2c6affd640 1 -- 172.21.15.78:0/786509297 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c700043c0 con 0x7f2c741018b0 2024-08-18T23:44:52.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.174+0000 7f2c7c03c640 1 -- 172.21.15.78:0/786509297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7419d350 con 0x7f2c741018b0 2024-08-18T23:44:52.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.174+0000 7f2c6affd640 1 -- 172.21.15.78:0/786509297 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c700049e0 con 0x7f2c741018b0 2024-08-18T23:44:52.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.174+0000 7f2c7c03c640 1 -- 172.21.15.78:0/786509297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c741a4c30 con 0x7f2c741018b0 2024-08-18T23:44:52.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.177+0000 7f2c6affd640 1 -- 172.21.15.78:0/786509297 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f2c700079b0 con 0x7f2c741018b0 2024-08-18T23:44:52.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.178+0000 7f2c7e43a740 1 Processor -- start 2024-08-18T23:44:52.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.178+0000 7f2c7e43a740 1 -- start start 2024-08-18T23:44:52.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.178+0000 7f2c6affd640 1 --2- 172.21.15.78:0/786509297 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2c600a2d10 0x7f2c600a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:52.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.179+0000 7f2c6affd640 1 -- 172.21.15.78:0/786509297 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2c700cf610 con 0x7f2c741018b0 2024-08-18T23:44:52.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.179+0000 7f2c7a5d9640 1 --2- 172.21.15.78:0/786509297 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2c600a2d10 0x7f2c600a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:52.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.179+0000 7f2c7e43a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6202ff0 0x564ef62033d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:52.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.179+0000 7f2c7e43a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6203910 0x564ef620dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:52.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.179+0000 7f2c7e43a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef620e1d0 0x564ef62105c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:52.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.179+0000 7f2c7e43a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564ef6168a40 con 0x564ef620e1d0 2024-08-18T23:44:52.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.179+0000 7f2c7e43a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564ef61009d0 con 0x564ef6203910 2024-08-18T23:44:52.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.179+0000 7f2c7e43a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564ef6164a00 con 0x564ef6202ff0 2024-08-18T23:44:52.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.179+0000 7f2c795d7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6203910 0x564ef620dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:52.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.180+0000 7f2c795d7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6203910 0x564ef620dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:51942/0 (socket says 172.21.15.78:51942) 2024-08-18T23:44:52.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.180+0000 7f2c795d7640 1 -- 172.21.15.78:0/9288054 learned_addr learned my addr 172.21.15.78:0/9288054 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:52.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.180+0000 7f2c79dd8640 1 --2- 172.21.15.78:0/9288054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6202ff0 0x564ef62033d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:52.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.180+0000 7f2c795d7640 1 -- 172.21.15.78:0/9288054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6202ff0 msgr2=0x564ef62033d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:52.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.180+0000 7f2c795d7640 1 --2- 172.21.15.78:0/9288054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6202ff0 0x564ef62033d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.180+0000 7f2c795d7640 1 -- 172.21.15.78:0/9288054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef620e1d0 msgr2=0x564ef62105c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T23:44:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.180+0000 7f2c795d7640 1 --2- 172.21.15.78:0/9288054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef620e1d0 0x564ef62105c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.180+0000 7f2c795d7640 1 -- 172.21.15.78:0/9288054 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef6107650 con 0x564ef6203910 2024-08-18T23:44:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.180+0000 7f2c7a5d9640 1 --2- 172.21.15.78:0/786509297 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2c600a2d10 0x7f2c600a51d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009a60 tx=0x7f2c6c009890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.180+0000 7f2c795d7640 1 --2- 172.21.15.78:0/9288054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6203910 0x564ef620dc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2c64008c10 tx=0x7f2c6400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fd5f9c0b2a95db1 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.180+0000 7f2c5ffff640 1 -- 172.21.15.78:0/9288054 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c64014070 con 0x564ef6203910 2024-08-18T23:44:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.180+0000 7f2c5ffff640 1 -- 172.21.15.78:0/9288054 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c640047a0 con 0x564ef6203910 2024-08-18T23:44:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.180+0000 7f2c5ffff640 1 -- 172.21.15.78:0/9288054 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c64010720 con 0x564ef6203910 2024-08-18T23:44:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.181+0000 7f2c7e43a740 1 -- 172.21.15.78:0/9288054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6203910 msgr2=0x564ef620dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.181+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/9288054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6203910 0x564ef620dc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2c64008c10 tx=0x7f2c6400e720 comp rx=0 tx=0).stop 2024-08-18T23:44:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.182+0000 7f2c7e43a740 1 -- 172.21.15.78:0/9288054 shutdown_connections 2024-08-18T23:44:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.182+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/9288054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef620e1d0 0x564ef62105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.182+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/9288054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6203910 0x564ef620dc90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.182+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/9288054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6202ff0 0x564ef62033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.182+0000 7f2c7e43a740 1 -- 172.21.15.78:0/9288054 >> 172.21.15.78:0/9288054 conn(0x564ef61ff580 msgr2=0x564ef61ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.182+0000 7f2c7e43a740 1 -- 172.21.15.78:0/9288054 shutdown_connections 2024-08-18T23:44:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.182+0000 7f2c7e43a740 1 -- 172.21.15.78:0/9288054 wait complete. 2024-08-18T23:44:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.182+0000 7f2c7e43a740 1 -- start start 2024-08-18T23:44:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.183+0000 7f2c7e43a740 1 Processor -- start 2024-08-18T23:44:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.183+0000 7f2c7e43a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6202ff0 0x564ef620b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.183+0000 7f2c7e43a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6203910 0x564ef620b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.183+0000 7f2c7e43a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef620e1d0 0x564ef620bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c7e43a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564ef6210df0 con 0x564ef620e1d0 2024-08-18T23:44:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c7e43a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564ef6123b20 con 0x564ef6202ff0 2024-08-18T23:44:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c7e43a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564ef6210f70 con 0x564ef6203910 2024-08-18T23:44:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c79dd8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6202ff0 0x564ef620b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c7e43a740 10 client.0 _ll_register_callbacks cb 0x564ef6121bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:52.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c79dd8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6202ff0 0x564ef620b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:51958/0 (socket says 172.21.15.78:51958) 2024-08-18T23:44:52.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c79dd8640 1 -- 172.21.15.78:0/600499876 learned_addr learned my addr 172.21.15.78:0/600499876 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:52.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c795d7640 1 --2- 172.21.15.78:0/600499876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6203910 0x564ef620b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:52.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c7a5d9640 1 --2- 172.21.15.78:0/600499876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef620e1d0 0x564ef620bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:52.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c79dd8640 1 -- 172.21.15.78:0/600499876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6203910 msgr2=0x564ef620b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:52.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c79dd8640 1 --2- 172.21.15.78:0/600499876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6203910 0x564ef620b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c79dd8640 1 -- 172.21.15.78:0/600499876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef620e1d0 msgr2=0x564ef620bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:52.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c79dd8640 1 --2- 172.21.15.78:0/600499876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef620e1d0 0x564ef620bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c79dd8640 1 -- 172.21.15.78:0/600499876 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef5eda3c0 con 0x564ef6202ff0 2024-08-18T23:44:52.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c7a5d9640 1 --2- 172.21.15.78:0/600499876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef620e1d0 0x564ef620bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:44:52.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c795d7640 1 --2- 172.21.15.78:0/600499876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6203910 0x564ef620b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:52.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c79dd8640 1 --2- 172.21.15.78:0/600499876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6202ff0 0x564ef620b060 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2c70000c00 tx=0x7f2c7009c0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b1da96de4489e0a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:52.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:44:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c5d7fa640 1 -- 172.21.15.78:0/600499876 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c700b4060 con 0x564ef6202ff0 2024-08-18T23:44:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c5d7fa640 1 -- 172.21.15.78:0/600499876 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2c70007ca0 con 0x564ef6202ff0 2024-08-18T23:44:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.184+0000 7f2c7e43a740 1 -- 172.21.15.78:0/600499876 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c640047a0 con 0x564ef6202ff0 2024-08-18T23:44:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.185+0000 7f2c5d7fa640 1 -- 172.21.15.78:0/600499876 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2c7001d040 con 0x564ef6202ff0 2024-08-18T23:44:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.185+0000 7f2c7e43a740 10 client.25016 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.185+0000 7f2c7e43a740 1 -- 172.21.15.78:0/600499876 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564ef621ca50 con 0x564ef6202ff0 2024-08-18T23:44:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.185+0000 7f2c5d7fa640 1 -- 172.21.15.78:0/600499876 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c640047a0 con 0x564ef6202ff0 2024-08-18T23:44:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.185+0000 7f2c5d7fa640 1 -- 172.21.15.78:0/600499876 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2c700ba170 con 0x564ef6202ff0 2024-08-18T23:44:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.185+0000 7f2c7e43a740 10 client.25016 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.185+0000 7f2c7e43a740 10 client.25016 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:44:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.185+0000 7f2c7e43a740 20 client.25016 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:52.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.185+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/600499876 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x564ef621e320 0x564ef6220750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:52.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.185+0000 7f2c7e43a740 4 client.25016 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:44:52.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.185+0000 7f2c7e43a740 1 -- 172.21.15.78:0/600499876 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ef6175fb0 con 0x564ef621e320 2024-08-18T23:44:52.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.186+0000 7f2c795d7640 1 --2- 172.21.15.78:0/600499876 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x564ef621e320 0x564ef6220750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:52.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.186+0000 7f2c5d7fa640 10 client.25016 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:44:52.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.186+0000 7f2c795d7640 1 --2- 172.21.15.78:0/600499876 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x564ef621e320 0x564ef6220750 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed951ffd7c7dbd9d server_cookie=7e10423bc299a2e6 in_seq=0 out_seq=0 2024-08-18T23:44:52.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.187+0000 7f2c5d7fa640 1 -- 172.21.15.78:0/600499876 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ef6175fb0 con 0x564ef621e320 2024-08-18T23:44:52.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.187+0000 7f2c5d7fa640 10 client.25016 handle_command_reply: tid=0 2024-08-18T23:44:52.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.187+0000 7f2c7e43a740 1 client.25016 shutdown 2024-08-18T23:44:52.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.187+0000 7f2c7e43a740 1 -- 172.21.15.78:0/600499876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6202ff0 msgr2=0x564ef620b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:52.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.187+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/600499876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6202ff0 0x564ef620b060 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2c70000c00 tx=0x7f2c7009c0e0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.187+0000 7f2c7e43a740 1 -- 172.21.15.78:0/600499876 shutdown_connections 2024-08-18T23:44:52.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.187+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/600499876 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x564ef621e320 0x564ef6220750 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.187+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/600499876 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef620e1d0 0x564ef620bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.187+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/600499876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6203910 0x564ef620b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.187+0000 7f2c5d7fa640 0 client.25016 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:44:52.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.187+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/600499876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6202ff0 0x564ef620b060 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.187+0000 7f2c7e43a740 1 -- 172.21.15.78:0/600499876 >> 172.21.15.78:0/600499876 conn(0x564ef61ff580 msgr2=0x564ef6205260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:52.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.188+0000 7f2c7e43a740 1 -- 172.21.15.78:0/600499876 shutdown_connections 2024-08-18T23:44:52.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.188+0000 7f2c7e43a740 1 -- 172.21.15.78:0/600499876 wait complete. 2024-08-18T23:44:52.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.188+0000 7f2c7e43a740 20 client.25016 trim_cache size 0 max 16384 2024-08-18T23:44:52.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.212+0000 7f2c7e43a740 1 Processor -- start 2024-08-18T23:44:52.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.212+0000 7f2c7e43a740 1 -- start start 2024-08-18T23:44:52.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.213+0000 7f2c7e43a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6202ff0 0x564ef620b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:52.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.213+0000 7f2c7e43a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6203910 0x564ef6226060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:52.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.213+0000 7f2c7e43a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef620e1d0 0x564ef62265a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:52.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.213+0000 7f2c7e43a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c741af120 con 0x564ef620e1d0 2024-08-18T23:44:52.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.213+0000 7f2c7e43a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c741ac9f0 con 0x564ef6202ff0 2024-08-18T23:44:52.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.213+0000 7f2c7e43a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c741aa840 con 0x564ef6203910 2024-08-18T23:44:52.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.213+0000 7f2c79dd8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6202ff0 0x564ef620b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:52.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.213+0000 7f2c7a5d9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef620e1d0 0x564ef62265a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:52.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.213+0000 7f2c7a5d9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef620e1d0 0x564ef62265a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43022/0 (socket says 172.21.15.78:43022) 2024-08-18T23:44:52.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.213+0000 7f2c79dd8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6202ff0 0x564ef620b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:51962/0 (socket says 172.21.15.78:51962) 2024-08-18T23:44:52.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.213+0000 7f2c7a5d9640 1 -- 172.21.15.78:0/3157022028 learned_addr learned my addr 172.21.15.78:0/3157022028 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:52.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.213+0000 7f2c795d7640 1 --2- 172.21.15.78:0/3157022028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6203910 0x564ef6226060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:52.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.213+0000 7f2c7a5d9640 1 -- 172.21.15.78:0/3157022028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6203910 msgr2=0x564ef6226060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:52.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.213+0000 7f2c7a5d9640 1 --2- 172.21.15.78:0/3157022028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6203910 0x564ef6226060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.213+0000 7f2c7a5d9640 1 -- 172.21.15.78:0/3157022028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6202ff0 msgr2=0x564ef620b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:52.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.213+0000 7f2c7a5d9640 1 --2- 172.21.15.78:0/3157022028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6202ff0 0x564ef620b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.213+0000 7f2c7a5d9640 1 -- 172.21.15.78:0/3157022028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef6203740 con 0x564ef620e1d0 2024-08-18T23:44:52.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.213+0000 7f2c79dd8640 1 --2- 172.21.15.78:0/3157022028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6202ff0 0x564ef620b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.213+0000 7f2c795d7640 1 --2- 172.21.15.78:0/3157022028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6203910 0x564ef6226060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:44:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.214+0000 7f2c7a5d9640 1 --2- 172.21.15.78:0/3157022028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef620e1d0 0x564ef62265a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c035000 tx=0x7f2c6c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b096ba3a2fc4de server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.214+0000 7f2c5f7fe640 1 -- 172.21.15.78:0/3157022028 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c041070 con 0x564ef620e1d0 2024-08-18T23:44:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.214+0000 7f2c5f7fe640 1 -- 172.21.15.78:0/3157022028 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c6c036040 con 0x564ef620e1d0 2024-08-18T23:44:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.214+0000 7f2c5f7fe640 1 -- 172.21.15.78:0/3157022028 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c030a60 con 0x564ef620e1d0 2024-08-18T23:44:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.215+0000 7f2c7e43a740 1 -- 172.21.15.78:0/3157022028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef620e1d0 msgr2=0x564ef62265a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.215+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/3157022028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef620e1d0 0x564ef62265a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c035000 tx=0x7f2c6c03d000 comp rx=0 tx=0).stop 2024-08-18T23:44:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.215+0000 7f2c7e43a740 1 -- 172.21.15.78:0/3157022028 shutdown_connections 2024-08-18T23:44:52.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.215+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/3157022028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef620e1d0 0x564ef62265a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.215+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/3157022028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6203910 0x564ef6226060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.215+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/3157022028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef6202ff0 0x564ef620b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.215+0000 7f2c7e43a740 1 -- 172.21.15.78:0/3157022028 >> 172.21.15.78:0/3157022028 conn(0x564ef61ff580 msgr2=0x564ef620a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:52.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.215+0000 7f2c7e43a740 1 -- 172.21.15.78:0/3157022028 shutdown_connections 2024-08-18T23:44:52.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.215+0000 7f2c7e43a740 1 -- 172.21.15.78:0/3157022028 wait complete. 2024-08-18T23:44:52.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.215+0000 7f2c7e43a740 1 -- start start 2024-08-18T23:44:52.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.215+0000 7f2c7e43a740 1 Processor -- start 2024-08-18T23:44:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.216+0000 7f2c7e43a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6202ff0 0x564ef6216210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.216+0000 7f2c7e43a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef6203910 0x564ef6216750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.216+0000 7f2c7e43a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef620e1d0 0x564ef6207470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.216+0000 7f2c7e43a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c741aaeb0 con 0x564ef6203910 2024-08-18T23:44:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.216+0000 7f2c7e43a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c741ac320 con 0x564ef620e1d0 2024-08-18T23:44:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.216+0000 7f2c7e43a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c741ab580 con 0x564ef6202ff0 2024-08-18T23:44:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.216+0000 7f2c7e43a740 10 client.0 _ll_register_callbacks cb 0x7f2c700058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.216+0000 7f2c795d7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef6203910 0x564ef6216750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.216+0000 7f2c795d7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef6203910 0x564ef6216750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43036/0 (socket says 172.21.15.78:43036) 2024-08-18T23:44:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.216+0000 7f2c795d7640 1 -- 172.21.15.78:0/3481792393 learned_addr learned my addr 172.21.15.78:0/3481792393 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.216+0000 7f2c79dd8640 1 --2- 172.21.15.78:0/3481792393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6202ff0 0x564ef6216210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.216+0000 7f2c795d7640 1 -- 172.21.15.78:0/3481792393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6202ff0 msgr2=0x564ef6216210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.216+0000 7f2c795d7640 1 --2- 172.21.15.78:0/3481792393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6202ff0 0x564ef6216210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.216+0000 7f2c795d7640 1 -- 172.21.15.78:0/3481792393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef620e1d0 msgr2=0x564ef6207470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.216+0000 7f2c795d7640 1 --2- 172.21.15.78:0/3481792393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef620e1d0 0x564ef6207470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.216+0000 7f2c795d7640 1 -- 172.21.15.78:0/3481792393 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef62079b0 con 0x564ef6203910 2024-08-18T23:44:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.216+0000 7f2c795d7640 1 --2- 172.21.15.78:0/3481792393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef6203910 0x564ef6216750 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2c64008b60 tx=0x7f2c640083f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ebd0e40a106054 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.216+0000 7f2c5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:44:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.217+0000 7f2c5d7fa640 1 -- 172.21.15.78:0/3481792393 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c64014040 con 0x564ef6203910 2024-08-18T23:44:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.217+0000 7f2c7e43a740 1 -- 172.21.15.78:0/3481792393 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c6c036040 con 0x564ef6203910 2024-08-18T23:44:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.217+0000 7f2c5d7fa640 1 -- 172.21.15.78:0/3481792393 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2c64010840 con 0x564ef6203910 2024-08-18T23:44:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.217+0000 7f2c5d7fa640 1 -- 172.21.15.78:0/3481792393 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2c64010a40 con 0x564ef6203910 2024-08-18T23:44:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.217+0000 7f2c7e43a740 10 client.15522 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.217+0000 7f2c7e43a740 1 -- 172.21.15.78:0/3481792393 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564ef6290f30 con 0x564ef6203910 2024-08-18T23:44:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.217+0000 7f2c5d7fa640 1 -- 172.21.15.78:0/3481792393 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c6c036040 con 0x564ef6203910 2024-08-18T23:44:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.217+0000 7f2c5d7fa640 1 -- 172.21.15.78:0/3481792393 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2c64018410 con 0x564ef6203910 2024-08-18T23:44:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.217+0000 7f2c7e43a740 10 client.15522 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.217+0000 7f2c7e43a740 10 client.15522 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:44:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.217+0000 7f2c7e43a740 20 client.15522 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.217+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/3481792393 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x564ef621e320 0x564ef6293610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.217+0000 7f2c7e43a740 4 client.15522 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:44:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.217+0000 7f2c7e43a740 1 -- 172.21.15.78:0/3481792393 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ef61704c0 con 0x564ef621e320 2024-08-18T23:44:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.218+0000 7f2c79dd8640 1 --2- 172.21.15.78:0/3481792393 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x564ef621e320 0x564ef6293610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:52.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.218+0000 7f2c5d7fa640 10 client.15522 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:44:52.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.218+0000 7f2c79dd8640 1 --2- 172.21.15.78:0/3481792393 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x564ef621e320 0x564ef6293610 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dfcf9352ea67850e server_cookie=eebc6b5bed6ae39d in_seq=0 out_seq=0 2024-08-18T23:44:52.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.219+0000 7f2c5d7fa640 1 -- 172.21.15.78:0/3481792393 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ef61704c0 con 0x564ef621e320 2024-08-18T23:44:52.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.219+0000 7f2c5d7fa640 10 client.15522 handle_command_reply: tid=0 2024-08-18T23:44:52.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.219+0000 7f2c7e43a740 1 client.15522 shutdown 2024-08-18T23:44:52.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.219+0000 7f2c7e43a740 1 -- 172.21.15.78:0/3481792393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef6203910 msgr2=0x564ef6216750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:52.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.219+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/3481792393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef6203910 0x564ef6216750 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2c64008b60 tx=0x7f2c640083f0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.219+0000 7f2c7e43a740 1 -- 172.21.15.78:0/3481792393 shutdown_connections 2024-08-18T23:44:52.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.219+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/3481792393 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x564ef621e320 0x564ef6293610 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.219+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/3481792393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ef620e1d0 0x564ef6207470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.219+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/3481792393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564ef6203910 0x564ef6216750 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.219+0000 7f2c7e43a740 1 --2- 172.21.15.78:0/3481792393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564ef6202ff0 0x564ef6216210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.219+0000 7f2c7e43a740 1 -- 172.21.15.78:0/3481792393 >> 172.21.15.78:0/3481792393 conn(0x564ef61ff580 msgr2=0x564ef6204a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:52.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.219+0000 7f2c5d7fa640 0 client.15522 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:44:52.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.219+0000 7f2c7e43a740 1 -- 172.21.15.78:0/3481792393 shutdown_connections 2024-08-18T23:44:52.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.219+0000 7f2c7e43a740 1 -- 172.21.15.78:0/3481792393 wait complete. 2024-08-18T23:44:52.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.220+0000 7f2c7e43a740 20 client.15522 trim_cache size 0 max 16384 2024-08-18T23:44:52.225 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:44:52.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.220+0000 7f2c7c03c640 1 -- 172.21.15.78:0/786509297 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2c600a2d10 msgr2=0x7f2c600a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:52.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.220+0000 7f2c7c03c640 1 --2- 172.21.15.78:0/786509297 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2c600a2d10 0x7f2c600a51d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009a60 tx=0x7f2c6c009890 comp rx=0 tx=0).stop 2024-08-18T23:44:52.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.220+0000 7f2c7c03c640 1 -- 172.21.15.78:0/786509297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c741018b0 msgr2=0x7f2c741980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:52.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.220+0000 7f2c7c03c640 1 --2- 172.21.15.78:0/786509297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c741018b0 0x7f2c741980b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000c890 tx=0x7f2c7000cd50 comp rx=0 tx=0).stop 2024-08-18T23:44:52.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.220+0000 7f2c7c03c640 1 -- 172.21.15.78:0/786509297 shutdown_connections 2024-08-18T23:44:52.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.220+0000 7f2c7c03c640 1 --2- 172.21.15.78:0/786509297 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2c600a2d10 0x7f2c600a51d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.220+0000 7f2c7c03c640 1 --2- 172.21.15.78:0/786509297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c741078b0 0x7f2c7419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.220+0000 7f2c7c03c640 1 --2- 172.21.15.78:0/786509297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c74102250 0x7f2c741985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.220+0000 7f2c7c03c640 1 --2- 172.21.15.78:0/786509297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c741018b0 0x7f2c741980b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:52.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.220+0000 7f2c7c03c640 1 -- 172.21.15.78:0/786509297 >> 172.21.15.78:0/786509297 conn(0x7f2c740fd5f0 msgr2=0x7f2c740fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:52.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.220+0000 7f2c7c03c640 1 -- 172.21.15.78:0/786509297 shutdown_connections 2024-08-18T23:44:52.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:52.220+0000 7f2c7c03c640 1 -- 172.21.15.78:0/786509297 wait complete. 2024-08-18T23:44:53.061 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-18T23:44:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:53.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:53 smithi078 ceph-mon[18440]: pgmap v367: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-18T23:44:53.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:53 smithi123 ceph-mon[21808]: pgmap v367: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-18T23:44:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:53 smithi161 ceph-mon[22027]: pgmap v367: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-18T23:44:53.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.967+0000 7f7d5d381640 1 -- 172.21.15.78:0/3283111571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d581068b0 msgr2=0x7f7d58106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:53.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.967+0000 7f7d5d381640 1 --2- 172.21.15.78:0/3283111571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d581068b0 0x7f7d58106c90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009900 tx=0x7f7d4802a410 comp rx=0 tx=0).stop 2024-08-18T23:44:53.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.967+0000 7f7d5d381640 1 -- 172.21.15.78:0/3283111571 shutdown_connections 2024-08-18T23:44:53.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.967+0000 7f7d5d381640 1 --2- 172.21.15.78:0/3283111571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d581007d0 0x7f7d5810e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:53.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.967+0000 7f7d5d381640 1 --2- 172.21.15.78:0/3283111571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d580ffe30 0x7f7d58100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:53.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.967+0000 7f7d5d381640 1 --2- 172.21.15.78:0/3283111571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d581068b0 0x7f7d58106c90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:53.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.967+0000 7f7d5d381640 1 -- 172.21.15.78:0/3283111571 >> 172.21.15.78:0/3283111571 conn(0x7f7d58076f60 msgr2=0x7f7d580fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:53.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.967+0000 7f7d5d381640 1 -- 172.21.15.78:0/3283111571 shutdown_connections 2024-08-18T23:44:53.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.967+0000 7f7d5d381640 1 -- 172.21.15.78:0/3283111571 wait complete. 2024-08-18T23:44:53.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.967+0000 7f7d5d381640 1 Processor -- start 2024-08-18T23:44:53.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.968+0000 7f7d5d381640 1 -- start start 2024-08-18T23:44:53.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.968+0000 7f7d5d381640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d580ffe30 0x7f7d58198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:53.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.968+0000 7f7d5d381640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d581007d0 0x7f7d58198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:53.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.968+0000 7f7d5d381640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d581068b0 0x7f7d5819c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:53.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.968+0000 7f7d5d381640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7d58111da0 con 0x7f7d580ffe30 2024-08-18T23:44:53.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.968+0000 7f7d5d381640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d58111c20 con 0x7f7d581068b0 2024-08-18T23:44:53.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.968+0000 7f7d5d381640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7d58111f20 con 0x7f7d581007d0 2024-08-18T23:44:53.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.968+0000 7f7d56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d580ffe30 0x7f7d58198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:53.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.968+0000 7f7d56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d580ffe30 0x7f7d58198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43060/0 (socket says 172.21.15.78:43060) 2024-08-18T23:44:53.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.968+0000 7f7d56ffd640 1 -- 172.21.15.78:0/4024523459 learned_addr learned my addr 172.21.15.78:0/4024523459 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:53.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.968+0000 7f7d4ffff640 1 --2- 172.21.15.78:0/4024523459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d581007d0 0x7f7d58198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:53.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.968+0000 7f7d577fe640 1 --2- 172.21.15.78:0/4024523459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d581068b0 0x7f7d5819c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:53.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.969+0000 7f7d4ffff640 1 -- 172.21.15.78:0/4024523459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d581068b0 msgr2=0x7f7d5819c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:53.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.969+0000 7f7d4ffff640 1 --2- 172.21.15.78:0/4024523459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d581068b0 0x7f7d5819c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:53.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.969+0000 7f7d4ffff640 1 -- 172.21.15.78:0/4024523459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d580ffe30 msgr2=0x7f7d58198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:53.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.969+0000 7f7d4ffff640 1 --2- 172.21.15.78:0/4024523459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d580ffe30 0x7f7d58198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:53.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.969+0000 7f7d4ffff640 1 -- 172.21.15.78:0/4024523459 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5819d0d0 con 0x7f7d581007d0 2024-08-18T23:44:53.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.969+0000 7f7d56ffd640 1 --2- 172.21.15.78:0/4024523459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d580ffe30 0x7f7d58198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:44:53.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.969+0000 7f7d4ffff640 1 --2- 172.21.15.78:0/4024523459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d581007d0 0x7f7d58198660 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000c970 tx=0x7f7d4000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:53.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.969+0000 7f7d54ff9640 1 -- 172.21.15.78:0/4024523459 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40007bb0 con 0x7f7d581007d0 2024-08-18T23:44:53.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.969+0000 7f7d54ff9640 1 -- 172.21.15.78:0/4024523459 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d40007d50 con 0x7f7d581007d0 2024-08-18T23:44:53.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.969+0000 7f7d5d381640 1 -- 172.21.15.78:0/4024523459 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5819d3c0 con 0x7f7d581007d0 2024-08-18T23:44:53.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.969+0000 7f7d54ff9640 1 -- 172.21.15.78:0/4024523459 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40018820 con 0x7f7d581007d0 2024-08-18T23:44:53.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.970+0000 7f7d5d381640 1 -- 172.21.15.78:0/4024523459 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d58100f40 con 0x7f7d581007d0 2024-08-18T23:44:53.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.973+0000 7f7d5f77f740 1 Processor -- start 2024-08-18T23:44:53.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.973+0000 7f7d54ff9640 1 -- 172.21.15.78:0/4024523459 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f7d400189c0 con 0x7f7d581007d0 2024-08-18T23:44:53.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.974+0000 7f7d5f77f740 1 -- start start 2024-08-18T23:44:53.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.974+0000 7f7d54ff9640 1 --2- 172.21.15.78:0/4024523459 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7d440a2c90 0x7f7d440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:53.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.974+0000 7f7d54ff9640 1 -- 172.21.15.78:0/4024523459 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f7d400cf5a0 con 0x7f7d581007d0 2024-08-18T23:44:53.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.974+0000 7f7d577fe640 1 --2- 172.21.15.78:0/4024523459 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7d440a2c90 0x7f7d440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:53.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.974+0000 7f7d5f77f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1accff0 0x55c8c1acd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:53.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.974+0000 7f7d5f77f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1acd910 0x55c8c1ad7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:53.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.974+0000 7f7d5f77f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1ad81d0 0x55c8c1ada5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:53.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.975+0000 7f7d56ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1accff0 0x55c8c1acd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:53.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.975+0000 7f7d5f77f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c8c1a32a40 con 0x55c8c1ad81d0 2024-08-18T23:44:53.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.975+0000 7f7d5f77f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c8c19ca9d0 con 0x55c8c1acd910 2024-08-18T23:44:53.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.975+0000 7f7d5f77f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c8c1a2ea00 con 0x55c8c1accff0 2024-08-18T23:44:53.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.975+0000 7f7d56ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1accff0 0x55c8c1acd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:48828/0 (socket says 172.21.15.78:48828) 2024-08-18T23:44:53.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.975+0000 7f7d56ffd640 1 -- 172.21.15.78:0/2102799977 learned_addr learned my addr 172.21.15.78:0/2102799977 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:53.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.975+0000 7f7d577fe640 1 --2- 172.21.15.78:0/2102799977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1ad81d0 0x55c8c1ada5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:53.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.975+0000 7f7d4ffff640 1 --2- 172.21.15.78:0/2102799977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1acd910 0x55c8c1ad7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:53.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.975+0000 7f7d4ffff640 1 -- 172.21.15.78:0/2102799977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1accff0 msgr2=0x55c8c1acd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:53.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.975+0000 7f7d4ffff640 1 --2- 172.21.15.78:0/2102799977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1accff0 0x55c8c1acd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:53.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.975+0000 7f7d4ffff640 1 -- 172.21.15.78:0/2102799977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1ad81d0 msgr2=0x55c8c1ada5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:53.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.975+0000 7f7d4ffff640 1 --2- 172.21.15.78:0/2102799977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1ad81d0 0x55c8c1ada5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:53.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.975+0000 7f7d4ffff640 1 -- 172.21.15.78:0/2102799977 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8c19d1650 con 0x55c8c1acd910 2024-08-18T23:44:53.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.976+0000 7f7d56ffd640 1 --2- 172.21.15.78:0/2102799977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1accff0 0x55c8c1acd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:53.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.976+0000 7f7d577fe640 1 --2- 172.21.15.78:0/2102799977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1ad81d0 0x55c8c1ada5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:44:53.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.976+0000 7f7d4ffff640 1 --2- 172.21.15.78:0/2102799977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1acd910 0x55c8c1ad7c90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7d400ccb60 tx=0x7f7d4009bfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7239619f618c9404 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:53.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.976+0000 7f7d577fe640 1 --2- 172.21.15.78:0/4024523459 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7d440a2c90 0x7f7d440a5150 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7d581074f0 tx=0x7f7d50007810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:53.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.976+0000 7f7d4d7fa640 1 -- 172.21.15.78:0/2102799977 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d400b4070 con 0x55c8c1acd910 2024-08-18T23:44:53.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.976+0000 7f7d4d7fa640 1 -- 172.21.15.78:0/2102799977 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d4009fc60 con 0x55c8c1acd910 2024-08-18T23:44:53.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.976+0000 7f7d4d7fa640 1 -- 172.21.15.78:0/2102799977 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d400ce030 con 0x55c8c1acd910 2024-08-18T23:44:53.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.977+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2102799977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1acd910 msgr2=0x55c8c1ad7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:53.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.977+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/2102799977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1acd910 0x55c8c1ad7c90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7d400ccb60 tx=0x7f7d4009bfb0 comp rx=0 tx=0).stop 2024-08-18T23:44:53.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.977+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2102799977 shutdown_connections 2024-08-18T23:44:53.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.977+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/2102799977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1ad81d0 0x55c8c1ada5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:53.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.977+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/2102799977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1acd910 0x55c8c1ad7c90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:53.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.977+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/2102799977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1accff0 0x55c8c1acd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:53.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.977+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2102799977 >> 172.21.15.78:0/2102799977 conn(0x55c8c1ac9580 msgr2=0x55c8c1ac9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:53.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.977+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2102799977 shutdown_connections 2024-08-18T23:44:53.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.977+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2102799977 wait complete. 2024-08-18T23:44:53.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.978+0000 7f7d5f77f740 1 -- start start 2024-08-18T23:44:53.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.978+0000 7f7d5f77f740 1 Processor -- start 2024-08-18T23:44:53.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.978+0000 7f7d5f77f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1accff0 0x55c8c1ad50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:53.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.979+0000 7f7d56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1accff0 0x55c8c1ad50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:53.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.979+0000 7f7d56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1accff0 0x55c8c1ad50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43082/0 (socket says 172.21.15.78:43082) 2024-08-18T23:44:53.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.979+0000 7f7d5f77f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1acd910 0x55c8c1ad5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:53.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.979+0000 7f7d56ffd640 1 -- 172.21.15.78:0/2682186909 learned_addr learned my addr 172.21.15.78:0/2682186909 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:53.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.979+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/2682186909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1ad81d0 0x55c8c1ad5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:53.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.979+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2682186909 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c8c1adada0 con 0x55c8c1accff0 2024-08-18T23:44:53.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.979+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2682186909 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c8c19edb20 con 0x55c8c1acd910 2024-08-18T23:44:53.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.979+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2682186909 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c8c1adaf20 con 0x55c8c1ad81d0 2024-08-18T23:44:53.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.979+0000 7f7d4ffff640 1 --2- 172.21.15.78:0/2682186909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1acd910 0x55c8c1ad5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:53.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.979+0000 7f7d56ffd640 1 -- 172.21.15.78:0/2682186909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1ad81d0 msgr2=0x55c8c1ad5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:53.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.979+0000 7f7d56ffd640 1 --2- 172.21.15.78:0/2682186909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1ad81d0 0x55c8c1ad5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:53.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.979+0000 7f7d56ffd640 1 -- 172.21.15.78:0/2682186909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1acd910 msgr2=0x55c8c1ad5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:53.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.979+0000 7f7d56ffd640 1 --2- 172.21.15.78:0/2682186909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1acd910 0x55c8c1ad5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:53.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.979+0000 7f7d56ffd640 1 -- 172.21.15.78:0/2682186909 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8c17a43c0 con 0x55c8c1accff0 2024-08-18T23:44:53.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.980+0000 7f7d5f77f740 10 client.0 _ll_register_callbacks cb 0x55c8c19ebbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:53.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.980+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2682186909 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d4009fc60 con 0x55c8c1accff0 2024-08-18T23:44:53.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.980+0000 7f7d56ffd640 1 --2- 172.21.15.78:0/2682186909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1accff0 0x55c8c1ad50e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7d48005a50 tx=0x7f7d48005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369cbd367e225092 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:53.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.980+0000 7f7d3e7fc640 10 client.15534 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:44:53.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.980+0000 7f7d3e7fc640 1 -- 172.21.15.78:0/2682186909 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4804b050 con 0x55c8c1accff0 2024-08-18T23:44:53.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.980+0000 7f7d3e7fc640 1 -- 172.21.15.78:0/2682186909 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f7d48004ad0 con 0x55c8c1accff0 2024-08-18T23:44:53.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.980+0000 7f7d3e7fc640 1 -- 172.21.15.78:0/2682186909 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d48035040 con 0x55c8c1accff0 2024-08-18T23:44:53.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.980+0000 7f7d5f77f740 10 client.15534 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:53.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.980+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2682186909 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55c8c1ae6800 con 0x55c8c1accff0 2024-08-18T23:44:53.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.981+0000 7f7d3e7fc640 1 -- 172.21.15.78:0/2682186909 <== mon.0 v2:172.21.15.78:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7d48004d10 con 0x55c8c1accff0 2024-08-18T23:44:53.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.981+0000 7f7d5f77f740 10 client.15534 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:53.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.981+0000 7f7d5f77f740 10 client.15534 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:44:53.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.981+0000 7f7d5f77f740 20 client.15534 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:53.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.981+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/2682186909 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55c8c1ae7c90 0x55c8c1aea120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:53.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.981+0000 7f7d5f77f740 4 client.15534 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:44:53.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.981+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2682186909 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8c1a3ffb0 con 0x55c8c1ae7c90 2024-08-18T23:44:53.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.981+0000 7f7d4ffff640 1 --2- 172.21.15.78:0/2682186909 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55c8c1ae7c90 0x55c8c1aea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:53.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.982+0000 7f7d4ffff640 1 --2- 172.21.15.78:0/2682186909 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55c8c1ae7c90 0x55c8c1aea120 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41598bd236702a55 server_cookie=2c84c649cfd9e05b in_seq=0 out_seq=0 2024-08-18T23:44:53.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.982+0000 7f7d3e7fc640 10 client.15534 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:44:53.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.982+0000 7f7d3e7fc640 1 -- 172.21.15.78:0/2682186909 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8c1a3ffb0 con 0x55c8c1ae7c90 2024-08-18T23:44:53.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.982+0000 7f7d3e7fc640 10 client.15534 handle_command_reply: tid=0 2024-08-18T23:44:53.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.982+0000 7f7d5f77f740 1 client.15534 shutdown 2024-08-18T23:44:53.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.983+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2682186909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1accff0 msgr2=0x55c8c1ad50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:53.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.983+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/2682186909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1accff0 0x55c8c1ad50e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7d48005a50 tx=0x7f7d48005360 comp rx=0 tx=0).stop 2024-08-18T23:44:53.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.983+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2682186909 shutdown_connections 2024-08-18T23:44:53.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.983+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/2682186909 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55c8c1ae7c90 0x55c8c1aea120 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:53.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.983+0000 7f7d3e7fc640 0 client.15534 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:44:53.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.983+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/2682186909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1ad81d0 0x55c8c1ad5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:53.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.983+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/2682186909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1acd910 0x55c8c1ad5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:53.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.983+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/2682186909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1accff0 0x55c8c1ad50e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:53.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.983+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2682186909 >> 172.21.15.78:0/2682186909 conn(0x55c8c1ac9580 msgr2=0x55c8c1acf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:53.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.983+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2682186909 shutdown_connections 2024-08-18T23:44:53.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.983+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2682186909 wait complete. 2024-08-18T23:44:53.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:53.983+0000 7f7d5f77f740 20 client.15534 trim_cache size 0 max 16384 2024-08-18T23:44:54.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.008+0000 7f7d5f77f740 1 Processor -- start 2024-08-18T23:44:54.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.008+0000 7f7d5f77f740 1 -- start start 2024-08-18T23:44:54.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.008+0000 7f7d5f77f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1accff0 0x55c8c1acfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:54.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.008+0000 7f7d5f77f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1acd910 0x55c8c1ad6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:54.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.008+0000 7f7d5f77f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1ad81d0 0x55c8c1aefa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:54.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.009+0000 7f7d5f77f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581aeff0 con 0x55c8c1accff0 2024-08-18T23:44:54.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.009+0000 7f7d5f77f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581ac8c0 con 0x55c8c1acd910 2024-08-18T23:44:54.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.009+0000 7f7d5f77f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581aa710 con 0x55c8c1ad81d0 2024-08-18T23:44:54.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.009+0000 7f7d56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1accff0 0x55c8c1acfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:54.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.009+0000 7f7d56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1accff0 0x55c8c1acfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43088/0 (socket says 172.21.15.78:43088) 2024-08-18T23:44:54.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.009+0000 7f7d56ffd640 1 -- 172.21.15.78:0/2637465201 learned_addr learned my addr 172.21.15.78:0/2637465201 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:54.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.009+0000 7f7d4ffff640 1 --2- 172.21.15.78:0/2637465201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1acd910 0x55c8c1ad6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:54.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.009+0000 7f7d577fe640 1 --2- 172.21.15.78:0/2637465201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1ad81d0 0x55c8c1aefa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:54.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.009+0000 7f7d4ffff640 1 -- 172.21.15.78:0/2637465201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1ad81d0 msgr2=0x55c8c1aefa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:54.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.009+0000 7f7d4ffff640 1 --2- 172.21.15.78:0/2637465201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1ad81d0 0x55c8c1aefa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:54.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.009+0000 7f7d4ffff640 1 -- 172.21.15.78:0/2637465201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1accff0 msgr2=0x55c8c1acfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:54.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.009+0000 7f7d4ffff640 1 --2- 172.21.15.78:0/2637465201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1accff0 0x55c8c1acfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:54.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.009+0000 7f7d4ffff640 1 -- 172.21.15.78:0/2637465201 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8c1acdf60 con 0x55c8c1acd910 2024-08-18T23:44:54.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.009+0000 7f7d56ffd640 1 --2- 172.21.15.78:0/2637465201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1accff0 0x55c8c1acfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:44:54.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.009+0000 7f7d4ffff640 1 --2- 172.21.15.78:0/2637465201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1acd910 0x55c8c1ad6f40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7d40099b60 tx=0x7f7d40086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35c449977c6686be server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.010+0000 7f7d4cff9640 1 -- 172.21.15.78:0/2637465201 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d400ce040 con 0x55c8c1acd910 2024-08-18T23:44:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.010+0000 7f7d4cff9640 1 -- 172.21.15.78:0/2637465201 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d40097f20 con 0x55c8c1acd910 2024-08-18T23:44:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.010+0000 7f7d4cff9640 1 -- 172.21.15.78:0/2637465201 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d400b4040 con 0x55c8c1acd910 2024-08-18T23:44:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.011+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2637465201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1acd910 msgr2=0x55c8c1ad6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.011+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/2637465201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1acd910 0x55c8c1ad6f40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7d40099b60 tx=0x7f7d40086040 comp rx=0 tx=0).stop 2024-08-18T23:44:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.011+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2637465201 shutdown_connections 2024-08-18T23:44:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.011+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/2637465201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1ad81d0 0x55c8c1aefa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.011+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/2637465201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1acd910 0x55c8c1ad6f40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.011+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/2637465201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1accff0 0x55c8c1acfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.011+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2637465201 >> 172.21.15.78:0/2637465201 conn(0x55c8c1ac9580 msgr2=0x55c8c1ad6830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.011+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2637465201 shutdown_connections 2024-08-18T23:44:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.011+0000 7f7d5f77f740 1 -- 172.21.15.78:0/2637465201 wait complete. 2024-08-18T23:44:54.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.012+0000 7f7d5f77f740 1 -- start start 2024-08-18T23:44:54.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.012+0000 7f7d5f77f740 1 Processor -- start 2024-08-18T23:44:54.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.012+0000 7f7d5f77f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1accff0 0x55c8c1ae08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:54.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.012+0000 7f7d5f77f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1acd910 0x55c8c1adbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:54.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.012+0000 7f7d4ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1acd910 0x55c8c1adbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:54.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.012+0000 7f7d4ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1acd910 0x55c8c1adbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43102/0 (socket says 172.21.15.78:43102) 2024-08-18T23:44:54.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.012+0000 7f7d5f77f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1ad81d0 0x55c8c1adc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:54.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.012+0000 7f7d4ffff640 1 -- 172.21.15.78:0/822971184 learned_addr learned my addr 172.21.15.78:0/822971184 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:54.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.012+0000 7f7d5f77f740 1 -- 172.21.15.78:0/822971184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581aad80 con 0x55c8c1acd910 2024-08-18T23:44:54.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.012+0000 7f7d5f77f740 1 -- 172.21.15.78:0/822971184 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581ac1f0 con 0x55c8c1ad81d0 2024-08-18T23:44:54.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.012+0000 7f7d5f77f740 1 -- 172.21.15.78:0/822971184 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581ab450 con 0x55c8c1accff0 2024-08-18T23:44:54.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.013+0000 7f7d577fe640 1 --2- 172.21.15.78:0/822971184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1ad81d0 0x55c8c1adc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:54.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.013+0000 7f7d4ffff640 1 -- 172.21.15.78:0/822971184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1accff0 msgr2=0x55c8c1ae08a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T23:44:54.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.013+0000 7f7d5f77f740 10 client.0 _ll_register_callbacks cb 0x7f7d48009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:54.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.013+0000 7f7d56ffd640 1 --2- 172.21.15.78:0/822971184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1accff0 0x55c8c1ae08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:54.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.013+0000 7f7d4ffff640 1 --2- 172.21.15.78:0/822971184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1accff0 0x55c8c1ae08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:54.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.013+0000 7f7d4ffff640 1 -- 172.21.15.78:0/822971184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1ad81d0 msgr2=0x55c8c1adc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:54.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.013+0000 7f7d4ffff640 1 --2- 172.21.15.78:0/822971184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1ad81d0 0x55c8c1adc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:54.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.013+0000 7f7d4ffff640 1 -- 172.21.15.78:0/822971184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8c1adc940 con 0x55c8c1acd910 2024-08-18T23:44:54.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.013+0000 7f7d577fe640 1 --2- 172.21.15.78:0/822971184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1ad81d0 0x55c8c1adc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:54.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.013+0000 7f7d5f77f740 1 -- 172.21.15.78:0/822971184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d40097f20 con 0x55c8c1acd910 2024-08-18T23:44:54.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.013+0000 7f7d4ffff640 1 --2- 172.21.15.78:0/822971184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1acd910 0x55c8c1adbbd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7d40098280 tx=0x7f7d4009b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5e2d541a6cbc3f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:54.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.013+0000 7f7d3e7fc640 10 client.15540 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:44:54.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.013+0000 7f7d3e7fc640 1 -- 172.21.15.78:0/822971184 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d400ce030 con 0x55c8c1acd910 2024-08-18T23:44:54.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.014+0000 7f7d3e7fc640 1 -- 172.21.15.78:0/822971184 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f7d40099bb0 con 0x55c8c1acd910 2024-08-18T23:44:54.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.014+0000 7f7d3e7fc640 1 -- 172.21.15.78:0/822971184 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d400b4050 con 0x55c8c1acd910 2024-08-18T23:44:54.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.014+0000 7f7d5f77f740 10 client.15540 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:54.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.014+0000 7f7d5f77f740 1 -- 172.21.15.78:0/822971184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55c8c1b528f0 con 0x55c8c1acd910 2024-08-18T23:44:54.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.014+0000 7f7d3e7fc640 1 -- 172.21.15.78:0/822971184 <== mon.0 v2:172.21.15.78:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7d400ccdb0 con 0x55c8c1acd910 2024-08-18T23:44:54.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.014+0000 7f7d5f77f740 10 client.15540 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:54.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.014+0000 7f7d5f77f740 10 client.15540 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:44:54.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.014+0000 7f7d5f77f740 20 client.15540 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:54.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.014+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/822971184 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55c8c1ae7c90 0x55c8c1b54a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:54.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.014+0000 7f7d5f77f740 4 client.15540 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:44:54.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.014+0000 7f7d5f77f740 1 -- 172.21.15.78:0/822971184 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c8c1a3a4c0 con 0x55c8c1ae7c90 2024-08-18T23:44:54.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.015+0000 7f7d56ffd640 1 --2- 172.21.15.78:0/822971184 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55c8c1ae7c90 0x55c8c1b54a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:54.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.015+0000 7f7d3e7fc640 10 client.15540 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:44:54.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.015+0000 7f7d56ffd640 1 --2- 172.21.15.78:0/822971184 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55c8c1ae7c90 0x55c8c1b54a10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=70d74c6dca5a1dca server_cookie=3d3f23138333e042 in_seq=0 out_seq=0 2024-08-18T23:44:54.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.015+0000 7f7d3e7fc640 1 -- 172.21.15.78:0/822971184 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c8c1a3a4c0 con 0x55c8c1ae7c90 2024-08-18T23:44:54.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.015+0000 7f7d3e7fc640 10 client.15540 handle_command_reply: tid=0 2024-08-18T23:44:54.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.016+0000 7f7d5f77f740 1 client.15540 shutdown 2024-08-18T23:44:54.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.016+0000 7f7d5f77f740 1 -- 172.21.15.78:0/822971184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1acd910 msgr2=0x55c8c1adbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:54.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.016+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/822971184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1acd910 0x55c8c1adbbd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7d40098280 tx=0x7f7d4009b770 comp rx=0 tx=0).stop 2024-08-18T23:44:54.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.016+0000 7f7d5f77f740 1 -- 172.21.15.78:0/822971184 shutdown_connections 2024-08-18T23:44:54.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.016+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/822971184 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55c8c1ae7c90 0x55c8c1b54a10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:54.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.016+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/822971184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8c1ad81d0 0x55c8c1adc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:54.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.016+0000 7f7d3e7fc640 0 client.15540 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:44:54.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.016+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/822971184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c8c1acd910 0x55c8c1adbbd0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:54.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.016+0000 7f7d5f77f740 1 --2- 172.21.15.78:0/822971184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c1accff0 0x55c8c1ae08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:54.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.016+0000 7f7d5f77f740 1 -- 172.21.15.78:0/822971184 >> 172.21.15.78:0/822971184 conn(0x55c8c1ac9580 msgr2=0x55c8c1ad5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:54.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.016+0000 7f7d5f77f740 1 -- 172.21.15.78:0/822971184 shutdown_connections 2024-08-18T23:44:54.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.016+0000 7f7d5f77f740 1 -- 172.21.15.78:0/822971184 wait complete. 2024-08-18T23:44:54.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.017+0000 7f7d5f77f740 20 client.15540 trim_cache size 0 max 16384 2024-08-18T23:44:54.021 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:44:54.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.019+0000 7f7d5d381640 1 -- 172.21.15.78:0/4024523459 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7d440a2c90 msgr2=0x7f7d440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:54.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.019+0000 7f7d5d381640 1 --2- 172.21.15.78:0/4024523459 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7d440a2c90 0x7f7d440a5150 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7d581074f0 tx=0x7f7d50007810 comp rx=0 tx=0).stop 2024-08-18T23:44:54.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.019+0000 7f7d5d381640 1 -- 172.21.15.78:0/4024523459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d581007d0 msgr2=0x7f7d58198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:54.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.019+0000 7f7d5d381640 1 --2- 172.21.15.78:0/4024523459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d581007d0 0x7f7d58198660 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000c970 tx=0x7f7d4000ce30 comp rx=0 tx=0).stop 2024-08-18T23:44:54.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.019+0000 7f7d5d381640 1 -- 172.21.15.78:0/4024523459 shutdown_connections 2024-08-18T23:44:54.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.019+0000 7f7d5d381640 1 --2- 172.21.15.78:0/4024523459 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7d440a2c90 0x7f7d440a5150 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:54.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.019+0000 7f7d5d381640 1 --2- 172.21.15.78:0/4024523459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d581068b0 0x7f7d5819c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:54.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.019+0000 7f7d5d381640 1 --2- 172.21.15.78:0/4024523459 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d581007d0 0x7f7d58198660 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:54.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.019+0000 7f7d5d381640 1 --2- 172.21.15.78:0/4024523459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d580ffe30 0x7f7d58198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:54.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.019+0000 7f7d5d381640 1 -- 172.21.15.78:0/4024523459 >> 172.21.15.78:0/4024523459 conn(0x7f7d58076f60 msgr2=0x7f7d58104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:54.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.019+0000 7f7d5d381640 1 -- 172.21.15.78:0/4024523459 shutdown_connections 2024-08-18T23:44:54.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:54.019+0000 7f7d5d381640 1 -- 172.21.15.78:0/4024523459 wait complete. 2024-08-18T23:44:54.836 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-18T23:44:55.135 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:55.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:55 smithi123 ceph-mon[21808]: pgmap v368: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-18T23:44:55.595 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:55 smithi078 ceph-mon[18440]: pgmap v368: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-18T23:44:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:55 smithi161 ceph-mon[22027]: pgmap v368: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-08-18T23:44:55.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.678+0000 7fa93fd13640 1 -- 172.21.15.78:0/3356501579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa938108030 msgr2=0x7fa93810e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:55.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.678+0000 7fa93fd13640 1 --2- 172.21.15.78:0/3356501579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa938108030 0x7fa93810e9a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa930009cd0 tx=0x7fa93002a8d0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.678+0000 7fa93fd13640 1 -- 172.21.15.78:0/3356501579 shutdown_connections 2024-08-18T23:44:55.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.678+0000 7fa93fd13640 1 --2- 172.21.15.78:0/3356501579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa938108030 0x7fa93810e9a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.678+0000 7fa93fd13640 1 --2- 172.21.15.78:0/3356501579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9381034b0 0x7fa9381078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.678+0000 7fa93fd13640 1 --2- 172.21.15.78:0/3356501579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa938102b00 0x7fa938102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.678+0000 7fa93fd13640 1 -- 172.21.15.78:0/3356501579 >> 172.21.15.78:0/3356501579 conn(0x7fa9380fc500 msgr2=0x7fa9380fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:55.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.678+0000 7fa93fd13640 1 -- 172.21.15.78:0/3356501579 shutdown_connections 2024-08-18T23:44:55.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.678+0000 7fa93fd13640 1 -- 172.21.15.78:0/3356501579 wait complete. 2024-08-18T23:44:55.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.679+0000 7fa93fd13640 1 Processor -- start 2024-08-18T23:44:55.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.679+0000 7fa93fd13640 1 -- start start 2024-08-18T23:44:55.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.679+0000 7fa93fd13640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa938102b00 0x7fa9381982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:55.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.679+0000 7fa93fd13640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9381034b0 0x7fa938198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:55.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.679+0000 7fa93fd13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa938108030 0x7fa93819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:55.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.679+0000 7fa93fd13640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa938111e50 con 0x7fa938108030 2024-08-18T23:44:55.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.680+0000 7fa93e2b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa938108030 0x7fa93819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:55.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.680+0000 7fa93d2ae640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9381034b0 0x7fa938198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:55.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.680+0000 7fa93fd13640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa938111cd0 con 0x7fa938102b00 2024-08-18T23:44:55.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.680+0000 7fa93d2ae640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9381034b0 0x7fa938198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:48876/0 (socket says 172.21.15.78:48876) 2024-08-18T23:44:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.680+0000 7fa93d2ae640 1 -- 172.21.15.78:0/2058894842 learned_addr learned my addr 172.21.15.78:0/2058894842 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.680+0000 7fa93fd13640 1 -- 172.21.15.78:0/2058894842 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa938111fd0 con 0x7fa9381034b0 2024-08-18T23:44:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.680+0000 7fa93d2ae640 1 -- 172.21.15.78:0/2058894842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa938102b00 msgr2=0x7fa9381982d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-18T23:44:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.680+0000 7fa93d2ae640 1 --2- 172.21.15.78:0/2058894842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa938102b00 0x7fa9381982d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.680+0000 7fa93d2ae640 1 -- 172.21.15.78:0/2058894842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa938108030 msgr2=0x7fa93819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.680+0000 7fa93d2ae640 1 --2- 172.21.15.78:0/2058894842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa938108030 0x7fa93819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.680+0000 7fa93d2ae640 1 -- 172.21.15.78:0/2058894842 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93819d280 con 0x7fa9381034b0 2024-08-18T23:44:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.680+0000 7fa93e2b0640 1 --2- 172.21.15.78:0/2058894842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa938108030 0x7fa93819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:44:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.680+0000 7fa93d2ae640 1 --2- 172.21.15.78:0/2058894842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9381034b0 0x7fa938198810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa92800d750 tx=0x7fa92800dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.680+0000 7fa92effd640 1 -- 172.21.15.78:0/2058894842 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928017070 con 0x7fa9381034b0 2024-08-18T23:44:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.681+0000 7fa92effd640 1 -- 172.21.15.78:0/2058894842 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa928005500 con 0x7fa9381034b0 2024-08-18T23:44:55.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.681+0000 7fa93fd13640 1 -- 172.21.15.78:0/2058894842 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa93819d570 con 0x7fa9381034b0 2024-08-18T23:44:55.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.681+0000 7fa92effd640 1 -- 172.21.15.78:0/2058894842 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928004e00 con 0x7fa9381034b0 2024-08-18T23:44:55.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.681+0000 7fa93fd13640 1 -- 172.21.15.78:0/2058894842 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9381a4e50 con 0x7fa9381034b0 2024-08-18T23:44:55.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.683+0000 7fa92effd640 1 -- 172.21.15.78:0/2058894842 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fa928004fa0 con 0x7fa9381034b0 2024-08-18T23:44:55.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.684+0000 7fa92effd640 1 --2- 172.21.15.78:0/2058894842 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa9240a2d10 0x7fa9240a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:55.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.684+0000 7fa92effd640 1 -- 172.21.15.78:0/2058894842 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa9280cef40 con 0x7fa9381034b0 2024-08-18T23:44:55.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.684+0000 7fa93daaf640 1 --2- 172.21.15.78:0/2058894842 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa9240a2d10 0x7fa9240a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:55.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.684+0000 7fa942111740 1 Processor -- start 2024-08-18T23:44:55.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.684+0000 7fa942111740 1 -- start start 2024-08-18T23:44:55.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.685+0000 7fa942111740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb34230 0x559a8fb34610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:55.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.685+0000 7fa942111740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb2eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:55.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.685+0000 7fa93daaf640 1 --2- 172.21.15.78:0/2058894842 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa9240a2d10 0x7fa9240a51d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa93400b990 tx=0x7fa93400b370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:55.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.685+0000 7fa942111740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb2f1f0 0x559a8fb2f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:55.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.685+0000 7fa942111740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559a8fa8aa40 con 0x559a8fb34b50 2024-08-18T23:44:55.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.685+0000 7fa942111740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559a8fa229d0 con 0x559a8fb2f1f0 2024-08-18T23:44:55.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.685+0000 7fa942111740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559a8fa86a00 con 0x559a8fb34230 2024-08-18T23:44:55.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.685+0000 7fa93daaf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb34230 0x559a8fb34610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:55.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.685+0000 7fa93d2ae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb2eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:55.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.685+0000 7fa93daaf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb34230 0x559a8fb34610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:48878/0 (socket says 172.21.15.78:48878) 2024-08-18T23:44:55.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.685+0000 7fa93daaf640 1 -- 172.21.15.78:0/1548160708 learned_addr learned my addr 172.21.15.78:0/1548160708 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:55.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.685+0000 7fa93e2b0640 1 --2- 172.21.15.78:0/1548160708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb2f1f0 0x559a8fb2f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:55.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.685+0000 7fa93d2ae640 1 -- 172.21.15.78:0/1548160708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb34230 msgr2=0x559a8fb34610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:55.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.685+0000 7fa93d2ae640 1 --2- 172.21.15.78:0/1548160708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb34230 0x559a8fb34610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.685+0000 7fa93d2ae640 1 -- 172.21.15.78:0/1548160708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb2f1f0 msgr2=0x559a8fb2f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:55.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.685+0000 7fa93d2ae640 1 --2- 172.21.15.78:0/1548160708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb2f1f0 0x559a8fb2f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.685+0000 7fa93d2ae640 1 -- 172.21.15.78:0/1548160708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a8fa29650 con 0x559a8fb34b50 2024-08-18T23:44:55.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.685+0000 7fa93daaf640 1 --2- 172.21.15.78:0/1548160708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb34230 0x559a8fb34610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:44:55.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.686+0000 7fa93d2ae640 1 --2- 172.21.15.78:0/1548160708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb2eb80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa9280cc500 tx=0x7fa928097620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0003d125f55951b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:55.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.686+0000 7fa923fff640 1 -- 172.21.15.78:0/1548160708 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928020070 con 0x559a8fb34b50 2024-08-18T23:44:55.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.686+0000 7fa923fff640 1 -- 172.21.15.78:0/1548160708 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa92809f600 con 0x559a8fb34b50 2024-08-18T23:44:55.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.686+0000 7fa923fff640 1 -- 172.21.15.78:0/1548160708 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9280cc750 con 0x559a8fb34b50 2024-08-18T23:44:55.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.687+0000 7fa942111740 1 -- 172.21.15.78:0/1548160708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 msgr2=0x559a8fb2eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:55.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.687+0000 7fa942111740 1 --2- 172.21.15.78:0/1548160708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb2eb80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa9280cc500 tx=0x7fa928097620 comp rx=0 tx=0).stop 2024-08-18T23:44:55.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.687+0000 7fa942111740 1 -- 172.21.15.78:0/1548160708 shutdown_connections 2024-08-18T23:44:55.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.687+0000 7fa942111740 1 --2- 172.21.15.78:0/1548160708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb2f1f0 0x559a8fb2f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.687+0000 7fa942111740 1 --2- 172.21.15.78:0/1548160708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb2eb80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.687+0000 7fa942111740 1 --2- 172.21.15.78:0/1548160708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb34230 0x559a8fb34610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.687+0000 7fa942111740 1 -- 172.21.15.78:0/1548160708 >> 172.21.15.78:0/1548160708 conn(0x559a8fb21030 msgr2=0x559a8fb21410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:55.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.687+0000 7fa942111740 1 -- 172.21.15.78:0/1548160708 shutdown_connections 2024-08-18T23:44:55.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.687+0000 7fa942111740 1 -- 172.21.15.78:0/1548160708 wait complete. 2024-08-18T23:44:55.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.687+0000 7fa942111740 1 -- start start 2024-08-18T23:44:55.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.688+0000 7fa942111740 1 Processor -- start 2024-08-18T23:44:55.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.688+0000 7fa942111740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb2f1f0 0x559a8fb380c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:55.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.688+0000 7fa942111740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb34230 0x559a8fb38600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:55.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.688+0000 7fa942111740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb38b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:55.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.688+0000 7fa93daaf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb2f1f0 0x559a8fb380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:55.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.688+0000 7fa93daaf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb2f1f0 0x559a8fb380c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:48892/0 (socket says 172.21.15.78:48892) 2024-08-18T23:44:55.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.688+0000 7fa93e2b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb38b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:55.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.688+0000 7fa93e2b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb38b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43130/0 (socket says 172.21.15.78:43130) 2024-08-18T23:44:55.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.688+0000 7fa93daaf640 1 -- 172.21.15.78:0/2242829406 learned_addr learned my addr 172.21.15.78:0/2242829406 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:55.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.689+0000 7fa93d2ae640 1 --2- 172.21.15.78:0/2242829406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb34230 0x559a8fb38600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:55.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.689+0000 7fa942111740 1 -- 172.21.15.78:0/2242829406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559a8fb3b570 con 0x559a8fb34b50 2024-08-18T23:44:55.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.689+0000 7fa942111740 1 -- 172.21.15.78:0/2242829406 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559a8fa45b20 con 0x559a8fb34230 2024-08-18T23:44:55.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.689+0000 7fa942111740 1 -- 172.21.15.78:0/2242829406 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559a8fb3b6f0 con 0x559a8fb2f1f0 2024-08-18T23:44:55.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.689+0000 7fa942111740 10 client.0 _ll_register_callbacks cb 0x559a8fa43bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:55.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.689+0000 7fa93daaf640 1 -- 172.21.15.78:0/2242829406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb34230 msgr2=0x559a8fb38600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:55.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.689+0000 7fa93daaf640 1 --2- 172.21.15.78:0/2242829406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb34230 0x559a8fb38600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.689+0000 7fa93daaf640 1 -- 172.21.15.78:0/2242829406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 msgr2=0x559a8fb38b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:55.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.689+0000 7fa93daaf640 1 --2- 172.21.15.78:0/2242829406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb38b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.689+0000 7fa93daaf640 1 -- 172.21.15.78:0/2242829406 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a8f7fc3c0 con 0x559a8fb2f1f0 2024-08-18T23:44:55.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.689+0000 7fa942111740 1 -- 172.21.15.78:0/2242829406 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa92809f600 con 0x559a8fb2f1f0 2024-08-18T23:44:55.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.689+0000 7fa93daaf640 1 --2- 172.21.15.78:0/2242829406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb2f1f0 0x559a8fb380c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9340104e0 tx=0x7fa934004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22049cd2b195b28a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:55.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.689+0000 7fa9217fa640 10 client.24973 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:44:55.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.689+0000 7fa9217fa640 1 -- 172.21.15.78:0/2242829406 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934026600 con 0x559a8fb2f1f0 2024-08-18T23:44:55.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.689+0000 7fa9217fa640 1 -- 172.21.15.78:0/2242829406 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa9340267a0 con 0x559a8fb2f1f0 2024-08-18T23:44:55.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.690+0000 7fa9217fa640 1 -- 172.21.15.78:0/2242829406 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa9340269a0 con 0x559a8fb2f1f0 2024-08-18T23:44:55.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.690+0000 7fa942111740 10 client.24973 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:55.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.690+0000 7fa942111740 1 -- 172.21.15.78:0/2242829406 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({fsmap=25}) -- 0x559a8fb471c0 con 0x559a8fb2f1f0 2024-08-18T23:44:55.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.690+0000 7fa9217fa640 1 -- 172.21.15.78:0/2242829406 <== mon.1 v2:172.21.15.161:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa934026b40 con 0x559a8fb2f1f0 2024-08-18T23:44:55.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.690+0000 7fa942111740 10 client.24973 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:55.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.690+0000 7fa942111740 10 client.24973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:44:55.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.690+0000 7fa942111740 20 client.24973 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:55.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.690+0000 7fa942111740 1 --2- 172.21.15.78:0/2242829406 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x559a8fb48570 0x559a8fb4a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:55.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.690+0000 7fa942111740 4 client.24973 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:44:55.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.690+0000 7fa942111740 1 -- 172.21.15.78:0/2242829406 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a8fa97fb0 con 0x559a8fb48570 2024-08-18T23:44:55.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.690+0000 7fa93d2ae640 1 --2- 172.21.15.78:0/2242829406 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x559a8fb48570 0x559a8fb4a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:55.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.691+0000 7fa9217fa640 10 client.24973 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:55.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.691+0000 7fa93d2ae640 1 --2- 172.21.15.78:0/2242829406 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x559a8fb48570 0x559a8fb4a9a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65cec997bef07cf7 server_cookie=9af4542ed1b6cd13 in_seq=0 out_seq=0 2024-08-18T23:44:55.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.692+0000 7fa9217fa640 1 -- 172.21.15.78:0/2242829406 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a8fa97fb0 con 0x559a8fb48570 2024-08-18T23:44:55.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.692+0000 7fa9217fa640 10 client.24973 handle_command_reply: tid=0 2024-08-18T23:44:55.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.692+0000 7fa942111740 1 client.24973 shutdown 2024-08-18T23:44:55.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.692+0000 7fa942111740 1 -- 172.21.15.78:0/2242829406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb2f1f0 msgr2=0x559a8fb380c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:55.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.692+0000 7fa942111740 1 --2- 172.21.15.78:0/2242829406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb2f1f0 0x559a8fb380c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9340104e0 tx=0x7fa934004730 comp rx=0 tx=0).stop 2024-08-18T23:44:55.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.692+0000 7fa942111740 1 -- 172.21.15.78:0/2242829406 shutdown_connections 2024-08-18T23:44:55.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.692+0000 7fa942111740 1 --2- 172.21.15.78:0/2242829406 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x559a8fb48570 0x559a8fb4a9a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.692+0000 7fa942111740 1 --2- 172.21.15.78:0/2242829406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb38b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.692+0000 7fa942111740 1 --2- 172.21.15.78:0/2242829406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb34230 0x559a8fb38600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.692+0000 7fa9217fa640 0 client.24973 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:55.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.692+0000 7fa942111740 1 --2- 172.21.15.78:0/2242829406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb2f1f0 0x559a8fb380c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.692+0000 7fa942111740 1 -- 172.21.15.78:0/2242829406 >> 172.21.15.78:0/2242829406 conn(0x559a8fb21030 msgr2=0x559a8fb20600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:55.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.692+0000 7fa942111740 1 -- 172.21.15.78:0/2242829406 shutdown_connections 2024-08-18T23:44:55.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.692+0000 7fa942111740 1 -- 172.21.15.78:0/2242829406 wait complete. 2024-08-18T23:44:55.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.692+0000 7fa942111740 20 client.24973 trim_cache size 0 max 16384 2024-08-18T23:44:55.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.717+0000 7fa942111740 1 Processor -- start 2024-08-18T23:44:55.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.717+0000 7fa942111740 1 -- start start 2024-08-18T23:44:55.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.717+0000 7fa942111740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb2f1f0 0x559a8fb2eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:55.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.717+0000 7fa942111740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb34230 0x559a8fb33250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:55.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.717+0000 7fa942111740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb33790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:55.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.718+0000 7fa942111740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa9381af170 con 0x559a8fb34b50 2024-08-18T23:44:55.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.718+0000 7fa942111740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa9381aca40 con 0x559a8fb34230 2024-08-18T23:44:55.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.718+0000 7fa942111740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa9381aa890 con 0x559a8fb2f1f0 2024-08-18T23:44:55.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.718+0000 7fa93daaf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb2f1f0 0x559a8fb2eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:55.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.718+0000 7fa93e2b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb33790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:55.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.718+0000 7fa93daaf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb2f1f0 0x559a8fb2eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:48906/0 (socket says 172.21.15.78:48906) 2024-08-18T23:44:55.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.718+0000 7fa93daaf640 1 -- 172.21.15.78:0/3468283073 learned_addr learned my addr 172.21.15.78:0/3468283073 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:55.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.718+0000 7fa93e2b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb33790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43144/0 (socket says 172.21.15.78:43144) 2024-08-18T23:44:55.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.718+0000 7fa93d2ae640 1 --2- 172.21.15.78:0/3468283073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb34230 0x559a8fb33250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:55.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.718+0000 7fa93daaf640 1 -- 172.21.15.78:0/3468283073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb34230 msgr2=0x559a8fb33250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:55.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.718+0000 7fa93daaf640 1 --2- 172.21.15.78:0/3468283073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb34230 0x559a8fb33250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.718+0000 7fa93daaf640 1 -- 172.21.15.78:0/3468283073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 msgr2=0x559a8fb33790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:55.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.718+0000 7fa93daaf640 1 --2- 172.21.15.78:0/3468283073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb33790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.718+0000 7fa93daaf640 1 -- 172.21.15.78:0/3468283073 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a8fb22690 con 0x559a8fb2f1f0 2024-08-18T23:44:55.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.718+0000 7fa93d2ae640 1 --2- 172.21.15.78:0/3468283073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb34230 0x559a8fb33250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:55.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.718+0000 7fa93e2b0640 1 --2- 172.21.15.78:0/3468283073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb33790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:44:55.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.719+0000 7fa93daaf640 1 --2- 172.21.15.78:0/3468283073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb2f1f0 0x559a8fb2eb80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa934000c00 tx=0x7fa934020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=578e74e3b4ffd651 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:55.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.719+0000 7fa9237fe640 1 -- 172.21.15.78:0/3468283073 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934010cb0 con 0x559a8fb2f1f0 2024-08-18T23:44:55.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.719+0000 7fa9237fe640 1 -- 172.21.15.78:0/3468283073 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa92809f600 con 0x559a8fb2f1f0 2024-08-18T23:44:55.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.719+0000 7fa9237fe640 1 -- 172.21.15.78:0/3468283073 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934030df0 con 0x559a8fb2f1f0 2024-08-18T23:44:55.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.720+0000 7fa942111740 1 -- 172.21.15.78:0/3468283073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb2f1f0 msgr2=0x559a8fb2eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:55.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.720+0000 7fa942111740 1 --2- 172.21.15.78:0/3468283073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb2f1f0 0x559a8fb2eb80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa934000c00 tx=0x7fa934020000 comp rx=0 tx=0).stop 2024-08-18T23:44:55.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.720+0000 7fa942111740 1 -- 172.21.15.78:0/3468283073 shutdown_connections 2024-08-18T23:44:55.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.720+0000 7fa942111740 1 --2- 172.21.15.78:0/3468283073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb33790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.720+0000 7fa942111740 1 --2- 172.21.15.78:0/3468283073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb34230 0x559a8fb33250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.720+0000 7fa942111740 1 --2- 172.21.15.78:0/3468283073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb2f1f0 0x559a8fb2eb80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.720+0000 7fa942111740 1 -- 172.21.15.78:0/3468283073 >> 172.21.15.78:0/3468283073 conn(0x559a8fb21030 msgr2=0x559a8fb369b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:55.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.720+0000 7fa942111740 1 -- 172.21.15.78:0/3468283073 shutdown_connections 2024-08-18T23:44:55.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.720+0000 7fa942111740 1 -- 172.21.15.78:0/3468283073 wait complete. 2024-08-18T23:44:55.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.720+0000 7fa942111740 1 -- start start 2024-08-18T23:44:55.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.720+0000 7fa942111740 1 Processor -- start 2024-08-18T23:44:55.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.721+0000 7fa942111740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb2f1f0 0x559a8fb40e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:55.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.721+0000 7fa942111740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb34230 0x559a8fb4f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.721+0000 7fa942111740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb4fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.721+0000 7fa942111740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa9381aaf00 con 0x559a8fb34b50 2024-08-18T23:44:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.721+0000 7fa942111740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa9381ac370 con 0x559a8fb34230 2024-08-18T23:44:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.721+0000 7fa942111740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa9381ab5d0 con 0x559a8fb2f1f0 2024-08-18T23:44:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.721+0000 7fa942111740 10 client.0 _ll_register_callbacks cb 0x7fa93400da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.721+0000 7fa93e2b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb4fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.721+0000 7fa93e2b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb4fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43152/0 (socket says 172.21.15.78:43152) 2024-08-18T23:44:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.721+0000 7fa93e2b0640 1 -- 172.21.15.78:0/3277245597 learned_addr learned my addr 172.21.15.78:0/3277245597 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.721+0000 7fa93d2ae640 1 --2- 172.21.15.78:0/3277245597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb34230 0x559a8fb4f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.721+0000 7fa93daaf640 1 --2- 172.21.15.78:0/3277245597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb2f1f0 0x559a8fb40e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.721+0000 7fa93e2b0640 1 -- 172.21.15.78:0/3277245597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb2f1f0 msgr2=0x559a8fb40e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:55.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.721+0000 7fa93e2b0640 1 --2- 172.21.15.78:0/3277245597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb2f1f0 0x559a8fb40e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.721+0000 7fa93e2b0640 1 -- 172.21.15.78:0/3277245597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb34230 msgr2=0x559a8fb4f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:55.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.721+0000 7fa93e2b0640 1 --2- 172.21.15.78:0/3277245597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb34230 0x559a8fb4f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.721+0000 7fa93e2b0640 1 -- 172.21.15.78:0/3277245597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a8fb503c0 con 0x559a8fb34b50 2024-08-18T23:44:55.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.722+0000 7fa93e2b0640 1 --2- 172.21.15.78:0/3277245597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb4fe80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa93002aec0 tx=0x7fa930031ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0017748407b323 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:55.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.722+0000 7fa9217fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:44:55.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.722+0000 7fa9217fa640 1 -- 172.21.15.78:0/3277245597 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930031830 con 0x559a8fb34b50 2024-08-18T23:44:55.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.722+0000 7fa9217fa640 1 -- 172.21.15.78:0/3277245597 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa930003970 con 0x559a8fb34b50 2024-08-18T23:44:55.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.722+0000 7fa942111740 1 -- 172.21.15.78:0/3277245597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa92809f600 con 0x559a8fb34b50 2024-08-18T23:44:55.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.722+0000 7fa9217fa640 1 -- 172.21.15.78:0/3277245597 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa930003b70 con 0x559a8fb34b50 2024-08-18T23:44:55.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.722+0000 7fa942111740 10 client.15555 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:55.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.722+0000 7fa942111740 1 -- 172.21.15.78:0/3277245597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559a8fbb2f70 con 0x559a8fb34b50 2024-08-18T23:44:55.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.723+0000 7fa9217fa640 1 -- 172.21.15.78:0/3277245597 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa92809f600 con 0x559a8fb34b50 2024-08-18T23:44:55.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.723+0000 7fa9217fa640 1 -- 172.21.15.78:0/3277245597 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa93002f670 con 0x559a8fb34b50 2024-08-18T23:44:55.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.723+0000 7fa942111740 10 client.15555 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:55.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.723+0000 7fa942111740 10 client.15555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:44:55.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.723+0000 7fa942111740 20 client.15555 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:55.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.723+0000 7fa942111740 1 --2- 172.21.15.78:0/3277245597 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x559a8fbb3730 0x559a8fbb5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:55.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.723+0000 7fa942111740 4 client.15555 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:44:55.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.723+0000 7fa942111740 1 -- 172.21.15.78:0/3277245597 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a8fa924c0 con 0x559a8fbb3730 2024-08-18T23:44:55.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.723+0000 7fa93daaf640 1 --2- 172.21.15.78:0/3277245597 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x559a8fbb3730 0x559a8fbb5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:55.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.724+0000 7fa9217fa640 10 client.15555 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:55.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.724+0000 7fa93daaf640 1 --2- 172.21.15.78:0/3277245597 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x559a8fbb3730 0x559a8fbb5b20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40de1a87e7c84acd server_cookie=96dc667fe42b874a in_seq=0 out_seq=0 2024-08-18T23:44:55.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.724+0000 7fa9217fa640 1 -- 172.21.15.78:0/3277245597 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a8fa924c0 con 0x559a8fbb3730 2024-08-18T23:44:55.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.724+0000 7fa9217fa640 10 client.15555 handle_command_reply: tid=0 2024-08-18T23:44:55.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.724+0000 7fa942111740 1 client.15555 shutdown 2024-08-18T23:44:55.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.724+0000 7fa942111740 1 -- 172.21.15.78:0/3277245597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 msgr2=0x559a8fb4fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:55.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.724+0000 7fa942111740 1 --2- 172.21.15.78:0/3277245597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb4fe80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa93002aec0 tx=0x7fa930031ce0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.724+0000 7fa942111740 1 -- 172.21.15.78:0/3277245597 shutdown_connections 2024-08-18T23:44:55.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.724+0000 7fa942111740 1 --2- 172.21.15.78:0/3277245597 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x559a8fbb3730 0x559a8fbb5b20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.724+0000 7fa942111740 1 --2- 172.21.15.78:0/3277245597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a8fb34b50 0x559a8fb4fe80 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.743 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:44:55.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.724+0000 7fa942111740 1 --2- 172.21.15.78:0/3277245597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a8fb34230 0x559a8fb4f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.724+0000 7fa942111740 1 --2- 172.21.15.78:0/3277245597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559a8fb2f1f0 0x559a8fb40e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.724+0000 7fa9217fa640 0 client.15555 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:55.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.724+0000 7fa942111740 1 -- 172.21.15.78:0/3277245597 >> 172.21.15.78:0/3277245597 conn(0x559a8fb21030 msgr2=0x559a8fb33250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:55.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.725+0000 7fa942111740 1 -- 172.21.15.78:0/3277245597 shutdown_connections 2024-08-18T23:44:55.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.725+0000 7fa942111740 1 -- 172.21.15.78:0/3277245597 wait complete. 2024-08-18T23:44:55.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.725+0000 7fa942111740 20 client.15555 trim_cache size 0 max 16384 2024-08-18T23:44:55.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.725+0000 7fa93fd13640 1 -- 172.21.15.78:0/2058894842 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa9240a2d10 msgr2=0x7fa9240a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:55.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.725+0000 7fa93fd13640 1 --2- 172.21.15.78:0/2058894842 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa9240a2d10 0x7fa9240a51d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa93400b990 tx=0x7fa93400b370 comp rx=0 tx=0).stop 2024-08-18T23:44:55.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.725+0000 7fa93fd13640 1 -- 172.21.15.78:0/2058894842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9381034b0 msgr2=0x7fa938198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:55.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.725+0000 7fa93fd13640 1 --2- 172.21.15.78:0/2058894842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9381034b0 0x7fa938198810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa92800d750 tx=0x7fa92800dc10 comp rx=0 tx=0).stop 2024-08-18T23:44:55.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.725+0000 7fa93fd13640 1 -- 172.21.15.78:0/2058894842 shutdown_connections 2024-08-18T23:44:55.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.725+0000 7fa93fd13640 1 --2- 172.21.15.78:0/2058894842 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa9240a2d10 0x7fa9240a51d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.725+0000 7fa93fd13640 1 --2- 172.21.15.78:0/2058894842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa938108030 0x7fa93819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.725+0000 7fa93fd13640 1 --2- 172.21.15.78:0/2058894842 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9381034b0 0x7fa938198810 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.725+0000 7fa93fd13640 1 --2- 172.21.15.78:0/2058894842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa938102b00 0x7fa9381982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:55.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.725+0000 7fa93fd13640 1 -- 172.21.15.78:0/2058894842 >> 172.21.15.78:0/2058894842 conn(0x7fa9380fc500 msgr2=0x7fa9381060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:55.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.725+0000 7fa93fd13640 1 -- 172.21.15.78:0/2058894842 shutdown_connections 2024-08-18T23:44:55.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:55.725+0000 7fa93fd13640 1 -- 172.21.15.78:0/2058894842 wait complete. 2024-08-18T23:44:57.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:57 smithi123 ceph-mon[21808]: pgmap v369: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-18T23:44:57.566 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-18T23:44:57.566 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-18T23:44:57.592 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:57 smithi078 ceph-mon[18440]: pgmap v369: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-18T23:44:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:57 smithi161 ceph-mon[22027]: pgmap v369: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-18T23:44:57.840 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:58.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.384+0000 7fc7f9364640 1 -- 172.21.15.78:0/348661156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f41078c0 msgr2=0x7fc7f4107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:58.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.384+0000 7fc7f9364640 1 --2- 172.21.15.78:0/348661156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f41078c0 0x7fc7f4107ca0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009930 tx=0x7fc7e402a420 comp rx=0 tx=0).stop 2024-08-18T23:44:58.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.384+0000 7fc7f9364640 1 -- 172.21.15.78:0/348661156 shutdown_connections 2024-08-18T23:44:58.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.385+0000 7fc7f9364640 1 --2- 172.21.15.78:0/348661156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7f4102260 0x7fc7f410e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.385+0000 7fc7f9364640 1 --2- 172.21.15.78:0/348661156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f41018c0 0x7fc7f4101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.385+0000 7fc7f9364640 1 --2- 172.21.15.78:0/348661156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f41078c0 0x7fc7f4107ca0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.385+0000 7fc7f9364640 1 -- 172.21.15.78:0/348661156 >> 172.21.15.78:0/348661156 conn(0x7fc7f40fd620 msgr2=0x7fc7f40ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:58.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.385+0000 7fc7f9364640 1 -- 172.21.15.78:0/348661156 shutdown_connections 2024-08-18T23:44:58.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.385+0000 7fc7f9364640 1 -- 172.21.15.78:0/348661156 wait complete. 2024-08-18T23:44:58.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.385+0000 7fc7f9364640 1 Processor -- start 2024-08-18T23:44:58.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.386+0000 7fc7f9364640 1 -- start start 2024-08-18T23:44:58.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.386+0000 7fc7f9364640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f41018c0 0x7fc7f4074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:58.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.386+0000 7fc7f9364640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f4102260 0x7fc7f4074d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:58.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.386+0000 7fc7f9364640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7f41078c0 0x7fc7f41aa550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:58.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.386+0000 7fc7f9364640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f4111cc0 con 0x7fc7f4102260 2024-08-18T23:44:58.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.386+0000 7fc7f9364640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f4111b40 con 0x7fc7f41018c0 2024-08-18T23:44:58.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.386+0000 7fc7f9364640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f4111e40 con 0x7fc7f41078c0 2024-08-18T23:44:58.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.386+0000 7fc7f27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f4102260 0x7fc7f4074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:58.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.386+0000 7fc7f27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f4102260 0x7fc7f4074d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40662/0 (socket says 172.21.15.78:40662) 2024-08-18T23:44:58.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.386+0000 7fc7f27fc640 1 -- 172.21.15.78:0/1319187291 learned_addr learned my addr 172.21.15.78:0/1319187291 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:58.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.386+0000 7fc7f37fe640 1 --2- 172.21.15.78:0/1319187291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7f41078c0 0x7fc7f41aa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:58.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.386+0000 7fc7f2ffd640 1 --2- 172.21.15.78:0/1319187291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f41018c0 0x7fc7f4074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:58.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.386+0000 7fc7f27fc640 1 -- 172.21.15.78:0/1319187291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7f41078c0 msgr2=0x7fc7f41aa550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:58.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.386+0000 7fc7f27fc640 1 --2- 172.21.15.78:0/1319187291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7f41078c0 0x7fc7f41aa550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.386+0000 7fc7f27fc640 1 -- 172.21.15.78:0/1319187291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f41018c0 msgr2=0x7fc7f4074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:58.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.386+0000 7fc7f27fc640 1 --2- 172.21.15.78:0/1319187291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f41018c0 0x7fc7f4074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.386+0000 7fc7f27fc640 1 -- 172.21.15.78:0/1319187291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f41aaa90 con 0x7fc7f4102260 2024-08-18T23:44:58.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.386+0000 7fc7f2ffd640 1 --2- 172.21.15.78:0/1319187291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f41018c0 0x7fc7f4074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:44:58.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.387+0000 7fc7f27fc640 1 --2- 172.21.15.78:0/1319187291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f4102260 0x7fc7f4074d80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc7e800d980 tx=0x7fc7e800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:58.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.387+0000 7fc7dbfff640 1 -- 172.21.15.78:0/1319187291 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8017070 con 0x7fc7f4102260 2024-08-18T23:44:58.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.387+0000 7fc7dbfff640 1 -- 172.21.15.78:0/1319187291 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7e8009c30 con 0x7fc7f4102260 2024-08-18T23:44:58.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.387+0000 7fc7f9364640 1 -- 172.21.15.78:0/1319187291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f41aad20 con 0x7fc7f4102260 2024-08-18T23:44:58.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.387+0000 7fc7f9364640 1 -- 172.21.15.78:0/1319187291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f41ab1f0 con 0x7fc7f4102260 2024-08-18T23:44:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.387+0000 7fc7dbfff640 1 -- 172.21.15.78:0/1319187291 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e800f620 con 0x7fc7f4102260 2024-08-18T23:44:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.391+0000 7fc7dbfff640 1 -- 172.21.15.78:0/1319187291 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fc7e800f800 con 0x7fc7f4102260 2024-08-18T23:44:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.392+0000 7fc7fb762740 1 Processor -- start 2024-08-18T23:44:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.392+0000 7fc7dbfff640 1 --2- 172.21.15.78:0/1319187291 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc7dc0a2c90 0x7fc7dc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.392+0000 7fc7fb762740 1 -- start start 2024-08-18T23:44:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.392+0000 7fc7f37fe640 1 --2- 172.21.15.78:0/1319187291 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc7dc0a2c90 0x7fc7dc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.392+0000 7fc7dbfff640 1 -- 172.21.15.78:0/1319187291 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc7e80cf010 con 0x7fc7f4102260 2024-08-18T23:44:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.392+0000 7fc7fb762740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72a2200 0x55ffd72a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.392+0000 7fc7fb762740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2b20 0x55ffd72acea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.392+0000 7fc7fb762740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72ad3e0 0x55ffd72af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.392+0000 7fc7fb762740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ffd7265e40 con 0x55ffd72a2200 2024-08-18T23:44:58.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.392+0000 7fc7fb762740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ffd72077f0 con 0x55ffd72ad3e0 2024-08-18T23:44:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.392+0000 7fc7fb762740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ffd71aefb0 con 0x55ffd72a2b20 2024-08-18T23:44:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.392+0000 7fc7f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72a2200 0x55ffd72a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.392+0000 7fc7f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72a2200 0x55ffd72a25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40670/0 (socket says 172.21.15.78:40670) 2024-08-18T23:44:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.392+0000 7fc7f2ffd640 1 -- 172.21.15.78:0/1631924170 learned_addr learned my addr 172.21.15.78:0/1631924170 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.392+0000 7fc7f37fe640 1 --2- 172.21.15.78:0/1319187291 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc7dc0a2c90 0x7fc7dc0a5150 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec005fd0 tx=0x7fc7ec009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.392+0000 7fc7f37fe640 1 --2- 172.21.15.78:0/1631924170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72ad3e0 0x55ffd72af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.393+0000 7fc7f2ffd640 1 -- 172.21.15.78:0/1631924170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2b20 msgr2=0x55ffd72acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:58.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.393+0000 7fc7f27fc640 1 --2- 172.21.15.78:0/1631924170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2b20 0x55ffd72acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.393+0000 7fc7f2ffd640 1 --2- 172.21.15.78:0/1631924170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2b20 0x55ffd72acea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.393+0000 7fc7f2ffd640 1 -- 172.21.15.78:0/1631924170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72ad3e0 msgr2=0x55ffd72af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.393+0000 7fc7f2ffd640 1 --2- 172.21.15.78:0/1631924170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72ad3e0 0x55ffd72af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.393+0000 7fc7f2ffd640 1 -- 172.21.15.78:0/1631924170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffd6f796a0 con 0x55ffd72a2200 2024-08-18T23:44:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.393+0000 7fc7f37fe640 1 --2- 172.21.15.78:0/1631924170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72ad3e0 0x55ffd72af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:44:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.393+0000 7fc7f27fc640 1 --2- 172.21.15.78:0/1631924170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2b20 0x55ffd72acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:44:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.393+0000 7fc7f2ffd640 1 --2- 172.21.15.78:0/1631924170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72a2200 0x55ffd72a25e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009930 tx=0x7fc7e4032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53e168b297eefa server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.393+0000 7fc7d97fa640 1 -- 172.21.15.78:0/1631924170 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e403f070 con 0x55ffd72a2200 2024-08-18T23:44:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.393+0000 7fc7d97fa640 1 -- 172.21.15.78:0/1631924170 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7e40317b0 con 0x55ffd72a2200 2024-08-18T23:44:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.393+0000 7fc7d97fa640 1 -- 172.21.15.78:0/1631924170 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4035070 con 0x55ffd72a2200 2024-08-18T23:44:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.394+0000 7fc7fb762740 1 -- 172.21.15.78:0/1631924170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72a2200 msgr2=0x55ffd72a25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.394+0000 7fc7fb762740 1 --2- 172.21.15.78:0/1631924170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72a2200 0x55ffd72a25e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009930 tx=0x7fc7e4032cb0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.394+0000 7fc7fb762740 1 -- 172.21.15.78:0/1631924170 shutdown_connections 2024-08-18T23:44:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.394+0000 7fc7fb762740 1 --2- 172.21.15.78:0/1631924170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72ad3e0 0x55ffd72af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.394+0000 7fc7fb762740 1 --2- 172.21.15.78:0/1631924170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2b20 0x55ffd72acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.394+0000 7fc7fb762740 1 --2- 172.21.15.78:0/1631924170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72a2200 0x55ffd72a25e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.394+0000 7fc7fb762740 1 -- 172.21.15.78:0/1631924170 >> 172.21.15.78:0/1631924170 conn(0x55ffd729e790 msgr2=0x55ffd729eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.394+0000 7fc7fb762740 1 -- 172.21.15.78:0/1631924170 shutdown_connections 2024-08-18T23:44:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.394+0000 7fc7fb762740 1 -- 172.21.15.78:0/1631924170 wait complete. 2024-08-18T23:44:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.395+0000 7fc7fb762740 1 -- start start 2024-08-18T23:44:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.395+0000 7fc7fb762740 1 Processor -- start 2024-08-18T23:44:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.395+0000 7fc7fb762740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2200 0x55ffd72aa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.395+0000 7fc7fb762740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72a2b20 0x55ffd72aa780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.395+0000 7fc7fb762740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72ad3e0 0x55ffd72aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.395+0000 7fc7fb762740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ffd71c2cb0 con 0x55ffd72a2b20 2024-08-18T23:44:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.395+0000 7fc7fb762740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ffd72037b0 con 0x55ffd72ad3e0 2024-08-18T23:44:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.395+0000 7fc7fb762740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ffd72affb0 con 0x55ffd72a2200 2024-08-18T23:44:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.395+0000 7fc7f2ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2200 0x55ffd72aa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.395+0000 7fc7f2ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2200 0x55ffd72aa240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:45376/0 (socket says 172.21.15.78:45376) 2024-08-18T23:44:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.395+0000 7fc7f2ffd640 1 -- 172.21.15.78:0/3634052731 learned_addr learned my addr 172.21.15.78:0/3634052731 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.395+0000 7fc7fb762740 10 client.0 _ll_register_callbacks cb 0x55ffd6f16ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.396+0000 7fc7f37fe640 1 --2- 172.21.15.78:0/3634052731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72ad3e0 0x55ffd72aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.396+0000 7fc7f27fc640 1 --2- 172.21.15.78:0/3634052731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72a2b20 0x55ffd72aa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.396+0000 7fc7f2ffd640 1 -- 172.21.15.78:0/3634052731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72ad3e0 msgr2=0x55ffd72aacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.396+0000 7fc7f2ffd640 1 --2- 172.21.15.78:0/3634052731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72ad3e0 0x55ffd72aacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.396+0000 7fc7f2ffd640 1 -- 172.21.15.78:0/3634052731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72a2b20 msgr2=0x55ffd72aa780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.396+0000 7fc7f2ffd640 1 --2- 172.21.15.78:0/3634052731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72a2b20 0x55ffd72aa780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.396+0000 7fc7f2ffd640 1 -- 172.21.15.78:0/3634052731 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffd71c1bc0 con 0x55ffd72a2200 2024-08-18T23:44:58.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.396+0000 7fc7f2ffd640 1 --2- 172.21.15.78:0/3634052731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2200 0x55ffd72aa240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc7e402a940 tx=0x7fc7e4032f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6c78a1ad15d3a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.396+0000 7fc7d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:44:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.397+0000 7fc7d6ff5640 1 -- 172.21.15.78:0/3634052731 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4035070 con 0x55ffd72a2200 2024-08-18T23:44:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.397+0000 7fc7fb762740 1 -- 172.21.15.78:0/3634052731 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7e40317b0 con 0x55ffd72a2200 2024-08-18T23:44:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.397+0000 7fc7d6ff5640 1 -- 172.21.15.78:0/3634052731 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc7e40048d0 con 0x55ffd72a2200 2024-08-18T23:44:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.397+0000 7fc7d6ff5640 1 -- 172.21.15.78:0/3634052731 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc7e4055050 con 0x55ffd72a2200 2024-08-18T23:44:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.397+0000 7fc7fb762740 10 client.24988 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.397+0000 7fc7fb762740 1 -- 172.21.15.78:0/3634052731 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ffd72bbdd0 con 0x55ffd72a2200 2024-08-18T23:44:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.397+0000 7fc7d6ff5640 1 -- 172.21.15.78:0/3634052731 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7e40317b0 con 0x55ffd72a2200 2024-08-18T23:44:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.397+0000 7fc7d6ff5640 1 -- 172.21.15.78:0/3634052731 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc7e403bab0 con 0x55ffd72a2200 2024-08-18T23:44:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.397+0000 7fc7fb762740 10 client.24988 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.397+0000 7fc7fb762740 10 client.24988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:44:58.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.397+0000 7fc7fb762740 20 client.24988 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.398+0000 7fc7fb762740 1 --2- 172.21.15.78:0/3634052731 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ffd72bd120 0x55ffd72bf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.398+0000 7fc7fb762740 4 client.24988 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:44:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.398+0000 7fc7fb762740 1 -- 172.21.15.78:0/3634052731 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffd7214d60 con 0x55ffd72bd120 2024-08-18T23:44:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.398+0000 7fc7f27fc640 1 --2- 172.21.15.78:0/3634052731 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ffd72bd120 0x55ffd72bf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.398+0000 7fc7d6ff5640 10 client.24988 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.398+0000 7fc7f27fc640 1 --2- 172.21.15.78:0/3634052731 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ffd72bd120 0x55ffd72bf510 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3a21b4260d1ae2c server_cookie=ffb18725c17d5b66 in_seq=0 out_seq=0 2024-08-18T23:44:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.399+0000 7fc7d6ff5640 1 -- 172.21.15.78:0/3634052731 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffd7214d60 con 0x55ffd72bd120 2024-08-18T23:44:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.399+0000 7fc7d6ff5640 10 client.24988 handle_command_reply: tid=0 2024-08-18T23:44:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.399+0000 7fc7fb762740 1 client.24988 shutdown 2024-08-18T23:44:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.399+0000 7fc7fb762740 1 -- 172.21.15.78:0/3634052731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2200 msgr2=0x55ffd72aa240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.399+0000 7fc7fb762740 1 --2- 172.21.15.78:0/3634052731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2200 0x55ffd72aa240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc7e402a940 tx=0x7fc7e4032f10 comp rx=0 tx=0).stop 2024-08-18T23:44:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.399+0000 7fc7fb762740 1 -- 172.21.15.78:0/3634052731 shutdown_connections 2024-08-18T23:44:58.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.399+0000 7fc7fb762740 1 --2- 172.21.15.78:0/3634052731 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ffd72bd120 0x55ffd72bf510 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.399+0000 7fc7fb762740 1 --2- 172.21.15.78:0/3634052731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72ad3e0 0x55ffd72aacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.399+0000 7fc7fb762740 1 --2- 172.21.15.78:0/3634052731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72a2b20 0x55ffd72aa780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.399+0000 7fc7fb762740 1 --2- 172.21.15.78:0/3634052731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2200 0x55ffd72aa240 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.399+0000 7fc7fb762740 1 -- 172.21.15.78:0/3634052731 >> 172.21.15.78:0/3634052731 conn(0x55ffd729e790 msgr2=0x55ffd729d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.399+0000 7fc7d6ff5640 0 client.24988 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.399+0000 7fc7fb762740 1 -- 172.21.15.78:0/3634052731 shutdown_connections 2024-08-18T23:44:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.399+0000 7fc7fb762740 1 -- 172.21.15.78:0/3634052731 wait complete. 2024-08-18T23:44:58.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.399+0000 7fc7fb762740 20 client.24988 trim_cache size 0 max 16384 2024-08-18T23:44:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.433+0000 7fc7fb762740 1 Processor -- start 2024-08-18T23:44:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.433+0000 7fc7fb762740 1 -- start start 2024-08-18T23:44:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.433+0000 7fc7fb762740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2200 0x55ffd72aab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.433+0000 7fc7fb762740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72a2b20 0x55ffd72a74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:58.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.433+0000 7fc7fb762740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72ad3e0 0x55ffd72a79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.433+0000 7fc7fb762740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0012460 con 0x55ffd72ad3e0 2024-08-18T23:44:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.433+0000 7fc7fb762740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e000fd30 con 0x55ffd72a2b20 2024-08-18T23:44:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.433+0000 7fc7fb762740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e000db80 con 0x55ffd72a2200 2024-08-18T23:44:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.433+0000 7fc7f37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72ad3e0 0x55ffd72a79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.433+0000 7fc7f37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72ad3e0 0x55ffd72a79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40700/0 (socket says 172.21.15.78:40700) 2024-08-18T23:44:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.433+0000 7fc7f37fe640 1 -- 172.21.15.78:0/2892479199 learned_addr learned my addr 172.21.15.78:0/2892479199 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:58.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.434+0000 7fc7f2ffd640 1 --2- 172.21.15.78:0/2892479199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2200 0x55ffd72aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.434+0000 7fc7f37fe640 1 -- 172.21.15.78:0/2892479199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2200 msgr2=0x55ffd72aab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.434+0000 7fc7f27fc640 1 --2- 172.21.15.78:0/2892479199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72a2b20 0x55ffd72a74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.434+0000 7fc7f37fe640 1 --2- 172.21.15.78:0/2892479199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2200 0x55ffd72aab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.434+0000 7fc7f37fe640 1 -- 172.21.15.78:0/2892479199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72a2b20 msgr2=0x55ffd72a74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.434+0000 7fc7f37fe640 1 --2- 172.21.15.78:0/2892479199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72a2b20 0x55ffd72a74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.434+0000 7fc7f37fe640 1 -- 172.21.15.78:0/2892479199 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffd729e230 con 0x55ffd72ad3e0 2024-08-18T23:44:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.434+0000 7fc7f27fc640 1 --2- 172.21.15.78:0/2892479199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72a2b20 0x55ffd72a74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:44:58.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.434+0000 7fc7f37fe640 1 --2- 172.21.15.78:0/2892479199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72ad3e0 0x55ffd72a79e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0127e0 tx=0x7fc7ec013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc814957e5ed3ea server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.435+0000 7fc7d8ff9640 1 -- 172.21.15.78:0/2892479199 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec012d60 con 0x55ffd72ad3e0 2024-08-18T23:44:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.435+0000 7fc7d8ff9640 1 -- 172.21.15.78:0/2892479199 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7ec020e10 con 0x55ffd72ad3e0 2024-08-18T23:44:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.435+0000 7fc7d8ff9640 1 -- 172.21.15.78:0/2892479199 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec0113e0 con 0x55ffd72ad3e0 2024-08-18T23:44:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.436+0000 7fc7fb762740 1 -- 172.21.15.78:0/2892479199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72ad3e0 msgr2=0x55ffd72a79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.436+0000 7fc7fb762740 1 --2- 172.21.15.78:0/2892479199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72ad3e0 0x55ffd72a79e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0127e0 tx=0x7fc7ec013000 comp rx=0 tx=0).stop 2024-08-18T23:44:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.436+0000 7fc7fb762740 1 -- 172.21.15.78:0/2892479199 shutdown_connections 2024-08-18T23:44:58.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.436+0000 7fc7fb762740 1 --2- 172.21.15.78:0/2892479199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72ad3e0 0x55ffd72a79e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.436+0000 7fc7fb762740 1 --2- 172.21.15.78:0/2892479199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72a2b20 0x55ffd72a74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.436+0000 7fc7fb762740 1 --2- 172.21.15.78:0/2892479199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2200 0x55ffd72aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.436+0000 7fc7fb762740 1 -- 172.21.15.78:0/2892479199 >> 172.21.15.78:0/2892479199 conn(0x55ffd729e790 msgr2=0x55ffd72ac2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:58.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.436+0000 7fc7fb762740 1 -- 172.21.15.78:0/2892479199 shutdown_connections 2024-08-18T23:44:58.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.436+0000 7fc7fb762740 1 -- 172.21.15.78:0/2892479199 wait complete. 2024-08-18T23:44:58.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.436+0000 7fc7fb762740 1 -- start start 2024-08-18T23:44:58.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.436+0000 7fc7fb762740 1 Processor -- start 2024-08-18T23:44:58.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.437+0000 7fc7fb762740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2200 0x55ffd72be430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.437+0000 7fc7fb762740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72a2b20 0x55ffd72be970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.437+0000 7fc7fb762740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72ad3e0 0x55ffd72beeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.437+0000 7fc7f27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72a2b20 0x55ffd72be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.437+0000 7fc7fb762740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e000e1f0 con 0x55ffd72a2b20 2024-08-18T23:44:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.437+0000 7fc7f27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72a2b20 0x55ffd72be970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40714/0 (socket says 172.21.15.78:40714) 2024-08-18T23:44:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.437+0000 7fc7f27fc640 1 -- 172.21.15.78:0/3798354836 learned_addr learned my addr 172.21.15.78:0/3798354836 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:44:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.437+0000 7fc7f2ffd640 1 --2- 172.21.15.78:0/3798354836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2200 0x55ffd72be430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.437+0000 7fc7fb762740 1 -- 172.21.15.78:0/3798354836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e000f660 con 0x55ffd72ad3e0 2024-08-18T23:44:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.437+0000 7fc7fb762740 1 -- 172.21.15.78:0/3798354836 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e000e8c0 con 0x55ffd72a2200 2024-08-18T23:44:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.437+0000 7fc7fb762740 10 client.0 _ll_register_callbacks cb 0x7fc7ec016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:44:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.437+0000 7fc7f37fe640 1 --2- 172.21.15.78:0/3798354836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72ad3e0 0x55ffd72beeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.437+0000 7fc7f2ffd640 1 -- 172.21.15.78:0/3798354836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72ad3e0 msgr2=0x55ffd72beeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:58.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.437+0000 7fc7f2ffd640 1 --2- 172.21.15.78:0/3798354836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72ad3e0 0x55ffd72beeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.437+0000 7fc7f2ffd640 1 -- 172.21.15.78:0/3798354836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72a2b20 msgr2=0x55ffd72be970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:44:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.437+0000 7fc7f2ffd640 1 --2- 172.21.15.78:0/3798354836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72a2b20 0x55ffd72be970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.437+0000 7fc7f2ffd640 1 -- 172.21.15.78:0/3798354836 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffd7326bf0 con 0x55ffd72a2200 2024-08-18T23:44:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.437+0000 7fc7f27fc640 1 --2- 172.21.15.78:0/3798354836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72a2b20 0x55ffd72be970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:44:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.438+0000 7fc7f2ffd640 1 --2- 172.21.15.78:0/3798354836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2200 0x55ffd72be430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc7e403be80 tx=0x7fc7e4032e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd7be22dea905ef9 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:44:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.438+0000 7fc7d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:44:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.438+0000 7fc7d6ff5640 1 -- 172.21.15.78:0/3798354836 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e406f060 con 0x55ffd72a2200 2024-08-18T23:44:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.438+0000 7fc7d6ff5640 1 -- 172.21.15.78:0/3798354836 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc7e4004230 con 0x55ffd72a2200 2024-08-18T23:44:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.438+0000 7fc7fb762740 1 -- 172.21.15.78:0/3798354836 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7ec020e10 con 0x55ffd72a2200 2024-08-18T23:44:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.438+0000 7fc7d6ff5640 1 -- 172.21.15.78:0/3798354836 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc7e4076050 con 0x55ffd72a2200 2024-08-18T23:44:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.438+0000 7fc7fb762740 10 client.24994 fetch_fsmap learned FSMap version 25 2024-08-18T23:44:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.438+0000 7fc7fb762740 1 -- 172.21.15.78:0/3798354836 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ffd7327e60 con 0x55ffd72a2200 2024-08-18T23:44:58.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.439+0000 7fc7d6ff5640 1 -- 172.21.15.78:0/3798354836 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7ec020e10 con 0x55ffd72a2200 2024-08-18T23:44:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.439+0000 7fc7d6ff5640 1 -- 172.21.15.78:0/3798354836 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc7e40048d0 con 0x55ffd72a2200 2024-08-18T23:44:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.439+0000 7fc7fb762740 10 client.24994 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:44:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.439+0000 7fc7fb762740 10 client.24994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:44:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.439+0000 7fc7fb762740 20 client.24994 populate_metadata read hostname 'smithi078' 2024-08-18T23:44:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.439+0000 7fc7fb762740 1 --2- 172.21.15.78:0/3798354836 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ffd7328520 0x55ffd732a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:44:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.439+0000 7fc7fb762740 4 client.24994 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-18T23:44:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.439+0000 7fc7fb762740 1 -- 172.21.15.78:0/3798354836 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ffd720f270 con 0x55ffd7328520 2024-08-18T23:44:58.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.439+0000 7fc7f27fc640 1 --2- 172.21.15.78:0/3798354836 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ffd7328520 0x55ffd732a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:44:58.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.440+0000 7fc7f27fc640 1 --2- 172.21.15.78:0/3798354836 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ffd7328520 0x55ffd732a930 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2db6ca6dc0a5676 server_cookie=35995c5778963212 in_seq=0 out_seq=0 2024-08-18T23:44:58.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.440+0000 7fc7d6ff5640 10 client.24994 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.441+0000 7fc7d6ff5640 1 -- 172.21.15.78:0/3798354836 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ffd720f270 con 0x55ffd7328520 2024-08-18T23:44:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.441+0000 7fc7d6ff5640 10 client.24994 handle_command_reply: tid=0 2024-08-18T23:44:58.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.441+0000 7fc7fb762740 1 client.24994 shutdown 2024-08-18T23:44:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.442+0000 7fc7fb762740 1 -- 172.21.15.78:0/3798354836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2200 msgr2=0x55ffd72be430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.442+0000 7fc7fb762740 1 --2- 172.21.15.78:0/3798354836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2200 0x55ffd72be430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc7e403be80 tx=0x7fc7e4032e20 comp rx=0 tx=0).stop 2024-08-18T23:44:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.442+0000 7fc7fb762740 1 -- 172.21.15.78:0/3798354836 shutdown_connections 2024-08-18T23:44:58.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.442+0000 7fc7fb762740 1 --2- 172.21.15.78:0/3798354836 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ffd7328520 0x55ffd732a930 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.442+0000 7fc7fb762740 1 --2- 172.21.15.78:0/3798354836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ffd72ad3e0 0x55ffd72beeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.442+0000 7fc7d6ff5640 0 client.24994 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:44:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.442+0000 7fc7fb762740 1 --2- 172.21.15.78:0/3798354836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ffd72a2b20 0x55ffd72be970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.442+0000 7fc7fb762740 1 --2- 172.21.15.78:0/3798354836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffd72a2200 0x55ffd72be430 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.442+0000 7fc7fb762740 1 -- 172.21.15.78:0/3798354836 >> 172.21.15.78:0/3798354836 conn(0x55ffd729e790 msgr2=0x55ffd72b8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.442+0000 7fc7fb762740 1 -- 172.21.15.78:0/3798354836 shutdown_connections 2024-08-18T23:44:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.442+0000 7fc7fb762740 1 -- 172.21.15.78:0/3798354836 wait complete. 2024-08-18T23:44:58.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.442+0000 7fc7fb762740 20 client.24994 trim_cache size 0 max 16384 2024-08-18T23:44:58.450 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:44:58.450 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-08-18T23:44:58.450 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "582d906a-c4f8-494f-a632-0eee0a5f385c", 2024-08-18T23:44:58.450 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-08-18T23:44:58.451 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:44:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.443+0000 7fc7f9364640 1 -- 172.21.15.78:0/1319187291 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc7dc0a2c90 msgr2=0x7fc7dc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.443+0000 7fc7f9364640 1 --2- 172.21.15.78:0/1319187291 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc7dc0a2c90 0x7fc7dc0a5150 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec005fd0 tx=0x7fc7ec009600 comp rx=0 tx=0).stop 2024-08-18T23:44:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.443+0000 7fc7f9364640 1 -- 172.21.15.78:0/1319187291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f4102260 msgr2=0x7fc7f4074d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:44:58.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.443+0000 7fc7f9364640 1 --2- 172.21.15.78:0/1319187291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f4102260 0x7fc7f4074d80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc7e800d980 tx=0x7fc7e800de40 comp rx=0 tx=0).stop 2024-08-18T23:44:58.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.444+0000 7fc7f9364640 1 -- 172.21.15.78:0/1319187291 shutdown_connections 2024-08-18T23:44:58.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.444+0000 7fc7f9364640 1 --2- 172.21.15.78:0/1319187291 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc7dc0a2c90 0x7fc7dc0a5150 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.444+0000 7fc7f9364640 1 --2- 172.21.15.78:0/1319187291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7f41078c0 0x7fc7f41aa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.444+0000 7fc7f9364640 1 --2- 172.21.15.78:0/1319187291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f4102260 0x7fc7f4074d80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.444+0000 7fc7f9364640 1 --2- 172.21.15.78:0/1319187291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f41018c0 0x7fc7f4074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:44:58.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.444+0000 7fc7f9364640 1 -- 172.21.15.78:0/1319187291 >> 172.21.15.78:0/1319187291 conn(0x7fc7f40fd620 msgr2=0x7fc7f40fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:44:58.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.444+0000 7fc7f9364640 1 -- 172.21.15.78:0/1319187291 shutdown_connections 2024-08-18T23:44:58.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:44:58.444+0000 7fc7f9364640 1 -- 172.21.15.78:0/1319187291 wait complete. 2024-08-18T23:44:59.287 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:44:59.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:59 smithi123 ceph-mon[21808]: pgmap v370: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-18T23:44:59.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:59 smithi123 ceph-mon[21808]: scrub queued for path: / 2024-08-18T23:44:59.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:59 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:44:59.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:59 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:44:59.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:44:59 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:44:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:44:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:59 smithi161 ceph-mon[22027]: pgmap v370: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-18T23:44:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:59 smithi161 ceph-mon[22027]: scrub queued for path: / 2024-08-18T23:44:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:59 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:44:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:59 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:44:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:44:59 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:44:59.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:59 smithi078 ceph-mon[18440]: pgmap v370: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-18T23:44:59.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:59 smithi078 ceph-mon[18440]: scrub queued for path: / 2024-08-18T23:44:59.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:59 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:44:59.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:59 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:44:59.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:44:59 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:45:00.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.241+0000 7fde5013a640 1 -- 172.21.15.78:0/35105907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde48100b50 msgr2=0x7fde48100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:00.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.241+0000 7fde5013a640 1 --2- 172.21.15.78:0/35105907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde48100b50 0x7fde48100f30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fde44009900 tx=0x7fde4402a480 comp rx=0 tx=0).stop 2024-08-18T23:45:00.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.241+0000 7fde5013a640 1 -- 172.21.15.78:0/35105907 shutdown_connections 2024-08-18T23:45:00.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.241+0000 7fde5013a640 1 --2- 172.21.15.78:0/35105907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde48102c20 0x7fde4810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.241+0000 7fde5013a640 1 --2- 172.21.15.78:0/35105907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde481022a0 0x7fde481026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.241+0000 7fde5013a640 1 --2- 172.21.15.78:0/35105907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde48100b50 0x7fde48100f30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.241+0000 7fde5013a640 1 -- 172.21.15.78:0/35105907 >> 172.21.15.78:0/35105907 conn(0x7fde480fc500 msgr2=0x7fde480fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:00.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.241+0000 7fde5013a640 1 -- 172.21.15.78:0/35105907 shutdown_connections 2024-08-18T23:45:00.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.241+0000 7fde5013a640 1 -- 172.21.15.78:0/35105907 wait complete. 2024-08-18T23:45:00.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.242+0000 7fde5013a640 1 Processor -- start 2024-08-18T23:45:00.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.242+0000 7fde5013a640 1 -- start start 2024-08-18T23:45:00.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.243+0000 7fde5013a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde48100b50 0x7fde48197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:00.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.243+0000 7fde5013a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde481022a0 0x7fde481975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:00.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.243+0000 7fde5013a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde48102c20 0x7fde4819b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:00.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.243+0000 7fde5013a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fde48110cb0 con 0x7fde481022a0 2024-08-18T23:45:00.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.243+0000 7fde5013a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde48110b30 con 0x7fde48100b50 2024-08-18T23:45:00.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.243+0000 7fde5013a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fde48110e30 con 0x7fde48102c20 2024-08-18T23:45:00.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.243+0000 7fde4d6d5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde481022a0 0x7fde481975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:00.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.243+0000 7fde4d6d5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde481022a0 0x7fde481975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40738/0 (socket says 172.21.15.78:40738) 2024-08-18T23:45:00.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.243+0000 7fde4d6d5640 1 -- 172.21.15.78:0/3036710572 learned_addr learned my addr 172.21.15.78:0/3036710572 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:00.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.243+0000 7fde4ded6640 1 --2- 172.21.15.78:0/3036710572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde48100b50 0x7fde48197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:00.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.243+0000 7fde4e6d7640 1 --2- 172.21.15.78:0/3036710572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde48102c20 0x7fde4819b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:00.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.243+0000 7fde4d6d5640 1 -- 172.21.15.78:0/3036710572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde48102c20 msgr2=0x7fde4819b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:00.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.243+0000 7fde4d6d5640 1 --2- 172.21.15.78:0/3036710572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde48102c20 0x7fde4819b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.243+0000 7fde4d6d5640 1 -- 172.21.15.78:0/3036710572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde48100b50 msgr2=0x7fde48197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:00.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.243+0000 7fde4d6d5640 1 --2- 172.21.15.78:0/3036710572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde48100b50 0x7fde48197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.243+0000 7fde4d6d5640 1 -- 172.21.15.78:0/3036710572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4819c020 con 0x7fde481022a0 2024-08-18T23:45:00.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.243+0000 7fde4e6d7640 1 --2- 172.21.15.78:0/3036710572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde48102c20 0x7fde4819b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:45:00.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.244+0000 7fde4d6d5640 1 --2- 172.21.15.78:0/3036710572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde481022a0 0x7fde481975b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fde3800c920 tx=0x7fde3800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:00.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.244+0000 7fde3effd640 1 -- 172.21.15.78:0/3036710572 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38007c70 con 0x7fde481022a0 2024-08-18T23:45:00.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.244+0000 7fde3effd640 1 -- 172.21.15.78:0/3036710572 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde38007e10 con 0x7fde481022a0 2024-08-18T23:45:00.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.244+0000 7fde5013a640 1 -- 172.21.15.78:0/3036710572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4819c310 con 0x7fde481022a0 2024-08-18T23:45:00.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.244+0000 7fde3effd640 1 -- 172.21.15.78:0/3036710572 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38018890 con 0x7fde481022a0 2024-08-18T23:45:00.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.244+0000 7fde5013a640 1 -- 172.21.15.78:0/3036710572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde48069ca0 con 0x7fde481022a0 2024-08-18T23:45:00.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.247+0000 7fde3effd640 1 -- 172.21.15.78:0/3036710572 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fde38018a70 con 0x7fde481022a0 2024-08-18T23:45:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.248+0000 7fde52538740 1 Processor -- start 2024-08-18T23:45:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.248+0000 7fde52538740 1 -- start start 2024-08-18T23:45:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.248+0000 7fde3effd640 1 --2- 172.21.15.78:0/3036710572 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fde340a2c90 0x7fde340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.249+0000 7fde52538740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45f0210 0x55b9c45f05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.249+0000 7fde52538740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45f0b30 0x55b9c45faeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.249+0000 7fde52538740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45fb3f0 0x55b9c45fd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.249+0000 7fde52538740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b9c4555980 con 0x55b9c45fb3f0 2024-08-18T23:45:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.249+0000 7fde52538740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b9c44eda70 con 0x55b9c45f0b30 2024-08-18T23:45:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.249+0000 7fde52538740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b9c4551940 con 0x55b9c45f0210 2024-08-18T23:45:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.249+0000 7fde3effd640 1 -- 172.21.15.78:0/3036710572 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fde380cf660 con 0x7fde481022a0 2024-08-18T23:45:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.249+0000 7fde4e6d7640 1 --2- 172.21.15.78:0/3036710572 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fde340a2c90 0x7fde340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.249+0000 7fde4ded6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45f0210 0x55b9c45f05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.249+0000 7fde4e6d7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45fb3f0 0x55b9c45fd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.249+0000 7fde4d6d5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45f0b30 0x55b9c45faeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.249+0000 7fde4ded6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45f0210 0x55b9c45f05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:45424/0 (socket says 172.21.15.78:45424) 2024-08-18T23:45:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.249+0000 7fde4ded6640 1 -- 172.21.15.78:0/1896745282 learned_addr learned my addr 172.21.15.78:0/1896745282 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.249+0000 7fde4d6d5640 1 -- 172.21.15.78:0/1896745282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45f0210 msgr2=0x55b9c45f05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.249+0000 7fde4d6d5640 1 --2- 172.21.15.78:0/1896745282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45f0210 0x55b9c45f05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.249+0000 7fde4d6d5640 1 -- 172.21.15.78:0/1896745282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45fb3f0 msgr2=0x55b9c45fd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.249+0000 7fde4d6d5640 1 --2- 172.21.15.78:0/1896745282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45fb3f0 0x55b9c45fd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.249+0000 7fde4d6d5640 1 -- 172.21.15.78:0/1896745282 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c45506e0 con 0x55b9c45f0b30 2024-08-18T23:45:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.249+0000 7fde4e6d7640 1 --2- 172.21.15.78:0/1896745282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45fb3f0 0x55b9c45fd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:45:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.250+0000 7fde4d6d5640 1 --2- 172.21.15.78:0/1896745282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45f0b30 0x55b9c45faeb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fde380ccc80 tx=0x7fde3804a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb28b60feef2a463 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.250+0000 7fde33fff640 1 -- 172.21.15.78:0/1896745282 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3809b9d0 con 0x55b9c45f0b30 2024-08-18T23:45:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.250+0000 7fde4e6d7640 1 --2- 172.21.15.78:0/3036710572 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fde340a2c90 0x7fde340a5150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55b9c45edb20 tx=0x7fde40009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.250+0000 7fde33fff640 1 -- 172.21.15.78:0/1896745282 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde3809fd80 con 0x55b9c45f0b30 2024-08-18T23:45:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.250+0000 7fde33fff640 1 -- 172.21.15.78:0/1896745282 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38098040 con 0x55b9c45f0b30 2024-08-18T23:45:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.251+0000 7fde52538740 1 -- 172.21.15.78:0/1896745282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45f0b30 msgr2=0x55b9c45faeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.251+0000 7fde52538740 1 --2- 172.21.15.78:0/1896745282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45f0b30 0x55b9c45faeb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fde380ccc80 tx=0x7fde3804a410 comp rx=0 tx=0).stop 2024-08-18T23:45:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.251+0000 7fde52538740 1 -- 172.21.15.78:0/1896745282 shutdown_connections 2024-08-18T23:45:00.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.251+0000 7fde52538740 1 --2- 172.21.15.78:0/1896745282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45fb3f0 0x55b9c45fd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.251+0000 7fde52538740 1 --2- 172.21.15.78:0/1896745282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45f0b30 0x55b9c45faeb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.251+0000 7fde52538740 1 --2- 172.21.15.78:0/1896745282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45f0210 0x55b9c45f05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.251+0000 7fde52538740 1 -- 172.21.15.78:0/1896745282 >> 172.21.15.78:0/1896745282 conn(0x55b9c45ec7a0 msgr2=0x55b9c45ecb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:00.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.251+0000 7fde52538740 1 -- 172.21.15.78:0/1896745282 shutdown_connections 2024-08-18T23:45:00.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.251+0000 7fde52538740 1 -- 172.21.15.78:0/1896745282 wait complete. 2024-08-18T23:45:00.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.251+0000 7fde52538740 1 -- start start 2024-08-18T23:45:00.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.252+0000 7fde52538740 1 Processor -- start 2024-08-18T23:45:00.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.252+0000 7fde52538740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45f0210 0x55b9c45f8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:00.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.252+0000 7fde52538740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45f0b30 0x55b9c45f8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:00.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.252+0000 7fde52538740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45fb3f0 0x55b9c45f8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:00.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.252+0000 7fde52538740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b9c45fdfc0 con 0x55b9c45fb3f0 2024-08-18T23:45:00.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.252+0000 7fde52538740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b9c4510aa0 con 0x55b9c45f0210 2024-08-18T23:45:00.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.252+0000 7fde52538740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b9c45fe140 con 0x55b9c45f0b30 2024-08-18T23:45:00.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.252+0000 7fde52538740 10 client.0 _ll_register_callbacks cb 0x55b9c44ffe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:45:00.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.252+0000 7fde4e6d7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45fb3f0 0x55b9c45f8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:00.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.252+0000 7fde4e6d7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45fb3f0 0x55b9c45f8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40762/0 (socket says 172.21.15.78:40762) 2024-08-18T23:45:00.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.252+0000 7fde4e6d7640 1 -- 172.21.15.78:0/1274401247 learned_addr learned my addr 172.21.15.78:0/1274401247 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:00.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.253+0000 7fde4ded6640 1 --2- 172.21.15.78:0/1274401247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45f0210 0x55b9c45f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:00.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.253+0000 7fde4d6d5640 1 --2- 172.21.15.78:0/1274401247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45f0b30 0x55b9c45f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:00.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.253+0000 7fde4e6d7640 1 -- 172.21.15.78:0/1274401247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45f0b30 msgr2=0x55b9c45f8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:00.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.253+0000 7fde4e6d7640 1 --2- 172.21.15.78:0/1274401247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45f0b30 0x55b9c45f8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.253+0000 7fde4e6d7640 1 -- 172.21.15.78:0/1274401247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45f0210 msgr2=0x55b9c45f8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:00.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.253+0000 7fde4e6d7640 1 --2- 172.21.15.78:0/1274401247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45f0210 0x55b9c45f8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.253+0000 7fde4e6d7640 1 -- 172.21.15.78:0/1274401247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9c44f4650 con 0x55b9c45fb3f0 2024-08-18T23:45:00.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.253+0000 7fde4d6d5640 1 --2- 172.21.15.78:0/1274401247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45f0b30 0x55b9c45f8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:45:00.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.253+0000 7fde52538740 1 -- 172.21.15.78:0/1274401247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde3809fd80 con 0x55b9c45fb3f0 2024-08-18T23:45:00.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.253+0000 7fde4e6d7640 1 --2- 172.21.15.78:0/1274401247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45fb3f0 0x55b9c45f8cb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fde40011ad0 tx=0x7fde40004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1499a8de15d4c0f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:00.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.253+0000 7fde317fa640 10 client.15585 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:45:00.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.254+0000 7fde317fa640 1 -- 172.21.15.78:0/1274401247 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40024030 con 0x55b9c45fb3f0 2024-08-18T23:45:00.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.254+0000 7fde317fa640 1 -- 172.21.15.78:0/1274401247 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fde40012c40 con 0x55b9c45fb3f0 2024-08-18T23:45:00.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.254+0000 7fde317fa640 1 -- 172.21.15.78:0/1274401247 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fde40012e40 con 0x55b9c45fb3f0 2024-08-18T23:45:00.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.254+0000 7fde52538740 10 client.15585 fetch_fsmap learned FSMap version 25 2024-08-18T23:45:00.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.254+0000 7fde52538740 1 -- 172.21.15.78:0/1274401247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55b9c42c73c0 con 0x55b9c45fb3f0 2024-08-18T23:45:00.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.254+0000 7fde317fa640 1 -- 172.21.15.78:0/1274401247 <== mon.0 v2:172.21.15.78:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fde4001d050 con 0x55b9c45fb3f0 2024-08-18T23:45:00.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.254+0000 7fde52538740 10 client.15585 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:45:00.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.254+0000 7fde52538740 10 client.15585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:45:00.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.254+0000 7fde52538740 20 client.15585 populate_metadata read hostname 'smithi078' 2024-08-18T23:45:00.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.254+0000 7fde52538740 1 --2- 172.21.15.78:0/1274401247 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b9c460aca0 0x55b9c460d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:00.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.254+0000 7fde52538740 4 client.15585 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:45:00.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.255+0000 7fde52538740 1 -- 172.21.15.78:0/1274401247 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9c4562ef0 con 0x55b9c460aca0 2024-08-18T23:45:00.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.255+0000 7fde4ded6640 1 --2- 172.21.15.78:0/1274401247 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b9c460aca0 0x55b9c460d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:00.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.255+0000 7fde4ded6640 1 --2- 172.21.15.78:0/1274401247 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b9c460aca0 0x55b9c460d130 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee138230cd467b82 server_cookie=e462f30626fa0e23 in_seq=0 out_seq=0 2024-08-18T23:45:00.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.255+0000 7fde317fa640 10 client.15585 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:00.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.256+0000 7fde317fa640 1 -- 172.21.15.78:0/1274401247 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9c4562ef0 con 0x55b9c460aca0 2024-08-18T23:45:00.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.256+0000 7fde317fa640 10 client.15585 handle_command_reply: tid=0 2024-08-18T23:45:00.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.256+0000 7fde52538740 1 client.15585 shutdown 2024-08-18T23:45:00.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.256+0000 7fde52538740 1 -- 172.21.15.78:0/1274401247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45fb3f0 msgr2=0x55b9c45f8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:00.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.256+0000 7fde52538740 1 --2- 172.21.15.78:0/1274401247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45fb3f0 0x55b9c45f8cb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fde40011ad0 tx=0x7fde40004ab0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.256+0000 7fde52538740 1 -- 172.21.15.78:0/1274401247 shutdown_connections 2024-08-18T23:45:00.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.256+0000 7fde52538740 1 --2- 172.21.15.78:0/1274401247 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b9c460aca0 0x55b9c460d130 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.256+0000 7fde52538740 1 --2- 172.21.15.78:0/1274401247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45fb3f0 0x55b9c45f8cb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.256+0000 7fde317fa640 0 client.15585 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:00.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.256+0000 7fde52538740 1 --2- 172.21.15.78:0/1274401247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45f0b30 0x55b9c45f8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.256+0000 7fde52538740 1 --2- 172.21.15.78:0/1274401247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45f0210 0x55b9c45f8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.256+0000 7fde52538740 1 -- 172.21.15.78:0/1274401247 >> 172.21.15.78:0/1274401247 conn(0x55b9c45ec7a0 msgr2=0x55b9c45f24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:00.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.257+0000 7fde52538740 1 -- 172.21.15.78:0/1274401247 shutdown_connections 2024-08-18T23:45:00.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.257+0000 7fde52538740 1 -- 172.21.15.78:0/1274401247 wait complete. 2024-08-18T23:45:00.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.257+0000 7fde52538740 20 client.15585 trim_cache size 0 max 16384 2024-08-18T23:45:00.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.290+0000 7fde52538740 1 Processor -- start 2024-08-18T23:45:00.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.290+0000 7fde52538740 1 -- start start 2024-08-18T23:45:00.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.291+0000 7fde52538740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45f0210 0x55b9c4612a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:00.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.291+0000 7fde52538740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45f0b30 0x55b9c4612f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:00.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.291+0000 7fde52538740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45fb3f0 0x55b9c45f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:00.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.291+0000 7fde52538740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fde481b3400 con 0x55b9c45f0210 2024-08-18T23:45:00.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.291+0000 7fde52538740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde481b0cd0 con 0x55b9c45f0b30 2024-08-18T23:45:00.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.291+0000 7fde52538740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fde481aeb20 con 0x55b9c45fb3f0 2024-08-18T23:45:00.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.291+0000 7fde4ded6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45f0210 0x55b9c4612a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:00.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.291+0000 7fde4ded6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45f0210 0x55b9c4612a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40772/0 (socket says 172.21.15.78:40772) 2024-08-18T23:45:00.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.291+0000 7fde4ded6640 1 -- 172.21.15.78:0/2864692673 learned_addr learned my addr 172.21.15.78:0/2864692673 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:00.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.291+0000 7fde4e6d7640 1 --2- 172.21.15.78:0/2864692673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45fb3f0 0x55b9c45f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:00.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.292+0000 7fde4d6d5640 1 --2- 172.21.15.78:0/2864692673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45f0b30 0x55b9c4612f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:00.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.292+0000 7fde4ded6640 1 -- 172.21.15.78:0/2864692673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45fb3f0 msgr2=0x55b9c45f69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:00.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.292+0000 7fde4ded6640 1 --2- 172.21.15.78:0/2864692673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45fb3f0 0x55b9c45f69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.292+0000 7fde4ded6640 1 -- 172.21.15.78:0/2864692673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45f0b30 msgr2=0x55b9c4612f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:00.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.292+0000 7fde4ded6640 1 --2- 172.21.15.78:0/2864692673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45f0b30 0x55b9c4612f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.292+0000 7fde4ded6640 1 -- 172.21.15.78:0/2864692673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c45ec5d0 con 0x55b9c45f0210 2024-08-18T23:45:00.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.292+0000 7fde4ded6640 1 --2- 172.21.15.78:0/2864692673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45f0210 0x55b9c4612a00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fde44009900 tx=0x7fde440036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524eb98b13b51e34 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:00.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.292+0000 7fde337fe640 1 -- 172.21.15.78:0/2864692673 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde44048030 con 0x55b9c45f0210 2024-08-18T23:45:00.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.292+0000 7fde337fe640 1 -- 172.21.15.78:0/2864692673 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde440055d0 con 0x55b9c45f0210 2024-08-18T23:45:00.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.292+0000 7fde337fe640 1 -- 172.21.15.78:0/2864692673 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde44035040 con 0x55b9c45f0210 2024-08-18T23:45:00.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.293+0000 7fde52538740 1 -- 172.21.15.78:0/2864692673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45f0210 msgr2=0x55b9c4612a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:00.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.293+0000 7fde52538740 1 --2- 172.21.15.78:0/2864692673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45f0210 0x55b9c4612a00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fde44009900 tx=0x7fde440036f0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.293+0000 7fde52538740 1 -- 172.21.15.78:0/2864692673 shutdown_connections 2024-08-18T23:45:00.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.293+0000 7fde52538740 1 --2- 172.21.15.78:0/2864692673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45fb3f0 0x55b9c45f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.293+0000 7fde52538740 1 --2- 172.21.15.78:0/2864692673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45f0b30 0x55b9c4612f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.293+0000 7fde52538740 1 --2- 172.21.15.78:0/2864692673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45f0210 0x55b9c4612a00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.293+0000 7fde52538740 1 -- 172.21.15.78:0/2864692673 >> 172.21.15.78:0/2864692673 conn(0x55b9c45ec7a0 msgr2=0x55b9c45eb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:00.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.294+0000 7fde52538740 1 -- 172.21.15.78:0/2864692673 shutdown_connections 2024-08-18T23:45:00.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.294+0000 7fde52538740 1 -- 172.21.15.78:0/2864692673 wait complete. 2024-08-18T23:45:00.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.294+0000 7fde52538740 1 -- start start 2024-08-18T23:45:00.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.294+0000 7fde52538740 1 Processor -- start 2024-08-18T23:45:00.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.295+0000 7fde52538740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45f0210 0x55b9c4603990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:00.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.295+0000 7fde52538740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45f0b30 0x55b9c45f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:00.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.295+0000 7fde52538740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45fb3f0 0x55b9c45f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:00.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.295+0000 7fde52538740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fde481af190 con 0x55b9c45f0210 2024-08-18T23:45:00.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.296+0000 7fde4ded6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45f0210 0x55b9c4603990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:00.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.296+0000 7fde52538740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde481b0600 con 0x55b9c45fb3f0 2024-08-18T23:45:00.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.296+0000 7fde52538740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fde481af860 con 0x55b9c45f0b30 2024-08-18T23:45:00.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.296+0000 7fde4d6d5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45f0b30 0x55b9c45f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:00.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.296+0000 7fde4e6d7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45fb3f0 0x55b9c45f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:00.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.296+0000 7fde4ded6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45f0210 0x55b9c4603990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40788/0 (socket says 172.21.15.78:40788) 2024-08-18T23:45:00.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.296+0000 7fde4ded6640 1 -- 172.21.15.78:0/3566337518 learned_addr learned my addr 172.21.15.78:0/3566337518 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:00.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.296+0000 7fde52538740 10 client.0 _ll_register_callbacks cb 0x7fde40014560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:45:00.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.296+0000 7fde4e6d7640 1 -- 172.21.15.78:0/3566337518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45f0b30 msgr2=0x55b9c45f4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:00.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.296+0000 7fde4e6d7640 1 --2- 172.21.15.78:0/3566337518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45f0b30 0x55b9c45f4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.296+0000 7fde4e6d7640 1 -- 172.21.15.78:0/3566337518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45f0210 msgr2=0x55b9c4603990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:00.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.296+0000 7fde4e6d7640 1 --2- 172.21.15.78:0/3566337518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45f0210 0x55b9c4603990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.296+0000 7fde4e6d7640 1 -- 172.21.15.78:0/3566337518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9c45f5960 con 0x55b9c45fb3f0 2024-08-18T23:45:00.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.296+0000 7fde4ded6640 1 --2- 172.21.15.78:0/3566337518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45f0210 0x55b9c4603990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:45:00.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.296+0000 7fde4e6d7640 1 --2- 172.21.15.78:0/3566337518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45fb3f0 0x55b9c45f52f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fde40006010 tx=0x7fde4000f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1905d5b8380d8c27 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:00.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.296+0000 7fde317fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:45:00.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.297+0000 7fde317fa640 1 -- 172.21.15.78:0/3566337518 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40037070 con 0x55b9c45fb3f0 2024-08-18T23:45:00.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.297+0000 7fde317fa640 1 -- 172.21.15.78:0/3566337518 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fde40020d30 con 0x55b9c45fb3f0 2024-08-18T23:45:00.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.297+0000 7fde52538740 1 -- 172.21.15.78:0/3566337518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde440055d0 con 0x55b9c45fb3f0 2024-08-18T23:45:00.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.297+0000 7fde317fa640 1 -- 172.21.15.78:0/3566337518 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fde40043050 con 0x55b9c45fb3f0 2024-08-18T23:45:00.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.297+0000 7fde52538740 10 client.25076 fetch_fsmap learned FSMap version 25 2024-08-18T23:45:00.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.297+0000 7fde52538740 1 -- 172.21.15.78:0/3566337518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b9c46022a0 con 0x55b9c45fb3f0 2024-08-18T23:45:00.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.297+0000 7fde317fa640 1 -- 172.21.15.78:0/3566337518 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde440055d0 con 0x55b9c45fb3f0 2024-08-18T23:45:00.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.297+0000 7fde317fa640 1 -- 172.21.15.78:0/3566337518 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fde4001d070 con 0x55b9c45fb3f0 2024-08-18T23:45:00.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.297+0000 7fde52538740 10 client.25076 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:45:00.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.297+0000 7fde52538740 10 client.25076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:45:00.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.297+0000 7fde52538740 20 client.25076 populate_metadata read hostname 'smithi078' 2024-08-18T23:45:00.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.297+0000 7fde52538740 1 --2- 172.21.15.78:0/3566337518 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b9c460aca0 0x55b9c4677f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.298+0000 7fde52538740 4 client.25076 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.298+0000 7fde52538740 1 -- 172.21.15.78:0/3566337518 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b9c455d400 con 0x55b9c460aca0 2024-08-18T23:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.298+0000 7fde4ded6640 1 --2- 172.21.15.78:0/3566337518 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b9c460aca0 0x55b9c4677f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.298+0000 7fde4ded6640 1 --2- 172.21.15.78:0/3566337518 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b9c460aca0 0x55b9c4677f50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fa33ad26c4a8384 server_cookie=ccf62ee2b92ad7b3 in_seq=0 out_seq=0 2024-08-18T23:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.298+0000 7fde317fa640 10 client.25076 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.298+0000 7fde317fa640 1 -- 172.21.15.78:0/3566337518 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b9c455d400 con 0x55b9c460aca0 2024-08-18T23:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.298+0000 7fde317fa640 10 client.25076 handle_command_reply: tid=0 2024-08-18T23:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.298+0000 7fde52538740 1 client.25076 shutdown 2024-08-18T23:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.299+0000 7fde52538740 1 -- 172.21.15.78:0/3566337518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45fb3f0 msgr2=0x55b9c45f52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.299+0000 7fde52538740 1 --2- 172.21.15.78:0/3566337518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45fb3f0 0x55b9c45f52f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fde40006010 tx=0x7fde4000f000 comp rx=0 tx=0).stop 2024-08-18T23:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.299+0000 7fde52538740 1 -- 172.21.15.78:0/3566337518 shutdown_connections 2024-08-18T23:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.299+0000 7fde52538740 1 --2- 172.21.15.78:0/3566337518 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b9c460aca0 0x55b9c4677f50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.299+0000 7fde52538740 1 --2- 172.21.15.78:0/3566337518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b9c45fb3f0 0x55b9c45f52f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.299+0000 7fde52538740 1 --2- 172.21.15.78:0/3566337518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b9c45f0b30 0x55b9c45f4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.299+0000 7fde317fa640 0 client.25076 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:00.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.299+0000 7fde52538740 1 --2- 172.21.15.78:0/3566337518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9c45f0210 0x55b9c4603990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.299+0000 7fde52538740 1 -- 172.21.15.78:0/3566337518 >> 172.21.15.78:0/3566337518 conn(0x55b9c45ec7a0 msgr2=0x55b9c45f69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:00.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.299+0000 7fde52538740 1 -- 172.21.15.78:0/3566337518 shutdown_connections 2024-08-18T23:45:00.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.299+0000 7fde52538740 1 -- 172.21.15.78:0/3566337518 wait complete. 2024-08-18T23:45:00.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.299+0000 7fde52538740 20 client.25076 trim_cache size 0 max 16384 2024-08-18T23:45:00.304 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:45:00.305 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-18T23:45:00.305 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-08-18T23:45:00.305 INFO:teuthology.orchestra.run.smithi078.stdout: "582d906a-c4f8-494f-a632-0eee0a5f385c": { 2024-08-18T23:45:00.305 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-08-18T23:45:00.305 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "582d906a-c4f8-494f-a632-0eee0a5f385c", 2024-08-18T23:45:00.305 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-08-18T23:45:00.305 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:45:00.305 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:45:00.305 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:45:00.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.300+0000 7fde5013a640 1 -- 172.21.15.78:0/3036710572 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fde340a2c90 msgr2=0x7fde340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:00.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.300+0000 7fde5013a640 1 --2- 172.21.15.78:0/3036710572 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fde340a2c90 0x7fde340a5150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55b9c45edb20 tx=0x7fde40009530 comp rx=0 tx=0).stop 2024-08-18T23:45:00.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.300+0000 7fde5013a640 1 -- 172.21.15.78:0/3036710572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde481022a0 msgr2=0x7fde481975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:00.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.300+0000 7fde5013a640 1 --2- 172.21.15.78:0/3036710572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde481022a0 0x7fde481975b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fde3800c920 tx=0x7fde3800cde0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.300+0000 7fde5013a640 1 -- 172.21.15.78:0/3036710572 shutdown_connections 2024-08-18T23:45:00.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.300+0000 7fde5013a640 1 --2- 172.21.15.78:0/3036710572 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fde340a2c90 0x7fde340a5150 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.300+0000 7fde5013a640 1 --2- 172.21.15.78:0/3036710572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde48102c20 0x7fde4819b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.300+0000 7fde5013a640 1 --2- 172.21.15.78:0/3036710572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde481022a0 0x7fde481975b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.300+0000 7fde5013a640 1 --2- 172.21.15.78:0/3036710572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde48100b50 0x7fde48197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:00.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.300+0000 7fde5013a640 1 -- 172.21.15.78:0/3036710572 >> 172.21.15.78:0/3036710572 conn(0x7fde480fc500 msgr2=0x7fde48106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:00.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.300+0000 7fde5013a640 1 -- 172.21.15.78:0/3036710572 shutdown_connections 2024-08-18T23:45:00.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:00.300+0000 7fde5013a640 1 -- 172.21.15.78:0/3036710572 wait complete. 2024-08-18T23:45:01.107 INFO:tasks.cephfs.filesystem:scrub status for tag:582d906a-c4f8-494f-a632-0eee0a5f385c - {'path': '/', 'tag': '582d906a-c4f8-494f-a632-0eee0a5f385c', 'options': 'recursive,force'} 2024-08-18T23:45:01.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:01 smithi078 ceph-mon[18440]: pgmap v371: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-08-18T23:45:01.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:01 smithi123 ceph-mon[21808]: pgmap v371: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-08-18T23:45:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:01 smithi161 ceph-mon[22027]: pgmap v371: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 7 op/s 2024-08-18T23:45:03.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:03 smithi123 ceph-mon[21808]: pgmap v372: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-18T23:45:03.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:03 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:45:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:03 smithi161 ceph-mon[22027]: pgmap v372: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-18T23:45:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:03 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:45:03.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:03 smithi078 ceph-mon[18440]: pgmap v372: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-18T23:45:03.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:03 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:45:05.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:05 smithi123 ceph-mon[21808]: pgmap v373: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-18T23:45:05.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:05 smithi161 ceph-mon[22027]: pgmap v373: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-18T23:45:05.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:05 smithi078 ceph-mon[18440]: pgmap v373: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-18T23:45:07.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:07 smithi123 ceph-mon[21808]: pgmap v374: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 6 op/s 2024-08-18T23:45:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:07 smithi161 ceph-mon[22027]: pgmap v374: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 6 op/s 2024-08-18T23:45:07.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:07 smithi078 ceph-mon[18440]: pgmap v374: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 6 op/s 2024-08-18T23:45:09.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:09 smithi123 ceph-mon[21808]: pgmap v375: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-18T23:45:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:09 smithi161 ceph-mon[22027]: pgmap v375: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-18T23:45:09.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:09 smithi078 ceph-mon[18440]: pgmap v375: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-18T23:45:11.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:11 smithi123 ceph-mon[21808]: pgmap v376: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-08-18T23:45:11.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:11 smithi161 ceph-mon[22027]: pgmap v376: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-08-18T23:45:11.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:11 smithi078 ceph-mon[18440]: pgmap v376: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-08-18T23:45:13.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:13 smithi123 ceph-mon[21808]: pgmap v377: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-08-18T23:45:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:13 smithi161 ceph-mon[22027]: pgmap v377: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-08-18T23:45:13.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:13 smithi078 ceph-mon[18440]: pgmap v377: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-08-18T23:45:15.372 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:45:15.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:15 smithi123 ceph-mon[21808]: pgmap v378: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-18T23:45:15.626 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:45:15.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:15 smithi161 ceph-mon[22027]: pgmap v378: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-18T23:45:15.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:15 smithi078 ceph-mon[18440]: pgmap v378: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-18T23:45:15.789 INFO:tasks.workunit.client.0.smithi078.stderr:Note: switching to '78215f5a1cf5eaf321d506a66539942e12851f34'. 2024-08-18T23:45:15.789 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-08-18T23:45:15.789 INFO:tasks.workunit.client.0.smithi078.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-08-18T23:45:15.789 INFO:tasks.workunit.client.0.smithi078.stderr:changes and commit them, and you can discard any commits you make in this 2024-08-18T23:45:15.789 INFO:tasks.workunit.client.0.smithi078.stderr:state without impacting any branches by switching back to a branch. 2024-08-18T23:45:15.789 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-08-18T23:45:15.790 INFO:tasks.workunit.client.0.smithi078.stderr:If you want to create a new branch to retain commits you create, you may 2024-08-18T23:45:15.790 INFO:tasks.workunit.client.0.smithi078.stderr:do so (now or later) by using -c with the switch command. Example: 2024-08-18T23:45:15.790 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-08-18T23:45:15.790 INFO:tasks.workunit.client.0.smithi078.stderr: git switch -c 2024-08-18T23:45:15.790 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-08-18T23:45:15.790 INFO:tasks.workunit.client.0.smithi078.stderr:Or undo this operation with: 2024-08-18T23:45:15.790 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-08-18T23:45:15.790 INFO:tasks.workunit.client.0.smithi078.stderr: git switch - 2024-08-18T23:45:15.790 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-08-18T23:45:15.790 INFO:tasks.workunit.client.0.smithi078.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-08-18T23:45:15.790 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-08-18T23:45:15.790 INFO:tasks.workunit.client.0.smithi078.stderr:HEAD is now at 78215f5a1cf Merge pull request #59291 from zdover23/wip-doc-2024-08-19-backport-59290-to-squid 2024-08-18T23:45:15.795 DEBUG:teuthology.orchestra.run.smithi078:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-08-18T23:45:15.841 INFO:tasks.workunit.client.0.smithi078.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-08-18T23:45:15.843 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-18T23:45:15.843 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-08-18T23:45:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.304+0000 7fa71daa7640 1 -- 172.21.15.78:0/3195621704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7181078b0 msgr2=0x7fa718107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.304+0000 7fa71daa7640 1 --2- 172.21.15.78:0/3195621704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7181078b0 0x7fa718107c90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa708009930 tx=0x7fa70802a420 comp rx=0 tx=0).stop 2024-08-18T23:45:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.304+0000 7fa71daa7640 1 -- 172.21.15.78:0/3195621704 shutdown_connections 2024-08-18T23:45:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.304+0000 7fa71daa7640 1 --2- 172.21.15.78:0/3195621704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa718102250 0x7fa71810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.304+0000 7fa71daa7640 1 --2- 172.21.15.78:0/3195621704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7181018b0 0x7fa718101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.304+0000 7fa71daa7640 1 --2- 172.21.15.78:0/3195621704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7181078b0 0x7fa718107c90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.304+0000 7fa71daa7640 1 -- 172.21.15.78:0/3195621704 >> 172.21.15.78:0/3195621704 conn(0x7fa7180fd5f0 msgr2=0x7fa7180ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.304+0000 7fa71daa7640 1 -- 172.21.15.78:0/3195621704 shutdown_connections 2024-08-18T23:45:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.304+0000 7fa71daa7640 1 -- 172.21.15.78:0/3195621704 wait complete. 2024-08-18T23:45:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.305+0000 7fa71daa7640 1 Processor -- start 2024-08-18T23:45:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.305+0000 7fa71daa7640 1 -- start start 2024-08-18T23:45:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.305+0000 7fa71daa7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7181018b0 0x7fa7181a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.305+0000 7fa71daa7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa718102250 0x7fa7181a3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.305+0000 7fa71daa7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7181078b0 0x7fa71810d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.305+0000 7fa71daa7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa718111cd0 con 0x7fa7181078b0 2024-08-18T23:45:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.305+0000 7fa71daa7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa718111b50 con 0x7fa7181018b0 2024-08-18T23:45:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.305+0000 7fa71daa7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa718111e50 con 0x7fa718102250 2024-08-18T23:45:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.306+0000 7fa717fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7181078b0 0x7fa71810d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.306+0000 7fa7177fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7181018b0 0x7fa7181a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.306+0000 7fa716ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa718102250 0x7fa7181a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:16.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.306+0000 7fa717fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7181078b0 0x7fa71810d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42760/0 (socket says 172.21.15.78:42760) 2024-08-18T23:45:16.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.306+0000 7fa717fff640 1 -- 172.21.15.78:0/3668563252 learned_addr learned my addr 172.21.15.78:0/3668563252 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:16.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.306+0000 7fa7177fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7181018b0 0x7fa7181a3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:55040/0 (socket says 172.21.15.78:55040) 2024-08-18T23:45:16.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.306+0000 7fa717fff640 1 -- 172.21.15.78:0/3668563252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa718102250 msgr2=0x7fa7181a3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:16.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.306+0000 7fa717fff640 1 --2- 172.21.15.78:0/3668563252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa718102250 0x7fa7181a3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:16.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.306+0000 7fa717fff640 1 -- 172.21.15.78:0/3668563252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7181018b0 msgr2=0x7fa7181a3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:16.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.306+0000 7fa717fff640 1 --2- 172.21.15.78:0/3668563252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7181018b0 0x7fa7181a3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:16.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.306+0000 7fa717fff640 1 -- 172.21.15.78:0/3668563252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71810e080 con 0x7fa7181078b0 2024-08-18T23:45:16.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.306+0000 7fa717fff640 1 --2- 172.21.15.78:0/3668563252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7181078b0 0x7fa71810d930 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa71000b970 tx=0x7fa71000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:16.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.306+0000 7fa714ff9640 1 -- 172.21.15.78:0/3668563252 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710019070 con 0x7fa7181078b0 2024-08-18T23:45:16.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.306+0000 7fa71daa7640 1 -- 172.21.15.78:0/3668563252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71810e370 con 0x7fa7181078b0 2024-08-18T23:45:16.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.306+0000 7fa714ff9640 1 -- 172.21.15.78:0/3668563252 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa71000cb40 con 0x7fa7181078b0 2024-08-18T23:45:16.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.306+0000 7fa714ff9640 1 -- 172.21.15.78:0/3668563252 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710007650 con 0x7fa7181078b0 2024-08-18T23:45:16.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.306+0000 7fa71daa7640 1 -- 172.21.15.78:0/3668563252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7181ab340 con 0x7fa7181078b0 2024-08-18T23:45:16.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.308+0000 7fa71daa7640 1 -- 172.21.15.78:0/3668563252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa718101d10 con 0x7fa7181078b0 2024-08-18T23:45:16.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.309+0000 7fa714ff9640 1 -- 172.21.15.78:0/3668563252 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fa71001c070 con 0x7fa7181078b0 2024-08-18T23:45:16.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.309+0000 7fa714ff9640 1 --2- 172.21.15.78:0/3668563252 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa7000a2c10 0x7fa7000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:16.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.310+0000 7fa7177fe640 1 --2- 172.21.15.78:0/3668563252 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa7000a2c10 0x7fa7000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:16.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.310+0000 7fa714ff9640 1 -- 172.21.15.78:0/3668563252 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa7100cea30 con 0x7fa7181078b0 2024-08-18T23:45:16.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.310+0000 7fa7177fe640 1 --2- 172.21.15.78:0/3668563252 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa7000a2c10 0x7fa7000a50d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa708009900 tx=0x7fa7080058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:16.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.314+0000 7fa714ff9640 1 -- 172.21.15.78:0/3668563252 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa71009af60 con 0x7fa7181078b0 2024-08-18T23:45:16.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.518+0000 7fa71daa7640 1 -- 172.21.15.78:0/3668563252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7181ab610 con 0x7fa7181078b0 2024-08-18T23:45:16.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.518+0000 7fa714ff9640 1 -- 172.21.15.78:0/3668563252 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7fa71009f260 con 0x7fa7181078b0 2024-08-18T23:45:16.520 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:45:16.520 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":25,"btime":"2024-08-18T23:43:57:245845+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:43:57.114120+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24428,24427]},"id":1}]} 2024-08-18T23:45:16.520 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 25 2024-08-18T23:45:16.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.520+0000 7fa71daa7640 1 -- 172.21.15.78:0/3668563252 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa7000a2c10 msgr2=0x7fa7000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:16.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.520+0000 7fa71daa7640 1 --2- 172.21.15.78:0/3668563252 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa7000a2c10 0x7fa7000a50d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa708009900 tx=0x7fa7080058d0 comp rx=0 tx=0).stop 2024-08-18T23:45:16.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.520+0000 7fa71daa7640 1 -- 172.21.15.78:0/3668563252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7181078b0 msgr2=0x7fa71810d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:16.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.520+0000 7fa71daa7640 1 --2- 172.21.15.78:0/3668563252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7181078b0 0x7fa71810d930 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa71000b970 tx=0x7fa71000be30 comp rx=0 tx=0).stop 2024-08-18T23:45:16.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.520+0000 7fa71daa7640 1 -- 172.21.15.78:0/3668563252 shutdown_connections 2024-08-18T23:45:16.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.520+0000 7fa71daa7640 1 --2- 172.21.15.78:0/3668563252 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa7000a2c10 0x7fa7000a50d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:16.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.520+0000 7fa71daa7640 1 --2- 172.21.15.78:0/3668563252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7181078b0 0x7fa71810d930 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:16.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.520+0000 7fa71daa7640 1 --2- 172.21.15.78:0/3668563252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa718102250 0x7fa7181a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:16.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.520+0000 7fa71daa7640 1 --2- 172.21.15.78:0/3668563252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7181018b0 0x7fa7181a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:16.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.520+0000 7fa71daa7640 1 -- 172.21.15.78:0/3668563252 >> 172.21.15.78:0/3668563252 conn(0x7fa7180fd5f0 msgr2=0x7fa7180fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:16.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.520+0000 7fa71daa7640 1 -- 172.21.15.78:0/3668563252 shutdown_connections 2024-08-18T23:45:16.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:16.520+0000 7fa71daa7640 1 -- 172.21.15.78:0/3668563252 wait complete. 2024-08-18T23:45:17.341 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id f246fd05 --release --await' 2024-08-18T23:45:17.341 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id f246fd05 --release --await 2024-08-18T23:45:17.537 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-08-18T23:45:17.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:17 smithi123 ceph-mon[21808]: pgmap v379: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-18T23:45:17.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:17 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3668563252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:45:17.602 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:45:17.614 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-08-18T23:45:17.655 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-18T23:45:17.657 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-18T23:45:17.657 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-08-18T23:45:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:17 smithi161 ceph-mon[22027]: pgmap v379: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-18T23:45:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:17 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3668563252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:45:17.695 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-18T23:45:17.698 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:45:17.698 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-08-18T23:45:17.714 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-08-18T23:45:17.715 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-08-18T23:45:17.716 DEBUG:teuthology.orchestra.run.smithi078:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=78215f5a1cf5eaf321d506a66539942e12851f34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-08-18T23:45:17.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:17 smithi078 ceph-mon[18440]: pgmap v379: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-18T23:45:17.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:17 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3668563252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:45:18.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.113+0000 7fb8cfbd7640 1 -- 172.21.15.78:0/1607462450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c81007b0 msgr2=0x7fb8c8100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:18.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.113+0000 7fb8cfbd7640 1 --2- 172.21.15.78:0/1607462450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c81007b0 0x7fb8c8100c10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009930 tx=0x7fb8b802a420 comp rx=0 tx=0).stop 2024-08-18T23:45:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.113+0000 7fb8cfbd7640 1 -- 172.21.15.78:0/1607462450 shutdown_connections 2024-08-18T23:45:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.113+0000 7fb8cfbd7640 1 --2- 172.21.15.78:0/1607462450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8c8101150 0x7fb8c810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.113+0000 7fb8cfbd7640 1 --2- 172.21.15.78:0/1607462450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c81007b0 0x7fb8c8100c10 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.113+0000 7fb8cfbd7640 1 --2- 172.21.15.78:0/1607462450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8c81067b0 0x7fb8c8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.113+0000 7fb8cfbd7640 1 -- 172.21.15.78:0/1607462450 >> 172.21.15.78:0/1607462450 conn(0x7fb8c80fc4d0 msgr2=0x7fb8c80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.113+0000 7fb8cfbd7640 1 -- 172.21.15.78:0/1607462450 shutdown_connections 2024-08-18T23:45:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.113+0000 7fb8cfbd7640 1 -- 172.21.15.78:0/1607462450 wait complete. 2024-08-18T23:45:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.114+0000 7fb8cfbd7640 1 Processor -- start 2024-08-18T23:45:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.114+0000 7fb8cfbd7640 1 -- start start 2024-08-18T23:45:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.114+0000 7fb8cfbd7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8c81007b0 0x7fb8c8198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.114+0000 7fb8cfbd7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8c8101150 0x7fb8c81987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:18.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.114+0000 7fb8cfbd7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c81067b0 0x7fb8c819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.114+0000 7fb8cfbd7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c8110bf0 con 0x7fb8c81067b0 2024-08-18T23:45:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.114+0000 7fb8cfbd7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c8110a70 con 0x7fb8c81007b0 2024-08-18T23:45:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.114+0000 7fb8cfbd7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c8110d70 con 0x7fb8c8101150 2024-08-18T23:45:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.114+0000 7fb8ce174640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c81067b0 0x7fb8c819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.114+0000 7fb8ce174640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c81067b0 0x7fb8c819cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42786/0 (socket says 172.21.15.78:42786) 2024-08-18T23:45:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.114+0000 7fb8ce174640 1 -- 172.21.15.78:0/1916874548 learned_addr learned my addr 172.21.15.78:0/1916874548 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.114+0000 7fb8cd973640 1 --2- 172.21.15.78:0/1916874548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8c81007b0 0x7fb8c8198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.114+0000 7fb8cd172640 1 --2- 172.21.15.78:0/1916874548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8c8101150 0x7fb8c81987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.114+0000 7fb8ce174640 1 -- 172.21.15.78:0/1916874548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8c8101150 msgr2=0x7fb8c81987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.115+0000 7fb8ce174640 1 --2- 172.21.15.78:0/1916874548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8c8101150 0x7fb8c81987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.115+0000 7fb8ce174640 1 -- 172.21.15.78:0/1916874548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8c81007b0 msgr2=0x7fb8c8198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.115+0000 7fb8ce174640 1 --2- 172.21.15.78:0/1916874548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8c81007b0 0x7fb8c8198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.115+0000 7fb8ce174640 1 -- 172.21.15.78:0/1916874548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c819d210 con 0x7fb8c81067b0 2024-08-18T23:45:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.115+0000 7fb8ce174640 1 --2- 172.21.15.78:0/1916874548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c81067b0 0x7fb8c819cb30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000cad0 tx=0x7fb8c000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.115+0000 7fb8beffd640 1 -- 172.21.15.78:0/1916874548 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c000ae20 con 0x7fb8c81067b0 2024-08-18T23:45:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.115+0000 7fb8cfbd7640 1 -- 172.21.15.78:0/1916874548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c819d500 con 0x7fb8c81067b0 2024-08-18T23:45:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.115+0000 7fb8beffd640 1 -- 172.21.15.78:0/1916874548 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c0005020 con 0x7fb8c81067b0 2024-08-18T23:45:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.115+0000 7fb8beffd640 1 -- 172.21.15.78:0/1916874548 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c000f6b0 con 0x7fb8c81067b0 2024-08-18T23:45:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.115+0000 7fb8cfbd7640 1 -- 172.21.15.78:0/1916874548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c81a4de0 con 0x7fb8c81067b0 2024-08-18T23:45:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.117+0000 7fb8beffd640 1 -- 172.21.15.78:0/1916874548 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fb8c00051d0 con 0x7fb8c81067b0 2024-08-18T23:45:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.117+0000 7fb8d1fd5740 1 Processor -- start 2024-08-18T23:45:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.117+0000 7fb8d1fd5740 1 -- start start 2024-08-18T23:45:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8d1fd5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efaf990 0x56147efafd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8d1fd5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efb02b0 0x56147efba630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8d1fd5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56147efbab70 0x56147efbcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:18.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8d1fd5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56147ef15c60 con 0x56147efaf990 2024-08-18T23:45:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8d1fd5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56147eeadb90 con 0x56147efb02b0 2024-08-18T23:45:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8d1fd5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56147ef11c20 con 0x56147efbab70 2024-08-18T23:45:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8beffd640 1 --2- 172.21.15.78:0/1916874548 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb8b40a2c10 0x7fb8b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8cd172640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efaf990 0x56147efafd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8cd172640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efaf990 0x56147efafd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42802/0 (socket says 172.21.15.78:42802) 2024-08-18T23:45:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8cd172640 1 -- 172.21.15.78:0/1610943218 learned_addr learned my addr 172.21.15.78:0/1610943218 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8ce174640 1 --2- 172.21.15.78:0/1916874548 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb8b40a2c10 0x7fb8b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8cd973640 1 --2- 172.21.15.78:0/1610943218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efb02b0 0x56147efba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8beffd640 1 -- 172.21.15.78:0/1916874548 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fb8c00cf9a0 con 0x7fb8c81067b0 2024-08-18T23:45:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8cd172640 1 -- 172.21.15.78:0/1610943218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56147efbab70 msgr2=0x56147efbcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8cd172640 1 --2- 172.21.15.78:0/1610943218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56147efbab70 0x56147efbcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8cd172640 1 -- 172.21.15.78:0/1610943218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efb02b0 msgr2=0x56147efba630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8cd172640 1 --2- 172.21.15.78:0/1610943218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efb02b0 0x56147efba630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8cd172640 1 -- 172.21.15.78:0/1610943218 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56147ec25580 con 0x56147efaf990 2024-08-18T23:45:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8cd172640 1 --2- 172.21.15.78:0/1610943218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efaf990 0x56147efafd70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80096c0 tx=0x7fb8b8032d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3cafc98182602b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.118+0000 7fb8ce174640 1 --2- 172.21.15.78:0/1916874548 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb8b40a2c10 0x7fb8b40a50d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb8c00986e0 tx=0x7fb8c0005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.119+0000 7fb8b3fff640 1 -- 172.21.15.78:0/1610943218 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b803f070 con 0x56147efaf990 2024-08-18T23:45:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.119+0000 7fb8b3fff640 1 -- 172.21.15.78:0/1610943218 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8b80317b0 con 0x56147efaf990 2024-08-18T23:45:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.119+0000 7fb8b3fff640 1 -- 172.21.15.78:0/1610943218 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8035070 con 0x56147efaf990 2024-08-18T23:45:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.119+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/1610943218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efaf990 msgr2=0x56147efafd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.119+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/1610943218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efaf990 0x56147efafd70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80096c0 tx=0x7fb8b8032d00 comp rx=0 tx=0).stop 2024-08-18T23:45:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.119+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/1610943218 shutdown_connections 2024-08-18T23:45:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.119+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/1610943218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56147efbab70 0x56147efbcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.119+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/1610943218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efb02b0 0x56147efba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.119+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/1610943218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efaf990 0x56147efafd70 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.119+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/1610943218 >> 172.21.15.78:0/1610943218 conn(0x56147efabf20 msgr2=0x56147efac300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:18.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.119+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/1610943218 shutdown_connections 2024-08-18T23:45:18.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.120+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/1610943218 wait complete. 2024-08-18T23:45:18.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.120+0000 7fb8d1fd5740 1 -- start start 2024-08-18T23:45:18.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.120+0000 7fb8d1fd5740 1 Processor -- start 2024-08-18T23:45:18.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.120+0000 7fb8d1fd5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efaf990 0x56147efb7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:18.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.120+0000 7fb8d1fd5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56147efb02b0 0x56147efb8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:18.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.120+0000 7fb8d1fd5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efbab70 0x56147efb8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:18.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.120+0000 7fb8d1fd5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56147efbd740 con 0x56147efbab70 2024-08-18T23:45:18.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.120+0000 7fb8d1fd5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56147eeb37c0 con 0x56147efaf990 2024-08-18T23:45:18.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.120+0000 7fb8d1fd5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56147efbd8c0 con 0x56147efb02b0 2024-08-18T23:45:18.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.120+0000 7fb8d1fd5740 10 client.0 _ll_register_callbacks cb 0x56147eebcf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:45:18.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.120+0000 7fb8cd172640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efbab70 0x56147efb8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:18.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.120+0000 7fb8cd172640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efbab70 0x56147efb8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42818/0 (socket says 172.21.15.78:42818) 2024-08-18T23:45:18.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.120+0000 7fb8cd172640 1 -- 172.21.15.78:0/17131952 learned_addr learned my addr 172.21.15.78:0/17131952 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:18.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.120+0000 7fb8cd172640 1 --2- 172.21.15.78:0/17131952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efaf990 0x56147efb7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:18.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.121+0000 7fb8cd973640 1 --2- 172.21.15.78:0/17131952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56147efb02b0 0x56147efb8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:18.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.121+0000 7fb8cd172640 1 -- 172.21.15.78:0/17131952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56147efb02b0 msgr2=0x56147efb8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:18.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.121+0000 7fb8cd172640 1 --2- 172.21.15.78:0/17131952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56147efb02b0 0x56147efb8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.121+0000 7fb8cd172640 1 -- 172.21.15.78:0/17131952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efaf990 msgr2=0x56147efb7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:18.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.121+0000 7fb8cd172640 1 --2- 172.21.15.78:0/17131952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efaf990 0x56147efb7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.121+0000 7fb8cd172640 1 -- 172.21.15.78:0/17131952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56147ecef960 con 0x56147efbab70 2024-08-18T23:45:18.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.121+0000 7fb8cd172640 1 --2- 172.21.15.78:0/17131952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efbab70 0x56147efb8550 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009a50 tx=0x7fb8b802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a175858c4f51f146 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:18.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.121+0000 7fb8b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:45:18.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.121+0000 7fb8b17fa640 1 -- 172.21.15.78:0/17131952 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b803f070 con 0x56147efbab70 2024-08-18T23:45:18.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.121+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/17131952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8b80317b0 con 0x56147efbab70 2024-08-18T23:45:18.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.121+0000 7fb8b17fa640 1 -- 172.21.15.78:0/17131952 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fb8b8004030 con 0x56147efbab70 2024-08-18T23:45:18.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.121+0000 7fb8b17fa640 1 -- 172.21.15.78:0/17131952 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb8b8035040 con 0x56147efbab70 2024-08-18T23:45:18.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.121+0000 7fb8d1fd5740 10 client.15609 fetch_fsmap learned FSMap version 25 2024-08-18T23:45:18.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.121+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/17131952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56147efc9800 con 0x56147efbab70 2024-08-18T23:45:18.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.121+0000 7fb8b17fa640 1 -- 172.21.15.78:0/17131952 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8b80317b0 con 0x56147efbab70 2024-08-18T23:45:18.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.122+0000 7fb8b17fa640 1 -- 172.21.15.78:0/17131952 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb8b80035f0 con 0x56147efbab70 2024-08-18T23:45:18.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.122+0000 7fb8d1fd5740 10 client.15609 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:45:18.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.122+0000 7fb8d1fd5740 10 client.15609 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:45:18.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.122+0000 7fb8d1fd5740 20 client.15609 populate_metadata read hostname 'smithi078' 2024-08-18T23:45:18.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.122+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/17131952 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56147efcab50 0x56147efccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:18.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.122+0000 7fb8d1fd5740 4 client.15609 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:45:18.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.122+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/17131952 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56147ef231d0 con 0x56147efcab50 2024-08-18T23:45:18.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.122+0000 7fb8cd973640 1 --2- 172.21.15.78:0/17131952 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56147efcab50 0x56147efccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:18.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.122+0000 7fb8cd973640 1 --2- 172.21.15.78:0/17131952 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56147efcab50 0x56147efccf40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ad795ef340b5354 server_cookie=168c8c26f058d10e in_seq=0 out_seq=0 2024-08-18T23:45:18.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.122+0000 7fb8b17fa640 10 client.15609 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:18.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.123+0000 7fb8b17fa640 1 -- 172.21.15.78:0/17131952 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56147ef231d0 con 0x56147efcab50 2024-08-18T23:45:18.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.123+0000 7fb8b17fa640 10 client.15609 handle_command_reply: tid=0 2024-08-18T23:45:18.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.123+0000 7fb8d1fd5740 1 client.15609 shutdown 2024-08-18T23:45:18.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.123+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/17131952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efbab70 msgr2=0x56147efb8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:18.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.123+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/17131952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efbab70 0x56147efb8550 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009a50 tx=0x7fb8b802a420 comp rx=0 tx=0).stop 2024-08-18T23:45:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.123+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/17131952 shutdown_connections 2024-08-18T23:45:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.123+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/17131952 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56147efcab50 0x56147efccf40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.123+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/17131952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efbab70 0x56147efb8550 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.123+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/17131952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56147efb02b0 0x56147efb8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.123+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/17131952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efaf990 0x56147efb7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.123+0000 7fb8b17fa640 0 client.15609 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.123+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/17131952 >> 172.21.15.78:0/17131952 conn(0x56147efabf20 msgr2=0x56147efb1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.123+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/17131952 shutdown_connections 2024-08-18T23:45:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.123+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/17131952 wait complete. 2024-08-18T23:45:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.123+0000 7fb8d1fd5740 20 client.15609 trim_cache size 0 max 16384 2024-08-18T23:45:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.140+0000 7fb8d1fd5740 1 Processor -- start 2024-08-18T23:45:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.140+0000 7fb8d1fd5740 1 -- start start 2024-08-18T23:45:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.140+0000 7fb8d1fd5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56147efaf990 0x56147efb23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.140+0000 7fb8d1fd5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efb02b0 0x56147efcff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.140+0000 7fb8d1fd5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efbab70 0x56147efd04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.140+0000 7fb8d1fd5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c81b13c0 con 0x56147efb02b0 2024-08-18T23:45:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.140+0000 7fb8d1fd5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c81aec90 con 0x56147efbab70 2024-08-18T23:45:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.140+0000 7fb8d1fd5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c81acae0 con 0x56147efaf990 2024-08-18T23:45:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.140+0000 7fb8cd973640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efb02b0 0x56147efcff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.140+0000 7fb8cd973640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efb02b0 0x56147efcff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42820/0 (socket says 172.21.15.78:42820) 2024-08-18T23:45:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.140+0000 7fb8cd973640 1 -- 172.21.15.78:0/1163752457 learned_addr learned my addr 172.21.15.78:0/1163752457 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.141+0000 7fb8cd973640 1 -- 172.21.15.78:0/1163752457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56147efaf990 msgr2=0x56147efb23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.141+0000 7fb8cd973640 1 --2- 172.21.15.78:0/1163752457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56147efaf990 0x56147efb23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.141+0000 7fb8cd973640 1 -- 172.21.15.78:0/1163752457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efbab70 msgr2=0x56147efd04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.141+0000 7fb8cd973640 1 --2- 172.21.15.78:0/1163752457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efbab70 0x56147efd04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.141+0000 7fb8cd973640 1 -- 172.21.15.78:0/1163752457 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56147efacfd0 con 0x56147efb02b0 2024-08-18T23:45:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.141+0000 7fb8cd973640 1 --2- 172.21.15.78:0/1163752457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efb02b0 0x56147efcff60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x56147efb9110 tx=0x7fb8c4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1cf79aa917550d server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.141+0000 7fb8b37fe640 1 -- 172.21.15.78:0/1163752457 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4020070 con 0x56147efb02b0 2024-08-18T23:45:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.141+0000 7fb8b37fe640 1 -- 172.21.15.78:0/1163752457 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c400bc40 con 0x56147efb02b0 2024-08-18T23:45:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.141+0000 7fb8b37fe640 1 -- 172.21.15.78:0/1163752457 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40143b0 con 0x56147efb02b0 2024-08-18T23:45:18.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.142+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/1163752457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efb02b0 msgr2=0x56147efcff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:18.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.142+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/1163752457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efb02b0 0x56147efcff60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x56147efb9110 tx=0x7fb8c4008040 comp rx=0 tx=0).stop 2024-08-18T23:45:18.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.142+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/1163752457 shutdown_connections 2024-08-18T23:45:18.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.142+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/1163752457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efbab70 0x56147efd04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.142+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/1163752457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efb02b0 0x56147efcff60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.142+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/1163752457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56147efaf990 0x56147efb23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.142+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/1163752457 >> 172.21.15.78:0/1163752457 conn(0x56147efabf20 msgr2=0x56147efb9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:18.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.142+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/1163752457 shutdown_connections 2024-08-18T23:45:18.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.142+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/1163752457 wait complete. 2024-08-18T23:45:18.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.142+0000 7fb8d1fd5740 1 -- start start 2024-08-18T23:45:18.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.142+0000 7fb8d1fd5740 1 Processor -- start 2024-08-18T23:45:18.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.142+0000 7fb8d1fd5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efaf990 0x56147efd0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:18.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.142+0000 7fb8d1fd5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efb02b0 0x56147efcd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:18.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.142+0000 7fb8d1fd5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56147efbab70 0x56147efcddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:18.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.142+0000 7fb8d1fd5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c81ad150 con 0x56147efaf990 2024-08-18T23:45:18.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.142+0000 7fb8d1fd5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c81ae5c0 con 0x56147efb02b0 2024-08-18T23:45:18.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.142+0000 7fb8d1fd5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c81ad820 con 0x56147efbab70 2024-08-18T23:45:18.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.143+0000 7fb8d1fd5740 10 client.0 _ll_register_callbacks cb 0x7fb8b80051a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:45:18.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.143+0000 7fb8cd172640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efaf990 0x56147efd0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:18.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.143+0000 7fb8cd172640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efaf990 0x56147efd0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42826/0 (socket says 172.21.15.78:42826) 2024-08-18T23:45:18.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.143+0000 7fb8cd172640 1 -- 172.21.15.78:0/1440302963 learned_addr learned my addr 172.21.15.78:0/1440302963 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:18.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.143+0000 7fb8cd973640 1 --2- 172.21.15.78:0/1440302963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efb02b0 0x56147efcd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:18.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.143+0000 7fb8cd172640 1 --2- 172.21.15.78:0/1440302963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56147efbab70 0x56147efcddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:18.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.143+0000 7fb8cd172640 1 -- 172.21.15.78:0/1440302963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56147efbab70 msgr2=0x56147efcddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:18.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.143+0000 7fb8cd172640 1 --2- 172.21.15.78:0/1440302963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56147efbab70 0x56147efcddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.143+0000 7fb8cd172640 1 -- 172.21.15.78:0/1440302963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efb02b0 msgr2=0x56147efcd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:18.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.143+0000 7fb8cd172640 1 --2- 172.21.15.78:0/1440302963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efb02b0 0x56147efcd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.143+0000 7fb8cd172640 1 -- 172.21.15.78:0/1440302963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56147efce420 con 0x56147efaf990 2024-08-18T23:45:18.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.143+0000 7fb8cd973640 1 --2- 172.21.15.78:0/1440302963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efb02b0 0x56147efcd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:45:18.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.143+0000 7fb8cd172640 1 --2- 172.21.15.78:0/1440302963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efaf990 0x56147efd0520 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8031ac0 tx=0x7fb8b8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc47532aec30e057 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:18.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.143+0000 7fb8b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:45:18.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.143+0000 7fb8b17fa640 1 -- 172.21.15.78:0/1440302963 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b803f040 con 0x56147efaf990 2024-08-18T23:45:18.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.143+0000 7fb8b17fa640 1 -- 172.21.15.78:0/1440302963 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fb8b80035f0 con 0x56147efaf990 2024-08-18T23:45:18.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.143+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/1440302963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8c400bc40 con 0x56147efaf990 2024-08-18T23:45:18.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.143+0000 7fb8b17fa640 1 -- 172.21.15.78:0/1440302963 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb8b802aa60 con 0x56147efaf990 2024-08-18T23:45:18.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.144+0000 7fb8d1fd5740 10 client.15615 fetch_fsmap learned FSMap version 25 2024-08-18T23:45:18.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.144+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/1440302963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56147f035340 con 0x56147efaf990 2024-08-18T23:45:18.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.144+0000 7fb8b17fa640 1 -- 172.21.15.78:0/1440302963 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c400bc40 con 0x56147efaf990 2024-08-18T23:45:18.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.144+0000 7fb8b17fa640 1 -- 172.21.15.78:0/1440302963 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb8b8065030 con 0x56147efaf990 2024-08-18T23:45:18.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.144+0000 7fb8d1fd5740 10 client.15615 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:45:18.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.144+0000 7fb8d1fd5740 10 client.15615 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:45:18.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.144+0000 7fb8d1fd5740 20 client.15615 populate_metadata read hostname 'smithi078' 2024-08-18T23:45:18.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.144+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/1440302963 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56147f035870 0x56147f037c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:18.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.144+0000 7fb8d1fd5740 4 client.15615 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f246fd05", "release": true, "await": true}] 2024-08-18T23:45:18.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.144+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/1440302963 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f246fd05", "release": true, "await": true}) -- 0x56147ef1d6e0 con 0x56147f035870 2024-08-18T23:45:18.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.144+0000 7fb8cd973640 1 --2- 172.21.15.78:0/1440302963 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56147f035870 0x56147f037c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:18.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.145+0000 7fb8b17fa640 10 client.15615 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:18.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.145+0000 7fb8cd973640 1 --2- 172.21.15.78:0/1440302963 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56147f035870 0x56147f037c60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dadd8b9ea28029f4 server_cookie=89225521357ed6bf in_seq=0 out_seq=0 2024-08-18T23:45:18.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.148+0000 7fb8b17fa640 1 -- 172.21.15.78:0/1440302963 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56147ef1d6e0 con 0x56147f035870 2024-08-18T23:45:18.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.148+0000 7fb8b17fa640 10 client.15615 handle_command_reply: tid=0 2024-08-18T23:45:18.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.148+0000 7fb8d1fd5740 1 client.15615 shutdown 2024-08-18T23:45:18.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.148+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/1440302963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efaf990 msgr2=0x56147efd0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:18.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.148+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/1440302963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efaf990 0x56147efd0520 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8031ac0 tx=0x7fb8b8005fd0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.148+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/1440302963 shutdown_connections 2024-08-18T23:45:18.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.148+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/1440302963 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56147f035870 0x56147f037c60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.148+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/1440302963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56147efbab70 0x56147efcddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.148+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/1440302963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56147efb02b0 0x56147efcd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.148+0000 7fb8d1fd5740 1 --2- 172.21.15.78:0/1440302963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56147efaf990 0x56147efd0520 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.148+0000 7fb8b17fa640 0 client.15615 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:18.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.148+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/1440302963 >> 172.21.15.78:0/1440302963 conn(0x56147efabf20 msgr2=0x56147efb9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:18.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.148+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/1440302963 shutdown_connections 2024-08-18T23:45:18.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.148+0000 7fb8d1fd5740 1 -- 172.21.15.78:0/1440302963 wait complete. 2024-08-18T23:45:18.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.148+0000 7fb8d1fd5740 20 client.15615 trim_cache size 0 max 16384 2024-08-18T23:45:18.153 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:45:18.153 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 25, 2024-08-18T23:45:18.153 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24452, 2024-08-18T23:45:18.153 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 4, 2024-08-18T23:45:18.153 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-08-18T23:45:18.153 INFO:teuthology.orchestra.run.smithi078.stdout: "f246fd05": { 2024-08-18T23:45:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 4, 2024-08-18T23:45:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-08-18T23:45:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:45:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-08-18T23:45:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:45:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:45:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-08-18T23:45:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 154.7, 2024-08-18T23:45:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-08-18T23:45:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-08-18T23:45:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-08-18T23:45:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:45:18.154 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-08-18T23:45:18.155 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:45:18.155 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:45:18.155 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:45:18.155 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:45:18.155 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:45:18.155 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:45:18.155 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:45:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.149+0000 7fb8cfbd7640 1 -- 172.21.15.78:0/1916874548 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb8b40a2c10 msgr2=0x7fb8b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.149+0000 7fb8cfbd7640 1 --2- 172.21.15.78:0/1916874548 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb8b40a2c10 0x7fb8b40a50d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb8c00986e0 tx=0x7fb8c0005960 comp rx=0 tx=0).stop 2024-08-18T23:45:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.149+0000 7fb8cfbd7640 1 -- 172.21.15.78:0/1916874548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c81067b0 msgr2=0x7fb8c819cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.149+0000 7fb8cfbd7640 1 --2- 172.21.15.78:0/1916874548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c81067b0 0x7fb8c819cb30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000cad0 tx=0x7fb8c000cf90 comp rx=0 tx=0).stop 2024-08-18T23:45:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.149+0000 7fb8cfbd7640 1 -- 172.21.15.78:0/1916874548 shutdown_connections 2024-08-18T23:45:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.149+0000 7fb8cfbd7640 1 --2- 172.21.15.78:0/1916874548 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb8b40a2c10 0x7fb8b40a50d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.149+0000 7fb8cfbd7640 1 --2- 172.21.15.78:0/1916874548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c81067b0 0x7fb8c819cb30 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.149+0000 7fb8cfbd7640 1 --2- 172.21.15.78:0/1916874548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8c8101150 0x7fb8c81987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.149+0000 7fb8cfbd7640 1 --2- 172.21.15.78:0/1916874548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8c81007b0 0x7fb8c8198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.149+0000 7fb8cfbd7640 1 -- 172.21.15.78:0/1916874548 >> 172.21.15.78:0/1916874548 conn(0x7fb8c80fc4d0 msgr2=0x7fb8c8104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.149+0000 7fb8cfbd7640 1 -- 172.21.15.78:0/1916874548 shutdown_connections 2024-08-18T23:45:18.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:18.149+0000 7fb8cfbd7640 1 -- 172.21.15.78:0/1916874548 wait complete. 2024-08-18T23:45:18.157 INFO:tasks.workunit.client.0.smithi078.stdout:getting blogbench 2024-08-18T23:45:18.157 INFO:tasks.workunit.client.0.smithi078.stderr:+ echo 'getting blogbench' 2024-08-18T23:45:18.157 INFO:tasks.workunit.client.0.smithi078.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-08-18T23:45:18.213 INFO:tasks.workunit.client.0.smithi078.stderr:--2024-08-18 23:45:18-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-08-18T23:45:18.215 INFO:tasks.workunit.client.0.smithi078.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-08-18T23:45:18.236 INFO:tasks.workunit.client.0.smithi078.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-08-18T23:45:18.257 INFO:tasks.workunit.client.0.smithi078.stderr:HTTP request sent, awaiting response... 200 OK 2024-08-18T23:45:18.257 INFO:tasks.workunit.client.0.smithi078.stderr:Length: 103355 (101K) [application/octet-stream] 2024-08-18T23:45:18.258 INFO:tasks.workunit.client.0.smithi078.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-08-18T23:45:18.261 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-08-18T23:45:18.299 INFO:tasks.workunit.client.0.smithi078.stderr: 0K .......... .......... .......... .......... .......... 49% 1.23M 0s 2024-08-18T23:45:18.319 INFO:tasks.workunit.client.0.smithi078.stderr: 50K .......... .......... .......... .......... .......... 99% 2.34M 0s 2024-08-18T23:45:18.319 INFO:tasks.workunit.client.0.smithi078.stderr: 100K 100% 1.74T=0.06s 2024-08-18T23:45:18.319 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-08-18T23:45:18.319 INFO:tasks.workunit.client.0.smithi078.stderr:2024-08-18 23:45:18 (1.63 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-08-18T23:45:18.320 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-08-18T23:45:18.320 INFO:tasks.workunit.client.0.smithi078.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-08-18T23:45:18.407 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/ 2024-08-18T23:45:18.408 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/m4/ 2024-08-18T23:45:18.409 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-08-18T23:45:18.410 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/man/ 2024-08-18T23:45:18.411 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/man/blogbench.8 2024-08-18T23:45:18.412 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/man/Makefile.am 2024-08-18T23:45:18.413 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/man/Makefile.in 2024-08-18T23:45:18.413 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/src/ 2024-08-18T23:45:18.414 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/src/mysnprintf.c 2024-08-18T23:45:18.417 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/src/mysnprintf.h 2024-08-18T23:45:18.418 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/src/blogbench.c 2024-08-18T23:45:18.419 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/src/blogbench.h 2024-08-18T23:45:18.420 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/src/fakesnprintf.c 2024-08-18T23:45:18.421 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/src/fakesnprintf.h 2024-08-18T23:45:18.422 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/src/blogbench_p.h 2024-08-18T23:45:18.423 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-08-18T23:45:18.423 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-08-18T23:45:18.424 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/src/Makefile.am 2024-08-18T23:45:18.425 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/src/Makefile.in 2024-08-18T23:45:18.426 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/src/process.c 2024-08-18T23:45:18.427 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/src/globals.h 2024-08-18T23:45:18.427 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/src/commenter.c 2024-08-18T23:45:18.428 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/src/rewriter.c 2024-08-18T23:45:18.429 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/src/reader.c 2024-08-18T23:45:18.430 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/src/writer.c 2024-08-18T23:45:18.431 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/src/helpers.c 2024-08-18T23:45:18.432 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/NEWS 2024-08-18T23:45:18.433 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/compile 2024-08-18T23:45:18.434 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/depcomp 2024-08-18T23:45:18.435 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/aclocal.m4 2024-08-18T23:45:18.436 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/README 2024-08-18T23:45:18.437 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/configure 2024-08-18T23:45:18.441 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/configure.ac 2024-08-18T23:45:18.442 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/config.guess 2024-08-18T23:45:18.443 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/install-sh 2024-08-18T23:45:18.443 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/config.sub 2024-08-18T23:45:18.444 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/missing 2024-08-18T23:45:18.445 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/mkinstalldirs 2024-08-18T23:45:18.446 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/Makefile.am 2024-08-18T23:45:18.447 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/Makefile.in 2024-08-18T23:45:18.447 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/config.h.in 2024-08-18T23:45:18.448 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/AUTHORS 2024-08-18T23:45:18.449 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/INSTALL 2024-08-18T23:45:18.450 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/ChangeLog 2024-08-18T23:45:18.451 INFO:tasks.workunit.client.0.smithi078.stdout:blogbench-1.0/COPYING 2024-08-18T23:45:18.452 INFO:tasks.workunit.client.0.smithi078.stderr:+ cd blogbench-1.0/ 2024-08-18T23:45:18.452 INFO:tasks.workunit.client.0.smithi078.stdout:making blogbench 2024-08-18T23:45:18.453 INFO:tasks.workunit.client.0.smithi078.stderr:+ echo 'making blogbench' 2024-08-18T23:45:18.453 INFO:tasks.workunit.client.0.smithi078.stderr:+ ./configure 2024-08-18T23:45:18.599 INFO:tasks.workunit.client.0.smithi078.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-08-18T23:45:18.964 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f246fd05', total seconds elapsed: 38.2 2024-08-18T23:45:19.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:19 smithi123 ceph-mon[21808]: pgmap v380: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:45:19.607 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether build environment is sane... yes 2024-08-18T23:45:19.615 INFO:tasks.workunit.client.0.smithi078.stdout:checking for gawk... gawk 2024-08-18T23:45:19.629 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether make sets $(MAKE)... yes 2024-08-18T23:45:19.637 INFO:tasks.workunit.client.0.smithi078.stdout:checking for gcc... gcc 2024-08-18T23:45:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:19 smithi161 ceph-mon[22027]: pgmap v380: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:45:19.683 INFO:tasks.workunit.client.0.smithi078.stdout:checking for C compiler default output file name... a.out 2024-08-18T23:45:19.684 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether the C compiler works... yes 2024-08-18T23:45:19.687 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether we are cross compiling... no 2024-08-18T23:45:19.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:19 smithi078 ceph-mon[18440]: pgmap v380: 129 pgs: 129 active+clean; 663 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:45:19.722 INFO:tasks.workunit.client.0.smithi078.stdout:checking for suffix of executables... 2024-08-18T23:45:19.754 INFO:tasks.workunit.client.0.smithi078.stdout:checking for suffix of object files... o 2024-08-18T23:45:19.785 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether we are using the GNU C compiler... yes 2024-08-18T23:45:19.816 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether gcc accepts -g... yes 2024-08-18T23:45:19.885 INFO:tasks.workunit.client.0.smithi078.stdout:checking for gcc option to accept ANSI C... none needed 2024-08-18T23:45:19.927 INFO:tasks.workunit.client.0.smithi078.stdout:checking for style of include used by make... GNU 2024-08-18T23:45:20.017 INFO:tasks.workunit.client.0.smithi078.stdout:checking dependency style of gcc... gcc3 2024-08-18T23:45:20.017 INFO:tasks.workunit.client.0.smithi078.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-08-18T23:45:20.059 INFO:tasks.workunit.client.0.smithi078.stdout:checking how to run the C preprocessor... gcc -E 2024-08-18T23:45:20.098 INFO:tasks.workunit.client.0.smithi078.stdout:checking for egrep... grep -E 2024-08-18T23:45:20.107 INFO:tasks.workunit.client.0.smithi078.stdout:checking for AIX... no 2024-08-18T23:45:20.167 INFO:tasks.workunit.client.0.smithi078.stdout:checking for library containing strerror... none required 2024-08-18T23:45:20.296 INFO:tasks.workunit.client.0.smithi078.stdout:checking for ANSI C header files... yes 2024-08-18T23:45:20.333 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-08-18T23:45:20.377 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sys/types.h... yes 2024-08-18T23:45:20.422 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sys/stat.h... yes 2024-08-18T23:45:20.467 INFO:tasks.workunit.client.0.smithi078.stdout:checking for stdlib.h... yes 2024-08-18T23:45:20.514 INFO:tasks.workunit.client.0.smithi078.stdout:checking for string.h... yes 2024-08-18T23:45:20.560 INFO:tasks.workunit.client.0.smithi078.stdout:checking for memory.h... yes 2024-08-18T23:45:20.606 INFO:tasks.workunit.client.0.smithi078.stdout:checking for strings.h... yes 2024-08-18T23:45:20.653 INFO:tasks.workunit.client.0.smithi078.stdout:checking for inttypes.h... yes 2024-08-18T23:45:20.699 INFO:tasks.workunit.client.0.smithi078.stdout:checking for stdint.h... yes 2024-08-18T23:45:20.749 INFO:tasks.workunit.client.0.smithi078.stdout:checking for unistd.h... yes 2024-08-18T23:45:20.760 INFO:tasks.workunit.client.0.smithi078.stdout:checking for memory.h... (cached) yes 2024-08-18T23:45:20.765 INFO:tasks.workunit.client.0.smithi078.stdout:checking for strings.h... (cached) yes 2024-08-18T23:45:20.770 INFO:tasks.workunit.client.0.smithi078.stdout:checking for unistd.h... (cached) yes 2024-08-18T23:45:20.823 INFO:tasks.workunit.client.0.smithi078.stdout:checking locale.h usability... yes 2024-08-18T23:45:20.843 INFO:tasks.workunit.client.0.smithi078.stdout:checking locale.h presence... yes 2024-08-18T23:45:20.843 INFO:tasks.workunit.client.0.smithi078.stdout:checking for locale.h... yes 2024-08-18T23:45:20.856 INFO:tasks.workunit.client.0.smithi078.stdout:checking for stdlib.h... (cached) yes 2024-08-18T23:45:20.861 INFO:tasks.workunit.client.0.smithi078.stdout:checking for string.h... (cached) yes 2024-08-18T23:45:20.907 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/socket.h usability... yes 2024-08-18T23:45:20.929 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/socket.h presence... yes 2024-08-18T23:45:20.929 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sys/socket.h... yes 2024-08-18T23:45:20.980 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/time.h usability... yes 2024-08-18T23:45:20.999 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/time.h presence... yes 2024-08-18T23:45:20.999 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sys/time.h... yes 2024-08-18T23:45:21.053 INFO:tasks.workunit.client.0.smithi078.stdout:checking fcntl.h usability... yes 2024-08-18T23:45:21.070 INFO:tasks.workunit.client.0.smithi078.stdout:checking fcntl.h presence... yes 2024-08-18T23:45:21.071 INFO:tasks.workunit.client.0.smithi078.stdout:checking for fcntl.h... yes 2024-08-18T23:45:21.131 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/fcntl.h usability... yes 2024-08-18T23:45:21.150 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/fcntl.h presence... yes 2024-08-18T23:45:21.151 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sys/fcntl.h... yes 2024-08-18T23:45:21.217 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/param.h usability... yes 2024-08-18T23:45:21.239 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/param.h presence... yes 2024-08-18T23:45:21.240 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sys/param.h... yes 2024-08-18T23:45:21.289 INFO:tasks.workunit.client.0.smithi078.stdout:checking ioctl.h usability... no 2024-08-18T23:45:21.308 INFO:tasks.workunit.client.0.smithi078.stdout:checking ioctl.h presence... no 2024-08-18T23:45:21.308 INFO:tasks.workunit.client.0.smithi078.stdout:checking for ioctl.h... no 2024-08-18T23:45:21.357 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/ioctl.h usability... yes 2024-08-18T23:45:21.376 INFO:tasks.workunit.client.0.smithi078.stdout:checking sys/ioctl.h presence... yes 2024-08-18T23:45:21.376 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sys/ioctl.h... yes 2024-08-18T23:45:21.428 INFO:tasks.workunit.client.0.smithi078.stdout:checking return type of signal handlers... void 2024-08-18T23:45:21.472 INFO:tasks.workunit.client.0.smithi078.stdout:checking for size_t... yes 2024-08-18T23:45:21.509 INFO:tasks.workunit.client.0.smithi078.stdout:checking for dev_t... yes 2024-08-18T23:45:21.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:21 smithi123 ceph-mon[21808]: pgmap v381: 129 pgs: 129 active+clean; 789 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-08-18T23:45:21.551 INFO:tasks.workunit.client.0.smithi078.stdout:checking for short... yes 2024-08-18T23:45:21.637 INFO:tasks.workunit.client.0.smithi078.stdout:checking size of short... 2 2024-08-18T23:45:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:21 smithi161 ceph-mon[22027]: pgmap v381: 129 pgs: 129 active+clean; 789 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-08-18T23:45:21.681 INFO:tasks.workunit.client.0.smithi078.stdout:checking for int... yes 2024-08-18T23:45:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:21 smithi078 ceph-mon[18440]: pgmap v381: 129 pgs: 129 active+clean; 789 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-08-18T23:45:21.747 INFO:tasks.workunit.client.0.smithi078.stdout:checking size of int... 4 2024-08-18T23:45:21.788 INFO:tasks.workunit.client.0.smithi078.stdout:checking for long... yes 2024-08-18T23:45:21.853 INFO:tasks.workunit.client.0.smithi078.stdout:checking size of long... 8 2024-08-18T23:45:21.897 INFO:tasks.workunit.client.0.smithi078.stdout:checking for long long... yes 2024-08-18T23:45:21.962 INFO:tasks.workunit.client.0.smithi078.stdout:checking size of long long... 8 2024-08-18T23:45:22.001 INFO:tasks.workunit.client.0.smithi078.stdout:checking for ssize_t... yes 2024-08-18T23:45:22.039 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sig_atomic_t... yes 2024-08-18T23:45:22.089 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether gcc needs -traditional... no 2024-08-18T23:45:22.125 INFO:tasks.workunit.client.0.smithi078.stdout:checking for an ANSI C-conforming const... yes 2024-08-18T23:45:22.147 INFO:tasks.workunit.client.0.smithi078.stdout:checking for inline... inline 2024-08-18T23:45:22.201 INFO:tasks.workunit.client.0.smithi078.stdout:checking for working alloca.h... yes 2024-08-18T23:45:22.247 INFO:tasks.workunit.client.0.smithi078.stdout:checking for alloca... yes 2024-08-18T23:45:22.316 INFO:tasks.workunit.client.0.smithi078.stdout:checking for working memcmp... yes 2024-08-18T23:45:22.368 INFO:tasks.workunit.client.0.smithi078.stdout:checking for strftime... yes 2024-08-18T23:45:22.436 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-08-18T23:45:22.493 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether stat accepts an empty string... no 2024-08-18T23:45:22.547 INFO:tasks.workunit.client.0.smithi078.stdout:checking for vprintf... yes 2024-08-18T23:45:22.604 INFO:tasks.workunit.client.0.smithi078.stdout:checking for _doprnt... no 2024-08-18T23:45:22.626 INFO:tasks.workunit.client.0.smithi078.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-08-18T23:45:22.637 INFO:tasks.workunit.client.0.smithi078.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-08-18T23:45:22.687 INFO:tasks.workunit.client.0.smithi078.stdout:checking for the pthreads library -lpthreads... no 2024-08-18T23:45:22.740 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether pthreads work without any flags... yes 2024-08-18T23:45:22.785 INFO:tasks.workunit.client.0.smithi078.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-08-18T23:45:22.786 INFO:tasks.workunit.client.0.smithi078.stdout:checking if more special flags are required for pthreads... no 2024-08-18T23:45:22.786 INFO:tasks.workunit.client.0.smithi078.stdout:checking for cc_r... gcc 2024-08-18T23:45:22.866 INFO:tasks.workunit.client.0.smithi078.stdout:checking for setrlimit... yes 2024-08-18T23:45:22.925 INFO:tasks.workunit.client.0.smithi078.stdout:checking for setproctitle... no 2024-08-18T23:45:22.981 INFO:tasks.workunit.client.0.smithi078.stdout:checking for getopt_long... yes 2024-08-18T23:45:23.040 INFO:tasks.workunit.client.0.smithi078.stdout:checking for strtoull... yes 2024-08-18T23:45:23.099 INFO:tasks.workunit.client.0.smithi078.stdout:checking for strtoq... yes 2024-08-18T23:45:23.158 INFO:tasks.workunit.client.0.smithi078.stdout:checking for snprintf... yes 2024-08-18T23:45:23.220 INFO:tasks.workunit.client.0.smithi078.stdout:checking for vsnprintf... yes 2024-08-18T23:45:23.279 INFO:tasks.workunit.client.0.smithi078.stdout:checking for vfprintf... yes 2024-08-18T23:45:23.338 INFO:tasks.workunit.client.0.smithi078.stdout:checking for gethostname... yes 2024-08-18T23:45:23.397 INFO:tasks.workunit.client.0.smithi078.stdout:checking for setlocale... yes 2024-08-18T23:45:23.522 INFO:tasks.workunit.client.0.smithi078.stdout:checking for sync... yes 2024-08-18T23:45:23.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:23 smithi123 ceph-mon[21808]: pgmap v382: 129 pgs: 129 active+clean; 789 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-08-18T23:45:23.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:23 smithi123 ceph-mon[21808]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-08-18T23:45:23.592 INFO:tasks.workunit.client.0.smithi078.stdout:checking whether snprintf is C99 conformant... done 2024-08-18T23:45:23.604 INFO:tasks.workunit.client.0.smithi078.stdout:configure: creating ./config.status 2024-08-18T23:45:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:23 smithi161 ceph-mon[22027]: pgmap v382: 129 pgs: 129 active+clean; 789 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-08-18T23:45:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:23 smithi161 ceph-mon[22027]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-08-18T23:45:23.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:23 smithi078 ceph-mon[18440]: pgmap v382: 129 pgs: 129 active+clean; 789 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-08-18T23:45:23.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:23 smithi078 ceph-mon[18440]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-08-18T23:45:23.764 INFO:tasks.workunit.client.0.smithi078.stdout:config.status: creating Makefile 2024-08-18T23:45:23.781 INFO:tasks.workunit.client.0.smithi078.stdout:config.status: creating src/Makefile 2024-08-18T23:45:23.799 INFO:tasks.workunit.client.0.smithi078.stdout:config.status: creating man/Makefile 2024-08-18T23:45:23.810 INFO:tasks.workunit.client.0.smithi078.stdout:config.status: creating config.h 2024-08-18T23:45:23.859 INFO:tasks.workunit.client.0.smithi078.stdout:config.status: executing depfiles commands 2024-08-18T23:45:23.978 INFO:tasks.workunit.client.0.smithi078.stderr:+ make 2024-08-18T23:45:23.987 INFO:tasks.workunit.client.0.smithi078.stdout:make all-recursive 2024-08-18T23:45:23.989 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-08-18T23:45:23.991 INFO:tasks.workunit.client.0.smithi078.stdout:Making all in src 2024-08-18T23:45:23.997 INFO:tasks.workunit.client.0.smithi078.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-08-18T23:45:23.997 INFO:tasks.workunit.client.0.smithi078.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-08-18T23:45:23.997 INFO:tasks.workunit.client.0.smithi078.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-08-18T23:45:24.098 INFO:tasks.workunit.client.0.smithi078.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-08-18T23:45:24.099 INFO:tasks.workunit.client.0.smithi078.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-08-18T23:45:24.166 INFO:tasks.workunit.client.0.smithi078.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-08-18T23:45:24.166 INFO:tasks.workunit.client.0.smithi078.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-08-18T23:45:24.218 INFO:tasks.workunit.client.0.smithi078.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-08-18T23:45:24.218 INFO:tasks.workunit.client.0.smithi078.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-08-18T23:45:24.271 INFO:tasks.workunit.client.0.smithi078.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-08-18T23:45:24.271 INFO:tasks.workunit.client.0.smithi078.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-08-18T23:45:24.346 INFO:tasks.workunit.client.0.smithi078.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-08-18T23:45:24.346 INFO:tasks.workunit.client.0.smithi078.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-08-18T23:45:24.416 INFO:tasks.workunit.client.0.smithi078.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-08-18T23:45:24.416 INFO:tasks.workunit.client.0.smithi078.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-08-18T23:45:24.490 INFO:tasks.workunit.client.0.smithi078.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-08-18T23:45:24.490 INFO:tasks.workunit.client.0.smithi078.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-08-18T23:45:24.526 INFO:tasks.workunit.client.0.smithi078.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-08-18T23:45:24.526 INFO:tasks.workunit.client.0.smithi078.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-08-18T23:45:24.560 INFO:tasks.workunit.client.0.smithi078.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-08-18T23:45:24.560 INFO:tasks.workunit.client.0.smithi078.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-08-18T23:45:24.598 INFO:tasks.workunit.client.0.smithi078.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-08-18T23:45:24.617 INFO:tasks.workunit.client.0.smithi078.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-08-18T23:45:24.617 INFO:tasks.workunit.client.0.smithi078.stdout:Making all in man 2024-08-18T23:45:24.656 INFO:tasks.workunit.client.0.smithi078.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-08-18T23:45:24.656 INFO:tasks.workunit.client.0.smithi078.stdout:make[2]: Nothing to be done for 'all'. 2024-08-18T23:45:24.656 INFO:tasks.workunit.client.0.smithi078.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-08-18T23:45:24.657 INFO:tasks.workunit.client.0.smithi078.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-08-18T23:45:24.658 INFO:tasks.workunit.client.0.smithi078.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-08-18T23:45:24.658 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-08-18T23:45:24.659 INFO:tasks.workunit.client.0.smithi078.stderr:+ cd src 2024-08-18T23:45:24.659 INFO:tasks.workunit.client.0.smithi078.stderr:+ mkdir blogtest_in 2024-08-18T23:45:24.662 INFO:tasks.workunit.client.0.smithi078.stdout:running blogbench 2024-08-18T23:45:24.662 INFO:tasks.workunit.client.0.smithi078.stderr:+ echo 'running blogbench' 2024-08-18T23:45:24.662 INFO:tasks.workunit.client.0.smithi078.stderr:+ ./blogbench -d blogtest_in 2024-08-18T23:45:24.662 INFO:tasks.workunit.client.0.smithi078.stdout: 2024-08-18T23:45:24.662 INFO:tasks.workunit.client.0.smithi078.stdout:Frequency = 10 secs 2024-08-18T23:45:24.663 INFO:tasks.workunit.client.0.smithi078.stdout:Scratch dir = [blogtest_in] 2024-08-18T23:45:25.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:25 smithi123 ceph-mon[21808]: pgmap v383: 129 pgs: 129 active+clean; 1.2 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 48 KiB/s wr, 8 op/s 2024-08-18T23:45:25.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:25 smithi161 ceph-mon[22027]: pgmap v383: 129 pgs: 129 active+clean; 1.2 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 48 KiB/s wr, 8 op/s 2024-08-18T23:45:25.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:25 smithi078 ceph-mon[18440]: pgmap v383: 129 pgs: 129 active+clean; 1.2 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 48 KiB/s wr, 8 op/s 2024-08-18T23:45:26.979 INFO:tasks.workunit.client.0.smithi078.stdout:Spawning 3 writers... 2024-08-18T23:45:26.979 INFO:tasks.workunit.client.0.smithi078.stdout:Spawning 1 rewriters... 2024-08-18T23:45:26.979 INFO:tasks.workunit.client.0.smithi078.stdout:Spawning 5 commenters... 2024-08-18T23:45:26.979 INFO:tasks.workunit.client.0.smithi078.stdout:Spawning 100 readers... 2024-08-18T23:45:26.981 INFO:tasks.workunit.client.0.smithi078.stdout:Benchmarking for 30 iterations. 2024-08-18T23:45:26.981 INFO:tasks.workunit.client.0.smithi078.stdout:The test will run during 5 minutes. 2024-08-18T23:45:26.981 INFO:tasks.workunit.client.0.smithi078.stdout: 2024-08-18T23:45:26.982 INFO:tasks.workunit.client.0.smithi078.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-08-18T23:45:27.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:27 smithi161 ceph-mon[22027]: pgmap v384: 129 pgs: 129 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 288 KiB/s wr, 29 op/s 2024-08-18T23:45:27.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:27 smithi078 ceph-mon[18440]: pgmap v384: 129 pgs: 129 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 288 KiB/s wr, 29 op/s 2024-08-18T23:45:27.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:27 smithi123 ceph-mon[21808]: pgmap v384: 129 pgs: 129 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 288 KiB/s wr, 29 op/s 2024-08-18T23:45:29.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:29 smithi161 ceph-mon[22027]: pgmap v385: 129 pgs: 129 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 378 KiB/s wr, 51 op/s 2024-08-18T23:45:29.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:29 smithi078 ceph-mon[18440]: pgmap v385: 129 pgs: 129 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 378 KiB/s wr, 51 op/s 2024-08-18T23:45:29.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:29 smithi123 ceph-mon[21808]: pgmap v385: 129 pgs: 129 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 378 KiB/s wr, 51 op/s 2024-08-18T23:45:31.109 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:45:31.382 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:45:31.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:31 smithi078 ceph-mon[18440]: pgmap v386: 129 pgs: 129 active+clean; 5.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 525 KiB/s wr, 61 op/s 2024-08-18T23:45:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:31 smithi161 ceph-mon[22027]: pgmap v386: 129 pgs: 129 active+clean; 5.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 525 KiB/s wr, 61 op/s 2024-08-18T23:45:31.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:31 smithi123 ceph-mon[21808]: pgmap v386: 129 pgs: 129 active+clean; 5.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 525 KiB/s wr, 61 op/s 2024-08-18T23:45:31.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.987+0000 7f244d152640 1 -- 172.21.15.78:0/3805756911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2448108f20 msgr2=0x7f2448109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:31.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.987+0000 7f244d152640 1 --2- 172.21.15.78:0/3805756911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2448108f20 0x7f2448109300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2438009860 tx=0x7f243802a3d0 comp rx=0 tx=0).stop 2024-08-18T23:45:31.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.987+0000 7f244d152640 1 -- 172.21.15.78:0/3805756911 shutdown_connections 2024-08-18T23:45:31.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.987+0000 7f244d152640 1 --2- 172.21.15.78:0/3805756911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24481147e0 0x7f2448116bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:31.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.987+0000 7f244d152640 1 --2- 172.21.15.78:0/3805756911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2448109840 0x7f24481142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:31.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.987+0000 7f244d152640 1 --2- 172.21.15.78:0/3805756911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2448108f20 0x7f2448109300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:31.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.987+0000 7f244d152640 1 -- 172.21.15.78:0/3805756911 >> 172.21.15.78:0/3805756911 conn(0x7f244806de70 msgr2=0x7f244806e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:31.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.987+0000 7f244d152640 1 -- 172.21.15.78:0/3805756911 shutdown_connections 2024-08-18T23:45:31.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.987+0000 7f244d152640 1 -- 172.21.15.78:0/3805756911 wait complete. 2024-08-18T23:45:31.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.988+0000 7f244d152640 1 Processor -- start 2024-08-18T23:45:31.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.988+0000 7f244d152640 1 -- start start 2024-08-18T23:45:31.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.988+0000 7f244d152640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2448108f20 0x7f24481a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:31.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.988+0000 7f244d152640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2448109840 0x7f24481a6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:31.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.988+0000 7f244d152640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24481147e0 0x7f24481a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:31.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.988+0000 7f244d152640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f244811a520 con 0x7f2448109840 2024-08-18T23:45:31.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.988+0000 7f244d152640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f244811a3a0 con 0x7f24481147e0 2024-08-18T23:45:31.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.988+0000 7f244d152640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f244811a6a0 con 0x7f2448108f20 2024-08-18T23:45:31.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.989+0000 7f244759e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24481147e0 0x7f24481a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:31.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.989+0000 7f244759e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24481147e0 0x7f24481a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:57496/0 (socket says 172.21.15.78:57496) 2024-08-18T23:45:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.989+0000 7f244759e640 1 -- 172.21.15.78:0/3479934658 learned_addr learned my addr 172.21.15.78:0/3479934658 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.989+0000 7f244659c640 1 --2- 172.21.15.78:0/3479934658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2448109840 0x7f24481a6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.989+0000 7f2446d9d640 1 --2- 172.21.15.78:0/3479934658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2448108f20 0x7f24481a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.989+0000 7f244659c640 1 -- 172.21.15.78:0/3479934658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2448108f20 msgr2=0x7f24481a66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.989+0000 7f244659c640 1 --2- 172.21.15.78:0/3479934658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2448108f20 0x7f24481a66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.989+0000 7f244659c640 1 -- 172.21.15.78:0/3479934658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24481147e0 msgr2=0x7f24481a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.989+0000 7f244659c640 1 --2- 172.21.15.78:0/3479934658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24481147e0 0x7f24481a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.989+0000 7f244659c640 1 -- 172.21.15.78:0/3479934658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24481a0fd0 con 0x7f2448109840 2024-08-18T23:45:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.989+0000 7f2446d9d640 1 --2- 172.21.15.78:0/3479934658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2448108f20 0x7f24481a66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:45:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.990+0000 7f244659c640 1 --2- 172.21.15.78:0/3479934658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2448109840 0x7f24481a6bf0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f243c00d6d0 tx=0x7f243c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.990+0000 7f242ffff640 1 -- 172.21.15.78:0/3479934658 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c017070 con 0x7f2448109840 2024-08-18T23:45:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.990+0000 7f242ffff640 1 -- 172.21.15.78:0/3479934658 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f243c009820 con 0x7f2448109840 2024-08-18T23:45:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.990+0000 7f242ffff640 1 -- 172.21.15.78:0/3479934658 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c004ce0 con 0x7f2448109840 2024-08-18T23:45:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.990+0000 7f244d152640 1 -- 172.21.15.78:0/3479934658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24481a12c0 con 0x7f2448109840 2024-08-18T23:45:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.991+0000 7f244d152640 1 -- 172.21.15.78:0/3479934658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f244810a030 con 0x7f2448109840 2024-08-18T23:45:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.994+0000 7f242ffff640 1 -- 172.21.15.78:0/3479934658 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f243c010670 con 0x7f2448109840 2024-08-18T23:45:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.995+0000 7f242ffff640 1 --2- 172.21.15.78:0/3479934658 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f24300a2c00 0x7f24300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.995+0000 7f244f550740 1 Processor -- start 2024-08-18T23:45:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.996+0000 7f244f550740 1 -- start start 2024-08-18T23:45:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.996+0000 7f242ffff640 1 -- 172.21.15.78:0/3479934658 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f243c0ceb20 con 0x7f2448109840 2024-08-18T23:45:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.996+0000 7f2446d9d640 1 --2- 172.21.15.78:0/3479934658 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f24300a2c00 0x7f24300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.996+0000 7f244f550740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55916858e130 0x55916858e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.996+0000 7f244f550740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858ea50 0x5591685a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.996+0000 7f244f550740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591685a1c00 0x5591685a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.996+0000 7f244f550740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5591684f3980 con 0x55916858ea50 2024-08-18T23:45:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.996+0000 7f244f550740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55916848ba70 con 0x55916858e130 2024-08-18T23:45:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.996+0000 7f244f550740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5591684ef940 con 0x5591685a1c00 2024-08-18T23:45:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.996+0000 7f244659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858ea50 0x5591685a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.996+0000 7f244659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858ea50 0x5591685a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37774/0 (socket says 172.21.15.78:37774) 2024-08-18T23:45:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.996+0000 7f244659c640 1 -- 172.21.15.78:0/1684854650 learned_addr learned my addr 172.21.15.78:0/1684854650 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.996+0000 7f244759e640 1 --2- 172.21.15.78:0/1684854650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591685a1c00 0x5591685a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.997+0000 7f2446d9d640 1 --2- 172.21.15.78:0/3479934658 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f24300a2c00 0x7f24300a50c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f24380095f0 tx=0x7f2438005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.997+0000 7f244659c640 1 -- 172.21.15.78:0/1684854650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591685a1c00 msgr2=0x5591685a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.997+0000 7f244659c640 1 --2- 172.21.15.78:0/1684854650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591685a1c00 0x5591685a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.997+0000 7f2446d9d640 1 --2- 172.21.15.78:0/1684854650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55916858e130 0x55916858e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.997+0000 7f244659c640 1 -- 172.21.15.78:0/1684854650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55916858e130 msgr2=0x55916858e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.997+0000 7f244659c640 1 --2- 172.21.15.78:0/1684854650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55916858e130 0x55916858e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.997+0000 7f244659c640 1 -- 172.21.15.78:0/1684854650 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591684ee6e0 con 0x55916858ea50 2024-08-18T23:45:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.997+0000 7f244759e640 1 --2- 172.21.15.78:0/1684854650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591685a1c00 0x5591685a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:45:31.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.997+0000 7f244659c640 1 --2- 172.21.15.78:0/1684854650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858ea50 0x5591685a16c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f243c09b030 tx=0x7f243c09f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf1f9ece6d9f79b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:32.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.997+0000 7f242d7fa640 1 -- 172.21.15.78:0/1684854650 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c020040 con 0x55916858ea50 2024-08-18T23:45:32.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.997+0000 7f242d7fa640 1 -- 172.21.15.78:0/1684854650 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f243c099140 con 0x55916858ea50 2024-08-18T23:45:32.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.998+0000 7f244f550740 1 -- 172.21.15.78:0/1684854650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858ea50 msgr2=0x5591685a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:32.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.998+0000 7f244f550740 1 --2- 172.21.15.78:0/1684854650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858ea50 0x5591685a16c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f243c09b030 tx=0x7f243c09f3a0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.998+0000 7f242d7fa640 1 -- 172.21.15.78:0/1684854650 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c09a4f0 con 0x55916858ea50 2024-08-18T23:45:32.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.999+0000 7f244f550740 1 -- 172.21.15.78:0/1684854650 shutdown_connections 2024-08-18T23:45:32.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.999+0000 7f244f550740 1 --2- 172.21.15.78:0/1684854650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5591685a1c00 0x5591685a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.999+0000 7f244f550740 1 --2- 172.21.15.78:0/1684854650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858ea50 0x5591685a16c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.999+0000 7f244f550740 1 --2- 172.21.15.78:0/1684854650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55916858e130 0x55916858e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.999+0000 7f244f550740 1 -- 172.21.15.78:0/1684854650 >> 172.21.15.78:0/1684854650 conn(0x55916858a7a0 msgr2=0x55916858ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:32.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.999+0000 7f244f550740 1 -- 172.21.15.78:0/1684854650 shutdown_connections 2024-08-18T23:45:32.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.999+0000 7f244f550740 1 -- 172.21.15.78:0/1684854650 wait complete. 2024-08-18T23:45:32.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.999+0000 7f244f550740 1 -- start start 2024-08-18T23:45:32.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:31.999+0000 7f244f550740 1 Processor -- start 2024-08-18T23:45:32.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.000+0000 7f244f550740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55916858e130 0x55916859eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:32.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.000+0000 7f244f550740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858ea50 0x55916859f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:32.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.000+0000 7f244f550740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591685a1c00 0x55916859f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:32.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.000+0000 7f244f550740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5591685a47b0 con 0x55916858ea50 2024-08-18T23:45:32.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.000+0000 7f244f550740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5591684aeaa0 con 0x5591685a1c00 2024-08-18T23:45:32.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.000+0000 7f244f550740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5591685a4930 con 0x55916858e130 2024-08-18T23:45:32.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.000+0000 7f244f550740 10 client.0 _ll_register_callbacks cb 0x55916849de60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:45:32.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.000+0000 7f244659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858ea50 0x55916859f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:32.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.000+0000 7f244659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858ea50 0x55916859f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37788/0 (socket says 172.21.15.78:37788) 2024-08-18T23:45:32.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.000+0000 7f244659c640 1 -- 172.21.15.78:0/268314887 learned_addr learned my addr 172.21.15.78:0/268314887 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:32.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.000+0000 7f2446d9d640 1 --2- 172.21.15.78:0/268314887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55916858e130 0x55916859eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:32.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.000+0000 7f244759e640 1 --2- 172.21.15.78:0/268314887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591685a1c00 0x55916859f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:32.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.001+0000 7f244659c640 1 -- 172.21.15.78:0/268314887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55916858e130 msgr2=0x55916859eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:32.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.001+0000 7f244659c640 1 --2- 172.21.15.78:0/268314887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55916858e130 0x55916859eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.001+0000 7f244659c640 1 -- 172.21.15.78:0/268314887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591685a1c00 msgr2=0x55916859f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:32.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.001+0000 7f244659c640 1 --2- 172.21.15.78:0/268314887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591685a1c00 0x55916859f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.001+0000 7f244659c640 1 -- 172.21.15.78:0/268314887 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559168492650 con 0x55916858ea50 2024-08-18T23:45:32.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.001+0000 7f244659c640 1 --2- 172.21.15.78:0/268314887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858ea50 0x55916859f0c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f243c09ada0 tx=0x7f243c0bb130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a9a41af4060542 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:32.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.001+0000 7f242aff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:45:32.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.001+0000 7f242aff5640 1 -- 172.21.15.78:0/268314887 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c0b3030 con 0x55916858ea50 2024-08-18T23:45:32.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.001+0000 7f244f550740 1 -- 172.21.15.78:0/268314887 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f243c099140 con 0x55916858ea50 2024-08-18T23:45:32.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.001+0000 7f242aff5640 1 -- 172.21.15.78:0/268314887 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f243c09a6f0 con 0x55916858ea50 2024-08-18T23:45:32.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.002+0000 7f242aff5640 1 -- 172.21.15.78:0/268314887 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f243c081050 con 0x55916858ea50 2024-08-18T23:45:32.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.002+0000 7f244f550740 10 client.15627 fetch_fsmap learned FSMap version 25 2024-08-18T23:45:32.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.002+0000 7f244f550740 1 -- 172.21.15.78:0/268314887 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5591682653c0 con 0x55916858ea50 2024-08-18T23:45:32.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.002+0000 7f242aff5640 1 -- 172.21.15.78:0/268314887 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f243c099140 con 0x55916858ea50 2024-08-18T23:45:32.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.002+0000 7f242aff5640 1 -- 172.21.15.78:0/268314887 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f243c023060 con 0x55916858ea50 2024-08-18T23:45:32.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.002+0000 7f244f550740 10 client.15627 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:45:32.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.002+0000 7f244f550740 10 client.15627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:45:32.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.002+0000 7f244f550740 20 client.15627 populate_metadata read hostname 'smithi078' 2024-08-18T23:45:32.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.002+0000 7f244f550740 1 --2- 172.21.15.78:0/268314887 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5591685b1670 0x5591685b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:32.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.002+0000 7f244f550740 4 client.15627 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:45:32.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.002+0000 7f244f550740 1 -- 172.21.15.78:0/268314887 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559168500ef0 con 0x5591685b1670 2024-08-18T23:45:32.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.003+0000 7f2446d9d640 1 --2- 172.21.15.78:0/268314887 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5591685b1670 0x5591685b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:32.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.003+0000 7f2446d9d640 1 --2- 172.21.15.78:0/268314887 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5591685b1670 0x5591685b3a60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ab7f49fdb5af34f server_cookie=b715ab8a830e3e86 in_seq=0 out_seq=0 2024-08-18T23:45:32.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.003+0000 7f242aff5640 10 client.15627 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:32.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.007+0000 7f242aff5640 1 -- 172.21.15.78:0/268314887 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559168500ef0 con 0x5591685b1670 2024-08-18T23:45:32.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.007+0000 7f242aff5640 10 client.15627 handle_command_reply: tid=0 2024-08-18T23:45:32.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.007+0000 7f244f550740 1 client.15627 shutdown 2024-08-18T23:45:32.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.007+0000 7f244f550740 1 -- 172.21.15.78:0/268314887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858ea50 msgr2=0x55916859f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:32.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.007+0000 7f244f550740 1 --2- 172.21.15.78:0/268314887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858ea50 0x55916859f0c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f243c09ada0 tx=0x7f243c0bb130 comp rx=0 tx=0).stop 2024-08-18T23:45:32.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.007+0000 7f244f550740 1 -- 172.21.15.78:0/268314887 shutdown_connections 2024-08-18T23:45:32.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.007+0000 7f244f550740 1 --2- 172.21.15.78:0/268314887 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5591685b1670 0x5591685b3a60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.007+0000 7f244f550740 1 --2- 172.21.15.78:0/268314887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591685a1c00 0x55916859f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.007+0000 7f244f550740 1 --2- 172.21.15.78:0/268314887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858ea50 0x55916859f0c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.007+0000 7f244f550740 1 --2- 172.21.15.78:0/268314887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55916858e130 0x55916859eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.007+0000 7f244f550740 1 -- 172.21.15.78:0/268314887 >> 172.21.15.78:0/268314887 conn(0x55916858a7a0 msgr2=0x559168589900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:32.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.007+0000 7f242aff5640 0 client.15627 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:32.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.007+0000 7f244f550740 1 -- 172.21.15.78:0/268314887 shutdown_connections 2024-08-18T23:45:32.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.007+0000 7f244f550740 1 -- 172.21.15.78:0/268314887 wait complete. 2024-08-18T23:45:32.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.008+0000 7f244f550740 20 client.15627 trim_cache size 0 max 16384 2024-08-18T23:45:32.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.025+0000 7f244f550740 1 Processor -- start 2024-08-18T23:45:32.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.025+0000 7f244f550740 1 -- start start 2024-08-18T23:45:32.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.025+0000 7f244f550740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858e130 0x55916859f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.025+0000 7f244f550740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55916858ea50 0x55916859bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.025+0000 7f244f550740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591685a1c00 0x55916859c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.025+0000 7f244f550740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f24481b79b0 con 0x55916858e130 2024-08-18T23:45:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.025+0000 7f244f550740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24481b5280 con 0x5591685a1c00 2024-08-18T23:45:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.025+0000 7f244f550740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f24481b30d0 con 0x55916858ea50 2024-08-18T23:45:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.026+0000 7f2446d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858e130 0x55916859f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.026+0000 7f2446d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858e130 0x55916859f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37794/0 (socket says 172.21.15.78:37794) 2024-08-18T23:45:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.026+0000 7f2446d9d640 1 -- 172.21.15.78:0/1853992057 learned_addr learned my addr 172.21.15.78:0/1853992057 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.026+0000 7f244759e640 1 --2- 172.21.15.78:0/1853992057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591685a1c00 0x55916859c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.026+0000 7f244659c640 1 --2- 172.21.15.78:0/1853992057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55916858ea50 0x55916859bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.026+0000 7f2446d9d640 1 -- 172.21.15.78:0/1853992057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55916858ea50 msgr2=0x55916859bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:32.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.026+0000 7f2446d9d640 1 --2- 172.21.15.78:0/1853992057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55916858ea50 0x55916859bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.026+0000 7f2446d9d640 1 -- 172.21.15.78:0/1853992057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591685a1c00 msgr2=0x55916859c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:32.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.026+0000 7f2446d9d640 1 --2- 172.21.15.78:0/1853992057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591685a1c00 0x55916859c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.026+0000 7f2446d9d640 1 -- 172.21.15.78:0/1853992057 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55916858e840 con 0x55916858e130 2024-08-18T23:45:32.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.026+0000 7f2446d9d640 1 --2- 172.21.15.78:0/1853992057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858e130 0x55916859f450 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2438009780 tx=0x7f243803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a4762eea059508 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:32.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.026+0000 7f242cff9640 1 -- 172.21.15.78:0/1853992057 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243805a070 con 0x55916858e130 2024-08-18T23:45:32.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.026+0000 7f242cff9640 1 -- 172.21.15.78:0/1853992057 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2438041070 con 0x55916858e130 2024-08-18T23:45:32.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.026+0000 7f242cff9640 1 -- 172.21.15.78:0/1853992057 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438032700 con 0x55916858e130 2024-08-18T23:45:32.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.027+0000 7f244f550740 1 -- 172.21.15.78:0/1853992057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858e130 msgr2=0x55916859f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:32.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.027+0000 7f244f550740 1 --2- 172.21.15.78:0/1853992057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858e130 0x55916859f450 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2438009780 tx=0x7f243803f000 comp rx=0 tx=0).stop 2024-08-18T23:45:32.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.027+0000 7f244f550740 1 -- 172.21.15.78:0/1853992057 shutdown_connections 2024-08-18T23:45:32.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.027+0000 7f244f550740 1 --2- 172.21.15.78:0/1853992057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5591685a1c00 0x55916859c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.027+0000 7f244f550740 1 --2- 172.21.15.78:0/1853992057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55916858ea50 0x55916859bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.027+0000 7f244f550740 1 --2- 172.21.15.78:0/1853992057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55916858e130 0x55916859f450 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.027+0000 7f244f550740 1 -- 172.21.15.78:0/1853992057 >> 172.21.15.78:0/1853992057 conn(0x55916858a7a0 msgr2=0x55916859dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:32.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.027+0000 7f244f550740 1 -- 172.21.15.78:0/1853992057 shutdown_connections 2024-08-18T23:45:32.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.027+0000 7f244f550740 1 -- 172.21.15.78:0/1853992057 wait complete. 2024-08-18T23:45:32.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.027+0000 7f244f550740 1 -- start start 2024-08-18T23:45:32.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f244f550740 1 Processor -- start 2024-08-18T23:45:32.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f244f550740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55916858e130 0x5591685b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:32.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f244f550740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55916858ea50 0x5591685b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:32.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f244f550740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591685a1c00 0x5591685b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:32.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f244f550740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f24481b3740 con 0x5591685a1c00 2024-08-18T23:45:32.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f244f550740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24481b4bb0 con 0x55916858e130 2024-08-18T23:45:32.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f244f550740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f24481b3e10 con 0x55916858ea50 2024-08-18T23:45:32.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f244f550740 10 client.0 _ll_register_callbacks cb 0x7f243c09aa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:45:32.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f2446d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55916858e130 0x5591685b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:32.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f244759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591685a1c00 0x5591685b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:32.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f2446d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55916858e130 0x5591685b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:57538/0 (socket says 172.21.15.78:57538) 2024-08-18T23:45:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f2446d9d640 1 -- 172.21.15.78:0/520630733 learned_addr learned my addr 172.21.15.78:0/520630733 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f244659c640 1 --2- 172.21.15.78:0/520630733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55916858ea50 0x5591685b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f2446d9d640 1 -- 172.21.15.78:0/520630733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55916858ea50 msgr2=0x5591685b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f2446d9d640 1 --2- 172.21.15.78:0/520630733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55916858ea50 0x5591685b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f2446d9d640 1 -- 172.21.15.78:0/520630733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591685a1c00 msgr2=0x5591685b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f2446d9d640 1 --2- 172.21.15.78:0/520630733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591685a1c00 0x5591685b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f2446d9d640 1 -- 172.21.15.78:0/520630733 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55916861a290 con 0x55916858e130 2024-08-18T23:45:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f244659c640 1 --2- 172.21.15.78:0/520630733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55916858ea50 0x5591685b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:45:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f2446d9d640 1 --2- 172.21.15.78:0/520630733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55916858e130 0x5591685b3730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2438003f30 tx=0x7f2438005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a908a035467c28e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.028+0000 7f242aff5640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:45:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.029+0000 7f242aff5640 1 -- 172.21.15.78:0/520630733 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243805a070 con 0x55916858e130 2024-08-18T23:45:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.029+0000 7f242aff5640 1 -- 172.21.15.78:0/520630733 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2438003800 con 0x55916858e130 2024-08-18T23:45:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.029+0000 7f244f550740 1 -- 172.21.15.78:0/520630733 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2438041070 con 0x55916858e130 2024-08-18T23:45:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.029+0000 7f242aff5640 1 -- 172.21.15.78:0/520630733 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2438003a00 con 0x55916858e130 2024-08-18T23:45:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.029+0000 7f244f550740 10 client.25109 fetch_fsmap learned FSMap version 25 2024-08-18T23:45:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.029+0000 7f244f550740 1 -- 172.21.15.78:0/520630733 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55916861b5a0 con 0x55916858e130 2024-08-18T23:45:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.029+0000 7f242aff5640 1 -- 172.21.15.78:0/520630733 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2438041070 con 0x55916858e130 2024-08-18T23:45:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.029+0000 7f242aff5640 1 -- 172.21.15.78:0/520630733 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f243805f460 con 0x55916858e130 2024-08-18T23:45:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.029+0000 7f244f550740 10 client.25109 fetch_fsmap finished waiting for FSMap version 25 2024-08-18T23:45:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.029+0000 7f244f550740 10 client.25109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:45:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.029+0000 7f244f550740 20 client.25109 populate_metadata read hostname 'smithi078' 2024-08-18T23:45:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.029+0000 7f244f550740 1 --2- 172.21.15.78:0/520630733 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55916861bbf0 0x55916861e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:32.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.029+0000 7f244f550740 4 client.25109 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:45:32.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.029+0000 7f244f550740 1 -- 172.21.15.78:0/520630733 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5591684fb400 con 0x55916861bbf0 2024-08-18T23:45:32.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.030+0000 7f244659c640 1 --2- 172.21.15.78:0/520630733 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55916861bbf0 0x55916861e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:32.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.030+0000 7f242aff5640 10 client.25109 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:32.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.030+0000 7f244659c640 1 --2- 172.21.15.78:0/520630733 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55916861bbf0 0x55916861e010 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5398d25b5dfb402 server_cookie=ddd6a2a67549abab in_seq=0 out_seq=0 2024-08-18T23:45:32.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.036+0000 7f242aff5640 1 -- 172.21.15.78:0/520630733 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5591684fb400 con 0x55916861bbf0 2024-08-18T23:45:32.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.036+0000 7f242aff5640 10 client.25109 handle_command_reply: tid=0 2024-08-18T23:45:32.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.036+0000 7f244f550740 1 client.25109 shutdown 2024-08-18T23:45:32.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.037+0000 7f244f550740 1 -- 172.21.15.78:0/520630733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55916858e130 msgr2=0x5591685b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.037+0000 7f244f550740 1 --2- 172.21.15.78:0/520630733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55916858e130 0x5591685b3730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2438003f30 tx=0x7f2438005fa0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.037+0000 7f244f550740 1 -- 172.21.15.78:0/520630733 shutdown_connections 2024-08-18T23:45:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.037+0000 7f244f550740 1 --2- 172.21.15.78:0/520630733 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55916861bbf0 0x55916861e010 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.037+0000 7f244f550740 1 --2- 172.21.15.78:0/520630733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5591685a1c00 0x5591685b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.037+0000 7f242aff5640 0 client.25109 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.037+0000 7f244f550740 1 --2- 172.21.15.78:0/520630733 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55916858ea50 0x5591685b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.037+0000 7f244f550740 1 --2- 172.21.15.78:0/520630733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55916858e130 0x5591685b3730 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.037+0000 7f244f550740 1 -- 172.21.15.78:0/520630733 >> 172.21.15.78:0/520630733 conn(0x55916858a7a0 msgr2=0x5591685acc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.037+0000 7f244f550740 1 -- 172.21.15.78:0/520630733 shutdown_connections 2024-08-18T23:45:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.037+0000 7f244f550740 1 -- 172.21.15.78:0/520630733 wait complete. 2024-08-18T23:45:32.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.037+0000 7f244f550740 20 client.25109 trim_cache size 0 max 16384 2024-08-18T23:45:32.041 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:45:32.041 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-08-18T23:45:32.041 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-08-18T23:45:32.041 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:45:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.037+0000 7f244d152640 1 -- 172.21.15.78:0/3479934658 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f24300a2c00 msgr2=0x7f24300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.037+0000 7f244d152640 1 --2- 172.21.15.78:0/3479934658 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f24300a2c00 0x7f24300a50c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f24380095f0 tx=0x7f2438005c90 comp rx=0 tx=0).stop 2024-08-18T23:45:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.037+0000 7f244d152640 1 -- 172.21.15.78:0/3479934658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2448109840 msgr2=0x7f24481a6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.037+0000 7f244d152640 1 --2- 172.21.15.78:0/3479934658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2448109840 0x7f24481a6bf0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f243c00d6d0 tx=0x7f243c00db90 comp rx=0 tx=0).stop 2024-08-18T23:45:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.038+0000 7f244d152640 1 -- 172.21.15.78:0/3479934658 shutdown_connections 2024-08-18T23:45:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.038+0000 7f244d152640 1 --2- 172.21.15.78:0/3479934658 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f24300a2c00 0x7f24300a50c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.038+0000 7f244d152640 1 --2- 172.21.15.78:0/3479934658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24481147e0 0x7f24481a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.038+0000 7f244d152640 1 --2- 172.21.15.78:0/3479934658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2448109840 0x7f24481a6bf0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.038+0000 7f244d152640 1 --2- 172.21.15.78:0/3479934658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2448108f20 0x7f24481a66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:32.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.038+0000 7f244d152640 1 -- 172.21.15.78:0/3479934658 >> 172.21.15.78:0/3479934658 conn(0x7f244806de70 msgr2=0x7f2448070630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:32.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.038+0000 7f244d152640 1 -- 172.21.15.78:0/3479934658 shutdown_connections 2024-08-18T23:45:32.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:32.038+0000 7f244d152640 1 -- 172.21.15.78:0/3479934658 wait complete. 2024-08-18T23:45:32.755 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-18T23:45:32.755 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:45:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:45:33.441 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:33 smithi078 ceph-mon[18440]: pgmap v387: 129 pgs: 129 active+clean; 9.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 894 KiB/s wr, 64 op/s 2024-08-18T23:45:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.522+0000 7f31a5144640 1 -- 172.21.15.78:0/754399221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a0103470 msgr2=0x7f31a01078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.522+0000 7f31a5144640 1 --2- 172.21.15.78:0/754399221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a0103470 0x7f31a01078b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3194009930 tx=0x7f319402a420 comp rx=0 tx=0).stop 2024-08-18T23:45:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.522+0000 7f31a5144640 1 -- 172.21.15.78:0/754399221 shutdown_connections 2024-08-18T23:45:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.522+0000 7f31a5144640 1 --2- 172.21.15.78:0/754399221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31a0107fc0 0x7f31a010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.522+0000 7f31a5144640 1 --2- 172.21.15.78:0/754399221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a0103470 0x7f31a01078b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.522+0000 7f31a5144640 1 --2- 172.21.15.78:0/754399221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31a0102ac0 0x7f31a0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.522+0000 7f31a5144640 1 -- 172.21.15.78:0/754399221 >> 172.21.15.78:0/754399221 conn(0x7f31a00fc500 msgr2=0x7f31a00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.522+0000 7f31a5144640 1 -- 172.21.15.78:0/754399221 shutdown_connections 2024-08-18T23:45:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.522+0000 7f31a5144640 1 -- 172.21.15.78:0/754399221 wait complete. 2024-08-18T23:45:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.523+0000 7f31a5144640 1 Processor -- start 2024-08-18T23:45:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.523+0000 7f31a5144640 1 -- start start 2024-08-18T23:45:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.523+0000 7f31a5144640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31a0102ac0 0x7f31a01982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f31a5144640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31a0103470 0x7f31a0198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f31a5144640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a0107fc0 0x7f31a019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f31a5144640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31a0111d90 con 0x7f31a0107fc0 2024-08-18T23:45:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f31a5144640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f31a0111c10 con 0x7f31a0102ac0 2024-08-18T23:45:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f31a5144640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f31a0111f10 con 0x7f31a0103470 2024-08-18T23:45:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f319ed9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31a0102ac0 0x7f31a01982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f319f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a0107fc0 0x7f31a019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f319ed9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31a0102ac0 0x7f31a01982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:57570/0 (socket says 172.21.15.78:57570) 2024-08-18T23:45:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f319f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a0107fc0 0x7f31a019cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37814/0 (socket says 172.21.15.78:37814) 2024-08-18T23:45:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f319ed9d640 1 -- 172.21.15.78:0/1485590016 learned_addr learned my addr 172.21.15.78:0/1485590016 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f319e59c640 1 --2- 172.21.15.78:0/1485590016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31a0103470 0x7f31a0198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f319f59e640 1 -- 172.21.15.78:0/1485590016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31a0103470 msgr2=0x7f31a0198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f319f59e640 1 --2- 172.21.15.78:0/1485590016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31a0103470 0x7f31a0198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f319f59e640 1 -- 172.21.15.78:0/1485590016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31a0102ac0 msgr2=0x7f31a01982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f319f59e640 1 --2- 172.21.15.78:0/1485590016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31a0102ac0 0x7f31a01982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f319f59e640 1 -- 172.21.15.78:0/1485590016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a019d290 con 0x7f31a0107fc0 2024-08-18T23:45:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f319ed9d640 1 --2- 172.21.15.78:0/1485590016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31a0102ac0 0x7f31a01982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:45:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f319f59e640 1 --2- 172.21.15.78:0/1485590016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a0107fc0 0x7f31a019cbb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3198009fd0 tx=0x7f319800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f3187fff640 1 -- 172.21.15.78:0/1485590016 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319800ce20 con 0x7f31a0107fc0 2024-08-18T23:45:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f3187fff640 1 -- 172.21.15.78:0/1485590016 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3198005020 con 0x7f31a0107fc0 2024-08-18T23:45:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f31a5144640 1 -- 172.21.15.78:0/1485590016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a019d580 con 0x7f31a0107fc0 2024-08-18T23:45:33.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f3187fff640 1 -- 172.21.15.78:0/1485590016 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3198010720 con 0x7f31a0107fc0 2024-08-18T23:45:33.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.524+0000 7f31a5144640 1 -- 172.21.15.78:0/1485590016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a0100650 con 0x7f31a0107fc0 2024-08-18T23:45:33.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.526+0000 7f31a5144640 1 -- 172.21.15.78:0/1485590016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a006a520 con 0x7f31a0107fc0 2024-08-18T23:45:33.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.526+0000 7f3187fff640 1 -- 172.21.15.78:0/1485590016 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3198004b20 con 0x7f31a0107fc0 2024-08-18T23:45:33.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.527+0000 7f3187fff640 1 --2- 172.21.15.78:0/1485590016 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f31880a2b80 0x7f31880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:33.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.527+0000 7f319ed9d640 1 --2- 172.21.15.78:0/1485590016 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f31880a2b80 0x7f31880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:33.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.527+0000 7f3187fff640 1 -- 172.21.15.78:0/1485590016 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f31980cf9e0 con 0x7f31a0107fc0 2024-08-18T23:45:33.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.528+0000 7f319ed9d640 1 --2- 172.21.15.78:0/1485590016 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f31880a2b80 0x7f31880a5040 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3190005fd0 tx=0x7f319000b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:33.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.531+0000 7f3187fff640 1 -- 172.21.15.78:0/1485590016 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f319809c5b0 con 0x7f31a0107fc0 2024-08-18T23:45:33.602 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:33 smithi123 ceph-mon[21808]: pgmap v387: 129 pgs: 129 active+clean; 9.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 894 KiB/s wr, 64 op/s 2024-08-18T23:45:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:33 smithi161 ceph-mon[22027]: pgmap v387: 129 pgs: 129 active+clean; 9.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 894 KiB/s wr, 64 op/s 2024-08-18T23:45:33.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.760+0000 7f31a5144640 1 -- 172.21.15.78:0/1485590016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31a0199640 con 0x7f31a0107fc0 2024-08-18T23:45:33.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.761+0000 7f3187fff640 1 -- 172.21.15.78:0/1485590016 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11933 (secure 0 0 0) 0x7f319809beb0 con 0x7f31a0107fc0 2024-08-18T23:45:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:45:33.762 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":26,"btime":"2024-08-18T23:45:33:306257+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:45:32.491054+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:45:33.763 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 26 2024-08-18T23:45:33.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.763+0000 7f31a5144640 1 -- 172.21.15.78:0/1485590016 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f31880a2b80 msgr2=0x7f31880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:33.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.763+0000 7f31a5144640 1 --2- 172.21.15.78:0/1485590016 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f31880a2b80 0x7f31880a5040 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3190005fd0 tx=0x7f319000b480 comp rx=0 tx=0).stop 2024-08-18T23:45:33.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.763+0000 7f31a5144640 1 -- 172.21.15.78:0/1485590016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a0107fc0 msgr2=0x7f31a019cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:33.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.763+0000 7f31a5144640 1 --2- 172.21.15.78:0/1485590016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a0107fc0 0x7f31a019cbb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3198009fd0 tx=0x7f319800ef10 comp rx=0 tx=0).stop 2024-08-18T23:45:33.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.763+0000 7f31a5144640 1 -- 172.21.15.78:0/1485590016 shutdown_connections 2024-08-18T23:45:33.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.763+0000 7f31a5144640 1 --2- 172.21.15.78:0/1485590016 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f31880a2b80 0x7f31880a5040 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:33.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.763+0000 7f31a5144640 1 --2- 172.21.15.78:0/1485590016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a0107fc0 0x7f31a019cbb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:33.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.763+0000 7f31a5144640 1 --2- 172.21.15.78:0/1485590016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31a0103470 0x7f31a0198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:33.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.763+0000 7f31a5144640 1 --2- 172.21.15.78:0/1485590016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31a0102ac0 0x7f31a01982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:33.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.763+0000 7f31a5144640 1 -- 172.21.15.78:0/1485590016 >> 172.21.15.78:0/1485590016 conn(0x7f31a00fc500 msgr2=0x7f31a0105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:33.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.763+0000 7f31a5144640 1 -- 172.21.15.78:0/1485590016 shutdown_connections 2024-08-18T23:45:33.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:33.763+0000 7f31a5144640 1 -- 172.21.15.78:0/1485590016 wait complete. 2024-08-18T23:45:34.412 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:34 smithi078 ceph-mon[18440]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:45:34.412 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:34 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1485590016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:45:34.412 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:34 smithi078 ceph-mon[18440]: pgmap v388: 129 pgs: 129 active+clean; 21 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.9 MiB/s wr, 105 op/s 2024-08-18T23:45:34.439 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-18T23:45:34.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:34 smithi161 ceph-mon[22027]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:45:34.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:34 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1485590016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:45:34.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:34 smithi161 ceph-mon[22027]: pgmap v388: 129 pgs: 129 active+clean; 21 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.9 MiB/s wr, 105 op/s 2024-08-18T23:45:34.729 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:45:34.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:34 smithi123 ceph-mon[21808]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:45:34.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:34 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1485590016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:45:34.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:34 smithi123 ceph-mon[21808]: pgmap v388: 129 pgs: 129 active+clean; 21 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.9 MiB/s wr, 105 op/s 2024-08-18T23:45:35.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.227+0000 7f63b77d6640 1 -- 172.21.15.78:0/1131381596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63b01018b0 msgr2=0x7f63b0101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:35.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.227+0000 7f63b77d6640 1 --2- 172.21.15.78:0/1131381596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63b01018b0 0x7f63b0101d10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009930 tx=0x7f63a002a420 comp rx=0 tx=0).stop 2024-08-18T23:45:35.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.227+0000 7f63b77d6640 1 -- 172.21.15.78:0/1131381596 shutdown_connections 2024-08-18T23:45:35.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.227+0000 7f63b77d6640 1 --2- 172.21.15.78:0/1131381596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63b0102250 0x7f63b010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.227+0000 7f63b77d6640 1 --2- 172.21.15.78:0/1131381596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63b01018b0 0x7f63b0101d10 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.227+0000 7f63b77d6640 1 --2- 172.21.15.78:0/1131381596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63b01078b0 0x7f63b0107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.227+0000 7f63b77d6640 1 -- 172.21.15.78:0/1131381596 >> 172.21.15.78:0/1131381596 conn(0x7f63b00fd5f0 msgr2=0x7f63b00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:35.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.228+0000 7f63b77d6640 1 -- 172.21.15.78:0/1131381596 shutdown_connections 2024-08-18T23:45:35.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.228+0000 7f63b77d6640 1 -- 172.21.15.78:0/1131381596 wait complete. 2024-08-18T23:45:35.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.228+0000 7f63b77d6640 1 Processor -- start 2024-08-18T23:45:35.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.228+0000 7f63b77d6640 1 -- start start 2024-08-18T23:45:35.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63b77d6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63b01018b0 0x7f63b01981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:35.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63b77d6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63b0102250 0x7f63b01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:35.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63b77d6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63b01078b0 0x7f63b019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:35.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63b77d6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0111cb0 con 0x7f63b01078b0 2024-08-18T23:45:35.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63b77d6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0111b30 con 0x7f63b01018b0 2024-08-18T23:45:35.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63b77d6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0111e30 con 0x7f63b0102250 2024-08-18T23:45:35.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63b5d73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63b01078b0 0x7f63b019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:35.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63b5d73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63b01078b0 0x7f63b019ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37840/0 (socket says 172.21.15.78:37840) 2024-08-18T23:45:35.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63b5572640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63b01018b0 0x7f63b01981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:35.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63b5d73640 1 -- 172.21.15.78:0/3849657372 learned_addr learned my addr 172.21.15.78:0/3849657372 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:35.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63b5d73640 1 -- 172.21.15.78:0/3849657372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63b0102250 msgr2=0x7f63b01986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:35.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63b4d71640 1 --2- 172.21.15.78:0/3849657372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63b0102250 0x7f63b01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:35.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63b5d73640 1 --2- 172.21.15.78:0/3849657372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63b0102250 0x7f63b01986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63b5d73640 1 -- 172.21.15.78:0/3849657372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63b01018b0 msgr2=0x7f63b01981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:35.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63b5d73640 1 --2- 172.21.15.78:0/3849657372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63b01018b0 0x7f63b01981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63b5d73640 1 -- 172.21.15.78:0/3849657372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b019d160 con 0x7f63b01078b0 2024-08-18T23:45:35.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63b5d73640 1 --2- 172.21.15.78:0/3849657372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63b01078b0 0x7f63b019ca80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f63a800eaa0 tx=0x7f63a800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:35.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63a67fc640 1 -- 172.21.15.78:0/3849657372 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a800cdb0 con 0x7f63b01078b0 2024-08-18T23:45:35.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63a67fc640 1 -- 172.21.15.78:0/3849657372 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a8005020 con 0x7f63b01078b0 2024-08-18T23:45:35.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.229+0000 7f63a67fc640 1 -- 172.21.15.78:0/3849657372 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a80116b0 con 0x7f63b01078b0 2024-08-18T23:45:35.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.230+0000 7f63b77d6640 1 -- 172.21.15.78:0/3849657372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b019d450 con 0x7f63b01078b0 2024-08-18T23:45:35.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.230+0000 7f63b77d6640 1 -- 172.21.15.78:0/3849657372 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b01a4d30 con 0x7f63b01078b0 2024-08-18T23:45:35.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.232+0000 7f63a67fc640 1 -- 172.21.15.78:0/3849657372 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f63a8005560 con 0x7f63b01078b0 2024-08-18T23:45:35.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.232+0000 7f63b9bd4740 1 Processor -- start 2024-08-18T23:45:35.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.232+0000 7f63b9bd4740 1 -- start start 2024-08-18T23:45:35.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63b9bd4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed76072ff0 0x55ed760733d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:35.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63b9bd4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 0x55ed7607dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:35.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63b9bd4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7607e1d0 0x55ed760805c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:35.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63b9bd4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ed75fd8a40 con 0x55ed76072ff0 2024-08-18T23:45:35.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63b9bd4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ed75f709d0 con 0x55ed76073910 2024-08-18T23:45:35.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63b9bd4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ed75fd4a00 con 0x55ed7607e1d0 2024-08-18T23:45:35.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63b4d71640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed76072ff0 0x55ed760733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:35.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63b5572640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 0x55ed7607dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:35.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63b4d71640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed76072ff0 0x55ed760733d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37856/0 (socket says 172.21.15.78:37856) 2024-08-18T23:45:35.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63b4d71640 1 -- 172.21.15.78:0/1445035706 learned_addr learned my addr 172.21.15.78:0/1445035706 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:35.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63b5572640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 0x55ed7607dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:57600/0 (socket says 172.21.15.78:57600) 2024-08-18T23:45:35.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63a67fc640 1 --2- 172.21.15.78:0/3849657372 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f639c0a2c00 0x7f639c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:35.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63b4d71640 1 --2- 172.21.15.78:0/1445035706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7607e1d0 0x55ed760805c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:35.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63b5d73640 1 --2- 172.21.15.78:0/3849657372 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f639c0a2c00 0x7f639c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:35.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63b5572640 1 -- 172.21.15.78:0/1445035706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7607e1d0 msgr2=0x55ed760805c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:35.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63b5572640 1 --2- 172.21.15.78:0/1445035706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7607e1d0 0x55ed760805c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63b5572640 1 -- 172.21.15.78:0/1445035706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed76072ff0 msgr2=0x55ed760733d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:35.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63b5572640 1 --2- 172.21.15.78:0/1445035706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed76072ff0 0x55ed760733d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63b5572640 1 -- 172.21.15.78:0/1445035706 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed75f77650 con 0x55ed76073910 2024-08-18T23:45:35.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63b4d71640 1 --2- 172.21.15.78:0/1445035706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed76072ff0 0x55ed760733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:45:35.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.233+0000 7f63a67fc640 1 -- 172.21.15.78:0/3849657372 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f63a80cf5a0 con 0x7f63b01078b0 2024-08-18T23:45:35.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.234+0000 7f63b5572640 1 --2- 172.21.15.78:0/1445035706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 0x55ed7607dc90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f63ac005fd0 tx=0x7f63ac0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82b5f062e29c3597 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:35.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.234+0000 7f63b5d73640 1 --2- 172.21.15.78:0/3849657372 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f639c0a2c00 0x7f639c0a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009270 tx=0x7f63a80c6600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:35.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.234+0000 7f639b7fe640 1 -- 172.21.15.78:0/1445035706 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac01c480 con 0x55ed76073910 2024-08-18T23:45:35.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.234+0000 7f639b7fe640 1 -- 172.21.15.78:0/1445035706 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63ac01cb00 con 0x55ed76073910 2024-08-18T23:45:35.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.234+0000 7f639b7fe640 1 -- 172.21.15.78:0/1445035706 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac011b00 con 0x55ed76073910 2024-08-18T23:45:35.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.234+0000 7f63b9bd4740 1 -- 172.21.15.78:0/1445035706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 msgr2=0x55ed7607dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:35.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.234+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/1445035706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 0x55ed7607dc90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f63ac005fd0 tx=0x7f63ac0107a0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.235+0000 7f63b9bd4740 1 -- 172.21.15.78:0/1445035706 shutdown_connections 2024-08-18T23:45:35.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.235+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/1445035706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7607e1d0 0x55ed760805c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.235+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/1445035706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 0x55ed7607dc90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.235+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/1445035706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed76072ff0 0x55ed760733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.235+0000 7f63b9bd4740 1 -- 172.21.15.78:0/1445035706 >> 172.21.15.78:0/1445035706 conn(0x55ed7606f580 msgr2=0x55ed7606f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:35.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.235+0000 7f63b9bd4740 1 -- 172.21.15.78:0/1445035706 shutdown_connections 2024-08-18T23:45:35.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.235+0000 7f63b9bd4740 1 -- 172.21.15.78:0/1445035706 wait complete. 2024-08-18T23:45:35.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.235+0000 7f63b9bd4740 1 -- start start 2024-08-18T23:45:35.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.235+0000 7f63b9bd4740 1 Processor -- start 2024-08-18T23:45:35.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b9bd4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed76072ff0 0x55ed7607b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:35.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b9bd4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 0x55ed7607b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:35.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b9bd4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7607e1d0 0x55ed7607bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:35.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b9bd4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ed76080da0 con 0x55ed7607e1d0 2024-08-18T23:45:35.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b9bd4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ed75f93b20 con 0x55ed76073910 2024-08-18T23:45:35.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b9bd4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ed76080f20 con 0x55ed76072ff0 2024-08-18T23:45:35.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b9bd4740 10 client.0 _ll_register_callbacks cb 0x55ed75f91bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:45:35.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b4d71640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7607e1d0 0x55ed7607bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:35.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b4d71640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7607e1d0 0x55ed7607bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37862/0 (socket says 172.21.15.78:37862) 2024-08-18T23:45:35.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b4d71640 1 -- 172.21.15.78:0/891225935 learned_addr learned my addr 172.21.15.78:0/891225935 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:35.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b5572640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 0x55ed7607b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:35.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b4d71640 1 --2- 172.21.15.78:0/891225935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed76072ff0 0x55ed7607b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:35.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b4d71640 1 -- 172.21.15.78:0/891225935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed76072ff0 msgr2=0x55ed7607b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:35.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b4d71640 1 --2- 172.21.15.78:0/891225935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed76072ff0 0x55ed7607b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b4d71640 1 -- 172.21.15.78:0/891225935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 msgr2=0x55ed7607b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:35.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b4d71640 1 --2- 172.21.15.78:0/891225935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 0x55ed7607b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b4d71640 1 -- 172.21.15.78:0/891225935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed75d4a3c0 con 0x55ed7607e1d0 2024-08-18T23:45:35.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b5572640 1 --2- 172.21.15.78:0/891225935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 0x55ed7607b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:45:35.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b4d71640 1 --2- 172.21.15.78:0/891225935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7607e1d0 0x55ed7607bae0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f63a0031550 tx=0x7f63a00058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538f49d97a3701ae server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:35.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f6398ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:45:35.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f6398ff9640 1 -- 172.21.15.78:0/891225935 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0043070 con 0x55ed7607e1d0 2024-08-18T23:45:35.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f6398ff9640 1 -- 172.21.15.78:0/891225935 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f63a0004450 con 0x55ed7607e1d0 2024-08-18T23:45:35.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.236+0000 7f63b9bd4740 1 -- 172.21.15.78:0/891225935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63ac01cb00 con 0x55ed7607e1d0 2024-08-18T23:45:35.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.237+0000 7f6398ff9640 1 -- 172.21.15.78:0/891225935 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f63a0035040 con 0x55ed7607e1d0 2024-08-18T23:45:35.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.237+0000 7f63b9bd4740 10 client.15651 fetch_fsmap learned FSMap version 26 2024-08-18T23:45:35.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.237+0000 7f63b9bd4740 1 -- 172.21.15.78:0/891225935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ed7608c950 con 0x55ed7607e1d0 2024-08-18T23:45:35.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.237+0000 7f6398ff9640 1 -- 172.21.15.78:0/891225935 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63ac01cb00 con 0x55ed7607e1d0 2024-08-18T23:45:35.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.237+0000 7f6398ff9640 1 -- 172.21.15.78:0/891225935 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f63a003a040 con 0x55ed7607e1d0 2024-08-18T23:45:35.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.237+0000 7f63b9bd4740 10 client.15651 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:45:35.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.237+0000 7f63b9bd4740 10 client.15651 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:45:35.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.237+0000 7f63b9bd4740 20 client.15651 populate_metadata read hostname 'smithi078' 2024-08-18T23:45:35.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.237+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/891225935 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55ed7608dd60 0x55ed76090150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:35.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.237+0000 7f63b9bd4740 4 client.15651 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:45:35.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.237+0000 7f63b9bd4740 1 -- 172.21.15.78:0/891225935 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed75fe5fb0 con 0x55ed7608dd60 2024-08-18T23:45:35.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.238+0000 7f63b5572640 1 --2- 172.21.15.78:0/891225935 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55ed7608dd60 0x55ed76090150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:35.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.238+0000 7f63b5572640 1 --2- 172.21.15.78:0/891225935 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55ed7608dd60 0x55ed76090150 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73dd783452895bc5 server_cookie=fcf2ebebecf5cbd1 in_seq=0 out_seq=0 2024-08-18T23:45:35.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.238+0000 7f6398ff9640 10 client.15651 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:45:35.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.239+0000 7f6398ff9640 1 -- 172.21.15.78:0/891225935 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed75fe5fb0 con 0x55ed7608dd60 2024-08-18T23:45:35.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.239+0000 7f6398ff9640 10 client.15651 handle_command_reply: tid=0 2024-08-18T23:45:35.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.239+0000 7f63b9bd4740 1 client.15651 shutdown 2024-08-18T23:45:35.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.239+0000 7f63b9bd4740 1 -- 172.21.15.78:0/891225935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7607e1d0 msgr2=0x55ed7607bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:35.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.239+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/891225935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7607e1d0 0x55ed7607bae0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f63a0031550 tx=0x7f63a00058d0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.239+0000 7f63b9bd4740 1 -- 172.21.15.78:0/891225935 shutdown_connections 2024-08-18T23:45:35.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.239+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/891225935 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55ed7608dd60 0x55ed76090150 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.239+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/891225935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7607e1d0 0x55ed7607bae0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.239+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/891225935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 0x55ed7607b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.239+0000 7f6398ff9640 0 client.15651 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:45:35.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.239+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/891225935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed76072ff0 0x55ed7607b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.239+0000 7f63b9bd4740 1 -- 172.21.15.78:0/891225935 >> 172.21.15.78:0/891225935 conn(0x55ed7606f580 msgr2=0x55ed760752b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:35.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.239+0000 7f63b9bd4740 1 -- 172.21.15.78:0/891225935 shutdown_connections 2024-08-18T23:45:35.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.239+0000 7f63b9bd4740 1 -- 172.21.15.78:0/891225935 wait complete. 2024-08-18T23:45:35.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.239+0000 7f63b9bd4740 20 client.15651 trim_cache size 0 max 16384 2024-08-18T23:45:35.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.253+0000 7f63b9bd4740 1 Processor -- start 2024-08-18T23:45:35.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.253+0000 7f63b9bd4740 1 -- start start 2024-08-18T23:45:35.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.253+0000 7f63b9bd4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76072ff0 0x55ed76095a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:35.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.253+0000 7f63b9bd4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed76073910 0x55ed76095f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:35.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.253+0000 7f63b9bd4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7607e1d0 0x55ed7607c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:35.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.253+0000 7f63b9bd4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01af1a0 con 0x55ed76073910 2024-08-18T23:45:35.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.253+0000 7f63b9bd4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01aca70 con 0x55ed76072ff0 2024-08-18T23:45:35.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.253+0000 7f63b9bd4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01aa8e0 con 0x55ed7607e1d0 2024-08-18T23:45:35.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.254+0000 7f63b5572640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed76073910 0x55ed76095f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:35.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.254+0000 7f63b4d71640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76072ff0 0x55ed76095a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:35.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.254+0000 7f63b5572640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed76073910 0x55ed76095f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37866/0 (socket says 172.21.15.78:37866) 2024-08-18T23:45:35.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.254+0000 7f63b5572640 1 -- 172.21.15.78:0/3071255762 learned_addr learned my addr 172.21.15.78:0/3071255762 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:35.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.254+0000 7f63b4d71640 1 --2- 172.21.15.78:0/3071255762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7607e1d0 0x55ed7607c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:35.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.254+0000 7f63b5572640 1 -- 172.21.15.78:0/3071255762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7607e1d0 msgr2=0x55ed7607c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:35.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.254+0000 7f63b5572640 1 --2- 172.21.15.78:0/3071255762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7607e1d0 0x55ed7607c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.254+0000 7f63b5572640 1 -- 172.21.15.78:0/3071255762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76072ff0 msgr2=0x55ed76095a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:35.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.254+0000 7f63b5572640 1 --2- 172.21.15.78:0/3071255762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76072ff0 0x55ed76095a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.254+0000 7f63b5572640 1 -- 172.21.15.78:0/3071255762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed760904c0 con 0x55ed76073910 2024-08-18T23:45:35.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.254+0000 7f63b5572640 1 --2- 172.21.15.78:0/3071255762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed76073910 0x55ed76095f40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f63ac005fd0 tx=0x7f63ac01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d2559b7e9ba45b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:35.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.254+0000 7f639affd640 1 -- 172.21.15.78:0/3071255762 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac005240 con 0x55ed76073910 2024-08-18T23:45:35.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.254+0000 7f639affd640 1 -- 172.21.15.78:0/3071255762 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63ac01c960 con 0x55ed76073910 2024-08-18T23:45:35.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.254+0000 7f639affd640 1 -- 172.21.15.78:0/3071255762 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac011930 con 0x55ed76073910 2024-08-18T23:45:35.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.255+0000 7f63b9bd4740 1 -- 172.21.15.78:0/3071255762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed76073910 msgr2=0x55ed76095f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:35.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.255+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/3071255762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed76073910 0x55ed76095f40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f63ac005fd0 tx=0x7f63ac01e040 comp rx=0 tx=0).stop 2024-08-18T23:45:35.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.255+0000 7f63b9bd4740 1 -- 172.21.15.78:0/3071255762 shutdown_connections 2024-08-18T23:45:35.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.255+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/3071255762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed7607e1d0 0x55ed7607c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.255+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/3071255762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed76073910 0x55ed76095f40 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.255+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/3071255762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76072ff0 0x55ed76095a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.255+0000 7f63b9bd4740 1 -- 172.21.15.78:0/3071255762 >> 172.21.15.78:0/3071255762 conn(0x55ed7606f580 msgr2=0x55ed7607a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:35.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.255+0000 7f63b9bd4740 1 -- 172.21.15.78:0/3071255762 shutdown_connections 2024-08-18T23:45:35.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.255+0000 7f63b9bd4740 1 -- 172.21.15.78:0/3071255762 wait complete. 2024-08-18T23:45:35.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.256+0000 7f63b9bd4740 1 -- start start 2024-08-18T23:45:35.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.256+0000 7f63b9bd4740 1 Processor -- start 2024-08-18T23:45:35.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.256+0000 7f63b9bd4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed76072ff0 0x55ed76081510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:35.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.256+0000 7f63b9bd4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 0x55ed76081a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:35.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.256+0000 7f63b9bd4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7607e1d0 0x55ed760f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:35.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.256+0000 7f63b9bd4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01aaf30 con 0x55ed7607e1d0 2024-08-18T23:45:35.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.256+0000 7f63b9bd4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01ac3a0 con 0x55ed76073910 2024-08-18T23:45:35.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.256+0000 7f63b9bd4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01ab600 con 0x55ed76072ff0 2024-08-18T23:45:35.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.256+0000 7f63b9bd4740 10 client.0 _ll_register_callbacks cb 0x7f63a0040040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:45:35.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.256+0000 7f63b5572640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 0x55ed76081a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:35.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.256+0000 7f63b4d71640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7607e1d0 0x55ed760f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:35.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.256+0000 7f63b5572640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 0x55ed76081a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:57620/0 (socket says 172.21.15.78:57620) 2024-08-18T23:45:35.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.256+0000 7f63b5572640 1 -- 172.21.15.78:0/2743847903 learned_addr learned my addr 172.21.15.78:0/2743847903 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:35.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.256+0000 7f63b4d71640 1 --2- 172.21.15.78:0/2743847903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed76072ff0 0x55ed76081510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:35.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.257+0000 7f63b5572640 1 -- 172.21.15.78:0/2743847903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed76072ff0 msgr2=0x55ed76081510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:35.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.257+0000 7f63b5572640 1 --2- 172.21.15.78:0/2743847903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed76072ff0 0x55ed76081510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.257+0000 7f63b5572640 1 -- 172.21.15.78:0/2743847903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7607e1d0 msgr2=0x55ed760f69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:35.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.257+0000 7f63b5572640 1 --2- 172.21.15.78:0/2743847903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7607e1d0 0x55ed760f69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.257+0000 7f63b5572640 1 -- 172.21.15.78:0/2743847903 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed760f6f00 con 0x55ed76073910 2024-08-18T23:45:35.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.257+0000 7f63b5572640 1 --2- 172.21.15.78:0/2743847903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 0x55ed76081a50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00b960 tx=0x7f63ac00b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1690a33609542868 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:35.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.257+0000 7f6398ff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:45:35.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.257+0000 7f6398ff9640 1 -- 172.21.15.78:0/2743847903 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac01cb10 con 0x55ed76073910 2024-08-18T23:45:35.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.257+0000 7f63b9bd4740 1 -- 172.21.15.78:0/2743847903 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63ac01c960 con 0x55ed76073910 2024-08-18T23:45:35.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.257+0000 7f6398ff9640 1 -- 172.21.15.78:0/2743847903 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f63ac01c680 con 0x55ed76073910 2024-08-18T23:45:35.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.257+0000 7f6398ff9640 1 -- 172.21.15.78:0/2743847903 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f63ac005040 con 0x55ed76073910 2024-08-18T23:45:35.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.257+0000 7f63b9bd4740 10 client.25133 fetch_fsmap learned FSMap version 26 2024-08-18T23:45:35.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.257+0000 7f63b9bd4740 1 -- 172.21.15.78:0/2743847903 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ed760f8100 con 0x55ed76073910 2024-08-18T23:45:35.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.258+0000 7f6398ff9640 1 -- 172.21.15.78:0/2743847903 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63ac01c960 con 0x55ed76073910 2024-08-18T23:45:35.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.258+0000 7f6398ff9640 1 -- 172.21.15.78:0/2743847903 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f63ac03a060 con 0x55ed76073910 2024-08-18T23:45:35.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.258+0000 7f63b9bd4740 10 client.25133 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:45:35.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.258+0000 7f63b9bd4740 10 client.25133 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:45:35.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.258+0000 7f63b9bd4740 20 client.25133 populate_metadata read hostname 'smithi078' 2024-08-18T23:45:35.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.258+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/2743847903 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55ed760f85f0 0x55ed760faa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:35.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.258+0000 7f63b9bd4740 4 client.25133 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:45:35.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.258+0000 7f63b9bd4740 1 -- 172.21.15.78:0/2743847903 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed75fe04c0 con 0x55ed760f85f0 2024-08-18T23:45:35.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.258+0000 7f63b4d71640 1 --2- 172.21.15.78:0/2743847903 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55ed760f85f0 0x55ed760faa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:35.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.258+0000 7f63b4d71640 1 --2- 172.21.15.78:0/2743847903 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55ed760f85f0 0x55ed760faa10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a12a4e09bc2c1dc0 server_cookie=9933eb87da8b1601 in_seq=0 out_seq=0 2024-08-18T23:45:35.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.258+0000 7f6398ff9640 10 client.25133 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:45:35.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.259+0000 7f6398ff9640 1 -- 172.21.15.78:0/2743847903 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed75fe04c0 con 0x55ed760f85f0 2024-08-18T23:45:35.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.259+0000 7f6398ff9640 10 client.25133 handle_command_reply: tid=0 2024-08-18T23:45:35.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.259+0000 7f63b9bd4740 1 client.25133 shutdown 2024-08-18T23:45:35.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.259+0000 7f63b9bd4740 1 -- 172.21.15.78:0/2743847903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 msgr2=0x55ed76081a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:35.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.259+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/2743847903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 0x55ed76081a50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00b960 tx=0x7f63ac00b630 comp rx=0 tx=0).stop 2024-08-18T23:45:35.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.259+0000 7f63b9bd4740 1 -- 172.21.15.78:0/2743847903 shutdown_connections 2024-08-18T23:45:35.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.259+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/2743847903 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55ed760f85f0 0x55ed760faa10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.259+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/2743847903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ed7607e1d0 0x55ed760f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.259+0000 7f6398ff9640 0 client.25133 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:45:35.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.259+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/2743847903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed76073910 0x55ed76081a50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.259+0000 7f63b9bd4740 1 --2- 172.21.15.78:0/2743847903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed76072ff0 0x55ed76081510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.259+0000 7f63b9bd4740 1 -- 172.21.15.78:0/2743847903 >> 172.21.15.78:0/2743847903 conn(0x55ed7606f580 msgr2=0x55ed76074a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:35.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.259+0000 7f63b9bd4740 1 -- 172.21.15.78:0/2743847903 shutdown_connections 2024-08-18T23:45:35.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.259+0000 7f63b9bd4740 1 -- 172.21.15.78:0/2743847903 wait complete. 2024-08-18T23:45:35.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.259+0000 7f63b9bd4740 20 client.25133 trim_cache size 0 max 16384 2024-08-18T23:45:35.266 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:45:35.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.260+0000 7f63b77d6640 1 -- 172.21.15.78:0/3849657372 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f639c0a2c00 msgr2=0x7f639c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:35.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.260+0000 7f63b77d6640 1 --2- 172.21.15.78:0/3849657372 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f639c0a2c00 0x7f639c0a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009270 tx=0x7f63a80c6600 comp rx=0 tx=0).stop 2024-08-18T23:45:35.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.260+0000 7f63b77d6640 1 -- 172.21.15.78:0/3849657372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63b01078b0 msgr2=0x7f63b019ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:35.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.260+0000 7f63b77d6640 1 --2- 172.21.15.78:0/3849657372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63b01078b0 0x7f63b019ca80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f63a800eaa0 tx=0x7f63a800ef60 comp rx=0 tx=0).stop 2024-08-18T23:45:35.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.260+0000 7f63b77d6640 1 -- 172.21.15.78:0/3849657372 shutdown_connections 2024-08-18T23:45:35.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.260+0000 7f63b77d6640 1 --2- 172.21.15.78:0/3849657372 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f639c0a2c00 0x7f639c0a50c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.260+0000 7f63b77d6640 1 --2- 172.21.15.78:0/3849657372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63b01078b0 0x7f63b019ca80 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.260+0000 7f63b77d6640 1 --2- 172.21.15.78:0/3849657372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63b0102250 0x7f63b01986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.260+0000 7f63b77d6640 1 --2- 172.21.15.78:0/3849657372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63b01018b0 0x7f63b01981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:35.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.260+0000 7f63b77d6640 1 -- 172.21.15.78:0/3849657372 >> 172.21.15.78:0/3849657372 conn(0x7f63b00fd5f0 msgr2=0x7f63b00fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:35.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.260+0000 7f63b77d6640 1 -- 172.21.15.78:0/3849657372 shutdown_connections 2024-08-18T23:45:35.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:35.260+0000 7f63b77d6640 1 -- 172.21.15.78:0/3849657372 wait complete. 2024-08-18T23:45:35.974 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-18T23:45:36.242 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:45:36.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.761+0000 7f1a06dd6640 1 -- 172.21.15.78:0/1512303709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a001033f0 msgr2=0x7f1a0010b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:36.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.761+0000 7f1a06dd6640 1 --2- 172.21.15.78:0/1512303709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a001033f0 0x7f1a0010b8e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009900 tx=0x7f19f002a480 comp rx=0 tx=0).stop 2024-08-18T23:45:36.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.762+0000 7f1a06dd6640 1 -- 172.21.15.78:0/1512303709 shutdown_connections 2024-08-18T23:45:36.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.762+0000 7f1a06dd6640 1 --2- 172.21.15.78:0/1512303709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a0010beb0 0x7f1a0010e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.762+0000 7f1a06dd6640 1 --2- 172.21.15.78:0/1512303709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a001033f0 0x7f1a0010b8e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.762+0000 7f1a06dd6640 1 --2- 172.21.15.78:0/1512303709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a00102ad0 0x7f1a00102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.762+0000 7f1a06dd6640 1 -- 172.21.15.78:0/1512303709 >> 172.21.15.78:0/1512303709 conn(0x7f1a000fc4d0 msgr2=0x7f1a000fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:36.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.762+0000 7f1a06dd6640 1 -- 172.21.15.78:0/1512303709 shutdown_connections 2024-08-18T23:45:36.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.762+0000 7f1a06dd6640 1 -- 172.21.15.78:0/1512303709 wait complete. 2024-08-18T23:45:36.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.762+0000 7f1a06dd6640 1 Processor -- start 2024-08-18T23:45:36.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.762+0000 7f1a06dd6640 1 -- start start 2024-08-18T23:45:36.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.762+0000 7f1a06dd6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a00102ad0 0x7f1a0019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:36.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.762+0000 7f1a06dd6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a001033f0 0x7f1a0019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:36.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.762+0000 7f1a06dd6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a0010beb0 0x7f1a0019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:36.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.763+0000 7f1a06dd6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1a00111c90 con 0x7f1a001033f0 2024-08-18T23:45:36.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.763+0000 7f19f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a001033f0 0x7f1a0019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:36.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.763+0000 7f1a06dd6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1a00111b10 con 0x7f1a00102ad0 2024-08-18T23:45:36.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.763+0000 7f1a06dd6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1a00111e10 con 0x7f1a0010beb0 2024-08-18T23:45:36.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.763+0000 7f19f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a001033f0 0x7f1a0019a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55114/0 (socket says 172.21.15.78:55114) 2024-08-18T23:45:36.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.763+0000 7f19f7fff640 1 -- 172.21.15.78:0/1813409716 learned_addr learned my addr 172.21.15.78:0/1813409716 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:36.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.763+0000 7f1a04b72640 1 --2- 172.21.15.78:0/1813409716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a00102ad0 0x7f1a0019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:36.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.763+0000 7f1a05373640 1 --2- 172.21.15.78:0/1813409716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a0010beb0 0x7f1a0019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:36.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.763+0000 7f19f7fff640 1 -- 172.21.15.78:0/1813409716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a0010beb0 msgr2=0x7f1a0019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:36.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.763+0000 7f19f7fff640 1 --2- 172.21.15.78:0/1813409716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a0010beb0 0x7f1a0019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.763+0000 7f19f7fff640 1 -- 172.21.15.78:0/1813409716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a00102ad0 msgr2=0x7f1a0019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:36.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.763+0000 7f19f7fff640 1 --2- 172.21.15.78:0/1813409716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a00102ad0 0x7f1a0019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.763+0000 7f19f7fff640 1 -- 172.21.15.78:0/1813409716 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0019f300 con 0x7f1a001033f0 2024-08-18T23:45:36.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.763+0000 7f19f7fff640 1 --2- 172.21.15.78:0/1813409716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a001033f0 0x7f1a0019a890 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f19f0006010 tx=0x7f19f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:36.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.763+0000 7f19f5ffb640 1 -- 172.21.15.78:0/1813409716 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f003f030 con 0x7f1a001033f0 2024-08-18T23:45:36.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.763+0000 7f19f5ffb640 1 -- 172.21.15.78:0/1813409716 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f0003ce0 con 0x7f1a001033f0 2024-08-18T23:45:36.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.763+0000 7f1a06dd6640 1 -- 172.21.15.78:0/1813409716 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a0019f590 con 0x7f1a001033f0 2024-08-18T23:45:36.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.763+0000 7f19f5ffb640 1 -- 172.21.15.78:0/1813409716 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0035070 con 0x7f1a001033f0 2024-08-18T23:45:36.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.764+0000 7f1a06dd6640 1 -- 172.21.15.78:0/1813409716 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a001a6ed0 con 0x7f1a001033f0 2024-08-18T23:45:36.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.766+0000 7f19f5ffb640 1 -- 172.21.15.78:0/1813409716 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f19f0042030 con 0x7f1a001033f0 2024-08-18T23:45:36.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.767+0000 7f1a091d4740 1 Processor -- start 2024-08-18T23:45:36.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.767+0000 7f19f5ffb640 1 --2- 172.21.15.78:0/1813409716 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f19e80a2b80 0x7f19e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:36.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.767+0000 7f1a091d4740 1 -- start start 2024-08-18T23:45:36.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.767+0000 7f19f5ffb640 1 -- 172.21.15.78:0/1813409716 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f19f00ef900 con 0x7f1a001033f0 2024-08-18T23:45:36.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.767+0000 7f1a091d4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564397346f10 0x5643973472f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:36.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.767+0000 7f1a091d4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397347830 0x56439735a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:36.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.767+0000 7f1a05373640 1 --2- 172.21.15.78:0/1813409716 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f19e80a2b80 0x7f19e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:36.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.767+0000 7f1a091d4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56439735a9e0 0x56439735cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:36.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.767+0000 7f1a091d4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5643972aca40 con 0x564397347830 2024-08-18T23:45:36.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.767+0000 7f1a091d4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5643972449d0 con 0x56439735a9e0 2024-08-18T23:45:36.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.767+0000 7f1a091d4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5643972a8a00 con 0x564397346f10 2024-08-18T23:45:36.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.767+0000 7f19f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397347830 0x56439735a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:36.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.767+0000 7f19f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397347830 0x56439735a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55124/0 (socket says 172.21.15.78:55124) 2024-08-18T23:45:36.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.767+0000 7f19f7fff640 1 -- 172.21.15.78:0/3165015430 learned_addr learned my addr 172.21.15.78:0/3165015430 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:36.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.767+0000 7f1a05373640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56439735a9e0 0x56439735cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:36.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.768+0000 7f19f7fff640 1 -- 172.21.15.78:0/3165015430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564397346f10 msgr2=0x5643973472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:36.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.768+0000 7f19f7fff640 1 --2- 172.21.15.78:0/3165015430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564397346f10 0x5643973472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.768+0000 7f19f7fff640 1 -- 172.21.15.78:0/3165015430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56439735a9e0 msgr2=0x56439735cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:36.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.768+0000 7f19f7fff640 1 --2- 172.21.15.78:0/3165015430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56439735a9e0 0x56439735cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.768+0000 7f19f7fff640 1 -- 172.21.15.78:0/3165015430 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56439724b650 con 0x564397347830 2024-08-18T23:45:36.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.768+0000 7f1a05373640 1 --2- 172.21.15.78:0/3165015430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56439735a9e0 0x56439735cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:45:36.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.768+0000 7f1a05373640 1 --2- 172.21.15.78:0/1813409716 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f19e80a2b80 0x7f19e80a5040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f19f8005fd0 tx=0x7f19f8009e60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:36.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.768+0000 7f19f7fff640 1 --2- 172.21.15.78:0/3165015430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397347830 0x56439735a4a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f19f00c45d0 tx=0x7f19f0031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42b4bb4dc449962 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:36.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.768+0000 7f19e6ffd640 1 -- 172.21.15.78:0/3165015430 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f00d9070 con 0x564397347830 2024-08-18T23:45:36.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.768+0000 7f19e6ffd640 1 -- 172.21.15.78:0/3165015430 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f00bbf40 con 0x564397347830 2024-08-18T23:45:36.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.768+0000 7f19e6ffd640 1 -- 172.21.15.78:0/3165015430 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0049040 con 0x564397347830 2024-08-18T23:45:36.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.769+0000 7f1a091d4740 1 -- 172.21.15.78:0/3165015430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397347830 msgr2=0x56439735a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:36.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.769+0000 7f1a091d4740 1 --2- 172.21.15.78:0/3165015430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397347830 0x56439735a4a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f19f00c45d0 tx=0x7f19f0031a60 comp rx=0 tx=0).stop 2024-08-18T23:45:36.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.769+0000 7f1a091d4740 1 -- 172.21.15.78:0/3165015430 shutdown_connections 2024-08-18T23:45:36.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.769+0000 7f1a091d4740 1 --2- 172.21.15.78:0/3165015430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56439735a9e0 0x56439735cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.769+0000 7f1a091d4740 1 --2- 172.21.15.78:0/3165015430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397347830 0x56439735a4a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.769+0000 7f1a091d4740 1 --2- 172.21.15.78:0/3165015430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564397346f10 0x5643973472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.769+0000 7f1a091d4740 1 -- 172.21.15.78:0/3165015430 >> 172.21.15.78:0/3165015430 conn(0x564397343580 msgr2=0x564397343960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:36.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.769+0000 7f1a091d4740 1 -- 172.21.15.78:0/3165015430 shutdown_connections 2024-08-18T23:45:36.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.769+0000 7f1a091d4740 1 -- 172.21.15.78:0/3165015430 wait complete. 2024-08-18T23:45:36.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.770+0000 7f1a091d4740 1 -- start start 2024-08-18T23:45:36.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.770+0000 7f1a091d4740 1 Processor -- start 2024-08-18T23:45:36.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.770+0000 7f1a091d4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564397346f10 0x564397357a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:36.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.771+0000 7f1a091d4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397347830 0x564397357f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:36.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.771+0000 7f1a091d4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56439735a9e0 0x564397358490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:36.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.771+0000 7f1a091d4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56439735d310 con 0x564397347830 2024-08-18T23:45:36.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.771+0000 7f19f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397347830 0x564397357f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:36.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.771+0000 7f1a091d4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564397267b20 con 0x56439735a9e0 2024-08-18T23:45:36.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.771+0000 7f1a091d4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56439735d490 con 0x564397346f10 2024-08-18T23:45:36.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.771+0000 7f1a04b72640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564397346f10 0x564397357a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:36.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.771+0000 7f19f7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397347830 0x564397357f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55128/0 (socket says 172.21.15.78:55128) 2024-08-18T23:45:36.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.771+0000 7f19f7fff640 1 -- 172.21.15.78:0/3061759980 learned_addr learned my addr 172.21.15.78:0/3061759980 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:36.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.771+0000 7f1a091d4740 10 client.0 _ll_register_callbacks cb 0x564397265bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:45:36.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.771+0000 7f1a05373640 1 --2- 172.21.15.78:0/3061759980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56439735a9e0 0x564397358490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:36.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.771+0000 7f1a04b72640 1 -- 172.21.15.78:0/3061759980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56439735a9e0 msgr2=0x564397358490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:36.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.771+0000 7f1a04b72640 1 --2- 172.21.15.78:0/3061759980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56439735a9e0 0x564397358490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.771+0000 7f1a04b72640 1 -- 172.21.15.78:0/3061759980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397347830 msgr2=0x564397357f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:36.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.771+0000 7f1a04b72640 1 --2- 172.21.15.78:0/3061759980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397347830 0x564397357f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.771+0000 7f1a04b72640 1 -- 172.21.15.78:0/3061759980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56439701e3c0 con 0x564397346f10 2024-08-18T23:45:36.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.771+0000 7f1a04b72640 1 --2- 172.21.15.78:0/3061759980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564397346f10 0x564397357a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0116c0 tx=0x7f19fc011b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fae4d9eb21ae5411 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:36.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.771+0000 7f19dffff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:45:36.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.772+0000 7f19dffff640 1 -- 172.21.15.78:0/3061759980 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc00fcb0 con 0x564397346f10 2024-08-18T23:45:36.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.772+0000 7f1a091d4740 1 -- 172.21.15.78:0/3061759980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f00bbf40 con 0x564397346f10 2024-08-18T23:45:36.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.772+0000 7f19dffff640 1 -- 172.21.15.78:0/3061759980 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f19fc005020 con 0x564397346f10 2024-08-18T23:45:36.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.772+0000 7f19dffff640 1 -- 172.21.15.78:0/3061759980 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f19fc024050 con 0x564397346f10 2024-08-18T23:45:36.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.772+0000 7f1a091d4740 10 client.25063 fetch_fsmap learned FSMap version 26 2024-08-18T23:45:36.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.772+0000 7f1a091d4740 1 -- 172.21.15.78:0/3061759980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564397369050 con 0x564397346f10 2024-08-18T23:45:36.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.772+0000 7f19dffff640 1 -- 172.21.15.78:0/3061759980 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f00bbf40 con 0x564397346f10 2024-08-18T23:45:36.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.772+0000 7f19dffff640 1 -- 172.21.15.78:0/3061759980 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f19fc019430 con 0x564397346f10 2024-08-18T23:45:36.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.772+0000 7f1a091d4740 10 client.25063 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:45:36.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.772+0000 7f1a091d4740 10 client.25063 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:45:36.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.772+0000 7f1a091d4740 20 client.25063 populate_metadata read hostname 'smithi078' 2024-08-18T23:45:36.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.772+0000 7f1a091d4740 1 --2- 172.21.15.78:0/3061759980 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56439736a4b0 0x56439736c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:36.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.772+0000 7f1a091d4740 4 client.25063 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:45:36.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.772+0000 7f1a091d4740 1 -- 172.21.15.78:0/3061759980 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643972b9fb0 con 0x56439736a4b0 2024-08-18T23:45:36.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.773+0000 7f19f7fff640 1 --2- 172.21.15.78:0/3061759980 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56439736a4b0 0x56439736c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:36.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.773+0000 7f19f7fff640 1 --2- 172.21.15.78:0/3061759980 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56439736a4b0 0x56439736c940 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bbba432e943b4657 server_cookie=1ea935e9738386f in_seq=0 out_seq=0 2024-08-18T23:45:36.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.773+0000 7f19dffff640 10 client.25063 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:45:36.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.776+0000 7f19dffff640 1 -- 172.21.15.78:0/3061759980 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5643972b9fb0 con 0x56439736a4b0 2024-08-18T23:45:36.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.776+0000 7f19dffff640 10 client.25063 handle_command_reply: tid=0 2024-08-18T23:45:36.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.776+0000 7f1a091d4740 1 client.25063 shutdown 2024-08-18T23:45:36.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.776+0000 7f1a091d4740 1 -- 172.21.15.78:0/3061759980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564397346f10 msgr2=0x564397357a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:36.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.776+0000 7f1a091d4740 1 --2- 172.21.15.78:0/3061759980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564397346f10 0x564397357a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0116c0 tx=0x7f19fc011b80 comp rx=0 tx=0).stop 2024-08-18T23:45:36.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.776+0000 7f1a091d4740 1 -- 172.21.15.78:0/3061759980 shutdown_connections 2024-08-18T23:45:36.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.776+0000 7f1a091d4740 1 --2- 172.21.15.78:0/3061759980 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56439736a4b0 0x56439736c940 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.776+0000 7f1a091d4740 1 --2- 172.21.15.78:0/3061759980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56439735a9e0 0x564397358490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.776+0000 7f1a091d4740 1 --2- 172.21.15.78:0/3061759980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397347830 0x564397357f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.776+0000 7f19dffff640 0 client.25063 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:45:36.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.776+0000 7f1a091d4740 1 --2- 172.21.15.78:0/3061759980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564397346f10 0x564397357a10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.776+0000 7f1a091d4740 1 -- 172.21.15.78:0/3061759980 >> 172.21.15.78:0/3061759980 conn(0x564397343580 msgr2=0x564397349210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:36.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.776+0000 7f1a091d4740 1 -- 172.21.15.78:0/3061759980 shutdown_connections 2024-08-18T23:45:36.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.776+0000 7f1a091d4740 1 -- 172.21.15.78:0/3061759980 wait complete. 2024-08-18T23:45:36.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.776+0000 7f1a091d4740 20 client.25063 trim_cache size 0 max 16384 2024-08-18T23:45:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.790+0000 7f1a091d4740 1 Processor -- start 2024-08-18T23:45:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.790+0000 7f1a091d4740 1 -- start start 2024-08-18T23:45:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.790+0000 7f1a091d4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397346f10 0x564397372210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.790+0000 7f1a091d4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564397347830 0x564397372750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.790+0000 7f1a091d4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56439735a9e0 0x564397348960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.790+0000 7f1a091d4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001af0b0 con 0x564397346f10 2024-08-18T23:45:36.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.790+0000 7f1a091d4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001ac980 con 0x564397347830 2024-08-18T23:45:36.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.790+0000 7f1a091d4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001aa830 con 0x56439735a9e0 2024-08-18T23:45:36.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.790+0000 7f1a04b72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397346f10 0x564397372210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:36.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.790+0000 7f1a04b72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397346f10 0x564397372210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55134/0 (socket says 172.21.15.78:55134) 2024-08-18T23:45:36.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.790+0000 7f19f7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564397347830 0x564397372750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:36.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.790+0000 7f1a04b72640 1 -- 172.21.15.78:0/3994534231 learned_addr learned my addr 172.21.15.78:0/3994534231 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:36.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.790+0000 7f1a05373640 1 --2- 172.21.15.78:0/3994534231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56439735a9e0 0x564397348960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:36.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.790+0000 7f19f7fff640 1 -- 172.21.15.78:0/3994534231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56439735a9e0 msgr2=0x564397348960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:36.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.790+0000 7f19f7fff640 1 --2- 172.21.15.78:0/3994534231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56439735a9e0 0x564397348960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.790+0000 7f19f7fff640 1 -- 172.21.15.78:0/3994534231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397346f10 msgr2=0x564397372210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:36.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.790+0000 7f19f7fff640 1 --2- 172.21.15.78:0/3994534231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397346f10 0x564397372210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.790+0000 7f19f7fff640 1 -- 172.21.15.78:0/3994534231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564397348420 con 0x564397347830 2024-08-18T23:45:36.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.790+0000 7f1a04b72640 1 --2- 172.21.15.78:0/3994534231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397346f10 0x564397372210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:45:36.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.791+0000 7f19f7fff640 1 --2- 172.21.15.78:0/3994534231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564397347830 0x564397372750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f19f0000c00 tx=0x7f19f0005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=170aba24265ad557 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:36.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.791+0000 7f19e67fc640 1 -- 172.21.15.78:0/3994534231 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f00d9070 con 0x564397347830 2024-08-18T23:45:36.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.791+0000 7f19e67fc640 1 -- 172.21.15.78:0/3994534231 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f00c4b00 con 0x564397347830 2024-08-18T23:45:36.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.791+0000 7f19e67fc640 1 -- 172.21.15.78:0/3994534231 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0049040 con 0x564397347830 2024-08-18T23:45:36.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.792+0000 7f1a091d4740 1 -- 172.21.15.78:0/3994534231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564397347830 msgr2=0x564397372750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:36.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.792+0000 7f1a091d4740 1 --2- 172.21.15.78:0/3994534231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564397347830 0x564397372750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f19f0000c00 tx=0x7f19f0005e50 comp rx=0 tx=0).stop 2024-08-18T23:45:36.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.792+0000 7f1a091d4740 1 -- 172.21.15.78:0/3994534231 shutdown_connections 2024-08-18T23:45:36.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.792+0000 7f1a091d4740 1 --2- 172.21.15.78:0/3994534231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56439735a9e0 0x564397348960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.792+0000 7f1a091d4740 1 --2- 172.21.15.78:0/3994534231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564397347830 0x564397372750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.792+0000 7f1a091d4740 1 --2- 172.21.15.78:0/3994534231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397346f10 0x564397372210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.792+0000 7f1a091d4740 1 -- 172.21.15.78:0/3994534231 >> 172.21.15.78:0/3994534231 conn(0x564397343580 msgr2=0x564397352d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:36.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.792+0000 7f1a091d4740 1 -- 172.21.15.78:0/3994534231 shutdown_connections 2024-08-18T23:45:36.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.792+0000 7f1a091d4740 1 -- 172.21.15.78:0/3994534231 wait complete. 2024-08-18T23:45:36.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.792+0000 7f1a091d4740 1 -- start start 2024-08-18T23:45:36.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.792+0000 7f1a091d4740 1 Processor -- start 2024-08-18T23:45:36.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f1a091d4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397346f10 0x564397362b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:36.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f1a091d4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564397347830 0x564397354450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:36.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f1a091d4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56439735a9e0 0x564397354990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:36.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f1a091d4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001aae80 con 0x564397346f10 2024-08-18T23:45:36.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f1a04b72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397346f10 0x564397362b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:36.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f1a04b72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397346f10 0x564397362b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55142/0 (socket says 172.21.15.78:55142) 2024-08-18T23:45:36.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f1a04b72640 1 -- 172.21.15.78:0/2362894754 learned_addr learned my addr 172.21.15.78:0/2362894754 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:36.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f1a091d4740 1 -- 172.21.15.78:0/2362894754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001ac2b0 con 0x56439735a9e0 2024-08-18T23:45:36.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f1a091d4740 1 -- 172.21.15.78:0/2362894754 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001ab510 con 0x564397347830 2024-08-18T23:45:36.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f1a05373640 1 --2- 172.21.15.78:0/2362894754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56439735a9e0 0x564397354990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:36.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f19f7fff640 1 --2- 172.21.15.78:0/2362894754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564397347830 0x564397354450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:36.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f1a05373640 1 -- 172.21.15.78:0/2362894754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564397347830 msgr2=0x564397354450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:36.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f1a05373640 1 --2- 172.21.15.78:0/2362894754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564397347830 0x564397354450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f1a05373640 1 -- 172.21.15.78:0/2362894754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397346f10 msgr2=0x564397362b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:36.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f1a05373640 1 --2- 172.21.15.78:0/2362894754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397346f10 0x564397362b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f1a05373640 1 -- 172.21.15.78:0/2362894754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564397354ed0 con 0x56439735a9e0 2024-08-18T23:45:36.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f1a091d4740 10 client.0 _ll_register_callbacks cb 0x7f19fc009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:45:36.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f1a091d4740 1 -- 172.21.15.78:0/2362894754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f00c4b00 con 0x56439735a9e0 2024-08-18T23:45:36.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f1a05373640 1 --2- 172.21.15.78:0/2362894754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56439735a9e0 0x564397354990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x564397346e90 tx=0x7f19f8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ff1886eb4a3540a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:36.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.793+0000 7f19dffff640 10 client.25151 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:45:36.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.794+0000 7f19dffff640 1 -- 172.21.15.78:0/2362894754 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f8004710 con 0x56439735a9e0 2024-08-18T23:45:36.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.794+0000 7f19dffff640 1 -- 172.21.15.78:0/2362894754 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f19f8022440 con 0x56439735a9e0 2024-08-18T23:45:36.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.794+0000 7f19dffff640 1 -- 172.21.15.78:0/2362894754 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f19f80226a0 con 0x56439735a9e0 2024-08-18T23:45:36.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.794+0000 7f1a091d4740 10 client.25151 fetch_fsmap learned FSMap version 26 2024-08-18T23:45:36.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.794+0000 7f1a091d4740 1 -- 172.21.15.78:0/2362894754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5643973d4de0 con 0x56439735a9e0 2024-08-18T23:45:36.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.794+0000 7f19dffff640 1 -- 172.21.15.78:0/2362894754 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f19f800f040 con 0x56439735a9e0 2024-08-18T23:45:36.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.794+0000 7f1a091d4740 10 client.25151 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:45:36.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.794+0000 7f1a091d4740 10 client.25151 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:45:36.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.794+0000 7f1a091d4740 20 client.25151 populate_metadata read hostname 'smithi078' 2024-08-18T23:45:36.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.794+0000 7f1a091d4740 1 --2- 172.21.15.78:0/2362894754 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x5643973d5420 0x5643973d7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:36.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.794+0000 7f1a091d4740 4 client.25151 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:45:36.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.794+0000 7f1a091d4740 1 -- 172.21.15.78:0/2362894754 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5643972b44c0 con 0x5643973d5420 2024-08-18T23:45:36.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.794+0000 7f1a04b72640 1 --2- 172.21.15.78:0/2362894754 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x5643973d5420 0x5643973d7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:36.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.795+0000 7f19dffff640 10 client.25151 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:45:36.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.795+0000 7f1a04b72640 1 --2- 172.21.15.78:0/2362894754 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x5643973d5420 0x5643973d7840 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e19b697fbe0491e server_cookie=75366dffbc078c7a in_seq=0 out_seq=0 2024-08-18T23:45:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.798+0000 7f19dffff640 1 -- 172.21.15.78:0/2362894754 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5643972b44c0 con 0x5643973d5420 2024-08-18T23:45:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.798+0000 7f19dffff640 10 client.25151 handle_command_reply: tid=0 2024-08-18T23:45:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.798+0000 7f1a091d4740 1 client.25151 shutdown 2024-08-18T23:45:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.798+0000 7f1a091d4740 1 -- 172.21.15.78:0/2362894754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56439735a9e0 msgr2=0x564397354990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.798+0000 7f1a091d4740 1 --2- 172.21.15.78:0/2362894754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56439735a9e0 0x564397354990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x564397346e90 tx=0x7f19f8004ab0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.798+0000 7f1a091d4740 1 -- 172.21.15.78:0/2362894754 shutdown_connections 2024-08-18T23:45:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.798+0000 7f1a091d4740 1 --2- 172.21.15.78:0/2362894754 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x5643973d5420 0x5643973d7840 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.798+0000 7f1a091d4740 1 --2- 172.21.15.78:0/2362894754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56439735a9e0 0x564397354990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.798+0000 7f1a091d4740 1 --2- 172.21.15.78:0/2362894754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564397347830 0x564397354450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.798+0000 7f1a091d4740 1 --2- 172.21.15.78:0/2362894754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564397346f10 0x564397362b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.798+0000 7f19dffff640 0 client.25151 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:45:36.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.798+0000 7f1a091d4740 1 -- 172.21.15.78:0/2362894754 >> 172.21.15.78:0/2362894754 conn(0x564397343580 msgr2=0x564397365560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:36.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.798+0000 7f1a091d4740 1 -- 172.21.15.78:0/2362894754 shutdown_connections 2024-08-18T23:45:36.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.798+0000 7f1a091d4740 1 -- 172.21.15.78:0/2362894754 wait complete. 2024-08-18T23:45:36.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.798+0000 7f1a091d4740 20 client.25151 trim_cache size 0 max 16384 2024-08-18T23:45:36.805 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:45:36.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.799+0000 7f1a06dd6640 1 -- 172.21.15.78:0/1813409716 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f19e80a2b80 msgr2=0x7f19e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:36.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.799+0000 7f1a06dd6640 1 --2- 172.21.15.78:0/1813409716 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f19e80a2b80 0x7f19e80a5040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f19f8005fd0 tx=0x7f19f8009e60 comp rx=0 tx=0).stop 2024-08-18T23:45:36.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.799+0000 7f1a06dd6640 1 -- 172.21.15.78:0/1813409716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a001033f0 msgr2=0x7f1a0019a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:36.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.799+0000 7f1a06dd6640 1 --2- 172.21.15.78:0/1813409716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a001033f0 0x7f1a0019a890 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f19f0006010 tx=0x7f19f00042f0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.799+0000 7f1a06dd6640 1 -- 172.21.15.78:0/1813409716 shutdown_connections 2024-08-18T23:45:36.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.799+0000 7f1a06dd6640 1 --2- 172.21.15.78:0/1813409716 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f19e80a2b80 0x7f19e80a5040 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.799+0000 7f1a06dd6640 1 --2- 172.21.15.78:0/1813409716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a0010beb0 0x7f1a0019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.799+0000 7f1a06dd6640 1 --2- 172.21.15.78:0/1813409716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1a001033f0 0x7f1a0019a890 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.799+0000 7f1a06dd6640 1 --2- 172.21.15.78:0/1813409716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a00102ad0 0x7f1a0019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:36.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.799+0000 7f1a06dd6640 1 -- 172.21.15.78:0/1813409716 >> 172.21.15.78:0/1813409716 conn(0x7f1a000fc4d0 msgr2=0x7f1a00100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:36.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.799+0000 7f1a06dd6640 1 -- 172.21.15.78:0/1813409716 shutdown_connections 2024-08-18T23:45:36.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:36.799+0000 7f1a06dd6640 1 -- 172.21.15.78:0/1813409716 wait complete. 2024-08-18T23:45:36.981 INFO:tasks.workunit.client.0.smithi078.stdout: 16 259526 958 95855 708 26089 1152 2024-08-18T23:45:37.487 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:37 smithi078 ceph-mon[18440]: pgmap v389: 129 pgs: 129 active+clean; 33 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 157 op/s 2024-08-18T23:45:37.516 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-18T23:45:37.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:37 smithi123 ceph-mon[21808]: pgmap v389: 129 pgs: 129 active+clean; 33 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 157 op/s 2024-08-18T23:45:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:37 smithi161 ceph-mon[22027]: pgmap v389: 129 pgs: 129 active+clean; 33 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 157 op/s 2024-08-18T23:45:37.782 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:45:38.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.324+0000 7fcc8f327640 1 -- 172.21.15.78:0/2105694515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc881076f0 msgr2=0x7fcc88107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:38.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.324+0000 7fcc8f327640 1 --2- 172.21.15.78:0/2105694515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc881076f0 0x7fcc88107ad0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009cd0 tx=0x7fcc8002a910 comp rx=0 tx=0).stop 2024-08-18T23:45:38.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.338+0000 7fcc8f327640 1 -- 172.21.15.78:0/2105694515 shutdown_connections 2024-08-18T23:45:38.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.338+0000 7fcc8f327640 1 --2- 172.21.15.78:0/2105694515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc88102090 0x7fcc8810e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.338+0000 7fcc8f327640 1 --2- 172.21.15.78:0/2105694515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc881016f0 0x7fcc88101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.338+0000 7fcc8f327640 1 --2- 172.21.15.78:0/2105694515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc881076f0 0x7fcc88107ad0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.338+0000 7fcc8f327640 1 -- 172.21.15.78:0/2105694515 >> 172.21.15.78:0/2105694515 conn(0x7fcc880fd470 msgr2=0x7fcc880ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:38.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.338+0000 7fcc8f327640 1 -- 172.21.15.78:0/2105694515 shutdown_connections 2024-08-18T23:45:38.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.338+0000 7fcc8f327640 1 -- 172.21.15.78:0/2105694515 wait complete. 2024-08-18T23:45:38.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.339+0000 7fcc8f327640 1 Processor -- start 2024-08-18T23:45:38.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.342+0000 7fcc8f327640 1 -- start start 2024-08-18T23:45:38.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.342+0000 7fcc8f327640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc881016f0 0x7fcc88074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:38.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.342+0000 7fcc8f327640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc88102090 0x7fcc88074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:38.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.342+0000 7fcc8f327640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc881076f0 0x7fcc88074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:38.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.342+0000 7fcc8f327640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc88111c00 con 0x7fcc881016f0 2024-08-18T23:45:38.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.343+0000 7fcc8f327640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcc88111a80 con 0x7fcc88102090 2024-08-18T23:45:38.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.343+0000 7fcc8f327640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcc88111d80 con 0x7fcc881076f0 2024-08-18T23:45:38.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.344+0000 7fcc8db24640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc88102090 0x7fcc88074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:38.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.344+0000 7fcc8db24640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc88102090 0x7fcc88074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:33054/0 (socket says 172.21.15.78:33054) 2024-08-18T23:45:38.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.344+0000 7fcc8db24640 1 -- 172.21.15.78:0/2017944392 learned_addr learned my addr 172.21.15.78:0/2017944392 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:38.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.345+0000 7fcc8e325640 1 --2- 172.21.15.78:0/2017944392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc881016f0 0x7fcc88074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:38.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.345+0000 7fcc8db24640 1 -- 172.21.15.78:0/2017944392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc881076f0 msgr2=0x7fcc88074ca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-18T23:45:38.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.345+0000 7fcc8eb26640 1 --2- 172.21.15.78:0/2017944392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc881076f0 0x7fcc88074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:38.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.345+0000 7fcc8db24640 1 --2- 172.21.15.78:0/2017944392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc881076f0 0x7fcc88074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.345+0000 7fcc8db24640 1 -- 172.21.15.78:0/2017944392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc881016f0 msgr2=0x7fcc88074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:38.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.345+0000 7fcc8db24640 1 --2- 172.21.15.78:0/2017944392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc881016f0 0x7fcc88074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.345+0000 7fcc8db24640 1 -- 172.21.15.78:0/2017944392 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8819c2f0 con 0x7fcc88102090 2024-08-18T23:45:38.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.346+0000 7fcc8e325640 1 --2- 172.21.15.78:0/2017944392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc881016f0 0x7fcc88074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:45:38.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.346+0000 7fcc8db24640 1 --2- 172.21.15.78:0/2017944392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc88102090 0x7fcc88074760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400b450 tx=0x7fcc8400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:38.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.346+0000 7fcc8eb26640 1 --2- 172.21.15.78:0/2017944392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc881076f0 0x7fcc88074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:45:38.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.347+0000 7fcc7f7fe640 1 -- 172.21.15.78:0/2017944392 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc84015020 con 0x7fcc88102090 2024-08-18T23:45:38.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.347+0000 7fcc8f327640 1 -- 172.21.15.78:0/2017944392 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc8819c5e0 con 0x7fcc88102090 2024-08-18T23:45:38.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.347+0000 7fcc8f327640 1 -- 172.21.15.78:0/2017944392 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc8819cb20 con 0x7fcc88102090 2024-08-18T23:45:38.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.347+0000 7fcc7f7fe640 1 -- 172.21.15.78:0/2017944392 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc84011d70 con 0x7fcc88102090 2024-08-18T23:45:38.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.348+0000 7fcc7f7fe640 1 -- 172.21.15.78:0/2017944392 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8400f340 con 0x7fcc88102090 2024-08-18T23:45:38.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.353+0000 7fcc7f7fe640 1 -- 172.21.15.78:0/2017944392 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fcc84018020 con 0x7fcc88102090 2024-08-18T23:45:38.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.354+0000 7fcc92987740 1 Processor -- start 2024-08-18T23:45:38.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.354+0000 7fcc92987740 1 -- start start 2024-08-18T23:45:38.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.354+0000 7fcc92987740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e8f1d0 0x55ee96e8f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:38.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.354+0000 7fcc92987740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e8faf0 0x55ee96e99e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:38.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.354+0000 7fcc92987740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96e9a3b0 0x55ee96e9c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:38.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.354+0000 7fcc92987740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ee96df4a40 con 0x55ee96e9a3b0 2024-08-18T23:45:38.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.354+0000 7fcc92987740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ee96d8c9d0 con 0x55ee96e8f1d0 2024-08-18T23:45:38.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.354+0000 7fcc92987740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ee96df0a00 con 0x55ee96e8faf0 2024-08-18T23:45:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.355+0000 7fcc8db24640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96e9a3b0 0x55ee96e9c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.355+0000 7fcc8db24640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96e9a3b0 0x55ee96e9c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55166/0 (socket says 172.21.15.78:55166) 2024-08-18T23:45:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.355+0000 7fcc8db24640 1 -- 172.21.15.78:0/632226946 learned_addr learned my addr 172.21.15.78:0/632226946 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.355+0000 7fcc8eb26640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e8f1d0 0x55ee96e8f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.355+0000 7fcc8e325640 1 --2- 172.21.15.78:0/632226946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e8faf0 0x55ee96e99e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.357+0000 7fcc8eb26640 1 -- 172.21.15.78:0/632226946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e8faf0 msgr2=0x55ee96e99e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.357+0000 7fcc8eb26640 1 --2- 172.21.15.78:0/632226946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e8faf0 0x55ee96e99e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.357+0000 7fcc8eb26640 1 -- 172.21.15.78:0/632226946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96e9a3b0 msgr2=0x55ee96e9c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.357+0000 7fcc8eb26640 1 --2- 172.21.15.78:0/632226946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96e9a3b0 0x55ee96e9c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.357+0000 7fcc8eb26640 1 -- 172.21.15.78:0/632226946 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee96d93650 con 0x55ee96e8f1d0 2024-08-18T23:45:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.357+0000 7fcc8eb26640 1 --2- 172.21.15.78:0/632226946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e8f1d0 0x55ee96e8f5b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800a8f0 tx=0x7fcc7800adb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5ebe7eccde2194e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.357+0000 7fcc8e325640 1 --2- 172.21.15.78:0/632226946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e8faf0 0x55ee96e99e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:45:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.357+0000 7fcc7f7fe640 1 --2- 172.21.15.78:0/2017944392 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcc740a2c20 0x7fcc740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.358+0000 7fcc7f7fe640 1 -- 172.21.15.78:0/2017944392 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fcc840ceed0 con 0x7fcc88102090 2024-08-18T23:45:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.358+0000 7fcc7cff9640 1 -- 172.21.15.78:0/632226946 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc78015070 con 0x55ee96e8f1d0 2024-08-18T23:45:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.358+0000 7fcc7cff9640 1 -- 172.21.15.78:0/632226946 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc7800cb40 con 0x55ee96e8f1d0 2024-08-18T23:45:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.360+0000 7fcc92987740 1 -- 172.21.15.78:0/632226946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e8f1d0 msgr2=0x55ee96e8f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.360+0000 7fcc92987740 1 --2- 172.21.15.78:0/632226946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e8f1d0 0x55ee96e8f5b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800a8f0 tx=0x7fcc7800adb0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.360+0000 7fcc8e325640 1 --2- 172.21.15.78:0/2017944392 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcc740a2c20 0x7fcc740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.361+0000 7fcc8e325640 1 --2- 172.21.15.78:0/2017944392 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcc740a2c20 0x7fcc740a50e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009430 tx=0x7fcc80009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.364+0000 7fcc92987740 1 -- 172.21.15.78:0/632226946 shutdown_connections 2024-08-18T23:45:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.364+0000 7fcc92987740 1 --2- 172.21.15.78:0/632226946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96e9a3b0 0x55ee96e9c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.364+0000 7fcc92987740 1 --2- 172.21.15.78:0/632226946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e8faf0 0x55ee96e99e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.364+0000 7fcc92987740 1 --2- 172.21.15.78:0/632226946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e8f1d0 0x55ee96e8f5b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.364+0000 7fcc92987740 1 -- 172.21.15.78:0/632226946 >> 172.21.15.78:0/632226946 conn(0x55ee96e8b760 msgr2=0x55ee96e8bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:38.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.370+0000 7fcc92987740 1 -- 172.21.15.78:0/632226946 shutdown_connections 2024-08-18T23:45:38.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.370+0000 7fcc92987740 1 -- 172.21.15.78:0/632226946 wait complete. 2024-08-18T23:45:38.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.370+0000 7fcc92987740 1 -- start start 2024-08-18T23:45:38.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.370+0000 7fcc92987740 1 Processor -- start 2024-08-18T23:45:38.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.371+0000 7fcc92987740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e8f1d0 0x55ee96e971e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:38.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.371+0000 7fcc92987740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e8faf0 0x55ee96e97720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:38.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.371+0000 7fcc92987740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96e9a3b0 0x55ee96e97c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:38.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.371+0000 7fcc92987740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ee96eadfe0 con 0x55ee96e9a3b0 2024-08-18T23:45:38.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.371+0000 7fcc92987740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ee96dafb20 con 0x55ee96e8faf0 2024-08-18T23:45:38.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.371+0000 7fcc92987740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ee96eae160 con 0x55ee96e8f1d0 2024-08-18T23:45:38.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.371+0000 7fcc92987740 10 client.0 _ll_register_callbacks cb 0x55ee96dadbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:45:38.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.373+0000 7fcc8db24640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e8faf0 0x55ee96e97720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:38.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.373+0000 7fcc8db24640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e8faf0 0x55ee96e97720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:33080/0 (socket says 172.21.15.78:33080) 2024-08-18T23:45:38.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.373+0000 7fcc8db24640 1 -- 172.21.15.78:0/1549412097 learned_addr learned my addr 172.21.15.78:0/1549412097 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:38.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.373+0000 7fcc8eb26640 1 --2- 172.21.15.78:0/1549412097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96e9a3b0 0x55ee96e97c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:38.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.374+0000 7fcc8db24640 1 -- 172.21.15.78:0/1549412097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e8f1d0 msgr2=0x55ee96e971e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T23:45:38.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.374+0000 7fcc8db24640 1 --2- 172.21.15.78:0/1549412097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e8f1d0 0x55ee96e971e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.374+0000 7fcc8db24640 1 -- 172.21.15.78:0/1549412097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96e9a3b0 msgr2=0x55ee96e97c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:38.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.374+0000 7fcc8db24640 1 --2- 172.21.15.78:0/1549412097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96e9a3b0 0x55ee96e97c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.374+0000 7fcc8db24640 1 -- 172.21.15.78:0/1549412097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee96b663c0 con 0x55ee96e8faf0 2024-08-18T23:45:38.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.375+0000 7fcc8db24640 1 --2- 172.21.15.78:0/1549412097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e8faf0 0x55ee96e97720 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009fd0 tx=0x7fcc8409b1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df285c72290123 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:38.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.375+0000 7fcc71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:45:38.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.375+0000 7fcc71ffb640 1 -- 172.21.15.78:0/1549412097 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8409a990 con 0x55ee96e8faf0 2024-08-18T23:45:38.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.376+0000 7fcc92987740 1 -- 172.21.15.78:0/1549412097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc7800cb40 con 0x55ee96e8faf0 2024-08-18T23:45:38.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.376+0000 7fcc71ffb640 1 -- 172.21.15.78:0/1549412097 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fcc84011890 con 0x55ee96e8faf0 2024-08-18T23:45:38.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.376+0000 7fcc71ffb640 1 -- 172.21.15.78:0/1549412097 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcc8409b340 con 0x55ee96e8faf0 2024-08-18T23:45:38.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.378+0000 7fcc92987740 10 client.25160 fetch_fsmap learned FSMap version 26 2024-08-18T23:45:38.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.378+0000 7fcc92987740 1 -- 172.21.15.78:0/1549412097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ee96eb9c40 con 0x55ee96e8faf0 2024-08-18T23:45:38.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.378+0000 7fcc71ffb640 1 -- 172.21.15.78:0/1549412097 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc7800cb40 con 0x55ee96e8faf0 2024-08-18T23:45:38.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.378+0000 7fcc71ffb640 1 -- 172.21.15.78:0/1549412097 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fcc840b40f0 con 0x55ee96e8faf0 2024-08-18T23:45:38.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.378+0000 7fcc92987740 10 client.25160 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:45:38.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.378+0000 7fcc92987740 10 client.25160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:45:38.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.378+0000 7fcc92987740 20 client.25160 populate_metadata read hostname 'smithi078' 2024-08-18T23:45:38.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.378+0000 7fcc92987740 1 --2- 172.21.15.78:0/1549412097 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ee96ebb5a0 0x55ee96ebda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:38.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.378+0000 7fcc92987740 4 client.25160 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:45:38.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.378+0000 7fcc92987740 1 -- 172.21.15.78:0/1549412097 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee96e01fb0 con 0x55ee96ebb5a0 2024-08-18T23:45:38.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.379+0000 7fcc8e325640 1 --2- 172.21.15.78:0/1549412097 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ee96ebb5a0 0x55ee96ebda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:38.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.379+0000 7fcc71ffb640 10 client.25160 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:38.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.380+0000 7fcc8e325640 1 --2- 172.21.15.78:0/1549412097 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ee96ebb5a0 0x55ee96ebda40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c14a4ffee6b0a1 server_cookie=72ebce6fe2d5a8 in_seq=0 out_seq=0 2024-08-18T23:45:38.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.384+0000 7fcc71ffb640 1 -- 172.21.15.78:0/1549412097 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee96e01fb0 con 0x55ee96ebb5a0 2024-08-18T23:45:38.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.384+0000 7fcc71ffb640 10 client.25160 handle_command_reply: tid=0 2024-08-18T23:45:38.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.384+0000 7fcc92987740 1 client.25160 shutdown 2024-08-18T23:45:38.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.386+0000 7fcc92987740 1 -- 172.21.15.78:0/1549412097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e8faf0 msgr2=0x55ee96e97720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:38.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.386+0000 7fcc92987740 1 --2- 172.21.15.78:0/1549412097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e8faf0 0x55ee96e97720 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009fd0 tx=0x7fcc8409b1d0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.386+0000 7fcc92987740 1 -- 172.21.15.78:0/1549412097 shutdown_connections 2024-08-18T23:45:38.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.386+0000 7fcc92987740 1 --2- 172.21.15.78:0/1549412097 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ee96ebb5a0 0x55ee96ebda40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.386+0000 7fcc92987740 1 --2- 172.21.15.78:0/1549412097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96e9a3b0 0x55ee96e97c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.386+0000 7fcc92987740 1 --2- 172.21.15.78:0/1549412097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e8faf0 0x55ee96e97720 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009fd0 tx=0x7fcc8409b1d0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.386+0000 7fcc92987740 1 --2- 172.21.15.78:0/1549412097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e8f1d0 0x55ee96e971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.386+0000 7fcc92987740 1 -- 172.21.15.78:0/1549412097 >> 172.21.15.78:0/1549412097 conn(0x55ee96e8b760 msgr2=0x55ee96e8a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:38.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.386+0000 7fcc71ffb640 0 client.25160 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:38.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.387+0000 7fcc92987740 1 -- 172.21.15.78:0/1549412097 shutdown_connections 2024-08-18T23:45:38.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.388+0000 7fcc92987740 1 -- 172.21.15.78:0/1549412097 wait complete. 2024-08-18T23:45:38.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.388+0000 7fcc92987740 20 client.25160 trim_cache size 0 max 16384 2024-08-18T23:45:38.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.420+0000 7fcc92987740 1 Processor -- start 2024-08-18T23:45:38.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.420+0000 7fcc92987740 1 -- start start 2024-08-18T23:45:38.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.421+0000 7fcc92987740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e8f1d0 0x55ee96e97ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:38.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.421+0000 7fcc92987740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e9a3b0 0x55ee96ec3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:38.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.421+0000 7fcc92987740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96ec3890 0x55ee96e8a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:38.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.421+0000 7fcc8db24640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e9a3b0 0x55ee96ec3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:38.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.421+0000 7fcc8db24640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e9a3b0 0x55ee96ec3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:33094/0 (socket says 172.21.15.78:33094) 2024-08-18T23:45:38.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.421+0000 7fcc8e325640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e8f1d0 0x55ee96e97ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:38.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.421+0000 7fcc8db24640 1 -- 172.21.15.78:0/500224415 learned_addr learned my addr 172.21.15.78:0/500224415 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.421+0000 7fcc8eb26640 1 --2- 172.21.15.78:0/500224415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96ec3890 0x55ee96e8a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.422+0000 7fcc92987740 1 -- 172.21.15.78:0/500224415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c012520 con 0x55ee96ec3890 2024-08-18T23:45:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.422+0000 7fcc92987740 1 -- 172.21.15.78:0/500224415 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c00fdf0 con 0x55ee96e9a3b0 2024-08-18T23:45:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.422+0000 7fcc92987740 1 -- 172.21.15.78:0/500224415 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c00dc40 con 0x55ee96e8f1d0 2024-08-18T23:45:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.422+0000 7fcc8db24640 1 -- 172.21.15.78:0/500224415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e8f1d0 msgr2=0x55ee96e97ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.422+0000 7fcc8db24640 1 --2- 172.21.15.78:0/500224415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e8f1d0 0x55ee96e97ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.422+0000 7fcc8db24640 1 -- 172.21.15.78:0/500224415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96ec3890 msgr2=0x55ee96e8a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.422+0000 7fcc8db24640 1 --2- 172.21.15.78:0/500224415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96ec3890 0x55ee96e8a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.422+0000 7fcc8db24640 1 -- 172.21.15.78:0/500224415 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee96e8f920 con 0x55ee96e9a3b0 2024-08-18T23:45:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.422+0000 7fcc8db24640 1 --2- 172.21.15.78:0/500224415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e9a3b0 0x55ee96ec3350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcc84010a80 tx=0x7fcc8409bb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7122a0b56c41115 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.422+0000 7fcc73fff640 1 -- 172.21.15.78:0/500224415 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc840b8070 con 0x55ee96e9a3b0 2024-08-18T23:45:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.423+0000 7fcc73fff640 1 -- 172.21.15.78:0/500224415 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc840d2600 con 0x55ee96e9a3b0 2024-08-18T23:45:38.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.423+0000 7fcc73fff640 1 -- 172.21.15.78:0/500224415 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8409a790 con 0x55ee96e9a3b0 2024-08-18T23:45:38.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.423+0000 7fcc92987740 1 -- 172.21.15.78:0/500224415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e9a3b0 msgr2=0x55ee96ec3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:38.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.423+0000 7fcc92987740 1 --2- 172.21.15.78:0/500224415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e9a3b0 0x55ee96ec3350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcc84010a80 tx=0x7fcc8409bb00 comp rx=0 tx=0).stop 2024-08-18T23:45:38.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.423+0000 7fcc92987740 1 -- 172.21.15.78:0/500224415 shutdown_connections 2024-08-18T23:45:38.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.423+0000 7fcc92987740 1 --2- 172.21.15.78:0/500224415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96ec3890 0x55ee96e8a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.423+0000 7fcc92987740 1 --2- 172.21.15.78:0/500224415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e9a3b0 0x55ee96ec3350 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.423+0000 7fcc92987740 1 --2- 172.21.15.78:0/500224415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e8f1d0 0x55ee96e97ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.423+0000 7fcc92987740 1 -- 172.21.15.78:0/500224415 >> 172.21.15.78:0/500224415 conn(0x55ee96e8b760 msgr2=0x55ee96e8bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:38.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.423+0000 7fcc92987740 1 -- 172.21.15.78:0/500224415 shutdown_connections 2024-08-18T23:45:38.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.424+0000 7fcc92987740 1 -- 172.21.15.78:0/500224415 wait complete. 2024-08-18T23:45:38.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.424+0000 7fcc92987740 1 -- start start 2024-08-18T23:45:38.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.424+0000 7fcc92987740 1 Processor -- start 2024-08-18T23:45:38.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.424+0000 7fcc92987740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e8f1d0 0x55ee96eaea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:38.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.424+0000 7fcc92987740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e9a3b0 0x55ee96eaef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:38.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.424+0000 7fcc92987740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96ec3890 0x55ee96f24310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:38.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.424+0000 7fcc8db24640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e9a3b0 0x55ee96eaef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:38.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.424+0000 7fcc8e325640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e8f1d0 0x55ee96eaea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:38.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.424+0000 7fcc8eb26640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96ec3890 0x55ee96f24310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:38.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.424+0000 7fcc8db24640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e9a3b0 0x55ee96eaef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:55110/0 (socket says 172.21.15.78:55110) 2024-08-18T23:45:38.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.424+0000 7fcc8e325640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e8f1d0 0x55ee96eaea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:33108/0 (socket says 172.21.15.78:33108) 2024-08-18T23:45:38.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.424+0000 7fcc8eb26640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96ec3890 0x55ee96f24310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55188/0 (socket says 172.21.15.78:55188) 2024-08-18T23:45:38.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.424+0000 7fcc8db24640 1 -- 172.21.15.78:0/1572527292 learned_addr learned my addr 172.21.15.78:0/1572527292 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:38.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.424+0000 7fcc92987740 1 -- 172.21.15.78:0/1572527292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c00e2b0 con 0x55ee96ec3890 2024-08-18T23:45:38.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.424+0000 7fcc92987740 1 -- 172.21.15.78:0/1572527292 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c00f720 con 0x55ee96e8f1d0 2024-08-18T23:45:38.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.424+0000 7fcc92987740 1 -- 172.21.15.78:0/1572527292 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c00e980 con 0x55ee96e9a3b0 2024-08-18T23:45:38.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.424+0000 7fcc92987740 10 client.0 _ll_register_callbacks cb 0x7fcc840975e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:45:38.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.425+0000 7fcc8db24640 1 -- 172.21.15.78:0/1572527292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e8f1d0 msgr2=0x55ee96eaea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:38.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.425+0000 7fcc8db24640 1 --2- 172.21.15.78:0/1572527292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e8f1d0 0x55ee96eaea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.425+0000 7fcc8db24640 1 -- 172.21.15.78:0/1572527292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96ec3890 msgr2=0x55ee96f24310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:38.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.425+0000 7fcc8db24640 1 --2- 172.21.15.78:0/1572527292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96ec3890 0x55ee96f24310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.425+0000 7fcc8db24640 1 -- 172.21.15.78:0/1572527292 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee96f24850 con 0x55ee96e9a3b0 2024-08-18T23:45:38.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.425+0000 7fcc8eb26640 1 --2- 172.21.15.78:0/1572527292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96ec3890 0x55ee96f24310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:45:38.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.425+0000 7fcc8db24640 1 --2- 172.21.15.78:0/1572527292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e9a3b0 0x55ee96eaef80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcc8409bbf0 tx=0x7fcc840bb4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdcffe16cd290e22 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:38.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.425+0000 7fcc71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:45:38.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.425+0000 7fcc71ffb640 1 -- 172.21.15.78:0/1572527292 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc840b8070 con 0x55ee96e9a3b0 2024-08-18T23:45:38.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.425+0000 7fcc92987740 1 -- 172.21.15.78:0/1572527292 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc840d2600 con 0x55ee96e9a3b0 2024-08-18T23:45:38.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.425+0000 7fcc71ffb640 1 -- 172.21.15.78:0/1572527292 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fcc8409a990 con 0x55ee96e9a3b0 2024-08-18T23:45:38.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.425+0000 7fcc71ffb640 1 -- 172.21.15.78:0/1572527292 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcc840b4300 con 0x55ee96e9a3b0 2024-08-18T23:45:38.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.425+0000 7fcc92987740 10 client.25081 fetch_fsmap learned FSMap version 26 2024-08-18T23:45:38.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.425+0000 7fcc92987740 1 -- 172.21.15.78:0/1572527292 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ee96f25ac0 con 0x55ee96e9a3b0 2024-08-18T23:45:38.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.425+0000 7fcc71ffb640 1 -- 172.21.15.78:0/1572527292 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc840d2600 con 0x55ee96e9a3b0 2024-08-18T23:45:38.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.426+0000 7fcc71ffb640 1 -- 172.21.15.78:0/1572527292 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fcc8409f1b0 con 0x55ee96e9a3b0 2024-08-18T23:45:38.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.426+0000 7fcc92987740 10 client.25081 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:45:38.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.426+0000 7fcc92987740 10 client.25081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:45:38.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.426+0000 7fcc92987740 20 client.25081 populate_metadata read hostname 'smithi078' 2024-08-18T23:45:38.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.426+0000 7fcc92987740 1 --2- 172.21.15.78:0/1572527292 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ee96f25fc0 0x55ee96f283e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:38.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.426+0000 7fcc92987740 4 client.25081 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:45:38.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.426+0000 7fcc92987740 1 -- 172.21.15.78:0/1572527292 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee96dfc4c0 con 0x55ee96f25fc0 2024-08-18T23:45:38.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.426+0000 7fcc8e325640 1 --2- 172.21.15.78:0/1572527292 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ee96f25fc0 0x55ee96f283e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:38.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.426+0000 7fcc8e325640 1 --2- 172.21.15.78:0/1572527292 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ee96f25fc0 0x55ee96f283e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c4b99881d6d02b3 server_cookie=51d169444f67d5d3 in_seq=0 out_seq=0 2024-08-18T23:45:38.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.426+0000 7fcc71ffb640 10 client.25081 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:38.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.428+0000 7fcc71ffb640 1 -- 172.21.15.78:0/1572527292 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee96dfc4c0 con 0x55ee96f25fc0 2024-08-18T23:45:38.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.428+0000 7fcc71ffb640 10 client.25081 handle_command_reply: tid=0 2024-08-18T23:45:38.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.428+0000 7fcc92987740 1 client.25081 shutdown 2024-08-18T23:45:38.435 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:45:38.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.428+0000 7fcc92987740 1 -- 172.21.15.78:0/1572527292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e9a3b0 msgr2=0x55ee96eaef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:38.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.428+0000 7fcc92987740 1 --2- 172.21.15.78:0/1572527292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e9a3b0 0x55ee96eaef80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcc8409bbf0 tx=0x7fcc840bb4e0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc92987740 1 -- 172.21.15.78:0/1572527292 shutdown_connections 2024-08-18T23:45:38.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc92987740 1 --2- 172.21.15.78:0/1572527292 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ee96f25fc0 0x55ee96f283e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc92987740 1 --2- 172.21.15.78:0/1572527292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee96ec3890 0x55ee96f24310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc92987740 1 --2- 172.21.15.78:0/1572527292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee96e9a3b0 0x55ee96eaef80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc92987740 1 --2- 172.21.15.78:0/1572527292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee96e8f1d0 0x55ee96eaea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc92987740 1 -- 172.21.15.78:0/1572527292 >> 172.21.15.78:0/1572527292 conn(0x55ee96e8b760 msgr2=0x55ee96e90c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:38.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc71ffb640 0 client.25081 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:38.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc92987740 1 -- 172.21.15.78:0/1572527292 shutdown_connections 2024-08-18T23:45:38.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc92987740 1 -- 172.21.15.78:0/1572527292 wait complete. 2024-08-18T23:45:38.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc92987740 20 client.25081 trim_cache size 0 max 16384 2024-08-18T23:45:38.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc8f327640 1 -- 172.21.15.78:0/2017944392 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcc740a2c20 msgr2=0x7fcc740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:38.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc8f327640 1 --2- 172.21.15.78:0/2017944392 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcc740a2c20 0x7fcc740a50e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009430 tx=0x7fcc80009360 comp rx=0 tx=0).stop 2024-08-18T23:45:38.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc8f327640 1 -- 172.21.15.78:0/2017944392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc88102090 msgr2=0x7fcc88074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:38.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc8f327640 1 --2- 172.21.15.78:0/2017944392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc88102090 0x7fcc88074760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400b450 tx=0x7fcc8400b910 comp rx=0 tx=0).stop 2024-08-18T23:45:38.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc8f327640 1 -- 172.21.15.78:0/2017944392 shutdown_connections 2024-08-18T23:45:38.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc8f327640 1 --2- 172.21.15.78:0/2017944392 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fcc740a2c20 0x7fcc740a50e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc8f327640 1 --2- 172.21.15.78:0/2017944392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc881076f0 0x7fcc88074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc8f327640 1 --2- 172.21.15.78:0/2017944392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc88102090 0x7fcc88074760 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc8f327640 1 --2- 172.21.15.78:0/2017944392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc881016f0 0x7fcc88074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:38.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc8f327640 1 -- 172.21.15.78:0/2017944392 >> 172.21.15.78:0/2017944392 conn(0x7fcc880fd470 msgr2=0x7fcc880fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:38.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc8f327640 1 -- 172.21.15.78:0/2017944392 shutdown_connections 2024-08-18T23:45:38.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:38.429+0000 7fcc8f327640 1 -- 172.21.15.78:0/2017944392 wait complete. 2024-08-18T23:45:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:45:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:45:38.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:38 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:45:38.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:38 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:45:38.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:38 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:45:38.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:38 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:45:38.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:38 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:45:38.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:38 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:45:38.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:38 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:45:39.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:39 smithi123 ceph-mon[21808]: pgmap v390: 129 pgs: 129 active+clean; 53 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 242 op/s 2024-08-18T23:45:39.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:45:39.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:45:39.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:45:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:39 smithi161 ceph-mon[22027]: pgmap v390: 129 pgs: 129 active+clean; 53 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 242 op/s 2024-08-18T23:45:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:45:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:45:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:45:39.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:39 smithi078 ceph-mon[18440]: pgmap v390: 129 pgs: 129 active+clean; 53 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 242 op/s 2024-08-18T23:45:39.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:45:39.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:45:39.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:45:40.367 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-18T23:45:40.368 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-18T23:45:40.670 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:45:41.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.239+0000 7fd56a35a640 1 -- 172.21.15.78:0/902956623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd564100b50 msgr2=0x7fd564100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:41.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.239+0000 7fd56a35a640 1 --2- 172.21.15.78:0/902956623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd564100b50 0x7fd564100f30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd554009900 tx=0x7fd55402a480 comp rx=0 tx=0).stop 2024-08-18T23:45:41.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.239+0000 7fd56a35a640 1 -- 172.21.15.78:0/902956623 shutdown_connections 2024-08-18T23:45:41.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.239+0000 7fd56a35a640 1 --2- 172.21.15.78:0/902956623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd564102c20 0x7fd56410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.239+0000 7fd56a35a640 1 --2- 172.21.15.78:0/902956623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5641022a0 0x7fd5641026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.239+0000 7fd56a35a640 1 --2- 172.21.15.78:0/902956623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd564100b50 0x7fd564100f30 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.239+0000 7fd56a35a640 1 -- 172.21.15.78:0/902956623 >> 172.21.15.78:0/902956623 conn(0x7fd5640fc500 msgr2=0x7fd5640fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:41.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.239+0000 7fd56a35a640 1 -- 172.21.15.78:0/902956623 shutdown_connections 2024-08-18T23:45:41.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.239+0000 7fd56a35a640 1 -- 172.21.15.78:0/902956623 wait complete. 2024-08-18T23:45:41.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.240+0000 7fd56a35a640 1 Processor -- start 2024-08-18T23:45:41.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.240+0000 7fd56a35a640 1 -- start start 2024-08-18T23:45:41.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.240+0000 7fd56a35a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd564100b50 0x7fd56419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:41.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.240+0000 7fd56a35a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5641022a0 0x7fd56419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:41.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.240+0000 7fd56a35a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd564102c20 0x7fd56419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:41.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.240+0000 7fd56a35a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd564110d00 con 0x7fd5641022a0 2024-08-18T23:45:41.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.240+0000 7fd56a35a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd564110b80 con 0x7fd564102c20 2024-08-18T23:45:41.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.240+0000 7fd56a35a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd564110e80 con 0x7fd564100b50 2024-08-18T23:45:41.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.240+0000 7fd5637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5641022a0 0x7fd56419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:41.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.241+0000 7fd5688f7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd564102c20 0x7fd56419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:41.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.241+0000 7fd5688f7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd564102c20 0x7fd56419ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:33126/0 (socket says 172.21.15.78:33126) 2024-08-18T23:45:41.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.241+0000 7fd5688f7640 1 -- 172.21.15.78:0/3604104570 learned_addr learned my addr 172.21.15.78:0/3604104570 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:41.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.241+0000 7fd563fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd564100b50 0x7fd56419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:41.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.241+0000 7fd5688f7640 1 -- 172.21.15.78:0/3604104570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd564100b50 msgr2=0x7fd56419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:41.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.241+0000 7fd5688f7640 1 --2- 172.21.15.78:0/3604104570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd564100b50 0x7fd56419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.241+0000 7fd5688f7640 1 -- 172.21.15.78:0/3604104570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5641022a0 msgr2=0x7fd56419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:41.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.241+0000 7fd5688f7640 1 --2- 172.21.15.78:0/3604104570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5641022a0 0x7fd56419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.241+0000 7fd5688f7640 1 -- 172.21.15.78:0/3604104570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56419f430 con 0x7fd564102c20 2024-08-18T23:45:41.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.241+0000 7fd563fff640 1 --2- 172.21.15.78:0/3604104570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd564100b50 0x7fd56419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:45:41.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.242+0000 7fd5688f7640 1 --2- 172.21.15.78:0/3604104570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd564102c20 0x7fd56419ed50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009fd0 tx=0x7fd55c00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:41.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.242+0000 7fd5617fa640 1 -- 172.21.15.78:0/3604104570 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c017070 con 0x7fd564102c20 2024-08-18T23:45:41.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.242+0000 7fd5617fa640 1 -- 172.21.15.78:0/3604104570 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd55c003e50 con 0x7fd564102c20 2024-08-18T23:45:41.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.242+0000 7fd5617fa640 1 -- 172.21.15.78:0/3604104570 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c0106c0 con 0x7fd564102c20 2024-08-18T23:45:41.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.242+0000 7fd56a35a640 1 -- 172.21.15.78:0/3604104570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd56419f720 con 0x7fd564102c20 2024-08-18T23:45:41.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.242+0000 7fd56a35a640 1 -- 172.21.15.78:0/3604104570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5641076b0 con 0x7fd564102c20 2024-08-18T23:45:41.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.245+0000 7fd5617fa640 1 -- 172.21.15.78:0/3604104570 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd55c003970 con 0x7fd564102c20 2024-08-18T23:45:41.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.246+0000 7fd56c758740 1 Processor -- start 2024-08-18T23:45:41.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.246+0000 7fd56c758740 1 -- start start 2024-08-18T23:45:41.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.246+0000 7fd5617fa640 1 --2- 172.21.15.78:0/3604104570 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd54c0a2c00 0x7fd54c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:41.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.246+0000 7fd56c758740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6200 0x55a7c46b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:41.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.246+0000 7fd56c758740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46b6b20 0x55a7c46c0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:41.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.246+0000 7fd56c758740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46c13e0 0x55a7c46c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:41.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.246+0000 7fd56c758740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a7c4679e40 con 0x55a7c46b6b20 2024-08-18T23:45:41.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.246+0000 7fd56c758740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a7c461b7f0 con 0x55a7c46c13e0 2024-08-18T23:45:41.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.246+0000 7fd56c758740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a7c45c2fb0 con 0x55a7c46b6200 2024-08-18T23:45:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.246+0000 7fd5637fe640 1 --2- 172.21.15.78:0/3604104570 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd54c0a2c00 0x7fd54c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.246+0000 7fd5688f7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46c13e0 0x55a7c46c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.246+0000 7fd5688f7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46c13e0 0x55a7c46c37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:33142/0 (socket says 172.21.15.78:33142) 2024-08-18T23:45:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.246+0000 7fd5688f7640 1 -- 172.21.15.78:0/1821424743 learned_addr learned my addr 172.21.15.78:0/1821424743 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.246+0000 7fd563fff640 1 --2- 172.21.15.78:0/1821424743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6200 0x55a7c46b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.246+0000 7fd5688f7640 1 -- 172.21.15.78:0/1821424743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6200 msgr2=0x55a7c46b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.247+0000 7fd5688f7640 1 --2- 172.21.15.78:0/1821424743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6200 0x55a7c46b65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.247+0000 7fd5688f7640 1 -- 172.21.15.78:0/1821424743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46b6b20 msgr2=0x55a7c46c0ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T23:45:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.247+0000 7fd5688f7640 1 --2- 172.21.15.78:0/1821424743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46b6b20 0x55a7c46c0ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.247+0000 7fd5688f7640 1 -- 172.21.15.78:0/1821424743 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7c438d6a0 con 0x55a7c46c13e0 2024-08-18T23:45:41.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.247+0000 7fd5617fa640 1 -- 172.21.15.78:0/3604104570 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fd55c0cf070 con 0x7fd564102c20 2024-08-18T23:45:41.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.247+0000 7fd5637fe640 1 --2- 172.21.15.78:0/3604104570 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd54c0a2c00 0x7fd54c0a50c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd5640ffa70 tx=0x7fd558005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:41.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.247+0000 7fd5688f7640 1 --2- 172.21.15.78:0/1821424743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46c13e0 0x55a7c46c37d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0a3c80 tx=0x7fd55c0661c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9edaa0d14b801f8c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:41.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.247+0000 7fd54a7fc640 1 -- 172.21.15.78:0/1821424743 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c09bed0 con 0x55a7c46c13e0 2024-08-18T23:45:41.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.247+0000 7fd54a7fc640 1 -- 172.21.15.78:0/1821424743 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd55c0bb020 con 0x55a7c46c13e0 2024-08-18T23:45:41.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.247+0000 7fd54a7fc640 1 -- 172.21.15.78:0/1821424743 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c09bc40 con 0x55a7c46c13e0 2024-08-18T23:45:41.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.248+0000 7fd56c758740 1 -- 172.21.15.78:0/1821424743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46c13e0 msgr2=0x55a7c46c37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:41.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.248+0000 7fd56c758740 1 --2- 172.21.15.78:0/1821424743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46c13e0 0x55a7c46c37d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0a3c80 tx=0x7fd55c0661c0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.248+0000 7fd56c758740 1 -- 172.21.15.78:0/1821424743 shutdown_connections 2024-08-18T23:45:41.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.248+0000 7fd56c758740 1 --2- 172.21.15.78:0/1821424743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46c13e0 0x55a7c46c37d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.248+0000 7fd56c758740 1 --2- 172.21.15.78:0/1821424743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46b6b20 0x55a7c46c0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.248+0000 7fd56c758740 1 --2- 172.21.15.78:0/1821424743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6200 0x55a7c46b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.248+0000 7fd56c758740 1 -- 172.21.15.78:0/1821424743 >> 172.21.15.78:0/1821424743 conn(0x55a7c46b2790 msgr2=0x55a7c46b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:41.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.248+0000 7fd56c758740 1 -- 172.21.15.78:0/1821424743 shutdown_connections 2024-08-18T23:45:41.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.248+0000 7fd56c758740 1 -- 172.21.15.78:0/1821424743 wait complete. 2024-08-18T23:45:41.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.249+0000 7fd56c758740 1 -- start start 2024-08-18T23:45:41.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.249+0000 7fd56c758740 1 Processor -- start 2024-08-18T23:45:41.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.249+0000 7fd56c758740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6200 0x55a7c46be380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:41.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.249+0000 7fd56c758740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46b6b20 0x55a7c46be8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:41.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.249+0000 7fd56c758740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46c13e0 0x55a7c46bee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:41.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.249+0000 7fd56c758740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a7c45d6cb0 con 0x55a7c46c13e0 2024-08-18T23:45:41.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.249+0000 7fd56c758740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a7c46177b0 con 0x55a7c46b6b20 2024-08-18T23:45:41.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.249+0000 7fd56c758740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a7c46c3f50 con 0x55a7c46b6200 2024-08-18T23:45:41.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.249+0000 7fd56c758740 10 client.0 _ll_register_callbacks cb 0x55a7c432aba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:45:41.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.250+0000 7fd5688f7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46c13e0 0x55a7c46bee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:41.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.250+0000 7fd5688f7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46c13e0 0x55a7c46bee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55218/0 (socket says 172.21.15.78:55218) 2024-08-18T23:45:41.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.250+0000 7fd5688f7640 1 -- 172.21.15.78:0/1941142245 learned_addr learned my addr 172.21.15.78:0/1941142245 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:41.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.250+0000 7fd5637fe640 1 --2- 172.21.15.78:0/1941142245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46b6b20 0x55a7c46be8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:41.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.250+0000 7fd563fff640 1 --2- 172.21.15.78:0/1941142245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6200 0x55a7c46be380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:41.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.250+0000 7fd5688f7640 1 -- 172.21.15.78:0/1941142245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6200 msgr2=0x55a7c46be380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:41.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.250+0000 7fd5688f7640 1 --2- 172.21.15.78:0/1941142245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6200 0x55a7c46be380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.250+0000 7fd5688f7640 1 -- 172.21.15.78:0/1941142245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46b6b20 msgr2=0x55a7c46be8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:41.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.250+0000 7fd5688f7640 1 --2- 172.21.15.78:0/1941142245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46b6b20 0x55a7c46be8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.250+0000 7fd5688f7640 1 -- 172.21.15.78:0/1941142245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7c45d5bc0 con 0x55a7c46c13e0 2024-08-18T23:45:41.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.251+0000 7fd5688f7640 1 --2- 172.21.15.78:0/1941142245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46c13e0 0x55a7c46bee00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd55c09b1f0 tx=0x7fd55c0d27b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb895d55807a88c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:41.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.251+0000 7fd5437fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:45:41.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.251+0000 7fd5437fe640 1 -- 172.21.15.78:0/1941142245 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c020040 con 0x55a7c46c13e0 2024-08-18T23:45:41.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.251+0000 7fd5437fe640 1 -- 172.21.15.78:0/1941142245 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fd55c097bb0 con 0x55a7c46c13e0 2024-08-18T23:45:41.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.251+0000 7fd56c758740 1 -- 172.21.15.78:0/1941142245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd55c0bb020 con 0x55a7c46c13e0 2024-08-18T23:45:41.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.251+0000 7fd5437fe640 1 -- 172.21.15.78:0/1941142245 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd55c09bed0 con 0x55a7c46c13e0 2024-08-18T23:45:41.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.252+0000 7fd56c758740 10 client.15696 fetch_fsmap learned FSMap version 26 2024-08-18T23:45:41.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.252+0000 7fd56c758740 1 -- 172.21.15.78:0/1941142245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a7c46cfeb0 con 0x55a7c46c13e0 2024-08-18T23:45:41.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.252+0000 7fd5437fe640 1 -- 172.21.15.78:0/1941142245 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd55c0bb020 con 0x55a7c46c13e0 2024-08-18T23:45:41.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.252+0000 7fd5437fe640 1 -- 172.21.15.78:0/1941142245 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fd55c0d4ce0 con 0x55a7c46c13e0 2024-08-18T23:45:41.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.252+0000 7fd56c758740 10 client.15696 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:45:41.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.252+0000 7fd56c758740 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:45:41.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.253+0000 7fd56c758740 20 client.15696 populate_metadata read hostname 'smithi078' 2024-08-18T23:45:41.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.253+0000 7fd56c758740 1 --2- 172.21.15.78:0/1941142245 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a7c46d1200 0x55a7c46d35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:41.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.253+0000 7fd56c758740 4 client.15696 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:45:41.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.253+0000 7fd56c758740 1 -- 172.21.15.78:0/1941142245 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7c4628d60 con 0x55a7c46d1200 2024-08-18T23:45:41.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.253+0000 7fd563fff640 1 --2- 172.21.15.78:0/1941142245 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a7c46d1200 0x55a7c46d35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:41.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.253+0000 7fd5437fe640 10 client.15696 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:41.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.254+0000 7fd563fff640 1 --2- 172.21.15.78:0/1941142245 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a7c46d1200 0x55a7c46d35f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e4406f4cf3a24fd server_cookie=fc759190a37d1c03 in_seq=0 out_seq=0 2024-08-18T23:45:41.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.255+0000 7fd5437fe640 1 -- 172.21.15.78:0/1941142245 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7c4628d60 con 0x55a7c46d1200 2024-08-18T23:45:41.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.255+0000 7fd5437fe640 10 client.15696 handle_command_reply: tid=0 2024-08-18T23:45:41.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.255+0000 7fd56c758740 1 client.15696 shutdown 2024-08-18T23:45:41.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.256+0000 7fd56c758740 1 -- 172.21.15.78:0/1941142245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46c13e0 msgr2=0x55a7c46bee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:41.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.256+0000 7fd56c758740 1 --2- 172.21.15.78:0/1941142245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46c13e0 0x55a7c46bee00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd55c09b1f0 tx=0x7fd55c0d27b0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.256+0000 7fd56c758740 1 -- 172.21.15.78:0/1941142245 shutdown_connections 2024-08-18T23:45:41.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.256+0000 7fd56c758740 1 --2- 172.21.15.78:0/1941142245 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a7c46d1200 0x55a7c46d35f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.256+0000 7fd56c758740 1 --2- 172.21.15.78:0/1941142245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46c13e0 0x55a7c46bee00 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.256+0000 7fd56c758740 1 --2- 172.21.15.78:0/1941142245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46b6b20 0x55a7c46be8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.256+0000 7fd56c758740 1 --2- 172.21.15.78:0/1941142245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6200 0x55a7c46be380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.256+0000 7fd5437fe640 0 client.15696 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:41.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.256+0000 7fd56c758740 1 -- 172.21.15.78:0/1941142245 >> 172.21.15.78:0/1941142245 conn(0x55a7c46b2790 msgr2=0x55a7c46b84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:41.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.256+0000 7fd56c758740 1 -- 172.21.15.78:0/1941142245 shutdown_connections 2024-08-18T23:45:41.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.256+0000 7fd56c758740 1 -- 172.21.15.78:0/1941142245 wait complete. 2024-08-18T23:45:41.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.256+0000 7fd56c758740 20 client.15696 trim_cache size 0 max 16384 2024-08-18T23:45:41.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.279+0000 7fd56c758740 1 Processor -- start 2024-08-18T23:45:41.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.279+0000 7fd56c758740 1 -- start start 2024-08-18T23:45:41.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.279+0000 7fd56c758740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6200 0x55a7c46c02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:41.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.279+0000 7fd56c758740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46b6b20 0x55a7c46d8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:41.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.279+0000 7fd56c758740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46c13e0 0x55a7c46d9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:41.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.279+0000 7fd56c758740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5641af220 con 0x55a7c46c13e0 2024-08-18T23:45:41.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.279+0000 7fd56c758740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5641acaf0 con 0x55a7c46b6b20 2024-08-18T23:45:41.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.279+0000 7fd56c758740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd5641aa960 con 0x55a7c46b6200 2024-08-18T23:45:41.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.280+0000 7fd5688f7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46c13e0 0x55a7c46d9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:41.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.280+0000 7fd563fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6200 0x55a7c46c02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:41.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.280+0000 7fd5637fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46b6b20 0x55a7c46d8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:41.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.280+0000 7fd5688f7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46c13e0 0x55a7c46d9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55232/0 (socket says 172.21.15.78:55232) 2024-08-18T23:45:41.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.280+0000 7fd5688f7640 1 -- 172.21.15.78:0/1940384424 learned_addr learned my addr 172.21.15.78:0/1940384424 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:41.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.280+0000 7fd5688f7640 1 -- 172.21.15.78:0/1940384424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6200 msgr2=0x55a7c46c02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:41.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.280+0000 7fd5688f7640 1 --2- 172.21.15.78:0/1940384424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6200 0x55a7c46c02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.280+0000 7fd5688f7640 1 -- 172.21.15.78:0/1940384424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46b6b20 msgr2=0x55a7c46d8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:41.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.280+0000 7fd5688f7640 1 --2- 172.21.15.78:0/1940384424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46b6b20 0x55a7c46d8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.280+0000 7fd5688f7640 1 -- 172.21.15.78:0/1940384424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7c46b91b0 con 0x55a7c46c13e0 2024-08-18T23:45:41.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.280+0000 7fd5637fe640 1 --2- 172.21.15.78:0/1940384424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46b6b20 0x55a7c46d8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:45:41.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.280+0000 7fd5688f7640 1 --2- 172.21.15.78:0/1940384424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46c13e0 0x55a7c46d9100 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd55c005c40 tx=0x7fd55c0cc830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f22165931761b6 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:41.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.280+0000 7fd549ffb640 1 -- 172.21.15.78:0/1940384424 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c020070 con 0x55a7c46c13e0 2024-08-18T23:45:41.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.280+0000 7fd549ffb640 1 -- 172.21.15.78:0/1940384424 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd55c0bb070 con 0x55a7c46c13e0 2024-08-18T23:45:41.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.280+0000 7fd549ffb640 1 -- 172.21.15.78:0/1940384424 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c09f8f0 con 0x55a7c46c13e0 2024-08-18T23:45:41.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.281+0000 7fd56c758740 1 -- 172.21.15.78:0/1940384424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46c13e0 msgr2=0x55a7c46d9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:41.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.281+0000 7fd56c758740 1 --2- 172.21.15.78:0/1940384424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46c13e0 0x55a7c46d9100 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd55c005c40 tx=0x7fd55c0cc830 comp rx=0 tx=0).stop 2024-08-18T23:45:41.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.281+0000 7fd56c758740 1 -- 172.21.15.78:0/1940384424 shutdown_connections 2024-08-18T23:45:41.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.281+0000 7fd56c758740 1 --2- 172.21.15.78:0/1940384424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46c13e0 0x55a7c46d9100 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.281+0000 7fd56c758740 1 --2- 172.21.15.78:0/1940384424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46b6b20 0x55a7c46d8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.281+0000 7fd56c758740 1 --2- 172.21.15.78:0/1940384424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6200 0x55a7c46c02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.281+0000 7fd56c758740 1 -- 172.21.15.78:0/1940384424 >> 172.21.15.78:0/1940384424 conn(0x55a7c46b2790 msgr2=0x55a7c46bfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:41.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.281+0000 7fd56c758740 1 -- 172.21.15.78:0/1940384424 shutdown_connections 2024-08-18T23:45:41.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.281+0000 7fd56c758740 1 -- 172.21.15.78:0/1940384424 wait complete. 2024-08-18T23:45:41.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.282+0000 7fd56c758740 1 -- start start 2024-08-18T23:45:41.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.282+0000 7fd56c758740 1 Processor -- start 2024-08-18T23:45:41.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.282+0000 7fd56c758740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46b6200 0x55a7c46c5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:41.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.282+0000 7fd56c758740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6b20 0x55a7c46c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:41.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.282+0000 7fd56c758740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46c13e0 0x55a7c473ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:41.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.282+0000 7fd56c758740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5641aafb0 con 0x55a7c46b6200 2024-08-18T23:45:41.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.282+0000 7fd56c758740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5641ac420 con 0x55a7c46c13e0 2024-08-18T23:45:41.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.282+0000 7fd563fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46b6200 0x55a7c46c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:41.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.282+0000 7fd563fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46b6200 0x55a7c46c5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55238/0 (socket says 172.21.15.78:55238) 2024-08-18T23:45:41.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.282+0000 7fd563fff640 1 -- 172.21.15.78:0/692973098 learned_addr learned my addr 172.21.15.78:0/692973098 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:41.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.282+0000 7fd56c758740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd5641ab680 con 0x55a7c46b6b20 2024-08-18T23:45:41.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.283+0000 7fd5688f7640 1 --2- 172.21.15.78:0/692973098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46c13e0 0x55a7c473ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:41.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.283+0000 7fd5637fe640 1 --2- 172.21.15.78:0/692973098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6b20 0x55a7c46c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:41.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.283+0000 7fd56c758740 10 client.0 _ll_register_callbacks cb 0x7fd558005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:45:41.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.283+0000 7fd5688f7640 1 -- 172.21.15.78:0/692973098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6b20 msgr2=0x55a7c46c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:41.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.283+0000 7fd5688f7640 1 --2- 172.21.15.78:0/692973098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6b20 0x55a7c46c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.283+0000 7fd5688f7640 1 -- 172.21.15.78:0/692973098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46b6200 msgr2=0x55a7c46c5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:41.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.283+0000 7fd5688f7640 1 --2- 172.21.15.78:0/692973098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46b6200 0x55a7c46c5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.283+0000 7fd5688f7640 1 -- 172.21.15.78:0/692973098 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7c46c60d0 con 0x55a7c46c13e0 2024-08-18T23:45:41.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.283+0000 7fd5688f7640 1 --2- 172.21.15.78:0/692973098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46c13e0 0x55a7c473ab90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0979a0 tx=0x7fd55c09b680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=205dc27bd5a00eb2 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:41.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.283+0000 7fd5437fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:45:41.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.283+0000 7fd5437fe640 1 -- 172.21.15.78:0/692973098 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c020070 con 0x55a7c46c13e0 2024-08-18T23:45:41.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.283+0000 7fd5437fe640 1 -- 172.21.15.78:0/692973098 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fd55c09b9a0 con 0x55a7c46c13e0 2024-08-18T23:45:41.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.283+0000 7fd56c758740 1 -- 172.21.15.78:0/692973098 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd55c0bb070 con 0x55a7c46c13e0 2024-08-18T23:45:41.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.284+0000 7fd5437fe640 1 -- 172.21.15.78:0/692973098 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd55c09c670 con 0x55a7c46c13e0 2024-08-18T23:45:41.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.284+0000 7fd56c758740 10 client.25184 fetch_fsmap learned FSMap version 26 2024-08-18T23:45:41.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.284+0000 7fd56c758740 1 -- 172.21.15.78:0/692973098 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a7c46c60d0 con 0x55a7c46c13e0 2024-08-18T23:45:41.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.284+0000 7fd5437fe640 1 -- 172.21.15.78:0/692973098 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd55c0bb070 con 0x55a7c46c13e0 2024-08-18T23:45:41.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.284+0000 7fd5437fe640 1 -- 172.21.15.78:0/692973098 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fd55c0b4230 con 0x55a7c46c13e0 2024-08-18T23:45:41.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.284+0000 7fd56c758740 10 client.25184 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:45:41.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.284+0000 7fd56c758740 10 client.25184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:45:41.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.285+0000 7fd56c758740 20 client.25184 populate_metadata read hostname 'smithi078' 2024-08-18T23:45:41.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.285+0000 7fd56c758740 1 --2- 172.21.15.78:0/692973098 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a7c473c4d0 0x55a7c473e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:41.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.285+0000 7fd56c758740 4 client.25184 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-18T23:45:41.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.285+0000 7fd56c758740 1 -- 172.21.15.78:0/692973098 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a7c4623270 con 0x55a7c473c4d0 2024-08-18T23:45:41.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.285+0000 7fd563fff640 1 --2- 172.21.15.78:0/692973098 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a7c473c4d0 0x55a7c473e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:41.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.285+0000 7fd563fff640 1 --2- 172.21.15.78:0/692973098 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a7c473c4d0 0x55a7c473e8c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=638b3aa305420a78 server_cookie=3ed178c35959b6bf in_seq=0 out_seq=0 2024-08-18T23:45:41.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.285+0000 7fd5437fe640 10 client.25184 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:41.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.287+0000 7fd5437fe640 1 -- 172.21.15.78:0/692973098 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a7c4623270 con 0x55a7c473c4d0 2024-08-18T23:45:41.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.288+0000 7fd5437fe640 10 client.25184 handle_command_reply: tid=0 2024-08-18T23:45:41.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.288+0000 7fd56c758740 1 client.25184 shutdown 2024-08-18T23:45:41.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.288+0000 7fd56c758740 1 -- 172.21.15.78:0/692973098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46c13e0 msgr2=0x55a7c473ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:41.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.288+0000 7fd56c758740 1 --2- 172.21.15.78:0/692973098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46c13e0 0x55a7c473ab90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0979a0 tx=0x7fd55c09b680 comp rx=0 tx=0).stop 2024-08-18T23:45:41.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.288+0000 7fd56c758740 1 -- 172.21.15.78:0/692973098 shutdown_connections 2024-08-18T23:45:41.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.288+0000 7fd56c758740 1 --2- 172.21.15.78:0/692973098 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a7c473c4d0 0x55a7c473e8c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.288+0000 7fd56c758740 1 --2- 172.21.15.78:0/692973098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7c46c13e0 0x55a7c473ab90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.288+0000 7fd5437fe640 0 client.25184 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:41.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.288+0000 7fd56c758740 1 --2- 172.21.15.78:0/692973098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7c46b6b20 0x55a7c46c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.288+0000 7fd56c758740 1 --2- 172.21.15.78:0/692973098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7c46b6200 0x55a7c46c5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.288+0000 7fd56c758740 1 -- 172.21.15.78:0/692973098 >> 172.21.15.78:0/692973098 conn(0x55a7c46b2790 msgr2=0x55a7c46b8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:41.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.288+0000 7fd56c758740 1 -- 172.21.15.78:0/692973098 shutdown_connections 2024-08-18T23:45:41.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.288+0000 7fd56c758740 1 -- 172.21.15.78:0/692973098 wait complete. 2024-08-18T23:45:41.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.288+0000 7fd56c758740 20 client.25184 trim_cache size 0 max 16384 2024-08-18T23:45:41.293 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:45:41.293 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-08-18T23:45:41.293 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "d714a85d-b4be-401f-a08c-11368554a659", 2024-08-18T23:45:41.294 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-08-18T23:45:41.294 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:45:41.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.289+0000 7fd56a35a640 1 -- 172.21.15.78:0/3604104570 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd54c0a2c00 msgr2=0x7fd54c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:41.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.289+0000 7fd56a35a640 1 --2- 172.21.15.78:0/3604104570 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd54c0a2c00 0x7fd54c0a50c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd5640ffa70 tx=0x7fd558005f90 comp rx=0 tx=0).stop 2024-08-18T23:45:41.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.289+0000 7fd56a35a640 1 -- 172.21.15.78:0/3604104570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd564102c20 msgr2=0x7fd56419ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:41.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.289+0000 7fd56a35a640 1 --2- 172.21.15.78:0/3604104570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd564102c20 0x7fd56419ed50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009fd0 tx=0x7fd55c00b260 comp rx=0 tx=0).stop 2024-08-18T23:45:41.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.289+0000 7fd56a35a640 1 -- 172.21.15.78:0/3604104570 shutdown_connections 2024-08-18T23:45:41.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.289+0000 7fd56a35a640 1 --2- 172.21.15.78:0/3604104570 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd54c0a2c00 0x7fd54c0a50c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.289+0000 7fd56a35a640 1 --2- 172.21.15.78:0/3604104570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd564102c20 0x7fd56419ed50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.289+0000 7fd56a35a640 1 --2- 172.21.15.78:0/3604104570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5641022a0 0x7fd56419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.289+0000 7fd56a35a640 1 --2- 172.21.15.78:0/3604104570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd564100b50 0x7fd56419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:41.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.289+0000 7fd56a35a640 1 -- 172.21.15.78:0/3604104570 >> 172.21.15.78:0/3604104570 conn(0x7fd5640fc500 msgr2=0x7fd564069d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:41.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.289+0000 7fd56a35a640 1 -- 172.21.15.78:0/3604104570 shutdown_connections 2024-08-18T23:45:41.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:41.289+0000 7fd56a35a640 1 -- 172.21.15.78:0/3604104570 wait complete. 2024-08-18T23:45:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:41 smithi078 ceph-mon[18440]: pgmap v391: 129 pgs: 129 active+clean; 54 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 241 op/s 2024-08-18T23:45:41.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:41 smithi123 ceph-mon[21808]: pgmap v391: 129 pgs: 129 active+clean; 54 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 241 op/s 2024-08-18T23:45:41.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:41 smithi161 ceph-mon[22027]: pgmap v391: 129 pgs: 129 active+clean; 54 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 241 op/s 2024-08-18T23:45:42.097 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:45:42.395 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:45:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:42 smithi078 ceph-mon[18440]: scrub queued for path: / 2024-08-18T23:45:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:42 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:45:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:42 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:45:42.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:42 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:45:42.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:42 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:45:42.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:42 smithi123 ceph-mon[21808]: scrub queued for path: / 2024-08-18T23:45:42.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:42 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:45:42.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:42 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:45:42.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:42 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:45:42.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:42 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:45:42.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:42 smithi161 ceph-mon[22027]: scrub queued for path: / 2024-08-18T23:45:42.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:42 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:45:42.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:42 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:45:42.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:42 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:45:42.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:42 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:45:42.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.888+0000 7f1f02b41640 1 -- 172.21.15.78:0/1874939165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1efc1022a0 msgr2=0x7f1efc1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:42.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.888+0000 7f1f02b41640 1 --2- 172.21.15.78:0/1874939165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1efc1022a0 0x7f1efc1026e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009900 tx=0x7f1ee802a480 comp rx=0 tx=0).stop 2024-08-18T23:45:42.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.888+0000 7f1f02b41640 1 -- 172.21.15.78:0/1874939165 shutdown_connections 2024-08-18T23:45:42.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.888+0000 7f1f02b41640 1 --2- 172.21.15.78:0/1874939165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1efc102c20 0x7f1efc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.888+0000 7f1f02b41640 1 --2- 172.21.15.78:0/1874939165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1efc1022a0 0x7f1efc1026e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.888+0000 7f1f02b41640 1 --2- 172.21.15.78:0/1874939165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1efc100b50 0x7f1efc100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.888+0000 7f1f02b41640 1 -- 172.21.15.78:0/1874939165 >> 172.21.15.78:0/1874939165 conn(0x7f1efc0fc500 msgr2=0x7f1efc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.888+0000 7f1f02b41640 1 -- 172.21.15.78:0/1874939165 shutdown_connections 2024-08-18T23:45:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.888+0000 7f1f02b41640 1 -- 172.21.15.78:0/1874939165 wait complete. 2024-08-18T23:45:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.889+0000 7f1f02b41640 1 Processor -- start 2024-08-18T23:45:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.889+0000 7f1f02b41640 1 -- start start 2024-08-18T23:45:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.889+0000 7f1f02b41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1efc100b50 0x7f1efc199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.889+0000 7f1f02b41640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1efc1022a0 0x7f1efc199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.889+0000 7f1f02b41640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1efc102c20 0x7f1efc1a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.889+0000 7f1f02b41640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc110cd0 con 0x7f1efc100b50 2024-08-18T23:45:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.889+0000 7f1f02b41640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc110b50 con 0x7f1efc102c20 2024-08-18T23:45:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.889+0000 7f1f02b41640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc110e50 con 0x7f1efc1022a0 2024-08-18T23:45:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.889+0000 7f1f008dd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1efc100b50 0x7f1efc199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.889+0000 7f1f008dd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1efc100b50 0x7f1efc199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55266/0 (socket says 172.21.15.78:55266) 2024-08-18T23:45:42.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.889+0000 7f1f008dd640 1 -- 172.21.15.78:0/2861048732 learned_addr learned my addr 172.21.15.78:0/2861048732 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.889+0000 7f1f010de640 1 --2- 172.21.15.78:0/2861048732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1efc102c20 0x7f1efc1a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.889+0000 7f1efbfff640 1 --2- 172.21.15.78:0/2861048732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1efc1022a0 0x7f1efc199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.889+0000 7f1f008dd640 1 -- 172.21.15.78:0/2861048732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1efc1022a0 msgr2=0x7f1efc199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.889+0000 7f1f008dd640 1 --2- 172.21.15.78:0/2861048732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1efc1022a0 0x7f1efc199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.889+0000 7f1f008dd640 1 -- 172.21.15.78:0/2861048732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1efc102c20 msgr2=0x7f1efc1a4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.889+0000 7f1f008dd640 1 --2- 172.21.15.78:0/2861048732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1efc102c20 0x7f1efc1a4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.889+0000 7f1f008dd640 1 -- 172.21.15.78:0/2861048732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1efc1a46c0 con 0x7f1efc100b50 2024-08-18T23:45:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.890+0000 7f1f008dd640 1 --2- 172.21.15.78:0/2861048732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1efc100b50 0x7f1efc199100 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000e9f0 tx=0x7f1ef000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.890+0000 7f1ef9ffb640 1 -- 172.21.15.78:0/2861048732 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef000cd80 con 0x7f1efc100b50 2024-08-18T23:45:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.890+0000 7f1f02b41640 1 -- 172.21.15.78:0/2861048732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1efc1a4890 con 0x7f1efc100b50 2024-08-18T23:45:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.890+0000 7f1ef9ffb640 1 -- 172.21.15.78:0/2861048732 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ef0010910 con 0x7f1efc100b50 2024-08-18T23:45:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.890+0000 7f1ef9ffb640 1 -- 172.21.15.78:0/2861048732 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00188f0 con 0x7f1efc100b50 2024-08-18T23:45:42.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.890+0000 7f1f02b41640 1 -- 172.21.15.78:0/2861048732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1efc1a4dd0 con 0x7f1efc100b50 2024-08-18T23:45:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.892+0000 7f1ef9ffb640 1 -- 172.21.15.78:0/2861048732 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1ef0010ac0 con 0x7f1efc100b50 2024-08-18T23:45:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.893+0000 7f1f04f3f740 1 Processor -- start 2024-08-18T23:45:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.893+0000 7f1f04f3f740 1 -- start start 2024-08-18T23:45:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.893+0000 7f1f04f3f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fca0d0 0x563b58fca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.893+0000 7f1f04f3f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fca9f0 0x563b58fd92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.893+0000 7f1f04f3f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fd9800 0x563b58fdbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.893+0000 7f1ef9ffb640 1 --2- 172.21.15.78:0/2861048732 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1eec0a2b80 0x7f1eec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.893+0000 7f1f04f3f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563b58f2f980 con 0x563b58fca0d0 2024-08-18T23:45:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.893+0000 7f1f04f3f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563b58ec7a70 con 0x563b58fca9f0 2024-08-18T23:45:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.893+0000 7f1f04f3f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563b58f2b940 con 0x563b58fd9800 2024-08-18T23:45:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.893+0000 7f1f008dd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fca9f0 0x563b58fd92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.893+0000 7f1f010de640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fca0d0 0x563b58fca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:42.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.893+0000 7f1efbfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fd9800 0x563b58fdbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.893+0000 7f1f010de640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fca0d0 0x563b58fca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55268/0 (socket says 172.21.15.78:55268) 2024-08-18T23:45:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.893+0000 7f1f008dd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fca9f0 0x563b58fd92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:33188/0 (socket says 172.21.15.78:33188) 2024-08-18T23:45:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.893+0000 7f1f010de640 1 -- 172.21.15.78:0/4049261899 learned_addr learned my addr 172.21.15.78:0/4049261899 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.893+0000 7f1ef9ffb640 1 -- 172.21.15.78:0/2861048732 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f1ef00cf810 con 0x7f1efc100b50 2024-08-18T23:45:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.893+0000 7f1f010de640 1 --2- 172.21.15.78:0/2861048732 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1eec0a2b80 0x7f1eec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.893+0000 7f1f008dd640 1 -- 172.21.15.78:0/4049261899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fd9800 msgr2=0x563b58fdbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.893+0000 7f1f008dd640 1 --2- 172.21.15.78:0/4049261899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fd9800 0x563b58fdbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.894+0000 7f1f008dd640 1 -- 172.21.15.78:0/4049261899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fca0d0 msgr2=0x563b58fca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.894+0000 7f1f008dd640 1 --2- 172.21.15.78:0/4049261899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fca0d0 0x563b58fca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.894+0000 7f1f008dd640 1 -- 172.21.15.78:0/4049261899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b58f2a6e0 con 0x563b58fca9f0 2024-08-18T23:45:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.894+0000 7f1f010de640 1 --2- 172.21.15.78:0/2861048732 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1eec0a2b80 0x7f1eec0a5040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009900 tx=0x7f1ef4009650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:42.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.894+0000 7f1f008dd640 1 --2- 172.21.15.78:0/4049261899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fca9f0 0x563b58fd92c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0098140 tx=0x7f1ef0097980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c27f698debdb4365 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.894+0000 7f1ee6ffd640 1 -- 172.21.15.78:0/4049261899 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef009bce0 con 0x563b58fca9f0 2024-08-18T23:45:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.894+0000 7f1ee6ffd640 1 -- 172.21.15.78:0/4049261899 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ef00a0090 con 0x563b58fca9f0 2024-08-18T23:45:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.894+0000 7f1ee6ffd640 1 -- 172.21.15.78:0/4049261899 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0097a30 con 0x563b58fca9f0 2024-08-18T23:45:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.895+0000 7f1f04f3f740 1 -- 172.21.15.78:0/4049261899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fca9f0 msgr2=0x563b58fd92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.895+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/4049261899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fca9f0 0x563b58fd92c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0098140 tx=0x7f1ef0097980 comp rx=0 tx=0).stop 2024-08-18T23:45:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.895+0000 7f1f04f3f740 1 -- 172.21.15.78:0/4049261899 shutdown_connections 2024-08-18T23:45:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.895+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/4049261899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fd9800 0x563b58fdbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.895+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/4049261899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fca9f0 0x563b58fd92c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.895+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/4049261899 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fca0d0 0x563b58fca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.895+0000 7f1f04f3f740 1 -- 172.21.15.78:0/4049261899 >> 172.21.15.78:0/4049261899 conn(0x563b58fc67a0 msgr2=0x563b58fc6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.895+0000 7f1f04f3f740 1 -- 172.21.15.78:0/4049261899 shutdown_connections 2024-08-18T23:45:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.895+0000 7f1f04f3f740 1 -- 172.21.15.78:0/4049261899 wait complete. 2024-08-18T23:45:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.895+0000 7f1f04f3f740 1 -- start start 2024-08-18T23:45:42.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.895+0000 7f1f04f3f740 1 Processor -- start 2024-08-18T23:45:42.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f04f3f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fca0d0 0x563b58fd66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:42.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f04f3f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fca9f0 0x563b58fd6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:42.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f04f3f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fd9800 0x563b58fd7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:42.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f04f3f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563b58fdc310 con 0x563b58fca0d0 2024-08-18T23:45:42.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f04f3f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563b58eeaaa0 con 0x563b58fd9800 2024-08-18T23:45:42.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f04f3f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563b58fdc490 con 0x563b58fca9f0 2024-08-18T23:45:42.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f04f3f740 10 client.0 _ll_register_callbacks cb 0x563b58ed9e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:45:42.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f008dd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fca0d0 0x563b58fd66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:42.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f008dd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fca0d0 0x563b58fd66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55274/0 (socket says 172.21.15.78:55274) 2024-08-18T23:45:42.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f008dd640 1 -- 172.21.15.78:0/3916034800 learned_addr learned my addr 172.21.15.78:0/3916034800 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:42.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f010de640 1 --2- 172.21.15.78:0/3916034800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fd9800 0x563b58fd7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:42.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1efbfff640 1 --2- 172.21.15.78:0/3916034800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fca9f0 0x563b58fd6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:42.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f008dd640 1 -- 172.21.15.78:0/3916034800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fca9f0 msgr2=0x563b58fd6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f008dd640 1 --2- 172.21.15.78:0/3916034800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fca9f0 0x563b58fd6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f008dd640 1 -- 172.21.15.78:0/3916034800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fd9800 msgr2=0x563b58fd7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f008dd640 1 --2- 172.21.15.78:0/3916034800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fd9800 0x563b58fd7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f008dd640 1 -- 172.21.15.78:0/3916034800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b58ece650 con 0x563b58fca0d0 2024-08-18T23:45:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f008dd640 1 --2- 172.21.15.78:0/3916034800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fca0d0 0x563b58fd66e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00a4420 tx=0x7f1ef00a49c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603707e978f62f49 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1edbfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:45:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1edbfff640 1 -- 172.21.15.78:0/3916034800 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef008a070 con 0x563b58fca0d0 2024-08-18T23:45:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1edbfff640 1 -- 172.21.15.78:0/3916034800 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f1ef0010430 con 0x563b58fca0d0 2024-08-18T23:45:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f04f3f740 1 -- 172.21.15.78:0/3916034800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef00a0090 con 0x563b58fca0d0 2024-08-18T23:45:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1edbfff640 1 -- 172.21.15.78:0/3916034800 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1ef009bce0 con 0x563b58fca0d0 2024-08-18T23:45:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f04f3f740 10 client.15714 fetch_fsmap learned FSMap version 26 2024-08-18T23:45:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.896+0000 7f1f04f3f740 1 -- 172.21.15.78:0/3916034800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563b58ca13c0 con 0x563b58fca0d0 2024-08-18T23:45:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.897+0000 7f1edbfff640 1 -- 172.21.15.78:0/3916034800 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ef00a0090 con 0x563b58fca0d0 2024-08-18T23:45:42.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.897+0000 7f1edbfff640 1 -- 172.21.15.78:0/3916034800 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f1ef009aba0 con 0x563b58fca0d0 2024-08-18T23:45:42.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.897+0000 7f1f04f3f740 10 client.15714 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:45:42.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.897+0000 7f1f04f3f740 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:45:42.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.897+0000 7f1f04f3f740 20 client.15714 populate_metadata read hostname 'smithi078' 2024-08-18T23:45:42.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.897+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/3916034800 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563b58fe90d0 0x563b58feb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:42.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.897+0000 7f1f04f3f740 4 client.15714 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:45:42.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.897+0000 7f1f04f3f740 1 -- 172.21.15.78:0/3916034800 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b58f3cef0 con 0x563b58fe90d0 2024-08-18T23:45:42.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.897+0000 7f1efbfff640 1 --2- 172.21.15.78:0/3916034800 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563b58fe90d0 0x563b58feb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:42.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.897+0000 7f1edbfff640 10 client.15714 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:42.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.897+0000 7f1efbfff640 1 --2- 172.21.15.78:0/3916034800 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563b58fe90d0 0x563b58feb4c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9a09cbaf8a45e5d server_cookie=b0a2236d87020886 in_seq=0 out_seq=0 2024-08-18T23:45:42.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.902+0000 7f1edbfff640 1 -- 172.21.15.78:0/3916034800 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b58f3cef0 con 0x563b58fe90d0 2024-08-18T23:45:42.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.902+0000 7f1edbfff640 10 client.15714 handle_command_reply: tid=0 2024-08-18T23:45:42.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.902+0000 7f1f04f3f740 1 client.15714 shutdown 2024-08-18T23:45:42.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.902+0000 7f1f04f3f740 1 -- 172.21.15.78:0/3916034800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fca0d0 msgr2=0x563b58fd66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:42.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.902+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/3916034800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fca0d0 0x563b58fd66e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00a4420 tx=0x7f1ef00a49c0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.902+0000 7f1f04f3f740 1 -- 172.21.15.78:0/3916034800 shutdown_connections 2024-08-18T23:45:42.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.902+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/3916034800 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563b58fe90d0 0x563b58feb4c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.902+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/3916034800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fd9800 0x563b58fd7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.902+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/3916034800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fca9f0 0x563b58fd6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.902+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/3916034800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fca0d0 0x563b58fd66e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.902+0000 7f1f04f3f740 1 -- 172.21.15.78:0/3916034800 >> 172.21.15.78:0/3916034800 conn(0x563b58fc67a0 msgr2=0x563b58fcc500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:42.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.902+0000 7f1edbfff640 0 client.15714 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:42.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.902+0000 7f1f04f3f740 1 -- 172.21.15.78:0/3916034800 shutdown_connections 2024-08-18T23:45:42.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.902+0000 7f1f04f3f740 1 -- 172.21.15.78:0/3916034800 wait complete. 2024-08-18T23:45:42.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.902+0000 7f1f04f3f740 20 client.15714 trim_cache size 0 max 16384 2024-08-18T23:45:42.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.919+0000 7f1f04f3f740 1 Processor -- start 2024-08-18T23:45:42.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.919+0000 7f1f04f3f740 1 -- start start 2024-08-18T23:45:42.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.919+0000 7f1f04f3f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fca0d0 0x563b58fd8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:42.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.919+0000 7f1f04f3f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fca9f0 0x563b58fd6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:42.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.919+0000 7f1f04f3f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fd9800 0x563b58fd39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:42.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.919+0000 7f1f04f3f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc1af180 con 0x563b58fd9800 2024-08-18T23:45:42.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.919+0000 7f1f04f3f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc1aca50 con 0x563b58fca0d0 2024-08-18T23:45:42.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.919+0000 7f1f04f3f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc1aa8a0 con 0x563b58fca9f0 2024-08-18T23:45:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.919+0000 7f1f010de640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fd9800 0x563b58fd39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.919+0000 7f1f010de640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fd9800 0x563b58fd39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55286/0 (socket says 172.21.15.78:55286) 2024-08-18T23:45:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.919+0000 7f1f010de640 1 -- 172.21.15.78:0/1628954867 learned_addr learned my addr 172.21.15.78:0/1628954867 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.919+0000 7f1f008dd640 1 --2- 172.21.15.78:0/1628954867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fca0d0 0x563b58fd8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.920+0000 7f1efbfff640 1 --2- 172.21.15.78:0/1628954867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fca9f0 0x563b58fd6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.920+0000 7f1f010de640 1 -- 172.21.15.78:0/1628954867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fca9f0 msgr2=0x563b58fd6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.920+0000 7f1f010de640 1 --2- 172.21.15.78:0/1628954867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fca9f0 0x563b58fd6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.920+0000 7f1f010de640 1 -- 172.21.15.78:0/1628954867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fca0d0 msgr2=0x563b58fd8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.920+0000 7f1f010de640 1 --2- 172.21.15.78:0/1628954867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fca0d0 0x563b58fd8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.920+0000 7f1f010de640 1 -- 172.21.15.78:0/1628954867 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b58fd4c70 con 0x563b58fd9800 2024-08-18T23:45:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.920+0000 7f1f010de640 1 --2- 172.21.15.78:0/1628954867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fd9800 0x563b58fd39a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400baf0 tx=0x7f1ef4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de84688a13df5f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.920+0000 7f1ee67fc640 1 -- 172.21.15.78:0/1628954867 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef402b070 con 0x563b58fd9800 2024-08-18T23:45:42.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.920+0000 7f1ee67fc640 1 -- 172.21.15.78:0/1628954867 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ef401dab0 con 0x563b58fd9800 2024-08-18T23:45:42.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.920+0000 7f1ee67fc640 1 -- 172.21.15.78:0/1628954867 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4014040 con 0x563b58fd9800 2024-08-18T23:45:42.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.921+0000 7f1f04f3f740 1 -- 172.21.15.78:0/1628954867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fd9800 msgr2=0x563b58fd39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:42.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.921+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/1628954867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fd9800 0x563b58fd39a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400baf0 tx=0x7f1ef4009bb0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.921+0000 7f1f04f3f740 1 -- 172.21.15.78:0/1628954867 shutdown_connections 2024-08-18T23:45:42.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.921+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/1628954867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fd9800 0x563b58fd39a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.921+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/1628954867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fca9f0 0x563b58fd6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.921+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/1628954867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fca0d0 0x563b58fd8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.921+0000 7f1f04f3f740 1 -- 172.21.15.78:0/1628954867 >> 172.21.15.78:0/1628954867 conn(0x563b58fc67a0 msgr2=0x563b58fd5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:42.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.921+0000 7f1f04f3f740 1 -- 172.21.15.78:0/1628954867 shutdown_connections 2024-08-18T23:45:42.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.921+0000 7f1f04f3f740 1 -- 172.21.15.78:0/1628954867 wait complete. 2024-08-18T23:45:42.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.921+0000 7f1f04f3f740 1 -- start start 2024-08-18T23:45:42.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.921+0000 7f1f04f3f740 1 Processor -- start 2024-08-18T23:45:42.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.922+0000 7f1f04f3f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fca0d0 0x563b58fee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:42.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.922+0000 7f1f04f3f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fca9f0 0x563b58fee710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:42.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.922+0000 7f1f04f3f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fd9800 0x563b58feb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:42.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.922+0000 7f1f04f3f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc1aaf10 con 0x563b58fd9800 2024-08-18T23:45:42.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.922+0000 7f1f04f3f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc1ac380 con 0x563b58fca9f0 2024-08-18T23:45:42.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.922+0000 7f1f04f3f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc1ab5e0 con 0x563b58fca0d0 2024-08-18T23:45:42.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.922+0000 7f1f04f3f740 10 client.0 _ll_register_callbacks cb 0x7f1ef400b510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:45:42.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.922+0000 7f1efbfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fca9f0 0x563b58fee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:42.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.922+0000 7f1f010de640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fd9800 0x563b58feb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:42.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.922+0000 7f1efbfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fca9f0 0x563b58fee710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:33200/0 (socket says 172.21.15.78:33200) 2024-08-18T23:45:42.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.922+0000 7f1efbfff640 1 -- 172.21.15.78:0/1212575090 learned_addr learned my addr 172.21.15.78:0/1212575090 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:45:42.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.922+0000 7f1f008dd640 1 --2- 172.21.15.78:0/1212575090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fca0d0 0x563b58fee1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:42.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.922+0000 7f1efbfff640 1 -- 172.21.15.78:0/1212575090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fca0d0 msgr2=0x563b58fee1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:42.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.922+0000 7f1efbfff640 1 --2- 172.21.15.78:0/1212575090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fca0d0 0x563b58fee1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.922+0000 7f1efbfff640 1 -- 172.21.15.78:0/1212575090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fd9800 msgr2=0x563b58feb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:45:42.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.922+0000 7f1efbfff640 1 --2- 172.21.15.78:0/1212575090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fd9800 0x563b58feb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.922+0000 7f1efbfff640 1 -- 172.21.15.78:0/1212575090 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b58febe50 con 0x563b58fca9f0 2024-08-18T23:45:42.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.922+0000 7f1efbfff640 1 --2- 172.21.15.78:0/1212575090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fca9f0 0x563b58fee710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80314e0 tx=0x7f1ee80044f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a4225f8708e8683 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:45:42.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.922+0000 7f1edbfff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:45:42.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.923+0000 7f1edbfff640 1 -- 172.21.15.78:0/1212575090 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8035040 con 0x563b58fca9f0 2024-08-18T23:45:42.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.923+0000 7f1edbfff640 1 -- 172.21.15.78:0/1212575090 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f1ee8005450 con 0x563b58fca9f0 2024-08-18T23:45:42.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.923+0000 7f1f04f3f740 1 -- 172.21.15.78:0/1212575090 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef401dab0 con 0x563b58fca9f0 2024-08-18T23:45:42.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.923+0000 7f1edbfff640 1 -- 172.21.15.78:0/1212575090 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1ee8054050 con 0x563b58fca9f0 2024-08-18T23:45:42.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.923+0000 7f1f04f3f740 10 client.25202 fetch_fsmap learned FSMap version 26 2024-08-18T23:45:42.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.923+0000 7f1f04f3f740 1 -- 172.21.15.78:0/1212575090 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563b59053560 con 0x563b58fca9f0 2024-08-18T23:45:42.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.923+0000 7f1edbfff640 1 -- 172.21.15.78:0/1212575090 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ef401dab0 con 0x563b58fca9f0 2024-08-18T23:45:42.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.923+0000 7f1edbfff640 1 -- 172.21.15.78:0/1212575090 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f1ee8003920 con 0x563b58fca9f0 2024-08-18T23:45:42.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.923+0000 7f1f04f3f740 10 client.25202 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:45:42.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.923+0000 7f1f04f3f740 10 client.25202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:45:42.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.923+0000 7f1f04f3f740 20 client.25202 populate_metadata read hostname 'smithi078' 2024-08-18T23:45:42.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.923+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/1212575090 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563b59053d40 0x563b59056160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:45:42.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.923+0000 7f1f04f3f740 4 client.25202 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:45:42.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.923+0000 7f1f04f3f740 1 -- 172.21.15.78:0/1212575090 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b58f37400 con 0x563b59053d40 2024-08-18T23:45:42.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.923+0000 7f1f008dd640 1 --2- 172.21.15.78:0/1212575090 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563b59053d40 0x563b59056160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:45:42.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.924+0000 7f1edbfff640 10 client.25202 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:42.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.924+0000 7f1f008dd640 1 --2- 172.21.15.78:0/1212575090 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563b59053d40 0x563b59056160 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f40394fdde4ad7db server_cookie=54d9de1d03a46904 in_seq=0 out_seq=0 2024-08-18T23:45:42.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.929+0000 7f1edbfff640 1 -- 172.21.15.78:0/1212575090 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x563b58f37400 con 0x563b59053d40 2024-08-18T23:45:42.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.929+0000 7f1edbfff640 10 client.25202 handle_command_reply: tid=0 2024-08-18T23:45:42.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.929+0000 7f1f04f3f740 1 client.25202 shutdown 2024-08-18T23:45:42.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.929+0000 7f1f04f3f740 1 -- 172.21.15.78:0/1212575090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fca9f0 msgr2=0x563b58fee710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:42.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.929+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/1212575090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fca9f0 0x563b58fee710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80314e0 tx=0x7f1ee80044f0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.929+0000 7f1f04f3f740 1 -- 172.21.15.78:0/1212575090 shutdown_connections 2024-08-18T23:45:42.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.929+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/1212575090 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563b59053d40 0x563b59056160 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.929+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/1212575090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b58fd9800 0x563b58feb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.929+0000 7f1edbfff640 0 client.25202 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:45:42.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.929+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/1212575090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b58fca9f0 0x563b58fee710 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.929+0000 7f1f04f3f740 1 --2- 172.21.15.78:0/1212575090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b58fca0d0 0x563b58fee1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.929+0000 7f1f04f3f740 1 -- 172.21.15.78:0/1212575090 >> 172.21.15.78:0/1212575090 conn(0x563b58fc67a0 msgr2=0x563b58fd7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:42.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.929+0000 7f1f04f3f740 1 -- 172.21.15.78:0/1212575090 shutdown_connections 2024-08-18T23:45:42.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.929+0000 7f1f04f3f740 1 -- 172.21.15.78:0/1212575090 wait complete. 2024-08-18T23:45:42.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.929+0000 7f1f04f3f740 20 client.25202 trim_cache size 0 max 16384 2024-08-18T23:45:42.932 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:45:42.932 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "scrub active (1420 inodes in the stack)", 2024-08-18T23:45:42.932 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-08-18T23:45:42.932 INFO:teuthology.orchestra.run.smithi078.stdout: "d714a85d-b4be-401f-a08c-11368554a659": { 2024-08-18T23:45:42.932 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-08-18T23:45:42.932 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "d714a85d-b4be-401f-a08c-11368554a659", 2024-08-18T23:45:42.932 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-08-18T23:45:42.932 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:45:42.932 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:45:42.933 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:45:42.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.930+0000 7f1f02b41640 1 -- 172.21.15.78:0/2861048732 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1eec0a2b80 msgr2=0x7f1eec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:42.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.930+0000 7f1f02b41640 1 --2- 172.21.15.78:0/2861048732 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1eec0a2b80 0x7f1eec0a5040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009900 tx=0x7f1ef4009650 comp rx=0 tx=0).stop 2024-08-18T23:45:42.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.930+0000 7f1f02b41640 1 -- 172.21.15.78:0/2861048732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1efc100b50 msgr2=0x7f1efc199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:45:42.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.930+0000 7f1f02b41640 1 --2- 172.21.15.78:0/2861048732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1efc100b50 0x7f1efc199100 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000e9f0 tx=0x7f1ef000eeb0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.930+0000 7f1f02b41640 1 -- 172.21.15.78:0/2861048732 shutdown_connections 2024-08-18T23:45:42.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.930+0000 7f1f02b41640 1 --2- 172.21.15.78:0/2861048732 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1eec0a2b80 0x7f1eec0a5040 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.930+0000 7f1f02b41640 1 --2- 172.21.15.78:0/2861048732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1efc102c20 0x7f1efc1a4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.930+0000 7f1f02b41640 1 --2- 172.21.15.78:0/2861048732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1efc1022a0 0x7f1efc199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.930+0000 7f1f02b41640 1 --2- 172.21.15.78:0/2861048732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1efc100b50 0x7f1efc199100 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:45:42.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.930+0000 7f1f02b41640 1 -- 172.21.15.78:0/2861048732 >> 172.21.15.78:0/2861048732 conn(0x7f1efc0fc500 msgr2=0x7f1efc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:45:42.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.930+0000 7f1f02b41640 1 -- 172.21.15.78:0/2861048732 shutdown_connections 2024-08-18T23:45:42.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:45:42.930+0000 7f1f02b41640 1 -- 172.21.15.78:0/2861048732 wait complete. 2024-08-18T23:45:43.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:43 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:45:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:43 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:45:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:43 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:45:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:43 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:45:43.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:43 smithi123 ceph-mon[21808]: pgmap v392: 129 pgs: 129 active+clean; 55 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 319 op/s 2024-08-18T23:45:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:43 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:45:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:43 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:45:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:43 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:45:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:43 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:45:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:43 smithi161 ceph-mon[22027]: pgmap v392: 129 pgs: 129 active+clean; 55 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 319 op/s 2024-08-18T23:45:43.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:43 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:45:43.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:43 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:45:43.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:43 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:45:43.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:43 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:45:43.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:43 smithi078 ceph-mon[18440]: pgmap v392: 129 pgs: 129 active+clean; 55 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 319 op/s 2024-08-18T23:45:43.824 INFO:tasks.cephfs.filesystem:scrub status for tag:d714a85d-b4be-401f-a08c-11368554a659 - {'path': '/', 'tag': 'd714a85d-b4be-401f-a08c-11368554a659', 'options': 'recursive,force'} 2024-08-18T23:45:45.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:45 smithi123 ceph-mon[21808]: pgmap v393: 129 pgs: 129 active+clean; 65 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 436 op/s 2024-08-18T23:45:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:45 smithi161 ceph-mon[22027]: pgmap v393: 129 pgs: 129 active+clean; 65 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 436 op/s 2024-08-18T23:45:45.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:45 smithi078 ceph-mon[18440]: pgmap v393: 129 pgs: 129 active+clean; 65 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 436 op/s 2024-08-18T23:45:46.981 INFO:tasks.workunit.client.0.smithi078.stdout: 28 710612 676 418265 875 281509 1762 2024-08-18T23:45:47.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:47 smithi123 ceph-mon[21808]: pgmap v394: 129 pgs: 129 active+clean; 65 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.2 MiB/s wr, 483 op/s 2024-08-18T23:45:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:47 smithi161 ceph-mon[22027]: pgmap v394: 129 pgs: 129 active+clean; 65 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.2 MiB/s wr, 483 op/s 2024-08-18T23:45:47.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:47 smithi078 ceph-mon[18440]: pgmap v394: 129 pgs: 129 active+clean; 65 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.2 MiB/s wr, 483 op/s 2024-08-18T23:45:48.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:48 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:45:48.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:48 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:45:48.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:48 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:45:48.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:48 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:45:48.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:48 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:45:48.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:48 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:45:48.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:48 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:45:48.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:48 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:45:48.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:48 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:45:49.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:49 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:45:49.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:49 smithi123 ceph-mon[21808]: pgmap v395: 129 pgs: 129 active+clean; 69 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.5 MiB/s wr, 437 op/s 2024-08-18T23:45:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:49 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:45:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:49 smithi161 ceph-mon[22027]: pgmap v395: 129 pgs: 129 active+clean; 69 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.5 MiB/s wr, 437 op/s 2024-08-18T23:45:49.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:49 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:45:49.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:49 smithi078 ceph-mon[18440]: pgmap v395: 129 pgs: 129 active+clean; 69 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.5 MiB/s wr, 437 op/s 2024-08-18T23:45:51.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:51 smithi123 ceph-mon[21808]: pgmap v396: 129 pgs: 129 active+clean; 95 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.0 MiB/s wr, 487 op/s 2024-08-18T23:45:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:51 smithi161 ceph-mon[22027]: pgmap v396: 129 pgs: 129 active+clean; 95 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.0 MiB/s wr, 487 op/s 2024-08-18T23:45:51.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:51 smithi078 ceph-mon[18440]: pgmap v396: 129 pgs: 129 active+clean; 95 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.0 MiB/s wr, 487 op/s 2024-08-18T23:45:53.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:53 smithi123 ceph-mon[21808]: pgmap v397: 129 pgs: 129 active+clean; 135 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.3 MiB/s wr, 637 op/s 2024-08-18T23:45:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:53 smithi161 ceph-mon[22027]: pgmap v397: 129 pgs: 129 active+clean; 135 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.3 MiB/s wr, 637 op/s 2024-08-18T23:45:53.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:53 smithi078 ceph-mon[18440]: pgmap v397: 129 pgs: 129 active+clean; 135 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.3 MiB/s wr, 637 op/s 2024-08-18T23:45:55.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:55 smithi123 ceph-mon[21808]: pgmap v398: 129 pgs: 129 active+clean; 144 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 8.1 MiB/s wr, 640 op/s 2024-08-18T23:45:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:55 smithi161 ceph-mon[22027]: pgmap v398: 129 pgs: 129 active+clean; 144 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 8.1 MiB/s wr, 640 op/s 2024-08-18T23:45:55.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:55 smithi078 ceph-mon[18440]: pgmap v398: 129 pgs: 129 active+clean; 144 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 8.1 MiB/s wr, 640 op/s 2024-08-18T23:45:56.981 INFO:tasks.workunit.client.0.smithi078.stdout: 33 380651 328 226172 178 162895 834 2024-08-18T23:45:57.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:57 smithi123 ceph-mon[21808]: pgmap v399: 129 pgs: 129 active+clean; 143 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.4 MiB/s wr, 564 op/s 2024-08-18T23:45:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:57 smithi161 ceph-mon[22027]: pgmap v399: 129 pgs: 129 active+clean; 143 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.4 MiB/s wr, 564 op/s 2024-08-18T23:45:57.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:57 smithi078 ceph-mon[18440]: pgmap v399: 129 pgs: 129 active+clean; 143 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.4 MiB/s wr, 564 op/s 2024-08-18T23:45:59.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:45:59 smithi078 ceph-mon[18440]: pgmap v400: 129 pgs: 129 active+clean; 148 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.1 MiB/s wr, 550 op/s 2024-08-18T23:45:59.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:45:59 smithi123 ceph-mon[21808]: pgmap v400: 129 pgs: 129 active+clean; 148 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.1 MiB/s wr, 550 op/s 2024-08-18T23:45:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:45:59 smithi161 ceph-mon[22027]: pgmap v400: 129 pgs: 129 active+clean; 148 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.1 MiB/s wr, 550 op/s 2024-08-18T23:46:01.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:01 smithi078 ceph-mon[18440]: pgmap v401: 129 pgs: 129 active+clean; 147 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.7 MiB/s wr, 555 op/s 2024-08-18T23:46:01.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:01 smithi123 ceph-mon[21808]: pgmap v401: 129 pgs: 129 active+clean; 147 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.7 MiB/s wr, 555 op/s 2024-08-18T23:46:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:01 smithi161 ceph-mon[22027]: pgmap v401: 129 pgs: 129 active+clean; 147 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.7 MiB/s wr, 555 op/s 2024-08-18T23:46:03.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:03 smithi123 ceph-mon[21808]: pgmap v402: 129 pgs: 129 active+clean; 151 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 428 op/s 2024-08-18T23:46:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:03 smithi161 ceph-mon[22027]: pgmap v402: 129 pgs: 129 active+clean; 151 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 428 op/s 2024-08-18T23:46:03.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:03 smithi078 ceph-mon[18440]: pgmap v402: 129 pgs: 129 active+clean; 151 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 428 op/s 2024-08-18T23:46:05.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:05 smithi123 ceph-mon[21808]: pgmap v403: 129 pgs: 129 active+clean; 151 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 268 op/s 2024-08-18T23:46:05.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:05 smithi161 ceph-mon[22027]: pgmap v403: 129 pgs: 129 active+clean; 151 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 268 op/s 2024-08-18T23:46:05.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:05 smithi078 ceph-mon[18440]: pgmap v403: 129 pgs: 129 active+clean; 151 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 268 op/s 2024-08-18T23:46:06.982 INFO:tasks.workunit.client.0.smithi078.stdout: 40 1066687 482 651266 638 508117 1334 2024-08-18T23:46:07.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:07 smithi123 ceph-mon[21808]: pgmap v404: 129 pgs: 129 active+clean; 151 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-08-18T23:46:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:07 smithi161 ceph-mon[22027]: pgmap v404: 129 pgs: 129 active+clean; 151 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-08-18T23:46:07.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:07 smithi078 ceph-mon[18440]: pgmap v404: 129 pgs: 129 active+clean; 151 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-08-18T23:46:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:09 smithi161 ceph-mon[22027]: pgmap v405: 129 pgs: 129 active+clean; 160 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 199 op/s 2024-08-18T23:46:09.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:09 smithi078 ceph-mon[18440]: pgmap v405: 129 pgs: 129 active+clean; 160 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 199 op/s 2024-08-18T23:46:09.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:09 smithi123 ceph-mon[21808]: pgmap v405: 129 pgs: 129 active+clean; 160 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 199 op/s 2024-08-18T23:46:11.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:11 smithi078 ceph-mon[18440]: pgmap v406: 129 pgs: 129 active+clean; 162 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 138 op/s 2024-08-18T23:46:11.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:11 smithi161 ceph-mon[22027]: pgmap v406: 129 pgs: 129 active+clean; 162 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 138 op/s 2024-08-18T23:46:11.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:11 smithi123 ceph-mon[21808]: pgmap v406: 129 pgs: 129 active+clean; 162 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 138 op/s 2024-08-18T23:46:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:13 smithi161 ceph-mon[22027]: pgmap v407: 129 pgs: 129 active+clean; 163 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 137 op/s 2024-08-18T23:46:13.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:13 smithi078 ceph-mon[18440]: pgmap v407: 129 pgs: 129 active+clean; 163 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 137 op/s 2024-08-18T23:46:13.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:13 smithi123 ceph-mon[21808]: pgmap v407: 129 pgs: 129 active+clean; 163 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 137 op/s 2024-08-18T23:46:13.826 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:46:14.118 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:46:14.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.650+0000 7f1b05617640 1 -- 172.21.15.78:0/889629729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b001022a0 msgr2=0x7f1b001026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:14.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.650+0000 7f1b05617640 1 --2- 172.21.15.78:0/889629729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b001022a0 0x7f1b001026e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009900 tx=0x7f1af402a480 comp rx=0 tx=0).stop 2024-08-18T23:46:14.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.652+0000 7f1b05617640 1 -- 172.21.15.78:0/889629729 shutdown_connections 2024-08-18T23:46:14.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.652+0000 7f1b05617640 1 --2- 172.21.15.78:0/889629729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b00102c20 0x7f1b0010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.652+0000 7f1b05617640 1 --2- 172.21.15.78:0/889629729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b001022a0 0x7f1b001026e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.652+0000 7f1b05617640 1 --2- 172.21.15.78:0/889629729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b00100b50 0x7f1b00100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.652+0000 7f1b05617640 1 -- 172.21.15.78:0/889629729 >> 172.21.15.78:0/889629729 conn(0x7f1b000fc500 msgr2=0x7f1b000fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:14.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.653+0000 7f1b05617640 1 -- 172.21.15.78:0/889629729 shutdown_connections 2024-08-18T23:46:14.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.653+0000 7f1b05617640 1 -- 172.21.15.78:0/889629729 wait complete. 2024-08-18T23:46:14.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.653+0000 7f1b05617640 1 Processor -- start 2024-08-18T23:46:14.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.654+0000 7f1b05617640 1 -- start start 2024-08-18T23:46:14.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.654+0000 7f1b05617640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b00100b50 0x7f1b00198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:14.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.654+0000 7f1b05617640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b001022a0 0x7f1b001986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:14.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.654+0000 7f1b05617640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b00102c20 0x7f1b0019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:14.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.654+0000 7f1b05617640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00110cb0 con 0x7f1b001022a0 2024-08-18T23:46:14.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.654+0000 7f1b05617640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00110b30 con 0x7f1b00102c20 2024-08-18T23:46:14.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.654+0000 7f1b05617640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00110e30 con 0x7f1b00100b50 2024-08-18T23:46:14.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.654+0000 7f1afe7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b001022a0 0x7f1b001986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:14.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.654+0000 7f1afe7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b001022a0 0x7f1b001986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60526/0 (socket says 172.21.15.78:60526) 2024-08-18T23:46:14.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.654+0000 7f1afe7fc640 1 -- 172.21.15.78:0/1246239397 learned_addr learned my addr 172.21.15.78:0/1246239397 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:14.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.654+0000 7f1aff7fe640 1 --2- 172.21.15.78:0/1246239397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b00102c20 0x7f1b0019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:14.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.654+0000 7f1afeffd640 1 --2- 172.21.15.78:0/1246239397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b00100b50 0x7f1b00198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:14.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.654+0000 7f1afe7fc640 1 -- 172.21.15.78:0/1246239397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b00100b50 msgr2=0x7f1b00198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:14.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.654+0000 7f1afe7fc640 1 --2- 172.21.15.78:0/1246239397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b00100b50 0x7f1b00198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.654+0000 7f1afe7fc640 1 -- 172.21.15.78:0/1246239397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b00102c20 msgr2=0x7f1b0019ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:14.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.654+0000 7f1afe7fc640 1 --2- 172.21.15.78:0/1246239397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b00102c20 0x7f1b0019ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.654+0000 7f1afe7fc640 1 -- 172.21.15.78:0/1246239397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0019d140 con 0x7f1b001022a0 2024-08-18T23:46:14.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.655+0000 7f1afe7fc640 1 --2- 172.21.15.78:0/1246239397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b001022a0 0x7f1b001986d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1af4006010 tx=0x7f1af4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:14.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.655+0000 7f1ae7fff640 1 -- 172.21.15.78:0/1246239397 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af403f070 con 0x7f1b001022a0 2024-08-18T23:46:14.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.655+0000 7f1b05617640 1 -- 172.21.15.78:0/1246239397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b0019d3d0 con 0x7f1b001022a0 2024-08-18T23:46:14.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.655+0000 7f1b05617640 1 -- 172.21.15.78:0/1246239397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b001a4d10 con 0x7f1b001022a0 2024-08-18T23:46:14.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.655+0000 7f1ae7fff640 1 -- 172.21.15.78:0/1246239397 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1af4031c30 con 0x7f1b001022a0 2024-08-18T23:46:14.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.655+0000 7f1ae7fff640 1 -- 172.21.15.78:0/1246239397 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af4049060 con 0x7f1b001022a0 2024-08-18T23:46:14.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.658+0000 7f1ae7fff640 1 -- 172.21.15.78:0/1246239397 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1af4042070 con 0x7f1b001022a0 2024-08-18T23:46:14.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.658+0000 7f1b07a15740 1 Processor -- start 2024-08-18T23:46:14.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.658+0000 7f1b07a15740 1 -- start start 2024-08-18T23:46:14.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.658+0000 7f1b07a15740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac93210 0x55e42ac935f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:14.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.658+0000 7f1b07a15740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac93b30 0x55e42ac9deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:14.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.658+0000 7f1b07a15740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac9e3f0 0x55e42aca07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:14.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.658+0000 7f1b07a15740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e42abf8980 con 0x55e42ac93b30 2024-08-18T23:46:14.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.658+0000 7f1b07a15740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e42ab90a70 con 0x55e42ac9e3f0 2024-08-18T23:46:14.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.658+0000 7f1b07a15740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e42abf4940 con 0x55e42ac93210 2024-08-18T23:46:14.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.658+0000 7f1afe7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac9e3f0 0x55e42aca07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:14.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.658+0000 7f1aff7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac93210 0x55e42ac935f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:14.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.658+0000 7f1aff7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac93210 0x55e42ac935f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:58918/0 (socket says 172.21.15.78:58918) 2024-08-18T23:46:14.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.658+0000 7f1aff7fe640 1 -- 172.21.15.78:0/642134762 learned_addr learned my addr 172.21.15.78:0/642134762 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:14.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.658+0000 7f1afeffd640 1 --2- 172.21.15.78:0/642134762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac93b30 0x55e42ac9deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:14.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.659+0000 7f1aff7fe640 1 -- 172.21.15.78:0/642134762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac9e3f0 msgr2=0x55e42aca07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:14.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.659+0000 7f1aff7fe640 1 --2- 172.21.15.78:0/642134762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac9e3f0 0x55e42aca07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.659+0000 7f1aff7fe640 1 -- 172.21.15.78:0/642134762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac93b30 msgr2=0x55e42ac9deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:14.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.659+0000 7f1aff7fe640 1 --2- 172.21.15.78:0/642134762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac93b30 0x55e42ac9deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.659+0000 7f1ae7fff640 1 --2- 172.21.15.78:0/1246239397 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1ae80a2b80 0x7f1ae80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:14.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.659+0000 7f1aff7fe640 1 -- 172.21.15.78:0/642134762 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e42abf36e0 con 0x55e42ac93210 2024-08-18T23:46:14.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.659+0000 7f1ae7fff640 1 -- 172.21.15.78:0/1246239397 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f1af40ef9f0 con 0x7f1b001022a0 2024-08-18T23:46:14.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.659+0000 7f1aff7fe640 1 --2- 172.21.15.78:0/642134762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac93210 0x55e42ac935f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1af8007ec0 tx=0x7f1af800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c4bb8cb4d766c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:14.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.659+0000 7f1ae57fa640 1 -- 172.21.15.78:0/642134762 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8014c50 con 0x55e42ac93210 2024-08-18T23:46:14.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.659+0000 7f1ae57fa640 1 -- 172.21.15.78:0/642134762 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1af8014df0 con 0x55e42ac93210 2024-08-18T23:46:14.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.659+0000 7f1ae57fa640 1 -- 172.21.15.78:0/642134762 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af801b840 con 0x55e42ac93210 2024-08-18T23:46:14.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.660+0000 7f1b07a15740 1 -- 172.21.15.78:0/642134762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac93210 msgr2=0x55e42ac935f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:14.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.660+0000 7f1b07a15740 1 --2- 172.21.15.78:0/642134762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac93210 0x55e42ac935f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1af8007ec0 tx=0x7f1af800f790 comp rx=0 tx=0).stop 2024-08-18T23:46:14.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.660+0000 7f1b07a15740 1 -- 172.21.15.78:0/642134762 shutdown_connections 2024-08-18T23:46:14.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.660+0000 7f1b07a15740 1 --2- 172.21.15.78:0/642134762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac9e3f0 0x55e42aca07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.660+0000 7f1b07a15740 1 --2- 172.21.15.78:0/642134762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac93b30 0x55e42ac9deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.660+0000 7f1b07a15740 1 --2- 172.21.15.78:0/642134762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac93210 0x55e42ac935f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.660+0000 7f1b07a15740 1 -- 172.21.15.78:0/642134762 >> 172.21.15.78:0/642134762 conn(0x55e42ac8f7a0 msgr2=0x55e42ac8fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:14.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.661+0000 7f1afe7fc640 1 --2- 172.21.15.78:0/1246239397 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1ae80a2b80 0x7f1ae80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:14.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.661+0000 7f1b07a15740 1 -- 172.21.15.78:0/642134762 shutdown_connections 2024-08-18T23:46:14.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.661+0000 7f1b07a15740 1 -- 172.21.15.78:0/642134762 wait complete. 2024-08-18T23:46:14.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.661+0000 7f1afe7fc640 1 --2- 172.21.15.78:0/1246239397 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1ae80a2b80 0x7f1ae80a5040 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1af40ed4d0 tx=0x7f1af403b460 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:14.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.662+0000 7f1b07a15740 1 -- start start 2024-08-18T23:46:14.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.662+0000 7f1b07a15740 1 Processor -- start 2024-08-18T23:46:14.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.662+0000 7f1b07a15740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac93210 0x55e42ac9b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:14.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.662+0000 7f1b07a15740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac93b30 0x55e42ac9b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:14.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.662+0000 7f1b07a15740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac9be20 0x55e42acaa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:14.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.662+0000 7f1b07a15740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e42aca0fc0 con 0x55e42ac93210 2024-08-18T23:46:14.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.662+0000 7f1b07a15740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e42abb3aa0 con 0x55e42ac93b30 2024-08-18T23:46:14.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.662+0000 7f1b07a15740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e42aca1140 con 0x55e42ac9be20 2024-08-18T23:46:14.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.662+0000 7f1afeffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac93b30 0x55e42ac9b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:14.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.662+0000 7f1afeffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac93b30 0x55e42ac9b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:55564/0 (socket says 172.21.15.78:55564) 2024-08-18T23:46:14.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.662+0000 7f1afeffd640 1 -- 172.21.15.78:0/2576999805 learned_addr learned my addr 172.21.15.78:0/2576999805 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:14.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.662+0000 7f1b07a15740 10 client.0 _ll_register_callbacks cb 0x55e42aba2e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:46:14.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.663+0000 7f1afeffd640 1 -- 172.21.15.78:0/2576999805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac9be20 msgr2=0x55e42acaa430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T23:46:14.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.663+0000 7f1afeffd640 1 --2- 172.21.15.78:0/2576999805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac9be20 0x55e42acaa430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.663+0000 7f1afeffd640 1 -- 172.21.15.78:0/2576999805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac93210 msgr2=0x55e42ac9b3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T23:46:14.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.663+0000 7f1afeffd640 1 --2- 172.21.15.78:0/2576999805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac93210 0x55e42ac9b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.663+0000 7f1afeffd640 1 -- 172.21.15.78:0/2576999805 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e42ab97650 con 0x55e42ac93b30 2024-08-18T23:46:14.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.663+0000 7f1afeffd640 1 --2- 172.21.15.78:0/2576999805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac93b30 0x55e42ac9b8e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1af000df50 tx=0x7f1af00128e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76bc19f113a794b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:14.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.663+0000 7f1ae2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:46:14.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.664+0000 7f1ae2ff5640 1 -- 172.21.15.78:0/2576999805 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af001f060 con 0x55e42ac93b30 2024-08-18T23:46:14.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.664+0000 7f1ae2ff5640 1 -- 172.21.15.78:0/2576999805 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f1af0010960 con 0x55e42ac93b30 2024-08-18T23:46:14.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.664+0000 7f1b07a15740 1 -- 172.21.15.78:0/2576999805 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1af8014df0 con 0x55e42ac93b30 2024-08-18T23:46:14.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.664+0000 7f1ae2ff5640 1 -- 172.21.15.78:0/2576999805 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1af0026050 con 0x55e42ac93b30 2024-08-18T23:46:14.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.664+0000 7f1b07a15740 10 client.25214 fetch_fsmap learned FSMap version 26 2024-08-18T23:46:14.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.664+0000 7f1b07a15740 1 -- 172.21.15.78:0/2576999805 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e42a96a3c0 con 0x55e42ac93b30 2024-08-18T23:46:14.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.664+0000 7f1ae2ff5640 1 -- 172.21.15.78:0/2576999805 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1af001a3f0 con 0x55e42ac93b30 2024-08-18T23:46:14.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.665+0000 7f1ae2ff5640 1 -- 172.21.15.78:0/2576999805 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f1af0019030 con 0x55e42ac93b30 2024-08-18T23:46:14.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.665+0000 7f1b07a15740 10 client.25214 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:46:14.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.665+0000 7f1b07a15740 10 client.25214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:46:14.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.665+0000 7f1b07a15740 20 client.25214 populate_metadata read hostname 'smithi078' 2024-08-18T23:46:14.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.665+0000 7f1b07a15740 1 --2- 172.21.15.78:0/2576999805 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55e42acae480 0x55e42acb08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:14.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.665+0000 7f1b07a15740 4 client.25214 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:46:14.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.665+0000 7f1b07a15740 1 -- 172.21.15.78:0/2576999805 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1af8014df0 con 0x55e42acae480 2024-08-18T23:46:14.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.679+0000 7f1aff7fe640 1 --2- 172.21.15.78:0/2576999805 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55e42acae480 0x55e42acb08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:14.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.683+0000 7f1aff7fe640 1 --2- 172.21.15.78:0/2576999805 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55e42acae480 0x55e42acb08b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d530d8439ecc792e server_cookie=f63044e75d3a3015 in_seq=0 out_seq=0 2024-08-18T23:46:14.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.683+0000 7f1ae2ff5640 10 client.25214 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:14.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.687+0000 7f1ae2ff5640 1 -- 172.21.15.78:0/2576999805 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f1af8014df0 con 0x55e42acae480 2024-08-18T23:46:14.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.687+0000 7f1ae2ff5640 10 client.25214 handle_command_reply: tid=0 2024-08-18T23:46:14.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.693+0000 7f1b07a15740 1 client.25214 shutdown 2024-08-18T23:46:14.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.695+0000 7f1b07a15740 1 -- 172.21.15.78:0/2576999805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac93b30 msgr2=0x55e42ac9b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:14.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.695+0000 7f1b07a15740 1 --2- 172.21.15.78:0/2576999805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac93b30 0x55e42ac9b8e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1af000df50 tx=0x7f1af00128e0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.698+0000 7f1b07a15740 1 -- 172.21.15.78:0/2576999805 shutdown_connections 2024-08-18T23:46:14.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.699+0000 7f1b07a15740 1 --2- 172.21.15.78:0/2576999805 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55e42acae480 0x55e42acb08b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.699+0000 7f1b07a15740 1 --2- 172.21.15.78:0/2576999805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac9be20 0x55e42acaa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.699+0000 7f1b07a15740 1 --2- 172.21.15.78:0/2576999805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac93b30 0x55e42ac9b8e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.699+0000 7f1b07a15740 1 --2- 172.21.15.78:0/2576999805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac93210 0x55e42ac9b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.699+0000 7f1b07a15740 1 -- 172.21.15.78:0/2576999805 >> 172.21.15.78:0/2576999805 conn(0x55e42ac8f7a0 msgr2=0x55e42ac954d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:14.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.699+0000 7f1ae2ff5640 0 client.25214 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.709+0000 7f1b07a15740 1 -- 172.21.15.78:0/2576999805 shutdown_connections 2024-08-18T23:46:14.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.712+0000 7f1b07a15740 1 -- 172.21.15.78:0/2576999805 wait complete. 2024-08-18T23:46:14.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.712+0000 7f1b07a15740 20 client.25214 trim_cache size 0 max 16384 2024-08-18T23:46:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.758+0000 7f1b07a15740 1 Processor -- start 2024-08-18T23:46:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.758+0000 7f1b07a15740 1 -- start start 2024-08-18T23:46:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.759+0000 7f1b07a15740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac8f7a0 0x55e42ac9b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.759+0000 7f1b07a15740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac93210 0x55e42acaa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.759+0000 7f1b07a15740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac93b30 0x55e42acb61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.759+0000 7f1b07a15740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001af000 con 0x55e42ac93b30 2024-08-18T23:46:14.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.759+0000 7f1b07a15740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001ac8d0 con 0x55e42ac93210 2024-08-18T23:46:14.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.759+0000 7f1b07a15740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001aa720 con 0x55e42ac8f7a0 2024-08-18T23:46:14.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.759+0000 7f1afeffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac93210 0x55e42acaa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.759+0000 7f1afeffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac93210 0x55e42acaa220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:55574/0 (socket says 172.21.15.78:55574) 2024-08-18T23:46:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.759+0000 7f1afeffd640 1 -- 172.21.15.78:0/3622261250 learned_addr learned my addr 172.21.15.78:0/3622261250 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.760+0000 7f1aff7fe640 1 --2- 172.21.15.78:0/3622261250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac8f7a0 0x55e42ac9b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.760+0000 7f1aff7fe640 1 --2- 172.21.15.78:0/3622261250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac93b30 0x55e42acb61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.760+0000 7f1aff7fe640 1 -- 172.21.15.78:0/3622261250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac93210 msgr2=0x55e42acaa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.760+0000 7f1aff7fe640 1 --2- 172.21.15.78:0/3622261250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac93210 0x55e42acaa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.760+0000 7f1aff7fe640 1 -- 172.21.15.78:0/3622261250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac93b30 msgr2=0x55e42acb61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.760+0000 7f1aff7fe640 1 --2- 172.21.15.78:0/3622261250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac93b30 0x55e42acb61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.760+0000 7f1aff7fe640 1 -- 172.21.15.78:0/3622261250 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e42acb0f30 con 0x55e42ac8f7a0 2024-08-18T23:46:14.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.761+0000 7f1aff7fe640 1 --2- 172.21.15.78:0/3622261250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac8f7a0 0x55e42ac9b360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55e42acb1130 tx=0x7f1af801b440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ec2e961a1c70ab9 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:14.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.763+0000 7f1ae4ff9640 1 -- 172.21.15.78:0/3622261250 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8014c50 con 0x55e42ac8f7a0 2024-08-18T23:46:14.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.763+0000 7f1ae4ff9640 1 -- 172.21.15.78:0/3622261250 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1af8003e50 con 0x55e42ac8f7a0 2024-08-18T23:46:14.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.763+0000 7f1ae4ff9640 1 -- 172.21.15.78:0/3622261250 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af801cc20 con 0x55e42ac8f7a0 2024-08-18T23:46:14.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.766+0000 7f1b07a15740 1 -- 172.21.15.78:0/3622261250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac8f7a0 msgr2=0x55e42ac9b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:14.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.766+0000 7f1b07a15740 1 --2- 172.21.15.78:0/3622261250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac8f7a0 0x55e42ac9b360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55e42acb1130 tx=0x7f1af801b440 comp rx=0 tx=0).stop 2024-08-18T23:46:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.770+0000 7f1b07a15740 1 -- 172.21.15.78:0/3622261250 shutdown_connections 2024-08-18T23:46:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.770+0000 7f1b07a15740 1 --2- 172.21.15.78:0/3622261250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac93b30 0x55e42acb61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.770+0000 7f1b07a15740 1 --2- 172.21.15.78:0/3622261250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac93210 0x55e42acaa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.770+0000 7f1b07a15740 1 --2- 172.21.15.78:0/3622261250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac8f7a0 0x55e42ac9b360 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.770+0000 7f1b07a15740 1 -- 172.21.15.78:0/3622261250 >> 172.21.15.78:0/3622261250 conn(0x55e42ac9ba40 msgr2=0x55e42ac99c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:14.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.776+0000 7f1b07a15740 1 -- 172.21.15.78:0/3622261250 shutdown_connections 2024-08-18T23:46:14.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.778+0000 7f1b07a15740 1 -- 172.21.15.78:0/3622261250 wait complete. 2024-08-18T23:46:14.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.778+0000 7f1b07a15740 1 -- start start 2024-08-18T23:46:14.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.779+0000 7f1b07a15740 1 Processor -- start 2024-08-18T23:46:14.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.779+0000 7f1b07a15740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac8f7a0 0x55e42ac983d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.779+0000 7f1b07a15740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac93210 0x55e42ac98910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.779+0000 7f1aff7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac8f7a0 0x55e42ac983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.779+0000 7f1aff7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac8f7a0 0x55e42ac983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60576/0 (socket says 172.21.15.78:60576) 2024-08-18T23:46:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.780+0000 7f1aff7fe640 1 -- 172.21.15.78:0/3894611794 learned_addr learned my addr 172.21.15.78:0/3894611794 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.780+0000 7f1b07a15740 1 --2- 172.21.15.78:0/3894611794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac93b30 0x55e42ac98e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.780+0000 7f1b07a15740 1 -- 172.21.15.78:0/3894611794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001aad90 con 0x55e42ac8f7a0 2024-08-18T23:46:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.780+0000 7f1b07a15740 1 -- 172.21.15.78:0/3894611794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001ac200 con 0x55e42ac93b30 2024-08-18T23:46:14.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.780+0000 7f1b07a15740 1 -- 172.21.15.78:0/3894611794 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1b001ab460 con 0x55e42ac93210 2024-08-18T23:46:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.780+0000 7f1aff7fe640 1 --2- 172.21.15.78:0/3894611794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac93b30 0x55e42ac98e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.780+0000 7f1aff7fe640 1 -- 172.21.15.78:0/3894611794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac93210 msgr2=0x55e42ac98910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-18T23:46:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.780+0000 7f1aff7fe640 1 --2- 172.21.15.78:0/3894611794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac93210 0x55e42ac98910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.780+0000 7f1aff7fe640 1 -- 172.21.15.78:0/3894611794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac93b30 msgr2=0x55e42ac98e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:14.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.780+0000 7f1aff7fe640 1 --2- 172.21.15.78:0/3894611794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac93b30 0x55e42ac98e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.780+0000 7f1aff7fe640 1 -- 172.21.15.78:0/3894611794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e42ad181c0 con 0x55e42ac8f7a0 2024-08-18T23:46:14.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.781+0000 7f1aff7fe640 1 --2- 172.21.15.78:0/3894611794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac8f7a0 0x55e42ac983d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1af80077f0 tx=0x7f1af8007760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82ae9d1b211dd66 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:14.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.781+0000 7f1b07a15740 10 client.0 _ll_register_callbacks cb 0x7f1af000d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:46:14.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.781+0000 7f1b07a15740 1 -- 172.21.15.78:0/3894611794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1af8003e50 con 0x55e42ac8f7a0 2024-08-18T23:46:14.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.782+0000 7f1ae2ff5640 10 client.15735 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:46:14.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.782+0000 7f1ae2ff5640 1 -- 172.21.15.78:0/3894611794 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8036070 con 0x55e42ac8f7a0 2024-08-18T23:46:14.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.782+0000 7f1ae2ff5640 1 -- 172.21.15.78:0/3894611794 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f1af8003c40 con 0x55e42ac8f7a0 2024-08-18T23:46:14.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.782+0000 7f1ae2ff5640 1 -- 172.21.15.78:0/3894611794 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1af8046050 con 0x55e42ac8f7a0 2024-08-18T23:46:14.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.785+0000 7f1b07a15740 10 client.15735 fetch_fsmap learned FSMap version 26 2024-08-18T23:46:14.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.785+0000 7f1b07a15740 1 -- 172.21.15.78:0/3894611794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55e42ad19190 con 0x55e42ac8f7a0 2024-08-18T23:46:14.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.789+0000 7f1ae2ff5640 1 -- 172.21.15.78:0/3894611794 <== mon.0 v2:172.21.15.78:3300/0 4 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f1af8014a50 con 0x55e42ac8f7a0 2024-08-18T23:46:14.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.789+0000 7f1b07a15740 10 client.15735 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:46:14.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.789+0000 7f1b07a15740 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:46:14.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.789+0000 7f1b07a15740 20 client.15735 populate_metadata read hostname 'smithi078' 2024-08-18T23:46:14.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.789+0000 7f1b07a15740 1 --2- 172.21.15.78:0/3894611794 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55e42ac9be20 0x55e42ad1b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:14.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.789+0000 7f1b07a15740 4 client.15735 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:46:14.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.789+0000 7f1b07a15740 1 -- 172.21.15.78:0/3894611794 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e42ac05ef0 con 0x55e42ac9be20 2024-08-18T23:46:14.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.793+0000 7f1afeffd640 1 --2- 172.21.15.78:0/3894611794 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55e42ac9be20 0x55e42ad1b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:14.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.796+0000 7f1afeffd640 1 --2- 172.21.15.78:0/3894611794 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55e42ac9be20 0x55e42ad1b640 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8c222a5dfdc5467 server_cookie=3caa220b189a7b5a in_seq=0 out_seq=0 2024-08-18T23:46:14.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.799+0000 7f1ae2ff5640 10 client.15735 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.803+0000 7f1ae2ff5640 1 -- 172.21.15.78:0/3894611794 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e42ac05ef0 con 0x55e42ac9be20 2024-08-18T23:46:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.803+0000 7f1ae2ff5640 10 client.15735 handle_command_reply: tid=0 2024-08-18T23:46:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.806+0000 7f1b07a15740 1 client.15735 shutdown 2024-08-18T23:46:14.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.813+0000 7f1b07a15740 1 -- 172.21.15.78:0/3894611794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac8f7a0 msgr2=0x55e42ac983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:14.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.813+0000 7f1b07a15740 1 --2- 172.21.15.78:0/3894611794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac8f7a0 0x55e42ac983d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1af80077f0 tx=0x7f1af8007760 comp rx=0 tx=0).stop 2024-08-18T23:46:14.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.815+0000 7f1b07a15740 1 -- 172.21.15.78:0/3894611794 shutdown_connections 2024-08-18T23:46:14.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.815+0000 7f1b07a15740 1 --2- 172.21.15.78:0/3894611794 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55e42ac9be20 0x55e42ad1b640 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.815+0000 7f1b07a15740 1 --2- 172.21.15.78:0/3894611794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e42ac93b30 0x55e42ac98e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.815+0000 7f1b07a15740 1 --2- 172.21.15.78:0/3894611794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e42ac93210 0x55e42ac98910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.815+0000 7f1b07a15740 1 --2- 172.21.15.78:0/3894611794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e42ac8f7a0 0x55e42ac983d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.816+0000 7f1b07a15740 1 -- 172.21.15.78:0/3894611794 >> 172.21.15.78:0/3894611794 conn(0x55e42ac9ba40 msgr2=0x55e42ac9cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:14.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.817+0000 7f1ae2ff5640 0 client.15735 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:14.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.822+0000 7f1b07a15740 1 -- 172.21.15.78:0/3894611794 shutdown_connections 2024-08-18T23:46:14.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.822+0000 7f1b07a15740 1 -- 172.21.15.78:0/3894611794 wait complete. 2024-08-18T23:46:14.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.823+0000 7f1b07a15740 20 client.15735 trim_cache size 0 max 16384 2024-08-18T23:46:14.829 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:46:14.829 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-08-18T23:46:14.829 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-08-18T23:46:14.829 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:46:14.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.833+0000 7f1b05617640 1 -- 172.21.15.78:0/1246239397 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1ae80a2b80 msgr2=0x7f1ae80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:14.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.833+0000 7f1b05617640 1 --2- 172.21.15.78:0/1246239397 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1ae80a2b80 0x7f1ae80a5040 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1af40ed4d0 tx=0x7f1af403b460 comp rx=0 tx=0).stop 2024-08-18T23:46:14.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.833+0000 7f1b05617640 1 -- 172.21.15.78:0/1246239397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b001022a0 msgr2=0x7f1b001986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:14.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.833+0000 7f1b05617640 1 --2- 172.21.15.78:0/1246239397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b001022a0 0x7f1b001986d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1af4006010 tx=0x7f1af4003c10 comp rx=0 tx=0).stop 2024-08-18T23:46:14.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.833+0000 7f1b05617640 1 -- 172.21.15.78:0/1246239397 shutdown_connections 2024-08-18T23:46:14.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.833+0000 7f1b05617640 1 --2- 172.21.15.78:0/1246239397 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1ae80a2b80 0x7f1ae80a5040 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.833+0000 7f1b05617640 1 --2- 172.21.15.78:0/1246239397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b00102c20 0x7f1b0019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.833+0000 7f1b05617640 1 --2- 172.21.15.78:0/1246239397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b001022a0 0x7f1b001986d0 secure :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1af4006010 tx=0x7f1af4003c10 comp rx=0 tx=0).stop 2024-08-18T23:46:14.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.833+0000 7f1b05617640 1 --2- 172.21.15.78:0/1246239397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b00100b50 0x7f1b00198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:14.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.833+0000 7f1b05617640 1 -- 172.21.15.78:0/1246239397 >> 172.21.15.78:0/1246239397 conn(0x7f1b000fc500 msgr2=0x7f1b00103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:14.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.834+0000 7f1b05617640 1 -- 172.21.15.78:0/1246239397 shutdown_connections 2024-08-18T23:46:14.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:14.834+0000 7f1b05617640 1 -- 172.21.15.78:0/1246239397 wait complete. 2024-08-18T23:46:15.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:15 smithi078 ceph-mon[18440]: pgmap v408: 129 pgs: 129 active+clean; 170 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 124 op/s 2024-08-18T23:46:15.664 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-18T23:46:15.664 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:46:15.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:15 smithi161 ceph-mon[22027]: pgmap v408: 129 pgs: 129 active+clean; 170 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 124 op/s 2024-08-18T23:46:15.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:15 smithi123 ceph-mon[21808]: pgmap v408: 129 pgs: 129 active+clean; 170 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 124 op/s 2024-08-18T23:46:15.994 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:46:16.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.662+0000 7f790759e640 1 -- 172.21.15.78:0/1362089727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f790009aa50 msgr2=0x7f79000a1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:16.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.662+0000 7f790759e640 1 --2- 172.21.15.78:0/1362089727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f790009aa50 0x7f79000a1430 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7908067690 tx=0x7f7908096f70 comp rx=0 tx=0).stop 2024-08-18T23:46:16.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.662+0000 7f790759e640 1 -- 172.21.15.78:0/1362089727 shutdown_connections 2024-08-18T23:46:16.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.662+0000 7f790759e640 1 --2- 172.21.15.78:0/1362089727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f790009aa50 0x7f79000a1430 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:16.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.662+0000 7f790759e640 1 --2- 172.21.15.78:0/1362089727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7900095e70 0x7f790009a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:16.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.662+0000 7f790759e640 1 --2- 172.21.15.78:0/1362089727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79000954c0 0x7f79000958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:16.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.663+0000 7f790759e640 1 -- 172.21.15.78:0/1362089727 >> 172.21.15.78:0/1362089727 conn(0x7f7900009a10 msgr2=0x7f7900009e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:16.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.663+0000 7f790759e640 1 -- 172.21.15.78:0/1362089727 shutdown_connections 2024-08-18T23:46:16.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.663+0000 7f790759e640 1 -- 172.21.15.78:0/1362089727 wait complete. 2024-08-18T23:46:16.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.664+0000 7f790759e640 1 Processor -- start 2024-08-18T23:46:16.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.664+0000 7f790759e640 1 -- start start 2024-08-18T23:46:16.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.664+0000 7f790759e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79000954c0 0x7f79000a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:16.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.664+0000 7f790759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7900095e70 0x7f79000a6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:16.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.664+0000 7f790759e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f790009aa50 0x7f79000a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:16.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.664+0000 7f790759e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79000a47a0 con 0x7f7900095e70 2024-08-18T23:46:16.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.664+0000 7f790759e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79000a4620 con 0x7f790009aa50 2024-08-18T23:46:16.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.664+0000 7f790759e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f79000a4920 con 0x7f79000954c0 2024-08-18T23:46:16.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.664+0000 7f7905d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7900095e70 0x7f79000a6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:16.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.664+0000 7f7906d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f790009aa50 0x7f79000a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:16.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.664+0000 7f7906d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f790009aa50 0x7f79000a5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:35130/0 (socket says 172.21.15.78:35130) 2024-08-18T23:46:16.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.664+0000 7f7906d9d640 1 -- 172.21.15.78:0/3686630943 learned_addr learned my addr 172.21.15.78:0/3686630943 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:16.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.665+0000 7f790659c640 1 --2- 172.21.15.78:0/3686630943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79000954c0 0x7f79000a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:16.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.665+0000 7f7905d9b640 1 -- 172.21.15.78:0/3686630943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79000954c0 msgr2=0x7f79000a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:16.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.665+0000 7f7905d9b640 1 --2- 172.21.15.78:0/3686630943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79000954c0 0x7f79000a66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:16.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.665+0000 7f7905d9b640 1 -- 172.21.15.78:0/3686630943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f790009aa50 msgr2=0x7f79000a5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:16.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.665+0000 7f7905d9b640 1 --2- 172.21.15.78:0/3686630943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f790009aa50 0x7f79000a5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:16.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.665+0000 7f7905d9b640 1 -- 172.21.15.78:0/3686630943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79000a5a00 con 0x7f7900095e70 2024-08-18T23:46:16.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.665+0000 7f7905d9b640 1 --2- 172.21.15.78:0/3686630943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7900095e70 0x7f79000a6c20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00d6d0 tx=0x7f78fc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:16.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.665+0000 7f78f77fe640 1 -- 172.21.15.78:0/3686630943 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc017070 con 0x7f7900095e70 2024-08-18T23:46:16.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.666+0000 7f790759e640 1 -- 172.21.15.78:0/3686630943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79000a5cf0 con 0x7f7900095e70 2024-08-18T23:46:16.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.666+0000 7f78f77fe640 1 -- 172.21.15.78:0/3686630943 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78fc009820 con 0x7f7900095e70 2024-08-18T23:46:16.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.666+0000 7f78f77fe640 1 -- 172.21.15.78:0/3686630943 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc004cd0 con 0x7f7900095e70 2024-08-18T23:46:16.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.666+0000 7f790759e640 1 -- 172.21.15.78:0/3686630943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f790014fd60 con 0x7f7900095e70 2024-08-18T23:46:16.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.668+0000 7f790759e640 1 -- 172.21.15.78:0/3686630943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79000a6030 con 0x7f7900095e70 2024-08-18T23:46:16.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.670+0000 7f78f77fe640 1 -- 172.21.15.78:0/3686630943 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f78fc0099d0 con 0x7f7900095e70 2024-08-18T23:46:16.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.671+0000 7f78f77fe640 1 --2- 172.21.15.78:0/3686630943 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f78f00a2b90 0x7f78f00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:16.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.672+0000 7f78f77fe640 1 -- 172.21.15.78:0/3686630943 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f78fc0cea50 con 0x7f7900095e70 2024-08-18T23:46:16.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.672+0000 7f790659c640 1 --2- 172.21.15.78:0/3686630943 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f78f00a2b90 0x7f78f00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:16.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.673+0000 7f790659c640 1 --2- 172.21.15.78:0/3686630943 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f78f00a2b90 0x7f78f00a5050 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f78f80095f0 tx=0x7f78f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:16.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.680+0000 7f78f77fe640 1 -- 172.21.15.78:0/3686630943 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f78fc09afd0 con 0x7f7900095e70 2024-08-18T23:46:16.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.919+0000 7f790759e640 1 -- 172.21.15.78:0/3686630943 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79001501e0 con 0x7f7900095e70 2024-08-18T23:46:16.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.920+0000 7f78f77fe640 1 -- 172.21.15.78:0/3686630943 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11933 (secure 0 0 0) 0x7f78fc09f2d0 con 0x7f7900095e70 2024-08-18T23:46:16.921 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:46:16.921 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":26,"btime":"2024-08-18T23:45:33:306257+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:45:32.491054+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:46:16.922 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 26 2024-08-18T23:46:16.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.923+0000 7f790759e640 1 -- 172.21.15.78:0/3686630943 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f78f00a2b90 msgr2=0x7f78f00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:16.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.923+0000 7f790759e640 1 --2- 172.21.15.78:0/3686630943 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f78f00a2b90 0x7f78f00a5050 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f78f80095f0 tx=0x7f78f8005f90 comp rx=0 tx=0).stop 2024-08-18T23:46:16.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.923+0000 7f790759e640 1 -- 172.21.15.78:0/3686630943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7900095e70 msgr2=0x7f79000a6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:16.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.923+0000 7f790759e640 1 --2- 172.21.15.78:0/3686630943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7900095e70 0x7f79000a6c20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00d6d0 tx=0x7f78fc00db90 comp rx=0 tx=0).stop 2024-08-18T23:46:16.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.923+0000 7f790759e640 1 -- 172.21.15.78:0/3686630943 shutdown_connections 2024-08-18T23:46:16.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.923+0000 7f790759e640 1 --2- 172.21.15.78:0/3686630943 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f78f00a2b90 0x7f78f00a5050 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:16.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.923+0000 7f790759e640 1 --2- 172.21.15.78:0/3686630943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f790009aa50 0x7f79000a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:16.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.923+0000 7f790759e640 1 --2- 172.21.15.78:0/3686630943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7900095e70 0x7f79000a6c20 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:16.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.923+0000 7f790759e640 1 --2- 172.21.15.78:0/3686630943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79000954c0 0x7f79000a66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:16.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.923+0000 7f790759e640 1 -- 172.21.15.78:0/3686630943 >> 172.21.15.78:0/3686630943 conn(0x7f7900009a10 msgr2=0x7f7900092f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:16.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.924+0000 7f790759e640 1 -- 172.21.15.78:0/3686630943 shutdown_connections 2024-08-18T23:46:16.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:16.924+0000 7f790759e640 1 -- 172.21.15.78:0/3686630943 wait complete. 2024-08-18T23:46:16.981 INFO:tasks.workunit.client.0.smithi078.stdout: 49 1028159 704 675188 480 521182 1740 2024-08-18T23:46:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:17 smithi161 ceph-mon[22027]: pgmap v409: 129 pgs: 129 active+clean; 169 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 119 op/s 2024-08-18T23:46:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:17 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3686630943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:46:17.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:17 smithi078 ceph-mon[18440]: pgmap v409: 129 pgs: 129 active+clean; 169 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 119 op/s 2024-08-18T23:46:17.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:17 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3686630943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:46:17.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:17 smithi123 ceph-mon[21808]: pgmap v409: 129 pgs: 129 active+clean; 169 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 119 op/s 2024-08-18T23:46:17.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:17 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3686630943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:46:17.959 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-18T23:46:18.344 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:46:19.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.222+0000 7f3457daf640 1 -- 172.21.15.78:0/3156706084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3450109180 msgr2=0x7f345010ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:19.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.222+0000 7f3457daf640 1 --2- 172.21.15.78:0/3156706084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3450109180 0x7f345010ff80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3448009be0 tx=0x7f344802a710 comp rx=0 tx=0).stop 2024-08-18T23:46:19.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.223+0000 7f3457daf640 1 -- 172.21.15.78:0/3156706084 shutdown_connections 2024-08-18T23:46:19.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.223+0000 7f3457daf640 1 --2- 172.21.15.78:0/3156706084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f345011ae70 0x7f345011d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.223+0000 7f3457daf640 1 --2- 172.21.15.78:0/3156706084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3450109560 0x7f34501099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.223+0000 7f3457daf640 1 --2- 172.21.15.78:0/3156706084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3450109180 0x7f345010ff80 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3448009be0 tx=0x7f344802a710 comp rx=0 tx=0).stop 2024-08-18T23:46:19.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.223+0000 7f3457daf640 1 -- 172.21.15.78:0/3156706084 >> 172.21.15.78:0/3156706084 conn(0x7f345006de70 msgr2=0x7f345006e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:19.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.231+0000 7f3457daf640 1 -- 172.21.15.78:0/3156706084 shutdown_connections 2024-08-18T23:46:19.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.231+0000 7f3457daf640 1 -- 172.21.15.78:0/3156706084 wait complete. 2024-08-18T23:46:19.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.232+0000 7f3457daf640 1 Processor -- start 2024-08-18T23:46:19.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.238+0000 7f3457daf640 1 -- start start 2024-08-18T23:46:19.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.238+0000 7f3457daf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3450109560 0x7f3450118670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:19.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.238+0000 7f3457daf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345011ae70 0x7f3450111700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:19.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.238+0000 7f3457daf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3450111c40 0x7f34501120f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:19.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.238+0000 7f3457daf640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3450120b20 con 0x7f3450111c40 2024-08-18T23:46:19.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.238+0000 7f3457daf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f34501209a0 con 0x7f345011ae70 2024-08-18T23:46:19.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.238+0000 7f3457daf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3450120ca0 con 0x7f3450109560 2024-08-18T23:46:19.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.239+0000 7f345634c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3450111c40 0x7f34501120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:19.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.239+0000 7f345634c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3450111c40 0x7f34501120f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54860/0 (socket says 172.21.15.78:54860) 2024-08-18T23:46:19.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.239+0000 7f345634c640 1 -- 172.21.15.78:0/733927869 learned_addr learned my addr 172.21.15.78:0/733927869 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:19.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.240+0000 7f345634c640 1 -- 172.21.15.78:0/733927869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3450109560 msgr2=0x7f3450118670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-18T23:46:19.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.240+0000 7f345634c640 1 --2- 172.21.15.78:0/733927869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3450109560 0x7f3450118670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.240+0000 7f345634c640 1 -- 172.21.15.78:0/733927869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345011ae70 msgr2=0x7f3450111700 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-18T23:46:19.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.240+0000 7f345634c640 1 --2- 172.21.15.78:0/733927869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345011ae70 0x7f3450111700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.240+0000 7f345634c640 1 -- 172.21.15.78:0/733927869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34501162d0 con 0x7f3450111c40 2024-08-18T23:46:19.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.240+0000 7f345634c640 1 --2- 172.21.15.78:0/733927869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3450111c40 0x7f34501120f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f344c00e970 tx=0x7f344c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:19.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.244+0000 7f3446ffd640 1 -- 172.21.15.78:0/733927869 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c00ccc0 con 0x7f3450111c40 2024-08-18T23:46:19.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.244+0000 7f3446ffd640 1 -- 172.21.15.78:0/733927869 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f344c005500 con 0x7f3450111c40 2024-08-18T23:46:19.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.245+0000 7f3446ffd640 1 -- 172.21.15.78:0/733927869 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c010430 con 0x7f3450111c40 2024-08-18T23:46:19.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.245+0000 7f3457daf640 1 -- 172.21.15.78:0/733927869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34501165c0 con 0x7f3450111c40 2024-08-18T23:46:19.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.245+0000 7f3457daf640 1 -- 172.21.15.78:0/733927869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3450116a80 con 0x7f3450111c40 2024-08-18T23:46:19.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.255+0000 7f3446ffd640 1 -- 172.21.15.78:0/733927869 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f344c005020 con 0x7f3450111c40 2024-08-18T23:46:19.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.256+0000 7f3446ffd640 1 --2- 172.21.15.78:0/733927869 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f343c0a2c00 0x7f343c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:19.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.256+0000 7f3446ffd640 1 -- 172.21.15.78:0/733927869 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f344c0cf6c0 con 0x7f3450111c40 2024-08-18T23:46:19.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.260+0000 7f345a1ad740 1 Processor -- start 2024-08-18T23:46:19.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.260+0000 7f3455b4b640 1 --2- 172.21.15.78:0/733927869 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f343c0a2c00 0x7f343c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:19.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.262+0000 7f345a1ad740 1 -- start start 2024-08-18T23:46:19.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.262+0000 7f345a1ad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90dff0 0x5592cc90e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:19.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.262+0000 7f345a1ad740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc90e910 0x5592cc918c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:19.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.262+0000 7f345a1ad740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc9191d0 0x5592cc91b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:19.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.262+0000 7f345a1ad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5592cc873a40 con 0x5592cc90dff0 2024-08-18T23:46:19.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.262+0000 7f345a1ad740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5592cc80b9d0 con 0x5592cc9191d0 2024-08-18T23:46:19.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.262+0000 7f345a1ad740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5592cc86fa00 con 0x5592cc90e910 2024-08-18T23:46:19.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.263+0000 7f345534a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc90e910 0x5592cc918c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:19.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.263+0000 7f345534a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc90e910 0x5592cc918c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:49068/0 (socket says 172.21.15.78:49068) 2024-08-18T23:46:19.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.263+0000 7f345534a640 1 -- 172.21.15.78:0/337859044 learned_addr learned my addr 172.21.15.78:0/337859044 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:19.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.269+0000 7f3455b4b640 1 --2- 172.21.15.78:0/337859044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90dff0 0x5592cc90e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:19.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.269+0000 7f345534a640 1 -- 172.21.15.78:0/337859044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc9191d0 msgr2=0x5592cc91b5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-18T23:46:19.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.269+0000 7f345534a640 1 --2- 172.21.15.78:0/337859044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc9191d0 0x5592cc91b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.269+0000 7f345534a640 1 -- 172.21.15.78:0/337859044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90dff0 msgr2=0x5592cc90e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:19.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.269+0000 7f345534a640 1 --2- 172.21.15.78:0/337859044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90dff0 0x5592cc90e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.269+0000 7f345534a640 1 -- 172.21.15.78:0/337859044 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592cc812650 con 0x5592cc90e910 2024-08-18T23:46:19.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.269+0000 7f345534a640 1 --2- 172.21.15.78:0/337859044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc90e910 0x5592cc918c90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3440009b40 tx=0x7f344000e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ae5aa2a4c78539f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:19.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.276+0000 7f343bfff640 1 -- 172.21.15.78:0/337859044 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344000ed80 con 0x5592cc90e910 2024-08-18T23:46:19.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.276+0000 7f343bfff640 1 -- 172.21.15.78:0/337859044 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3440005500 con 0x5592cc90e910 2024-08-18T23:46:19.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.277+0000 7f345a1ad740 1 -- 172.21.15.78:0/337859044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc90e910 msgr2=0x5592cc918c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:19.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.277+0000 7f345a1ad740 1 --2- 172.21.15.78:0/337859044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc90e910 0x5592cc918c90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3440009b40 tx=0x7f344000e350 comp rx=0 tx=0).stop 2024-08-18T23:46:19.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.277+0000 7f3455b4b640 1 --2- 172.21.15.78:0/733927869 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f343c0a2c00 0x7f343c0a50c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3448009b30 tx=0x7f3448000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:19.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.278+0000 7f345a1ad740 1 -- 172.21.15.78:0/337859044 shutdown_connections 2024-08-18T23:46:19.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.278+0000 7f345a1ad740 1 --2- 172.21.15.78:0/337859044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc9191d0 0x5592cc91b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.278+0000 7f345a1ad740 1 --2- 172.21.15.78:0/337859044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc90e910 0x5592cc918c90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.278+0000 7f345a1ad740 1 --2- 172.21.15.78:0/337859044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90dff0 0x5592cc90e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.278+0000 7f345a1ad740 1 -- 172.21.15.78:0/337859044 >> 172.21.15.78:0/337859044 conn(0x5592cc90a580 msgr2=0x5592cc90a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:19.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.278+0000 7f345a1ad740 1 -- 172.21.15.78:0/337859044 shutdown_connections 2024-08-18T23:46:19.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.278+0000 7f345a1ad740 1 -- 172.21.15.78:0/337859044 wait complete. 2024-08-18T23:46:19.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.278+0000 7f345a1ad740 1 -- start start 2024-08-18T23:46:19.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.278+0000 7f345a1ad740 1 Processor -- start 2024-08-18T23:46:19.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.279+0000 7f345a1ad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90dff0 0x5592cc916010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:19.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.279+0000 7f345a1ad740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc90e910 0x5592cc916550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:19.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.279+0000 7f345a1ad740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc9191d0 0x5592cc916a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:19.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.279+0000 7f345a1ad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5592cc91bdf0 con 0x5592cc90dff0 2024-08-18T23:46:19.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.279+0000 7f345a1ad740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5592cc82eb20 con 0x5592cc9191d0 2024-08-18T23:46:19.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.279+0000 7f345a1ad740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5592cc91bf70 con 0x5592cc90e910 2024-08-18T23:46:19.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.279+0000 7f345a1ad740 10 client.0 _ll_register_callbacks cb 0x5592cc82cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:46:19.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.279+0000 7f3455b4b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90dff0 0x5592cc916010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:19.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.279+0000 7f3455b4b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90dff0 0x5592cc916010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54874/0 (socket says 172.21.15.78:54874) 2024-08-18T23:46:19.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.279+0000 7f3455b4b640 1 -- 172.21.15.78:0/3144015012 learned_addr learned my addr 172.21.15.78:0/3144015012 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:19.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.280+0000 7f3455b4b640 1 -- 172.21.15.78:0/3144015012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc90e910 msgr2=0x5592cc916550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:19.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.280+0000 7f3455b4b640 1 --2- 172.21.15.78:0/3144015012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc90e910 0x5592cc916550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.280+0000 7f3455b4b640 1 -- 172.21.15.78:0/3144015012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc9191d0 msgr2=0x5592cc916a90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-18T23:46:19.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.280+0000 7f3455b4b640 1 --2- 172.21.15.78:0/3144015012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc9191d0 0x5592cc916a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.280+0000 7f3455b4b640 1 -- 172.21.15.78:0/3144015012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592cc5e53c0 con 0x5592cc90dff0 2024-08-18T23:46:19.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.281+0000 7f3455b4b640 1 --2- 172.21.15.78:0/3144015012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90dff0 0x5592cc916010 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3448033000 tx=0x7f3448009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24894eded343efc server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:19.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.281+0000 7f34397fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:46:19.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.281+0000 7f34397fa640 1 -- 172.21.15.78:0/3144015012 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448045040 con 0x5592cc90dff0 2024-08-18T23:46:19.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.281+0000 7f34397fa640 1 -- 172.21.15.78:0/3144015012 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f34480316e0 con 0x5592cc90dff0 2024-08-18T23:46:19.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.282+0000 7f345a1ad740 1 -- 172.21.15.78:0/3144015012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3440005500 con 0x5592cc90dff0 2024-08-18T23:46:19.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.284+0000 7f34397fa640 1 -- 172.21.15.78:0/3144015012 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3448036040 con 0x5592cc90dff0 2024-08-18T23:46:19.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.288+0000 7f345a1ad740 10 client.15753 fetch_fsmap learned FSMap version 26 2024-08-18T23:46:19.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.288+0000 7f345a1ad740 1 -- 172.21.15.78:0/3144015012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5592cc949a10 con 0x5592cc90dff0 2024-08-18T23:46:19.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.289+0000 7f34397fa640 1 -- 172.21.15.78:0/3144015012 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3440005500 con 0x5592cc90dff0 2024-08-18T23:46:19.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.289+0000 7f34397fa640 1 -- 172.21.15.78:0/3144015012 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f344803c070 con 0x5592cc90dff0 2024-08-18T23:46:19.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.289+0000 7f345a1ad740 10 client.15753 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:46:19.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.289+0000 7f345a1ad740 10 client.15753 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:46:19.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.289+0000 7f345a1ad740 20 client.15753 populate_metadata read hostname 'smithi078' 2024-08-18T23:46:19.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.289+0000 7f345a1ad740 1 --2- 172.21.15.78:0/3144015012 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5592cc94b380 0x5592cc94d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:19.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.289+0000 7f345a1ad740 4 client.15753 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:46:19.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.289+0000 7f345a1ad740 1 -- 172.21.15.78:0/3144015012 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592cc880fb0 con 0x5592cc94b380 2024-08-18T23:46:19.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.289+0000 7f345534a640 1 --2- 172.21.15.78:0/3144015012 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5592cc94b380 0x5592cc94d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:19.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.292+0000 7f345534a640 1 --2- 172.21.15.78:0/3144015012 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5592cc94b380 0x5592cc94d840 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f581546b2598f5e4 server_cookie=f054dcc0e59fda in_seq=0 out_seq=0 2024-08-18T23:46:19.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.292+0000 7f34397fa640 10 client.15753 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:46:19.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.293+0000 7f34397fa640 1 -- 172.21.15.78:0/3144015012 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592cc880fb0 con 0x5592cc94b380 2024-08-18T23:46:19.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.294+0000 7f34397fa640 10 client.15753 handle_command_reply: tid=0 2024-08-18T23:46:19.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.294+0000 7f345a1ad740 1 client.15753 shutdown 2024-08-18T23:46:19.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.295+0000 7f345a1ad740 1 -- 172.21.15.78:0/3144015012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90dff0 msgr2=0x5592cc916010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:19.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.295+0000 7f345a1ad740 1 --2- 172.21.15.78:0/3144015012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90dff0 0x5592cc916010 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3448033000 tx=0x7f3448009500 comp rx=0 tx=0).stop 2024-08-18T23:46:19.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.297+0000 7f345a1ad740 1 -- 172.21.15.78:0/3144015012 shutdown_connections 2024-08-18T23:46:19.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.297+0000 7f345a1ad740 1 --2- 172.21.15.78:0/3144015012 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5592cc94b380 0x5592cc94d840 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.297+0000 7f345a1ad740 1 --2- 172.21.15.78:0/3144015012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc9191d0 0x5592cc916a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.297+0000 7f345a1ad740 1 --2- 172.21.15.78:0/3144015012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc90e910 0x5592cc916550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.297+0000 7f345a1ad740 1 --2- 172.21.15.78:0/3144015012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90dff0 0x5592cc916010 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.297+0000 7f345a1ad740 1 -- 172.21.15.78:0/3144015012 >> 172.21.15.78:0/3144015012 conn(0x5592cc90a580 msgr2=0x5592cc9102b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:19.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.297+0000 7f34397fa640 0 client.15753 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:46:19.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.297+0000 7f345a1ad740 1 -- 172.21.15.78:0/3144015012 shutdown_connections 2024-08-18T23:46:19.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.297+0000 7f345a1ad740 1 -- 172.21.15.78:0/3144015012 wait complete. 2024-08-18T23:46:19.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.297+0000 7f345a1ad740 20 client.15753 trim_cache size 0 max 16384 2024-08-18T23:46:19.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.326+0000 7f345a1ad740 1 Processor -- start 2024-08-18T23:46:19.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.326+0000 7f345a1ad740 1 -- start start 2024-08-18T23:46:19.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.327+0000 7f345a1ad740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc90dff0 0x5592cc953150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:19.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.327+0000 7f345a1ad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90e910 0x5592cc953690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:19.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.327+0000 7f345a1ad740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc9191d0 0x5592cc914920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:19.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.328+0000 7f3455b4b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc90dff0 0x5592cc953150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:19.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.328+0000 7f3455b4b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc90dff0 0x5592cc953150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:35192/0 (socket says 172.21.15.78:35192) 2024-08-18T23:46:19.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.328+0000 7f3455b4b640 1 -- 172.21.15.78:0/136344086 learned_addr learned my addr 172.21.15.78:0/136344086 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:19.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.328+0000 7f345634c640 1 --2- 172.21.15.78:0/136344086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc9191d0 0x5592cc914920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:19.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.328+0000 7f345534a640 1 --2- 172.21.15.78:0/136344086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90e910 0x5592cc953690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:19.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.328+0000 7f345a1ad740 1 -- 172.21.15.78:0/136344086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f345007fee0 con 0x5592cc90e910 2024-08-18T23:46:19.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.329+0000 7f345a1ad740 1 -- 172.21.15.78:0/136344086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f345007d7b0 con 0x5592cc90dff0 2024-08-18T23:46:19.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.329+0000 7f345a1ad740 1 -- 172.21.15.78:0/136344086 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f345007b600 con 0x5592cc9191d0 2024-08-18T23:46:19.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.331+0000 7f3455b4b640 1 -- 172.21.15.78:0/136344086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc9191d0 msgr2=0x5592cc914920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:19.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.331+0000 7f3455b4b640 1 --2- 172.21.15.78:0/136344086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc9191d0 0x5592cc914920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.331+0000 7f3455b4b640 1 -- 172.21.15.78:0/136344086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90e910 msgr2=0x5592cc953690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:19.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.331+0000 7f3455b4b640 1 --2- 172.21.15.78:0/136344086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90e910 0x5592cc953690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.331+0000 7f3455b4b640 1 -- 172.21.15.78:0/136344086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592cc90b6f0 con 0x5592cc90dff0 2024-08-18T23:46:19.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.331+0000 7f3455b4b640 1 --2- 172.21.15.78:0/136344086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc90dff0 0x5592cc953150 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f34480315e0 tx=0x7f34480307c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f605e7062304673 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:19.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.332+0000 7f343b7fe640 1 -- 172.21.15.78:0/136344086 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448045070 con 0x5592cc90dff0 2024-08-18T23:46:19.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.332+0000 7f345634c640 1 --2- 172.21.15.78:0/136344086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc9191d0 0x5592cc914920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:46:19.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.332+0000 7f343b7fe640 1 -- 172.21.15.78:0/136344086 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f344804e070 con 0x5592cc90dff0 2024-08-18T23:46:19.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.333+0000 7f343b7fe640 1 -- 172.21.15.78:0/136344086 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34480536b0 con 0x5592cc90dff0 2024-08-18T23:46:19.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.333+0000 7f345534a640 1 --2- 172.21.15.78:0/136344086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90e910 0x5592cc953690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:46:19.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.333+0000 7f345a1ad740 1 -- 172.21.15.78:0/136344086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc90dff0 msgr2=0x5592cc953150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:19.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.333+0000 7f345a1ad740 1 --2- 172.21.15.78:0/136344086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc90dff0 0x5592cc953150 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f34480315e0 tx=0x7f34480307c0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.333+0000 7f345a1ad740 1 -- 172.21.15.78:0/136344086 shutdown_connections 2024-08-18T23:46:19.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.333+0000 7f345a1ad740 1 --2- 172.21.15.78:0/136344086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc9191d0 0x5592cc914920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.333+0000 7f345a1ad740 1 --2- 172.21.15.78:0/136344086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90e910 0x5592cc953690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.333+0000 7f345a1ad740 1 --2- 172.21.15.78:0/136344086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc90dff0 0x5592cc953150 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.333+0000 7f345a1ad740 1 -- 172.21.15.78:0/136344086 >> 172.21.15.78:0/136344086 conn(0x5592cc90a580 msgr2=0x5592cc911370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:19.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.333+0000 7f345a1ad740 1 -- 172.21.15.78:0/136344086 shutdown_connections 2024-08-18T23:46:19.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.333+0000 7f345a1ad740 1 -- 172.21.15.78:0/136344086 wait complete. 2024-08-18T23:46:19.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.334+0000 7f345a1ad740 1 -- start start 2024-08-18T23:46:19.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.334+0000 7f345a1ad740 1 Processor -- start 2024-08-18T23:46:19.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.335+0000 7f345a1ad740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc90dff0 0x5592cc9138b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:19.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.335+0000 7f345a1ad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90e910 0x5592cc921210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:19.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.335+0000 7f3455b4b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc90dff0 0x5592cc9138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:19.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.335+0000 7f345a1ad740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc9191d0 0x5592cc921750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:19.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.335+0000 7f345a1ad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f345007bc70 con 0x5592cc90e910 2024-08-18T23:46:19.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.335+0000 7f345a1ad740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f345007d0e0 con 0x5592cc9191d0 2024-08-18T23:46:19.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.335+0000 7f345a1ad740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f345007c340 con 0x5592cc90dff0 2024-08-18T23:46:19.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.335+0000 7f345a1ad740 10 client.0 _ll_register_callbacks cb 0x7f3448042040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:46:19.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.335+0000 7f3455b4b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc90dff0 0x5592cc9138b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:49092/0 (socket says 172.21.15.78:49092) 2024-08-18T23:46:19.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.335+0000 7f3455b4b640 1 -- 172.21.15.78:0/1117135337 learned_addr learned my addr 172.21.15.78:0/1117135337 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:19.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.335+0000 7f345634c640 1 --2- 172.21.15.78:0/1117135337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc9191d0 0x5592cc921750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:19.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.336+0000 7f3455b4b640 1 -- 172.21.15.78:0/1117135337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc9191d0 msgr2=0x5592cc921750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:19.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.336+0000 7f3455b4b640 1 --2- 172.21.15.78:0/1117135337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc9191d0 0x5592cc921750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.336+0000 7f3455b4b640 1 -- 172.21.15.78:0/1117135337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90e910 msgr2=0x5592cc921210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:19.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.336+0000 7f3455b4b640 1 --2- 172.21.15.78:0/1117135337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90e910 0x5592cc921210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.336+0000 7f3455b4b640 1 -- 172.21.15.78:0/1117135337 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592cc94dbb0 con 0x5592cc90dff0 2024-08-18T23:46:19.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.336+0000 7f3455b4b640 1 --2- 172.21.15.78:0/1117135337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc90dff0 0x5592cc9138b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f344802ae60 tx=0x7f3448030fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25487aa5ddb6cde1 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:19.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.337+0000 7f34397fa640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:46:19.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.338+0000 7f34397fa640 1 -- 172.21.15.78:0/1117135337 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448045070 con 0x5592cc90dff0 2024-08-18T23:46:19.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.338+0000 7f34397fa640 1 -- 172.21.15.78:0/1117135337 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f3448053450 con 0x5592cc90dff0 2024-08-18T23:46:19.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.340+0000 7f345a1ad740 1 -- 172.21.15.78:0/1117135337 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f344804e070 con 0x5592cc90dff0 2024-08-18T23:46:19.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.341+0000 7f34397fa640 1 -- 172.21.15.78:0/1117135337 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3448053650 con 0x5592cc90dff0 2024-08-18T23:46:19.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.341+0000 7f345a1ad740 10 client.25138 fetch_fsmap learned FSMap version 26 2024-08-18T23:46:19.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.341+0000 7f345a1ad740 1 -- 172.21.15.78:0/1117135337 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5592cc912fa0 con 0x5592cc90dff0 2024-08-18T23:46:19.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.342+0000 7f34397fa640 1 -- 172.21.15.78:0/1117135337 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f344804e070 con 0x5592cc90dff0 2024-08-18T23:46:19.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.342+0000 7f34397fa640 1 -- 172.21.15.78:0/1117135337 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f344803c040 con 0x5592cc90dff0 2024-08-18T23:46:19.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.342+0000 7f345a1ad740 10 client.25138 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:46:19.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.342+0000 7f345a1ad740 10 client.25138 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:46:19.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.342+0000 7f345a1ad740 20 client.25138 populate_metadata read hostname 'smithi078' 2024-08-18T23:46:19.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.342+0000 7f345a1ad740 1 --2- 172.21.15.78:0/1117135337 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5592cc9b5470 0x5592cc9b7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:19.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.342+0000 7f345a1ad740 4 client.25138 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:46:19.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.342+0000 7f345a1ad740 1 -- 172.21.15.78:0/1117135337 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5592cc87b4c0 con 0x5592cc9b5470 2024-08-18T23:46:19.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.344+0000 7f345534a640 1 --2- 172.21.15.78:0/1117135337 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5592cc9b5470 0x5592cc9b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:19.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.361+0000 7f34397fa640 10 client.25138 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:46:19.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.361+0000 7f345534a640 1 --2- 172.21.15.78:0/1117135337 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5592cc9b5470 0x5592cc9b7890 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c548f318db7f296f server_cookie=4f6d03d63e7af5ed in_seq=0 out_seq=0 2024-08-18T23:46:19.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.362+0000 7f34397fa640 1 -- 172.21.15.78:0/1117135337 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5592cc87b4c0 con 0x5592cc9b5470 2024-08-18T23:46:19.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.362+0000 7f34397fa640 10 client.25138 handle_command_reply: tid=0 2024-08-18T23:46:19.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.362+0000 7f345a1ad740 1 client.25138 shutdown 2024-08-18T23:46:19.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.363+0000 7f345a1ad740 1 -- 172.21.15.78:0/1117135337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc90dff0 msgr2=0x5592cc9138b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:19.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.363+0000 7f345a1ad740 1 --2- 172.21.15.78:0/1117135337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc90dff0 0x5592cc9138b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f344802ae60 tx=0x7f3448030fd0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.364+0000 7f345a1ad740 1 -- 172.21.15.78:0/1117135337 shutdown_connections 2024-08-18T23:46:19.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.364+0000 7f345a1ad740 1 --2- 172.21.15.78:0/1117135337 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5592cc9b5470 0x5592cc9b7890 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.364+0000 7f345a1ad740 1 --2- 172.21.15.78:0/1117135337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592cc9191d0 0x5592cc921750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.364+0000 7f345a1ad740 1 --2- 172.21.15.78:0/1117135337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592cc90e910 0x5592cc921210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.364+0000 7f345a1ad740 1 --2- 172.21.15.78:0/1117135337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5592cc90dff0 0x5592cc9138b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.364+0000 7f345a1ad740 1 -- 172.21.15.78:0/1117135337 >> 172.21.15.78:0/1117135337 conn(0x5592cc90a580 msgr2=0x5592cc914920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:19.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.364+0000 7f34397fa640 0 client.25138 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:46:19.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.365+0000 7f345a1ad740 1 -- 172.21.15.78:0/1117135337 shutdown_connections 2024-08-18T23:46:19.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.365+0000 7f345a1ad740 1 -- 172.21.15.78:0/1117135337 wait complete. 2024-08-18T23:46:19.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.365+0000 7f345a1ad740 20 client.25138 trim_cache size 0 max 16384 2024-08-18T23:46:19.368 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:46:19.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.371+0000 7f3457daf640 1 -- 172.21.15.78:0/733927869 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f343c0a2c00 msgr2=0x7f343c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:19.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.371+0000 7f3457daf640 1 --2- 172.21.15.78:0/733927869 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f343c0a2c00 0x7f343c0a50c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3448009b30 tx=0x7f3448000f40 comp rx=0 tx=0).stop 2024-08-18T23:46:19.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.371+0000 7f3457daf640 1 -- 172.21.15.78:0/733927869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3450111c40 msgr2=0x7f34501120f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:19.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.371+0000 7f3457daf640 1 --2- 172.21.15.78:0/733927869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3450111c40 0x7f34501120f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f344c00e970 tx=0x7f344c00ee30 comp rx=0 tx=0).stop 2024-08-18T23:46:19.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.372+0000 7f3457daf640 1 -- 172.21.15.78:0/733927869 shutdown_connections 2024-08-18T23:46:19.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.372+0000 7f3457daf640 1 --2- 172.21.15.78:0/733927869 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f343c0a2c00 0x7f343c0a50c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.372+0000 7f3457daf640 1 --2- 172.21.15.78:0/733927869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3450111c40 0x7f34501120f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.372+0000 7f3457daf640 1 --2- 172.21.15.78:0/733927869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345011ae70 0x7f3450111700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.372+0000 7f3457daf640 1 --2- 172.21.15.78:0/733927869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3450109560 0x7f3450118670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:19.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.372+0000 7f3457daf640 1 -- 172.21.15.78:0/733927869 >> 172.21.15.78:0/733927869 conn(0x7f345006de70 msgr2=0x7f345011bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:19.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.373+0000 7f3457daf640 1 -- 172.21.15.78:0/733927869 shutdown_connections 2024-08-18T23:46:19.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:19.373+0000 7f3457daf640 1 -- 172.21.15.78:0/733927869 wait complete. 2024-08-18T23:46:19.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:19 smithi078 ceph-mon[18440]: pgmap v410: 129 pgs: 129 active+clean; 170 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 121 op/s 2024-08-18T23:46:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:19 smithi161 ceph-mon[22027]: pgmap v410: 129 pgs: 129 active+clean; 170 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 121 op/s 2024-08-18T23:46:19.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:19 smithi123 ceph-mon[21808]: pgmap v410: 129 pgs: 129 active+clean; 170 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 121 op/s 2024-08-18T23:46:20.151 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-18T23:46:20.552 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:46:21.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.078+0000 7f7d585e5640 1 -- 172.21.15.78:0/2451966715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d50103470 msgr2=0x7f7d501078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:21.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.078+0000 7f7d585e5640 1 --2- 172.21.15.78:0/2451966715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d50103470 0x7f7d501078b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009930 tx=0x7f7d4002a420 comp rx=0 tx=0).stop 2024-08-18T23:46:21.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.078+0000 7f7d585e5640 1 -- 172.21.15.78:0/2451966715 shutdown_connections 2024-08-18T23:46:21.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.078+0000 7f7d585e5640 1 --2- 172.21.15.78:0/2451966715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d50107fc0 0x7f7d5010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.078+0000 7f7d585e5640 1 --2- 172.21.15.78:0/2451966715 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d50103470 0x7f7d501078b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.078+0000 7f7d585e5640 1 --2- 172.21.15.78:0/2451966715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d50102ac0 0x7f7d50102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.078+0000 7f7d585e5640 1 -- 172.21.15.78:0/2451966715 >> 172.21.15.78:0/2451966715 conn(0x7f7d500fc500 msgr2=0x7f7d500fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:21.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.078+0000 7f7d585e5640 1 -- 172.21.15.78:0/2451966715 shutdown_connections 2024-08-18T23:46:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.078+0000 7f7d585e5640 1 -- 172.21.15.78:0/2451966715 wait complete. 2024-08-18T23:46:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.079+0000 7f7d585e5640 1 Processor -- start 2024-08-18T23:46:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.079+0000 7f7d585e5640 1 -- start start 2024-08-18T23:46:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.079+0000 7f7d585e5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d50102ac0 0x7f7d50198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.079+0000 7f7d585e5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d50103470 0x7f7d501988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.079+0000 7f7d585e5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d50107fc0 0x7f7d5019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.079+0000 7f7d585e5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50111dc0 con 0x7f7d50103470 2024-08-18T23:46:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.079+0000 7f7d585e5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50111c40 con 0x7f7d50102ac0 2024-08-18T23:46:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.079+0000 7f7d585e5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50111f40 con 0x7f7d50107fc0 2024-08-18T23:46:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.079+0000 7f7d55b80640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d50103470 0x7f7d501988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.079+0000 7f7d56381640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d50102ac0 0x7f7d50198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.079+0000 7f7d55b80640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d50103470 0x7f7d501988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54906/0 (socket says 172.21.15.78:54906) 2024-08-18T23:46:21.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.079+0000 7f7d56381640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d50102ac0 0x7f7d50198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:35208/0 (socket says 172.21.15.78:35208) 2024-08-18T23:46:21.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.080+0000 7f7d55b80640 1 -- 172.21.15.78:0/3027597516 learned_addr learned my addr 172.21.15.78:0/3027597516 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:21.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.080+0000 7f7d56b82640 1 --2- 172.21.15.78:0/3027597516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d50107fc0 0x7f7d5019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:21.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.080+0000 7f7d56381640 1 -- 172.21.15.78:0/3027597516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d50107fc0 msgr2=0x7f7d5019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:21.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.080+0000 7f7d56381640 1 --2- 172.21.15.78:0/3027597516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d50107fc0 0x7f7d5019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.080+0000 7f7d56381640 1 -- 172.21.15.78:0/3027597516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d50103470 msgr2=0x7f7d501988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:21.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.080+0000 7f7d56381640 1 --2- 172.21.15.78:0/3027597516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d50103470 0x7f7d501988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.080+0000 7f7d56381640 1 -- 172.21.15.78:0/3027597516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5019d330 con 0x7f7d50102ac0 2024-08-18T23:46:21.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.080+0000 7f7d55b80640 1 --2- 172.21.15.78:0/3027597516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d50103470 0x7f7d501988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:46:21.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.080+0000 7f7d56381640 1 --2- 172.21.15.78:0/3027597516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d50102ac0 0x7f7d50198380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00e9f0 tx=0x7f7d4c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:21.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.080+0000 7f7d477fe640 1 -- 172.21.15.78:0/3027597516 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c00ccc0 con 0x7f7d50102ac0 2024-08-18T23:46:21.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.080+0000 7f7d477fe640 1 -- 172.21.15.78:0/3027597516 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d4c010910 con 0x7f7d50102ac0 2024-08-18T23:46:21.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.080+0000 7f7d585e5640 1 -- 172.21.15.78:0/3027597516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5019d620 con 0x7f7d50102ac0 2024-08-18T23:46:21.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.080+0000 7f7d477fe640 1 -- 172.21.15.78:0/3027597516 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c018960 con 0x7f7d50102ac0 2024-08-18T23:46:21.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.080+0000 7f7d585e5640 1 -- 172.21.15.78:0/3027597516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d501a4f00 con 0x7f7d50102ac0 2024-08-18T23:46:21.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.082+0000 7f7d477fe640 1 -- 172.21.15.78:0/3027597516 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7d4c010ac0 con 0x7f7d50102ac0 2024-08-18T23:46:21.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.083+0000 7f7d5a9e3740 1 Processor -- start 2024-08-18T23:46:21.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.083+0000 7f7d5a9e3740 1 -- start start 2024-08-18T23:46:21.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.083+0000 7f7d5a9e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f3ff0 0x56541c9f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:21.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.083+0000 7f7d5a9e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9f4910 0x56541c9fec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:21.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.083+0000 7f7d5a9e3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9ff1d0 0x56541ca015c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:21.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.083+0000 7f7d5a9e3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56541c959a40 con 0x56541c9f3ff0 2024-08-18T23:46:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.083+0000 7f7d5a9e3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56541c8f19d0 con 0x56541c9ff1d0 2024-08-18T23:46:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.083+0000 7f7d5a9e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56541c955a00 con 0x56541c9f4910 2024-08-18T23:46:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.083+0000 7f7d56b82640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f3ff0 0x56541c9f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.083+0000 7f7d56b82640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f3ff0 0x56541c9f43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54920/0 (socket says 172.21.15.78:54920) 2024-08-18T23:46:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.083+0000 7f7d56b82640 1 -- 172.21.15.78:0/2307956186 learned_addr learned my addr 172.21.15.78:0/2307956186 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.083+0000 7f7d55b80640 1 --2- 172.21.15.78:0/2307956186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9ff1d0 0x56541ca015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.083+0000 7f7d477fe640 1 --2- 172.21.15.78:0/3027597516 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7d3c0a2b80 0x7f7d3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.083+0000 7f7d56381640 1 --2- 172.21.15.78:0/2307956186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9f4910 0x56541c9fec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.084+0000 7f7d56b82640 1 --2- 172.21.15.78:0/3027597516 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7d3c0a2b80 0x7f7d3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.084+0000 7f7d55b80640 1 -- 172.21.15.78:0/2307956186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9f4910 msgr2=0x56541c9fec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.084+0000 7f7d55b80640 1 --2- 172.21.15.78:0/2307956186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9f4910 0x56541c9fec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.084+0000 7f7d55b80640 1 -- 172.21.15.78:0/2307956186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f3ff0 msgr2=0x56541c9f43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.084+0000 7f7d55b80640 1 --2- 172.21.15.78:0/2307956186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f3ff0 0x56541c9f43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.084+0000 7f7d55b80640 1 -- 172.21.15.78:0/2307956186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56541c8f8650 con 0x56541c9ff1d0 2024-08-18T23:46:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.084+0000 7f7d56b82640 1 --2- 172.21.15.78:0/2307956186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f3ff0 0x56541c9f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:46:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.084+0000 7f7d477fe640 1 -- 172.21.15.78:0/3027597516 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f7d4c0cf7c0 con 0x7f7d50102ac0 2024-08-18T23:46:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.084+0000 7f7d55b80640 1 --2- 172.21.15.78:0/2307956186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9ff1d0 0x56541ca015c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009900 tx=0x7f7d400315b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41b281add5f68ea3 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.084+0000 7f7d56b82640 1 --2- 172.21.15.78:0/3027597516 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7d3c0a2b80 0x7f7d3c0a5040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800afd0 tx=0x7f7d48004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.084+0000 7f7d44ff9640 1 -- 172.21.15.78:0/2307956186 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40035040 con 0x56541c9ff1d0 2024-08-18T23:46:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.084+0000 7f7d44ff9640 1 -- 172.21.15.78:0/2307956186 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d40005020 con 0x56541c9ff1d0 2024-08-18T23:46:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.084+0000 7f7d44ff9640 1 -- 172.21.15.78:0/2307956186 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40004a90 con 0x56541c9ff1d0 2024-08-18T23:46:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.085+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2307956186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9ff1d0 msgr2=0x56541ca015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.085+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/2307956186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9ff1d0 0x56541ca015c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009900 tx=0x7f7d400315b0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.085+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2307956186 shutdown_connections 2024-08-18T23:46:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.085+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/2307956186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9ff1d0 0x56541ca015c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.085+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/2307956186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9f4910 0x56541c9fec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.085+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/2307956186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f3ff0 0x56541c9f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.085+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2307956186 >> 172.21.15.78:0/2307956186 conn(0x56541c9f0580 msgr2=0x56541c9f0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:21.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.085+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2307956186 shutdown_connections 2024-08-18T23:46:21.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.085+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2307956186 wait complete. 2024-08-18T23:46:21.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.085+0000 7f7d5a9e3740 1 -- start start 2024-08-18T23:46:21.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.085+0000 7f7d5a9e3740 1 Processor -- start 2024-08-18T23:46:21.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d5a9e3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9f3ff0 0x56541c9fec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:21.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d5a9e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f4910 0x56541c9fbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:21.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d5a9e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9ff1d0 0x56541c9fc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:21.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d5a9e3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56541ca01da0 con 0x56541c9f4910 2024-08-18T23:46:21.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d5a9e3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56541c914b20 con 0x56541c9f3ff0 2024-08-18T23:46:21.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d5a9e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56541ca01f20 con 0x56541c9ff1d0 2024-08-18T23:46:21.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d5a9e3740 10 client.0 _ll_register_callbacks cb 0x56541c912bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:46:21.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d55b80640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f4910 0x56541c9fbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:21.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d56381640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9f3ff0 0x56541c9fec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:21.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d56381640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9f3ff0 0x56541c9fec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:35230/0 (socket says 172.21.15.78:35230) 2024-08-18T23:46:21.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d55b80640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f4910 0x56541c9fbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54932/0 (socket says 172.21.15.78:54932) 2024-08-18T23:46:21.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d56381640 1 -- 172.21.15.78:0/2197604613 learned_addr learned my addr 172.21.15.78:0/2197604613 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:21.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d56b82640 1 --2- 172.21.15.78:0/2197604613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9ff1d0 0x56541c9fc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:21.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d56381640 1 -- 172.21.15.78:0/2197604613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9ff1d0 msgr2=0x56541c9fc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:21.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d56381640 1 --2- 172.21.15.78:0/2197604613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9ff1d0 0x56541c9fc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d56381640 1 -- 172.21.15.78:0/2197604613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f4910 msgr2=0x56541c9fbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:21.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d56381640 1 --2- 172.21.15.78:0/2197604613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f4910 0x56541c9fbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d56381640 1 -- 172.21.15.78:0/2197604613 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56541c6cb3c0 con 0x56541c9f3ff0 2024-08-18T23:46:21.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d55b80640 1 --2- 172.21.15.78:0/2197604613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f4910 0x56541c9fbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:46:21.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d56381640 1 --2- 172.21.15.78:0/2197604613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9f3ff0 0x56541c9fec60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c0986b0 tx=0x7f7d4c09b640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3b78e3e20dc4194 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:21.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:46:21.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d39ffb640 1 -- 172.21.15.78:0/2197604613 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c0a48a0 con 0x56541c9f3ff0 2024-08-18T23:46:21.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d39ffb640 1 -- 172.21.15.78:0/2197604613 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f7d4c0cfbb0 con 0x56541c9f3ff0 2024-08-18T23:46:21.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.086+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2197604613 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d40005020 con 0x56541c9f3ff0 2024-08-18T23:46:21.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.087+0000 7f7d39ffb640 1 -- 172.21.15.78:0/2197604613 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7d4c0cfdb0 con 0x56541c9f3ff0 2024-08-18T23:46:21.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.087+0000 7f7d5a9e3740 10 client.25247 fetch_fsmap learned FSMap version 26 2024-08-18T23:46:21.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.087+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2197604613 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56541ca0d8b0 con 0x56541c9f3ff0 2024-08-18T23:46:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.087+0000 7f7d39ffb640 1 -- 172.21.15.78:0/2197604613 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d40005020 con 0x56541c9f3ff0 2024-08-18T23:46:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.087+0000 7f7d39ffb640 1 -- 172.21.15.78:0/2197604613 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f7d4c018cc0 con 0x56541c9f3ff0 2024-08-18T23:46:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.087+0000 7f7d5a9e3740 10 client.25247 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:46:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.087+0000 7f7d5a9e3740 10 client.25247 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:46:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.087+0000 7f7d5a9e3740 20 client.25247 populate_metadata read hostname 'smithi078' 2024-08-18T23:46:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.087+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/2197604613 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56541ca0ed70 0x56541ca111e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.087+0000 7f7d5a9e3740 4 client.25247 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:46:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.087+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2197604613 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56541c966fb0 con 0x56541ca0ed70 2024-08-18T23:46:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.087+0000 7f7d55b80640 1 --2- 172.21.15.78:0/2197604613 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56541ca0ed70 0x56541ca111e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.088+0000 7f7d39ffb640 10 client.25247 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:46:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.088+0000 7f7d55b80640 1 --2- 172.21.15.78:0/2197604613 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56541ca0ed70 0x56541ca111e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f30de33edd50b8a9 server_cookie=1e366e29961bb6c9 in_seq=0 out_seq=0 2024-08-18T23:46:21.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.088+0000 7f7d39ffb640 1 -- 172.21.15.78:0/2197604613 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56541c966fb0 con 0x56541ca0ed70 2024-08-18T23:46:21.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.088+0000 7f7d39ffb640 10 client.25247 handle_command_reply: tid=0 2024-08-18T23:46:21.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.088+0000 7f7d5a9e3740 1 client.25247 shutdown 2024-08-18T23:46:21.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.088+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2197604613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9f3ff0 msgr2=0x56541c9fec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:21.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.088+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/2197604613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9f3ff0 0x56541c9fec60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c0986b0 tx=0x7f7d4c09b640 comp rx=0 tx=0).stop 2024-08-18T23:46:21.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.088+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2197604613 shutdown_connections 2024-08-18T23:46:21.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.088+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/2197604613 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56541ca0ed70 0x56541ca111e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.088+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/2197604613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9ff1d0 0x56541c9fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.088+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/2197604613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f4910 0x56541c9fbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.088+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/2197604613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9f3ff0 0x56541c9fec60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.088+0000 7f7d39ffb640 0 client.25247 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:46:21.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.088+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2197604613 >> 172.21.15.78:0/2197604613 conn(0x56541c9f0580 msgr2=0x56541c9f62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:21.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.089+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2197604613 shutdown_connections 2024-08-18T23:46:21.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.089+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2197604613 wait complete. 2024-08-18T23:46:21.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.089+0000 7f7d5a9e3740 20 client.25247 trim_cache size 0 max 16384 2024-08-18T23:46:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.101+0000 7f7d5a9e3740 1 Processor -- start 2024-08-18T23:46:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.101+0000 7f7d5a9e3740 1 -- start start 2024-08-18T23:46:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.101+0000 7f7d5a9e3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9f3ff0 0x56541c9fbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.101+0000 7f7d5a9e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f4910 0x56541c9f7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.101+0000 7f7d5a9e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9ff1d0 0x56541c9f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.101+0000 7f7d5a9e3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501b13b0 con 0x56541c9f4910 2024-08-18T23:46:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.101+0000 7f7d5a9e3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501aec80 con 0x56541c9f3ff0 2024-08-18T23:46:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.101+0000 7f7d5a9e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501acad0 con 0x56541c9ff1d0 2024-08-18T23:46:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.102+0000 7f7d55b80640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f4910 0x56541c9f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.102+0000 7f7d56381640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9f3ff0 0x56541c9fbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.102+0000 7f7d55b80640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f4910 0x56541c9f7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54942/0 (socket says 172.21.15.78:54942) 2024-08-18T23:46:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.102+0000 7f7d55b80640 1 -- 172.21.15.78:0/849642966 learned_addr learned my addr 172.21.15.78:0/849642966 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.102+0000 7f7d55b80640 1 -- 172.21.15.78:0/849642966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9ff1d0 msgr2=0x56541c9f77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.102+0000 7f7d56b82640 1 --2- 172.21.15.78:0/849642966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9ff1d0 0x56541c9f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.102+0000 7f7d55b80640 1 --2- 172.21.15.78:0/849642966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9ff1d0 0x56541c9f77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.102+0000 7f7d55b80640 1 -- 172.21.15.78:0/849642966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9f3ff0 msgr2=0x56541c9fbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.102+0000 7f7d55b80640 1 --2- 172.21.15.78:0/849642966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9f3ff0 0x56541c9fbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.102+0000 7f7d55b80640 1 -- 172.21.15.78:0/849642966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56541c9fcab0 con 0x56541c9f4910 2024-08-18T23:46:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.102+0000 7f7d56381640 1 --2- 172.21.15.78:0/849642966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9f3ff0 0x56541c9fbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:46:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.102+0000 7f7d55b80640 1 --2- 172.21.15.78:0/849642966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f4910 0x56541c9f7270 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009900 tx=0x7f7d400058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d34f04dc8472939 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.102+0000 7f7d3bfff640 1 -- 172.21.15.78:0/849642966 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40035040 con 0x56541c9f4910 2024-08-18T23:46:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.102+0000 7f7d3bfff640 1 -- 172.21.15.78:0/849642966 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d40031dc0 con 0x56541c9f4910 2024-08-18T23:46:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.102+0000 7f7d3bfff640 1 -- 172.21.15.78:0/849642966 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d400047a0 con 0x56541c9f4910 2024-08-18T23:46:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.103+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/849642966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f4910 msgr2=0x56541c9f7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.103+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/849642966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f4910 0x56541c9f7270 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009900 tx=0x7f7d400058d0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.103+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/849642966 shutdown_connections 2024-08-18T23:46:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.103+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/849642966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9ff1d0 0x56541c9f77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.103+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/849642966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f4910 0x56541c9f7270 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.103+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/849642966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9f3ff0 0x56541c9fbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.103+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/849642966 >> 172.21.15.78:0/849642966 conn(0x56541c9f0580 msgr2=0x56541c9eb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.103+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/849642966 shutdown_connections 2024-08-18T23:46:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.103+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/849642966 wait complete. 2024-08-18T23:46:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.103+0000 7f7d5a9e3740 1 -- start start 2024-08-18T23:46:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.103+0000 7f7d5a9e3740 1 Processor -- start 2024-08-18T23:46:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d5a9e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f3ff0 0x56541ca07350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d5a9e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9f4910 0x56541ca07890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d5a9e3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9ff1d0 0x56541c9f9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d5a9e3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501ad140 con 0x56541c9f3ff0 2024-08-18T23:46:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d5a9e3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501ae5b0 con 0x56541c9ff1d0 2024-08-18T23:46:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d5a9e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501ad810 con 0x56541c9f4910 2024-08-18T23:46:21.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d5a9e3740 10 client.0 _ll_register_callbacks cb 0x7f7d4c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:46:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d56381640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f3ff0 0x56541ca07350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d56381640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f3ff0 0x56541ca07350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54958/0 (socket says 172.21.15.78:54958) 2024-08-18T23:46:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d56381640 1 -- 172.21.15.78:0/2833751437 learned_addr learned my addr 172.21.15.78:0/2833751437 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d56b82640 1 --2- 172.21.15.78:0/2833751437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9ff1d0 0x56541c9f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d55b80640 1 --2- 172.21.15.78:0/2833751437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9f4910 0x56541ca07890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d56381640 1 -- 172.21.15.78:0/2833751437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9f4910 msgr2=0x56541ca07890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d56381640 1 --2- 172.21.15.78:0/2833751437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9f4910 0x56541ca07890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d56381640 1 -- 172.21.15.78:0/2833751437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9ff1d0 msgr2=0x56541c9f9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d56381640 1 --2- 172.21.15.78:0/2833751437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9ff1d0 0x56541c9f9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d56381640 1 -- 172.21.15.78:0/2833751437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56541c9f4740 con 0x56541c9f3ff0 2024-08-18T23:46:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d56381640 1 --2- 172.21.15.78:0/2833751437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f3ff0 0x56541ca07350 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c09bc70 tx=0x7f7d4c09b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779318bfd6312a66 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:46:21.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d39ffb640 1 -- 172.21.15.78:0/2833751437 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c09bce0 con 0x56541c9f3ff0 2024-08-18T23:46:21.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d39ffb640 1 -- 172.21.15.78:0/2833751437 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f7d4c010430 con 0x56541c9f3ff0 2024-08-18T23:46:21.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2833751437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d40031dc0 con 0x56541c9f3ff0 2024-08-18T23:46:21.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d39ffb640 1 -- 172.21.15.78:0/2833751437 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7d4c081050 con 0x56541c9f3ff0 2024-08-18T23:46:21.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d5a9e3740 10 client.15771 fetch_fsmap learned FSMap version 26 2024-08-18T23:46:21.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.104+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2833751437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56541c9f4740 con 0x56541c9f3ff0 2024-08-18T23:46:21.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.105+0000 7f7d39ffb640 1 -- 172.21.15.78:0/2833751437 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d40031dc0 con 0x56541c9f3ff0 2024-08-18T23:46:21.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.105+0000 7f7d39ffb640 1 -- 172.21.15.78:0/2833751437 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f7d4c097a30 con 0x56541c9f3ff0 2024-08-18T23:46:21.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.105+0000 7f7d5a9e3740 10 client.15771 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:46:21.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.105+0000 7f7d5a9e3740 10 client.15771 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:46:21.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.105+0000 7f7d5a9e3740 20 client.15771 populate_metadata read hostname 'smithi078' 2024-08-18T23:46:21.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.105+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/2833751437 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56541ca004f0 0x56541ca7b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:21.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.105+0000 7f7d5a9e3740 4 client.15771 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:46:21.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.105+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2833751437 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56541c9614c0 con 0x56541ca004f0 2024-08-18T23:46:21.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.105+0000 7f7d55b80640 1 --2- 172.21.15.78:0/2833751437 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56541ca004f0 0x56541ca7b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:21.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.105+0000 7f7d39ffb640 10 client.15771 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:46:21.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.105+0000 7f7d55b80640 1 --2- 172.21.15.78:0/2833751437 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56541ca004f0 0x56541ca7b9f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=714d94ef73a55832 server_cookie=355d9d42d4897868 in_seq=0 out_seq=0 2024-08-18T23:46:21.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d39ffb640 1 -- 172.21.15.78:0/2833751437 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56541c9614c0 con 0x56541ca004f0 2024-08-18T23:46:21.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d39ffb640 10 client.15771 handle_command_reply: tid=0 2024-08-18T23:46:21.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d5a9e3740 1 client.15771 shutdown 2024-08-18T23:46:21.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2833751437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f3ff0 msgr2=0x56541ca07350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:21.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/2833751437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f3ff0 0x56541ca07350 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c09bc70 tx=0x7f7d4c09b770 comp rx=0 tx=0).stop 2024-08-18T23:46:21.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2833751437 shutdown_connections 2024-08-18T23:46:21.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/2833751437 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56541ca004f0 0x56541ca7b9f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/2833751437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56541c9ff1d0 0x56541c9f9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/2833751437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56541c9f4910 0x56541ca07890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d39ffb640 0 client.15771 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:46:21.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d5a9e3740 1 --2- 172.21.15.78:0/2833751437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56541c9f3ff0 0x56541ca07350 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2833751437 >> 172.21.15.78:0/2833751437 conn(0x56541c9f0580 msgr2=0x56541c9f7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:21.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2833751437 shutdown_connections 2024-08-18T23:46:21.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d5a9e3740 1 -- 172.21.15.78:0/2833751437 wait complete. 2024-08-18T23:46:21.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d5a9e3740 20 client.15771 trim_cache size 0 max 16384 2024-08-18T23:46:21.117 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:46:21.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d585e5640 1 -- 172.21.15.78:0/3027597516 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7d3c0a2b80 msgr2=0x7f7d3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:21.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d585e5640 1 --2- 172.21.15.78:0/3027597516 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7d3c0a2b80 0x7f7d3c0a5040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800afd0 tx=0x7f7d48004ab0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d585e5640 1 -- 172.21.15.78:0/3027597516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d50102ac0 msgr2=0x7f7d50198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:21.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d585e5640 1 --2- 172.21.15.78:0/3027597516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d50102ac0 0x7f7d50198380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00e9f0 tx=0x7f7d4c00eeb0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d585e5640 1 -- 172.21.15.78:0/3027597516 shutdown_connections 2024-08-18T23:46:21.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d585e5640 1 --2- 172.21.15.78:0/3027597516 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7d3c0a2b80 0x7f7d3c0a5040 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d585e5640 1 --2- 172.21.15.78:0/3027597516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d50107fc0 0x7f7d5019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d585e5640 1 --2- 172.21.15.78:0/3027597516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d50103470 0x7f7d501988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d585e5640 1 --2- 172.21.15.78:0/3027597516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d50102ac0 0x7f7d50198380 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:21.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.106+0000 7f7d585e5640 1 -- 172.21.15.78:0/3027597516 >> 172.21.15.78:0/3027597516 conn(0x7f7d500fc500 msgr2=0x7f7d50105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:21.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.107+0000 7f7d585e5640 1 -- 172.21.15.78:0/3027597516 shutdown_connections 2024-08-18T23:46:21.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:21.107+0000 7f7d585e5640 1 -- 172.21.15.78:0/3027597516 wait complete. 2024-08-18T23:46:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:21 smithi161 ceph-mon[22027]: pgmap v411: 129 pgs: 129 active+clean; 196 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 205 op/s 2024-08-18T23:46:21.711 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:21 smithi078 ceph-mon[18440]: pgmap v411: 129 pgs: 129 active+clean; 196 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 205 op/s 2024-08-18T23:46:21.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:21 smithi123 ceph-mon[21808]: pgmap v411: 129 pgs: 129 active+clean; 196 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 205 op/s 2024-08-18T23:46:21.917 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-18T23:46:22.176 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:46:23.408 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-08-18T23:46:23.409 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:46:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:23 smithi161 ceph-mon[22027]: pgmap v412: 129 pgs: 129 active+clean; 242 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.3 MiB/s wr, 459 op/s 2024-08-18T23:46:23.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:23 smithi078 ceph-mon[18440]: pgmap v412: 129 pgs: 129 active+clean; 242 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.3 MiB/s wr, 459 op/s 2024-08-18T23:46:23.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:23 smithi123 ceph-mon[21808]: pgmap v412: 129 pgs: 129 active+clean; 242 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.3 MiB/s wr, 459 op/s 2024-08-18T23:46:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:24 smithi161 ceph-mon[22027]: pgmap v413: 129 pgs: 129 active+clean; 262 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.9 MiB/s wr, 558 op/s 2024-08-18T23:46:24.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:24 smithi078 ceph-mon[18440]: pgmap v413: 129 pgs: 129 active+clean; 262 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.9 MiB/s wr, 558 op/s 2024-08-18T23:46:24.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:24 smithi123 ceph-mon[21808]: pgmap v413: 129 pgs: 129 active+clean; 262 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.9 MiB/s wr, 558 op/s 2024-08-18T23:46:25.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.162+0000 7f3860a1c640 1 -- 172.21.15.78:0/166121523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f385c1007b0 msgr2=0x7f385c100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:25.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.162+0000 7f3860a1c640 1 --2- 172.21.15.78:0/166121523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f385c1007b0 0x7f385c100c10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f384c009860 tx=0x7f384c02a340 comp rx=0 tx=0).stop 2024-08-18T23:46:25.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.164+0000 7f3860a1c640 1 -- 172.21.15.78:0/166121523 shutdown_connections 2024-08-18T23:46:25.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.164+0000 7f3860a1c640 1 --2- 172.21.15.78:0/166121523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f385c101150 0x7f385c06a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.164+0000 7f3860a1c640 1 --2- 172.21.15.78:0/166121523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f385c1007b0 0x7f385c100c10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.164+0000 7f3860a1c640 1 --2- 172.21.15.78:0/166121523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f385c1067b0 0x7f385c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.164+0000 7f3860a1c640 1 -- 172.21.15.78:0/166121523 >> 172.21.15.78:0/166121523 conn(0x7f385c0fc4f0 msgr2=0x7f385c0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:25.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.168+0000 7f3860a1c640 1 -- 172.21.15.78:0/166121523 shutdown_connections 2024-08-18T23:46:25.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.169+0000 7f3860a1c640 1 -- 172.21.15.78:0/166121523 wait complete. 2024-08-18T23:46:25.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.170+0000 7f3860a1c640 1 Processor -- start 2024-08-18T23:46:25.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.171+0000 7f3860a1c640 1 -- start start 2024-08-18T23:46:25.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.171+0000 7f3860a1c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f385c1007b0 0x7f385c1a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:25.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.171+0000 7f3860a1c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f385c101150 0x7f385c1aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:25.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.171+0000 7f3860a1c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f385c1067b0 0x7f385c1b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:25.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.171+0000 7f3860a1c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f385c06bbc0 con 0x7f385c1007b0 2024-08-18T23:46:25.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.171+0000 7f3860a1c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f385c06ba40 con 0x7f385c101150 2024-08-18T23:46:25.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.171+0000 7f3860a1c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f385c06bd40 con 0x7f385c1067b0 2024-08-18T23:46:25.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.171+0000 7f385ad9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f385c1067b0 0x7f385c1b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:25.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.172+0000 7f385ad9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f385c1067b0 0x7f385c1b0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:49156/0 (socket says 172.21.15.78:49156) 2024-08-18T23:46:25.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.172+0000 7f385ad9d640 1 -- 172.21.15.78:0/856274700 learned_addr learned my addr 172.21.15.78:0/856274700 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:25.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.172+0000 7f385ad9d640 1 -- 172.21.15.78:0/856274700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f385c101150 msgr2=0x7f385c1aa0d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-18T23:46:25.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.172+0000 7f385ad9d640 1 --2- 172.21.15.78:0/856274700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f385c101150 0x7f385c1aa0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.172+0000 7f385ad9d640 1 -- 172.21.15.78:0/856274700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f385c1007b0 msgr2=0x7f385c1a9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:25.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.172+0000 7f385ad9d640 1 --2- 172.21.15.78:0/856274700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f385c1007b0 0x7f385c1a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.172+0000 7f385ad9d640 1 -- 172.21.15.78:0/856274700 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385c1b0f90 con 0x7f385c1067b0 2024-08-18T23:46:25.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.175+0000 7f385ad9d640 1 --2- 172.21.15.78:0/856274700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f385c1067b0 0x7f385c1b0a20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f385000d750 tx=0x7f385000dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:25.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.175+0000 7f384b7fe640 1 -- 172.21.15.78:0/856274700 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3850017070 con 0x7f385c1067b0 2024-08-18T23:46:25.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.175+0000 7f3860a1c640 1 -- 172.21.15.78:0/856274700 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385c1b1220 con 0x7f385c1067b0 2024-08-18T23:46:25.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.175+0000 7f3860a1c640 1 -- 172.21.15.78:0/856274700 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f385c1b1700 con 0x7f385c1067b0 2024-08-18T23:46:25.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.179+0000 7f3862e1a740 1 Processor -- start 2024-08-18T23:46:25.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.180+0000 7f3862e1a740 1 -- start start 2024-08-18T23:46:25.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.180+0000 7f3862e1a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38df930 0x5566b38de100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:25.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.180+0000 7f3862e1a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38de640 0x5566b38dea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:25.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.180+0000 7f3862e1a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38e8fa0 0x5566b38eb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:25.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.180+0000 7f3862e1a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5566b3843a40 con 0x5566b38df930 2024-08-18T23:46:25.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.180+0000 7f3862e1a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5566b37db9d0 con 0x5566b38e8fa0 2024-08-18T23:46:25.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.180+0000 7f3862e1a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5566b383fa00 con 0x5566b38de640 2024-08-18T23:46:25.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.180+0000 7f384b7fe640 1 -- 172.21.15.78:0/856274700 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3850005500 con 0x7f385c1067b0 2024-08-18T23:46:25.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.180+0000 7f384b7fe640 1 -- 172.21.15.78:0/856274700 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3850004d90 con 0x7f385c1067b0 2024-08-18T23:46:25.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.184+0000 7f3859d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38e8fa0 0x5566b38eb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:25.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.184+0000 7f3859d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38df930 0x5566b38de100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:25.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.184+0000 7f3859d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38df930 0x5566b38de100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54998/0 (socket says 172.21.15.78:54998) 2024-08-18T23:46:25.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.184+0000 7f3859d9b640 1 -- 172.21.15.78:0/926368717 learned_addr learned my addr 172.21.15.78:0/926368717 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:25.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.184+0000 7f385a59c640 1 --2- 172.21.15.78:0/926368717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38de640 0x5566b38dea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:25.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.185+0000 7f3859d9b640 1 -- 172.21.15.78:0/926368717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38de640 msgr2=0x5566b38dea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:25.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.185+0000 7f3859d9b640 1 --2- 172.21.15.78:0/926368717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38de640 0x5566b38dea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.185+0000 7f3859d9b640 1 -- 172.21.15.78:0/926368717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38df930 msgr2=0x5566b38de100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:25.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.185+0000 7f3859d9b640 1 --2- 172.21.15.78:0/926368717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38df930 0x5566b38de100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.185+0000 7f384b7fe640 1 -- 172.21.15.78:0/856274700 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3850010460 con 0x7f385c1067b0 2024-08-18T23:46:25.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.186+0000 7f3859d9b640 1 -- 172.21.15.78:0/926368717 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566b37e2650 con 0x5566b38e8fa0 2024-08-18T23:46:25.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.187+0000 7f3859d9b640 1 --2- 172.21.15.78:0/926368717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38e8fa0 0x5566b38eb390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f384c034000 tx=0x7f384c02a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfcb6a6888434f47 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:25.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.188+0000 7f384b7fe640 1 --2- 172.21.15.78:0/856274700 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f38440a2cc0 0x7f38440a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:25.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.188+0000 7f384b7fe640 1 -- 172.21.15.78:0/856274700 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f38500cfe90 con 0x7f385c1067b0 2024-08-18T23:46:25.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.189+0000 7f3860a1c640 1 -- 172.21.15.78:0/926368717 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c044020 con 0x5566b38e8fa0 2024-08-18T23:46:25.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.189+0000 7f3860a1c640 1 -- 172.21.15.78:0/926368717 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f384c030e50 con 0x5566b38e8fa0 2024-08-18T23:46:25.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.189+0000 7f3860a1c640 1 -- 172.21.15.78:0/926368717 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c037040 con 0x5566b38e8fa0 2024-08-18T23:46:25.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.192+0000 7f3862e1a740 1 -- 172.21.15.78:0/926368717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38e8fa0 msgr2=0x5566b38eb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:25.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.192+0000 7f3862e1a740 1 --2- 172.21.15.78:0/926368717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38e8fa0 0x5566b38eb390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f384c034000 tx=0x7f384c02a340 comp rx=0 tx=0).stop 2024-08-18T23:46:25.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.193+0000 7f3862e1a740 1 -- 172.21.15.78:0/926368717 shutdown_connections 2024-08-18T23:46:25.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.193+0000 7f3862e1a740 1 --2- 172.21.15.78:0/926368717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38e8fa0 0x5566b38eb390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.193+0000 7f3862e1a740 1 --2- 172.21.15.78:0/926368717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38de640 0x5566b38dea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.193+0000 7f3862e1a740 1 --2- 172.21.15.78:0/926368717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38df930 0x5566b38de100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.193+0000 7f3862e1a740 1 -- 172.21.15.78:0/926368717 >> 172.21.15.78:0/926368717 conn(0x5566b38da890 msgr2=0x5566b38dac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:25.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.197+0000 7f385a59c640 1 --2- 172.21.15.78:0/856274700 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f38440a2cc0 0x7f38440a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:25.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.199+0000 7f3862e1a740 1 -- 172.21.15.78:0/926368717 shutdown_connections 2024-08-18T23:46:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.201+0000 7f3862e1a740 1 -- 172.21.15.78:0/926368717 wait complete. 2024-08-18T23:46:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.202+0000 7f3862e1a740 1 -- start start 2024-08-18T23:46:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.202+0000 7f3862e1a740 1 Processor -- start 2024-08-18T23:46:25.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.203+0000 7f385a59c640 1 --2- 172.21.15.78:0/856274700 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f38440a2cc0 0x7f38440a5180 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3854005c40 tx=0x7f3854005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:25.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.210+0000 7f3862e1a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38de640 0x5566b38e5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:25.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.210+0000 7f3862e1a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38df930 0x5566b38e6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:25.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.210+0000 7f3862e1a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38e8fa0 0x5566b38e6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:25.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.210+0000 7f3862e1a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5566b38ebdb0 con 0x5566b38df930 2024-08-18T23:46:25.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.210+0000 7f3862e1a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5566b37feb20 con 0x5566b38de640 2024-08-18T23:46:25.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.210+0000 7f3862e1a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5566b38ebf30 con 0x5566b38e8fa0 2024-08-18T23:46:25.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.210+0000 7f3862e1a740 10 client.0 _ll_register_callbacks cb 0x5566b37fcbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:46:25.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.211+0000 7f3859d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38df930 0x5566b38e6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:25.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.211+0000 7f3859d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38df930 0x5566b38e6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55012/0 (socket says 172.21.15.78:55012) 2024-08-18T23:46:25.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.211+0000 7f3859d9b640 1 -- 172.21.15.78:0/1269049675 learned_addr learned my addr 172.21.15.78:0/1269049675 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:25.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.211+0000 7f385ad9d640 1 --2- 172.21.15.78:0/1269049675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38e8fa0 0x5566b38e6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:25.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.212+0000 7f3859d9b640 1 -- 172.21.15.78:0/1269049675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38e8fa0 msgr2=0x5566b38e6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:25.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.212+0000 7f3859d9b640 1 --2- 172.21.15.78:0/1269049675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38e8fa0 0x5566b38e6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.212+0000 7f3859d9b640 1 -- 172.21.15.78:0/1269049675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38de640 msgr2=0x5566b38e5ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-18T23:46:25.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.212+0000 7f3859d9b640 1 --2- 172.21.15.78:0/1269049675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38de640 0x5566b38e5ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.212+0000 7f3859d9b640 1 -- 172.21.15.78:0/1269049675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566b35b53c0 con 0x5566b38df930 2024-08-18T23:46:25.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.213+0000 7f3859d9b640 1 --2- 172.21.15.78:0/1269049675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38df930 0x5566b38e6420 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f384c000c00 tx=0x7f384c030a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2418bbca173f4eb7 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:25.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.213+0000 7f3841ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:46:25.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.213+0000 7f3841ffb640 1 -- 172.21.15.78:0/1269049675 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c044070 con 0x5566b38df930 2024-08-18T23:46:25.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.213+0000 7f3841ffb640 1 -- 172.21.15.78:0/1269049675 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f384c005340 con 0x5566b38df930 2024-08-18T23:46:25.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.213+0000 7f3862e1a740 1 -- 172.21.15.78:0/1269049675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f384c030e50 con 0x5566b38df930 2024-08-18T23:46:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.215+0000 7f3841ffb640 1 -- 172.21.15.78:0/1269049675 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f384c005540 con 0x5566b38df930 2024-08-18T23:46:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.219+0000 7f3862e1a740 10 client.15780 fetch_fsmap learned FSMap version 26 2024-08-18T23:46:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.219+0000 7f3862e1a740 1 -- 172.21.15.78:0/1269049675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5566b38f7ae0 con 0x5566b38df930 2024-08-18T23:46:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.221+0000 7f3841ffb640 1 -- 172.21.15.78:0/1269049675 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f384c030e50 con 0x5566b38df930 2024-08-18T23:46:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.222+0000 7f3841ffb640 1 -- 172.21.15.78:0/1269049675 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f384c0035f0 con 0x5566b38df930 2024-08-18T23:46:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.222+0000 7f3862e1a740 10 client.15780 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:46:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.222+0000 7f3862e1a740 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:46:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.222+0000 7f3862e1a740 20 client.15780 populate_metadata read hostname 'smithi078' 2024-08-18T23:46:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.222+0000 7f3862e1a740 1 --2- 172.21.15.78:0/1269049675 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5566b38f8e30 0x5566b38fb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.222+0000 7f3862e1a740 4 client.15780 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:46:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.222+0000 7f3862e1a740 1 -- 172.21.15.78:0/1269049675 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566b3850fb0 con 0x5566b38f8e30 2024-08-18T23:46:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.230+0000 7f385a59c640 1 --2- 172.21.15.78:0/1269049675 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5566b38f8e30 0x5566b38fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:25.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.247+0000 7f3841ffb640 10 client.15780 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:25.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.247+0000 7f385a59c640 1 --2- 172.21.15.78:0/1269049675 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5566b38f8e30 0x5566b38fb220 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afbcfd7b825a39a7 server_cookie=efdd694322e17e8b in_seq=0 out_seq=0 2024-08-18T23:46:25.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.252+0000 7f3841ffb640 1 -- 172.21.15.78:0/1269049675 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566b3850fb0 con 0x5566b38f8e30 2024-08-18T23:46:25.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.252+0000 7f3841ffb640 10 client.15780 handle_command_reply: tid=0 2024-08-18T23:46:25.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.252+0000 7f3862e1a740 1 client.15780 shutdown 2024-08-18T23:46:25.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.256+0000 7f3862e1a740 1 -- 172.21.15.78:0/1269049675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38df930 msgr2=0x5566b38e6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:25.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.256+0000 7f3862e1a740 1 --2- 172.21.15.78:0/1269049675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38df930 0x5566b38e6420 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f384c000c00 tx=0x7f384c030a70 comp rx=0 tx=0).stop 2024-08-18T23:46:25.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.256+0000 7f3862e1a740 1 -- 172.21.15.78:0/1269049675 shutdown_connections 2024-08-18T23:46:25.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.256+0000 7f3862e1a740 1 --2- 172.21.15.78:0/1269049675 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5566b38f8e30 0x5566b38fb220 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.256+0000 7f3862e1a740 1 --2- 172.21.15.78:0/1269049675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38e8fa0 0x5566b38e6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.256+0000 7f3862e1a740 1 --2- 172.21.15.78:0/1269049675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38df930 0x5566b38e6420 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.256+0000 7f3862e1a740 1 --2- 172.21.15.78:0/1269049675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38de640 0x5566b38e5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.256+0000 7f3862e1a740 1 -- 172.21.15.78:0/1269049675 >> 172.21.15.78:0/1269049675 conn(0x5566b38da890 msgr2=0x5566b38e05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:25.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.256+0000 7f3841ffb640 0 client.15780 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:25.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.259+0000 7f3862e1a740 1 -- 172.21.15.78:0/1269049675 shutdown_connections 2024-08-18T23:46:25.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.259+0000 7f3862e1a740 1 -- 172.21.15.78:0/1269049675 wait complete. 2024-08-18T23:46:25.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.260+0000 7f3862e1a740 20 client.15780 trim_cache size 0 max 16384 2024-08-18T23:46:25.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.292+0000 7f3862e1a740 1 Processor -- start 2024-08-18T23:46:25.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.292+0000 7f3862e1a740 1 -- start start 2024-08-18T23:46:25.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.293+0000 7f3862e1a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38de640 0x5566b38ea390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:25.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.293+0000 7f3862e1a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38df930 0x5566b38ea8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:25.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.293+0000 7f3862e1a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38e8fa0 0x5566b38e7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:25.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.293+0000 7f3859d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38df930 0x5566b38ea8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:25.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.293+0000 7f3859d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38df930 0x5566b38ea8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:49192/0 (socket says 172.21.15.78:49192) 2024-08-18T23:46:25.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.294+0000 7f385a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38de640 0x5566b38ea390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:25.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.294+0000 7f385a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38de640 0x5566b38ea390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55014/0 (socket says 172.21.15.78:55014) 2024-08-18T23:46:25.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.294+0000 7f385ad9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38e8fa0 0x5566b38e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:25.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.294+0000 7f385ad9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38e8fa0 0x5566b38e7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:35284/0 (socket says 172.21.15.78:35284) 2024-08-18T23:46:25.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.294+0000 7f3859d9b640 1 -- 172.21.15.78:0/4255029888 learned_addr learned my addr 172.21.15.78:0/4255029888 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:25.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.294+0000 7f3862e1a740 1 -- 172.21.15.78:0/4255029888 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f383c012530 con 0x5566b38de640 2024-08-18T23:46:25.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.294+0000 7f3862e1a740 1 -- 172.21.15.78:0/4255029888 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f383c00fe00 con 0x5566b38e8fa0 2024-08-18T23:46:25.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.294+0000 7f3862e1a740 1 -- 172.21.15.78:0/4255029888 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f383c00dc50 con 0x5566b38df930 2024-08-18T23:46:25.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.295+0000 7f3859d9b640 1 -- 172.21.15.78:0/4255029888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38e8fa0 msgr2=0x5566b38e7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:25.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.295+0000 7f3859d9b640 1 --2- 172.21.15.78:0/4255029888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38e8fa0 0x5566b38e7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.295+0000 7f3859d9b640 1 -- 172.21.15.78:0/4255029888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38de640 msgr2=0x5566b38ea390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:25.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.295+0000 7f3859d9b640 1 --2- 172.21.15.78:0/4255029888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38de640 0x5566b38ea390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.295+0000 7f3859d9b640 1 -- 172.21.15.78:0/4255029888 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566b38e4450 con 0x5566b38df930 2024-08-18T23:46:25.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.296+0000 7f3859d9b640 1 --2- 172.21.15.78:0/4255029888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38df930 0x5566b38ea8d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f384c005c40 tx=0x7f384c005b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b63caba239608c98 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:25.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.299+0000 7f3843fff640 1 -- 172.21.15.78:0/4255029888 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c037030 con 0x5566b38df930 2024-08-18T23:46:25.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.299+0000 7f3843fff640 1 -- 172.21.15.78:0/4255029888 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f384c04d070 con 0x5566b38df930 2024-08-18T23:46:25.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.299+0000 7f3843fff640 1 -- 172.21.15.78:0/4255029888 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c044040 con 0x5566b38df930 2024-08-18T23:46:25.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.300+0000 7f3862e1a740 1 -- 172.21.15.78:0/4255029888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38df930 msgr2=0x5566b38ea8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:25.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.300+0000 7f3862e1a740 1 --2- 172.21.15.78:0/4255029888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38df930 0x5566b38ea8d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f384c005c40 tx=0x7f384c005b10 comp rx=0 tx=0).stop 2024-08-18T23:46:25.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.301+0000 7f3862e1a740 1 -- 172.21.15.78:0/4255029888 shutdown_connections 2024-08-18T23:46:25.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.301+0000 7f3862e1a740 1 --2- 172.21.15.78:0/4255029888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38e8fa0 0x5566b38e7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.301+0000 7f3862e1a740 1 --2- 172.21.15.78:0/4255029888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38df930 0x5566b38ea8d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.301+0000 7f3862e1a740 1 --2- 172.21.15.78:0/4255029888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38de640 0x5566b38ea390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.301+0000 7f3862e1a740 1 -- 172.21.15.78:0/4255029888 >> 172.21.15.78:0/4255029888 conn(0x5566b38da890 msgr2=0x5566b38e4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:25.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.302+0000 7f3862e1a740 1 -- 172.21.15.78:0/4255029888 shutdown_connections 2024-08-18T23:46:25.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.302+0000 7f3862e1a740 1 -- 172.21.15.78:0/4255029888 wait complete. 2024-08-18T23:46:25.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.302+0000 7f3862e1a740 1 -- start start 2024-08-18T23:46:25.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.302+0000 7f3862e1a740 1 Processor -- start 2024-08-18T23:46:25.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.303+0000 7f3862e1a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38de640 0x5566b38f1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:25.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.303+0000 7f3862e1a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38df930 0x5566b38f1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:25.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.303+0000 7f3862e1a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38e8fa0 0x5566b38e3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:25.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.303+0000 7f3862e1a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f383c00e2c0 con 0x5566b38de640 2024-08-18T23:46:25.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.303+0000 7f3862e1a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f383c00f730 con 0x5566b38df930 2024-08-18T23:46:25.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.303+0000 7f3862e1a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f383c00e990 con 0x5566b38e8fa0 2024-08-18T23:46:25.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.303+0000 7f3862e1a740 10 client.0 _ll_register_callbacks cb 0x7f3854009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:46:25.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.303+0000 7f385ad9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38e8fa0 0x5566b38e3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:25.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.303+0000 7f385ad9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38e8fa0 0x5566b38e3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:49200/0 (socket says 172.21.15.78:49200) 2024-08-18T23:46:25.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.303+0000 7f385ad9d640 1 -- 172.21.15.78:0/3879646430 learned_addr learned my addr 172.21.15.78:0/3879646430 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:25.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.304+0000 7f385ad9d640 1 -- 172.21.15.78:0/3879646430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38df930 msgr2=0x5566b38f1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:25.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.304+0000 7f385ad9d640 1 --2- 172.21.15.78:0/3879646430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38df930 0x5566b38f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.304+0000 7f385ad9d640 1 -- 172.21.15.78:0/3879646430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38de640 msgr2=0x5566b38f1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:25.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.304+0000 7f385ad9d640 1 --2- 172.21.15.78:0/3879646430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38de640 0x5566b38f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.304+0000 7f385ad9d640 1 -- 172.21.15.78:0/3879646430 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566b38e3870 con 0x5566b38e8fa0 2024-08-18T23:46:25.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.305+0000 7f385ad9d640 1 --2- 172.21.15.78:0/3879646430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38e8fa0 0x5566b38e3180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f385009c290 tx=0x7f38500bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51877a71e4d3152e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:25.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.305+0000 7f3841ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:46:25.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.305+0000 7f3841ffb640 1 -- 172.21.15.78:0/3879646430 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38500b4060 con 0x5566b38e8fa0 2024-08-18T23:46:25.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.305+0000 7f3862e1a740 1 -- 172.21.15.78:0/3879646430 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f384c04d070 con 0x5566b38e8fa0 2024-08-18T23:46:25.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.305+0000 7f3841ffb640 1 -- 172.21.15.78:0/3879646430 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f38500a4ae0 con 0x5566b38e8fa0 2024-08-18T23:46:25.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.305+0000 7f3841ffb640 1 -- 172.21.15.78:0/3879646430 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f385007c050 con 0x5566b38e8fa0 2024-08-18T23:46:25.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.310+0000 7f3862e1a740 10 client.25162 fetch_fsmap learned FSMap version 26 2024-08-18T23:46:25.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.310+0000 7f3862e1a740 1 -- 172.21.15.78:0/3879646430 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5566b3900510 con 0x5566b38e8fa0 2024-08-18T23:46:25.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.311+0000 7f3841ffb640 1 -- 172.21.15.78:0/3879646430 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f384c04d070 con 0x5566b38e8fa0 2024-08-18T23:46:25.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.311+0000 7f3841ffb640 1 -- 172.21.15.78:0/3879646430 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f385009b750 con 0x5566b38e8fa0 2024-08-18T23:46:25.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.311+0000 7f3862e1a740 10 client.25162 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:46:25.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.311+0000 7f3862e1a740 10 client.25162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:46:25.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.311+0000 7f3862e1a740 20 client.25162 populate_metadata read hostname 'smithi078' 2024-08-18T23:46:25.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.311+0000 7f3862e1a740 1 --2- 172.21.15.78:0/3879646430 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5566b3963970 0x5566b3965e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:25.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.311+0000 7f3862e1a740 4 client.25162 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:46:25.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.311+0000 7f3862e1a740 1 -- 172.21.15.78:0/3879646430 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566b384b4c0 con 0x5566b3963970 2024-08-18T23:46:25.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.312+0000 7f385a59c640 1 --2- 172.21.15.78:0/3879646430 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5566b3963970 0x5566b3965e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:25.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.312+0000 7f385a59c640 1 --2- 172.21.15.78:0/3879646430 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5566b3963970 0x5566b3965e10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51c7f02673b0791d server_cookie=bd7cb5db171eb382 in_seq=0 out_seq=0 2024-08-18T23:46:25.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.312+0000 7f3841ffb640 10 client.25162 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:25.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.315+0000 7f3841ffb640 1 -- 172.21.15.78:0/3879646430 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566b384b4c0 con 0x5566b3963970 2024-08-18T23:46:25.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.315+0000 7f3841ffb640 10 client.25162 handle_command_reply: tid=0 2024-08-18T23:46:25.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.315+0000 7f3862e1a740 1 client.25162 shutdown 2024-08-18T23:46:25.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.317+0000 7f3862e1a740 1 -- 172.21.15.78:0/3879646430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38e8fa0 msgr2=0x5566b38e3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:25.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.317+0000 7f3862e1a740 1 --2- 172.21.15.78:0/3879646430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38e8fa0 0x5566b38e3180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f385009c290 tx=0x7f38500bc040 comp rx=0 tx=0).stop 2024-08-18T23:46:25.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.317+0000 7f3862e1a740 1 -- 172.21.15.78:0/3879646430 shutdown_connections 2024-08-18T23:46:25.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.317+0000 7f3862e1a740 1 --2- 172.21.15.78:0/3879646430 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5566b3963970 0x5566b3965e10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.318+0000 7f3862e1a740 1 --2- 172.21.15.78:0/3879646430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5566b38e8fa0 0x5566b38e3180 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.318+0000 7f3862e1a740 1 --2- 172.21.15.78:0/3879646430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566b38df930 0x5566b38f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.318+0000 7f3862e1a740 1 --2- 172.21.15.78:0/3879646430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5566b38de640 0x5566b38f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.318+0000 7f3862e1a740 1 -- 172.21.15.78:0/3879646430 >> 172.21.15.78:0/3879646430 conn(0x5566b38da890 msgr2=0x5566b38dfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:25.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.318+0000 7f3841ffb640 0 client.25162 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:25.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.320+0000 7f3862e1a740 1 -- 172.21.15.78:0/3879646430 shutdown_connections 2024-08-18T23:46:25.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.320+0000 7f3862e1a740 1 -- 172.21.15.78:0/3879646430 wait complete. 2024-08-18T23:46:25.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.320+0000 7f3862e1a740 20 client.25162 trim_cache size 0 max 16384 2024-08-18T23:46:25.325 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:46:25.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.326+0000 7f38497fa640 1 -- 172.21.15.78:0/856274700 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f38440a2cc0 msgr2=0x7f38440a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:25.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.326+0000 7f38497fa640 1 --2- 172.21.15.78:0/856274700 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f38440a2cc0 0x7f38440a5180 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3854005c40 tx=0x7f3854005bd0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.326+0000 7f38497fa640 1 -- 172.21.15.78:0/856274700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f385c1067b0 msgr2=0x7f385c1b0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:25.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.326+0000 7f38497fa640 1 --2- 172.21.15.78:0/856274700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f385c1067b0 0x7f385c1b0a20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f385000d750 tx=0x7f385000dc10 comp rx=0 tx=0).stop 2024-08-18T23:46:25.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.327+0000 7f38497fa640 1 -- 172.21.15.78:0/856274700 shutdown_connections 2024-08-18T23:46:25.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.327+0000 7f38497fa640 1 --2- 172.21.15.78:0/856274700 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f38440a2cc0 0x7f38440a5180 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.327+0000 7f38497fa640 1 --2- 172.21.15.78:0/856274700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f385c1067b0 0x7f385c1b0a20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.327+0000 7f38497fa640 1 --2- 172.21.15.78:0/856274700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f385c101150 0x7f385c1aa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.327+0000 7f38497fa640 1 --2- 172.21.15.78:0/856274700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f385c1007b0 0x7f385c1a9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:25.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.327+0000 7f38497fa640 1 -- 172.21.15.78:0/856274700 >> 172.21.15.78:0/856274700 conn(0x7f385c0fc4f0 msgr2=0x7f385c06c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:25.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.327+0000 7f38497fa640 1 -- 172.21.15.78:0/856274700 shutdown_connections 2024-08-18T23:46:25.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:25.327+0000 7f38497fa640 1 -- 172.21.15.78:0/856274700 wait complete. 2024-08-18T23:46:26.254 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:46:26.982 INFO:tasks.workunit.client.0.smithi078.stdout: 56 876775 292 578863 303 454243 735 2024-08-18T23:46:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.004+0000 7f8341714640 1 -- 172.21.15.78:0/3028538208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f833c101150 msgr2=0x7f833c10d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.004+0000 7f8341714640 1 --2- 172.21.15.78:0/3028538208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f833c101150 0x7f833c10d680 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8334009cd0 tx=0x7f833402a8e0 comp rx=0 tx=0).stop 2024-08-18T23:46:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.004+0000 7f8341714640 1 -- 172.21.15.78:0/3028538208 shutdown_connections 2024-08-18T23:46:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.004+0000 7f8341714640 1 --2- 172.21.15.78:0/3028538208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f833c101150 0x7f833c10d680 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.004+0000 7f8341714640 1 --2- 172.21.15.78:0/3028538208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f833c1007b0 0x7f833c100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.004+0000 7f8341714640 1 --2- 172.21.15.78:0/3028538208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c1067b0 0x7f833c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.004+0000 7f8341714640 1 -- 172.21.15.78:0/3028538208 >> 172.21.15.78:0/3028538208 conn(0x7f833c0fc4d0 msgr2=0x7f833c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.005+0000 7f8341714640 1 -- 172.21.15.78:0/3028538208 shutdown_connections 2024-08-18T23:46:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.005+0000 7f8341714640 1 -- 172.21.15.78:0/3028538208 wait complete. 2024-08-18T23:46:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.005+0000 7f8341714640 1 Processor -- start 2024-08-18T23:46:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.005+0000 7f8341714640 1 -- start start 2024-08-18T23:46:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.005+0000 7f8341714640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c1007b0 0x7f833c198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.005+0000 7f8341714640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f833c101150 0x7f833c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.005+0000 7f8341714640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f833c1067b0 0x7f833c19ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.005+0000 7f8341714640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f833c110bb0 con 0x7f833c101150 2024-08-18T23:46:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.005+0000 7f8341714640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f833c110a30 con 0x7f833c1007b0 2024-08-18T23:46:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.005+0000 7f8341714640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f833c110d30 con 0x7f833c1067b0 2024-08-18T23:46:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.006+0000 7f833affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c1007b0 0x7f833c198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.006+0000 7f8333fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f833c101150 0x7f833c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.006+0000 7f833affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c1007b0 0x7f833c198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:39192/0 (socket says 172.21.15.78:39192) 2024-08-18T23:46:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.006+0000 7f833affd640 1 -- 172.21.15.78:0/2414093779 learned_addr learned my addr 172.21.15.78:0/2414093779 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.006+0000 7f8333fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f833c101150 0x7f833c198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35398/0 (socket says 172.21.15.78:35398) 2024-08-18T23:46:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.006+0000 7f833b7fe640 1 --2- 172.21.15.78:0/2414093779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f833c1067b0 0x7f833c19ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.006+0000 7f8333fff640 1 -- 172.21.15.78:0/2414093779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f833c1067b0 msgr2=0x7f833c19ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.006+0000 7f8333fff640 1 --2- 172.21.15.78:0/2414093779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f833c1067b0 0x7f833c19ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.006+0000 7f8333fff640 1 -- 172.21.15.78:0/2414093779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c1007b0 msgr2=0x7f833c198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.006+0000 7f8333fff640 1 --2- 172.21.15.78:0/2414093779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c1007b0 0x7f833c198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.006+0000 7f8333fff640 1 -- 172.21.15.78:0/2414093779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833c19d0e0 con 0x7f833c101150 2024-08-18T23:46:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.006+0000 7f8333fff640 1 --2- 172.21.15.78:0/2414093779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f833c101150 0x7f833c198670 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f832400d6d0 tx=0x7f832400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.006+0000 7f8338ff9640 1 -- 172.21.15.78:0/2414093779 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324017070 con 0x7f833c101150 2024-08-18T23:46:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.006+0000 7f8338ff9640 1 -- 172.21.15.78:0/2414093779 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8324009820 con 0x7f833c101150 2024-08-18T23:46:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.006+0000 7f8341714640 1 -- 172.21.15.78:0/2414093779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833c19d3d0 con 0x7f833c101150 2024-08-18T23:46:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.006+0000 7f8338ff9640 1 -- 172.21.15.78:0/2414093779 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324004d00 con 0x7f833c101150 2024-08-18T23:46:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.006+0000 7f8341714640 1 -- 172.21.15.78:0/2414093779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f833c1a4cb0 con 0x7f833c101150 2024-08-18T23:46:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.008+0000 7f8341714640 1 -- 172.21.15.78:0/2414093779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f833c100c10 con 0x7f833c101150 2024-08-18T23:46:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.008+0000 7f8338ff9640 1 -- 172.21.15.78:0/2414093779 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f83240099d0 con 0x7f833c101150 2024-08-18T23:46:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.009+0000 7f8338ff9640 1 --2- 172.21.15.78:0/2414093779 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f83280a2b80 0x7f83280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.009+0000 7f833affd640 1 --2- 172.21.15.78:0/2414093779 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f83280a2b80 0x7f83280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.009+0000 7f8338ff9640 1 -- 172.21.15.78:0/2414093779 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f83240ceef0 con 0x7f833c101150 2024-08-18T23:46:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.010+0000 7f833affd640 1 --2- 172.21.15.78:0/2414093779 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f83280a2b80 0x7f83280a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f832c009c10 tx=0x7f832c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.013+0000 7f8338ff9640 1 -- 172.21.15.78:0/2414093779 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f832409b470 con 0x7f833c101150 2024-08-18T23:46:27.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.240+0000 7f8341714640 1 -- 172.21.15.78:0/2414093779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f833c1097a0 con 0x7f833c101150 2024-08-18T23:46:27.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.242+0000 7f8338ff9640 1 -- 172.21.15.78:0/2414093779 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11933 (secure 0 0 0) 0x7f832409f770 con 0x7f833c101150 2024-08-18T23:46:27.244 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:46:27.244 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":26,"btime":"2024-08-18T23:45:33:306257+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:45:32.491054+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:46:27.244 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 26 2024-08-18T23:46:27.244 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-18T23:46:27.245 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-18T23:46:27.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.246+0000 7f8341714640 1 -- 172.21.15.78:0/2414093779 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f83280a2b80 msgr2=0x7f83280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:27.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.246+0000 7f8341714640 1 --2- 172.21.15.78:0/2414093779 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f83280a2b80 0x7f83280a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f832c009c10 tx=0x7f832c005f90 comp rx=0 tx=0).stop 2024-08-18T23:46:27.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.246+0000 7f8341714640 1 -- 172.21.15.78:0/2414093779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f833c101150 msgr2=0x7f833c198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:27.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.246+0000 7f8341714640 1 --2- 172.21.15.78:0/2414093779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f833c101150 0x7f833c198670 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f832400d6d0 tx=0x7f832400db90 comp rx=0 tx=0).stop 2024-08-18T23:46:27.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.246+0000 7f8341714640 1 -- 172.21.15.78:0/2414093779 shutdown_connections 2024-08-18T23:46:27.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.246+0000 7f8341714640 1 --2- 172.21.15.78:0/2414093779 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f83280a2b80 0x7f83280a5040 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:27.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.246+0000 7f8341714640 1 --2- 172.21.15.78:0/2414093779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f833c1067b0 0x7f833c19ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:27.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.246+0000 7f8341714640 1 --2- 172.21.15.78:0/2414093779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f833c101150 0x7f833c198670 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:27.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.246+0000 7f8341714640 1 --2- 172.21.15.78:0/2414093779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c1007b0 0x7f833c198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:27.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.246+0000 7f8341714640 1 -- 172.21.15.78:0/2414093779 >> 172.21.15.78:0/2414093779 conn(0x7f833c0fc4d0 msgr2=0x7f833c06a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:27.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.246+0000 7f8341714640 1 -- 172.21.15.78:0/2414093779 shutdown_connections 2024-08-18T23:46:27.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:27.246+0000 7f8341714640 1 -- 172.21.15.78:0/2414093779 wait complete. 2024-08-18T23:46:27.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:27 smithi123 ceph-mon[21808]: pgmap v414: 129 pgs: 129 active+clean; 283 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 702 op/s 2024-08-18T23:46:27.626 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:27 smithi078 ceph-mon[18440]: pgmap v414: 129 pgs: 129 active+clean; 283 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 702 op/s 2024-08-18T23:46:27.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:27 smithi161 ceph-mon[22027]: pgmap v414: 129 pgs: 129 active+clean; 283 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 702 op/s 2024-08-18T23:46:28.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:28 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2414093779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:46:28.609 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:28 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2414093779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:46:28.639 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-08-18T23:46:28.639 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-08-18T23:46:28.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:28 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2414093779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:46:28.745 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:46:29.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:29 smithi123 ceph-mon[21808]: pgmap v415: 129 pgs: 129 active+clean; 279 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 763 op/s 2024-08-18T23:46:29.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:29 smithi161 ceph-mon[22027]: pgmap v415: 129 pgs: 129 active+clean; 279 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 763 op/s 2024-08-18T23:46:29.724 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:29 smithi078 ceph-mon[18440]: pgmap v415: 129 pgs: 129 active+clean; 279 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 763 op/s 2024-08-18T23:46:29.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.907+0000 7f119be6d640 1 -- 172.21.15.78:0/1354922055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11941018e0 msgr2=0x7f1194101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:29.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.907+0000 7f119be6d640 1 --2- 172.21.15.78:0/1354922055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11941018e0 0x7f1194101d40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1184009930 tx=0x7f118402a420 comp rx=0 tx=0).stop 2024-08-18T23:46:29.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.907+0000 7f119be6d640 1 -- 172.21.15.78:0/1354922055 shutdown_connections 2024-08-18T23:46:29.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.907+0000 7f119be6d640 1 --2- 172.21.15.78:0/1354922055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1194102280 0x7f119410e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.907+0000 7f119be6d640 1 --2- 172.21.15.78:0/1354922055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11941018e0 0x7f1194101d40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.907+0000 7f119be6d640 1 --2- 172.21.15.78:0/1354922055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11941078e0 0x7f1194107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.907+0000 7f119be6d640 1 -- 172.21.15.78:0/1354922055 >> 172.21.15.78:0/1354922055 conn(0x7f11940fd600 msgr2=0x7f11940ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:29.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.907+0000 7f119be6d640 1 -- 172.21.15.78:0/1354922055 shutdown_connections 2024-08-18T23:46:29.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.907+0000 7f119be6d640 1 -- 172.21.15.78:0/1354922055 wait complete. 2024-08-18T23:46:29.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.908+0000 7f119be6d640 1 Processor -- start 2024-08-18T23:46:29.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.908+0000 7f119be6d640 1 -- start start 2024-08-18T23:46:29.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.908+0000 7f119be6d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11941018e0 0x7f11941981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:29.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.908+0000 7f119be6d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1194102280 0x7f1194198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:29.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.908+0000 7f119be6d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11941078e0 0x7f119419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:29.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.908+0000 7f119be6d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1194111ca0 con 0x7f11941018e0 2024-08-18T23:46:29.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.908+0000 7f119be6d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1194111b20 con 0x7f1194102280 2024-08-18T23:46:29.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.908+0000 7f119be6d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1194111e20 con 0x7f11941078e0 2024-08-18T23:46:29.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.908+0000 7f1199c09640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11941018e0 0x7f11941981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:29.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.908+0000 7f1199c09640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11941018e0 0x7f11941981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35414/0 (socket says 172.21.15.78:35414) 2024-08-18T23:46:29.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.908+0000 7f1199c09640 1 -- 172.21.15.78:0/2875388308 learned_addr learned my addr 172.21.15.78:0/2875388308 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:29.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.908+0000 7f1199408640 1 --2- 172.21.15.78:0/2875388308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1194102280 0x7f1194198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:29.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.908+0000 7f119a40a640 1 --2- 172.21.15.78:0/2875388308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11941078e0 0x7f119419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:29.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.909+0000 7f1199c09640 1 -- 172.21.15.78:0/2875388308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11941078e0 msgr2=0x7f119419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:29.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.909+0000 7f1199c09640 1 --2- 172.21.15.78:0/2875388308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11941078e0 0x7f119419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.909+0000 7f1199c09640 1 -- 172.21.15.78:0/2875388308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1194102280 msgr2=0x7f1194198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:29.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.909+0000 7f1199c09640 1 --2- 172.21.15.78:0/2875388308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1194102280 0x7f1194198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.909+0000 7f1199c09640 1 -- 172.21.15.78:0/2875388308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119419d170 con 0x7f11941018e0 2024-08-18T23:46:29.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.909+0000 7f1199c09640 1 --2- 172.21.15.78:0/2875388308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11941018e0 0x7f11941981c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f119000d950 tx=0x7f119000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:29.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.909+0000 7f118affd640 1 -- 172.21.15.78:0/2875388308 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190017030 con 0x7f11941018e0 2024-08-18T23:46:29.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.909+0000 7f118affd640 1 -- 172.21.15.78:0/2875388308 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1190004020 con 0x7f11941018e0 2024-08-18T23:46:29.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.909+0000 7f119be6d640 1 -- 172.21.15.78:0/2875388308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f119419d460 con 0x7f11941018e0 2024-08-18T23:46:29.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.909+0000 7f118affd640 1 -- 172.21.15.78:0/2875388308 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11900047a0 con 0x7f11941018e0 2024-08-18T23:46:29.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.909+0000 7f119be6d640 1 -- 172.21.15.78:0/2875388308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11941a4d40 con 0x7f11941018e0 2024-08-18T23:46:29.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.911+0000 7f118affd640 1 -- 172.21.15.78:0/2875388308 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f11900041d0 con 0x7f11941018e0 2024-08-18T23:46:29.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.912+0000 7f119e26b740 1 Processor -- start 2024-08-18T23:46:29.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.912+0000 7f119e26b740 1 -- start start 2024-08-18T23:46:29.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.912+0000 7f119e26b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a641c0c0 0x5608a641c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:29.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.912+0000 7f119e26b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a641c9e0 0x5608a642b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:29.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.912+0000 7f118affd640 1 --2- 172.21.15.78:0/2875388308 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f11800a2b80 0x7f11800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:29.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.912+0000 7f119e26b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a642b7f0 0x5608a642dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:29.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.912+0000 7f119e26b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5608a63dfe40 con 0x5608a641c0c0 2024-08-18T23:46:29.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.912+0000 7f119e26b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5608a63817f0 con 0x5608a641c9e0 2024-08-18T23:46:29.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.912+0000 7f119e26b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5608a6328fb0 con 0x5608a642b7f0 2024-08-18T23:46:29.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.913+0000 7f119a40a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a641c0c0 0x5608a641c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:29.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.913+0000 7f1199c09640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a641c9e0 0x5608a642b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:29.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.913+0000 7f119a40a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a641c0c0 0x5608a641c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35420/0 (socket says 172.21.15.78:35420) 2024-08-18T23:46:29.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.913+0000 7f119a40a640 1 -- 172.21.15.78:0/2823649091 learned_addr learned my addr 172.21.15.78:0/2823649091 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:29.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.913+0000 7f1199408640 1 --2- 172.21.15.78:0/2875388308 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f11800a2b80 0x7f11800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:29.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.913+0000 7f119a40a640 1 --2- 172.21.15.78:0/2823649091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a642b7f0 0x5608a642dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:29.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.913+0000 7f118affd640 1 -- 172.21.15.78:0/2875388308 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f11900ced00 con 0x7f11941018e0 2024-08-18T23:46:29.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.913+0000 7f1199c09640 1 -- 172.21.15.78:0/2823649091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a642b7f0 msgr2=0x5608a642dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:29.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.913+0000 7f1199c09640 1 --2- 172.21.15.78:0/2823649091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a642b7f0 0x5608a642dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.913+0000 7f1199c09640 1 -- 172.21.15.78:0/2823649091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a641c0c0 msgr2=0x5608a641c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.913+0000 7f1199c09640 1 --2- 172.21.15.78:0/2823649091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a641c0c0 0x5608a641c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.913+0000 7f1199c09640 1 -- 172.21.15.78:0/2823649091 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608a60f36a0 con 0x5608a641c9e0 2024-08-18T23:46:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.913+0000 7f1199c09640 1 --2- 172.21.15.78:0/2823649091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a641c9e0 0x5608a642b2b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f119009ae70 tx=0x7f1190000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78b4d2586be9db24 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.913+0000 7f117ffff640 1 -- 172.21.15.78:0/2823649091 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119009b870 con 0x5608a641c9e0 2024-08-18T23:46:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.913+0000 7f1199408640 1 --2- 172.21.15.78:0/2875388308 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f11800a2b80 0x7f11800a5040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1184009900 tx=0x7f11840058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.913+0000 7f117ffff640 1 -- 172.21.15.78:0/2823649091 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f119009b170 con 0x5608a641c9e0 2024-08-18T23:46:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.913+0000 7f117ffff640 1 -- 172.21.15.78:0/2823649091 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11900bb0d0 con 0x5608a641c9e0 2024-08-18T23:46:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.914+0000 7f119e26b740 1 -- 172.21.15.78:0/2823649091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a641c9e0 msgr2=0x5608a642b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.914+0000 7f119e26b740 1 --2- 172.21.15.78:0/2823649091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a641c9e0 0x5608a642b2b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f119009ae70 tx=0x7f1190000bc0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.914+0000 7f119e26b740 1 -- 172.21.15.78:0/2823649091 shutdown_connections 2024-08-18T23:46:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.914+0000 7f119e26b740 1 --2- 172.21.15.78:0/2823649091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a642b7f0 0x5608a642dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.914+0000 7f119e26b740 1 --2- 172.21.15.78:0/2823649091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a641c9e0 0x5608a642b2b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.914+0000 7f119e26b740 1 --2- 172.21.15.78:0/2823649091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a641c0c0 0x5608a641c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.914+0000 7f119e26b740 1 -- 172.21.15.78:0/2823649091 >> 172.21.15.78:0/2823649091 conn(0x5608a6418790 msgr2=0x5608a6418b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:29.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.914+0000 7f119e26b740 1 -- 172.21.15.78:0/2823649091 shutdown_connections 2024-08-18T23:46:29.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.914+0000 7f119e26b740 1 -- 172.21.15.78:0/2823649091 wait complete. 2024-08-18T23:46:29.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.915+0000 7f119e26b740 1 -- start start 2024-08-18T23:46:29.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.915+0000 7f119e26b740 1 Processor -- start 2024-08-18T23:46:29.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.915+0000 7f119e26b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a641c0c0 0x5608a64287d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:29.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.915+0000 7f119e26b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a641c9e0 0x5608a6428d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:29.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.915+0000 7f119e26b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a642b7f0 0x5608a6429250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:29.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.915+0000 7f119e26b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5608a633ccb0 con 0x5608a642b7f0 2024-08-18T23:46:29.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.915+0000 7f119e26b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5608a637d7b0 con 0x5608a641c0c0 2024-08-18T23:46:29.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.915+0000 7f119e26b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5608a642e300 con 0x5608a641c9e0 2024-08-18T23:46:29.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.915+0000 7f119e26b740 10 client.0 _ll_register_callbacks cb 0x5608a6090ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:46:29.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.915+0000 7f1199c09640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a641c0c0 0x5608a64287d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:29.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.915+0000 7f1199c09640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a641c0c0 0x5608a64287d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:39234/0 (socket says 172.21.15.78:39234) 2024-08-18T23:46:29.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.915+0000 7f119a40a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a642b7f0 0x5608a6429250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:29.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.915+0000 7f1199c09640 1 -- 172.21.15.78:0/4040555857 learned_addr learned my addr 172.21.15.78:0/4040555857 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:29.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.915+0000 7f1199408640 1 --2- 172.21.15.78:0/4040555857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a641c9e0 0x5608a6428d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:29.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.916+0000 7f1199c09640 1 -- 172.21.15.78:0/4040555857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a641c9e0 msgr2=0x5608a6428d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:29.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.916+0000 7f1199c09640 1 --2- 172.21.15.78:0/4040555857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a641c9e0 0x5608a6428d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.916+0000 7f1199c09640 1 -- 172.21.15.78:0/4040555857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a642b7f0 msgr2=0x5608a6429250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:29.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.916+0000 7f1199c09640 1 --2- 172.21.15.78:0/4040555857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a642b7f0 0x5608a6429250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.916+0000 7f1199c09640 1 -- 172.21.15.78:0/4040555857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608a633bbc0 con 0x5608a641c0c0 2024-08-18T23:46:29.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.916+0000 7f119a40a640 1 --2- 172.21.15.78:0/4040555857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a642b7f0 0x5608a6429250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:46:29.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.916+0000 7f1199c09640 1 --2- 172.21.15.78:0/4040555857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a641c0c0 0x5608a64287d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1190005fd0 tx=0x7f1190095b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee266384fc0999dd server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:29.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.916+0000 7f117d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:46:29.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.916+0000 7f117d7fa640 1 -- 172.21.15.78:0/4040555857 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11900a3de0 con 0x5608a641c0c0 2024-08-18T23:46:29.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.916+0000 7f117d7fa640 1 -- 172.21.15.78:0/4040555857 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f11900bb0d0 con 0x5608a641c0c0 2024-08-18T23:46:29.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.916+0000 7f119e26b740 1 -- 172.21.15.78:0/4040555857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f119009b170 con 0x5608a641c0c0 2024-08-18T23:46:29.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.916+0000 7f117d7fa640 1 -- 172.21.15.78:0/4040555857 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f119009b870 con 0x5608a641c0c0 2024-08-18T23:46:29.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.916+0000 7f119e26b740 10 client.25280 fetch_fsmap learned FSMap version 26 2024-08-18T23:46:29.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.916+0000 7f119e26b740 1 -- 172.21.15.78:0/4040555857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608a643a2d0 con 0x5608a641c0c0 2024-08-18T23:46:29.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.916+0000 7f117d7fa640 1 -- 172.21.15.78:0/4040555857 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f119009b170 con 0x5608a641c0c0 2024-08-18T23:46:29.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.917+0000 7f117d7fa640 1 -- 172.21.15.78:0/4040555857 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f119009a5c0 con 0x5608a641c0c0 2024-08-18T23:46:29.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.917+0000 7f119e26b740 10 client.25280 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:46:29.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.917+0000 7f119e26b740 10 client.25280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:46:29.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.917+0000 7f119e26b740 20 client.25280 populate_metadata read hostname 'smithi078' 2024-08-18T23:46:29.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.917+0000 7f119e26b740 1 --2- 172.21.15.78:0/4040555857 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5608a643b620 0x5608a643da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:29.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.917+0000 7f119e26b740 4 client.25280 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:46:29.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.917+0000 7f119e26b740 1 -- 172.21.15.78:0/4040555857 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608a638ed60 con 0x5608a643b620 2024-08-18T23:46:29.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.917+0000 7f1199408640 1 --2- 172.21.15.78:0/4040555857 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5608a643b620 0x5608a643da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:29.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.917+0000 7f117d7fa640 10 client.25280 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:29.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.917+0000 7f1199408640 1 --2- 172.21.15.78:0/4040555857 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5608a643b620 0x5608a643da10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f773ae7339b957d9 server_cookie=a163006acfd75104 in_seq=0 out_seq=0 2024-08-18T23:46:29.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.920+0000 7f117d7fa640 1 -- 172.21.15.78:0/4040555857 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5608a638ed60 con 0x5608a643b620 2024-08-18T23:46:29.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.920+0000 7f117d7fa640 10 client.25280 handle_command_reply: tid=0 2024-08-18T23:46:29.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.920+0000 7f119e26b740 1 client.25280 shutdown 2024-08-18T23:46:29.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.921+0000 7f119e26b740 1 -- 172.21.15.78:0/4040555857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a641c0c0 msgr2=0x5608a64287d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:29.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.921+0000 7f119e26b740 1 --2- 172.21.15.78:0/4040555857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a641c0c0 0x5608a64287d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1190005fd0 tx=0x7f1190095b40 comp rx=0 tx=0).stop 2024-08-18T23:46:29.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.921+0000 7f119e26b740 1 -- 172.21.15.78:0/4040555857 shutdown_connections 2024-08-18T23:46:29.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.921+0000 7f119e26b740 1 --2- 172.21.15.78:0/4040555857 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5608a643b620 0x5608a643da10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.921+0000 7f119e26b740 1 --2- 172.21.15.78:0/4040555857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a642b7f0 0x5608a6429250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.921+0000 7f119e26b740 1 --2- 172.21.15.78:0/4040555857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a641c9e0 0x5608a6428d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.921+0000 7f119e26b740 1 --2- 172.21.15.78:0/4040555857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a641c0c0 0x5608a64287d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.921+0000 7f117d7fa640 0 client.25280 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:29.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.921+0000 7f119e26b740 1 -- 172.21.15.78:0/4040555857 >> 172.21.15.78:0/4040555857 conn(0x5608a6418790 msgr2=0x5608a641e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:29.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.921+0000 7f119e26b740 1 -- 172.21.15.78:0/4040555857 shutdown_connections 2024-08-18T23:46:29.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.921+0000 7f119e26b740 1 -- 172.21.15.78:0/4040555857 wait complete. 2024-08-18T23:46:29.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.921+0000 7f119e26b740 20 client.25280 trim_cache size 0 max 16384 2024-08-18T23:46:29.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.938+0000 7f119e26b740 1 Processor -- start 2024-08-18T23:46:29.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.938+0000 7f119e26b740 1 -- start start 2024-08-18T23:46:29.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.938+0000 7f119e26b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a641c0c0 0x5608a642b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:29.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.938+0000 7f119e26b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a641c9e0 0x5608a6425a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:29.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.938+0000 7f119e26b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a642b7f0 0x5608a6425f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:29.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.938+0000 7f119e26b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11941af170 con 0x5608a641c9e0 2024-08-18T23:46:29.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.938+0000 7f119e26b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f11941aca40 con 0x5608a642b7f0 2024-08-18T23:46:29.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.938+0000 7f119e26b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f11941aa890 con 0x5608a641c0c0 2024-08-18T23:46:29.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.939+0000 7f119a40a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a642b7f0 0x5608a6425f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:29.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.939+0000 7f1199408640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a641c9e0 0x5608a6425a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:29.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.939+0000 7f1199408640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a641c9e0 0x5608a6425a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35442/0 (socket says 172.21.15.78:35442) 2024-08-18T23:46:29.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.939+0000 7f1199408640 1 -- 172.21.15.78:0/4098896975 learned_addr learned my addr 172.21.15.78:0/4098896975 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:29.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.939+0000 7f1199c09640 1 --2- 172.21.15.78:0/4098896975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a641c0c0 0x5608a642b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:29.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.939+0000 7f1199408640 1 -- 172.21.15.78:0/4098896975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a641c0c0 msgr2=0x5608a642b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:29.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.939+0000 7f1199408640 1 --2- 172.21.15.78:0/4098896975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a641c0c0 0x5608a642b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.939+0000 7f1199408640 1 -- 172.21.15.78:0/4098896975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a642b7f0 msgr2=0x5608a6425f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:29.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.939+0000 7f1199408640 1 --2- 172.21.15.78:0/4098896975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a642b7f0 0x5608a6425f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.939+0000 7f1199408640 1 -- 172.21.15.78:0/4098896975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608a6423580 con 0x5608a641c9e0 2024-08-18T23:46:29.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.939+0000 7f119a40a640 1 --2- 172.21.15.78:0/4098896975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a642b7f0 0x5608a6425f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:46:29.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.939+0000 7f1199408640 1 --2- 172.21.15.78:0/4098896975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a641c9e0 0x5608a6425a30 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1184005ec0 tx=0x7f1184040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf7e70c04e761a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:29.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.939+0000 7f117f7fe640 1 -- 172.21.15.78:0/4098896975 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1184056070 con 0x5608a641c9e0 2024-08-18T23:46:29.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.939+0000 7f117f7fe640 1 -- 172.21.15.78:0/4098896975 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11840316b0 con 0x5608a641c9e0 2024-08-18T23:46:29.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.939+0000 7f117f7fe640 1 -- 172.21.15.78:0/4098896975 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1184035040 con 0x5608a641c9e0 2024-08-18T23:46:29.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.940+0000 7f119e26b740 1 -- 172.21.15.78:0/4098896975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a641c9e0 msgr2=0x5608a6425a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:29.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.940+0000 7f119e26b740 1 --2- 172.21.15.78:0/4098896975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a641c9e0 0x5608a6425a30 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1184005ec0 tx=0x7f1184040000 comp rx=0 tx=0).stop 2024-08-18T23:46:29.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.940+0000 7f119e26b740 1 -- 172.21.15.78:0/4098896975 shutdown_connections 2024-08-18T23:46:29.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.940+0000 7f119e26b740 1 --2- 172.21.15.78:0/4098896975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a642b7f0 0x5608a6425f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.940+0000 7f119e26b740 1 --2- 172.21.15.78:0/4098896975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a641c9e0 0x5608a6425a30 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.940+0000 7f119e26b740 1 --2- 172.21.15.78:0/4098896975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a641c0c0 0x5608a642b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.940+0000 7f119e26b740 1 -- 172.21.15.78:0/4098896975 >> 172.21.15.78:0/4098896975 conn(0x5608a6418790 msgr2=0x5608a64278c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:29.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.940+0000 7f119e26b740 1 -- 172.21.15.78:0/4098896975 shutdown_connections 2024-08-18T23:46:29.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.940+0000 7f119e26b740 1 -- 172.21.15.78:0/4098896975 wait complete. 2024-08-18T23:46:29.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.940+0000 7f119e26b740 1 -- start start 2024-08-18T23:46:29.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.940+0000 7f119e26b740 1 Processor -- start 2024-08-18T23:46:29.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.941+0000 7f119e26b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a641c0c0 0x5608a6440ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:29.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.941+0000 7f119e26b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a641c9e0 0x5608a643d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:29.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.941+0000 7f119e26b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a642b7f0 0x5608a643d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:29.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.941+0000 7f119e26b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11941aaf00 con 0x5608a642b7f0 2024-08-18T23:46:29.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.941+0000 7f119e26b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f11941ac370 con 0x5608a641c9e0 2024-08-18T23:46:29.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.941+0000 7f119e26b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f11941ab5d0 con 0x5608a641c0c0 2024-08-18T23:46:29.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.941+0000 7f119e26b740 10 client.0 _ll_register_callbacks cb 0x7f118c009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:46:29.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.941+0000 7f119a40a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a642b7f0 0x5608a643d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:29.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.941+0000 7f119a40a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a642b7f0 0x5608a643d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35456/0 (socket says 172.21.15.78:35456) 2024-08-18T23:46:29.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.941+0000 7f119a40a640 1 -- 172.21.15.78:0/1753073987 learned_addr learned my addr 172.21.15.78:0/1753073987 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:29.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.941+0000 7f1199c09640 1 --2- 172.21.15.78:0/1753073987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a641c0c0 0x5608a6440ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:29.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.941+0000 7f1199408640 1 --2- 172.21.15.78:0/1753073987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a641c9e0 0x5608a643d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:29.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.941+0000 7f119a40a640 1 -- 172.21.15.78:0/1753073987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a641c0c0 msgr2=0x5608a6440ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:29.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.941+0000 7f119a40a640 1 --2- 172.21.15.78:0/1753073987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a641c0c0 0x5608a6440ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.941+0000 7f119a40a640 1 -- 172.21.15.78:0/1753073987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a641c9e0 msgr2=0x5608a643d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:29.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.941+0000 7f119a40a640 1 --2- 172.21.15.78:0/1753073987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a641c9e0 0x5608a643d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:29.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.941+0000 7f119a40a640 1 -- 172.21.15.78:0/1753073987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608a643e030 con 0x5608a642b7f0 2024-08-18T23:46:29.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.942+0000 7f119a40a640 1 --2- 172.21.15.78:0/1753073987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a642b7f0 0x5608a643d800 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f118c010460 tx=0x7f118c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fb537f04617960 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:29.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.942+0000 7f117d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:46:29.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.942+0000 7f117d7fa640 1 -- 172.21.15.78:0/1753073987 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c003b80 con 0x5608a642b7f0 2024-08-18T23:46:29.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.942+0000 7f117d7fa640 1 -- 172.21.15.78:0/1753073987 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f118c01c040 con 0x5608a642b7f0 2024-08-18T23:46:29.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.942+0000 7f119e26b740 1 -- 172.21.15.78:0/1753073987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11840316b0 con 0x5608a642b7f0 2024-08-18T23:46:29.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.942+0000 7f117d7fa640 1 -- 172.21.15.78:0/1753073987 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f118c02c050 con 0x5608a642b7f0 2024-08-18T23:46:29.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.942+0000 7f119e26b740 10 client.15807 fetch_fsmap learned FSMap version 26 2024-08-18T23:46:29.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.942+0000 7f119e26b740 1 -- 172.21.15.78:0/1753073987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608a64a50a0 con 0x5608a642b7f0 2024-08-18T23:46:29.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.942+0000 7f117d7fa640 1 -- 172.21.15.78:0/1753073987 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11840316b0 con 0x5608a642b7f0 2024-08-18T23:46:29.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.942+0000 7f117d7fa640 1 -- 172.21.15.78:0/1753073987 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f118c0046e0 con 0x5608a642b7f0 2024-08-18T23:46:29.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.942+0000 7f119e26b740 10 client.15807 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:46:29.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.942+0000 7f119e26b740 10 client.15807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:46:29.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.942+0000 7f119e26b740 20 client.15807 populate_metadata read hostname 'smithi078' 2024-08-18T23:46:29.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.942+0000 7f119e26b740 1 --2- 172.21.15.78:0/1753073987 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5608a64a58d0 0x5608a64a7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:29.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.942+0000 7f119e26b740 4 client.15807 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-18T23:46:29.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.942+0000 7f119e26b740 1 -- 172.21.15.78:0/1753073987 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5608a6389270 con 0x5608a64a58d0 2024-08-18T23:46:29.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.943+0000 7f1199c09640 1 --2- 172.21.15.78:0/1753073987 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5608a64a58d0 0x5608a64a7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:29.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.943+0000 7f1199c09640 1 --2- 172.21.15.78:0/1753073987 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5608a64a58d0 0x5608a64a7cf0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fecf4633b8e5422 server_cookie=b685ce33bb34ff72 in_seq=0 out_seq=0 2024-08-18T23:46:29.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:29.943+0000 7f117d7fa640 10 client.15807 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:30.042 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:46:30.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.145+0000 7f117d7fa640 1 -- 172.21.15.78:0/1753073987 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5608a6389270 con 0x5608a64a58d0 2024-08-18T23:46:30.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.145+0000 7f117d7fa640 10 client.15807 handle_command_reply: tid=0 2024-08-18T23:46:30.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.145+0000 7f119e26b740 1 client.15807 shutdown 2024-08-18T23:46:30.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.146+0000 7f119e26b740 1 -- 172.21.15.78:0/1753073987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a642b7f0 msgr2=0x5608a643d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:30.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.146+0000 7f119e26b740 1 --2- 172.21.15.78:0/1753073987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a642b7f0 0x5608a643d800 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f118c010460 tx=0x7f118c010860 comp rx=0 tx=0).stop 2024-08-18T23:46:30.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.146+0000 7f119e26b740 1 -- 172.21.15.78:0/1753073987 shutdown_connections 2024-08-18T23:46:30.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.146+0000 7f119e26b740 1 --2- 172.21.15.78:0/1753073987 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5608a64a58d0 0x5608a64a7cf0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:30.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.146+0000 7f119e26b740 1 --2- 172.21.15.78:0/1753073987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5608a642b7f0 0x5608a643d800 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:30.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.146+0000 7f117d7fa640 0 client.15807 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:30.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.146+0000 7f119e26b740 1 --2- 172.21.15.78:0/1753073987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608a641c9e0 0x5608a643d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:30.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.146+0000 7f119e26b740 1 --2- 172.21.15.78:0/1753073987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5608a641c0c0 0x5608a6440ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:30.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.146+0000 7f119e26b740 1 -- 172.21.15.78:0/1753073987 >> 172.21.15.78:0/1753073987 conn(0x5608a6418790 msgr2=0x5608a6436590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:30.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.146+0000 7f119e26b740 1 -- 172.21.15.78:0/1753073987 shutdown_connections 2024-08-18T23:46:30.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.146+0000 7f119e26b740 1 -- 172.21.15.78:0/1753073987 wait complete. 2024-08-18T23:46:30.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.146+0000 7f119e26b740 20 client.15807 trim_cache size 0 max 16384 2024-08-18T23:46:30.149 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:46:30.149 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-08-18T23:46:30.149 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "00dbb5e1-ab64-413e-abca-e8a570e1b9fa", 2024-08-18T23:46:30.149 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-08-18T23:46:30.149 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:46:30.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.147+0000 7f119be6d640 1 -- 172.21.15.78:0/2875388308 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f11800a2b80 msgr2=0x7f11800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:30.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.147+0000 7f119be6d640 1 --2- 172.21.15.78:0/2875388308 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f11800a2b80 0x7f11800a5040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1184009900 tx=0x7f11840058d0 comp rx=0 tx=0).stop 2024-08-18T23:46:30.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.147+0000 7f119be6d640 1 -- 172.21.15.78:0/2875388308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11941018e0 msgr2=0x7f11941981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:30.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.147+0000 7f119be6d640 1 --2- 172.21.15.78:0/2875388308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11941018e0 0x7f11941981c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f119000d950 tx=0x7f119000de10 comp rx=0 tx=0).stop 2024-08-18T23:46:30.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.147+0000 7f119be6d640 1 -- 172.21.15.78:0/2875388308 shutdown_connections 2024-08-18T23:46:30.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.147+0000 7f119be6d640 1 --2- 172.21.15.78:0/2875388308 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f11800a2b80 0x7f11800a5040 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:30.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.147+0000 7f119be6d640 1 --2- 172.21.15.78:0/2875388308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11941078e0 0x7f119419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:30.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.147+0000 7f119be6d640 1 --2- 172.21.15.78:0/2875388308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1194102280 0x7f1194198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:30.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.147+0000 7f119be6d640 1 --2- 172.21.15.78:0/2875388308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11941018e0 0x7f11941981c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:30.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.147+0000 7f119be6d640 1 -- 172.21.15.78:0/2875388308 >> 172.21.15.78:0/2875388308 conn(0x7f11940fd600 msgr2=0x7f11940fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:30.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.147+0000 7f119be6d640 1 -- 172.21.15.78:0/2875388308 shutdown_connections 2024-08-18T23:46:30.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:30.147+0000 7f119be6d640 1 -- 172.21.15.78:0/2875388308 wait complete. 2024-08-18T23:46:31.135 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:46:31.321 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:31 smithi078 ceph-mon[18440]: scrub queued for path: / 2024-08-18T23:46:31.321 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:31 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:46:31.321 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:31 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:46:31.321 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:31 smithi078 ceph-mon[18440]: pgmap v416: 129 pgs: 129 active+clean; 280 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 794 op/s 2024-08-18T23:46:31.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:31 smithi123 ceph-mon[21808]: scrub queued for path: / 2024-08-18T23:46:31.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:31 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:46:31.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:31 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:46:31.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:31 smithi123 ceph-mon[21808]: pgmap v416: 129 pgs: 129 active+clean; 280 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 794 op/s 2024-08-18T23:46:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:31 smithi161 ceph-mon[22027]: scrub queued for path: / 2024-08-18T23:46:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:31 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:46:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:31 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:46:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:31 smithi161 ceph-mon[22027]: pgmap v416: 129 pgs: 129 active+clean; 280 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 794 op/s 2024-08-18T23:46:31.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.806+0000 7fbb5f04b640 1 -- 172.21.15.78:0/1062954066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb58109180 msgr2=0x7fbb5810ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:31.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.806+0000 7fbb5f04b640 1 --2- 172.21.15.78:0/1062954066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb58109180 0x7fbb5810ff80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbb5001acb0 tx=0x7fbb5003b920 comp rx=0 tx=0).stop 2024-08-18T23:46:31.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.811+0000 7fbb5f04b640 1 -- 172.21.15.78:0/1062954066 shutdown_connections 2024-08-18T23:46:31.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.811+0000 7fbb5f04b640 1 --2- 172.21.15.78:0/1062954066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb5811ae70 0x7fbb5811d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.811+0000 7fbb5f04b640 1 --2- 172.21.15.78:0/1062954066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb58109560 0x7fbb581099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.811+0000 7fbb5f04b640 1 --2- 172.21.15.78:0/1062954066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb58109180 0x7fbb5810ff80 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.811+0000 7fbb5f04b640 1 -- 172.21.15.78:0/1062954066 >> 172.21.15.78:0/1062954066 conn(0x7fbb5806de70 msgr2=0x7fbb5806e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:31.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.811+0000 7fbb5f04b640 1 -- 172.21.15.78:0/1062954066 shutdown_connections 2024-08-18T23:46:31.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.815+0000 7fbb5f04b640 1 -- 172.21.15.78:0/1062954066 wait complete. 2024-08-18T23:46:31.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.815+0000 7fbb5f04b640 1 Processor -- start 2024-08-18T23:46:31.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.818+0000 7fbb5f04b640 1 -- start start 2024-08-18T23:46:31.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.818+0000 7fbb5f04b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb58109180 0x7fbb5811a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:31.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.818+0000 7fbb5f04b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb58109560 0x7fbb581167b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:31.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.818+0000 7fbb5f04b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb5811ae70 0x7fbb58116cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:31.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.818+0000 7fbb5f04b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb58113110 con 0x7fbb58109180 2024-08-18T23:46:31.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.818+0000 7fbb5f04b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbb58112f90 con 0x7fbb58109560 2024-08-18T23:46:31.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.818+0000 7fbb5f04b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbb58113290 con 0x7fbb5811ae70 2024-08-18T23:46:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.819+0000 7fbb57fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb58109560 0x7fbb581167b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.819+0000 7fbb57fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb58109560 0x7fbb581167b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:39248/0 (socket says 172.21.15.78:39248) 2024-08-18T23:46:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.819+0000 7fbb5d5e8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb5811ae70 0x7fbb58116cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.819+0000 7fbb5d5e8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb5811ae70 0x7fbb58116cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:53678/0 (socket says 172.21.15.78:53678) 2024-08-18T23:46:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.819+0000 7fbb5d5e8640 1 -- 172.21.15.78:0/15547722 learned_addr learned my addr 172.21.15.78:0/15547722 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.820+0000 7fbb5cde7640 1 --2- 172.21.15.78:0/15547722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb58109180 0x7fbb5811a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.820+0000 7fbb57fff640 1 -- 172.21.15.78:0/15547722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb5811ae70 msgr2=0x7fbb58116cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.820+0000 7fbb57fff640 1 --2- 172.21.15.78:0/15547722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb5811ae70 0x7fbb58116cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.820+0000 7fbb57fff640 1 -- 172.21.15.78:0/15547722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb58109180 msgr2=0x7fbb5811a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:31.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.820+0000 7fbb57fff640 1 --2- 172.21.15.78:0/15547722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb58109180 0x7fbb5811a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.820+0000 7fbb57fff640 1 -- 172.21.15.78:0/15547722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb58117560 con 0x7fbb58109560 2024-08-18T23:46:31.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.820+0000 7fbb5d5e8640 1 --2- 172.21.15.78:0/15547722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb5811ae70 0x7fbb58116cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:46:31.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.821+0000 7fbb57fff640 1 --2- 172.21.15.78:0/15547722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb58109560 0x7fbb581167b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009fd0 tx=0x7fbb4c00b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:31.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.823+0000 7fbb55ffb640 1 -- 172.21.15.78:0/15547722 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c015020 con 0x7fbb58109560 2024-08-18T23:46:31.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.823+0000 7fbb5f04b640 1 -- 172.21.15.78:0/15547722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb581aaf00 con 0x7fbb58109560 2024-08-18T23:46:31.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.823+0000 7fbb5f04b640 1 -- 172.21.15.78:0/15547722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb581ab2a0 con 0x7fbb58109560 2024-08-18T23:46:31.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.824+0000 7fbb55ffb640 1 -- 172.21.15.78:0/15547722 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb4c011d10 con 0x7fbb58109560 2024-08-18T23:46:31.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.824+0000 7fbb55ffb640 1 -- 172.21.15.78:0/15547722 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c00f2e0 con 0x7fbb58109560 2024-08-18T23:46:31.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.845+0000 7fbb61449740 1 Processor -- start 2024-08-18T23:46:31.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.847+0000 7fbb55ffb640 1 -- 172.21.15.78:0/15547722 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbb4c018020 con 0x7fbb58109560 2024-08-18T23:46:31.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.848+0000 7fbb55ffb640 1 --2- 172.21.15.78:0/15547722 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbb400a2d10 0x7fbb400a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:31.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.849+0000 7fbb55ffb640 1 -- 172.21.15.78:0/15547722 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fbb4c0ce820 con 0x7fbb58109560 2024-08-18T23:46:31.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.849+0000 7fbb61449740 1 -- start start 2024-08-18T23:46:31.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.849+0000 7fbb61449740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260a8130 0x55aa260a8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:31.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.849+0000 7fbb61449740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260a8a50 0x55aa260b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:31.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.849+0000 7fbb61449740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260b3310 0x55aa260b5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:31.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.849+0000 7fbb61449740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55aa2600da70 con 0x55aa260a8a50 2024-08-18T23:46:31.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.849+0000 7fbb61449740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aa25fa5780 con 0x55aa260a8130 2024-08-18T23:46:31.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.849+0000 7fbb61449740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55aa26009a30 con 0x55aa260b3310 2024-08-18T23:46:31.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.853+0000 7fbb57fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260a8a50 0x55aa260b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:31.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.853+0000 7fbb57fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260a8a50 0x55aa260b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35482/0 (socket says 172.21.15.78:35482) 2024-08-18T23:46:31.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.853+0000 7fbb57fff640 1 -- 172.21.15.78:0/4187073752 learned_addr learned my addr 172.21.15.78:0/4187073752 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:31.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.854+0000 7fbb5cde7640 1 --2- 172.21.15.78:0/4187073752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260a8130 0x55aa260a8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:31.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.854+0000 7fbb5d5e8640 1 --2- 172.21.15.78:0/4187073752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260b3310 0x55aa260b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:31.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.854+0000 7fbb5d5e8640 1 --2- 172.21.15.78:0/15547722 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbb400a2d10 0x7fbb400a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:31.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.854+0000 7fbb57fff640 1 -- 172.21.15.78:0/4187073752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260b3310 msgr2=0x55aa260b5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:31.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.854+0000 7fbb57fff640 1 --2- 172.21.15.78:0/4187073752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260b3310 0x55aa260b5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.854+0000 7fbb57fff640 1 -- 172.21.15.78:0/4187073752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260a8130 msgr2=0x55aa260a8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:31.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.854+0000 7fbb57fff640 1 --2- 172.21.15.78:0/4187073752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260a8130 0x55aa260a8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.854+0000 7fbb57fff640 1 -- 172.21.15.78:0/4187073752 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa25d7fb90 con 0x55aa260a8a50 2024-08-18T23:46:31.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.855+0000 7fbb57fff640 1 --2- 172.21.15.78:0/4187073752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260a8a50 0x55aa260b2dd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c09a9f0 tx=0x7fbb4c09b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927ab625a2b03922 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:31.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.860+0000 7fbb3effd640 1 -- 172.21.15.78:0/4187073752 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c0b8070 con 0x55aa260a8a50 2024-08-18T23:46:31.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.860+0000 7fbb3effd640 1 -- 172.21.15.78:0/4187073752 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb4c0a3960 con 0x55aa260a8a50 2024-08-18T23:46:31.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.861+0000 7fbb61449740 1 -- 172.21.15.78:0/4187073752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260a8a50 msgr2=0x55aa260b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:31.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.861+0000 7fbb61449740 1 --2- 172.21.15.78:0/4187073752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260a8a50 0x55aa260b2dd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c09a9f0 tx=0x7fbb4c09b4b0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.865+0000 7fbb5d5e8640 1 --2- 172.21.15.78:0/15547722 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbb400a2d10 0x7fbb400a51d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55aa260a5a40 tx=0x7fbb480090e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:31.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.866+0000 7fbb61449740 1 -- 172.21.15.78:0/4187073752 shutdown_connections 2024-08-18T23:46:31.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.866+0000 7fbb61449740 1 --2- 172.21.15.78:0/4187073752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260b3310 0x55aa260b5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.866+0000 7fbb61449740 1 --2- 172.21.15.78:0/4187073752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260a8a50 0x55aa260b2dd0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.866+0000 7fbb61449740 1 --2- 172.21.15.78:0/4187073752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260a8130 0x55aa260a8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.866+0000 7fbb61449740 1 -- 172.21.15.78:0/4187073752 >> 172.21.15.78:0/4187073752 conn(0x55aa260a46c0 msgr2=0x55aa260a4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:31.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.871+0000 7fbb61449740 1 -- 172.21.15.78:0/4187073752 shutdown_connections 2024-08-18T23:46:31.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.875+0000 7fbb61449740 1 -- 172.21.15.78:0/4187073752 wait complete. 2024-08-18T23:46:31.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.875+0000 7fbb61449740 1 -- start start 2024-08-18T23:46:31.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.875+0000 7fbb61449740 1 Processor -- start 2024-08-18T23:46:31.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.876+0000 7fbb61449740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260a8130 0x55aa260b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:31.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.876+0000 7fbb61449740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260a8a50 0x55aa260b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:31.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.876+0000 7fbb61449740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260b3310 0x55aa260b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:31.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.876+0000 7fbb61449740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55aa260b5d60 con 0x55aa260a8a50 2024-08-18T23:46:31.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.876+0000 7fbb61449740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aa25fc8990 con 0x55aa260b3310 2024-08-18T23:46:31.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.876+0000 7fbb61449740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55aa260b5ee0 con 0x55aa260a8130 2024-08-18T23:46:31.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.876+0000 7fbb61449740 10 client.0 _ll_register_callbacks cb 0x55aa25d1d090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:46:31.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.876+0000 7fbb5d5e8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260b3310 0x55aa260b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:31.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.876+0000 7fbb5d5e8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260b3310 0x55aa260b0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:39276/0 (socket says 172.21.15.78:39276) 2024-08-18T23:46:31.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.876+0000 7fbb5d5e8640 1 -- 172.21.15.78:0/2459060765 learned_addr learned my addr 172.21.15.78:0/2459060765 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:31.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.876+0000 7fbb5cde7640 1 --2- 172.21.15.78:0/2459060765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260a8130 0x55aa260b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:31.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.877+0000 7fbb5cde7640 1 -- 172.21.15.78:0/2459060765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260b3310 msgr2=0x55aa260b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:31.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.877+0000 7fbb5cde7640 1 --2- 172.21.15.78:0/2459060765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260b3310 0x55aa260b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.877+0000 7fbb5cde7640 1 -- 172.21.15.78:0/2459060765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260a8a50 msgr2=0x55aa260b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:31.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.877+0000 7fbb5cde7640 1 --2- 172.21.15.78:0/2459060765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260a8a50 0x55aa260b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.877+0000 7fbb5cde7640 1 -- 172.21.15.78:0/2459060765 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa25fc76c0 con 0x55aa260a8130 2024-08-18T23:46:31.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.879+0000 7fbb5cde7640 1 --2- 172.21.15.78:0/2459060765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260a8130 0x55aa260b0190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbb50006010 tx=0x7fbb50003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc6b329a51e58537 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:31.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.879+0000 7fbb37fff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:46:31.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.883+0000 7fbb37fff640 1 -- 172.21.15.78:0/2459060765 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb50045070 con 0x55aa260a8130 2024-08-18T23:46:31.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.883+0000 7fbb61449740 1 -- 172.21.15.78:0/2459060765 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb4c0a3960 con 0x55aa260a8130 2024-08-18T23:46:31.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.883+0000 7fbb37fff640 1 -- 172.21.15.78:0/2459060765 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fbb50019040 con 0x55aa260a8130 2024-08-18T23:46:31.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.887+0000 7fbb37fff640 1 -- 172.21.15.78:0/2459060765 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbb5005f050 con 0x55aa260a8130 2024-08-18T23:46:31.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.887+0000 7fbb61449740 10 client.25192 fetch_fsmap learned FSMap version 26 2024-08-18T23:46:31.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.887+0000 7fbb61449740 1 -- 172.21.15.78:0/2459060765 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aa260c1aa0 con 0x55aa260a8130 2024-08-18T23:46:31.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.891+0000 7fbb37fff640 1 -- 172.21.15.78:0/2459060765 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb4c0a3960 con 0x55aa260a8130 2024-08-18T23:46:31.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.891+0000 7fbb37fff640 1 -- 172.21.15.78:0/2459060765 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fbb50004680 con 0x55aa260a8130 2024-08-18T23:46:31.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.891+0000 7fbb61449740 10 client.25192 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:46:31.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.891+0000 7fbb61449740 10 client.25192 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:46:31.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.891+0000 7fbb61449740 20 client.25192 populate_metadata read hostname 'smithi078' 2024-08-18T23:46:31.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.891+0000 7fbb61449740 1 --2- 172.21.15.78:0/2459060765 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55aa260c33b0 0x55aa260c5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:31.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.891+0000 7fbb61449740 4 client.25192 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:46:31.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.891+0000 7fbb61449740 1 -- 172.21.15.78:0/2459060765 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa2601b1a0 con 0x55aa260c33b0 2024-08-18T23:46:31.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.893+0000 7fbb57fff640 1 --2- 172.21.15.78:0/2459060765 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55aa260c33b0 0x55aa260c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:31.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.899+0000 7fbb57fff640 1 --2- 172.21.15.78:0/2459060765 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55aa260c33b0 0x55aa260c5870 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2652e99381a76fe server_cookie=91ebb1e44a2bbac6 in_seq=0 out_seq=0 2024-08-18T23:46:31.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.901+0000 7fbb37fff640 10 client.25192 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:31.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.920+0000 7fbb37fff640 1 -- 172.21.15.78:0/2459060765 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aa2601b1a0 con 0x55aa260c33b0 2024-08-18T23:46:31.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.920+0000 7fbb37fff640 10 client.25192 handle_command_reply: tid=0 2024-08-18T23:46:31.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.920+0000 7fbb61449740 1 client.25192 shutdown 2024-08-18T23:46:31.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.922+0000 7fbb61449740 1 -- 172.21.15.78:0/2459060765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260a8130 msgr2=0x55aa260b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:31.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.922+0000 7fbb61449740 1 --2- 172.21.15.78:0/2459060765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260a8130 0x55aa260b0190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbb50006010 tx=0x7fbb50003b80 comp rx=0 tx=0).stop 2024-08-18T23:46:31.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.922+0000 7fbb61449740 1 -- 172.21.15.78:0/2459060765 shutdown_connections 2024-08-18T23:46:31.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.922+0000 7fbb61449740 1 --2- 172.21.15.78:0/2459060765 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55aa260c33b0 0x55aa260c5870 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.922+0000 7fbb61449740 1 --2- 172.21.15.78:0/2459060765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260b3310 0x55aa260b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.922+0000 7fbb61449740 1 --2- 172.21.15.78:0/2459060765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260a8a50 0x55aa260b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.922+0000 7fbb61449740 1 --2- 172.21.15.78:0/2459060765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260a8130 0x55aa260b0190 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbb50006010 tx=0x7fbb50003b80 comp rx=0 tx=0).stop 2024-08-18T23:46:31.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.922+0000 7fbb61449740 1 -- 172.21.15.78:0/2459060765 >> 172.21.15.78:0/2459060765 conn(0x55aa260a46c0 msgr2=0x55aa260aa3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:31.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.922+0000 7fbb37fff640 0 client.25192 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:31.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.922+0000 7fbb61449740 1 -- 172.21.15.78:0/2459060765 shutdown_connections 2024-08-18T23:46:31.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.922+0000 7fbb61449740 1 -- 172.21.15.78:0/2459060765 wait complete. 2024-08-18T23:46:31.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.923+0000 7fbb61449740 20 client.25192 trim_cache size 0 max 16384 2024-08-18T23:46:31.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.964+0000 7fbb61449740 1 Processor -- start 2024-08-18T23:46:31.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.964+0000 7fbb61449740 1 -- start start 2024-08-18T23:46:31.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.964+0000 7fbb61449740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260a8a50 0x55aa260cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:31.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.964+0000 7fbb61449740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260b3310 0x55aa260cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:31.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.964+0000 7fbb61449740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260b19f0 0x55aa260b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:31.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.964+0000 7fbb57fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260b3310 0x55aa260cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:31.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.964+0000 7fbb57fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260b3310 0x55aa260cb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:39286/0 (socket says 172.21.15.78:39286) 2024-08-18T23:46:31.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.964+0000 7fbb57fff640 1 -- 172.21.15.78:0/646199426 learned_addr learned my addr 172.21.15.78:0/646199426 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:31.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.964+0000 7fbb61449740 1 -- 172.21.15.78:0/646199426 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb440125b0 con 0x55aa260b19f0 2024-08-18T23:46:31.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.964+0000 7fbb61449740 1 -- 172.21.15.78:0/646199426 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4400fe80 con 0x55aa260b3310 2024-08-18T23:46:31.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.964+0000 7fbb61449740 1 -- 172.21.15.78:0/646199426 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4400dcd0 con 0x55aa260a8a50 2024-08-18T23:46:31.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.964+0000 7fbb5d5e8640 1 --2- 172.21.15.78:0/646199426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260b19f0 0x55aa260b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:31.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.966+0000 7fbb57fff640 1 -- 172.21.15.78:0/646199426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260a8a50 msgr2=0x55aa260cb180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-18T23:46:31.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.966+0000 7fbb57fff640 1 --2- 172.21.15.78:0/646199426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260a8a50 0x55aa260cb180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.966+0000 7fbb57fff640 1 -- 172.21.15.78:0/646199426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260b19f0 msgr2=0x55aa260b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:31.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.966+0000 7fbb57fff640 1 --2- 172.21.15.78:0/646199426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260b19f0 0x55aa260b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.966+0000 7fbb57fff640 1 -- 172.21.15.78:0/646199426 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa260a2c70 con 0x55aa260b3310 2024-08-18T23:46:31.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.970+0000 7fbb57fff640 1 --2- 172.21.15.78:0/646199426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260b3310 0x55aa260cb6c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0cbfe0 tx=0x7fbb4c0cbf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ffaf12947f2a703 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:31.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.972+0000 7fbb3e7fc640 1 -- 172.21.15.78:0/646199426 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c0a7070 con 0x55aa260b3310 2024-08-18T23:46:31.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.972+0000 7fbb3e7fc640 1 -- 172.21.15.78:0/646199426 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb4c098ff0 con 0x55aa260b3310 2024-08-18T23:46:31.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.972+0000 7fbb3e7fc640 1 -- 172.21.15.78:0/646199426 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c0b8030 con 0x55aa260b3310 2024-08-18T23:46:31.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.974+0000 7fbb61449740 1 -- 172.21.15.78:0/646199426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260b3310 msgr2=0x55aa260cb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:31.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.974+0000 7fbb61449740 1 --2- 172.21.15.78:0/646199426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260b3310 0x55aa260cb6c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0cbfe0 tx=0x7fbb4c0cbf00 comp rx=0 tx=0).stop 2024-08-18T23:46:31.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.978+0000 7fbb61449740 1 -- 172.21.15.78:0/646199426 shutdown_connections 2024-08-18T23:46:31.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.978+0000 7fbb61449740 1 --2- 172.21.15.78:0/646199426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260b19f0 0x55aa260b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.978+0000 7fbb61449740 1 --2- 172.21.15.78:0/646199426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260b3310 0x55aa260cb6c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.978+0000 7fbb61449740 1 --2- 172.21.15.78:0/646199426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260a8a50 0x55aa260cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.978+0000 7fbb61449740 1 -- 172.21.15.78:0/646199426 >> 172.21.15.78:0/646199426 conn(0x55aa260a46c0 msgr2=0x55aa260a4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.982+0000 7fbb61449740 1 -- 172.21.15.78:0/646199426 shutdown_connections 2024-08-18T23:46:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.984+0000 7fbb61449740 1 -- 172.21.15.78:0/646199426 wait complete. 2024-08-18T23:46:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.984+0000 7fbb61449740 1 -- start start 2024-08-18T23:46:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.984+0000 7fbb61449740 1 Processor -- start 2024-08-18T23:46:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.985+0000 7fbb61449740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260a8a50 0x55aa260ad280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.985+0000 7fbb61449740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260b19f0 0x55aa260ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.985+0000 7fbb61449740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260b3310 0x55aa260add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.985+0000 7fbb61449740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4400e340 con 0x55aa260b3310 2024-08-18T23:46:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.985+0000 7fbb61449740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4400f7b0 con 0x55aa260a8a50 2024-08-18T23:46:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.985+0000 7fbb61449740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4400ea10 con 0x55aa260b19f0 2024-08-18T23:46:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.985+0000 7fbb61449740 10 client.0 _ll_register_callbacks cb 0x7fbb5001a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:46:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.985+0000 7fbb57fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260b19f0 0x55aa260ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.986+0000 7fbb57fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260b19f0 0x55aa260ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:53722/0 (socket says 172.21.15.78:53722) 2024-08-18T23:46:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.986+0000 7fbb57fff640 1 -- 172.21.15.78:0/2042994835 learned_addr learned my addr 172.21.15.78:0/2042994835 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.986+0000 7fbb5d5e8640 1 --2- 172.21.15.78:0/2042994835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260b3310 0x55aa260add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.987+0000 7fbb5d5e8640 1 -- 172.21.15.78:0/2042994835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260b19f0 msgr2=0x55aa260ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.987+0000 7fbb5d5e8640 1 --2- 172.21.15.78:0/2042994835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260b19f0 0x55aa260ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.987+0000 7fbb5d5e8640 1 -- 172.21.15.78:0/2042994835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260a8a50 msgr2=0x55aa260ad280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T23:46:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.987+0000 7fbb5d5e8640 1 --2- 172.21.15.78:0/2042994835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260a8a50 0x55aa260ad280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.987+0000 7fbb5d5e8640 1 -- 172.21.15.78:0/2042994835 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa2612d200 con 0x55aa260b3310 2024-08-18T23:46:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.988+0000 7fbb5d5e8640 1 --2- 172.21.15.78:0/2042994835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260b3310 0x55aa260add00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbb48014e20 tx=0x7fbb480047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954b901e5299a900 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.990+0000 7fbb37fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:46:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.990+0000 7fbb37fff640 1 -- 172.21.15.78:0/2042994835 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4801f850 con 0x55aa260b3310 2024-08-18T23:46:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.990+0000 7fbb61449740 1 -- 172.21.15.78:0/2042994835 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb4c098ff0 con 0x55aa260b3310 2024-08-18T23:46:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.990+0000 7fbb37fff640 1 -- 172.21.15.78:0/2042994835 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fbb4801f9f0 con 0x55aa260b3310 2024-08-18T23:46:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.992+0000 7fbb37fff640 1 -- 172.21.15.78:0/2042994835 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbb48010e50 con 0x55aa260b3310 2024-08-18T23:46:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.992+0000 7fbb61449740 10 client.15822 fetch_fsmap learned FSMap version 26 2024-08-18T23:46:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.992+0000 7fbb61449740 1 -- 172.21.15.78:0/2042994835 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aa26150530 con 0x55aa260b3310 2024-08-18T23:46:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.993+0000 7fbb37fff640 1 -- 172.21.15.78:0/2042994835 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb4c098ff0 con 0x55aa260b3310 2024-08-18T23:46:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.993+0000 7fbb37fff640 1 -- 172.21.15.78:0/2042994835 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fbb48028070 con 0x55aa260b3310 2024-08-18T23:46:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.995+0000 7fbb61449740 10 client.15822 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:46:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.995+0000 7fbb61449740 10 client.15822 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:46:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.995+0000 7fbb61449740 20 client.15822 populate_metadata read hostname 'smithi078' 2024-08-18T23:46:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.995+0000 7fbb61449740 1 --2- 172.21.15.78:0/2042994835 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55aa26150d00 0x55aa26153120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:31.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.995+0000 7fbb61449740 4 client.15822 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-08-18T23:46:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.995+0000 7fbb61449740 1 -- 172.21.15.78:0/2042994835 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x55aa260154f0 con 0x55aa26150d00 2024-08-18T23:46:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.995+0000 7fbb5cde7640 1 --2- 172.21.15.78:0/2042994835 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55aa26150d00 0x55aa26153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.995+0000 7fbb5cde7640 1 --2- 172.21.15.78:0/2042994835 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55aa26150d00 0x55aa26153120 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abfc05b2adc6bab3 server_cookie=676bd718143f2744 in_seq=0 out_seq=0 2024-08-18T23:46:31.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:31.996+0000 7fbb37fff640 10 client.15822 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:32.320 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:46:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.817+0000 7fe9d215f640 1 -- 172.21.15.78:0/708940425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc102a70 msgr2=0x7fe9cc102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.817+0000 7fe9d215f640 1 --2- 172.21.15.78:0/708940425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc102a70 0x7fe9cc102e50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009900 tx=0x7fe9bc02a480 comp rx=0 tx=0).stop 2024-08-18T23:46:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.817+0000 7fe9d215f640 1 -- 172.21.15.78:0/708940425 shutdown_connections 2024-08-18T23:46:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.817+0000 7fe9d215f640 1 --2- 172.21.15.78:0/708940425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9cc10bea0 0x7fe9cc10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.817+0000 7fe9d215f640 1 --2- 172.21.15.78:0/708940425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9cc103390 0x7fe9cc10b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.817+0000 7fe9d215f640 1 --2- 172.21.15.78:0/708940425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc102a70 0x7fe9cc102e50 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.817+0000 7fe9d215f640 1 -- 172.21.15.78:0/708940425 >> 172.21.15.78:0/708940425 conn(0x7fe9cc0fc4d0 msgr2=0x7fe9cc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.817+0000 7fe9d215f640 1 -- 172.21.15.78:0/708940425 shutdown_connections 2024-08-18T23:46:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.817+0000 7fe9d215f640 1 -- 172.21.15.78:0/708940425 wait complete. 2024-08-18T23:46:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.817+0000 7fe9d215f640 1 Processor -- start 2024-08-18T23:46:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.818+0000 7fe9d215f640 1 -- start start 2024-08-18T23:46:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.818+0000 7fe9d215f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9cc102a70 0x7fe9cc198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.818+0000 7fe9d215f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9cc103390 0x7fe9cc198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.818+0000 7fe9d215f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc10bea0 0x7fe9cc19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.818+0000 7fe9d215f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc111ca0 con 0x7fe9cc10bea0 2024-08-18T23:46:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.818+0000 7fe9d215f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc111b20 con 0x7fe9cc102a70 2024-08-18T23:46:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.818+0000 7fe9d215f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc111e20 con 0x7fe9cc103390 2024-08-18T23:46:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.818+0000 7fe9cbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc10bea0 0x7fe9cc19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.818+0000 7fe9cb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9cc102a70 0x7fe9cc198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.818+0000 7fe9cbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc10bea0 0x7fe9cc19ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35542/0 (socket says 172.21.15.78:35542) 2024-08-18T23:46:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.818+0000 7fe9cbfff640 1 -- 172.21.15.78:0/3207489036 learned_addr learned my addr 172.21.15.78:0/3207489036 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.818+0000 7fe9caffd640 1 --2- 172.21.15.78:0/3207489036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9cc103390 0x7fe9cc198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.818+0000 7fe9cb7fe640 1 -- 172.21.15.78:0/3207489036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9cc103390 msgr2=0x7fe9cc198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.818+0000 7fe9cb7fe640 1 --2- 172.21.15.78:0/3207489036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9cc103390 0x7fe9cc198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.818+0000 7fe9cb7fe640 1 -- 172.21.15.78:0/3207489036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc10bea0 msgr2=0x7fe9cc19ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.818+0000 7fe9cb7fe640 1 --2- 172.21.15.78:0/3207489036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc10bea0 0x7fe9cc19ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.818+0000 7fe9cb7fe640 1 -- 172.21.15.78:0/3207489036 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9cc19d100 con 0x7fe9cc102a70 2024-08-18T23:46:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.819+0000 7fe9cbfff640 1 --2- 172.21.15.78:0/3207489036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc10bea0 0x7fe9cc19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:46:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.819+0000 7fe9cb7fe640 1 --2- 172.21.15.78:0/3207489036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9cc102a70 0x7fe9cc198150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc005940 tx=0x7fe9bc0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.819+0000 7fe9c8ff9640 1 -- 172.21.15.78:0/3207489036 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc03f030 con 0x7fe9cc102a70 2024-08-18T23:46:32.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.819+0000 7fe9d215f640 1 -- 172.21.15.78:0/3207489036 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9cc19d390 con 0x7fe9cc102a70 2024-08-18T23:46:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.819+0000 7fe9c8ff9640 1 -- 172.21.15.78:0/3207489036 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9bc0044c0 con 0x7fe9cc102a70 2024-08-18T23:46:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.819+0000 7fe9c8ff9640 1 -- 172.21.15.78:0/3207489036 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc035070 con 0x7fe9cc102a70 2024-08-18T23:46:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.819+0000 7fe9d215f640 1 -- 172.21.15.78:0/3207489036 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9cc1a4cd0 con 0x7fe9cc102a70 2024-08-18T23:46:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.821+0000 7fe9c8ff9640 1 -- 172.21.15.78:0/3207489036 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe9bc042030 con 0x7fe9cc102a70 2024-08-18T23:46:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.821+0000 7fe9d455d740 1 Processor -- start 2024-08-18T23:46:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.821+0000 7fe9d455d740 1 -- start start 2024-08-18T23:46:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9d455d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3990d0 0x5626dc3994b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9d455d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3999f0 0x5626dc3a82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:32.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9d455d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3a8800 0x5626dc3aabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:32.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9d455d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5626dc2fe980 con 0x5626dc3a8800 2024-08-18T23:46:32.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9d455d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5626dc296a70 con 0x5626dc3999f0 2024-08-18T23:46:32.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9d455d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5626dc2fa940 con 0x5626dc3990d0 2024-08-18T23:46:32.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9caffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3a8800 0x5626dc3aabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:32.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9c8ff9640 1 --2- 172.21.15.78:0/3207489036 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe9b40a2c00 0x7fe9b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:32.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9caffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3a8800 0x5626dc3aabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35556/0 (socket says 172.21.15.78:35556) 2024-08-18T23:46:32.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9cb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3999f0 0x5626dc3a82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:32.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9caffd640 1 -- 172.21.15.78:0/3764619213 learned_addr learned my addr 172.21.15.78:0/3764619213 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:32.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9cbfff640 1 --2- 172.21.15.78:0/3764619213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3990d0 0x5626dc3994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:32.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9cbfff640 1 --2- 172.21.15.78:0/3207489036 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe9b40a2c00 0x7fe9b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:32.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9c8ff9640 1 -- 172.21.15.78:0/3207489036 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fe9bc0ef730 con 0x7fe9cc102a70 2024-08-18T23:46:32.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9caffd640 1 -- 172.21.15.78:0/3764619213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3990d0 msgr2=0x5626dc3994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:32.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9caffd640 1 --2- 172.21.15.78:0/3764619213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3990d0 0x5626dc3994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9caffd640 1 -- 172.21.15.78:0/3764619213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3999f0 msgr2=0x5626dc3a82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:32.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9caffd640 1 --2- 172.21.15.78:0/3764619213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3999f0 0x5626dc3a82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9caffd640 1 -- 172.21.15.78:0/3764619213 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626dc2f96e0 con 0x5626dc3a8800 2024-08-18T23:46:32.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.822+0000 7fe9cbfff640 1 --2- 172.21.15.78:0/3764619213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3990d0 0x5626dc3994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:46:32.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.823+0000 7fe9caffd640 1 --2- 172.21.15.78:0/3764619213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3a8800 0x5626dc3aabf0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009c80 tx=0x7fe9c0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f27065eb98c3f22 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:32.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.823+0000 7fe9cbfff640 1 --2- 172.21.15.78:0/3207489036 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe9b40a2c00 0x7fe9b40a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4009aa0 tx=0x7fe9c4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:32.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.823+0000 7fe9b1ffb640 1 -- 172.21.15.78:0/3764619213 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c001e070 con 0x5626dc3a8800 2024-08-18T23:46:32.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.823+0000 7fe9b1ffb640 1 -- 172.21.15.78:0/3764619213 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c0019b10 con 0x5626dc3a8800 2024-08-18T23:46:32.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.823+0000 7fe9b1ffb640 1 -- 172.21.15.78:0/3764619213 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c000ea60 con 0x5626dc3a8800 2024-08-18T23:46:32.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.823+0000 7fe9d455d740 1 -- 172.21.15.78:0/3764619213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3a8800 msgr2=0x5626dc3aabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:32.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.823+0000 7fe9d455d740 1 --2- 172.21.15.78:0/3764619213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3a8800 0x5626dc3aabf0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009c80 tx=0x7fe9c0011810 comp rx=0 tx=0).stop 2024-08-18T23:46:32.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.824+0000 7fe9d455d740 1 -- 172.21.15.78:0/3764619213 shutdown_connections 2024-08-18T23:46:32.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.824+0000 7fe9d455d740 1 --2- 172.21.15.78:0/3764619213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3a8800 0x5626dc3aabf0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.824+0000 7fe9d455d740 1 --2- 172.21.15.78:0/3764619213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3999f0 0x5626dc3a82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.824+0000 7fe9d455d740 1 --2- 172.21.15.78:0/3764619213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3990d0 0x5626dc3994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.824+0000 7fe9d455d740 1 -- 172.21.15.78:0/3764619213 >> 172.21.15.78:0/3764619213 conn(0x5626dc3957a0 msgr2=0x5626dc395b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:32.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.824+0000 7fe9d455d740 1 -- 172.21.15.78:0/3764619213 shutdown_connections 2024-08-18T23:46:32.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.824+0000 7fe9d455d740 1 -- 172.21.15.78:0/3764619213 wait complete. 2024-08-18T23:46:32.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.824+0000 7fe9d455d740 1 -- start start 2024-08-18T23:46:32.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.824+0000 7fe9d455d740 1 Processor -- start 2024-08-18T23:46:32.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9d455d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 0x5626dc3a5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:32.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9d455d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3999f0 0x5626dc3a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:32.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9d455d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3a8800 0x5626dc3a6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:32.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9d455d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5626dc3ab3d0 con 0x5626dc3990d0 2024-08-18T23:46:32.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9d455d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5626dc2b9aa0 con 0x5626dc3a8800 2024-08-18T23:46:32.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9d455d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5626dc3ab550 con 0x5626dc3999f0 2024-08-18T23:46:32.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9d455d740 10 client.0 _ll_register_callbacks cb 0x5626dc2a8e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:46:32.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9cb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 0x5626dc3a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:32.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9cb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 0x5626dc3a5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35572/0 (socket says 172.21.15.78:35572) 2024-08-18T23:46:32.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9cb7fe640 1 -- 172.21.15.78:0/2820104647 learned_addr learned my addr 172.21.15.78:0/2820104647 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:32.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9cbfff640 1 --2- 172.21.15.78:0/2820104647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3a8800 0x5626dc3a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:32.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9caffd640 1 --2- 172.21.15.78:0/2820104647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3999f0 0x5626dc3a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:32.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9cb7fe640 1 -- 172.21.15.78:0/2820104647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3999f0 msgr2=0x5626dc3a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:32.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9cb7fe640 1 --2- 172.21.15.78:0/2820104647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3999f0 0x5626dc3a5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9cb7fe640 1 -- 172.21.15.78:0/2820104647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3a8800 msgr2=0x5626dc3a6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:32.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9cb7fe640 1 --2- 172.21.15.78:0/2820104647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3a8800 0x5626dc3a6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9cb7fe640 1 -- 172.21.15.78:0/2820104647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626dc29d650 con 0x5626dc3990d0 2024-08-18T23:46:32.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9cbfff640 1 --2- 172.21.15.78:0/2820104647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3a8800 0x5626dc3a6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:46:32.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9cb7fe640 1 --2- 172.21.15.78:0/2820104647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 0x5626dc3a5780 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0b8620 tx=0x7fe9bc0bb790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a282681a34fa1966 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:32.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:46:32.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9aaffd640 1 -- 172.21.15.78:0/2820104647 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc049040 con 0x5626dc3990d0 2024-08-18T23:46:32.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9aaffd640 1 -- 172.21.15.78:0/2820104647 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fe9bc0b9f50 con 0x5626dc3990d0 2024-08-18T23:46:32.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9d455d740 1 -- 172.21.15.78:0/2820104647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9c0019b10 con 0x5626dc3990d0 2024-08-18T23:46:32.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9aaffd640 1 -- 172.21.15.78:0/2820104647 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe9bc0c4810 con 0x5626dc3990d0 2024-08-18T23:46:32.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9d455d740 10 client.15834 fetch_fsmap learned FSMap version 26 2024-08-18T23:46:32.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.825+0000 7fe9d455d740 1 -- 172.21.15.78:0/2820104647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5626dc0703c0 con 0x5626dc3990d0 2024-08-18T23:46:32.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.826+0000 7fe9aaffd640 1 -- 172.21.15.78:0/2820104647 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c0019b10 con 0x5626dc3990d0 2024-08-18T23:46:32.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.826+0000 7fe9aaffd640 1 -- 172.21.15.78:0/2820104647 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fe9bc0b82c0 con 0x5626dc3990d0 2024-08-18T23:46:32.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.826+0000 7fe9d455d740 10 client.15834 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:46:32.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.826+0000 7fe9d455d740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:46:32.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.826+0000 7fe9d455d740 20 client.15834 populate_metadata read hostname 'smithi078' 2024-08-18T23:46:32.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.826+0000 7fe9d455d740 1 --2- 172.21.15.78:0/2820104647 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5626dc3b8300 0x5626dc3ba760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:32.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.826+0000 7fe9d455d740 4 client.15834 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:46:32.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.826+0000 7fe9d455d740 1 -- 172.21.15.78:0/2820104647 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626dc30bef0 con 0x5626dc3b8300 2024-08-18T23:46:32.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.826+0000 7fe9caffd640 1 --2- 172.21.15.78:0/2820104647 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5626dc3b8300 0x5626dc3ba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:32.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.827+0000 7fe9aaffd640 10 client.15834 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:32.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.827+0000 7fe9caffd640 1 --2- 172.21.15.78:0/2820104647 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5626dc3b8300 0x5626dc3ba760 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be0c5bb542833319 server_cookie=c404ba0a94948017 in_seq=0 out_seq=0 2024-08-18T23:46:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.859+0000 7fe9aaffd640 1 -- 172.21.15.78:0/2820104647 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626dc30bef0 con 0x5626dc3b8300 2024-08-18T23:46:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.859+0000 7fe9aaffd640 10 client.15834 handle_command_reply: tid=0 2024-08-18T23:46:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.859+0000 7fe9d455d740 1 client.15834 shutdown 2024-08-18T23:46:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.859+0000 7fe9d455d740 1 -- 172.21.15.78:0/2820104647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 msgr2=0x5626dc3a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.859+0000 7fe9d455d740 1 --2- 172.21.15.78:0/2820104647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 0x5626dc3a5780 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0b8620 tx=0x7fe9bc0bb790 comp rx=0 tx=0).stop 2024-08-18T23:46:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.859+0000 7fe9d455d740 1 -- 172.21.15.78:0/2820104647 shutdown_connections 2024-08-18T23:46:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.859+0000 7fe9d455d740 1 --2- 172.21.15.78:0/2820104647 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5626dc3b8300 0x5626dc3ba760 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.859+0000 7fe9d455d740 1 --2- 172.21.15.78:0/2820104647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3a8800 0x5626dc3a6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.859+0000 7fe9d455d740 1 --2- 172.21.15.78:0/2820104647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3999f0 0x5626dc3a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.859+0000 7fe9d455d740 1 --2- 172.21.15.78:0/2820104647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 0x5626dc3a5780 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.859+0000 7fe9aaffd640 0 client.15834 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:32.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.859+0000 7fe9d455d740 1 -- 172.21.15.78:0/2820104647 >> 172.21.15.78:0/2820104647 conn(0x5626dc3957a0 msgr2=0x5626dc39b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:32.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.860+0000 7fe9d455d740 1 -- 172.21.15.78:0/2820104647 shutdown_connections 2024-08-18T23:46:32.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.860+0000 7fe9d455d740 1 -- 172.21.15.78:0/2820104647 wait complete. 2024-08-18T23:46:32.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.860+0000 7fe9d455d740 20 client.15834 trim_cache size 0 max 16384 2024-08-18T23:46:32.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.876+0000 7fe9d455d740 1 Processor -- start 2024-08-18T23:46:32.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.876+0000 7fe9d455d740 1 -- start start 2024-08-18T23:46:32.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.877+0000 7fe9d455d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 0x5626dc3a7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:32.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.877+0000 7fe9d455d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3999f0 0x5626dc3947f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:32.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.877+0000 7fe9d455d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3a8800 0x5626dc3c0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:32.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.877+0000 7fe9d455d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1af050 con 0x5626dc3990d0 2024-08-18T23:46:32.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.877+0000 7fe9d455d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1ac920 con 0x5626dc3999f0 2024-08-18T23:46:32.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.877+0000 7fe9d455d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1aa770 con 0x5626dc3a8800 2024-08-18T23:46:32.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.877+0000 7fe9cb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 0x5626dc3a7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:32.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.877+0000 7fe9caffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3999f0 0x5626dc3947f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:32.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.877+0000 7fe9cb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 0x5626dc3a7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35574/0 (socket says 172.21.15.78:35574) 2024-08-18T23:46:32.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.877+0000 7fe9cb7fe640 1 -- 172.21.15.78:0/184824503 learned_addr learned my addr 172.21.15.78:0/184824503 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:32.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.877+0000 7fe9cbfff640 1 --2- 172.21.15.78:0/184824503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3a8800 0x5626dc3c0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:32.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.877+0000 7fe9cb7fe640 1 -- 172.21.15.78:0/184824503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3a8800 msgr2=0x5626dc3c0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:32.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.877+0000 7fe9cb7fe640 1 --2- 172.21.15.78:0/184824503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3a8800 0x5626dc3c0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.877+0000 7fe9cb7fe640 1 -- 172.21.15.78:0/184824503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3999f0 msgr2=0x5626dc3947f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:32.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.877+0000 7fe9cb7fe640 1 --2- 172.21.15.78:0/184824503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3999f0 0x5626dc3947f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.877+0000 7fe9cb7fe640 1 -- 172.21.15.78:0/184824503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626dc3a3cd0 con 0x5626dc3990d0 2024-08-18T23:46:32.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.877+0000 7fe9caffd640 1 --2- 172.21.15.78:0/184824503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3999f0 0x5626dc3947f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:46:32.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.877+0000 7fe9cb7fe640 1 --2- 172.21.15.78:0/184824503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 0x5626dc3a7ee0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009900 tx=0x7fe9bc0bb8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f7425b146f7231 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:32.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.877+0000 7fe9b17fa640 1 -- 172.21.15.78:0/184824503 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc049070 con 0x5626dc3990d0 2024-08-18T23:46:32.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.878+0000 7fe9b17fa640 1 -- 172.21.15.78:0/184824503 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9bc09f070 con 0x5626dc3990d0 2024-08-18T23:46:32.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.878+0000 7fe9b17fa640 1 -- 172.21.15.78:0/184824503 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0ef440 con 0x5626dc3990d0 2024-08-18T23:46:32.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.878+0000 7fe9d455d740 1 -- 172.21.15.78:0/184824503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 msgr2=0x5626dc3a7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:32.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.878+0000 7fe9d455d740 1 --2- 172.21.15.78:0/184824503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 0x5626dc3a7ee0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009900 tx=0x7fe9bc0bb8c0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.878+0000 7fe9d455d740 1 -- 172.21.15.78:0/184824503 shutdown_connections 2024-08-18T23:46:32.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.878+0000 7fe9d455d740 1 --2- 172.21.15.78:0/184824503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3a8800 0x5626dc3c0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.878+0000 7fe9d455d740 1 --2- 172.21.15.78:0/184824503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3999f0 0x5626dc3947f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.878+0000 7fe9d455d740 1 --2- 172.21.15.78:0/184824503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 0x5626dc3a7ee0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.878+0000 7fe9d455d740 1 -- 172.21.15.78:0/184824503 >> 172.21.15.78:0/184824503 conn(0x5626dc3957a0 msgr2=0x5626dc3940e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:32.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.878+0000 7fe9d455d740 1 -- 172.21.15.78:0/184824503 shutdown_connections 2024-08-18T23:46:32.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.878+0000 7fe9d455d740 1 -- 172.21.15.78:0/184824503 wait complete. 2024-08-18T23:46:32.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.879+0000 7fe9d455d740 1 -- start start 2024-08-18T23:46:32.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.879+0000 7fe9d455d740 1 Processor -- start 2024-08-18T23:46:32.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.879+0000 7fe9d455d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 0x5626dc3b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:32.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.879+0000 7fe9d455d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3999f0 0x5626dc3b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:32.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.879+0000 7fe9d455d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3a8800 0x5626dc3ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:32.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.879+0000 7fe9d455d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1aade0 con 0x5626dc3990d0 2024-08-18T23:46:32.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.879+0000 7fe9d455d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1ac250 con 0x5626dc3a8800 2024-08-18T23:46:32.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.879+0000 7fe9d455d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1ab4b0 con 0x5626dc3999f0 2024-08-18T23:46:32.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.879+0000 7fe9cb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 0x5626dc3b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:32.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.879+0000 7fe9cb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 0x5626dc3b0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35578/0 (socket says 172.21.15.78:35578) 2024-08-18T23:46:32.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.879+0000 7fe9cb7fe640 1 -- 172.21.15.78:0/3399034254 learned_addr learned my addr 172.21.15.78:0/3399034254 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:32.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.879+0000 7fe9d455d740 10 client.0 _ll_register_callbacks cb 0x7fe9bc0bba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:46:32.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.879+0000 7fe9caffd640 1 --2- 172.21.15.78:0/3399034254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3999f0 0x5626dc3b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:32.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.879+0000 7fe9cbfff640 1 --2- 172.21.15.78:0/3399034254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3a8800 0x5626dc3ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:32.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.880+0000 7fe9cb7fe640 1 -- 172.21.15.78:0/3399034254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3999f0 msgr2=0x5626dc3b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:32.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.880+0000 7fe9cb7fe640 1 --2- 172.21.15.78:0/3399034254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3999f0 0x5626dc3b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.880+0000 7fe9cb7fe640 1 -- 172.21.15.78:0/3399034254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3a8800 msgr2=0x5626dc3ac0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:32.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.880+0000 7fe9cb7fe640 1 --2- 172.21.15.78:0/3399034254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3a8800 0x5626dc3ac0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.880+0000 7fe9cb7fe640 1 -- 172.21.15.78:0/3399034254 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626dc3ac920 con 0x5626dc3990d0 2024-08-18T23:46:32.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.880+0000 7fe9cb7fe640 1 --2- 172.21.15.78:0/3399034254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 0x5626dc3b0b70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc03b580 tx=0x7fe9bc0bb790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a41fd827be6df7 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:32.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.880+0000 7fe9aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:46:32.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.880+0000 7fe9aaffd640 1 -- 172.21.15.78:0/3399034254 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc049040 con 0x5626dc3990d0 2024-08-18T23:46:32.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.880+0000 7fe9aaffd640 1 -- 172.21.15.78:0/3399034254 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fe9bc0ef440 con 0x5626dc3990d0 2024-08-18T23:46:32.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.880+0000 7fe9d455d740 1 -- 172.21.15.78:0/3399034254 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9bc09f070 con 0x5626dc3990d0 2024-08-18T23:46:32.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.880+0000 7fe9aaffd640 1 -- 172.21.15.78:0/3399034254 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe9bc0c45c0 con 0x5626dc3990d0 2024-08-18T23:46:32.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.880+0000 7fe9d455d740 10 client.15840 fetch_fsmap learned FSMap version 26 2024-08-18T23:46:32.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.880+0000 7fe9d455d740 1 -- 172.21.15.78:0/3399034254 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5626dc3ac920 con 0x5626dc3990d0 2024-08-18T23:46:32.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.881+0000 7fe9aaffd640 1 -- 172.21.15.78:0/3399034254 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9bc09f070 con 0x5626dc3990d0 2024-08-18T23:46:32.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.881+0000 7fe9aaffd640 1 -- 172.21.15.78:0/3399034254 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fe9bc03bb30 con 0x5626dc3990d0 2024-08-18T23:46:32.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.881+0000 7fe9d455d740 10 client.15840 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:46:32.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.881+0000 7fe9d455d740 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:46:32.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.881+0000 7fe9d455d740 20 client.15840 populate_metadata read hostname 'smithi078' 2024-08-18T23:46:32.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.881+0000 7fe9d455d740 1 --2- 172.21.15.78:0/3399034254 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5626dc422fe0 0x5626dc425400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:32.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.881+0000 7fe9d455d740 4 client.15840 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:46:32.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.881+0000 7fe9d455d740 1 -- 172.21.15.78:0/3399034254 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5626dc306400 con 0x5626dc422fe0 2024-08-18T23:46:32.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.881+0000 7fe9caffd640 1 --2- 172.21.15.78:0/3399034254 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5626dc422fe0 0x5626dc425400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:32.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.881+0000 7fe9aaffd640 10 client.15840 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:32.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.881+0000 7fe9caffd640 1 --2- 172.21.15.78:0/3399034254 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5626dc422fe0 0x5626dc425400 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a540ef4aff3b4361 server_cookie=ecdb60c8b9f55f3 in_seq=0 out_seq=0 2024-08-18T23:46:32.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.911+0000 7fe9aaffd640 1 -- 172.21.15.78:0/3399034254 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5626dc306400 con 0x5626dc422fe0 2024-08-18T23:46:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.911+0000 7fe9aaffd640 10 client.15840 handle_command_reply: tid=0 2024-08-18T23:46:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.911+0000 7fe9d455d740 1 client.15840 shutdown 2024-08-18T23:46:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.911+0000 7fe9d455d740 1 -- 172.21.15.78:0/3399034254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 msgr2=0x5626dc3b0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.911+0000 7fe9d455d740 1 --2- 172.21.15.78:0/3399034254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 0x5626dc3b0b70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc03b580 tx=0x7fe9bc0bb790 comp rx=0 tx=0).stop 2024-08-18T23:46:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.911+0000 7fe9d455d740 1 -- 172.21.15.78:0/3399034254 shutdown_connections 2024-08-18T23:46:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.911+0000 7fe9d455d740 1 --2- 172.21.15.78:0/3399034254 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5626dc422fe0 0x5626dc425400 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.911+0000 7fe9d455d740 1 --2- 172.21.15.78:0/3399034254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5626dc3a8800 0x5626dc3ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.911+0000 7fe9d455d740 1 --2- 172.21.15.78:0/3399034254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5626dc3999f0 0x5626dc3b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.911+0000 7fe9d455d740 1 --2- 172.21.15.78:0/3399034254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5626dc3990d0 0x5626dc3b0b70 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.911+0000 7fe9d455d740 1 -- 172.21.15.78:0/3399034254 >> 172.21.15.78:0/3399034254 conn(0x5626dc3957a0 msgr2=0x5626dc394900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.911+0000 7fe9aaffd640 0 client.15840 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.911+0000 7fe9d455d740 1 -- 172.21.15.78:0/3399034254 shutdown_connections 2024-08-18T23:46:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.911+0000 7fe9d455d740 1 -- 172.21.15.78:0/3399034254 wait complete. 2024-08-18T23:46:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.911+0000 7fe9d455d740 20 client.15840 trim_cache size 0 max 16384 2024-08-18T23:46:32.914 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:46:32.914 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "scrub active (9300 inodes in the stack)", 2024-08-18T23:46:32.914 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-08-18T23:46:32.915 INFO:teuthology.orchestra.run.smithi078.stdout: "00dbb5e1-ab64-413e-abca-e8a570e1b9fa": { 2024-08-18T23:46:32.915 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-08-18T23:46:32.915 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "00dbb5e1-ab64-413e-abca-e8a570e1b9fa", 2024-08-18T23:46:32.915 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-08-18T23:46:32.915 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:46:32.915 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:46:32.915 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:46:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.912+0000 7fe9d215f640 1 -- 172.21.15.78:0/3207489036 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe9b40a2c00 msgr2=0x7fe9b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.912+0000 7fe9d215f640 1 --2- 172.21.15.78:0/3207489036 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe9b40a2c00 0x7fe9b40a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4009aa0 tx=0x7fe9c4009500 comp rx=0 tx=0).stop 2024-08-18T23:46:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.912+0000 7fe9d215f640 1 -- 172.21.15.78:0/3207489036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9cc102a70 msgr2=0x7fe9cc198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.912+0000 7fe9d215f640 1 --2- 172.21.15.78:0/3207489036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9cc102a70 0x7fe9cc198150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc005940 tx=0x7fe9bc0040c0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.912+0000 7fe9d215f640 1 -- 172.21.15.78:0/3207489036 shutdown_connections 2024-08-18T23:46:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.912+0000 7fe9d215f640 1 --2- 172.21.15.78:0/3207489036 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe9b40a2c00 0x7fe9b40a50c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.912+0000 7fe9d215f640 1 --2- 172.21.15.78:0/3207489036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9cc10bea0 0x7fe9cc19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.912+0000 7fe9d215f640 1 --2- 172.21.15.78:0/3207489036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9cc103390 0x7fe9cc198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.912+0000 7fe9d215f640 1 --2- 172.21.15.78:0/3207489036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9cc102a70 0x7fe9cc198150 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.912+0000 7fe9d215f640 1 -- 172.21.15.78:0/3207489036 >> 172.21.15.78:0/3207489036 conn(0x7fe9cc0fc4d0 msgr2=0x7fe9cc0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.912+0000 7fe9d215f640 1 -- 172.21.15.78:0/3207489036 shutdown_connections 2024-08-18T23:46:32.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:32.912+0000 7fe9d215f640 1 -- 172.21.15.78:0/3207489036 wait complete. 2024-08-18T23:46:33.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:33 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:46:33.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:33 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:46:33.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:33 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:46:33.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:33 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:46:33.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:33 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:46:33.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:33 smithi123 ceph-mon[21808]: pgmap v417: 129 pgs: 129 active+clean; 281 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.8 MiB/s wr, 687 op/s 2024-08-18T23:46:33.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:33 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:46:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:33 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:46:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:33 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:46:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:33 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:46:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:33 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:46:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:33 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:46:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:33 smithi161 ceph-mon[22027]: pgmap v417: 129 pgs: 129 active+clean; 281 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.8 MiB/s wr, 687 op/s 2024-08-18T23:46:33.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:33 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:46:33.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:33 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:46:33.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:33 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:46:33.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:33 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:46:33.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:33 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:46:33.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:33 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:46:33.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:33 smithi078 ceph-mon[18440]: pgmap v417: 129 pgs: 129 active+clean; 281 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.8 MiB/s wr, 687 op/s 2024-08-18T23:46:33.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:33 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:46:33.893 INFO:tasks.cephfs.filesystem:scrub status for tag:00dbb5e1-ab64-413e-abca-e8a570e1b9fa - {'path': '/', 'tag': '00dbb5e1-ab64-413e-abca-e8a570e1b9fa', 'options': 'recursive,force'} 2024-08-18T23:46:34.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:34 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:46:34.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:34 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:46:34.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:34 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:46:34.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:34 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:46:34.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:34 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:46:34.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:34 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:46:35.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:35 smithi123 ceph-mon[21808]: pgmap v418: 129 pgs: 129 active+clean; 284 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 433 op/s 2024-08-18T23:46:35.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:35 smithi161 ceph-mon[22027]: pgmap v418: 129 pgs: 129 active+clean; 284 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 433 op/s 2024-08-18T23:46:35.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:35 smithi078 ceph-mon[18440]: pgmap v418: 129 pgs: 129 active+clean; 284 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 433 op/s 2024-08-18T23:46:36.982 INFO:tasks.workunit.client.0.smithi078.stdout: 60 785884 396 521062 263 382655 804 2024-08-18T23:46:37.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:37 smithi123 ceph-mon[21808]: pgmap v419: 129 pgs: 129 active+clean; 287 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 379 op/s 2024-08-18T23:46:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:37 smithi161 ceph-mon[22027]: pgmap v419: 129 pgs: 129 active+clean; 287 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 379 op/s 2024-08-18T23:46:37.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:37 smithi078 ceph-mon[18440]: pgmap v419: 129 pgs: 129 active+clean; 287 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 379 op/s 2024-08-18T23:46:39.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:39 smithi123 ceph-mon[21808]: pgmap v420: 129 pgs: 129 active+clean; 288 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 233 op/s 2024-08-18T23:46:39.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:39 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:46:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:39 smithi161 ceph-mon[22027]: pgmap v420: 129 pgs: 129 active+clean; 288 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 233 op/s 2024-08-18T23:46:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:39 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:46:39.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:39 smithi078 ceph-mon[18440]: pgmap v420: 129 pgs: 129 active+clean; 288 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 233 op/s 2024-08-18T23:46:39.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:39 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:46:40.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:40 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:46:40.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:40 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:46:40.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:40 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:46:40.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:40 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:46:40.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:40 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:46:40.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:40 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:46:40.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:40 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:46:40.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:40 smithi161 ceph-mon[22027]: pgmap v421: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 350 op/s 2024-08-18T23:46:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:40 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:46:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:40 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:46:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:40 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:46:40.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:40 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:46:40.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:40 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:46:40.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:40 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:46:40.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:40 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:46:40.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:40 smithi078 ceph-mon[18440]: pgmap v421: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 350 op/s 2024-08-18T23:46:40.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:40 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:46:40.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:40 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:46:40.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:40 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:46:40.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:40 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:46:40.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:40 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:46:40.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:40 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:46:40.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:40 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:46:40.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:40 smithi123 ceph-mon[21808]: pgmap v421: 129 pgs: 129 active+clean; 293 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 350 op/s 2024-08-18T23:46:42.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:41 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:46:42.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:41 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:46:42.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:41 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:46:43.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:42 smithi078 ceph-mon[18440]: pgmap v422: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 306 op/s 2024-08-18T23:46:43.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:42 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:46:43.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:42 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:46:43.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:42 smithi123 ceph-mon[21808]: pgmap v422: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 306 op/s 2024-08-18T23:46:43.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:42 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:46:43.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:42 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:46:43.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:42 smithi161 ceph-mon[22027]: pgmap v422: 129 pgs: 129 active+clean; 294 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 306 op/s 2024-08-18T23:46:43.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:42 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:46:43.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:42 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:46:43.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.784+0000 7fbb37fff640 1 -- 172.21.15.78:0/2042994835 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55aa260154f0 con 0x55aa26150d00 2024-08-18T23:46:43.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.784+0000 7fbb37fff640 10 client.15822 handle_command_reply: tid=0 2024-08-18T23:46:43.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.784+0000 7fbb61449740 1 client.15822 shutdown 2024-08-18T23:46:43.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.785+0000 7fbb61449740 1 -- 172.21.15.78:0/2042994835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260b3310 msgr2=0x55aa260add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:43.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.785+0000 7fbb61449740 1 --2- 172.21.15.78:0/2042994835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260b3310 0x55aa260add00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbb48014e20 tx=0x7fbb480047a0 comp rx=0 tx=0).stop 2024-08-18T23:46:43.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.785+0000 7fbb61449740 1 -- 172.21.15.78:0/2042994835 shutdown_connections 2024-08-18T23:46:43.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.785+0000 7fbb61449740 1 --2- 172.21.15.78:0/2042994835 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55aa26150d00 0x55aa26153120 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:43.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.785+0000 7fbb61449740 1 --2- 172.21.15.78:0/2042994835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa260b3310 0x55aa260add00 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:43.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.785+0000 7fbb61449740 1 --2- 172.21.15.78:0/2042994835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa260b19f0 0x55aa260ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:43.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.785+0000 7fbb61449740 1 --2- 172.21.15.78:0/2042994835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa260a8a50 0x55aa260ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:43.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.785+0000 7fbb37fff640 0 client.15822 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:43.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.785+0000 7fbb61449740 1 -- 172.21.15.78:0/2042994835 >> 172.21.15.78:0/2042994835 conn(0x55aa260a46c0 msgr2=0x55aa260b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:43.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.785+0000 7fbb61449740 1 -- 172.21.15.78:0/2042994835 shutdown_connections 2024-08-18T23:46:43.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.785+0000 7fbb61449740 1 -- 172.21.15.78:0/2042994835 wait complete. 2024-08-18T23:46:43.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.785+0000 7fbb61449740 20 client.15822 trim_cache size 0 max 16384 2024-08-18T23:46:43.788 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:46:43.788 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 26, 2024-08-18T23:46:43.788 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24452, 2024-08-18T23:46:43.788 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 6, 2024-08-18T23:46:43.789 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-08-18T23:46:43.789 INFO:teuthology.orchestra.run.smithi078.stdout: "73198cf": { 2024-08-18T23:46:43.789 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 6, 2024-08-18T23:46:43.789 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-08-18T23:46:43.789 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:46:43.789 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-08-18T23:46:43.789 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:46:43.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:46:43.789 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-08-18T23:46:43.789 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 130.0, 2024-08-18T23:46:43.789 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-08-18T23:46:43.790 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-08-18T23:46:43.790 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-08-18T23:46:43.790 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:46:43.790 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-08-18T23:46:43.790 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:46:43.790 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:46:43.790 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:46:43.790 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:46:43.790 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:46:43.790 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:46:43.790 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:46:43.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.786+0000 7fbb5f04b640 1 -- 172.21.15.78:0/15547722 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbb400a2d10 msgr2=0x7fbb400a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:43.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.786+0000 7fbb5f04b640 1 --2- 172.21.15.78:0/15547722 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbb400a2d10 0x7fbb400a51d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55aa260a5a40 tx=0x7fbb480090e0 comp rx=0 tx=0).stop 2024-08-18T23:46:43.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.786+0000 7fbb5f04b640 1 -- 172.21.15.78:0/15547722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb58109560 msgr2=0x7fbb581167b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:43.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.786+0000 7fbb5f04b640 1 --2- 172.21.15.78:0/15547722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb58109560 0x7fbb581167b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009fd0 tx=0x7fbb4c00b870 comp rx=0 tx=0).stop 2024-08-18T23:46:43.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.786+0000 7fbb5f04b640 1 -- 172.21.15.78:0/15547722 shutdown_connections 2024-08-18T23:46:43.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.786+0000 7fbb5f04b640 1 --2- 172.21.15.78:0/15547722 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbb400a2d10 0x7fbb400a51d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:43.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.786+0000 7fbb5f04b640 1 --2- 172.21.15.78:0/15547722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbb5811ae70 0x7fbb58116cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:43.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.786+0000 7fbb5f04b640 1 --2- 172.21.15.78:0/15547722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb58109560 0x7fbb581167b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:43.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.786+0000 7fbb5f04b640 1 --2- 172.21.15.78:0/15547722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb58109180 0x7fbb5811a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.786+0000 7fbb5f04b640 1 -- 172.21.15.78:0/15547722 >> 172.21.15.78:0/15547722 conn(0x7fbb5806de70 msgr2=0x7fbb580709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.786+0000 7fbb5f04b640 1 -- 172.21.15.78:0/15547722 shutdown_connections 2024-08-18T23:46:43.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:43.786+0000 7fbb5f04b640 1 -- 172.21.15.78:0/15547722 wait complete. 2024-08-18T23:46:44.124 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:43 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:46:44.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:43 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:46:44.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:43 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:46:44.689 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '73198cf', quiesce took 21.3 seconds. Will release after: -11.3 2024-08-18T23:46:44.689 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:46:44.950 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:46:45.233 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:44 smithi078 ceph-mon[18440]: pgmap v423: 129 pgs: 129 active+clean; 298 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 289 op/s 2024-08-18T23:46:45.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:44 smithi123 ceph-mon[21808]: pgmap v423: 129 pgs: 129 active+clean; 298 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 289 op/s 2024-08-18T23:46:45.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:44 smithi161 ceph-mon[22027]: pgmap v423: 129 pgs: 129 active+clean; 298 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 289 op/s 2024-08-18T23:46:45.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.452+0000 7f24cd0fd640 1 -- 172.21.15.78:0/1779463995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24c81018c0 msgr2=0x7f24c8101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:45.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.452+0000 7f24cd0fd640 1 --2- 172.21.15.78:0/1779463995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24c81018c0 0x7f24c8101d20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009930 tx=0x7f24b002a440 comp rx=0 tx=0).stop 2024-08-18T23:46:45.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.452+0000 7f24cd0fd640 1 -- 172.21.15.78:0/1779463995 shutdown_connections 2024-08-18T23:46:45.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.452+0000 7f24cd0fd640 1 --2- 172.21.15.78:0/1779463995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24c8102260 0x7f24c810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:45.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.453+0000 7f24cd0fd640 1 --2- 172.21.15.78:0/1779463995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24c81018c0 0x7f24c8101d20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:45.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.453+0000 7f24cd0fd640 1 --2- 172.21.15.78:0/1779463995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c81078c0 0x7f24c8107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:45.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.453+0000 7f24cd0fd640 1 -- 172.21.15.78:0/1779463995 >> 172.21.15.78:0/1779463995 conn(0x7f24c80fd5e0 msgr2=0x7f24c80ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:45.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.453+0000 7f24cd0fd640 1 -- 172.21.15.78:0/1779463995 shutdown_connections 2024-08-18T23:46:45.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.453+0000 7f24cd0fd640 1 -- 172.21.15.78:0/1779463995 wait complete. 2024-08-18T23:46:45.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.453+0000 7f24cd0fd640 1 Processor -- start 2024-08-18T23:46:45.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.453+0000 7f24cd0fd640 1 -- start start 2024-08-18T23:46:45.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24cd0fd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24c81018c0 0x7f24c8198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:45.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24cd0fd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24c8102260 0x7f24c81986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:45.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24cd0fd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c81078c0 0x7f24c819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:45.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24cd0fd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f24c8111c80 con 0x7f24c81018c0 2024-08-18T23:46:45.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24cd0fd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24c8111b00 con 0x7f24c81078c0 2024-08-18T23:46:45.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24cd0fd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f24c8111e00 con 0x7f24c8102260 2024-08-18T23:46:45.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24c6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24c81018c0 0x7f24c8198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:45.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24c6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24c81018c0 0x7f24c8198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41724/0 (socket says 172.21.15.78:41724) 2024-08-18T23:46:45.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24c6d9d640 1 -- 172.21.15.78:0/3646344532 learned_addr learned my addr 172.21.15.78:0/3646344532 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:45.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24c759e640 1 --2- 172.21.15.78:0/3646344532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c81078c0 0x7f24c819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:45.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24c6d9d640 1 -- 172.21.15.78:0/3646344532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24c8102260 msgr2=0x7f24c81986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:45.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24c659c640 1 --2- 172.21.15.78:0/3646344532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24c8102260 0x7f24c81986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:45.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24c6d9d640 1 --2- 172.21.15.78:0/3646344532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24c8102260 0x7f24c81986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:45.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24c6d9d640 1 -- 172.21.15.78:0/3646344532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c81078c0 msgr2=0x7f24c819ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:45.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24c6d9d640 1 --2- 172.21.15.78:0/3646344532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c81078c0 0x7f24c819ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:45.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24c6d9d640 1 -- 172.21.15.78:0/3646344532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c819d130 con 0x7f24c81018c0 2024-08-18T23:46:45.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24c659c640 1 --2- 172.21.15.78:0/3646344532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24c8102260 0x7f24c81986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:46:45.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24c6d9d640 1 --2- 172.21.15.78:0/3646344532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24c81018c0 0x7f24c8198180 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f24c000db80 tx=0x7f24c000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:45.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24bffff640 1 -- 172.21.15.78:0/3646344532 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0017070 con 0x7f24c81018c0 2024-08-18T23:46:45.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24cd0fd640 1 -- 172.21.15.78:0/3646344532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c819d420 con 0x7f24c81018c0 2024-08-18T23:46:45.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24bffff640 1 -- 172.21.15.78:0/3646344532 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24c0003b70 con 0x7f24c81018c0 2024-08-18T23:46:45.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24cd0fd640 1 -- 172.21.15.78:0/3646344532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c81a4d00 con 0x7f24c81018c0 2024-08-18T23:46:45.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.454+0000 7f24bffff640 1 -- 172.21.15.78:0/3646344532 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c00106c0 con 0x7f24c81018c0 2024-08-18T23:46:45.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.455+0000 7f24cd0fd640 1 -- 172.21.15.78:0/3646344532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c8101d20 con 0x7f24c81018c0 2024-08-18T23:46:45.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.457+0000 7f24bffff640 1 -- 172.21.15.78:0/3646344532 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f24c0004020 con 0x7f24c81018c0 2024-08-18T23:46:45.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.457+0000 7f24bffff640 1 --2- 172.21.15.78:0/3646344532 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f24b40a2b80 0x7f24b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:45.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.457+0000 7f24bffff640 1 -- 172.21.15.78:0/3646344532 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f24c00cf480 con 0x7f24c81018c0 2024-08-18T23:46:45.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.457+0000 7f24c659c640 1 --2- 172.21.15.78:0/3646344532 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f24b40a2b80 0x7f24b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:45.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.458+0000 7f24c659c640 1 --2- 172.21.15.78:0/3646344532 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f24b40a2b80 0x7f24b40a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f24b0000c00 tx=0x7f24b0032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:45.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.460+0000 7f24bffff640 1 -- 172.21.15.78:0/3646344532 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f24c0020070 con 0x7f24c81018c0 2024-08-18T23:46:45.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.681+0000 7f24cd0fd640 1 -- 172.21.15.78:0/3646344532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24c8109e50 con 0x7f24c81018c0 2024-08-18T23:46:45.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.681+0000 7f24bffff640 1 -- 172.21.15.78:0/3646344532 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11933 (secure 0 0 0) 0x7f24c009c050 con 0x7f24c81018c0 2024-08-18T23:46:45.684 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:46:45.684 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":26,"btime":"2024-08-18T23:45:33:306257+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:45:32.491054+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:46:45.684 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 26 2024-08-18T23:46:45.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.684+0000 7f24cd0fd640 1 -- 172.21.15.78:0/3646344532 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f24b40a2b80 msgr2=0x7f24b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:45.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.684+0000 7f24cd0fd640 1 --2- 172.21.15.78:0/3646344532 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f24b40a2b80 0x7f24b40a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f24b0000c00 tx=0x7f24b0032c70 comp rx=0 tx=0).stop 2024-08-18T23:46:45.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.684+0000 7f24cd0fd640 1 -- 172.21.15.78:0/3646344532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24c81018c0 msgr2=0x7f24c8198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:45.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.684+0000 7f24cd0fd640 1 --2- 172.21.15.78:0/3646344532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24c81018c0 0x7f24c8198180 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f24c000db80 tx=0x7f24c000b630 comp rx=0 tx=0).stop 2024-08-18T23:46:45.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.684+0000 7f24cd0fd640 1 -- 172.21.15.78:0/3646344532 shutdown_connections 2024-08-18T23:46:45.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.684+0000 7f24cd0fd640 1 --2- 172.21.15.78:0/3646344532 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f24b40a2b80 0x7f24b40a5040 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:45.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.684+0000 7f24cd0fd640 1 --2- 172.21.15.78:0/3646344532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c81078c0 0x7f24c819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:45.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.684+0000 7f24cd0fd640 1 --2- 172.21.15.78:0/3646344532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24c8102260 0x7f24c81986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:45.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.684+0000 7f24cd0fd640 1 --2- 172.21.15.78:0/3646344532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24c81018c0 0x7f24c8198180 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:45.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.684+0000 7f24cd0fd640 1 -- 172.21.15.78:0/3646344532 >> 172.21.15.78:0/3646344532 conn(0x7f24c80fd5e0 msgr2=0x7f24c80fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:45.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.684+0000 7f24cd0fd640 1 -- 172.21.15.78:0/3646344532 shutdown_connections 2024-08-18T23:46:45.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:45.684+0000 7f24cd0fd640 1 -- 172.21.15.78:0/3646344532 wait complete. 2024-08-18T23:46:46.100 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:45 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3646344532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:46:46.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:45 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3646344532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:46:46.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:45 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3646344532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:46:46.516 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 73198cf --release --await' 2024-08-18T23:46:46.517 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 73198cf --release --await 2024-08-18T23:46:46.784 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:46:46.982 INFO:tasks.workunit.client.0.smithi078.stdout: 60 846 0 703 0 1107 0 2024-08-18T23:46:47.046 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:46 smithi078 ceph-mon[18440]: pgmap v424: 129 pgs: 129 active+clean; 298 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-08-18T23:46:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.293+0000 7fc67ea15640 1 -- 172.21.15.78:0/474148100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc678102260 msgr2=0x7fc67810e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.293+0000 7fc67ea15640 1 --2- 172.21.15.78:0/474148100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc678102260 0x7fc67810e790 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc670009cd0 tx=0x7fc67002a8e0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.293+0000 7fc67ea15640 1 -- 172.21.15.78:0/474148100 shutdown_connections 2024-08-18T23:46:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.293+0000 7fc67ea15640 1 --2- 172.21.15.78:0/474148100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc678102260 0x7fc67810e790 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.293+0000 7fc67ea15640 1 --2- 172.21.15.78:0/474148100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6781018c0 0x7fc678101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.293+0000 7fc67ea15640 1 --2- 172.21.15.78:0/474148100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6781078c0 0x7fc678107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.293+0000 7fc67ea15640 1 -- 172.21.15.78:0/474148100 >> 172.21.15.78:0/474148100 conn(0x7fc6780fd5e0 msgr2=0x7fc6780ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:47.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.293+0000 7fc67ea15640 1 -- 172.21.15.78:0/474148100 shutdown_connections 2024-08-18T23:46:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.293+0000 7fc67ea15640 1 -- 172.21.15.78:0/474148100 wait complete. 2024-08-18T23:46:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.294+0000 7fc67ea15640 1 Processor -- start 2024-08-18T23:46:47.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.294+0000 7fc67ea15640 1 -- start start 2024-08-18T23:46:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.294+0000 7fc67ea15640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6781018c0 0x7fc678074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.294+0000 7fc67ea15640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc678102260 0x7fc678074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.294+0000 7fc67ea15640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6781078c0 0x7fc678074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.294+0000 7fc67ea15640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc678111ce0 con 0x7fc6781018c0 2024-08-18T23:46:47.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.294+0000 7fc67ea15640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc678111b60 con 0x7fc678102260 2024-08-18T23:46:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.294+0000 7fc67ea15640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc678111e60 con 0x7fc6781078c0 2024-08-18T23:46:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.294+0000 7fc677fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6781018c0 0x7fc678074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.294+0000 7fc6777fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc678102260 0x7fc678074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.294+0000 7fc677fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6781018c0 0x7fc678074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58106/0 (socket says 172.21.15.78:58106) 2024-08-18T23:46:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.294+0000 7fc677fff640 1 -- 172.21.15.78:0/2866957044 learned_addr learned my addr 172.21.15.78:0/2866957044 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.294+0000 7fc67cfb2640 1 --2- 172.21.15.78:0/2866957044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6781078c0 0x7fc678074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.295+0000 7fc6777fe640 1 -- 172.21.15.78:0/2866957044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6781078c0 msgr2=0x7fc678074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.295+0000 7fc6777fe640 1 --2- 172.21.15.78:0/2866957044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6781078c0 0x7fc678074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.295+0000 7fc6777fe640 1 -- 172.21.15.78:0/2866957044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6781018c0 msgr2=0x7fc678074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.295+0000 7fc6777fe640 1 --2- 172.21.15.78:0/2866957044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6781018c0 0x7fc678074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.295+0000 7fc6777fe640 1 -- 172.21.15.78:0/2866957044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6781a6680 con 0x7fc678102260 2024-08-18T23:46:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.295+0000 7fc6777fe640 1 --2- 172.21.15.78:0/2866957044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc678102260 0x7fc678074760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00b3f0 tx=0x7fc66c00b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:47.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.295+0000 7fc6757fa640 1 -- 172.21.15.78:0/2866957044 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c015020 con 0x7fc678102260 2024-08-18T23:46:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.295+0000 7fc67ea15640 1 -- 172.21.15.78:0/2866957044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6781a6970 con 0x7fc678102260 2024-08-18T23:46:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.295+0000 7fc6757fa640 1 -- 172.21.15.78:0/2866957044 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc66c011d10 con 0x7fc678102260 2024-08-18T23:46:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.295+0000 7fc6757fa640 1 -- 172.21.15.78:0/2866957044 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c00f2e0 con 0x7fc678102260 2024-08-18T23:46:47.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.295+0000 7fc67ea15640 1 -- 172.21.15.78:0/2866957044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6781a6f00 con 0x7fc678102260 2024-08-18T23:46:47.300 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:46 smithi123 ceph-mon[21808]: pgmap v424: 129 pgs: 129 active+clean; 298 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-08-18T23:46:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.298+0000 7fc6757fa640 1 -- 172.21.15.78:0/2866957044 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc66c018020 con 0x7fc678102260 2024-08-18T23:46:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.298+0000 7fc680e13740 1 Processor -- start 2024-08-18T23:46:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.298+0000 7fc680e13740 1 -- start start 2024-08-18T23:46:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.298+0000 7fc680e13740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5def990 0x55c3a5defd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.298+0000 7fc680e13740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5df02b0 0x55c3a5dfa630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.298+0000 7fc680e13740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5dfab70 0x55c3a5dfcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.299+0000 7fc67cfb2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5def990 0x55c3a5defd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.299+0000 7fc67cfb2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5def990 0x55c3a5defd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58114/0 (socket says 172.21.15.78:58114) 2024-08-18T23:46:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.299+0000 7fc67cfb2640 1 -- 172.21.15.78:0/3507466892 learned_addr learned my addr 172.21.15.78:0/3507466892 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.299+0000 7fc680e13740 1 -- 172.21.15.78:0/3507466892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c3a5d55c60 con 0x55c3a5def990 2024-08-18T23:46:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.299+0000 7fc680e13740 1 -- 172.21.15.78:0/3507466892 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c3a5cedb90 con 0x55c3a5dfab70 2024-08-18T23:46:47.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.299+0000 7fc680e13740 1 -- 172.21.15.78:0/3507466892 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c3a5d51c20 con 0x55c3a5df02b0 2024-08-18T23:46:47.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.299+0000 7fc677fff640 1 --2- 172.21.15.78:0/3507466892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5df02b0 0x55c3a5dfa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:47.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.299+0000 7fc6777fe640 1 --2- 172.21.15.78:0/3507466892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5dfab70 0x55c3a5dfcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:47.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.299+0000 7fc6757fa640 1 --2- 172.21.15.78:0/2866957044 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc6600a2b80 0x7fc6600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:47.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.299+0000 7fc67cfb2640 1 --2- 172.21.15.78:0/2866957044 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc6600a2b80 0x7fc6600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:47.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.299+0000 7fc677fff640 1 -- 172.21.15.78:0/3507466892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5dfab70 msgr2=0x55c3a5dfcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:47.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.299+0000 7fc677fff640 1 --2- 172.21.15.78:0/3507466892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5dfab70 0x55c3a5dfcf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.299+0000 7fc677fff640 1 -- 172.21.15.78:0/3507466892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5def990 msgr2=0x55c3a5defd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:47.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.299+0000 7fc677fff640 1 --2- 172.21.15.78:0/3507466892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5def990 0x55c3a5defd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.299+0000 7fc677fff640 1 -- 172.21.15.78:0/3507466892 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3a5a65580 con 0x55c3a5df02b0 2024-08-18T23:46:47.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.299+0000 7fc6757fa640 1 -- 172.21.15.78:0/2866957044 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc66c0cebe0 con 0x7fc678102260 2024-08-18T23:46:47.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.299+0000 7fc67cfb2640 1 --2- 172.21.15.78:0/3507466892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5def990 0x55c3a5defd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:46:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.299+0000 7fc677fff640 1 --2- 172.21.15.78:0/3507466892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5df02b0 0x55c3a5dfa630 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc668009c80 tx=0x7fc668011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9b71e1f435c53d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.300+0000 7fc67cfb2640 1 --2- 172.21.15.78:0/2866957044 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc6600a2b80 0x7fc6600a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc670005fe0 tx=0x7fc67002a8e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.300+0000 7fc65e7fc640 1 -- 172.21.15.78:0/3507466892 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc668004230 con 0x55c3a5df02b0 2024-08-18T23:46:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.300+0000 7fc65e7fc640 1 -- 172.21.15.78:0/3507466892 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6680043d0 con 0x55c3a5df02b0 2024-08-18T23:46:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.300+0000 7fc65e7fc640 1 -- 172.21.15.78:0/3507466892 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc668004d90 con 0x55c3a5df02b0 2024-08-18T23:46:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.300+0000 7fc680e13740 1 -- 172.21.15.78:0/3507466892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5df02b0 msgr2=0x55c3a5dfa630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.300+0000 7fc680e13740 1 --2- 172.21.15.78:0/3507466892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5df02b0 0x55c3a5dfa630 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc668009c80 tx=0x7fc668011810 comp rx=0 tx=0).stop 2024-08-18T23:46:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.300+0000 7fc680e13740 1 -- 172.21.15.78:0/3507466892 shutdown_connections 2024-08-18T23:46:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.300+0000 7fc680e13740 1 --2- 172.21.15.78:0/3507466892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5dfab70 0x55c3a5dfcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.300+0000 7fc680e13740 1 --2- 172.21.15.78:0/3507466892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5df02b0 0x55c3a5dfa630 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.300+0000 7fc680e13740 1 --2- 172.21.15.78:0/3507466892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5def990 0x55c3a5defd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.300+0000 7fc680e13740 1 -- 172.21.15.78:0/3507466892 >> 172.21.15.78:0/3507466892 conn(0x55c3a5debf20 msgr2=0x55c3a5dec300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.301+0000 7fc680e13740 1 -- 172.21.15.78:0/3507466892 shutdown_connections 2024-08-18T23:46:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.301+0000 7fc680e13740 1 -- 172.21.15.78:0/3507466892 wait complete. 2024-08-18T23:46:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.301+0000 7fc680e13740 1 -- start start 2024-08-18T23:46:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.301+0000 7fc680e13740 1 Processor -- start 2024-08-18T23:46:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.301+0000 7fc680e13740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5def990 0x55c3a5df7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.301+0000 7fc680e13740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5df02b0 0x55c3a5df7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.301+0000 7fc680e13740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5dfab70 0x55c3a5df83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.301+0000 7fc680e13740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c3a5dfd740 con 0x55c3a5dfab70 2024-08-18T23:46:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.301+0000 7fc680e13740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c3a5cf37c0 con 0x55c3a5df02b0 2024-08-18T23:46:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.301+0000 7fc680e13740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c3a5dfd8c0 con 0x55c3a5def990 2024-08-18T23:46:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.301+0000 7fc680e13740 10 client.0 _ll_register_callbacks cb 0x55c3a5cfcf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:46:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.301+0000 7fc67cfb2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5dfab70 0x55c3a5df83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.302+0000 7fc67cfb2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5dfab70 0x55c3a5df83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58116/0 (socket says 172.21.15.78:58116) 2024-08-18T23:46:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.302+0000 7fc67cfb2640 1 -- 172.21.15.78:0/4189328176 learned_addr learned my addr 172.21.15.78:0/4189328176 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:47.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.302+0000 7fc677fff640 1 --2- 172.21.15.78:0/4189328176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5def990 0x55c3a5df7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:47.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.302+0000 7fc6777fe640 1 --2- 172.21.15.78:0/4189328176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5df02b0 0x55c3a5df7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:47.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.302+0000 7fc67cfb2640 1 -- 172.21.15.78:0/4189328176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5def990 msgr2=0x55c3a5df7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:47.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.302+0000 7fc67cfb2640 1 --2- 172.21.15.78:0/4189328176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5def990 0x55c3a5df7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.302+0000 7fc67cfb2640 1 -- 172.21.15.78:0/4189328176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5df02b0 msgr2=0x55c3a5df7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:47.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.302+0000 7fc67cfb2640 1 --2- 172.21.15.78:0/4189328176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5df02b0 0x55c3a5df7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.302+0000 7fc67cfb2640 1 -- 172.21.15.78:0/4189328176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3a5b2f960 con 0x55c3a5dfab70 2024-08-18T23:46:47.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.302+0000 7fc6777fe640 1 --2- 172.21.15.78:0/4189328176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5df02b0 0x55c3a5df7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:46:47.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.302+0000 7fc677fff640 1 --2- 172.21.15.78:0/4189328176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5def990 0x55c3a5df7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:46:47.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.302+0000 7fc67cfb2640 1 --2- 172.21.15.78:0/4189328176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5dfab70 0x55c3a5df83c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc670006010 tx=0x7fc6700095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829bf37e8eb06f95 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:47.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.302+0000 7fc6577fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:46:47.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.302+0000 7fc6577fe640 1 -- 172.21.15.78:0/4189328176 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67004e030 con 0x55c3a5dfab70 2024-08-18T23:46:47.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.302+0000 7fc680e13740 1 -- 172.21.15.78:0/4189328176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6680043d0 con 0x55c3a5dfab70 2024-08-18T23:46:47.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.302+0000 7fc6577fe640 1 -- 172.21.15.78:0/4189328176 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc670003e00 con 0x55c3a5dfab70 2024-08-18T23:46:47.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.302+0000 7fc6577fe640 1 -- 172.21.15.78:0/4189328176 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc67005a050 con 0x55c3a5dfab70 2024-08-18T23:46:47.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.302+0000 7fc680e13740 10 client.15858 fetch_fsmap learned FSMap version 26 2024-08-18T23:46:47.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.302+0000 7fc680e13740 1 -- 172.21.15.78:0/4189328176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c3a5e09710 con 0x55c3a5dfab70 2024-08-18T23:46:47.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.303+0000 7fc6577fe640 1 -- 172.21.15.78:0/4189328176 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6680043d0 con 0x55c3a5dfab70 2024-08-18T23:46:47.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.303+0000 7fc6577fe640 1 -- 172.21.15.78:0/4189328176 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fc670030070 con 0x55c3a5dfab70 2024-08-18T23:46:47.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.303+0000 7fc680e13740 10 client.15858 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:46:47.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.303+0000 7fc680e13740 10 client.15858 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:46:47.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.303+0000 7fc680e13740 20 client.15858 populate_metadata read hostname 'smithi078' 2024-08-18T23:46:47.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.303+0000 7fc680e13740 1 --2- 172.21.15.78:0/4189328176 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55c3a5e0ab20 0x55c3a5e0cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:47.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.303+0000 7fc680e13740 4 client.15858 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:46:47.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.303+0000 7fc680e13740 1 -- 172.21.15.78:0/4189328176 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3a5d631d0 con 0x55c3a5e0ab20 2024-08-18T23:46:47.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.303+0000 7fc677fff640 1 --2- 172.21.15.78:0/4189328176 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55c3a5e0ab20 0x55c3a5e0cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:47.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.303+0000 7fc6577fe640 10 client.15858 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:47.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.303+0000 7fc677fff640 1 --2- 172.21.15.78:0/4189328176 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55c3a5e0ab20 0x55c3a5e0cfc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cc2342950d501b5 server_cookie=6105ec9d93cca9cb in_seq=0 out_seq=0 2024-08-18T23:46:47.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.305+0000 7fc6577fe640 1 -- 172.21.15.78:0/4189328176 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c3a5d631d0 con 0x55c3a5e0ab20 2024-08-18T23:46:47.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.305+0000 7fc6577fe640 10 client.15858 handle_command_reply: tid=0 2024-08-18T23:46:47.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.305+0000 7fc680e13740 1 client.15858 shutdown 2024-08-18T23:46:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.305+0000 7fc680e13740 1 -- 172.21.15.78:0/4189328176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5dfab70 msgr2=0x55c3a5df83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.305+0000 7fc680e13740 1 --2- 172.21.15.78:0/4189328176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5dfab70 0x55c3a5df83c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc670006010 tx=0x7fc6700095c0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.306+0000 7fc680e13740 1 -- 172.21.15.78:0/4189328176 shutdown_connections 2024-08-18T23:46:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.306+0000 7fc680e13740 1 --2- 172.21.15.78:0/4189328176 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55c3a5e0ab20 0x55c3a5e0cfc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.306+0000 7fc680e13740 1 --2- 172.21.15.78:0/4189328176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5dfab70 0x55c3a5df83c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.306+0000 7fc680e13740 1 --2- 172.21.15.78:0/4189328176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5df02b0 0x55c3a5df7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.306+0000 7fc680e13740 1 --2- 172.21.15.78:0/4189328176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5def990 0x55c3a5df7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.306+0000 7fc680e13740 1 -- 172.21.15.78:0/4189328176 >> 172.21.15.78:0/4189328176 conn(0x55c3a5debf20 msgr2=0x55c3a5deb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.306+0000 7fc6577fe640 0 client.15858 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:47.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.306+0000 7fc680e13740 1 -- 172.21.15.78:0/4189328176 shutdown_connections 2024-08-18T23:46:47.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.306+0000 7fc680e13740 1 -- 172.21.15.78:0/4189328176 wait complete. 2024-08-18T23:46:47.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.306+0000 7fc680e13740 20 client.15858 trim_cache size 0 max 16384 2024-08-18T23:46:47.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.324+0000 7fc680e13740 1 Processor -- start 2024-08-18T23:46:47.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.324+0000 7fc680e13740 1 -- start start 2024-08-18T23:46:47.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.324+0000 7fc680e13740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5debf20 0x55c3a5e12890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:47.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.324+0000 7fc680e13740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5def990 0x55c3a5e12dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:47.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.324+0000 7fc680e13740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5df02b0 0x55c3a5df4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:47.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.324+0000 7fc680e13740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc6781b34f0 con 0x55c3a5df02b0 2024-08-18T23:46:47.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.324+0000 7fc680e13740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6781b0dc0 con 0x55c3a5debf20 2024-08-18T23:46:47.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.324+0000 7fc680e13740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc6781aec10 con 0x55c3a5def990 2024-08-18T23:46:47.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.324+0000 7fc67cfb2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5df02b0 0x55c3a5df4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:47.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.324+0000 7fc677fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5debf20 0x55c3a5e12890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:47.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.324+0000 7fc677fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5debf20 0x55c3a5e12890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:35214/0 (socket says 172.21.15.78:35214) 2024-08-18T23:46:47.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.324+0000 7fc677fff640 1 -- 172.21.15.78:0/1038822309 learned_addr learned my addr 172.21.15.78:0/1038822309 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:47.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.324+0000 7fc6777fe640 1 --2- 172.21.15.78:0/1038822309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5def990 0x55c3a5e12dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:47.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.324+0000 7fc67cfb2640 1 -- 172.21.15.78:0/1038822309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5def990 msgr2=0x55c3a5e12dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:47.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.324+0000 7fc67cfb2640 1 --2- 172.21.15.78:0/1038822309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5def990 0x55c3a5e12dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.324+0000 7fc67cfb2640 1 -- 172.21.15.78:0/1038822309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5debf20 msgr2=0x55c3a5e12890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:47.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.324+0000 7fc67cfb2640 1 --2- 172.21.15.78:0/1038822309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5debf20 0x55c3a5e12890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.324+0000 7fc67cfb2640 1 -- 172.21.15.78:0/1038822309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3a5debd50 con 0x55c3a5df02b0 2024-08-18T23:46:47.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.325+0000 7fc677fff640 1 --2- 172.21.15.78:0/1038822309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5debf20 0x55c3a5e12890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:46:47.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.325+0000 7fc67cfb2640 1 --2- 172.21.15.78:0/1038822309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5df02b0 0x55c3a5df4ba0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc670005e30 tx=0x7fc670031e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3fa9caca4a62ab server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:47.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.325+0000 7fc65dffb640 1 -- 172.21.15.78:0/1038822309 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67004e030 con 0x55c3a5df02b0 2024-08-18T23:46:47.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.325+0000 7fc65dffb640 1 -- 172.21.15.78:0/1038822309 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc670036050 con 0x55c3a5df02b0 2024-08-18T23:46:47.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.325+0000 7fc65dffb640 1 -- 172.21.15.78:0/1038822309 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc670030a80 con 0x55c3a5df02b0 2024-08-18T23:46:47.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.326+0000 7fc680e13740 1 -- 172.21.15.78:0/1038822309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5df02b0 msgr2=0x55c3a5df4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:47.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.326+0000 7fc680e13740 1 --2- 172.21.15.78:0/1038822309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5df02b0 0x55c3a5df4ba0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc670005e30 tx=0x7fc670031e70 comp rx=0 tx=0).stop 2024-08-18T23:46:47.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.326+0000 7fc680e13740 1 -- 172.21.15.78:0/1038822309 shutdown_connections 2024-08-18T23:46:47.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.326+0000 7fc680e13740 1 --2- 172.21.15.78:0/1038822309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5df02b0 0x55c3a5df4ba0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.326+0000 7fc680e13740 1 --2- 172.21.15.78:0/1038822309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5def990 0x55c3a5e12dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.326+0000 7fc680e13740 1 --2- 172.21.15.78:0/1038822309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5debf20 0x55c3a5e12890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.326+0000 7fc680e13740 1 -- 172.21.15.78:0/1038822309 >> 172.21.15.78:0/1038822309 conn(0x55c3a5df3330 msgr2=0x55c3a5df99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:47.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.326+0000 7fc680e13740 1 -- 172.21.15.78:0/1038822309 shutdown_connections 2024-08-18T23:46:47.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.326+0000 7fc680e13740 1 -- 172.21.15.78:0/1038822309 wait complete. 2024-08-18T23:46:47.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.326+0000 7fc680e13740 1 -- start start 2024-08-18T23:46:47.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.326+0000 7fc680e13740 1 Processor -- start 2024-08-18T23:46:47.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.327+0000 7fc680e13740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5debf20 0x55c3a5df4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:47.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.327+0000 7fc680e13740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5def990 0x55c3a5df4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:47.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.327+0000 7fc680e13740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5df02b0 0x55c3a5df52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:47.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.327+0000 7fc680e13740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc6781af280 con 0x55c3a5debf20 2024-08-18T23:46:47.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.327+0000 7fc680e13740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6781b06f0 con 0x55c3a5df02b0 2024-08-18T23:46:47.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.327+0000 7fc680e13740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc6781af950 con 0x55c3a5def990 2024-08-18T23:46:47.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.327+0000 7fc680e13740 10 client.0 _ll_register_callbacks cb 0x7fc670031700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:46:47.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.327+0000 7fc677fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5debf20 0x55c3a5df4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:47.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.327+0000 7fc677fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5debf20 0x55c3a5df4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58124/0 (socket says 172.21.15.78:58124) 2024-08-18T23:46:47.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.327+0000 7fc677fff640 1 -- 172.21.15.78:0/1968939412 learned_addr learned my addr 172.21.15.78:0/1968939412 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:46:47.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.327+0000 7fc67cfb2640 1 --2- 172.21.15.78:0/1968939412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5df02b0 0x55c3a5df52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:47.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.327+0000 7fc6777fe640 1 --2- 172.21.15.78:0/1968939412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5def990 0x55c3a5df4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:47.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.327+0000 7fc677fff640 1 -- 172.21.15.78:0/1968939412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5def990 msgr2=0x55c3a5df4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:47.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.327+0000 7fc677fff640 1 --2- 172.21.15.78:0/1968939412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5def990 0x55c3a5df4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.327+0000 7fc677fff640 1 -- 172.21.15.78:0/1968939412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5df02b0 msgr2=0x55c3a5df52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:46:47.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.327+0000 7fc677fff640 1 --2- 172.21.15.78:0/1968939412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5df02b0 0x55c3a5df52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.327+0000 7fc677fff640 1 -- 172.21.15.78:0/1968939412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3a5e00f90 con 0x55c3a5debf20 2024-08-18T23:46:47.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.327+0000 7fc677fff640 1 --2- 172.21.15.78:0/1968939412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5debf20 0x55c3a5df4870 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc668009c80 tx=0x7fc668009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82678e18afa9fb2 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:46:47.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.327+0000 7fc6577fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:46:47.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.328+0000 7fc6577fe640 1 -- 172.21.15.78:0/1968939412 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66801e040 con 0x55c3a5debf20 2024-08-18T23:46:47.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.328+0000 7fc6577fe640 1 -- 172.21.15.78:0/1968939412 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc668019910 con 0x55c3a5debf20 2024-08-18T23:46:47.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.328+0000 7fc680e13740 1 -- 172.21.15.78:0/1968939412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc670036050 con 0x55c3a5debf20 2024-08-18T23:46:47.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.328+0000 7fc6577fe640 1 -- 172.21.15.78:0/1968939412 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc66803a050 con 0x55c3a5debf20 2024-08-18T23:46:47.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.328+0000 7fc680e13740 10 client.15864 fetch_fsmap learned FSMap version 26 2024-08-18T23:46:47.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.328+0000 7fc680e13740 1 -- 172.21.15.78:0/1968939412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c3a5e75ad0 con 0x55c3a5debf20 2024-08-18T23:46:47.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.328+0000 7fc6577fe640 1 -- 172.21.15.78:0/1968939412 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc670036050 con 0x55c3a5debf20 2024-08-18T23:46:47.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.328+0000 7fc6577fe640 1 -- 172.21.15.78:0/1968939412 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fc668024040 con 0x55c3a5debf20 2024-08-18T23:46:47.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.328+0000 7fc680e13740 10 client.15864 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:46:47.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.328+0000 7fc680e13740 10 client.15864 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:46:47.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.328+0000 7fc680e13740 20 client.15864 populate_metadata read hostname 'smithi078' 2024-08-18T23:46:47.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.328+0000 7fc680e13740 1 --2- 172.21.15.78:0/1968939412 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55c3a5dfab70 0x55c3a5e780f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:46:47.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.328+0000 7fc680e13740 4 client.15864 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "73198cf", "release": true, "await": true}] 2024-08-18T23:46:47.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.328+0000 7fc680e13740 1 -- 172.21.15.78:0/1968939412 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "quiesce db", "set_id": "73198cf", "release": true, "await": true}) -- 0x55c3a5d5d6e0 con 0x55c3a5dfab70 2024-08-18T23:46:47.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.329+0000 7fc6777fe640 1 --2- 172.21.15.78:0/1968939412 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55c3a5dfab70 0x55c3a5e780f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:46:47.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.329+0000 7fc6577fe640 10 client.15864 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:47.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.329+0000 7fc6777fe640 1 --2- 172.21.15.78:0/1968939412 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55c3a5dfab70 0x55c3a5e780f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3beb967ae01d3e9 server_cookie=28b0be94349d7fd1 in_seq=0 out_seq=0 2024-08-18T23:46:47.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:46 smithi161 ceph-mon[22027]: pgmap v424: 129 pgs: 129 active+clean; 298 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-08-18T23:46:47.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.855+0000 7fc6577fe640 1 -- 172.21.15.78:0/1968939412 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55c3a5d5d6e0 con 0x55c3a5dfab70 2024-08-18T23:46:47.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.855+0000 7fc6577fe640 10 client.15864 handle_command_reply: tid=0 2024-08-18T23:46:47.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.855+0000 7fc680e13740 1 client.15864 shutdown 2024-08-18T23:46:47.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.855+0000 7fc680e13740 1 -- 172.21.15.78:0/1968939412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5debf20 msgr2=0x55c3a5df4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:47.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.855+0000 7fc680e13740 1 --2- 172.21.15.78:0/1968939412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5debf20 0x55c3a5df4870 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc668009c80 tx=0x7fc668009ef0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.856+0000 7fc680e13740 1 -- 172.21.15.78:0/1968939412 shutdown_connections 2024-08-18T23:46:47.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.856+0000 7fc680e13740 1 --2- 172.21.15.78:0/1968939412 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55c3a5dfab70 0x55c3a5e780f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.856+0000 7fc680e13740 1 --2- 172.21.15.78:0/1968939412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3a5df02b0 0x55c3a5df52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.856+0000 7fc6577fe640 0 client.15864 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:46:47.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.856+0000 7fc680e13740 1 --2- 172.21.15.78:0/1968939412 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3a5def990 0x55c3a5df4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.856+0000 7fc680e13740 1 --2- 172.21.15.78:0/1968939412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c3a5debf20 0x55c3a5df4870 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.856+0000 7fc680e13740 1 -- 172.21.15.78:0/1968939412 >> 172.21.15.78:0/1968939412 conn(0x55c3a5df3330 msgr2=0x55c3a5df7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:47.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.856+0000 7fc680e13740 1 -- 172.21.15.78:0/1968939412 shutdown_connections 2024-08-18T23:46:47.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.856+0000 7fc680e13740 1 -- 172.21.15.78:0/1968939412 wait complete. 2024-08-18T23:46:47.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.856+0000 7fc680e13740 20 client.15864 trim_cache size 0 max 16384 2024-08-18T23:46:47.859 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:46:47.859 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 26, 2024-08-18T23:46:47.859 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24452, 2024-08-18T23:46:47.859 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 8, 2024-08-18T23:46:47.859 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-08-18T23:46:47.859 INFO:teuthology.orchestra.run.smithi078.stdout: "73198cf": { 2024-08-18T23:46:47.859 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 8, 2024-08-18T23:46:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-08-18T23:46:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:46:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-08-18T23:46:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:46:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:46:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-08-18T23:46:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 130.0, 2024-08-18T23:46:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-08-18T23:46:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-08-18T23:46:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-08-18T23:46:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:46:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-08-18T23:46:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:46:47.861 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:46:47.861 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:46:47.861 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:46:47.861 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:46:47.861 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:46:47.861 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:46:47.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.856+0000 7fc67ea15640 1 -- 172.21.15.78:0/2866957044 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc6600a2b80 msgr2=0x7fc6600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:47.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.856+0000 7fc67ea15640 1 --2- 172.21.15.78:0/2866957044 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc6600a2b80 0x7fc6600a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc670005fe0 tx=0x7fc67002a8e0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.856+0000 7fc67ea15640 1 -- 172.21.15.78:0/2866957044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc678102260 msgr2=0x7fc678074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:46:47.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.856+0000 7fc67ea15640 1 --2- 172.21.15.78:0/2866957044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc678102260 0x7fc678074760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00b3f0 tx=0x7fc66c00b8b0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.857+0000 7fc67ea15640 1 -- 172.21.15.78:0/2866957044 shutdown_connections 2024-08-18T23:46:47.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.857+0000 7fc67ea15640 1 --2- 172.21.15.78:0/2866957044 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc6600a2b80 0x7fc6600a5040 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.857+0000 7fc67ea15640 1 --2- 172.21.15.78:0/2866957044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6781078c0 0x7fc678074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.857+0000 7fc67ea15640 1 --2- 172.21.15.78:0/2866957044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc678102260 0x7fc678074760 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.857+0000 7fc67ea15640 1 --2- 172.21.15.78:0/2866957044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6781018c0 0x7fc678074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:46:47.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.857+0000 7fc67ea15640 1 -- 172.21.15.78:0/2866957044 >> 172.21.15.78:0/2866957044 conn(0x7fc6780fd5e0 msgr2=0x7fc6780fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:46:47.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.857+0000 7fc67ea15640 1 -- 172.21.15.78:0/2866957044 shutdown_connections 2024-08-18T23:46:47.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:46:47.857+0000 7fc67ea15640 1 -- 172.21.15.78:0/2866957044 wait complete. 2024-08-18T23:46:48.717 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '73198cf', total seconds elapsed: 25.3 2024-08-18T23:46:49.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:49 smithi123 ceph-mon[21808]: pgmap v425: 129 pgs: 129 active+clean; 306 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 249 op/s 2024-08-18T23:46:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:49 smithi161 ceph-mon[22027]: pgmap v425: 129 pgs: 129 active+clean; 306 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 249 op/s 2024-08-18T23:46:49.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:49 smithi078 ceph-mon[18440]: pgmap v425: 129 pgs: 129 active+clean; 306 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 249 op/s 2024-08-18T23:46:51.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:51 smithi123 ceph-mon[21808]: pgmap v426: 129 pgs: 129 active+clean; 314 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 283 op/s 2024-08-18T23:46:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:51 smithi161 ceph-mon[22027]: pgmap v426: 129 pgs: 129 active+clean; 314 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 283 op/s 2024-08-18T23:46:51.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:51 smithi078 ceph-mon[18440]: pgmap v426: 129 pgs: 129 active+clean; 314 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 283 op/s 2024-08-18T23:46:53.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:53 smithi123 ceph-mon[21808]: pgmap v427: 129 pgs: 129 active+clean; 329 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 220 op/s 2024-08-18T23:46:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:53 smithi161 ceph-mon[22027]: pgmap v427: 129 pgs: 129 active+clean; 329 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 220 op/s 2024-08-18T23:46:53.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:53 smithi078 ceph-mon[18440]: pgmap v427: 129 pgs: 129 active+clean; 329 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 220 op/s 2024-08-18T23:46:55.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:55 smithi123 ceph-mon[21808]: pgmap v428: 129 pgs: 129 active+clean; 336 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 276 op/s 2024-08-18T23:46:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:55 smithi161 ceph-mon[22027]: pgmap v428: 129 pgs: 129 active+clean; 336 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 276 op/s 2024-08-18T23:46:55.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:55 smithi078 ceph-mon[18440]: pgmap v428: 129 pgs: 129 active+clean; 336 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 276 op/s 2024-08-18T23:46:56.982 INFO:tasks.workunit.client.0.smithi078.stdout: 64 31947 158 21806 165 16110 251 2024-08-18T23:46:57.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:57 smithi123 ceph-mon[21808]: pgmap v429: 129 pgs: 129 active+clean; 349 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 370 op/s 2024-08-18T23:46:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:57 smithi161 ceph-mon[22027]: pgmap v429: 129 pgs: 129 active+clean; 349 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 370 op/s 2024-08-18T23:46:57.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:57 smithi078 ceph-mon[18440]: pgmap v429: 129 pgs: 129 active+clean; 349 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 370 op/s 2024-08-18T23:46:59.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:46:59 smithi078 ceph-mon[18440]: pgmap v430: 129 pgs: 129 active+clean; 350 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 382 op/s 2024-08-18T23:46:59.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:46:59 smithi123 ceph-mon[21808]: pgmap v430: 129 pgs: 129 active+clean; 350 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 382 op/s 2024-08-18T23:46:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:46:59 smithi161 ceph-mon[22027]: pgmap v430: 129 pgs: 129 active+clean; 350 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 382 op/s 2024-08-18T23:47:01.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:01 smithi078 ceph-mon[18440]: pgmap v431: 129 pgs: 129 active+clean; 350 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 397 op/s 2024-08-18T23:47:01.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:01 smithi123 ceph-mon[21808]: pgmap v431: 129 pgs: 129 active+clean; 350 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 397 op/s 2024-08-18T23:47:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:01 smithi161 ceph-mon[22027]: pgmap v431: 129 pgs: 129 active+clean; 350 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 397 op/s 2024-08-18T23:47:03.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:03 smithi123 ceph-mon[21808]: pgmap v432: 129 pgs: 129 active+clean; 355 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 377 op/s 2024-08-18T23:47:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:03 smithi161 ceph-mon[22027]: pgmap v432: 129 pgs: 129 active+clean; 355 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 377 op/s 2024-08-18T23:47:03.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:03 smithi078 ceph-mon[18440]: pgmap v432: 129 pgs: 129 active+clean; 355 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 377 op/s 2024-08-18T23:47:03.894 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:47:04.654 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:47:05.431 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:05 smithi078 ceph-mon[18440]: pgmap v433: 129 pgs: 129 active+clean; 355 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 276 op/s 2024-08-18T23:47:05.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.427+0000 7fd2e61d2640 1 -- 172.21.15.78:0/1029136568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e0109560 msgr2=0x7fd2e01099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:05.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.427+0000 7fd2e61d2640 1 --2- 172.21.15.78:0/1029136568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e0109560 0x7fd2e01099c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009ca0 tx=0x7fd2d802a8d0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.429+0000 7fd2e61d2640 1 -- 172.21.15.78:0/1029136568 shutdown_connections 2024-08-18T23:47:05.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.429+0000 7fd2e61d2640 1 --2- 172.21.15.78:0/1029136568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2e011ae70 0x7fd2e011d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.429+0000 7fd2e61d2640 1 --2- 172.21.15.78:0/1029136568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e0109560 0x7fd2e01099c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.429+0000 7fd2e61d2640 1 --2- 172.21.15.78:0/1029136568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e0109180 0x7fd2e010ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.429+0000 7fd2e61d2640 1 -- 172.21.15.78:0/1029136568 >> 172.21.15.78:0/1029136568 conn(0x7fd2e006de70 msgr2=0x7fd2e006e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:05.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.437+0000 7fd2e61d2640 1 -- 172.21.15.78:0/1029136568 shutdown_connections 2024-08-18T23:47:05.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.440+0000 7fd2e61d2640 1 -- 172.21.15.78:0/1029136568 wait complete. 2024-08-18T23:47:05.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.441+0000 7fd2e61d2640 1 Processor -- start 2024-08-18T23:47:05.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.446+0000 7fd2e61d2640 1 -- start start 2024-08-18T23:47:05.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.447+0000 7fd2e61d2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e0109180 0x7fd2e01a6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:05.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.447+0000 7fd2e61d2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e0109560 0x7fd2e01a7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:05.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.447+0000 7fd2e61d2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2e011ae70 0x7fd2e01ab890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:05.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.447+0000 7fd2e61d2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e0120b20 con 0x7fd2e0109560 2024-08-18T23:47:05.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.447+0000 7fd2e61d2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e01209a0 con 0x7fd2e0109180 2024-08-18T23:47:05.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.448+0000 7fd2e61d2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e0120ca0 con 0x7fd2e011ae70 2024-08-18T23:47:05.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.449+0000 7fd2deffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e0109560 0x7fd2e01a7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:05.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.449+0000 7fd2df7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e0109180 0x7fd2e01a6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:05.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.449+0000 7fd2df7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e0109180 0x7fd2e01a6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:53908/0 (socket says 172.21.15.78:53908) 2024-08-18T23:47:05.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.449+0000 7fd2df7fe640 1 -- 172.21.15.78:0/1706089239 learned_addr learned my addr 172.21.15.78:0/1706089239 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:05.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.450+0000 7fd2df7fe640 1 -- 172.21.15.78:0/1706089239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2e011ae70 msgr2=0x7fd2e01ab890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-18T23:47:05.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.450+0000 7fd2df7fe640 1 --2- 172.21.15.78:0/1706089239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2e011ae70 0x7fd2e01ab890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.450+0000 7fd2df7fe640 1 -- 172.21.15.78:0/1706089239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e0109560 msgr2=0x7fd2e01a7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:05.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.450+0000 7fd2df7fe640 1 --2- 172.21.15.78:0/1706089239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e0109560 0x7fd2e01a7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.450+0000 7fd2df7fe640 1 -- 172.21.15.78:0/1706089239 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e01abf70 con 0x7fd2e0109180 2024-08-18T23:47:05.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.453+0000 7fd2df7fe640 1 --2- 172.21.15.78:0/1706089239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e0109180 0x7fd2e01a6fc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000c920 tx=0x7fd2d000cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:05.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.455+0000 7fd2dcff9640 1 -- 172.21.15.78:0/1706089239 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0007c70 con 0x7fd2e0109180 2024-08-18T23:47:05.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.456+0000 7fd2e61d2640 1 -- 172.21.15.78:0/1706089239 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e01ac260 con 0x7fd2e0109180 2024-08-18T23:47:05.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.456+0000 7fd2e61d2640 1 -- 172.21.15.78:0/1706089239 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e01b3b40 con 0x7fd2e0109180 2024-08-18T23:47:05.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.456+0000 7fd2dcff9640 1 -- 172.21.15.78:0/1706089239 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2d0007e10 con 0x7fd2e0109180 2024-08-18T23:47:05.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.456+0000 7fd2dcff9640 1 -- 172.21.15.78:0/1706089239 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0018960 con 0x7fd2e0109180 2024-08-18T23:47:05.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.460+0000 7fd2e85d0740 1 Processor -- start 2024-08-18T23:47:05.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.461+0000 7fd2e85d0740 1 -- start start 2024-08-18T23:47:05.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.465+0000 7fd2e85d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98210 0x55f88ea985f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:05.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.465+0000 7fd2e85d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88ea98b30 0x55f88eaa2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:05.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.465+0000 7fd2e85d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88eaa33f0 0x55f88eaa57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:05.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.465+0000 7fd2e85d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f88e9fd980 con 0x55f88eaa33f0 2024-08-18T23:47:05.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.465+0000 7fd2e85d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f88e995a70 con 0x55f88ea98210 2024-08-18T23:47:05.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.465+0000 7fd2e85d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f88e9f9940 con 0x55f88ea98b30 2024-08-18T23:47:05.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.466+0000 7fd2dffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98210 0x55f88ea985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:05.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.466+0000 7fd2dffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98210 0x55f88ea985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:53914/0 (socket says 172.21.15.78:53914) 2024-08-18T23:47:05.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.466+0000 7fd2dffff640 1 -- 172.21.15.78:0/336475746 learned_addr learned my addr 172.21.15.78:0/336475746 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:05.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.467+0000 7fd2dffff640 1 -- 172.21.15.78:0/336475746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88ea98b30 msgr2=0x55f88eaa2eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T23:47:05.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.467+0000 7fd2dffff640 1 --2- 172.21.15.78:0/336475746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88ea98b30 0x55f88eaa2eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.467+0000 7fd2dffff640 1 -- 172.21.15.78:0/336475746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88eaa33f0 msgr2=0x55f88eaa57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:05.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.467+0000 7fd2dffff640 1 --2- 172.21.15.78:0/336475746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88eaa33f0 0x55f88eaa57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.467+0000 7fd2dffff640 1 -- 172.21.15.78:0/336475746 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f88e9f86e0 con 0x55f88ea98210 2024-08-18T23:47:05.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.468+0000 7fd2dffff640 1 --2- 172.21.15.78:0/336475746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98210 0x55f88ea985f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400a850 tx=0x7fd2d400ad10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf9888800ff45a6e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:05.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.469+0000 7fd2c5ffb640 1 -- 172.21.15.78:0/336475746 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d400ccc0 con 0x55f88ea98210 2024-08-18T23:47:05.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.469+0000 7fd2c5ffb640 1 -- 172.21.15.78:0/336475746 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2d400f930 con 0x55f88ea98210 2024-08-18T23:47:05.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.469+0000 7fd2c5ffb640 1 -- 172.21.15.78:0/336475746 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4018930 con 0x55f88ea98210 2024-08-18T23:47:05.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.472+0000 7fd2e85d0740 1 -- 172.21.15.78:0/336475746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98210 msgr2=0x55f88ea985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:05.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.472+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/336475746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98210 0x55f88ea985f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400a850 tx=0x7fd2d400ad10 comp rx=0 tx=0).stop 2024-08-18T23:47:05.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.472+0000 7fd2dcff9640 1 -- 172.21.15.78:0/1706089239 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd2d0018b00 con 0x7fd2e0109180 2024-08-18T23:47:05.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.473+0000 7fd2dcff9640 1 --2- 172.21.15.78:0/1706089239 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2c80a2d90 0x7fd2c80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:05.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.474+0000 7fd2dcff9640 1 -- 172.21.15.78:0/1706089239 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fd2d00cf370 con 0x7fd2e0109180 2024-08-18T23:47:05.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.474+0000 7fd2dffff640 1 --2- 172.21.15.78:0/1706089239 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2c80a2d90 0x7fd2c80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:05.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.474+0000 7fd2e85d0740 1 -- 172.21.15.78:0/336475746 shutdown_connections 2024-08-18T23:47:05.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.474+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/336475746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88eaa33f0 0x55f88eaa57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.474+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/336475746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88ea98b30 0x55f88eaa2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.474+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/336475746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98210 0x55f88ea985f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.474+0000 7fd2e85d0740 1 -- 172.21.15.78:0/336475746 >> 172.21.15.78:0/336475746 conn(0x55f88ea947a0 msgr2=0x55f88ea94b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:05.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.475+0000 7fd2dffff640 1 --2- 172.21.15.78:0/1706089239 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2c80a2d90 0x7fd2c80a5250 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400c6a0 tx=0x7fd2d400a540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:05.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.480+0000 7fd2e85d0740 1 -- 172.21.15.78:0/336475746 shutdown_connections 2024-08-18T23:47:05.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.482+0000 7fd2e85d0740 1 -- 172.21.15.78:0/336475746 wait complete. 2024-08-18T23:47:05.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.482+0000 7fd2e85d0740 1 -- start start 2024-08-18T23:47:05.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.483+0000 7fd2e85d0740 1 Processor -- start 2024-08-18T23:47:05.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.483+0000 7fd2e85d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98210 0x55f88eaa0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:05.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.483+0000 7fd2e85d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88ea98b30 0x55f88eaa0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:05.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.483+0000 7fd2e85d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88eaa33f0 0x55f88eaa0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:05.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.483+0000 7fd2e85d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f88eaa5fc0 con 0x55f88eaa33f0 2024-08-18T23:47:05.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.483+0000 7fd2e85d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f88e9b8aa0 con 0x55f88ea98210 2024-08-18T23:47:05.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.483+0000 7fd2e85d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f88eaa6140 con 0x55f88ea98b30 2024-08-18T23:47:05.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.483+0000 7fd2e85d0740 10 client.0 _ll_register_callbacks cb 0x55f88e9a7e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:05.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.484+0000 7fd2deffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88ea98b30 0x55f88eaa0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:05.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.484+0000 7fd2deffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88ea98b30 0x55f88eaa0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:41042/0 (socket says 172.21.15.78:41042) 2024-08-18T23:47:05.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.484+0000 7fd2deffd640 1 -- 172.21.15.78:0/1479868679 learned_addr learned my addr 172.21.15.78:0/1479868679 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:05.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.484+0000 7fd2deffd640 1 -- 172.21.15.78:0/1479868679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98210 msgr2=0x55f88eaa0250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T23:47:05.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.484+0000 7fd2deffd640 1 --2- 172.21.15.78:0/1479868679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98210 0x55f88eaa0250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.484+0000 7fd2deffd640 1 -- 172.21.15.78:0/1479868679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88eaa33f0 msgr2=0x55f88eaa0cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T23:47:05.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.484+0000 7fd2deffd640 1 --2- 172.21.15.78:0/1479868679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88eaa33f0 0x55f88eaa0cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.484+0000 7fd2deffd640 1 -- 172.21.15.78:0/1479868679 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f88e99c650 con 0x55f88ea98b30 2024-08-18T23:47:05.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.484+0000 7fd2deffd640 1 --2- 172.21.15.78:0/1479868679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88ea98b30 0x55f88eaa0790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009ca0 tx=0x7fd2d8031e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ada31b16a829c31 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:05.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.488+0000 7fd2beffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:47:05.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.488+0000 7fd2beffd640 1 -- 172.21.15.78:0/1479868679 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8030c10 con 0x55f88ea98b30 2024-08-18T23:47:05.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.488+0000 7fd2beffd640 1 -- 172.21.15.78:0/1479868679 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fd2d8030db0 con 0x55f88ea98b30 2024-08-18T23:47:05.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.489+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1479868679 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2d400f930 con 0x55f88ea98b30 2024-08-18T23:47:05.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.495+0000 7fd2beffd640 1 -- 172.21.15.78:0/1479868679 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd2d8003970 con 0x55f88ea98b30 2024-08-18T23:47:05.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.495+0000 7fd2e85d0740 10 client.25231 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:05.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.495+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1479868679 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f88e76f3c0 con 0x55f88ea98b30 2024-08-18T23:47:05.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.496+0000 7fd2beffd640 1 -- 172.21.15.78:0/1479868679 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2d400f930 con 0x55f88ea98b30 2024-08-18T23:47:05.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.496+0000 7fd2beffd640 1 -- 172.21.15.78:0/1479868679 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fd2d8004260 con 0x55f88ea98b30 2024-08-18T23:47:05.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.496+0000 7fd2e85d0740 10 client.25231 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.496+0000 7fd2e85d0740 10 client.25231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:47:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.496+0000 7fd2e85d0740 20 client.25231 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.496+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/1479868679 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f88eab3350 0x55f88eab57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.496+0000 7fd2e85d0740 4 client.25231 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:47:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.496+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1479868679 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f88ea0aef0 con 0x55f88eab3350 2024-08-18T23:47:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.502+0000 7fd2df7fe640 1 --2- 172.21.15.78:0/1479868679 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f88eab3350 0x55f88eab57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.512+0000 7fd2df7fe640 1 --2- 172.21.15.78:0/1479868679 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f88eab3350 0x55f88eab57b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daf2d5e38da8af56 server_cookie=262ffde718cd443e in_seq=0 out_seq=0 2024-08-18T23:47:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.512+0000 7fd2beffd640 10 client.25231 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.517+0000 7fd2beffd640 1 -- 172.21.15.78:0/1479868679 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f88ea0aef0 con 0x55f88eab3350 2024-08-18T23:47:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.517+0000 7fd2beffd640 10 client.25231 handle_command_reply: tid=0 2024-08-18T23:47:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.517+0000 7fd2e85d0740 1 client.25231 shutdown 2024-08-18T23:47:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.522+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1479868679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88ea98b30 msgr2=0x55f88eaa0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.522+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/1479868679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88ea98b30 0x55f88eaa0790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009ca0 tx=0x7fd2d8031e20 comp rx=0 tx=0).stop 2024-08-18T23:47:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.522+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1479868679 shutdown_connections 2024-08-18T23:47:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.522+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/1479868679 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f88eab3350 0x55f88eab57b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.522+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/1479868679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88eaa33f0 0x55f88eaa0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.522+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/1479868679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88ea98b30 0x55f88eaa0790 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.522+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/1479868679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98210 0x55f88eaa0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.522+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1479868679 >> 172.21.15.78:0/1479868679 conn(0x55f88ea947a0 msgr2=0x55f88ea9a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.522+0000 7fd2beffd640 0 client.25231 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.522+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1479868679 shutdown_connections 2024-08-18T23:47:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.522+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1479868679 wait complete. 2024-08-18T23:47:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.523+0000 7fd2e85d0740 20 client.25231 trim_cache size 0 max 16384 2024-08-18T23:47:05.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:05 smithi123 ceph-mon[21808]: pgmap v433: 129 pgs: 129 active+clean; 355 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 276 op/s 2024-08-18T23:47:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.571+0000 7fd2e85d0740 1 Processor -- start 2024-08-18T23:47:05.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.571+0000 7fd2e85d0740 1 -- start start 2024-08-18T23:47:05.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.572+0000 7fd2e85d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88ea98210 0x55f88ea9ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:05.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.572+0000 7fd2df7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88ea98210 0x55f88ea9ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:05.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.573+0000 7fd2df7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88ea98210 0x55f88ea9ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43918/0 (socket says 172.21.15.78:43918) 2024-08-18T23:47:05.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.573+0000 7fd2df7fe640 1 -- 172.21.15.78:0/1006098825 learned_addr learned my addr 172.21.15.78:0/1006098825 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:05.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.573+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/1006098825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98b30 0x55f88eaa0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:05.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.573+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/1006098825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88eaa33f0 0x55f88eabb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:05.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.573+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1006098825 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc012580 con 0x55f88ea98210 2024-08-18T23:47:05.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.573+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1006098825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc00fe50 con 0x55f88ea98b30 2024-08-18T23:47:05.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.574+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1006098825 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc00dca0 con 0x55f88eaa33f0 2024-08-18T23:47:05.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.574+0000 7fd2df7fe640 1 -- 172.21.15.78:0/1006098825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88eaa33f0 msgr2=0x55f88eabb080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-18T23:47:05.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.574+0000 7fd2df7fe640 1 --2- 172.21.15.78:0/1006098825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88eaa33f0 0x55f88eabb080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.574+0000 7fd2df7fe640 1 -- 172.21.15.78:0/1006098825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98b30 msgr2=0x55f88eaa0b20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-18T23:47:05.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.574+0000 7fd2df7fe640 1 --2- 172.21.15.78:0/1006098825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98b30 0x55f88eaa0b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.574+0000 7fd2df7fe640 1 -- 172.21.15.78:0/1006098825 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f88eab1a70 con 0x55f88ea98210 2024-08-18T23:47:05.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.575+0000 7fd2df7fe640 1 --2- 172.21.15.78:0/1006098825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88ea98210 0x55f88ea9ac70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00981b0 tx=0x7fd2d00cc950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f942f3054fb9e1e8 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:05.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.576+0000 7fd2c57fa640 1 -- 172.21.15.78:0/1006098825 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d00b3070 con 0x55f88ea98210 2024-08-18T23:47:05.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.576+0000 7fd2c57fa640 1 -- 172.21.15.78:0/1006098825 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2d00a4450 con 0x55f88ea98210 2024-08-18T23:47:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.577+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1006098825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88ea98210 msgr2=0x55f88ea9ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.577+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/1006098825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88ea98210 0x55f88ea9ac70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00981b0 tx=0x7fd2d00cc950 comp rx=0 tx=0).stop 2024-08-18T23:47:05.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.580+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1006098825 shutdown_connections 2024-08-18T23:47:05.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.580+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/1006098825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88eaa33f0 0x55f88eabb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.580+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/1006098825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98b30 0x55f88eaa0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.580+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/1006098825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88ea98210 0x55f88ea9ac70 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.580+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1006098825 >> 172.21.15.78:0/1006098825 conn(0x55f88ea947a0 msgr2=0x55f88ea9f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:05.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.583+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1006098825 shutdown_connections 2024-08-18T23:47:05.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.583+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1006098825 wait complete. 2024-08-18T23:47:05.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.583+0000 7fd2e85d0740 1 -- start start 2024-08-18T23:47:05.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.584+0000 7fd2e85d0740 1 Processor -- start 2024-08-18T23:47:05.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.584+0000 7fd2e85d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88ea98210 0x55f88eaa6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:05.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.584+0000 7fd2e85d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98b30 0x55f88eaa7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:05.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.584+0000 7fd2e85d0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88eaa33f0 0x55f88eb16080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:05.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.584+0000 7fd2e85d0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc00e310 con 0x55f88eaa33f0 2024-08-18T23:47:05.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.584+0000 7fd2e85d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc00f780 con 0x55f88ea98b30 2024-08-18T23:47:05.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.584+0000 7fd2e85d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc00e9e0 con 0x55f88ea98210 2024-08-18T23:47:05.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.585+0000 7fd2e85d0740 10 client.0 _ll_register_callbacks cb 0x7fd2d8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:05.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.587+0000 7fd2df7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88ea98210 0x55f88eaa6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:05.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.587+0000 7fd2deffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98b30 0x55f88eaa7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:05.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.587+0000 7fd2df7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88ea98210 0x55f88eaa6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:41064/0 (socket says 172.21.15.78:41064) 2024-08-18T23:47:05.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.587+0000 7fd2df7fe640 1 -- 172.21.15.78:0/1100608591 learned_addr learned my addr 172.21.15.78:0/1100608591 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:05.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.587+0000 7fd2dffff640 1 --2- 172.21.15.78:0/1100608591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88eaa33f0 0x55f88eb16080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:05.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.588+0000 7fd2deffd640 1 -- 172.21.15.78:0/1100608591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88ea98210 msgr2=0x55f88eaa6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:05.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.588+0000 7fd2deffd640 1 --2- 172.21.15.78:0/1100608591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88ea98210 0x55f88eaa6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.588+0000 7fd2deffd640 1 -- 172.21.15.78:0/1100608591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88eaa33f0 msgr2=0x55f88eb16080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:05.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.588+0000 7fd2deffd640 1 --2- 172.21.15.78:0/1100608591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88eaa33f0 0x55f88eb16080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.588+0000 7fd2deffd640 1 -- 172.21.15.78:0/1100608591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f88eb16640 con 0x55f88ea98b30 2024-08-18T23:47:05.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.590+0000 7fd2deffd640 1 --2- 172.21.15.78:0/1100608591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98b30 0x55f88eaa7070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55f88eaa1890 tx=0x7fd2d80021d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20f0fd2ed738a007 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:05.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.590+0000 7fd2beffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:47:05.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.590+0000 7fd2beffd640 1 -- 172.21.15.78:0/1100608591 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8030c20 con 0x55f88ea98b30 2024-08-18T23:47:05.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.590+0000 7fd2beffd640 1 -- 172.21.15.78:0/1100608591 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fd2d8030dc0 con 0x55f88ea98b30 2024-08-18T23:47:05.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.590+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1100608591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2d00a4450 con 0x55f88ea98b30 2024-08-18T23:47:05.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.591+0000 7fd2beffd640 1 -- 172.21.15.78:0/1100608591 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd2d8004b40 con 0x55f88ea98b30 2024-08-18T23:47:05.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.591+0000 7fd2e85d0740 10 client.25349 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:05.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.591+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1100608591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f88eb2ea10 con 0x55f88ea98b30 2024-08-18T23:47:05.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.592+0000 7fd2beffd640 1 -- 172.21.15.78:0/1100608591 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2d00a4450 con 0x55f88ea98b30 2024-08-18T23:47:05.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.592+0000 7fd2beffd640 1 -- 172.21.15.78:0/1100608591 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fd2d804faf0 con 0x55f88ea98b30 2024-08-18T23:47:05.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.594+0000 7fd2e85d0740 10 client.25349 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:05.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.594+0000 7fd2e85d0740 10 client.25349 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:47:05.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.594+0000 7fd2e85d0740 20 client.25349 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:05.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.594+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/1100608591 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f88eb2f0e0 0x55f88eb31500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:05.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.594+0000 7fd2e85d0740 4 client.25349 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:47:05.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.594+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1100608591 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f88ea05400 con 0x55f88eb2f0e0 2024-08-18T23:47:05.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.595+0000 7fd2df7fe640 1 --2- 172.21.15.78:0/1100608591 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f88eb2f0e0 0x55f88eb31500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:05.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.606+0000 7fd2df7fe640 1 --2- 172.21.15.78:0/1100608591 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f88eb2f0e0 0x55f88eb31500 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c1b3cec900f4f2e server_cookie=7af8e440e8c1042c in_seq=0 out_seq=0 2024-08-18T23:47:05.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.606+0000 7fd2beffd640 10 client.25349 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:05.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.607+0000 7fd2beffd640 1 -- 172.21.15.78:0/1100608591 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f88ea05400 con 0x55f88eb2f0e0 2024-08-18T23:47:05.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.607+0000 7fd2beffd640 10 client.25349 handle_command_reply: tid=0 2024-08-18T23:47:05.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.607+0000 7fd2e85d0740 1 client.25349 shutdown 2024-08-18T23:47:05.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.610+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1100608591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98b30 msgr2=0x55f88eaa7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:05.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.610+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/1100608591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98b30 0x55f88eaa7070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55f88eaa1890 tx=0x7fd2d80021d0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.614+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1100608591 shutdown_connections 2024-08-18T23:47:05.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.614+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/1100608591 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f88eb2f0e0 0x55f88eb31500 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.614+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/1100608591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f88eaa33f0 0x55f88eb16080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.614+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/1100608591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f88ea98b30 0x55f88eaa7070 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.614+0000 7fd2e85d0740 1 --2- 172.21.15.78:0/1100608591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f88ea98210 0x55f88eaa6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.614+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1100608591 >> 172.21.15.78:0/1100608591 conn(0x55f88ea947a0 msgr2=0x55f88ea930e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:05.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.614+0000 7fd2beffd640 0 client.25349 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:05.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.614+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1100608591 shutdown_connections 2024-08-18T23:47:05.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.615+0000 7fd2e85d0740 1 -- 172.21.15.78:0/1100608591 wait complete. 2024-08-18T23:47:05.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.617+0000 7fd2e85d0740 20 client.25349 trim_cache size 0 max 16384 2024-08-18T23:47:05.622 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:47:05.622 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-08-18T23:47:05.622 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-08-18T23:47:05.622 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:47:05.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.621+0000 7fd2e61d2640 1 -- 172.21.15.78:0/1706089239 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2c80a2d90 msgr2=0x7fd2c80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:05.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.621+0000 7fd2e61d2640 1 --2- 172.21.15.78:0/1706089239 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2c80a2d90 0x7fd2c80a5250 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400c6a0 tx=0x7fd2d400a540 comp rx=0 tx=0).stop 2024-08-18T23:47:05.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.621+0000 7fd2e61d2640 1 -- 172.21.15.78:0/1706089239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e0109180 msgr2=0x7fd2e01a6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:05.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.621+0000 7fd2e61d2640 1 --2- 172.21.15.78:0/1706089239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e0109180 0x7fd2e01a6fc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000c920 tx=0x7fd2d000cde0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.623+0000 7fd2e61d2640 1 -- 172.21.15.78:0/1706089239 shutdown_connections 2024-08-18T23:47:05.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.623+0000 7fd2e61d2640 1 --2- 172.21.15.78:0/1706089239 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2c80a2d90 0x7fd2c80a5250 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.623+0000 7fd2e61d2640 1 --2- 172.21.15.78:0/1706089239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2e011ae70 0x7fd2e01ab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.623+0000 7fd2e61d2640 1 --2- 172.21.15.78:0/1706089239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2e0109560 0x7fd2e01a7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.623+0000 7fd2e61d2640 1 --2- 172.21.15.78:0/1706089239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2e0109180 0x7fd2e01a6fc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:05.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.623+0000 7fd2e61d2640 1 -- 172.21.15.78:0/1706089239 >> 172.21.15.78:0/1706089239 conn(0x7fd2e006de70 msgr2=0x7fd2e011bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:05.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.623+0000 7fd2e61d2640 1 -- 172.21.15.78:0/1706089239 shutdown_connections 2024-08-18T23:47:05.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:05.623+0000 7fd2e61d2640 1 -- 172.21.15.78:0/1706089239 wait complete. 2024-08-18T23:47:05.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:05 smithi161 ceph-mon[22027]: pgmap v433: 129 pgs: 129 active+clean; 355 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 276 op/s 2024-08-18T23:47:06.982 INFO:tasks.workunit.client.0.smithi078.stdout: 74 1443368 434 965430 627 714720 1095 2024-08-18T23:47:07.130 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-18T23:47:07.130 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:47:07.289 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-18T23:47:07.290 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-08-18T23:47:07.290 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:47:07.436 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:07 smithi078 ceph-mon[18440]: pgmap v434: 129 pgs: 129 active+clean; 364 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 286 op/s 2024-08-18T23:47:07.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:07 smithi123 ceph-mon[21808]: pgmap v434: 129 pgs: 129 active+clean; 364 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 286 op/s 2024-08-18T23:47:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:07 smithi161 ceph-mon[22027]: pgmap v434: 129 pgs: 129 active+clean; 364 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 286 op/s 2024-08-18T23:47:07.887 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:47:07.894 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:47:08.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.530+0000 7f8a9000e640 1 -- 172.21.15.78:0/355646018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a8811ae70 msgr2=0x7f8a8811d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:08.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.530+0000 7f8a9000e640 1 --2- 172.21.15.78:0/355646018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a8811ae70 0x7f8a8811d260 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009ca0 tx=0x7f8a8002a8b0 comp rx=0 tx=0).stop 2024-08-18T23:47:08.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.534+0000 7f8a9000e640 1 -- 172.21.15.78:0/355646018 shutdown_connections 2024-08-18T23:47:08.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.534+0000 7f8a9000e640 1 --2- 172.21.15.78:0/355646018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a8811ae70 0x7f8a8811d260 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:08.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.534+0000 7f8a9000e640 1 --2- 172.21.15.78:0/355646018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a88109560 0x7f8a881099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:08.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.534+0000 7f8a9000e640 1 --2- 172.21.15.78:0/355646018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a88109180 0x7f8a8810ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:08.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.534+0000 7f8a9000e640 1 -- 172.21.15.78:0/355646018 >> 172.21.15.78:0/355646018 conn(0x7f8a8806de70 msgr2=0x7f8a8806e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.540+0000 7f8a9000e640 1 -- 172.21.15.78:0/355646018 shutdown_connections 2024-08-18T23:47:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.541+0000 7f8a9000e640 1 -- 172.21.15.78:0/355646018 wait complete. 2024-08-18T23:47:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.541+0000 7f8a9000e640 1 Processor -- start 2024-08-18T23:47:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.542+0000 7f8a9000e640 1 -- start start 2024-08-18T23:47:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.542+0000 7f8a9000e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a88109180 0x7f8a88118390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.542+0000 7f8a9000e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a88109560 0x7f8a881188d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.542+0000 7f8a9000e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a8811ae70 0x7f8a881115b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.542+0000 7f8a9000e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8a88120af0 con 0x7f8a8811ae70 2024-08-18T23:47:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.542+0000 7f8a9000e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a88120970 con 0x7f8a88109180 2024-08-18T23:47:08.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.542+0000 7f8a9000e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8a88120c70 con 0x7f8a88109560 2024-08-18T23:47:08.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.543+0000 7f8a8e5ab640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a8811ae70 0x7f8a881115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:08.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.543+0000 7f8a8e5ab640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a8811ae70 0x7f8a881115b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41974/0 (socket says 172.21.15.78:41974) 2024-08-18T23:47:08.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.543+0000 7f8a8e5ab640 1 -- 172.21.15.78:0/2567104960 learned_addr learned my addr 172.21.15.78:0/2567104960 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:08.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.543+0000 7f8a8e5ab640 1 -- 172.21.15.78:0/2567104960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a88109560 msgr2=0x7f8a881188d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:08.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.543+0000 7f8a8e5ab640 1 --2- 172.21.15.78:0/2567104960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a88109560 0x7f8a881188d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:08.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.543+0000 7f8a8e5ab640 1 -- 172.21.15.78:0/2567104960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a88109180 msgr2=0x7f8a88118390 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-18T23:47:08.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.543+0000 7f8a8e5ab640 1 --2- 172.21.15.78:0/2567104960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a88109180 0x7f8a88118390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:08.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.543+0000 7f8a8e5ab640 1 -- 172.21.15.78:0/2567104960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a88111e70 con 0x7f8a8811ae70 2024-08-18T23:47:08.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.547+0000 7f8a8e5ab640 1 --2- 172.21.15.78:0/2567104960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a8811ae70 0x7f8a881115b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8a80003f40 tx=0x7f8a80003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:08.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.548+0000 7f8a7affd640 1 -- 172.21.15.78:0/2567104960 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80030d40 con 0x7f8a8811ae70 2024-08-18T23:47:08.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.548+0000 7f8a7affd640 1 -- 172.21.15.78:0/2567104960 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a80040070 con 0x7f8a8811ae70 2024-08-18T23:47:08.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.548+0000 7f8a7affd640 1 -- 172.21.15.78:0/2567104960 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80004780 con 0x7f8a8811ae70 2024-08-18T23:47:08.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.553+0000 7f8a9000e640 1 -- 172.21.15.78:0/2567104960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a88112100 con 0x7f8a8811ae70 2024-08-18T23:47:08.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.557+0000 7f8a9000e640 1 -- 172.21.15.78:0/2567104960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a88116090 con 0x7f8a8811ae70 2024-08-18T23:47:08.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.559+0000 7f8a7affd640 1 -- 172.21.15.78:0/2567104960 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8a80031880 con 0x7f8a8811ae70 2024-08-18T23:47:08.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.560+0000 7f8a9000e640 1 -- 172.21.15.78:0/2567104960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a88073830 con 0x7f8a8811ae70 2024-08-18T23:47:08.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.564+0000 7f8a7affd640 1 --2- 172.21.15.78:0/2567104960 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8a740a2d10 0x7f8a740a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:08.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.569+0000 7f8a8ddaa640 1 --2- 172.21.15.78:0/2567104960 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8a740a2d10 0x7f8a740a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:08.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.570+0000 7f8a8ddaa640 1 --2- 172.21.15.78:0/2567104960 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8a740a2d10 0x7f8a740a51d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400a8e0 tx=0x7f8a84005b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:08.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.577+0000 7f8a7affd640 1 -- 172.21.15.78:0/2567104960 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f8a80009890 con 0x7f8a8811ae70 2024-08-18T23:47:08.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.577+0000 7f8a7affd640 1 -- 172.21.15.78:0/2567104960 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8a80044e90 con 0x7f8a8811ae70 2024-08-18T23:47:08.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.881+0000 7f8a9000e640 1 -- 172.21.15.78:0/2567104960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a88116ca0 con 0x7f8a8811ae70 2024-08-18T23:47:08.884 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:47:08.884 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":26,"btime":"2024-08-18T23:45:33:306257+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:45:32.491054+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:47:08.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.882+0000 7f8a7affd640 1 -- 172.21.15.78:0/2567104960 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11933 (secure 0 0 0) 0x7f8a800bc980 con 0x7f8a8811ae70 2024-08-18T23:47:08.884 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 26 2024-08-18T23:47:08.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.884+0000 7f8a9000e640 1 -- 172.21.15.78:0/2567104960 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8a740a2d10 msgr2=0x7f8a740a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:08.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.884+0000 7f8a9000e640 1 --2- 172.21.15.78:0/2567104960 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8a740a2d10 0x7f8a740a51d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400a8e0 tx=0x7f8a84005b80 comp rx=0 tx=0).stop 2024-08-18T23:47:08.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.884+0000 7f8a9000e640 1 -- 172.21.15.78:0/2567104960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a8811ae70 msgr2=0x7f8a881115b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:08.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.884+0000 7f8a9000e640 1 --2- 172.21.15.78:0/2567104960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a8811ae70 0x7f8a881115b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8a80003f40 tx=0x7f8a80003f70 comp rx=0 tx=0).stop 2024-08-18T23:47:08.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.884+0000 7f8a9000e640 1 -- 172.21.15.78:0/2567104960 shutdown_connections 2024-08-18T23:47:08.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.884+0000 7f8a9000e640 1 --2- 172.21.15.78:0/2567104960 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8a740a2d10 0x7f8a740a51d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:08.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.884+0000 7f8a9000e640 1 --2- 172.21.15.78:0/2567104960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a8811ae70 0x7f8a881115b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:08.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.884+0000 7f8a9000e640 1 --2- 172.21.15.78:0/2567104960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a88109560 0x7f8a881188d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:08.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.884+0000 7f8a9000e640 1 --2- 172.21.15.78:0/2567104960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a88109180 0x7f8a88118390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:08.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.884+0000 7f8a9000e640 1 -- 172.21.15.78:0/2567104960 >> 172.21.15.78:0/2567104960 conn(0x7f8a8806de70 msgr2=0x7f8a8811bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:08.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.885+0000 7f8a9000e640 1 -- 172.21.15.78:0/2567104960 shutdown_connections 2024-08-18T23:47:08.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:08.885+0000 7f8a9000e640 1 -- 172.21.15.78:0/2567104960 wait complete. 2024-08-18T23:47:09.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.056+0000 7f2b5cca4640 1 -- 172.21.15.78:0/3227997182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b5810e200 msgr2=0x7f2b58110610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:09.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.056+0000 7f2b5cca4640 1 --2- 172.21.15.78:0/3227997182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b5810e200 0x7f2b58110610 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009ca0 tx=0x7f2b5002a8c0 comp rx=0 tx=0).stop 2024-08-18T23:47:09.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.056+0000 7f2b5cca4640 1 -- 172.21.15.78:0/3227997182 shutdown_connections 2024-08-18T23:47:09.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.056+0000 7f2b5cca4640 1 --2- 172.21.15.78:0/3227997182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b5810e200 0x7f2b58110610 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:09.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.056+0000 7f2b5cca4640 1 --2- 172.21.15.78:0/3227997182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b58101090 0x7f2b5810dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:09.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.056+0000 7f2b5cca4640 1 --2- 172.21.15.78:0/3227997182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b581006e0 0x7f2b58100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:09.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.056+0000 7f2b5cca4640 1 -- 172.21.15.78:0/3227997182 >> 172.21.15.78:0/3227997182 conn(0x7f2b580fc500 msgr2=0x7f2b580fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:09.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.056+0000 7f2b5cca4640 1 -- 172.21.15.78:0/3227997182 shutdown_connections 2024-08-18T23:47:09.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.056+0000 7f2b5cca4640 1 -- 172.21.15.78:0/3227997182 wait complete. 2024-08-18T23:47:09.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.057+0000 7f2b5cca4640 1 Processor -- start 2024-08-18T23:47:09.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.057+0000 7f2b5cca4640 1 -- start start 2024-08-18T23:47:09.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.057+0000 7f2b5cca4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b581006e0 0x7f2b5819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:09.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.057+0000 7f2b5cca4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b58101090 0x7f2b5819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:09.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.057+0000 7f2b5cca4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b5810e200 0x7f2b5819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:09.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.057+0000 7f2b5cca4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b58113f70 con 0x7f2b5810e200 2024-08-18T23:47:09.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.057+0000 7f2b5cca4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b58113df0 con 0x7f2b581006e0 2024-08-18T23:47:09.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.057+0000 7f2b5cca4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2b581140f0 con 0x7f2b58101090 2024-08-18T23:47:09.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.058+0000 7f2b5659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b581006e0 0x7f2b5819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:09.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.058+0000 7f2b56d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b5810e200 0x7f2b5819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:09.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.058+0000 7f2b5659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b581006e0 0x7f2b5819a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:60752/0 (socket says 172.21.15.78:60752) 2024-08-18T23:47:09.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.058+0000 7f2b5659c640 1 -- 172.21.15.78:0/1037134417 learned_addr learned my addr 172.21.15.78:0/1037134417 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:09.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.058+0000 7f2b55d9b640 1 --2- 172.21.15.78:0/1037134417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b58101090 0x7f2b5819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:09.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.058+0000 7f2b5659c640 1 -- 172.21.15.78:0/1037134417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b58101090 msgr2=0x7f2b5819a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:09.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.058+0000 7f2b5659c640 1 --2- 172.21.15.78:0/1037134417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b58101090 0x7f2b5819a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:09.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.058+0000 7f2b5659c640 1 -- 172.21.15.78:0/1037134417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b5810e200 msgr2=0x7f2b5819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:09.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.058+0000 7f2b5659c640 1 --2- 172.21.15.78:0/1037134417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b5810e200 0x7f2b5819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:09.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.058+0000 7f2b5659c640 1 -- 172.21.15.78:0/1037134417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5819f3a0 con 0x7f2b581006e0 2024-08-18T23:47:09.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.058+0000 7f2b5659c640 1 --2- 172.21.15.78:0/1037134417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b581006e0 0x7f2b5819a3f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800d770 tx=0x7f2b4800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:09.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.058+0000 7f2b477fe640 1 -- 172.21.15.78:0/1037134417 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48017070 con 0x7f2b581006e0 2024-08-18T23:47:09.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.058+0000 7f2b477fe640 1 -- 172.21.15.78:0/1037134417 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b48005500 con 0x7f2b581006e0 2024-08-18T23:47:09.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.058+0000 7f2b477fe640 1 -- 172.21.15.78:0/1037134417 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48004e00 con 0x7f2b581006e0 2024-08-18T23:47:09.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.059+0000 7f2b5cca4640 1 -- 172.21.15.78:0/1037134417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b5819f690 con 0x7f2b581006e0 2024-08-18T23:47:09.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.059+0000 7f2b5cca4640 1 -- 172.21.15.78:0/1037134417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b581a6f70 con 0x7f2b581006e0 2024-08-18T23:47:09.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.060+0000 7f2b5cca4640 1 -- 172.21.15.78:0/1037134417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b58100ac0 con 0x7f2b581006e0 2024-08-18T23:47:09.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.061+0000 7f2b477fe640 1 -- 172.21.15.78:0/1037134417 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2b48004fa0 con 0x7f2b581006e0 2024-08-18T23:47:09.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.062+0000 7f2b477fe640 1 --2- 172.21.15.78:0/1037134417 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2b400a2b80 0x7f2b400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:09.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.062+0000 7f2b477fe640 1 -- 172.21.15.78:0/1037134417 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2b480cedb0 con 0x7f2b581006e0 2024-08-18T23:47:09.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.062+0000 7f2b55d9b640 1 --2- 172.21.15.78:0/1037134417 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2b400a2b80 0x7f2b400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:09.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.062+0000 7f2b55d9b640 1 --2- 172.21.15.78:0/1037134417 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2b400a2b80 0x7f2b400a5040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0095f0 tx=0x7f2b4c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:09.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.065+0000 7f2b477fe640 1 -- 172.21.15.78:0/1037134417 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2b4809b2d0 con 0x7f2b581006e0 2024-08-18T23:47:09.295 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:09 smithi078 ceph-mon[18440]: pgmap v435: 129 pgs: 129 active+clean; 385 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 285 op/s 2024-08-18T23:47:09.295 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:09 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2567104960' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:47:09.487 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:47:09.487 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":26,"btime":"2024-08-18T23:45:33:306257+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:45:32.491054+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:47:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.454+0000 7f2b5cca4640 1 -- 172.21.15.78:0/1037134417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b58109390 con 0x7f2b581006e0 2024-08-18T23:47:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.464+0000 7f2b477fe640 1 -- 172.21.15.78:0/1037134417 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11933 (secure 0 0 0) 0x7f2b4809f5d0 con 0x7f2b581006e0 2024-08-18T23:47:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 26 2024-08-18T23:47:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.471+0000 7f2b5cca4640 1 -- 172.21.15.78:0/1037134417 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2b400a2b80 msgr2=0x7f2b400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:09.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.471+0000 7f2b5cca4640 1 --2- 172.21.15.78:0/1037134417 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2b400a2b80 0x7f2b400a5040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0095f0 tx=0x7f2b4c005f90 comp rx=0 tx=0).stop 2024-08-18T23:47:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.471+0000 7f2b5cca4640 1 -- 172.21.15.78:0/1037134417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b581006e0 msgr2=0x7f2b5819a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.471+0000 7f2b5cca4640 1 --2- 172.21.15.78:0/1037134417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b581006e0 0x7f2b5819a3f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800d770 tx=0x7f2b4800dc30 comp rx=0 tx=0).stop 2024-08-18T23:47:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.472+0000 7f2b5cca4640 1 -- 172.21.15.78:0/1037134417 shutdown_connections 2024-08-18T23:47:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.472+0000 7f2b5cca4640 1 --2- 172.21.15.78:0/1037134417 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2b400a2b80 0x7f2b400a5040 secure :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0095f0 tx=0x7f2b4c005f90 comp rx=0 tx=0).stop 2024-08-18T23:47:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.472+0000 7f2b5cca4640 1 --2- 172.21.15.78:0/1037134417 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b5810e200 0x7f2b5819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.472+0000 7f2b5cca4640 1 --2- 172.21.15.78:0/1037134417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b58101090 0x7f2b5819a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.472+0000 7f2b5cca4640 1 --2- 172.21.15.78:0/1037134417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b581006e0 0x7f2b5819a3f0 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800d770 tx=0x7f2b4800dc30 comp rx=0 tx=0).stop 2024-08-18T23:47:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.472+0000 7f2b5cca4640 1 -- 172.21.15.78:0/1037134417 >> 172.21.15.78:0/1037134417 conn(0x7f2b580fc500 msgr2=0x7f2b580fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.478+0000 7f2b5cca4640 1 -- 172.21.15.78:0/1037134417 shutdown_connections 2024-08-18T23:47:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:09.483+0000 7f2b5cca4640 1 -- 172.21.15.78:0/1037134417 wait complete. 2024-08-18T23:47:09.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:09 smithi123 ceph-mon[21808]: pgmap v435: 129 pgs: 129 active+clean; 385 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 285 op/s 2024-08-18T23:47:09.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:09 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2567104960' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:47:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:09 smithi161 ceph-mon[22027]: pgmap v435: 129 pgs: 129 active+clean; 385 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 285 op/s 2024-08-18T23:47:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:09 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2567104960' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:47:10.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:10 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1037134417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:47:10.457 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-18T23:47:10.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:10 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1037134417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:47:10.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:10 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1037134417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:47:10.806 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-08-18T23:47:10.807 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-08-18T23:47:10.945 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:47:11.431 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:11 smithi078 ceph-mon[18440]: pgmap v436: 129 pgs: 129 active+clean; 403 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 346 op/s 2024-08-18T23:47:11.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.525+0000 7ffa321bd640 1 -- 172.21.15.78:0/1321090639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa2c1022a0 msgr2=0x7ffa2c1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:11.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.525+0000 7ffa321bd640 1 --2- 172.21.15.78:0/1321090639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa2c1022a0 0x7ffa2c1026e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ffa20009900 tx=0x7ffa2002a480 comp rx=0 tx=0).stop 2024-08-18T23:47:11.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.525+0000 7ffa321bd640 1 -- 172.21.15.78:0/1321090639 shutdown_connections 2024-08-18T23:47:11.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.525+0000 7ffa321bd640 1 --2- 172.21.15.78:0/1321090639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa2c102c20 0x7ffa2c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.525+0000 7ffa321bd640 1 --2- 172.21.15.78:0/1321090639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa2c1022a0 0x7ffa2c1026e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.525+0000 7ffa321bd640 1 --2- 172.21.15.78:0/1321090639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa2c100b50 0x7ffa2c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.525+0000 7ffa321bd640 1 -- 172.21.15.78:0/1321090639 >> 172.21.15.78:0/1321090639 conn(0x7ffa2c0fc500 msgr2=0x7ffa2c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.525+0000 7ffa321bd640 1 -- 172.21.15.78:0/1321090639 shutdown_connections 2024-08-18T23:47:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.525+0000 7ffa321bd640 1 -- 172.21.15.78:0/1321090639 wait complete. 2024-08-18T23:47:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.526+0000 7ffa321bd640 1 Processor -- start 2024-08-18T23:47:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.526+0000 7ffa321bd640 1 -- start start 2024-08-18T23:47:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.526+0000 7ffa321bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa2c100b50 0x7ffa2c198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.526+0000 7ffa321bd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa2c1022a0 0x7ffa2c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.526+0000 7ffa321bd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa2c102c20 0x7ffa2c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.526+0000 7ffa321bd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c110cb0 con 0x7ffa2c100b50 2024-08-18T23:47:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.526+0000 7ffa321bd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c110b30 con 0x7ffa2c102c20 2024-08-18T23:47:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.526+0000 7ffa321bd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c110e30 con 0x7ffa2c1022a0 2024-08-18T23:47:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.526+0000 7ffa2b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa2c100b50 0x7ffa2c198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.526+0000 7ffa2b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa2c100b50 0x7ffa2c198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42006/0 (socket says 172.21.15.78:42006) 2024-08-18T23:47:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.526+0000 7ffa2b7fe640 1 -- 172.21.15.78:0/900282463 learned_addr learned my addr 172.21.15.78:0/900282463 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.526+0000 7ffa2bfff640 1 --2- 172.21.15.78:0/900282463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa2c102c20 0x7ffa2c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.527+0000 7ffa2affd640 1 --2- 172.21.15.78:0/900282463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa2c1022a0 0x7ffa2c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.527+0000 7ffa2b7fe640 1 -- 172.21.15.78:0/900282463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa2c1022a0 msgr2=0x7ffa2c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.527+0000 7ffa2b7fe640 1 --2- 172.21.15.78:0/900282463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa2c1022a0 0x7ffa2c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.527+0000 7ffa2b7fe640 1 -- 172.21.15.78:0/900282463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa2c102c20 msgr2=0x7ffa2c19cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.527+0000 7ffa2b7fe640 1 --2- 172.21.15.78:0/900282463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa2c102c20 0x7ffa2c19cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.527+0000 7ffa2b7fe640 1 -- 172.21.15.78:0/900282463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2c19d1b0 con 0x7ffa2c100b50 2024-08-18T23:47:11.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.527+0000 7ffa2b7fe640 1 --2- 172.21.15.78:0/900282463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa2c100b50 0x7ffa2c198200 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00e9f0 tx=0x7ffa1c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:11.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.527+0000 7ffa28ff9640 1 -- 172.21.15.78:0/900282463 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c00cd80 con 0x7ffa2c100b50 2024-08-18T23:47:11.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.527+0000 7ffa321bd640 1 -- 172.21.15.78:0/900282463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2c19d4a0 con 0x7ffa2c100b50 2024-08-18T23:47:11.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.527+0000 7ffa28ff9640 1 -- 172.21.15.78:0/900282463 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1c005500 con 0x7ffa2c100b50 2024-08-18T23:47:11.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.527+0000 7ffa28ff9640 1 -- 172.21.15.78:0/900282463 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c010690 con 0x7ffa2c100b50 2024-08-18T23:47:11.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.527+0000 7ffa321bd640 1 -- 172.21.15.78:0/900282463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa2c1a4d80 con 0x7ffa2c100b50 2024-08-18T23:47:11.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.530+0000 7ffa28ff9640 1 -- 172.21.15.78:0/900282463 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ffa1c0046c0 con 0x7ffa2c100b50 2024-08-18T23:47:11.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.530+0000 7ffa345bb740 1 Processor -- start 2024-08-18T23:47:11.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.530+0000 7ffa345bb740 1 -- start start 2024-08-18T23:47:11.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.530+0000 7ffa28ff9640 1 --2- 172.21.15.78:0/900282463 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ffa140a2b80 0x7ffa140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:11.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.530+0000 7ffa345bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076825ff0 0x55b0768263d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:11.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.530+0000 7ffa345bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b076826910 0x55b076830c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:11.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.530+0000 7ffa345bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0768311d0 0x55b0768335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:11.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.530+0000 7ffa28ff9640 1 -- 172.21.15.78:0/900282463 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7ffa1c0cf810 con 0x7ffa2c100b50 2024-08-18T23:47:11.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.530+0000 7ffa345bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b07678ba40 con 0x55b076825ff0 2024-08-18T23:47:11.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.530+0000 7ffa345bb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b0767239d0 con 0x55b0768311d0 2024-08-18T23:47:11.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.530+0000 7ffa345bb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b076787a00 con 0x55b076826910 2024-08-18T23:47:11.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.530+0000 7ffa2bfff640 1 --2- 172.21.15.78:0/900282463 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ffa140a2b80 0x7ffa140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:11.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.531+0000 7ffa2b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076825ff0 0x55b0768263d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:11.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.531+0000 7ffa2b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076825ff0 0x55b0768263d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42008/0 (socket says 172.21.15.78:42008) 2024-08-18T23:47:11.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.531+0000 7ffa2b7fe640 1 -- 172.21.15.78:0/878246210 learned_addr learned my addr 172.21.15.78:0/878246210 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:11.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.531+0000 7ffa2affd640 1 --2- 172.21.15.78:0/878246210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b076826910 0x55b076830c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:11.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.531+0000 7ffa2bfff640 1 --2- 172.21.15.78:0/878246210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0768311d0 0x55b0768335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:11.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.531+0000 7ffa2b7fe640 1 -- 172.21.15.78:0/878246210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b076826910 msgr2=0x55b076830c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:11.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.531+0000 7ffa2b7fe640 1 --2- 172.21.15.78:0/878246210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b076826910 0x55b076830c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.531+0000 7ffa2b7fe640 1 -- 172.21.15.78:0/878246210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0768311d0 msgr2=0x55b0768335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:11.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.531+0000 7ffa2b7fe640 1 --2- 172.21.15.78:0/878246210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0768311d0 0x55b0768335c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.531+0000 7ffa2b7fe640 1 -- 172.21.15.78:0/878246210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b07672a650 con 0x55b076825ff0 2024-08-18T23:47:11.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.531+0000 7ffa2b7fe640 1 --2- 172.21.15.78:0/878246210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076825ff0 0x55b0768263d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c09bc10 tx=0x7ffa1c09b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b899eed422d787 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.531+0000 7ffa2bfff640 1 --2- 172.21.15.78:0/900282463 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ffa140a2b80 0x7ffa140a5040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009fd0 tx=0x7ffa24014bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.531+0000 7ffa11ffb640 1 -- 172.21.15.78:0/878246210 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c0a0030 con 0x55b076825ff0 2024-08-18T23:47:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.531+0000 7ffa11ffb640 1 -- 172.21.15.78:0/878246210 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1c0a48f0 con 0x55b076825ff0 2024-08-18T23:47:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.531+0000 7ffa11ffb640 1 -- 172.21.15.78:0/878246210 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c09b0d0 con 0x55b076825ff0 2024-08-18T23:47:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.532+0000 7ffa345bb740 1 -- 172.21.15.78:0/878246210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076825ff0 msgr2=0x55b0768263d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.532+0000 7ffa345bb740 1 --2- 172.21.15.78:0/878246210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076825ff0 0x55b0768263d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c09bc10 tx=0x7ffa1c09b5e0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.532+0000 7ffa345bb740 1 -- 172.21.15.78:0/878246210 shutdown_connections 2024-08-18T23:47:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.532+0000 7ffa345bb740 1 --2- 172.21.15.78:0/878246210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0768311d0 0x55b0768335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.532+0000 7ffa345bb740 1 --2- 172.21.15.78:0/878246210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b076826910 0x55b076830c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.532+0000 7ffa345bb740 1 --2- 172.21.15.78:0/878246210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076825ff0 0x55b0768263d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.532+0000 7ffa345bb740 1 -- 172.21.15.78:0/878246210 >> 172.21.15.78:0/878246210 conn(0x55b076822580 msgr2=0x55b076822960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.532+0000 7ffa345bb740 1 -- 172.21.15.78:0/878246210 shutdown_connections 2024-08-18T23:47:11.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.532+0000 7ffa345bb740 1 -- 172.21.15.78:0/878246210 wait complete. 2024-08-18T23:47:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.533+0000 7ffa345bb740 1 -- start start 2024-08-18T23:47:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.533+0000 7ffa345bb740 1 Processor -- start 2024-08-18T23:47:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.533+0000 7ffa345bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b076825ff0 0x55b07682e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.533+0000 7ffa345bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076826910 0x55b07682e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.533+0000 7ffa345bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0768311d0 0x55b07682ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.534+0000 7ffa345bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b0768381b0 con 0x55b076826910 2024-08-18T23:47:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.534+0000 7ffa345bb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b076746b20 con 0x55b076825ff0 2024-08-18T23:47:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.534+0000 7ffa345bb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b076838330 con 0x55b0768311d0 2024-08-18T23:47:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.534+0000 7ffa345bb740 10 client.0 _ll_register_callbacks cb 0x55b076744bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.534+0000 7ffa2affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076826910 0x55b07682e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.534+0000 7ffa2b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b076825ff0 0x55b07682e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.534+0000 7ffa2affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076826910 0x55b07682e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42014/0 (socket says 172.21.15.78:42014) 2024-08-18T23:47:11.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.534+0000 7ffa2affd640 1 -- 172.21.15.78:0/3641252245 learned_addr learned my addr 172.21.15.78:0/3641252245 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.534+0000 7ffa2bfff640 1 --2- 172.21.15.78:0/3641252245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0768311d0 0x55b07682ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.534+0000 7ffa2b7fe640 1 -- 172.21.15.78:0/3641252245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0768311d0 msgr2=0x55b07682ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.534+0000 7ffa2b7fe640 1 --2- 172.21.15.78:0/3641252245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0768311d0 0x55b07682ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.534+0000 7ffa2b7fe640 1 -- 172.21.15.78:0/3641252245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076826910 msgr2=0x55b07682e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.534+0000 7ffa2b7fe640 1 --2- 172.21.15.78:0/3641252245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076826910 0x55b07682e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.534+0000 7ffa2b7fe640 1 -- 172.21.15.78:0/3641252245 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0764fd3c0 con 0x55b076825ff0 2024-08-18T23:47:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.534+0000 7ffa2bfff640 1 --2- 172.21.15.78:0/3641252245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0768311d0 0x55b07682ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:47:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.534+0000 7ffa2affd640 1 --2- 172.21.15.78:0/3641252245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076826910 0x55b07682e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.534+0000 7ffa2b7fe640 1 --2- 172.21.15.78:0/3641252245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b076825ff0 0x55b07682e1a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c099f30 tx=0x7ffa1c09be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bfe20aaa22e5764 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.534+0000 7ffa0affd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:47:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.534+0000 7ffa0affd640 1 -- 172.21.15.78:0/3641252245 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c0ba050 con 0x55b076825ff0 2024-08-18T23:47:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.535+0000 7ffa345bb740 1 -- 172.21.15.78:0/3641252245 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa1c0a48f0 con 0x55b076825ff0 2024-08-18T23:47:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.535+0000 7ffa0affd640 1 -- 172.21.15.78:0/3641252245 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7ffa1c0a4b40 con 0x55b076825ff0 2024-08-18T23:47:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.535+0000 7ffa0affd640 1 -- 172.21.15.78:0/3641252245 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffa1c0a4d40 con 0x55b076825ff0 2024-08-18T23:47:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.535+0000 7ffa345bb740 10 client.25367 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.535+0000 7ffa345bb740 1 -- 172.21.15.78:0/3641252245 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b0768381b0 con 0x55b076825ff0 2024-08-18T23:47:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.535+0000 7ffa0affd640 1 -- 172.21.15.78:0/3641252245 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1c0a48f0 con 0x55b076825ff0 2024-08-18T23:47:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.535+0000 7ffa0affd640 1 -- 172.21.15.78:0/3641252245 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7ffa1c0b4030 con 0x55b076825ff0 2024-08-18T23:47:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.535+0000 7ffa345bb740 10 client.25367 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.535+0000 7ffa345bb740 10 client.25367 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:47:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.535+0000 7ffa345bb740 20 client.25367 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:11.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.535+0000 7ffa345bb740 1 --2- 172.21.15.78:0/3641252245 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55b076834c80 0x55b0768370d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.535+0000 7ffa345bb740 4 client.25367 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:47:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.535+0000 7ffa345bb740 1 -- 172.21.15.78:0/3641252245 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b076798fb0 con 0x55b076834c80 2024-08-18T23:47:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.536+0000 7ffa2affd640 1 --2- 172.21.15.78:0/3641252245 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55b076834c80 0x55b0768370d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.536+0000 7ffa0affd640 10 client.25367 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:47:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.536+0000 7ffa2affd640 1 --2- 172.21.15.78:0/3641252245 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55b076834c80 0x55b0768370d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6c080bb35879f4a server_cookie=5ea9d164c33e9de5 in_seq=0 out_seq=0 2024-08-18T23:47:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.537+0000 7ffa0affd640 1 -- 172.21.15.78:0/3641252245 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b076798fb0 con 0x55b076834c80 2024-08-18T23:47:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.537+0000 7ffa0affd640 10 client.25367 handle_command_reply: tid=0 2024-08-18T23:47:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.537+0000 7ffa345bb740 1 client.25367 shutdown 2024-08-18T23:47:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.537+0000 7ffa345bb740 1 -- 172.21.15.78:0/3641252245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b076825ff0 msgr2=0x55b07682e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.537+0000 7ffa345bb740 1 --2- 172.21.15.78:0/3641252245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b076825ff0 0x55b07682e1a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c099f30 tx=0x7ffa1c09be00 comp rx=0 tx=0).stop 2024-08-18T23:47:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.537+0000 7ffa345bb740 1 -- 172.21.15.78:0/3641252245 shutdown_connections 2024-08-18T23:47:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.537+0000 7ffa345bb740 1 --2- 172.21.15.78:0/3641252245 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55b076834c80 0x55b0768370d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.537+0000 7ffa345bb740 1 --2- 172.21.15.78:0/3641252245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b0768311d0 0x55b07682ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.537+0000 7ffa345bb740 1 --2- 172.21.15.78:0/3641252245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076826910 0x55b07682e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.537+0000 7ffa345bb740 1 --2- 172.21.15.78:0/3641252245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b076825ff0 0x55b07682e1a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.537+0000 7ffa345bb740 1 -- 172.21.15.78:0/3641252245 >> 172.21.15.78:0/3641252245 conn(0x55b076822580 msgr2=0x55b0768282b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:11.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.537+0000 7ffa0affd640 0 client.25367 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:47:11.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.537+0000 7ffa345bb740 1 -- 172.21.15.78:0/3641252245 shutdown_connections 2024-08-18T23:47:11.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.537+0000 7ffa345bb740 1 -- 172.21.15.78:0/3641252245 wait complete. 2024-08-18T23:47:11.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.537+0000 7ffa345bb740 20 client.25367 trim_cache size 0 max 16384 2024-08-18T23:47:11.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.579+0000 7ffa345bb740 1 Processor -- start 2024-08-18T23:47:11.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.582+0000 7ffa345bb740 1 -- start start 2024-08-18T23:47:11.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.582+0000 7ffa345bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076825ff0 0x55b076828a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:11.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.582+0000 7ffa345bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b076826910 0x55b07682d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:11.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.582+0000 7ffa345bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0768311d0 0x55b07682b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:11.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.582+0000 7ffa345bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1aefb0 con 0x55b076825ff0 2024-08-18T23:47:11.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.582+0000 7ffa345bb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1ac880 con 0x55b0768311d0 2024-08-18T23:47:11.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.582+0000 7ffa345bb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1aa6d0 con 0x55b076826910 2024-08-18T23:47:11.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.583+0000 7ffa2b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076825ff0 0x55b076828a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:11.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.583+0000 7ffa2b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076825ff0 0x55b076828a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42018/0 (socket says 172.21.15.78:42018) 2024-08-18T23:47:11.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.583+0000 7ffa2b7fe640 1 -- 172.21.15.78:0/3097816544 learned_addr learned my addr 172.21.15.78:0/3097816544 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:11.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.584+0000 7ffa2b7fe640 1 -- 172.21.15.78:0/3097816544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b076826910 msgr2=0x55b07682d1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-18T23:47:11.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.584+0000 7ffa2b7fe640 1 --2- 172.21.15.78:0/3097816544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b076826910 0x55b07682d1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.584+0000 7ffa2b7fe640 1 -- 172.21.15.78:0/3097816544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0768311d0 msgr2=0x55b07682b440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T23:47:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.584+0000 7ffa2b7fe640 1 --2- 172.21.15.78:0/3097816544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0768311d0 0x55b07682b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.584+0000 7ffa2b7fe640 1 -- 172.21.15.78:0/3097816544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b076823630 con 0x55b076825ff0 2024-08-18T23:47:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.584+0000 7ffa2b7fe640 1 --2- 172.21.15.78:0/3097816544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076825ff0 0x55b076828a50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0049d0 tx=0x7ffa1c09b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f5d736a436a5f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.585+0000 7ffa117fa640 1 -- 172.21.15.78:0/3097816544 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c09b3e0 con 0x55b076825ff0 2024-08-18T23:47:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.585+0000 7ffa117fa640 1 -- 172.21.15.78:0/3097816544 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1c0a9070 con 0x55b076825ff0 2024-08-18T23:47:11.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.585+0000 7ffa117fa640 1 -- 172.21.15.78:0/3097816544 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c0a48f0 con 0x55b076825ff0 2024-08-18T23:47:11.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.587+0000 7ffa345bb740 1 -- 172.21.15.78:0/3097816544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076825ff0 msgr2=0x55b076828a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:11.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.587+0000 7ffa345bb740 1 --2- 172.21.15.78:0/3097816544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076825ff0 0x55b076828a50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0049d0 tx=0x7ffa1c09b710 comp rx=0 tx=0).stop 2024-08-18T23:47:11.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.591+0000 7ffa345bb740 1 -- 172.21.15.78:0/3097816544 shutdown_connections 2024-08-18T23:47:11.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.591+0000 7ffa345bb740 1 --2- 172.21.15.78:0/3097816544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0768311d0 0x55b07682b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.591+0000 7ffa345bb740 1 --2- 172.21.15.78:0/3097816544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b076826910 0x55b07682d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.591+0000 7ffa345bb740 1 --2- 172.21.15.78:0/3097816544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b076825ff0 0x55b076828a50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.591+0000 7ffa345bb740 1 -- 172.21.15.78:0/3097816544 >> 172.21.15.78:0/3097816544 conn(0x55b076822580 msgr2=0x55b07682ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:11.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.597+0000 7ffa345bb740 1 -- 172.21.15.78:0/3097816544 shutdown_connections 2024-08-18T23:47:11.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.601+0000 7ffa345bb740 1 -- 172.21.15.78:0/3097816544 wait complete. 2024-08-18T23:47:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.601+0000 7ffa345bb740 1 -- start start 2024-08-18T23:47:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.601+0000 7ffa345bb740 1 Processor -- start 2024-08-18T23:47:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.602+0000 7ffa345bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b076825ff0 0x55b076849350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.602+0000 7ffa345bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b076826910 0x55b076845d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.602+0000 7ffa345bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0768311d0 0x55b0768462a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.602+0000 7ffa345bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1aad40 con 0x55b0768311d0 2024-08-18T23:47:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.602+0000 7ffa345bb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1ac1b0 con 0x55b076825ff0 2024-08-18T23:47:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.602+0000 7ffa345bb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1ab410 con 0x55b076826910 2024-08-18T23:47:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.602+0000 7ffa345bb740 10 client.0 _ll_register_callbacks cb 0x7ffa20009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.602+0000 7ffa2bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0768311d0 0x55b0768462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.602+0000 7ffa2affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b076826910 0x55b076845d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:11.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.602+0000 7ffa2bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0768311d0 0x55b0768462a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42024/0 (socket says 172.21.15.78:42024) 2024-08-18T23:47:11.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.602+0000 7ffa2bfff640 1 -- 172.21.15.78:0/1065462192 learned_addr learned my addr 172.21.15.78:0/1065462192 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:11.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.603+0000 7ffa2b7fe640 1 --2- 172.21.15.78:0/1065462192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b076825ff0 0x55b076849350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:11.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.603+0000 7ffa2affd640 1 -- 172.21.15.78:0/1065462192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b076825ff0 msgr2=0x55b076849350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:11.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.603+0000 7ffa2affd640 1 --2- 172.21.15.78:0/1065462192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b076825ff0 0x55b076849350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.603+0000 7ffa2affd640 1 -- 172.21.15.78:0/1065462192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0768311d0 msgr2=0x55b0768462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:11.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.603+0000 7ffa2affd640 1 --2- 172.21.15.78:0/1065462192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0768311d0 0x55b0768462a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.603+0000 7ffa2affd640 1 -- 172.21.15.78:0/1065462192 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0768467e0 con 0x55b076826910 2024-08-18T23:47:11.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.609+0000 7ffa2affd640 1 --2- 172.21.15.78:0/1065462192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b076826910 0x55b076845d60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa200314e0 tx=0x7ffa2002ac00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9225964b7ac8f92 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:11.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.609+0000 7ffa0affd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:47:11.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.613+0000 7ffa0affd640 1 -- 172.21.15.78:0/1065462192 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2004c060 con 0x55b076826910 2024-08-18T23:47:11.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.613+0000 7ffa345bb740 1 -- 172.21.15.78:0/1065462192 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa1c0a9070 con 0x55b076826910 2024-08-18T23:47:11.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.613+0000 7ffa0affd640 1 -- 172.21.15.78:0/1065462192 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7ffa20046680 con 0x55b076826910 2024-08-18T23:47:11.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.613+0000 7ffa0affd640 1 -- 172.21.15.78:0/1065462192 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffa20053050 con 0x55b076826910 2024-08-18T23:47:11.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.614+0000 7ffa345bb740 10 client.25255 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:11.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.614+0000 7ffa345bb740 1 -- 172.21.15.78:0/1065462192 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b0768aac00 con 0x55b076826910 2024-08-18T23:47:11.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.615+0000 7ffa0affd640 1 -- 172.21.15.78:0/1065462192 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa200035f0 con 0x55b076826910 2024-08-18T23:47:11.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.615+0000 7ffa0affd640 1 -- 172.21.15.78:0/1065462192 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7ffa20031540 con 0x55b076826910 2024-08-18T23:47:11.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.615+0000 7ffa345bb740 10 client.25255 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:11.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.615+0000 7ffa345bb740 10 client.25255 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:47:11.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.616+0000 7ffa345bb740 20 client.25255 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:11.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.616+0000 7ffa345bb740 1 --2- 172.21.15.78:0/1065462192 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55b0768ab360 0x55b0768ad780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:11.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.616+0000 7ffa345bb740 4 client.25255 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:47:11.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.616+0000 7ffa345bb740 1 -- 172.21.15.78:0/1065462192 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ffa1c0a9070 con 0x55b0768ab360 2024-08-18T23:47:11.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.619+0000 7ffa2b7fe640 1 --2- 172.21.15.78:0/1065462192 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55b0768ab360 0x55b0768ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:11.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.635+0000 7ffa2b7fe640 1 --2- 172.21.15.78:0/1065462192 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55b0768ab360 0x55b0768ad780 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c36d4a436aa03a5 server_cookie=aecbe7f55ea66df4 in_seq=0 out_seq=0 2024-08-18T23:47:11.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.635+0000 7ffa0affd640 10 client.25255 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:47:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.638+0000 7ffa0affd640 1 -- 172.21.15.78:0/1065462192 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ffa1c0a9070 con 0x55b0768ab360 2024-08-18T23:47:11.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.638+0000 7ffa0affd640 10 client.25255 handle_command_reply: tid=0 2024-08-18T23:47:11.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.639+0000 7ffa345bb740 1 client.25255 shutdown 2024-08-18T23:47:11.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.646+0000 7ffa345bb740 1 -- 172.21.15.78:0/1065462192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b076826910 msgr2=0x55b076845d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:11.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.646+0000 7ffa345bb740 1 --2- 172.21.15.78:0/1065462192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b076826910 0x55b076845d60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa200314e0 tx=0x7ffa2002ac00 comp rx=0 tx=0).stop 2024-08-18T23:47:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.651+0000 7ffa345bb740 1 -- 172.21.15.78:0/1065462192 shutdown_connections 2024-08-18T23:47:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.651+0000 7ffa345bb740 1 --2- 172.21.15.78:0/1065462192 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55b0768ab360 0x55b0768ad780 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.651+0000 7ffa345bb740 1 --2- 172.21.15.78:0/1065462192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b0768311d0 0x55b0768462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.651+0000 7ffa0affd640 0 client.25255 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:47:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.651+0000 7ffa345bb740 1 --2- 172.21.15.78:0/1065462192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b076826910 0x55b076845d60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.651+0000 7ffa345bb740 1 --2- 172.21.15.78:0/1065462192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b076825ff0 0x55b076849350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.651+0000 7ffa345bb740 1 -- 172.21.15.78:0/1065462192 >> 172.21.15.78:0/1065462192 conn(0x55b076822580 msgr2=0x55b07682d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:11.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.657+0000 7ffa345bb740 1 -- 172.21.15.78:0/1065462192 shutdown_connections 2024-08-18T23:47:11.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.657+0000 7ffa345bb740 1 -- 172.21.15.78:0/1065462192 wait complete. 2024-08-18T23:47:11.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.657+0000 7ffa345bb740 20 client.25255 trim_cache size 0 max 16384 2024-08-18T23:47:11.658 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:47:11.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.660+0000 7ffa321bd640 1 -- 172.21.15.78:0/900282463 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ffa140a2b80 msgr2=0x7ffa140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:11.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.660+0000 7ffa321bd640 1 --2- 172.21.15.78:0/900282463 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ffa140a2b80 0x7ffa140a5040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009fd0 tx=0x7ffa24014bf0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.660+0000 7ffa321bd640 1 -- 172.21.15.78:0/900282463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa2c100b50 msgr2=0x7ffa2c198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:11.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.660+0000 7ffa321bd640 1 --2- 172.21.15.78:0/900282463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa2c100b50 0x7ffa2c198200 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00e9f0 tx=0x7ffa1c00edf0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.662+0000 7ffa321bd640 1 -- 172.21.15.78:0/900282463 shutdown_connections 2024-08-18T23:47:11.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.662+0000 7ffa321bd640 1 --2- 172.21.15.78:0/900282463 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ffa140a2b80 0x7ffa140a5040 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.662+0000 7ffa321bd640 1 --2- 172.21.15.78:0/900282463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa2c102c20 0x7ffa2c19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.662+0000 7ffa321bd640 1 --2- 172.21.15.78:0/900282463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa2c1022a0 0x7ffa2c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.662+0000 7ffa321bd640 1 --2- 172.21.15.78:0/900282463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa2c100b50 0x7ffa2c198200 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:11.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.662+0000 7ffa321bd640 1 -- 172.21.15.78:0/900282463 >> 172.21.15.78:0/900282463 conn(0x7ffa2c0fc500 msgr2=0x7ffa2c069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:11.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.663+0000 7ffa321bd640 1 -- 172.21.15.78:0/900282463 shutdown_connections 2024-08-18T23:47:11.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:11.667+0000 7ffa321bd640 1 -- 172.21.15.78:0/900282463 wait complete. 2024-08-18T23:47:11.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:11 smithi161 ceph-mon[22027]: pgmap v436: 129 pgs: 129 active+clean; 403 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 346 op/s 2024-08-18T23:47:11.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:11 smithi123 ceph-mon[21808]: pgmap v436: 129 pgs: 129 active+clean; 403 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 346 op/s 2024-08-18T23:47:12.634 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-18T23:47:12.669 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:47:13.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:13 smithi078 ceph-mon[18440]: pgmap v437: 129 pgs: 129 active+clean; 405 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 355 op/s 2024-08-18T23:47:13.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.598+0000 7f630c827640 1 -- 172.21.15.78:0/1029585369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f630811ae70 msgr2=0x7f630811d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:13.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.598+0000 7f630c827640 1 --2- 172.21.15.78:0/1029585369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f630811ae70 0x7f630811d260 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009900 tx=0x7f62fc02a440 comp rx=0 tx=0).stop 2024-08-18T23:47:13.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.601+0000 7f630c827640 1 -- 172.21.15.78:0/1029585369 shutdown_connections 2024-08-18T23:47:13.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.601+0000 7f630c827640 1 --2- 172.21.15.78:0/1029585369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f630811ae70 0x7f630811d260 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.601+0000 7f630c827640 1 --2- 172.21.15.78:0/1029585369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6308109560 0x7f63081099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.601+0000 7f630c827640 1 --2- 172.21.15.78:0/1029585369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6308109180 0x7f630810ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.601+0000 7f630c827640 1 -- 172.21.15.78:0/1029585369 >> 172.21.15.78:0/1029585369 conn(0x7f630806de70 msgr2=0x7f630806e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:13.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.605+0000 7f630c827640 1 -- 172.21.15.78:0/1029585369 shutdown_connections 2024-08-18T23:47:13.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.609+0000 7f630c827640 1 -- 172.21.15.78:0/1029585369 wait complete. 2024-08-18T23:47:13.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.610+0000 7f630c827640 1 Processor -- start 2024-08-18T23:47:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.613+0000 7f630c827640 1 -- start start 2024-08-18T23:47:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.613+0000 7f630c827640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6308109180 0x7f630811a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.613+0000 7f630c827640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6308109560 0x7f630811a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.613+0000 7f630c827640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f630811ae70 0x7f63081134e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.613+0000 7f630c827640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6308120b10 con 0x7f6308109560 2024-08-18T23:47:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.613+0000 7f630c827640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6308120990 con 0x7f630811ae70 2024-08-18T23:47:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.613+0000 7f630c827640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6308120c90 con 0x7f6308109180 2024-08-18T23:47:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.614+0000 7f6305d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6308109560 0x7f630811a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.614+0000 7f6305d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6308109560 0x7f630811a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42042/0 (socket says 172.21.15.78:42042) 2024-08-18T23:47:13.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.614+0000 7f6305d9b640 1 -- 172.21.15.78:0/4207621691 learned_addr learned my addr 172.21.15.78:0/4207621691 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:13.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.615+0000 7f6305d9b640 1 -- 172.21.15.78:0/4207621691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6308109180 msgr2=0x7f630811a330 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-18T23:47:13.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.615+0000 7f6305d9b640 1 --2- 172.21.15.78:0/4207621691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6308109180 0x7f630811a330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.615+0000 7f6305d9b640 1 -- 172.21.15.78:0/4207621691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f630811ae70 msgr2=0x7f63081134e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-18T23:47:13.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.615+0000 7f6305d9b640 1 --2- 172.21.15.78:0/4207621691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f630811ae70 0x7f63081134e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.615+0000 7f6305d9b640 1 -- 172.21.15.78:0/4207621691 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6308113da0 con 0x7f6308109560 2024-08-18T23:47:13.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.616+0000 7f6305d9b640 1 --2- 172.21.15.78:0/4207621691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6308109560 0x7f630811a870 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f630000ba50 tx=0x7f630000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:13.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.618+0000 7f62f77fe640 1 -- 172.21.15.78:0/4207621691 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300019070 con 0x7f6308109560 2024-08-18T23:47:13.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.618+0000 7f62f77fe640 1 -- 172.21.15.78:0/4207621691 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f630000cc10 con 0x7f6308109560 2024-08-18T23:47:13.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.619+0000 7f62f77fe640 1 -- 172.21.15.78:0/4207621691 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300007660 con 0x7f6308109560 2024-08-18T23:47:13.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.619+0000 7f630c827640 1 -- 172.21.15.78:0/4207621691 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6308114090 con 0x7f6308109560 2024-08-18T23:47:13.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.619+0000 7f630c827640 1 -- 172.21.15.78:0/4207621691 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6308118030 con 0x7f6308109560 2024-08-18T23:47:13.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.625+0000 7f630ec25740 1 Processor -- start 2024-08-18T23:47:13.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.625+0000 7f630ec25740 1 -- start start 2024-08-18T23:47:13.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.625+0000 7f630ec25740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9bed130 0x5559a9bed510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:13.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.625+0000 7f630ec25740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9beda50 0x5559a9bf7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:13.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.625+0000 7f630ec25740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9bf8310 0x5559a9bfa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:13.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.625+0000 7f630ec25740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5559a9b52a70 con 0x5559a9bed130 2024-08-18T23:47:13.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.625+0000 7f630ec25740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5559a9aea780 con 0x5559a9bf8310 2024-08-18T23:47:13.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.625+0000 7f630ec25740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5559a9b4ea30 con 0x5559a9beda50 2024-08-18T23:47:13.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.627+0000 7f630659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9beda50 0x5559a9bf7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:13.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.628+0000 7f630659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9beda50 0x5559a9bf7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:57540/0 (socket says 172.21.15.78:57540) 2024-08-18T23:47:13.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.628+0000 7f630659c640 1 -- 172.21.15.78:0/4117306972 learned_addr learned my addr 172.21.15.78:0/4117306972 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:13.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.629+0000 7f62f77fe640 1 -- 172.21.15.78:0/4207621691 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f63000149b0 con 0x7f6308109560 2024-08-18T23:47:13.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.630+0000 7f6306d9d640 1 --2- 172.21.15.78:0/4117306972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9bed130 0x5559a9bed510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:13.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.633+0000 7f62f77fe640 1 --2- 172.21.15.78:0/4207621691 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f62f00a2c00 0x7f62f00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:13.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.637+0000 7f6305d9b640 1 --2- 172.21.15.78:0/4117306972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9bf8310 0x5559a9bfa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:13.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.639+0000 7f62f77fe640 1 -- 172.21.15.78:0/4207621691 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f63000ce9f0 con 0x7f6308109560 2024-08-18T23:47:13.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.639+0000 7f6306d9d640 1 --2- 172.21.15.78:0/4207621691 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f62f00a2c00 0x7f62f00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:13.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.639+0000 7f6306d9d640 1 -- 172.21.15.78:0/4117306972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9beda50 msgr2=0x5559a9bf7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:13.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.639+0000 7f6306d9d640 1 --2- 172.21.15.78:0/4117306972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9beda50 0x5559a9bf7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.639+0000 7f6306d9d640 1 -- 172.21.15.78:0/4117306972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9bf8310 msgr2=0x5559a9bfa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:13.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.640+0000 7f6306d9d640 1 --2- 172.21.15.78:0/4117306972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9bf8310 0x5559a9bfa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.640+0000 7f6306d9d640 1 -- 172.21.15.78:0/4117306972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559a98c4b90 con 0x5559a9bed130 2024-08-18T23:47:13.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.640+0000 7f6305d9b640 1 --2- 172.21.15.78:0/4117306972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9bf8310 0x5559a9bfa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:47:13.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.640+0000 7f630659c640 1 --2- 172.21.15.78:0/4117306972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9beda50 0x5559a9bf7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:47:13.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.643+0000 7f6306d9d640 1 --2- 172.21.15.78:0/4117306972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9bed130 0x5559a9bed510 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f62fc02a9b0 tx=0x7f62fc004190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5826a66b442a7f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:13.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.643+0000 7f6306d9d640 1 --2- 172.21.15.78:0/4207621691 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f62f00a2c00 0x7f62f00a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f62fc043000 tx=0x7f62fc046000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:13.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.646+0000 7f62f4ff9640 1 -- 172.21.15.78:0/4117306972 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc035040 con 0x5559a9bed130 2024-08-18T23:47:13.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.646+0000 7f62f4ff9640 1 -- 172.21.15.78:0/4117306972 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62fc0035f0 con 0x5559a9bed130 2024-08-18T23:47:13.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.646+0000 7f62f4ff9640 1 -- 172.21.15.78:0/4117306972 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc0496c0 con 0x5559a9bed130 2024-08-18T23:47:13.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.649+0000 7f630ec25740 1 -- 172.21.15.78:0/4117306972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9bed130 msgr2=0x5559a9bed510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:13.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.649+0000 7f630ec25740 1 --2- 172.21.15.78:0/4117306972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9bed130 0x5559a9bed510 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f62fc02a9b0 tx=0x7f62fc004190 comp rx=0 tx=0).stop 2024-08-18T23:47:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.657+0000 7f630ec25740 1 -- 172.21.15.78:0/4117306972 shutdown_connections 2024-08-18T23:47:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.657+0000 7f630ec25740 1 --2- 172.21.15.78:0/4117306972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9bf8310 0x5559a9bfa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.657+0000 7f630ec25740 1 --2- 172.21.15.78:0/4117306972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9beda50 0x5559a9bf7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.657+0000 7f630ec25740 1 --2- 172.21.15.78:0/4117306972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9bed130 0x5559a9bed510 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.657+0000 7f630ec25740 1 -- 172.21.15.78:0/4117306972 >> 172.21.15.78:0/4117306972 conn(0x5559a9be96c0 msgr2=0x5559a9be9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.661+0000 7f630ec25740 1 -- 172.21.15.78:0/4117306972 shutdown_connections 2024-08-18T23:47:13.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.661+0000 7f630ec25740 1 -- 172.21.15.78:0/4117306972 wait complete. 2024-08-18T23:47:13.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.661+0000 7f630ec25740 1 -- start start 2024-08-18T23:47:13.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.662+0000 7f630ec25740 1 Processor -- start 2024-08-18T23:47:13.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.663+0000 7f630ec25740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9bed130 0x5559a9bf5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:13.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.663+0000 7f630ec25740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9beda50 0x5559a9bf5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:13.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.663+0000 7f630ec25740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9bf8310 0x5559a9bf5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:13.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.663+0000 7f630ec25740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5559a9bfaee0 con 0x5559a9bed130 2024-08-18T23:47:13.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.663+0000 7f630ec25740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5559a9b0d990 con 0x5559a9bf8310 2024-08-18T23:47:13.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.663+0000 7f630ec25740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5559a9bfb060 con 0x5559a9beda50 2024-08-18T23:47:13.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.663+0000 7f630ec25740 10 client.0 _ll_register_callbacks cb 0x5559a9862090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:13.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.664+0000 7f630659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9bed130 0x5559a9bf5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:13.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.664+0000 7f630659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9bed130 0x5559a9bf5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42066/0 (socket says 172.21.15.78:42066) 2024-08-18T23:47:13.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.664+0000 7f630659c640 1 -- 172.21.15.78:0/1808735512 learned_addr learned my addr 172.21.15.78:0/1808735512 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:13.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.665+0000 7f630659c640 1 -- 172.21.15.78:0/1808735512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9beda50 msgr2=0x5559a9bf5860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-18T23:47:13.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.665+0000 7f630659c640 1 --2- 172.21.15.78:0/1808735512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9beda50 0x5559a9bf5860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.665+0000 7f630659c640 1 -- 172.21.15.78:0/1808735512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9bf8310 msgr2=0x5559a9bf5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:13.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.665+0000 7f630659c640 1 --2- 172.21.15.78:0/1808735512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9bf8310 0x5559a9bf5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.665+0000 7f630659c640 1 -- 172.21.15.78:0/1808735512 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559a9b0c6c0 con 0x5559a9bed130 2024-08-18T23:47:13.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.666+0000 7f630659c640 1 --2- 172.21.15.78:0/1808735512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9bed130 0x5559a9bf5320 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f62f800e650 tx=0x7f62f800eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec4e11b641cf56 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:13.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.667+0000 7f62edffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:47:13.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.667+0000 7f62edffb640 1 -- 172.21.15.78:0/1808735512 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8004220 con 0x5559a9bed130 2024-08-18T23:47:13.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.667+0000 7f62edffb640 1 -- 172.21.15.78:0/1808735512 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f62f80043c0 con 0x5559a9bed130 2024-08-18T23:47:13.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.670+0000 7f630ec25740 1 -- 172.21.15.78:0/1808735512 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62fc0035f0 con 0x5559a9bed130 2024-08-18T23:47:13.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.671+0000 7f62edffb640 1 -- 172.21.15.78:0/1808735512 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f62f800ee50 con 0x5559a9bed130 2024-08-18T23:47:13.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.673+0000 7f630ec25740 10 client.15918 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:13.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.673+0000 7f630ec25740 1 -- 172.21.15.78:0/1808735512 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5559a9c06c80 con 0x5559a9bed130 2024-08-18T23:47:13.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.674+0000 7f62edffb640 1 -- 172.21.15.78:0/1808735512 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62fc0035f0 con 0x5559a9bed130 2024-08-18T23:47:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:13 smithi161 ceph-mon[22027]: pgmap v437: 129 pgs: 129 active+clean; 405 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 355 op/s 2024-08-18T23:47:13.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.675+0000 7f62edffb640 1 -- 172.21.15.78:0/1808735512 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f62f8003bf0 con 0x5559a9bed130 2024-08-18T23:47:13.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.675+0000 7f630ec25740 10 client.15918 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:13.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.675+0000 7f630ec25740 10 client.15918 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:47:13.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.675+0000 7f630ec25740 20 client.15918 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:13.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.675+0000 7f630ec25740 1 --2- 172.21.15.78:0/1808735512 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5559a9c085b0 0x5559a9c0aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:13.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.675+0000 7f630ec25740 4 client.15918 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:47:13.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.675+0000 7f630ec25740 1 -- 172.21.15.78:0/1808735512 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559a9b601a0 con 0x5559a9c085b0 2024-08-18T23:47:13.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.677+0000 7f6305d9b640 1 --2- 172.21.15.78:0/1808735512 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5559a9c085b0 0x5559a9c0aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:13.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.685+0000 7f6305d9b640 1 --2- 172.21.15.78:0/1808735512 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5559a9c085b0 0x5559a9c0aa70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b787e30f969e7c5 server_cookie=f82784fbf47c70a in_seq=0 out_seq=0 2024-08-18T23:47:13.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.685+0000 7f62edffb640 10 client.15918 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:13.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.689+0000 7f62edffb640 1 -- 172.21.15.78:0/1808735512 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559a9b601a0 con 0x5559a9c085b0 2024-08-18T23:47:13.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.689+0000 7f62edffb640 10 client.15918 handle_command_reply: tid=0 2024-08-18T23:47:13.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.689+0000 7f630ec25740 1 client.15918 shutdown 2024-08-18T23:47:13.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.698+0000 7f630ec25740 1 -- 172.21.15.78:0/1808735512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9bed130 msgr2=0x5559a9bf5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:13.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.698+0000 7f630ec25740 1 --2- 172.21.15.78:0/1808735512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9bed130 0x5559a9bf5320 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f62f800e650 tx=0x7f62f800eb10 comp rx=0 tx=0).stop 2024-08-18T23:47:13.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.702+0000 7f630ec25740 1 -- 172.21.15.78:0/1808735512 shutdown_connections 2024-08-18T23:47:13.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.702+0000 7f630ec25740 1 --2- 172.21.15.78:0/1808735512 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5559a9c085b0 0x5559a9c0aa70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.702+0000 7f630ec25740 1 --2- 172.21.15.78:0/1808735512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9bf8310 0x5559a9bf5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.702+0000 7f630ec25740 1 --2- 172.21.15.78:0/1808735512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9beda50 0x5559a9bf5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.702+0000 7f630ec25740 1 --2- 172.21.15.78:0/1808735512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9bed130 0x5559a9bf5320 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.702+0000 7f630ec25740 1 -- 172.21.15.78:0/1808735512 >> 172.21.15.78:0/1808735512 conn(0x5559a9be96c0 msgr2=0x5559a9bef3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:13.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.702+0000 7f62edffb640 0 client.15918 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:13.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.708+0000 7f630ec25740 1 -- 172.21.15.78:0/1808735512 shutdown_connections 2024-08-18T23:47:13.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.712+0000 7f630ec25740 1 -- 172.21.15.78:0/1808735512 wait complete. 2024-08-18T23:47:13.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.712+0000 7f630ec25740 20 client.15918 trim_cache size 0 max 16384 2024-08-18T23:47:13.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.746+0000 7f630ec25740 1 Processor -- start 2024-08-18T23:47:13.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.746+0000 7f630ec25740 1 -- start start 2024-08-18T23:47:13.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.747+0000 7f630ec25740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9bed130 0x5559a9bf7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:13.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.747+0000 7f630ec25740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9beda50 0x5559a9c10380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:13.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.747+0000 7f6305d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9beda50 0x5559a9c10380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:13.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.747+0000 7f630659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9bed130 0x5559a9bf7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:13.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.747+0000 7f630659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9bed130 0x5559a9bf7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:60844/0 (socket says 172.21.15.78:60844) 2024-08-18T23:47:13.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.747+0000 7f6305d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9beda50 0x5559a9c10380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42074/0 (socket says 172.21.15.78:42074) 2024-08-18T23:47:13.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.748+0000 7f630ec25740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9bf8310 0x5559a9c108c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:13.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.748+0000 7f630ec25740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63081c8650 con 0x5559a9beda50 2024-08-18T23:47:13.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.748+0000 7f630ec25740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f63081c5f20 con 0x5559a9bed130 2024-08-18T23:47:13.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.748+0000 7f630ec25740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f63081c3d70 con 0x5559a9bf8310 2024-08-18T23:47:13.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.748+0000 7f6305d9b640 1 -- 172.21.15.78:0/2240990288 learned_addr learned my addr 172.21.15.78:0/2240990288 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:13.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.749+0000 7f6305d9b640 1 -- 172.21.15.78:0/2240990288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9bf8310 msgr2=0x5559a9c108c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:13.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.749+0000 7f6306d9d640 1 --2- 172.21.15.78:0/2240990288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9bf8310 0x5559a9c108c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:13.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.760+0000 7f6305d9b640 1 --2- 172.21.15.78:0/2240990288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9bf8310 0x5559a9c108c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.760+0000 7f6305d9b640 1 -- 172.21.15.78:0/2240990288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9bed130 msgr2=0x5559a9bf7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:13.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.760+0000 7f6305d9b640 1 --2- 172.21.15.78:0/2240990288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9bed130 0x5559a9bf7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.760+0000 7f6305d9b640 1 -- 172.21.15.78:0/2240990288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559a9bf42f0 con 0x5559a9beda50 2024-08-18T23:47:13.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.760+0000 7f6306d9d640 1 --2- 172.21.15.78:0/2240990288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9bf8310 0x5559a9c108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:47:13.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.761+0000 7f630659c640 1 --2- 172.21.15.78:0/2240990288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9bed130 0x5559a9bf7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:13.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.762+0000 7f6305d9b640 1 --2- 172.21.15.78:0/2240990288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9beda50 0x5559a9c10380 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6300014df0 tx=0x7f63000cec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b5842b071da435 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:13.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.763+0000 7f62effff640 1 -- 172.21.15.78:0/2240990288 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300098f00 con 0x5559a9beda50 2024-08-18T23:47:13.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.763+0000 7f62effff640 1 -- 172.21.15.78:0/2240990288 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6300099160 con 0x5559a9beda50 2024-08-18T23:47:13.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.763+0000 7f62effff640 1 -- 172.21.15.78:0/2240990288 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630009a2b0 con 0x5559a9beda50 2024-08-18T23:47:13.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.765+0000 7f630ec25740 1 -- 172.21.15.78:0/2240990288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9beda50 msgr2=0x5559a9c10380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:13.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.765+0000 7f630ec25740 1 --2- 172.21.15.78:0/2240990288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9beda50 0x5559a9c10380 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6300014df0 tx=0x7f63000cec40 comp rx=0 tx=0).stop 2024-08-18T23:47:13.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.765+0000 7f630ec25740 1 -- 172.21.15.78:0/2240990288 shutdown_connections 2024-08-18T23:47:13.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.765+0000 7f630ec25740 1 --2- 172.21.15.78:0/2240990288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9bf8310 0x5559a9c108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.765+0000 7f630ec25740 1 --2- 172.21.15.78:0/2240990288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9beda50 0x5559a9c10380 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.765+0000 7f630ec25740 1 --2- 172.21.15.78:0/2240990288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9bed130 0x5559a9bf7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.765+0000 7f630ec25740 1 -- 172.21.15.78:0/2240990288 >> 172.21.15.78:0/2240990288 conn(0x5559a9be96c0 msgr2=0x5559a9bf6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:13.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.768+0000 7f630ec25740 1 -- 172.21.15.78:0/2240990288 shutdown_connections 2024-08-18T23:47:13.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.769+0000 7f630ec25740 1 -- 172.21.15.78:0/2240990288 wait complete. 2024-08-18T23:47:13.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.769+0000 7f630ec25740 1 -- start start 2024-08-18T23:47:13.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.770+0000 7f630ec25740 1 Processor -- start 2024-08-18T23:47:13.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.770+0000 7f630ec25740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9bed130 0x5559a9c00bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:13.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.771+0000 7f630ec25740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9beda50 0x5559a9bf2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:13.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.771+0000 7f630659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9bed130 0x5559a9c00bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:13.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.771+0000 7f630659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9bed130 0x5559a9c00bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:57578/0 (socket says 172.21.15.78:57578) 2024-08-18T23:47:13.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.771+0000 7f630659c640 1 -- 172.21.15.78:0/715894465 learned_addr learned my addr 172.21.15.78:0/715894465 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:13.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.771+0000 7f6305d9b640 1 --2- 172.21.15.78:0/715894465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9beda50 0x5559a9bf2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:13.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.771+0000 7f630ec25740 1 --2- 172.21.15.78:0/715894465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9bf8310 0x5559a9bf29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:13.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.772+0000 7f6306d9d640 1 --2- 172.21.15.78:0/715894465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9bf8310 0x5559a9bf29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:13.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.772+0000 7f630ec25740 1 -- 172.21.15.78:0/715894465 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63081c43e0 con 0x5559a9bf8310 2024-08-18T23:47:13.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.772+0000 7f630ec25740 1 -- 172.21.15.78:0/715894465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f63081c5850 con 0x5559a9beda50 2024-08-18T23:47:13.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.772+0000 7f630ec25740 1 -- 172.21.15.78:0/715894465 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f63081c4ab0 con 0x5559a9bed130 2024-08-18T23:47:13.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.777+0000 7f6305d9b640 1 -- 172.21.15.78:0/715894465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9bed130 msgr2=0x5559a9c00bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:13.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.777+0000 7f6305d9b640 1 --2- 172.21.15.78:0/715894465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9bed130 0x5559a9c00bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.777+0000 7f6305d9b640 1 -- 172.21.15.78:0/715894465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9bf8310 msgr2=0x5559a9bf29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:13.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.777+0000 7f6305d9b640 1 --2- 172.21.15.78:0/715894465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9bf8310 0x5559a9bf29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:13.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.777+0000 7f6305d9b640 1 -- 172.21.15.78:0/715894465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559a9c72350 con 0x5559a9beda50 2024-08-18T23:47:13.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.777+0000 7f6306d9d640 1 --2- 172.21.15.78:0/715894465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9bf8310 0x5559a9bf29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:13.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.778+0000 7f6305d9b640 1 --2- 172.21.15.78:0/715894465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9beda50 0x5559a9bf2230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x5559a9bf0010 tx=0x7f63000cc0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17cc921a0fe5e394 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:13.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.778+0000 7f630ec25740 10 client.0 _ll_register_callbacks cb 0x7f62f8005f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:13.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.778+0000 7f630ec25740 1 -- 172.21.15.78:0/715894465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6300099160 con 0x5559a9beda50 2024-08-18T23:47:13.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.779+0000 7f62edffb640 10 client.25373 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:47:13.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.779+0000 7f62edffb640 1 -- 172.21.15.78:0/715894465 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63000b9060 con 0x5559a9beda50 2024-08-18T23:47:13.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.779+0000 7f62edffb640 1 -- 172.21.15.78:0/715894465 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f63000a3640 con 0x5559a9beda50 2024-08-18T23:47:13.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.779+0000 7f62edffb640 1 -- 172.21.15.78:0/715894465 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f630009a2b0 con 0x5559a9beda50 2024-08-18T23:47:13.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.779+0000 7f630ec25740 10 client.25373 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:13.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.779+0000 7f630ec25740 1 -- 172.21.15.78:0/715894465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5559a9c731e0 con 0x5559a9beda50 2024-08-18T23:47:13.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.782+0000 7f62edffb640 1 -- 172.21.15.78:0/715894465 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f630009a450 con 0x5559a9beda50 2024-08-18T23:47:13.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.782+0000 7f630ec25740 10 client.25373 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:13.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.782+0000 7f630ec25740 10 client.25373 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:47:13.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.782+0000 7f630ec25740 20 client.25373 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:13.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.782+0000 7f630ec25740 1 --2- 172.21.15.78:0/715894465 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5559a9c73900 0x5559a9c75dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:13.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.782+0000 7f630ec25740 4 client.25373 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-08-18T23:47:13.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.782+0000 7f630ec25740 1 -- 172.21.15.78:0/715894465 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7f6300099160 con 0x5559a9c73900 2024-08-18T23:47:13.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.782+0000 7f630659c640 1 --2- 172.21.15.78:0/715894465 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5559a9c73900 0x5559a9c75dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:13.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.783+0000 7f62edffb640 10 client.25373 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:13.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:13.783+0000 7f630659c640 1 --2- 172.21.15.78:0/715894465 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5559a9c73900 0x5559a9c75dc0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61a58eb071e0fb47 server_cookie=b3d59eb34bf8800b in_seq=0 out_seq=0 2024-08-18T23:47:13.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:13 smithi123 ceph-mon[21808]: pgmap v437: 129 pgs: 129 active+clean; 405 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 355 op/s 2024-08-18T23:47:13.883 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:47:14.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.401+0000 7f911a435640 1 -- 172.21.15.78:0/1990035458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91140ffe30 msgr2=0x7f9114100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:14.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.401+0000 7f911a435640 1 --2- 172.21.15.78:0/1990035458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91140ffe30 0x7f9114100290 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009900 tx=0x7f90fc02a410 comp rx=0 tx=0).stop 2024-08-18T23:47:14.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.401+0000 7f911a435640 1 -- 172.21.15.78:0/1990035458 shutdown_connections 2024-08-18T23:47:14.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.401+0000 7f911a435640 1 --2- 172.21.15.78:0/1990035458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91141007d0 0x7f911410e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.401+0000 7f911a435640 1 --2- 172.21.15.78:0/1990035458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91140ffe30 0x7f9114100290 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.401+0000 7f911a435640 1 --2- 172.21.15.78:0/1990035458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91141068b0 0x7f9114106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.401+0000 7f911a435640 1 -- 172.21.15.78:0/1990035458 >> 172.21.15.78:0/1990035458 conn(0x7f9114076f60 msgr2=0x7f91140fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:14.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.401+0000 7f911a435640 1 -- 172.21.15.78:0/1990035458 shutdown_connections 2024-08-18T23:47:14.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.401+0000 7f911a435640 1 -- 172.21.15.78:0/1990035458 wait complete. 2024-08-18T23:47:14.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.402+0000 7f911a435640 1 Processor -- start 2024-08-18T23:47:14.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.402+0000 7f911a435640 1 -- start start 2024-08-18T23:47:14.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.402+0000 7f911a435640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91140ffe30 0x7f9114101010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:14.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.402+0000 7f911a435640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91141007d0 0x7f9114101550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:14.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.402+0000 7f911a435640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91141068b0 0x7f9114101a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:14.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.402+0000 7f911a435640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9114111d50 con 0x7f91140ffe30 2024-08-18T23:47:14.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.402+0000 7f911a435640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9114111bd0 con 0x7f91141068b0 2024-08-18T23:47:14.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.402+0000 7f911a435640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9114111ed0 con 0x7f91141007d0 2024-08-18T23:47:14.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.402+0000 7f9113fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91140ffe30 0x7f9114101010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:14.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.402+0000 7f91189d2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91141068b0 0x7f9114101a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:14.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.402+0000 7f9113fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91140ffe30 0x7f9114101010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42098/0 (socket says 172.21.15.78:42098) 2024-08-18T23:47:14.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.402+0000 7f9113fff640 1 -- 172.21.15.78:0/2335497305 learned_addr learned my addr 172.21.15.78:0/2335497305 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:14.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.402+0000 7f910bfff640 1 --2- 172.21.15.78:0/2335497305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91141007d0 0x7f9114101550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:14.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.403+0000 7f9113fff640 1 -- 172.21.15.78:0/2335497305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91141007d0 msgr2=0x7f9114101550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:14.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.403+0000 7f9113fff640 1 --2- 172.21.15.78:0/2335497305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91141007d0 0x7f9114101550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.403+0000 7f9113fff640 1 -- 172.21.15.78:0/2335497305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91141068b0 msgr2=0x7f9114101a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:14.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.403+0000 7f9113fff640 1 --2- 172.21.15.78:0/2335497305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91141068b0 0x7f9114101a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.403+0000 7f9113fff640 1 -- 172.21.15.78:0/2335497305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91141a66f0 con 0x7f91140ffe30 2024-08-18T23:47:14.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.403+0000 7f91189d2640 1 --2- 172.21.15.78:0/2335497305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91141068b0 0x7f9114101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:14.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.403+0000 7f9113fff640 1 --2- 172.21.15.78:0/2335497305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91140ffe30 0x7f9114101010 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f910400d950 tx=0x7f910400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:14.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.403+0000 7f9111ffb640 1 -- 172.21.15.78:0/2335497305 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104017070 con 0x7f91140ffe30 2024-08-18T23:47:14.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.403+0000 7f911a435640 1 -- 172.21.15.78:0/2335497305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91141a69e0 con 0x7f91140ffe30 2024-08-18T23:47:14.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.403+0000 7f9111ffb640 1 -- 172.21.15.78:0/2335497305 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9104009c30 con 0x7f91140ffe30 2024-08-18T23:47:14.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.403+0000 7f9111ffb640 1 -- 172.21.15.78:0/2335497305 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910400f6b0 con 0x7f91140ffe30 2024-08-18T23:47:14.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.403+0000 7f911a435640 1 -- 172.21.15.78:0/2335497305 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91141a6f40 con 0x7f91140ffe30 2024-08-18T23:47:14.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.405+0000 7f9111ffb640 1 -- 172.21.15.78:0/2335497305 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f91040109d0 con 0x7f91140ffe30 2024-08-18T23:47:14.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.406+0000 7f911c833740 1 Processor -- start 2024-08-18T23:47:14.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.406+0000 7f911c833740 1 -- start start 2024-08-18T23:47:14.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.406+0000 7f911c833740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad1bff0 0x55987ad1c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:14.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.406+0000 7f911c833740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1c910 0x55987ad26c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:14.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.406+0000 7f911c833740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987ad271d0 0x55987ad295c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:14.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.406+0000 7f911c833740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55987ac81a40 con 0x55987ad1c910 2024-08-18T23:47:14.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.406+0000 7f911c833740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55987ac199d0 con 0x55987ad1bff0 2024-08-18T23:47:14.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.406+0000 7f911c833740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55987ac7da00 con 0x55987ad271d0 2024-08-18T23:47:14.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.406+0000 7f9111ffb640 1 --2- 172.21.15.78:0/2335497305 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f91000a2b80 0x7f91000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:14.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.406+0000 7f9113fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1c910 0x55987ad26c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:14.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.406+0000 7f91189d2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad1bff0 0x55987ad1c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:14.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.406+0000 7f91189d2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad1bff0 0x55987ad1c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:60874/0 (socket says 172.21.15.78:60874) 2024-08-18T23:47:14.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.406+0000 7f91189d2640 1 -- 172.21.15.78:0/1397431941 learned_addr learned my addr 172.21.15.78:0/1397431941 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:14.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.406+0000 7f910bfff640 1 --2- 172.21.15.78:0/1397431941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987ad271d0 0x55987ad295c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:14.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.407+0000 7f91189d2640 1 --2- 172.21.15.78:0/2335497305 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f91000a2b80 0x7f91000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:14.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.407+0000 7f9111ffb640 1 -- 172.21.15.78:0/2335497305 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f91040cef10 con 0x7f91140ffe30 2024-08-18T23:47:14.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.407+0000 7f9113fff640 1 -- 172.21.15.78:0/1397431941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987ad271d0 msgr2=0x55987ad295c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:14.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.407+0000 7f9113fff640 1 --2- 172.21.15.78:0/1397431941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987ad271d0 0x55987ad295c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.407+0000 7f9113fff640 1 -- 172.21.15.78:0/1397431941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad1bff0 msgr2=0x55987ad1c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:14.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.407+0000 7f9113fff640 1 --2- 172.21.15.78:0/1397431941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad1bff0 0x55987ad1c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.407+0000 7f9113fff640 1 -- 172.21.15.78:0/1397431941 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55987ac20650 con 0x55987ad1c910 2024-08-18T23:47:14.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.407+0000 7f91189d2640 1 --2- 172.21.15.78:0/1397431941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad1bff0 0x55987ad1c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:14.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.407+0000 7f9113fff640 1 --2- 172.21.15.78:0/1397431941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1c910 0x55987ad26c90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f910409f270 tx=0x7f9104085fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0c973a99096207 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:14.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.407+0000 7f910a7fc640 1 -- 172.21.15.78:0/1397431941 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104020070 con 0x55987ad1c910 2024-08-18T23:47:14.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.407+0000 7f91189d2640 1 --2- 172.21.15.78:0/2335497305 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f91000a2b80 0x7f91000a5040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f910c005c40 tx=0x7f910c004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:14.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.407+0000 7f910a7fc640 1 -- 172.21.15.78:0/1397431941 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91040a3ff0 con 0x55987ad1c910 2024-08-18T23:47:14.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.407+0000 7f910a7fc640 1 -- 172.21.15.78:0/1397431941 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91040a3b60 con 0x55987ad1c910 2024-08-18T23:47:14.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.408+0000 7f911c833740 1 -- 172.21.15.78:0/1397431941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1c910 msgr2=0x55987ad26c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:14.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.408+0000 7f911c833740 1 --2- 172.21.15.78:0/1397431941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1c910 0x55987ad26c90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f910409f270 tx=0x7f9104085fb0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.408+0000 7f911c833740 1 -- 172.21.15.78:0/1397431941 shutdown_connections 2024-08-18T23:47:14.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.408+0000 7f911c833740 1 --2- 172.21.15.78:0/1397431941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987ad271d0 0x55987ad295c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.408+0000 7f911c833740 1 --2- 172.21.15.78:0/1397431941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1c910 0x55987ad26c90 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.408+0000 7f911c833740 1 --2- 172.21.15.78:0/1397431941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad1bff0 0x55987ad1c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.408+0000 7f911c833740 1 -- 172.21.15.78:0/1397431941 >> 172.21.15.78:0/1397431941 conn(0x55987ad18580 msgr2=0x55987ad18960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:14.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.408+0000 7f911c833740 1 -- 172.21.15.78:0/1397431941 shutdown_connections 2024-08-18T23:47:14.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.408+0000 7f911c833740 1 -- 172.21.15.78:0/1397431941 wait complete. 2024-08-18T23:47:14.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.408+0000 7f911c833740 1 -- start start 2024-08-18T23:47:14.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f911c833740 1 Processor -- start 2024-08-18T23:47:14.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f911c833740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 0x55987ad240a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:14.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f911c833740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad1c910 0x55987ad245e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:14.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f911c833740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987ad271d0 0x55987ad24b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:14.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f911c833740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55987ad29da0 con 0x55987ad1bff0 2024-08-18T23:47:14.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f911c833740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55987ac3cb20 con 0x55987ad1c910 2024-08-18T23:47:14.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f911c833740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55987ad29f20 con 0x55987ad271d0 2024-08-18T23:47:14.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f911c833740 10 client.0 _ll_register_callbacks cb 0x55987ac3abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:14.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f9113fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 0x55987ad240a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:14.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f9113fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 0x55987ad240a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42118/0 (socket says 172.21.15.78:42118) 2024-08-18T23:47:14.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f9113fff640 1 -- 172.21.15.78:0/3250699730 learned_addr learned my addr 172.21.15.78:0/3250699730 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:14.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f910bfff640 1 --2- 172.21.15.78:0/3250699730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad1c910 0x55987ad245e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:14.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f9113fff640 1 -- 172.21.15.78:0/3250699730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987ad271d0 msgr2=0x55987ad24b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:14.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f91189d2640 1 --2- 172.21.15.78:0/3250699730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987ad271d0 0x55987ad24b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:14.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f9113fff640 1 --2- 172.21.15.78:0/3250699730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987ad271d0 0x55987ad24b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f9113fff640 1 -- 172.21.15.78:0/3250699730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad1c910 msgr2=0x55987ad245e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:14.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f9113fff640 1 --2- 172.21.15.78:0/3250699730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad1c910 0x55987ad245e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f9113fff640 1 -- 172.21.15.78:0/3250699730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55987a9f33c0 con 0x55987ad1bff0 2024-08-18T23:47:14.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f910bfff640 1 --2- 172.21.15.78:0/3250699730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad1c910 0x55987ad245e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:47:14.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.409+0000 7f9113fff640 1 --2- 172.21.15.78:0/3250699730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 0x55987ad240a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f910409b3c0 tx=0x7f9104097cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19543e276e28898d server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:14.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.410+0000 7f90f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:47:14.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.410+0000 7f90f37fe640 1 -- 172.21.15.78:0/3250699730 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104020040 con 0x55987ad1bff0 2024-08-18T23:47:14.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.410+0000 7f90f37fe640 1 -- 172.21.15.78:0/3250699730 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f91040a3d70 con 0x55987ad1bff0 2024-08-18T23:47:14.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.410+0000 7f911c833740 1 -- 172.21.15.78:0/3250699730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91040a3ff0 con 0x55987ad1bff0 2024-08-18T23:47:14.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.410+0000 7f90f37fe640 1 -- 172.21.15.78:0/3250699730 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f910409b430 con 0x55987ad1bff0 2024-08-18T23:47:14.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.410+0000 7f911c833740 10 client.15936 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:14.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.410+0000 7f911c833740 1 -- 172.21.15.78:0/3250699730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55987ad359a0 con 0x55987ad1bff0 2024-08-18T23:47:14.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.410+0000 7f90f37fe640 1 -- 172.21.15.78:0/3250699730 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91040a3ff0 con 0x55987ad1bff0 2024-08-18T23:47:14.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.410+0000 7f90f37fe640 1 -- 172.21.15.78:0/3250699730 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f9104087360 con 0x55987ad1bff0 2024-08-18T23:47:14.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.410+0000 7f911c833740 10 client.15936 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:14.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.410+0000 7f911c833740 10 client.15936 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:47:14.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.410+0000 7f911c833740 20 client.15936 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:14.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.410+0000 7f911c833740 1 --2- 172.21.15.78:0/3250699730 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55987ad36e80 0x55987ad39270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:14.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.410+0000 7f911c833740 4 client.15936 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:47:14.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.410+0000 7f911c833740 1 -- 172.21.15.78:0/3250699730 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55987ac8efb0 con 0x55987ad36e80 2024-08-18T23:47:14.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.410+0000 7f910bfff640 1 --2- 172.21.15.78:0/3250699730 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55987ad36e80 0x55987ad39270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:14.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.411+0000 7f910bfff640 1 --2- 172.21.15.78:0/3250699730 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55987ad36e80 0x55987ad39270 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48aec309f053d00b server_cookie=48053fa68117ea1a in_seq=0 out_seq=0 2024-08-18T23:47:14.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.411+0000 7f90f37fe640 10 client.15936 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:47:14.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.411+0000 7f90f37fe640 1 -- 172.21.15.78:0/3250699730 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55987ac8efb0 con 0x55987ad36e80 2024-08-18T23:47:14.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.411+0000 7f90f37fe640 10 client.15936 handle_command_reply: tid=0 2024-08-18T23:47:14.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.412+0000 7f911c833740 1 client.15936 shutdown 2024-08-18T23:47:14.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.412+0000 7f911c833740 1 -- 172.21.15.78:0/3250699730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 msgr2=0x55987ad240a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:14.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.412+0000 7f911c833740 1 --2- 172.21.15.78:0/3250699730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 0x55987ad240a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f910409b3c0 tx=0x7f9104097cd0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.412+0000 7f911c833740 1 -- 172.21.15.78:0/3250699730 shutdown_connections 2024-08-18T23:47:14.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.412+0000 7f911c833740 1 --2- 172.21.15.78:0/3250699730 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55987ad36e80 0x55987ad39270 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.412+0000 7f911c833740 1 --2- 172.21.15.78:0/3250699730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987ad271d0 0x55987ad24b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.412+0000 7f911c833740 1 --2- 172.21.15.78:0/3250699730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad1c910 0x55987ad245e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.412+0000 7f90f37fe640 0 client.15936 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:47:14.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.412+0000 7f911c833740 1 --2- 172.21.15.78:0/3250699730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 0x55987ad240a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.412+0000 7f911c833740 1 -- 172.21.15.78:0/3250699730 >> 172.21.15.78:0/3250699730 conn(0x55987ad18580 msgr2=0x55987ad1e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:14.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.412+0000 7f911c833740 1 -- 172.21.15.78:0/3250699730 shutdown_connections 2024-08-18T23:47:14.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.412+0000 7f911c833740 1 -- 172.21.15.78:0/3250699730 wait complete. 2024-08-18T23:47:14.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.412+0000 7f911c833740 20 client.15936 trim_cache size 0 max 16384 2024-08-18T23:47:14.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.427+0000 7f911c833740 1 Processor -- start 2024-08-18T23:47:14.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.427+0000 7f911c833740 1 -- start start 2024-08-18T23:47:14.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.427+0000 7f911c833740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 0x55987ad21300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:14.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.427+0000 7f911c833740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987ad1c910 0x55987ad21840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:14.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.427+0000 7f911c833740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad271d0 0x55987ad257f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:14.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.427+0000 7f911c833740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91141b34f0 con 0x55987ad1bff0 2024-08-18T23:47:14.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.427+0000 7f911c833740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91141b0dc0 con 0x55987ad271d0 2024-08-18T23:47:14.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.427+0000 7f911c833740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f91141aec10 con 0x55987ad1c910 2024-08-18T23:47:14.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.427+0000 7f9113fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 0x55987ad21300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:14.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.427+0000 7f9113fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 0x55987ad21300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42128/0 (socket says 172.21.15.78:42128) 2024-08-18T23:47:14.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.427+0000 7f9113fff640 1 -- 172.21.15.78:0/2818197751 learned_addr learned my addr 172.21.15.78:0/2818197751 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:14.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.427+0000 7f91189d2640 1 --2- 172.21.15.78:0/2818197751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad271d0 0x55987ad257f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:14.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.427+0000 7f9113fff640 1 -- 172.21.15.78:0/2818197751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987ad1c910 msgr2=0x55987ad21840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:14.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.427+0000 7f9113fff640 1 --2- 172.21.15.78:0/2818197751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987ad1c910 0x55987ad21840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.427+0000 7f9113fff640 1 -- 172.21.15.78:0/2818197751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad271d0 msgr2=0x55987ad257f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:14.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.427+0000 7f9113fff640 1 --2- 172.21.15.78:0/2818197751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad271d0 0x55987ad257f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.427+0000 7f9113fff640 1 -- 172.21.15.78:0/2818197751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55987ad1c740 con 0x55987ad1bff0 2024-08-18T23:47:14.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.428+0000 7f9113fff640 1 --2- 172.21.15.78:0/2818197751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 0x55987ad21300 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9104097840 tx=0x7f9104005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13a680f7e25bc25 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:14.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.428+0000 7f9109ffb640 1 -- 172.21.15.78:0/2818197751 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104020070 con 0x55987ad1bff0 2024-08-18T23:47:14.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.428+0000 7f9109ffb640 1 -- 172.21.15.78:0/2818197751 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91040055d0 con 0x55987ad1bff0 2024-08-18T23:47:14.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.428+0000 7f9109ffb640 1 -- 172.21.15.78:0/2818197751 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104097200 con 0x55987ad1bff0 2024-08-18T23:47:14.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.429+0000 7f911c833740 1 -- 172.21.15.78:0/2818197751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 msgr2=0x55987ad21300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:14.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.429+0000 7f911c833740 1 --2- 172.21.15.78:0/2818197751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 0x55987ad21300 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9104097840 tx=0x7f9104005bb0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.429+0000 7f911c833740 1 -- 172.21.15.78:0/2818197751 shutdown_connections 2024-08-18T23:47:14.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.429+0000 7f911c833740 1 --2- 172.21.15.78:0/2818197751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad271d0 0x55987ad257f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.429+0000 7f911c833740 1 --2- 172.21.15.78:0/2818197751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987ad1c910 0x55987ad21840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.429+0000 7f911c833740 1 --2- 172.21.15.78:0/2818197751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 0x55987ad21300 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.429+0000 7f911c833740 1 -- 172.21.15.78:0/2818197751 >> 172.21.15.78:0/2818197751 conn(0x55987ad18580 msgr2=0x55987ad22970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:14.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.429+0000 7f911c833740 1 -- 172.21.15.78:0/2818197751 shutdown_connections 2024-08-18T23:47:14.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.429+0000 7f911c833740 1 -- 172.21.15.78:0/2818197751 wait complete. 2024-08-18T23:47:14.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.430+0000 7f911c833740 1 -- start start 2024-08-18T23:47:14.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.430+0000 7f911c833740 1 Processor -- start 2024-08-18T23:47:14.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.430+0000 7f911c833740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 0x55987ad39160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:14.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.430+0000 7f911c833740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987ad1c910 0x55987ad396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:14.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.430+0000 7f911c833740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad271d0 0x55987ad39be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:14.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.430+0000 7f911c833740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91141af280 con 0x55987ad1bff0 2024-08-18T23:47:14.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.430+0000 7f911c833740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91141b06f0 con 0x55987ad271d0 2024-08-18T23:47:14.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.430+0000 7f911c833740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f91141af950 con 0x55987ad1c910 2024-08-18T23:47:14.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.430+0000 7f911c833740 10 client.0 _ll_register_callbacks cb 0x7f910c009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:14.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.430+0000 7f9113fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 0x55987ad39160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:14.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.430+0000 7f9113fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 0x55987ad39160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42140/0 (socket says 172.21.15.78:42140) 2024-08-18T23:47:14.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.430+0000 7f9113fff640 1 -- 172.21.15.78:0/1211154732 learned_addr learned my addr 172.21.15.78:0/1211154732 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:14.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.430+0000 7f91189d2640 1 --2- 172.21.15.78:0/1211154732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad271d0 0x55987ad39be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:14.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.431+0000 7f910bfff640 1 --2- 172.21.15.78:0/1211154732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987ad1c910 0x55987ad396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:14.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.431+0000 7f9113fff640 1 -- 172.21.15.78:0/1211154732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987ad1c910 msgr2=0x55987ad396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:14.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.431+0000 7f9113fff640 1 --2- 172.21.15.78:0/1211154732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987ad1c910 0x55987ad396a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.431+0000 7f9113fff640 1 -- 172.21.15.78:0/1211154732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad271d0 msgr2=0x55987ad39be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:14.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.431+0000 7f9113fff640 1 --2- 172.21.15.78:0/1211154732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad271d0 0x55987ad39be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.431+0000 7f9113fff640 1 -- 172.21.15.78:0/1211154732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55987ad9f860 con 0x55987ad1bff0 2024-08-18T23:47:14.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.431+0000 7f9113fff640 1 --2- 172.21.15.78:0/1211154732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 0x55987ad39160 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9104009610 tx=0x7f910400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601208ddffc1d4f6 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:14.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.431+0000 7f90f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:47:14.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.431+0000 7f90f37fe640 1 -- 172.21.15.78:0/1211154732 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104020070 con 0x55987ad1bff0 2024-08-18T23:47:14.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.431+0000 7f90f37fe640 1 -- 172.21.15.78:0/1211154732 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f910409ab50 con 0x55987ad1bff0 2024-08-18T23:47:14.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.431+0000 7f911c833740 1 -- 172.21.15.78:0/1211154732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91040055d0 con 0x55987ad1bff0 2024-08-18T23:47:14.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.431+0000 7f90f37fe640 1 -- 172.21.15.78:0/1211154732 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f91040a3ff0 con 0x55987ad1bff0 2024-08-18T23:47:14.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.431+0000 7f911c833740 10 client.15942 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:14.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.431+0000 7f911c833740 1 -- 172.21.15.78:0/1211154732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55987ad9f860 con 0x55987ad1bff0 2024-08-18T23:47:14.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.431+0000 7f90f37fe640 1 -- 172.21.15.78:0/1211154732 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91040055d0 con 0x55987ad1bff0 2024-08-18T23:47:14.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.432+0000 7f90f37fe640 1 -- 172.21.15.78:0/1211154732 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f910407c070 con 0x55987ad1bff0 2024-08-18T23:47:14.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.432+0000 7f911c833740 10 client.15942 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:14.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.432+0000 7f911c833740 10 client.15942 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:47:14.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.432+0000 7f911c833740 20 client.15942 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:14.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.432+0000 7f911c833740 1 --2- 172.21.15.78:0/1211154732 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55987ad36e80 0x55987ada2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:14.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.432+0000 7f911c833740 4 client.15942 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:47:14.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.432+0000 7f911c833740 1 -- 172.21.15.78:0/1211154732 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55987ac894c0 con 0x55987ad36e80 2024-08-18T23:47:14.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.432+0000 7f910bfff640 1 --2- 172.21.15.78:0/1211154732 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55987ad36e80 0x55987ada2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:14.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.432+0000 7f90f37fe640 10 client.15942 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:47:14.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.432+0000 7f910bfff640 1 --2- 172.21.15.78:0/1211154732 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55987ad36e80 0x55987ada2ea0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=abaec3710e8a0ef5 server_cookie=3b16f453d3aa5c31 in_seq=0 out_seq=0 2024-08-18T23:47:14.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.432+0000 7f90f37fe640 1 -- 172.21.15.78:0/1211154732 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55987ac894c0 con 0x55987ad36e80 2024-08-18T23:47:14.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.432+0000 7f90f37fe640 10 client.15942 handle_command_reply: tid=0 2024-08-18T23:47:14.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.432+0000 7f911c833740 1 client.15942 shutdown 2024-08-18T23:47:14.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.433+0000 7f911c833740 1 -- 172.21.15.78:0/1211154732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 msgr2=0x55987ad39160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:14.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.433+0000 7f911c833740 1 --2- 172.21.15.78:0/1211154732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 0x55987ad39160 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9104009610 tx=0x7f910400bf60 comp rx=0 tx=0).stop 2024-08-18T23:47:14.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.433+0000 7f911c833740 1 -- 172.21.15.78:0/1211154732 shutdown_connections 2024-08-18T23:47:14.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.433+0000 7f911c833740 1 --2- 172.21.15.78:0/1211154732 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55987ad36e80 0x55987ada2ea0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.433+0000 7f911c833740 1 --2- 172.21.15.78:0/1211154732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55987ad271d0 0x55987ad39be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.433+0000 7f911c833740 1 --2- 172.21.15.78:0/1211154732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55987ad1c910 0x55987ad396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.433+0000 7f90f37fe640 0 client.15942 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:47:14.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.433+0000 7f911c833740 1 --2- 172.21.15.78:0/1211154732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55987ad1bff0 0x55987ad39160 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.433+0000 7f911c833740 1 -- 172.21.15.78:0/1211154732 >> 172.21.15.78:0/1211154732 conn(0x55987ad18580 msgr2=0x55987ad32170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:14.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.433+0000 7f911c833740 1 -- 172.21.15.78:0/1211154732 shutdown_connections 2024-08-18T23:47:14.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.433+0000 7f911c833740 1 -- 172.21.15.78:0/1211154732 wait complete. 2024-08-18T23:47:14.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.433+0000 7f911c833740 20 client.15942 trim_cache size 0 max 16384 2024-08-18T23:47:14.452 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:47:14.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.433+0000 7f911a435640 1 -- 172.21.15.78:0/2335497305 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f91000a2b80 msgr2=0x7f91000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:14.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.433+0000 7f911a435640 1 --2- 172.21.15.78:0/2335497305 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f91000a2b80 0x7f91000a5040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f910c005c40 tx=0x7f910c004ab0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.433+0000 7f911a435640 1 -- 172.21.15.78:0/2335497305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91140ffe30 msgr2=0x7f9114101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:14.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.433+0000 7f911a435640 1 --2- 172.21.15.78:0/2335497305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91140ffe30 0x7f9114101010 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f910400d950 tx=0x7f910400de10 comp rx=0 tx=0).stop 2024-08-18T23:47:14.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.434+0000 7f911a435640 1 -- 172.21.15.78:0/2335497305 shutdown_connections 2024-08-18T23:47:14.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.434+0000 7f911a435640 1 --2- 172.21.15.78:0/2335497305 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f91000a2b80 0x7f91000a5040 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.434+0000 7f911a435640 1 --2- 172.21.15.78:0/2335497305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91141068b0 0x7f9114101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.434+0000 7f911a435640 1 --2- 172.21.15.78:0/2335497305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91141007d0 0x7f9114101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.434+0000 7f911a435640 1 --2- 172.21.15.78:0/2335497305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91140ffe30 0x7f9114101010 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:14.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.434+0000 7f911a435640 1 -- 172.21.15.78:0/2335497305 >> 172.21.15.78:0/2335497305 conn(0x7f9114076f60 msgr2=0x7f9114104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:14.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.434+0000 7f911a435640 1 -- 172.21.15.78:0/2335497305 shutdown_connections 2024-08-18T23:47:14.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:14.434+0000 7f911a435640 1 -- 172.21.15.78:0/2335497305 wait complete. 2024-08-18T23:47:15.329 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-18T23:47:15.629 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:47:15.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:15 smithi161 ceph-mon[22027]: pgmap v438: 129 pgs: 129 active+clean; 408 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 350 op/s 2024-08-18T23:47:15.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:15 smithi078 ceph-mon[18440]: pgmap v438: 129 pgs: 129 active+clean; 408 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 350 op/s 2024-08-18T23:47:15.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:15 smithi123 ceph-mon[21808]: pgmap v438: 129 pgs: 129 active+clean; 408 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 350 op/s 2024-08-18T23:47:16.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.105+0000 7fdc784ca640 1 -- 172.21.15.78:0/1081990587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc70103410 msgr2=0x7fdc701078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:16.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.105+0000 7fdc784ca640 1 --2- 172.21.15.78:0/1081990587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc70103410 0x7fdc701078c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009930 tx=0x7fdc6002a420 comp rx=0 tx=0).stop 2024-08-18T23:47:16.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.105+0000 7fdc784ca640 1 -- 172.21.15.78:0/1081990587 shutdown_connections 2024-08-18T23:47:16.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.105+0000 7fdc784ca640 1 --2- 172.21.15.78:0/1081990587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc70107fd0 0x7fdc7010e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.105+0000 7fdc784ca640 1 --2- 172.21.15.78:0/1081990587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc70103410 0x7fdc701078c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.105+0000 7fdc784ca640 1 --2- 172.21.15.78:0/1081990587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc70102a60 0x7fdc70102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.105+0000 7fdc784ca640 1 -- 172.21.15.78:0/1081990587 >> 172.21.15.78:0/1081990587 conn(0x7fdc700fc500 msgr2=0x7fdc700fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:16.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.105+0000 7fdc784ca640 1 -- 172.21.15.78:0/1081990587 shutdown_connections 2024-08-18T23:47:16.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.105+0000 7fdc784ca640 1 -- 172.21.15.78:0/1081990587 wait complete. 2024-08-18T23:47:16.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.106+0000 7fdc784ca640 1 Processor -- start 2024-08-18T23:47:16.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.106+0000 7fdc784ca640 1 -- start start 2024-08-18T23:47:16.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.106+0000 7fdc784ca640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc70102a60 0x7fdc70198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:16.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.106+0000 7fdc784ca640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc70103410 0x7fdc701988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:16.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.106+0000 7fdc784ca640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc70107fd0 0x7fdc7019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:16.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.106+0000 7fdc784ca640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc70111df0 con 0x7fdc70103410 2024-08-18T23:47:16.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.106+0000 7fdc784ca640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc70111c70 con 0x7fdc70102a60 2024-08-18T23:47:16.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.106+0000 7fdc784ca640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdc70111f70 con 0x7fdc70107fd0 2024-08-18T23:47:16.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.106+0000 7fdc75a65640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc70103410 0x7fdc701988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:16.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.106+0000 7fdc75a65640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc70103410 0x7fdc701988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57994/0 (socket says 172.21.15.78:57994) 2024-08-18T23:47:16.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.106+0000 7fdc75a65640 1 -- 172.21.15.78:0/2747187664 learned_addr learned my addr 172.21.15.78:0/2747187664 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:16.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.106+0000 7fdc76266640 1 --2- 172.21.15.78:0/2747187664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc70102a60 0x7fdc70198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:16.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.106+0000 7fdc76a67640 1 --2- 172.21.15.78:0/2747187664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc70107fd0 0x7fdc7019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:16.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.106+0000 7fdc75a65640 1 -- 172.21.15.78:0/2747187664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc70107fd0 msgr2=0x7fdc7019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:16.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.106+0000 7fdc75a65640 1 --2- 172.21.15.78:0/2747187664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc70107fd0 0x7fdc7019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.106+0000 7fdc75a65640 1 -- 172.21.15.78:0/2747187664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc70102a60 msgr2=0x7fdc70198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:16.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.106+0000 7fdc75a65640 1 --2- 172.21.15.78:0/2747187664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc70102a60 0x7fdc70198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.106+0000 7fdc75a65640 1 -- 172.21.15.78:0/2747187664 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7019d320 con 0x7fdc70103410 2024-08-18T23:47:16.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.107+0000 7fdc75a65640 1 --2- 172.21.15.78:0/2747187664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc70103410 0x7fdc701988b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdc60005940 tx=0x7fdc600040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:16.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.107+0000 7fdc677fe640 1 -- 172.21.15.78:0/2747187664 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6003f030 con 0x7fdc70103410 2024-08-18T23:47:16.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.107+0000 7fdc677fe640 1 -- 172.21.15.78:0/2747187664 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc600044c0 con 0x7fdc70103410 2024-08-18T23:47:16.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.107+0000 7fdc677fe640 1 -- 172.21.15.78:0/2747187664 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60035070 con 0x7fdc70103410 2024-08-18T23:47:16.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.107+0000 7fdc784ca640 1 -- 172.21.15.78:0/2747187664 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7019d5b0 con 0x7fdc70103410 2024-08-18T23:47:16.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.107+0000 7fdc784ca640 1 -- 172.21.15.78:0/2747187664 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc701a4ef0 con 0x7fdc70103410 2024-08-18T23:47:16.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.109+0000 7fdc677fe640 1 -- 172.21.15.78:0/2747187664 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdc60042030 con 0x7fdc70103410 2024-08-18T23:47:16.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.109+0000 7fdc7a8c8740 1 Processor -- start 2024-08-18T23:47:16.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.109+0000 7fdc677fe640 1 --2- 172.21.15.78:0/2747187664 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdc5c0a2b80 0x7fdc5c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:16.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc76266640 1 --2- 172.21.15.78:0/2747187664 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdc5c0a2b80 0x7fdc5c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:16.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc677fe640 1 -- 172.21.15.78:0/2747187664 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fdc600ef7d0 con 0x7fdc70103410 2024-08-18T23:47:16.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc7a8c8740 1 -- start start 2024-08-18T23:47:16.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc7a8c8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:16.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc7a8c8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:16.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc7a8c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6b05e0 0x55bb1e6b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:16.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc7a8c8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bb1e606a40 con 0x55bb1e6a0ed0 2024-08-18T23:47:16.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc7a8c8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bb1e59e9d0 con 0x55bb1e6a17f0 2024-08-18T23:47:16.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc7a8c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bb1e602a00 con 0x55bb1e6b05e0 2024-08-18T23:47:16.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc76266640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:16.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc75a65640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:16.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc76266640 1 --2- 172.21.15.78:0/2747187664 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdc5c0a2b80 0x7fdc5c0a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009990 tx=0x7fdc6c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:16.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc76266640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6a12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57998/0 (socket says 172.21.15.78:57998) 2024-08-18T23:47:16.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc76266640 1 -- 172.21.15.78:0/2391966939 learned_addr learned my addr 172.21.15.78:0/2391966939 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:16.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc76a67640 1 --2- 172.21.15.78:0/2391966939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6b05e0 0x55bb1e6b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:16.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc75a65640 1 -- 172.21.15.78:0/2391966939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6b05e0 msgr2=0x55bb1e6b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:16.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc75a65640 1 --2- 172.21.15.78:0/2391966939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6b05e0 0x55bb1e6b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc75a65640 1 -- 172.21.15.78:0/2391966939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 msgr2=0x55bb1e6a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:16.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc75a65640 1 --2- 172.21.15.78:0/2391966939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc75a65640 1 -- 172.21.15.78:0/2391966939 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb1e5a5650 con 0x55bb1e6a17f0 2024-08-18T23:47:16.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc76a67640 1 --2- 172.21.15.78:0/2391966939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6b05e0 0x55bb1e6b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:47:16.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.110+0000 7fdc76266640 1 --2- 172.21.15.78:0/2391966939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:16.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.111+0000 7fdc75a65640 1 --2- 172.21.15.78:0/2391966939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6b00a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdc600ecf40 tx=0x7fdc60031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d67292ee9e51f0e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:16.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.111+0000 7fdc64ff9640 1 -- 172.21.15.78:0/2391966939 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60049040 con 0x55bb1e6a17f0 2024-08-18T23:47:16.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.111+0000 7fdc64ff9640 1 -- 172.21.15.78:0/2391966939 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc600c0050 con 0x55bb1e6a17f0 2024-08-18T23:47:16.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.111+0000 7fdc64ff9640 1 -- 172.21.15.78:0/2391966939 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc600c46e0 con 0x55bb1e6a17f0 2024-08-18T23:47:16.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.112+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/2391966939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6a17f0 msgr2=0x55bb1e6b00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:16.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.112+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/2391966939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6b00a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdc600ecf40 tx=0x7fdc60031400 comp rx=0 tx=0).stop 2024-08-18T23:47:16.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.112+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/2391966939 shutdown_connections 2024-08-18T23:47:16.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.112+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/2391966939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6b05e0 0x55bb1e6b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.112+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/2391966939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6b00a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.112+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/2391966939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.112+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/2391966939 >> 172.21.15.78:0/2391966939 conn(0x55bb1e69d580 msgr2=0x55bb1e69d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:16.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.112+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/2391966939 shutdown_connections 2024-08-18T23:47:16.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.112+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/2391966939 wait complete. 2024-08-18T23:47:16.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.112+0000 7fdc7a8c8740 1 -- start start 2024-08-18T23:47:16.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.113+0000 7fdc7a8c8740 1 Processor -- start 2024-08-18T23:47:16.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.113+0000 7fdc7a8c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6affe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:16.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.113+0000 7fdc7a8c8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:16.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.113+0000 7fdc7a8c8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6b05e0 0x55bb1e6adae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:16.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.113+0000 7fdc7a8c8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bb1e6b2f70 con 0x55bb1e6a17f0 2024-08-18T23:47:16.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.113+0000 7fdc7a8c8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bb1e5c1b20 con 0x55bb1e6b05e0 2024-08-18T23:47:16.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.113+0000 7fdc7a8c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bb1e6b30f0 con 0x55bb1e6a0ed0 2024-08-18T23:47:16.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.113+0000 7fdc7a8c8740 10 client.0 _ll_register_callbacks cb 0x55bb1e5bfbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:16.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.113+0000 7fdc75a65640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:16.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.113+0000 7fdc75a65640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6ad5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58004/0 (socket says 172.21.15.78:58004) 2024-08-18T23:47:16.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.113+0000 7fdc75a65640 1 -- 172.21.15.78:0/239985059 learned_addr learned my addr 172.21.15.78:0/239985059 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:16.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.113+0000 7fdc76a67640 1 --2- 172.21.15.78:0/239985059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6b05e0 0x55bb1e6adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:16.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.113+0000 7fdc76266640 1 --2- 172.21.15.78:0/239985059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:16.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.113+0000 7fdc75a65640 1 -- 172.21.15.78:0/239985059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6a0ed0 msgr2=0x55bb1e6affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:16.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.113+0000 7fdc75a65640 1 --2- 172.21.15.78:0/239985059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.113+0000 7fdc75a65640 1 -- 172.21.15.78:0/239985059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6b05e0 msgr2=0x55bb1e6adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.113+0000 7fdc75a65640 1 --2- 172.21.15.78:0/239985059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6b05e0 0x55bb1e6adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.113+0000 7fdc75a65640 1 -- 172.21.15.78:0/239985059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb1e3783c0 con 0x55bb1e6a17f0 2024-08-18T23:47:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.113+0000 7fdc76a67640 1 --2- 172.21.15.78:0/239985059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6b05e0 0x55bb1e6adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.114+0000 7fdc75a65640 1 --2- 172.21.15.78:0/239985059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6ad5a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdc600ecf90 tx=0x7fdc6003b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4fb547c2cfa520 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.114+0000 7fdc59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:47:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.114+0000 7fdc59ffb640 1 -- 172.21.15.78:0/239985059 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60049070 con 0x55bb1e6a17f0 2024-08-18T23:47:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.114+0000 7fdc59ffb640 1 -- 172.21.15.78:0/239985059 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fdc60031bd0 con 0x55bb1e6a17f0 2024-08-18T23:47:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.114+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/239985059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc600c0050 con 0x55bb1e6a17f0 2024-08-18T23:47:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.114+0000 7fdc59ffb640 1 -- 172.21.15.78:0/239985059 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdc60031dd0 con 0x55bb1e6a17f0 2024-08-18T23:47:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.114+0000 7fdc7a8c8740 10 client.15954 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.114+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/239985059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bb1e6a7a20 con 0x55bb1e6a17f0 2024-08-18T23:47:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.114+0000 7fdc59ffb640 1 -- 172.21.15.78:0/239985059 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc600c4480 con 0x55bb1e6a17f0 2024-08-18T23:47:16.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.115+0000 7fdc59ffb640 1 -- 172.21.15.78:0/239985059 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fdc600b79f0 con 0x55bb1e6a17f0 2024-08-18T23:47:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.115+0000 7fdc7a8c8740 10 client.15954 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.115+0000 7fdc7a8c8740 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:47:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.115+0000 7fdc7a8c8740 20 client.15954 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.115+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/239985059 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55bb1e6c43b0 0x55bb1e6c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.115+0000 7fdc7a8c8740 4 client.15954 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:47:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.115+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/239985059 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdc600c0050 con 0x55bb1e6c43b0 2024-08-18T23:47:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.115+0000 7fdc76266640 1 --2- 172.21.15.78:0/239985059 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55bb1e6c43b0 0x55bb1e6c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.116+0000 7fdc76266640 1 --2- 172.21.15.78:0/239985059 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55bb1e6c43b0 0x55bb1e6c67a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75bf5df576678a7d server_cookie=112a93a5bfdcf33f in_seq=0 out_seq=0 2024-08-18T23:47:16.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.116+0000 7fdc59ffb640 10 client.15954 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:16.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.126+0000 7fdc59ffb640 1 -- 172.21.15.78:0/239985059 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fdc600c0050 con 0x55bb1e6c43b0 2024-08-18T23:47:16.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.126+0000 7fdc59ffb640 10 client.15954 handle_command_reply: tid=0 2024-08-18T23:47:16.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.126+0000 7fdc7a8c8740 1 client.15954 shutdown 2024-08-18T23:47:16.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.126+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/239985059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a17f0 msgr2=0x55bb1e6ad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:16.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.126+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/239985059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6ad5a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdc600ecf90 tx=0x7fdc6003b4d0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.126+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/239985059 shutdown_connections 2024-08-18T23:47:16.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.126+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/239985059 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55bb1e6c43b0 0x55bb1e6c67a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.126+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/239985059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6b05e0 0x55bb1e6adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.126+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/239985059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6ad5a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.126+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/239985059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.126+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/239985059 >> 172.21.15.78:0/239985059 conn(0x55bb1e69d580 msgr2=0x55bb1e6a32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:16.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.126+0000 7fdc59ffb640 0 client.15954 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:16.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.126+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/239985059 shutdown_connections 2024-08-18T23:47:16.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.126+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/239985059 wait complete. 2024-08-18T23:47:16.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.126+0000 7fdc7a8c8740 20 client.15954 trim_cache size 0 max 16384 2024-08-18T23:47:16.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.139+0000 7fdc7a8c8740 1 Processor -- start 2024-08-18T23:47:16.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.139+0000 7fdc7a8c8740 1 -- start start 2024-08-18T23:47:16.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.139+0000 7fdc7a8c8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6afd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:16.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.139+0000 7fdc7a8c8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:16.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.139+0000 7fdc7a8c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6b05e0 0x55bb1e6cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:16.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.139+0000 7fdc7a8c8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701af0e0 con 0x55bb1e6a0ed0 2024-08-18T23:47:16.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.139+0000 7fdc7a8c8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701ac9b0 con 0x55bb1e6a17f0 2024-08-18T23:47:16.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.139+0000 7fdc7a8c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701aa800 con 0x55bb1e6b05e0 2024-08-18T23:47:16.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.139+0000 7fdc76266640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:16.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.139+0000 7fdc76266640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58020/0 (socket says 172.21.15.78:58020) 2024-08-18T23:47:16.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.139+0000 7fdc76266640 1 -- 172.21.15.78:0/2682667824 learned_addr learned my addr 172.21.15.78:0/2682667824 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:16.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.139+0000 7fdc75a65640 1 --2- 172.21.15.78:0/2682667824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:16.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.139+0000 7fdc76a67640 1 --2- 172.21.15.78:0/2682667824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6b05e0 0x55bb1e6cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:16.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.139+0000 7fdc76266640 1 -- 172.21.15.78:0/2682667824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6b05e0 msgr2=0x55bb1e6cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:16.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.139+0000 7fdc76266640 1 --2- 172.21.15.78:0/2682667824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6b05e0 0x55bb1e6cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.140+0000 7fdc76266640 1 -- 172.21.15.78:0/2682667824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6a17f0 msgr2=0x55bb1e6cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:16.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.140+0000 7fdc76266640 1 --2- 172.21.15.78:0/2682667824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6cbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.140+0000 7fdc76266640 1 -- 172.21.15.78:0/2682667824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb1e6ae540 con 0x55bb1e6a0ed0 2024-08-18T23:47:16.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.140+0000 7fdc76266640 1 --2- 172.21.15.78:0/2682667824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6afd00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00ea30 tx=0x7fdc6c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11ec481d176cafc server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:16.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.140+0000 7fdc5bfff640 1 -- 172.21.15.78:0/2682667824 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c01b040 con 0x55bb1e6a0ed0 2024-08-18T23:47:16.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.140+0000 7fdc5bfff640 1 -- 172.21.15.78:0/2682667824 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc6c02f070 con 0x55bb1e6a0ed0 2024-08-18T23:47:16.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.140+0000 7fdc5bfff640 1 -- 172.21.15.78:0/2682667824 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c022d80 con 0x55bb1e6a0ed0 2024-08-18T23:47:16.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.141+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/2682667824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 msgr2=0x55bb1e6afd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:16.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.141+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/2682667824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6afd00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00ea30 tx=0x7fdc6c013000 comp rx=0 tx=0).stop 2024-08-18T23:47:16.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.141+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/2682667824 shutdown_connections 2024-08-18T23:47:16.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.141+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/2682667824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6b05e0 0x55bb1e6cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.141+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/2682667824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.141+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/2682667824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6afd00 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.141+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/2682667824 >> 172.21.15.78:0/2682667824 conn(0x55bb1e69d580 msgr2=0x55bb1e6a8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:16.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.141+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/2682667824 shutdown_connections 2024-08-18T23:47:16.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.141+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/2682667824 wait complete. 2024-08-18T23:47:16.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.141+0000 7fdc7a8c8740 1 -- start start 2024-08-18T23:47:16.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.141+0000 7fdc7a8c8740 1 Processor -- start 2024-08-18T23:47:16.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.142+0000 7fdc7a8c8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6bd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:16.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.142+0000 7fdc7a8c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6bdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:16.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.142+0000 7fdc7a8c8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6b05e0 0x55bb1e6be460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:16.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.142+0000 7fdc7a8c8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701aae70 con 0x55bb1e6a0ed0 2024-08-18T23:47:16.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.142+0000 7fdc7a8c8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701ac2e0 con 0x55bb1e6b05e0 2024-08-18T23:47:16.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.142+0000 7fdc7a8c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701ab540 con 0x55bb1e6a17f0 2024-08-18T23:47:16.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.142+0000 7fdc7a8c8740 10 client.0 _ll_register_callbacks cb 0x7fdc6c0095e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:16.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.142+0000 7fdc75a65640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6bdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:16.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.142+0000 7fdc76266640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6bd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:16.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.142+0000 7fdc75a65640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6bdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:35626/0 (socket says 172.21.15.78:35626) 2024-08-18T23:47:16.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.142+0000 7fdc75a65640 1 -- 172.21.15.78:0/395373540 learned_addr learned my addr 172.21.15.78:0/395373540 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:16.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.142+0000 7fdc76a67640 1 --2- 172.21.15.78:0/395373540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6b05e0 0x55bb1e6be460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:16.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.142+0000 7fdc76266640 1 -- 172.21.15.78:0/395373540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6a17f0 msgr2=0x55bb1e6bdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:16.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.142+0000 7fdc76266640 1 --2- 172.21.15.78:0/395373540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6bdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.142+0000 7fdc76266640 1 -- 172.21.15.78:0/395373540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6b05e0 msgr2=0x55bb1e6be460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:16.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.142+0000 7fdc76266640 1 --2- 172.21.15.78:0/395373540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6b05e0 0x55bb1e6be460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.142+0000 7fdc76266640 1 -- 172.21.15.78:0/395373540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb1e72ce30 con 0x55bb1e6a0ed0 2024-08-18T23:47:16.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.142+0000 7fdc76266640 1 --2- 172.21.15.78:0/395373540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6bd9e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009850 tx=0x7fdc6c015f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71063b98dc4e4168 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:16.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.142+0000 7fdc59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:47:16.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.143+0000 7fdc59ffb640 1 -- 172.21.15.78:0/395373540 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c023e60 con 0x55bb1e6a0ed0 2024-08-18T23:47:16.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.143+0000 7fdc59ffb640 1 -- 172.21.15.78:0/395373540 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fdc6c022430 con 0x55bb1e6a0ed0 2024-08-18T23:47:16.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.143+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/395373540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc6c02f070 con 0x55bb1e6a0ed0 2024-08-18T23:47:16.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.143+0000 7fdc59ffb640 1 -- 172.21.15.78:0/395373540 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdc6c022630 con 0x55bb1e6a0ed0 2024-08-18T23:47:16.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.143+0000 7fdc7a8c8740 10 client.15960 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:16.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.143+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/395373540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bb1e72e0b0 con 0x55bb1e6a0ed0 2024-08-18T23:47:16.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.143+0000 7fdc59ffb640 1 -- 172.21.15.78:0/395373540 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc6c02f070 con 0x55bb1e6a0ed0 2024-08-18T23:47:16.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.143+0000 7fdc59ffb640 1 -- 172.21.15.78:0/395373540 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fdc6c013450 con 0x55bb1e6a0ed0 2024-08-18T23:47:16.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.143+0000 7fdc7a8c8740 10 client.15960 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:16.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.143+0000 7fdc7a8c8740 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:47:16.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.143+0000 7fdc7a8c8740 20 client.15960 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:16.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.143+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/395373540 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55bb1e72e7f0 0x55bb1e730c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:16.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.143+0000 7fdc7a8c8740 4 client.15960 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:47:16.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.143+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/395373540 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb1e613fb0 con 0x55bb1e72e7f0 2024-08-18T23:47:16.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.143+0000 7fdc75a65640 1 --2- 172.21.15.78:0/395373540 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55bb1e72e7f0 0x55bb1e730c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:16.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.144+0000 7fdc75a65640 1 --2- 172.21.15.78:0/395373540 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55bb1e72e7f0 0x55bb1e730c10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2d114ab2e0745a2 server_cookie=b14916e7a5dbd875 in_seq=0 out_seq=0 2024-08-18T23:47:16.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.144+0000 7fdc59ffb640 10 client.15960 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:16.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.156+0000 7fdc59ffb640 1 -- 172.21.15.78:0/395373540 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb1e613fb0 con 0x55bb1e72e7f0 2024-08-18T23:47:16.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.156+0000 7fdc59ffb640 10 client.15960 handle_command_reply: tid=0 2024-08-18T23:47:16.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.156+0000 7fdc7a8c8740 1 client.15960 shutdown 2024-08-18T23:47:16.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.156+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/395373540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 msgr2=0x55bb1e6bd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:16.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.156+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/395373540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6bd9e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009850 tx=0x7fdc6c015f00 comp rx=0 tx=0).stop 2024-08-18T23:47:16.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.156+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/395373540 shutdown_connections 2024-08-18T23:47:16.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.156+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/395373540 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55bb1e72e7f0 0x55bb1e730c10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.156+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/395373540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb1e6b05e0 0x55bb1e6be460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.156+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/395373540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bb1e6a17f0 0x55bb1e6bdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.156+0000 7fdc7a8c8740 1 --2- 172.21.15.78:0/395373540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bb1e6a0ed0 0x55bb1e6bd9e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.156+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/395373540 >> 172.21.15.78:0/395373540 conn(0x55bb1e69d580 msgr2=0x55bb1e69bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:16.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.156+0000 7fdc59ffb640 0 client.15960 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:16.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.156+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/395373540 shutdown_connections 2024-08-18T23:47:16.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.156+0000 7fdc7a8c8740 1 -- 172.21.15.78:0/395373540 wait complete. 2024-08-18T23:47:16.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.157+0000 7fdc7a8c8740 20 client.15960 trim_cache size 0 max 16384 2024-08-18T23:47:16.160 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:47:16.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.157+0000 7fdc784ca640 1 -- 172.21.15.78:0/2747187664 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdc5c0a2b80 msgr2=0x7fdc5c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:16.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.157+0000 7fdc784ca640 1 --2- 172.21.15.78:0/2747187664 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdc5c0a2b80 0x7fdc5c0a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009990 tx=0x7fdc6c009110 comp rx=0 tx=0).stop 2024-08-18T23:47:16.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.157+0000 7fdc784ca640 1 -- 172.21.15.78:0/2747187664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc70103410 msgr2=0x7fdc701988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:16.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.157+0000 7fdc784ca640 1 --2- 172.21.15.78:0/2747187664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc70103410 0x7fdc701988b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdc60005940 tx=0x7fdc600040c0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.157+0000 7fdc784ca640 1 -- 172.21.15.78:0/2747187664 shutdown_connections 2024-08-18T23:47:16.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.157+0000 7fdc784ca640 1 --2- 172.21.15.78:0/2747187664 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdc5c0a2b80 0x7fdc5c0a5040 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.157+0000 7fdc784ca640 1 --2- 172.21.15.78:0/2747187664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc70107fd0 0x7fdc7019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.157+0000 7fdc784ca640 1 --2- 172.21.15.78:0/2747187664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc70103410 0x7fdc701988b0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.157+0000 7fdc784ca640 1 --2- 172.21.15.78:0/2747187664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc70102a60 0x7fdc70198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:16.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.157+0000 7fdc784ca640 1 -- 172.21.15.78:0/2747187664 >> 172.21.15.78:0/2747187664 conn(0x7fdc700fc500 msgr2=0x7fdc70105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:16.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.157+0000 7fdc784ca640 1 -- 172.21.15.78:0/2747187664 shutdown_connections 2024-08-18T23:47:16.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:16.157+0000 7fdc784ca640 1 -- 172.21.15.78:0/2747187664 wait complete. 2024-08-18T23:47:16.982 INFO:tasks.workunit.client.0.smithi078.stdout: 77 1168743 339 802525 405 610277 1124 2024-08-18T23:47:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:17 smithi161 ceph-mon[22027]: pgmap v439: 129 pgs: 129 active+clean; 410 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.5 MiB/s wr, 370 op/s 2024-08-18T23:47:17.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:17 smithi078 ceph-mon[18440]: pgmap v439: 129 pgs: 129 active+clean; 410 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.5 MiB/s wr, 370 op/s 2024-08-18T23:47:17.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:17 smithi123 ceph-mon[21808]: pgmap v439: 129 pgs: 129 active+clean; 410 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.5 MiB/s wr, 370 op/s 2024-08-18T23:47:18.089 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-18T23:47:18.089 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-18T23:47:18.364 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:47:18.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.894+0000 7fbe52d50640 1 -- 172.21.15.78:0/2994470231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe4c100b50 msgr2=0x7fbe4c100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.894+0000 7fbe52d50640 1 --2- 172.21.15.78:0/2994470231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe4c100b50 0x7fbe4c100f30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009900 tx=0x7fbe4002a480 comp rx=0 tx=0).stop 2024-08-18T23:47:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.894+0000 7fbe52d50640 1 -- 172.21.15.78:0/2994470231 shutdown_connections 2024-08-18T23:47:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.894+0000 7fbe52d50640 1 --2- 172.21.15.78:0/2994470231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe4c102c20 0x7fbe4c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.894+0000 7fbe52d50640 1 --2- 172.21.15.78:0/2994470231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe4c1022a0 0x7fbe4c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.894+0000 7fbe52d50640 1 --2- 172.21.15.78:0/2994470231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe4c100b50 0x7fbe4c100f30 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.894+0000 7fbe52d50640 1 -- 172.21.15.78:0/2994470231 >> 172.21.15.78:0/2994470231 conn(0x7fbe4c0fc500 msgr2=0x7fbe4c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.894+0000 7fbe52d50640 1 -- 172.21.15.78:0/2994470231 shutdown_connections 2024-08-18T23:47:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.894+0000 7fbe52d50640 1 -- 172.21.15.78:0/2994470231 wait complete. 2024-08-18T23:47:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.895+0000 7fbe52d50640 1 Processor -- start 2024-08-18T23:47:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.895+0000 7fbe52d50640 1 -- start start 2024-08-18T23:47:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.895+0000 7fbe52d50640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe4c100b50 0x7fbe4c198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.895+0000 7fbe52d50640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe4c1022a0 0x7fbe4c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.896+0000 7fbe52d50640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe4c102c20 0x7fbe4c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.896+0000 7fbe52d50640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c110c70 con 0x7fbe4c1022a0 2024-08-18T23:47:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.896+0000 7fbe52d50640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c110af0 con 0x7fbe4c102c20 2024-08-18T23:47:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.896+0000 7fbe52d50640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c110df0 con 0x7fbe4c100b50 2024-08-18T23:47:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.896+0000 7fbe4bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe4c1022a0 0x7fbe4c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.896+0000 7fbe50aec640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe4c100b50 0x7fbe4c198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.896+0000 7fbe50aec640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe4c100b50 0x7fbe4c198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:35646/0 (socket says 172.21.15.78:35646) 2024-08-18T23:47:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.896+0000 7fbe4bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe4c1022a0 0x7fbe4c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58040/0 (socket says 172.21.15.78:58040) 2024-08-18T23:47:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.896+0000 7fbe50aec640 1 -- 172.21.15.78:0/926369922 learned_addr learned my addr 172.21.15.78:0/926369922 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:18.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.896+0000 7fbe512ed640 1 --2- 172.21.15.78:0/926369922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe4c102c20 0x7fbe4c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:18.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.896+0000 7fbe50aec640 1 -- 172.21.15.78:0/926369922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe4c102c20 msgr2=0x7fbe4c19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:18.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.896+0000 7fbe50aec640 1 --2- 172.21.15.78:0/926369922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe4c102c20 0x7fbe4c19cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:18.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.896+0000 7fbe50aec640 1 -- 172.21.15.78:0/926369922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe4c1022a0 msgr2=0x7fbe4c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:18.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.896+0000 7fbe50aec640 1 --2- 172.21.15.78:0/926369922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe4c1022a0 0x7fbe4c198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:18.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.896+0000 7fbe50aec640 1 -- 172.21.15.78:0/926369922 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4c19d1c0 con 0x7fbe4c100b50 2024-08-18T23:47:18.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.896+0000 7fbe4bfff640 1 --2- 172.21.15.78:0/926369922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe4c1022a0 0x7fbe4c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:18.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.896+0000 7fbe50aec640 1 --2- 172.21.15.78:0/926369922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe4c100b50 0x7fbe4c198210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbe40000c00 tx=0x7fbe40003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:18.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.896+0000 7fbe49ffb640 1 -- 172.21.15.78:0/926369922 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4003f070 con 0x7fbe4c100b50 2024-08-18T23:47:18.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.897+0000 7fbe52d50640 1 -- 172.21.15.78:0/926369922 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe4c19d450 con 0x7fbe4c100b50 2024-08-18T23:47:18.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.897+0000 7fbe49ffb640 1 -- 172.21.15.78:0/926369922 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe40031c30 con 0x7fbe4c100b50 2024-08-18T23:47:18.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.897+0000 7fbe49ffb640 1 -- 172.21.15.78:0/926369922 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40035030 con 0x7fbe4c100b50 2024-08-18T23:47:18.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.897+0000 7fbe52d50640 1 -- 172.21.15.78:0/926369922 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe4c0fc8e0 con 0x7fbe4c100b50 2024-08-18T23:47:18.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.900+0000 7fbe5514e740 1 Processor -- start 2024-08-18T23:47:18.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.900+0000 7fbe5514e740 1 -- start start 2024-08-18T23:47:18.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.900+0000 7fbe5514e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 0x5615d4536500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:18.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.900+0000 7fbe5514e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4536a40 0x5615d45496b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.900+0000 7fbe5514e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4549bf0 0x5615d454bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.900+0000 7fbe5514e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5615d44f9e40 con 0x5615d4549bf0 2024-08-18T23:47:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.900+0000 7fbe5514e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5615d449b7f0 con 0x5615d4536120 2024-08-18T23:47:18.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.900+0000 7fbe5514e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5615d4442fb0 con 0x5615d4536a40 2024-08-18T23:47:18.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.901+0000 7fbe49ffb640 1 -- 172.21.15.78:0/926369922 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbe40042070 con 0x7fbe4c100b50 2024-08-18T23:47:18.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.901+0000 7fbe4bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4549bf0 0x5615d454bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:18.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.901+0000 7fbe512ed640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 0x5615d4536500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:18.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.901+0000 7fbe512ed640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 0x5615d4536500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:58936/0 (socket says 172.21.15.78:58936) 2024-08-18T23:47:18.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.901+0000 7fbe512ed640 1 -- 172.21.15.78:0/474833189 learned_addr learned my addr 172.21.15.78:0/474833189 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:18.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.901+0000 7fbe4bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4549bf0 0x5615d454bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58052/0 (socket says 172.21.15.78:58052) 2024-08-18T23:47:18.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.901+0000 7fbe50aec640 1 --2- 172.21.15.78:0/474833189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4536a40 0x5615d45496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:18.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.901+0000 7fbe512ed640 1 -- 172.21.15.78:0/474833189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4536a40 msgr2=0x5615d45496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:18.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.901+0000 7fbe512ed640 1 --2- 172.21.15.78:0/474833189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4536a40 0x5615d45496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:18.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.901+0000 7fbe512ed640 1 -- 172.21.15.78:0/474833189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4549bf0 msgr2=0x5615d454bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:18.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.901+0000 7fbe512ed640 1 --2- 172.21.15.78:0/474833189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4549bf0 0x5615d454bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:18.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.901+0000 7fbe512ed640 1 -- 172.21.15.78:0/474833189 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615d420d6a0 con 0x5615d4536120 2024-08-18T23:47:18.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.901+0000 7fbe4bfff640 1 --2- 172.21.15.78:0/474833189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4549bf0 0x5615d454bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:18.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.902+0000 7fbe50aec640 1 --2- 172.21.15.78:0/474833189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4536a40 0x5615d45496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:47:18.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.902+0000 7fbe512ed640 1 --2- 172.21.15.78:0/474833189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 0x5615d4536500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009f50 tx=0x7fbe440118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66f580c8e3df854f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:18.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.902+0000 7fbe36ffd640 1 -- 172.21.15.78:0/474833189 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44004b20 con 0x5615d4536120 2024-08-18T23:47:18.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.902+0000 7fbe49ffb640 1 --2- 172.21.15.78:0/926369922 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbe3c0a2b80 0x7fbe3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:18.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.902+0000 7fbe36ffd640 1 -- 172.21.15.78:0/474833189 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe44004cc0 con 0x5615d4536120 2024-08-18T23:47:18.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.903+0000 7fbe36ffd640 1 -- 172.21.15.78:0/474833189 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44005020 con 0x5615d4536120 2024-08-18T23:47:18.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.903+0000 7fbe50aec640 1 --2- 172.21.15.78:0/926369922 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbe3c0a2b80 0x7fbe3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:18.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.903+0000 7fbe49ffb640 1 -- 172.21.15.78:0/926369922 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fbe400efa50 con 0x7fbe4c100b50 2024-08-18T23:47:18.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.903+0000 7fbe5514e740 1 -- 172.21.15.78:0/474833189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 msgr2=0x5615d4536500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:18.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.903+0000 7fbe50aec640 1 --2- 172.21.15.78:0/926369922 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbe3c0a2b80 0x7fbe3c0a5040 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbe400bbcd0 tx=0x7fbe400f4040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:18.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.903+0000 7fbe5514e740 1 --2- 172.21.15.78:0/474833189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 0x5615d4536500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009f50 tx=0x7fbe440118d0 comp rx=0 tx=0).stop 2024-08-18T23:47:18.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.903+0000 7fbe5514e740 1 -- 172.21.15.78:0/474833189 shutdown_connections 2024-08-18T23:47:18.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.903+0000 7fbe5514e740 1 --2- 172.21.15.78:0/474833189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4549bf0 0x5615d454bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:18.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.903+0000 7fbe5514e740 1 --2- 172.21.15.78:0/474833189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4536a40 0x5615d45496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:18.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.903+0000 7fbe5514e740 1 --2- 172.21.15.78:0/474833189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 0x5615d4536500 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:18.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.903+0000 7fbe5514e740 1 -- 172.21.15.78:0/474833189 >> 172.21.15.78:0/474833189 conn(0x5615d4532790 msgr2=0x5615d4532b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:18.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.904+0000 7fbe5514e740 1 -- 172.21.15.78:0/474833189 shutdown_connections 2024-08-18T23:47:18.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.904+0000 7fbe5514e740 1 -- 172.21.15.78:0/474833189 wait complete. 2024-08-18T23:47:18.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.904+0000 7fbe5514e740 1 -- start start 2024-08-18T23:47:18.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.904+0000 7fbe5514e740 1 Processor -- start 2024-08-18T23:47:18.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.905+0000 7fbe5514e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536120 0x5615d4546c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:18.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.905+0000 7fbe5514e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4536a40 0x5615d4547160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:18.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.905+0000 7fbe5514e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4549bf0 0x5615d45476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:18.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.905+0000 7fbe512ed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536120 0x5615d4546c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:18.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.905+0000 7fbe512ed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536120 0x5615d4546c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58064/0 (socket says 172.21.15.78:58064) 2024-08-18T23:47:18.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.905+0000 7fbe512ed640 1 -- 172.21.15.78:0/2019431176 learned_addr learned my addr 172.21.15.78:0/2019431176 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:18.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.905+0000 7fbe5514e740 1 -- 172.21.15.78:0/2019431176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5615d4456cb0 con 0x5615d4536120 2024-08-18T23:47:18.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.905+0000 7fbe5514e740 1 -- 172.21.15.78:0/2019431176 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5615d44977b0 con 0x5615d4549bf0 2024-08-18T23:47:18.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.905+0000 7fbe5514e740 1 -- 172.21.15.78:0/2019431176 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5615d454c760 con 0x5615d4536a40 2024-08-18T23:47:18.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.905+0000 7fbe4bfff640 1 --2- 172.21.15.78:0/2019431176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4536a40 0x5615d4547160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:18.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.905+0000 7fbe512ed640 1 --2- 172.21.15.78:0/2019431176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4549bf0 0x5615d45476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:18.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.905+0000 7fbe5514e740 10 client.0 _ll_register_callbacks cb 0x5615d41aaba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:18.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.905+0000 7fbe512ed640 1 -- 172.21.15.78:0/2019431176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4536a40 msgr2=0x5615d4547160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:18.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.905+0000 7fbe512ed640 1 --2- 172.21.15.78:0/2019431176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4536a40 0x5615d4547160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:18.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.905+0000 7fbe512ed640 1 -- 172.21.15.78:0/2019431176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4549bf0 msgr2=0x5615d45476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:18.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.905+0000 7fbe512ed640 1 --2- 172.21.15.78:0/2019431176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4549bf0 0x5615d45476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:18.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.905+0000 7fbe512ed640 1 -- 172.21.15.78:0/2019431176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615d4455bc0 con 0x5615d4536120 2024-08-18T23:47:18.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.906+0000 7fbe5514e740 1 -- 172.21.15.78:0/2019431176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe44004cc0 con 0x5615d4536120 2024-08-18T23:47:18.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.906+0000 7fbe512ed640 1 --2- 172.21.15.78:0/2019431176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536120 0x5615d4546c20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x5615d4549bc0 tx=0x7fbe440045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2b28c266571548 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:18.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.906+0000 7fbe2bfff640 10 client.15972 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:47:18.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.906+0000 7fbe2bfff640 1 -- 172.21.15.78:0/2019431176 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4402c020 con 0x5615d4536120 2024-08-18T23:47:18.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.906+0000 7fbe2bfff640 1 -- 172.21.15.78:0/2019431176 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fbe44003c40 con 0x5615d4536120 2024-08-18T23:47:18.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.906+0000 7fbe2bfff640 1 -- 172.21.15.78:0/2019431176 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbe44028450 con 0x5615d4536120 2024-08-18T23:47:18.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.906+0000 7fbe5514e740 10 client.15972 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:18.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.906+0000 7fbe5514e740 1 -- 172.21.15.78:0/2019431176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5615d4558550 con 0x5615d4536120 2024-08-18T23:47:18.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.907+0000 7fbe2bfff640 1 -- 172.21.15.78:0/2019431176 <== mon.0 v2:172.21.15.78:3300/0 4 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fbe440285f0 con 0x5615d4536120 2024-08-18T23:47:18.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.907+0000 7fbe5514e740 10 client.15972 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:18.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.907+0000 7fbe5514e740 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:47:18.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.907+0000 7fbe5514e740 20 client.15972 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:18.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.907+0000 7fbe5514e740 1 --2- 172.21.15.78:0/2019431176 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5615d4559900 0x5615d455bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:18.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.907+0000 7fbe5514e740 4 client.15972 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:47:18.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.907+0000 7fbe5514e740 1 -- 172.21.15.78:0/2019431176 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615d44a8d60 con 0x5615d4559900 2024-08-18T23:47:18.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.907+0000 7fbe4bfff640 1 --2- 172.21.15.78:0/2019431176 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5615d4559900 0x5615d455bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:18.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.907+0000 7fbe2bfff640 10 client.15972 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:18.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.907+0000 7fbe4bfff640 1 --2- 172.21.15.78:0/2019431176 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5615d4559900 0x5615d455bcf0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1207b25326a3a29 server_cookie=daa85eafdd22dfa7 in_seq=0 out_seq=0 2024-08-18T23:47:19.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.912+0000 7fbe2bfff640 1 -- 172.21.15.78:0/2019431176 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5615d44a8d60 con 0x5615d4559900 2024-08-18T23:47:19.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.912+0000 7fbe2bfff640 10 client.15972 handle_command_reply: tid=0 2024-08-18T23:47:19.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.912+0000 7fbe5514e740 1 client.15972 shutdown 2024-08-18T23:47:19.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.912+0000 7fbe5514e740 1 -- 172.21.15.78:0/2019431176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536120 msgr2=0x5615d4546c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:19.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.912+0000 7fbe5514e740 1 --2- 172.21.15.78:0/2019431176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536120 0x5615d4546c20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x5615d4549bc0 tx=0x7fbe440045e0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.912+0000 7fbe5514e740 1 -- 172.21.15.78:0/2019431176 shutdown_connections 2024-08-18T23:47:19.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.912+0000 7fbe5514e740 1 --2- 172.21.15.78:0/2019431176 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5615d4559900 0x5615d455bcf0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.912+0000 7fbe5514e740 1 --2- 172.21.15.78:0/2019431176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4549bf0 0x5615d45476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.912+0000 7fbe5514e740 1 --2- 172.21.15.78:0/2019431176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4536a40 0x5615d4547160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.912+0000 7fbe5514e740 1 --2- 172.21.15.78:0/2019431176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536120 0x5615d4546c20 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.912+0000 7fbe2bfff640 0 client.15972 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:19.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.912+0000 7fbe5514e740 1 -- 172.21.15.78:0/2019431176 >> 172.21.15.78:0/2019431176 conn(0x5615d4532790 msgr2=0x5615d4538420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:19.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.913+0000 7fbe5514e740 1 -- 172.21.15.78:0/2019431176 shutdown_connections 2024-08-18T23:47:19.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.913+0000 7fbe5514e740 1 -- 172.21.15.78:0/2019431176 wait complete. 2024-08-18T23:47:19.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.913+0000 7fbe5514e740 20 client.15972 trim_cache size 0 max 16384 2024-08-18T23:47:19.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.936+0000 7fbe5514e740 1 Processor -- start 2024-08-18T23:47:19.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.936+0000 7fbe5514e740 1 -- start start 2024-08-18T23:47:19.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.936+0000 7fbe5514e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 0x5615d45320e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:19.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.936+0000 7fbe5514e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536a40 0x5615d4561540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:19.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.937+0000 7fbe5514e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4549bf0 0x5615d4561a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:19.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.937+0000 7fbe5514e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c1af080 con 0x5615d4536a40 2024-08-18T23:47:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.937+0000 7fbe5514e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c1ac950 con 0x5615d4536120 2024-08-18T23:47:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.937+0000 7fbe5514e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c1aa7c0 con 0x5615d4549bf0 2024-08-18T23:47:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.937+0000 7fbe4bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536a40 0x5615d4561540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.937+0000 7fbe512ed640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 0x5615d45320e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.937+0000 7fbe4bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536a40 0x5615d4561540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58080/0 (socket says 172.21.15.78:58080) 2024-08-18T23:47:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.937+0000 7fbe4bfff640 1 -- 172.21.15.78:0/1843706887 learned_addr learned my addr 172.21.15.78:0/1843706887 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.937+0000 7fbe512ed640 1 --2- 172.21.15.78:0/1843706887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4549bf0 0x5615d4561a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.937+0000 7fbe4bfff640 1 -- 172.21.15.78:0/1843706887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4549bf0 msgr2=0x5615d4561a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.937+0000 7fbe4bfff640 1 --2- 172.21.15.78:0/1843706887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4549bf0 0x5615d4561a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.937+0000 7fbe4bfff640 1 -- 172.21.15.78:0/1843706887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 msgr2=0x5615d45320e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.937+0000 7fbe4bfff640 1 --2- 172.21.15.78:0/1843706887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 0x5615d45320e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.937+0000 7fbe4bfff640 1 -- 172.21.15.78:0/1843706887 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615d454c350 con 0x5615d4536a40 2024-08-18T23:47:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.937+0000 7fbe512ed640 1 --2- 172.21.15.78:0/1843706887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 0x5615d45320e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.938+0000 7fbe4bfff640 1 --2- 172.21.15.78:0/1843706887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536a40 0x5615d4561540 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbe38005d00 tx=0x7fbe380107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7edde38838e353 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.938+0000 7fbe367fc640 1 -- 172.21.15.78:0/1843706887 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38023030 con 0x5615d4536a40 2024-08-18T23:47:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.938+0000 7fbe367fc640 1 -- 172.21.15.78:0/1843706887 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe38010e40 con 0x5615d4536a40 2024-08-18T23:47:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.938+0000 7fbe367fc640 1 -- 172.21.15.78:0/1843706887 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38004c70 con 0x5615d4536a40 2024-08-18T23:47:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.939+0000 7fbe5514e740 1 -- 172.21.15.78:0/1843706887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536a40 msgr2=0x5615d4561540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.939+0000 7fbe5514e740 1 --2- 172.21.15.78:0/1843706887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536a40 0x5615d4561540 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbe38005d00 tx=0x7fbe380107a0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.939+0000 7fbe5514e740 1 -- 172.21.15.78:0/1843706887 shutdown_connections 2024-08-18T23:47:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.939+0000 7fbe5514e740 1 --2- 172.21.15.78:0/1843706887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4549bf0 0x5615d4561a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.939+0000 7fbe5514e740 1 --2- 172.21.15.78:0/1843706887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536a40 0x5615d4561540 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.939+0000 7fbe5514e740 1 --2- 172.21.15.78:0/1843706887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 0x5615d45320e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.939+0000 7fbe5514e740 1 -- 172.21.15.78:0/1843706887 >> 172.21.15.78:0/1843706887 conn(0x5615d4532790 msgr2=0x5615d4547400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.939+0000 7fbe5514e740 1 -- 172.21.15.78:0/1843706887 shutdown_connections 2024-08-18T23:47:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.939+0000 7fbe5514e740 1 -- 172.21.15.78:0/1843706887 wait complete. 2024-08-18T23:47:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.940+0000 7fbe5514e740 1 -- start start 2024-08-18T23:47:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.940+0000 7fbe5514e740 1 Processor -- start 2024-08-18T23:47:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.940+0000 7fbe5514e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 0x5615d4552280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.940+0000 7fbe5514e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536a40 0x5615d4543a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.940+0000 7fbe5514e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4549bf0 0x5615d4544090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.940+0000 7fbe5514e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c1aae10 con 0x5615d4536a40 2024-08-18T23:47:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.940+0000 7fbe5514e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c1ac280 con 0x5615d4536120 2024-08-18T23:47:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.940+0000 7fbe5514e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c1ab4e0 con 0x5615d4549bf0 2024-08-18T23:47:19.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.940+0000 7fbe5514e740 10 client.0 _ll_register_callbacks cb 0x7fbe38011c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:19.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.941+0000 7fbe512ed640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 0x5615d4552280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:19.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.941+0000 7fbe512ed640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4549bf0 0x5615d4544090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:19.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.941+0000 7fbe4bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536a40 0x5615d4543a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:19.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.941+0000 7fbe512ed640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 0x5615d4552280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:58962/0 (socket says 172.21.15.78:58962) 2024-08-18T23:47:19.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.941+0000 7fbe4bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536a40 0x5615d4543a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58088/0 (socket says 172.21.15.78:58088) 2024-08-18T23:47:19.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.941+0000 7fbe512ed640 1 -- 172.21.15.78:0/3927282668 learned_addr learned my addr 172.21.15.78:0/3927282668 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:19.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.941+0000 7fbe512ed640 1 -- 172.21.15.78:0/3927282668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4549bf0 msgr2=0x5615d4544090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:19.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.941+0000 7fbe512ed640 1 --2- 172.21.15.78:0/3927282668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4549bf0 0x5615d4544090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.941+0000 7fbe512ed640 1 -- 172.21.15.78:0/3927282668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536a40 msgr2=0x5615d4543a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:19.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.941+0000 7fbe512ed640 1 --2- 172.21.15.78:0/3927282668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536a40 0x5615d4543a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.941+0000 7fbe512ed640 1 -- 172.21.15.78:0/3927282668 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615d45448c0 con 0x5615d4536120 2024-08-18T23:47:19.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.941+0000 7fbe512ed640 1 --2- 172.21.15.78:0/3927282668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 0x5615d4552280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbe44006010 tx=0x7fbe44020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c170d2dea68826ba server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:19.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.941+0000 7fbe2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:47:19.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.942+0000 7fbe2bfff640 1 -- 172.21.15.78:0/3927282668 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4402c070 con 0x5615d4536120 2024-08-18T23:47:19.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.942+0000 7fbe5514e740 1 -- 172.21.15.78:0/3927282668 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe38010e40 con 0x5615d4536120 2024-08-18T23:47:19.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.942+0000 7fbe2bfff640 1 -- 172.21.15.78:0/3927282668 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fbe44004100 con 0x5615d4536120 2024-08-18T23:47:19.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.942+0000 7fbe2bfff640 1 -- 172.21.15.78:0/3927282668 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbe4404b050 con 0x5615d4536120 2024-08-18T23:47:19.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.942+0000 7fbe5514e740 10 client.25427 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:19.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.942+0000 7fbe5514e740 1 -- 172.21.15.78:0/3927282668 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5615d45c4680 con 0x5615d4536120 2024-08-18T23:47:19.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.942+0000 7fbe2bfff640 1 -- 172.21.15.78:0/3927282668 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe38010e40 con 0x5615d4536120 2024-08-18T23:47:19.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.943+0000 7fbe2bfff640 1 -- 172.21.15.78:0/3927282668 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fbe44046030 con 0x5615d4536120 2024-08-18T23:47:19.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.943+0000 7fbe5514e740 10 client.25427 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:19.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.943+0000 7fbe5514e740 10 client.25427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:47:19.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.943+0000 7fbe5514e740 20 client.25427 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:19.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.943+0000 7fbe5514e740 1 --2- 172.21.15.78:0/3927282668 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5615d45c4d40 0x5615d45c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:19.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.943+0000 7fbe5514e740 4 client.25427 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-18T23:47:19.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.943+0000 7fbe5514e740 1 -- 172.21.15.78:0/3927282668 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5615d44a3270 con 0x5615d45c4d40 2024-08-18T23:47:19.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.943+0000 7fbe4bfff640 1 --2- 172.21.15.78:0/3927282668 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5615d45c4d40 0x5615d45c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:19.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.944+0000 7fbe4bfff640 1 --2- 172.21.15.78:0/3927282668 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5615d45c4d40 0x5615d45c7130 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa33a3831f9c08a4 server_cookie=70f4ee6efcd17532 in_seq=0 out_seq=0 2024-08-18T23:47:19.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.944+0000 7fbe2bfff640 10 client.25427 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:19.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.949+0000 7fbe2bfff640 1 -- 172.21.15.78:0/3927282668 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5615d44a3270 con 0x5615d45c4d40 2024-08-18T23:47:19.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.949+0000 7fbe2bfff640 10 client.25427 handle_command_reply: tid=0 2024-08-18T23:47:19.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.949+0000 7fbe5514e740 1 client.25427 shutdown 2024-08-18T23:47:19.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.949+0000 7fbe5514e740 1 -- 172.21.15.78:0/3927282668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 msgr2=0x5615d4552280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:19.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.949+0000 7fbe5514e740 1 --2- 172.21.15.78:0/3927282668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 0x5615d4552280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbe44006010 tx=0x7fbe44020000 comp rx=0 tx=0).stop 2024-08-18T23:47:19.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.949+0000 7fbe5514e740 1 -- 172.21.15.78:0/3927282668 shutdown_connections 2024-08-18T23:47:19.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.949+0000 7fbe5514e740 1 --2- 172.21.15.78:0/3927282668 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5615d45c4d40 0x5615d45c7130 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.949+0000 7fbe5514e740 1 --2- 172.21.15.78:0/3927282668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615d4549bf0 0x5615d4544090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.949+0000 7fbe5514e740 1 --2- 172.21.15.78:0/3927282668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615d4536a40 0x5615d4543a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.949+0000 7fbe5514e740 1 --2- 172.21.15.78:0/3927282668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d4536120 0x5615d4552280 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.949+0000 7fbe2bfff640 0 client.25427 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:19.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.949+0000 7fbe5514e740 1 -- 172.21.15.78:0/3927282668 >> 172.21.15.78:0/3927282668 conn(0x5615d4532790 msgr2=0x5615d4546be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:19.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.950+0000 7fbe5514e740 1 -- 172.21.15.78:0/3927282668 shutdown_connections 2024-08-18T23:47:19.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.950+0000 7fbe5514e740 1 -- 172.21.15.78:0/3927282668 wait complete. 2024-08-18T23:47:19.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.950+0000 7fbe5514e740 20 client.25427 trim_cache size 0 max 16384 2024-08-18T23:47:19.013 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:47:19.013 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-08-18T23:47:19.013 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "e1c540b0-f677-4ba3-b186-e94e8b511070", 2024-08-18T23:47:19.013 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-08-18T23:47:19.013 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:47:19.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.951+0000 7fbe52d50640 1 -- 172.21.15.78:0/926369922 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbe3c0a2b80 msgr2=0x7fbe3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:19.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.951+0000 7fbe52d50640 1 --2- 172.21.15.78:0/926369922 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbe3c0a2b80 0x7fbe3c0a5040 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbe400bbcd0 tx=0x7fbe400f4040 comp rx=0 tx=0).stop 2024-08-18T23:47:19.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.951+0000 7fbe52d50640 1 -- 172.21.15.78:0/926369922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe4c100b50 msgr2=0x7fbe4c198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:19.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.951+0000 7fbe52d50640 1 --2- 172.21.15.78:0/926369922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe4c100b50 0x7fbe4c198210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbe40000c00 tx=0x7fbe40003c10 comp rx=0 tx=0).stop 2024-08-18T23:47:19.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.951+0000 7fbe52d50640 1 -- 172.21.15.78:0/926369922 shutdown_connections 2024-08-18T23:47:19.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.951+0000 7fbe52d50640 1 --2- 172.21.15.78:0/926369922 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbe3c0a2b80 0x7fbe3c0a5040 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.951+0000 7fbe52d50640 1 --2- 172.21.15.78:0/926369922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe4c102c20 0x7fbe4c19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.951+0000 7fbe52d50640 1 --2- 172.21.15.78:0/926369922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe4c1022a0 0x7fbe4c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.951+0000 7fbe52d50640 1 --2- 172.21.15.78:0/926369922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe4c100b50 0x7fbe4c198210 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:19.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.951+0000 7fbe52d50640 1 -- 172.21.15.78:0/926369922 >> 172.21.15.78:0/926369922 conn(0x7fbe4c0fc500 msgr2=0x7fbe4c069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:19.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.951+0000 7fbe52d50640 1 -- 172.21.15.78:0/926369922 shutdown_connections 2024-08-18T23:47:19.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:18.951+0000 7fbe52d50640 1 -- 172.21.15.78:0/926369922 wait complete. 2024-08-18T23:47:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:19 smithi161 ceph-mon[22027]: pgmap v440: 129 pgs: 129 active+clean; 409 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 319 op/s 2024-08-18T23:47:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:19 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:47:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:19 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:19.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:19 smithi078 ceph-mon[18440]: pgmap v440: 129 pgs: 129 active+clean; 409 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 319 op/s 2024-08-18T23:47:19.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:19 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:47:19.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:19 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:19.767 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:47:19.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:19 smithi123 ceph-mon[21808]: pgmap v440: 129 pgs: 129 active+clean; 409 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 319 op/s 2024-08-18T23:47:19.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:19 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:47:19.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:19 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:20.044 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:47:20.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:20 smithi078 ceph-mon[18440]: scrub queued for path: / 2024-08-18T23:47:20.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:20 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:20.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:20 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:47:20.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:20 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:47:20.468 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:20 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:20.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:20 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:47:20.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:20 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:20.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:20 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:47:20.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:20 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:20.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:20 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:47:20.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:20 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:20.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:20 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:47:20.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:20 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:20.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.555+0000 7fc44c373640 1 -- 172.21.15.78:0/4077108293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc444102c20 msgr2=0x7fc44410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:20.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.555+0000 7fc44c373640 1 --2- 172.21.15.78:0/4077108293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc444102c20 0x7fc44410d770 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009ca0 tx=0x7fc43c02a950 comp rx=0 tx=0).stop 2024-08-18T23:47:20.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.555+0000 7fc44c373640 1 -- 172.21.15.78:0/4077108293 shutdown_connections 2024-08-18T23:47:20.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.555+0000 7fc44c373640 1 --2- 172.21.15.78:0/4077108293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc444102c20 0x7fc44410d770 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.555+0000 7fc44c373640 1 --2- 172.21.15.78:0/4077108293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4441022a0 0x7fc4441026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.555+0000 7fc44c373640 1 --2- 172.21.15.78:0/4077108293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc444100b50 0x7fc444100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.555+0000 7fc44c373640 1 -- 172.21.15.78:0/4077108293 >> 172.21.15.78:0/4077108293 conn(0x7fc4440fc500 msgr2=0x7fc4440fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:20.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.555+0000 7fc44c373640 1 -- 172.21.15.78:0/4077108293 shutdown_connections 2024-08-18T23:47:20.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.555+0000 7fc44c373640 1 -- 172.21.15.78:0/4077108293 wait complete. 2024-08-18T23:47:20.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.556+0000 7fc44c373640 1 Processor -- start 2024-08-18T23:47:20.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.556+0000 7fc44c373640 1 -- start start 2024-08-18T23:47:20.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.556+0000 7fc44c373640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc444100b50 0x7fc4441981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:20.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.556+0000 7fc44c373640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4441022a0 0x7fc4441986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:20.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.556+0000 7fc44c373640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc444102c20 0x7fc44419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:20.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.556+0000 7fc44c373640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc444110cb0 con 0x7fc444102c20 2024-08-18T23:47:20.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.556+0000 7fc44c373640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc444110b30 con 0x7fc4441022a0 2024-08-18T23:47:20.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.556+0000 7fc44c373640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc444110e30 con 0x7fc444100b50 2024-08-18T23:47:20.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.556+0000 7fc44a910640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc444102c20 0x7fc44419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:20.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.556+0000 7fc44a910640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc444102c20 0x7fc44419ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58106/0 (socket says 172.21.15.78:58106) 2024-08-18T23:47:20.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.556+0000 7fc44a910640 1 -- 172.21.15.78:0/572565916 learned_addr learned my addr 172.21.15.78:0/572565916 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:20.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.556+0000 7fc44990e640 1 --2- 172.21.15.78:0/572565916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4441022a0 0x7fc4441986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:20.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.556+0000 7fc44a10f640 1 --2- 172.21.15.78:0/572565916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc444100b50 0x7fc4441981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:20.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.556+0000 7fc44a910640 1 -- 172.21.15.78:0/572565916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc444100b50 msgr2=0x7fc4441981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:20.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.556+0000 7fc44a910640 1 --2- 172.21.15.78:0/572565916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc444100b50 0x7fc4441981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.556+0000 7fc44a910640 1 -- 172.21.15.78:0/572565916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4441022a0 msgr2=0x7fc4441986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:20.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.556+0000 7fc44a910640 1 --2- 172.21.15.78:0/572565916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4441022a0 0x7fc4441986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.556+0000 7fc44a910640 1 -- 172.21.15.78:0/572565916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44419d150 con 0x7fc444102c20 2024-08-18T23:47:20.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.556+0000 7fc44990e640 1 --2- 172.21.15.78:0/572565916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4441022a0 0x7fc4441986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:20.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.557+0000 7fc44a910640 1 --2- 172.21.15.78:0/572565916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc444102c20 0x7fc44419ca70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc43c004af0 tx=0x7fc43c02f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:20.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.557+0000 7fc43b7fe640 1 -- 172.21.15.78:0/572565916 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c034040 con 0x7fc444102c20 2024-08-18T23:47:20.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.557+0000 7fc44c373640 1 -- 172.21.15.78:0/572565916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc44419d3e0 con 0x7fc444102c20 2024-08-18T23:47:20.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.557+0000 7fc43b7fe640 1 -- 172.21.15.78:0/572565916 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc43c030920 con 0x7fc444102c20 2024-08-18T23:47:20.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.557+0000 7fc43b7fe640 1 -- 172.21.15.78:0/572565916 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c004630 con 0x7fc444102c20 2024-08-18T23:47:20.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.557+0000 7fc44c373640 1 -- 172.21.15.78:0/572565916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4441a4d20 con 0x7fc444102c20 2024-08-18T23:47:20.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.559+0000 7fc43b7fe640 1 -- 172.21.15.78:0/572565916 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc43c037020 con 0x7fc444102c20 2024-08-18T23:47:20.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.560+0000 7fc43b7fe640 1 --2- 172.21.15.78:0/572565916 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc4300a2b80 0x7fc4300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:20.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.560+0000 7fc44e771740 1 Processor -- start 2024-08-18T23:47:20.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.560+0000 7fc44a10f640 1 --2- 172.21.15.78:0/572565916 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc4300a2b80 0x7fc4300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:20.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.560+0000 7fc43b7fe640 1 -- 172.21.15.78:0/572565916 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc43c0f0110 con 0x7fc444102c20 2024-08-18T23:47:20.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.560+0000 7fc44e771740 1 -- start start 2024-08-18T23:47:20.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.560+0000 7fc44e771740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 0x557cc881b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:20.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.560+0000 7fc44e771740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 0x557cc8825eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:20.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.560+0000 7fc44e771740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc88263f0 0x557cc88287e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:20.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.560+0000 7fc44e771740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557cc8780980 con 0x557cc881bb30 2024-08-18T23:47:20.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.560+0000 7fc44e771740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557cc8718a70 con 0x557cc88263f0 2024-08-18T23:47:20.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.560+0000 7fc44e771740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557cc877c940 con 0x557cc881b210 2024-08-18T23:47:20.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.560+0000 7fc44a10f640 1 --2- 172.21.15.78:0/572565916 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc4300a2b80 0x7fc4300a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc440009bd0 tx=0x7fc440005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:20.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.560+0000 7fc44990e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 0x557cc8825eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:20.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.560+0000 7fc44990e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 0x557cc8825eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58118/0 (socket says 172.21.15.78:58118) 2024-08-18T23:47:20.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.560+0000 7fc44990e640 1 -- 172.21.15.78:0/2035680001 learned_addr learned my addr 172.21.15.78:0/2035680001 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:20.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.561+0000 7fc44a910640 1 --2- 172.21.15.78:0/2035680001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc88263f0 0x557cc88287e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:20.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.561+0000 7fc44a10f640 1 --2- 172.21.15.78:0/2035680001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 0x557cc881b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:20.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.561+0000 7fc44990e640 1 -- 172.21.15.78:0/2035680001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 msgr2=0x557cc881b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:20.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.561+0000 7fc44990e640 1 --2- 172.21.15.78:0/2035680001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 0x557cc881b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.561+0000 7fc44990e640 1 -- 172.21.15.78:0/2035680001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc88263f0 msgr2=0x557cc88287e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:20.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.561+0000 7fc44990e640 1 --2- 172.21.15.78:0/2035680001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc88263f0 0x557cc88287e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.561+0000 7fc44990e640 1 -- 172.21.15.78:0/2035680001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cc877b6e0 con 0x557cc881bb30 2024-08-18T23:47:20.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.561+0000 7fc44a910640 1 --2- 172.21.15.78:0/2035680001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc88263f0 0x557cc88287e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:47:20.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.561+0000 7fc44990e640 1 --2- 172.21.15.78:0/2035680001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 0x557cc8825eb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc4340114f0 tx=0x7fc4340119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc398249d664e6c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:20.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.561+0000 7fc438dea640 1 -- 172.21.15.78:0/2035680001 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43401e070 con 0x557cc881bb30 2024-08-18T23:47:20.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.561+0000 7fc438dea640 1 -- 172.21.15.78:0/2035680001 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4340035f0 con 0x557cc881bb30 2024-08-18T23:47:20.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.561+0000 7fc438dea640 1 -- 172.21.15.78:0/2035680001 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43400f7c0 con 0x557cc881bb30 2024-08-18T23:47:20.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.562+0000 7fc44e771740 1 -- 172.21.15.78:0/2035680001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 msgr2=0x557cc8825eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:20.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.562+0000 7fc44e771740 1 --2- 172.21.15.78:0/2035680001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 0x557cc8825eb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc4340114f0 tx=0x7fc4340119b0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.562+0000 7fc44e771740 1 -- 172.21.15.78:0/2035680001 shutdown_connections 2024-08-18T23:47:20.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.562+0000 7fc44e771740 1 --2- 172.21.15.78:0/2035680001 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc88263f0 0x557cc88287e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.562+0000 7fc44e771740 1 --2- 172.21.15.78:0/2035680001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 0x557cc8825eb0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.562+0000 7fc44e771740 1 --2- 172.21.15.78:0/2035680001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 0x557cc881b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.562+0000 7fc44e771740 1 -- 172.21.15.78:0/2035680001 >> 172.21.15.78:0/2035680001 conn(0x557cc88177a0 msgr2=0x557cc8817b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:20.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.562+0000 7fc44e771740 1 -- 172.21.15.78:0/2035680001 shutdown_connections 2024-08-18T23:47:20.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.562+0000 7fc44e771740 1 -- 172.21.15.78:0/2035680001 wait complete. 2024-08-18T23:47:20.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.562+0000 7fc44e771740 1 -- start start 2024-08-18T23:47:20.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc44e771740 1 Processor -- start 2024-08-18T23:47:20.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc44e771740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 0x557cc88232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:20.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc44e771740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 0x557cc8823800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:20.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc44e771740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc88263f0 0x557cc8823d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:20.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc44e771740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557cc8828fc0 con 0x557cc881bb30 2024-08-18T23:47:20.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc44e771740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557cc873baa0 con 0x557cc88263f0 2024-08-18T23:47:20.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc44e771740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557cc8829140 con 0x557cc881b210 2024-08-18T23:47:20.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc44e771740 10 client.0 _ll_register_callbacks cb 0x557cc872ae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:20.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc44990e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 0x557cc8823800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:20.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc44990e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 0x557cc8823800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58132/0 (socket says 172.21.15.78:58132) 2024-08-18T23:47:20.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc44990e640 1 -- 172.21.15.78:0/1042683500 learned_addr learned my addr 172.21.15.78:0/1042683500 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:20.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc44a910640 1 --2- 172.21.15.78:0/1042683500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc88263f0 0x557cc8823d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:20.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc44a10f640 1 --2- 172.21.15.78:0/1042683500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 0x557cc88232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:20.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc44990e640 1 -- 172.21.15.78:0/1042683500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 msgr2=0x557cc88232c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:20.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc44990e640 1 --2- 172.21.15.78:0/1042683500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 0x557cc88232c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc44990e640 1 -- 172.21.15.78:0/1042683500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc88263f0 msgr2=0x557cc8823d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:20.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc44990e640 1 --2- 172.21.15.78:0/1042683500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc88263f0 0x557cc8823d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc44990e640 1 -- 172.21.15.78:0/1042683500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cc871f650 con 0x557cc881bb30 2024-08-18T23:47:20.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc44990e640 1 --2- 172.21.15.78:0/1042683500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 0x557cc8823800 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc43401c040 tx=0x7fc434003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6961c7c0ff7d3521 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:20.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.563+0000 7fc42dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:47:20.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.564+0000 7fc42dffb640 1 -- 172.21.15.78:0/1042683500 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434026020 con 0x557cc881bb30 2024-08-18T23:47:20.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.564+0000 7fc42dffb640 1 -- 172.21.15.78:0/1042683500 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc434005040 con 0x557cc881bb30 2024-08-18T23:47:20.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.564+0000 7fc44e771740 1 -- 172.21.15.78:0/1042683500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4340035f0 con 0x557cc881bb30 2024-08-18T23:47:20.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.564+0000 7fc42dffb640 1 -- 172.21.15.78:0/1042683500 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc434031050 con 0x557cc881bb30 2024-08-18T23:47:20.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.564+0000 7fc44e771740 10 client.15990 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:20.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.564+0000 7fc44e771740 1 -- 172.21.15.78:0/1042683500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557cc84f23c0 con 0x557cc881bb30 2024-08-18T23:47:20.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.564+0000 7fc42dffb640 1 -- 172.21.15.78:0/1042683500 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4340035f0 con 0x557cc881bb30 2024-08-18T23:47:20.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.564+0000 7fc42dffb640 1 -- 172.21.15.78:0/1042683500 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fc434021030 con 0x557cc881bb30 2024-08-18T23:47:20.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.564+0000 7fc44e771740 10 client.15990 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:20.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.564+0000 7fc44e771740 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:47:20.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.564+0000 7fc44e771740 20 client.15990 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:20.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.564+0000 7fc44e771740 1 --2- 172.21.15.78:0/1042683500 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x557cc8835f10 0x557cc8838300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:20.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.564+0000 7fc44e771740 4 client.15990 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:47:20.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.564+0000 7fc44e771740 1 -- 172.21.15.78:0/1042683500 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cc878def0 con 0x557cc8835f10 2024-08-18T23:47:20.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.564+0000 7fc44a10f640 1 --2- 172.21.15.78:0/1042683500 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x557cc8835f10 0x557cc8838300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:20.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.565+0000 7fc42dffb640 10 client.15990 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:20.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.565+0000 7fc44a10f640 1 --2- 172.21.15.78:0/1042683500 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x557cc8835f10 0x557cc8838300 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cad8388c47735d06 server_cookie=684ea9bd679abc84 in_seq=0 out_seq=0 2024-08-18T23:47:20.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.591+0000 7fc42dffb640 1 -- 172.21.15.78:0/1042683500 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557cc878def0 con 0x557cc8835f10 2024-08-18T23:47:20.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.591+0000 7fc42dffb640 10 client.15990 handle_command_reply: tid=0 2024-08-18T23:47:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.591+0000 7fc44e771740 1 client.15990 shutdown 2024-08-18T23:47:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.591+0000 7fc44e771740 1 -- 172.21.15.78:0/1042683500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 msgr2=0x557cc8823800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.591+0000 7fc44e771740 1 --2- 172.21.15.78:0/1042683500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 0x557cc8823800 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc43401c040 tx=0x7fc434003a50 comp rx=0 tx=0).stop 2024-08-18T23:47:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.591+0000 7fc44e771740 1 -- 172.21.15.78:0/1042683500 shutdown_connections 2024-08-18T23:47:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.591+0000 7fc44e771740 1 --2- 172.21.15.78:0/1042683500 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x557cc8835f10 0x557cc8838300 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.591+0000 7fc42dffb640 0 client.15990 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.591+0000 7fc44e771740 1 --2- 172.21.15.78:0/1042683500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc88263f0 0x557cc8823d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.591+0000 7fc44e771740 1 --2- 172.21.15.78:0/1042683500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 0x557cc8823800 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.591+0000 7fc44e771740 1 --2- 172.21.15.78:0/1042683500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 0x557cc88232c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.591+0000 7fc44e771740 1 -- 172.21.15.78:0/1042683500 >> 172.21.15.78:0/1042683500 conn(0x557cc88177a0 msgr2=0x557cc881d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.591+0000 7fc44e771740 1 -- 172.21.15.78:0/1042683500 shutdown_connections 2024-08-18T23:47:20.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.591+0000 7fc44e771740 1 -- 172.21.15.78:0/1042683500 wait complete. 2024-08-18T23:47:20.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.591+0000 7fc44e771740 20 client.15990 trim_cache size 0 max 16384 2024-08-18T23:47:20.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.610+0000 7fc44e771740 1 Processor -- start 2024-08-18T23:47:20.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.610+0000 7fc44e771740 1 -- start start 2024-08-18T23:47:20.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.610+0000 7fc44e771740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc88177a0 0x557cc883db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:20.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.610+0000 7fc44e771740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 0x557cc883e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:20.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.610+0000 7fc44e771740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 0x557cc88160e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:20.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.610+0000 7fc44e771740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441b1150 con 0x557cc881bb30 2024-08-18T23:47:20.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.610+0000 7fc44e771740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441aea20 con 0x557cc88177a0 2024-08-18T23:47:20.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.610+0000 7fc44e771740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441ac870 con 0x557cc881b210 2024-08-18T23:47:20.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.610+0000 7fc44a910640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 0x557cc88160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:20.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.610+0000 7fc44a910640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 0x557cc88160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58140/0 (socket says 172.21.15.78:58140) 2024-08-18T23:47:20.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.610+0000 7fc44a910640 1 -- 172.21.15.78:0/3121870611 learned_addr learned my addr 172.21.15.78:0/3121870611 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:20.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.610+0000 7fc44a10f640 1 --2- 172.21.15.78:0/3121870611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc88177a0 0x557cc883db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:20.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.610+0000 7fc44990e640 1 --2- 172.21.15.78:0/3121870611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 0x557cc883e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:20.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.610+0000 7fc44a910640 1 -- 172.21.15.78:0/3121870611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 msgr2=0x557cc883e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:20.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.610+0000 7fc44a910640 1 --2- 172.21.15.78:0/3121870611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 0x557cc883e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.610+0000 7fc44a910640 1 -- 172.21.15.78:0/3121870611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc88177a0 msgr2=0x557cc883db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:20.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.610+0000 7fc44a910640 1 --2- 172.21.15.78:0/3121870611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc88177a0 0x557cc883db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.610+0000 7fc44a910640 1 -- 172.21.15.78:0/3121870611 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cc8818930 con 0x557cc881bb30 2024-08-18T23:47:20.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.611+0000 7fc44a910640 1 --2- 172.21.15.78:0/3121870611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 0x557cc88160e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0b8fb0 tx=0x7fc43c03c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89d0f521c412471 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:20.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.611+0000 7fc42ffff640 1 -- 172.21.15.78:0/3121870611 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c0bcce0 con 0x557cc881bb30 2024-08-18T23:47:20.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.611+0000 7fc42ffff640 1 -- 172.21.15.78:0/3121870611 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc43c0bc5e0 con 0x557cc881bb30 2024-08-18T23:47:20.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.611+0000 7fc42ffff640 1 -- 172.21.15.78:0/3121870611 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c0b3050 con 0x557cc881bb30 2024-08-18T23:47:20.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.611+0000 7fc44e771740 1 -- 172.21.15.78:0/3121870611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 msgr2=0x557cc88160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:20.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.611+0000 7fc44e771740 1 --2- 172.21.15.78:0/3121870611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 0x557cc88160e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0b8fb0 tx=0x7fc43c03c040 comp rx=0 tx=0).stop 2024-08-18T23:47:20.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.611+0000 7fc44e771740 1 -- 172.21.15.78:0/3121870611 shutdown_connections 2024-08-18T23:47:20.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.611+0000 7fc44e771740 1 --2- 172.21.15.78:0/3121870611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc881bb30 0x557cc88160e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.611+0000 7fc44e771740 1 --2- 172.21.15.78:0/3121870611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 0x557cc883e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.611+0000 7fc44e771740 1 --2- 172.21.15.78:0/3121870611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc88177a0 0x557cc883db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.611+0000 7fc44e771740 1 -- 172.21.15.78:0/3121870611 >> 172.21.15.78:0/3121870611 conn(0x557cc8838670 msgr2=0x557cc881d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:20.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.612+0000 7fc44e771740 1 -- 172.21.15.78:0/3121870611 shutdown_connections 2024-08-18T23:47:20.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.612+0000 7fc44e771740 1 -- 172.21.15.78:0/3121870611 wait complete. 2024-08-18T23:47:20.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.612+0000 7fc44e771740 1 -- start start 2024-08-18T23:47:20.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.612+0000 7fc44e771740 1 Processor -- start 2024-08-18T23:47:20.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.612+0000 7fc44e771740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc88177a0 0x557cc882e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:20.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.612+0000 7fc44e771740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 0x557cc8829c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:20.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.612+0000 7fc44e771740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc881bb30 0x557cc882a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:20.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.612+0000 7fc44e771740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441acee0 con 0x557cc88177a0 2024-08-18T23:47:20.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.612+0000 7fc44e771740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441ae350 con 0x557cc881bb30 2024-08-18T23:47:20.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.612+0000 7fc44e771740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441ad5b0 con 0x557cc881b210 2024-08-18T23:47:20.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.612+0000 7fc44a10f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc88177a0 0x557cc882e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:20.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.612+0000 7fc44e771740 10 client.0 _ll_register_callbacks cb 0x7fc43401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:20.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.612+0000 7fc44a10f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc88177a0 0x557cc882e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58150/0 (socket says 172.21.15.78:58150) 2024-08-18T23:47:20.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.612+0000 7fc44a10f640 1 -- 172.21.15.78:0/686780351 learned_addr learned my addr 172.21.15.78:0/686780351 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:20.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.612+0000 7fc44990e640 1 --2- 172.21.15.78:0/686780351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 0x557cc8829c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:20.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.612+0000 7fc44a910640 1 --2- 172.21.15.78:0/686780351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc881bb30 0x557cc882a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:20.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.613+0000 7fc44a10f640 1 -- 172.21.15.78:0/686780351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 msgr2=0x557cc8829c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:20.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.613+0000 7fc44a10f640 1 --2- 172.21.15.78:0/686780351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 0x557cc8829c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.613+0000 7fc44a10f640 1 -- 172.21.15.78:0/686780351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc881bb30 msgr2=0x557cc882a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:20.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.613+0000 7fc44a10f640 1 --2- 172.21.15.78:0/686780351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc881bb30 0x557cc882a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.613+0000 7fc44a10f640 1 -- 172.21.15.78:0/686780351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cc882a870 con 0x557cc88177a0 2024-08-18T23:47:20.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.613+0000 7fc44a910640 1 --2- 172.21.15.78:0/686780351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc881bb30 0x557cc882a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:20.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.613+0000 7fc44990e640 1 --2- 172.21.15.78:0/686780351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 0x557cc8829c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:20.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.613+0000 7fc44a10f640 1 --2- 172.21.15.78:0/686780351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc88177a0 0x557cc882e960 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x557cc8838ab0 tx=0x7fc440013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3842b9fc6939dc server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:20.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.613+0000 7fc42dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:47:20.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.613+0000 7fc42dffb640 1 -- 172.21.15.78:0/686780351 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44002d070 con 0x557cc88177a0 2024-08-18T23:47:20.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.613+0000 7fc42dffb640 1 -- 172.21.15.78:0/686780351 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc440005220 con 0x557cc88177a0 2024-08-18T23:47:20.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.613+0000 7fc44e771740 1 -- 172.21.15.78:0/686780351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc43c0bc5e0 con 0x557cc88177a0 2024-08-18T23:47:20.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.613+0000 7fc42dffb640 1 -- 172.21.15.78:0/686780351 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc440038050 con 0x557cc88177a0 2024-08-18T23:47:20.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.613+0000 7fc44e771740 10 client.15996 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:20.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.613+0000 7fc44e771740 1 -- 172.21.15.78:0/686780351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557cc88a4dd0 con 0x557cc88177a0 2024-08-18T23:47:20.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.613+0000 7fc42dffb640 1 -- 172.21.15.78:0/686780351 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc43c0bc5e0 con 0x557cc88177a0 2024-08-18T23:47:20.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.614+0000 7fc42dffb640 1 -- 172.21.15.78:0/686780351 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fc440004a00 con 0x557cc88177a0 2024-08-18T23:47:20.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.614+0000 7fc44e771740 10 client.15996 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:20.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.614+0000 7fc44e771740 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:47:20.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.614+0000 7fc44e771740 20 client.15996 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:20.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.614+0000 7fc44e771740 1 --2- 172.21.15.78:0/686780351 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x557cc88263f0 0x557cc88a75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:20.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.614+0000 7fc44e771740 4 client.15996 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:47:20.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.614+0000 7fc44e771740 1 -- 172.21.15.78:0/686780351 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557cc8788400 con 0x557cc88263f0 2024-08-18T23:47:20.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.614+0000 7fc44990e640 1 --2- 172.21.15.78:0/686780351 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x557cc88263f0 0x557cc88a75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:20.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.614+0000 7fc42dffb640 10 client.15996 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:20.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.614+0000 7fc44990e640 1 --2- 172.21.15.78:0/686780351 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x557cc88263f0 0x557cc88a75a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2894234f88fcee server_cookie=289cc154743ecce2 in_seq=0 out_seq=0 2024-08-18T23:47:20.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.633+0000 7fc42dffb640 1 -- 172.21.15.78:0/686780351 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x557cc8788400 con 0x557cc88263f0 2024-08-18T23:47:20.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.633+0000 7fc42dffb640 10 client.15996 handle_command_reply: tid=0 2024-08-18T23:47:20.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.633+0000 7fc44e771740 1 client.15996 shutdown 2024-08-18T23:47:20.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.633+0000 7fc44e771740 1 -- 172.21.15.78:0/686780351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc88177a0 msgr2=0x557cc882e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:20.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.633+0000 7fc44e771740 1 --2- 172.21.15.78:0/686780351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc88177a0 0x557cc882e960 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x557cc8838ab0 tx=0x7fc440013040 comp rx=0 tx=0).stop 2024-08-18T23:47:20.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.633+0000 7fc44e771740 1 -- 172.21.15.78:0/686780351 shutdown_connections 2024-08-18T23:47:20.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.633+0000 7fc44e771740 1 --2- 172.21.15.78:0/686780351 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x557cc88263f0 0x557cc88a75a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.633+0000 7fc44e771740 1 --2- 172.21.15.78:0/686780351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557cc881bb30 0x557cc882a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.633+0000 7fc44e771740 1 --2- 172.21.15.78:0/686780351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557cc881b210 0x557cc8829c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.633+0000 7fc44e771740 1 --2- 172.21.15.78:0/686780351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557cc88177a0 0x557cc882e960 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.633+0000 7fc44e771740 1 -- 172.21.15.78:0/686780351 >> 172.21.15.78:0/686780351 conn(0x557cc8838670 msgr2=0x557cc88160e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:20.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.633+0000 7fc42dffb640 0 client.15996 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:20.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.633+0000 7fc44e771740 1 -- 172.21.15.78:0/686780351 shutdown_connections 2024-08-18T23:47:20.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.633+0000 7fc44e771740 1 -- 172.21.15.78:0/686780351 wait complete. 2024-08-18T23:47:20.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.633+0000 7fc44e771740 20 client.15996 trim_cache size 0 max 16384 2024-08-18T23:47:20.637 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:47:20.637 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "scrub active (10162 inodes in the stack)", 2024-08-18T23:47:20.637 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-08-18T23:47:20.637 INFO:teuthology.orchestra.run.smithi078.stdout: "e1c540b0-f677-4ba3-b186-e94e8b511070": { 2024-08-18T23:47:20.637 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-08-18T23:47:20.638 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "e1c540b0-f677-4ba3-b186-e94e8b511070", 2024-08-18T23:47:20.638 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-08-18T23:47:20.638 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:47:20.638 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:47:20.638 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:47:20.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.634+0000 7fc44c373640 1 -- 172.21.15.78:0/572565916 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc4300a2b80 msgr2=0x7fc4300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:20.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.634+0000 7fc44c373640 1 --2- 172.21.15.78:0/572565916 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc4300a2b80 0x7fc4300a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc440009bd0 tx=0x7fc440005f90 comp rx=0 tx=0).stop 2024-08-18T23:47:20.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.634+0000 7fc44c373640 1 -- 172.21.15.78:0/572565916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc444102c20 msgr2=0x7fc44419ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:20.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.634+0000 7fc44c373640 1 --2- 172.21.15.78:0/572565916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc444102c20 0x7fc44419ca70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc43c004af0 tx=0x7fc43c02f3a0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.634+0000 7fc44c373640 1 -- 172.21.15.78:0/572565916 shutdown_connections 2024-08-18T23:47:20.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.634+0000 7fc44c373640 1 --2- 172.21.15.78:0/572565916 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc4300a2b80 0x7fc4300a5040 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.634+0000 7fc44c373640 1 --2- 172.21.15.78:0/572565916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc444102c20 0x7fc44419ca70 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.634+0000 7fc44c373640 1 --2- 172.21.15.78:0/572565916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4441022a0 0x7fc4441986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.634+0000 7fc44c373640 1 --2- 172.21.15.78:0/572565916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc444100b50 0x7fc4441981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:20.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.634+0000 7fc44c373640 1 -- 172.21.15.78:0/572565916 >> 172.21.15.78:0/572565916 conn(0x7fc4440fc500 msgr2=0x7fc444069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:20.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.634+0000 7fc44c373640 1 -- 172.21.15.78:0/572565916 shutdown_connections 2024-08-18T23:47:20.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:20.634+0000 7fc44c373640 1 -- 172.21.15.78:0/572565916 wait complete. 2024-08-18T23:47:20.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:20 smithi161 ceph-mon[22027]: scrub queued for path: / 2024-08-18T23:47:20.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:20 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:20.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:20 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:47:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:20 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:47:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:20 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:20 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:47:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:20 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:20 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:47:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:20 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:20 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:47:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:20 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:20.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:20 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:47:20.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:20 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:20.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:20 smithi123 ceph-mon[21808]: scrub queued for path: / 2024-08-18T23:47:20.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:20 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:20.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:20 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:47:20.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:20 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:47:20.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:20 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:20.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:20 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:47:20.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:20 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:20.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:20 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:47:20.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:20 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:20.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:20 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:47:20.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:20 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:20.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:20 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:47:20.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:20 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:21.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:21 smithi078 ceph-mon[18440]: pgmap v441: 129 pgs: 129 active+clean; 410 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-08-18T23:47:21.543 INFO:tasks.cephfs.filesystem:scrub status for tag:e1c540b0-f677-4ba3-b186-e94e8b511070 - {'path': '/', 'tag': 'e1c540b0-f677-4ba3-b186-e94e8b511070', 'options': 'recursive,force'} 2024-08-18T23:47:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:21 smithi161 ceph-mon[22027]: pgmap v441: 129 pgs: 129 active+clean; 410 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-08-18T23:47:21.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:21 smithi123 ceph-mon[21808]: pgmap v441: 129 pgs: 129 active+clean; 410 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-08-18T23:47:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:23 smithi161 ceph-mon[22027]: pgmap v442: 129 pgs: 129 active+clean; 415 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 202 op/s 2024-08-18T23:47:23.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:23 smithi078 ceph-mon[18440]: pgmap v442: 129 pgs: 129 active+clean; 415 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 202 op/s 2024-08-18T23:47:23.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:23 smithi123 ceph-mon[21808]: pgmap v442: 129 pgs: 129 active+clean; 415 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 202 op/s 2024-08-18T23:47:25.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.466+0000 7f62edffb640 1 -- 172.21.15.78:0/715894465 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f6300099160 con 0x5559a9c73900 2024-08-18T23:47:25.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.466+0000 7f62edffb640 10 client.25373 handle_command_reply: tid=0 2024-08-18T23:47:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.466+0000 7f630ec25740 1 client.25373 shutdown 2024-08-18T23:47:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.466+0000 7f630ec25740 1 -- 172.21.15.78:0/715894465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9beda50 msgr2=0x5559a9bf2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.466+0000 7f630ec25740 1 --2- 172.21.15.78:0/715894465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9beda50 0x5559a9bf2230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x5559a9bf0010 tx=0x7f63000cc0a0 comp rx=0 tx=0).stop 2024-08-18T23:47:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.466+0000 7f630ec25740 1 -- 172.21.15.78:0/715894465 shutdown_connections 2024-08-18T23:47:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.466+0000 7f630ec25740 1 --2- 172.21.15.78:0/715894465 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5559a9c73900 0x5559a9c75dc0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.466+0000 7f630ec25740 1 --2- 172.21.15.78:0/715894465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5559a9bf8310 0x5559a9bf29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.466+0000 7f630ec25740 1 --2- 172.21.15.78:0/715894465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559a9beda50 0x5559a9bf2230 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:25.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.466+0000 7f62edffb640 0 client.25373 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:25.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.466+0000 7f630ec25740 1 --2- 172.21.15.78:0/715894465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559a9bed130 0x5559a9c00bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:25.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.466+0000 7f630ec25740 1 -- 172.21.15.78:0/715894465 >> 172.21.15.78:0/715894465 conn(0x5559a9be96c0 msgr2=0x5559a9be86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:25.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.467+0000 7f630ec25740 1 -- 172.21.15.78:0/715894465 shutdown_connections 2024-08-18T23:47:25.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.467+0000 7f630ec25740 1 -- 172.21.15.78:0/715894465 wait complete. 2024-08-18T23:47:25.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.467+0000 7f630ec25740 20 client.25373 trim_cache size 0 max 16384 2024-08-18T23:47:25.470 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:47:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 26, 2024-08-18T23:47:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24452, 2024-08-18T23:47:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 10, 2024-08-18T23:47:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-08-18T23:47:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: "edfda978": { 2024-08-18T23:47:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 10, 2024-08-18T23:47:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-08-18T23:47:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:47:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-08-18T23:47:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:47:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:47:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-08-18T23:47:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 150.0, 2024-08-18T23:47:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-08-18T23:47:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-08-18T23:47:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-08-18T23:47:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:47:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-08-18T23:47:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:47:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:47:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:47:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:47:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:47:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:47:25.473 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:47:25.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.467+0000 7f630c827640 1 -- 172.21.15.78:0/4207621691 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f62f00a2c00 msgr2=0x7f62f00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:25.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.467+0000 7f630c827640 1 --2- 172.21.15.78:0/4207621691 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f62f00a2c00 0x7f62f00a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f62fc043000 tx=0x7f62fc046000 comp rx=0 tx=0).stop 2024-08-18T23:47:25.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.467+0000 7f630c827640 1 -- 172.21.15.78:0/4207621691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6308109560 msgr2=0x7f630811a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:25.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.467+0000 7f630c827640 1 --2- 172.21.15.78:0/4207621691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6308109560 0x7f630811a870 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f630000ba50 tx=0x7f630000bf10 comp rx=0 tx=0).stop 2024-08-18T23:47:25.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.467+0000 7f630c827640 1 -- 172.21.15.78:0/4207621691 shutdown_connections 2024-08-18T23:47:25.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.467+0000 7f630c827640 1 --2- 172.21.15.78:0/4207621691 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f62f00a2c00 0x7f62f00a50c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:25.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.467+0000 7f630c827640 1 --2- 172.21.15.78:0/4207621691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f630811ae70 0x7f63081134e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:25.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.467+0000 7f630c827640 1 --2- 172.21.15.78:0/4207621691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6308109560 0x7f630811a870 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:25.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.468+0000 7f630c827640 1 --2- 172.21.15.78:0/4207621691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6308109180 0x7f630811a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:25.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.468+0000 7f630c827640 1 -- 172.21.15.78:0/4207621691 >> 172.21.15.78:0/4207621691 conn(0x7f630806de70 msgr2=0x7f6308071fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:25.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.468+0000 7f630c827640 1 -- 172.21.15.78:0/4207621691 shutdown_connections 2024-08-18T23:47:25.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:25.468+0000 7f630c827640 1 -- 172.21.15.78:0/4207621691 wait complete. 2024-08-18T23:47:25.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:25 smithi161 ceph-mon[22027]: pgmap v443: 129 pgs: 129 active+clean; 419 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-08-18T23:47:25.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:25 smithi078 ceph-mon[18440]: pgmap v443: 129 pgs: 129 active+clean; 419 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-08-18T23:47:25.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:25 smithi123 ceph-mon[21808]: pgmap v443: 129 pgs: 129 active+clean; 419 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-08-18T23:47:26.415 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'edfda978', quiesce took 19.1 seconds. Will release after: 10.899999999999999 2024-08-18T23:47:26.982 INFO:tasks.workunit.client.0.smithi078.stdout: 77 3026 0 2006 0 853 0 2024-08-18T23:47:27.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:27 smithi161 ceph-mon[22027]: pgmap v444: 129 pgs: 129 active+clean; 431 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 327 op/s 2024-08-18T23:47:27.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:27 smithi078 ceph-mon[18440]: pgmap v444: 129 pgs: 129 active+clean; 431 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 327 op/s 2024-08-18T23:47:27.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:27 smithi123 ceph-mon[21808]: pgmap v444: 129 pgs: 129 active+clean; 431 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 327 op/s 2024-08-18T23:47:29.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:29 smithi161 ceph-mon[22027]: pgmap v445: 129 pgs: 129 active+clean; 441 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 343 op/s 2024-08-18T23:47:29.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:29 smithi078 ceph-mon[18440]: pgmap v445: 129 pgs: 129 active+clean; 441 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 343 op/s 2024-08-18T23:47:29.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:29 smithi123 ceph-mon[21808]: pgmap v445: 129 pgs: 129 active+clean; 441 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 343 op/s 2024-08-18T23:47:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:31 smithi161 ceph-mon[22027]: pgmap v446: 129 pgs: 129 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 357 op/s 2024-08-18T23:47:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:31 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:31.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:31 smithi078 ceph-mon[18440]: pgmap v446: 129 pgs: 129 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 357 op/s 2024-08-18T23:47:31.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:31 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:31.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:31 smithi123 ceph-mon[21808]: pgmap v446: 129 pgs: 129 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 357 op/s 2024-08-18T23:47:31.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:31 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:47:33.599 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:33 smithi123 ceph-mon[21808]: pgmap v447: 129 pgs: 129 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 382 op/s 2024-08-18T23:47:33.599 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:33 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:47:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:33 smithi161 ceph-mon[22027]: pgmap v447: 129 pgs: 129 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 382 op/s 2024-08-18T23:47:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:33 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:47:33.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:33 smithi078 ceph-mon[18440]: pgmap v447: 129 pgs: 129 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 382 op/s 2024-08-18T23:47:33.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:33 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:47:34.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:34 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:47:34.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:34 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:47:34.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:34 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:47:34.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:34 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:47:34.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:34 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:47:34.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:34 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:47:35.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:35 smithi161 ceph-mon[22027]: pgmap v448: 129 pgs: 129 active+clean; 446 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 330 op/s 2024-08-18T23:47:35.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:35 smithi078 ceph-mon[18440]: pgmap v448: 129 pgs: 129 active+clean; 446 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 330 op/s 2024-08-18T23:47:35.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:35 smithi123 ceph-mon[21808]: pgmap v448: 129 pgs: 129 active+clean; 446 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 330 op/s 2024-08-18T23:47:36.982 INFO:tasks.workunit.client.0.smithi078.stdout: 77 72397 0 50675 0 41521 0 2024-08-18T23:47:37.316 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:47:37.570 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:47:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:37 smithi161 ceph-mon[22027]: pgmap v449: 129 pgs: 129 active+clean; 446 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 317 op/s 2024-08-18T23:47:37.692 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:37 smithi078 ceph-mon[18440]: pgmap v449: 129 pgs: 129 active+clean; 446 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 317 op/s 2024-08-18T23:47:37.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:37 smithi123 ceph-mon[21808]: pgmap v449: 129 pgs: 129 active+clean; 446 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 317 op/s 2024-08-18T23:47:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.066+0000 7f9193a1b640 1 -- 172.21.15.78:0/3883299974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f918c102250 msgr2=0x7f918c10e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.066+0000 7f9193a1b640 1 --2- 172.21.15.78:0/3883299974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f918c102250 0x7f918c10e780 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9184009930 tx=0x7f918402a420 comp rx=0 tx=0).stop 2024-08-18T23:47:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.067+0000 7f9193a1b640 1 -- 172.21.15.78:0/3883299974 shutdown_connections 2024-08-18T23:47:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.067+0000 7f9193a1b640 1 --2- 172.21.15.78:0/3883299974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f918c102250 0x7f918c10e780 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.067+0000 7f9193a1b640 1 --2- 172.21.15.78:0/3883299974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f918c1018b0 0x7f918c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.067+0000 7f9193a1b640 1 --2- 172.21.15.78:0/3883299974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f918c1078b0 0x7f918c107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.067+0000 7f9193a1b640 1 -- 172.21.15.78:0/3883299974 >> 172.21.15.78:0/3883299974 conn(0x7f918c0fd5f0 msgr2=0x7f918c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.067+0000 7f9193a1b640 1 -- 172.21.15.78:0/3883299974 shutdown_connections 2024-08-18T23:47:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.067+0000 7f9193a1b640 1 -- 172.21.15.78:0/3883299974 wait complete. 2024-08-18T23:47:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.067+0000 7f9193a1b640 1 Processor -- start 2024-08-18T23:47:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.067+0000 7f9193a1b640 1 -- start start 2024-08-18T23:47:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.067+0000 7f9193a1b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f918c1018b0 0x7f918c074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.067+0000 7f9193a1b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f918c102250 0x7f918c074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.067+0000 7f9193a1b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f918c1078b0 0x7f918c074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.067+0000 7f9193a1b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f918c111cd0 con 0x7f918c102250 2024-08-18T23:47:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.067+0000 7f9193a1b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f918c111b50 con 0x7f918c1078b0 2024-08-18T23:47:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.067+0000 7f9193a1b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f918c111e50 con 0x7f918c1018b0 2024-08-18T23:47:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.068+0000 7f9190fb6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f918c102250 0x7f918c074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.068+0000 7f9190fb6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f918c102250 0x7f918c074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47442/0 (socket says 172.21.15.78:47442) 2024-08-18T23:47:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.068+0000 7f9190fb6640 1 -- 172.21.15.78:0/1503569957 learned_addr learned my addr 172.21.15.78:0/1503569957 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.068+0000 7f9191fb8640 1 --2- 172.21.15.78:0/1503569957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f918c1078b0 0x7f918c074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.068+0000 7f91917b7640 1 --2- 172.21.15.78:0/1503569957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f918c1018b0 0x7f918c074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.068+0000 7f9190fb6640 1 -- 172.21.15.78:0/1503569957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f918c1018b0 msgr2=0x7f918c074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.068+0000 7f9190fb6640 1 --2- 172.21.15.78:0/1503569957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f918c1018b0 0x7f918c074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.068+0000 7f9190fb6640 1 -- 172.21.15.78:0/1503569957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f918c1078b0 msgr2=0x7f918c074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.068+0000 7f9190fb6640 1 --2- 172.21.15.78:0/1503569957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f918c1078b0 0x7f918c074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.068+0000 7f9190fb6640 1 -- 172.21.15.78:0/1503569957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c1a6690 con 0x7f918c102250 2024-08-18T23:47:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.068+0000 7f9190fb6640 1 --2- 172.21.15.78:0/1503569957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f918c102250 0x7f918c074760 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f918000c890 tx=0x7f918000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.068+0000 7f918a7fc640 1 -- 172.21.15.78:0/1503569957 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9180004220 con 0x7f918c102250 2024-08-18T23:47:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.068+0000 7f918a7fc640 1 -- 172.21.15.78:0/1503569957 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91800043c0 con 0x7f918c102250 2024-08-18T23:47:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.068+0000 7f9193a1b640 1 -- 172.21.15.78:0/1503569957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918c1a6980 con 0x7f918c102250 2024-08-18T23:47:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.068+0000 7f918a7fc640 1 -- 172.21.15.78:0/1503569957 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91800049e0 con 0x7f918c102250 2024-08-18T23:47:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.068+0000 7f9193a1b640 1 -- 172.21.15.78:0/1503569957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f918c1a6ec0 con 0x7f918c102250 2024-08-18T23:47:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.069+0000 7f9193a1b640 1 -- 172.21.15.78:0/1503569957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f918c101d10 con 0x7f918c102250 2024-08-18T23:47:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.070+0000 7f918a7fc640 1 -- 172.21.15.78:0/1503569957 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f91800079b0 con 0x7f918c102250 2024-08-18T23:47:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.071+0000 7f918a7fc640 1 --2- 172.21.15.78:0/1503569957 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f91740a2b80 0x7f91740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.071+0000 7f91917b7640 1 --2- 172.21.15.78:0/1503569957 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f91740a2b80 0x7f91740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.071+0000 7f918a7fc640 1 -- 172.21.15.78:0/1503569957 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f91800cf4e0 con 0x7f918c102250 2024-08-18T23:47:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.071+0000 7f91917b7640 1 --2- 172.21.15.78:0/1503569957 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f91740a2b80 0x7f91740a5040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f917c009f20 tx=0x7f917c0094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.075+0000 7f918a7fc640 1 -- 172.21.15.78:0/1503569957 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9180014030 con 0x7f918c102250 2024-08-18T23:47:38.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.291+0000 7f9193a1b640 1 -- 172.21.15.78:0/1503569957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f918c109e40 con 0x7f918c102250 2024-08-18T23:47:38.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.292+0000 7f918a7fc640 1 -- 172.21.15.78:0/1503569957 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11933 (secure 0 0 0) 0x7f918009c0b0 con 0x7f918c102250 2024-08-18T23:47:38.293 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:47:38.293 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":26,"btime":"2024-08-18T23:45:33:306257+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:45:32.491054+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:47:38.294 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 26 2024-08-18T23:47:38.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.294+0000 7f9193a1b640 1 -- 172.21.15.78:0/1503569957 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f91740a2b80 msgr2=0x7f91740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:38.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.294+0000 7f9193a1b640 1 --2- 172.21.15.78:0/1503569957 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f91740a2b80 0x7f91740a5040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f917c009f20 tx=0x7f917c0094f0 comp rx=0 tx=0).stop 2024-08-18T23:47:38.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.294+0000 7f9193a1b640 1 -- 172.21.15.78:0/1503569957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f918c102250 msgr2=0x7f918c074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:38.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.294+0000 7f9193a1b640 1 --2- 172.21.15.78:0/1503569957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f918c102250 0x7f918c074760 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f918000c890 tx=0x7f918000cd50 comp rx=0 tx=0).stop 2024-08-18T23:47:38.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.294+0000 7f9193a1b640 1 -- 172.21.15.78:0/1503569957 shutdown_connections 2024-08-18T23:47:38.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.294+0000 7f9193a1b640 1 --2- 172.21.15.78:0/1503569957 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f91740a2b80 0x7f91740a5040 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:38.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.294+0000 7f9193a1b640 1 --2- 172.21.15.78:0/1503569957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f918c1078b0 0x7f918c074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:38.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.294+0000 7f9193a1b640 1 --2- 172.21.15.78:0/1503569957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f918c102250 0x7f918c074760 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:38.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.294+0000 7f9193a1b640 1 --2- 172.21.15.78:0/1503569957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f918c1018b0 0x7f918c074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:38.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.294+0000 7f9193a1b640 1 -- 172.21.15.78:0/1503569957 >> 172.21.15.78:0/1503569957 conn(0x7f918c0fd5f0 msgr2=0x7f918c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:38.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.294+0000 7f9193a1b640 1 -- 172.21.15.78:0/1503569957 shutdown_connections 2024-08-18T23:47:38.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:38.294+0000 7f9193a1b640 1 -- 172.21.15.78:0/1503569957 wait complete. 2024-08-18T23:47:38.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:38 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1503569957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:47:38.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:38 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1503569957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:47:38.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:38 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1503569957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:47:39.134 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id edfda978 --release --await' 2024-08-18T23:47:39.134 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id edfda978 --release --await 2024-08-18T23:47:39.388 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:47:39.658 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:39 smithi078 ceph-mon[18440]: pgmap v450: 129 pgs: 129 active+clean; 446 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 261 op/s 2024-08-18T23:47:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:39 smithi161 ceph-mon[22027]: pgmap v450: 129 pgs: 129 active+clean; 446 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 261 op/s 2024-08-18T23:47:39.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:39 smithi123 ceph-mon[21808]: pgmap v450: 129 pgs: 129 active+clean; 446 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 261 op/s 2024-08-18T23:47:39.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.917+0000 7f21b4560640 1 -- 172.21.15.78:0/4044739801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21ac102a70 msgr2=0x7f21ac102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:39.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.917+0000 7f21b4560640 1 --2- 172.21.15.78:0/4044739801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21ac102a70 0x7f21ac102e50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f219c009ca0 tx=0x7f219c02a950 comp rx=0 tx=0).stop 2024-08-18T23:47:39.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.917+0000 7f21b4560640 1 -- 172.21.15.78:0/4044739801 shutdown_connections 2024-08-18T23:47:39.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.917+0000 7f21b4560640 1 --2- 172.21.15.78:0/4044739801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21ac10bea0 0x7f21ac10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.917+0000 7f21b4560640 1 --2- 172.21.15.78:0/4044739801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ac103390 0x7f21ac10b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.918+0000 7f21b4560640 1 --2- 172.21.15.78:0/4044739801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21ac102a70 0x7f21ac102e50 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.918+0000 7f21b4560640 1 -- 172.21.15.78:0/4044739801 >> 172.21.15.78:0/4044739801 conn(0x7f21ac0fc4d0 msgr2=0x7f21ac0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:39.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.918+0000 7f21b4560640 1 -- 172.21.15.78:0/4044739801 shutdown_connections 2024-08-18T23:47:39.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.918+0000 7f21b4560640 1 -- 172.21.15.78:0/4044739801 wait complete. 2024-08-18T23:47:39.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.918+0000 7f21b4560640 1 Processor -- start 2024-08-18T23:47:39.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.918+0000 7f21b4560640 1 -- start start 2024-08-18T23:47:39.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.918+0000 7f21b4560640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21ac102a70 0x7f21ac198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:39.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.918+0000 7f21b4560640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ac103390 0x7f21ac198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:39.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.918+0000 7f21b4560640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21ac10bea0 0x7f21ac19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:39.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.918+0000 7f21b4560640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac111ce0 con 0x7f21ac102a70 2024-08-18T23:47:39.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.918+0000 7f21b4560640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac111b60 con 0x7f21ac10bea0 2024-08-18T23:47:39.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.918+0000 7f21b4560640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac111e60 con 0x7f21ac103390 2024-08-18T23:47:39.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.919+0000 7f21b22fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21ac102a70 0x7f21ac198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:39.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.919+0000 7f21b22fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21ac102a70 0x7f21ac198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47470/0 (socket says 172.21.15.78:47470) 2024-08-18T23:47:39.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.919+0000 7f21b22fc640 1 -- 172.21.15.78:0/3236812532 learned_addr learned my addr 172.21.15.78:0/3236812532 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:39.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.919+0000 7f21b1afb640 1 --2- 172.21.15.78:0/3236812532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ac103390 0x7f21ac198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:39.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.919+0000 7f21b2afd640 1 --2- 172.21.15.78:0/3236812532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21ac10bea0 0x7f21ac19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:39.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.919+0000 7f21b22fc640 1 -- 172.21.15.78:0/3236812532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ac103390 msgr2=0x7f21ac198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:39.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.919+0000 7f21b22fc640 1 --2- 172.21.15.78:0/3236812532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ac103390 0x7f21ac198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.919+0000 7f21b22fc640 1 -- 172.21.15.78:0/3236812532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21ac10bea0 msgr2=0x7f21ac19cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:39.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.919+0000 7f21b22fc640 1 --2- 172.21.15.78:0/3236812532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21ac10bea0 0x7f21ac19cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.919+0000 7f21b22fc640 1 -- 172.21.15.78:0/3236812532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ac19d1f0 con 0x7f21ac102a70 2024-08-18T23:47:39.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.919+0000 7f21b1afb640 1 --2- 172.21.15.78:0/3236812532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ac103390 0x7f21ac198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:47:39.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.919+0000 7f21b22fc640 1 --2- 172.21.15.78:0/3236812532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21ac102a70 0x7f21ac198240 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f219c005cc0 tx=0x7f219c02f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:39.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.919+0000 7f21ab7fe640 1 -- 172.21.15.78:0/3236812532 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219c034040 con 0x7f21ac102a70 2024-08-18T23:47:39.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.919+0000 7f21ab7fe640 1 -- 172.21.15.78:0/3236812532 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f219c003e40 con 0x7f21ac102a70 2024-08-18T23:47:39.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.919+0000 7f21b4560640 1 -- 172.21.15.78:0/3236812532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21ac19d480 con 0x7f21ac102a70 2024-08-18T23:47:39.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.919+0000 7f21ab7fe640 1 -- 172.21.15.78:0/3236812532 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219c045480 con 0x7f21ac102a70 2024-08-18T23:47:39.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.919+0000 7f21b4560640 1 -- 172.21.15.78:0/3236812532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21ac1a4dc0 con 0x7f21ac102a70 2024-08-18T23:47:39.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.921+0000 7f21ab7fe640 1 -- 172.21.15.78:0/3236812532 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f219c037020 con 0x7f21ac102a70 2024-08-18T23:47:39.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.922+0000 7f21b695e740 1 Processor -- start 2024-08-18T23:47:39.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.922+0000 7f21b695e740 1 -- start start 2024-08-18T23:47:39.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.922+0000 7f21ab7fe640 1 --2- 172.21.15.78:0/3236812532 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f21940a2b80 0x7f21940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:39.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.922+0000 7f21b695e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2832d990 0x55fb2832dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:39.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.922+0000 7f21b695e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832e2b0 0x55fb28338630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:39.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.922+0000 7f21b695e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb28338b70 0x55fb2833af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:39.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.922+0000 7f21b695e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fb28293c60 con 0x55fb28338b70 2024-08-18T23:47:39.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.922+0000 7f21b695e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fb2822bb90 con 0x55fb2832e2b0 2024-08-18T23:47:39.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.922+0000 7f21b695e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fb2828fc20 con 0x55fb2832d990 2024-08-18T23:47:39.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.922+0000 7f21ab7fe640 1 -- 172.21.15.78:0/3236812532 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f219c0f06b0 con 0x7f21ac102a70 2024-08-18T23:47:39.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.922+0000 7f21b2afd640 1 --2- 172.21.15.78:0/3236812532 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f21940a2b80 0x7f21940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:39.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.922+0000 7f21b2afd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb28338b70 0x55fb2833af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:39.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.922+0000 7f21b2afd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb28338b70 0x55fb2833af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47486/0 (socket says 172.21.15.78:47486) 2024-08-18T23:47:39.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.922+0000 7f21b2afd640 1 -- 172.21.15.78:0/954897112 learned_addr learned my addr 172.21.15.78:0/954897112 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:39.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.923+0000 7f21b1afb640 1 --2- 172.21.15.78:0/954897112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832e2b0 0x55fb28338630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:39.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.923+0000 7f21b22fc640 1 --2- 172.21.15.78:0/954897112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2832d990 0x55fb2832dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:39.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.923+0000 7f21b2afd640 1 -- 172.21.15.78:0/954897112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2832d990 msgr2=0x55fb2832dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:39.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.923+0000 7f21b2afd640 1 --2- 172.21.15.78:0/954897112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2832d990 0x55fb2832dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.923+0000 7f21b2afd640 1 -- 172.21.15.78:0/954897112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832e2b0 msgr2=0x55fb28338630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:39.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.923+0000 7f21b2afd640 1 --2- 172.21.15.78:0/954897112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832e2b0 0x55fb28338630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.923+0000 7f21b2afd640 1 -- 172.21.15.78:0/954897112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb27fa3580 con 0x55fb28338b70 2024-08-18T23:47:39.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.923+0000 7f21b22fc640 1 --2- 172.21.15.78:0/954897112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2832d990 0x55fb2832dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:47:39.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.923+0000 7f21b2afd640 1 --2- 172.21.15.78:0/3236812532 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f21940a2b80 0x7f21940a5040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f21a40138a0 tx=0x7f21a400ba80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:39.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.923+0000 7f21b2afd640 1 --2- 172.21.15.78:0/954897112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb28338b70 0x55fb2833af60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f21a40107a0 tx=0x7f21a401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e328d6ef9134ec server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:39.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.923+0000 7f21a8ff9640 1 -- 172.21.15.78:0/954897112 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a4021070 con 0x55fb28338b70 2024-08-18T23:47:39.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.923+0000 7f21a8ff9640 1 -- 172.21.15.78:0/954897112 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21a400ee50 con 0x55fb28338b70 2024-08-18T23:47:39.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.923+0000 7f21a8ff9640 1 -- 172.21.15.78:0/954897112 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a4003850 con 0x55fb28338b70 2024-08-18T23:47:39.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.924+0000 7f21b695e740 1 -- 172.21.15.78:0/954897112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb28338b70 msgr2=0x55fb2833af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:39.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.924+0000 7f21b695e740 1 --2- 172.21.15.78:0/954897112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb28338b70 0x55fb2833af60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f21a40107a0 tx=0x7f21a401a000 comp rx=0 tx=0).stop 2024-08-18T23:47:39.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.924+0000 7f21b695e740 1 -- 172.21.15.78:0/954897112 shutdown_connections 2024-08-18T23:47:39.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.924+0000 7f21b695e740 1 --2- 172.21.15.78:0/954897112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb28338b70 0x55fb2833af60 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.924+0000 7f21b695e740 1 --2- 172.21.15.78:0/954897112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832e2b0 0x55fb28338630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.924+0000 7f21b695e740 1 --2- 172.21.15.78:0/954897112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb2832d990 0x55fb2832dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.924+0000 7f21b695e740 1 -- 172.21.15.78:0/954897112 >> 172.21.15.78:0/954897112 conn(0x55fb28329f20 msgr2=0x55fb2832a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:39.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.924+0000 7f21b695e740 1 -- 172.21.15.78:0/954897112 shutdown_connections 2024-08-18T23:47:39.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.924+0000 7f21b695e740 1 -- 172.21.15.78:0/954897112 wait complete. 2024-08-18T23:47:39.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.924+0000 7f21b695e740 1 -- start start 2024-08-18T23:47:39.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.924+0000 7f21b695e740 1 Processor -- start 2024-08-18T23:47:39.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b695e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832d990 0x55fb28335a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:39.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b695e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832e2b0 0x55fb28335f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:39.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b695e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb28338b70 0x55fb283364d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:39.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b695e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fb2833fb50 con 0x55fb2832e2b0 2024-08-18T23:47:39.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b695e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fb282317c0 con 0x55fb2832d990 2024-08-18T23:47:39.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b695e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fb2833fcd0 con 0x55fb28338b70 2024-08-18T23:47:39.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b695e740 10 client.0 _ll_register_callbacks cb 0x55fb2823af50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:39.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b1afb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832e2b0 0x55fb28335f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:39.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b1afb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832e2b0 0x55fb28335f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47488/0 (socket says 172.21.15.78:47488) 2024-08-18T23:47:39.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b1afb640 1 -- 172.21.15.78:0/853393300 learned_addr learned my addr 172.21.15.78:0/853393300 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:39.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b22fc640 1 --2- 172.21.15.78:0/853393300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832d990 0x55fb28335a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:39.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b2afd640 1 --2- 172.21.15.78:0/853393300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb28338b70 0x55fb283364d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:39.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b1afb640 1 -- 172.21.15.78:0/853393300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb28338b70 msgr2=0x55fb283364d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b1afb640 1 --2- 172.21.15.78:0/853393300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb28338b70 0x55fb283364d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b1afb640 1 -- 172.21.15.78:0/853393300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832d990 msgr2=0x55fb28335a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b1afb640 1 --2- 172.21.15.78:0/853393300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832d990 0x55fb28335a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b1afb640 1 -- 172.21.15.78:0/853393300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb2806d960 con 0x55fb2832e2b0 2024-08-18T23:47:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b2afd640 1 --2- 172.21.15.78:0/853393300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb28338b70 0x55fb283364d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:47:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b1afb640 1 --2- 172.21.15.78:0/853393300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832e2b0 0x55fb28335f90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f21a0010530 tx=0x7f21a00109f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464e51a4666bd41e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f2191ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:47:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f2191ffb640 1 -- 172.21.15.78:0/853393300 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0018070 con 0x55fb2832e2b0 2024-08-18T23:47:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f2191ffb640 1 -- 172.21.15.78:0/853393300 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f21a0014640 con 0x55fb2832e2b0 2024-08-18T23:47:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.925+0000 7f21b695e740 1 -- 172.21.15.78:0/853393300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21a400ee50 con 0x55fb2832e2b0 2024-08-18T23:47:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.926+0000 7f2191ffb640 1 -- 172.21.15.78:0/853393300 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f21a0024050 con 0x55fb2832e2b0 2024-08-18T23:47:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.926+0000 7f21b695e740 10 client.16014 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.926+0000 7f21b695e740 1 -- 172.21.15.78:0/853393300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fb2834bc40 con 0x55fb2832e2b0 2024-08-18T23:47:39.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.926+0000 7f2191ffb640 1 -- 172.21.15.78:0/853393300 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21a400ee50 con 0x55fb2832e2b0 2024-08-18T23:47:39.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.926+0000 7f2191ffb640 1 -- 172.21.15.78:0/853393300 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f21a0012ae0 con 0x55fb2832e2b0 2024-08-18T23:47:39.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.926+0000 7f21b695e740 10 client.16014 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:39.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.926+0000 7f21b695e740 10 client.16014 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:47:39.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.926+0000 7f21b695e740 20 client.16014 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:39.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.926+0000 7f21b695e740 1 --2- 172.21.15.78:0/853393300 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55fb2834d160 0x55fb2834f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:39.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.926+0000 7f21b695e740 4 client.16014 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:47:39.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.926+0000 7f21b695e740 1 -- 172.21.15.78:0/853393300 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb282a11d0 con 0x55fb2834d160 2024-08-18T23:47:39.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.926+0000 7f21b22fc640 1 --2- 172.21.15.78:0/853393300 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55fb2834d160 0x55fb2834f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:39.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.927+0000 7f2191ffb640 10 client.16014 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:39.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.927+0000 7f21b22fc640 1 --2- 172.21.15.78:0/853393300 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55fb2834d160 0x55fb2834f590 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bda21842c35e29e server_cookie=c3c598d0b4543712 in_seq=0 out_seq=0 2024-08-18T23:47:39.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.927+0000 7f2191ffb640 1 -- 172.21.15.78:0/853393300 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb282a11d0 con 0x55fb2834d160 2024-08-18T23:47:39.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.927+0000 7f2191ffb640 10 client.16014 handle_command_reply: tid=0 2024-08-18T23:47:39.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.927+0000 7f21b695e740 1 client.16014 shutdown 2024-08-18T23:47:39.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.928+0000 7f21b695e740 1 -- 172.21.15.78:0/853393300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832e2b0 msgr2=0x55fb28335f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:39.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.928+0000 7f21b695e740 1 --2- 172.21.15.78:0/853393300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832e2b0 0x55fb28335f90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f21a0010530 tx=0x7f21a00109f0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.928+0000 7f21b695e740 1 -- 172.21.15.78:0/853393300 shutdown_connections 2024-08-18T23:47:39.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.928+0000 7f21b695e740 1 --2- 172.21.15.78:0/853393300 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55fb2834d160 0x55fb2834f590 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.928+0000 7f21b695e740 1 --2- 172.21.15.78:0/853393300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb28338b70 0x55fb283364d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.928+0000 7f21b695e740 1 --2- 172.21.15.78:0/853393300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832e2b0 0x55fb28335f90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.928+0000 7f21b695e740 1 --2- 172.21.15.78:0/853393300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832d990 0x55fb28335a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.928+0000 7f21b695e740 1 -- 172.21.15.78:0/853393300 >> 172.21.15.78:0/853393300 conn(0x55fb28329f20 msgr2=0x55fb2832fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:39.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.928+0000 7f2191ffb640 0 client.16014 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:39.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.928+0000 7f21b695e740 1 -- 172.21.15.78:0/853393300 shutdown_connections 2024-08-18T23:47:39.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.928+0000 7f21b695e740 1 -- 172.21.15.78:0/853393300 wait complete. 2024-08-18T23:47:39.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.928+0000 7f21b695e740 20 client.16014 trim_cache size 0 max 16384 2024-08-18T23:47:39.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.945+0000 7f21b695e740 1 Processor -- start 2024-08-18T23:47:39.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.945+0000 7f21b695e740 1 -- start start 2024-08-18T23:47:39.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.945+0000 7f21b695e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832d990 0x55fb283349d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:39.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.945+0000 7f21b695e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832e2b0 0x55fb28354e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:39.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.945+0000 7f21b695e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb28338b70 0x55fb283553a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:39.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.945+0000 7f21b695e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac1af0e0 con 0x55fb2832d990 2024-08-18T23:47:39.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.945+0000 7f21b695e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac1ac9b0 con 0x55fb2832e2b0 2024-08-18T23:47:39.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.945+0000 7f21b695e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac1aa800 con 0x55fb28338b70 2024-08-18T23:47:39.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.945+0000 7f21b22fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832d990 0x55fb283349d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:39.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.945+0000 7f21b22fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832d990 0x55fb283349d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47492/0 (socket says 172.21.15.78:47492) 2024-08-18T23:47:39.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.945+0000 7f21b22fc640 1 -- 172.21.15.78:0/1299516980 learned_addr learned my addr 172.21.15.78:0/1299516980 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:39.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.945+0000 7f21b1afb640 1 --2- 172.21.15.78:0/1299516980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832e2b0 0x55fb28354e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:39.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.945+0000 7f21b2afd640 1 --2- 172.21.15.78:0/1299516980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb28338b70 0x55fb283553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:39.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.945+0000 7f21b22fc640 1 -- 172.21.15.78:0/1299516980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb28338b70 msgr2=0x55fb283553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:39.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.945+0000 7f21b22fc640 1 --2- 172.21.15.78:0/1299516980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb28338b70 0x55fb283553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.945+0000 7f21b22fc640 1 -- 172.21.15.78:0/1299516980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832e2b0 msgr2=0x55fb28354e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:39.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.945+0000 7f21b22fc640 1 --2- 172.21.15.78:0/1299516980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832e2b0 0x55fb28354e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.945+0000 7f21b22fc640 1 -- 172.21.15.78:0/1299516980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb283385a0 con 0x55fb2832d990 2024-08-18T23:47:39.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.946+0000 7f21b22fc640 1 --2- 172.21.15.78:0/1299516980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832d990 0x55fb283349d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f219c004450 tx=0x7f219c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63a5c9b4a45c3af server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:39.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.946+0000 7f2193fff640 1 -- 172.21.15.78:0/1299516980 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219c0bcb20 con 0x55fb2832d990 2024-08-18T23:47:39.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.946+0000 7f2193fff640 1 -- 172.21.15.78:0/1299516980 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f219c0bde50 con 0x55fb2832d990 2024-08-18T23:47:39.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.946+0000 7f2193fff640 1 -- 172.21.15.78:0/1299516980 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219c004040 con 0x55fb2832d990 2024-08-18T23:47:39.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.947+0000 7f21b695e740 1 -- 172.21.15.78:0/1299516980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832d990 msgr2=0x55fb283349d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:39.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.947+0000 7f21b695e740 1 --2- 172.21.15.78:0/1299516980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832d990 0x55fb283349d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f219c004450 tx=0x7f219c0095c0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.947+0000 7f21b695e740 1 -- 172.21.15.78:0/1299516980 shutdown_connections 2024-08-18T23:47:39.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.947+0000 7f21b695e740 1 --2- 172.21.15.78:0/1299516980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb28338b70 0x55fb283553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.947+0000 7f21b695e740 1 --2- 172.21.15.78:0/1299516980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832e2b0 0x55fb28354e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.947+0000 7f21b695e740 1 --2- 172.21.15.78:0/1299516980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832d990 0x55fb283349d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.947+0000 7f21b695e740 1 -- 172.21.15.78:0/1299516980 >> 172.21.15.78:0/1299516980 conn(0x55fb28329f20 msgr2=0x55fb28337ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:39.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.947+0000 7f21b695e740 1 -- 172.21.15.78:0/1299516980 shutdown_connections 2024-08-18T23:47:39.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.947+0000 7f21b695e740 1 -- 172.21.15.78:0/1299516980 wait complete. 2024-08-18T23:47:39.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.947+0000 7f21b695e740 1 -- start start 2024-08-18T23:47:39.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.947+0000 7f21b695e740 1 Processor -- start 2024-08-18T23:47:39.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.947+0000 7f21b695e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832d990 0x55fb28332c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:39.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.947+0000 7f21b695e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832e2b0 0x55fb28333190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:39.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.947+0000 7f21b695e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb28338b70 0x55fb283336d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:39.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f21b695e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac1aae70 con 0x55fb2832d990 2024-08-18T23:47:39.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f21b695e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac1ac2e0 con 0x55fb2832e2b0 2024-08-18T23:47:39.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f21b695e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f21ac1ab540 con 0x55fb28338b70 2024-08-18T23:47:39.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f21b22fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832d990 0x55fb28332c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:39.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f21b695e740 10 client.0 _ll_register_callbacks cb 0x7f21a0007df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:39.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f21b22fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832d990 0x55fb28332c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47506/0 (socket says 172.21.15.78:47506) 2024-08-18T23:47:39.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f21b22fc640 1 -- 172.21.15.78:0/1872687189 learned_addr learned my addr 172.21.15.78:0/1872687189 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:39.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f21b2afd640 1 --2- 172.21.15.78:0/1872687189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb28338b70 0x55fb283336d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:39.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f21b1afb640 1 --2- 172.21.15.78:0/1872687189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832e2b0 0x55fb28333190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:39.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f21b22fc640 1 -- 172.21.15.78:0/1872687189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb28338b70 msgr2=0x55fb283336d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:39.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f21b22fc640 1 --2- 172.21.15.78:0/1872687189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb28338b70 0x55fb283336d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f21b22fc640 1 -- 172.21.15.78:0/1872687189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832e2b0 msgr2=0x55fb28333190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:39.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f21b22fc640 1 --2- 172.21.15.78:0/1872687189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832e2b0 0x55fb28333190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:39.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f21b22fc640 1 -- 172.21.15.78:0/1872687189 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb283b6f90 con 0x55fb2832d990 2024-08-18T23:47:39.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f21b22fc640 1 --2- 172.21.15.78:0/1872687189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832d990 0x55fb28332c50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f219c02ae70 tx=0x7f219c02fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38f2257e5f65eef server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:39.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f2191ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:47:39.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f2191ffb640 1 -- 172.21.15.78:0/1872687189 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219c0bcb20 con 0x55fb2832d990 2024-08-18T23:47:39.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f2191ffb640 1 -- 172.21.15.78:0/1872687189 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f219c0f0360 con 0x55fb2832d990 2024-08-18T23:47:39.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f21b695e740 1 -- 172.21.15.78:0/1872687189 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f219c0bde50 con 0x55fb2832d990 2024-08-18T23:47:39.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f2191ffb640 1 -- 172.21.15.78:0/1872687189 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f219c0a3050 con 0x55fb2832d990 2024-08-18T23:47:39.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f21b695e740 10 client.16020 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:39.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.948+0000 7f21b695e740 1 -- 172.21.15.78:0/1872687189 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fb283b82b0 con 0x55fb2832d990 2024-08-18T23:47:39.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.949+0000 7f2191ffb640 1 -- 172.21.15.78:0/1872687189 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f219c0bde50 con 0x55fb2832d990 2024-08-18T23:47:39.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.949+0000 7f2191ffb640 1 -- 172.21.15.78:0/1872687189 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f219c0f0040 con 0x55fb2832d990 2024-08-18T23:47:39.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.949+0000 7f21b695e740 10 client.16020 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:39.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.949+0000 7f21b695e740 10 client.16020 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:47:39.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.949+0000 7f21b695e740 20 client.16020 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:39.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.949+0000 7f21b695e740 1 --2- 172.21.15.78:0/1872687189 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55fb283b89c0 0x55fb283badb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:39.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.949+0000 7f21b695e740 4 client.16020 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "edfda978", "release": true, "await": true}] 2024-08-18T23:47:39.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.949+0000 7f21b695e740 1 -- 172.21.15.78:0/1872687189 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "quiesce db", "set_id": "edfda978", "release": true, "await": true}) -- 0x55fb2829b6e0 con 0x55fb283b89c0 2024-08-18T23:47:39.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.949+0000 7f21b1afb640 1 --2- 172.21.15.78:0/1872687189 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55fb283b89c0 0x55fb283badb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:39.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.950+0000 7f21b1afb640 1 --2- 172.21.15.78:0/1872687189 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55fb283b89c0 0x55fb283badb0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40000b6105d73549 server_cookie=19b99e77ebbf92ba in_seq=0 out_seq=0 2024-08-18T23:47:39.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:39.950+0000 7f2191ffb640 10 client.16020 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:40.621 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:40 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:47:40.621 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:40 smithi078 ceph-mon[18440]: pgmap v451: 129 pgs: 129 active+clean; 447 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 470 KiB/s wr, 223 op/s 2024-08-18T23:47:40.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.619+0000 7f2191ffb640 1 -- 172.21.15.78:0/1872687189 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fb2829b6e0 con 0x55fb283b89c0 2024-08-18T23:47:40.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.619+0000 7f2191ffb640 10 client.16020 handle_command_reply: tid=0 2024-08-18T23:47:40.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.619+0000 7f21b695e740 1 client.16020 shutdown 2024-08-18T23:47:40.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.619+0000 7f21b695e740 1 -- 172.21.15.78:0/1872687189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832d990 msgr2=0x55fb28332c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:40.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.619+0000 7f21b695e740 1 --2- 172.21.15.78:0/1872687189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832d990 0x55fb28332c50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f219c02ae70 tx=0x7f219c02fdc0 comp rx=0 tx=0).stop 2024-08-18T23:47:40.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.619+0000 7f21b695e740 1 -- 172.21.15.78:0/1872687189 shutdown_connections 2024-08-18T23:47:40.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.619+0000 7f21b695e740 1 --2- 172.21.15.78:0/1872687189 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55fb283b89c0 0x55fb283badb0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:40.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.619+0000 7f21b695e740 1 --2- 172.21.15.78:0/1872687189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb28338b70 0x55fb283336d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:40.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.619+0000 7f21b695e740 1 --2- 172.21.15.78:0/1872687189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fb2832e2b0 0x55fb28333190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:40.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.619+0000 7f21b695e740 1 --2- 172.21.15.78:0/1872687189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb2832d990 0x55fb28332c50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:40.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.619+0000 7f2191ffb640 0 client.16020 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:40.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.619+0000 7f21b695e740 1 -- 172.21.15.78:0/1872687189 >> 172.21.15.78:0/1872687189 conn(0x55fb28329f20 msgr2=0x55fb28328860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:40.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.620+0000 7f21b695e740 1 -- 172.21.15.78:0/1872687189 shutdown_connections 2024-08-18T23:47:40.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.620+0000 7f21b695e740 1 -- 172.21.15.78:0/1872687189 wait complete. 2024-08-18T23:47:40.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.620+0000 7f21b695e740 20 client.16020 trim_cache size 0 max 16384 2024-08-18T23:47:40.623 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:47:40.623 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 26, 2024-08-18T23:47:40.623 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24452, 2024-08-18T23:47:40.623 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 12, 2024-08-18T23:47:40.623 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-08-18T23:47:40.624 INFO:teuthology.orchestra.run.smithi078.stdout: "edfda978": { 2024-08-18T23:47:40.624 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 12, 2024-08-18T23:47:40.624 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-08-18T23:47:40.624 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:47:40.624 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-08-18T23:47:40.624 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:47:40.624 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:47:40.624 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-08-18T23:47:40.624 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 150.0, 2024-08-18T23:47:40.624 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-08-18T23:47:40.624 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-08-18T23:47:40.624 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-08-18T23:47:40.625 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:47:40.625 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-08-18T23:47:40.625 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:47:40.625 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:47:40.625 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:47:40.625 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:47:40.625 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:47:40.625 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:47:40.625 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:47:40.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.620+0000 7f21b4560640 1 -- 172.21.15.78:0/3236812532 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f21940a2b80 msgr2=0x7f21940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:40.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.620+0000 7f21b4560640 1 --2- 172.21.15.78:0/3236812532 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f21940a2b80 0x7f21940a5040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f21a40138a0 tx=0x7f21a400ba80 comp rx=0 tx=0).stop 2024-08-18T23:47:40.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.620+0000 7f21b4560640 1 -- 172.21.15.78:0/3236812532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21ac102a70 msgr2=0x7f21ac198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:40.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.620+0000 7f21b4560640 1 --2- 172.21.15.78:0/3236812532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21ac102a70 0x7f21ac198240 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f219c005cc0 tx=0x7f219c02f770 comp rx=0 tx=0).stop 2024-08-18T23:47:40.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.620+0000 7f21b4560640 1 -- 172.21.15.78:0/3236812532 shutdown_connections 2024-08-18T23:47:40.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.620+0000 7f21b4560640 1 --2- 172.21.15.78:0/3236812532 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f21940a2b80 0x7f21940a5040 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:40.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.620+0000 7f21b4560640 1 --2- 172.21.15.78:0/3236812532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21ac10bea0 0x7f21ac19cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:40.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.620+0000 7f21b4560640 1 --2- 172.21.15.78:0/3236812532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21ac103390 0x7f21ac198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:40.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.620+0000 7f21b4560640 1 --2- 172.21.15.78:0/3236812532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21ac102a70 0x7f21ac198240 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:40.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.620+0000 7f21b4560640 1 -- 172.21.15.78:0/3236812532 >> 172.21.15.78:0/3236812532 conn(0x7f21ac0fc4d0 msgr2=0x7f21ac0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:40.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.621+0000 7f21b4560640 1 -- 172.21.15.78:0/3236812532 shutdown_connections 2024-08-18T23:47:40.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:40.621+0000 7f21b4560640 1 -- 172.21.15.78:0/3236812532 wait complete. 2024-08-18T23:47:40.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:40 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:47:40.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:40 smithi161 ceph-mon[22027]: pgmap v451: 129 pgs: 129 active+clean; 447 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 470 KiB/s wr, 223 op/s 2024-08-18T23:47:40.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:40 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:47:40.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:40 smithi123 ceph-mon[21808]: pgmap v451: 129 pgs: 129 active+clean; 447 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 470 KiB/s wr, 223 op/s 2024-08-18T23:47:41.438 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'edfda978', total seconds elapsed: 34.1 2024-08-18T23:47:41.438 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-08-18T23:47:41.439 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:47:41.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:41 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:47:41.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:41 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:47:41.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:41 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:47:41.718 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:47:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:41 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:47:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:41 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:47:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:41 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:47:41.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:41 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:47:41.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:41 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:47:41.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:41 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:47:42.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.263+0000 7f253205e640 1 -- 172.21.15.78:0/2854362799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f252c1078c0 msgr2=0x7f252c107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:42.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.263+0000 7f253205e640 1 --2- 172.21.15.78:0/2854362799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f252c1078c0 0x7f252c107ca0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f251c009930 tx=0x7f251c02a420 comp rx=0 tx=0).stop 2024-08-18T23:47:42.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.263+0000 7f253205e640 1 -- 172.21.15.78:0/2854362799 shutdown_connections 2024-08-18T23:47:42.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.263+0000 7f253205e640 1 --2- 172.21.15.78:0/2854362799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f252c102260 0x7f252c10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:42.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.263+0000 7f253205e640 1 --2- 172.21.15.78:0/2854362799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f252c1018c0 0x7f252c101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:42.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.263+0000 7f253205e640 1 --2- 172.21.15.78:0/2854362799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f252c1078c0 0x7f252c107ca0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:42.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.263+0000 7f253205e640 1 -- 172.21.15.78:0/2854362799 >> 172.21.15.78:0/2854362799 conn(0x7f252c0fd5e0 msgr2=0x7f252c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:42.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.263+0000 7f253205e640 1 -- 172.21.15.78:0/2854362799 shutdown_connections 2024-08-18T23:47:42.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.263+0000 7f253205e640 1 -- 172.21.15.78:0/2854362799 wait complete. 2024-08-18T23:47:42.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.264+0000 7f253205e640 1 Processor -- start 2024-08-18T23:47:42.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.264+0000 7f253205e640 1 -- start start 2024-08-18T23:47:42.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.264+0000 7f253205e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f252c1018c0 0x7f252c19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:42.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.264+0000 7f253205e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f252c102260 0x7f252c19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:42.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.264+0000 7f253205e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f252c1078c0 0x7f252c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:42.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.264+0000 7f253205e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f252c111cc0 con 0x7f252c1018c0 2024-08-18T23:47:42.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.264+0000 7f253205e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f252c111b40 con 0x7f252c1078c0 2024-08-18T23:47:42.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.264+0000 7f253205e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f252c111e40 con 0x7f252c102260 2024-08-18T23:47:42.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.264+0000 7f252b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f252c1018c0 0x7f252c19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:42.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.264+0000 7f252b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f252c1018c0 0x7f252c19a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47526/0 (socket says 172.21.15.78:47526) 2024-08-18T23:47:42.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.264+0000 7f252b7fe640 1 -- 172.21.15.78:0/189522572 learned_addr learned my addr 172.21.15.78:0/189522572 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:42.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.264+0000 7f252bfff640 1 --2- 172.21.15.78:0/189522572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f252c1078c0 0x7f252c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:42.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.264+0000 7f2523fff640 1 --2- 172.21.15.78:0/189522572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f252c102260 0x7f252c19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:42.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.265+0000 7f252b7fe640 1 -- 172.21.15.78:0/189522572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f252c102260 msgr2=0x7f252c19a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:42.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.265+0000 7f252b7fe640 1 --2- 172.21.15.78:0/189522572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f252c102260 0x7f252c19a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:42.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.265+0000 7f252b7fe640 1 -- 172.21.15.78:0/189522572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f252c1078c0 msgr2=0x7f252c19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:42.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.265+0000 7f252b7fe640 1 --2- 172.21.15.78:0/189522572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f252c1078c0 0x7f252c19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:42.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.265+0000 7f252b7fe640 1 -- 172.21.15.78:0/189522572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252c19f290 con 0x7f252c1018c0 2024-08-18T23:47:42.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.265+0000 7f252b7fe640 1 --2- 172.21.15.78:0/189522572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f252c1018c0 0x7f252c19a2e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f251c009230 tx=0x7f251c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:42.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.265+0000 7f25297fa640 1 -- 172.21.15.78:0/189522572 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c03f030 con 0x7f252c1018c0 2024-08-18T23:47:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.265+0000 7f253205e640 1 -- 172.21.15.78:0/189522572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f252c19f520 con 0x7f252c1018c0 2024-08-18T23:47:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.265+0000 7f25297fa640 1 -- 172.21.15.78:0/189522572 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f251c003ad0 con 0x7f252c1018c0 2024-08-18T23:47:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.265+0000 7f25297fa640 1 -- 172.21.15.78:0/189522572 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c035070 con 0x7f252c1018c0 2024-08-18T23:47:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.265+0000 7f253205e640 1 -- 172.21.15.78:0/189522572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f252c1a6e60 con 0x7f252c1018c0 2024-08-18T23:47:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.266+0000 7f253205e640 1 -- 172.21.15.78:0/189522572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f252c103000 con 0x7f252c1018c0 2024-08-18T23:47:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.267+0000 7f25297fa640 1 -- 172.21.15.78:0/189522572 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f251c042030 con 0x7f252c1018c0 2024-08-18T23:47:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.268+0000 7f25297fa640 1 --2- 172.21.15.78:0/189522572 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f25180a2b80 0x7f25180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.268+0000 7f25297fa640 1 -- 172.21.15.78:0/189522572 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f251c0efa50 con 0x7f252c1018c0 2024-08-18T23:47:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.268+0000 7f2523fff640 1 --2- 172.21.15.78:0/189522572 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f25180a2b80 0x7f25180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.268+0000 7f2523fff640 1 --2- 172.21.15.78:0/189522572 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f25180a2b80 0x7f25180a5040 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f252c10e440 tx=0x7f2514005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.272+0000 7f25297fa640 1 -- 172.21.15.78:0/189522572 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f251c0bc620 con 0x7f252c1018c0 2024-08-18T23:47:42.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:42 smithi078 ceph-mon[18440]: pgmap v452: 129 pgs: 129 active+clean; 450 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 472 KiB/s wr, 224 op/s 2024-08-18T23:47:42.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.485+0000 7f253205e640 1 -- 172.21.15.78:0/189522572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f252c1a7030 con 0x7f252c1018c0 2024-08-18T23:47:42.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.486+0000 7f25297fa640 1 -- 172.21.15.78:0/189522572 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11933 (secure 0 0 0) 0x7f251c0bbf20 con 0x7f252c1018c0 2024-08-18T23:47:42.487 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:47:42.487 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":26,"btime":"2024-08-18T23:45:33:306257+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:45:32.491054+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:47:42.488 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 26 2024-08-18T23:47:42.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.488+0000 7f253205e640 1 -- 172.21.15.78:0/189522572 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f25180a2b80 msgr2=0x7f25180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:42.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.488+0000 7f253205e640 1 --2- 172.21.15.78:0/189522572 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f25180a2b80 0x7f25180a5040 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f252c10e440 tx=0x7f2514005f90 comp rx=0 tx=0).stop 2024-08-18T23:47:42.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.488+0000 7f253205e640 1 -- 172.21.15.78:0/189522572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f252c1018c0 msgr2=0x7f252c19a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:42.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.488+0000 7f253205e640 1 --2- 172.21.15.78:0/189522572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f252c1018c0 0x7f252c19a2e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f251c009230 tx=0x7f251c004270 comp rx=0 tx=0).stop 2024-08-18T23:47:42.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.488+0000 7f253205e640 1 -- 172.21.15.78:0/189522572 shutdown_connections 2024-08-18T23:47:42.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.488+0000 7f253205e640 1 --2- 172.21.15.78:0/189522572 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f25180a2b80 0x7f25180a5040 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:42.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.488+0000 7f253205e640 1 --2- 172.21.15.78:0/189522572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f252c1078c0 0x7f252c19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:42.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.488+0000 7f253205e640 1 --2- 172.21.15.78:0/189522572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f252c102260 0x7f252c19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:42.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.488+0000 7f253205e640 1 --2- 172.21.15.78:0/189522572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f252c1018c0 0x7f252c19a2e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:42.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.488+0000 7f253205e640 1 -- 172.21.15.78:0/189522572 >> 172.21.15.78:0/189522572 conn(0x7f252c0fd5e0 msgr2=0x7f252c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:42.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.488+0000 7f253205e640 1 -- 172.21.15.78:0/189522572 shutdown_connections 2024-08-18T23:47:42.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:42.488+0000 7f253205e640 1 -- 172.21.15.78:0/189522572 wait complete. 2024-08-18T23:47:42.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:42 smithi161 ceph-mon[22027]: pgmap v452: 129 pgs: 129 active+clean; 450 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 472 KiB/s wr, 224 op/s 2024-08-18T23:47:42.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:42 smithi123 ceph-mon[21808]: pgmap v452: 129 pgs: 129 active+clean; 450 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 472 KiB/s wr, 224 op/s 2024-08-18T23:47:43.231 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-08-18T23:47:43.231 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-08-18T23:47:43.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:43 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/189522572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:47:43.514 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:47:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:43 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/189522572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:47:43.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:43 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/189522572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:47:44.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.078+0000 7fc698a14640 1 -- 172.21.15.78:0/1553931085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6941078e0 msgr2=0x7fc694107cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:44.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.078+0000 7fc698a14640 1 --2- 172.21.15.78:0/1553931085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6941078e0 0x7fc694107cc0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc684009930 tx=0x7fc68402a420 comp rx=0 tx=0).stop 2024-08-18T23:47:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.078+0000 7fc698a14640 1 -- 172.21.15.78:0/1553931085 shutdown_connections 2024-08-18T23:47:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.078+0000 7fc698a14640 1 --2- 172.21.15.78:0/1553931085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc694102280 0x7fc69410e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.078+0000 7fc698a14640 1 --2- 172.21.15.78:0/1553931085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6941018e0 0x7fc694101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.078+0000 7fc698a14640 1 --2- 172.21.15.78:0/1553931085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6941078e0 0x7fc694107cc0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.078+0000 7fc698a14640 1 -- 172.21.15.78:0/1553931085 >> 172.21.15.78:0/1553931085 conn(0x7fc6940fd600 msgr2=0x7fc6940ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.078+0000 7fc698a14640 1 -- 172.21.15.78:0/1553931085 shutdown_connections 2024-08-18T23:47:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.078+0000 7fc698a14640 1 -- 172.21.15.78:0/1553931085 wait complete. 2024-08-18T23:47:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.079+0000 7fc698a14640 1 Processor -- start 2024-08-18T23:47:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.079+0000 7fc698a14640 1 -- start start 2024-08-18T23:47:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.079+0000 7fc698a14640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6941018e0 0x7fc694074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.079+0000 7fc698a14640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc694102280 0x7fc694074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.079+0000 7fc698a14640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6941078e0 0x7fc694074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.079+0000 7fc698a14640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc694111ce0 con 0x7fc694102280 2024-08-18T23:47:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.079+0000 7fc698a14640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc694111b60 con 0x7fc6941018e0 2024-08-18T23:47:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.079+0000 7fc698a14640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc694111e60 con 0x7fc6941078e0 2024-08-18T23:47:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.079+0000 7fc691d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc694102280 0x7fc694074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.079+0000 7fc691d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc694102280 0x7fc694074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47538/0 (socket says 172.21.15.78:47538) 2024-08-18T23:47:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.079+0000 7fc691d9b640 1 -- 172.21.15.78:0/469891406 learned_addr learned my addr 172.21.15.78:0/469891406 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.079+0000 7fc69259c640 1 --2- 172.21.15.78:0/469891406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6941018e0 0x7fc694074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.080+0000 7fc692d9d640 1 --2- 172.21.15.78:0/469891406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6941078e0 0x7fc694074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.080+0000 7fc691d9b640 1 -- 172.21.15.78:0/469891406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6941078e0 msgr2=0x7fc694074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.080+0000 7fc691d9b640 1 --2- 172.21.15.78:0/469891406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6941078e0 0x7fc694074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.080+0000 7fc691d9b640 1 -- 172.21.15.78:0/469891406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6941018e0 msgr2=0x7fc694074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.080+0000 7fc691d9b640 1 --2- 172.21.15.78:0/469891406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6941018e0 0x7fc694074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.080+0000 7fc691d9b640 1 -- 172.21.15.78:0/469891406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6941a66a0 con 0x7fc694102280 2024-08-18T23:47:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.080+0000 7fc691d9b640 1 --2- 172.21.15.78:0/469891406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc694102280 0x7fc694074760 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00d770 tx=0x7fc67c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.080+0000 7fc68b7fe640 1 -- 172.21.15.78:0/469891406 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c017070 con 0x7fc694102280 2024-08-18T23:47:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.080+0000 7fc68b7fe640 1 -- 172.21.15.78:0/469891406 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67c005500 con 0x7fc694102280 2024-08-18T23:47:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.080+0000 7fc698a14640 1 -- 172.21.15.78:0/469891406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6941a6990 con 0x7fc694102280 2024-08-18T23:47:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.080+0000 7fc68b7fe640 1 -- 172.21.15.78:0/469891406 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c004e00 con 0x7fc694102280 2024-08-18T23:47:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.080+0000 7fc698a14640 1 -- 172.21.15.78:0/469891406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6941a6ed0 con 0x7fc694102280 2024-08-18T23:47:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.082+0000 7fc68b7fe640 1 -- 172.21.15.78:0/469891406 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc67c004fa0 con 0x7fc694102280 2024-08-18T23:47:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.083+0000 7fc69ae12740 1 Processor -- start 2024-08-18T23:47:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.083+0000 7fc69ae12740 1 -- start start 2024-08-18T23:47:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.083+0000 7fc69ae12740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05c130 0x55f4ae05c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.083+0000 7fc69ae12740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae05ca50 0x55f4ae066dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.083+0000 7fc69ae12740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae067310 0x55f4ae069700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.083+0000 7fc69ae12740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f4adfc1a70 con 0x55f4ae05c130 2024-08-18T23:47:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.083+0000 7fc69ae12740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f4adf59780 con 0x55f4ae067310 2024-08-18T23:47:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.083+0000 7fc69ae12740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f4adfbda30 con 0x55f4ae05ca50 2024-08-18T23:47:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.083+0000 7fc692d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05c130 0x55f4ae05c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.083+0000 7fc692d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05c130 0x55f4ae05c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47554/0 (socket says 172.21.15.78:47554) 2024-08-18T23:47:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.083+0000 7fc68b7fe640 1 --2- 172.21.15.78:0/469891406 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc6800a2c00 0x7fc6800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.083+0000 7fc692d9d640 1 -- 172.21.15.78:0/3171475205 learned_addr learned my addr 172.21.15.78:0/3171475205 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.083+0000 7fc691d9b640 1 --2- 172.21.15.78:0/3171475205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae067310 0x55f4ae069700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.084+0000 7fc69259c640 1 --2- 172.21.15.78:0/3171475205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae05ca50 0x55f4ae066dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.084+0000 7fc692d9d640 1 --2- 172.21.15.78:0/469891406 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc6800a2c00 0x7fc6800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.084+0000 7fc68b7fe640 1 -- 172.21.15.78:0/469891406 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc67c0ceda0 con 0x7fc694102280 2024-08-18T23:47:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.084+0000 7fc692d9d640 1 -- 172.21.15.78:0/3171475205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae05ca50 msgr2=0x55f4ae066dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.084+0000 7fc692d9d640 1 --2- 172.21.15.78:0/3171475205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae05ca50 0x55f4ae066dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.084+0000 7fc692d9d640 1 -- 172.21.15.78:0/3171475205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae067310 msgr2=0x55f4ae069700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.084+0000 7fc692d9d640 1 --2- 172.21.15.78:0/3171475205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae067310 0x55f4ae069700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.084+0000 7fc692d9d640 1 -- 172.21.15.78:0/3171475205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4add33b90 con 0x55f4ae05c130 2024-08-18T23:47:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.084+0000 7fc69259c640 1 --2- 172.21.15.78:0/3171475205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae05ca50 0x55f4ae066dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:47:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.084+0000 7fc692d9d640 1 --2- 172.21.15.78:0/3171475205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05c130 0x55f4ae05c510 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0129f0 tx=0x7fc68c012eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce498845c20058ad server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.084+0000 7fc688ff9640 1 -- 172.21.15.78:0/3171475205 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c023070 con 0x55f4ae05c130 2024-08-18T23:47:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.084+0000 7fc692d9d640 1 --2- 172.21.15.78:0/469891406 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc6800a2c00 0x7fc6800a50c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc68c022000 tx=0x7fc68c021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.084+0000 7fc688ff9640 1 -- 172.21.15.78:0/3171475205 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc68c004710 con 0x55f4ae05c130 2024-08-18T23:47:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.084+0000 7fc688ff9640 1 -- 172.21.15.78:0/3171475205 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c01fba0 con 0x55f4ae05c130 2024-08-18T23:47:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.085+0000 7fc69ae12740 1 -- 172.21.15.78:0/3171475205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05c130 msgr2=0x55f4ae05c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.085+0000 7fc69ae12740 1 --2- 172.21.15.78:0/3171475205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05c130 0x55f4ae05c510 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0129f0 tx=0x7fc68c012eb0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.085+0000 7fc69ae12740 1 -- 172.21.15.78:0/3171475205 shutdown_connections 2024-08-18T23:47:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.085+0000 7fc69ae12740 1 --2- 172.21.15.78:0/3171475205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae067310 0x55f4ae069700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.085+0000 7fc69ae12740 1 --2- 172.21.15.78:0/3171475205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae05ca50 0x55f4ae066dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.085+0000 7fc69ae12740 1 --2- 172.21.15.78:0/3171475205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05c130 0x55f4ae05c510 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.085+0000 7fc69ae12740 1 -- 172.21.15.78:0/3171475205 >> 172.21.15.78:0/3171475205 conn(0x55f4ae0586c0 msgr2=0x55f4ae058aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.085+0000 7fc69ae12740 1 -- 172.21.15.78:0/3171475205 shutdown_connections 2024-08-18T23:47:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.085+0000 7fc69ae12740 1 -- 172.21.15.78:0/3171475205 wait complete. 2024-08-18T23:47:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.085+0000 7fc69ae12740 1 -- start start 2024-08-18T23:47:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc69ae12740 1 Processor -- start 2024-08-18T23:47:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc69ae12740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae05c130 0x55f4ae064290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc69ae12740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 0x55f4ae0647d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc69ae12740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae067310 0x55f4ae064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc69ae12740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f4ae069ee0 con 0x55f4ae05ca50 2024-08-18T23:47:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc69ae12740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f4adf7c990 con 0x55f4ae067310 2024-08-18T23:47:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc69ae12740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f4ae06a060 con 0x55f4ae05c130 2024-08-18T23:47:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc69ae12740 10 client.0 _ll_register_callbacks cb 0x55f4adcd1090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc691d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 0x55f4ae0647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc691d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 0x55f4ae0647d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47564/0 (socket says 172.21.15.78:47564) 2024-08-18T23:47:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc691d9b640 1 -- 172.21.15.78:0/1280702147 learned_addr learned my addr 172.21.15.78:0/1280702147 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc692d9d640 1 --2- 172.21.15.78:0/1280702147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae067310 0x55f4ae064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc69259c640 1 --2- 172.21.15.78:0/1280702147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae05c130 0x55f4ae064290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc691d9b640 1 -- 172.21.15.78:0/1280702147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae05c130 msgr2=0x55f4ae064290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc691d9b640 1 --2- 172.21.15.78:0/1280702147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae05c130 0x55f4ae064290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc691d9b640 1 -- 172.21.15.78:0/1280702147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae067310 msgr2=0x55f4ae064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc691d9b640 1 --2- 172.21.15.78:0/1280702147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae067310 0x55f4ae064d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc691d9b640 1 -- 172.21.15.78:0/1280702147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4adf7b6c0 con 0x55f4ae05ca50 2024-08-18T23:47:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc691d9b640 1 --2- 172.21.15.78:0/1280702147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 0x55f4ae0647d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0cc510 tx=0x7fc67c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f128cf95982caec server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.086+0000 7fc679ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:47:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.087+0000 7fc679ffb640 1 -- 172.21.15.78:0/1280702147 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c020070 con 0x55f4ae05ca50 2024-08-18T23:47:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.087+0000 7fc679ffb640 1 -- 172.21.15.78:0/1280702147 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc67c009cc0 con 0x55f4ae05ca50 2024-08-18T23:47:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.087+0000 7fc69ae12740 1 -- 172.21.15.78:0/1280702147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc68c004710 con 0x55f4ae05ca50 2024-08-18T23:47:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.087+0000 7fc679ffb640 1 -- 172.21.15.78:0/1280702147 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc67c09f620 con 0x55f4ae05ca50 2024-08-18T23:47:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.087+0000 7fc69ae12740 10 client.16038 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.087+0000 7fc69ae12740 1 -- 172.21.15.78:0/1280702147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f4ae075c10 con 0x55f4ae05ca50 2024-08-18T23:47:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.087+0000 7fc679ffb640 1 -- 172.21.15.78:0/1280702147 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc68c004710 con 0x55f4ae05ca50 2024-08-18T23:47:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.087+0000 7fc679ffb640 1 -- 172.21.15.78:0/1280702147 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fc67c0bb0b0 con 0x55f4ae05ca50 2024-08-18T23:47:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.087+0000 7fc69ae12740 10 client.16038 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.087+0000 7fc69ae12740 10 client.16038 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:47:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.087+0000 7fc69ae12740 20 client.16038 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.087+0000 7fc69ae12740 1 --2- 172.21.15.78:0/1280702147 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f4ae077020 0x55f4ae079460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.087+0000 7fc69ae12740 4 client.16038 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:47:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.087+0000 7fc69ae12740 1 -- 172.21.15.78:0/1280702147 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4adfcf1a0 con 0x55f4ae077020 2024-08-18T23:47:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.087+0000 7fc69259c640 1 --2- 172.21.15.78:0/1280702147 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f4ae077020 0x55f4ae079460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.088+0000 7fc69259c640 1 --2- 172.21.15.78:0/1280702147 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f4ae077020 0x55f4ae079460 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8d30cd3cc94a9ff server_cookie=1063eb1bde319c8f in_seq=0 out_seq=0 2024-08-18T23:47:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.088+0000 7fc679ffb640 10 client.16038 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.089+0000 7fc679ffb640 1 -- 172.21.15.78:0/1280702147 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4adfcf1a0 con 0x55f4ae077020 2024-08-18T23:47:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.089+0000 7fc679ffb640 10 client.16038 handle_command_reply: tid=0 2024-08-18T23:47:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.089+0000 7fc69ae12740 1 client.16038 shutdown 2024-08-18T23:47:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.090+0000 7fc69ae12740 1 -- 172.21.15.78:0/1280702147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 msgr2=0x55f4ae0647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.090+0000 7fc69ae12740 1 --2- 172.21.15.78:0/1280702147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 0x55f4ae0647d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0cc510 tx=0x7fc67c009c30 comp rx=0 tx=0).stop 2024-08-18T23:47:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.090+0000 7fc69ae12740 1 -- 172.21.15.78:0/1280702147 shutdown_connections 2024-08-18T23:47:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.090+0000 7fc69ae12740 1 --2- 172.21.15.78:0/1280702147 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f4ae077020 0x55f4ae079460 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.090+0000 7fc69ae12740 1 --2- 172.21.15.78:0/1280702147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae067310 0x55f4ae064d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.090+0000 7fc69ae12740 1 --2- 172.21.15.78:0/1280702147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 0x55f4ae0647d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.090+0000 7fc69ae12740 1 --2- 172.21.15.78:0/1280702147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae05c130 0x55f4ae064290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.090+0000 7fc679ffb640 0 client.16038 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.090+0000 7fc69ae12740 1 -- 172.21.15.78:0/1280702147 >> 172.21.15.78:0/1280702147 conn(0x55f4ae0586c0 msgr2=0x55f4ae05e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.090+0000 7fc69ae12740 1 -- 172.21.15.78:0/1280702147 shutdown_connections 2024-08-18T23:47:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.090+0000 7fc69ae12740 1 -- 172.21.15.78:0/1280702147 wait complete. 2024-08-18T23:47:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.090+0000 7fc69ae12740 20 client.16038 trim_cache size 0 max 16384 2024-08-18T23:47:44.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.108+0000 7fc69ae12740 1 Processor -- start 2024-08-18T23:47:44.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.108+0000 7fc69ae12740 1 -- start start 2024-08-18T23:47:44.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.108+0000 7fc69ae12740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae05c130 0x55f4ae05e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.108+0000 7fc69ae12740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 0x55f4ae07ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.108+0000 7fc69ae12740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae067310 0x55f4ae07f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.108+0000 7fc69ae12740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941b3460 con 0x55f4ae05ca50 2024-08-18T23:47:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.108+0000 7fc69ae12740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941b0d30 con 0x55f4ae05c130 2024-08-18T23:47:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.108+0000 7fc69ae12740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941aeb80 con 0x55f4ae067310 2024-08-18T23:47:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.108+0000 7fc691d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 0x55f4ae07ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.109+0000 7fc691d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 0x55f4ae07ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47570/0 (socket says 172.21.15.78:47570) 2024-08-18T23:47:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.109+0000 7fc691d9b640 1 -- 172.21.15.78:0/1951193268 learned_addr learned my addr 172.21.15.78:0/1951193268 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.109+0000 7fc69259c640 1 --2- 172.21.15.78:0/1951193268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae05c130 0x55f4ae05e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.109+0000 7fc692d9d640 1 --2- 172.21.15.78:0/1951193268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae067310 0x55f4ae07f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.109+0000 7fc691d9b640 1 -- 172.21.15.78:0/1951193268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae067310 msgr2=0x55f4ae07f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.109+0000 7fc691d9b640 1 --2- 172.21.15.78:0/1951193268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae067310 0x55f4ae07f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.109+0000 7fc691d9b640 1 -- 172.21.15.78:0/1951193268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae05c130 msgr2=0x55f4ae05e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.109+0000 7fc691d9b640 1 --2- 172.21.15.78:0/1951193268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae05c130 0x55f4ae05e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.109+0000 7fc691d9b640 1 -- 172.21.15.78:0/1951193268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4ae0627e0 con 0x55f4ae05ca50 2024-08-18T23:47:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.109+0000 7fc692d9d640 1 --2- 172.21.15.78:0/1951193268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae067310 0x55f4ae07f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:47:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.109+0000 7fc691d9b640 1 --2- 172.21.15.78:0/1951193268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 0x55f4ae07ed30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0976d0 tx=0x7fc67c0048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcf4fe23fd18158 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.109+0000 7fc67bfff640 1 -- 172.21.15.78:0/1951193268 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c020070 con 0x55f4ae05ca50 2024-08-18T23:47:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.109+0000 7fc67bfff640 1 -- 172.21.15.78:0/1951193268 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67c0a3ee0 con 0x55f4ae05ca50 2024-08-18T23:47:44.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.109+0000 7fc67bfff640 1 -- 172.21.15.78:0/1951193268 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c0978e0 con 0x55f4ae05ca50 2024-08-18T23:47:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.110+0000 7fc69ae12740 1 -- 172.21.15.78:0/1951193268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 msgr2=0x55f4ae07ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.110+0000 7fc69ae12740 1 --2- 172.21.15.78:0/1951193268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 0x55f4ae07ed30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0976d0 tx=0x7fc67c0048a0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.110+0000 7fc69ae12740 1 -- 172.21.15.78:0/1951193268 shutdown_connections 2024-08-18T23:47:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.110+0000 7fc69ae12740 1 --2- 172.21.15.78:0/1951193268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae067310 0x55f4ae07f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.110+0000 7fc69ae12740 1 --2- 172.21.15.78:0/1951193268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 0x55f4ae07ed30 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.110+0000 7fc69ae12740 1 --2- 172.21.15.78:0/1951193268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae05c130 0x55f4ae05e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.110+0000 7fc69ae12740 1 -- 172.21.15.78:0/1951193268 >> 172.21.15.78:0/1951193268 conn(0x55f4ae0586c0 msgr2=0x55f4ae063380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.110+0000 7fc69ae12740 1 -- 172.21.15.78:0/1951193268 shutdown_connections 2024-08-18T23:47:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.110+0000 7fc69ae12740 1 -- 172.21.15.78:0/1951193268 wait complete. 2024-08-18T23:47:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.110+0000 7fc69ae12740 1 -- start start 2024-08-18T23:47:44.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.110+0000 7fc69ae12740 1 Processor -- start 2024-08-18T23:47:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc69ae12740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae05c130 0x55f4ae06b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc69ae12740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 0x55f4ae06bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc69ae12740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae067310 0x55f4ae0e0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc69ae12740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941af1f0 con 0x55f4ae05ca50 2024-08-18T23:47:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc69ae12740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941b0660 con 0x55f4ae067310 2024-08-18T23:47:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc69ae12740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941af8c0 con 0x55f4ae05c130 2024-08-18T23:47:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc69ae12740 10 client.0 _ll_register_callbacks cb 0x7fc67c00de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc691d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 0x55f4ae06bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc691d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 0x55f4ae06bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47572/0 (socket says 172.21.15.78:47572) 2024-08-18T23:47:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc691d9b640 1 -- 172.21.15.78:0/3431335236 learned_addr learned my addr 172.21.15.78:0/3431335236 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc692d9d640 1 --2- 172.21.15.78:0/3431335236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae067310 0x55f4ae0e0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc69259c640 1 --2- 172.21.15.78:0/3431335236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae05c130 0x55f4ae06b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:44.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc691d9b640 1 -- 172.21.15.78:0/3431335236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae05c130 msgr2=0x55f4ae06b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc691d9b640 1 --2- 172.21.15.78:0/3431335236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae05c130 0x55f4ae06b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc691d9b640 1 -- 172.21.15.78:0/3431335236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae067310 msgr2=0x55f4ae0e0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc691d9b640 1 --2- 172.21.15.78:0/3431335236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae067310 0x55f4ae0e0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc691d9b640 1 -- 172.21.15.78:0/3431335236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4ae0e1240 con 0x55f4ae05ca50 2024-08-18T23:47:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc69259c640 1 --2- 172.21.15.78:0/3431335236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae05c130 0x55f4ae06b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:47:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc691d9b640 1 --2- 172.21.15.78:0/3431335236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 0x55f4ae06bc70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0cc880 tx=0x7fc67c0cf0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d18c76757393bc server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc679ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:47:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc679ffb640 1 -- 172.21.15.78:0/3431335236 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c020070 con 0x55f4ae05ca50 2024-08-18T23:47:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc679ffb640 1 -- 172.21.15.78:0/3431335236 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc67c0978e0 con 0x55f4ae05ca50 2024-08-18T23:47:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.111+0000 7fc69ae12740 1 -- 172.21.15.78:0/3431335236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc67c0a3ee0 con 0x55f4ae05ca50 2024-08-18T23:47:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.112+0000 7fc679ffb640 1 -- 172.21.15.78:0/3431335236 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc67c09a6c0 con 0x55f4ae05ca50 2024-08-18T23:47:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.112+0000 7fc69ae12740 10 client.16044 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:44.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.112+0000 7fc69ae12740 1 -- 172.21.15.78:0/3431335236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f4ae0e2460 con 0x55f4ae05ca50 2024-08-18T23:47:44.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.112+0000 7fc679ffb640 1 -- 172.21.15.78:0/3431335236 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67c0a3ee0 con 0x55f4ae05ca50 2024-08-18T23:47:44.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.112+0000 7fc679ffb640 1 -- 172.21.15.78:0/3431335236 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fc67c09a2f0 con 0x55f4ae05ca50 2024-08-18T23:47:44.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.112+0000 7fc69ae12740 10 client.16044 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:44.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.112+0000 7fc69ae12740 10 client.16044 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:47:44.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.112+0000 7fc69ae12740 20 client.16044 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:44.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.112+0000 7fc69ae12740 1 --2- 172.21.15.78:0/3431335236 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f4ae077020 0x55f4ae0e4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:44.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.112+0000 7fc69ae12740 4 client.16044 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-08-18T23:47:44.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.112+0000 7fc69ae12740 1 -- 172.21.15.78:0/3431335236 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55f4adfc94f0 con 0x55f4ae077020 2024-08-18T23:47:44.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.112+0000 7fc69259c640 1 --2- 172.21.15.78:0/3431335236 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f4ae077020 0x55f4ae0e4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:44.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.113+0000 7fc679ffb640 10 client.16044 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:44.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:44.113+0000 7fc69259c640 1 --2- 172.21.15.78:0/3431335236 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f4ae077020 0x55f4ae0e4a40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1d5c098295b2db9 server_cookie=946b66755c7c2240 in_seq=0 out_seq=0 2024-08-18T23:47:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:44 smithi161 ceph-mon[22027]: pgmap v453: 129 pgs: 129 active+clean; 451 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 612 KiB/s wr, 198 op/s 2024-08-18T23:47:44.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:44 smithi078 ceph-mon[18440]: pgmap v453: 129 pgs: 129 active+clean; 451 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 612 KiB/s wr, 198 op/s 2024-08-18T23:47:44.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:44 smithi123 ceph-mon[21808]: pgmap v453: 129 pgs: 129 active+clean; 451 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 612 KiB/s wr, 198 op/s 2024-08-18T23:47:46.982 INFO:tasks.workunit.client.0.smithi078.stdout: 79 75185 90 54418 61 40299 169 2024-08-18T23:47:47.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:47 smithi123 ceph-mon[21808]: pgmap v454: 129 pgs: 129 active+clean; 454 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 821 KiB/s wr, 223 op/s 2024-08-18T23:47:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:47 smithi161 ceph-mon[22027]: pgmap v454: 129 pgs: 129 active+clean; 454 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 821 KiB/s wr, 223 op/s 2024-08-18T23:47:47.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:47 smithi078 ceph-mon[18440]: pgmap v454: 129 pgs: 129 active+clean; 454 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 821 KiB/s wr, 223 op/s 2024-08-18T23:47:49.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:49 smithi123 ceph-mon[21808]: pgmap v455: 129 pgs: 129 active+clean; 458 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2024-08-18T23:47:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:49 smithi161 ceph-mon[22027]: pgmap v455: 129 pgs: 129 active+clean; 458 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2024-08-18T23:47:49.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:49 smithi078 ceph-mon[18440]: pgmap v455: 129 pgs: 129 active+clean; 458 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2024-08-18T23:47:50.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.924+0000 7fc679ffb640 1 -- 172.21.15.78:0/3431335236 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f4adfc94f0 con 0x55f4ae077020 2024-08-18T23:47:50.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.924+0000 7fc679ffb640 10 client.16044 handle_command_reply: tid=0 2024-08-18T23:47:50.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.924+0000 7fc69ae12740 1 client.16044 shutdown 2024-08-18T23:47:50.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.924+0000 7fc69ae12740 1 -- 172.21.15.78:0/3431335236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 msgr2=0x55f4ae06bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:50.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.924+0000 7fc69ae12740 1 --2- 172.21.15.78:0/3431335236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 0x55f4ae06bc70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0cc880 tx=0x7fc67c0cf0a0 comp rx=0 tx=0).stop 2024-08-18T23:47:50.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.924+0000 7fc69ae12740 1 -- 172.21.15.78:0/3431335236 shutdown_connections 2024-08-18T23:47:50.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.924+0000 7fc69ae12740 1 --2- 172.21.15.78:0/3431335236 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f4ae077020 0x55f4ae0e4a40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:50.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.924+0000 7fc69ae12740 1 --2- 172.21.15.78:0/3431335236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4ae067310 0x55f4ae0e0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:50.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.924+0000 7fc69ae12740 1 --2- 172.21.15.78:0/3431335236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4ae05ca50 0x55f4ae06bc70 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:50.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.924+0000 7fc69ae12740 1 --2- 172.21.15.78:0/3431335236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4ae05c130 0x55f4ae06b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:50.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.924+0000 7fc679ffb640 0 client.16044 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:50.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.924+0000 7fc69ae12740 1 -- 172.21.15.78:0/3431335236 >> 172.21.15.78:0/3431335236 conn(0x55f4ae0586c0 msgr2=0x55f4ae065af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:50.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.924+0000 7fc69ae12740 1 -- 172.21.15.78:0/3431335236 shutdown_connections 2024-08-18T23:47:50.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.924+0000 7fc69ae12740 1 -- 172.21.15.78:0/3431335236 wait complete. 2024-08-18T23:47:50.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.924+0000 7fc69ae12740 20 client.16044 trim_cache size 0 max 16384 2024-08-18T23:47:50.927 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:47:50.927 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 26, 2024-08-18T23:47:50.927 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24452, 2024-08-18T23:47:50.928 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 14, 2024-08-18T23:47:50.928 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-08-18T23:47:50.928 INFO:teuthology.orchestra.run.smithi078.stdout: "f640679b": { 2024-08-18T23:47:50.928 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 14, 2024-08-18T23:47:50.928 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-08-18T23:47:50.928 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:47:50.928 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-08-18T23:47:50.928 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:47:50.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:47:50.928 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-08-18T23:47:50.928 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 150.0, 2024-08-18T23:47:50.928 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-08-18T23:47:50.928 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-08-18T23:47:50.929 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-08-18T23:47:50.929 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:47:50.929 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-08-18T23:47:50.929 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:47:50.929 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:47:50.929 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:47:50.929 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:47:50.929 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:47:50.929 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:47:50.929 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:47:50.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.925+0000 7fc698a14640 1 -- 172.21.15.78:0/469891406 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc6800a2c00 msgr2=0x7fc6800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:50.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.925+0000 7fc698a14640 1 --2- 172.21.15.78:0/469891406 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc6800a2c00 0x7fc6800a50c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc68c022000 tx=0x7fc68c021000 comp rx=0 tx=0).stop 2024-08-18T23:47:50.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.925+0000 7fc698a14640 1 -- 172.21.15.78:0/469891406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc694102280 msgr2=0x7fc694074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:50.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.925+0000 7fc698a14640 1 --2- 172.21.15.78:0/469891406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc694102280 0x7fc694074760 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00d770 tx=0x7fc67c00dc30 comp rx=0 tx=0).stop 2024-08-18T23:47:50.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.925+0000 7fc698a14640 1 -- 172.21.15.78:0/469891406 shutdown_connections 2024-08-18T23:47:50.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.925+0000 7fc698a14640 1 --2- 172.21.15.78:0/469891406 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc6800a2c00 0x7fc6800a50c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:50.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.925+0000 7fc698a14640 1 --2- 172.21.15.78:0/469891406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6941078e0 0x7fc694074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:50.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.925+0000 7fc698a14640 1 --2- 172.21.15.78:0/469891406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc694102280 0x7fc694074760 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:50.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.925+0000 7fc698a14640 1 --2- 172.21.15.78:0/469891406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6941018e0 0x7fc694074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:50.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.925+0000 7fc698a14640 1 -- 172.21.15.78:0/469891406 >> 172.21.15.78:0/469891406 conn(0x7fc6940fd600 msgr2=0x7fc6940fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:50.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.925+0000 7fc698a14640 1 -- 172.21.15.78:0/469891406 shutdown_connections 2024-08-18T23:47:50.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:50.925+0000 7fc698a14640 1 -- 172.21.15.78:0/469891406 wait complete. 2024-08-18T23:47:51.544 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:47:51.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:51 smithi123 ceph-mon[21808]: pgmap v456: 129 pgs: 129 active+clean; 460 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-08-18T23:47:51.568 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:51 smithi078 ceph-mon[18440]: pgmap v456: 129 pgs: 129 active+clean; 460 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-08-18T23:47:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:51 smithi161 ceph-mon[22027]: pgmap v456: 129 pgs: 129 active+clean; 460 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-08-18T23:47:51.771 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f640679b', quiesce took 10.3 seconds. Will release after: 19.7 2024-08-18T23:47:51.810 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:47:52.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.292+0000 7fe18dc53640 1 -- 172.21.15.78:0/876482688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1881022a0 msgr2=0x7fe1881026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:52.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.292+0000 7fe18dc53640 1 --2- 172.21.15.78:0/876482688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1881022a0 0x7fe1881026e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe178009900 tx=0x7fe17802a480 comp rx=0 tx=0).stop 2024-08-18T23:47:52.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.292+0000 7fe18dc53640 1 -- 172.21.15.78:0/876482688 shutdown_connections 2024-08-18T23:47:52.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.292+0000 7fe18dc53640 1 --2- 172.21.15.78:0/876482688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe188102c20 0x7fe18810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.292+0000 7fe18dc53640 1 --2- 172.21.15.78:0/876482688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1881022a0 0x7fe1881026e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.292+0000 7fe18dc53640 1 --2- 172.21.15.78:0/876482688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe188100b50 0x7fe188100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.292+0000 7fe18dc53640 1 -- 172.21.15.78:0/876482688 >> 172.21.15.78:0/876482688 conn(0x7fe1880fc500 msgr2=0x7fe1880fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:52.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.293+0000 7fe18dc53640 1 -- 172.21.15.78:0/876482688 shutdown_connections 2024-08-18T23:47:52.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.293+0000 7fe18dc53640 1 -- 172.21.15.78:0/876482688 wait complete. 2024-08-18T23:47:52.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.293+0000 7fe18dc53640 1 Processor -- start 2024-08-18T23:47:52.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.293+0000 7fe18dc53640 1 -- start start 2024-08-18T23:47:52.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.293+0000 7fe18dc53640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe188100b50 0x7fe1880fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:52.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.293+0000 7fe18dc53640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1881022a0 0x7fe1880fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:52.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.293+0000 7fe18dc53640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe188102c20 0x7fe1880fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:52.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.293+0000 7fe18dc53640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe188110cd0 con 0x7fe188102c20 2024-08-18T23:47:52.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.293+0000 7fe18dc53640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe188110b50 con 0x7fe188100b50 2024-08-18T23:47:52.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.293+0000 7fe18dc53640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe188110e50 con 0x7fe1881022a0 2024-08-18T23:47:52.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.294+0000 7fe187fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe188102c20 0x7fe1880fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:52.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.294+0000 7fe1877fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe188100b50 0x7fe1880fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:52.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.294+0000 7fe187fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe188102c20 0x7fe1880fd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42842/0 (socket says 172.21.15.78:42842) 2024-08-18T23:47:52.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.294+0000 7fe1877fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe188100b50 0x7fe1880fc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:56180/0 (socket says 172.21.15.78:56180) 2024-08-18T23:47:52.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.294+0000 7fe187fff640 1 -- 172.21.15.78:0/2906400373 learned_addr learned my addr 172.21.15.78:0/2906400373 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:52.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.294+0000 7fe186ffd640 1 --2- 172.21.15.78:0/2906400373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1881022a0 0x7fe1880fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:52.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.294+0000 7fe187fff640 1 -- 172.21.15.78:0/2906400373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1881022a0 msgr2=0x7fe1880fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:52.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.294+0000 7fe187fff640 1 --2- 172.21.15.78:0/2906400373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1881022a0 0x7fe1880fcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.294+0000 7fe187fff640 1 -- 172.21.15.78:0/2906400373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe188100b50 msgr2=0x7fe1880fc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:52.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.294+0000 7fe187fff640 1 --2- 172.21.15.78:0/2906400373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe188100b50 0x7fe1880fc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.294+0000 7fe187fff640 1 -- 172.21.15.78:0/2906400373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1881a67f0 con 0x7fe188102c20 2024-08-18T23:47:52.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.294+0000 7fe186ffd640 1 --2- 172.21.15.78:0/2906400373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1881022a0 0x7fe1880fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:47:52.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.294+0000 7fe1877fe640 1 --2- 172.21.15.78:0/2906400373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe188100b50 0x7fe1880fc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:52.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.294+0000 7fe187fff640 1 --2- 172.21.15.78:0/2906400373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe188102c20 0x7fe1880fd430 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe18000b970 tx=0x7fe18000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:52.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.294+0000 7fe184ff9640 1 -- 172.21.15.78:0/2906400373 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180019070 con 0x7fe188102c20 2024-08-18T23:47:52.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.294+0000 7fe184ff9640 1 -- 172.21.15.78:0/2906400373 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe18000cb40 con 0x7fe188102c20 2024-08-18T23:47:52.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.294+0000 7fe18dc53640 1 -- 172.21.15.78:0/2906400373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1881a6ae0 con 0x7fe188102c20 2024-08-18T23:47:52.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.294+0000 7fe184ff9640 1 -- 172.21.15.78:0/2906400373 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180007650 con 0x7fe188102c20 2024-08-18T23:47:52.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.294+0000 7fe18dc53640 1 -- 172.21.15.78:0/2906400373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1881a6fd0 con 0x7fe188102c20 2024-08-18T23:47:52.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.297+0000 7fe184ff9640 1 -- 172.21.15.78:0/2906400373 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe18001c070 con 0x7fe188102c20 2024-08-18T23:47:52.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.297+0000 7fe184ff9640 1 --2- 172.21.15.78:0/2906400373 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe1740a2b80 0x7fe1740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:52.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.297+0000 7fe190051740 1 Processor -- start 2024-08-18T23:47:52.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.297+0000 7fe1877fe640 1 --2- 172.21.15.78:0/2906400373 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe1740a2b80 0x7fe1740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:52.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.297+0000 7fe184ff9640 1 -- 172.21.15.78:0/2906400373 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fe1800ce9b0 con 0x7fe188102c20 2024-08-18T23:47:52.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.297+0000 7fe190051740 1 -- start start 2024-08-18T23:47:52.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.298+0000 7fe190051740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42130 0x560296c42510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:52.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.298+0000 7fe190051740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c42a50 0x560296c556c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:52.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.298+0000 7fe190051740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c55c00 0x560296c57ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:52.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.298+0000 7fe190051740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560296ba7980 con 0x560296c42130 2024-08-18T23:47:52.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.298+0000 7fe190051740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560296b3fa70 con 0x560296c42a50 2024-08-18T23:47:52.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.298+0000 7fe190051740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560296ba3940 con 0x560296c55c00 2024-08-18T23:47:52.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.298+0000 7fe1877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42130 0x560296c42510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:52.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.298+0000 7fe1877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42130 0x560296c42510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42852/0 (socket says 172.21.15.78:42852) 2024-08-18T23:47:52.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.298+0000 7fe1877fe640 1 -- 172.21.15.78:0/86970998 learned_addr learned my addr 172.21.15.78:0/86970998 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:52.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.298+0000 7fe186ffd640 1 --2- 172.21.15.78:0/86970998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c42a50 0x560296c556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:52.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.298+0000 7fe1877fe640 1 --2- 172.21.15.78:0/2906400373 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe1740a2b80 0x7fe1740a5040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe170009fc0 tx=0x7fe170005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:52.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.298+0000 7fe187fff640 1 --2- 172.21.15.78:0/86970998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c55c00 0x560296c57ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:52.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.298+0000 7fe1877fe640 1 -- 172.21.15.78:0/86970998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c55c00 msgr2=0x560296c57ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:52.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.298+0000 7fe1877fe640 1 --2- 172.21.15.78:0/86970998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c55c00 0x560296c57ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.298+0000 7fe1877fe640 1 -- 172.21.15.78:0/86970998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c42a50 msgr2=0x560296c556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:52.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.298+0000 7fe1877fe640 1 --2- 172.21.15.78:0/86970998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c42a50 0x560296c556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.298+0000 7fe1877fe640 1 -- 172.21.15.78:0/86970998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560296ba26e0 con 0x560296c42130 2024-08-18T23:47:52.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.299+0000 7fe1877fe640 1 --2- 172.21.15.78:0/86970998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42130 0x560296c42510 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe1700042c0 tx=0x7fe170014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33719cc2820f8fca server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:52.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.299+0000 7fe17dffb640 1 -- 172.21.15.78:0/86970998 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170026470 con 0x560296c42130 2024-08-18T23:47:52.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.299+0000 7fe17dffb640 1 -- 172.21.15.78:0/86970998 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe170026af0 con 0x560296c42130 2024-08-18T23:47:52.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.299+0000 7fe17dffb640 1 -- 172.21.15.78:0/86970998 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17000fcd0 con 0x560296c42130 2024-08-18T23:47:52.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.299+0000 7fe190051740 1 -- 172.21.15.78:0/86970998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42130 msgr2=0x560296c42510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:52.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.299+0000 7fe190051740 1 --2- 172.21.15.78:0/86970998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42130 0x560296c42510 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe1700042c0 tx=0x7fe170014000 comp rx=0 tx=0).stop 2024-08-18T23:47:52.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.300+0000 7fe190051740 1 -- 172.21.15.78:0/86970998 shutdown_connections 2024-08-18T23:47:52.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.300+0000 7fe190051740 1 --2- 172.21.15.78:0/86970998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c55c00 0x560296c57ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.300+0000 7fe190051740 1 --2- 172.21.15.78:0/86970998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c42a50 0x560296c556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.300+0000 7fe190051740 1 --2- 172.21.15.78:0/86970998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42130 0x560296c42510 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.300+0000 7fe190051740 1 -- 172.21.15.78:0/86970998 >> 172.21.15.78:0/86970998 conn(0x560296c3e7a0 msgr2=0x560296c3eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:52.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.300+0000 7fe190051740 1 -- 172.21.15.78:0/86970998 shutdown_connections 2024-08-18T23:47:52.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.300+0000 7fe190051740 1 -- 172.21.15.78:0/86970998 wait complete. 2024-08-18T23:47:52.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.300+0000 7fe190051740 1 -- start start 2024-08-18T23:47:52.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.300+0000 7fe190051740 1 Processor -- start 2024-08-18T23:47:52.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.300+0000 7fe190051740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c42130 0x560296c52c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:52.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.300+0000 7fe190051740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42a50 0x560296c53140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:52.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.300+0000 7fe190051740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c55c00 0x560296c53680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:52.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.300+0000 7fe190051740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560296c58770 con 0x560296c42a50 2024-08-18T23:47:52.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.300+0000 7fe190051740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560296b62aa0 con 0x560296c55c00 2024-08-18T23:47:52.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.300+0000 7fe190051740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560296c588f0 con 0x560296c42130 2024-08-18T23:47:52.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.300+0000 7fe186ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42a50 0x560296c53140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:52.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.301+0000 7fe186ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42a50 0x560296c53140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42854/0 (socket says 172.21.15.78:42854) 2024-08-18T23:47:52.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.301+0000 7fe186ffd640 1 -- 172.21.15.78:0/2978429007 learned_addr learned my addr 172.21.15.78:0/2978429007 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:52.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.301+0000 7fe190051740 10 client.0 _ll_register_callbacks cb 0x560296b51e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:52.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.301+0000 7fe1877fe640 1 --2- 172.21.15.78:0/2978429007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c42130 0x560296c52c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:52.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.301+0000 7fe187fff640 1 --2- 172.21.15.78:0/2978429007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c55c00 0x560296c53680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:52.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.301+0000 7fe186ffd640 1 -- 172.21.15.78:0/2978429007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c42130 msgr2=0x560296c52c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:52.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.301+0000 7fe186ffd640 1 --2- 172.21.15.78:0/2978429007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c42130 0x560296c52c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.301+0000 7fe186ffd640 1 -- 172.21.15.78:0/2978429007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c55c00 msgr2=0x560296c53680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:52.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.301+0000 7fe186ffd640 1 --2- 172.21.15.78:0/2978429007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c55c00 0x560296c53680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.301+0000 7fe186ffd640 1 -- 172.21.15.78:0/2978429007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560296b46650 con 0x560296c42a50 2024-08-18T23:47:52.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.301+0000 7fe186ffd640 1 --2- 172.21.15.78:0/2978429007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42a50 0x560296c53140 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe178005c40 tx=0x7fe178005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb82ae6bf310d59 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:52.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.301+0000 7fe166ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:47:52.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.301+0000 7fe166ffd640 1 -- 172.21.15.78:0/2978429007 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe178035040 con 0x560296c42a50 2024-08-18T23:47:52.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.301+0000 7fe190051740 1 -- 172.21.15.78:0/2978429007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe170026af0 con 0x560296c42a50 2024-08-18T23:47:52.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.301+0000 7fe166ffd640 1 -- 172.21.15.78:0/2978429007 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fe1780048c0 con 0x560296c42a50 2024-08-18T23:47:52.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.301+0000 7fe166ffd640 1 -- 172.21.15.78:0/2978429007 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe17804e050 con 0x560296c42a50 2024-08-18T23:47:52.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.301+0000 7fe190051740 10 client.16056 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:52.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.301+0000 7fe190051740 1 -- 172.21.15.78:0/2978429007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5602969193c0 con 0x560296c42a50 2024-08-18T23:47:52.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.302+0000 7fe166ffd640 1 -- 172.21.15.78:0/2978429007 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe170026af0 con 0x560296c42a50 2024-08-18T23:47:52.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.302+0000 7fe166ffd640 1 -- 172.21.15.78:0/2978429007 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fe178031540 con 0x560296c42a50 2024-08-18T23:47:52.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.302+0000 7fe190051740 10 client.16056 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:52.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.302+0000 7fe190051740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:47:52.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.302+0000 7fe190051740 20 client.16056 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:52.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.302+0000 7fe190051740 1 --2- 172.21.15.78:0/2978429007 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x560296c657c0 0x560296c67c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:52.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.302+0000 7fe190051740 4 client.16056 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:47:52.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.302+0000 7fe190051740 1 -- 172.21.15.78:0/2978429007 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560296bb4ef0 con 0x560296c657c0 2024-08-18T23:47:52.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.302+0000 7fe1877fe640 1 --2- 172.21.15.78:0/2978429007 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x560296c657c0 0x560296c67c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:52.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.302+0000 7fe166ffd640 10 client.16056 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:52.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.303+0000 7fe1877fe640 1 --2- 172.21.15.78:0/2978429007 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x560296c657c0 0x560296c67c00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd72bfcf7b567697 server_cookie=8887af6065f96afc in_seq=0 out_seq=0 2024-08-18T23:47:52.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.304+0000 7fe166ffd640 1 -- 172.21.15.78:0/2978429007 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560296bb4ef0 con 0x560296c657c0 2024-08-18T23:47:52.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.304+0000 7fe166ffd640 10 client.16056 handle_command_reply: tid=0 2024-08-18T23:47:52.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.304+0000 7fe190051740 1 client.16056 shutdown 2024-08-18T23:47:52.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.304+0000 7fe190051740 1 -- 172.21.15.78:0/2978429007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42a50 msgr2=0x560296c53140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:52.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.304+0000 7fe190051740 1 --2- 172.21.15.78:0/2978429007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42a50 0x560296c53140 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe178005c40 tx=0x7fe178005250 comp rx=0 tx=0).stop 2024-08-18T23:47:52.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.304+0000 7fe190051740 1 -- 172.21.15.78:0/2978429007 shutdown_connections 2024-08-18T23:47:52.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.304+0000 7fe190051740 1 --2- 172.21.15.78:0/2978429007 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x560296c657c0 0x560296c67c00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.304+0000 7fe190051740 1 --2- 172.21.15.78:0/2978429007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c55c00 0x560296c53680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.304+0000 7fe190051740 1 --2- 172.21.15.78:0/2978429007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42a50 0x560296c53140 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.304+0000 7fe190051740 1 --2- 172.21.15.78:0/2978429007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c42130 0x560296c52c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.304+0000 7fe166ffd640 0 client.16056 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:52.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.304+0000 7fe190051740 1 -- 172.21.15.78:0/2978429007 >> 172.21.15.78:0/2978429007 conn(0x560296c3e7a0 msgr2=0x560296c44430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:52.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.304+0000 7fe190051740 1 -- 172.21.15.78:0/2978429007 shutdown_connections 2024-08-18T23:47:52.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.304+0000 7fe190051740 1 -- 172.21.15.78:0/2978429007 wait complete. 2024-08-18T23:47:52.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.304+0000 7fe190051740 20 client.16056 trim_cache size 0 max 16384 2024-08-18T23:47:52.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.323+0000 7fe190051740 1 Processor -- start 2024-08-18T23:47:52.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.323+0000 7fe190051740 1 -- start start 2024-08-18T23:47:52.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.323+0000 7fe190051740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c42130 0x560296c534d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:52.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.323+0000 7fe190051740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c42a50 0x560296c6d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:52.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.323+0000 7fe190051740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c55c00 0x560296c6da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:52.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.323+0000 7fe190051740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881af1a0 con 0x560296c55c00 2024-08-18T23:47:52.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.323+0000 7fe190051740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881aca70 con 0x560296c42a50 2024-08-18T23:47:52.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.323+0000 7fe190051740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881aa920 con 0x560296c42130 2024-08-18T23:47:52.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.323+0000 7fe187fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c55c00 0x560296c6da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:52.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.323+0000 7fe186ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c42a50 0x560296c6d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:52.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.323+0000 7fe187fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c55c00 0x560296c6da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42866/0 (socket says 172.21.15.78:42866) 2024-08-18T23:47:52.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.323+0000 7fe187fff640 1 -- 172.21.15.78:0/1912964190 learned_addr learned my addr 172.21.15.78:0/1912964190 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:52.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.323+0000 7fe1877fe640 1 --2- 172.21.15.78:0/1912964190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c42130 0x560296c534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:52.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.323+0000 7fe186ffd640 1 -- 172.21.15.78:0/1912964190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c42130 msgr2=0x560296c534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:52.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.323+0000 7fe186ffd640 1 --2- 172.21.15.78:0/1912964190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c42130 0x560296c534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.323+0000 7fe186ffd640 1 -- 172.21.15.78:0/1912964190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c55c00 msgr2=0x560296c6da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:52.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.323+0000 7fe186ffd640 1 --2- 172.21.15.78:0/1912964190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c55c00 0x560296c6da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.323+0000 7fe186ffd640 1 -- 172.21.15.78:0/1912964190 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560296c64380 con 0x560296c42a50 2024-08-18T23:47:52.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.323+0000 7fe187fff640 1 --2- 172.21.15.78:0/1912964190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c55c00 0x560296c6da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:52.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.324+0000 7fe186ffd640 1 --2- 172.21.15.78:0/1912964190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c42a50 0x560296c6d520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe178006010 tx=0x7fe178001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f21271b3e987c479 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:52.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.324+0000 7fe17d7fa640 1 -- 172.21.15.78:0/1912964190 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe178035070 con 0x560296c42a50 2024-08-18T23:47:52.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.324+0000 7fe17d7fa640 1 -- 172.21.15.78:0/1912964190 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe178040070 con 0x560296c42a50 2024-08-18T23:47:52.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.324+0000 7fe17d7fa640 1 -- 172.21.15.78:0/1912964190 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe178004ba0 con 0x560296c42a50 2024-08-18T23:47:52.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.325+0000 7fe190051740 1 -- 172.21.15.78:0/1912964190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c42a50 msgr2=0x560296c6d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:52.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.325+0000 7fe190051740 1 --2- 172.21.15.78:0/1912964190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c42a50 0x560296c6d520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe178006010 tx=0x7fe178001e70 comp rx=0 tx=0).stop 2024-08-18T23:47:52.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.325+0000 7fe190051740 1 -- 172.21.15.78:0/1912964190 shutdown_connections 2024-08-18T23:47:52.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.325+0000 7fe190051740 1 --2- 172.21.15.78:0/1912964190 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c55c00 0x560296c6da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.325+0000 7fe190051740 1 --2- 172.21.15.78:0/1912964190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c42a50 0x560296c6d520 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.325+0000 7fe190051740 1 --2- 172.21.15.78:0/1912964190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c42130 0x560296c534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.325+0000 7fe190051740 1 -- 172.21.15.78:0/1912964190 >> 172.21.15.78:0/1912964190 conn(0x560296c3e7a0 msgr2=0x560296c51cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:52.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.325+0000 7fe190051740 1 -- 172.21.15.78:0/1912964190 shutdown_connections 2024-08-18T23:47:52.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.325+0000 7fe190051740 1 -- 172.21.15.78:0/1912964190 wait complete. 2024-08-18T23:47:52.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.325+0000 7fe190051740 1 -- start start 2024-08-18T23:47:52.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.325+0000 7fe190051740 1 Processor -- start 2024-08-18T23:47:52.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe190051740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c42130 0x560296c592f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:52.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe190051740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42a50 0x560296c59830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:52.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe190051740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c55c00 0x560296cc84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:52.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe190051740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881aaf70 con 0x560296c42a50 2024-08-18T23:47:52.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe190051740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881ac3a0 con 0x560296c55c00 2024-08-18T23:47:52.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe190051740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881ab600 con 0x560296c42130 2024-08-18T23:47:52.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe190051740 10 client.0 _ll_register_callbacks cb 0x7fe178009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:52.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe186ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42a50 0x560296c59830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:52.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe1877fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c42130 0x560296c592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:52.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe187fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c55c00 0x560296cc84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:52.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe186ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42a50 0x560296c59830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42870/0 (socket says 172.21.15.78:42870) 2024-08-18T23:47:52.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe186ffd640 1 -- 172.21.15.78:0/1810916062 learned_addr learned my addr 172.21.15.78:0/1810916062 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:52.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe186ffd640 1 -- 172.21.15.78:0/1810916062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c42130 msgr2=0x560296c592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:52.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe186ffd640 1 --2- 172.21.15.78:0/1810916062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c42130 0x560296c592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe186ffd640 1 -- 172.21.15.78:0/1810916062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c55c00 msgr2=0x560296cc84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:52.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe186ffd640 1 --2- 172.21.15.78:0/1810916062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c55c00 0x560296cc84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe186ffd640 1 -- 172.21.15.78:0/1810916062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560296cc8af0 con 0x560296c42a50 2024-08-18T23:47:52.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe187fff640 1 --2- 172.21.15.78:0/1810916062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c55c00 0x560296cc84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:52.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe186ffd640 1 --2- 172.21.15.78:0/1810916062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42a50 0x560296c59830 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe1780314e0 tx=0x7fe178031ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9acf643ce435cc server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:52.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe166ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:47:52.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe166ffd640 1 -- 172.21.15.78:0/1810916062 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe178035040 con 0x560296c42a50 2024-08-18T23:47:52.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe190051740 1 -- 172.21.15.78:0/1810916062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe178040070 con 0x560296c42a50 2024-08-18T23:47:52.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.326+0000 7fe166ffd640 1 -- 172.21.15.78:0/1810916062 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fe17804b040 con 0x560296c42a50 2024-08-18T23:47:52.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.327+0000 7fe166ffd640 1 -- 172.21.15.78:0/1810916062 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe178005650 con 0x560296c42a50 2024-08-18T23:47:52.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.327+0000 7fe190051740 10 client.16062 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:52.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.327+0000 7fe190051740 1 -- 172.21.15.78:0/1810916062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560296ccfe50 con 0x560296c42a50 2024-08-18T23:47:52.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.327+0000 7fe166ffd640 1 -- 172.21.15.78:0/1810916062 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe178040070 con 0x560296c42a50 2024-08-18T23:47:52.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.327+0000 7fe166ffd640 1 -- 172.21.15.78:0/1810916062 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fe1780037b0 con 0x560296c42a50 2024-08-18T23:47:52.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.327+0000 7fe190051740 10 client.16062 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:52.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.327+0000 7fe190051740 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:47:52.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.327+0000 7fe190051740 20 client.16062 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:52.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.327+0000 7fe190051740 1 --2- 172.21.15.78:0/1810916062 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x560296cd0590 0x560296cd29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:52.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.327+0000 7fe190051740 4 client.16062 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:47:52.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.327+0000 7fe190051740 1 -- 172.21.15.78:0/1810916062 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560296baf400 con 0x560296cd0590 2024-08-18T23:47:52.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.327+0000 7fe1877fe640 1 --2- 172.21.15.78:0/1810916062 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x560296cd0590 0x560296cd29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:52.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.328+0000 7fe1877fe640 1 --2- 172.21.15.78:0/1810916062 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x560296cd0590 0x560296cd29b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b7decefa2a7144b server_cookie=fe28a99bc68fb1ef in_seq=0 out_seq=0 2024-08-18T23:47:52.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.328+0000 7fe166ffd640 10 client.16062 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:52.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.329+0000 7fe166ffd640 1 -- 172.21.15.78:0/1810916062 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560296baf400 con 0x560296cd0590 2024-08-18T23:47:52.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.329+0000 7fe166ffd640 10 client.16062 handle_command_reply: tid=0 2024-08-18T23:47:52.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.329+0000 7fe190051740 1 client.16062 shutdown 2024-08-18T23:47:52.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.329+0000 7fe190051740 1 -- 172.21.15.78:0/1810916062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42a50 msgr2=0x560296c59830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:52.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.329+0000 7fe190051740 1 --2- 172.21.15.78:0/1810916062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42a50 0x560296c59830 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe1780314e0 tx=0x7fe178031ec0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe190051740 1 -- 172.21.15.78:0/1810916062 shutdown_connections 2024-08-18T23:47:52.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe190051740 1 --2- 172.21.15.78:0/1810916062 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x560296cd0590 0x560296cd29b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe190051740 1 --2- 172.21.15.78:0/1810916062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560296c55c00 0x560296cc84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe190051740 1 --2- 172.21.15.78:0/1810916062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560296c42a50 0x560296c59830 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe190051740 1 --2- 172.21.15.78:0/1810916062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560296c42130 0x560296c592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe190051740 1 -- 172.21.15.78:0/1810916062 >> 172.21.15.78:0/1810916062 conn(0x560296c3e7a0 msgr2=0x560296c3d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:52.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe166ffd640 0 client.16062 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:52.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe190051740 1 -- 172.21.15.78:0/1810916062 shutdown_connections 2024-08-18T23:47:52.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe190051740 1 -- 172.21.15.78:0/1810916062 wait complete. 2024-08-18T23:47:52.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe190051740 20 client.16062 trim_cache size 0 max 16384 2024-08-18T23:47:52.360 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:47:52.360 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-08-18T23:47:52.360 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-08-18T23:47:52.361 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:47:52.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe18dc53640 1 -- 172.21.15.78:0/2906400373 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe1740a2b80 msgr2=0x7fe1740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:52.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe18dc53640 1 --2- 172.21.15.78:0/2906400373 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe1740a2b80 0x7fe1740a5040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe170009fc0 tx=0x7fe170005f90 comp rx=0 tx=0).stop 2024-08-18T23:47:52.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe18dc53640 1 -- 172.21.15.78:0/2906400373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe188102c20 msgr2=0x7fe1880fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:52.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe18dc53640 1 --2- 172.21.15.78:0/2906400373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe188102c20 0x7fe1880fd430 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe18000b970 tx=0x7fe18000be30 comp rx=0 tx=0).stop 2024-08-18T23:47:52.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe18dc53640 1 -- 172.21.15.78:0/2906400373 shutdown_connections 2024-08-18T23:47:52.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe18dc53640 1 --2- 172.21.15.78:0/2906400373 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe1740a2b80 0x7fe1740a5040 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe18dc53640 1 --2- 172.21.15.78:0/2906400373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe188102c20 0x7fe1880fd430 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe18dc53640 1 --2- 172.21.15.78:0/2906400373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1881022a0 0x7fe1880fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe18dc53640 1 --2- 172.21.15.78:0/2906400373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe188100b50 0x7fe1880fc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:52.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.330+0000 7fe18dc53640 1 -- 172.21.15.78:0/2906400373 >> 172.21.15.78:0/2906400373 conn(0x7fe1880fc500 msgr2=0x7fe188069ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:52.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.331+0000 7fe18dc53640 1 -- 172.21.15.78:0/2906400373 shutdown_connections 2024-08-18T23:47:52.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:52.331+0000 7fe18dc53640 1 -- 172.21.15.78:0/2906400373 wait complete. 2024-08-18T23:47:53.038 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-18T23:47:53.039 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:47:53.345 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:47:53.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:53 smithi123 ceph-mon[21808]: pgmap v457: 129 pgs: 129 active+clean; 467 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-08-18T23:47:53.606 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:53 smithi078 ceph-mon[18440]: pgmap v457: 129 pgs: 129 active+clean; 467 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-08-18T23:47:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:53 smithi161 ceph-mon[22027]: pgmap v457: 129 pgs: 129 active+clean; 467 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-08-18T23:47:53.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.839+0000 7ff5e68b4640 1 -- 172.21.15.78:0/4050702587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5e0100b50 msgr2=0x7ff5e0100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:53.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.839+0000 7ff5e68b4640 1 --2- 172.21.15.78:0/4050702587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5e0100b50 0x7ff5e0100f30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0009900 tx=0x7ff5d002a480 comp rx=0 tx=0).stop 2024-08-18T23:47:53.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.839+0000 7ff5e68b4640 1 -- 172.21.15.78:0/4050702587 shutdown_connections 2024-08-18T23:47:53.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.839+0000 7ff5e68b4640 1 --2- 172.21.15.78:0/4050702587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e0102c20 0x7ff5e010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:53.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.839+0000 7ff5e68b4640 1 --2- 172.21.15.78:0/4050702587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e01022a0 0x7ff5e01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:53.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.839+0000 7ff5e68b4640 1 --2- 172.21.15.78:0/4050702587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5e0100b50 0x7ff5e0100f30 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:53.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.839+0000 7ff5e68b4640 1 -- 172.21.15.78:0/4050702587 >> 172.21.15.78:0/4050702587 conn(0x7ff5e00fc500 msgr2=0x7ff5e00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:53.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.839+0000 7ff5e68b4640 1 -- 172.21.15.78:0/4050702587 shutdown_connections 2024-08-18T23:47:53.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.839+0000 7ff5e68b4640 1 -- 172.21.15.78:0/4050702587 wait complete. 2024-08-18T23:47:53.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.839+0000 7ff5e68b4640 1 Processor -- start 2024-08-18T23:47:53.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.839+0000 7ff5e68b4640 1 -- start start 2024-08-18T23:47:53.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5e68b4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e0100b50 0x7ff5e0197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:53.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5e68b4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5e01022a0 0x7ff5e0197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5e68b4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e0102c20 0x7ff5e019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5e68b4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e0110cb0 con 0x7ff5e01022a0 2024-08-18T23:47:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5e68b4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e0110b30 con 0x7ff5e0102c20 2024-08-18T23:47:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5e68b4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e0110e30 con 0x7ff5e0100b50 2024-08-18T23:47:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5df7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5e01022a0 0x7ff5e0197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5df7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5e01022a0 0x7ff5e0197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42888/0 (socket says 172.21.15.78:42888) 2024-08-18T23:47:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5df7fe640 1 -- 172.21.15.78:0/1553163464 learned_addr learned my addr 172.21.15.78:0/1553163464 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5e4e51640 1 --2- 172.21.15.78:0/1553163464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e0102c20 0x7ff5e019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5df7fe640 1 -- 172.21.15.78:0/1553163464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e0100b50 msgr2=0x7ff5e0197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5dffff640 1 --2- 172.21.15.78:0/1553163464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e0100b50 0x7ff5e0197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5df7fe640 1 --2- 172.21.15.78:0/1553163464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e0100b50 0x7ff5e0197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:53.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5df7fe640 1 -- 172.21.15.78:0/1553163464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e0102c20 msgr2=0x7ff5e019ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5df7fe640 1 --2- 172.21.15.78:0/1553163464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e0102c20 0x7ff5e019ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5df7fe640 1 -- 172.21.15.78:0/1553163464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e019c0f0 con 0x7ff5e01022a0 2024-08-18T23:47:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5e4e51640 1 --2- 172.21.15.78:0/1553163464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e0102c20 0x7ff5e019ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5dffff640 1 --2- 172.21.15.78:0/1553163464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e0100b50 0x7ff5e0197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:47:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5df7fe640 1 --2- 172.21.15.78:0/1553163464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5e01022a0 0x7ff5e0197680 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400d770 tx=0x7ff5d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5dd7fa640 1 -- 172.21.15.78:0/1553163464 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4017070 con 0x7ff5e01022a0 2024-08-18T23:47:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5dd7fa640 1 -- 172.21.15.78:0/1553163464 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5d4005500 con 0x7ff5e01022a0 2024-08-18T23:47:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.840+0000 7ff5e68b4640 1 -- 172.21.15.78:0/1553163464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e019c3e0 con 0x7ff5e01022a0 2024-08-18T23:47:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.841+0000 7ff5dd7fa640 1 -- 172.21.15.78:0/1553163464 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4004e00 con 0x7ff5e01022a0 2024-08-18T23:47:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.841+0000 7ff5e68b4640 1 -- 172.21.15.78:0/1553163464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e01a3cc0 con 0x7ff5e01022a0 2024-08-18T23:47:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.841+0000 7ff5e68b4640 1 -- 172.21.15.78:0/1553163464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e0100f30 con 0x7ff5e01022a0 2024-08-18T23:47:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.843+0000 7ff5dd7fa640 1 -- 172.21.15.78:0/1553163464 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff5d4004fa0 con 0x7ff5e01022a0 2024-08-18T23:47:53.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.843+0000 7ff5dd7fa640 1 --2- 172.21.15.78:0/1553163464 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff5c80a2c00 0x7ff5c80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:53.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.844+0000 7ff5dffff640 1 --2- 172.21.15.78:0/1553163464 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff5c80a2c00 0x7ff5c80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:53.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.844+0000 7ff5dd7fa640 1 -- 172.21.15.78:0/1553163464 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7ff5d40cf210 con 0x7ff5e01022a0 2024-08-18T23:47:53.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.844+0000 7ff5dffff640 1 --2- 172.21.15.78:0/1553163464 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff5c80a2c00 0x7ff5c80a50c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0000c00 tx=0x7ff5d0004100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:53.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:53.847+0000 7ff5dd7fa640 1 -- 172.21.15.78:0/1553163464 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff5d409be60 con 0x7ff5e01022a0 2024-08-18T23:47:54.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:54.071+0000 7ff5e68b4640 1 -- 172.21.15.78:0/1553163464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5e0108df0 con 0x7ff5e01022a0 2024-08-18T23:47:54.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:54.072+0000 7ff5dd7fa640 1 -- 172.21.15.78:0/1553163464 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11933 (secure 0 0 0) 0x7ff5d409b760 con 0x7ff5e01022a0 2024-08-18T23:47:54.074 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:47:54.074 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":26,"btime":"2024-08-18T23:45:33:306257+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:45:32.491054+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:47:54.074 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 26 2024-08-18T23:47:54.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:54.074+0000 7ff5e68b4640 1 -- 172.21.15.78:0/1553163464 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff5c80a2c00 msgr2=0x7ff5c80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:54.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:54.074+0000 7ff5e68b4640 1 --2- 172.21.15.78:0/1553163464 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff5c80a2c00 0x7ff5c80a50c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0000c00 tx=0x7ff5d0004100 comp rx=0 tx=0).stop 2024-08-18T23:47:54.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:54.074+0000 7ff5e68b4640 1 -- 172.21.15.78:0/1553163464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5e01022a0 msgr2=0x7ff5e0197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:54.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:54.074+0000 7ff5e68b4640 1 --2- 172.21.15.78:0/1553163464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5e01022a0 0x7ff5e0197680 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400d770 tx=0x7ff5d400dc30 comp rx=0 tx=0).stop 2024-08-18T23:47:54.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:54.074+0000 7ff5e68b4640 1 -- 172.21.15.78:0/1553163464 shutdown_connections 2024-08-18T23:47:54.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:54.074+0000 7ff5e68b4640 1 --2- 172.21.15.78:0/1553163464 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff5c80a2c00 0x7ff5c80a50c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:54.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:54.074+0000 7ff5e68b4640 1 --2- 172.21.15.78:0/1553163464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5e0102c20 0x7ff5e019ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:54.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:54.074+0000 7ff5e68b4640 1 --2- 172.21.15.78:0/1553163464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5e01022a0 0x7ff5e0197680 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:54.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:54.074+0000 7ff5e68b4640 1 --2- 172.21.15.78:0/1553163464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5e0100b50 0x7ff5e0197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:54.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:54.075+0000 7ff5e68b4640 1 -- 172.21.15.78:0/1553163464 >> 172.21.15.78:0/1553163464 conn(0x7ff5e00fc500 msgr2=0x7ff5e00fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:54.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:54.075+0000 7ff5e68b4640 1 -- 172.21.15.78:0/1553163464 shutdown_connections 2024-08-18T23:47:54.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:54.075+0000 7ff5e68b4640 1 -- 172.21.15.78:0/1553163464 wait complete. 2024-08-18T23:47:54.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:54 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1553163464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:47:54.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:54 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1553163464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:47:54.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:54 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1553163464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:47:55.007 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-18T23:47:55.276 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:47:55.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:55 smithi123 ceph-mon[21808]: pgmap v458: 129 pgs: 129 active+clean; 467 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-08-18T23:47:55.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:55 smithi078 ceph-mon[18440]: pgmap v458: 129 pgs: 129 active+clean; 467 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-08-18T23:47:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:55 smithi161 ceph-mon[22027]: pgmap v458: 129 pgs: 129 active+clean; 467 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-08-18T23:47:55.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.773+0000 7f398aef6640 1 -- 172.21.15.78:0/4148256059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3984103410 msgr2=0x7f39841078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:55.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.773+0000 7f398aef6640 1 --2- 172.21.15.78:0/4148256059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3984103410 0x7f39841078c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3970009930 tx=0x7f397002a420 comp rx=0 tx=0).stop 2024-08-18T23:47:55.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.773+0000 7f398aef6640 1 -- 172.21.15.78:0/4148256059 shutdown_connections 2024-08-18T23:47:55.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.774+0000 7f398aef6640 1 --2- 172.21.15.78:0/4148256059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3984107fd0 0x7f398410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.774+0000 7f398aef6640 1 --2- 172.21.15.78:0/4148256059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3984103410 0x7f39841078c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.774+0000 7f398aef6640 1 --2- 172.21.15.78:0/4148256059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3984102a60 0x7f3984102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.774+0000 7f398aef6640 1 -- 172.21.15.78:0/4148256059 >> 172.21.15.78:0/4148256059 conn(0x7f39840fc500 msgr2=0x7f39840fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:55.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.774+0000 7f398aef6640 1 -- 172.21.15.78:0/4148256059 shutdown_connections 2024-08-18T23:47:55.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.774+0000 7f398aef6640 1 -- 172.21.15.78:0/4148256059 wait complete. 2024-08-18T23:47:55.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.774+0000 7f398aef6640 1 Processor -- start 2024-08-18T23:47:55.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.774+0000 7f398aef6640 1 -- start start 2024-08-18T23:47:55.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.774+0000 7f398aef6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3984102a60 0x7f3984198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:55.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.774+0000 7f398aef6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3984103410 0x7f3984198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:55.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.774+0000 7f398aef6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3984107fd0 0x7f398419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:55.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.774+0000 7f398aef6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3984111dd0 con 0x7f3984107fd0 2024-08-18T23:47:55.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.774+0000 7f398aef6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3984111c50 con 0x7f3984102a60 2024-08-18T23:47:55.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.774+0000 7f398aef6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3984111f50 con 0x7f3984103410 2024-08-18T23:47:55.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.775+0000 7f3989493640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3984107fd0 0x7f398419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:55.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.775+0000 7f3988c92640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3984102a60 0x7f3984198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:55.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.775+0000 7f3983fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3984103410 0x7f3984198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:55.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.775+0000 7f3988c92640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3984102a60 0x7f3984198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:46394/0 (socket says 172.21.15.78:46394) 2024-08-18T23:47:55.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.775+0000 7f3983fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3984103410 0x7f3984198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:44662/0 (socket says 172.21.15.78:44662) 2024-08-18T23:47:55.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.775+0000 7f3983fff640 1 -- 172.21.15.78:0/1592096712 learned_addr learned my addr 172.21.15.78:0/1592096712 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:55.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.775+0000 7f3989493640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3984107fd0 0x7f398419cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46518/0 (socket says 172.21.15.78:46518) 2024-08-18T23:47:55.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.775+0000 7f3989493640 1 -- 172.21.15.78:0/1592096712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3984103410 msgr2=0x7f3984198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:55.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.775+0000 7f3989493640 1 --2- 172.21.15.78:0/1592096712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3984103410 0x7f3984198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.775+0000 7f3989493640 1 -- 172.21.15.78:0/1592096712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3984102a60 msgr2=0x7f3984198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:55.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.775+0000 7f3989493640 1 --2- 172.21.15.78:0/1592096712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3984102a60 0x7f3984198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.775+0000 7f3989493640 1 -- 172.21.15.78:0/1592096712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398419d2c0 con 0x7f3984107fd0 2024-08-18T23:47:55.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.775+0000 7f3989493640 1 --2- 172.21.15.78:0/1592096712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3984107fd0 0x7f398419cbe0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f397c009fd0 tx=0x7f397c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:55.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.775+0000 7f3981ffb640 1 -- 172.21.15.78:0/1592096712 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c00ce20 con 0x7f3984107fd0 2024-08-18T23:47:55.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.775+0000 7f398aef6640 1 -- 172.21.15.78:0/1592096712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398419d5b0 con 0x7f3984107fd0 2024-08-18T23:47:55.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.775+0000 7f3981ffb640 1 -- 172.21.15.78:0/1592096712 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f397c005020 con 0x7f3984107fd0 2024-08-18T23:47:55.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.775+0000 7f3981ffb640 1 -- 172.21.15.78:0/1592096712 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c010720 con 0x7f3984107fd0 2024-08-18T23:47:55.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.775+0000 7f398aef6640 1 -- 172.21.15.78:0/1592096712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39841a4e90 con 0x7f3984107fd0 2024-08-18T23:47:55.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.778+0000 7f3981ffb640 1 -- 172.21.15.78:0/1592096712 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f397c005540 con 0x7f3984107fd0 2024-08-18T23:47:55.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.778+0000 7f398d2f4740 1 Processor -- start 2024-08-18T23:47:55.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.778+0000 7f398d2f4740 1 -- start start 2024-08-18T23:47:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.778+0000 7f398d2f4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc49ff0 0x55889cc4a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.778+0000 7f398d2f4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc4a910 0x55889cc54c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.778+0000 7f398d2f4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc551d0 0x55889cc575c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.778+0000 7f398d2f4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55889cbafa40 con 0x55889cc4a910 2024-08-18T23:47:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.778+0000 7f398d2f4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55889cb479d0 con 0x55889cc551d0 2024-08-18T23:47:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.778+0000 7f398d2f4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55889cbaba00 con 0x55889cc49ff0 2024-08-18T23:47:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.779+0000 7f3988c92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc4a910 0x55889cc54c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.779+0000 7f3988c92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc4a910 0x55889cc54c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46528/0 (socket says 172.21.15.78:46528) 2024-08-18T23:47:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.779+0000 7f3988c92640 1 -- 172.21.15.78:0/415594432 learned_addr learned my addr 172.21.15.78:0/415594432 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.779+0000 7f3983fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc49ff0 0x55889cc4a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.779+0000 7f3983fff640 1 --2- 172.21.15.78:0/415594432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc551d0 0x55889cc575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.779+0000 7f3988c92640 1 -- 172.21.15.78:0/415594432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc49ff0 msgr2=0x55889cc4a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.779+0000 7f3988c92640 1 --2- 172.21.15.78:0/415594432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc49ff0 0x55889cc4a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.779+0000 7f3988c92640 1 -- 172.21.15.78:0/415594432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc551d0 msgr2=0x55889cc575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.779+0000 7f3988c92640 1 --2- 172.21.15.78:0/415594432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc551d0 0x55889cc575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.779+0000 7f3988c92640 1 -- 172.21.15.78:0/415594432 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55889cb4e650 con 0x55889cc4a910 2024-08-18T23:47:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.779+0000 7f3983fff640 1 --2- 172.21.15.78:0/415594432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc49ff0 0x55889cc4a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:47:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.779+0000 7f3981ffb640 1 --2- 172.21.15.78:0/1592096712 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f39740a2b80 0x7f39740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.779+0000 7f3988c92640 1 --2- 172.21.15.78:0/415594432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc4a910 0x55889cc54c90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3978009d20 tx=0x7f3978011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe9719d6e9f34e2 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.779+0000 7f3983fff640 1 --2- 172.21.15.78:0/1592096712 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f39740a2b80 0x7f39740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.779+0000 7f3981ffb640 1 -- 172.21.15.78:0/1592096712 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f397c0cfaa0 con 0x7f3984107fd0 2024-08-18T23:47:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.779+0000 7f396effd640 1 -- 172.21.15.78:0/415594432 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397801e070 con 0x55889cc4a910 2024-08-18T23:47:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.779+0000 7f396effd640 1 -- 172.21.15.78:0/415594432 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3978019b10 con 0x55889cc4a910 2024-08-18T23:47:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.779+0000 7f396effd640 1 -- 172.21.15.78:0/415594432 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397800eb90 con 0x55889cc4a910 2024-08-18T23:47:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.780+0000 7f398d2f4740 1 -- 172.21.15.78:0/415594432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc4a910 msgr2=0x55889cc54c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.780+0000 7f3983fff640 1 --2- 172.21.15.78:0/1592096712 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f39740a2b80 0x7f39740a5040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3970000c00 tx=0x7f3970009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.780+0000 7f398d2f4740 1 --2- 172.21.15.78:0/415594432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc4a910 0x55889cc54c90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3978009d20 tx=0x7f3978011810 comp rx=0 tx=0).stop 2024-08-18T23:47:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.780+0000 7f398d2f4740 1 -- 172.21.15.78:0/415594432 shutdown_connections 2024-08-18T23:47:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.780+0000 7f398d2f4740 1 --2- 172.21.15.78:0/415594432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc551d0 0x55889cc575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.780+0000 7f398d2f4740 1 --2- 172.21.15.78:0/415594432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc4a910 0x55889cc54c90 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.780+0000 7f398d2f4740 1 --2- 172.21.15.78:0/415594432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc49ff0 0x55889cc4a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.780+0000 7f398d2f4740 1 -- 172.21.15.78:0/415594432 >> 172.21.15.78:0/415594432 conn(0x55889cc46580 msgr2=0x55889cc46960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.780+0000 7f398d2f4740 1 -- 172.21.15.78:0/415594432 shutdown_connections 2024-08-18T23:47:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.780+0000 7f398d2f4740 1 -- 172.21.15.78:0/415594432 wait complete. 2024-08-18T23:47:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.781+0000 7f398d2f4740 1 -- start start 2024-08-18T23:47:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.781+0000 7f398d2f4740 1 Processor -- start 2024-08-18T23:47:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.781+0000 7f398d2f4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc49ff0 0x55889cc52040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.781+0000 7f398d2f4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc4a910 0x55889cc52580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.781+0000 7f398d2f4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc551d0 0x55889cc52ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.781+0000 7f398d2f4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55889cc57da0 con 0x55889cc4a910 2024-08-18T23:47:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.781+0000 7f398d2f4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55889cb6ab20 con 0x55889cc551d0 2024-08-18T23:47:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.781+0000 7f398d2f4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55889cc57f20 con 0x55889cc49ff0 2024-08-18T23:47:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.781+0000 7f398d2f4740 10 client.0 _ll_register_callbacks cb 0x55889cb68bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.781+0000 7f3983fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc4a910 0x55889cc52580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.781+0000 7f3989493640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc551d0 0x55889cc52ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.781+0000 7f3988c92640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc49ff0 0x55889cc52040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.781+0000 7f3983fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc4a910 0x55889cc52580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46532/0 (socket says 172.21.15.78:46532) 2024-08-18T23:47:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.781+0000 7f3983fff640 1 -- 172.21.15.78:0/247706481 learned_addr learned my addr 172.21.15.78:0/247706481 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.782+0000 7f3988c92640 1 -- 172.21.15.78:0/247706481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc551d0 msgr2=0x55889cc52ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.782+0000 7f3988c92640 1 --2- 172.21.15.78:0/247706481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc551d0 0x55889cc52ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.782+0000 7f3988c92640 1 -- 172.21.15.78:0/247706481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc4a910 msgr2=0x55889cc52580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.782+0000 7f3988c92640 1 --2- 172.21.15.78:0/247706481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc4a910 0x55889cc52580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.782+0000 7f3988c92640 1 -- 172.21.15.78:0/247706481 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55889c9213c0 con 0x55889cc49ff0 2024-08-18T23:47:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.782+0000 7f3989493640 1 --2- 172.21.15.78:0/247706481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc551d0 0x55889cc52ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.782+0000 7f3983fff640 1 --2- 172.21.15.78:0/247706481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc4a910 0x55889cc52580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.782+0000 7f3988c92640 1 --2- 172.21.15.78:0/247706481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc49ff0 0x55889cc52040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3978011d10 tx=0x7f3978019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abc902b704812b4c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.782+0000 7f3963fff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:47:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.782+0000 7f3963fff640 1 -- 172.21.15.78:0/247706481 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978027020 con 0x55889cc49ff0 2024-08-18T23:47:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.782+0000 7f3963fff640 1 -- 172.21.15.78:0/247706481 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f397800ead0 con 0x55889cc49ff0 2024-08-18T23:47:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.782+0000 7f398d2f4740 1 -- 172.21.15.78:0/247706481 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3978019b10 con 0x55889cc49ff0 2024-08-18T23:47:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.782+0000 7f3963fff640 1 -- 172.21.15.78:0/247706481 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3978032050 con 0x55889cc49ff0 2024-08-18T23:47:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.782+0000 7f398d2f4740 10 client.25381 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.782+0000 7f398d2f4740 1 -- 172.21.15.78:0/247706481 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55889cc63930 con 0x55889cc49ff0 2024-08-18T23:47:55.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.783+0000 7f3963fff640 1 -- 172.21.15.78:0/247706481 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3978019b10 con 0x55889cc49ff0 2024-08-18T23:47:55.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.783+0000 7f3963fff640 1 -- 172.21.15.78:0/247706481 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f39780046c0 con 0x55889cc49ff0 2024-08-18T23:47:55.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.783+0000 7f398d2f4740 10 client.25381 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:55.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.783+0000 7f398d2f4740 10 client.25381 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:47:55.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.783+0000 7f398d2f4740 20 client.25381 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:55.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.783+0000 7f398d2f4740 1 --2- 172.21.15.78:0/247706481 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55889cc64df0 0x55889cc671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:55.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.783+0000 7f398d2f4740 4 client.25381 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:47:55.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.783+0000 7f398d2f4740 1 -- 172.21.15.78:0/247706481 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55889cbbcfb0 con 0x55889cc64df0 2024-08-18T23:47:55.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.783+0000 7f3983fff640 1 --2- 172.21.15.78:0/247706481 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55889cc64df0 0x55889cc671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:55.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.783+0000 7f3983fff640 1 --2- 172.21.15.78:0/247706481 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55889cc64df0 0x55889cc671e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=62a1d2505167688a server_cookie=92218604e547ade2 in_seq=0 out_seq=0 2024-08-18T23:47:55.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.783+0000 7f3963fff640 10 client.25381 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:47:55.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.784+0000 7f3963fff640 1 -- 172.21.15.78:0/247706481 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55889cbbcfb0 con 0x55889cc64df0 2024-08-18T23:47:55.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.784+0000 7f3963fff640 10 client.25381 handle_command_reply: tid=0 2024-08-18T23:47:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.784+0000 7f398d2f4740 1 client.25381 shutdown 2024-08-18T23:47:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.784+0000 7f398d2f4740 1 -- 172.21.15.78:0/247706481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc49ff0 msgr2=0x55889cc52040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.784+0000 7f398d2f4740 1 --2- 172.21.15.78:0/247706481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc49ff0 0x55889cc52040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3978011d10 tx=0x7f3978019660 comp rx=0 tx=0).stop 2024-08-18T23:47:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.784+0000 7f398d2f4740 1 -- 172.21.15.78:0/247706481 shutdown_connections 2024-08-18T23:47:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.784+0000 7f398d2f4740 1 --2- 172.21.15.78:0/247706481 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55889cc64df0 0x55889cc671e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.784+0000 7f398d2f4740 1 --2- 172.21.15.78:0/247706481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc551d0 0x55889cc52ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.784+0000 7f398d2f4740 1 --2- 172.21.15.78:0/247706481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc4a910 0x55889cc52580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.784+0000 7f3963fff640 0 client.25381 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:47:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.784+0000 7f398d2f4740 1 --2- 172.21.15.78:0/247706481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc49ff0 0x55889cc52040 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.784+0000 7f398d2f4740 1 -- 172.21.15.78:0/247706481 >> 172.21.15.78:0/247706481 conn(0x55889cc46580 msgr2=0x55889cc4c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.784+0000 7f398d2f4740 1 -- 172.21.15.78:0/247706481 shutdown_connections 2024-08-18T23:47:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.784+0000 7f398d2f4740 1 -- 172.21.15.78:0/247706481 wait complete. 2024-08-18T23:47:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.784+0000 7f398d2f4740 20 client.25381 trim_cache size 0 max 16384 2024-08-18T23:47:55.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.798+0000 7f398d2f4740 1 Processor -- start 2024-08-18T23:47:55.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.798+0000 7f398d2f4740 1 -- start start 2024-08-18T23:47:55.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.798+0000 7f398d2f4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc49ff0 0x55889cc4ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:55.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.798+0000 7f398d2f4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc4a910 0x55889cc4bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:55.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.798+0000 7f398d2f4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc551d0 0x55889cc4c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:55.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.798+0000 7f398d2f4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39841b12a0 con 0x55889cc49ff0 2024-08-18T23:47:55.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.799+0000 7f398d2f4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f39841aeb70 con 0x55889cc551d0 2024-08-18T23:47:55.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.799+0000 7f398d2f4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f39841ac9c0 con 0x55889cc4a910 2024-08-18T23:47:55.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.799+0000 7f3988c92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc49ff0 0x55889cc4ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:55.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.799+0000 7f3988c92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc49ff0 0x55889cc4ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46544/0 (socket says 172.21.15.78:46544) 2024-08-18T23:47:55.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.799+0000 7f3988c92640 1 -- 172.21.15.78:0/3452673240 learned_addr learned my addr 172.21.15.78:0/3452673240 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:55.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.799+0000 7f3983fff640 1 --2- 172.21.15.78:0/3452673240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc4a910 0x55889cc4bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:55.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.799+0000 7f3989493640 1 --2- 172.21.15.78:0/3452673240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc551d0 0x55889cc4c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:55.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.799+0000 7f3988c92640 1 -- 172.21.15.78:0/3452673240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc4a910 msgr2=0x55889cc4bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:55.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.799+0000 7f3988c92640 1 --2- 172.21.15.78:0/3452673240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc4a910 0x55889cc4bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.799+0000 7f3988c92640 1 -- 172.21.15.78:0/3452673240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc551d0 msgr2=0x55889cc4c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:55.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.799+0000 7f3988c92640 1 --2- 172.21.15.78:0/3452673240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc551d0 0x55889cc4c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.799+0000 7f3988c92640 1 -- 172.21.15.78:0/3452673240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55889cc4a740 con 0x55889cc49ff0 2024-08-18T23:47:55.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.799+0000 7f3988c92640 1 --2- 172.21.15.78:0/3452673240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc49ff0 0x55889cc4ba10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3978004c80 tx=0x7f39780096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6eb9610ce20c0e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:55.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.799+0000 7f396e7fc640 1 -- 172.21.15.78:0/3452673240 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978015020 con 0x55889cc49ff0 2024-08-18T23:47:55.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.799+0000 7f396e7fc640 1 -- 172.21.15.78:0/3452673240 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f397801f040 con 0x55889cc49ff0 2024-08-18T23:47:55.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.799+0000 7f396e7fc640 1 -- 172.21.15.78:0/3452673240 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978027040 con 0x55889cc49ff0 2024-08-18T23:47:55.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.800+0000 7f398d2f4740 1 -- 172.21.15.78:0/3452673240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc49ff0 msgr2=0x55889cc4ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:55.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.800+0000 7f398d2f4740 1 --2- 172.21.15.78:0/3452673240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc49ff0 0x55889cc4ba10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3978004c80 tx=0x7f39780096a0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.800+0000 7f398d2f4740 1 -- 172.21.15.78:0/3452673240 shutdown_connections 2024-08-18T23:47:55.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.800+0000 7f398d2f4740 1 --2- 172.21.15.78:0/3452673240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc551d0 0x55889cc4c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.800+0000 7f398d2f4740 1 --2- 172.21.15.78:0/3452673240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc4a910 0x55889cc4bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.800+0000 7f398d2f4740 1 --2- 172.21.15.78:0/3452673240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc49ff0 0x55889cc4ba10 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.800+0000 7f398d2f4740 1 -- 172.21.15.78:0/3452673240 >> 172.21.15.78:0/3452673240 conn(0x55889cc46580 msgr2=0x55889cc411b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:55.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.800+0000 7f398d2f4740 1 -- 172.21.15.78:0/3452673240 shutdown_connections 2024-08-18T23:47:55.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.800+0000 7f398d2f4740 1 -- 172.21.15.78:0/3452673240 wait complete. 2024-08-18T23:47:55.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.801+0000 7f398d2f4740 1 -- start start 2024-08-18T23:47:55.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.801+0000 7f398d2f4740 1 Processor -- start 2024-08-18T23:47:55.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.801+0000 7f398d2f4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc49ff0 0x55889cc5d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:55.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.801+0000 7f398d2f4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc4a910 0x55889cc5d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:55.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.801+0000 7f398d2f4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc551d0 0x55889cc4e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:55.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.801+0000 7f398d2f4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39841ad030 con 0x55889cc551d0 2024-08-18T23:47:55.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.801+0000 7f398d2f4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f39841ae4a0 con 0x55889cc49ff0 2024-08-18T23:47:55.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.801+0000 7f398d2f4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f39841ad700 con 0x55889cc4a910 2024-08-18T23:47:55.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.801+0000 7f398d2f4740 10 client.0 _ll_register_callbacks cb 0x7f397801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:55.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.801+0000 7f3988c92640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc49ff0 0x55889cc5d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:55.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.801+0000 7f3989493640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc551d0 0x55889cc4e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:55.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.801+0000 7f3988c92640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc49ff0 0x55889cc5d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:46424/0 (socket says 172.21.15.78:46424) 2024-08-18T23:47:55.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.801+0000 7f3988c92640 1 -- 172.21.15.78:0/1495054595 learned_addr learned my addr 172.21.15.78:0/1495054595 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:55.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.801+0000 7f3989493640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc551d0 0x55889cc4e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46548/0 (socket says 172.21.15.78:46548) 2024-08-18T23:47:55.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.801+0000 7f3983fff640 1 --2- 172.21.15.78:0/1495054595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc4a910 0x55889cc5d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:55.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.802+0000 7f3989493640 1 -- 172.21.15.78:0/1495054595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc4a910 msgr2=0x55889cc5d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:55.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.802+0000 7f3989493640 1 --2- 172.21.15.78:0/1495054595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc4a910 0x55889cc5d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.802+0000 7f3989493640 1 -- 172.21.15.78:0/1495054595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc49ff0 msgr2=0x55889cc5d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:55.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.802+0000 7f3989493640 1 --2- 172.21.15.78:0/1495054595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc49ff0 0x55889cc5d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.802+0000 7f3989493640 1 -- 172.21.15.78:0/1495054595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55889cc4e870 con 0x55889cc551d0 2024-08-18T23:47:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.802+0000 7f3988c92640 1 --2- 172.21.15.78:0/1495054595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc49ff0 0x55889cc5d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.802+0000 7f3989493640 1 --2- 172.21.15.78:0/1495054595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc551d0 0x55889cc4e2b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f397c09bea0 tx=0x7f397c09c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1aac70fbf71ce6 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.802+0000 7f3963fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:47:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.802+0000 7f3963fff640 1 -- 172.21.15.78:0/1495054595 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c0b7070 con 0x55889cc551d0 2024-08-18T23:47:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.802+0000 7f3963fff640 1 -- 172.21.15.78:0/1495054595 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f397c0bab50 con 0x55889cc551d0 2024-08-18T23:47:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.802+0000 7f398d2f4740 1 -- 172.21.15.78:0/1495054595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f397801f040 con 0x55889cc551d0 2024-08-18T23:47:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.802+0000 7f3963fff640 1 -- 172.21.15.78:0/1495054595 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f397c083050 con 0x55889cc551d0 2024-08-18T23:47:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.802+0000 7f398d2f4740 10 client.16086 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.802+0000 7f398d2f4740 1 -- 172.21.15.78:0/1495054595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55889cc4fbd0 con 0x55889cc551d0 2024-08-18T23:47:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.803+0000 7f3963fff640 1 -- 172.21.15.78:0/1495054595 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f397801f040 con 0x55889cc551d0 2024-08-18T23:47:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.803+0000 7f3963fff640 1 -- 172.21.15.78:0/1495054595 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f397c09add0 con 0x55889cc551d0 2024-08-18T23:47:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.803+0000 7f398d2f4740 10 client.16086 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:55.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.803+0000 7f398d2f4740 10 client.16086 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:47:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.803+0000 7f398d2f4740 20 client.16086 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.803+0000 7f398d2f4740 1 --2- 172.21.15.78:0/1495054595 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55889ccd2e10 0x55889ccd5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.803+0000 7f398d2f4740 4 client.16086 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:47:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.803+0000 7f398d2f4740 1 -- 172.21.15.78:0/1495054595 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55889cbb74c0 con 0x55889ccd2e10 2024-08-18T23:47:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.803+0000 7f3988c92640 1 --2- 172.21.15.78:0/1495054595 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55889ccd2e10 0x55889ccd5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.804+0000 7f3988c92640 1 --2- 172.21.15.78:0/1495054595 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55889ccd2e10 0x55889ccd5230 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=759ec2dbbd9ca87d server_cookie=52f6a8fbb3c6e6e5 in_seq=0 out_seq=0 2024-08-18T23:47:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.804+0000 7f3963fff640 10 client.16086 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:47:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.804+0000 7f3963fff640 1 -- 172.21.15.78:0/1495054595 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55889cbb74c0 con 0x55889ccd2e10 2024-08-18T23:47:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.804+0000 7f3963fff640 10 client.16086 handle_command_reply: tid=0 2024-08-18T23:47:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.804+0000 7f398d2f4740 1 client.16086 shutdown 2024-08-18T23:47:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.804+0000 7f398d2f4740 1 -- 172.21.15.78:0/1495054595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc551d0 msgr2=0x55889cc4e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.804+0000 7f398d2f4740 1 --2- 172.21.15.78:0/1495054595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc551d0 0x55889cc4e2b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f397c09bea0 tx=0x7f397c09c610 comp rx=0 tx=0).stop 2024-08-18T23:47:55.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.804+0000 7f398d2f4740 1 -- 172.21.15.78:0/1495054595 shutdown_connections 2024-08-18T23:47:55.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.804+0000 7f398d2f4740 1 --2- 172.21.15.78:0/1495054595 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55889ccd2e10 0x55889ccd5230 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.804+0000 7f3963fff640 0 client.16086 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:47:55.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.804+0000 7f398d2f4740 1 --2- 172.21.15.78:0/1495054595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55889cc551d0 0x55889cc4e2b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.804+0000 7f398d2f4740 1 --2- 172.21.15.78:0/1495054595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55889cc4a910 0x55889cc5d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.804+0000 7f398d2f4740 1 --2- 172.21.15.78:0/1495054595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55889cc49ff0 0x55889cc5d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.804+0000 7f398d2f4740 1 -- 172.21.15.78:0/1495054595 >> 172.21.15.78:0/1495054595 conn(0x55889cc46580 msgr2=0x55889cc44d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:55.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.804+0000 7f398d2f4740 1 -- 172.21.15.78:0/1495054595 shutdown_connections 2024-08-18T23:47:55.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.804+0000 7f398d2f4740 1 -- 172.21.15.78:0/1495054595 wait complete. 2024-08-18T23:47:55.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.805+0000 7f398d2f4740 20 client.16086 trim_cache size 0 max 16384 2024-08-18T23:47:55.812 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:47:55.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.805+0000 7f398aef6640 1 -- 172.21.15.78:0/1592096712 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f39740a2b80 msgr2=0x7f39740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:55.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.805+0000 7f398aef6640 1 --2- 172.21.15.78:0/1592096712 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f39740a2b80 0x7f39740a5040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3970000c00 tx=0x7f3970009260 comp rx=0 tx=0).stop 2024-08-18T23:47:55.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.805+0000 7f398aef6640 1 -- 172.21.15.78:0/1592096712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3984107fd0 msgr2=0x7f398419cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:55.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.805+0000 7f398aef6640 1 --2- 172.21.15.78:0/1592096712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3984107fd0 0x7f398419cbe0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f397c009fd0 tx=0x7f397c00ef10 comp rx=0 tx=0).stop 2024-08-18T23:47:55.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.805+0000 7f398aef6640 1 -- 172.21.15.78:0/1592096712 shutdown_connections 2024-08-18T23:47:55.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.805+0000 7f398aef6640 1 --2- 172.21.15.78:0/1592096712 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f39740a2b80 0x7f39740a5040 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.805+0000 7f398aef6640 1 --2- 172.21.15.78:0/1592096712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3984107fd0 0x7f398419cbe0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.805+0000 7f398aef6640 1 --2- 172.21.15.78:0/1592096712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3984103410 0x7f3984198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.805+0000 7f398aef6640 1 --2- 172.21.15.78:0/1592096712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3984102a60 0x7f3984198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:55.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.805+0000 7f398aef6640 1 -- 172.21.15.78:0/1592096712 >> 172.21.15.78:0/1592096712 conn(0x7f39840fc500 msgr2=0x7f3984105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:55.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.805+0000 7f398aef6640 1 -- 172.21.15.78:0/1592096712 shutdown_connections 2024-08-18T23:47:55.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:55.805+0000 7f398aef6640 1 -- 172.21.15.78:0/1592096712 wait complete. 2024-08-18T23:47:56.607 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-18T23:47:56.875 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:47:56.982 INFO:tasks.workunit.client.0.smithi078.stdout: 79 8597 0 6720 0 4793 0 2024-08-18T23:47:57.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.418+0000 7f0860bb8640 1 -- 172.21.15.78:0/3560117689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f085c102c20 msgr2=0x7f085c10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:57.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.418+0000 7f0860bb8640 1 --2- 172.21.15.78:0/3560117689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f085c102c20 0x7f085c10d770 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0854009ca0 tx=0x7f085402a950 comp rx=0 tx=0).stop 2024-08-18T23:47:57.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.418+0000 7f0860bb8640 1 -- 172.21.15.78:0/3560117689 shutdown_connections 2024-08-18T23:47:57.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.418+0000 7f0860bb8640 1 --2- 172.21.15.78:0/3560117689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f085c102c20 0x7f085c10d770 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.418+0000 7f0860bb8640 1 --2- 172.21.15.78:0/3560117689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f085c1022a0 0x7f085c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.418+0000 7f0860bb8640 1 --2- 172.21.15.78:0/3560117689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f085c100b50 0x7f085c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.418+0000 7f0860bb8640 1 -- 172.21.15.78:0/3560117689 >> 172.21.15.78:0/3560117689 conn(0x7f085c0fc500 msgr2=0x7f085c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:57.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.418+0000 7f0860bb8640 1 -- 172.21.15.78:0/3560117689 shutdown_connections 2024-08-18T23:47:57.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.418+0000 7f0860bb8640 1 -- 172.21.15.78:0/3560117689 wait complete. 2024-08-18T23:47:57.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.419+0000 7f0860bb8640 1 Processor -- start 2024-08-18T23:47:57.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.419+0000 7f0860bb8640 1 -- start start 2024-08-18T23:47:57.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.419+0000 7f0860bb8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f085c100b50 0x7f085c0fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.419+0000 7f0860bb8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f085c1022a0 0x7f085c0fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.419+0000 7f0860bb8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f085c102c20 0x7f085c0fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.419+0000 7f0860bb8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f085c110cb0 con 0x7f085c102c20 2024-08-18T23:47:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.419+0000 7f0860bb8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f085c110b30 con 0x7f085c1022a0 2024-08-18T23:47:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.419+0000 7f0860bb8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f085c110e30 con 0x7f085c100b50 2024-08-18T23:47:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.419+0000 7f085ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f085c102c20 0x7f085c0fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.419+0000 7f085ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f085c102c20 0x7f085c0fd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46556/0 (socket says 172.21.15.78:46556) 2024-08-18T23:47:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.420+0000 7f085ad9d640 1 -- 172.21.15.78:0/391413554 learned_addr learned my addr 172.21.15.78:0/391413554 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.420+0000 7f0859d9b640 1 --2- 172.21.15.78:0/391413554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f085c1022a0 0x7f085c0fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.420+0000 7f085a59c640 1 --2- 172.21.15.78:0/391413554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f085c100b50 0x7f085c0fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.420+0000 7f085ad9d640 1 -- 172.21.15.78:0/391413554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f085c100b50 msgr2=0x7f085c0fc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.420+0000 7f085ad9d640 1 --2- 172.21.15.78:0/391413554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f085c100b50 0x7f085c0fc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.420+0000 7f085ad9d640 1 -- 172.21.15.78:0/391413554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f085c1022a0 msgr2=0x7f085c0fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:57.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.420+0000 7f085ad9d640 1 --2- 172.21.15.78:0/391413554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f085c1022a0 0x7f085c0fcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.420+0000 7f085ad9d640 1 -- 172.21.15.78:0/391413554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c1a6680 con 0x7f085c102c20 2024-08-18T23:47:57.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.420+0000 7f085ad9d640 1 --2- 172.21.15.78:0/391413554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f085c102c20 0x7f085c0fd430 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0854004af0 tx=0x7f085402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:57.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.420+0000 7f084b7fe640 1 -- 172.21.15.78:0/391413554 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854034040 con 0x7f085c102c20 2024-08-18T23:47:57.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.420+0000 7f0860bb8640 1 -- 172.21.15.78:0/391413554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085c1a6910 con 0x7f085c102c20 2024-08-18T23:47:57.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.420+0000 7f084b7fe640 1 -- 172.21.15.78:0/391413554 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0854030920 con 0x7f085c102c20 2024-08-18T23:47:57.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.420+0000 7f084b7fe640 1 -- 172.21.15.78:0/391413554 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854004630 con 0x7f085c102c20 2024-08-18T23:47:57.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.420+0000 7f0860bb8640 1 -- 172.21.15.78:0/391413554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f085c1a6df0 con 0x7f085c102c20 2024-08-18T23:47:57.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.423+0000 7f0862fb6740 1 Processor -- start 2024-08-18T23:47:57.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.423+0000 7f084b7fe640 1 -- 172.21.15.78:0/391413554 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0854037020 con 0x7f085c102c20 2024-08-18T23:47:57.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.423+0000 7f0862fb6740 1 -- start start 2024-08-18T23:47:57.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.423+0000 7f0862fb6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb64ff0 0x5604fbb653d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:57.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.423+0000 7f0862fb6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb65910 0x5604fbb6fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:57.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.423+0000 7f0862fb6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb701d0 0x5604fbb725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:57.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.423+0000 7f0862fb6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5604fbacaa40 con 0x5604fbb701d0 2024-08-18T23:47:57.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.423+0000 7f0862fb6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5604fba629d0 con 0x5604fbb64ff0 2024-08-18T23:47:57.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.423+0000 7f0862fb6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5604fbac6a00 con 0x5604fbb65910 2024-08-18T23:47:57.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.424+0000 7f0859d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb64ff0 0x5604fbb653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:57.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.424+0000 7f085a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb65910 0x5604fbb6fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:57.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.424+0000 7f085a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb65910 0x5604fbb6fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:44740/0 (socket says 172.21.15.78:44740) 2024-08-18T23:47:57.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.424+0000 7f085a59c640 1 -- 172.21.15.78:0/4292935766 learned_addr learned my addr 172.21.15.78:0/4292935766 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:57.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.424+0000 7f0859d9b640 1 --2- 172.21.15.78:0/4292935766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb701d0 0x5604fbb725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:57.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.424+0000 7f085a59c640 1 -- 172.21.15.78:0/4292935766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb64ff0 msgr2=0x5604fbb653d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:57.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.424+0000 7f085a59c640 1 --2- 172.21.15.78:0/4292935766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb64ff0 0x5604fbb653d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.424+0000 7f085a59c640 1 -- 172.21.15.78:0/4292935766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb701d0 msgr2=0x5604fbb725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:57.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.424+0000 7f085a59c640 1 --2- 172.21.15.78:0/4292935766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb701d0 0x5604fbb725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.424+0000 7f085a59c640 1 -- 172.21.15.78:0/4292935766 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604fba69650 con 0x5604fbb65910 2024-08-18T23:47:57.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.424+0000 7f084b7fe640 1 --2- 172.21.15.78:0/391413554 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f08440a2b80 0x7f08440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:57.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.424+0000 7f0859d9b640 1 --2- 172.21.15.78:0/4292935766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb701d0 0x5604fbb725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:47:57.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.424+0000 7f084b7fe640 1 -- 172.21.15.78:0/391413554 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f08540f0110 con 0x7f085c102c20 2024-08-18T23:47:57.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.424+0000 7f0859d9b640 1 --2- 172.21.15.78:0/391413554 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f08440a2b80 0x7f08440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:57.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.424+0000 7f085a59c640 1 --2- 172.21.15.78:0/4292935766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb65910 0x5604fbb6fc90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f084c009d20 tx=0x7f084c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92b86b92c85db389 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:57.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.424+0000 7f0848ff9640 1 -- 172.21.15.78:0/4292935766 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c01e070 con 0x5604fbb65910 2024-08-18T23:47:57.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.424+0000 7f0848ff9640 1 -- 172.21.15.78:0/4292935766 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084c019b10 con 0x5604fbb65910 2024-08-18T23:47:57.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.424+0000 7f0848ff9640 1 -- 172.21.15.78:0/4292935766 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c00eb30 con 0x5604fbb65910 2024-08-18T23:47:57.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.425+0000 7f0862fb6740 1 -- 172.21.15.78:0/4292935766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb65910 msgr2=0x5604fbb6fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:57.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.425+0000 7f0859d9b640 1 --2- 172.21.15.78:0/391413554 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f08440a2b80 0x7f08440a5040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5604fbb62900 tx=0x7f0850013040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:57.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.425+0000 7f0862fb6740 1 --2- 172.21.15.78:0/4292935766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb65910 0x5604fbb6fc90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f084c009d20 tx=0x7f084c011810 comp rx=0 tx=0).stop 2024-08-18T23:47:57.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.425+0000 7f0862fb6740 1 -- 172.21.15.78:0/4292935766 shutdown_connections 2024-08-18T23:47:57.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.425+0000 7f0862fb6740 1 --2- 172.21.15.78:0/4292935766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb701d0 0x5604fbb725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.425+0000 7f0862fb6740 1 --2- 172.21.15.78:0/4292935766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb65910 0x5604fbb6fc90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.425+0000 7f0862fb6740 1 --2- 172.21.15.78:0/4292935766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb64ff0 0x5604fbb653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.425+0000 7f0862fb6740 1 -- 172.21.15.78:0/4292935766 >> 172.21.15.78:0/4292935766 conn(0x5604fbb61580 msgr2=0x5604fbb61960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:57.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.425+0000 7f0862fb6740 1 -- 172.21.15.78:0/4292935766 shutdown_connections 2024-08-18T23:47:57.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.425+0000 7f0862fb6740 1 -- 172.21.15.78:0/4292935766 wait complete. 2024-08-18T23:47:57.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.426+0000 7f0862fb6740 1 -- start start 2024-08-18T23:47:57.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.426+0000 7f0862fb6740 1 Processor -- start 2024-08-18T23:47:57.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.426+0000 7f0862fb6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb64ff0 0x5604fbb6d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:57.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.426+0000 7f0862fb6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb65910 0x5604fbb6d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:57.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.426+0000 7f0862fb6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb701d0 0x5604fbb6dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:57.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.426+0000 7f0862fb6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5604fbb72da0 con 0x5604fbb64ff0 2024-08-18T23:47:57.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.426+0000 7f0862fb6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5604fba85b20 con 0x5604fbb701d0 2024-08-18T23:47:57.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.426+0000 7f0862fb6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5604fbb72f20 con 0x5604fbb65910 2024-08-18T23:47:57.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.426+0000 7f085a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb64ff0 0x5604fbb6d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:57.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.426+0000 7f085a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb64ff0 0x5604fbb6d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46574/0 (socket says 172.21.15.78:46574) 2024-08-18T23:47:57.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.426+0000 7f085a59c640 1 -- 172.21.15.78:0/4113373365 learned_addr learned my addr 172.21.15.78:0/4113373365 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:57.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.426+0000 7f0859d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb65910 0x5604fbb6d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:57.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.426+0000 7f0862fb6740 10 client.0 _ll_register_callbacks cb 0x5604fba83bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:57.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.427+0000 7f085ad9d640 1 --2- 172.21.15.78:0/4113373365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb701d0 0x5604fbb6dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:57.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.427+0000 7f0859d9b640 1 -- 172.21.15.78:0/4113373365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb701d0 msgr2=0x5604fbb6dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:57.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.427+0000 7f0859d9b640 1 --2- 172.21.15.78:0/4113373365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb701d0 0x5604fbb6dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.427+0000 7f0859d9b640 1 -- 172.21.15.78:0/4113373365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb64ff0 msgr2=0x5604fbb6d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:57.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.427+0000 7f0859d9b640 1 --2- 172.21.15.78:0/4113373365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb64ff0 0x5604fbb6d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.427+0000 7f0859d9b640 1 -- 172.21.15.78:0/4113373365 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604fb83c3c0 con 0x5604fbb65910 2024-08-18T23:47:57.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.427+0000 7f085a59c640 1 --2- 172.21.15.78:0/4113373365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb64ff0 0x5604fbb6d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:57.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.427+0000 7f0859d9b640 1 --2- 172.21.15.78:0/4113373365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb65910 0x5604fbb6d570 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f085c0fe440 tx=0x7f08500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84a66527fddf092c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:57.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.427+0000 7f0841ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:47:57.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.427+0000 7f0841ffb640 1 -- 172.21.15.78:0/4113373365 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085001fce0 con 0x5604fbb65910 2024-08-18T23:47:57.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.427+0000 7f0841ffb640 1 -- 172.21.15.78:0/4113373365 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f085001a070 con 0x5604fbb65910 2024-08-18T23:47:57.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.427+0000 7f0862fb6740 1 -- 172.21.15.78:0/4113373365 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f084c019b10 con 0x5604fbb65910 2024-08-18T23:47:57.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.427+0000 7f0841ffb640 1 -- 172.21.15.78:0/4113373365 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0850036050 con 0x5604fbb65910 2024-08-18T23:47:57.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.427+0000 7f0862fb6740 10 client.25399 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.427+0000 7f0862fb6740 1 -- 172.21.15.78:0/4113373365 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5604fbb7e970 con 0x5604fbb65910 2024-08-18T23:47:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.428+0000 7f0841ffb640 1 -- 172.21.15.78:0/4113373365 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084c019b10 con 0x5604fbb65910 2024-08-18T23:47:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.428+0000 7f0841ffb640 1 -- 172.21.15.78:0/4113373365 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f085002a430 con 0x5604fbb65910 2024-08-18T23:47:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.428+0000 7f0862fb6740 10 client.25399 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.428+0000 7f0862fb6740 10 client.25399 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:47:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.428+0000 7f0862fb6740 20 client.25399 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.428+0000 7f0862fb6740 1 --2- 172.21.15.78:0/4113373365 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x5604fbb7fe80 0x5604fbb82310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.428+0000 7f0862fb6740 4 client.25399 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:47:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.428+0000 7f0862fb6740 1 -- 172.21.15.78:0/4113373365 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604fbad7fb0 con 0x5604fbb7fe80 2024-08-18T23:47:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.428+0000 7f085a59c640 1 --2- 172.21.15.78:0/4113373365 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x5604fbb7fe80 0x5604fbb82310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.428+0000 7f085a59c640 1 --2- 172.21.15.78:0/4113373365 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x5604fbb7fe80 0x5604fbb82310 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e342b800d58cc49 server_cookie=f80739e1e4b931e1 in_seq=0 out_seq=0 2024-08-18T23:47:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.428+0000 7f0841ffb640 10 client.25399 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:47:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.429+0000 7f0841ffb640 1 -- 172.21.15.78:0/4113373365 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604fbad7fb0 con 0x5604fbb7fe80 2024-08-18T23:47:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.429+0000 7f0841ffb640 10 client.25399 handle_command_reply: tid=0 2024-08-18T23:47:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.429+0000 7f0862fb6740 1 client.25399 shutdown 2024-08-18T23:47:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.430+0000 7f0862fb6740 1 -- 172.21.15.78:0/4113373365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb65910 msgr2=0x5604fbb6d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.430+0000 7f0862fb6740 1 --2- 172.21.15.78:0/4113373365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb65910 0x5604fbb6d570 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f085c0fe440 tx=0x7f08500046c0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.430+0000 7f0862fb6740 1 -- 172.21.15.78:0/4113373365 shutdown_connections 2024-08-18T23:47:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.430+0000 7f0862fb6740 1 --2- 172.21.15.78:0/4113373365 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x5604fbb7fe80 0x5604fbb82310 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.430+0000 7f0862fb6740 1 --2- 172.21.15.78:0/4113373365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb701d0 0x5604fbb6dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.430+0000 7f0862fb6740 1 --2- 172.21.15.78:0/4113373365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb65910 0x5604fbb6d570 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.430+0000 7f0841ffb640 0 client.25399 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:47:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.430+0000 7f0862fb6740 1 --2- 172.21.15.78:0/4113373365 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb64ff0 0x5604fbb6d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.430+0000 7f0862fb6740 1 -- 172.21.15.78:0/4113373365 >> 172.21.15.78:0/4113373365 conn(0x5604fbb61580 msgr2=0x5604fbb67260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.430+0000 7f0862fb6740 1 -- 172.21.15.78:0/4113373365 shutdown_connections 2024-08-18T23:47:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.430+0000 7f0862fb6740 1 -- 172.21.15.78:0/4113373365 wait complete. 2024-08-18T23:47:57.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.430+0000 7f0862fb6740 20 client.25399 trim_cache size 0 max 16384 2024-08-18T23:47:57.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.443+0000 7f0862fb6740 1 Processor -- start 2024-08-18T23:47:57.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.443+0000 7f0862fb6740 1 -- start start 2024-08-18T23:47:57.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.443+0000 7f0862fb6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb64ff0 0x5604fbb87b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:57.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.443+0000 7f0862fb6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb65910 0x5604fbb880d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:57.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.444+0000 7f0862fb6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb701d0 0x5604fbb6b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:57.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.444+0000 7f0862fb6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1b3480 con 0x5604fbb65910 2024-08-18T23:47:57.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.444+0000 7f0862fb6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1b0d50 con 0x5604fbb701d0 2024-08-18T23:47:57.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.444+0000 7f0862fb6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1aeba0 con 0x5604fbb64ff0 2024-08-18T23:47:57.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.444+0000 7f0859d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb65910 0x5604fbb880d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:57.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.444+0000 7f0859d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb65910 0x5604fbb880d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46582/0 (socket says 172.21.15.78:46582) 2024-08-18T23:47:57.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.444+0000 7f0859d9b640 1 -- 172.21.15.78:0/3130246197 learned_addr learned my addr 172.21.15.78:0/3130246197 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:57.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.444+0000 7f085ad9d640 1 --2- 172.21.15.78:0/3130246197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb701d0 0x5604fbb6b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:57.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.444+0000 7f085a59c640 1 --2- 172.21.15.78:0/3130246197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb64ff0 0x5604fbb87b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:57.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.444+0000 7f0859d9b640 1 -- 172.21.15.78:0/3130246197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb64ff0 msgr2=0x5604fbb87b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:57.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.444+0000 7f0859d9b640 1 --2- 172.21.15.78:0/3130246197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb64ff0 0x5604fbb87b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.444+0000 7f0859d9b640 1 -- 172.21.15.78:0/3130246197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb701d0 msgr2=0x5604fbb6b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:57.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.444+0000 7f0859d9b640 1 --2- 172.21.15.78:0/3130246197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb701d0 0x5604fbb6b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.444+0000 7f0859d9b640 1 -- 172.21.15.78:0/3130246197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604fbb5fb30 con 0x5604fbb65910 2024-08-18T23:47:57.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.444+0000 7f0859d9b640 1 --2- 172.21.15.78:0/3130246197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb65910 0x5604fbb880d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f085002a820 tx=0x7f0850005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123695a4b25ac362 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:57.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.444+0000 7f0843fff640 1 -- 172.21.15.78:0/3130246197 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085001fce0 con 0x5604fbb65910 2024-08-18T23:47:57.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.444+0000 7f0843fff640 1 -- 172.21.15.78:0/3130246197 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f085003f070 con 0x5604fbb65910 2024-08-18T23:47:57.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.444+0000 7f0843fff640 1 -- 172.21.15.78:0/3130246197 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08500052a0 con 0x5604fbb65910 2024-08-18T23:47:57.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.445+0000 7f0862fb6740 1 -- 172.21.15.78:0/3130246197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb65910 msgr2=0x5604fbb880d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:57.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.445+0000 7f0862fb6740 1 --2- 172.21.15.78:0/3130246197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb65910 0x5604fbb880d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f085002a820 tx=0x7f0850005d30 comp rx=0 tx=0).stop 2024-08-18T23:47:57.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.445+0000 7f0862fb6740 1 -- 172.21.15.78:0/3130246197 shutdown_connections 2024-08-18T23:47:57.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.445+0000 7f0862fb6740 1 --2- 172.21.15.78:0/3130246197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb701d0 0x5604fbb6b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.445+0000 7f0862fb6740 1 --2- 172.21.15.78:0/3130246197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb65910 0x5604fbb880d0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.445+0000 7f0862fb6740 1 --2- 172.21.15.78:0/3130246197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb64ff0 0x5604fbb87b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.445+0000 7f0862fb6740 1 -- 172.21.15.78:0/3130246197 >> 172.21.15.78:0/3130246197 conn(0x5604fbb61580 msgr2=0x5604fbb5c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:57.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.445+0000 7f0862fb6740 1 -- 172.21.15.78:0/3130246197 shutdown_connections 2024-08-18T23:47:57.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.445+0000 7f0862fb6740 1 -- 172.21.15.78:0/3130246197 wait complete. 2024-08-18T23:47:57.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f0862fb6740 1 -- start start 2024-08-18T23:47:57.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f0862fb6740 1 Processor -- start 2024-08-18T23:47:57.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f0862fb6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb64ff0 0x5604fbb78170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:57.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f0862fb6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb65910 0x5604fbb786b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:57.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f0862fb6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb701d0 0x5604fbb69680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:57.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f0862fb6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1af210 con 0x5604fbb64ff0 2024-08-18T23:47:57.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f0862fb6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1b0680 con 0x5604fbb65910 2024-08-18T23:47:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f0862fb6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1af8e0 con 0x5604fbb701d0 2024-08-18T23:47:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f0862fb6740 10 client.0 _ll_register_callbacks cb 0x7f0850009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f085a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb64ff0 0x5604fbb78170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f085a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb64ff0 0x5604fbb78170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46592/0 (socket says 172.21.15.78:46592) 2024-08-18T23:47:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f085a59c640 1 -- 172.21.15.78:0/3555133296 learned_addr learned my addr 172.21.15.78:0/3555133296 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f0859d9b640 1 --2- 172.21.15.78:0/3555133296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb65910 0x5604fbb786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f085ad9d640 1 --2- 172.21.15.78:0/3555133296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb701d0 0x5604fbb69680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f085a59c640 1 -- 172.21.15.78:0/3555133296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb701d0 msgr2=0x5604fbb69680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f085a59c640 1 --2- 172.21.15.78:0/3555133296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb701d0 0x5604fbb69680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f085a59c640 1 -- 172.21.15.78:0/3555133296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb65910 msgr2=0x5604fbb786b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f085a59c640 1 --2- 172.21.15.78:0/3555133296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb65910 0x5604fbb786b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f085a59c640 1 -- 172.21.15.78:0/3555133296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604fbb69bc0 con 0x5604fbb64ff0 2024-08-18T23:47:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f085a59c640 1 --2- 172.21.15.78:0/3555133296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb64ff0 0x5604fbb78170 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f084c019fd0 tx=0x7f084c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac9be75dacad09c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.446+0000 7f0841ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:47:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.447+0000 7f0841ffb640 1 -- 172.21.15.78:0/3555133296 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c01e040 con 0x5604fbb64ff0 2024-08-18T23:47:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.447+0000 7f0862fb6740 1 -- 172.21.15.78:0/3555133296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f085003f070 con 0x5604fbb64ff0 2024-08-18T23:47:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.447+0000 7f0841ffb640 1 -- 172.21.15.78:0/3555133296 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f084c0050c0 con 0x5604fbb64ff0 2024-08-18T23:47:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.447+0000 7f0841ffb640 1 -- 172.21.15.78:0/3555133296 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f084c038050 con 0x5604fbb64ff0 2024-08-18T23:47:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.447+0000 7f0862fb6740 10 client.16104 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.447+0000 7f0862fb6740 1 -- 172.21.15.78:0/3555133296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5604fbbea360 con 0x5604fbb64ff0 2024-08-18T23:47:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.447+0000 7f0841ffb640 1 -- 172.21.15.78:0/3555133296 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f085003f070 con 0x5604fbb64ff0 2024-08-18T23:47:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.447+0000 7f0841ffb640 1 -- 172.21.15.78:0/3555133296 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f084c019430 con 0x5604fbb64ff0 2024-08-18T23:47:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.447+0000 7f0862fb6740 10 client.16104 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.447+0000 7f0862fb6740 10 client.16104 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:47:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.447+0000 7f0862fb6740 20 client.16104 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.447+0000 7f0862fb6740 1 --2- 172.21.15.78:0/3555133296 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x5604fbbea890 0x5604fbbecc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.447+0000 7f0862fb6740 4 client.16104 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:47:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.447+0000 7f0862fb6740 1 -- 172.21.15.78:0/3555133296 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5604fbad24c0 con 0x5604fbbea890 2024-08-18T23:47:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.448+0000 7f0859d9b640 1 --2- 172.21.15.78:0/3555133296 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x5604fbbea890 0x5604fbbecc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.448+0000 7f0841ffb640 10 client.16104 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:47:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.448+0000 7f0859d9b640 1 --2- 172.21.15.78:0/3555133296 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x5604fbbea890 0x5604fbbecc80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ee5554408ef011 server_cookie=88ed73e916f61459 in_seq=0 out_seq=0 2024-08-18T23:47:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.448+0000 7f0841ffb640 1 -- 172.21.15.78:0/3555133296 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5604fbad24c0 con 0x5604fbbea890 2024-08-18T23:47:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.448+0000 7f0841ffb640 10 client.16104 handle_command_reply: tid=0 2024-08-18T23:47:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.448+0000 7f0862fb6740 1 client.16104 shutdown 2024-08-18T23:47:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.448+0000 7f0862fb6740 1 -- 172.21.15.78:0/3555133296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb64ff0 msgr2=0x5604fbb78170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.448+0000 7f0862fb6740 1 --2- 172.21.15.78:0/3555133296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb64ff0 0x5604fbb78170 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f084c019fd0 tx=0x7f084c009c20 comp rx=0 tx=0).stop 2024-08-18T23:47:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.448+0000 7f0862fb6740 1 -- 172.21.15.78:0/3555133296 shutdown_connections 2024-08-18T23:47:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.448+0000 7f0862fb6740 1 --2- 172.21.15.78:0/3555133296 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x5604fbbea890 0x5604fbbecc80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.448+0000 7f0862fb6740 1 --2- 172.21.15.78:0/3555133296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604fbb701d0 0x5604fbb69680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.448+0000 7f0862fb6740 1 --2- 172.21.15.78:0/3555133296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604fbb65910 0x5604fbb786b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.448+0000 7f0841ffb640 0 client.16104 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:47:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.448+0000 7f0862fb6740 1 --2- 172.21.15.78:0/3555133296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604fbb64ff0 0x5604fbb78170 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.448+0000 7f0862fb6740 1 -- 172.21.15.78:0/3555133296 >> 172.21.15.78:0/3555133296 conn(0x5604fbb61580 msgr2=0x5604fbb6b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.448+0000 7f0862fb6740 1 -- 172.21.15.78:0/3555133296 shutdown_connections 2024-08-18T23:47:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.449+0000 7f0862fb6740 1 -- 172.21.15.78:0/3555133296 wait complete. 2024-08-18T23:47:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.449+0000 7f0862fb6740 20 client.16104 trim_cache size 0 max 16384 2024-08-18T23:47:57.463 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:47:57.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.449+0000 7f0860bb8640 1 -- 172.21.15.78:0/391413554 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f08440a2b80 msgr2=0x7f08440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:57.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.449+0000 7f0860bb8640 1 --2- 172.21.15.78:0/391413554 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f08440a2b80 0x7f08440a5040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5604fbb62900 tx=0x7f0850013040 comp rx=0 tx=0).stop 2024-08-18T23:47:57.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.449+0000 7f0860bb8640 1 -- 172.21.15.78:0/391413554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f085c102c20 msgr2=0x7f085c0fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:57.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.449+0000 7f0860bb8640 1 --2- 172.21.15.78:0/391413554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f085c102c20 0x7f085c0fd430 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0854004af0 tx=0x7f085402f3a0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.449+0000 7f0860bb8640 1 -- 172.21.15.78:0/391413554 shutdown_connections 2024-08-18T23:47:57.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.449+0000 7f0860bb8640 1 --2- 172.21.15.78:0/391413554 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f08440a2b80 0x7f08440a5040 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.449+0000 7f0860bb8640 1 --2- 172.21.15.78:0/391413554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f085c102c20 0x7f085c0fd430 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.449+0000 7f0860bb8640 1 --2- 172.21.15.78:0/391413554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f085c1022a0 0x7f085c0fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.449+0000 7f0860bb8640 1 --2- 172.21.15.78:0/391413554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f085c100b50 0x7f085c0fc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:57.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.449+0000 7f0860bb8640 1 -- 172.21.15.78:0/391413554 >> 172.21.15.78:0/391413554 conn(0x7f085c0fc500 msgr2=0x7f085c069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:57.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.450+0000 7f0860bb8640 1 -- 172.21.15.78:0/391413554 shutdown_connections 2024-08-18T23:47:57.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:57.450+0000 7f0860bb8640 1 -- 172.21.15.78:0/391413554 wait complete. 2024-08-18T23:47:57.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:57 smithi123 ceph-mon[21808]: pgmap v459: 129 pgs: 129 active+clean; 468 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-08-18T23:47:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:57 smithi161 ceph-mon[22027]: pgmap v459: 129 pgs: 129 active+clean; 468 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-08-18T23:47:57.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:57 smithi078 ceph-mon[18440]: pgmap v459: 129 pgs: 129 active+clean; 468 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-08-18T23:47:58.150 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-18T23:47:58.429 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:47:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.937+0000 7fb939e38640 1 -- 172.21.15.78:0/1401399145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb934101e40 msgr2=0x7fb93410d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.937+0000 7fb939e38640 1 --2- 172.21.15.78:0/1401399145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb934101e40 0x7fb93410d5c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009ca0 tx=0x7fb92c02a950 comp rx=0 tx=0).stop 2024-08-18T23:47:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.937+0000 7fb939e38640 1 -- 172.21.15.78:0/1401399145 shutdown_connections 2024-08-18T23:47:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.937+0000 7fb939e38640 1 --2- 172.21.15.78:0/1401399145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb934101e40 0x7fb93410d5c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.937+0000 7fb939e38640 1 --2- 172.21.15.78:0/1401399145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9341014c0 0x7fb934101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.937+0000 7fb939e38640 1 --2- 172.21.15.78:0/1401399145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9341058d0 0x7fb934105cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.937+0000 7fb939e38640 1 -- 172.21.15.78:0/1401399145 >> 172.21.15.78:0/1401399145 conn(0x7fb934076f40 msgr2=0x7fb934077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:58.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.937+0000 7fb939e38640 1 -- 172.21.15.78:0/1401399145 shutdown_connections 2024-08-18T23:47:58.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.937+0000 7fb939e38640 1 -- 172.21.15.78:0/1401399145 wait complete. 2024-08-18T23:47:58.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.938+0000 7fb939e38640 1 Processor -- start 2024-08-18T23:47:58.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.938+0000 7fb939e38640 1 -- start start 2024-08-18T23:47:58.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.938+0000 7fb939e38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9341014c0 0x7fb934196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:58.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.938+0000 7fb939e38640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934101e40 0x7fb9341971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:58.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.938+0000 7fb939e38640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9341058d0 0x7fb934197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:58.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.938+0000 7fb939e38640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb934110bd0 con 0x7fb9341014c0 2024-08-18T23:47:58.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.938+0000 7fb939e38640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb934110a50 con 0x7fb934101e40 2024-08-18T23:47:58.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.938+0000 7fb939e38640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb934110d50 con 0x7fb9341058d0 2024-08-18T23:47:58.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.938+0000 7fb9337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9341014c0 0x7fb934196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:58.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.938+0000 7fb9337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9341014c0 0x7fb934196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46620/0 (socket says 172.21.15.78:46620) 2024-08-18T23:47:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.938+0000 7fb9337fe640 1 -- 172.21.15.78:0/316730776 learned_addr learned my addr 172.21.15.78:0/316730776 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.938+0000 7fb92bfff640 1 --2- 172.21.15.78:0/316730776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934101e40 0x7fb9341971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.938+0000 7fb933fff640 1 --2- 172.21.15.78:0/316730776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9341058d0 0x7fb934197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.939+0000 7fb9337fe640 1 -- 172.21.15.78:0/316730776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9341058d0 msgr2=0x7fb934197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.939+0000 7fb9337fe640 1 --2- 172.21.15.78:0/316730776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9341058d0 0x7fb934197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.939+0000 7fb9337fe640 1 -- 172.21.15.78:0/316730776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934101e40 msgr2=0x7fb9341971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.939+0000 7fb9337fe640 1 --2- 172.21.15.78:0/316730776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934101e40 0x7fb9341971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.939+0000 7fb9337fe640 1 -- 172.21.15.78:0/316730776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9341a6520 con 0x7fb9341014c0 2024-08-18T23:47:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.939+0000 7fb92bfff640 1 --2- 172.21.15.78:0/316730776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934101e40 0x7fb9341971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.939+0000 7fb9337fe640 1 --2- 172.21.15.78:0/316730776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9341014c0 0x7fb934196cb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb92400e9f0 tx=0x7fb92400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.939+0000 7fb9317fa640 1 -- 172.21.15.78:0/316730776 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92400cd80 con 0x7fb9341014c0 2024-08-18T23:47:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.939+0000 7fb939e38640 1 -- 172.21.15.78:0/316730776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9341a6810 con 0x7fb9341014c0 2024-08-18T23:47:58.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.939+0000 7fb9317fa640 1 -- 172.21.15.78:0/316730776 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb924010910 con 0x7fb9341014c0 2024-08-18T23:47:58.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.939+0000 7fb9317fa640 1 -- 172.21.15.78:0/316730776 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240188f0 con 0x7fb9341014c0 2024-08-18T23:47:58.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.939+0000 7fb939e38640 1 -- 172.21.15.78:0/316730776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9341a6da0 con 0x7fb9341014c0 2024-08-18T23:47:58.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.941+0000 7fb9317fa640 1 -- 172.21.15.78:0/316730776 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb924010ac0 con 0x7fb9341014c0 2024-08-18T23:47:58.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb93c236740 1 Processor -- start 2024-08-18T23:47:58.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb93c236740 1 -- start start 2024-08-18T23:47:58.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb93c236740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad3d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:58.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb93c236740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 0x55ee4ad47c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:58.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb93c236740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4ad481d0 0x55ee4ad4a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:58.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb93c236740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ee4aca2a40 con 0x55ee4ad3cff0 2024-08-18T23:47:58.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb93c236740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ee4ac3a9d0 con 0x55ee4ad3d910 2024-08-18T23:47:58.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb93c236740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ee4ac9ea00 con 0x55ee4ad481d0 2024-08-18T23:47:58.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb9317fa640 1 --2- 172.21.15.78:0/316730776 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb9200a2b80 0x7fb9200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:58.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb9337fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 0x55ee4ad47c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:58.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb9337fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 0x55ee4ad47c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:46490/0 (socket says 172.21.15.78:46490) 2024-08-18T23:47:58.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb9337fe640 1 -- 172.21.15.78:0/3373809909 learned_addr learned my addr 172.21.15.78:0/3373809909 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:58.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb933fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad3d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:58.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb933fff640 1 --2- 172.21.15.78:0/316730776 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb9200a2b80 0x7fb9200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:58.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb92bfff640 1 --2- 172.21.15.78:0/3373809909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4ad481d0 0x55ee4ad4a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:58.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb9317fa640 1 -- 172.21.15.78:0/316730776 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fb9240cf7f0 con 0x7fb9341014c0 2024-08-18T23:47:58.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb9337fe640 1 -- 172.21.15.78:0/3373809909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4ad481d0 msgr2=0x55ee4ad4a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:58.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb9337fe640 1 --2- 172.21.15.78:0/3373809909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4ad481d0 0x55ee4ad4a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb9337fe640 1 -- 172.21.15.78:0/3373809909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad3cff0 msgr2=0x55ee4ad3d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:58.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.942+0000 7fb9337fe640 1 --2- 172.21.15.78:0/3373809909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad3d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.943+0000 7fb9337fe640 1 -- 172.21.15.78:0/3373809909 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee4ac41650 con 0x55ee4ad3d910 2024-08-18T23:47:58.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.943+0000 7fb933fff640 1 --2- 172.21.15.78:0/316730776 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb9200a2b80 0x7fb9200a5040 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb92c005fa0 tx=0x7fb92c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:58.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.943+0000 7fb9337fe640 1 --2- 172.21.15.78:0/3373809909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 0x55ee4ad47c90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb92400e9c0 tx=0x7fb92400cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83c1a6d22e5a0165 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:58.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.943+0000 7fb929ffb640 1 -- 172.21.15.78:0/3373809909 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240a4930 con 0x55ee4ad3d910 2024-08-18T23:47:58.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.943+0000 7fb929ffb640 1 -- 172.21.15.78:0/3373809909 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb924099fc0 con 0x55ee4ad3d910 2024-08-18T23:47:58.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.943+0000 7fb929ffb640 1 -- 172.21.15.78:0/3373809909 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92409b370 con 0x55ee4ad3d910 2024-08-18T23:47:58.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.944+0000 7fb93c236740 1 -- 172.21.15.78:0/3373809909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 msgr2=0x55ee4ad47c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:58.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.944+0000 7fb93c236740 1 --2- 172.21.15.78:0/3373809909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 0x55ee4ad47c90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb92400e9c0 tx=0x7fb92400cf90 comp rx=0 tx=0).stop 2024-08-18T23:47:58.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.944+0000 7fb93c236740 1 -- 172.21.15.78:0/3373809909 shutdown_connections 2024-08-18T23:47:58.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.944+0000 7fb93c236740 1 --2- 172.21.15.78:0/3373809909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4ad481d0 0x55ee4ad4a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.944+0000 7fb93c236740 1 --2- 172.21.15.78:0/3373809909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 0x55ee4ad47c90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.944+0000 7fb93c236740 1 --2- 172.21.15.78:0/3373809909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad3d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.944+0000 7fb93c236740 1 -- 172.21.15.78:0/3373809909 >> 172.21.15.78:0/3373809909 conn(0x55ee4ad39580 msgr2=0x55ee4ad39960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:58.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.944+0000 7fb93c236740 1 -- 172.21.15.78:0/3373809909 shutdown_connections 2024-08-18T23:47:58.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.944+0000 7fb93c236740 1 -- 172.21.15.78:0/3373809909 wait complete. 2024-08-18T23:47:58.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.944+0000 7fb93c236740 1 -- start start 2024-08-18T23:47:58.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.944+0000 7fb93c236740 1 Processor -- start 2024-08-18T23:47:58.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.944+0000 7fb93c236740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad45060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:58.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.944+0000 7fb93c236740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad3d910 0x55ee4ad455a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:58.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb93c236740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4ad481d0 0x55ee4ad45ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:58.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb93c236740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ee4ad4ad40 con 0x55ee4ad3d910 2024-08-18T23:47:58.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb93c236740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ee4ac5db20 con 0x55ee4ad3cff0 2024-08-18T23:47:58.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb93c236740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ee4ad4aec0 con 0x55ee4ad481d0 2024-08-18T23:47:58.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb93c236740 10 client.0 _ll_register_callbacks cb 0x55ee4ac5bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:58.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb9337fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad45060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:58.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb92bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad3d910 0x55ee4ad455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:58.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb9337fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad45060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:46504/0 (socket says 172.21.15.78:46504) 2024-08-18T23:47:58.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb9337fe640 1 -- 172.21.15.78:0/2105953252 learned_addr learned my addr 172.21.15.78:0/2105953252 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:58.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb933fff640 1 --2- 172.21.15.78:0/2105953252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4ad481d0 0x55ee4ad45ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:58.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb92bfff640 1 -- 172.21.15.78:0/2105953252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4ad481d0 msgr2=0x55ee4ad45ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:58.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb92bfff640 1 --2- 172.21.15.78:0/2105953252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4ad481d0 0x55ee4ad45ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb92bfff640 1 -- 172.21.15.78:0/2105953252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3cff0 msgr2=0x55ee4ad45060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:58.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb92bfff640 1 --2- 172.21.15.78:0/2105953252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad45060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb92bfff640 1 -- 172.21.15.78:0/2105953252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee4aa143c0 con 0x55ee4ad3d910 2024-08-18T23:47:58.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb9337fe640 1 --2- 172.21.15.78:0/2105953252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad45060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:58.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb92bfff640 1 --2- 172.21.15.78:0/2105953252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad3d910 0x55ee4ad455a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb934198e10 tx=0x7fb91c012dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616c5d1a9cf92556 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:58.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:47:58.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb912ffd640 1 -- 172.21.15.78:0/2105953252 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c004050 con 0x55ee4ad3d910 2024-08-18T23:47:58.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb912ffd640 1 -- 172.21.15.78:0/2105953252 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fb91c0041f0 con 0x55ee4ad3d910 2024-08-18T23:47:58.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb93c236740 1 -- 172.21.15.78:0/2105953252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb924099fc0 con 0x55ee4ad3d910 2024-08-18T23:47:58.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.945+0000 7fb912ffd640 1 -- 172.21.15.78:0/2105953252 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb91c025050 con 0x55ee4ad3d910 2024-08-18T23:47:58.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.946+0000 7fb93c236740 10 client.16116 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:58.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.946+0000 7fb93c236740 1 -- 172.21.15.78:0/2105953252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ee4ad56950 con 0x55ee4ad3d910 2024-08-18T23:47:58.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.946+0000 7fb912ffd640 1 -- 172.21.15.78:0/2105953252 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb924099fc0 con 0x55ee4ad3d910 2024-08-18T23:47:58.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.946+0000 7fb912ffd640 1 -- 172.21.15.78:0/2105953252 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fb91c003ae0 con 0x55ee4ad3d910 2024-08-18T23:47:58.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.946+0000 7fb93c236740 10 client.16116 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:58.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.946+0000 7fb93c236740 10 client.16116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:47:58.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.946+0000 7fb93c236740 20 client.16116 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:58.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.946+0000 7fb93c236740 1 --2- 172.21.15.78:0/2105953252 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ee4ad57e10 0x55ee4ad5a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:58.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.946+0000 7fb93c236740 4 client.16116 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:47:58.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.946+0000 7fb93c236740 1 -- 172.21.15.78:0/2105953252 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee4acaffb0 con 0x55ee4ad57e10 2024-08-18T23:47:58.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.946+0000 7fb9337fe640 1 --2- 172.21.15.78:0/2105953252 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ee4ad57e10 0x55ee4ad5a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:58.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.946+0000 7fb912ffd640 10 client.16116 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:58.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.946+0000 7fb9337fe640 1 --2- 172.21.15.78:0/2105953252 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ee4ad57e10 0x55ee4ad5a280 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37369090e1c0b8d9 server_cookie=14ee00775709a02a in_seq=0 out_seq=0 2024-08-18T23:47:58.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.948+0000 7fb912ffd640 1 -- 172.21.15.78:0/2105953252 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee4acaffb0 con 0x55ee4ad57e10 2024-08-18T23:47:58.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.948+0000 7fb912ffd640 10 client.16116 handle_command_reply: tid=0 2024-08-18T23:47:58.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.948+0000 7fb93c236740 1 client.16116 shutdown 2024-08-18T23:47:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.948+0000 7fb93c236740 1 -- 172.21.15.78:0/2105953252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad3d910 msgr2=0x55ee4ad455a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.948+0000 7fb93c236740 1 --2- 172.21.15.78:0/2105953252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad3d910 0x55ee4ad455a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb934198e10 tx=0x7fb91c012dc0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.948+0000 7fb93c236740 1 -- 172.21.15.78:0/2105953252 shutdown_connections 2024-08-18T23:47:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.948+0000 7fb93c236740 1 --2- 172.21.15.78:0/2105953252 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ee4ad57e10 0x55ee4ad5a280 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.948+0000 7fb93c236740 1 --2- 172.21.15.78:0/2105953252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4ad481d0 0x55ee4ad45ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.948+0000 7fb93c236740 1 --2- 172.21.15.78:0/2105953252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad3d910 0x55ee4ad455a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.948+0000 7fb93c236740 1 --2- 172.21.15.78:0/2105953252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad45060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.948+0000 7fb912ffd640 0 client.16116 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.948+0000 7fb93c236740 1 -- 172.21.15.78:0/2105953252 >> 172.21.15.78:0/2105953252 conn(0x55ee4ad39580 msgr2=0x55ee4ad3f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.948+0000 7fb93c236740 1 -- 172.21.15.78:0/2105953252 shutdown_connections 2024-08-18T23:47:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.948+0000 7fb93c236740 1 -- 172.21.15.78:0/2105953252 wait complete. 2024-08-18T23:47:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.948+0000 7fb93c236740 20 client.16116 trim_cache size 0 max 16384 2024-08-18T23:47:58.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.962+0000 7fb93c236740 1 Processor -- start 2024-08-18T23:47:58.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.962+0000 7fb93c236740 1 -- start start 2024-08-18T23:47:58.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.962+0000 7fb93c236740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad3faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:58.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.962+0000 7fb93c236740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 0x55ee4ad5fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:58.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.962+0000 7fb93c236740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad481d0 0x55ee4ad60120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:58.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.962+0000 7fb93c236740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341b3540 con 0x55ee4ad481d0 2024-08-18T23:47:58.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.962+0000 7fb93c236740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341b0e10 con 0x55ee4ad3d910 2024-08-18T23:47:58.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.962+0000 7fb93c236740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341aec60 con 0x55ee4ad3cff0 2024-08-18T23:47:58.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.962+0000 7fb933fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad481d0 0x55ee4ad60120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:58.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.962+0000 7fb92bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 0x55ee4ad5fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:58.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.962+0000 7fb92bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 0x55ee4ad5fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:46516/0 (socket says 172.21.15.78:46516) 2024-08-18T23:47:58.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.962+0000 7fb933fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad481d0 0x55ee4ad60120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46646/0 (socket says 172.21.15.78:46646) 2024-08-18T23:47:58.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.962+0000 7fb92bfff640 1 -- 172.21.15.78:0/2611126983 learned_addr learned my addr 172.21.15.78:0/2611126983 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:58.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.962+0000 7fb9337fe640 1 --2- 172.21.15.78:0/2611126983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad3faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:58.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.962+0000 7fb92bfff640 1 -- 172.21.15.78:0/2611126983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4ad3cff0 msgr2=0x55ee4ad3faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:58.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.962+0000 7fb92bfff640 1 --2- 172.21.15.78:0/2611126983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad3faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.962+0000 7fb92bfff640 1 -- 172.21.15.78:0/2611126983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad481d0 msgr2=0x55ee4ad60120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:58.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.962+0000 7fb92bfff640 1 --2- 172.21.15.78:0/2611126983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad481d0 0x55ee4ad60120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.962+0000 7fb92bfff640 1 -- 172.21.15.78:0/2611126983 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee4ad435b0 con 0x55ee4ad3d910 2024-08-18T23:47:58.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.963+0000 7fb92bfff640 1 --2- 172.21.15.78:0/2611126983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 0x55ee4ad5fbe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb91c003eb0 tx=0x7fb91c00f7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43de89ccfed90879 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:58.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.963+0000 7fb9297fa640 1 -- 172.21.15.78:0/2611126983 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c00f980 con 0x55ee4ad3d910 2024-08-18T23:47:58.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.963+0000 7fb9297fa640 1 -- 172.21.15.78:0/2611126983 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb91c005070 con 0x55ee4ad3d910 2024-08-18T23:47:58.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.963+0000 7fb9297fa640 1 -- 172.21.15.78:0/2611126983 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c003af0 con 0x55ee4ad3d910 2024-08-18T23:47:58.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.964+0000 7fb93c236740 1 -- 172.21.15.78:0/2611126983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 msgr2=0x55ee4ad5fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:58.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.964+0000 7fb93c236740 1 --2- 172.21.15.78:0/2611126983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 0x55ee4ad5fbe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb91c003eb0 tx=0x7fb91c00f7b0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.964+0000 7fb93c236740 1 -- 172.21.15.78:0/2611126983 shutdown_connections 2024-08-18T23:47:58.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.964+0000 7fb93c236740 1 --2- 172.21.15.78:0/2611126983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad481d0 0x55ee4ad60120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.964+0000 7fb93c236740 1 --2- 172.21.15.78:0/2611126983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 0x55ee4ad5fbe0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.964+0000 7fb93c236740 1 --2- 172.21.15.78:0/2611126983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad3faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.964+0000 7fb93c236740 1 -- 172.21.15.78:0/2611126983 >> 172.21.15.78:0/2611126983 conn(0x55ee4ad39580 msgr2=0x55ee4ad44150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:58.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.964+0000 7fb93c236740 1 -- 172.21.15.78:0/2611126983 shutdown_connections 2024-08-18T23:47:58.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.964+0000 7fb93c236740 1 -- 172.21.15.78:0/2611126983 wait complete. 2024-08-18T23:47:58.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.964+0000 7fb93c236740 1 -- start start 2024-08-18T23:47:58.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.964+0000 7fb93c236740 1 Processor -- start 2024-08-18T23:47:58.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb93c236740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad4b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:58.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb93c236740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 0x55ee4ad4bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:58.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb93c236740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4ad481d0 0x55ee4adc3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:58.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb93c236740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341af2d0 con 0x55ee4ad3cff0 2024-08-18T23:47:58.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb93c236740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341b0740 con 0x55ee4ad3d910 2024-08-18T23:47:58.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb93c236740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341af9a0 con 0x55ee4ad481d0 2024-08-18T23:47:58.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb93c236740 10 client.0 _ll_register_callbacks cb 0x7fb91c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:47:58.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb9337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad4b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:58.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb9337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad4b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46660/0 (socket says 172.21.15.78:46660) 2024-08-18T23:47:58.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb92bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 0x55ee4ad4bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:58.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb9337fe640 1 -- 172.21.15.78:0/2083491875 learned_addr learned my addr 172.21.15.78:0/2083491875 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:47:58.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb9337fe640 1 -- 172.21.15.78:0/2083491875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4ad481d0 msgr2=0x55ee4adc3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:58.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb9337fe640 1 --2- 172.21.15.78:0/2083491875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4ad481d0 0x55ee4adc3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb9337fe640 1 -- 172.21.15.78:0/2083491875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 msgr2=0x55ee4ad4bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:47:58.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb9337fe640 1 --2- 172.21.15.78:0/2083491875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 0x55ee4ad4bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb9337fe640 1 -- 172.21.15.78:0/2083491875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee4adc39f0 con 0x55ee4ad3cff0 2024-08-18T23:47:58.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb92bfff640 1 --2- 172.21.15.78:0/2083491875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 0x55ee4ad4bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:47:58.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb9337fe640 1 --2- 172.21.15.78:0/2083491875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad4b850 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb92409b9c0 tx=0x7fb92409be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f5293644f73eed server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:47:58.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:47:58.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb912ffd640 1 -- 172.21.15.78:0/2083491875 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92409b110 con 0x55ee4ad3cff0 2024-08-18T23:47:58.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb912ffd640 1 -- 172.21.15.78:0/2083491875 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fb92409b2b0 con 0x55ee4ad3cff0 2024-08-18T23:47:58.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb93c236740 1 -- 172.21.15.78:0/2083491875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb91c005070 con 0x55ee4ad3cff0 2024-08-18T23:47:58.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb912ffd640 1 -- 172.21.15.78:0/2083491875 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb924010430 con 0x55ee4ad3cff0 2024-08-18T23:47:58.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.965+0000 7fb93c236740 10 client.16122 fetch_fsmap learned FSMap version 26 2024-08-18T23:47:58.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.966+0000 7fb93c236740 1 -- 172.21.15.78:0/2083491875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ee4adcad80 con 0x55ee4ad3cff0 2024-08-18T23:47:58.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.966+0000 7fb912ffd640 1 -- 172.21.15.78:0/2083491875 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb91c005070 con 0x55ee4ad3cff0 2024-08-18T23:47:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.966+0000 7fb912ffd640 1 -- 172.21.15.78:0/2083491875 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fb92409ab80 con 0x55ee4ad3cff0 2024-08-18T23:47:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.966+0000 7fb93c236740 10 client.16122 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:47:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.966+0000 7fb93c236740 10 client.16122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:47:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.966+0000 7fb93c236740 20 client.16122 populate_metadata read hostname 'smithi078' 2024-08-18T23:47:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.966+0000 7fb93c236740 1 --2- 172.21.15.78:0/2083491875 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ee4adcb0c0 0x55ee4adcd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:47:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.966+0000 7fb93c236740 4 client.16122 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:47:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.966+0000 7fb93c236740 1 -- 172.21.15.78:0/2083491875 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee4acaa4c0 con 0x55ee4adcb0c0 2024-08-18T23:47:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.966+0000 7fb92bfff640 1 --2- 172.21.15.78:0/2083491875 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ee4adcb0c0 0x55ee4adcd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:47:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.966+0000 7fb912ffd640 10 client.16122 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.966+0000 7fb92bfff640 1 --2- 172.21.15.78:0/2083491875 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ee4adcb0c0 0x55ee4adcd500 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27e2f8672cb92efc server_cookie=ba78e999c68127a in_seq=0 out_seq=0 2024-08-18T23:47:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.968+0000 7fb912ffd640 1 -- 172.21.15.78:0/2083491875 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee4acaa4c0 con 0x55ee4adcb0c0 2024-08-18T23:47:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.968+0000 7fb912ffd640 10 client.16122 handle_command_reply: tid=0 2024-08-18T23:47:58.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.968+0000 7fb93c236740 1 client.16122 shutdown 2024-08-18T23:47:58.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.968+0000 7fb93c236740 1 -- 172.21.15.78:0/2083491875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad3cff0 msgr2=0x55ee4ad4b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:58.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.968+0000 7fb93c236740 1 --2- 172.21.15.78:0/2083491875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad4b850 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb92409b9c0 tx=0x7fb92409be40 comp rx=0 tx=0).stop 2024-08-18T23:47:58.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.968+0000 7fb93c236740 1 -- 172.21.15.78:0/2083491875 shutdown_connections 2024-08-18T23:47:58.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.968+0000 7fb93c236740 1 --2- 172.21.15.78:0/2083491875 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ee4adcb0c0 0x55ee4adcd500 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.968+0000 7fb93c236740 1 --2- 172.21.15.78:0/2083491875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ee4ad481d0 0x55ee4adc3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.968+0000 7fb93c236740 1 --2- 172.21.15.78:0/2083491875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4ad3d910 0x55ee4ad4bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.968+0000 7fb912ffd640 0 client.16122 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:47:58.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.968+0000 7fb93c236740 1 --2- 172.21.15.78:0/2083491875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4ad3cff0 0x55ee4ad4b850 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.968+0000 7fb93c236740 1 -- 172.21.15.78:0/2083491875 >> 172.21.15.78:0/2083491875 conn(0x55ee4ad39580 msgr2=0x55ee4ad37d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:58.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.968+0000 7fb93c236740 1 -- 172.21.15.78:0/2083491875 shutdown_connections 2024-08-18T23:47:58.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.968+0000 7fb93c236740 1 -- 172.21.15.78:0/2083491875 wait complete. 2024-08-18T23:47:58.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.968+0000 7fb93c236740 20 client.16122 trim_cache size 0 max 16384 2024-08-18T23:47:58.980 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:47:58.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.969+0000 7fb939e38640 1 -- 172.21.15.78:0/316730776 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb9200a2b80 msgr2=0x7fb9200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:58.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.969+0000 7fb939e38640 1 --2- 172.21.15.78:0/316730776 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb9200a2b80 0x7fb9200a5040 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb92c005fa0 tx=0x7fb92c0095c0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.969+0000 7fb939e38640 1 -- 172.21.15.78:0/316730776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9341014c0 msgr2=0x7fb934196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:47:58.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.969+0000 7fb939e38640 1 --2- 172.21.15.78:0/316730776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9341014c0 0x7fb934196cb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb92400e9f0 tx=0x7fb92400eeb0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.969+0000 7fb939e38640 1 -- 172.21.15.78:0/316730776 shutdown_connections 2024-08-18T23:47:58.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.969+0000 7fb939e38640 1 --2- 172.21.15.78:0/316730776 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb9200a2b80 0x7fb9200a5040 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.969+0000 7fb939e38640 1 --2- 172.21.15.78:0/316730776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9341058d0 0x7fb934197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.969+0000 7fb939e38640 1 --2- 172.21.15.78:0/316730776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb934101e40 0x7fb9341971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.969+0000 7fb939e38640 1 --2- 172.21.15.78:0/316730776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9341014c0 0x7fb934196cb0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:47:58.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.969+0000 7fb939e38640 1 -- 172.21.15.78:0/316730776 >> 172.21.15.78:0/316730776 conn(0x7fb934076f40 msgr2=0x7fb9340fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:47:58.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.969+0000 7fb939e38640 1 -- 172.21.15.78:0/316730776 shutdown_connections 2024-08-18T23:47:58.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:47:58.969+0000 7fb939e38640 1 -- 172.21.15.78:0/316730776 wait complete. 2024-08-18T23:47:59.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:47:59 smithi123 ceph-mon[21808]: pgmap v460: 129 pgs: 129 active+clean; 468 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 196 op/s 2024-08-18T23:47:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:47:59 smithi161 ceph-mon[22027]: pgmap v460: 129 pgs: 129 active+clean; 468 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 196 op/s 2024-08-18T23:47:59.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:47:59 smithi078 ceph-mon[18440]: pgmap v460: 129 pgs: 129 active+clean; 468 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 196 op/s 2024-08-18T23:48:01.075 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-18T23:48:01.075 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-18T23:48:01.337 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:48:01.459 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:01 smithi078 ceph-mon[18440]: pgmap v461: 129 pgs: 129 active+clean; 468 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 953 KiB/s wr, 230 op/s 2024-08-18T23:48:01.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:01 smithi123 ceph-mon[21808]: pgmap v461: 129 pgs: 129 active+clean; 468 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 953 KiB/s wr, 230 op/s 2024-08-18T23:48:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:01 smithi161 ceph-mon[22027]: pgmap v461: 129 pgs: 129 active+clean; 468 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 953 KiB/s wr, 230 op/s 2024-08-18T23:48:01.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.856+0000 7f230c736640 1 -- 172.21.15.78:0/3895217990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2304102250 msgr2=0x7f230410e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:01.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.856+0000 7f230c736640 1 --2- 172.21.15.78:0/3895217990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2304102250 0x7f230410e780 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009cd0 tx=0x7f22fc02a8e0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.857+0000 7f230c736640 1 -- 172.21.15.78:0/3895217990 shutdown_connections 2024-08-18T23:48:01.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.857+0000 7f230c736640 1 --2- 172.21.15.78:0/3895217990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2304102250 0x7f230410e780 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.857+0000 7f230c736640 1 --2- 172.21.15.78:0/3895217990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23041018b0 0x7f2304101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.857+0000 7f230c736640 1 --2- 172.21.15.78:0/3895217990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23041078b0 0x7f2304107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.857+0000 7f230c736640 1 -- 172.21.15.78:0/3895217990 >> 172.21.15.78:0/3895217990 conn(0x7f23040fd5f0 msgr2=0x7f23040ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:01.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.857+0000 7f230c736640 1 -- 172.21.15.78:0/3895217990 shutdown_connections 2024-08-18T23:48:01.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.857+0000 7f230c736640 1 -- 172.21.15.78:0/3895217990 wait complete. 2024-08-18T23:48:01.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.857+0000 7f230c736640 1 Processor -- start 2024-08-18T23:48:01.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.857+0000 7f230c736640 1 -- start start 2024-08-18T23:48:01.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.857+0000 7f230c736640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23041018b0 0x7f2304074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:01.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.857+0000 7f230c736640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2304102250 0x7f2304074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:01.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.857+0000 7f230c736640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23041078b0 0x7f2304074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:01.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.857+0000 7f230c736640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2304111cb0 con 0x7f23041078b0 2024-08-18T23:48:01.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.857+0000 7f230c736640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2304111b30 con 0x7f23041018b0 2024-08-18T23:48:01.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.857+0000 7f230c736640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2304111e30 con 0x7f2304102250 2024-08-18T23:48:01.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.858+0000 7f230acd3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23041078b0 0x7f2304074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:01.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.858+0000 7f230acd3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23041078b0 0x7f2304074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46682/0 (socket says 172.21.15.78:46682) 2024-08-18T23:48:01.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.858+0000 7f230acd3640 1 -- 172.21.15.78:0/3812567751 learned_addr learned my addr 172.21.15.78:0/3812567751 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:01.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.858+0000 7f230a4d2640 1 --2- 172.21.15.78:0/3812567751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23041018b0 0x7f2304074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:01.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.858+0000 7f2309cd1640 1 --2- 172.21.15.78:0/3812567751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2304102250 0x7f2304074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:01.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.858+0000 7f230acd3640 1 -- 172.21.15.78:0/3812567751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2304102250 msgr2=0x7f2304074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:01.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.858+0000 7f230acd3640 1 --2- 172.21.15.78:0/3812567751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2304102250 0x7f2304074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.858+0000 7f230acd3640 1 -- 172.21.15.78:0/3812567751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23041018b0 msgr2=0x7f2304074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:01.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.858+0000 7f230acd3640 1 --2- 172.21.15.78:0/3812567751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23041018b0 0x7f2304074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.858+0000 7f230acd3640 1 -- 172.21.15.78:0/3812567751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23041a6620 con 0x7f23041078b0 2024-08-18T23:48:01.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.858+0000 7f230acd3640 1 --2- 172.21.15.78:0/3812567751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23041078b0 0x7f2304074ca0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f22fc005cc0 tx=0x7f22fc003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:01.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.858+0000 7f22fb7fe640 1 -- 172.21.15.78:0/3812567751 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc004040 con 0x7f23041078b0 2024-08-18T23:48:01.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.858+0000 7f230c736640 1 -- 172.21.15.78:0/3812567751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23041a68b0 con 0x7f23041078b0 2024-08-18T23:48:01.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.858+0000 7f22fb7fe640 1 -- 172.21.15.78:0/3812567751 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22fc0046c0 con 0x7f23041078b0 2024-08-18T23:48:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.858+0000 7f22fb7fe640 1 -- 172.21.15.78:0/3812567751 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc046480 con 0x7f23041078b0 2024-08-18T23:48:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.858+0000 7f230c736640 1 -- 172.21.15.78:0/3812567751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23041a6e40 con 0x7f23041078b0 2024-08-18T23:48:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.860+0000 7f22fb7fe640 1 -- 172.21.15.78:0/3812567751 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f22fc0305a0 con 0x7f23041078b0 2024-08-18T23:48:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.861+0000 7f230eb34740 1 Processor -- start 2024-08-18T23:48:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.861+0000 7f230eb34740 1 -- start start 2024-08-18T23:48:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.861+0000 7f230eb34740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f87a56f200 0x55f87a56f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.861+0000 7f230eb34740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87a56fb20 0x55f87a579ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.861+0000 7f230eb34740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a57a3e0 0x55f87a57c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.861+0000 7f230eb34740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f87a532e40 con 0x55f87a57a3e0 2024-08-18T23:48:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.861+0000 7f230eb34740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f87a4d47f0 con 0x55f87a56f200 2024-08-18T23:48:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.861+0000 7f230eb34740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f87a47bfb0 con 0x55f87a56fb20 2024-08-18T23:48:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.862+0000 7f22fb7fe640 1 --2- 172.21.15.78:0/3812567751 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f22f00a2b80 0x7f22f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.862+0000 7f2309cd1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a57a3e0 0x55f87a57c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.862+0000 7f2309cd1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a57a3e0 0x55f87a57c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46694/0 (socket says 172.21.15.78:46694) 2024-08-18T23:48:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.862+0000 7f2309cd1640 1 -- 172.21.15.78:0/1917612228 learned_addr learned my addr 172.21.15.78:0/1917612228 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.862+0000 7f230a4d2640 1 --2- 172.21.15.78:0/1917612228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87a56fb20 0x55f87a579ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.862+0000 7f2309cd1640 1 --2- 172.21.15.78:0/1917612228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f87a56f200 0x55f87a56f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.862+0000 7f230acd3640 1 --2- 172.21.15.78:0/3812567751 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f22f00a2b80 0x7f22f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.862+0000 7f2309cd1640 1 -- 172.21.15.78:0/1917612228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87a56fb20 msgr2=0x55f87a579ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.862+0000 7f2309cd1640 1 --2- 172.21.15.78:0/1917612228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87a56fb20 0x55f87a579ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.862+0000 7f2309cd1640 1 -- 172.21.15.78:0/1917612228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f87a56f200 msgr2=0x55f87a56f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:01.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.862+0000 7f2309cd1640 1 --2- 172.21.15.78:0/1917612228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f87a56f200 0x55f87a56f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.862+0000 7f22fb7fe640 1 -- 172.21.15.78:0/3812567751 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f22fc02e030 con 0x7f23041078b0 2024-08-18T23:48:01.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.862+0000 7f2309cd1640 1 -- 172.21.15.78:0/1917612228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f87a2466a0 con 0x55f87a57a3e0 2024-08-18T23:48:01.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.862+0000 7f2309cd1640 1 --2- 172.21.15.78:0/1917612228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a57a3e0 0x55f87a57c7d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2300013600 tx=0x7f230000c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9b81dcb868b46a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:01.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.862+0000 7f230acd3640 1 --2- 172.21.15.78:0/3812567751 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f22f00a2b80 0x7f22f00a5040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0b9aa0 tx=0x7f22fc0c0040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:01.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.862+0000 7f22f8ff9640 1 -- 172.21.15.78:0/1917612228 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2300019030 con 0x55f87a57a3e0 2024-08-18T23:48:01.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.862+0000 7f22f8ff9640 1 -- 172.21.15.78:0/1917612228 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2300011a00 con 0x55f87a57a3e0 2024-08-18T23:48:01.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.862+0000 7f22f8ff9640 1 -- 172.21.15.78:0/1917612228 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230000fa10 con 0x55f87a57a3e0 2024-08-18T23:48:01.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.863+0000 7f230eb34740 1 -- 172.21.15.78:0/1917612228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a57a3e0 msgr2=0x55f87a57c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:01.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.863+0000 7f230eb34740 1 --2- 172.21.15.78:0/1917612228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a57a3e0 0x55f87a57c7d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2300013600 tx=0x7f230000c6b0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.863+0000 7f230eb34740 1 -- 172.21.15.78:0/1917612228 shutdown_connections 2024-08-18T23:48:01.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.863+0000 7f230eb34740 1 --2- 172.21.15.78:0/1917612228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a57a3e0 0x55f87a57c7d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.863+0000 7f230eb34740 1 --2- 172.21.15.78:0/1917612228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87a56fb20 0x55f87a579ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.863+0000 7f230eb34740 1 --2- 172.21.15.78:0/1917612228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f87a56f200 0x55f87a56f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.863+0000 7f230eb34740 1 -- 172.21.15.78:0/1917612228 >> 172.21.15.78:0/1917612228 conn(0x55f87a56b790 msgr2=0x55f87a56bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:01.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.863+0000 7f230eb34740 1 -- 172.21.15.78:0/1917612228 shutdown_connections 2024-08-18T23:48:01.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.863+0000 7f230eb34740 1 -- 172.21.15.78:0/1917612228 wait complete. 2024-08-18T23:48:01.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.863+0000 7f230eb34740 1 -- start start 2024-08-18T23:48:01.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f230eb34740 1 Processor -- start 2024-08-18T23:48:01.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f230eb34740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a56f200 0x55f87a577330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:01.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f230eb34740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87a56fb20 0x55f87a577870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:01.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f230eb34740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f87a57a3e0 0x55f87a577db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:01.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f230eb34740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f87a48fcb0 con 0x55f87a56f200 2024-08-18T23:48:01.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f230eb34740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f87a4d07b0 con 0x55f87a57a3e0 2024-08-18T23:48:01.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f230eb34740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f87a57cfb0 con 0x55f87a56fb20 2024-08-18T23:48:01.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f230eb34740 10 client.0 _ll_register_callbacks cb 0x55f87a1e3ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:48:01.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f2309cd1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a56f200 0x55f87a577330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:01.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f2309cd1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a56f200 0x55f87a577330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46700/0 (socket says 172.21.15.78:46700) 2024-08-18T23:48:01.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f2309cd1640 1 -- 172.21.15.78:0/3971649868 learned_addr learned my addr 172.21.15.78:0/3971649868 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:01.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f2309cd1640 1 -- 172.21.15.78:0/3971649868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87a56fb20 msgr2=0x55f87a577870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:01.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f2309cd1640 1 --2- 172.21.15.78:0/3971649868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87a56fb20 0x55f87a577870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f2309cd1640 1 -- 172.21.15.78:0/3971649868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f87a57a3e0 msgr2=0x55f87a577db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:01.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f2309cd1640 1 --2- 172.21.15.78:0/3971649868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f87a57a3e0 0x55f87a577db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f2309cd1640 1 -- 172.21.15.78:0/3971649868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f87a48ebc0 con 0x55f87a56f200 2024-08-18T23:48:01.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f2309cd1640 1 --2- 172.21.15.78:0/3971649868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a56f200 0x55f87a577330 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2300013f50 tx=0x7f230000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8120d6ff21c02f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:01.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f22edffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:48:01.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f22edffb640 1 -- 172.21.15.78:0/3971649868 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2300019040 con 0x55f87a56f200 2024-08-18T23:48:01.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.864+0000 7f22edffb640 1 -- 172.21.15.78:0/3971649868 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2300004030 con 0x55f87a56f200 2024-08-18T23:48:01.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.865+0000 7f230eb34740 1 -- 172.21.15.78:0/3971649868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2300011a00 con 0x55f87a56f200 2024-08-18T23:48:01.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.865+0000 7f22edffb640 1 -- 172.21.15.78:0/3971649868 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2300004230 con 0x55f87a56f200 2024-08-18T23:48:01.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.865+0000 7f230eb34740 10 client.16134 fetch_fsmap learned FSMap version 26 2024-08-18T23:48:01.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.865+0000 7f230eb34740 1 -- 172.21.15.78:0/3971649868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f87a588f00 con 0x55f87a56f200 2024-08-18T23:48:01.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.865+0000 7f22edffb640 1 -- 172.21.15.78:0/3971649868 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2300011a00 con 0x55f87a56f200 2024-08-18T23:48:01.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.865+0000 7f22edffb640 1 -- 172.21.15.78:0/3971649868 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f23000035f0 con 0x55f87a56f200 2024-08-18T23:48:01.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.865+0000 7f230eb34740 10 client.16134 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:48:01.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.865+0000 7f230eb34740 10 client.16134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:48:01.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.865+0000 7f230eb34740 20 client.16134 populate_metadata read hostname 'smithi078' 2024-08-18T23:48:01.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.865+0000 7f230eb34740 1 --2- 172.21.15.78:0/3971649868 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f87a58a3c0 0x55f87a58c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:01.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.865+0000 7f230eb34740 4 client.16134 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:48:01.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.865+0000 7f230eb34740 1 -- 172.21.15.78:0/3971649868 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f87a4e1d60 con 0x55f87a58a3c0 2024-08-18T23:48:01.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.865+0000 7f230a4d2640 1 --2- 172.21.15.78:0/3971649868 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f87a58a3c0 0x55f87a58c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:01.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.866+0000 7f230a4d2640 1 --2- 172.21.15.78:0/3971649868 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f87a58a3c0 0x55f87a58c7b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b8e6bb5984aca0 server_cookie=2dbbbcc6c9df18bc in_seq=0 out_seq=0 2024-08-18T23:48:01.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.866+0000 7f22edffb640 10 client.16134 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:01.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.866+0000 7f22edffb640 1 -- 172.21.15.78:0/3971649868 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f87a4e1d60 con 0x55f87a58a3c0 2024-08-18T23:48:01.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.866+0000 7f22edffb640 10 client.16134 handle_command_reply: tid=0 2024-08-18T23:48:01.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.866+0000 7f230eb34740 1 client.16134 shutdown 2024-08-18T23:48:01.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.866+0000 7f230eb34740 1 -- 172.21.15.78:0/3971649868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a56f200 msgr2=0x55f87a577330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:01.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.866+0000 7f230eb34740 1 --2- 172.21.15.78:0/3971649868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a56f200 0x55f87a577330 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2300013f50 tx=0x7f230000cc80 comp rx=0 tx=0).stop 2024-08-18T23:48:01.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.867+0000 7f230eb34740 1 -- 172.21.15.78:0/3971649868 shutdown_connections 2024-08-18T23:48:01.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.867+0000 7f230eb34740 1 --2- 172.21.15.78:0/3971649868 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f87a58a3c0 0x55f87a58c7b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.867+0000 7f230eb34740 1 --2- 172.21.15.78:0/3971649868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f87a57a3e0 0x55f87a577db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.867+0000 7f230eb34740 1 --2- 172.21.15.78:0/3971649868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87a56fb20 0x55f87a577870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.867+0000 7f22edffb640 0 client.16134 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:01.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.867+0000 7f230eb34740 1 --2- 172.21.15.78:0/3971649868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a56f200 0x55f87a577330 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.867+0000 7f230eb34740 1 -- 172.21.15.78:0/3971649868 >> 172.21.15.78:0/3971649868 conn(0x55f87a56b790 msgr2=0x55f87a571470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:01.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.867+0000 7f230eb34740 1 -- 172.21.15.78:0/3971649868 shutdown_connections 2024-08-18T23:48:01.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.867+0000 7f230eb34740 1 -- 172.21.15.78:0/3971649868 wait complete. 2024-08-18T23:48:01.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.867+0000 7f230eb34740 20 client.16134 trim_cache size 0 max 16384 2024-08-18T23:48:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.884+0000 7f230eb34740 1 Processor -- start 2024-08-18T23:48:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.884+0000 7f230eb34740 1 -- start start 2024-08-18T23:48:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.884+0000 7f230eb34740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a56f200 0x55f87a571360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.884+0000 7f230eb34740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87a56fb20 0x55f87a591ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.884+0000 7f230eb34740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f87a57a3e0 0x55f87a5923e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.884+0000 7f230eb34740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23041b3460 con 0x55f87a56f200 2024-08-18T23:48:01.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.884+0000 7f230eb34740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f23041b0d30 con 0x55f87a57a3e0 2024-08-18T23:48:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.884+0000 7f230eb34740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f23041aeb80 con 0x55f87a56fb20 2024-08-18T23:48:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.884+0000 7f2309cd1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a56f200 0x55f87a571360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.884+0000 7f2309cd1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a56f200 0x55f87a571360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46704/0 (socket says 172.21.15.78:46704) 2024-08-18T23:48:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.884+0000 7f2309cd1640 1 -- 172.21.15.78:0/324824061 learned_addr learned my addr 172.21.15.78:0/324824061 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.884+0000 7f2309cd1640 1 -- 172.21.15.78:0/324824061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87a56fb20 msgr2=0x55f87a591ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.884+0000 7f230a4d2640 1 --2- 172.21.15.78:0/324824061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87a56fb20 0x55f87a591ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.884+0000 7f2309cd1640 1 --2- 172.21.15.78:0/324824061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87a56fb20 0x55f87a591ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.884+0000 7f2309cd1640 1 -- 172.21.15.78:0/324824061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f87a57a3e0 msgr2=0x55f87a5923e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.884+0000 7f2309cd1640 1 --2- 172.21.15.78:0/324824061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f87a57a3e0 0x55f87a5923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.884+0000 7f2309cd1640 1 -- 172.21.15.78:0/324824061 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f87a56f030 con 0x55f87a56f200 2024-08-18T23:48:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.884+0000 7f2309cd1640 1 --2- 172.21.15.78:0/324824061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a56f200 0x55f87a571360 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f23000039c0 tx=0x7f230000c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea2ce928ac8124b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:01.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.885+0000 7f22effff640 1 -- 172.21.15.78:0/324824061 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230003b070 con 0x55f87a56f200 2024-08-18T23:48:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.885+0000 7f22effff640 1 -- 172.21.15.78:0/324824061 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f230000f440 con 0x55f87a56f200 2024-08-18T23:48:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.885+0000 7f22effff640 1 -- 172.21.15.78:0/324824061 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230000ce70 con 0x55f87a56f200 2024-08-18T23:48:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.885+0000 7f230eb34740 1 -- 172.21.15.78:0/324824061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a56f200 msgr2=0x55f87a571360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.885+0000 7f230eb34740 1 --2- 172.21.15.78:0/324824061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a56f200 0x55f87a571360 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f23000039c0 tx=0x7f230000c570 comp rx=0 tx=0).stop 2024-08-18T23:48:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.885+0000 7f230eb34740 1 -- 172.21.15.78:0/324824061 shutdown_connections 2024-08-18T23:48:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.885+0000 7f230eb34740 1 --2- 172.21.15.78:0/324824061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f87a57a3e0 0x55f87a5923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.885+0000 7f230eb34740 1 --2- 172.21.15.78:0/324824061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87a56fb20 0x55f87a591ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.885+0000 7f230eb34740 1 --2- 172.21.15.78:0/324824061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a56f200 0x55f87a571360 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.885+0000 7f230eb34740 1 -- 172.21.15.78:0/324824061 >> 172.21.15.78:0/324824061 conn(0x55f87a56b790 msgr2=0x55f87a570c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.885+0000 7f230eb34740 1 -- 172.21.15.78:0/324824061 shutdown_connections 2024-08-18T23:48:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.885+0000 7f230eb34740 1 -- 172.21.15.78:0/324824061 wait complete. 2024-08-18T23:48:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.886+0000 7f230eb34740 1 -- start start 2024-08-18T23:48:01.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.886+0000 7f230eb34740 1 Processor -- start 2024-08-18T23:48:01.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.886+0000 7f230eb34740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f87a56f200 0x55f87a582860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:01.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.886+0000 7f230eb34740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87a56fb20 0x55f87a573fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:01.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.886+0000 7f230eb34740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a57a3e0 0x55f87a574500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:01.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.886+0000 7f230eb34740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23041af1f0 con 0x55f87a57a3e0 2024-08-18T23:48:01.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.886+0000 7f230eb34740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f23041b0660 con 0x55f87a56f200 2024-08-18T23:48:01.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.886+0000 7f230eb34740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f23041af8c0 con 0x55f87a56fb20 2024-08-18T23:48:01.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.886+0000 7f230eb34740 10 client.0 _ll_register_callbacks cb 0x7f22f4004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:48:01.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.886+0000 7f2309cd1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a57a3e0 0x55f87a574500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:01.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.886+0000 7f2309cd1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a57a3e0 0x55f87a574500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46708/0 (socket says 172.21.15.78:46708) 2024-08-18T23:48:01.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.886+0000 7f2309cd1640 1 -- 172.21.15.78:0/2055025000 learned_addr learned my addr 172.21.15.78:0/2055025000 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:01.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.886+0000 7f2309cd1640 1 --2- 172.21.15.78:0/2055025000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f87a56f200 0x55f87a582860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:01.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.886+0000 7f230a4d2640 1 --2- 172.21.15.78:0/2055025000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87a56fb20 0x55f87a573fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:01.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.886+0000 7f2309cd1640 1 -- 172.21.15.78:0/2055025000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87a56fb20 msgr2=0x55f87a573fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:01.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.886+0000 7f2309cd1640 1 --2- 172.21.15.78:0/2055025000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87a56fb20 0x55f87a573fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.886+0000 7f2309cd1640 1 -- 172.21.15.78:0/2055025000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f87a56f200 msgr2=0x55f87a582860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:01.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.886+0000 7f2309cd1640 1 --2- 172.21.15.78:0/2055025000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f87a56f200 0x55f87a582860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.886+0000 7f2309cd1640 1 -- 172.21.15.78:0/2055025000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f87a574bf0 con 0x55f87a57a3e0 2024-08-18T23:48:01.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.887+0000 7f2309cd1640 1 --2- 172.21.15.78:0/2055025000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a57a3e0 0x55f87a574500 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2300005cb0 tx=0x7f230000c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ea175de7a08977 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:01.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.887+0000 7f22edffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:48:01.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.887+0000 7f22edffb640 1 -- 172.21.15.78:0/2055025000 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2300019070 con 0x55f87a57a3e0 2024-08-18T23:48:01.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.887+0000 7f22edffb640 1 -- 172.21.15.78:0/2055025000 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2300011730 con 0x55f87a57a3e0 2024-08-18T23:48:01.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.887+0000 7f230eb34740 1 -- 172.21.15.78:0/2055025000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f230000f440 con 0x55f87a57a3e0 2024-08-18T23:48:01.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.887+0000 7f22edffb640 1 -- 172.21.15.78:0/2055025000 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f230003b040 con 0x55f87a57a3e0 2024-08-18T23:48:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.887+0000 7f230eb34740 10 client.16140 fetch_fsmap learned FSMap version 26 2024-08-18T23:48:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.887+0000 7f230eb34740 1 -- 172.21.15.78:0/2055025000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f87a5f4d90 con 0x55f87a57a3e0 2024-08-18T23:48:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.887+0000 7f22edffb640 1 -- 172.21.15.78:0/2055025000 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f230000f440 con 0x55f87a57a3e0 2024-08-18T23:48:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.887+0000 7f22edffb640 1 -- 172.21.15.78:0/2055025000 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f230000f6d0 con 0x55f87a57a3e0 2024-08-18T23:48:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.887+0000 7f230eb34740 10 client.16140 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:48:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.887+0000 7f230eb34740 10 client.16140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:48:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.887+0000 7f230eb34740 20 client.16140 populate_metadata read hostname 'smithi078' 2024-08-18T23:48:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.887+0000 7f230eb34740 1 --2- 172.21.15.78:0/2055025000 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f87a5f53f0 0x55f87a5f7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.887+0000 7f230eb34740 4 client.16140 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-18T23:48:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.887+0000 7f230eb34740 1 -- 172.21.15.78:0/2055025000 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f87a4dc270 con 0x55f87a5f53f0 2024-08-18T23:48:01.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.887+0000 7f230a4d2640 1 --2- 172.21.15.78:0/2055025000 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f87a5f53f0 0x55f87a5f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:01.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.888+0000 7f230a4d2640 1 --2- 172.21.15.78:0/2055025000 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f87a5f53f0 0x55f87a5f7810 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9fd0c2ee066ff8 server_cookie=fc7fd3406610e748 in_seq=0 out_seq=0 2024-08-18T23:48:01.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.888+0000 7f22edffb640 10 client.16140 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:01.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.889+0000 7f22edffb640 1 -- 172.21.15.78:0/2055025000 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f87a4dc270 con 0x55f87a5f53f0 2024-08-18T23:48:01.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.889+0000 7f22edffb640 10 client.16140 handle_command_reply: tid=0 2024-08-18T23:48:01.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.889+0000 7f230eb34740 1 client.16140 shutdown 2024-08-18T23:48:01.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.889+0000 7f230eb34740 1 -- 172.21.15.78:0/2055025000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a57a3e0 msgr2=0x55f87a574500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:01.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.889+0000 7f230eb34740 1 --2- 172.21.15.78:0/2055025000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a57a3e0 0x55f87a574500 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2300005cb0 tx=0x7f230000c4d0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.889+0000 7f230eb34740 1 -- 172.21.15.78:0/2055025000 shutdown_connections 2024-08-18T23:48:01.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.889+0000 7f230eb34740 1 --2- 172.21.15.78:0/2055025000 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f87a5f53f0 0x55f87a5f7810 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.889+0000 7f230eb34740 1 --2- 172.21.15.78:0/2055025000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f87a57a3e0 0x55f87a574500 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.889+0000 7f230eb34740 1 --2- 172.21.15.78:0/2055025000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f87a56fb20 0x55f87a573fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.889+0000 7f230eb34740 1 --2- 172.21.15.78:0/2055025000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f87a56f200 0x55f87a582860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.889+0000 7f230eb34740 1 -- 172.21.15.78:0/2055025000 >> 172.21.15.78:0/2055025000 conn(0x55f87a56b790 msgr2=0x55f87a56a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:01.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.890+0000 7f22edffb640 0 client.16140 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:01.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.890+0000 7f230eb34740 1 -- 172.21.15.78:0/2055025000 shutdown_connections 2024-08-18T23:48:01.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.890+0000 7f230eb34740 1 -- 172.21.15.78:0/2055025000 wait complete. 2024-08-18T23:48:01.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.890+0000 7f230eb34740 20 client.16140 trim_cache size 0 max 16384 2024-08-18T23:48:01.901 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:48:01.902 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-08-18T23:48:01.902 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "e6717d1b-b750-41b6-b544-e0812dd33e7c", 2024-08-18T23:48:01.902 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-08-18T23:48:01.902 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:48:01.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.890+0000 7f230c736640 1 -- 172.21.15.78:0/3812567751 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f22f00a2b80 msgr2=0x7f22f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:01.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.890+0000 7f230c736640 1 --2- 172.21.15.78:0/3812567751 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f22f00a2b80 0x7f22f00a5040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0b9aa0 tx=0x7f22fc0c0040 comp rx=0 tx=0).stop 2024-08-18T23:48:01.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.890+0000 7f230c736640 1 -- 172.21.15.78:0/3812567751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23041078b0 msgr2=0x7f2304074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:01.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.890+0000 7f230c736640 1 --2- 172.21.15.78:0/3812567751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23041078b0 0x7f2304074ca0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f22fc005cc0 tx=0x7f22fc003970 comp rx=0 tx=0).stop 2024-08-18T23:48:01.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.890+0000 7f230c736640 1 -- 172.21.15.78:0/3812567751 shutdown_connections 2024-08-18T23:48:01.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.890+0000 7f230c736640 1 --2- 172.21.15.78:0/3812567751 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f22f00a2b80 0x7f22f00a5040 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.890+0000 7f230c736640 1 --2- 172.21.15.78:0/3812567751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23041078b0 0x7f2304074ca0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.890+0000 7f230c736640 1 --2- 172.21.15.78:0/3812567751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2304102250 0x7f2304074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.890+0000 7f230c736640 1 --2- 172.21.15.78:0/3812567751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23041018b0 0x7f2304074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:01.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.890+0000 7f230c736640 1 -- 172.21.15.78:0/3812567751 >> 172.21.15.78:0/3812567751 conn(0x7f23040fd5f0 msgr2=0x7f23040fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:01.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.890+0000 7f230c736640 1 -- 172.21.15.78:0/3812567751 shutdown_connections 2024-08-18T23:48:01.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:01.890+0000 7f230c736640 1 -- 172.21.15.78:0/3812567751 wait complete. 2024-08-18T23:48:03.171 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:48:03.441 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:48:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:03 smithi078 ceph-mon[18440]: scrub queued for path: / 2024-08-18T23:48:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:03 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:03 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:48:03.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:03 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:48:03.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:03 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:03 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:48:03.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:03 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:03 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:48:03.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:03 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:48:03.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:03 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:03 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:03 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:48:03.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:03 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:03 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:48:03.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:03 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:03 smithi078 ceph-mon[18440]: pgmap v462: 129 pgs: 129 active+clean; 470 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 913 KiB/s wr, 250 op/s 2024-08-18T23:48:03.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:03 smithi123 ceph-mon[21808]: scrub queued for path: / 2024-08-18T23:48:03.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:03 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:03 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:48:03.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:03 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:48:03.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:03 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:03 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:48:03.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:03 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:03 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:48:03.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:03 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:48:03.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:03 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:03 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:03 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:48:03.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:03 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:03 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:48:03.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:03 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:03 smithi123 ceph-mon[21808]: pgmap v462: 129 pgs: 129 active+clean; 470 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 913 KiB/s wr, 250 op/s 2024-08-18T23:48:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:03 smithi161 ceph-mon[22027]: scrub queued for path: / 2024-08-18T23:48:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:03 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:03 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:48:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:03 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:48:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:03 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:03 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:48:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:03 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:03 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:48:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:03 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:48:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:03 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:03 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:03 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:48:03.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:03 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:03 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:48:03.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:03 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:03.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:03 smithi161 ceph-mon[22027]: pgmap v462: 129 pgs: 129 active+clean; 470 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 913 KiB/s wr, 250 op/s 2024-08-18T23:48:03.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.921+0000 7f62d91e8640 1 -- 172.21.15.78:0/1824822220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d41078b0 msgr2=0x7f62d4107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:03.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.921+0000 7f62d91e8640 1 --2- 172.21.15.78:0/1824822220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d41078b0 0x7f62d4107c90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009cd0 tx=0x7f62c402a8e0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.921+0000 7f62d91e8640 1 -- 172.21.15.78:0/1824822220 shutdown_connections 2024-08-18T23:48:03.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.921+0000 7f62d91e8640 1 --2- 172.21.15.78:0/1824822220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d4102250 0x7f62d410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.921+0000 7f62d91e8640 1 --2- 172.21.15.78:0/1824822220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62d41018b0 0x7f62d4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.921+0000 7f62d91e8640 1 --2- 172.21.15.78:0/1824822220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d41078b0 0x7f62d4107c90 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.921+0000 7f62d91e8640 1 -- 172.21.15.78:0/1824822220 >> 172.21.15.78:0/1824822220 conn(0x7f62d40fd5f0 msgr2=0x7f62d40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:03.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.921+0000 7f62d91e8640 1 -- 172.21.15.78:0/1824822220 shutdown_connections 2024-08-18T23:48:03.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.921+0000 7f62d91e8640 1 -- 172.21.15.78:0/1824822220 wait complete. 2024-08-18T23:48:03.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.922+0000 7f62d91e8640 1 Processor -- start 2024-08-18T23:48:03.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.922+0000 7f62d91e8640 1 -- start start 2024-08-18T23:48:03.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62d91e8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d41018b0 0x7f62d4198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:03.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62d91e8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62d4102250 0x7f62d4198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:03.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62d91e8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d41078b0 0x7f62d419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:03.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62d91e8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62d4111cb0 con 0x7f62d41018b0 2024-08-18T23:48:03.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62d91e8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f62d4111b30 con 0x7f62d41078b0 2024-08-18T23:48:03.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62d91e8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f62d4111e30 con 0x7f62d4102250 2024-08-18T23:48:03.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62d2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d41018b0 0x7f62d4198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:03.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62d2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d41018b0 0x7f62d4198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46738/0 (socket says 172.21.15.78:46738) 2024-08-18T23:48:03.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62d2d9d640 1 -- 172.21.15.78:0/4253188359 learned_addr learned my addr 172.21.15.78:0/4253188359 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:03.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62d259c640 1 --2- 172.21.15.78:0/4253188359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62d4102250 0x7f62d4198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:03.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62d359e640 1 --2- 172.21.15.78:0/4253188359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d41078b0 0x7f62d419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:03.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62d2d9d640 1 -- 172.21.15.78:0/4253188359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62d4102250 msgr2=0x7f62d4198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:03.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62d2d9d640 1 --2- 172.21.15.78:0/4253188359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62d4102250 0x7f62d4198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62d2d9d640 1 -- 172.21.15.78:0/4253188359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d41078b0 msgr2=0x7f62d419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:03.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62d2d9d640 1 --2- 172.21.15.78:0/4253188359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d41078b0 0x7f62d419cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62d2d9d640 1 -- 172.21.15.78:0/4253188359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d419d1e0 con 0x7f62d41018b0 2024-08-18T23:48:03.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62d359e640 1 --2- 172.21.15.78:0/4253188359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d41078b0 0x7f62d419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:48:03.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62d2d9d640 1 --2- 172.21.15.78:0/4253188359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d41018b0 0x7f62d4198230 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f62c402ae00 tx=0x7f62c4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:03.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62bbfff640 1 -- 172.21.15.78:0/4253188359 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4004040 con 0x7f62d41018b0 2024-08-18T23:48:03.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62bbfff640 1 -- 172.21.15.78:0/4253188359 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62c40046c0 con 0x7f62d41018b0 2024-08-18T23:48:03.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62d91e8640 1 -- 172.21.15.78:0/4253188359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d419d470 con 0x7f62d41018b0 2024-08-18T23:48:03.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.923+0000 7f62bbfff640 1 -- 172.21.15.78:0/4253188359 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4046480 con 0x7f62d41018b0 2024-08-18T23:48:03.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.924+0000 7f62d91e8640 1 -- 172.21.15.78:0/4253188359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d41a4db0 con 0x7f62d41018b0 2024-08-18T23:48:03.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.926+0000 7f62bbfff640 1 -- 172.21.15.78:0/4253188359 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f62c40305a0 con 0x7f62d41018b0 2024-08-18T23:48:03.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.926+0000 7f62db5e6740 1 Processor -- start 2024-08-18T23:48:03.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.926+0000 7f62db5e6740 1 -- start start 2024-08-18T23:48:03.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62db5e6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561506069210 0x5615060695f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:03.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62db5e6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561506069b30 0x561506073eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:03.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62db5e6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615060743f0 0x5615060767e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:03.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62db5e6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561505fce980 con 0x5615060743f0 2024-08-18T23:48:03.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62db5e6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561505f66a70 con 0x561506069b30 2024-08-18T23:48:03.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62db5e6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561505fca940 con 0x561506069210 2024-08-18T23:48:03.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62d359e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561506069210 0x5615060695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:03.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62d259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615060743f0 0x5615060767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:03.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62d359e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561506069210 0x5615060695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:44902/0 (socket says 172.21.15.78:44902) 2024-08-18T23:48:03.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62d359e640 1 -- 172.21.15.78:0/3123620452 learned_addr learned my addr 172.21.15.78:0/3123620452 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:03.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62bbfff640 1 --2- 172.21.15.78:0/4253188359 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f62bc0a2b80 0x7f62bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:03.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62d2d9d640 1 --2- 172.21.15.78:0/3123620452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561506069b30 0x561506073eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:03.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62d259c640 1 -- 172.21.15.78:0/3123620452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561506069210 msgr2=0x5615060695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:03.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62d259c640 1 --2- 172.21.15.78:0/3123620452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561506069210 0x5615060695f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62d259c640 1 -- 172.21.15.78:0/3123620452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561506069b30 msgr2=0x561506073eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:03.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62d259c640 1 --2- 172.21.15.78:0/3123620452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561506069b30 0x561506073eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62d259c640 1 -- 172.21.15.78:0/3123620452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561505fc96e0 con 0x5615060743f0 2024-08-18T23:48:03.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62d359e640 1 --2- 172.21.15.78:0/3123620452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561506069210 0x5615060695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:48:03.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62d359e640 1 --2- 172.21.15.78:0/4253188359 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f62bc0a2b80 0x7f62bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:03.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62bbfff640 1 -- 172.21.15.78:0/4253188359 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f62c402e030 con 0x7f62d41018b0 2024-08-18T23:48:03.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62d259c640 1 --2- 172.21.15.78:0/3123620452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615060743f0 0x5615060767e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f62c800bbb0 tx=0x7f62c800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c291e6b867403141 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:03.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.927+0000 7f62b97fa640 1 -- 172.21.15.78:0/3123620452 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c8015070 con 0x5615060743f0 2024-08-18T23:48:03.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.928+0000 7f62b97fa640 1 -- 172.21.15.78:0/3123620452 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62c8005500 con 0x5615060743f0 2024-08-18T23:48:03.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.928+0000 7f62b97fa640 1 -- 172.21.15.78:0/3123620452 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c80116a0 con 0x5615060743f0 2024-08-18T23:48:03.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.928+0000 7f62db5e6740 1 -- 172.21.15.78:0/3123620452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615060743f0 msgr2=0x5615060767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:03.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.928+0000 7f62d359e640 1 --2- 172.21.15.78:0/4253188359 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f62bc0a2b80 0x7f62bc0a5040 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00bdf0 tx=0x7f62cc0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:03.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.928+0000 7f62db5e6740 1 --2- 172.21.15.78:0/3123620452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615060743f0 0x5615060767e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f62c800bbb0 tx=0x7f62c800f730 comp rx=0 tx=0).stop 2024-08-18T23:48:03.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.928+0000 7f62db5e6740 1 -- 172.21.15.78:0/3123620452 shutdown_connections 2024-08-18T23:48:03.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.928+0000 7f62db5e6740 1 --2- 172.21.15.78:0/3123620452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615060743f0 0x5615060767e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.928+0000 7f62db5e6740 1 --2- 172.21.15.78:0/3123620452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561506069b30 0x561506073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.928+0000 7f62db5e6740 1 --2- 172.21.15.78:0/3123620452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561506069210 0x5615060695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.928+0000 7f62db5e6740 1 -- 172.21.15.78:0/3123620452 >> 172.21.15.78:0/3123620452 conn(0x5615060657a0 msgr2=0x561506065b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:03.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.928+0000 7f62db5e6740 1 -- 172.21.15.78:0/3123620452 shutdown_connections 2024-08-18T23:48:03.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.928+0000 7f62db5e6740 1 -- 172.21.15.78:0/3123620452 wait complete. 2024-08-18T23:48:03.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.929+0000 7f62db5e6740 1 -- start start 2024-08-18T23:48:03.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.929+0000 7f62db5e6740 1 Processor -- start 2024-08-18T23:48:03.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.929+0000 7f62db5e6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561506069210 0x5615060712c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:03.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.929+0000 7f62db5e6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561506069b30 0x561506071800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:03.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.929+0000 7f62db5e6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615060743f0 0x561506071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:03.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.929+0000 7f62db5e6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561506076fc0 con 0x561506069b30 2024-08-18T23:48:03.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.929+0000 7f62db5e6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561505f89aa0 con 0x561506069210 2024-08-18T23:48:03.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.929+0000 7f62db5e6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561506077140 con 0x5615060743f0 2024-08-18T23:48:03.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.929+0000 7f62db5e6740 10 client.0 _ll_register_callbacks cb 0x561505f78e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:48:03.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.929+0000 7f62d259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561506069b30 0x561506071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:03.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.929+0000 7f62d259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561506069b30 0x561506071800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46752/0 (socket says 172.21.15.78:46752) 2024-08-18T23:48:03.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.929+0000 7f62d259c640 1 -- 172.21.15.78:0/2222103059 learned_addr learned my addr 172.21.15.78:0/2222103059 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:03.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.929+0000 7f62d359e640 1 --2- 172.21.15.78:0/2222103059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615060743f0 0x561506071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:03.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.929+0000 7f62d2d9d640 1 --2- 172.21.15.78:0/2222103059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561506069210 0x5615060712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:03.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.929+0000 7f62d259c640 1 -- 172.21.15.78:0/2222103059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615060743f0 msgr2=0x561506071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:03.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.929+0000 7f62d259c640 1 --2- 172.21.15.78:0/2222103059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615060743f0 0x561506071d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.929+0000 7f62d259c640 1 -- 172.21.15.78:0/2222103059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561506069210 msgr2=0x5615060712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:03.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.929+0000 7f62d259c640 1 --2- 172.21.15.78:0/2222103059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561506069210 0x5615060712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.929+0000 7f62d259c640 1 -- 172.21.15.78:0/2222103059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561505f6d650 con 0x561506069b30 2024-08-18T23:48:03.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.930+0000 7f62d259c640 1 --2- 172.21.15.78:0/2222103059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561506069b30 0x561506071800 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f62c80049b0 tx=0x7f62c8004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf818b8325d97024 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:03.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.930+0000 7f62b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:48:03.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.930+0000 7f62b6ff5640 1 -- 172.21.15.78:0/2222103059 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c8028060 con 0x561506069b30 2024-08-18T23:48:03.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.930+0000 7f62b6ff5640 1 -- 172.21.15.78:0/2222103059 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f62c8004e10 con 0x561506069b30 2024-08-18T23:48:03.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.930+0000 7f62db5e6740 1 -- 172.21.15.78:0/2222103059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62c8005500 con 0x561506069b30 2024-08-18T23:48:03.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.930+0000 7f62b6ff5640 1 -- 172.21.15.78:0/2222103059 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f62c8015070 con 0x561506069b30 2024-08-18T23:48:03.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.930+0000 7f62db5e6740 10 client.16152 fetch_fsmap learned FSMap version 26 2024-08-18T23:48:03.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.930+0000 7f62db5e6740 1 -- 172.21.15.78:0/2222103059 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561505d403c0 con 0x561506069b30 2024-08-18T23:48:03.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.930+0000 7f62b6ff5640 1 -- 172.21.15.78:0/2222103059 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62c8005500 con 0x561506069b30 2024-08-18T23:48:03.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.930+0000 7f62b6ff5640 1 -- 172.21.15.78:0/2222103059 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f62c8011b70 con 0x561506069b30 2024-08-18T23:48:03.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.930+0000 7f62db5e6740 10 client.16152 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:48:03.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.930+0000 7f62db5e6740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:48:03.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.930+0000 7f62db5e6740 20 client.16152 populate_metadata read hostname 'smithi078' 2024-08-18T23:48:03.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.930+0000 7f62db5e6740 1 --2- 172.21.15.78:0/2222103059 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x561506083f10 0x561506086300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:03.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.930+0000 7f62db5e6740 4 client.16152 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:48:03.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.930+0000 7f62db5e6740 1 -- 172.21.15.78:0/2222103059 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561505fdbef0 con 0x561506083f10 2024-08-18T23:48:03.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.931+0000 7f62d2d9d640 1 --2- 172.21.15.78:0/2222103059 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x561506083f10 0x561506086300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:03.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.931+0000 7f62b6ff5640 10 client.16152 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:03.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.931+0000 7f62d2d9d640 1 --2- 172.21.15.78:0/2222103059 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x561506083f10 0x561506086300 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8d2e856d9df7f84 server_cookie=3021508275d4c1e4 in_seq=0 out_seq=0 2024-08-18T23:48:03.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.932+0000 7f62b6ff5640 1 -- 172.21.15.78:0/2222103059 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561505fdbef0 con 0x561506083f10 2024-08-18T23:48:03.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.932+0000 7f62b6ff5640 10 client.16152 handle_command_reply: tid=0 2024-08-18T23:48:03.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.932+0000 7f62db5e6740 1 client.16152 shutdown 2024-08-18T23:48:03.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.932+0000 7f62db5e6740 1 -- 172.21.15.78:0/2222103059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561506069b30 msgr2=0x561506071800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:03.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.932+0000 7f62db5e6740 1 --2- 172.21.15.78:0/2222103059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561506069b30 0x561506071800 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f62c80049b0 tx=0x7f62c8004a30 comp rx=0 tx=0).stop 2024-08-18T23:48:03.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.932+0000 7f62db5e6740 1 -- 172.21.15.78:0/2222103059 shutdown_connections 2024-08-18T23:48:03.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.932+0000 7f62db5e6740 1 --2- 172.21.15.78:0/2222103059 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x561506083f10 0x561506086300 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.932+0000 7f62db5e6740 1 --2- 172.21.15.78:0/2222103059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615060743f0 0x561506071d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.932+0000 7f62db5e6740 1 --2- 172.21.15.78:0/2222103059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561506069b30 0x561506071800 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.932+0000 7f62b6ff5640 0 client.16152 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:03.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.932+0000 7f62db5e6740 1 --2- 172.21.15.78:0/2222103059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561506069210 0x5615060712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.932+0000 7f62db5e6740 1 -- 172.21.15.78:0/2222103059 >> 172.21.15.78:0/2222103059 conn(0x5615060657a0 msgr2=0x56150606b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:03.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.932+0000 7f62db5e6740 1 -- 172.21.15.78:0/2222103059 shutdown_connections 2024-08-18T23:48:03.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.932+0000 7f62db5e6740 1 -- 172.21.15.78:0/2222103059 wait complete. 2024-08-18T23:48:03.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.932+0000 7f62db5e6740 20 client.16152 trim_cache size 0 max 16384 2024-08-18T23:48:03.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.949+0000 7f62db5e6740 1 Processor -- start 2024-08-18T23:48:03.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.949+0000 7f62db5e6740 1 -- start start 2024-08-18T23:48:03.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.949+0000 7f62db5e6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561506069210 0x56150608b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:03.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.949+0000 7f62db5e6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561506069b30 0x56150608be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:03.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.949+0000 7f62db5e6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615060743f0 0x56150606ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:03.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.949+0000 7f62db5e6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41b1260 con 0x561506069b30 2024-08-18T23:48:03.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.949+0000 7f62db5e6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41aeb30 con 0x561506069210 2024-08-18T23:48:03.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.949+0000 7f62db5e6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41ac980 con 0x5615060743f0 2024-08-18T23:48:03.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.949+0000 7f62d259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561506069b30 0x56150608be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:03.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.949+0000 7f62d259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561506069b30 0x56150608be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46760/0 (socket says 172.21.15.78:46760) 2024-08-18T23:48:03.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.949+0000 7f62d259c640 1 -- 172.21.15.78:0/1196205677 learned_addr learned my addr 172.21.15.78:0/1196205677 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:03.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.949+0000 7f62d259c640 1 -- 172.21.15.78:0/1196205677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615060743f0 msgr2=0x56150606ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:03.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.949+0000 7f62d259c640 1 --2- 172.21.15.78:0/1196205677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615060743f0 0x56150606ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.949+0000 7f62d259c640 1 -- 172.21.15.78:0/1196205677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561506069210 msgr2=0x56150608b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:03.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.949+0000 7f62d259c640 1 --2- 172.21.15.78:0/1196205677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561506069210 0x56150608b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.949+0000 7f62d259c640 1 -- 172.21.15.78:0/1196205677 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56150606ce80 con 0x561506069b30 2024-08-18T23:48:03.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.950+0000 7f62d259c640 1 --2- 172.21.15.78:0/1196205677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561506069b30 0x56150608be70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f62c8011f70 tx=0x7f62c8005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a76a62a1d7e122 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:03.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.950+0000 7f62b8ff9640 1 -- 172.21.15.78:0/1196205677 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c8038070 con 0x561506069b30 2024-08-18T23:48:03.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.950+0000 7f62b8ff9640 1 -- 172.21.15.78:0/1196205677 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62c801d030 con 0x561506069b30 2024-08-18T23:48:03.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.950+0000 7f62b8ff9640 1 -- 172.21.15.78:0/1196205677 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c80116a0 con 0x561506069b30 2024-08-18T23:48:03.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.950+0000 7f62db5e6740 1 -- 172.21.15.78:0/1196205677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561506069b30 msgr2=0x56150608be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:03.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.950+0000 7f62db5e6740 1 --2- 172.21.15.78:0/1196205677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561506069b30 0x56150608be70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f62c8011f70 tx=0x7f62c8005970 comp rx=0 tx=0).stop 2024-08-18T23:48:03.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.950+0000 7f62db5e6740 1 -- 172.21.15.78:0/1196205677 shutdown_connections 2024-08-18T23:48:03.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.950+0000 7f62db5e6740 1 --2- 172.21.15.78:0/1196205677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5615060743f0 0x56150606ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.950+0000 7f62db5e6740 1 --2- 172.21.15.78:0/1196205677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561506069b30 0x56150608be70 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.950+0000 7f62db5e6740 1 --2- 172.21.15.78:0/1196205677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561506069210 0x56150608b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.950+0000 7f62db5e6740 1 -- 172.21.15.78:0/1196205677 >> 172.21.15.78:0/1196205677 conn(0x5615060657a0 msgr2=0x5615060703b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:03.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.950+0000 7f62db5e6740 1 -- 172.21.15.78:0/1196205677 shutdown_connections 2024-08-18T23:48:03.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.950+0000 7f62db5e6740 1 -- 172.21.15.78:0/1196205677 wait complete. 2024-08-18T23:48:03.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.951+0000 7f62db5e6740 1 -- start start 2024-08-18T23:48:03.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.951+0000 7f62db5e6740 1 Processor -- start 2024-08-18T23:48:03.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.951+0000 7f62db5e6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561506069210 0x56150607c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:03.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.951+0000 7f62db5e6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561506069b30 0x561506077b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:03.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.951+0000 7f62db5e6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615060743f0 0x561506078040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:03.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.951+0000 7f62db5e6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41acff0 con 0x5615060743f0 2024-08-18T23:48:03.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.951+0000 7f62db5e6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41ae460 con 0x561506069210 2024-08-18T23:48:03.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.951+0000 7f62db5e6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41ad6c0 con 0x561506069b30 2024-08-18T23:48:03.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.951+0000 7f62db5e6740 10 client.0 _ll_register_callbacks cb 0x7f62cc005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:48:03.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.951+0000 7f62d359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615060743f0 0x561506078040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:03.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.951+0000 7f62d359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615060743f0 0x561506078040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46766/0 (socket says 172.21.15.78:46766) 2024-08-18T23:48:03.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.951+0000 7f62d359e640 1 -- 172.21.15.78:0/2583608034 learned_addr learned my addr 172.21.15.78:0/2583608034 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:03.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.951+0000 7f62d2d9d640 1 --2- 172.21.15.78:0/2583608034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561506069210 0x56150607c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:03.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.951+0000 7f62d259c640 1 --2- 172.21.15.78:0/2583608034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561506069b30 0x561506077b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:03.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.951+0000 7f62d359e640 1 -- 172.21.15.78:0/2583608034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561506069b30 msgr2=0x561506077b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:03.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.951+0000 7f62d359e640 1 --2- 172.21.15.78:0/2583608034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561506069b30 0x561506077b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.951+0000 7f62d359e640 1 -- 172.21.15.78:0/2583608034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561506069210 msgr2=0x56150607c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:03.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.951+0000 7f62d359e640 1 --2- 172.21.15.78:0/2583608034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561506069210 0x56150607c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.951+0000 7f62d359e640 1 -- 172.21.15.78:0/2583608034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561506078580 con 0x5615060743f0 2024-08-18T23:48:03.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.952+0000 7f62d359e640 1 --2- 172.21.15.78:0/2583608034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615060743f0 0x561506078040 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f62cc013e60 tx=0x7f62cc00b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15e44dd627fb198 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:03.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.952+0000 7f62b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:48:03.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.952+0000 7f62b6ff5640 1 -- 172.21.15.78:0/2583608034 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc02b060 con 0x5615060743f0 2024-08-18T23:48:03.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.952+0000 7f62b6ff5640 1 -- 172.21.15.78:0/2583608034 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f62cc01a420 con 0x5615060743f0 2024-08-18T23:48:03.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.952+0000 7f62db5e6740 1 -- 172.21.15.78:0/2583608034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62c801d030 con 0x5615060743f0 2024-08-18T23:48:03.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.952+0000 7f62b6ff5640 1 -- 172.21.15.78:0/2583608034 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f62cc032050 con 0x5615060743f0 2024-08-18T23:48:03.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.952+0000 7f62db5e6740 10 client.16158 fetch_fsmap learned FSMap version 26 2024-08-18T23:48:03.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.952+0000 7f62db5e6740 1 -- 172.21.15.78:0/2583608034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5615060ee660 con 0x5615060743f0 2024-08-18T23:48:03.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.952+0000 7f62b6ff5640 1 -- 172.21.15.78:0/2583608034 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62c801d030 con 0x5615060743f0 2024-08-18T23:48:03.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.952+0000 7f62b6ff5640 1 -- 172.21.15.78:0/2583608034 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f62cc0196e0 con 0x5615060743f0 2024-08-18T23:48:03.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.952+0000 7f62db5e6740 10 client.16158 fetch_fsmap finished waiting for FSMap version 26 2024-08-18T23:48:03.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.952+0000 7f62db5e6740 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:48:03.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.952+0000 7f62db5e6740 20 client.16158 populate_metadata read hostname 'smithi078' 2024-08-18T23:48:03.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.952+0000 7f62db5e6740 1 --2- 172.21.15.78:0/2583608034 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5615060eee40 0x5615060f1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:03.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.952+0000 7f62db5e6740 4 client.16158 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:48:03.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.952+0000 7f62db5e6740 1 -- 172.21.15.78:0/2583608034 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561505fd6400 con 0x5615060eee40 2024-08-18T23:48:03.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.953+0000 7f62d2d9d640 1 --2- 172.21.15.78:0/2583608034 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5615060eee40 0x5615060f1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:03.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.953+0000 7f62b6ff5640 10 client.16158 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:03.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.953+0000 7f62d2d9d640 1 --2- 172.21.15.78:0/2583608034 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5615060eee40 0x5615060f1260 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c54ac341498a0ea6 server_cookie=f0b4267243c37da3 in_seq=0 out_seq=0 2024-08-18T23:48:03.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.953+0000 7f62b6ff5640 1 -- 172.21.15.78:0/2583608034 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561505fd6400 con 0x5615060eee40 2024-08-18T23:48:03.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.953+0000 7f62b6ff5640 10 client.16158 handle_command_reply: tid=0 2024-08-18T23:48:03.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.953+0000 7f62db5e6740 1 client.16158 shutdown 2024-08-18T23:48:03.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.953+0000 7f62db5e6740 1 -- 172.21.15.78:0/2583608034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615060743f0 msgr2=0x561506078040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:03.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.953+0000 7f62db5e6740 1 --2- 172.21.15.78:0/2583608034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615060743f0 0x561506078040 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f62cc013e60 tx=0x7f62cc00b370 comp rx=0 tx=0).stop 2024-08-18T23:48:03.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.953+0000 7f62db5e6740 1 -- 172.21.15.78:0/2583608034 shutdown_connections 2024-08-18T23:48:03.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.953+0000 7f62db5e6740 1 --2- 172.21.15.78:0/2583608034 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5615060eee40 0x5615060f1260 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.953+0000 7f62db5e6740 1 --2- 172.21.15.78:0/2583608034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615060743f0 0x561506078040 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.953+0000 7f62b6ff5640 0 client.16158 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:03.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.953+0000 7f62db5e6740 1 --2- 172.21.15.78:0/2583608034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561506069b30 0x561506077b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.953+0000 7f62db5e6740 1 --2- 172.21.15.78:0/2583608034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561506069210 0x56150607c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.953+0000 7f62db5e6740 1 -- 172.21.15.78:0/2583608034 >> 172.21.15.78:0/2583608034 conn(0x5615060657a0 msgr2=0x56150606ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:03.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.953+0000 7f62db5e6740 1 -- 172.21.15.78:0/2583608034 shutdown_connections 2024-08-18T23:48:03.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.954+0000 7f62db5e6740 1 -- 172.21.15.78:0/2583608034 wait complete. 2024-08-18T23:48:03.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.954+0000 7f62db5e6740 20 client.16158 trim_cache size 0 max 16384 2024-08-18T23:48:03.975 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:48:03.975 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-18T23:48:03.976 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-08-18T23:48:03.976 INFO:teuthology.orchestra.run.smithi078.stdout: "e6717d1b-b750-41b6-b544-e0812dd33e7c": { 2024-08-18T23:48:03.976 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-08-18T23:48:03.976 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "e6717d1b-b750-41b6-b544-e0812dd33e7c", 2024-08-18T23:48:03.976 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-08-18T23:48:03.976 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:48:03.976 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:48:03.976 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:48:03.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.954+0000 7f62d91e8640 1 -- 172.21.15.78:0/4253188359 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f62bc0a2b80 msgr2=0x7f62bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:03.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.954+0000 7f62d91e8640 1 --2- 172.21.15.78:0/4253188359 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f62bc0a2b80 0x7f62bc0a5040 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00bdf0 tx=0x7f62cc0046c0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.954+0000 7f62d91e8640 1 -- 172.21.15.78:0/4253188359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d41018b0 msgr2=0x7f62d4198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:03.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.954+0000 7f62d91e8640 1 --2- 172.21.15.78:0/4253188359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d41018b0 0x7f62d4198230 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f62c402ae00 tx=0x7f62c4003970 comp rx=0 tx=0).stop 2024-08-18T23:48:03.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.954+0000 7f62d91e8640 1 -- 172.21.15.78:0/4253188359 shutdown_connections 2024-08-18T23:48:03.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.954+0000 7f62d91e8640 1 --2- 172.21.15.78:0/4253188359 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f62bc0a2b80 0x7f62bc0a5040 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.954+0000 7f62d91e8640 1 --2- 172.21.15.78:0/4253188359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62d41078b0 0x7f62d419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.954+0000 7f62d91e8640 1 --2- 172.21.15.78:0/4253188359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62d4102250 0x7f62d4198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.954+0000 7f62d91e8640 1 --2- 172.21.15.78:0/4253188359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62d41018b0 0x7f62d4198230 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:03.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.954+0000 7f62d91e8640 1 -- 172.21.15.78:0/4253188359 >> 172.21.15.78:0/4253188359 conn(0x7f62d40fd5f0 msgr2=0x7f62d40fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:03.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.954+0000 7f62d91e8640 1 -- 172.21.15.78:0/4253188359 shutdown_connections 2024-08-18T23:48:03.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:03.954+0000 7f62d91e8640 1 -- 172.21.15.78:0/4253188359 wait complete. 2024-08-18T23:48:04.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:04 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:04.594 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:04 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:04 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:04.998 INFO:tasks.cephfs.filesystem:scrub status for tag:e6717d1b-b750-41b6-b544-e0812dd33e7c - {'path': '/', 'tag': 'e6717d1b-b750-41b6-b544-e0812dd33e7c', 'options': 'recursive,force'} 2024-08-18T23:48:05.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:05 smithi123 ceph-mon[21808]: pgmap v463: 129 pgs: 129 active+clean; 470 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 295 KiB/s wr, 223 op/s 2024-08-18T23:48:05.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:05 smithi161 ceph-mon[22027]: pgmap v463: 129 pgs: 129 active+clean; 470 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 295 KiB/s wr, 223 op/s 2024-08-18T23:48:05.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:05 smithi078 ceph-mon[18440]: pgmap v463: 129 pgs: 129 active+clean; 470 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 295 KiB/s wr, 223 op/s 2024-08-18T23:48:06.982 INFO:tasks.workunit.client.0.smithi078.stdout: 79 30860 0 21936 0 17934 0 2024-08-18T23:48:07.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:07 smithi123 ceph-mon[21808]: pgmap v464: 129 pgs: 129 active+clean; 470 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 251 KiB/s wr, 219 op/s 2024-08-18T23:48:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:07 smithi161 ceph-mon[22027]: pgmap v464: 129 pgs: 129 active+clean; 470 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 251 KiB/s wr, 219 op/s 2024-08-18T23:48:07.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:07 smithi078 ceph-mon[18440]: pgmap v464: 129 pgs: 129 active+clean; 470 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 251 KiB/s wr, 219 op/s 2024-08-18T23:48:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:09 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:48:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:09 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:48:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:09 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:48:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:09 smithi161 ceph-mon[22027]: pgmap v465: 129 pgs: 129 active+clean; 471 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 287 KiB/s wr, 178 op/s 2024-08-18T23:48:09.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:09 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:48:09.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:09 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:48:09.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:09 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:48:09.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:09 smithi078 ceph-mon[18440]: pgmap v465: 129 pgs: 129 active+clean; 471 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 287 KiB/s wr, 178 op/s 2024-08-18T23:48:09.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:09 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:48:09.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:09 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:48:09.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:09 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:48:09.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:09 smithi123 ceph-mon[21808]: pgmap v465: 129 pgs: 129 active+clean; 471 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 287 KiB/s wr, 178 op/s 2024-08-18T23:48:11.472 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:48:11.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:11 smithi161 ceph-mon[22027]: pgmap v466: 129 pgs: 129 active+clean; 472 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 321 KiB/s wr, 127 op/s 2024-08-18T23:48:11.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:11 smithi078 ceph-mon[18440]: pgmap v466: 129 pgs: 129 active+clean; 472 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 321 KiB/s wr, 127 op/s 2024-08-18T23:48:11.739 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:48:11.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:11 smithi123 ceph-mon[21808]: pgmap v466: 129 pgs: 129 active+clean; 472 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 321 KiB/s wr, 127 op/s 2024-08-18T23:48:12.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.250+0000 7fe2a24c7640 1 -- 172.21.15.78:0/341239146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe29c0ffd00 msgr2=0x7fe29c100160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:12.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.250+0000 7fe2a24c7640 1 --2- 172.21.15.78:0/341239146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe29c0ffd00 0x7fe29c100160 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe290009900 tx=0x7fe29002a410 comp rx=0 tx=0).stop 2024-08-18T23:48:12.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.250+0000 7fe2a24c7640 1 -- 172.21.15.78:0/341239146 shutdown_connections 2024-08-18T23:48:12.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.250+0000 7fe2a24c7640 1 --2- 172.21.15.78:0/341239146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe29c1006a0 0x7fe29c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:12.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.250+0000 7fe2a24c7640 1 --2- 172.21.15.78:0/341239146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe29c0ffd00 0x7fe29c100160 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:12.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.250+0000 7fe2a24c7640 1 --2- 172.21.15.78:0/341239146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe29c1068e0 0x7fe29c106cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:12.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.250+0000 7fe2a24c7640 1 -- 172.21.15.78:0/341239146 >> 172.21.15.78:0/341239146 conn(0x7fe29c0fbc10 msgr2=0x7fe29c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:12.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.250+0000 7fe2a24c7640 1 -- 172.21.15.78:0/341239146 shutdown_connections 2024-08-18T23:48:12.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.250+0000 7fe2a24c7640 1 -- 172.21.15.78:0/341239146 wait complete. 2024-08-18T23:48:12.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.251+0000 7fe2a24c7640 1 Processor -- start 2024-08-18T23:48:12.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.251+0000 7fe2a24c7640 1 -- start start 2024-08-18T23:48:12.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.251+0000 7fe2a24c7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe29c0ffd00 0x7fe29c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:12.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.251+0000 7fe2a24c7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe29c1006a0 0x7fe29c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:12.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.251+0000 7fe2a24c7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe29c1068e0 0x7fe29c19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:12.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.251+0000 7fe2a24c7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c111df0 con 0x7fe29c1068e0 2024-08-18T23:48:12.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.251+0000 7fe2a24c7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c111c70 con 0x7fe29c1006a0 2024-08-18T23:48:12.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.251+0000 7fe2a24c7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c111f70 con 0x7fe29c0ffd00 2024-08-18T23:48:12.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.251+0000 7fe2a0a64640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe29c1068e0 0x7fe29c19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:12.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.251+0000 7fe2a0a64640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe29c1068e0 0x7fe29c19cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51238/0 (socket says 172.21.15.78:51238) 2024-08-18T23:48:12.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.251+0000 7fe2a0a64640 1 -- 172.21.15.78:0/698482053 learned_addr learned my addr 172.21.15.78:0/698482053 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:12.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.251+0000 7fe29b7fe640 1 --2- 172.21.15.78:0/698482053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe29c1006a0 0x7fe29c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:12.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.251+0000 7fe29bfff640 1 --2- 172.21.15.78:0/698482053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe29c0ffd00 0x7fe29c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:12.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.251+0000 7fe2a0a64640 1 -- 172.21.15.78:0/698482053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe29c0ffd00 msgr2=0x7fe29c1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:12.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.251+0000 7fe2a0a64640 1 --2- 172.21.15.78:0/698482053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe29c0ffd00 0x7fe29c1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:12.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.251+0000 7fe2a0a64640 1 -- 172.21.15.78:0/698482053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe29c1006a0 msgr2=0x7fe29c198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:12.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.251+0000 7fe2a0a64640 1 --2- 172.21.15.78:0/698482053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe29c1006a0 0x7fe29c198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:12.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.251+0000 7fe2a0a64640 1 -- 172.21.15.78:0/698482053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29c19d1a0 con 0x7fe29c1068e0 2024-08-18T23:48:12.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.251+0000 7fe29b7fe640 1 --2- 172.21.15.78:0/698482053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe29c1006a0 0x7fe29c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:48:12.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.252+0000 7fe2a0a64640 1 --2- 172.21.15.78:0/698482053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe29c1068e0 0x7fe29c19cac0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe29400b9e0 tx=0x7fe29400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:12.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.252+0000 7fe2997fa640 1 -- 172.21.15.78:0/698482053 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294019070 con 0x7fe29c1068e0 2024-08-18T23:48:12.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.252+0000 7fe2a24c7640 1 -- 172.21.15.78:0/698482053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe29c19d490 con 0x7fe29c1068e0 2024-08-18T23:48:12.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.252+0000 7fe2997fa640 1 -- 172.21.15.78:0/698482053 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe294014440 con 0x7fe29c1068e0 2024-08-18T23:48:12.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.252+0000 7fe2997fa640 1 -- 172.21.15.78:0/698482053 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294012580 con 0x7fe29c1068e0 2024-08-18T23:48:12.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.252+0000 7fe2a24c7640 1 -- 172.21.15.78:0/698482053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe29c1a4d70 con 0x7fe29c1068e0 2024-08-18T23:48:12.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.253+0000 7fe2a24c7640 1 -- 172.21.15.78:0/698482053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe29c100160 con 0x7fe29c1068e0 2024-08-18T23:48:12.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.254+0000 7fe2997fa640 1 -- 172.21.15.78:0/698482053 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe29401c070 con 0x7fe29c1068e0 2024-08-18T23:48:12.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.254+0000 7fe2997fa640 1 --2- 172.21.15.78:0/698482053 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe2840a2b80 0x7fe2840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:12.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.255+0000 7fe29bfff640 1 --2- 172.21.15.78:0/698482053 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe2840a2b80 0x7fe2840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:12.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.255+0000 7fe2997fa640 1 -- 172.21.15.78:0/698482053 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fe2940cea00 con 0x7fe29c1068e0 2024-08-18T23:48:12.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.255+0000 7fe29bfff640 1 --2- 172.21.15.78:0/698482053 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe2840a2b80 0x7fe2840a5040 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe28c005e10 tx=0x7fe28c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:12.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.259+0000 7fe2997fa640 1 -- 172.21.15.78:0/698482053 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe29409b5d0 con 0x7fe29c1068e0 2024-08-18T23:48:12.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.464+0000 7fe2a24c7640 1 -- 172.21.15.78:0/698482053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe29c1a5030 con 0x7fe29c1068e0 2024-08-18T23:48:12.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.464+0000 7fe2997fa640 1 -- 172.21.15.78:0/698482053 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11933 (secure 0 0 0) 0x7fe29409aed0 con 0x7fe29c1068e0 2024-08-18T23:48:12.466 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:48:12.466 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":26,"btime":"2024-08-18T23:45:33:306257+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:45:32.491054+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:48:12.466 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 26 2024-08-18T23:48:12.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.466+0000 7fe2a24c7640 1 -- 172.21.15.78:0/698482053 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe2840a2b80 msgr2=0x7fe2840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:12.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.466+0000 7fe2a24c7640 1 --2- 172.21.15.78:0/698482053 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe2840a2b80 0x7fe2840a5040 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe28c005e10 tx=0x7fe28c005da0 comp rx=0 tx=0).stop 2024-08-18T23:48:12.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.466+0000 7fe2a24c7640 1 -- 172.21.15.78:0/698482053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe29c1068e0 msgr2=0x7fe29c19cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:12.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.466+0000 7fe2a24c7640 1 --2- 172.21.15.78:0/698482053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe29c1068e0 0x7fe29c19cac0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe29400b9e0 tx=0x7fe29400bea0 comp rx=0 tx=0).stop 2024-08-18T23:48:12.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.467+0000 7fe2a24c7640 1 -- 172.21.15.78:0/698482053 shutdown_connections 2024-08-18T23:48:12.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.467+0000 7fe2a24c7640 1 --2- 172.21.15.78:0/698482053 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe2840a2b80 0x7fe2840a5040 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:12.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.467+0000 7fe2a24c7640 1 --2- 172.21.15.78:0/698482053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe29c1068e0 0x7fe29c19cac0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:12.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.467+0000 7fe2a24c7640 1 --2- 172.21.15.78:0/698482053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe29c1006a0 0x7fe29c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:12.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.467+0000 7fe2a24c7640 1 --2- 172.21.15.78:0/698482053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe29c0ffd00 0x7fe29c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:12.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.467+0000 7fe2a24c7640 1 -- 172.21.15.78:0/698482053 >> 172.21.15.78:0/698482053 conn(0x7fe29c0fbc10 msgr2=0x7fe29c0fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:12.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.467+0000 7fe2a24c7640 1 -- 172.21.15.78:0/698482053 shutdown_connections 2024-08-18T23:48:12.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:12.467+0000 7fe2a24c7640 1 -- 172.21.15.78:0/698482053 wait complete. 2024-08-18T23:48:13.299 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id f640679b --release --await' 2024-08-18T23:48:13.300 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id f640679b --release --await 2024-08-18T23:48:13.553 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:48:13.675 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:13 smithi078 ceph-mon[18440]: pgmap v467: 129 pgs: 129 active+clean; 472 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 372 KiB/s wr, 71 op/s 2024-08-18T23:48:13.675 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:13 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/698482053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:48:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:13 smithi161 ceph-mon[22027]: pgmap v467: 129 pgs: 129 active+clean; 472 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 372 KiB/s wr, 71 op/s 2024-08-18T23:48:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:13 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/698482053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:48:13.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:13 smithi123 ceph-mon[21808]: pgmap v467: 129 pgs: 129 active+clean; 472 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 372 KiB/s wr, 71 op/s 2024-08-18T23:48:13.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:13 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/698482053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:48:14.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.098+0000 7f18146a2640 1 -- 172.21.15.78:0/3924085835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f180c103390 msgr2=0x7f180c10b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:14.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.099+0000 7f18146a2640 1 --2- 172.21.15.78:0/3924085835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f180c103390 0x7f180c10b880 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1808009900 tx=0x7f180802a480 comp rx=0 tx=0).stop 2024-08-18T23:48:14.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.099+0000 7f18146a2640 1 -- 172.21.15.78:0/3924085835 shutdown_connections 2024-08-18T23:48:14.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.099+0000 7f18146a2640 1 --2- 172.21.15.78:0/3924085835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f180c10bea0 0x7f180c10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.099+0000 7f18146a2640 1 --2- 172.21.15.78:0/3924085835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f180c103390 0x7f180c10b880 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.099+0000 7f18146a2640 1 --2- 172.21.15.78:0/3924085835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f180c102a70 0x7f180c102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.099+0000 7f18146a2640 1 -- 172.21.15.78:0/3924085835 >> 172.21.15.78:0/3924085835 conn(0x7f180c0fc4d0 msgr2=0x7f180c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:14.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.099+0000 7f18146a2640 1 -- 172.21.15.78:0/3924085835 shutdown_connections 2024-08-18T23:48:14.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.099+0000 7f18146a2640 1 -- 172.21.15.78:0/3924085835 wait complete. 2024-08-18T23:48:14.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.099+0000 7f18146a2640 1 Processor -- start 2024-08-18T23:48:14.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.099+0000 7f18146a2640 1 -- start start 2024-08-18T23:48:14.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.099+0000 7f18146a2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f180c102a70 0x7f180c1006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:14.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.099+0000 7f18146a2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f180c103390 0x7f180c100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:14.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.099+0000 7f18146a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f180c10bea0 0x7f180c101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:14.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.100+0000 7f18146a2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f180c111ca0 con 0x7f180c10bea0 2024-08-18T23:48:14.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.100+0000 7f18146a2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f180c111b20 con 0x7f180c102a70 2024-08-18T23:48:14.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.100+0000 7f18146a2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f180c111e20 con 0x7f180c103390 2024-08-18T23:48:14.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.100+0000 7f1812c3f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f180c10bea0 0x7f180c101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:14.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.100+0000 7f1811c3d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f180c103390 0x7f180c100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:14.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.100+0000 7f1812c3f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f180c10bea0 0x7f180c101150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51254/0 (socket says 172.21.15.78:51254) 2024-08-18T23:48:14.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.100+0000 7f1812c3f640 1 -- 172.21.15.78:0/734570751 learned_addr learned my addr 172.21.15.78:0/734570751 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:14.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.100+0000 7f181243e640 1 --2- 172.21.15.78:0/734570751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f180c102a70 0x7f180c1006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:14.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.100+0000 7f1812c3f640 1 -- 172.21.15.78:0/734570751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f180c103390 msgr2=0x7f180c100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:14.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.100+0000 7f1812c3f640 1 --2- 172.21.15.78:0/734570751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f180c103390 0x7f180c100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.100+0000 7f1812c3f640 1 -- 172.21.15.78:0/734570751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f180c102a70 msgr2=0x7f180c1006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:14.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.100+0000 7f1812c3f640 1 --2- 172.21.15.78:0/734570751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f180c102a70 0x7f180c1006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.100+0000 7f1812c3f640 1 -- 172.21.15.78:0/734570751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180c1a6640 con 0x7f180c10bea0 2024-08-18T23:48:14.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.100+0000 7f1812c3f640 1 --2- 172.21.15.78:0/734570751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f180c10bea0 0x7f180c101150 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f180400eaa0 tx=0x7f180400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:14.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.100+0000 7f18037fe640 1 -- 172.21.15.78:0/734570751 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180400cdb0 con 0x7f180c10bea0 2024-08-18T23:48:14.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.100+0000 7f18146a2640 1 -- 172.21.15.78:0/734570751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180c1a6930 con 0x7f180c10bea0 2024-08-18T23:48:14.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.100+0000 7f18037fe640 1 -- 172.21.15.78:0/734570751 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1804005020 con 0x7f180c10bea0 2024-08-18T23:48:14.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.100+0000 7f18037fe640 1 -- 172.21.15.78:0/734570751 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18040116b0 con 0x7f180c10bea0 2024-08-18T23:48:14.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.100+0000 7f18146a2640 1 -- 172.21.15.78:0/734570751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f180c1a6e40 con 0x7f180c10bea0 2024-08-18T23:48:14.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.103+0000 7f18037fe640 1 -- 172.21.15.78:0/734570751 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1804005560 con 0x7f180c10bea0 2024-08-18T23:48:14.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.103+0000 7f1816aa0740 1 Processor -- start 2024-08-18T23:48:14.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.104+0000 7f1816aa0740 1 -- start start 2024-08-18T23:48:14.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.104+0000 7f1816aa0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eeebb8990 0x555eeebb8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:14.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.104+0000 7f1816aa0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb92b0 0x555eeebc3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:14.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.104+0000 7f1816aa0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 0x555eeebc5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:14.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.104+0000 7f1816aa0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555eeeb1ec60 con 0x555eeebc3b70 2024-08-18T23:48:14.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.104+0000 7f1816aa0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555eeeab6b90 con 0x555eeebb92b0 2024-08-18T23:48:14.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.104+0000 7f1816aa0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555eeeb1ac20 con 0x555eeebb8990 2024-08-18T23:48:14.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.104+0000 7f1811c3d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 0x555eeebc5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:14.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.104+0000 7f1811c3d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 0x555eeebc5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51266/0 (socket says 172.21.15.78:51266) 2024-08-18T23:48:14.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.104+0000 7f1811c3d640 1 -- 172.21.15.78:0/2215033275 learned_addr learned my addr 172.21.15.78:0/2215033275 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:14.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.104+0000 7f1811c3d640 1 --2- 172.21.15.78:0/2215033275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eeebb8990 0x555eeebb8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:14.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.104+0000 7f181243e640 1 --2- 172.21.15.78:0/2215033275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb92b0 0x555eeebc3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:14.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.104+0000 7f18037fe640 1 --2- 172.21.15.78:0/734570751 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f17f80a2b80 0x7f17f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:14.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.104+0000 7f1811c3d640 1 -- 172.21.15.78:0/2215033275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eeebb8990 msgr2=0x555eeebb8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:14.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.104+0000 7f1811c3d640 1 --2- 172.21.15.78:0/2215033275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eeebb8990 0x555eeebb8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.104+0000 7f1811c3d640 1 -- 172.21.15.78:0/2215033275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb92b0 msgr2=0x555eeebc3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:14.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.104+0000 7f1811c3d640 1 --2- 172.21.15.78:0/2215033275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb92b0 0x555eeebc3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.105+0000 7f1811c3d640 1 -- 172.21.15.78:0/2215033275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eee82e580 con 0x555eeebc3b70 2024-08-18T23:48:14.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.105+0000 7f181243e640 1 --2- 172.21.15.78:0/2215033275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb92b0 0x555eeebc3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:48:14.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.105+0000 7f18037fe640 1 -- 172.21.15.78:0/734570751 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f18040cf190 con 0x7f180c10bea0 2024-08-18T23:48:14.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.105+0000 7f1812c3f640 1 --2- 172.21.15.78:0/734570751 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f17f80a2b80 0x7f17f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:14.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.105+0000 7f1811c3d640 1 --2- 172.21.15.78:0/2215033275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 0x555eeebc5f60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1808000c00 tx=0x7f18080314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8667b5a3eeee42 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:14.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.105+0000 7f1800ff9640 1 -- 172.21.15.78:0/2215033275 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808035070 con 0x555eeebc3b70 2024-08-18T23:48:14.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.105+0000 7f1800ff9640 1 -- 172.21.15.78:0/2215033275 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1808005020 con 0x555eeebc3b70 2024-08-18T23:48:14.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.105+0000 7f1800ff9640 1 -- 172.21.15.78:0/2215033275 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808004a00 con 0x555eeebc3b70 2024-08-18T23:48:14.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.105+0000 7f1812c3f640 1 --2- 172.21.15.78:0/734570751 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f17f80a2b80 0x7f17f80a5040 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1804005910 tx=0x7f1804009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:14.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.106+0000 7f1816aa0740 1 -- 172.21.15.78:0/2215033275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 msgr2=0x555eeebc5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:14.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.106+0000 7f1816aa0740 1 --2- 172.21.15.78:0/2215033275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 0x555eeebc5f60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1808000c00 tx=0x7f18080314e0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.106+0000 7f1816aa0740 1 -- 172.21.15.78:0/2215033275 shutdown_connections 2024-08-18T23:48:14.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.106+0000 7f1816aa0740 1 --2- 172.21.15.78:0/2215033275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 0x555eeebc5f60 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.106+0000 7f1816aa0740 1 --2- 172.21.15.78:0/2215033275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb92b0 0x555eeebc3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.106+0000 7f1816aa0740 1 --2- 172.21.15.78:0/2215033275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eeebb8990 0x555eeebb8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.106+0000 7f1816aa0740 1 -- 172.21.15.78:0/2215033275 >> 172.21.15.78:0/2215033275 conn(0x555eeebb4f20 msgr2=0x555eeebb5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:14.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.106+0000 7f1816aa0740 1 -- 172.21.15.78:0/2215033275 shutdown_connections 2024-08-18T23:48:14.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.106+0000 7f1816aa0740 1 -- 172.21.15.78:0/2215033275 wait complete. 2024-08-18T23:48:14.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.106+0000 7f1816aa0740 1 -- start start 2024-08-18T23:48:14.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.106+0000 7f1816aa0740 1 Processor -- start 2024-08-18T23:48:14.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.107+0000 7f1816aa0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb8990 0x555eeebc0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:14.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.107+0000 7f1816aa0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebb92b0 0x555eeebc1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:14.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.107+0000 7f1816aa0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eeebc3b70 0x555eeebc1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:14.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.107+0000 7f1816aa0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555eeebc6740 con 0x555eeebb92b0 2024-08-18T23:48:14.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.107+0000 7f1816aa0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555eeeabc7c0 con 0x555eeebb8990 2024-08-18T23:48:14.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.107+0000 7f1816aa0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555eeebc68c0 con 0x555eeebc3b70 2024-08-18T23:48:14.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.107+0000 7f1816aa0740 10 client.0 _ll_register_callbacks cb 0x555eeeac5f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:48:14.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.107+0000 7f181243e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebb92b0 0x555eeebc1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:14.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.107+0000 7f181243e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebb92b0 0x555eeebc1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51276/0 (socket says 172.21.15.78:51276) 2024-08-18T23:48:14.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.107+0000 7f181243e640 1 -- 172.21.15.78:0/1922991518 learned_addr learned my addr 172.21.15.78:0/1922991518 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:14.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.107+0000 7f1811c3d640 1 --2- 172.21.15.78:0/1922991518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eeebc3b70 0x555eeebc1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:14.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.107+0000 7f181243e640 1 -- 172.21.15.78:0/1922991518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eeebc3b70 msgr2=0x555eeebc1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:14.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.107+0000 7f181243e640 1 --2- 172.21.15.78:0/1922991518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eeebc3b70 0x555eeebc1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.107+0000 7f1811c3d640 1 --2- 172.21.15.78:0/1922991518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb8990 0x555eeebc0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:14.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.107+0000 7f181243e640 1 -- 172.21.15.78:0/1922991518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb8990 msgr2=0x555eeebc0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:14.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.107+0000 7f181243e640 1 --2- 172.21.15.78:0/1922991518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb8990 0x555eeebc0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.107+0000 7f181243e640 1 -- 172.21.15.78:0/1922991518 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eee8f8960 con 0x555eeebb92b0 2024-08-18T23:48:14.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.107+0000 7f181243e640 1 --2- 172.21.15.78:0/1922991518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebb92b0 0x555eeebc1020 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00b9e0 tx=0x7f17fc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6b27fc78722e6d server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:14.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.107+0000 7f17f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:48:14.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.108+0000 7f17f5ffb640 1 -- 172.21.15.78:0/1922991518 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc01b640 con 0x555eeebb92b0 2024-08-18T23:48:14.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.108+0000 7f17f5ffb640 1 -- 172.21.15.78:0/1922991518 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f17fc01b7e0 con 0x555eeebb92b0 2024-08-18T23:48:14.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.108+0000 7f1816aa0740 1 -- 172.21.15.78:0/1922991518 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1808005020 con 0x555eeebb92b0 2024-08-18T23:48:14.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.108+0000 7f17f5ffb640 1 -- 172.21.15.78:0/1922991518 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f17fc024050 con 0x555eeebb92b0 2024-08-18T23:48:14.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.108+0000 7f1816aa0740 10 client.16176 fetch_fsmap learned FSMap version 27 2024-08-18T23:48:14.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.108+0000 7f1816aa0740 1 -- 172.21.15.78:0/1922991518 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555eeebd2890 con 0x555eeebb92b0 2024-08-18T23:48:14.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.108+0000 7f17f5ffb640 1 -- 172.21.15.78:0/1922991518 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1808005020 con 0x555eeebb92b0 2024-08-18T23:48:14.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.108+0000 7f17f5ffb640 1 -- 172.21.15.78:0/1922991518 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f17fc019020 con 0x555eeebb92b0 2024-08-18T23:48:14.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.108+0000 7f1816aa0740 10 client.16176 fetch_fsmap finished waiting for FSMap version 27 2024-08-18T23:48:14.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.108+0000 7f1816aa0740 10 client.16176 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:48:14.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.108+0000 7f1816aa0740 20 client.16176 populate_metadata read hostname 'smithi078' 2024-08-18T23:48:14.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.108+0000 7f1816aa0740 1 --2- 172.21.15.78:0/1922991518 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x555eeebd3d50 0x555eeebd61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:14.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.108+0000 7f1816aa0740 4 client.16176 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:48:14.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.108+0000 7f1816aa0740 1 -- 172.21.15.78:0/1922991518 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555eeeb2c1d0 con 0x555eeebd3d50 2024-08-18T23:48:14.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.108+0000 7f1811c3d640 1 --2- 172.21.15.78:0/1922991518 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x555eeebd3d50 0x555eeebd61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:14.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.109+0000 7f1811c3d640 1 --2- 172.21.15.78:0/1922991518 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x555eeebd3d50 0x555eeebd61d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd4c87392154f69c server_cookie=c835bf25e05d10d7 in_seq=0 out_seq=0 2024-08-18T23:48:14.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.109+0000 7f17f5ffb640 10 client.16176 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:14.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.110+0000 7f17f5ffb640 1 -- 172.21.15.78:0/1922991518 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555eeeb2c1d0 con 0x555eeebd3d50 2024-08-18T23:48:14.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.110+0000 7f17f5ffb640 10 client.16176 handle_command_reply: tid=0 2024-08-18T23:48:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.110+0000 7f1816aa0740 1 client.16176 shutdown 2024-08-18T23:48:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.110+0000 7f1816aa0740 1 -- 172.21.15.78:0/1922991518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebb92b0 msgr2=0x555eeebc1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.110+0000 7f1816aa0740 1 --2- 172.21.15.78:0/1922991518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebb92b0 0x555eeebc1020 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00b9e0 tx=0x7f17fc00bea0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.110+0000 7f1816aa0740 1 -- 172.21.15.78:0/1922991518 shutdown_connections 2024-08-18T23:48:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.110+0000 7f1816aa0740 1 --2- 172.21.15.78:0/1922991518 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x555eeebd3d50 0x555eeebd61d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.110+0000 7f1816aa0740 1 --2- 172.21.15.78:0/1922991518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eeebc3b70 0x555eeebc1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.110+0000 7f1816aa0740 1 --2- 172.21.15.78:0/1922991518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebb92b0 0x555eeebc1020 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.110+0000 7f1816aa0740 1 --2- 172.21.15.78:0/1922991518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb8990 0x555eeebc0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.110+0000 7f1816aa0740 1 -- 172.21.15.78:0/1922991518 >> 172.21.15.78:0/1922991518 conn(0x555eeebb4f20 msgr2=0x555eeebbac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.110+0000 7f17f5ffb640 0 client.16176 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.110+0000 7f1816aa0740 1 -- 172.21.15.78:0/1922991518 shutdown_connections 2024-08-18T23:48:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.110+0000 7f1816aa0740 1 -- 172.21.15.78:0/1922991518 wait complete. 2024-08-18T23:48:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.110+0000 7f1816aa0740 20 client.16176 trim_cache size 0 max 16384 2024-08-18T23:48:14.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.127+0000 7f1816aa0740 1 Processor -- start 2024-08-18T23:48:14.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.127+0000 7f1816aa0740 1 -- start start 2024-08-18T23:48:14.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.127+0000 7f1816aa0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eeebb8990 0x555eeebdbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:14.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.127+0000 7f1816aa0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb92b0 0x555eeebdc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:14.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.127+0000 7f1816aa0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 0x555eeebbdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:14.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.127+0000 7f1816aa0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1b3510 con 0x555eeebc3b70 2024-08-18T23:48:14.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.127+0000 7f1816aa0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1b0de0 con 0x555eeebb92b0 2024-08-18T23:48:14.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.127+0000 7f1816aa0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1aec30 con 0x555eeebb8990 2024-08-18T23:48:14.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.127+0000 7f1811c3d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 0x555eeebbdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:14.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.127+0000 7f1811c3d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 0x555eeebbdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51284/0 (socket says 172.21.15.78:51284) 2024-08-18T23:48:14.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.127+0000 7f1811c3d640 1 -- 172.21.15.78:0/579584524 learned_addr learned my addr 172.21.15.78:0/579584524 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:14.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.127+0000 7f1811c3d640 1 -- 172.21.15.78:0/579584524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eeebb8990 msgr2=0x555eeebdbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:14.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.127+0000 7f1811c3d640 1 --2- 172.21.15.78:0/579584524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eeebb8990 0x555eeebdbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.127+0000 7f181243e640 1 --2- 172.21.15.78:0/579584524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb92b0 0x555eeebdc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:14.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.127+0000 7f1811c3d640 1 -- 172.21.15.78:0/579584524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb92b0 msgr2=0x555eeebdc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:14.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.127+0000 7f1811c3d640 1 --2- 172.21.15.78:0/579584524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb92b0 0x555eeebdc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.127+0000 7f1811c3d640 1 -- 172.21.15.78:0/579584524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eeebb87c0 con 0x555eeebc3b70 2024-08-18T23:48:14.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.127+0000 7f1811c3d640 1 --2- 172.21.15.78:0/579584524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 0x555eeebbdd40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1808031550 tx=0x7f1808009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d349900fb94f2e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:14.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.128+0000 7f17f7fff640 1 -- 172.21.15.78:0/579584524 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808059070 con 0x555eeebc3b70 2024-08-18T23:48:14.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.128+0000 7f17f7fff640 1 -- 172.21.15.78:0/579584524 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1808005520 con 0x555eeebc3b70 2024-08-18T23:48:14.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.128+0000 7f17f7fff640 1 -- 172.21.15.78:0/579584524 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808035070 con 0x555eeebc3b70 2024-08-18T23:48:14.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.128+0000 7f1816aa0740 1 -- 172.21.15.78:0/579584524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 msgr2=0x555eeebbdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:14.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.128+0000 7f1816aa0740 1 --2- 172.21.15.78:0/579584524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 0x555eeebbdd40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1808031550 tx=0x7f1808009640 comp rx=0 tx=0).stop 2024-08-18T23:48:14.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.128+0000 7f1816aa0740 1 -- 172.21.15.78:0/579584524 shutdown_connections 2024-08-18T23:48:14.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.128+0000 7f1816aa0740 1 --2- 172.21.15.78:0/579584524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 0x555eeebbdd40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.128+0000 7f1816aa0740 1 --2- 172.21.15.78:0/579584524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb92b0 0x555eeebdc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.128+0000 7f1816aa0740 1 --2- 172.21.15.78:0/579584524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eeebb8990 0x555eeebdbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.128+0000 7f1816aa0740 1 -- 172.21.15.78:0/579584524 >> 172.21.15.78:0/579584524 conn(0x555eeebb4f20 msgr2=0x555eeebc2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:14.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.128+0000 7f1816aa0740 1 -- 172.21.15.78:0/579584524 shutdown_connections 2024-08-18T23:48:14.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.128+0000 7f1816aa0740 1 -- 172.21.15.78:0/579584524 wait complete. 2024-08-18T23:48:14.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f1816aa0740 1 -- start start 2024-08-18T23:48:14.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f1816aa0740 1 Processor -- start 2024-08-18T23:48:14.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f1816aa0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eeebb8990 0x555eeebcc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:14.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f1816aa0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb92b0 0x555eeebc75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:14.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f1816aa0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 0x555eeebc7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:14.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f1816aa0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1af2a0 con 0x555eeebc3b70 2024-08-18T23:48:14.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f1816aa0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1b0710 con 0x555eeebb92b0 2024-08-18T23:48:14.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f1816aa0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1af970 con 0x555eeebb8990 2024-08-18T23:48:14.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f1816aa0740 10 client.0 _ll_register_callbacks cb 0x7f17fc004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:48:14.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f1811c3d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 0x555eeebc7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:14.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f1811c3d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 0x555eeebc7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51292/0 (socket says 172.21.15.78:51292) 2024-08-18T23:48:14.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f1811c3d640 1 -- 172.21.15.78:0/945369002 learned_addr learned my addr 172.21.15.78:0/945369002 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:14.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f181243e640 1 --2- 172.21.15.78:0/945369002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb92b0 0x555eeebc75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:14.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f1811c3d640 1 -- 172.21.15.78:0/945369002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eeebb8990 msgr2=0x555eeebcc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:14.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f1811c3d640 1 --2- 172.21.15.78:0/945369002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eeebb8990 0x555eeebcc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f1811c3d640 1 -- 172.21.15.78:0/945369002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb92b0 msgr2=0x555eeebc75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:14.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f1811c3d640 1 --2- 172.21.15.78:0/945369002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb92b0 0x555eeebc75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f1811c3d640 1 -- 172.21.15.78:0/945369002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eeebc8370 con 0x555eeebc3b70 2024-08-18T23:48:14.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f181243e640 1 --2- 172.21.15.78:0/945369002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb92b0 0x555eeebc75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:48:14.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f1811c3d640 1 --2- 172.21.15.78:0/945369002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 0x555eeebc7b20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f180802a9a0 tx=0x7f18080097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a546759bd48f482b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:14.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.129+0000 7f17f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:48:14.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.130+0000 7f17f5ffb640 1 -- 172.21.15.78:0/945369002 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808035070 con 0x555eeebc3b70 2024-08-18T23:48:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.130+0000 7f17f5ffb640 1 -- 172.21.15.78:0/945369002 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f1808005040 con 0x555eeebc3b70 2024-08-18T23:48:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.130+0000 7f1816aa0740 1 -- 172.21.15.78:0/945369002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1808005520 con 0x555eeebc3b70 2024-08-18T23:48:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.130+0000 7f17f5ffb640 1 -- 172.21.15.78:0/945369002 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1808059040 con 0x555eeebc3b70 2024-08-18T23:48:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.130+0000 7f1816aa0740 10 client.16182 fetch_fsmap learned FSMap version 27 2024-08-18T23:48:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.130+0000 7f1816aa0740 1 -- 172.21.15.78:0/945369002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555eeec3eb20 con 0x555eeebc3b70 2024-08-18T23:48:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.130+0000 7f17f5ffb640 1 -- 172.21.15.78:0/945369002 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1808005520 con 0x555eeebc3b70 2024-08-18T23:48:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.130+0000 7f17f5ffb640 1 -- 172.21.15.78:0/945369002 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f180803d610 con 0x555eeebc3b70 2024-08-18T23:48:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.130+0000 7f1816aa0740 10 client.16182 fetch_fsmap finished waiting for FSMap version 27 2024-08-18T23:48:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.130+0000 7f1816aa0740 10 client.16182 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:48:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.130+0000 7f1816aa0740 20 client.16182 populate_metadata read hostname 'smithi078' 2024-08-18T23:48:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.130+0000 7f1816aa0740 1 --2- 172.21.15.78:0/945369002 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x555eeec3ed50 0x555eeec41140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.130+0000 7f1816aa0740 4 client.16182 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f640679b", "release": true, "await": true}] 2024-08-18T23:48:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.130+0000 7f1816aa0740 1 -- 172.21.15.78:0/945369002 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f640679b", "release": true, "await": true}) -- 0x555eeeb266e0 con 0x555eeec3ed50 2024-08-18T23:48:14.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.131+0000 7f181243e640 1 --2- 172.21.15.78:0/945369002 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x555eeec3ed50 0x555eeec41140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:14.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.131+0000 7f17f5ffb640 10 client.16182 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:14.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.131+0000 7f181243e640 1 --2- 172.21.15.78:0/945369002 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x555eeec3ed50 0x555eeec41140 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f062dbb29303a1cf server_cookie=31048a05fdf1ad50 in_seq=0 out_seq=0 2024-08-18T23:48:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:14 smithi161 ceph-mon[22027]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:48:14.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:14 smithi078 ceph-mon[18440]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:48:14.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.744+0000 7f17f5ffb640 1 -- 172.21.15.78:0/945369002 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555eeeb266e0 con 0x555eeec3ed50 2024-08-18T23:48:14.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.744+0000 7f17f5ffb640 10 client.16182 handle_command_reply: tid=0 2024-08-18T23:48:14.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.744+0000 7f1816aa0740 1 client.16182 shutdown 2024-08-18T23:48:14.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.744+0000 7f1816aa0740 1 -- 172.21.15.78:0/945369002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 msgr2=0x555eeebc7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:14.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.744+0000 7f1816aa0740 1 --2- 172.21.15.78:0/945369002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 0x555eeebc7b20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f180802a9a0 tx=0x7f18080097a0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.744+0000 7f1816aa0740 1 -- 172.21.15.78:0/945369002 shutdown_connections 2024-08-18T23:48:14.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.744+0000 7f1816aa0740 1 --2- 172.21.15.78:0/945369002 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x555eeec3ed50 0x555eeec41140 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.744+0000 7f1816aa0740 1 --2- 172.21.15.78:0/945369002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555eeebc3b70 0x555eeebc7b20 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.744+0000 7f1816aa0740 1 --2- 172.21.15.78:0/945369002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555eeebb92b0 0x555eeebc75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.744+0000 7f17f5ffb640 0 client.16182 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:14.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.744+0000 7f1816aa0740 1 --2- 172.21.15.78:0/945369002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555eeebb8990 0x555eeebcc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.744+0000 7f1816aa0740 1 -- 172.21.15.78:0/945369002 >> 172.21.15.78:0/945369002 conn(0x555eeebb4f20 msgr2=0x555eeebba350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:14.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.744+0000 7f1816aa0740 1 -- 172.21.15.78:0/945369002 shutdown_connections 2024-08-18T23:48:14.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.744+0000 7f1816aa0740 1 -- 172.21.15.78:0/945369002 wait complete. 2024-08-18T23:48:14.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.744+0000 7f1816aa0740 20 client.16182 trim_cache size 0 max 16384 2024-08-18T23:48:14.747 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:48:14.747 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 27, 2024-08-18T23:48:14.747 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24452, 2024-08-18T23:48:14.747 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 16, 2024-08-18T23:48:14.748 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-08-18T23:48:14.748 INFO:teuthology.orchestra.run.smithi078.stdout: "f640679b": { 2024-08-18T23:48:14.748 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 16, 2024-08-18T23:48:14.748 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-08-18T23:48:14.748 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:48:14.748 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-08-18T23:48:14.748 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:48:14.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:48:14.748 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-08-18T23:48:14.748 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 150.0, 2024-08-18T23:48:14.748 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-08-18T23:48:14.748 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-08-18T23:48:14.748 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-08-18T23:48:14.749 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:48:14.749 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-08-18T23:48:14.749 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:48:14.749 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:48:14.749 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:48:14.749 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:48:14.749 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:48:14.749 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:48:14.749 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:48:14.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.745+0000 7f18146a2640 1 -- 172.21.15.78:0/734570751 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f17f80a2b80 msgr2=0x7f17f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:14.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.745+0000 7f18146a2640 1 --2- 172.21.15.78:0/734570751 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f17f80a2b80 0x7f17f80a5040 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1804005910 tx=0x7f1804009270 comp rx=0 tx=0).stop 2024-08-18T23:48:14.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.745+0000 7f18146a2640 1 -- 172.21.15.78:0/734570751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f180c10bea0 msgr2=0x7f180c101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:14.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.745+0000 7f18146a2640 1 --2- 172.21.15.78:0/734570751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f180c10bea0 0x7f180c101150 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f180400eaa0 tx=0x7f180400ef60 comp rx=0 tx=0).stop 2024-08-18T23:48:14.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.745+0000 7f18146a2640 1 -- 172.21.15.78:0/734570751 shutdown_connections 2024-08-18T23:48:14.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.745+0000 7f18146a2640 1 --2- 172.21.15.78:0/734570751 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f17f80a2b80 0x7f17f80a5040 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.745+0000 7f18146a2640 1 --2- 172.21.15.78:0/734570751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f180c10bea0 0x7f180c101150 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.745+0000 7f18146a2640 1 --2- 172.21.15.78:0/734570751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f180c103390 0x7f180c100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.745+0000 7f18146a2640 1 --2- 172.21.15.78:0/734570751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f180c102a70 0x7f180c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:14.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.745+0000 7f18146a2640 1 -- 172.21.15.78:0/734570751 >> 172.21.15.78:0/734570751 conn(0x7f180c0fc4d0 msgr2=0x7f180c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:14.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.745+0000 7f18146a2640 1 -- 172.21.15.78:0/734570751 shutdown_connections 2024-08-18T23:48:14.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:14.745+0000 7f18146a2640 1 -- 172.21.15.78:0/734570751 wait complete. 2024-08-18T23:48:14.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:14 smithi123 ceph-mon[21808]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:48:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:15 smithi078 ceph-mon[18440]: pgmap v468: 129 pgs: 129 active+clean; 472 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 227 KiB/s wr, 32 op/s 2024-08-18T23:48:15.565 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f640679b', total seconds elapsed: 34.1 2024-08-18T23:48:15.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:15 smithi161 ceph-mon[22027]: pgmap v468: 129 pgs: 129 active+clean; 472 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 227 KiB/s wr, 32 op/s 2024-08-18T23:48:15.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:15 smithi123 ceph-mon[21808]: pgmap v468: 129 pgs: 129 active+clean; 472 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 227 KiB/s wr, 32 op/s 2024-08-18T23:48:16.982 INFO:tasks.workunit.client.0.smithi078.stdout: 80 3373 98 2499 26 985 205 2024-08-18T23:48:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:17 smithi161 ceph-mon[22027]: pgmap v469: 129 pgs: 129 active+clean; 472 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 227 KiB/s wr, 24 op/s 2024-08-18T23:48:17.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:17 smithi078 ceph-mon[18440]: pgmap v469: 129 pgs: 129 active+clean; 472 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 227 KiB/s wr, 24 op/s 2024-08-18T23:48:17.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:17 smithi123 ceph-mon[21808]: pgmap v469: 129 pgs: 129 active+clean; 472 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 227 KiB/s wr, 24 op/s 2024-08-18T23:48:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:19 smithi161 ceph-mon[22027]: pgmap v470: 129 pgs: 129 active+clean; 472 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 264 KiB/s wr, 35 op/s 2024-08-18T23:48:19.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:19 smithi078 ceph-mon[18440]: pgmap v470: 129 pgs: 129 active+clean; 472 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 264 KiB/s wr, 35 op/s 2024-08-18T23:48:19.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:19 smithi123 ceph-mon[21808]: pgmap v470: 129 pgs: 129 active+clean; 472 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 264 KiB/s wr, 35 op/s 2024-08-18T23:48:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:21 smithi161 ceph-mon[22027]: pgmap v471: 129 pgs: 129 active+clean; 472 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-08-18T23:48:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:21 smithi078 ceph-mon[18440]: pgmap v471: 129 pgs: 129 active+clean; 472 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-08-18T23:48:21.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:21 smithi123 ceph-mon[21808]: pgmap v471: 129 pgs: 129 active+clean; 472 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-08-18T23:48:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:23 smithi161 ceph-mon[22027]: pgmap v472: 129 pgs: 129 active+clean; 474 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 343 KiB/s wr, 82 op/s 2024-08-18T23:48:23.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:23 smithi078 ceph-mon[18440]: pgmap v472: 129 pgs: 129 active+clean; 474 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 343 KiB/s wr, 82 op/s 2024-08-18T23:48:23.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:23 smithi123 ceph-mon[21808]: pgmap v472: 129 pgs: 129 active+clean; 474 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 343 KiB/s wr, 82 op/s 2024-08-18T23:48:25.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:25 smithi161 ceph-mon[22027]: pgmap v473: 129 pgs: 129 active+clean; 477 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 655 KiB/s wr, 98 op/s 2024-08-18T23:48:25.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:25 smithi078 ceph-mon[18440]: pgmap v473: 129 pgs: 129 active+clean; 477 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 655 KiB/s wr, 98 op/s 2024-08-18T23:48:25.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:25 smithi123 ceph-mon[21808]: pgmap v473: 129 pgs: 129 active+clean; 477 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 655 KiB/s wr, 98 op/s 2024-08-18T23:48:26.983 INFO:tasks.workunit.client.0.smithi078.stdout: 84 522608 369 362983 220 271671 731 2024-08-18T23:48:27.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:27 smithi161 ceph-mon[22027]: pgmap v474: 129 pgs: 129 active+clean; 479 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 881 KiB/s wr, 141 op/s 2024-08-18T23:48:27.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:27 smithi078 ceph-mon[18440]: pgmap v474: 129 pgs: 129 active+clean; 479 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 881 KiB/s wr, 141 op/s 2024-08-18T23:48:27.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:27 smithi123 ceph-mon[21808]: pgmap v474: 129 pgs: 129 active+clean; 479 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 881 KiB/s wr, 141 op/s 2024-08-18T23:48:29.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:29 smithi161 ceph-mon[22027]: pgmap v475: 129 pgs: 129 active+clean; 490 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 209 op/s 2024-08-18T23:48:29.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:29 smithi078 ceph-mon[18440]: pgmap v475: 129 pgs: 129 active+clean; 490 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 209 op/s 2024-08-18T23:48:29.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:29 smithi123 ceph-mon[21808]: pgmap v475: 129 pgs: 129 active+clean; 490 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 209 op/s 2024-08-18T23:48:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:31 smithi161 ceph-mon[22027]: pgmap v476: 129 pgs: 129 active+clean; 497 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 256 op/s 2024-08-18T23:48:31.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:31 smithi078 ceph-mon[18440]: pgmap v476: 129 pgs: 129 active+clean; 497 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 256 op/s 2024-08-18T23:48:31.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:31 smithi123 ceph-mon[21808]: pgmap v476: 129 pgs: 129 active+clean; 497 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 256 op/s 2024-08-18T23:48:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:33 smithi161 ceph-mon[22027]: pgmap v477: 129 pgs: 129 active+clean; 504 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 254 op/s 2024-08-18T23:48:33.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:33 smithi078 ceph-mon[18440]: pgmap v477: 129 pgs: 129 active+clean; 504 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 254 op/s 2024-08-18T23:48:33.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:33 smithi123 ceph-mon[21808]: pgmap v477: 129 pgs: 129 active+clean; 504 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 254 op/s 2024-08-18T23:48:34.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:34 smithi161 ceph-mon[22027]: pgmap v478: 129 pgs: 129 active+clean; 504 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 224 op/s 2024-08-18T23:48:34.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:34 smithi078 ceph-mon[18440]: pgmap v478: 129 pgs: 129 active+clean; 504 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 224 op/s 2024-08-18T23:48:34.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:34 smithi123 ceph-mon[21808]: pgmap v478: 129 pgs: 129 active+clean; 504 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 224 op/s 2024-08-18T23:48:34.999 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:48:35.943 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:48:36.822 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:48:36.822 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-08-18T23:48:36.822 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-08-18T23:48:36.822 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:48:36.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.538+0000 7f3a83435640 1 -- 172.21.15.78:0/2309890357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a7c109180 msgr2=0x7f3a7c109560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:36.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.538+0000 7f3a83435640 1 --2- 172.21.15.78:0/2309890357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a7c109180 0x7f3a7c109560 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009900 tx=0x7f3a7802a440 comp rx=0 tx=0).stop 2024-08-18T23:48:36.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.538+0000 7f3a83435640 1 -- 172.21.15.78:0/2309890357 shutdown_connections 2024-08-18T23:48:36.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.538+0000 7f3a83435640 1 --2- 172.21.15.78:0/2309890357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a7c0746c0 0x7f3a7c07b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.538+0000 7f3a83435640 1 --2- 172.21.15.78:0/2309890357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a7c109b30 0x7f3a7c074000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.538+0000 7f3a83435640 1 --2- 172.21.15.78:0/2309890357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a7c109180 0x7f3a7c109560 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.538+0000 7f3a83435640 1 -- 172.21.15.78:0/2309890357 >> 172.21.15.78:0/2309890357 conn(0x7f3a7c06de70 msgr2=0x7f3a7c06e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:36.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.539+0000 7f3a83435640 1 -- 172.21.15.78:0/2309890357 shutdown_connections 2024-08-18T23:48:36.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.539+0000 7f3a83435640 1 -- 172.21.15.78:0/2309890357 wait complete. 2024-08-18T23:48:36.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.542+0000 7f3a83435640 1 Processor -- start 2024-08-18T23:48:36.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.546+0000 7f3a83435640 1 -- start start 2024-08-18T23:48:36.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.546+0000 7f3a83435640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a7c0746c0 0x7f3a7c07aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:36.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.546+0000 7f3a83435640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a7c109180 0x7f3a7c075f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:36.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.547+0000 7f3a83435640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a7c109b30 0x7f3a7c076490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:36.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.547+0000 7f3a811d1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a7c0746c0 0x7f3a7c07aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:36.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.547+0000 7f3a811d1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a7c0746c0 0x7f3a7c07aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:39390/0 (socket says 172.21.15.78:39390) 2024-08-18T23:48:36.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.547+0000 7f3a811d1640 1 -- 172.21.15.78:0/840402152 learned_addr learned my addr 172.21.15.78:0/840402152 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:36.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.547+0000 7f3a809d0640 1 --2- 172.21.15.78:0/840402152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a7c109180 0x7f3a7c075f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:36.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.547+0000 7f3a819d2640 1 --2- 172.21.15.78:0/840402152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a7c109b30 0x7f3a7c076490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:36.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.547+0000 7f3a83435640 1 -- 172.21.15.78:0/840402152 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c07e4b0 con 0x7f3a7c109b30 2024-08-18T23:48:36.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.547+0000 7f3a83435640 1 -- 172.21.15.78:0/840402152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c07e330 con 0x7f3a7c109180 2024-08-18T23:48:36.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.547+0000 7f3a83435640 1 -- 172.21.15.78:0/840402152 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c07e630 con 0x7f3a7c0746c0 2024-08-18T23:48:36.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.550+0000 7f3a811d1640 1 -- 172.21.15.78:0/840402152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a7c109180 msgr2=0x7f3a7c075f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:36.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.550+0000 7f3a811d1640 1 --2- 172.21.15.78:0/840402152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a7c109180 0x7f3a7c075f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.550+0000 7f3a811d1640 1 -- 172.21.15.78:0/840402152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a7c109b30 msgr2=0x7f3a7c076490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:36.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.550+0000 7f3a811d1640 1 --2- 172.21.15.78:0/840402152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a7c109b30 0x7f3a7c076490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.550+0000 7f3a811d1640 1 -- 172.21.15.78:0/840402152 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7c076d50 con 0x7f3a7c0746c0 2024-08-18T23:48:36.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.550+0000 7f3a809d0640 1 --2- 172.21.15.78:0/840402152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a7c109180 0x7f3a7c075f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:48:36.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.550+0000 7f3a819d2640 1 --2- 172.21.15.78:0/840402152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a7c109b30 0x7f3a7c076490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:48:36.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.551+0000 7f3a811d1640 1 --2- 172.21.15.78:0/840402152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a7c0746c0 0x7f3a7c07aec0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3a78005ab0 tx=0x7f3a78005230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:36.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.552+0000 7f3a727fc640 1 -- 172.21.15.78:0/840402152 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78040030 con 0x7f3a7c0746c0 2024-08-18T23:48:36.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.552+0000 7f3a727fc640 1 -- 172.21.15.78:0/840402152 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a78004a30 con 0x7f3a7c0746c0 2024-08-18T23:48:36.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.552+0000 7f3a727fc640 1 -- 172.21.15.78:0/840402152 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7803c7f0 con 0x7f3a7c0746c0 2024-08-18T23:48:36.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.554+0000 7f3a83435640 1 -- 172.21.15.78:0/840402152 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7c1bbd30 con 0x7f3a7c0746c0 2024-08-18T23:48:36.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.555+0000 7f3a83435640 1 -- 172.21.15.78:0/840402152 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7c1bc160 con 0x7f3a7c0746c0 2024-08-18T23:48:36.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.563+0000 7f3a727fc640 1 -- 172.21.15.78:0/840402152 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3a78043030 con 0x7f3a7c0746c0 2024-08-18T23:48:36.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.565+0000 7f3a727fc640 1 --2- 172.21.15.78:0/840402152 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3a680a2d90 0x7f3a680a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:36.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.565+0000 7f3a727fc640 1 -- 172.21.15.78:0/840402152 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f3a780efd40 con 0x7f3a7c0746c0 2024-08-18T23:48:36.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.566+0000 7f3a809d0640 1 --2- 172.21.15.78:0/840402152 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3a680a2d90 0x7f3a680a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:36.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.567+0000 7f3a809d0640 1 --2- 172.21.15.78:0/840402152 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3a680a2d90 0x7f3a680a5250 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c077710 tx=0x7f3a6c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:36.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.567+0000 7f3a85833740 1 Processor -- start 2024-08-18T23:48:36.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.570+0000 7f3a85833740 1 -- start start 2024-08-18T23:48:36.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.570+0000 7f3a85833740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55957749a210 0x55957749a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:36.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.570+0000 7f3a85833740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55957749ab30 0x5595774a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:36.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.570+0000 7f3a85833740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595774a53f0 0x5595774a77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:36.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.571+0000 7f3a85833740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5595773ff980 con 0x5595774a53f0 2024-08-18T23:48:36.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.571+0000 7f3a85833740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559577397a70 con 0x55957749a210 2024-08-18T23:48:36.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.571+0000 7f3a85833740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5595773fb940 con 0x55957749ab30 2024-08-18T23:48:36.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.571+0000 7f3a809d0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55957749ab30 0x5595774a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:36.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.571+0000 7f3a819d2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595774a53f0 0x5595774a77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:36.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.571+0000 7f3a819d2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595774a53f0 0x5595774a77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39148/0 (socket says 172.21.15.78:39148) 2024-08-18T23:48:36.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.571+0000 7f3a809d0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55957749ab30 0x5595774a4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:39402/0 (socket says 172.21.15.78:39402) 2024-08-18T23:48:36.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.571+0000 7f3a819d2640 1 -- 172.21.15.78:0/3506502121 learned_addr learned my addr 172.21.15.78:0/3506502121 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:36.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.571+0000 7f3a819d2640 1 -- 172.21.15.78:0/3506502121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55957749ab30 msgr2=0x5595774a4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:36.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.571+0000 7f3a819d2640 1 --2- 172.21.15.78:0/3506502121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55957749ab30 0x5595774a4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.571+0000 7f3a819d2640 1 -- 172.21.15.78:0/3506502121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55957749a210 msgr2=0x55957749a5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T23:48:36.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.571+0000 7f3a819d2640 1 --2- 172.21.15.78:0/3506502121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55957749a210 0x55957749a5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.571+0000 7f3a819d2640 1 -- 172.21.15.78:0/3506502121 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595773fa6e0 con 0x5595774a53f0 2024-08-18T23:48:36.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.572+0000 7f3a819d2640 1 --2- 172.21.15.78:0/3506502121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595774a53f0 0x5595774a77e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009f90 tx=0x7f3a7400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed864352dd46b4c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:36.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.572+0000 7f3a677fe640 1 -- 172.21.15.78:0/3506502121 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74003b70 con 0x5595774a53f0 2024-08-18T23:48:36.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.572+0000 7f3a677fe640 1 -- 172.21.15.78:0/3506502121 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a74003d10 con 0x5595774a53f0 2024-08-18T23:48:36.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.572+0000 7f3a677fe640 1 -- 172.21.15.78:0/3506502121 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74033a70 con 0x5595774a53f0 2024-08-18T23:48:36.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.574+0000 7f3a85833740 1 -- 172.21.15.78:0/3506502121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595774a53f0 msgr2=0x5595774a77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:36.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.574+0000 7f3a85833740 1 --2- 172.21.15.78:0/3506502121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595774a53f0 0x5595774a77e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009f90 tx=0x7f3a7400f7f0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.576+0000 7f3a85833740 1 -- 172.21.15.78:0/3506502121 shutdown_connections 2024-08-18T23:48:36.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.576+0000 7f3a85833740 1 --2- 172.21.15.78:0/3506502121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5595774a53f0 0x5595774a77e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.576+0000 7f3a85833740 1 --2- 172.21.15.78:0/3506502121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55957749ab30 0x5595774a4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.576+0000 7f3a85833740 1 --2- 172.21.15.78:0/3506502121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55957749a210 0x55957749a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.576+0000 7f3a85833740 1 -- 172.21.15.78:0/3506502121 >> 172.21.15.78:0/3506502121 conn(0x5595774967a0 msgr2=0x559577496b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:36.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.583+0000 7f3a85833740 1 -- 172.21.15.78:0/3506502121 shutdown_connections 2024-08-18T23:48:36.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.583+0000 7f3a85833740 1 -- 172.21.15.78:0/3506502121 wait complete. 2024-08-18T23:48:36.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.583+0000 7f3a85833740 1 -- start start 2024-08-18T23:48:36.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.583+0000 7f3a85833740 1 Processor -- start 2024-08-18T23:48:36.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.584+0000 7f3a85833740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55957749a210 0x5595774a2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:36.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.584+0000 7f3a85833740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55957749ab30 0x5595774a2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:36.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.584+0000 7f3a85833740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595774a53f0 0x5595774a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:36.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.585+0000 7f3a85833740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5595774a7fc0 con 0x55957749ab30 2024-08-18T23:48:36.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.585+0000 7f3a85833740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5595773baaa0 con 0x55957749a210 2024-08-18T23:48:36.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.585+0000 7f3a85833740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5595774a8140 con 0x5595774a53f0 2024-08-18T23:48:36.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.585+0000 7f3a819d2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595774a53f0 0x5595774a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:36.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.585+0000 7f3a819d2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595774a53f0 0x5595774a2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:39404/0 (socket says 172.21.15.78:39404) 2024-08-18T23:48:36.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.585+0000 7f3a819d2640 1 -- 172.21.15.78:0/3969641823 learned_addr learned my addr 172.21.15.78:0/3969641823 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:36.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.585+0000 7f3a811d1640 1 --2- 172.21.15.78:0/3969641823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55957749a210 0x5595774a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:36.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.586+0000 7f3a819d2640 1 -- 172.21.15.78:0/3969641823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55957749a210 msgr2=0x5595774a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:36.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.586+0000 7f3a819d2640 1 --2- 172.21.15.78:0/3969641823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55957749a210 0x5595774a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.586+0000 7f3a819d2640 1 -- 172.21.15.78:0/3969641823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55957749ab30 msgr2=0x5595774a2850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T23:48:36.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.586+0000 7f3a819d2640 1 --2- 172.21.15.78:0/3969641823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55957749ab30 0x5595774a2850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.586+0000 7f3a819d2640 1 -- 172.21.15.78:0/3969641823 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55957739e650 con 0x5595774a53f0 2024-08-18T23:48:36.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.586+0000 7f3a819d2640 1 --2- 172.21.15.78:0/3969641823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595774a53f0 0x5595774a2d90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3a74000f80 tx=0x7f3a74009d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dfea84b3bbd11cf server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:36.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.586+0000 7f3a85833740 10 client.0 _ll_register_callbacks cb 0x5595773a9e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:48:36.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.586+0000 7f3a85833740 1 -- 172.21.15.78:0/3969641823 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a74003d10 con 0x5595774a53f0 2024-08-18T23:48:36.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.587+0000 7f3a64ff9640 10 client.25468 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:48:36.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.587+0000 7f3a64ff9640 1 -- 172.21.15.78:0/3969641823 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74004a40 con 0x5595774a53f0 2024-08-18T23:48:36.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.587+0000 7f3a64ff9640 1 -- 172.21.15.78:0/3969641823 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f3a74004be0 con 0x5595774a53f0 2024-08-18T23:48:36.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.587+0000 7f3a64ff9640 1 -- 172.21.15.78:0/3969641823 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3a74050050 con 0x5595774a53f0 2024-08-18T23:48:36.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.592+0000 7f3a85833740 10 client.25468 fetch_fsmap learned FSMap version 27 2024-08-18T23:48:36.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.592+0000 7f3a85833740 1 -- 172.21.15.78:0/3969641823 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5595771713c0 con 0x5595774a53f0 2024-08-18T23:48:36.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.596+0000 7f3a64ff9640 1 -- 172.21.15.78:0/3969641823 <== mon.1 v2:172.21.15.161:3300/0 4 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f3a740337f0 con 0x5595774a53f0 2024-08-18T23:48:36.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.596+0000 7f3a85833740 10 client.25468 fetch_fsmap finished waiting for FSMap version 27 2024-08-18T23:48:36.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.596+0000 7f3a85833740 10 client.25468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:48:36.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.596+0000 7f3a85833740 20 client.25468 populate_metadata read hostname 'smithi078' 2024-08-18T23:48:36.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.596+0000 7f3a85833740 1 --2- 172.21.15.78:0/3969641823 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5595774b4be0 0x5595774b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:36.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.596+0000 7f3a85833740 4 client.25468 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:48:36.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.596+0000 7f3a85833740 1 -- 172.21.15.78:0/3969641823 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3a74003d10 con 0x5595774b4be0 2024-08-18T23:48:36.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.600+0000 7f3a811d1640 1 --2- 172.21.15.78:0/3969641823 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5595774b4be0 0x5595774b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:36.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.608+0000 7f3a811d1640 1 --2- 172.21.15.78:0/3969641823 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5595774b4be0 0x5595774b6fd0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17fc87d92883eba1 server_cookie=9a0722262c8f100 in_seq=0 out_seq=0 2024-08-18T23:48:36.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.608+0000 7f3a64ff9640 10 client.25468 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:36.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.612+0000 7f3a64ff9640 1 -- 172.21.15.78:0/3969641823 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f3a74003d10 con 0x5595774b4be0 2024-08-18T23:48:36.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.612+0000 7f3a64ff9640 10 client.25468 handle_command_reply: tid=0 2024-08-18T23:48:36.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.612+0000 7f3a85833740 1 client.25468 shutdown 2024-08-18T23:48:36.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.618+0000 7f3a85833740 1 -- 172.21.15.78:0/3969641823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595774a53f0 msgr2=0x5595774a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:36.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.618+0000 7f3a85833740 1 --2- 172.21.15.78:0/3969641823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595774a53f0 0x5595774a2d90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3a74000f80 tx=0x7f3a74009d50 comp rx=0 tx=0).stop 2024-08-18T23:48:36.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.622+0000 7f3a85833740 1 -- 172.21.15.78:0/3969641823 shutdown_connections 2024-08-18T23:48:36.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.622+0000 7f3a85833740 1 --2- 172.21.15.78:0/3969641823 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5595774b4be0 0x5595774b6fd0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.622+0000 7f3a85833740 1 --2- 172.21.15.78:0/3969641823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595774a53f0 0x5595774a2d90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.622+0000 7f3a85833740 1 --2- 172.21.15.78:0/3969641823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55957749ab30 0x5595774a2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.622+0000 7f3a85833740 1 --2- 172.21.15.78:0/3969641823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55957749a210 0x5595774a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.622+0000 7f3a85833740 1 -- 172.21.15.78:0/3969641823 >> 172.21.15.78:0/3969641823 conn(0x5595774967a0 msgr2=0x55957749c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:36.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.623+0000 7f3a64ff9640 0 client.25468 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:36.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.623+0000 7f3a85833740 1 -- 172.21.15.78:0/3969641823 shutdown_connections 2024-08-18T23:48:36.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.627+0000 7f3a85833740 1 -- 172.21.15.78:0/3969641823 wait complete. 2024-08-18T23:48:36.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.629+0000 7f3a85833740 20 client.25468 trim_cache size 0 max 16384 2024-08-18T23:48:36.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.668+0000 7f3a85833740 1 Processor -- start 2024-08-18T23:48:36.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.668+0000 7f3a85833740 1 -- start start 2024-08-18T23:48:36.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.668+0000 7f3a85833740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55957749a210 0x55957749cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:36.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.668+0000 7f3a85833740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55957749ab30 0x5595774a2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:36.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.668+0000 7f3a85833740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595774a53f0 0x5595774a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:36.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.668+0000 7f3a85833740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1c8a00 con 0x55957749a210 2024-08-18T23:48:36.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.668+0000 7f3a85833740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1c62d0 con 0x55957749ab30 2024-08-18T23:48:36.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.668+0000 7f3a85833740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1c4120 con 0x5595774a53f0 2024-08-18T23:48:36.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.668+0000 7f3a809d0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55957749ab30 0x5595774a2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:36.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.668+0000 7f3a809d0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55957749ab30 0x5595774a2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:33482/0 (socket says 172.21.15.78:33482) 2024-08-18T23:48:36.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.668+0000 7f3a809d0640 1 -- 172.21.15.78:0/1383779995 learned_addr learned my addr 172.21.15.78:0/1383779995 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:36.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.669+0000 7f3a819d2640 1 --2- 172.21.15.78:0/1383779995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595774a53f0 0x5595774a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:36.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.669+0000 7f3a809d0640 1 -- 172.21.15.78:0/1383779995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595774a53f0 msgr2=0x5595774a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:36.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.669+0000 7f3a809d0640 1 --2- 172.21.15.78:0/1383779995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595774a53f0 0x5595774a2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.669+0000 7f3a809d0640 1 -- 172.21.15.78:0/1383779995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55957749a210 msgr2=0x55957749cc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T23:48:36.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.669+0000 7f3a809d0640 1 --2- 172.21.15.78:0/1383779995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55957749a210 0x55957749cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.669+0000 7f3a809d0640 1 -- 172.21.15.78:0/1383779995 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595774a4990 con 0x55957749ab30 2024-08-18T23:48:36.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.669+0000 7f3a809d0640 1 --2- 172.21.15.78:0/1383779995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55957749ab30 0x5595774a2860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c073580 tx=0x7f3a6c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6c09c2a6399ac6f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:36.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.670+0000 7f3a66ffd640 1 -- 172.21.15.78:0/1383779995 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c01f900 con 0x55957749ab30 2024-08-18T23:48:36.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.670+0000 7f3a66ffd640 1 -- 172.21.15.78:0/1383779995 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a6c0273e0 con 0x55957749ab30 2024-08-18T23:48:36.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.670+0000 7f3a66ffd640 1 -- 172.21.15.78:0/1383779995 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c01a600 con 0x55957749ab30 2024-08-18T23:48:36.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.671+0000 7f3a85833740 1 -- 172.21.15.78:0/1383779995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55957749ab30 msgr2=0x5595774a2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:36.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.671+0000 7f3a85833740 1 --2- 172.21.15.78:0/1383779995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55957749ab30 0x5595774a2860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c073580 tx=0x7f3a6c013000 comp rx=0 tx=0).stop 2024-08-18T23:48:36.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.671+0000 7f3a85833740 1 -- 172.21.15.78:0/1383779995 shutdown_connections 2024-08-18T23:48:36.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.671+0000 7f3a85833740 1 --2- 172.21.15.78:0/1383779995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595774a53f0 0x5595774a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.671+0000 7f3a85833740 1 --2- 172.21.15.78:0/1383779995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55957749ab30 0x5595774a2860 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.671+0000 7f3a85833740 1 --2- 172.21.15.78:0/1383779995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55957749a210 0x55957749cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.671+0000 7f3a85833740 1 -- 172.21.15.78:0/1383779995 >> 172.21.15.78:0/1383779995 conn(0x5595774967a0 msgr2=0x5595774a4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:36.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.674+0000 7f3a85833740 1 -- 172.21.15.78:0/1383779995 shutdown_connections 2024-08-18T23:48:36.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.674+0000 7f3a85833740 1 -- 172.21.15.78:0/1383779995 wait complete. 2024-08-18T23:48:36.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.674+0000 7f3a85833740 1 -- start start 2024-08-18T23:48:36.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.674+0000 7f3a85833740 1 Processor -- start 2024-08-18T23:48:36.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.675+0000 7f3a85833740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55957749a210 0x5595774ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:36.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.675+0000 7f3a85833740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55957749ab30 0x55957749edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:36.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.675+0000 7f3a85833740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595774a53f0 0x55957749f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:36.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.675+0000 7f3a85833740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1c4790 con 0x55957749a210 2024-08-18T23:48:36.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.675+0000 7f3a85833740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1c5c00 con 0x5595774a53f0 2024-08-18T23:48:36.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.675+0000 7f3a85833740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1c4e60 con 0x55957749ab30 2024-08-18T23:48:36.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.675+0000 7f3a85833740 10 client.0 _ll_register_callbacks cb 0x7f3a74037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:48:36.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.677+0000 7f3a811d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55957749a210 0x5595774ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:36.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.677+0000 7f3a819d2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595774a53f0 0x55957749f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:36.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.677+0000 7f3a811d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55957749a210 0x5595774ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39162/0 (socket says 172.21.15.78:39162) 2024-08-18T23:48:36.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.677+0000 7f3a811d1640 1 -- 172.21.15.78:0/1507964104 learned_addr learned my addr 172.21.15.78:0/1507964104 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:36.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.677+0000 7f3a809d0640 1 --2- 172.21.15.78:0/1507964104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55957749ab30 0x55957749edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:36.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.678+0000 7f3a811d1640 1 -- 172.21.15.78:0/1507964104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55957749ab30 msgr2=0x55957749edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:36.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.678+0000 7f3a811d1640 1 --2- 172.21.15.78:0/1507964104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55957749ab30 0x55957749edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.678+0000 7f3a811d1640 1 -- 172.21.15.78:0/1507964104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595774a53f0 msgr2=0x55957749f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:36.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.678+0000 7f3a811d1640 1 --2- 172.21.15.78:0/1507964104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595774a53f0 0x55957749f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.678+0000 7f3a811d1640 1 -- 172.21.15.78:0/1507964104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55957749f860 con 0x55957749a210 2024-08-18T23:48:36.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.678+0000 7f3a811d1640 1 --2- 172.21.15.78:0/1507964104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55957749a210 0x5595774ad8a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3a780b8b80 tx=0x7f3a7802a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa00b394a037647 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:36.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.679+0000 7f3a64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:48:36.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.679+0000 7f3a64ff9640 1 -- 172.21.15.78:0/1507964104 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a780d51a0 con 0x55957749a210 2024-08-18T23:48:36.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.679+0000 7f3a85833740 1 -- 172.21.15.78:0/1507964104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a6c0273e0 con 0x55957749a210 2024-08-18T23:48:36.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.680+0000 7f3a64ff9640 1 -- 172.21.15.78:0/1507964104 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f3a780ba250 con 0x55957749a210 2024-08-18T23:48:36.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.680+0000 7f3a64ff9640 1 -- 172.21.15.78:0/1507964104 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3a780cd050 con 0x55957749a210 2024-08-18T23:48:36.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.681+0000 7f3a85833740 10 client.16191 fetch_fsmap learned FSMap version 27 2024-08-18T23:48:36.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.681+0000 7f3a85833740 1 -- 172.21.15.78:0/1507964104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55957751f3a0 con 0x55957749a210 2024-08-18T23:48:36.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.681+0000 7f3a64ff9640 1 -- 172.21.15.78:0/1507964104 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a6c0273e0 con 0x55957749a210 2024-08-18T23:48:36.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.681+0000 7f3a64ff9640 1 -- 172.21.15.78:0/1507964104 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f3a7804d060 con 0x55957749a210 2024-08-18T23:48:36.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.681+0000 7f3a85833740 10 client.16191 fetch_fsmap finished waiting for FSMap version 27 2024-08-18T23:48:36.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.681+0000 7f3a85833740 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:48:36.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.681+0000 7f3a85833740 20 client.16191 populate_metadata read hostname 'smithi078' 2024-08-18T23:48:36.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.681+0000 7f3a85833740 1 --2- 172.21.15.78:0/1507964104 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55957751fad0 0x559577521ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:36.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.682+0000 7f3a85833740 4 client.16191 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:48:36.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.682+0000 7f3a85833740 1 -- 172.21.15.78:0/1507964104 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559577497450 con 0x55957751fad0 2024-08-18T23:48:36.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.682+0000 7f3a809d0640 1 --2- 172.21.15.78:0/1507964104 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55957751fad0 0x559577521ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:36.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.683+0000 7f3a809d0640 1 --2- 172.21.15.78:0/1507964104 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55957751fad0 0x559577521ef0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1db85bd26e47a27c server_cookie=33fb69a72dbdb7e4 in_seq=0 out_seq=0 2024-08-18T23:48:36.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.683+0000 7f3a64ff9640 10 client.16191 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:36.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.687+0000 7f3a64ff9640 1 -- 172.21.15.78:0/1507964104 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559577497450 con 0x55957751fad0 2024-08-18T23:48:36.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.687+0000 7f3a64ff9640 10 client.16191 handle_command_reply: tid=0 2024-08-18T23:48:36.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.687+0000 7f3a85833740 1 client.16191 shutdown 2024-08-18T23:48:36.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.691+0000 7f3a85833740 1 -- 172.21.15.78:0/1507964104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55957749a210 msgr2=0x5595774ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:36.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.691+0000 7f3a85833740 1 --2- 172.21.15.78:0/1507964104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55957749a210 0x5595774ad8a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3a780b8b80 tx=0x7f3a7802a440 comp rx=0 tx=0).stop 2024-08-18T23:48:36.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.693+0000 7f3a85833740 1 -- 172.21.15.78:0/1507964104 shutdown_connections 2024-08-18T23:48:36.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.693+0000 7f3a85833740 1 --2- 172.21.15.78:0/1507964104 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55957751fad0 0x559577521ef0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.693+0000 7f3a85833740 1 --2- 172.21.15.78:0/1507964104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595774a53f0 0x55957749f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.693+0000 7f3a85833740 1 --2- 172.21.15.78:0/1507964104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55957749ab30 0x55957749edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.693+0000 7f3a85833740 1 --2- 172.21.15.78:0/1507964104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55957749a210 0x5595774ad8a0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.693+0000 7f3a85833740 1 -- 172.21.15.78:0/1507964104 >> 172.21.15.78:0/1507964104 conn(0x5595774967a0 msgr2=0x5595774b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:36.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.693+0000 7f3a64ff9640 0 client.16191 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:36.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.697+0000 7f3a85833740 1 -- 172.21.15.78:0/1507964104 shutdown_connections 2024-08-18T23:48:36.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.697+0000 7f3a85833740 1 -- 172.21.15.78:0/1507964104 wait complete. 2024-08-18T23:48:36.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.698+0000 7f3a85833740 20 client.16191 trim_cache size 0 max 16384 2024-08-18T23:48:36.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.703+0000 7f3a83435640 1 -- 172.21.15.78:0/840402152 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3a680a2d90 msgr2=0x7f3a680a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:36.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.704+0000 7f3a83435640 1 --2- 172.21.15.78:0/840402152 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3a680a2d90 0x7f3a680a5250 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c077710 tx=0x7f3a6c00b410 comp rx=0 tx=0).stop 2024-08-18T23:48:36.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.704+0000 7f3a83435640 1 -- 172.21.15.78:0/840402152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a7c0746c0 msgr2=0x7f3a7c07aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:36.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.704+0000 7f3a83435640 1 --2- 172.21.15.78:0/840402152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a7c0746c0 0x7f3a7c07aec0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3a78005ab0 tx=0x7f3a78005230 comp rx=0 tx=0).stop 2024-08-18T23:48:36.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.704+0000 7f3a83435640 1 -- 172.21.15.78:0/840402152 shutdown_connections 2024-08-18T23:48:36.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.704+0000 7f3a83435640 1 --2- 172.21.15.78:0/840402152 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3a680a2d90 0x7f3a680a5250 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.704+0000 7f3a83435640 1 --2- 172.21.15.78:0/840402152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a7c109b30 0x7f3a7c076490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.704+0000 7f3a83435640 1 --2- 172.21.15.78:0/840402152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a7c109180 0x7f3a7c075f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.704+0000 7f3a83435640 1 --2- 172.21.15.78:0/840402152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a7c0746c0 0x7f3a7c07aec0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:36.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.704+0000 7f3a83435640 1 -- 172.21.15.78:0/840402152 >> 172.21.15.78:0/840402152 conn(0x7f3a7c06de70 msgr2=0x7f3a7c072250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:36.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.704+0000 7f3a83435640 1 -- 172.21.15.78:0/840402152 shutdown_connections 2024-08-18T23:48:36.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:36.705+0000 7f3a83435640 1 -- 172.21.15.78:0/840402152 wait complete. 2024-08-18T23:48:36.983 INFO:tasks.workunit.client.0.smithi078.stdout: 90 1776502 448 1241638 506 911040 1428 2024-08-18T23:48:37.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:37 smithi123 ceph-mon[21808]: pgmap v479: 129 pgs: 129 active+clean; 507 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 256 op/s 2024-08-18T23:48:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:37 smithi161 ceph-mon[22027]: pgmap v479: 129 pgs: 129 active+clean; 507 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 256 op/s 2024-08-18T23:48:37.705 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:37 smithi078 ceph-mon[18440]: pgmap v479: 129 pgs: 129 active+clean; 507 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 256 op/s 2024-08-18T23:48:38.083 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-18T23:48:38.083 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:48:39.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:39 smithi123 ceph-mon[21808]: pgmap v480: 129 pgs: 129 active+clean; 505 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 319 op/s 2024-08-18T23:48:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:39 smithi161 ceph-mon[22027]: pgmap v480: 129 pgs: 129 active+clean; 505 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 319 op/s 2024-08-18T23:48:39.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:39 smithi078 ceph-mon[18440]: pgmap v480: 129 pgs: 129 active+clean; 505 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 319 op/s 2024-08-18T23:48:40.200 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:48:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.088+0000 7fbc04502640 1 -- 172.21.15.78:0/2785177342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc1078c0 msgr2=0x7fbbfc107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.088+0000 7fbc04502640 1 --2- 172.21.15.78:0/2785177342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc1078c0 0x7fbbfc107ca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009860 tx=0x7fbbf802a360 comp rx=0 tx=0).stop 2024-08-18T23:48:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.093+0000 7fbc04502640 1 -- 172.21.15.78:0/2785177342 shutdown_connections 2024-08-18T23:48:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.093+0000 7fbc04502640 1 --2- 172.21.15.78:0/2785177342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbfc102260 0x7fbbfc10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.093+0000 7fbc04502640 1 --2- 172.21.15.78:0/2785177342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbfc1018c0 0x7fbbfc101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.093+0000 7fbc04502640 1 --2- 172.21.15.78:0/2785177342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc1078c0 0x7fbbfc107ca0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.093+0000 7fbc04502640 1 -- 172.21.15.78:0/2785177342 >> 172.21.15.78:0/2785177342 conn(0x7fbbfc0fd620 msgr2=0x7fbbfc0ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:41.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.104+0000 7fbc04502640 1 -- 172.21.15.78:0/2785177342 shutdown_connections 2024-08-18T23:48:41.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.105+0000 7fbc04502640 1 -- 172.21.15.78:0/2785177342 wait complete. 2024-08-18T23:48:41.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.106+0000 7fbc04502640 1 Processor -- start 2024-08-18T23:48:41.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.106+0000 7fbc04502640 1 -- start start 2024-08-18T23:48:41.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.106+0000 7fbc04502640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc1018c0 0x7fbbfc10e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:41.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.106+0000 7fbc04502640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbfc102260 0x7fbbfc074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:41.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.106+0000 7fbc04502640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbfc1078c0 0x7fbbfc074690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:41.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.106+0000 7fbc04502640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc06e230 con 0x7fbbfc1078c0 2024-08-18T23:48:41.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.106+0000 7fbc04502640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc06e0b0 con 0x7fbbfc1018c0 2024-08-18T23:48:41.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.106+0000 7fbc04502640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc06e3b0 con 0x7fbbfc102260 2024-08-18T23:48:41.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.107+0000 7fbc01a9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbfc102260 0x7fbbfc074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:41.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.107+0000 7fbc01a9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbfc102260 0x7fbbfc074150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:39440/0 (socket says 172.21.15.78:39440) 2024-08-18T23:48:41.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.107+0000 7fbc01a9d640 1 -- 172.21.15.78:0/3972404432 learned_addr learned my addr 172.21.15.78:0/3972404432 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:41.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.108+0000 7fbc02a9f640 1 --2- 172.21.15.78:0/3972404432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbfc1078c0 0x7fbbfc074690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:41.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.109+0000 7fbc0229e640 1 --2- 172.21.15.78:0/3972404432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc1018c0 0x7fbbfc10e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:41.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.110+0000 7fbc0229e640 1 -- 172.21.15.78:0/3972404432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbfc102260 msgr2=0x7fbbfc074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:41.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.110+0000 7fbc0229e640 1 --2- 172.21.15.78:0/3972404432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbfc102260 0x7fbbfc074150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:41.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.110+0000 7fbc0229e640 1 -- 172.21.15.78:0/3972404432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbfc1078c0 msgr2=0x7fbbfc074690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:41.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.110+0000 7fbc0229e640 1 --2- 172.21.15.78:0/3972404432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbfc1078c0 0x7fbbfc074690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:41.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.110+0000 7fbc0229e640 1 -- 172.21.15.78:0/3972404432 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbfc074bd0 con 0x7fbbfc1018c0 2024-08-18T23:48:41.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.113+0000 7fbc0229e640 1 --2- 172.21.15.78:0/3972404432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc1018c0 0x7fbbfc10e3f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbbf802a880 tx=0x7fbbf80326f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:41.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.114+0000 7fbbf37fe640 1 -- 172.21.15.78:0/3972404432 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8037040 con 0x7fbbfc1018c0 2024-08-18T23:48:41.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.114+0000 7fbc04502640 1 -- 172.21.15.78:0/3972404432 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbfc074e60 con 0x7fbbfc1018c0 2024-08-18T23:48:41.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.114+0000 7fbc04502640 1 -- 172.21.15.78:0/3972404432 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbfc1a5270 con 0x7fbbfc1018c0 2024-08-18T23:48:41.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.116+0000 7fbbf37fe640 1 -- 172.21.15.78:0/3972404432 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbf8032cd0 con 0x7fbbfc1018c0 2024-08-18T23:48:41.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.116+0000 7fbbf37fe640 1 -- 172.21.15.78:0/3972404432 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8003950 con 0x7fbbfc1018c0 2024-08-18T23:48:41.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.118+0000 7fbc04502640 1 -- 172.21.15.78:0/3972404432 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe0005290 con 0x7fbbfc1018c0 2024-08-18T23:48:41.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.121+0000 7fbbf37fe640 1 -- 172.21.15.78:0/3972404432 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbbf8004050 con 0x7fbbfc1018c0 2024-08-18T23:48:41.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.122+0000 7fbbf37fe640 1 --2- 172.21.15.78:0/3972404432 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbbe80a2d50 0x7fbbe80a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:41.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.122+0000 7fbbf37fe640 1 -- 172.21.15.78:0/3972404432 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fbbf80eec50 con 0x7fbbfc1018c0 2024-08-18T23:48:41.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.125+0000 7fbc01a9d640 1 --2- 172.21.15.78:0/3972404432 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbbe80a2d50 0x7fbbe80a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:41.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.128+0000 7fbbf37fe640 1 -- 172.21.15.78:0/3972404432 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbbf8049020 con 0x7fbbfc1018c0 2024-08-18T23:48:41.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.163+0000 7fbc01a9d640 1 --2- 172.21.15.78:0/3972404432 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbbe80a2d50 0x7fbbe80a5210 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0758c0 tx=0x7fbbec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:41 smithi078 ceph-mon[18440]: pgmap v481: 129 pgs: 129 active+clean; 510 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 361 op/s 2024-08-18T23:48:41.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:41 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:48:41.565 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:48:41.565 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-08-18T23:48:13:304168+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:48:12.895899+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24428,24427]},"id":1}]} 2024-08-18T23:48:41.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.561+0000 7fbc04502640 1 -- 172.21.15.78:0/3972404432 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbe0005580 con 0x7fbbfc1018c0 2024-08-18T23:48:41.565 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-08-18T23:48:41.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.563+0000 7fbbf37fe640 1 -- 172.21.15.78:0/3972404432 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11931 (secure 0 0 0) 0x7fbbf8004340 con 0x7fbbfc1018c0 2024-08-18T23:48:41.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.569+0000 7fbc04502640 1 -- 172.21.15.78:0/3972404432 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbbe80a2d50 msgr2=0x7fbbe80a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:41.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.569+0000 7fbc04502640 1 --2- 172.21.15.78:0/3972404432 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbbe80a2d50 0x7fbbe80a5210 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0758c0 tx=0x7fbbec009260 comp rx=0 tx=0).stop 2024-08-18T23:48:41.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.569+0000 7fbc04502640 1 -- 172.21.15.78:0/3972404432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc1018c0 msgr2=0x7fbbfc10e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:41.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.569+0000 7fbc04502640 1 --2- 172.21.15.78:0/3972404432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc1018c0 0x7fbbfc10e3f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbbf802a880 tx=0x7fbbf80326f0 comp rx=0 tx=0).stop 2024-08-18T23:48:41.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.573+0000 7fbc04502640 1 -- 172.21.15.78:0/3972404432 shutdown_connections 2024-08-18T23:48:41.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.573+0000 7fbc04502640 1 --2- 172.21.15.78:0/3972404432 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fbbe80a2d50 0x7fbbe80a5210 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:41.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.573+0000 7fbc04502640 1 --2- 172.21.15.78:0/3972404432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbfc1078c0 0x7fbbfc074690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:41.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.573+0000 7fbc04502640 1 --2- 172.21.15.78:0/3972404432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbfc102260 0x7fbbfc074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:41.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.573+0000 7fbc04502640 1 --2- 172.21.15.78:0/3972404432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc1018c0 0x7fbbfc10e3f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:41.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.573+0000 7fbc04502640 1 -- 172.21.15.78:0/3972404432 >> 172.21.15.78:0/3972404432 conn(0x7fbbfc0fd620 msgr2=0x7fbbfc0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:41.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.573+0000 7fbc04502640 1 -- 172.21.15.78:0/3972404432 shutdown_connections 2024-08-18T23:48:41.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:41.573+0000 7fbc04502640 1 -- 172.21.15.78:0/3972404432 wait complete. 2024-08-18T23:48:41.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:41 smithi161 ceph-mon[22027]: pgmap v481: 129 pgs: 129 active+clean; 510 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 361 op/s 2024-08-18T23:48:41.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:41 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:48:41.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:41 smithi123 ceph-mon[21808]: pgmap v481: 129 pgs: 129 active+clean; 510 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 361 op/s 2024-08-18T23:48:41.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:41 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:48:42.543 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-18T23:48:42.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:42 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3972404432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:48:42.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:42 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3972404432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:48:42.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:42 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3972404432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:48:43.308 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:48:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:43 smithi161 ceph-mon[22027]: pgmap v482: 129 pgs: 129 active+clean; 517 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 395 op/s 2024-08-18T23:48:43.684 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:43 smithi078 ceph-mon[18440]: pgmap v482: 129 pgs: 129 active+clean; 517 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 395 op/s 2024-08-18T23:48:43.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:43 smithi123 ceph-mon[21808]: pgmap v482: 129 pgs: 129 active+clean; 517 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 395 op/s 2024-08-18T23:48:43.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.941+0000 7f2b0a3bb640 1 -- 172.21.15.78:0/1590483389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b0410a140 msgr2=0x7f2b0410a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:43.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.941+0000 7f2b0a3bb640 1 --2- 172.21.15.78:0/1590483389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b0410a140 0x7f2b0410a580 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2af4009900 tx=0x7f2af402a480 comp rx=0 tx=0).stop 2024-08-18T23:48:43.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.941+0000 7f2b0a3bb640 1 -- 172.21.15.78:0/1590483389 shutdown_connections 2024-08-18T23:48:43.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.941+0000 7f2b0a3bb640 1 --2- 172.21.15.78:0/1590483389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b0410aac0 0x7f2b04116f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:43.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.941+0000 7f2b0a3bb640 1 --2- 172.21.15.78:0/1590483389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b0410a140 0x7f2b0410a580 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:43.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.941+0000 7f2b0a3bb640 1 --2- 172.21.15.78:0/1590483389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b041101a0 0x7f2b04110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:43.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.941+0000 7f2b0a3bb640 1 -- 172.21.15.78:0/1590483389 >> 172.21.15.78:0/1590483389 conn(0x7f2b04071360 msgr2=0x7f2b04073780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:43.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.941+0000 7f2b0a3bb640 1 -- 172.21.15.78:0/1590483389 shutdown_connections 2024-08-18T23:48:43.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.941+0000 7f2b0a3bb640 1 -- 172.21.15.78:0/1590483389 wait complete. 2024-08-18T23:48:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.942+0000 7f2b0a3bb640 1 Processor -- start 2024-08-18T23:48:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.942+0000 7f2b0a3bb640 1 -- start start 2024-08-18T23:48:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.943+0000 7f2b0a3bb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b0410a140 0x7f2b0411e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.943+0000 7f2b0a3bb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b0410aac0 0x7f2b0411eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.943+0000 7f2b0a3bb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b041101a0 0x7f2b04119780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.943+0000 7f2b0a3bb640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041182e0 con 0x7f2b041101a0 2024-08-18T23:48:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.943+0000 7f2b0a3bb640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b04118160 con 0x7f2b0410aac0 2024-08-18T23:48:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.943+0000 7f2b0a3bb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2b04118460 con 0x7f2b0410a140 2024-08-18T23:48:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.943+0000 7f2b08958640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b041101a0 0x7f2b04119780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.943+0000 7f2b037fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b0410aac0 0x7f2b0411eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.943+0000 7f2b08958640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b041101a0 0x7f2b04119780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39208/0 (socket says 172.21.15.78:39208) 2024-08-18T23:48:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.943+0000 7f2b037fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b0410aac0 0x7f2b0411eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:33518/0 (socket says 172.21.15.78:33518) 2024-08-18T23:48:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.943+0000 7f2b08958640 1 -- 172.21.15.78:0/1746262324 learned_addr learned my addr 172.21.15.78:0/1746262324 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.943+0000 7f2b03fff640 1 --2- 172.21.15.78:0/1746262324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b0410a140 0x7f2b0411e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.944+0000 7f2b08958640 1 -- 172.21.15.78:0/1746262324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b0410a140 msgr2=0x7f2b0411e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.944+0000 7f2b08958640 1 --2- 172.21.15.78:0/1746262324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b0410a140 0x7f2b0411e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.944+0000 7f2b08958640 1 -- 172.21.15.78:0/1746262324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b0410aac0 msgr2=0x7f2b0411eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.944+0000 7f2b08958640 1 --2- 172.21.15.78:0/1746262324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b0410aac0 0x7f2b0411eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.944+0000 7f2b08958640 1 -- 172.21.15.78:0/1746262324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0411a040 con 0x7f2b041101a0 2024-08-18T23:48:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.944+0000 7f2b037fe640 1 --2- 172.21.15.78:0/1746262324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b0410aac0 0x7f2b0411eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:48:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.944+0000 7f2b08958640 1 --2- 172.21.15.78:0/1746262324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b041101a0 0x7f2b04119780 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2af800c970 tx=0x7f2af800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.944+0000 7f2b017fa640 1 -- 172.21.15.78:0/1746262324 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8007bb0 con 0x7f2b041101a0 2024-08-18T23:48:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.944+0000 7f2b017fa640 1 -- 172.21.15.78:0/1746262324 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2af8007d50 con 0x7f2b041101a0 2024-08-18T23:48:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.944+0000 7f2b017fa640 1 -- 172.21.15.78:0/1746262324 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8018800 con 0x7f2b041101a0 2024-08-18T23:48:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.944+0000 7f2b0a3bb640 1 -- 172.21.15.78:0/1746262324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0411a330 con 0x7f2b041101a0 2024-08-18T23:48:43.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.944+0000 7f2b0a3bb640 1 -- 172.21.15.78:0/1746262324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b04122370 con 0x7f2b041101a0 2024-08-18T23:48:43.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.948+0000 7f2b0c7b9740 1 Processor -- start 2024-08-18T23:48:43.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.948+0000 7f2b017fa640 1 -- 172.21.15.78:0/1746262324 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2af80213f0 con 0x7f2b041101a0 2024-08-18T23:48:43.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.949+0000 7f2b0c7b9740 1 -- start start 2024-08-18T23:48:43.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.949+0000 7f2b0c7b9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c612dff0 0x5652c612e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:43.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.949+0000 7f2b0c7b9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c612e910 0x5652c6138c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:43.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.949+0000 7f2b0c7b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c61391d0 0x5652c613b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:43.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.949+0000 7f2b0c7b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5652c6093a40 con 0x5652c61391d0 2024-08-18T23:48:43.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.949+0000 7f2b0c7b9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5652c602b9d0 con 0x5652c612e910 2024-08-18T23:48:43.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.949+0000 7f2b0c7b9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5652c608fa00 con 0x5652c612dff0 2024-08-18T23:48:43.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.949+0000 7f2b037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c61391d0 0x5652c613b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:43.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.949+0000 7f2b037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c61391d0 0x5652c613b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39224/0 (socket says 172.21.15.78:39224) 2024-08-18T23:48:43.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.949+0000 7f2b037fe640 1 -- 172.21.15.78:0/752807608 learned_addr learned my addr 172.21.15.78:0/752807608 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:43.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.949+0000 7f2b03fff640 1 --2- 172.21.15.78:0/752807608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c612e910 0x5652c6138c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:43.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.949+0000 7f2b037fe640 1 --2- 172.21.15.78:0/752807608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c612dff0 0x5652c612e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:43.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.950+0000 7f2b03fff640 1 -- 172.21.15.78:0/752807608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c612dff0 msgr2=0x5652c612e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:43.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.950+0000 7f2b03fff640 1 --2- 172.21.15.78:0/752807608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c612dff0 0x5652c612e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:43.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.950+0000 7f2b03fff640 1 -- 172.21.15.78:0/752807608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c61391d0 msgr2=0x5652c613b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:43.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.950+0000 7f2b03fff640 1 --2- 172.21.15.78:0/752807608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c61391d0 0x5652c613b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:43.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.950+0000 7f2b03fff640 1 -- 172.21.15.78:0/752807608 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652c6032650 con 0x5652c612e910 2024-08-18T23:48:43.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.950+0000 7f2b03fff640 1 --2- 172.21.15.78:0/752807608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c612e910 0x5652c6138c90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009f30 tx=0x7f2afc010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afa3a1c08115047f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:43.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.950+0000 7f2b017fa640 1 --2- 172.21.15.78:0/1746262324 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2aec0a2c40 0x7f2aec0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:43.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.951+0000 7f2aea7fc640 1 -- 172.21.15.78:0/752807608 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc016070 con 0x5652c612e910 2024-08-18T23:48:43.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.951+0000 7f2aea7fc640 1 -- 172.21.15.78:0/752807608 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2afc004d20 con 0x5652c612e910 2024-08-18T23:48:43.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.951+0000 7f2aea7fc640 1 -- 172.21.15.78:0/752807608 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc005350 con 0x5652c612e910 2024-08-18T23:48:43.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.951+0000 7f2b037fe640 1 --2- 172.21.15.78:0/1746262324 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2aec0a2c40 0x7f2aec0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:43.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.951+0000 7f2b017fa640 1 -- 172.21.15.78:0/1746262324 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2af80cf310 con 0x7f2b041101a0 2024-08-18T23:48:43.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.952+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/752807608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c612e910 msgr2=0x5652c6138c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:43.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.952+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/752807608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c612e910 0x5652c6138c90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009f30 tx=0x7f2afc010800 comp rx=0 tx=0).stop 2024-08-18T23:48:43.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.952+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/752807608 shutdown_connections 2024-08-18T23:48:43.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.952+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/752807608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c61391d0 0x5652c613b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:43.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.952+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/752807608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c612e910 0x5652c6138c90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:43.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.952+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/752807608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c612dff0 0x5652c612e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:43.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.952+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/752807608 >> 172.21.15.78:0/752807608 conn(0x5652c612a580 msgr2=0x5652c612a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:43.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.952+0000 7f2b037fe640 1 --2- 172.21.15.78:0/1746262324 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2aec0a2c40 0x7f2aec0a5100 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2af4006010 tx=0x7f2af4005b90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:43.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.952+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/752807608 shutdown_connections 2024-08-18T23:48:43.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.952+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/752807608 wait complete. 2024-08-18T23:48:43.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.953+0000 7f2b0c7b9740 1 -- start start 2024-08-18T23:48:43.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.953+0000 7f2b0c7b9740 1 Processor -- start 2024-08-18T23:48:43.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.954+0000 7f2b0c7b9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c612dff0 0x5652c61360d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.954+0000 7f2b0c7b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c612e910 0x5652c6136610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.954+0000 7f2b0c7b9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c61391d0 0x5652c6136b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.954+0000 7f2b037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c612e910 0x5652c6136610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.954+0000 7f2b037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c612e910 0x5652c6136610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39228/0 (socket says 172.21.15.78:39228) 2024-08-18T23:48:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.954+0000 7f2b03fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c612dff0 0x5652c61360d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.954+0000 7f2b03fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c612dff0 0x5652c61360d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:39484/0 (socket says 172.21.15.78:39484) 2024-08-18T23:48:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.954+0000 7f2b037fe640 1 -- 172.21.15.78:0/3040161824 learned_addr learned my addr 172.21.15.78:0/3040161824 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.954+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/3040161824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5652c613bda0 con 0x5652c612e910 2024-08-18T23:48:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.954+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/3040161824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5652c604eb20 con 0x5652c61391d0 2024-08-18T23:48:43.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.954+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/3040161824 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5652c613bf20 con 0x5652c612dff0 2024-08-18T23:48:43.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.955+0000 7f2b08958640 1 --2- 172.21.15.78:0/3040161824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c61391d0 0x5652c6136b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:43.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.955+0000 7f2b0c7b9740 10 client.0 _ll_register_callbacks cb 0x5652c604cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:48:43.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.955+0000 7f2b03fff640 1 -- 172.21.15.78:0/3040161824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c61391d0 msgr2=0x5652c6136b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:43.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.955+0000 7f2b03fff640 1 --2- 172.21.15.78:0/3040161824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c61391d0 0x5652c6136b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:43.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.955+0000 7f2b03fff640 1 -- 172.21.15.78:0/3040161824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c612e910 msgr2=0x5652c6136610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:43.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.955+0000 7f2b03fff640 1 --2- 172.21.15.78:0/3040161824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c612e910 0x5652c6136610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:43.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.955+0000 7f2b03fff640 1 -- 172.21.15.78:0/3040161824 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652c5e053c0 con 0x5652c612dff0 2024-08-18T23:48:43.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.955+0000 7f2b037fe640 1 --2- 172.21.15.78:0/3040161824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c612e910 0x5652c6136610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:48:43.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.955+0000 7f2b03fff640 1 --2- 172.21.15.78:0/3040161824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c612dff0 0x5652c61360d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2afc000f80 tx=0x7f2afc004c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fea7e9c0092aea7f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:43.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.956+0000 7f2ae37fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:48:43.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.956+0000 7f2ae37fe640 1 -- 172.21.15.78:0/3040161824 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc016070 con 0x5652c612dff0 2024-08-18T23:48:43.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.956+0000 7f2ae37fe640 1 -- 172.21.15.78:0/3040161824 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2afc003970 con 0x5652c612dff0 2024-08-18T23:48:43.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.956+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/3040161824 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2afc004d20 con 0x5652c612dff0 2024-08-18T23:48:43.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.957+0000 7f2ae37fe640 1 -- 172.21.15.78:0/3040161824 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2afc003b70 con 0x5652c612dff0 2024-08-18T23:48:43.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.958+0000 7f2b0c7b9740 10 client.25489 fetch_fsmap learned FSMap version 28 2024-08-18T23:48:43.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.958+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/3040161824 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5652c61479b0 con 0x5652c612dff0 2024-08-18T23:48:43.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.959+0000 7f2ae37fe640 1 -- 172.21.15.78:0/3040161824 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2afc012c00 con 0x5652c612dff0 2024-08-18T23:48:43.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.959+0000 7f2ae37fe640 1 -- 172.21.15.78:0/3040161824 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2afc00f040 con 0x5652c612dff0 2024-08-18T23:48:43.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.959+0000 7f2b0c7b9740 10 client.25489 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:48:43.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.959+0000 7f2b0c7b9740 10 client.25489 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:48:43.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.959+0000 7f2b0c7b9740 20 client.25489 populate_metadata read hostname 'smithi078' 2024-08-18T23:48:43.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.959+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/3040161824 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5652c6148dc0 0x5652c614b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:43.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.959+0000 7f2b0c7b9740 4 client.25489 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:48:43.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.959+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/3040161824 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2afc004d20 con 0x5652c6148dc0 2024-08-18T23:48:43.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.960+0000 7f2b037fe640 1 --2- 172.21.15.78:0/3040161824 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5652c6148dc0 0x5652c614b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:43.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.961+0000 7f2b037fe640 1 --2- 172.21.15.78:0/3040161824 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5652c6148dc0 0x5652c614b1b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e00cf11763202d9a server_cookie=a8861c48d7400082 in_seq=0 out_seq=0 2024-08-18T23:48:43.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.961+0000 7f2ae37fe640 10 client.25489 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:48:43.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.962+0000 7f2ae37fe640 1 -- 172.21.15.78:0/3040161824 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f2afc004d20 con 0x5652c6148dc0 2024-08-18T23:48:43.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.962+0000 7f2ae37fe640 10 client.25489 handle_command_reply: tid=0 2024-08-18T23:48:43.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.963+0000 7f2b0c7b9740 1 client.25489 shutdown 2024-08-18T23:48:43.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.963+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/3040161824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c612dff0 msgr2=0x5652c61360d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:43.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.963+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/3040161824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c612dff0 0x5652c61360d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2afc000f80 tx=0x7f2afc004c70 comp rx=0 tx=0).stop 2024-08-18T23:48:43.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.963+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/3040161824 shutdown_connections 2024-08-18T23:48:43.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.963+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/3040161824 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5652c6148dc0 0x5652c614b1b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:43.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.963+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/3040161824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c61391d0 0x5652c6136b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:43.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.963+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/3040161824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c612e910 0x5652c6136610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:43.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.963+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/3040161824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c612dff0 0x5652c61360d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:43.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.963+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/3040161824 >> 172.21.15.78:0/3040161824 conn(0x5652c612a580 msgr2=0x5652c6130260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:43.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.963+0000 7f2ae37fe640 0 client.25489 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:48:43.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.963+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/3040161824 shutdown_connections 2024-08-18T23:48:43.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.964+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/3040161824 wait complete. 2024-08-18T23:48:43.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.964+0000 7f2b0c7b9740 20 client.25489 trim_cache size 0 max 16384 2024-08-18T23:48:43.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.992+0000 7f2b0c7b9740 1 Processor -- start 2024-08-18T23:48:43.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.992+0000 7f2b0c7b9740 1 -- start start 2024-08-18T23:48:43.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.992+0000 7f2b0c7b9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c612dff0 0x5652c61313d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:43.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.992+0000 7f2b0c7b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c612e910 0x5652c6131910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:43.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.992+0000 7f2b0c7b9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c61391d0 0x5652c6128d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:43.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.992+0000 7f2b0c7b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041c8830 con 0x5652c612e910 2024-08-18T23:48:43.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.992+0000 7f2b0c7b9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041c6100 con 0x5652c61391d0 2024-08-18T23:48:43.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.992+0000 7f2b0c7b9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041c3f50 con 0x5652c612dff0 2024-08-18T23:48:43.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.994+0000 7f2b037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c612e910 0x5652c6131910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:43.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.994+0000 7f2b037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c612e910 0x5652c6131910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39244/0 (socket says 172.21.15.78:39244) 2024-08-18T23:48:43.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.994+0000 7f2b037fe640 1 -- 172.21.15.78:0/2720674655 learned_addr learned my addr 172.21.15.78:0/2720674655 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:43.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.994+0000 7f2b08958640 1 --2- 172.21.15.78:0/2720674655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c61391d0 0x5652c6128d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:43.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.994+0000 7f2b03fff640 1 --2- 172.21.15.78:0/2720674655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c612dff0 0x5652c61313d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:43.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.995+0000 7f2b037fe640 1 -- 172.21.15.78:0/2720674655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c612dff0 msgr2=0x5652c61313d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:43.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.995+0000 7f2b037fe640 1 --2- 172.21.15.78:0/2720674655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c612dff0 0x5652c61313d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:43.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.995+0000 7f2b037fe640 1 -- 172.21.15.78:0/2720674655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c61391d0 msgr2=0x5652c6128d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:43.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.995+0000 7f2b037fe640 1 --2- 172.21.15.78:0/2720674655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c61391d0 0x5652c6128d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:43.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.995+0000 7f2b037fe640 1 -- 172.21.15.78:0/2720674655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652c612f500 con 0x5652c612e910 2024-08-18T23:48:43.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.996+0000 7f2b037fe640 1 --2- 172.21.15.78:0/2720674655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c612e910 0x5652c6131910 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2af4005c40 tx=0x7f2af4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae955672e2910dd server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:43.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.996+0000 7f2ae9ffb640 1 -- 172.21.15.78:0/2720674655 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4031b40 con 0x5652c612e910 2024-08-18T23:48:43.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.997+0000 7f2ae9ffb640 1 -- 172.21.15.78:0/2720674655 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2af4031ce0 con 0x5652c612e910 2024-08-18T23:48:43.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.997+0000 7f2ae9ffb640 1 -- 172.21.15.78:0/2720674655 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af403d740 con 0x5652c612e910 2024-08-18T23:48:44.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.999+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/2720674655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c612e910 msgr2=0x5652c6131910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:44.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:43.999+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/2720674655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c612e910 0x5652c6131910 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2af4005c40 tx=0x7f2af4009260 comp rx=0 tx=0).stop 2024-08-18T23:48:44.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.005+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/2720674655 shutdown_connections 2024-08-18T23:48:44.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.005+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/2720674655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c61391d0 0x5652c6128d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:44.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.005+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/2720674655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c612e910 0x5652c6131910 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:44.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.005+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/2720674655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c612dff0 0x5652c61313d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:44.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.005+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/2720674655 >> 172.21.15.78:0/2720674655 conn(0x5652c612a580 msgr2=0x5652c61251b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:44.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.010+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/2720674655 shutdown_connections 2024-08-18T23:48:44.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.011+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/2720674655 wait complete. 2024-08-18T23:48:44.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.011+0000 7f2b0c7b9740 1 -- start start 2024-08-18T23:48:44.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.011+0000 7f2b0c7b9740 1 Processor -- start 2024-08-18T23:48:44.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.012+0000 7f2b0c7b9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c612dff0 0x5652c6141360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:44.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.012+0000 7f2b0c7b9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c612e910 0x5652c61418a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:44.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.012+0000 7f2b0c7b9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c61391d0 0x5652c613c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:44.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.012+0000 7f2b0c7b9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041c45c0 con 0x5652c612e910 2024-08-18T23:48:44.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.012+0000 7f2b0c7b9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041c5a30 con 0x5652c612dff0 2024-08-18T23:48:44.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.012+0000 7f2b0c7b9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041c4c90 con 0x5652c61391d0 2024-08-18T23:48:44.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.012+0000 7f2b0c7b9740 10 client.0 _ll_register_callbacks cb 0x7f2af4009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:48:44.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.012+0000 7f2b03fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c612dff0 0x5652c6141360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:44.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.013+0000 7f2b08958640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c61391d0 0x5652c613c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:44.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.013+0000 7f2b03fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c612dff0 0x5652c6141360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:33552/0 (socket says 172.21.15.78:33552) 2024-08-18T23:48:44.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.013+0000 7f2b03fff640 1 -- 172.21.15.78:0/822304267 learned_addr learned my addr 172.21.15.78:0/822304267 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:44.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.015+0000 7f2b08958640 1 -- 172.21.15.78:0/822304267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c612dff0 msgr2=0x5652c6141360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:44.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.015+0000 7f2b08958640 1 --2- 172.21.15.78:0/822304267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c612dff0 0x5652c6141360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:44.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.015+0000 7f2b08958640 1 -- 172.21.15.78:0/822304267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c612e910 msgr2=0x5652c61418a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:44.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.015+0000 7f2b08958640 1 --2- 172.21.15.78:0/822304267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c612e910 0x5652c61418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:44.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.015+0000 7f2b08958640 1 -- 172.21.15.78:0/822304267 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652c613c960 con 0x5652c61391d0 2024-08-18T23:48:44.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.017+0000 7f2b08958640 1 --2- 172.21.15.78:0/822304267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c61391d0 0x5652c613c130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2af800ffd0 tx=0x7f2af809c210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38ef298f57672fae server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:44.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.019+0000 7f2ae37fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:48:44.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.019+0000 7f2ae37fe640 1 -- 172.21.15.78:0/822304267 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af80a0be0 con 0x5652c61391d0 2024-08-18T23:48:44.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.020+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/822304267 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2af4031ce0 con 0x5652c61391d0 2024-08-18T23:48:44.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.020+0000 7f2ae37fe640 1 -- 172.21.15.78:0/822304267 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2af80870b0 con 0x5652c61391d0 2024-08-18T23:48:44.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.020+0000 7f2ae37fe640 1 -- 172.21.15.78:0/822304267 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2af80a54a0 con 0x5652c61391d0 2024-08-18T23:48:44.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.023+0000 7f2b0c7b9740 10 client.25495 fetch_fsmap learned FSMap version 28 2024-08-18T23:48:44.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.023+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/822304267 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5652c61b2280 con 0x5652c61391d0 2024-08-18T23:48:44.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.025+0000 7f2ae37fe640 1 -- 172.21.15.78:0/822304267 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2af4031ce0 con 0x5652c61391d0 2024-08-18T23:48:44.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.025+0000 7f2ae37fe640 1 -- 172.21.15.78:0/822304267 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2af8020030 con 0x5652c61391d0 2024-08-18T23:48:44.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.025+0000 7f2b0c7b9740 10 client.25495 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:48:44.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.025+0000 7f2b0c7b9740 10 client.25495 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:48:44.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.025+0000 7f2b0c7b9740 20 client.25495 populate_metadata read hostname 'smithi078' 2024-08-18T23:48:44.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.025+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/822304267 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5652c61b2ac0 0x5652c61b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:44.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.025+0000 7f2b0c7b9740 4 client.25495 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:48:44.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.025+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/822304267 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652c60a0fb0 con 0x5652c61b2ac0 2024-08-18T23:48:44.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.029+0000 7f2b03fff640 1 --2- 172.21.15.78:0/822304267 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5652c61b2ac0 0x5652c61b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:44.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.045+0000 7f2ae37fe640 10 client.25495 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:48:44.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.045+0000 7f2b03fff640 1 --2- 172.21.15.78:0/822304267 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5652c61b2ac0 0x5652c61b4f20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c52d997a54bb2602 server_cookie=5eb74b5e2c8e8c92 in_seq=0 out_seq=0 2024-08-18T23:48:44.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.048+0000 7f2ae37fe640 1 -- 172.21.15.78:0/822304267 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652c60a0fb0 con 0x5652c61b2ac0 2024-08-18T23:48:44.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.049+0000 7f2ae37fe640 10 client.25495 handle_command_reply: tid=0 2024-08-18T23:48:44.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.049+0000 7f2b0c7b9740 1 client.25495 shutdown 2024-08-18T23:48:44.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.050+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/822304267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c61391d0 msgr2=0x5652c613c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:44.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.050+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/822304267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c61391d0 0x5652c613c130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2af800ffd0 tx=0x7f2af809c210 comp rx=0 tx=0).stop 2024-08-18T23:48:44.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.050+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/822304267 shutdown_connections 2024-08-18T23:48:44.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.050+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/822304267 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5652c61b2ac0 0x5652c61b4f20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:44.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.051+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/822304267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652c61391d0 0x5652c613c130 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:44.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.051+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/822304267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5652c612e910 0x5652c61418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:44.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.051+0000 7f2b0c7b9740 1 --2- 172.21.15.78:0/822304267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c612dff0 0x5652c6141360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:44.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.051+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/822304267 >> 172.21.15.78:0/822304267 conn(0x5652c612a580 msgr2=0x5652c61313d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:44.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.051+0000 7f2ae37fe640 0 client.25495 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:48:44.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.057+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/822304267 shutdown_connections 2024-08-18T23:48:44.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.060+0000 7f2b0c7b9740 1 -- 172.21.15.78:0/822304267 wait complete. 2024-08-18T23:48:44.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.061+0000 7f2b0c7b9740 20 client.25495 trim_cache size 0 max 16384 2024-08-18T23:48:44.062 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:48:44.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.062+0000 7f2b0a3bb640 1 -- 172.21.15.78:0/1746262324 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2aec0a2c40 msgr2=0x7f2aec0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:44.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.062+0000 7f2b0a3bb640 1 --2- 172.21.15.78:0/1746262324 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2aec0a2c40 0x7f2aec0a5100 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2af4006010 tx=0x7f2af4005b90 comp rx=0 tx=0).stop 2024-08-18T23:48:44.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.062+0000 7f2b0a3bb640 1 -- 172.21.15.78:0/1746262324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b041101a0 msgr2=0x7f2b04119780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:44.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.062+0000 7f2b0a3bb640 1 --2- 172.21.15.78:0/1746262324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b041101a0 0x7f2b04119780 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2af800c970 tx=0x7f2af800ce30 comp rx=0 tx=0).stop 2024-08-18T23:48:44.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.062+0000 7f2b0a3bb640 1 -- 172.21.15.78:0/1746262324 shutdown_connections 2024-08-18T23:48:44.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.062+0000 7f2b0a3bb640 1 --2- 172.21.15.78:0/1746262324 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2aec0a2c40 0x7f2aec0a5100 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:44.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.062+0000 7f2b0a3bb640 1 --2- 172.21.15.78:0/1746262324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b041101a0 0x7f2b04119780 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:44.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.062+0000 7f2b0a3bb640 1 --2- 172.21.15.78:0/1746262324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b0410aac0 0x7f2b0411eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:44.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.062+0000 7f2b0a3bb640 1 --2- 172.21.15.78:0/1746262324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b0410a140 0x7f2b0411e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:44.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.062+0000 7f2b0a3bb640 1 -- 172.21.15.78:0/1746262324 >> 172.21.15.78:0/1746262324 conn(0x7f2b04071360 msgr2=0x7f2b04072930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:44.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.062+0000 7f2b0a3bb640 1 -- 172.21.15.78:0/1746262324 shutdown_connections 2024-08-18T23:48:44.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:44.063+0000 7f2b0a3bb640 1 -- 172.21.15.78:0/1746262324 wait complete. 2024-08-18T23:48:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:44 smithi161 ceph-mon[22027]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:48:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:44 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:48:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:44 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:48:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:44 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:48:44.723 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:44 smithi078 ceph-mon[18440]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:48:44.723 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:44 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:48:44.723 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:44 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:48:44.723 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:44 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:48:44.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:44 smithi123 ceph-mon[21808]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:48:44.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:44 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:48:44.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:44 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:48:44.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:44 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:48:45.168 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-18T23:48:45.443 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:45 smithi078 ceph-mon[18440]: pgmap v483: 129 pgs: 129 active+clean; 529 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 495 op/s 2024-08-18T23:48:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:45 smithi161 ceph-mon[22027]: pgmap v483: 129 pgs: 129 active+clean; 529 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 495 op/s 2024-08-18T23:48:45.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:45 smithi123 ceph-mon[21808]: pgmap v483: 129 pgs: 129 active+clean; 529 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 495 op/s 2024-08-18T23:48:45.804 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:48:46.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.383+0000 7f5054e72640 1 -- 172.21.15.78:0/3460758101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5050100b50 msgr2=0x7f5050100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:46.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.383+0000 7f5054e72640 1 --2- 172.21.15.78:0/3460758101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5050100b50 0x7f5050100f30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5040009900 tx=0x7f504002a480 comp rx=0 tx=0).stop 2024-08-18T23:48:46.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.384+0000 7f5054e72640 1 -- 172.21.15.78:0/3460758101 shutdown_connections 2024-08-18T23:48:46.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.384+0000 7f5054e72640 1 --2- 172.21.15.78:0/3460758101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5050102c20 0x7f505010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.384+0000 7f5054e72640 1 --2- 172.21.15.78:0/3460758101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50501022a0 0x7f50501026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.384+0000 7f5054e72640 1 --2- 172.21.15.78:0/3460758101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5050100b50 0x7f5050100f30 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.384+0000 7f5054e72640 1 -- 172.21.15.78:0/3460758101 >> 172.21.15.78:0/3460758101 conn(0x7f50500fc500 msgr2=0x7f50500fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:46.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.384+0000 7f5054e72640 1 -- 172.21.15.78:0/3460758101 shutdown_connections 2024-08-18T23:48:46.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.384+0000 7f5054e72640 1 -- 172.21.15.78:0/3460758101 wait complete. 2024-08-18T23:48:46.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.384+0000 7f5054e72640 1 Processor -- start 2024-08-18T23:48:46.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.384+0000 7f5054e72640 1 -- start start 2024-08-18T23:48:46.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.385+0000 7f5054e72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5050100b50 0x7f5050198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:46.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.385+0000 7f5054e72640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50501022a0 0x7f5050198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:46.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.385+0000 7f5054e72640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5050102c20 0x7f505019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:46.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.385+0000 7f5054e72640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5050110c70 con 0x7f5050100b50 2024-08-18T23:48:46.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.385+0000 7f5054e72640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5050110af0 con 0x7f50501022a0 2024-08-18T23:48:46.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.385+0000 7f5054e72640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5050110df0 con 0x7f5050102c20 2024-08-18T23:48:46.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.385+0000 7f504e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5050100b50 0x7f5050198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:46.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.385+0000 7f504e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5050100b50 0x7f5050198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49524/0 (socket says 172.21.15.78:49524) 2024-08-18T23:48:46.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.385+0000 7f504e59c640 1 -- 172.21.15.78:0/3604251478 learned_addr learned my addr 172.21.15.78:0/3604251478 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:46.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.385+0000 7f504dd9b640 1 --2- 172.21.15.78:0/3604251478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50501022a0 0x7f5050198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:46.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.385+0000 7f504ed9d640 1 --2- 172.21.15.78:0/3604251478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5050102c20 0x7f505019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:46.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.385+0000 7f504dd9b640 1 -- 172.21.15.78:0/3604251478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5050102c20 msgr2=0x7f505019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:46.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.385+0000 7f504dd9b640 1 --2- 172.21.15.78:0/3604251478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5050102c20 0x7f505019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.385+0000 7f504dd9b640 1 -- 172.21.15.78:0/3604251478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5050100b50 msgr2=0x7f5050198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:46.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.385+0000 7f504dd9b640 1 --2- 172.21.15.78:0/3604251478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5050100b50 0x7f5050198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.385+0000 7f504dd9b640 1 -- 172.21.15.78:0/3604251478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f505019d1c0 con 0x7f50501022a0 2024-08-18T23:48:46.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.385+0000 7f504dd9b640 1 --2- 172.21.15.78:0/3604251478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50501022a0 0x7f5050198750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5038005fd0 tx=0x7f503800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:46.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.385+0000 7f50477fe640 1 -- 172.21.15.78:0/3604251478 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038009c30 con 0x7f50501022a0 2024-08-18T23:48:46.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.386+0000 7f50477fe640 1 -- 172.21.15.78:0/3604251478 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5038009dd0 con 0x7f50501022a0 2024-08-18T23:48:46.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.386+0000 7f5054e72640 1 -- 172.21.15.78:0/3604251478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f505019d4b0 con 0x7f50501022a0 2024-08-18T23:48:46.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.386+0000 7f50477fe640 1 -- 172.21.15.78:0/3604251478 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038010460 con 0x7f50501022a0 2024-08-18T23:48:46.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.386+0000 7f5054e72640 1 -- 172.21.15.78:0/3604251478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50501a4d90 con 0x7f50501022a0 2024-08-18T23:48:46.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.388+0000 7f50477fe640 1 -- 172.21.15.78:0/3604251478 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5038003f50 con 0x7f50501022a0 2024-08-18T23:48:46.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.388+0000 7f5057270740 1 Processor -- start 2024-08-18T23:48:46.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.388+0000 7f5057270740 1 -- start start 2024-08-18T23:48:46.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.389+0000 7f5057270740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c08bff0 0x56063c08c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:46.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.389+0000 7f5057270740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08c910 0x56063c096c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:46.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.389+0000 7f5057270740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c0971d0 0x56063c0995c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:46.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.389+0000 7f5057270740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56063bff1a40 con 0x56063c0971d0 2024-08-18T23:48:46.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.389+0000 7f5057270740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56063bf899d0 con 0x56063c08c910 2024-08-18T23:48:46.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.389+0000 7f5057270740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56063bfeda00 con 0x56063c08bff0 2024-08-18T23:48:46.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.389+0000 7f504e59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08c910 0x56063c096c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:46.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.389+0000 7f504dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c0971d0 0x56063c0995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:46.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.389+0000 7f504e59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08c910 0x56063c096c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:51904/0 (socket says 172.21.15.78:51904) 2024-08-18T23:48:46.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.389+0000 7f504e59c640 1 -- 172.21.15.78:0/2814566754 learned_addr learned my addr 172.21.15.78:0/2814566754 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:46.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.389+0000 7f504ed9d640 1 --2- 172.21.15.78:0/2814566754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c08bff0 0x56063c08c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:46.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.389+0000 7f50477fe640 1 --2- 172.21.15.78:0/3604251478 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f503c0a2b80 0x7f503c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:46.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.389+0000 7f504e59c640 1 -- 172.21.15.78:0/2814566754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c08bff0 msgr2=0x56063c08c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:46.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.389+0000 7f504e59c640 1 --2- 172.21.15.78:0/2814566754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c08bff0 0x56063c08c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.389+0000 7f504e59c640 1 -- 172.21.15.78:0/2814566754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c0971d0 msgr2=0x56063c0995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:46.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.389+0000 7f504ed9d640 1 --2- 172.21.15.78:0/3604251478 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f503c0a2b80 0x7f503c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:46.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.389+0000 7f504e59c640 1 --2- 172.21.15.78:0/2814566754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c0971d0 0x56063c0995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.389+0000 7f504e59c640 1 -- 172.21.15.78:0/2814566754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56063bf90650 con 0x56063c08c910 2024-08-18T23:48:46.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.390+0000 7f50477fe640 1 -- 172.21.15.78:0/3604251478 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f50380cefc0 con 0x7f50501022a0 2024-08-18T23:48:46.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.390+0000 7f504e59c640 1 --2- 172.21.15.78:0/2814566754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08c910 0x56063c096c90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5040009a20 tx=0x7f5040004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cc2bf07b80d11f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:46.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.390+0000 7f504ed9d640 1 --2- 172.21.15.78:0/3604251478 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f503c0a2b80 0x7f503c0a5040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5048005c40 tx=0x7f5048004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:46.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.390+0000 7f5044ff9640 1 -- 172.21.15.78:0/2814566754 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040040070 con 0x56063c08c910 2024-08-18T23:48:46.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.390+0000 7f5044ff9640 1 -- 172.21.15.78:0/2814566754 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5040003ad0 con 0x56063c08c910 2024-08-18T23:48:46.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.390+0000 7f5044ff9640 1 -- 172.21.15.78:0/2814566754 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040035040 con 0x56063c08c910 2024-08-18T23:48:46.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.391+0000 7f5057270740 1 -- 172.21.15.78:0/2814566754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08c910 msgr2=0x56063c096c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:46.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.391+0000 7f5057270740 1 --2- 172.21.15.78:0/2814566754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08c910 0x56063c096c90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5040009a20 tx=0x7f5040004040 comp rx=0 tx=0).stop 2024-08-18T23:48:46.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.391+0000 7f5057270740 1 -- 172.21.15.78:0/2814566754 shutdown_connections 2024-08-18T23:48:46.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.391+0000 7f5057270740 1 --2- 172.21.15.78:0/2814566754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c0971d0 0x56063c0995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.391+0000 7f5057270740 1 --2- 172.21.15.78:0/2814566754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08c910 0x56063c096c90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.391+0000 7f5057270740 1 --2- 172.21.15.78:0/2814566754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c08bff0 0x56063c08c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.391+0000 7f5057270740 1 -- 172.21.15.78:0/2814566754 >> 172.21.15.78:0/2814566754 conn(0x56063c088580 msgr2=0x56063c088960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:46.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.391+0000 7f5057270740 1 -- 172.21.15.78:0/2814566754 shutdown_connections 2024-08-18T23:48:46.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.391+0000 7f5057270740 1 -- 172.21.15.78:0/2814566754 wait complete. 2024-08-18T23:48:46.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.391+0000 7f5057270740 1 -- start start 2024-08-18T23:48:46.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.391+0000 7f5057270740 1 Processor -- start 2024-08-18T23:48:46.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.391+0000 7f5057270740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08bff0 0x56063c094080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:46.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.391+0000 7f5057270740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08c910 0x56063c0945c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:46.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f5057270740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c0971d0 0x56063c094b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:46.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f5057270740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56063c099da0 con 0x56063c08c910 2024-08-18T23:48:46.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f5057270740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56063bfacb20 con 0x56063c08bff0 2024-08-18T23:48:46.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f5057270740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56063c099f20 con 0x56063c0971d0 2024-08-18T23:48:46.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f5057270740 10 client.0 _ll_register_callbacks cb 0x56063bfaabb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:48:46.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f504dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08c910 0x56063c0945c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:46.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f504e59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08bff0 0x56063c094080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:46.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f504dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08c910 0x56063c0945c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49554/0 (socket says 172.21.15.78:49554) 2024-08-18T23:48:46.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f504dd9b640 1 -- 172.21.15.78:0/4241671074 learned_addr learned my addr 172.21.15.78:0/4241671074 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:46.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f504e59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08bff0 0x56063c094080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:51914/0 (socket says 172.21.15.78:51914) 2024-08-18T23:48:46.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f504ed9d640 1 --2- 172.21.15.78:0/4241671074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c0971d0 0x56063c094b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f504dd9b640 1 -- 172.21.15.78:0/4241671074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c0971d0 msgr2=0x56063c094b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f504dd9b640 1 --2- 172.21.15.78:0/4241671074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c0971d0 0x56063c094b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f504dd9b640 1 -- 172.21.15.78:0/4241671074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08bff0 msgr2=0x56063c094080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f504dd9b640 1 --2- 172.21.15.78:0/4241671074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08bff0 0x56063c094080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f504dd9b640 1 -- 172.21.15.78:0/4241671074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56063bd633c0 con 0x56063c08c910 2024-08-18T23:48:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f504dd9b640 1 --2- 172.21.15.78:0/4241671074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08c910 0x56063c0945c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f50380ccaa0 tx=0x7f503809b140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08f174c224c5872 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f5035ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:48:46.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f5035ffb640 1 -- 172.21.15.78:0/4241671074 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038020050 con 0x56063c08c910 2024-08-18T23:48:46.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f5035ffb640 1 -- 172.21.15.78:0/4241671074 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f5038097bb0 con 0x56063c08c910 2024-08-18T23:48:46.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f5057270740 1 -- 172.21.15.78:0/4241671074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5040003ad0 con 0x56063c08c910 2024-08-18T23:48:46.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f5035ffb640 1 -- 172.21.15.78:0/4241671074 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f503809f840 con 0x56063c08c910 2024-08-18T23:48:46.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f5057270740 10 client.16224 fetch_fsmap learned FSMap version 28 2024-08-18T23:48:46.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.392+0000 7f5057270740 1 -- 172.21.15.78:0/4241671074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56063c0a5970 con 0x56063c08c910 2024-08-18T23:48:46.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.393+0000 7f5035ffb640 1 -- 172.21.15.78:0/4241671074 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5040003ad0 con 0x56063c08c910 2024-08-18T23:48:46.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.393+0000 7f5035ffb640 1 -- 172.21.15.78:0/4241671074 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f50380a3c70 con 0x56063c08c910 2024-08-18T23:48:46.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.393+0000 7f5057270740 10 client.16224 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:48:46.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.393+0000 7f5057270740 10 client.16224 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:48:46.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.393+0000 7f5057270740 20 client.16224 populate_metadata read hostname 'smithi078' 2024-08-18T23:48:46.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.393+0000 7f5057270740 1 --2- 172.21.15.78:0/4241671074 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56063c0a6ea0 0x56063c0a92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:46.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.393+0000 7f5057270740 4 client.16224 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:48:46.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.393+0000 7f5057270740 1 -- 172.21.15.78:0/4241671074 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56063bffefb0 con 0x56063c0a6ea0 2024-08-18T23:48:46.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.393+0000 7f504e59c640 1 --2- 172.21.15.78:0/4241671074 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56063c0a6ea0 0x56063c0a92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:46.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.394+0000 7f5035ffb640 10 client.16224 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:48:46.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.394+0000 7f504e59c640 1 --2- 172.21.15.78:0/4241671074 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56063c0a6ea0 0x56063c0a92e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d256c3c73346d4c4 server_cookie=91aac25f7cd56f11 in_seq=0 out_seq=0 2024-08-18T23:48:46.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.394+0000 7f5035ffb640 1 -- 172.21.15.78:0/4241671074 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56063bffefb0 con 0x56063c0a6ea0 2024-08-18T23:48:46.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.394+0000 7f5035ffb640 10 client.16224 handle_command_reply: tid=0 2024-08-18T23:48:46.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.394+0000 7f5057270740 1 client.16224 shutdown 2024-08-18T23:48:46.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.394+0000 7f5057270740 1 -- 172.21.15.78:0/4241671074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08c910 msgr2=0x56063c0945c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:46.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.394+0000 7f5057270740 1 --2- 172.21.15.78:0/4241671074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08c910 0x56063c0945c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f50380ccaa0 tx=0x7f503809b140 comp rx=0 tx=0).stop 2024-08-18T23:48:46.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.394+0000 7f5057270740 1 -- 172.21.15.78:0/4241671074 shutdown_connections 2024-08-18T23:48:46.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.394+0000 7f5057270740 1 --2- 172.21.15.78:0/4241671074 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56063c0a6ea0 0x56063c0a92e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.394+0000 7f5057270740 1 --2- 172.21.15.78:0/4241671074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c0971d0 0x56063c094b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.394+0000 7f5057270740 1 --2- 172.21.15.78:0/4241671074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08c910 0x56063c0945c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.394+0000 7f5057270740 1 --2- 172.21.15.78:0/4241671074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08bff0 0x56063c094080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.394+0000 7f5057270740 1 -- 172.21.15.78:0/4241671074 >> 172.21.15.78:0/4241671074 conn(0x56063c088580 msgr2=0x56063c08e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:46.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.394+0000 7f5035ffb640 0 client.16224 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:48:46.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.395+0000 7f5057270740 1 -- 172.21.15.78:0/4241671074 shutdown_connections 2024-08-18T23:48:46.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.395+0000 7f5057270740 1 -- 172.21.15.78:0/4241671074 wait complete. 2024-08-18T23:48:46.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.395+0000 7f5057270740 20 client.16224 trim_cache size 0 max 16384 2024-08-18T23:48:46.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.407+0000 7f5057270740 1 Processor -- start 2024-08-18T23:48:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.407+0000 7f5057270740 1 -- start start 2024-08-18T23:48:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f5057270740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08bff0 0x56063c086d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f5057270740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08c910 0x56063c0872d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f5057270740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c0971d0 0x56063c087810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f5057270740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50501af090 con 0x56063c08c910 2024-08-18T23:48:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f5057270740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50501ac960 con 0x56063c08bff0 2024-08-18T23:48:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f5057270740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f50501aa7b0 con 0x56063c0971d0 2024-08-18T23:48:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f504dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08c910 0x56063c0872d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f504e59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08bff0 0x56063c086d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f504dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08c910 0x56063c0872d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49556/0 (socket says 172.21.15.78:49556) 2024-08-18T23:48:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f504dd9b640 1 -- 172.21.15.78:0/2584419443 learned_addr learned my addr 172.21.15.78:0/2584419443 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f504ed9d640 1 --2- 172.21.15.78:0/2584419443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c0971d0 0x56063c087810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f504dd9b640 1 -- 172.21.15.78:0/2584419443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c0971d0 msgr2=0x56063c087810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f504dd9b640 1 --2- 172.21.15.78:0/2584419443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c0971d0 0x56063c087810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f504dd9b640 1 -- 172.21.15.78:0/2584419443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08bff0 msgr2=0x56063c086d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f504dd9b640 1 --2- 172.21.15.78:0/2584419443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08bff0 0x56063c086d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f504dd9b640 1 -- 172.21.15.78:0/2584419443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56063c08ee60 con 0x56063c08c910 2024-08-18T23:48:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f504e59c640 1 --2- 172.21.15.78:0/2584419443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08bff0 0x56063c086d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:48:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f504dd9b640 1 --2- 172.21.15.78:0/2584419443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08c910 0x56063c0872d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5038009610 tx=0x7f50380039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b9f246bcbde4f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f5037fff640 1 -- 172.21.15.78:0/2584419443 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038020070 con 0x56063c08c910 2024-08-18T23:48:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f5037fff640 1 -- 172.21.15.78:0/2584419443 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50380a3c70 con 0x56063c08c910 2024-08-18T23:48:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.408+0000 7f5037fff640 1 -- 172.21.15.78:0/2584419443 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50380d23e0 con 0x56063c08c910 2024-08-18T23:48:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.409+0000 7f5057270740 1 -- 172.21.15.78:0/2584419443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08c910 msgr2=0x56063c0872d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.409+0000 7f5057270740 1 --2- 172.21.15.78:0/2584419443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08c910 0x56063c0872d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5038009610 tx=0x7f50380039f0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.409+0000 7f5057270740 1 -- 172.21.15.78:0/2584419443 shutdown_connections 2024-08-18T23:48:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.409+0000 7f5057270740 1 --2- 172.21.15.78:0/2584419443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c0971d0 0x56063c087810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.409+0000 7f5057270740 1 --2- 172.21.15.78:0/2584419443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08c910 0x56063c0872d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.409+0000 7f5057270740 1 --2- 172.21.15.78:0/2584419443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08bff0 0x56063c086d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.409+0000 7f5057270740 1 -- 172.21.15.78:0/2584419443 >> 172.21.15.78:0/2584419443 conn(0x56063c088580 msgr2=0x56063c08f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.409+0000 7f5057270740 1 -- 172.21.15.78:0/2584419443 shutdown_connections 2024-08-18T23:48:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.409+0000 7f5057270740 1 -- 172.21.15.78:0/2584419443 wait complete. 2024-08-18T23:48:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.409+0000 7f5057270740 1 -- start start 2024-08-18T23:48:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.409+0000 7f5057270740 1 Processor -- start 2024-08-18T23:48:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.409+0000 7f5057270740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08bff0 0x56063c091ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.409+0000 7f5057270740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08c910 0x56063c09f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.409+0000 7f5057270740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c0971d0 0x56063c09f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f5057270740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50501aae20 con 0x56063c08bff0 2024-08-18T23:48:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f5057270740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50501ac290 con 0x56063c08c910 2024-08-18T23:48:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f5057270740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f50501ab4f0 con 0x56063c0971d0 2024-08-18T23:48:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f5057270740 10 client.0 _ll_register_callbacks cb 0x7f5038005d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:48:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f504e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08bff0 0x56063c091ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f504e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08bff0 0x56063c091ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49560/0 (socket says 172.21.15.78:49560) 2024-08-18T23:48:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f504e59c640 1 -- 172.21.15.78:0/3491563730 learned_addr learned my addr 172.21.15.78:0/3491563730 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f504dd9b640 1 --2- 172.21.15.78:0/3491563730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08c910 0x56063c09f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f504ed9d640 1 --2- 172.21.15.78:0/3491563730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c0971d0 0x56063c09f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f504e59c640 1 -- 172.21.15.78:0/3491563730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c0971d0 msgr2=0x56063c09f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f504e59c640 1 --2- 172.21.15.78:0/3491563730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c0971d0 0x56063c09f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f504e59c640 1 -- 172.21.15.78:0/3491563730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08c910 msgr2=0x56063c09f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f504e59c640 1 --2- 172.21.15.78:0/3491563730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08c910 0x56063c09f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f504e59c640 1 -- 172.21.15.78:0/3491563730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56063c0902f0 con 0x56063c08bff0 2024-08-18T23:48:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f504e59c640 1 --2- 172.21.15.78:0/3491563730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08bff0 0x56063c091ca0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5040000c00 tx=0x7f504002ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291c3739d26b1096 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f5035ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:48:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f5035ffb640 1 -- 172.21.15.78:0/3491563730 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040035040 con 0x56063c08bff0 2024-08-18T23:48:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f5035ffb640 1 -- 172.21.15.78:0/3491563730 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f5040031950 con 0x56063c08bff0 2024-08-18T23:48:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f5057270740 1 -- 172.21.15.78:0/3491563730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50380a3c70 con 0x56063c08bff0 2024-08-18T23:48:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f5035ffb640 1 -- 172.21.15.78:0/3491563730 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5040063050 con 0x56063c08bff0 2024-08-18T23:48:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.410+0000 7f5057270740 10 client.16230 fetch_fsmap learned FSMap version 28 2024-08-18T23:48:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.411+0000 7f5057270740 1 -- 172.21.15.78:0/3491563730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56063c110d70 con 0x56063c08bff0 2024-08-18T23:48:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.411+0000 7f5035ffb640 1 -- 172.21.15.78:0/3491563730 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50380a3c70 con 0x56063c08bff0 2024-08-18T23:48:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.411+0000 7f5035ffb640 1 -- 172.21.15.78:0/3491563730 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f50400543b0 con 0x56063c08bff0 2024-08-18T23:48:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.411+0000 7f5057270740 10 client.16230 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:48:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.411+0000 7f5057270740 10 client.16230 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:48:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.411+0000 7f5057270740 20 client.16230 populate_metadata read hostname 'smithi078' 2024-08-18T23:48:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.411+0000 7f5057270740 1 --2- 172.21.15.78:0/3491563730 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56063c111170 0x56063c113560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.411+0000 7f5057270740 4 client.16230 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:48:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.411+0000 7f5057270740 1 -- 172.21.15.78:0/3491563730 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56063bff94c0 con 0x56063c111170 2024-08-18T23:48:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.411+0000 7f504dd9b640 1 --2- 172.21.15.78:0/3491563730 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56063c111170 0x56063c113560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.411+0000 7f504dd9b640 1 --2- 172.21.15.78:0/3491563730 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56063c111170 0x56063c113560 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b6ac1a7edf64685 server_cookie=772f6288b320591a in_seq=0 out_seq=0 2024-08-18T23:48:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.411+0000 7f5035ffb640 10 client.16230 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:48:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.412+0000 7f5035ffb640 1 -- 172.21.15.78:0/3491563730 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56063bff94c0 con 0x56063c111170 2024-08-18T23:48:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.412+0000 7f5035ffb640 10 client.16230 handle_command_reply: tid=0 2024-08-18T23:48:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.412+0000 7f5057270740 1 client.16230 shutdown 2024-08-18T23:48:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.412+0000 7f5057270740 1 -- 172.21.15.78:0/3491563730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08bff0 msgr2=0x56063c091ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.412+0000 7f5057270740 1 --2- 172.21.15.78:0/3491563730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08bff0 0x56063c091ca0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5040000c00 tx=0x7f504002ac10 comp rx=0 tx=0).stop 2024-08-18T23:48:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.412+0000 7f5057270740 1 -- 172.21.15.78:0/3491563730 shutdown_connections 2024-08-18T23:48:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.412+0000 7f5057270740 1 --2- 172.21.15.78:0/3491563730 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x56063c111170 0x56063c113560 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.412+0000 7f5057270740 1 --2- 172.21.15.78:0/3491563730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56063c0971d0 0x56063c09f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.412+0000 7f5057270740 1 --2- 172.21.15.78:0/3491563730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56063c08c910 0x56063c09f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.412+0000 7f5057270740 1 --2- 172.21.15.78:0/3491563730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56063c08bff0 0x56063c091ca0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.412+0000 7f5057270740 1 -- 172.21.15.78:0/3491563730 >> 172.21.15.78:0/3491563730 conn(0x56063c088580 msgr2=0x56063c086d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.412+0000 7f5035ffb640 0 client.16230 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:48:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.412+0000 7f5057270740 1 -- 172.21.15.78:0/3491563730 shutdown_connections 2024-08-18T23:48:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.412+0000 7f5057270740 1 -- 172.21.15.78:0/3491563730 wait complete. 2024-08-18T23:48:46.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.412+0000 7f5057270740 20 client.16230 trim_cache size 0 max 16384 2024-08-18T23:48:46.433 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:48:46.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.412+0000 7f5054e72640 1 -- 172.21.15.78:0/3604251478 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f503c0a2b80 msgr2=0x7f503c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:46.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.412+0000 7f5054e72640 1 --2- 172.21.15.78:0/3604251478 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f503c0a2b80 0x7f503c0a5040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5048005c40 tx=0x7f5048004a40 comp rx=0 tx=0).stop 2024-08-18T23:48:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.412+0000 7f5054e72640 1 -- 172.21.15.78:0/3604251478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50501022a0 msgr2=0x7f5050198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.412+0000 7f5054e72640 1 --2- 172.21.15.78:0/3604251478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50501022a0 0x7f5050198750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5038005fd0 tx=0x7f503800dde0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.413+0000 7f5054e72640 1 -- 172.21.15.78:0/3604251478 shutdown_connections 2024-08-18T23:48:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.413+0000 7f5054e72640 1 --2- 172.21.15.78:0/3604251478 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f503c0a2b80 0x7f503c0a5040 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.413+0000 7f5054e72640 1 --2- 172.21.15.78:0/3604251478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5050102c20 0x7f505019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.413+0000 7f5054e72640 1 --2- 172.21.15.78:0/3604251478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50501022a0 0x7f5050198750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.413+0000 7f5054e72640 1 --2- 172.21.15.78:0/3604251478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5050100b50 0x7f5050198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.413+0000 7f5054e72640 1 -- 172.21.15.78:0/3604251478 >> 172.21.15.78:0/3604251478 conn(0x7f50500fc500 msgr2=0x7f5050069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.413+0000 7f5054e72640 1 -- 172.21.15.78:0/3604251478 shutdown_connections 2024-08-18T23:48:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:46.413+0000 7f5054e72640 1 -- 172.21.15.78:0/3604251478 wait complete. 2024-08-18T23:48:46.983 INFO:tasks.workunit.client.0.smithi078.stdout: 98 1476722 412 1062837 318 744020 1132 2024-08-18T23:48:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:47 smithi161 ceph-mon[22027]: pgmap v484: 129 pgs: 129 active+clean; 548 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.8 MiB/s wr, 678 op/s 2024-08-18T23:48:47.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:47 smithi078 ceph-mon[18440]: pgmap v484: 129 pgs: 129 active+clean; 548 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.8 MiB/s wr, 678 op/s 2024-08-18T23:48:47.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:47 smithi123 ceph-mon[21808]: pgmap v484: 129 pgs: 129 active+clean; 548 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.8 MiB/s wr, 678 op/s 2024-08-18T23:48:47.836 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-18T23:48:48.592 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:48:49.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.353+0000 7fdb3fa7a640 1 -- 172.21.15.78:0/4243979154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb38109840 msgr2=0x7fdb381142a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:49.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.353+0000 7fdb3fa7a640 1 --2- 172.21.15.78:0/4243979154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb38109840 0x7fdb381142a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fdb34009ca0 tx=0x7fdb3402a950 comp rx=0 tx=0).stop 2024-08-18T23:48:49.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.353+0000 7fdb3fa7a640 1 -- 172.21.15.78:0/4243979154 shutdown_connections 2024-08-18T23:48:49.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.353+0000 7fdb3fa7a640 1 --2- 172.21.15.78:0/4243979154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb381147e0 0x7fdb38116bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.353+0000 7fdb3fa7a640 1 --2- 172.21.15.78:0/4243979154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb38109840 0x7fdb381142a0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.353+0000 7fdb3fa7a640 1 --2- 172.21.15.78:0/4243979154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb38108f20 0x7fdb38109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.353+0000 7fdb3fa7a640 1 -- 172.21.15.78:0/4243979154 >> 172.21.15.78:0/4243979154 conn(0x7fdb3806de70 msgr2=0x7fdb3806e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:49.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.353+0000 7fdb3fa7a640 1 -- 172.21.15.78:0/4243979154 shutdown_connections 2024-08-18T23:48:49.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.353+0000 7fdb3fa7a640 1 -- 172.21.15.78:0/4243979154 wait complete. 2024-08-18T23:48:49.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.354+0000 7fdb3fa7a640 1 Processor -- start 2024-08-18T23:48:49.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.354+0000 7fdb3fa7a640 1 -- start start 2024-08-18T23:48:49.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.354+0000 7fdb3fa7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb38108f20 0x7fdb381a6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:49.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.354+0000 7fdb3fa7a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb38109840 0x7fdb381a6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:49.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.354+0000 7fdb3fa7a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb381147e0 0x7fdb381a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:49.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.354+0000 7fdb3fa7a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3811a4e0 con 0x7fdb38108f20 2024-08-18T23:48:49.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.354+0000 7fdb3fa7a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3811a360 con 0x7fdb38109840 2024-08-18T23:48:49.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.354+0000 7fdb3fa7a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3811a660 con 0x7fdb381147e0 2024-08-18T23:48:49.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.354+0000 7fdb3d816640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb38108f20 0x7fdb381a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:49.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.354+0000 7fdb3d015640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb38109840 0x7fdb381a6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:49.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.354+0000 7fdb3d816640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb38108f20 0x7fdb381a6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49586/0 (socket says 172.21.15.78:49586) 2024-08-18T23:48:49.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.354+0000 7fdb3d816640 1 -- 172.21.15.78:0/53157205 learned_addr learned my addr 172.21.15.78:0/53157205 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:49.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.354+0000 7fdb3e017640 1 --2- 172.21.15.78:0/53157205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb381147e0 0x7fdb381a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:49.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.354+0000 7fdb3d015640 1 -- 172.21.15.78:0/53157205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb381147e0 msgr2=0x7fdb381a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:49.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.354+0000 7fdb3d015640 1 --2- 172.21.15.78:0/53157205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb381147e0 0x7fdb381a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.354+0000 7fdb3d015640 1 -- 172.21.15.78:0/53157205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb38108f20 msgr2=0x7fdb381a6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:49.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.354+0000 7fdb3d015640 1 --2- 172.21.15.78:0/53157205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb38108f20 0x7fdb381a6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.354+0000 7fdb3d015640 1 -- 172.21.15.78:0/53157205 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb381a0fb0 con 0x7fdb38109840 2024-08-18T23:48:49.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.354+0000 7fdb3d816640 1 --2- 172.21.15.78:0/53157205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb38108f20 0x7fdb381a6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:48:49.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.355+0000 7fdb3d015640 1 --2- 172.21.15.78:0/53157205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb38109840 0x7fdb381a6bd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdb34006010 tx=0x7fdb3402f5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:49.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.355+0000 7fdb2effd640 1 -- 172.21.15.78:0/53157205 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb34034040 con 0x7fdb38109840 2024-08-18T23:48:49.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.355+0000 7fdb2effd640 1 -- 172.21.15.78:0/53157205 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb34030920 con 0x7fdb38109840 2024-08-18T23:48:49.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.355+0000 7fdb3fa7a640 1 -- 172.21.15.78:0/53157205 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb381a1240 con 0x7fdb38109840 2024-08-18T23:48:49.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.355+0000 7fdb2effd640 1 -- 172.21.15.78:0/53157205 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb34004710 con 0x7fdb38109840 2024-08-18T23:48:49.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.355+0000 7fdb3fa7a640 1 -- 172.21.15.78:0/53157205 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb381ad4f0 con 0x7fdb38109840 2024-08-18T23:48:49.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.357+0000 7fdb2effd640 1 -- 172.21.15.78:0/53157205 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdb34037020 con 0x7fdb38109840 2024-08-18T23:48:49.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.358+0000 7fdb41e78740 1 Processor -- start 2024-08-18T23:48:49.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.358+0000 7fdb41e78740 1 -- start start 2024-08-18T23:48:49.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.358+0000 7fdb41e78740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4bdff0 0x55d19a4be3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:49.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.358+0000 7fdb41e78740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4be910 0x55d19a4c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:49.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.358+0000 7fdb41e78740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4c91d0 0x55d19a4cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:49.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.358+0000 7fdb41e78740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d19a423a40 con 0x55d19a4c91d0 2024-08-18T23:48:49.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.358+0000 7fdb41e78740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d19a3bb9d0 con 0x55d19a4bdff0 2024-08-18T23:48:49.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.358+0000 7fdb41e78740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d19a41fa00 con 0x55d19a4be910 2024-08-18T23:48:49.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.358+0000 7fdb2effd640 1 --2- 172.21.15.78:0/53157205 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdb240a2b80 0x7fdb240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:49.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.358+0000 7fdb3e017640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4bdff0 0x55d19a4be3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:49.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.358+0000 7fdb3e017640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4bdff0 0x55d19a4be3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:51964/0 (socket says 172.21.15.78:51964) 2024-08-18T23:48:49.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.358+0000 7fdb3e017640 1 -- 172.21.15.78:0/3987716603 learned_addr learned my addr 172.21.15.78:0/3987716603 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:49.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.359+0000 7fdb3d816640 1 --2- 172.21.15.78:0/3987716603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4be910 0x55d19a4c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:49.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.359+0000 7fdb3e017640 1 --2- 172.21.15.78:0/53157205 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdb240a2b80 0x7fdb240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:49.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.359+0000 7fdb2effd640 1 -- 172.21.15.78:0/53157205 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fdb340f0130 con 0x7fdb38109840 2024-08-18T23:48:49.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.359+0000 7fdb3d015640 1 --2- 172.21.15.78:0/3987716603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4c91d0 0x55d19a4cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:49.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.359+0000 7fdb3e017640 1 -- 172.21.15.78:0/3987716603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4be910 msgr2=0x55d19a4c8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:49.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.359+0000 7fdb3e017640 1 --2- 172.21.15.78:0/3987716603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4be910 0x55d19a4c8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.359+0000 7fdb3e017640 1 -- 172.21.15.78:0/3987716603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4c91d0 msgr2=0x55d19a4cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:49.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.359+0000 7fdb3e017640 1 --2- 172.21.15.78:0/3987716603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4c91d0 0x55d19a4cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.359+0000 7fdb3e017640 1 -- 172.21.15.78:0/3987716603 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d19a3c2650 con 0x55d19a4bdff0 2024-08-18T23:48:49.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.359+0000 7fdb3d816640 1 --2- 172.21.15.78:0/3987716603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4be910 0x55d19a4c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:48:49.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.359+0000 7fdb3d015640 1 --2- 172.21.15.78:0/3987716603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4c91d0 0x55d19a4cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:48:49.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.359+0000 7fdb3e017640 1 --2- 172.21.15.78:0/53157205 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdb240a2b80 0x7fdb240a5040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdb300107a0 tx=0x7fdb30008000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:49.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.359+0000 7fdb3e017640 1 --2- 172.21.15.78:0/3987716603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4bdff0 0x55d19a4be3d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdb30012930 tx=0x7fdb30012df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12e4cdf18b867157 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:49.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.359+0000 7fdb23fff640 1 -- 172.21.15.78:0/3987716603 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30012050 con 0x55d19a4bdff0 2024-08-18T23:48:49.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.360+0000 7fdb23fff640 1 -- 172.21.15.78:0/3987716603 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb30009900 con 0x55d19a4bdff0 2024-08-18T23:48:49.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.360+0000 7fdb23fff640 1 -- 172.21.15.78:0/3987716603 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3000db60 con 0x55d19a4bdff0 2024-08-18T23:48:49.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.360+0000 7fdb41e78740 1 -- 172.21.15.78:0/3987716603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4bdff0 msgr2=0x55d19a4be3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:49.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.360+0000 7fdb41e78740 1 --2- 172.21.15.78:0/3987716603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4bdff0 0x55d19a4be3d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdb30012930 tx=0x7fdb30012df0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.360+0000 7fdb41e78740 1 -- 172.21.15.78:0/3987716603 shutdown_connections 2024-08-18T23:48:49.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.360+0000 7fdb41e78740 1 --2- 172.21.15.78:0/3987716603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4c91d0 0x55d19a4cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.360+0000 7fdb41e78740 1 --2- 172.21.15.78:0/3987716603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4be910 0x55d19a4c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.360+0000 7fdb41e78740 1 --2- 172.21.15.78:0/3987716603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4bdff0 0x55d19a4be3d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.360+0000 7fdb41e78740 1 -- 172.21.15.78:0/3987716603 >> 172.21.15.78:0/3987716603 conn(0x55d19a4ba580 msgr2=0x55d19a4ba960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.360+0000 7fdb41e78740 1 -- 172.21.15.78:0/3987716603 shutdown_connections 2024-08-18T23:48:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.360+0000 7fdb41e78740 1 -- 172.21.15.78:0/3987716603 wait complete. 2024-08-18T23:48:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.361+0000 7fdb41e78740 1 -- start start 2024-08-18T23:48:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.361+0000 7fdb41e78740 1 Processor -- start 2024-08-18T23:48:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.361+0000 7fdb41e78740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4bdff0 0x55d19a4c6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.361+0000 7fdb41e78740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4be910 0x55d19a4c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.361+0000 7fdb41e78740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4c91d0 0x55d19a4c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.361+0000 7fdb41e78740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d19a4cbda0 con 0x55d19a4be910 2024-08-18T23:48:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.361+0000 7fdb41e78740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d19a3deb20 con 0x55d19a4bdff0 2024-08-18T23:48:49.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.361+0000 7fdb41e78740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d19a4cbf20 con 0x55d19a4c91d0 2024-08-18T23:48:49.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.361+0000 7fdb41e78740 10 client.0 _ll_register_callbacks cb 0x55d19a3dcbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:48:49.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.361+0000 7fdb3d015640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4be910 0x55d19a4c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:49.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.361+0000 7fdb3d816640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4bdff0 0x55d19a4c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:49.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.361+0000 7fdb3d015640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4be910 0x55d19a4c65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49614/0 (socket says 172.21.15.78:49614) 2024-08-18T23:48:49.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.361+0000 7fdb3d015640 1 -- 172.21.15.78:0/640419511 learned_addr learned my addr 172.21.15.78:0/640419511 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:49.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.362+0000 7fdb3e017640 1 --2- 172.21.15.78:0/640419511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4c91d0 0x55d19a4c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:49.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.362+0000 7fdb3d015640 1 -- 172.21.15.78:0/640419511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4c91d0 msgr2=0x55d19a4c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:49.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.362+0000 7fdb3d015640 1 --2- 172.21.15.78:0/640419511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4c91d0 0x55d19a4c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.362+0000 7fdb3d015640 1 -- 172.21.15.78:0/640419511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4bdff0 msgr2=0x55d19a4c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:49.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.362+0000 7fdb3d015640 1 --2- 172.21.15.78:0/640419511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4bdff0 0x55d19a4c6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.362+0000 7fdb3d015640 1 -- 172.21.15.78:0/640419511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d19a1953c0 con 0x55d19a4be910 2024-08-18T23:48:49.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.362+0000 7fdb3d816640 1 --2- 172.21.15.78:0/640419511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4bdff0 0x55d19a4c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:48:49.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.362+0000 7fdb3d015640 1 --2- 172.21.15.78:0/640419511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4be910 0x55d19a4c65b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdb34005e30 tx=0x7fdb34004100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93aaf2d6a9b05884 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:49.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.362+0000 7fdb217fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:48:49.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.362+0000 7fdb217fa640 1 -- 172.21.15.78:0/640419511 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb340bbf00 con 0x55d19a4be910 2024-08-18T23:48:49.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.362+0000 7fdb217fa640 1 -- 172.21.15.78:0/640419511 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fdb340d5520 con 0x55d19a4be910 2024-08-18T23:48:49.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.362+0000 7fdb41e78740 1 -- 172.21.15.78:0/640419511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb30009900 con 0x55d19a4be910 2024-08-18T23:48:49.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.363+0000 7fdb217fa640 1 -- 172.21.15.78:0/640419511 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdb340bcca0 con 0x55d19a4be910 2024-08-18T23:48:49.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.363+0000 7fdb41e78740 10 client.16242 fetch_fsmap learned FSMap version 28 2024-08-18T23:48:49.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.363+0000 7fdb41e78740 1 -- 172.21.15.78:0/640419511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d19a4dbcf0 con 0x55d19a4be910 2024-08-18T23:48:49.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.363+0000 7fdb217fa640 1 -- 172.21.15.78:0/640419511 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb30009900 con 0x55d19a4be910 2024-08-18T23:48:49.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.363+0000 7fdb217fa640 1 -- 172.21.15.78:0/640419511 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdb3404d030 con 0x55d19a4be910 2024-08-18T23:48:49.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.363+0000 7fdb41e78740 10 client.16242 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:48:49.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.363+0000 7fdb41e78740 10 client.16242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:48:49.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.363+0000 7fdb41e78740 20 client.16242 populate_metadata read hostname 'smithi078' 2024-08-18T23:48:49.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.363+0000 7fdb41e78740 1 --2- 172.21.15.78:0/640419511 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55d19a4dd130 0x55d19a4df550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:49.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.363+0000 7fdb41e78740 4 client.16242 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:48:49.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.363+0000 7fdb41e78740 1 -- 172.21.15.78:0/640419511 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d19a430fb0 con 0x55d19a4dd130 2024-08-18T23:48:49.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.363+0000 7fdb3d816640 1 --2- 172.21.15.78:0/640419511 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55d19a4dd130 0x55d19a4df550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:49.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.364+0000 7fdb217fa640 10 client.16242 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:49.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.364+0000 7fdb3d816640 1 --2- 172.21.15.78:0/640419511 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55d19a4dd130 0x55d19a4df550 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=707921ca7f501fa6 server_cookie=d30c802f6738c534 in_seq=0 out_seq=0 2024-08-18T23:48:49.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.367+0000 7fdb217fa640 1 -- 172.21.15.78:0/640419511 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d19a430fb0 con 0x55d19a4dd130 2024-08-18T23:48:49.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.367+0000 7fdb217fa640 10 client.16242 handle_command_reply: tid=0 2024-08-18T23:48:49.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.367+0000 7fdb41e78740 1 client.16242 shutdown 2024-08-18T23:48:49.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.367+0000 7fdb41e78740 1 -- 172.21.15.78:0/640419511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4be910 msgr2=0x55d19a4c65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:49.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.367+0000 7fdb41e78740 1 --2- 172.21.15.78:0/640419511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4be910 0x55d19a4c65b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdb34005e30 tx=0x7fdb34004100 comp rx=0 tx=0).stop 2024-08-18T23:48:49.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.367+0000 7fdb41e78740 1 -- 172.21.15.78:0/640419511 shutdown_connections 2024-08-18T23:48:49.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.367+0000 7fdb41e78740 1 --2- 172.21.15.78:0/640419511 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55d19a4dd130 0x55d19a4df550 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.367+0000 7fdb41e78740 1 --2- 172.21.15.78:0/640419511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4c91d0 0x55d19a4c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.367+0000 7fdb41e78740 1 --2- 172.21.15.78:0/640419511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4be910 0x55d19a4c65b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.367+0000 7fdb41e78740 1 --2- 172.21.15.78:0/640419511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4bdff0 0x55d19a4c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.367+0000 7fdb217fa640 0 client.16242 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:49.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.367+0000 7fdb41e78740 1 -- 172.21.15.78:0/640419511 >> 172.21.15.78:0/640419511 conn(0x55d19a4ba580 msgr2=0x55d19a4c02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:49.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.367+0000 7fdb41e78740 1 -- 172.21.15.78:0/640419511 shutdown_connections 2024-08-18T23:48:49.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.367+0000 7fdb41e78740 1 -- 172.21.15.78:0/640419511 wait complete. 2024-08-18T23:48:49.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.367+0000 7fdb41e78740 20 client.16242 trim_cache size 0 max 16384 2024-08-18T23:48:49.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.388+0000 7fdb41e78740 1 Processor -- start 2024-08-18T23:48:49.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.388+0000 7fdb41e78740 1 -- start start 2024-08-18T23:48:49.390 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:49 smithi078 ceph-mon[18440]: pgmap v485: 129 pgs: 129 active+clean; 567 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 863 op/s 2024-08-18T23:48:49.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.388+0000 7fdb41e78740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4bdff0 0x55d19a4e4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.388+0000 7fdb41e78740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4be910 0x55d19a4e53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.389+0000 7fdb41e78740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4c91d0 0x55d19a4bfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.389+0000 7fdb41e78740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb381b77e0 con 0x55d19a4be910 2024-08-18T23:48:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.389+0000 7fdb41e78740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb381b50b0 con 0x55d19a4c91d0 2024-08-18T23:48:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.389+0000 7fdb41e78740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdb381b2f00 con 0x55d19a4bdff0 2024-08-18T23:48:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.389+0000 7fdb3d015640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4be910 0x55d19a4e53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.389+0000 7fdb3d816640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4bdff0 0x55d19a4e4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.389+0000 7fdb3d015640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4be910 0x55d19a4e53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49616/0 (socket says 172.21.15.78:49616) 2024-08-18T23:48:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.389+0000 7fdb3d015640 1 -- 172.21.15.78:0/892298132 learned_addr learned my addr 172.21.15.78:0/892298132 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.389+0000 7fdb3d816640 1 --2- 172.21.15.78:0/892298132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4bdff0 0x55d19a4e4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:33466/0 (socket says 172.21.15.78:33466) 2024-08-18T23:48:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.389+0000 7fdb3e017640 1 --2- 172.21.15.78:0/892298132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4c91d0 0x55d19a4bfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.389+0000 7fdb3d816640 1 -- 172.21.15.78:0/892298132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4c91d0 msgr2=0x55d19a4bfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.389+0000 7fdb3d816640 1 --2- 172.21.15.78:0/892298132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4c91d0 0x55d19a4bfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.389+0000 7fdb3d816640 1 -- 172.21.15.78:0/892298132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4be910 msgr2=0x55d19a4e53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.389+0000 7fdb3d816640 1 --2- 172.21.15.78:0/892298132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4be910 0x55d19a4e53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.389+0000 7fdb3d816640 1 -- 172.21.15.78:0/892298132 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d19a4dfbd0 con 0x55d19a4bdff0 2024-08-18T23:48:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.389+0000 7fdb3d015640 1 --2- 172.21.15.78:0/892298132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4be910 0x55d19a4e53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:48:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.389+0000 7fdb3d816640 1 --2- 172.21.15.78:0/892298132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4bdff0 0x55d19a4e4e60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55d19a4dd510 tx=0x7fdb28005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=411b6298555f9151 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.389+0000 7fdb237fe640 1 -- 172.21.15.78:0/892298132 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28012110 con 0x55d19a4bdff0 2024-08-18T23:48:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.390+0000 7fdb237fe640 1 -- 172.21.15.78:0/892298132 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb28005500 con 0x55d19a4bdff0 2024-08-18T23:48:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.390+0000 7fdb237fe640 1 -- 172.21.15.78:0/892298132 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28010430 con 0x55d19a4bdff0 2024-08-18T23:48:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.390+0000 7fdb41e78740 1 -- 172.21.15.78:0/892298132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4bdff0 msgr2=0x55d19a4e4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.390+0000 7fdb41e78740 1 --2- 172.21.15.78:0/892298132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4bdff0 0x55d19a4e4e60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55d19a4dd510 tx=0x7fdb28005f90 comp rx=0 tx=0).stop 2024-08-18T23:48:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.390+0000 7fdb41e78740 1 -- 172.21.15.78:0/892298132 shutdown_connections 2024-08-18T23:48:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.390+0000 7fdb41e78740 1 --2- 172.21.15.78:0/892298132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4c91d0 0x55d19a4bfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.390+0000 7fdb41e78740 1 --2- 172.21.15.78:0/892298132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4be910 0x55d19a4e53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.390+0000 7fdb41e78740 1 --2- 172.21.15.78:0/892298132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4bdff0 0x55d19a4e4e60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.390+0000 7fdb41e78740 1 -- 172.21.15.78:0/892298132 >> 172.21.15.78:0/892298132 conn(0x55d19a4ba580 msgr2=0x55d19a4c4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.390+0000 7fdb41e78740 1 -- 172.21.15.78:0/892298132 shutdown_connections 2024-08-18T23:48:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.390+0000 7fdb41e78740 1 -- 172.21.15.78:0/892298132 wait complete. 2024-08-18T23:48:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.391+0000 7fdb41e78740 1 -- start start 2024-08-18T23:48:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.391+0000 7fdb41e78740 1 Processor -- start 2024-08-18T23:48:49.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.392+0000 7fdb41e78740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4bdff0 0x55d19a4cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.392+0000 7fdb41e78740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4be910 0x55d19a4ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.392+0000 7fdb41e78740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4c91d0 0x55d19a545e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.392+0000 7fdb41e78740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb381b3570 con 0x55d19a4c91d0 2024-08-18T23:48:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.392+0000 7fdb41e78740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb381b49e0 con 0x55d19a4bdff0 2024-08-18T23:48:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.392+0000 7fdb41e78740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdb381b3c40 con 0x55d19a4be910 2024-08-18T23:48:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.392+0000 7fdb41e78740 10 client.0 _ll_register_callbacks cb 0x7fdb34009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:48:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.392+0000 7fdb3e017640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4c91d0 0x55d19a545e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.392+0000 7fdb3e017640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4c91d0 0x55d19a545e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49622/0 (socket says 172.21.15.78:49622) 2024-08-18T23:48:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.392+0000 7fdb3e017640 1 -- 172.21.15.78:0/3701055760 learned_addr learned my addr 172.21.15.78:0/3701055760 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.392+0000 7fdb3d816640 1 --2- 172.21.15.78:0/3701055760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4bdff0 0x55d19a4cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.392+0000 7fdb3d015640 1 --2- 172.21.15.78:0/3701055760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4be910 0x55d19a4ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:49.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.392+0000 7fdb3e017640 1 -- 172.21.15.78:0/3701055760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4be910 msgr2=0x55d19a4ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.392+0000 7fdb3e017640 1 --2- 172.21.15.78:0/3701055760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4be910 0x55d19a4ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.392+0000 7fdb3e017640 1 -- 172.21.15.78:0/3701055760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4bdff0 msgr2=0x55d19a4cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.392+0000 7fdb3e017640 1 --2- 172.21.15.78:0/3701055760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4bdff0 0x55d19a4cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.392+0000 7fdb3e017640 1 -- 172.21.15.78:0/3701055760 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d19a546360 con 0x55d19a4c91d0 2024-08-18T23:48:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.393+0000 7fdb3e017640 1 --2- 172.21.15.78:0/3701055760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4c91d0 0x55d19a545e20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdb30012930 tx=0x7fdb30012280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95456a152b5c0f60 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.393+0000 7fdb217fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:48:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.393+0000 7fdb217fa640 1 -- 172.21.15.78:0/3701055760 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30009900 con 0x55d19a4c91d0 2024-08-18T23:48:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.393+0000 7fdb217fa640 1 -- 172.21.15.78:0/3701055760 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fdb30005260 con 0x55d19a4c91d0 2024-08-18T23:48:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.393+0000 7fdb41e78740 1 -- 172.21.15.78:0/3701055760 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb28005500 con 0x55d19a4c91d0 2024-08-18T23:48:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.393+0000 7fdb217fa640 1 -- 172.21.15.78:0/3701055760 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdb30005bf0 con 0x55d19a4c91d0 2024-08-18T23:48:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.393+0000 7fdb41e78740 10 client.16248 fetch_fsmap learned FSMap version 28 2024-08-18T23:48:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.393+0000 7fdb41e78740 1 -- 172.21.15.78:0/3701055760 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d19a547600 con 0x55d19a4c91d0 2024-08-18T23:48:49.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.393+0000 7fdb217fa640 1 -- 172.21.15.78:0/3701055760 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb28005500 con 0x55d19a4c91d0 2024-08-18T23:48:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.394+0000 7fdb217fa640 1 -- 172.21.15.78:0/3701055760 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdb30003680 con 0x55d19a4c91d0 2024-08-18T23:48:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.394+0000 7fdb41e78740 10 client.16248 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:48:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.394+0000 7fdb41e78740 10 client.16248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:48:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.394+0000 7fdb41e78740 20 client.16248 populate_metadata read hostname 'smithi078' 2024-08-18T23:48:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.394+0000 7fdb41e78740 1 --2- 172.21.15.78:0/3701055760 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55d19a547d10 0x55d19a54a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.394+0000 7fdb41e78740 4 client.16248 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:48:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.394+0000 7fdb41e78740 1 -- 172.21.15.78:0/3701055760 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d19a42b4c0 con 0x55d19a547d10 2024-08-18T23:48:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.394+0000 7fdb3d816640 1 --2- 172.21.15.78:0/3701055760 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55d19a547d10 0x55d19a54a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.394+0000 7fdb217fa640 10 client.16248 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.394+0000 7fdb3d816640 1 --2- 172.21.15.78:0/3701055760 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55d19a547d10 0x55d19a54a130 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6539a66fc2d6dace server_cookie=40bbab84bf05aacc in_seq=0 out_seq=0 2024-08-18T23:48:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.397+0000 7fdb217fa640 1 -- 172.21.15.78:0/3701055760 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d19a42b4c0 con 0x55d19a547d10 2024-08-18T23:48:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.397+0000 7fdb217fa640 10 client.16248 handle_command_reply: tid=0 2024-08-18T23:48:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.397+0000 7fdb41e78740 1 client.16248 shutdown 2024-08-18T23:48:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.398+0000 7fdb41e78740 1 -- 172.21.15.78:0/3701055760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4c91d0 msgr2=0x55d19a545e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.398+0000 7fdb41e78740 1 --2- 172.21.15.78:0/3701055760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4c91d0 0x55d19a545e20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdb30012930 tx=0x7fdb30012280 comp rx=0 tx=0).stop 2024-08-18T23:48:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.398+0000 7fdb41e78740 1 -- 172.21.15.78:0/3701055760 shutdown_connections 2024-08-18T23:48:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.398+0000 7fdb41e78740 1 --2- 172.21.15.78:0/3701055760 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55d19a547d10 0x55d19a54a130 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.398+0000 7fdb41e78740 1 --2- 172.21.15.78:0/3701055760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d19a4c91d0 0x55d19a545e20 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.398+0000 7fdb217fa640 0 client.16248 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.398+0000 7fdb41e78740 1 --2- 172.21.15.78:0/3701055760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19a4be910 0x55d19a4ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.398+0000 7fdb41e78740 1 --2- 172.21.15.78:0/3701055760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d19a4bdff0 0x55d19a4cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.398+0000 7fdb41e78740 1 -- 172.21.15.78:0/3701055760 >> 172.21.15.78:0/3701055760 conn(0x55d19a4ba580 msgr2=0x55d19a4bfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.398+0000 7fdb41e78740 1 -- 172.21.15.78:0/3701055760 shutdown_connections 2024-08-18T23:48:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.398+0000 7fdb41e78740 1 -- 172.21.15.78:0/3701055760 wait complete. 2024-08-18T23:48:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.398+0000 7fdb41e78740 20 client.16248 trim_cache size 0 max 16384 2024-08-18T23:48:49.401 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:48:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.399+0000 7fdb3fa7a640 1 -- 172.21.15.78:0/53157205 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdb240a2b80 msgr2=0x7fdb240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.399+0000 7fdb3fa7a640 1 --2- 172.21.15.78:0/53157205 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdb240a2b80 0x7fdb240a5040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdb300107a0 tx=0x7fdb30008000 comp rx=0 tx=0).stop 2024-08-18T23:48:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.399+0000 7fdb3fa7a640 1 -- 172.21.15.78:0/53157205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb38109840 msgr2=0x7fdb381a6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.399+0000 7fdb3fa7a640 1 --2- 172.21.15.78:0/53157205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb38109840 0x7fdb381a6bd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdb34006010 tx=0x7fdb3402f5d0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.399+0000 7fdb3fa7a640 1 -- 172.21.15.78:0/53157205 shutdown_connections 2024-08-18T23:48:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.399+0000 7fdb3fa7a640 1 --2- 172.21.15.78:0/53157205 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fdb240a2b80 0x7fdb240a5040 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.399+0000 7fdb3fa7a640 1 --2- 172.21.15.78:0/53157205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb381147e0 0x7fdb381a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.399+0000 7fdb3fa7a640 1 --2- 172.21.15.78:0/53157205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb38109840 0x7fdb381a6bd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.399+0000 7fdb3fa7a640 1 --2- 172.21.15.78:0/53157205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb38108f20 0x7fdb381a6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.399+0000 7fdb3fa7a640 1 -- 172.21.15.78:0/53157205 >> 172.21.15.78:0/53157205 conn(0x7fdb3806de70 msgr2=0x7fdb38070670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.399+0000 7fdb3fa7a640 1 -- 172.21.15.78:0/53157205 shutdown_connections 2024-08-18T23:48:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:49.399+0000 7fdb3fa7a640 1 -- 172.21.15.78:0/53157205 wait complete. 2024-08-18T23:48:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:49 smithi161 ceph-mon[22027]: pgmap v485: 129 pgs: 129 active+clean; 567 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 863 op/s 2024-08-18T23:48:49.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:49 smithi123 ceph-mon[21808]: pgmap v485: 129 pgs: 129 active+clean; 567 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 863 op/s 2024-08-18T23:48:51.249 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-18T23:48:51.249 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-18T23:48:51.515 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:48:51.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:51 smithi078 ceph-mon[18440]: pgmap v486: 129 pgs: 129 active+clean; 571 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 855 op/s 2024-08-18T23:48:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:51 smithi161 ceph-mon[22027]: pgmap v486: 129 pgs: 129 active+clean; 571 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 855 op/s 2024-08-18T23:48:51.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:51 smithi123 ceph-mon[21808]: pgmap v486: 129 pgs: 129 active+clean; 571 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 855 op/s 2024-08-18T23:48:52.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.013+0000 7effaffe8640 1 -- 172.21.15.78:0/309727103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effa8103ac0 msgr2=0x7effa8103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:52.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.013+0000 7effaffe8640 1 --2- 172.21.15.78:0/309727103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effa8103ac0 0x7effa8103f20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff98009890 tx=0x7eff9802a380 comp rx=0 tx=0).stop 2024-08-18T23:48:52.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.013+0000 7effaffe8640 1 -- 172.21.15.78:0/309727103 shutdown_connections 2024-08-18T23:48:52.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.013+0000 7effaffe8640 1 --2- 172.21.15.78:0/309727103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effa8104460 0x7effa8110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.013+0000 7effaffe8640 1 --2- 172.21.15.78:0/309727103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effa8103ac0 0x7effa8103f20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.013+0000 7effaffe8640 1 --2- 172.21.15.78:0/309727103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa8109ac0 0x7effa8109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.013+0000 7effaffe8640 1 -- 172.21.15.78:0/309727103 >> 172.21.15.78:0/309727103 conn(0x7effa80ff7e0 msgr2=0x7effa8101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:52.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.014+0000 7effaffe8640 1 -- 172.21.15.78:0/309727103 shutdown_connections 2024-08-18T23:48:52.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.014+0000 7effaffe8640 1 -- 172.21.15.78:0/309727103 wait complete. 2024-08-18T23:48:52.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.015+0000 7effaffe8640 1 Processor -- start 2024-08-18T23:48:52.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.015+0000 7effaffe8640 1 -- start start 2024-08-18T23:48:52.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.015+0000 7effaffe8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effa8103ac0 0x7effa810ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:52.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.015+0000 7effaffe8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effa8104460 0x7effa811b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:52.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.015+0000 7effaffe8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa8109ac0 0x7effa810f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:52.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.016+0000 7effaffe8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7effa8077100 con 0x7effa8103ac0 2024-08-18T23:48:52.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.016+0000 7effaffe8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7effa8076f80 con 0x7effa8109ac0 2024-08-18T23:48:52.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.016+0000 7effaffe8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7effa8077280 con 0x7effa8104460 2024-08-18T23:48:52.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.016+0000 7effadd84640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effa8103ac0 0x7effa810ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:52.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.016+0000 7effadd84640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effa8103ac0 0x7effa810ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49636/0 (socket says 172.21.15.78:49636) 2024-08-18T23:48:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.016+0000 7effae585640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa8109ac0 0x7effa810f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.016+0000 7effadd84640 1 -- 172.21.15.78:0/1405316183 learned_addr learned my addr 172.21.15.78:0/1405316183 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.016+0000 7effad583640 1 --2- 172.21.15.78:0/1405316183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effa8104460 0x7effa811b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.017+0000 7effadd84640 1 -- 172.21.15.78:0/1405316183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effa8104460 msgr2=0x7effa811b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.017+0000 7effadd84640 1 --2- 172.21.15.78:0/1405316183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effa8104460 0x7effa811b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.017+0000 7effadd84640 1 -- 172.21.15.78:0/1405316183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa8109ac0 msgr2=0x7effa810f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.017+0000 7effadd84640 1 --2- 172.21.15.78:0/1405316183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa8109ac0 0x7effa810f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.017+0000 7effadd84640 1 -- 172.21.15.78:0/1405316183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa810faf0 con 0x7effa8103ac0 2024-08-18T23:48:52.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.019+0000 7effadd84640 1 --2- 172.21.15.78:0/1405316183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effa8103ac0 0x7effa810ee80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7effa000db80 tx=0x7effa000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:52.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.019+0000 7eff9effd640 1 -- 172.21.15.78:0/1405316183 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0017070 con 0x7effa8103ac0 2024-08-18T23:48:52.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.019+0000 7eff9effd640 1 -- 172.21.15.78:0/1405316183 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa0003e50 con 0x7effa8103ac0 2024-08-18T23:48:52.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.019+0000 7eff9effd640 1 -- 172.21.15.78:0/1405316183 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa00106c0 con 0x7effa8103ac0 2024-08-18T23:48:52.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.019+0000 7effaffe8640 1 -- 172.21.15.78:0/1405316183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa81b3670 con 0x7effa8103ac0 2024-08-18T23:48:52.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.019+0000 7effaffe8640 1 -- 172.21.15.78:0/1405316183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa81b3ad0 con 0x7effa8103ac0 2024-08-18T23:48:52.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.023+0000 7effb23e6740 1 Processor -- start 2024-08-18T23:48:52.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.023+0000 7eff9effd640 1 -- 172.21.15.78:0/1405316183 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7effa0003970 con 0x7effa8103ac0 2024-08-18T23:48:52.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.024+0000 7effb23e6740 1 -- start start 2024-08-18T23:48:52.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.024+0000 7effb23e6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 0x55cabc30e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:52.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.024+0000 7effb23e6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc30eb20 0x55cabc318ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:52.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.024+0000 7effae585640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 0x55cabc30e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:52.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.024+0000 7effae585640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 0x55cabc30e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:52024/0 (socket says 172.21.15.78:52024) 2024-08-18T23:48:52.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.024+0000 7effb23e6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc3193e0 0x55cabc31b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:52.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.024+0000 7effb23e6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cabc2d1e40 con 0x55cabc3193e0 2024-08-18T23:48:52.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.024+0000 7effb23e6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cabc2737f0 con 0x55cabc30e200 2024-08-18T23:48:52.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.024+0000 7effb23e6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cabc21afb0 con 0x55cabc30eb20 2024-08-18T23:48:52.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.024+0000 7effae585640 1 -- 172.21.15.78:0/925376911 learned_addr learned my addr 172.21.15.78:0/925376911 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:52.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.024+0000 7effadd84640 1 --2- 172.21.15.78:0/925376911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc30eb20 0x55cabc318ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:52.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.025+0000 7eff9effd640 1 --2- 172.21.15.78:0/1405316183 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7eff940a2cc0 0x7eff940a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:52.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.025+0000 7effae585640 1 -- 172.21.15.78:0/925376911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc30eb20 msgr2=0x55cabc318ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:52.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.025+0000 7effad583640 1 --2- 172.21.15.78:0/925376911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc3193e0 0x55cabc31b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:52.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.025+0000 7effae585640 1 --2- 172.21.15.78:0/925376911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc30eb20 0x55cabc318ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.025+0000 7effae585640 1 -- 172.21.15.78:0/925376911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc3193e0 msgr2=0x55cabc31b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:52.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.025+0000 7effae585640 1 --2- 172.21.15.78:0/925376911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc3193e0 0x55cabc31b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.025+0000 7effae585640 1 -- 172.21.15.78:0/925376911 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cabbfe56a0 con 0x55cabc30e200 2024-08-18T23:48:52.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.025+0000 7effad583640 1 --2- 172.21.15.78:0/925376911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc3193e0 0x55cabc31b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:48:52.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.025+0000 7eff9effd640 1 -- 172.21.15.78:0/1405316183 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7effa00cf2e0 con 0x7effa8103ac0 2024-08-18T23:48:52.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.025+0000 7effae585640 1 --2- 172.21.15.78:0/1405316183 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7eff940a2cc0 0x7eff940a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:52.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.025+0000 7effadd84640 1 --2- 172.21.15.78:0/925376911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc30eb20 0x55cabc318ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:48:52.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.025+0000 7effae585640 1 --2- 172.21.15.78:0/925376911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 0x55cabc30e5e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7effa4009e00 tx=0x7effa4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=784d6f32e64b0163 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:52.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.026+0000 7eff93fff640 1 -- 172.21.15.78:0/925376911 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4017050 con 0x55cabc30e200 2024-08-18T23:48:52.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.026+0000 7eff93fff640 1 -- 172.21.15.78:0/925376911 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa4026b20 con 0x55cabc30e200 2024-08-18T23:48:52.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.026+0000 7eff93fff640 1 -- 172.21.15.78:0/925376911 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa400f940 con 0x55cabc30e200 2024-08-18T23:48:52.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.026+0000 7effae585640 1 --2- 172.21.15.78:0/1405316183 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7eff940a2cc0 0x7eff940a5180 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7effa4023000 tx=0x7effa4024000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:52.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.026+0000 7effb23e6740 1 -- 172.21.15.78:0/925376911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 msgr2=0x55cabc30e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:52.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.026+0000 7effb23e6740 1 --2- 172.21.15.78:0/925376911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 0x55cabc30e5e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7effa4009e00 tx=0x7effa4011810 comp rx=0 tx=0).stop 2024-08-18T23:48:52.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.026+0000 7effb23e6740 1 -- 172.21.15.78:0/925376911 shutdown_connections 2024-08-18T23:48:52.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.026+0000 7effb23e6740 1 --2- 172.21.15.78:0/925376911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc3193e0 0x55cabc31b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.026+0000 7effb23e6740 1 --2- 172.21.15.78:0/925376911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc30eb20 0x55cabc318ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.026+0000 7effb23e6740 1 --2- 172.21.15.78:0/925376911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 0x55cabc30e5e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.027+0000 7effb23e6740 1 -- 172.21.15.78:0/925376911 >> 172.21.15.78:0/925376911 conn(0x55cabc30a790 msgr2=0x55cabc30ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:52.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.027+0000 7effb23e6740 1 -- 172.21.15.78:0/925376911 shutdown_connections 2024-08-18T23:48:52.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.027+0000 7effb23e6740 1 -- 172.21.15.78:0/925376911 wait complete. 2024-08-18T23:48:52.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.027+0000 7effb23e6740 1 -- start start 2024-08-18T23:48:52.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.027+0000 7effb23e6740 1 Processor -- start 2024-08-18T23:48:52.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.027+0000 7effb23e6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 0x55cabc3162f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:52.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.027+0000 7effb23e6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc30eb20 0x55cabc316830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:52.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.027+0000 7effb23e6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc3193e0 0x55cabc316d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:52.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.027+0000 7effb23e6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cabc22ecb0 con 0x55cabc30eb20 2024-08-18T23:48:52.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.027+0000 7effb23e6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cabc26f7b0 con 0x55cabc30e200 2024-08-18T23:48:52.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.027+0000 7effb23e6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cabc31bfb0 con 0x55cabc3193e0 2024-08-18T23:48:52.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.028+0000 7effb23e6740 10 client.0 _ll_register_callbacks cb 0x55cabbf82ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:48:52.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.028+0000 7effad583640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc30eb20 0x55cabc316830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:52.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.028+0000 7effad583640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc30eb20 0x55cabc316830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49656/0 (socket says 172.21.15.78:49656) 2024-08-18T23:48:52.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.028+0000 7effadd84640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 0x55cabc3162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:52.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.028+0000 7effad583640 1 -- 172.21.15.78:0/1404281981 learned_addr learned my addr 172.21.15.78:0/1404281981 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:52.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.028+0000 7effae585640 1 --2- 172.21.15.78:0/1404281981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc3193e0 0x55cabc316d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:52.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.028+0000 7effad583640 1 -- 172.21.15.78:0/1404281981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc3193e0 msgr2=0x55cabc316d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:52.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.028+0000 7effad583640 1 --2- 172.21.15.78:0/1404281981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc3193e0 0x55cabc316d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.028+0000 7effad583640 1 -- 172.21.15.78:0/1404281981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 msgr2=0x55cabc3162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:52.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.028+0000 7effad583640 1 --2- 172.21.15.78:0/1404281981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 0x55cabc3162f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.028+0000 7effad583640 1 -- 172.21.15.78:0/1404281981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cabc22dbc0 con 0x55cabc30eb20 2024-08-18T23:48:52.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.028+0000 7effadd84640 1 --2- 172.21.15.78:0/1404281981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 0x55cabc3162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:48:52.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.028+0000 7effad583640 1 --2- 172.21.15.78:0/1404281981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc30eb20 0x55cabc316830 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7eff98000c00 tx=0x7eff98004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346dd3635aed89f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:52.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.028+0000 7eff917fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:48:52.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.028+0000 7eff917fa640 1 -- 172.21.15.78:0/1404281981 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98037070 con 0x55cabc30eb20 2024-08-18T23:48:52.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.028+0000 7effb23e6740 1 -- 172.21.15.78:0/1404281981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effa4026b20 con 0x55cabc30eb20 2024-08-18T23:48:52.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.028+0000 7eff917fa640 1 -- 172.21.15.78:0/1404281981 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7eff98030750 con 0x55cabc30eb20 2024-08-18T23:48:52.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.028+0000 7eff917fa640 1 -- 172.21.15.78:0/1404281981 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff9804a050 con 0x55cabc30eb20 2024-08-18T23:48:52.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.028+0000 7effb23e6740 10 client.16257 fetch_fsmap learned FSMap version 28 2024-08-18T23:48:52.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.029+0000 7effb23e6740 1 -- 172.21.15.78:0/1404281981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cabc327e10 con 0x55cabc30eb20 2024-08-18T23:48:52.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.029+0000 7eff917fa640 1 -- 172.21.15.78:0/1404281981 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa4026b20 con 0x55cabc30eb20 2024-08-18T23:48:52.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.029+0000 7eff917fa640 1 -- 172.21.15.78:0/1404281981 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7eff98045030 con 0x55cabc30eb20 2024-08-18T23:48:52.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.029+0000 7effb23e6740 10 client.16257 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:48:52.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.029+0000 7effb23e6740 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:48:52.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.029+0000 7effb23e6740 20 client.16257 populate_metadata read hostname 'smithi078' 2024-08-18T23:48:52.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.029+0000 7effb23e6740 1 --2- 172.21.15.78:0/1404281981 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cabc329320 0x55cabc32b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:52.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.029+0000 7effb23e6740 4 client.16257 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:48:52.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.029+0000 7effb23e6740 1 -- 172.21.15.78:0/1404281981 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cabc280d60 con 0x55cabc329320 2024-08-18T23:48:52.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.029+0000 7effadd84640 1 --2- 172.21.15.78:0/1404281981 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cabc329320 0x55cabc32b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:52.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.029+0000 7eff917fa640 10 client.16257 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:52.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.029+0000 7effadd84640 1 --2- 172.21.15.78:0/1404281981 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cabc329320 0x55cabc32b7c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71dbdc626b55d902 server_cookie=3461e75f3db37923 in_seq=0 out_seq=0 2024-08-18T23:48:52.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.032+0000 7eff917fa640 1 -- 172.21.15.78:0/1404281981 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cabc280d60 con 0x55cabc329320 2024-08-18T23:48:52.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.032+0000 7eff917fa640 10 client.16257 handle_command_reply: tid=0 2024-08-18T23:48:52.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.032+0000 7effb23e6740 1 client.16257 shutdown 2024-08-18T23:48:52.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.033+0000 7effb23e6740 1 -- 172.21.15.78:0/1404281981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc30eb20 msgr2=0x55cabc316830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:52.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.033+0000 7effb23e6740 1 --2- 172.21.15.78:0/1404281981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc30eb20 0x55cabc316830 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7eff98000c00 tx=0x7eff98004270 comp rx=0 tx=0).stop 2024-08-18T23:48:52.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.033+0000 7effb23e6740 1 -- 172.21.15.78:0/1404281981 shutdown_connections 2024-08-18T23:48:52.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.033+0000 7effb23e6740 1 --2- 172.21.15.78:0/1404281981 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cabc329320 0x55cabc32b7c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.033+0000 7effb23e6740 1 --2- 172.21.15.78:0/1404281981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc3193e0 0x55cabc316d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.033+0000 7eff917fa640 0 client.16257 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:52.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.033+0000 7effb23e6740 1 --2- 172.21.15.78:0/1404281981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc30eb20 0x55cabc316830 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.033+0000 7effb23e6740 1 --2- 172.21.15.78:0/1404281981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 0x55cabc3162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.033+0000 7effb23e6740 1 -- 172.21.15.78:0/1404281981 >> 172.21.15.78:0/1404281981 conn(0x55cabc30a790 msgr2=0x55cabc3098f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:52.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.033+0000 7effb23e6740 1 -- 172.21.15.78:0/1404281981 shutdown_connections 2024-08-18T23:48:52.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.033+0000 7effb23e6740 1 -- 172.21.15.78:0/1404281981 wait complete. 2024-08-18T23:48:52.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.033+0000 7effb23e6740 20 client.16257 trim_cache size 0 max 16384 2024-08-18T23:48:52.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effb23e6740 1 Processor -- start 2024-08-18T23:48:52.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effb23e6740 1 -- start start 2024-08-18T23:48:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effb23e6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 0x55cabc3310d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effb23e6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc30eb20 0x55cabc331610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effb23e6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc3193e0 0x55cabc3090d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effb23e6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7effa81c0250 con 0x55cabc3193e0 2024-08-18T23:48:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effb23e6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7effa81bdb20 con 0x55cabc30e200 2024-08-18T23:48:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effb23e6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7effa81bb930 con 0x55cabc30eb20 2024-08-18T23:48:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effad583640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc30eb20 0x55cabc331610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effadd84640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 0x55cabc3310d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effae585640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc3193e0 0x55cabc3090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effadd84640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 0x55cabc3310d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:52054/0 (socket says 172.21.15.78:52054) 2024-08-18T23:48:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effad583640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc30eb20 0x55cabc331610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:33518/0 (socket says 172.21.15.78:33518) 2024-08-18T23:48:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effae585640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc3193e0 0x55cabc3090d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49668/0 (socket says 172.21.15.78:49668) 2024-08-18T23:48:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effadd84640 1 -- 172.21.15.78:0/418703233 learned_addr learned my addr 172.21.15.78:0/418703233 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effadd84640 1 -- 172.21.15.78:0/418703233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc30eb20 msgr2=0x55cabc331610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effadd84640 1 --2- 172.21.15.78:0/418703233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc30eb20 0x55cabc331610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effadd84640 1 -- 172.21.15.78:0/418703233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc3193e0 msgr2=0x55cabc3090d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effadd84640 1 --2- 172.21.15.78:0/418703233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc3193e0 0x55cabc3090d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.052+0000 7effadd84640 1 -- 172.21.15.78:0/418703233 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cabc30e950 con 0x55cabc30e200 2024-08-18T23:48:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.053+0000 7effadd84640 1 --2- 172.21.15.78:0/418703233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 0x55cabc3310d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7effa009b450 tx=0x7effa00cc8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cb32ac9bab1848c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:52.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.053+0000 7eff937fe640 1 -- 172.21.15.78:0/418703233 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa009be50 con 0x55cabc30e200 2024-08-18T23:48:52.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.053+0000 7eff937fe640 1 -- 172.21.15.78:0/418703233 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa009b750 con 0x55cabc30e200 2024-08-18T23:48:52.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.053+0000 7eff937fe640 1 -- 172.21.15.78:0/418703233 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa00860b0 con 0x55cabc30e200 2024-08-18T23:48:52.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.054+0000 7effb23e6740 1 -- 172.21.15.78:0/418703233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 msgr2=0x55cabc3310d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:52.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.054+0000 7effb23e6740 1 --2- 172.21.15.78:0/418703233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 0x55cabc3310d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7effa009b450 tx=0x7effa00cc8c0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.054+0000 7effb23e6740 1 -- 172.21.15.78:0/418703233 shutdown_connections 2024-08-18T23:48:52.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.054+0000 7effb23e6740 1 --2- 172.21.15.78:0/418703233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc3193e0 0x55cabc3090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.054+0000 7effb23e6740 1 --2- 172.21.15.78:0/418703233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc30eb20 0x55cabc331610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.054+0000 7effb23e6740 1 --2- 172.21.15.78:0/418703233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc30e200 0x55cabc3310d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.054+0000 7effb23e6740 1 -- 172.21.15.78:0/418703233 >> 172.21.15.78:0/418703233 conn(0x55cabc30a790 msgr2=0x55cabc314bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:52.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.054+0000 7effb23e6740 1 -- 172.21.15.78:0/418703233 shutdown_connections 2024-08-18T23:48:52.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.054+0000 7effb23e6740 1 -- 172.21.15.78:0/418703233 wait complete. 2024-08-18T23:48:52.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.054+0000 7effb23e6740 1 -- start start 2024-08-18T23:48:52.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.054+0000 7effb23e6740 1 Processor -- start 2024-08-18T23:48:52.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.055+0000 7effb23e6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc30e200 0x55cabc31c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:52.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.055+0000 7effb23e6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc30eb20 0x55cabc31cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:52.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.055+0000 7effb23e6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc3193e0 0x55cabc321020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:52.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.055+0000 7effb23e6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7effa81bbfe0 con 0x55cabc30e200 2024-08-18T23:48:52.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.055+0000 7effb23e6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7effa81bd450 con 0x55cabc3193e0 2024-08-18T23:48:52.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.055+0000 7effb23e6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7effa81bc6b0 con 0x55cabc30eb20 2024-08-18T23:48:52.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.055+0000 7effadd84640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc30e200 0x55cabc31c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:52.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.055+0000 7effb23e6740 10 client.0 _ll_register_callbacks cb 0x7eff98035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:48:52.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.055+0000 7effadd84640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc30e200 0x55cabc31c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49674/0 (socket says 172.21.15.78:49674) 2024-08-18T23:48:52.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.055+0000 7effae585640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc3193e0 0x55cabc321020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:52.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.055+0000 7effadd84640 1 -- 172.21.15.78:0/1067386270 learned_addr learned my addr 172.21.15.78:0/1067386270 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:52.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.055+0000 7effad583640 1 --2- 172.21.15.78:0/1067386270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc30eb20 0x55cabc31cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:52.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.055+0000 7effae585640 1 -- 172.21.15.78:0/1067386270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc30eb20 msgr2=0x55cabc31cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:52.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.055+0000 7effae585640 1 --2- 172.21.15.78:0/1067386270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc30eb20 0x55cabc31cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.055+0000 7effae585640 1 -- 172.21.15.78:0/1067386270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc30e200 msgr2=0x55cabc31c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:52.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.055+0000 7effae585640 1 --2- 172.21.15.78:0/1067386270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc30e200 0x55cabc31c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.055+0000 7effae585640 1 -- 172.21.15.78:0/1067386270 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cabc321710 con 0x55cabc3193e0 2024-08-18T23:48:52.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.055+0000 7effae585640 1 --2- 172.21.15.78:0/1067386270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc3193e0 0x55cabc321020 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7effa4011d10 tx=0x7effa4013030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf2d27cdde6ebcd6 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:52.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.055+0000 7eff917fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:48:52.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.056+0000 7eff917fa640 1 -- 172.21.15.78:0/1067386270 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4035070 con 0x55cabc3193e0 2024-08-18T23:48:52.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.056+0000 7eff917fa640 1 -- 172.21.15.78:0/1067386270 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7effa4026640 con 0x55cabc3193e0 2024-08-18T23:48:52.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.056+0000 7effb23e6740 1 -- 172.21.15.78:0/1067386270 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effa009b750 con 0x55cabc3193e0 2024-08-18T23:48:52.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.056+0000 7eff917fa640 1 -- 172.21.15.78:0/1067386270 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7effa4017020 con 0x55cabc3193e0 2024-08-18T23:48:52.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.056+0000 7effb23e6740 10 client.25691 fetch_fsmap learned FSMap version 28 2024-08-18T23:48:52.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.056+0000 7effb23e6740 1 -- 172.21.15.78:0/1067386270 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cabc393c80 con 0x55cabc3193e0 2024-08-18T23:48:52.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.056+0000 7eff917fa640 1 -- 172.21.15.78:0/1067386270 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa402c040 con 0x55cabc3193e0 2024-08-18T23:48:52.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.056+0000 7eff917fa640 1 -- 172.21.15.78:0/1067386270 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7effa40318d0 con 0x55cabc3193e0 2024-08-18T23:48:52.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.056+0000 7effb23e6740 10 client.25691 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:48:52.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.056+0000 7effb23e6740 10 client.25691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:48:52.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.056+0000 7effb23e6740 20 client.25691 populate_metadata read hostname 'smithi078' 2024-08-18T23:48:52.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.056+0000 7effb23e6740 1 --2- 172.21.15.78:0/1067386270 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cabc329320 0x55cabc3962a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:52.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.056+0000 7effb23e6740 4 client.25691 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-18T23:48:52.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.056+0000 7effb23e6740 1 -- 172.21.15.78:0/1067386270 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7effa009b750 con 0x55cabc329320 2024-08-18T23:48:52.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.056+0000 7effadd84640 1 --2- 172.21.15.78:0/1067386270 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cabc329320 0x55cabc3962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:52.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.057+0000 7eff917fa640 10 client.25691 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:52.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.057+0000 7effadd84640 1 --2- 172.21.15.78:0/1067386270 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cabc329320 0x55cabc3962a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f88473a39d79e8d0 server_cookie=4b4c1d8a371f7bb9 in_seq=0 out_seq=0 2024-08-18T23:48:52.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.067+0000 7eff917fa640 1 -- 172.21.15.78:0/1067386270 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7effa009b750 con 0x55cabc329320 2024-08-18T23:48:52.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.067+0000 7eff917fa640 10 client.25691 handle_command_reply: tid=0 2024-08-18T23:48:52.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.067+0000 7effb23e6740 1 client.25691 shutdown 2024-08-18T23:48:52.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.067+0000 7effb23e6740 1 -- 172.21.15.78:0/1067386270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc3193e0 msgr2=0x55cabc321020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:52.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.067+0000 7effb23e6740 1 --2- 172.21.15.78:0/1067386270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc3193e0 0x55cabc321020 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7effa4011d10 tx=0x7effa4013030 comp rx=0 tx=0).stop 2024-08-18T23:48:52.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.068+0000 7effb23e6740 1 -- 172.21.15.78:0/1067386270 shutdown_connections 2024-08-18T23:48:52.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.068+0000 7effb23e6740 1 --2- 172.21.15.78:0/1067386270 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cabc329320 0x55cabc3962a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.068+0000 7effb23e6740 1 --2- 172.21.15.78:0/1067386270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cabc3193e0 0x55cabc321020 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.068+0000 7effb23e6740 1 --2- 172.21.15.78:0/1067386270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cabc30eb20 0x55cabc31cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.068+0000 7eff917fa640 0 client.25691 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:52.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.068+0000 7effb23e6740 1 --2- 172.21.15.78:0/1067386270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cabc30e200 0x55cabc31c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.068+0000 7effb23e6740 1 -- 172.21.15.78:0/1067386270 >> 172.21.15.78:0/1067386270 conn(0x55cabc30a790 msgr2=0x55cabc3090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:52.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.068+0000 7effb23e6740 1 -- 172.21.15.78:0/1067386270 shutdown_connections 2024-08-18T23:48:52.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.068+0000 7effb23e6740 1 -- 172.21.15.78:0/1067386270 wait complete. 2024-08-18T23:48:52.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.068+0000 7effb23e6740 20 client.25691 trim_cache size 0 max 16384 2024-08-18T23:48:52.070 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:48:52.070 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-08-18T23:48:52.071 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "dd2e1456-db13-4844-a083-fe682462993f", 2024-08-18T23:48:52.071 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-08-18T23:48:52.071 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:48:52.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.069+0000 7effaffe8640 1 -- 172.21.15.78:0/1405316183 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7eff940a2cc0 msgr2=0x7eff940a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:52.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.069+0000 7effaffe8640 1 --2- 172.21.15.78:0/1405316183 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7eff940a2cc0 0x7eff940a5180 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7effa4023000 tx=0x7effa4024000 comp rx=0 tx=0).stop 2024-08-18T23:48:52.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.069+0000 7effaffe8640 1 -- 172.21.15.78:0/1405316183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effa8103ac0 msgr2=0x7effa810ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:52.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.069+0000 7effaffe8640 1 --2- 172.21.15.78:0/1405316183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effa8103ac0 0x7effa810ee80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7effa000db80 tx=0x7effa000df80 comp rx=0 tx=0).stop 2024-08-18T23:48:52.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.069+0000 7effaffe8640 1 -- 172.21.15.78:0/1405316183 shutdown_connections 2024-08-18T23:48:52.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.069+0000 7effaffe8640 1 --2- 172.21.15.78:0/1405316183 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7eff940a2cc0 0x7eff940a5180 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.069+0000 7effaffe8640 1 --2- 172.21.15.78:0/1405316183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa8109ac0 0x7effa810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.069+0000 7effaffe8640 1 --2- 172.21.15.78:0/1405316183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effa8104460 0x7effa811b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.069+0000 7effaffe8640 1 --2- 172.21.15.78:0/1405316183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effa8103ac0 0x7effa810ee80 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:52.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.069+0000 7effaffe8640 1 -- 172.21.15.78:0/1405316183 >> 172.21.15.78:0/1405316183 conn(0x7effa80ff7e0 msgr2=0x7effa8100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:52.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.069+0000 7effaffe8640 1 -- 172.21.15.78:0/1405316183 shutdown_connections 2024-08-18T23:48:52.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:52.069+0000 7effaffe8640 1 -- 172.21.15.78:0/1405316183 wait complete. 2024-08-18T23:48:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:52 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:48:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:52 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:52.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:52 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:48:52.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:52 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:52.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:52 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:48:52.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:52 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:52.798 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:48:52 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-b[51547]: 2024-08-18T23:48:52.556+0000 7f7b33a10640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000005059 [...2,head] /volumes/qa/sv_1/5f1302ca-91a6-4860-a5ba-eb68e67b3c86/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-82/ auth{1=1} v238155 ap=2 DIRTYPARENT RANDEPHEMERALPIN f(v3 m2024-08-18T23:48:39.020667+0000 85=85+0) n(v3 rc2024-08-18T23:48:39.020667+0000 b1626124 86=85+1) (inest sync->mix g=1) caps={24760=pAsLsXsFs/pAsLsXsFsx@110} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=1 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55f0e5041b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(2)0x10000005059:[<0x10000000399/blog-82 v238155>,<0x10000000208/blogtest_in v1139>,<0x10000000201/src v2363>,<0x10000000006/blogbench-1.0 v2058>,<0x10000000005/tmp v319>,<0x100000001fb/client.0 v312>,<0x100000001fa/5f1302ca-91a6-4860-a5ba-eb68e67b3c86 v323>,<0x10000000001/sv_1 v338>,<0x10000000000/qa v328>,<0x1/volumes v322>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-18T23:48:30.340798+0000 83=83+0)","ondisk_value.rstat":"n(v0 rc2024-08-18T23:48:39.020667+0000 b1626124 86=85+1)","memory_value.dirstat":"f(v3 m2024-08-18T23:48:39.020667+0000 85=85+0)","memory_value.rstat":"n(v3 rc2024-08-18T23:48:39.020667+0000 b1626124 86=85+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-08-18T23:48:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:53 smithi161 ceph-mon[22027]: scrub queued for path: / 2024-08-18T23:48:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:53 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:53 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:48:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:53 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:48:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:53 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:53 smithi161 ceph-mon[22027]: pgmap v487: 129 pgs: 129 active+clean; 570 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.5 MiB/s wr, 815 op/s 2024-08-18T23:48:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:53 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:48:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:53 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:48:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:53 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:53 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:48:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:53 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:53 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:53 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:48:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:53 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:53 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:48:53.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:53 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:53 smithi078 ceph-mon[18440]: scrub queued for path: / 2024-08-18T23:48:53.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:53 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:53 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:48:53.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:53 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:48:53.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:53 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:53 smithi078 ceph-mon[18440]: pgmap v487: 129 pgs: 129 active+clean; 570 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.5 MiB/s wr, 815 op/s 2024-08-18T23:48:53.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:53 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:48:53.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:53 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:48:53.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:53 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:53 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:48:53.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:53 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:53 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:53 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:48:53.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:53 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:53 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:48:53.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:53 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:53 smithi123 ceph-mon[21808]: scrub queued for path: / 2024-08-18T23:48:53.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:53 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:53 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:48:53.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:53 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:48:53.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:53 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:53 smithi123 ceph-mon[21808]: pgmap v487: 129 pgs: 129 active+clean; 570 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.5 MiB/s wr, 815 op/s 2024-08-18T23:48:53.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:53 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:48:53.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:53 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:48:53.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:53 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:53 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:48:53.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:53 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:53 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:53 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:48:53.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:53 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:53.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:53 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:48:53.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:53 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:48:54.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:54 smithi161 ceph-mon[22027]: Scrub error on inode 0x10000005059 (/volumes/qa/sv_1/5f1302ca-91a6-4860-a5ba-eb68e67b3c86/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-82) see mds.b log and `damage ls` output for details 2024-08-18T23:48:54.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:54 smithi078 ceph-mon[18440]: Scrub error on inode 0x10000005059 (/volumes/qa/sv_1/5f1302ca-91a6-4860-a5ba-eb68e67b3c86/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-82) see mds.b log and `damage ls` output for details 2024-08-18T23:48:54.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:54 smithi123 ceph-mon[21808]: Scrub error on inode 0x10000005059 (/volumes/qa/sv_1/5f1302ca-91a6-4860-a5ba-eb68e67b3c86/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-82) see mds.b log and `damage ls` output for details 2024-08-18T23:48:55.328 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:48:55.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:55 smithi078 ceph-mon[18440]: pgmap v488: 129 pgs: 129 active+clean; 556 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 783 op/s 2024-08-18T23:48:55.656 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:48:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:55 smithi161 ceph-mon[22027]: pgmap v488: 129 pgs: 129 active+clean; 556 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 783 op/s 2024-08-18T23:48:55.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:55 smithi123 ceph-mon[21808]: pgmap v488: 129 pgs: 129 active+clean; 556 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 783 op/s 2024-08-18T23:48:56.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.478+0000 7f3af5be9640 1 -- 172.21.15.78:0/655741667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af0100460 msgr2=0x7f3af010c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:56.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.478+0000 7f3af5be9640 1 --2- 172.21.15.78:0/655741667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af0100460 0x7f3af010c990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3ae001a940 tx=0x7f3ae003b490 comp rx=0 tx=0).stop 2024-08-18T23:48:56.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.478+0000 7f3af5be9640 1 -- 172.21.15.78:0/655741667 shutdown_connections 2024-08-18T23:48:56.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.478+0000 7f3af5be9640 1 --2- 172.21.15.78:0/655741667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af0100460 0x7f3af010c990 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.478+0000 7f3af5be9640 1 --2- 172.21.15.78:0/655741667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3af00ffac0 0x7f3af00fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.478+0000 7f3af5be9640 1 --2- 172.21.15.78:0/655741667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3af0105ac0 0x7f3af0105ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.478+0000 7f3af5be9640 1 -- 172.21.15.78:0/655741667 >> 172.21.15.78:0/655741667 conn(0x7f3af00fb7c0 msgr2=0x7f3af00fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:56.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.479+0000 7f3af5be9640 1 -- 172.21.15.78:0/655741667 shutdown_connections 2024-08-18T23:48:56.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.479+0000 7f3af5be9640 1 -- 172.21.15.78:0/655741667 wait complete. 2024-08-18T23:48:56.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.479+0000 7f3af5be9640 1 Processor -- start 2024-08-18T23:48:56.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.480+0000 7f3af5be9640 1 -- start start 2024-08-18T23:48:56.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.480+0000 7f3af5be9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3af00ffac0 0x7f3af0071610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:56.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.480+0000 7f3af5be9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af0105ac0 0x7f3af0071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:56.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.480+0000 7f3af5be9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3af010b980 0x7f3af010be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:56.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.480+0000 7f3af5be9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3af006e3b0 con 0x7f3af0105ac0 2024-08-18T23:48:56.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.480+0000 7f3af5be9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3af006e230 con 0x7f3af00ffac0 2024-08-18T23:48:56.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.480+0000 7f3af5be9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3af006e530 con 0x7f3af010b980 2024-08-18T23:48:56.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.480+0000 7f3aef7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3af00ffac0 0x7f3af0071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:56.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.480+0000 7f3aef7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3af00ffac0 0x7f3af0071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:47078/0 (socket says 172.21.15.78:47078) 2024-08-18T23:48:56.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.480+0000 7f3aef7fe640 1 -- 172.21.15.78:0/3896905149 learned_addr learned my addr 172.21.15.78:0/3896905149 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:56.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.481+0000 7f3aeffff640 1 --2- 172.21.15.78:0/3896905149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3af010b980 0x7f3af010be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:56.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.481+0000 7f3aeeffd640 1 --2- 172.21.15.78:0/3896905149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af0105ac0 0x7f3af0071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:56.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.482+0000 7f3aef7fe640 1 -- 172.21.15.78:0/3896905149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3af010b980 msgr2=0x7f3af010be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:56.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.482+0000 7f3aef7fe640 1 --2- 172.21.15.78:0/3896905149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3af010b980 0x7f3af010be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.482+0000 7f3aef7fe640 1 -- 172.21.15.78:0/3896905149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af0105ac0 msgr2=0x7f3af0071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:56.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.482+0000 7f3aef7fe640 1 --2- 172.21.15.78:0/3896905149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af0105ac0 0x7f3af0071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.482+0000 7f3aef7fe640 1 -- 172.21.15.78:0/3896905149 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af010c470 con 0x7f3af00ffac0 2024-08-18T23:48:56.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.483+0000 7f3aef7fe640 1 --2- 172.21.15.78:0/3896905149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3af00ffac0 0x7f3af0071610 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800ba80 tx=0x7f3ae800bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:56.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.483+0000 7f3aecff9640 1 -- 172.21.15.78:0/3896905149 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8019070 con 0x7f3af00ffac0 2024-08-18T23:48:56.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.483+0000 7f3af5be9640 1 -- 172.21.15.78:0/3896905149 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af010c760 con 0x7f3af00ffac0 2024-08-18T23:48:56.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.484+0000 7f3af5be9640 1 -- 172.21.15.78:0/3896905149 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af010cca0 con 0x7f3af00ffac0 2024-08-18T23:48:56.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.489+0000 7f3af7fe7740 1 Processor -- start 2024-08-18T23:48:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.490+0000 7f3af7fe7740 1 -- start start 2024-08-18T23:48:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.490+0000 7f3af7fe7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046dc210 0x5617046dc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.490+0000 7f3af7fe7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dcb30 0x5617046e6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.490+0000 7f3af7fe7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046e73f0 0x5617046e97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.490+0000 7f3af7fe7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561704641980 con 0x5617046dc210 2024-08-18T23:48:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.490+0000 7f3af7fe7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5617045d9a70 con 0x5617046e73f0 2024-08-18T23:48:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.490+0000 7f3af7fe7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56170463d940 con 0x5617046dcb30 2024-08-18T23:48:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.490+0000 7f3aecff9640 1 -- 172.21.15.78:0/3896905149 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae800cc90 con 0x7f3af00ffac0 2024-08-18T23:48:56.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.490+0000 7f3aecff9640 1 -- 172.21.15.78:0/3896905149 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8007730 con 0x7f3af00ffac0 2024-08-18T23:48:56.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.491+0000 7f3aeeffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046e73f0 0x5617046e97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:56.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.491+0000 7f3aeeffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046e73f0 0x5617046e97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:47088/0 (socket says 172.21.15.78:47088) 2024-08-18T23:48:56.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.491+0000 7f3aeeffd640 1 -- 172.21.15.78:0/2625151465 learned_addr learned my addr 172.21.15.78:0/2625151465 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:56.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.491+0000 7f3aeffff640 1 --2- 172.21.15.78:0/2625151465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046dc210 0x5617046dc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:56.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.491+0000 7f3aef7fe640 1 --2- 172.21.15.78:0/2625151465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dcb30 0x5617046e6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:56.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.491+0000 7f3aeeffd640 1 -- 172.21.15.78:0/2625151465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dcb30 msgr2=0x5617046e6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:56.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.491+0000 7f3aeeffd640 1 --2- 172.21.15.78:0/2625151465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dcb30 0x5617046e6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.491+0000 7f3aeeffd640 1 -- 172.21.15.78:0/2625151465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046dc210 msgr2=0x5617046dc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:56.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.491+0000 7f3aeeffd640 1 --2- 172.21.15.78:0/2625151465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046dc210 0x5617046dc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.491+0000 7f3aeeffd640 1 -- 172.21.15.78:0/2625151465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56170463c6e0 con 0x5617046e73f0 2024-08-18T23:48:56.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.492+0000 7f3aef7fe640 1 --2- 172.21.15.78:0/2625151465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dcb30 0x5617046e6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:48:56.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.492+0000 7f3aeeffd640 1 --2- 172.21.15.78:0/2625151465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046e73f0 0x5617046e97e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40099e0 tx=0x7f3ae4009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcae54f0bfdfb5fd server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:56.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.494+0000 7f3aecff9640 1 -- 172.21.15.78:0/3896905149 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3ae801c070 con 0x7f3af00ffac0 2024-08-18T23:48:56.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.495+0000 7f3aecff9640 1 --2- 172.21.15.78:0/3896905149 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3ad80a2cc0 0x7f3ad80a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:56.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.496+0000 7f3aecff9640 1 -- 172.21.15.78:0/3896905149 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f3ae80cec50 con 0x7f3af00ffac0 2024-08-18T23:48:56.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.496+0000 7f3ad5ffb640 1 -- 172.21.15.78:0/2625151465 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4017070 con 0x5617046e73f0 2024-08-18T23:48:56.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.496+0000 7f3ad5ffb640 1 -- 172.21.15.78:0/2625151465 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae4013470 con 0x5617046e73f0 2024-08-18T23:48:56.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.497+0000 7f3af7fe7740 1 -- 172.21.15.78:0/2625151465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046e73f0 msgr2=0x5617046e97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:56.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.497+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/2625151465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046e73f0 0x5617046e97e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40099e0 tx=0x7f3ae4009ea0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.498+0000 7f3af7fe7740 1 -- 172.21.15.78:0/2625151465 shutdown_connections 2024-08-18T23:48:56.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.498+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/2625151465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046e73f0 0x5617046e97e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.498+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/2625151465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dcb30 0x5617046e6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.498+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/2625151465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046dc210 0x5617046dc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.498+0000 7f3af7fe7740 1 -- 172.21.15.78:0/2625151465 >> 172.21.15.78:0/2625151465 conn(0x5617046d87a0 msgr2=0x5617046d8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:56.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.498+0000 7f3af7fe7740 1 -- 172.21.15.78:0/2625151465 shutdown_connections 2024-08-18T23:48:56.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.498+0000 7f3af7fe7740 1 -- 172.21.15.78:0/2625151465 wait complete. 2024-08-18T23:48:56.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.498+0000 7f3af7fe7740 1 -- start start 2024-08-18T23:48:56.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.499+0000 7f3af7fe7740 1 Processor -- start 2024-08-18T23:48:56.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.499+0000 7f3aeffff640 1 --2- 172.21.15.78:0/3896905149 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3ad80a2cc0 0x7f3ad80a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:56.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.499+0000 7f3af7fe7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dc210 0x5617046e9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:56.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.499+0000 7f3af7fe7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046dcb30 0x5617046e6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:56.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.499+0000 7f3af7fe7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046e6d80 0x5617046e7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:56.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.499+0000 7f3af7fe7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5617046ea010 con 0x5617046e6d80 2024-08-18T23:48:56.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.499+0000 7f3af7fe7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5617045fcaa0 con 0x5617046dcb30 2024-08-18T23:48:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.499+0000 7f3af7fe7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5617046ea190 con 0x5617046dc210 2024-08-18T23:48:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.499+0000 7f3af7fe7740 10 client.0 _ll_register_callbacks cb 0x5617045ebe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:48:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.500+0000 7f3aeffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046e6d80 0x5617046e7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.500+0000 7f3aeffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046e6d80 0x5617046e7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45644/0 (socket says 172.21.15.78:45644) 2024-08-18T23:48:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.500+0000 7f3aeffff640 1 -- 172.21.15.78:0/2695677228 learned_addr learned my addr 172.21.15.78:0/2695677228 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.500+0000 7f3aeeffd640 1 --2- 172.21.15.78:0/2695677228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046dcb30 0x5617046e6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.500+0000 7f3aef7fe640 1 --2- 172.21.15.78:0/2695677228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dc210 0x5617046e9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.501+0000 7f3aeeffd640 1 -- 172.21.15.78:0/2695677228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dc210 msgr2=0x5617046e9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.501+0000 7f3aeeffd640 1 --2- 172.21.15.78:0/2695677228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dc210 0x5617046e9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.501+0000 7f3aeeffd640 1 -- 172.21.15.78:0/2695677228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046e6d80 msgr2=0x5617046e7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.501+0000 7f3aeeffd640 1 --2- 172.21.15.78:0/2695677228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046e6d80 0x5617046e7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.501+0000 7f3aeeffd640 1 -- 172.21.15.78:0/2695677228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617045e0650 con 0x5617046dcb30 2024-08-18T23:48:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.501+0000 7f3aeeffd640 1 --2- 172.21.15.78:0/2695677228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046dcb30 0x5617046e6840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400a010 tx=0x7f3ae4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7a6f433a53c4eab server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:56.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.501+0000 7f3aceffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:48:56.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.502+0000 7f3aeffff640 1 --2- 172.21.15.78:0/3896905149 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3ad80a2cc0 0x7f3ad80a5180 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0006010 tx=0x7f3ae0040c10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:56.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.502+0000 7f3aceffd640 1 -- 172.21.15.78:0/2695677228 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4020070 con 0x5617046dcb30 2024-08-18T23:48:56.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.502+0000 7f3af7fe7740 1 -- 172.21.15.78:0/2695677228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56170464eef0 con 0x5617046dcb30 2024-08-18T23:48:56.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.502+0000 7f3aceffd640 1 -- 172.21.15.78:0/2695677228 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f3ae40129d0 con 0x5617046dcb30 2024-08-18T23:48:56.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.502+0000 7f3aceffd640 1 -- 172.21.15.78:0/2695677228 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3ae402c050 con 0x5617046dcb30 2024-08-18T23:48:56.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.502+0000 7f3af7fe7740 10 client.25700 fetch_fsmap learned FSMap version 28 2024-08-18T23:48:56.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.502+0000 7f3af7fe7740 1 -- 172.21.15.78:0/2695677228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5617043b33c0 con 0x5617046dcb30 2024-08-18T23:48:56.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.502+0000 7f3aceffd640 1 -- 172.21.15.78:0/2695677228 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56170464eef0 con 0x5617046dcb30 2024-08-18T23:48:56.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.502+0000 7f3aceffd640 1 -- 172.21.15.78:0/2695677228 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3ae4003c00 con 0x5617046dcb30 2024-08-18T23:48:56.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.503+0000 7f3af7fe7740 10 client.25700 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:48:56.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.503+0000 7f3af7fe7740 10 client.25700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:48:56.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.503+0000 7f3af7fe7740 20 client.25700 populate_metadata read hostname 'smithi078' 2024-08-18T23:48:56.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.503+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/2695677228 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5617046f74e0 0x5617046f9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:56.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.503+0000 7f3af7fe7740 4 client.25700 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:48:56.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.503+0000 7f3af7fe7740 1 -- 172.21.15.78:0/2695677228 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561704649400 con 0x5617046f74e0 2024-08-18T23:48:56.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.503+0000 7f3aef7fe640 1 --2- 172.21.15.78:0/2695677228 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5617046f74e0 0x5617046f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:56.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.503+0000 7f3aef7fe640 1 --2- 172.21.15.78:0/2695677228 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5617046f74e0 0x5617046f9920 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a02e7fea190000d server_cookie=8f583e511767d4fe in_seq=0 out_seq=0 2024-08-18T23:48:56.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.503+0000 7f3aceffd640 10 client.25700 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:56.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.506+0000 7f3aceffd640 1 -- 172.21.15.78:0/2695677228 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561704649400 con 0x5617046f74e0 2024-08-18T23:48:56.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.506+0000 7f3aceffd640 10 client.25700 handle_command_reply: tid=0 2024-08-18T23:48:56.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.506+0000 7f3af7fe7740 1 client.25700 shutdown 2024-08-18T23:48:56.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.507+0000 7f3af7fe7740 1 -- 172.21.15.78:0/2695677228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046dcb30 msgr2=0x5617046e6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:56.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.507+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/2695677228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046dcb30 0x5617046e6840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400a010 tx=0x7f3ae4004270 comp rx=0 tx=0).stop 2024-08-18T23:48:56.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.507+0000 7f3af7fe7740 1 -- 172.21.15.78:0/2695677228 shutdown_connections 2024-08-18T23:48:56.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.507+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/2695677228 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5617046f74e0 0x5617046f9920 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.507+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/2695677228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046e6d80 0x5617046e7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.507+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/2695677228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046dcb30 0x5617046e6840 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.507+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/2695677228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dc210 0x5617046e9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.507+0000 7f3af7fe7740 1 -- 172.21.15.78:0/2695677228 >> 172.21.15.78:0/2695677228 conn(0x5617046d87a0 msgr2=0x5617046de4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:56.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.507+0000 7f3aceffd640 0 client.25700 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:56.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.510+0000 7f3af7fe7740 1 -- 172.21.15.78:0/2695677228 shutdown_connections 2024-08-18T23:48:56.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.510+0000 7f3af7fe7740 1 -- 172.21.15.78:0/2695677228 wait complete. 2024-08-18T23:48:56.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.512+0000 7f3af7fe7740 20 client.25700 trim_cache size 0 max 16384 2024-08-18T23:48:56.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.575+0000 7f3af7fe7740 1 Processor -- start 2024-08-18T23:48:56.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.580+0000 7f3af7fe7740 1 -- start start 2024-08-18T23:48:56.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.580+0000 7f3af7fe7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046dc210 0x5617046f94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:56.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.580+0000 7f3af7fe7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dcb30 0x5617046e6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.580+0000 7f3af7fe7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046e6d80 0x5617046e6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.580+0000 7f3af7fe7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc012570 con 0x5617046e6d80 2024-08-18T23:48:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.580+0000 7f3af7fe7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc00fe40 con 0x5617046dc210 2024-08-18T23:48:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.580+0000 7f3af7fe7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc00dc90 con 0x5617046dcb30 2024-08-18T23:48:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.584+0000 7f3aef7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046dc210 0x5617046f94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.584+0000 7f3aef7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046dc210 0x5617046f94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:47108/0 (socket says 172.21.15.78:47108) 2024-08-18T23:48:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.584+0000 7f3aef7fe640 1 -- 172.21.15.78:0/230890557 learned_addr learned my addr 172.21.15.78:0/230890557 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.584+0000 7f3aeffff640 1 --2- 172.21.15.78:0/230890557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046e6d80 0x5617046e6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.584+0000 7f3aeeffd640 1 --2- 172.21.15.78:0/230890557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dcb30 0x5617046e6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:56.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.584+0000 7f3aeeffd640 1 -- 172.21.15.78:0/230890557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046dc210 msgr2=0x5617046f94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:56.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.584+0000 7f3aeeffd640 1 --2- 172.21.15.78:0/230890557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046dc210 0x5617046f94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.584+0000 7f3aeeffd640 1 -- 172.21.15.78:0/230890557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046e6d80 msgr2=0x5617046e6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:56.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.584+0000 7f3aeeffd640 1 --2- 172.21.15.78:0/230890557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046e6d80 0x5617046e6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.584+0000 7f3aeeffd640 1 -- 172.21.15.78:0/230890557 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617046e6ba0 con 0x5617046dcb30 2024-08-18T23:48:56.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.585+0000 7f3aeeffd640 1 --2- 172.21.15.78:0/230890557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dcb30 0x5617046e6120 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40099e0 tx=0x7f3ae40042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9711f9cfde44c62 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:56.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.589+0000 7f3ad57fa640 1 -- 172.21.15.78:0/230890557 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4029070 con 0x5617046dcb30 2024-08-18T23:48:56.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.589+0000 7f3ad57fa640 1 -- 172.21.15.78:0/230890557 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae4009350 con 0x5617046dcb30 2024-08-18T23:48:56.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.589+0000 7f3ad57fa640 1 -- 172.21.15.78:0/230890557 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4020050 con 0x5617046dcb30 2024-08-18T23:48:56.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.590+0000 7f3af7fe7740 1 -- 172.21.15.78:0/230890557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dcb30 msgr2=0x5617046e6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:56.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.590+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/230890557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dcb30 0x5617046e6120 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40099e0 tx=0x7f3ae40042e0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.590+0000 7f3af7fe7740 1 -- 172.21.15.78:0/230890557 shutdown_connections 2024-08-18T23:48:56.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.590+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/230890557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046e6d80 0x5617046e6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.590+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/230890557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dcb30 0x5617046e6120 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.590+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/230890557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046dc210 0x5617046f94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.590+0000 7f3af7fe7740 1 -- 172.21.15.78:0/230890557 >> 172.21.15.78:0/230890557 conn(0x5617046d87a0 msgr2=0x5617046e5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:56.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.593+0000 7f3af7fe7740 1 -- 172.21.15.78:0/230890557 shutdown_connections 2024-08-18T23:48:56.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.593+0000 7f3af7fe7740 1 -- 172.21.15.78:0/230890557 wait complete. 2024-08-18T23:48:56.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.593+0000 7f3af7fe7740 1 -- start start 2024-08-18T23:48:56.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.594+0000 7f3af7fe7740 1 Processor -- start 2024-08-18T23:48:56.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.594+0000 7f3af7fe7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dc210 0x5617046eabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:56.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.594+0000 7f3af7fe7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046dcb30 0x5617046eb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:56.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.595+0000 7f3aef7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dc210 0x5617046eabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:56.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.595+0000 7f3aef7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dc210 0x5617046eabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:44396/0 (socket says 172.21.15.78:44396) 2024-08-18T23:48:56.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.599+0000 7f3aeeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046dcb30 0x5617046eb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:56.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.599+0000 7f3aeeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046dcb30 0x5617046eb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45656/0 (socket says 172.21.15.78:45656) 2024-08-18T23:48:56.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.599+0000 7f3aef7fe640 1 -- 172.21.15.78:0/4108840144 learned_addr learned my addr 172.21.15.78:0/4108840144 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:48:56.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.599+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/4108840144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046e6d80 0x5617046ef240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:56.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.599+0000 7f3af7fe7740 1 -- 172.21.15.78:0/4108840144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc00e300 con 0x5617046dcb30 2024-08-18T23:48:56.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.599+0000 7f3aeffff640 1 --2- 172.21.15.78:0/4108840144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046e6d80 0x5617046ef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:56.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.599+0000 7f3af7fe7740 1 -- 172.21.15.78:0/4108840144 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc00f770 con 0x5617046e6d80 2024-08-18T23:48:56.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.600+0000 7f3af7fe7740 1 -- 172.21.15.78:0/4108840144 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc00e9d0 con 0x5617046dc210 2024-08-18T23:48:56.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.600+0000 7f3aef7fe640 1 -- 172.21.15.78:0/4108840144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046e6d80 msgr2=0x5617046ef240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:56.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.600+0000 7f3aef7fe640 1 --2- 172.21.15.78:0/4108840144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046e6d80 0x5617046ef240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.600+0000 7f3aef7fe640 1 -- 172.21.15.78:0/4108840144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046dcb30 msgr2=0x5617046eb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:48:56.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.601+0000 7f3aef7fe640 1 --2- 172.21.15.78:0/4108840144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046dcb30 0x5617046eb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.601+0000 7f3aef7fe640 1 -- 172.21.15.78:0/4108840144 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617046ef8b0 con 0x5617046dc210 2024-08-18T23:48:56.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.601+0000 7f3aeeffd640 1 --2- 172.21.15.78:0/4108840144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046dcb30 0x5617046eb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:48:56.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.601+0000 7f3aef7fe640 1 --2- 172.21.15.78:0/4108840144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dc210 0x5617046eabc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3ae801b000 tx=0x7f3ae80cc230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de5143c6592f9f69 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:48:56.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.601+0000 7f3af7fe7740 10 client.0 _ll_register_callbacks cb 0x7f3ae001a4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:48:56.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.601+0000 7f3af7fe7740 1 -- 172.21.15.78:0/4108840144 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ae4009350 con 0x5617046dc210 2024-08-18T23:48:56.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.602+0000 7f3aceffd640 10 client.25561 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:48:56.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.602+0000 7f3aceffd640 1 -- 172.21.15.78:0/4108840144 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8023040 con 0x5617046dc210 2024-08-18T23:48:56.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.602+0000 7f3aceffd640 1 -- 172.21.15.78:0/4108840144 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f3ae809f170 con 0x5617046dc210 2024-08-18T23:48:56.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.602+0000 7f3aceffd640 1 -- 172.21.15.78:0/4108840144 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3ae8099530 con 0x5617046dc210 2024-08-18T23:48:56.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.605+0000 7f3af7fe7740 10 client.25561 fetch_fsmap learned FSMap version 28 2024-08-18T23:48:56.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.606+0000 7f3af7fe7740 1 -- 172.21.15.78:0/4108840144 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({fsmap=28}) -- 0x561704761ab0 con 0x5617046dc210 2024-08-18T23:48:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.606+0000 7f3aceffd640 1 -- 172.21.15.78:0/4108840144 <== mon.1 v2:172.21.15.161:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3ae8095ce0 con 0x5617046dc210 2024-08-18T23:48:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.606+0000 7f3af7fe7740 10 client.25561 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:48:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.606+0000 7f3af7fe7740 10 client.25561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:48:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.606+0000 7f3af7fe7740 20 client.25561 populate_metadata read hostname 'smithi078' 2024-08-18T23:48:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.606+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/4108840144 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x561704762270 0x561704764730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:48:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.607+0000 7f3af7fe7740 4 client.25561 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:48:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.607+0000 7f3af7fe7740 1 -- 172.21.15.78:0/4108840144 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f3ae4009350 con 0x561704762270 2024-08-18T23:48:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.607+0000 7f3aeeffd640 1 --2- 172.21.15.78:0/4108840144 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x561704762270 0x561704764730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:48:56.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.609+0000 7f3aeeffd640 1 --2- 172.21.15.78:0/4108840144 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x561704762270 0x561704764730 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fbe3286a0d622a6 server_cookie=b664fc15e632e8a4 in_seq=0 out_seq=0 2024-08-18T23:48:56.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.609+0000 7f3aceffd640 10 client.25561 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:56.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.614+0000 7f3aceffd640 1 -- 172.21.15.78:0/4108840144 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x7f3ae4009350 con 0x561704762270 2024-08-18T23:48:56.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.614+0000 7f3aceffd640 10 client.25561 handle_command_reply: tid=0 2024-08-18T23:48:56.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.614+0000 7f3af7fe7740 1 client.25561 shutdown 2024-08-18T23:48:56.626 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:56 smithi078 ceph-mon[18440]: pgmap v489: 129 pgs: 129 active+clean; 547 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 729 op/s 2024-08-18T23:48:56.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.623+0000 7f3af7fe7740 1 -- 172.21.15.78:0/4108840144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dc210 msgr2=0x5617046eabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:56.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.623+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/4108840144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dc210 0x5617046eabc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3ae801b000 tx=0x7f3ae80cc230 comp rx=0 tx=0).stop 2024-08-18T23:48:56.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.631+0000 7f3af7fe7740 1 -- 172.21.15.78:0/4108840144 shutdown_connections 2024-08-18T23:48:56.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.631+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/4108840144 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x561704762270 0x561704764730 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.631+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/4108840144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617046e6d80 0x5617046ef240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.631+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/4108840144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5617046dcb30 0x5617046eb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.631+0000 7f3af7fe7740 1 --2- 172.21.15.78:0/4108840144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617046dc210 0x5617046eabc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.631+0000 7f3af7fe7740 1 -- 172.21.15.78:0/4108840144 >> 172.21.15.78:0/4108840144 conn(0x5617046d87a0 msgr2=0x5617046e4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:56.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.631+0000 7f3aceffd640 0 client.25561 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:48:56.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.636+0000 7f3af7fe7740 1 -- 172.21.15.78:0/4108840144 shutdown_connections 2024-08-18T23:48:56.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.636+0000 7f3af7fe7740 1 -- 172.21.15.78:0/4108840144 wait complete. 2024-08-18T23:48:56.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.638+0000 7f3af7fe7740 20 client.25561 trim_cache size 0 max 16384 2024-08-18T23:48:56.646 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:48:56.646 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "scrub active (12463 inodes in the stack)", 2024-08-18T23:48:56.646 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-08-18T23:48:56.646 INFO:teuthology.orchestra.run.smithi078.stdout: "dd2e1456-db13-4844-a083-fe682462993f": { 2024-08-18T23:48:56.646 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-08-18T23:48:56.646 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "dd2e1456-db13-4844-a083-fe682462993f", 2024-08-18T23:48:56.646 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-08-18T23:48:56.646 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:48:56.646 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:48:56.646 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:48:56.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.649+0000 7f3af5be9640 1 -- 172.21.15.78:0/3896905149 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3ad80a2cc0 msgr2=0x7f3ad80a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:56.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.649+0000 7f3af5be9640 1 --2- 172.21.15.78:0/3896905149 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3ad80a2cc0 0x7f3ad80a5180 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0006010 tx=0x7f3ae0040c10 comp rx=0 tx=0).stop 2024-08-18T23:48:56.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.649+0000 7f3af5be9640 1 -- 172.21.15.78:0/3896905149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3af00ffac0 msgr2=0x7f3af0071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:48:56.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.649+0000 7f3af5be9640 1 --2- 172.21.15.78:0/3896905149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3af00ffac0 0x7f3af0071610 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800ba80 tx=0x7f3ae800bf40 comp rx=0 tx=0).stop 2024-08-18T23:48:56.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.650+0000 7f3af5be9640 1 -- 172.21.15.78:0/3896905149 shutdown_connections 2024-08-18T23:48:56.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.650+0000 7f3af5be9640 1 --2- 172.21.15.78:0/3896905149 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3ad80a2cc0 0x7f3ad80a5180 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.650+0000 7f3af5be9640 1 --2- 172.21.15.78:0/3896905149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3af010b980 0x7f3af010be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.650+0000 7f3af5be9640 1 --2- 172.21.15.78:0/3896905149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3af0105ac0 0x7f3af0071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.650+0000 7f3af5be9640 1 --2- 172.21.15.78:0/3896905149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3af00ffac0 0x7f3af0071610 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:48:56.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.650+0000 7f3af5be9640 1 -- 172.21.15.78:0/3896905149 >> 172.21.15.78:0/3896905149 conn(0x7f3af00fb7c0 msgr2=0x7f3af01035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:48:56.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.651+0000 7f3af5be9640 1 -- 172.21.15.78:0/3896905149 shutdown_connections 2024-08-18T23:48:56.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:48:56.651+0000 7f3af5be9640 1 -- 172.21.15.78:0/3896905149 wait complete. 2024-08-18T23:48:56.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:56 smithi161 ceph-mon[22027]: pgmap v489: 129 pgs: 129 active+clean; 547 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 729 op/s 2024-08-18T23:48:56.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:56 smithi123 ceph-mon[21808]: pgmap v489: 129 pgs: 129 active+clean; 547 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 729 op/s 2024-08-18T23:48:56.983 INFO:tasks.workunit.client.0.smithi078.stdout: 106 1009387 423 715025 365 511111 1321 2024-08-18T23:48:57.699 INFO:tasks.cephfs.filesystem:scrub status for tag:dd2e1456-db13-4844-a083-fe682462993f - {'path': '/', 'tag': 'dd2e1456-db13-4844-a083-fe682462993f', 'options': 'recursive,force'} 2024-08-18T23:48:59.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:48:59 smithi123 ceph-mon[21808]: pgmap v490: 129 pgs: 129 active+clean; 547 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 625 op/s 2024-08-18T23:48:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:48:59 smithi161 ceph-mon[22027]: pgmap v490: 129 pgs: 129 active+clean; 547 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 625 op/s 2024-08-18T23:48:59.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:48:59 smithi078 ceph-mon[18440]: pgmap v490: 129 pgs: 129 active+clean; 547 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 625 op/s 2024-08-18T23:49:01.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:01 smithi123 ceph-mon[21808]: pgmap v491: 129 pgs: 129 active+clean; 552 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 447 op/s 2024-08-18T23:49:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:01 smithi161 ceph-mon[22027]: pgmap v491: 129 pgs: 129 active+clean; 552 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 447 op/s 2024-08-18T23:49:01.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:01 smithi078 ceph-mon[18440]: pgmap v491: 129 pgs: 129 active+clean; 552 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 447 op/s 2024-08-18T23:49:03.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:03 smithi123 ceph-mon[21808]: pgmap v492: 129 pgs: 129 active+clean; 558 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 421 op/s 2024-08-18T23:49:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:03 smithi161 ceph-mon[22027]: pgmap v492: 129 pgs: 129 active+clean; 558 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 421 op/s 2024-08-18T23:49:03.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:03 smithi078 ceph-mon[18440]: pgmap v492: 129 pgs: 129 active+clean; 558 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 421 op/s 2024-08-18T23:49:05.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:05 smithi161 ceph-mon[22027]: pgmap v493: 129 pgs: 129 active+clean; 573 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 442 op/s 2024-08-18T23:49:05.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:05 smithi078 ceph-mon[18440]: pgmap v493: 129 pgs: 129 active+clean; 573 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 442 op/s 2024-08-18T23:49:05.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:05 smithi123 ceph-mon[21808]: pgmap v493: 129 pgs: 129 active+clean; 573 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 442 op/s 2024-08-18T23:49:06.983 INFO:tasks.workunit.client.0.smithi078.stdout: 113 1357353 496 946710 431 685813 1089 2024-08-18T23:49:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:07 smithi161 ceph-mon[22027]: pgmap v494: 129 pgs: 129 active+clean; 585 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 551 op/s 2024-08-18T23:49:07.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:07 smithi078 ceph-mon[18440]: pgmap v494: 129 pgs: 129 active+clean; 585 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 551 op/s 2024-08-18T23:49:07.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:07 smithi123 ceph-mon[21808]: pgmap v494: 129 pgs: 129 active+clean; 585 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 551 op/s 2024-08-18T23:49:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:09 smithi161 ceph-mon[22027]: pgmap v495: 129 pgs: 129 active+clean; 604 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 713 op/s 2024-08-18T23:49:09.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:09 smithi078 ceph-mon[18440]: pgmap v495: 129 pgs: 129 active+clean; 604 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 713 op/s 2024-08-18T23:49:09.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:09 smithi123 ceph-mon[21808]: pgmap v495: 129 pgs: 129 active+clean; 604 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 713 op/s 2024-08-18T23:49:11.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:11 smithi161 ceph-mon[22027]: pgmap v496: 129 pgs: 129 active+clean; 614 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.5 MiB/s wr, 746 op/s 2024-08-18T23:49:11.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:11 smithi078 ceph-mon[18440]: pgmap v496: 129 pgs: 129 active+clean; 614 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.5 MiB/s wr, 746 op/s 2024-08-18T23:49:11.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:11 smithi123 ceph-mon[21808]: pgmap v496: 129 pgs: 129 active+clean; 614 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.5 MiB/s wr, 746 op/s 2024-08-18T23:49:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:13 smithi161 ceph-mon[22027]: pgmap v497: 129 pgs: 129 active+clean; 595 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.6 MiB/s wr, 776 op/s 2024-08-18T23:49:13.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:13 smithi078 ceph-mon[18440]: pgmap v497: 129 pgs: 129 active+clean; 595 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.6 MiB/s wr, 776 op/s 2024-08-18T23:49:13.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:13 smithi123 ceph-mon[21808]: pgmap v497: 129 pgs: 129 active+clean; 595 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.6 MiB/s wr, 776 op/s 2024-08-18T23:49:15.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:15 smithi161 ceph-mon[22027]: pgmap v498: 129 pgs: 129 active+clean; 595 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 766 op/s 2024-08-18T23:49:15.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:15 smithi078 ceph-mon[18440]: pgmap v498: 129 pgs: 129 active+clean; 595 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 766 op/s 2024-08-18T23:49:15.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:15 smithi123 ceph-mon[21808]: pgmap v498: 129 pgs: 129 active+clean; 595 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 766 op/s 2024-08-18T23:49:16.983 INFO:tasks.workunit.client.0.smithi078.stdout: 122 2015680 348 1426006 599 1047832 1624 2024-08-18T23:49:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:17 smithi161 ceph-mon[22027]: pgmap v499: 129 pgs: 129 active+clean; 596 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 758 op/s 2024-08-18T23:49:17.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:17 smithi078 ceph-mon[18440]: pgmap v499: 129 pgs: 129 active+clean; 596 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 758 op/s 2024-08-18T23:49:17.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:17 smithi123 ceph-mon[21808]: pgmap v499: 129 pgs: 129 active+clean; 596 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 758 op/s 2024-08-18T23:49:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:19 smithi161 ceph-mon[22027]: pgmap v500: 129 pgs: 129 active+clean; 602 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 712 op/s 2024-08-18T23:49:19.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:19 smithi078 ceph-mon[18440]: pgmap v500: 129 pgs: 129 active+clean; 602 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 712 op/s 2024-08-18T23:49:19.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:19 smithi123 ceph-mon[21808]: pgmap v500: 129 pgs: 129 active+clean; 602 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 712 op/s 2024-08-18T23:49:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:21 smithi161 ceph-mon[22027]: pgmap v501: 129 pgs: 129 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 593 op/s 2024-08-18T23:49:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:21 smithi078 ceph-mon[18440]: pgmap v501: 129 pgs: 129 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 593 op/s 2024-08-18T23:49:21.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:21 smithi123 ceph-mon[21808]: pgmap v501: 129 pgs: 129 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 593 op/s 2024-08-18T23:49:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:23 smithi161 ceph-mon[22027]: pgmap v502: 129 pgs: 129 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 555 op/s 2024-08-18T23:49:23.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:23 smithi078 ceph-mon[18440]: pgmap v502: 129 pgs: 129 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 555 op/s 2024-08-18T23:49:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:23 smithi123 ceph-mon[21808]: pgmap v502: 129 pgs: 129 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 555 op/s 2024-08-18T23:49:25.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:25 smithi161 ceph-mon[22027]: pgmap v503: 129 pgs: 129 active+clean; 614 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 609 op/s 2024-08-18T23:49:25.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:25 smithi078 ceph-mon[18440]: pgmap v503: 129 pgs: 129 active+clean; 614 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 609 op/s 2024-08-18T23:49:25.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:25 smithi123 ceph-mon[21808]: pgmap v503: 129 pgs: 129 active+clean; 614 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 609 op/s 2024-08-18T23:49:26.983 INFO:tasks.workunit.client.0.smithi078.stdout: 126 1017975 344 729343 291 545420 893 2024-08-18T23:49:27.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:27 smithi161 ceph-mon[22027]: pgmap v504: 129 pgs: 129 active+clean; 646 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 827 op/s 2024-08-18T23:49:27.701 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:49:27.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:27 smithi078 ceph-mon[18440]: pgmap v504: 129 pgs: 129 active+clean; 646 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 827 op/s 2024-08-18T23:49:27.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:27 smithi123 ceph-mon[21808]: pgmap v504: 129 pgs: 129 active+clean; 646 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 827 op/s 2024-08-18T23:49:27.984 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:49:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.539+0000 7f08ec87a640 1 -- 172.21.15.78:0/2202419337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08e8103470 msgr2=0x7f08e81078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:49:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.539+0000 7f08ec87a640 1 --2- 172.21.15.78:0/2202419337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08e8103470 0x7f08e81078b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009930 tx=0x7f08d002a420 comp rx=0 tx=0).stop 2024-08-18T23:49:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.540+0000 7f08ec87a640 1 -- 172.21.15.78:0/2202419337 shutdown_connections 2024-08-18T23:49:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.540+0000 7f08ec87a640 1 --2- 172.21.15.78:0/2202419337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08e8107fc0 0x7f08e810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.540+0000 7f08ec87a640 1 --2- 172.21.15.78:0/2202419337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08e8103470 0x7f08e81078b0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.540+0000 7f08ec87a640 1 --2- 172.21.15.78:0/2202419337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e8102ac0 0x7f08e8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.540+0000 7f08ec87a640 1 -- 172.21.15.78:0/2202419337 >> 172.21.15.78:0/2202419337 conn(0x7f08e80fc500 msgr2=0x7f08e80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:49:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.540+0000 7f08ec87a640 1 -- 172.21.15.78:0/2202419337 shutdown_connections 2024-08-18T23:49:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.540+0000 7f08ec87a640 1 -- 172.21.15.78:0/2202419337 wait complete. 2024-08-18T23:49:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.540+0000 7f08ec87a640 1 Processor -- start 2024-08-18T23:49:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.541+0000 7f08ec87a640 1 -- start start 2024-08-18T23:49:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.541+0000 7f08ec87a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08e8102ac0 0x7f08e819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:49:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.541+0000 7f08ec87a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08e8103470 0x7f08e819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:49:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.541+0000 7f08ec87a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e8107fc0 0x7f08e819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:49:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.541+0000 7f08ec87a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08e8111dc0 con 0x7f08e8103470 2024-08-18T23:49:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.541+0000 7f08ec87a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f08e8111c40 con 0x7f08e8102ac0 2024-08-18T23:49:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.541+0000 7f08ec87a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f08e8111f40 con 0x7f08e8107fc0 2024-08-18T23:49:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.541+0000 7f08e5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08e8103470 0x7f08e819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:49:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.541+0000 7f08e5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08e8103470 0x7f08e819aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46806/0 (socket says 172.21.15.78:46806) 2024-08-18T23:49:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.541+0000 7f08e5d9b640 1 -- 172.21.15.78:0/2873096921 learned_addr learned my addr 172.21.15.78:0/2873096921 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:49:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.541+0000 7f08e659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08e8102ac0 0x7f08e819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:49:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.541+0000 7f08e6d9d640 1 --2- 172.21.15.78:0/2873096921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e8107fc0 0x7f08e819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:49:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.541+0000 7f08e659c640 1 -- 172.21.15.78:0/2873096921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e8107fc0 msgr2=0x7f08e819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:49:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.541+0000 7f08e659c640 1 --2- 172.21.15.78:0/2873096921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e8107fc0 0x7f08e819ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.541+0000 7f08e659c640 1 -- 172.21.15.78:0/2873096921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08e8103470 msgr2=0x7f08e819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:49:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.541+0000 7f08e659c640 1 --2- 172.21.15.78:0/2873096921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08e8103470 0x7f08e819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.541+0000 7f08e659c640 1 -- 172.21.15.78:0/2873096921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e819f510 con 0x7f08e8102ac0 2024-08-18T23:49:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.541+0000 7f08e659c640 1 --2- 172.21.15.78:0/2873096921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08e8102ac0 0x7f08e819a560 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f08e000d720 tx=0x7f08e000dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:49:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.542+0000 7f08df7fe640 1 -- 172.21.15.78:0/2873096921 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0017070 con 0x7f08e8102ac0 2024-08-18T23:49:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.542+0000 7f08ec87a640 1 -- 172.21.15.78:0/2873096921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e819f800 con 0x7f08e8102ac0 2024-08-18T23:49:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.542+0000 7f08df7fe640 1 -- 172.21.15.78:0/2873096921 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08e000de30 con 0x7f08e8102ac0 2024-08-18T23:49:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.542+0000 7f08df7fe640 1 -- 172.21.15.78:0/2873096921 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e00047a0 con 0x7f08e8102ac0 2024-08-18T23:49:28.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.542+0000 7f08ec87a640 1 -- 172.21.15.78:0/2873096921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e81a70e0 con 0x7f08e8102ac0 2024-08-18T23:49:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.544+0000 7f08df7fe640 1 -- 172.21.15.78:0/2873096921 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f08e00099d0 con 0x7f08e8102ac0 2024-08-18T23:49:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.544+0000 7f08eec78740 1 Processor -- start 2024-08-18T23:49:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.544+0000 7f08eec78740 1 -- start start 2024-08-18T23:49:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08eec78740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f6b210 0x55da44f6b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:49:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08eec78740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f6bb30 0x55da44f75eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:49:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08eec78740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f763f0 0x55da44f787e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:49:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08eec78740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55da44ed0980 con 0x55da44f763f0 2024-08-18T23:49:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08eec78740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55da44e68a70 con 0x55da44f6bb30 2024-08-18T23:49:28.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08eec78740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55da44ecc940 con 0x55da44f6b210 2024-08-18T23:49:28.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08e659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f6bb30 0x55da44f75eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:49:28.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08e659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f6bb30 0x55da44f75eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:52896/0 (socket says 172.21.15.78:52896) 2024-08-18T23:49:28.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08e659c640 1 -- 172.21.15.78:0/1078185779 learned_addr learned my addr 172.21.15.78:0/1078185779 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:49:28.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08e5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f763f0 0x55da44f787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:49:28.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08e6d9d640 1 --2- 172.21.15.78:0/1078185779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f6b210 0x55da44f6b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:49:28.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08df7fe640 1 --2- 172.21.15.78:0/2873096921 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f08d40a2b80 0x7f08d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:49:28.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08e5d9b640 1 -- 172.21.15.78:0/1078185779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f6b210 msgr2=0x55da44f6b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:49:28.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08e5d9b640 1 --2- 172.21.15.78:0/1078185779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f6b210 0x55da44f6b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08e5d9b640 1 -- 172.21.15.78:0/1078185779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f6bb30 msgr2=0x55da44f75eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:49:28.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08e5d9b640 1 --2- 172.21.15.78:0/1078185779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f6bb30 0x55da44f75eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08e5d9b640 1 -- 172.21.15.78:0/1078185779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da44ecb6e0 con 0x55da44f763f0 2024-08-18T23:49:28.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08e659c640 1 --2- 172.21.15.78:0/1078185779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f6bb30 0x55da44f75eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:49:28.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08e6d9d640 1 --2- 172.21.15.78:0/2873096921 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f08d40a2b80 0x7f08d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:49:28.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08df7fe640 1 -- 172.21.15.78:0/2873096921 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f08e00ceff0 con 0x7f08e8102ac0 2024-08-18T23:49:28.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.545+0000 7f08e5d9b640 1 --2- 172.21.15.78:0/1078185779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f763f0 0x55da44f787e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009900 tx=0x7f08d0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c01296470ada9e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:49:28.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.546+0000 7f08dcff9640 1 -- 172.21.15.78:0/1078185779 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0040070 con 0x55da44f763f0 2024-08-18T23:49:28.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.546+0000 7f08dcff9640 1 -- 172.21.15.78:0/1078185779 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08d0031970 con 0x55da44f763f0 2024-08-18T23:49:28.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.546+0000 7f08dcff9640 1 -- 172.21.15.78:0/1078185779 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0035040 con 0x55da44f763f0 2024-08-18T23:49:28.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.546+0000 7f08eec78740 1 -- 172.21.15.78:0/1078185779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f763f0 msgr2=0x55da44f787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:49:28.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.546+0000 7f08e6d9d640 1 --2- 172.21.15.78:0/2873096921 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f08d40a2b80 0x7f08d40a5040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f08d8005fd0 tx=0x7f08d8014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:49:28.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.546+0000 7f08eec78740 1 --2- 172.21.15.78:0/1078185779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f763f0 0x55da44f787e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009900 tx=0x7f08d0005e80 comp rx=0 tx=0).stop 2024-08-18T23:49:28.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.546+0000 7f08eec78740 1 -- 172.21.15.78:0/1078185779 shutdown_connections 2024-08-18T23:49:28.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.546+0000 7f08eec78740 1 --2- 172.21.15.78:0/1078185779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f763f0 0x55da44f787e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.546+0000 7f08eec78740 1 --2- 172.21.15.78:0/1078185779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f6bb30 0x55da44f75eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.546+0000 7f08eec78740 1 --2- 172.21.15.78:0/1078185779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f6b210 0x55da44f6b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.546+0000 7f08eec78740 1 -- 172.21.15.78:0/1078185779 >> 172.21.15.78:0/1078185779 conn(0x55da44f677a0 msgr2=0x55da44f67b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:49:28.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.547+0000 7f08eec78740 1 -- 172.21.15.78:0/1078185779 shutdown_connections 2024-08-18T23:49:28.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.547+0000 7f08eec78740 1 -- 172.21.15.78:0/1078185779 wait complete. 2024-08-18T23:49:28.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.547+0000 7f08eec78740 1 -- start start 2024-08-18T23:49:28.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.547+0000 7f08eec78740 1 Processor -- start 2024-08-18T23:49:28.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.547+0000 7f08eec78740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f6b210 0x55da44f73260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:49:28.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.547+0000 7f08eec78740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6bb30 0x55da44f737a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:49:28.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.547+0000 7f08eec78740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f763f0 0x55da44f73ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:49:28.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.547+0000 7f08eec78740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55da44f78fc0 con 0x55da44f6bb30 2024-08-18T23:49:28.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.547+0000 7f08eec78740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55da44e8baa0 con 0x55da44f6b210 2024-08-18T23:49:28.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.547+0000 7f08eec78740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55da44f79140 con 0x55da44f763f0 2024-08-18T23:49:28.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.547+0000 7f08eec78740 10 client.0 _ll_register_callbacks cb 0x55da44e7ae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:49:28.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.547+0000 7f08e5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6bb30 0x55da44f737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:49:28.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.547+0000 7f08e659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f6b210 0x55da44f73260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:49:28.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.547+0000 7f08e5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6bb30 0x55da44f737a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46830/0 (socket says 172.21.15.78:46830) 2024-08-18T23:49:28.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.547+0000 7f08e5d9b640 1 -- 172.21.15.78:0/320772385 learned_addr learned my addr 172.21.15.78:0/320772385 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:49:28.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.547+0000 7f08e6d9d640 1 --2- 172.21.15.78:0/320772385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f763f0 0x55da44f73ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:49:28.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.548+0000 7f08e659c640 1 -- 172.21.15.78:0/320772385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f763f0 msgr2=0x55da44f73ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:49:28.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.548+0000 7f08e659c640 1 --2- 172.21.15.78:0/320772385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f763f0 0x55da44f73ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.548+0000 7f08e659c640 1 -- 172.21.15.78:0/320772385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6bb30 msgr2=0x55da44f737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:49:28.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.548+0000 7f08e659c640 1 --2- 172.21.15.78:0/320772385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6bb30 0x55da44f737a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.548+0000 7f08e659c640 1 -- 172.21.15.78:0/320772385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da44e6f650 con 0x55da44f6b210 2024-08-18T23:49:28.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.548+0000 7f08e6d9d640 1 --2- 172.21.15.78:0/320772385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f763f0 0x55da44f73ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:49:28.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.548+0000 7f08e5d9b640 1 --2- 172.21.15.78:0/320772385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6bb30 0x55da44f737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:49:28.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.548+0000 7f08e659c640 1 --2- 172.21.15.78:0/320772385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f6b210 0x55da44f73260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f08e0097e50 tx=0x7f08e0097060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db3e6fa0532ed746 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:49:28.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.548+0000 7f08cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:49:28.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.548+0000 7f08cdffb640 1 -- 172.21.15.78:0/320772385 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e009ade0 con 0x55da44f6b210 2024-08-18T23:49:28.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.548+0000 7f08cdffb640 1 -- 172.21.15.78:0/320772385 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f08e0009cc0 con 0x55da44f6b210 2024-08-18T23:49:28.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.548+0000 7f08eec78740 1 -- 172.21.15.78:0/320772385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08d0031970 con 0x55da44f6b210 2024-08-18T23:49:28.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.548+0000 7f08cdffb640 1 -- 172.21.15.78:0/320772385 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f08e009bb80 con 0x55da44f6b210 2024-08-18T23:49:28.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.548+0000 7f08eec78740 10 client.25718 fetch_fsmap learned FSMap version 28 2024-08-18T23:49:28.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.548+0000 7f08eec78740 1 -- 172.21.15.78:0/320772385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55da44c423c0 con 0x55da44f6b210 2024-08-18T23:49:28.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.548+0000 7f08cdffb640 1 -- 172.21.15.78:0/320772385 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08d0031970 con 0x55da44f6b210 2024-08-18T23:49:28.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.549+0000 7f08cdffb640 1 -- 172.21.15.78:0/320772385 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f08e00d10d0 con 0x55da44f6b210 2024-08-18T23:49:28.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.549+0000 7f08eec78740 10 client.25718 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:49:28.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.549+0000 7f08eec78740 10 client.25718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:49:28.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.549+0000 7f08eec78740 20 client.25718 populate_metadata read hostname 'smithi078' 2024-08-18T23:49:28.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.549+0000 7f08eec78740 1 --2- 172.21.15.78:0/320772385 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55da44f86280 0x55da44f88670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:49:28.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.549+0000 7f08eec78740 4 client.25718 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:49:28.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.549+0000 7f08eec78740 1 -- 172.21.15.78:0/320772385 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da44eddef0 con 0x55da44f86280 2024-08-18T23:49:28.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.549+0000 7f08e5d9b640 1 --2- 172.21.15.78:0/320772385 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55da44f86280 0x55da44f88670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:49:28.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.549+0000 7f08e5d9b640 1 --2- 172.21.15.78:0/320772385 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55da44f86280 0x55da44f88670 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a3cac19819c1e9e server_cookie=3562a1ca674e1e18 in_seq=0 out_seq=0 2024-08-18T23:49:28.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.549+0000 7f08cdffb640 10 client.25718 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:49:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.552+0000 7f08cdffb640 1 -- 172.21.15.78:0/320772385 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da44eddef0 con 0x55da44f86280 2024-08-18T23:49:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.552+0000 7f08cdffb640 10 client.25718 handle_command_reply: tid=0 2024-08-18T23:49:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.552+0000 7f08eec78740 1 client.25718 shutdown 2024-08-18T23:49:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.552+0000 7f08eec78740 1 -- 172.21.15.78:0/320772385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f6b210 msgr2=0x55da44f73260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:49:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.552+0000 7f08eec78740 1 --2- 172.21.15.78:0/320772385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f6b210 0x55da44f73260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f08e0097e50 tx=0x7f08e0097060 comp rx=0 tx=0).stop 2024-08-18T23:49:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.552+0000 7f08eec78740 1 -- 172.21.15.78:0/320772385 shutdown_connections 2024-08-18T23:49:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.552+0000 7f08eec78740 1 --2- 172.21.15.78:0/320772385 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55da44f86280 0x55da44f88670 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.552+0000 7f08eec78740 1 --2- 172.21.15.78:0/320772385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f763f0 0x55da44f73ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.553+0000 7f08eec78740 1 --2- 172.21.15.78:0/320772385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6bb30 0x55da44f737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.553+0000 7f08eec78740 1 --2- 172.21.15.78:0/320772385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f6b210 0x55da44f73260 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.553+0000 7f08cdffb640 0 client.25718 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:49:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.553+0000 7f08eec78740 1 -- 172.21.15.78:0/320772385 >> 172.21.15.78:0/320772385 conn(0x55da44f677a0 msgr2=0x55da44f6d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:49:28.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.553+0000 7f08eec78740 1 -- 172.21.15.78:0/320772385 shutdown_connections 2024-08-18T23:49:28.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.553+0000 7f08eec78740 1 -- 172.21.15.78:0/320772385 wait complete. 2024-08-18T23:49:28.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.553+0000 7f08eec78740 20 client.25718 trim_cache size 0 max 16384 2024-08-18T23:49:28.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.570+0000 7f08eec78740 1 Processor -- start 2024-08-18T23:49:28.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.570+0000 7f08eec78740 1 -- start start 2024-08-18T23:49:28.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.570+0000 7f08eec78740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f6b210 0x55da44f67b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:49:28.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.570+0000 7f08eec78740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6bb30 0x55da44f6d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:49:28.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.570+0000 7f08eec78740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f763f0 0x55da44f8dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:49:28.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.570+0000 7f08eec78740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08e81b3500 con 0x55da44f6bb30 2024-08-18T23:49:28.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.570+0000 7f08eec78740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f08e81b0dd0 con 0x55da44f763f0 2024-08-18T23:49:28.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.570+0000 7f08eec78740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f08e81aec20 con 0x55da44f6b210 2024-08-18T23:49:28.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.570+0000 7f08e5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6bb30 0x55da44f6d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:49:28.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.570+0000 7f08e5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6bb30 0x55da44f6d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46846/0 (socket says 172.21.15.78:46846) 2024-08-18T23:49:28.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.570+0000 7f08e5d9b640 1 -- 172.21.15.78:0/3834465873 learned_addr learned my addr 172.21.15.78:0/3834465873 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:49:28.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.570+0000 7f08e6d9d640 1 --2- 172.21.15.78:0/3834465873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f763f0 0x55da44f8dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:49:28.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.570+0000 7f08e659c640 1 --2- 172.21.15.78:0/3834465873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f6b210 0x55da44f67b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:49:28.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.570+0000 7f08e5d9b640 1 -- 172.21.15.78:0/3834465873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f6b210 msgr2=0x55da44f67b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:49:28.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.570+0000 7f08e5d9b640 1 --2- 172.21.15.78:0/3834465873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f6b210 0x55da44f67b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.570+0000 7f08e5d9b640 1 -- 172.21.15.78:0/3834465873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f763f0 msgr2=0x55da44f8dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:49:28.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.570+0000 7f08e5d9b640 1 --2- 172.21.15.78:0/3834465873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f763f0 0x55da44f8dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.570+0000 7f08e5d9b640 1 -- 172.21.15.78:0/3834465873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da44f84a10 con 0x55da44f6bb30 2024-08-18T23:49:28.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.570+0000 7f08e659c640 1 --2- 172.21.15.78:0/3834465873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f6b210 0x55da44f67b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:49:28.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.571+0000 7f08e5d9b640 1 --2- 172.21.15.78:0/3834465873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6bb30 0x55da44f6d370 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f08d0006010 tx=0x7f08d0005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3ed4c0050978d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:49:28.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.571+0000 7f08cffff640 1 -- 172.21.15.78:0/3834465873 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0040050 con 0x55da44f6bb30 2024-08-18T23:49:28.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.571+0000 7f08cffff640 1 -- 172.21.15.78:0/3834465873 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08d0031890 con 0x55da44f6bb30 2024-08-18T23:49:28.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.571+0000 7f08cffff640 1 -- 172.21.15.78:0/3834465873 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0035040 con 0x55da44f6bb30 2024-08-18T23:49:28.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.571+0000 7f08eec78740 1 -- 172.21.15.78:0/3834465873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6bb30 msgr2=0x55da44f6d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:49:28.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.571+0000 7f08eec78740 1 --2- 172.21.15.78:0/3834465873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6bb30 0x55da44f6d370 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f08d0006010 tx=0x7f08d0005540 comp rx=0 tx=0).stop 2024-08-18T23:49:28.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.571+0000 7f08eec78740 1 -- 172.21.15.78:0/3834465873 shutdown_connections 2024-08-18T23:49:28.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.571+0000 7f08eec78740 1 --2- 172.21.15.78:0/3834465873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f763f0 0x55da44f8dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.571+0000 7f08eec78740 1 --2- 172.21.15.78:0/3834465873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6bb30 0x55da44f6d370 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.571+0000 7f08eec78740 1 --2- 172.21.15.78:0/3834465873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f6b210 0x55da44f67b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.571+0000 7f08eec78740 1 -- 172.21.15.78:0/3834465873 >> 172.21.15.78:0/3834465873 conn(0x55da44f677a0 msgr2=0x55da44f6cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:49:28.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.571+0000 7f08eec78740 1 -- 172.21.15.78:0/3834465873 shutdown_connections 2024-08-18T23:49:28.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.571+0000 7f08eec78740 1 -- 172.21.15.78:0/3834465873 wait complete. 2024-08-18T23:49:28.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.572+0000 7f08eec78740 1 -- start start 2024-08-18T23:49:28.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.572+0000 7f08eec78740 1 Processor -- start 2024-08-18T23:49:28.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.572+0000 7f08eec78740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6b210 0x55da44fe8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:49:28.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.572+0000 7f08eec78740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f6bb30 0x55da44fe9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:49:28.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.572+0000 7f08eec78740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f763f0 0x55da44f799d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:49:28.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.572+0000 7f08eec78740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08e81af290 con 0x55da44f6b210 2024-08-18T23:49:28.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.572+0000 7f08eec78740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f08e81b0700 con 0x55da44f763f0 2024-08-18T23:49:28.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.572+0000 7f08eec78740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f08e81af960 con 0x55da44f6bb30 2024-08-18T23:49:28.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.572+0000 7f08eec78740 10 client.0 _ll_register_callbacks cb 0x7f08e00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:49:28.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.572+0000 7f08e659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6b210 0x55da44fe8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:49:28.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.572+0000 7f08e659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6b210 0x55da44fe8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46852/0 (socket says 172.21.15.78:46852) 2024-08-18T23:49:28.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.572+0000 7f08e659c640 1 -- 172.21.15.78:0/219828924 learned_addr learned my addr 172.21.15.78:0/219828924 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:49:28.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.572+0000 7f08e6d9d640 1 --2- 172.21.15.78:0/219828924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f763f0 0x55da44f799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:49:28.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.572+0000 7f08e659c640 1 -- 172.21.15.78:0/219828924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f6bb30 msgr2=0x55da44fe9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:49:28.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.572+0000 7f08e659c640 1 --2- 172.21.15.78:0/219828924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f6bb30 0x55da44fe9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.572+0000 7f08e659c640 1 -- 172.21.15.78:0/219828924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f763f0 msgr2=0x55da44f799d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:49:28.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.572+0000 7f08e659c640 1 --2- 172.21.15.78:0/219828924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f763f0 0x55da44f799d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.572+0000 7f08e659c640 1 -- 172.21.15.78:0/219828924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da44f7a0c0 con 0x55da44f6b210 2024-08-18T23:49:28.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.573+0000 7f08e659c640 1 --2- 172.21.15.78:0/219828924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6b210 0x55da44fe8f40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f08e00ccd20 tx=0x7f08e009ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbc8e62015dac6e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:49:28.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.573+0000 7f08cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:49:28.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.573+0000 7f08cdffb640 1 -- 172.21.15.78:0/219828924 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0004220 con 0x55da44f6b210 2024-08-18T23:49:28.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.573+0000 7f08eec78740 1 -- 172.21.15.78:0/219828924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08d0031890 con 0x55da44f6b210 2024-08-18T23:49:28.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.573+0000 7f08cdffb640 1 -- 172.21.15.78:0/219828924 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f08e0009cc0 con 0x55da44f6b210 2024-08-18T23:49:28.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.573+0000 7f08cdffb640 1 -- 172.21.15.78:0/219828924 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f08e009b480 con 0x55da44f6b210 2024-08-18T23:49:28.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.573+0000 7f08eec78740 10 client.16299 fetch_fsmap learned FSMap version 28 2024-08-18T23:49:28.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.573+0000 7f08eec78740 1 -- 172.21.15.78:0/219828924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55da44ff0720 con 0x55da44f6b210 2024-08-18T23:49:28.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.573+0000 7f08cdffb640 1 -- 172.21.15.78:0/219828924 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08d0031890 con 0x55da44f6b210 2024-08-18T23:49:28.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.573+0000 7f08cdffb640 1 -- 172.21.15.78:0/219828924 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f08e007b070 con 0x55da44f6b210 2024-08-18T23:49:28.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.573+0000 7f08eec78740 10 client.16299 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:49:28.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.573+0000 7f08eec78740 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:49:28.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.573+0000 7f08eec78740 20 client.16299 populate_metadata read hostname 'smithi078' 2024-08-18T23:49:28.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.573+0000 7f08eec78740 1 --2- 172.21.15.78:0/219828924 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55da44ff0e10 0x55da44ff32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:49:28.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.573+0000 7f08eec78740 4 client.16299 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:49:28.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.573+0000 7f08eec78740 1 -- 172.21.15.78:0/219828924 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55da44ed8400 con 0x55da44ff0e10 2024-08-18T23:49:28.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.574+0000 7f08e5d9b640 1 --2- 172.21.15.78:0/219828924 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55da44ff0e10 0x55da44ff32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:49:28.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.574+0000 7f08e5d9b640 1 --2- 172.21.15.78:0/219828924 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55da44ff0e10 0x55da44ff32b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=448bd922608a76bc server_cookie=a61ef3eddbc7c2bd in_seq=0 out_seq=0 2024-08-18T23:49:28.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.574+0000 7f08cdffb640 10 client.16299 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:49:28.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.578+0000 7f08cdffb640 1 -- 172.21.15.78:0/219828924 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55da44ed8400 con 0x55da44ff0e10 2024-08-18T23:49:28.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.578+0000 7f08cdffb640 10 client.16299 handle_command_reply: tid=0 2024-08-18T23:49:28.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.578+0000 7f08eec78740 1 client.16299 shutdown 2024-08-18T23:49:28.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.578+0000 7f08eec78740 1 -- 172.21.15.78:0/219828924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6b210 msgr2=0x55da44fe8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:49:28.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.578+0000 7f08eec78740 1 --2- 172.21.15.78:0/219828924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6b210 0x55da44fe8f40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f08e00ccd20 tx=0x7f08e009ae60 comp rx=0 tx=0).stop 2024-08-18T23:49:28.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.578+0000 7f08eec78740 1 -- 172.21.15.78:0/219828924 shutdown_connections 2024-08-18T23:49:28.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.578+0000 7f08eec78740 1 --2- 172.21.15.78:0/219828924 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55da44ff0e10 0x55da44ff32b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.578+0000 7f08eec78740 1 --2- 172.21.15.78:0/219828924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da44f763f0 0x55da44f799d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.578+0000 7f08eec78740 1 --2- 172.21.15.78:0/219828924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da44f6bb30 0x55da44fe9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.578+0000 7f08eec78740 1 --2- 172.21.15.78:0/219828924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da44f6b210 0x55da44fe8f40 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.578+0000 7f08eec78740 1 -- 172.21.15.78:0/219828924 >> 172.21.15.78:0/219828924 conn(0x55da44f677a0 msgr2=0x55da44f66900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:49:28.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.578+0000 7f08cdffb640 0 client.16299 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:49:28.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.578+0000 7f08eec78740 1 -- 172.21.15.78:0/219828924 shutdown_connections 2024-08-18T23:49:28.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.578+0000 7f08eec78740 1 -- 172.21.15.78:0/219828924 wait complete. 2024-08-18T23:49:28.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.578+0000 7f08eec78740 20 client.16299 trim_cache size 0 max 16384 2024-08-18T23:49:28.585 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:49:28.585 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "scrub active (3511 inodes in the stack)", 2024-08-18T23:49:28.585 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-08-18T23:49:28.585 INFO:teuthology.orchestra.run.smithi078.stdout: "dd2e1456-db13-4844-a083-fe682462993f": { 2024-08-18T23:49:28.586 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-08-18T23:49:28.586 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "dd2e1456-db13-4844-a083-fe682462993f", 2024-08-18T23:49:28.586 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-08-18T23:49:28.586 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:49:28.586 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:49:28.586 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:49:28.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.579+0000 7f08ec87a640 1 -- 172.21.15.78:0/2873096921 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f08d40a2b80 msgr2=0x7f08d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:49:28.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.579+0000 7f08ec87a640 1 --2- 172.21.15.78:0/2873096921 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f08d40a2b80 0x7f08d40a5040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f08d8005fd0 tx=0x7f08d8014040 comp rx=0 tx=0).stop 2024-08-18T23:49:28.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.579+0000 7f08ec87a640 1 -- 172.21.15.78:0/2873096921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08e8102ac0 msgr2=0x7f08e819a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:49:28.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.579+0000 7f08ec87a640 1 --2- 172.21.15.78:0/2873096921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08e8102ac0 0x7f08e819a560 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f08e000d720 tx=0x7f08e000dbe0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.579+0000 7f08ec87a640 1 -- 172.21.15.78:0/2873096921 shutdown_connections 2024-08-18T23:49:28.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.579+0000 7f08ec87a640 1 --2- 172.21.15.78:0/2873096921 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f08d40a2b80 0x7f08d40a5040 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.579+0000 7f08ec87a640 1 --2- 172.21.15.78:0/2873096921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e8107fc0 0x7f08e819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.579+0000 7f08ec87a640 1 --2- 172.21.15.78:0/2873096921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08e8103470 0x7f08e819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.579+0000 7f08ec87a640 1 --2- 172.21.15.78:0/2873096921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08e8102ac0 0x7f08e819a560 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:49:28.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.579+0000 7f08ec87a640 1 -- 172.21.15.78:0/2873096921 >> 172.21.15.78:0/2873096921 conn(0x7f08e80fc500 msgr2=0x7f08e8100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:49:28.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.579+0000 7f08ec87a640 1 -- 172.21.15.78:0/2873096921 shutdown_connections 2024-08-18T23:49:28.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:49:28.579+0000 7f08ec87a640 1 -- 172.21.15.78:0/2873096921 wait complete. 2024-08-18T23:49:29.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:29 smithi078 ceph-mon[18440]: pgmap v505: 129 pgs: 129 active+clean; 679 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.0 MiB/s wr, 1.12k op/s 2024-08-18T23:49:29.521 INFO:tasks.cephfs.filesystem:scrub status for tag:dd2e1456-db13-4844-a083-fe682462993f - {'path': '/', 'tag': 'dd2e1456-db13-4844-a083-fe682462993f', 'options': 'recursive,force'} 2024-08-18T23:49:29.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:29 smithi161 ceph-mon[22027]: pgmap v505: 129 pgs: 129 active+clean; 679 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.0 MiB/s wr, 1.12k op/s 2024-08-18T23:49:29.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:29 smithi123 ceph-mon[21808]: pgmap v505: 129 pgs: 129 active+clean; 679 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.0 MiB/s wr, 1.12k op/s 2024-08-18T23:49:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:31 smithi161 ceph-mon[22027]: pgmap v506: 129 pgs: 129 active+clean; 677 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.7 MiB/s wr, 1.13k op/s 2024-08-18T23:49:31.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:31 smithi078 ceph-mon[18440]: pgmap v506: 129 pgs: 129 active+clean; 677 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.7 MiB/s wr, 1.13k op/s 2024-08-18T23:49:31.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:31 smithi123 ceph-mon[21808]: pgmap v506: 129 pgs: 129 active+clean; 677 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.7 MiB/s wr, 1.13k op/s 2024-08-18T23:49:32.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:32 smithi161 ceph-mon[22027]: pgmap v507: 129 pgs: 129 active+clean; 663 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.5 MiB/s wr, 1.09k op/s 2024-08-18T23:49:32.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:32 smithi078 ceph-mon[18440]: pgmap v507: 129 pgs: 129 active+clean; 663 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.5 MiB/s wr, 1.09k op/s 2024-08-18T23:49:32.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:32 smithi123 ceph-mon[21808]: pgmap v507: 129 pgs: 129 active+clean; 663 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.5 MiB/s wr, 1.09k op/s 2024-08-18T23:49:35.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:35 smithi161 ceph-mon[22027]: pgmap v508: 129 pgs: 129 active+clean; 650 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.4 MiB/s wr, 1.05k op/s 2024-08-18T23:49:35.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:35 smithi078 ceph-mon[18440]: pgmap v508: 129 pgs: 129 active+clean; 650 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.4 MiB/s wr, 1.05k op/s 2024-08-18T23:49:35.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:35 smithi123 ceph-mon[21808]: pgmap v508: 129 pgs: 129 active+clean; 650 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.4 MiB/s wr, 1.05k op/s 2024-08-18T23:49:36.983 INFO:tasks.workunit.client.0.smithi078.stdout: 130 679604 468 486210 372 368940 1573 2024-08-18T23:49:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:37 smithi161 ceph-mon[22027]: pgmap v509: 129 pgs: 129 active+clean; 652 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.0 MiB/s wr, 953 op/s 2024-08-18T23:49:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:37 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:49:37.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:37 smithi078 ceph-mon[18440]: pgmap v509: 129 pgs: 129 active+clean; 652 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.0 MiB/s wr, 953 op/s 2024-08-18T23:49:37.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:37 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:49:37.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:37 smithi123 ceph-mon[21808]: pgmap v509: 129 pgs: 129 active+clean; 652 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.0 MiB/s wr, 953 op/s 2024-08-18T23:49:37.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:37 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:49:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:39 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:49:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:39 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:49:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:39 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:49:39.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:39 smithi161 ceph-mon[22027]: pgmap v510: 129 pgs: 129 active+clean; 655 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.8 MiB/s wr, 779 op/s 2024-08-18T23:49:39.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:39 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:49:39.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:39 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:49:39.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:39 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:49:39.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:39 smithi078 ceph-mon[18440]: pgmap v510: 129 pgs: 129 active+clean; 655 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.8 MiB/s wr, 779 op/s 2024-08-18T23:49:39.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:39 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:49:39.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:39 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:49:39.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:39 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:49:39.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:39 smithi123 ceph-mon[21808]: pgmap v510: 129 pgs: 129 active+clean; 655 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.8 MiB/s wr, 779 op/s 2024-08-18T23:49:41.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:41 smithi161 ceph-mon[22027]: pgmap v511: 129 pgs: 129 active+clean; 653 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 476 op/s 2024-08-18T23:49:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:41 smithi078 ceph-mon[18440]: pgmap v511: 129 pgs: 129 active+clean; 653 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 476 op/s 2024-08-18T23:49:41.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:41 smithi123 ceph-mon[21808]: pgmap v511: 129 pgs: 129 active+clean; 653 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 476 op/s 2024-08-18T23:49:43.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:43 smithi161 ceph-mon[22027]: pgmap v512: 129 pgs: 129 active+clean; 655 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 388 op/s 2024-08-18T23:49:43.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:43 smithi078 ceph-mon[18440]: pgmap v512: 129 pgs: 129 active+clean; 655 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 388 op/s 2024-08-18T23:49:43.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:43 smithi123 ceph-mon[21808]: pgmap v512: 129 pgs: 129 active+clean; 655 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 388 op/s 2024-08-18T23:49:44.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:44 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:49:44.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:44 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:49:44.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:44 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:49:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:45 smithi161 ceph-mon[22027]: pgmap v513: 129 pgs: 129 active+clean; 662 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 391 op/s 2024-08-18T23:49:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:45 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:49:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:45 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:49:45.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:45 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:49:45.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:45 smithi078 ceph-mon[18440]: pgmap v513: 129 pgs: 129 active+clean; 662 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 391 op/s 2024-08-18T23:49:45.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:45 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:49:45.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:45 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:49:45.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:45 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:49:45.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:45 smithi123 ceph-mon[21808]: pgmap v513: 129 pgs: 129 active+clean; 662 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 391 op/s 2024-08-18T23:49:45.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:45 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:49:45.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:45 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:49:45.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:45 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:49:46.983 INFO:tasks.workunit.client.0.smithi078.stdout: 138 1521073 335 1084846 248 852238 1100 2024-08-18T23:49:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:47 smithi161 ceph-mon[22027]: pgmap v514: 129 pgs: 129 active+clean; 677 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 531 op/s 2024-08-18T23:49:47.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:47 smithi078 ceph-mon[18440]: pgmap v514: 129 pgs: 129 active+clean; 677 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 531 op/s 2024-08-18T23:49:47.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:47 smithi123 ceph-mon[21808]: pgmap v514: 129 pgs: 129 active+clean; 677 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 531 op/s 2024-08-18T23:49:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:49 smithi161 ceph-mon[22027]: pgmap v515: 129 pgs: 129 active+clean; 694 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 734 op/s 2024-08-18T23:49:49.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:49 smithi078 ceph-mon[18440]: pgmap v515: 129 pgs: 129 active+clean; 694 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 734 op/s 2024-08-18T23:49:49.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:49 smithi123 ceph-mon[21808]: pgmap v515: 129 pgs: 129 active+clean; 694 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 734 op/s 2024-08-18T23:49:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:51 smithi161 ceph-mon[22027]: pgmap v516: 129 pgs: 129 active+clean; 691 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 823 op/s 2024-08-18T23:49:51.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:51 smithi078 ceph-mon[18440]: pgmap v516: 129 pgs: 129 active+clean; 691 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 823 op/s 2024-08-18T23:49:51.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:51 smithi123 ceph-mon[21808]: pgmap v516: 129 pgs: 129 active+clean; 691 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 823 op/s 2024-08-18T23:49:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:53 smithi161 ceph-mon[22027]: pgmap v517: 129 pgs: 129 active+clean; 697 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.6 MiB/s wr, 809 op/s 2024-08-18T23:49:53.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:53 smithi078 ceph-mon[18440]: pgmap v517: 129 pgs: 129 active+clean; 697 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.6 MiB/s wr, 809 op/s 2024-08-18T23:49:53.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:53 smithi123 ceph-mon[21808]: pgmap v517: 129 pgs: 129 active+clean; 697 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.6 MiB/s wr, 809 op/s 2024-08-18T23:49:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:55 smithi161 ceph-mon[22027]: pgmap v518: 129 pgs: 129 active+clean; 699 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.7 MiB/s wr, 793 op/s 2024-08-18T23:49:55.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:55 smithi078 ceph-mon[18440]: pgmap v518: 129 pgs: 129 active+clean; 699 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.7 MiB/s wr, 793 op/s 2024-08-18T23:49:55.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:55 smithi123 ceph-mon[21808]: pgmap v518: 129 pgs: 129 active+clean; 699 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.7 MiB/s wr, 793 op/s 2024-08-18T23:49:56.983 INFO:tasks.workunit.client.0.smithi078.stdout: 144 1848302 682 1312460 355 1046953 1628 2024-08-18T23:49:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:57 smithi161 ceph-mon[22027]: pgmap v519: 129 pgs: 129 active+clean; 698 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 749 op/s 2024-08-18T23:49:57.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:57 smithi078 ceph-mon[18440]: pgmap v519: 129 pgs: 129 active+clean; 698 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 749 op/s 2024-08-18T23:49:57.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:57 smithi123 ceph-mon[21808]: pgmap v519: 129 pgs: 129 active+clean; 698 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.0 MiB/s wr, 749 op/s 2024-08-18T23:49:59.522 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:49:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:49:59 smithi161 ceph-mon[22027]: pgmap v520: 129 pgs: 129 active+clean; 706 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 611 op/s 2024-08-18T23:49:59.726 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:49:59 smithi078 ceph-mon[18440]: pgmap v520: 129 pgs: 129 active+clean; 706 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 611 op/s 2024-08-18T23:49:59.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:49:59 smithi123 ceph-mon[21808]: pgmap v520: 129 pgs: 129 active+clean; 706 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 611 op/s 2024-08-18T23:50:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:00 smithi161 ceph-mon[22027]: overall HEALTH_OK 2024-08-18T23:50:00.726 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:00 smithi078 ceph-mon[18440]: overall HEALTH_OK 2024-08-18T23:50:00.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:00 smithi123 ceph-mon[21808]: overall HEALTH_OK 2024-08-18T23:50:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:01 smithi161 ceph-mon[22027]: pgmap v521: 129 pgs: 129 active+clean; 688 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 413 op/s 2024-08-18T23:50:01.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:01 smithi078 ceph-mon[18440]: pgmap v521: 129 pgs: 129 active+clean; 688 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 413 op/s 2024-08-18T23:50:01.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:01 smithi123 ceph-mon[21808]: pgmap v521: 129 pgs: 129 active+clean; 688 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 413 op/s 2024-08-18T23:50:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:03 smithi161 ceph-mon[22027]: pgmap v522: 129 pgs: 129 active+clean; 683 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2024-08-18T23:50:03.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:03 smithi078 ceph-mon[18440]: pgmap v522: 129 pgs: 129 active+clean; 683 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2024-08-18T23:50:03.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:03 smithi123 ceph-mon[21808]: pgmap v522: 129 pgs: 129 active+clean; 683 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2024-08-18T23:50:05.083 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:50:05.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:05 smithi161 ceph-mon[22027]: pgmap v523: 129 pgs: 129 active+clean; 678 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 260 op/s 2024-08-18T23:50:05.724 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:05 smithi078 ceph-mon[18440]: pgmap v523: 129 pgs: 129 active+clean; 678 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 260 op/s 2024-08-18T23:50:05.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:05 smithi123 ceph-mon[21808]: pgmap v523: 129 pgs: 129 active+clean; 678 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 260 op/s 2024-08-18T23:50:06.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.132+0000 7f6e350b7640 1 -- 172.21.15.78:0/1984998511 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2000a4b0 con 0x7f6e3010ec60 2024-08-18T23:50:06.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.133+0000 7f6e3831d640 1 -- 172.21.15.78:0/1984998511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3010ec60 msgr2=0x7f6e3010cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:06.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.133+0000 7f6e3831d640 1 --2- 172.21.15.78:0/1984998511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3010ec60 0x7f6e3010cfd0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6e2000af30 tx=0x7f6e2002e660 comp rx=0 tx=0).stop 2024-08-18T23:50:06.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.135+0000 7f6e3831d640 1 -- 172.21.15.78:0/1984998511 shutdown_connections 2024-08-18T23:50:06.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.135+0000 7f6e3831d640 1 --2- 172.21.15.78:0/1984998511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e3010d6e0 0x7f6e30115240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.135+0000 7f6e3831d640 1 --2- 172.21.15.78:0/1984998511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3010ec60 0x7f6e3010cfd0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.136+0000 7f6e3831d640 1 --2- 172.21.15.78:0/1984998511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e3010e2b0 0x7f6e3010e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.136+0000 7f6e3831d640 1 -- 172.21.15.78:0/1984998511 >> 172.21.15.78:0/1984998511 conn(0x7f6e3006de70 msgr2=0x7f6e3006e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:06.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.136+0000 7f6e3831d640 1 -- 172.21.15.78:0/1984998511 shutdown_connections 2024-08-18T23:50:06.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.136+0000 7f6e3831d640 1 -- 172.21.15.78:0/1984998511 wait complete. 2024-08-18T23:50:06.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.137+0000 7f6e3831d640 1 Processor -- start 2024-08-18T23:50:06.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.138+0000 7f6e3831d640 1 -- start start 2024-08-18T23:50:06.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.138+0000 7f6e3831d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3010d6e0 0x7f6e301a7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:06.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.138+0000 7f6e3831d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e3010e2b0 0x7f6e301a7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:06.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.138+0000 7f6e3831d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e3010ec60 0x7f6e301ab920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:06.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.138+0000 7f6e3831d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e301182f0 con 0x7f6e3010d6e0 2024-08-18T23:50:06.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.138+0000 7f6e3831d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6e30118170 con 0x7f6e3010ec60 2024-08-18T23:50:06.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.138+0000 7f6e3831d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6e30118470 con 0x7f6e3010e2b0 2024-08-18T23:50:06.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.140+0000 7f6e358b8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e3010e2b0 0x7f6e301a7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:06.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.140+0000 7f6e358b8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e3010e2b0 0x7f6e301a7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:54194/0 (socket says 172.21.15.78:54194) 2024-08-18T23:50:06.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.140+0000 7f6e358b8640 1 -- 172.21.15.78:0/4042298019 learned_addr learned my addr 172.21.15.78:0/4042298019 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:06.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.143+0000 7f6e358b8640 1 -- 172.21.15.78:0/4042298019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e3010ec60 msgr2=0x7f6e301ab920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-18T23:50:06.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.143+0000 7f6e358b8640 1 --2- 172.21.15.78:0/4042298019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e3010ec60 0x7f6e301ab920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.143+0000 7f6e358b8640 1 -- 172.21.15.78:0/4042298019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3010d6e0 msgr2=0x7f6e301a7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:06.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.143+0000 7f6e358b8640 1 --2- 172.21.15.78:0/4042298019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3010d6e0 0x7f6e301a7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.143+0000 7f6e358b8640 1 -- 172.21.15.78:0/4042298019 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e301ac020 con 0x7f6e3010e2b0 2024-08-18T23:50:06.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.147+0000 7f6e358b8640 1 --2- 172.21.15.78:0/4042298019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e3010e2b0 0x7f6e301a7590 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6e20037040 tx=0x7f6e200045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:06.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.149+0000 7f6e277fe640 1 -- 172.21.15.78:0/4042298019 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e200035f0 con 0x7f6e3010e2b0 2024-08-18T23:50:06.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.150+0000 7f6e3831d640 1 -- 172.21.15.78:0/4042298019 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e301ac2b0 con 0x7f6e3010e2b0 2024-08-18T23:50:06.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.150+0000 7f6e3831d640 1 -- 172.21.15.78:0/4042298019 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e3019e5a0 con 0x7f6e3010e2b0 2024-08-18T23:50:06.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.150+0000 7f6e277fe640 1 -- 172.21.15.78:0/4042298019 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e20042070 con 0x7f6e3010e2b0 2024-08-18T23:50:06.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.150+0000 7f6e277fe640 1 -- 172.21.15.78:0/4042298019 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2003db90 con 0x7f6e3010e2b0 2024-08-18T23:50:06.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.154+0000 7f6e3a71b740 1 Processor -- start 2024-08-18T23:50:06.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.158+0000 7f6e277fe640 1 -- 172.21.15.78:0/4042298019 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6e2000f030 con 0x7f6e3010e2b0 2024-08-18T23:50:06.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.159+0000 7f6e277fe640 1 --2- 172.21.15.78:0/4042298019 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6e1c0a2d50 0x7f6e1c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:06.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.159+0000 7f6e277fe640 1 -- 172.21.15.78:0/4042298019 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f6e200ef920 con 0x7f6e3010e2b0 2024-08-18T23:50:06.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.162+0000 7f6e3a71b740 1 -- start start 2024-08-18T23:50:06.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.162+0000 7f6e3a71b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef3212c210 0x55ef3212c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:06.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.162+0000 7f6e3a71b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3212cb30 0x55ef32136eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:06.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.162+0000 7f6e3a71b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef321373f0 0x55ef321397e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:06.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.162+0000 7f6e3a71b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ef32091980 con 0x55ef321373f0 2024-08-18T23:50:06.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.162+0000 7f6e3a71b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef32029a70 con 0x55ef3212cb30 2024-08-18T23:50:06.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.162+0000 7f6e3a71b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ef3208d940 con 0x55ef3212c210 2024-08-18T23:50:06.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.163+0000 7f6e368ba640 1 --2- 172.21.15.78:0/4042298019 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6e1c0a2d50 0x7f6e1c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:06.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.165+0000 7f6e368ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef321373f0 0x55ef321397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.165+0000 7f6e368ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef321373f0 0x55ef321397e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40606/0 (socket says 172.21.15.78:40606) 2024-08-18T23:50:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.165+0000 7f6e368ba640 1 -- 172.21.15.78:0/2200200641 learned_addr learned my addr 172.21.15.78:0/2200200641 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.165+0000 7f6e360b9640 1 --2- 172.21.15.78:0/2200200641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef3212c210 0x55ef3212c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:06.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.165+0000 7f6e358b8640 1 --2- 172.21.15.78:0/2200200641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3212cb30 0x55ef32136eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:06.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.166+0000 7f6e368ba640 1 -- 172.21.15.78:0/2200200641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef3212c210 msgr2=0x55ef3212c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:06.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.166+0000 7f6e368ba640 1 --2- 172.21.15.78:0/2200200641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef3212c210 0x55ef3212c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.166+0000 7f6e368ba640 1 -- 172.21.15.78:0/2200200641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3212cb30 msgr2=0x55ef32136eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:06.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.166+0000 7f6e368ba640 1 --2- 172.21.15.78:0/2200200641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3212cb30 0x55ef32136eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.166+0000 7f6e368ba640 1 -- 172.21.15.78:0/2200200641 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef3208c6e0 con 0x55ef321373f0 2024-08-18T23:50:06.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.166+0000 7f6e368ba640 1 --2- 172.21.15.78:0/2200200641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef321373f0 0x55ef321397e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800ec10 tx=0x7f6e28018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5630111482a3cc6e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:06.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.166+0000 7f6e368ba640 1 --2- 172.21.15.78:0/4042298019 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6e1c0a2d50 0x7f6e1c0a5210 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6e28011a10 tx=0x7f6e2800abf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:06.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.167+0000 7f6e24dea640 1 -- 172.21.15.78:0/2200200641 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2801bb70 con 0x55ef321373f0 2024-08-18T23:50:06.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.170+0000 7f6e24dea640 1 -- 172.21.15.78:0/2200200641 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e2801bd10 con 0x55ef321373f0 2024-08-18T23:50:06.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.170+0000 7f6e24dea640 1 -- 172.21.15.78:0/2200200641 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2800fb60 con 0x55ef321373f0 2024-08-18T23:50:06.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.172+0000 7f6e3a71b740 1 -- 172.21.15.78:0/2200200641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef321373f0 msgr2=0x55ef321397e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:06.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.173+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/2200200641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef321373f0 0x55ef321397e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800ec10 tx=0x7f6e28018000 comp rx=0 tx=0).stop 2024-08-18T23:50:06.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.173+0000 7f6e3a71b740 1 -- 172.21.15.78:0/2200200641 shutdown_connections 2024-08-18T23:50:06.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.173+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/2200200641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef321373f0 0x55ef321397e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.173+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/2200200641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3212cb30 0x55ef32136eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.173+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/2200200641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef3212c210 0x55ef3212c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.173+0000 7f6e3a71b740 1 -- 172.21.15.78:0/2200200641 >> 172.21.15.78:0/2200200641 conn(0x55ef321287a0 msgr2=0x55ef32128b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:06.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.178+0000 7f6e3a71b740 1 -- 172.21.15.78:0/2200200641 shutdown_connections 2024-08-18T23:50:06.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.182+0000 7f6e3a71b740 1 -- 172.21.15.78:0/2200200641 wait complete. 2024-08-18T23:50:06.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.182+0000 7f6e3a71b740 1 -- start start 2024-08-18T23:50:06.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.182+0000 7f6e3a71b740 1 Processor -- start 2024-08-18T23:50:06.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.185+0000 7f6e3a71b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef3212c210 0x55ef321342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:06.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.185+0000 7f6e3a71b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3212cb30 0x55ef32134830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:06.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.185+0000 7f6e3a71b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef321373f0 0x55ef32134d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:06.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.185+0000 7f6e3a71b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ef32139f60 con 0x55ef3212c210 2024-08-18T23:50:06.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.185+0000 7f6e3a71b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef3204caa0 con 0x55ef3212cb30 2024-08-18T23:50:06.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.185+0000 7f6e3a71b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ef3213a0e0 con 0x55ef321373f0 2024-08-18T23:50:06.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.185+0000 7f6e3a71b740 10 client.0 _ll_register_callbacks cb 0x55ef3203be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:06.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.187+0000 7f6e368ba640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef321373f0 0x55ef32134d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:06.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.187+0000 7f6e368ba640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef321373f0 0x55ef32134d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:54214/0 (socket says 172.21.15.78:54214) 2024-08-18T23:50:06.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.187+0000 7f6e368ba640 1 -- 172.21.15.78:0/3282822266 learned_addr learned my addr 172.21.15.78:0/3282822266 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:06.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.187+0000 7f6e358b8640 1 --2- 172.21.15.78:0/3282822266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3212cb30 0x55ef32134830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:06.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.189+0000 7f6e358b8640 1 -- 172.21.15.78:0/3282822266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef321373f0 msgr2=0x55ef32134d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:06.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.189+0000 7f6e358b8640 1 --2- 172.21.15.78:0/3282822266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef321373f0 0x55ef32134d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.189+0000 7f6e358b8640 1 -- 172.21.15.78:0/3282822266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef3212c210 msgr2=0x55ef321342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.189+0000 7f6e358b8640 1 --2- 172.21.15.78:0/3282822266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef3212c210 0x55ef321342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.189+0000 7f6e358b8640 1 -- 172.21.15.78:0/3282822266 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef32030650 con 0x55ef3212cb30 2024-08-18T23:50:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.189+0000 7f6e358b8640 1 --2- 172.21.15.78:0/3282822266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3212cb30 0x55ef32134830 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6e200ed030 tx=0x7f6e20005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9effbc50d386874 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.189+0000 7f6e19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:50:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.192+0000 7f6e19ffb640 1 -- 172.21.15.78:0/3282822266 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e200bbd90 con 0x55ef3212cb30 2024-08-18T23:50:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.192+0000 7f6e3a71b740 1 -- 172.21.15.78:0/3282822266 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e2801bd10 con 0x55ef3212cb30 2024-08-18T23:50:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.192+0000 7f6e19ffb640 1 -- 172.21.15.78:0/3282822266 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f6e200c4570 con 0x55ef3212cb30 2024-08-18T23:50:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.193+0000 7f6e19ffb640 1 -- 172.21.15.78:0/3282822266 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6e200c0140 con 0x55ef3212cb30 2024-08-18T23:50:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.193+0000 7f6e3a71b740 10 client.25727 fetch_fsmap learned FSMap version 28 2024-08-18T23:50:06.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.193+0000 7f6e3a71b740 1 -- 172.21.15.78:0/3282822266 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef31e033c0 con 0x55ef3212cb30 2024-08-18T23:50:06.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.195+0000 7f6e19ffb640 1 -- 172.21.15.78:0/3282822266 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e2801bd10 con 0x55ef3212cb30 2024-08-18T23:50:06.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.195+0000 7f6e19ffb640 1 -- 172.21.15.78:0/3282822266 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6e200bac50 con 0x55ef3212cb30 2024-08-18T23:50:06.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.196+0000 7f6e3a71b740 10 client.25727 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:50:06.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.196+0000 7f6e3a71b740 10 client.25727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:50:06.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.196+0000 7f6e3a71b740 20 client.25727 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:06.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.196+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/3282822266 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ef32146f60 0x55ef321493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:06.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.196+0000 7f6e3a71b740 4 client.25727 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:50:06.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.196+0000 7f6e3a71b740 1 -- 172.21.15.78:0/3282822266 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef3209eef0 con 0x55ef32146f60 2024-08-18T23:50:06.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.201+0000 7f6e360b9640 1 --2- 172.21.15.78:0/3282822266 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ef32146f60 0x55ef321493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:06.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.205+0000 7f6e360b9640 1 --2- 172.21.15.78:0/3282822266 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ef32146f60 0x55ef321493c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93d1da5f1d6a1fa1 server_cookie=b4b213f35bb2672 in_seq=0 out_seq=0 2024-08-18T23:50:06.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.205+0000 7f6e19ffb640 10 client.25727 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:06.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.206+0000 7f6e19ffb640 1 -- 172.21.15.78:0/3282822266 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef3209eef0 con 0x55ef32146f60 2024-08-18T23:50:06.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.206+0000 7f6e19ffb640 10 client.25727 handle_command_reply: tid=0 2024-08-18T23:50:06.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.207+0000 7f6e3a71b740 1 client.25727 shutdown 2024-08-18T23:50:06.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.216+0000 7f6e3a71b740 1 -- 172.21.15.78:0/3282822266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3212cb30 msgr2=0x55ef32134830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:06.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.216+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/3282822266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3212cb30 0x55ef32134830 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6e200ed030 tx=0x7f6e20005e80 comp rx=0 tx=0).stop 2024-08-18T23:50:06.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.216+0000 7f6e3a71b740 1 -- 172.21.15.78:0/3282822266 shutdown_connections 2024-08-18T23:50:06.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.216+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/3282822266 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ef32146f60 0x55ef321493c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.216+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/3282822266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef321373f0 0x55ef32134d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.216+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/3282822266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3212cb30 0x55ef32134830 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6e200ed030 tx=0x7f6e20005e80 comp rx=0 tx=0).stop 2024-08-18T23:50:06.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.216+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/3282822266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef3212c210 0x55ef321342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.216+0000 7f6e3a71b740 1 -- 172.21.15.78:0/3282822266 >> 172.21.15.78:0/3282822266 conn(0x55ef321287a0 msgr2=0x55ef3212e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:06.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.216+0000 7f6e19ffb640 0 client.25727 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:06.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.218+0000 7f6e3a71b740 1 -- 172.21.15.78:0/3282822266 shutdown_connections 2024-08-18T23:50:06.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.219+0000 7f6e3a71b740 1 -- 172.21.15.78:0/3282822266 wait complete. 2024-08-18T23:50:06.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.219+0000 7f6e3a71b740 20 client.25727 trim_cache size 0 max 16384 2024-08-18T23:50:06.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.266+0000 7f6e3a71b740 1 Processor -- start 2024-08-18T23:50:06.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.266+0000 7f6e3a71b740 1 -- start start 2024-08-18T23:50:06.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.266+0000 7f6e3a71b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef3212c210 0x55ef32132bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:06.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.267+0000 7f6e3a71b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef321373f0 0x55ef32134bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:06.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.267+0000 7f6e360b9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef3212c210 0x55ef32132bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:06.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.267+0000 7f6e360b9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef3212c210 0x55ef32132bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:54226/0 (socket says 172.21.15.78:54226) 2024-08-18T23:50:06.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.267+0000 7f6e358b8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef321373f0 0x55ef32134bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:06.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.267+0000 7f6e358b8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef321373f0 0x55ef32134bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:58412/0 (socket says 172.21.15.78:58412) 2024-08-18T23:50:06.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.267+0000 7f6e360b9640 1 -- 172.21.15.78:0/974755394 learned_addr learned my addr 172.21.15.78:0/974755394 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:06.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.267+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/974755394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef3214ec90 0x55ef3214f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:06.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.267+0000 7f6e3a71b740 1 -- 172.21.15.78:0/974755394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e140124e0 con 0x55ef3214ec90 2024-08-18T23:50:06.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.267+0000 7f6e3a71b740 1 -- 172.21.15.78:0/974755394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1400fdb0 con 0x55ef321373f0 2024-08-18T23:50:06.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.267+0000 7f6e3a71b740 1 -- 172.21.15.78:0/974755394 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1400dc00 con 0x55ef3212c210 2024-08-18T23:50:06.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.268+0000 7f6e368ba640 1 --2- 172.21.15.78:0/974755394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef3214ec90 0x55ef3214f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:06.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.269+0000 7f6e358b8640 1 -- 172.21.15.78:0/974755394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef3212c210 msgr2=0x55ef32132bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:06.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.269+0000 7f6e358b8640 1 --2- 172.21.15.78:0/974755394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef3212c210 0x55ef32132bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.269+0000 7f6e358b8640 1 -- 172.21.15.78:0/974755394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef3214ec90 msgr2=0x55ef3214f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:06.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.269+0000 7f6e358b8640 1 --2- 172.21.15.78:0/974755394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef3214ec90 0x55ef3214f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.269+0000 7f6e358b8640 1 -- 172.21.15.78:0/974755394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef321332a0 con 0x55ef321373f0 2024-08-18T23:50:06.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.270+0000 7f6e358b8640 1 --2- 172.21.15.78:0/974755394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef321373f0 0x55ef32134bc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55ef321358a0 tx=0x7f6e2002e660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5f545dabba85c34 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:06.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.272+0000 7f6e1bfff640 1 -- 172.21.15.78:0/974755394 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e200bbb60 con 0x55ef321373f0 2024-08-18T23:50:06.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.272+0000 7f6e1bfff640 1 -- 172.21.15.78:0/974755394 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e200bbd90 con 0x55ef321373f0 2024-08-18T23:50:06.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.272+0000 7f6e1bfff640 1 -- 172.21.15.78:0/974755394 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e200badc0 con 0x55ef321373f0 2024-08-18T23:50:06.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.273+0000 7f6e3a71b740 1 -- 172.21.15.78:0/974755394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef321373f0 msgr2=0x55ef32134bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:06.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.273+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/974755394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef321373f0 0x55ef32134bc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55ef321358a0 tx=0x7f6e2002e660 comp rx=0 tx=0).stop 2024-08-18T23:50:06.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.275+0000 7f6e3a71b740 1 -- 172.21.15.78:0/974755394 shutdown_connections 2024-08-18T23:50:06.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.275+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/974755394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef3214ec90 0x55ef3214f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.275+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/974755394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef321373f0 0x55ef32134bc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.275+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/974755394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef3212c210 0x55ef32132bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.275+0000 7f6e3a71b740 1 -- 172.21.15.78:0/974755394 >> 172.21.15.78:0/974755394 conn(0x55ef321287a0 msgr2=0x55ef32136260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:06.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.276+0000 7f6e3a71b740 1 -- 172.21.15.78:0/974755394 shutdown_connections 2024-08-18T23:50:06.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.276+0000 7f6e3a71b740 1 -- 172.21.15.78:0/974755394 wait complete. 2024-08-18T23:50:06.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.276+0000 7f6e3a71b740 1 -- start start 2024-08-18T23:50:06.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.277+0000 7f6e3a71b740 1 Processor -- start 2024-08-18T23:50:06.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.277+0000 7f6e3a71b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef3212c210 0x55ef32131500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:06.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.277+0000 7f6e3a71b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef321373f0 0x55ef32131a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:06.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.277+0000 7f6e360b9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef3212c210 0x55ef32131500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:06.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.277+0000 7f6e360b9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef3212c210 0x55ef32131500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:54236/0 (socket says 172.21.15.78:54236) 2024-08-18T23:50:06.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.278+0000 7f6e360b9640 1 -- 172.21.15.78:0/2895787375 learned_addr learned my addr 172.21.15.78:0/2895787375 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:06.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.278+0000 7f6e358b8640 1 --2- 172.21.15.78:0/2895787375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef321373f0 0x55ef32131a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:06.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.278+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/2895787375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef3214ec90 0x55ef32131f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:06.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.278+0000 7f6e3a71b740 1 -- 172.21.15.78:0/2895787375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1400e270 con 0x55ef3214ec90 2024-08-18T23:50:06.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.278+0000 7f6e3a71b740 1 -- 172.21.15.78:0/2895787375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1400f6e0 con 0x55ef321373f0 2024-08-18T23:50:06.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.278+0000 7f6e3a71b740 1 -- 172.21.15.78:0/2895787375 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1400e940 con 0x55ef3212c210 2024-08-18T23:50:06.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.278+0000 7f6e3a71b740 10 client.0 _ll_register_callbacks cb 0x7f6e2000ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:06.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.279+0000 7f6e360b9640 1 -- 172.21.15.78:0/2895787375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef321373f0 msgr2=0x55ef32131a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:06.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.279+0000 7f6e360b9640 1 --2- 172.21.15.78:0/2895787375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef321373f0 0x55ef32131a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.279+0000 7f6e360b9640 1 -- 172.21.15.78:0/2895787375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef3214ec90 msgr2=0x55ef32131f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:06.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.279+0000 7f6e360b9640 1 --2- 172.21.15.78:0/2895787375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef3214ec90 0x55ef32131f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.279+0000 7f6e360b9640 1 -- 172.21.15.78:0/2895787375 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef321b0e60 con 0x55ef3212c210 2024-08-18T23:50:06.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.279+0000 7f6e358b8640 1 --2- 172.21.15.78:0/2895787375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef321373f0 0x55ef32131a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:50:06.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.279+0000 7f6e360b9640 1 --2- 172.21.15.78:0/2895787375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef3212c210 0x55ef32131500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00cde0 tx=0x7f6e2c010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27416a91d274c05 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:06.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.280+0000 7f6e19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:50:06.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.281+0000 7f6e19ffb640 1 -- 172.21.15.78:0/2895787375 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c01c070 con 0x55ef3212c210 2024-08-18T23:50:06.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.281+0000 7f6e19ffb640 1 -- 172.21.15.78:0/2895787375 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f6e2c004220 con 0x55ef3212c210 2024-08-18T23:50:06.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.282+0000 7f6e3a71b740 1 -- 172.21.15.78:0/2895787375 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ef32099400 con 0x55ef3212c210 2024-08-18T23:50:06.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.283+0000 7f6e19ffb640 1 -- 172.21.15.78:0/2895787375 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6e2c010e30 con 0x55ef3212c210 2024-08-18T23:50:06.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.283+0000 7f6e3a71b740 10 client.25591 fetch_fsmap learned FSMap version 28 2024-08-18T23:50:06.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.283+0000 7f6e3a71b740 1 -- 172.21.15.78:0/2895787375 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef321b2170 con 0x55ef3212c210 2024-08-18T23:50:06.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.283+0000 7f6e19ffb640 1 -- 172.21.15.78:0/2895787375 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e2c004460 con 0x55ef3212c210 2024-08-18T23:50:06.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.283+0000 7f6e19ffb640 1 -- 172.21.15.78:0/2895787375 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6e2c003920 con 0x55ef3212c210 2024-08-18T23:50:06.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.283+0000 7f6e3a71b740 10 client.25591 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:50:06.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.284+0000 7f6e3a71b740 10 client.25591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:50:06.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.284+0000 7f6e3a71b740 20 client.25591 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:06.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.284+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/2895787375 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ef321b2960 0x55ef321b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:06.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.284+0000 7f6e3a71b740 4 client.25591 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:50:06.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.284+0000 7f6e3a71b740 1 -- 172.21.15.78:0/2895787375 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ef32099400 con 0x55ef321b2960 2024-08-18T23:50:06.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.284+0000 7f6e358b8640 1 --2- 172.21.15.78:0/2895787375 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ef321b2960 0x55ef321b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:06.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.284+0000 7f6e19ffb640 10 client.25591 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:06.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.285+0000 7f6e358b8640 1 --2- 172.21.15.78:0/2895787375 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ef321b2960 0x55ef321b4da0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b42abf57f45fb6 server_cookie=e9f2e6656c204bf3 in_seq=0 out_seq=0 2024-08-18T23:50:06.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.287+0000 7f6e19ffb640 1 -- 172.21.15.78:0/2895787375 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ef32099400 con 0x55ef321b2960 2024-08-18T23:50:06.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.287+0000 7f6e19ffb640 10 client.25591 handle_command_reply: tid=0 2024-08-18T23:50:06.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.287+0000 7f6e3a71b740 1 client.25591 shutdown 2024-08-18T23:50:06.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.292+0000 7f6e3a71b740 1 -- 172.21.15.78:0/2895787375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef3212c210 msgr2=0x55ef32131500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:06.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.292+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/2895787375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef3212c210 0x55ef32131500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00cde0 tx=0x7f6e2c010800 comp rx=0 tx=0).stop 2024-08-18T23:50:06.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.293+0000 7f6e3a71b740 1 -- 172.21.15.78:0/2895787375 shutdown_connections 2024-08-18T23:50:06.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.293+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/2895787375 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55ef321b2960 0x55ef321b4da0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.293+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/2895787375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ef3214ec90 0x55ef32131f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.293+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/2895787375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef321373f0 0x55ef32131a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.293+0000 7f6e3a71b740 1 --2- 172.21.15.78:0/2895787375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef3212c210 0x55ef32131500 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.293+0000 7f6e3a71b740 1 -- 172.21.15.78:0/2895787375 >> 172.21.15.78:0/2895787375 conn(0x55ef321287a0 msgr2=0x55ef3214e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:06.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.293+0000 7f6e19ffb640 0 client.25591 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:06.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.297+0000 7f6e3a71b740 1 -- 172.21.15.78:0/2895787375 shutdown_connections 2024-08-18T23:50:06.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.297+0000 7f6e3a71b740 1 -- 172.21.15.78:0/2895787375 wait complete. 2024-08-18T23:50:06.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.297+0000 7f6e3a71b740 20 client.25591 trim_cache size 0 max 16384 2024-08-18T23:50:06.302 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:50:06.303 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-08-18T23:50:06.303 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-08-18T23:50:06.303 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:50:06.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.302+0000 7f6e3831d640 1 -- 172.21.15.78:0/4042298019 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6e1c0a2d50 msgr2=0x7f6e1c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:06.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.302+0000 7f6e3831d640 1 --2- 172.21.15.78:0/4042298019 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6e1c0a2d50 0x7f6e1c0a5210 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6e28011a10 tx=0x7f6e2800abf0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.302+0000 7f6e3831d640 1 -- 172.21.15.78:0/4042298019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e3010e2b0 msgr2=0x7f6e301a7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:06.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.302+0000 7f6e3831d640 1 --2- 172.21.15.78:0/4042298019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e3010e2b0 0x7f6e301a7590 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6e20037040 tx=0x7f6e200045e0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.302+0000 7f6e3831d640 1 -- 172.21.15.78:0/4042298019 shutdown_connections 2024-08-18T23:50:06.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.302+0000 7f6e3831d640 1 --2- 172.21.15.78:0/4042298019 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6e1c0a2d50 0x7f6e1c0a5210 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.302+0000 7f6e3831d640 1 --2- 172.21.15.78:0/4042298019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e3010ec60 0x7f6e301ab920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.302+0000 7f6e3831d640 1 --2- 172.21.15.78:0/4042298019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e3010e2b0 0x7f6e301a7590 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6e20037040 tx=0x7f6e200045e0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.302+0000 7f6e3831d640 1 --2- 172.21.15.78:0/4042298019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3010d6e0 0x7f6e301a7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:06.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.302+0000 7f6e3831d640 1 -- 172.21.15.78:0/4042298019 >> 172.21.15.78:0/4042298019 conn(0x7f6e3006de70 msgr2=0x7f6e3010b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:06.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.305+0000 7f6e3831d640 1 -- 172.21.15.78:0/4042298019 shutdown_connections 2024-08-18T23:50:06.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:06.305+0000 7f6e3831d640 1 -- 172.21.15.78:0/4042298019 wait complete. 2024-08-18T23:50:06.985 INFO:tasks.workunit.client.0.smithi078.stdout: 148 1740414 289 1230172 347 1005437 1060 2024-08-18T23:50:06.994 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-18T23:50:06.994 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.2 2024-08-18T23:50:06.995 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:50:07.294 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-18T23:50:07.295 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:50:07.467 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:07 smithi078 ceph-mon[18440]: pgmap v524: 129 pgs: 129 active+clean; 703 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 456 op/s 2024-08-18T23:50:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:07 smithi161 ceph-mon[22027]: pgmap v524: 129 pgs: 129 active+clean; 703 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 456 op/s 2024-08-18T23:50:07.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:07 smithi123 ceph-mon[21808]: pgmap v524: 129 pgs: 129 active+clean; 703 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 456 op/s 2024-08-18T23:50:08.507 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:50:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:09 smithi161 ceph-mon[22027]: pgmap v525: 129 pgs: 129 active+clean; 721 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 619 op/s 2024-08-18T23:50:09.730 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:09 smithi078 ceph-mon[18440]: pgmap v525: 129 pgs: 129 active+clean; 721 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 619 op/s 2024-08-18T23:50:09.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:09 smithi123 ceph-mon[21808]: pgmap v525: 129 pgs: 129 active+clean; 721 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 619 op/s 2024-08-18T23:50:09.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.910+0000 7f12734ed640 1 -- 172.21.15.78:0/597089579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f126c109180 msgr2=0x7f126c10ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:09.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.910+0000 7f12734ed640 1 --2- 172.21.15.78:0/597089579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f126c109180 0x7f126c10ff80 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1268009900 tx=0x7f126802a420 comp rx=0 tx=0).stop 2024-08-18T23:50:09.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.914+0000 7f12734ed640 1 -- 172.21.15.78:0/597089579 shutdown_connections 2024-08-18T23:50:09.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.914+0000 7f12734ed640 1 --2- 172.21.15.78:0/597089579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f126c11ae70 0x7f126c11d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:09.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.914+0000 7f12734ed640 1 --2- 172.21.15.78:0/597089579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f126c109560 0x7f126c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:09.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.914+0000 7f12734ed640 1 --2- 172.21.15.78:0/597089579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f126c109180 0x7f126c10ff80 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:09.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.914+0000 7f12734ed640 1 -- 172.21.15.78:0/597089579 >> 172.21.15.78:0/597089579 conn(0x7f126c06de70 msgr2=0x7f126c06e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:09.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.915+0000 7f12734ed640 1 -- 172.21.15.78:0/597089579 shutdown_connections 2024-08-18T23:50:09.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.920+0000 7f12734ed640 1 -- 172.21.15.78:0/597089579 wait complete. 2024-08-18T23:50:09.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.920+0000 7f12734ed640 1 Processor -- start 2024-08-18T23:50:09.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.922+0000 7f12734ed640 1 -- start start 2024-08-18T23:50:09.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.922+0000 7f12734ed640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f126c109180 0x7f126c11a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:09.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.922+0000 7f12734ed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f126c109560 0x7f126c11a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:09.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.922+0000 7f12734ed640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f126c11ae70 0x7f126c1141e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:09.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.922+0000 7f12734ed640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f126c110c60 con 0x7f126c109560 2024-08-18T23:50:09.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.922+0000 7f12734ed640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f126c110ae0 con 0x7f126c11ae70 2024-08-18T23:50:09.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.922+0000 7f12734ed640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f126c110de0 con 0x7f126c109180 2024-08-18T23:50:09.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.924+0000 7f1271289640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f126c109180 0x7f126c11a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:09.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.928+0000 7f1271289640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f126c109180 0x7f126c11a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:54250/0 (socket says 172.21.15.78:54250) 2024-08-18T23:50:09.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.928+0000 7f1271289640 1 -- 172.21.15.78:0/3924717461 learned_addr learned my addr 172.21.15.78:0/3924717461 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:09.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.929+0000 7f1271a8a640 1 --2- 172.21.15.78:0/3924717461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f126c11ae70 0x7f126c1141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:09.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.930+0000 7f1270a88640 1 --2- 172.21.15.78:0/3924717461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f126c109560 0x7f126c11a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:09.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.934+0000 7f1271289640 1 -- 172.21.15.78:0/3924717461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f126c11ae70 msgr2=0x7f126c1141e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:09.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.934+0000 7f1271289640 1 --2- 172.21.15.78:0/3924717461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f126c11ae70 0x7f126c1141e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:09.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.934+0000 7f1271289640 1 -- 172.21.15.78:0/3924717461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f126c109560 msgr2=0x7f126c11a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:09.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.934+0000 7f1271289640 1 --2- 172.21.15.78:0/3924717461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f126c109560 0x7f126c11a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:09.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.934+0000 7f1271289640 1 -- 172.21.15.78:0/3924717461 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126c114a50 con 0x7f126c109180 2024-08-18T23:50:09.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.935+0000 7f1271289640 1 --2- 172.21.15.78:0/3924717461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f126c109180 0x7f126c11a160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1268031980 tx=0x7f1268005650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:09.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.939+0000 7f125e7fc640 1 -- 172.21.15.78:0/3924717461 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126803f030 con 0x7f126c109180 2024-08-18T23:50:09.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.939+0000 7f125e7fc640 1 -- 172.21.15.78:0/3924717461 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1268004db0 con 0x7f126c109180 2024-08-18T23:50:09.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.939+0000 7f12734ed640 1 -- 172.21.15.78:0/3924717461 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126c114ce0 con 0x7f126c109180 2024-08-18T23:50:09.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.939+0000 7f12734ed640 1 -- 172.21.15.78:0/3924717461 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f126c1bc2a0 con 0x7f126c109180 2024-08-18T23:50:09.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.942+0000 7f125e7fc640 1 -- 172.21.15.78:0/3924717461 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268035070 con 0x7f126c109180 2024-08-18T23:50:09.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.945+0000 7f12734ed640 1 -- 172.21.15.78:0/3924717461 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1250005290 con 0x7f126c109180 2024-08-18T23:50:09.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.959+0000 7f125e7fc640 1 -- 172.21.15.78:0/3924717461 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1268042030 con 0x7f126c109180 2024-08-18T23:50:09.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.960+0000 7f125e7fc640 1 --2- 172.21.15.78:0/3924717461 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f12580a2d10 0x7f12580a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:09.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.961+0000 7f125e7fc640 1 -- 172.21.15.78:0/3924717461 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f12680ef650 con 0x7f126c109180 2024-08-18T23:50:09.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.961+0000 7f1270a88640 1 --2- 172.21.15.78:0/3924717461 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f12580a2d10 0x7f12580a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:09.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.962+0000 7f1270a88640 1 --2- 172.21.15.78:0/3924717461 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f12580a2d10 0x7f12580a51d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f126c115a30 tx=0x7f1260009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:09.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:09.971+0000 7f125e7fc640 1 -- 172.21.15.78:0/3924717461 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f12680bcca0 con 0x7f126c109180 2024-08-18T23:50:10.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:10.401+0000 7f12734ed640 1 -- 172.21.15.78:0/3924717461 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1250005580 con 0x7f126c109180 2024-08-18T23:50:10.412 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:50:10.412 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"btime":"2024-08-18T23:48:43:326157+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:48:42.900232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:50:10.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:10.406+0000 7f125e7fc640 1 -- 172.21.15.78:0/3924717461 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11933 (secure 0 0 0) 0x7f12680c0fa0 con 0x7f126c109180 2024-08-18T23:50:10.412 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-08-18T23:50:10.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:10.411+0000 7f12734ed640 1 -- 172.21.15.78:0/3924717461 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f12580a2d10 msgr2=0x7f12580a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:10.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:10.411+0000 7f12734ed640 1 --2- 172.21.15.78:0/3924717461 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f12580a2d10 0x7f12580a51d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f126c115a30 tx=0x7f1260009680 comp rx=0 tx=0).stop 2024-08-18T23:50:10.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:10.411+0000 7f12734ed640 1 -- 172.21.15.78:0/3924717461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f126c109180 msgr2=0x7f126c11a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:10.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:10.411+0000 7f12734ed640 1 --2- 172.21.15.78:0/3924717461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f126c109180 0x7f126c11a160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1268031980 tx=0x7f1268005650 comp rx=0 tx=0).stop 2024-08-18T23:50:10.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:10.415+0000 7f12734ed640 1 -- 172.21.15.78:0/3924717461 shutdown_connections 2024-08-18T23:50:10.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:10.415+0000 7f12734ed640 1 --2- 172.21.15.78:0/3924717461 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f12580a2d10 0x7f12580a51d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:10.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:10.415+0000 7f12734ed640 1 --2- 172.21.15.78:0/3924717461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f126c11ae70 0x7f126c1141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:10.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:10.415+0000 7f12734ed640 1 --2- 172.21.15.78:0/3924717461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f126c109560 0x7f126c11a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:10.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:10.415+0000 7f12734ed640 1 --2- 172.21.15.78:0/3924717461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f126c109180 0x7f126c11a160 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:10.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:10.416+0000 7f12734ed640 1 -- 172.21.15.78:0/3924717461 >> 172.21.15.78:0/3924717461 conn(0x7f126c06de70 msgr2=0x7f126c0705d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:10.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:10.418+0000 7f12734ed640 1 -- 172.21.15.78:0/3924717461 shutdown_connections 2024-08-18T23:50:10.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:10.418+0000 7f12734ed640 1 -- 172.21.15.78:0/3924717461 wait complete. 2024-08-18T23:50:10.727 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:10 smithi078 ceph-mon[18440]: pgmap v526: 129 pgs: 129 active+clean; 743 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.4 MiB/s wr, 754 op/s 2024-08-18T23:50:10.728 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:10 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3924717461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:50:10.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:10 smithi123 ceph-mon[21808]: pgmap v526: 129 pgs: 129 active+clean; 743 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.4 MiB/s wr, 754 op/s 2024-08-18T23:50:10.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:10 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3924717461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:50:10.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:10 smithi161 ceph-mon[22027]: pgmap v526: 129 pgs: 129 active+clean; 743 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.4 MiB/s wr, 754 op/s 2024-08-18T23:50:10.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:10 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3924717461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:50:11.344 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:50:11.873 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 139.2 --await-for 45.0' 2024-08-18T23:50:11.873 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 139.2 --await-for 45.0 2024-08-18T23:50:12.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.387+0000 7f5925e24640 1 -- 172.21.15.78:0/175204092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f592010e120 msgr2=0x7f5920115ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:12.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.387+0000 7f5925e24640 1 --2- 172.21.15.78:0/175204092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f592010e120 0x7f5920115ce0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f590c009900 tx=0x7f590c02a410 comp rx=0 tx=0).stop 2024-08-18T23:50:12.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.401+0000 7f5925e24640 1 -- 172.21.15.78:0/175204092 shutdown_connections 2024-08-18T23:50:12.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.401+0000 7f5925e24640 1 --2- 172.21.15.78:0/175204092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f592010e120 0x7f5920115ce0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:12.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.401+0000 7f5925e24640 1 --2- 172.21.15.78:0/175204092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f592006ee30 0x7f592010d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:12.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.401+0000 7f5925e24640 1 --2- 172.21.15.78:0/175204092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f592006e480 0x7f592006e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:12.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.401+0000 7f5925e24640 1 -- 172.21.15.78:0/175204092 >> 172.21.15.78:0/175204092 conn(0x7f5920072eb0 msgr2=0x7f59200732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:12.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.406+0000 7f5925e24640 1 -- 172.21.15.78:0/175204092 shutdown_connections 2024-08-18T23:50:12.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.406+0000 7f5925e24640 1 -- 172.21.15.78:0/175204092 wait complete. 2024-08-18T23:50:12.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.407+0000 7f5925e24640 1 Processor -- start 2024-08-18T23:50:12.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.409+0000 7f5925e24640 1 -- start start 2024-08-18T23:50:12.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.409+0000 7f5925e24640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f592006e480 0x7f59201a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:12.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.409+0000 7f5925e24640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f592006ee30 0x7f59201a7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:12.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.410+0000 7f5925e24640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f592010e120 0x7f59201ab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:12.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.410+0000 7f5925e24640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5920118140 con 0x7f592006ee30 2024-08-18T23:50:12.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.410+0000 7f5925e24640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5920117fc0 con 0x7f592010e120 2024-08-18T23:50:12.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.410+0000 7f5925e24640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f59201182c0 con 0x7f592006e480 2024-08-18T23:50:12.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.412+0000 7f5924e22640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f592006e480 0x7f59201a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:12.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.412+0000 7f5924e22640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f592006e480 0x7f59201a6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:54258/0 (socket says 172.21.15.78:54258) 2024-08-18T23:50:12.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.412+0000 7f5924e22640 1 -- 172.21.15.78:0/1829476347 learned_addr learned my addr 172.21.15.78:0/1829476347 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:12.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.412+0000 7f591ffff640 1 --2- 172.21.15.78:0/1829476347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f592006ee30 0x7f59201a7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:12.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.412+0000 7f5925623640 1 --2- 172.21.15.78:0/1829476347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f592010e120 0x7f59201ab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:12.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.413+0000 7f5924e22640 1 -- 172.21.15.78:0/1829476347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f592010e120 msgr2=0x7f59201ab6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.413+0000 7f5924e22640 1 --2- 172.21.15.78:0/1829476347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f592010e120 0x7f59201ab6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.413+0000 7f5924e22640 1 -- 172.21.15.78:0/1829476347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f592006ee30 msgr2=0x7f59201a7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.413+0000 7f5924e22640 1 --2- 172.21.15.78:0/1829476347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f592006ee30 0x7f59201a7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.413+0000 7f5924e22640 1 -- 172.21.15.78:0/1829476347 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59201abda0 con 0x7f592006e480 2024-08-18T23:50:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.416+0000 7f5924e22640 1 --2- 172.21.15.78:0/1829476347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f592006e480 0x7f59201a6df0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f591000d950 tx=0x7f591000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.422+0000 7f591dffb640 1 -- 172.21.15.78:0/1829476347 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910017030 con 0x7f592006e480 2024-08-18T23:50:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.422+0000 7f591dffb640 1 -- 172.21.15.78:0/1829476347 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5910004020 con 0x7f592006e480 2024-08-18T23:50:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.422+0000 7f591dffb640 1 -- 172.21.15.78:0/1829476347 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59100047a0 con 0x7f592006e480 2024-08-18T23:50:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.423+0000 7f5925e24640 1 -- 172.21.15.78:0/1829476347 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59201ac090 con 0x7f592006e480 2024-08-18T23:50:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.424+0000 7f5925e24640 1 -- 172.21.15.78:0/1829476347 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59201b3970 con 0x7f592006e480 2024-08-18T23:50:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.428+0000 7f5925e24640 1 -- 172.21.15.78:0/1829476347 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f592006e860 con 0x7f592006e480 2024-08-18T23:50:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.433+0000 7f591dffb640 1 -- 172.21.15.78:0/1829476347 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f59100041d0 con 0x7f592006e480 2024-08-18T23:50:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.434+0000 7f591dffb640 1 --2- 172.21.15.78:0/1829476347 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f59040a2d40 0x7f59040a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.434+0000 7f591dffb640 1 -- 172.21.15.78:0/1829476347 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f59100ce7e0 con 0x7f592006e480 2024-08-18T23:50:12.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.435+0000 7f591ffff640 1 --2- 172.21.15.78:0/1829476347 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f59040a2d40 0x7f59040a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:12.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.436+0000 7f591ffff640 1 --2- 172.21.15.78:0/1829476347 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f59040a2d40 0x7f59040a5200 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f592006dcd0 tx=0x7f591800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:12.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.439+0000 7f591dffb640 1 -- 172.21.15.78:0/1829476347 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f591009b450 con 0x7f592006e480 2024-08-18T23:50:12.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.879+0000 7f5925e24640 1 -- 172.21.15.78:0/1829476347 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59201a8150 con 0x7f592006e480 2024-08-18T23:50:12.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.885+0000 7f591dffb640 1 -- 172.21.15.78:0/1829476347 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11933 (secure 0 0 0) 0x7f59201a8150 con 0x7f592006e480 2024-08-18T23:50:12.887 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:50:12.887 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"btime":"2024-08-18T23:48:43:326157+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:48:42.900232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:50:12.888 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-08-18T23:50:12.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.893+0000 7f5925e24640 1 -- 172.21.15.78:0/1829476347 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f59040a2d40 msgr2=0x7f59040a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:12.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.893+0000 7f5925e24640 1 --2- 172.21.15.78:0/1829476347 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f59040a2d40 0x7f59040a5200 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f592006dcd0 tx=0x7f591800d040 comp rx=0 tx=0).stop 2024-08-18T23:50:12.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.893+0000 7f5925e24640 1 -- 172.21.15.78:0/1829476347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f592006e480 msgr2=0x7f59201a6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:12.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.893+0000 7f5925e24640 1 --2- 172.21.15.78:0/1829476347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f592006e480 0x7f59201a6df0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f591000d950 tx=0x7f591000de10 comp rx=0 tx=0).stop 2024-08-18T23:50:12.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.893+0000 7f5925e24640 1 -- 172.21.15.78:0/1829476347 shutdown_connections 2024-08-18T23:50:12.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.893+0000 7f5925e24640 1 --2- 172.21.15.78:0/1829476347 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f59040a2d40 0x7f59040a5200 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:12.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.893+0000 7f5925e24640 1 --2- 172.21.15.78:0/1829476347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f592010e120 0x7f59201ab6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:12.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.893+0000 7f5925e24640 1 --2- 172.21.15.78:0/1829476347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f592006ee30 0x7f59201a7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:12.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.893+0000 7f5925e24640 1 --2- 172.21.15.78:0/1829476347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f592006e480 0x7f59201a6df0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:12.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.893+0000 7f5925e24640 1 -- 172.21.15.78:0/1829476347 >> 172.21.15.78:0/1829476347 conn(0x7f5920072eb0 msgr2=0x7f592006c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:12.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.898+0000 7f5925e24640 1 -- 172.21.15.78:0/1829476347 shutdown_connections 2024-08-18T23:50:12.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:12.901+0000 7f5925e24640 1 -- 172.21.15.78:0/1829476347 wait complete. 2024-08-18T23:50:13.613 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:13 smithi078 ceph-mon[18440]: pgmap v527: 129 pgs: 129 active+clean; 752 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.0 MiB/s wr, 829 op/s 2024-08-18T23:50:13.613 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:13 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1829476347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:50:13.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:13 smithi161 ceph-mon[22027]: pgmap v527: 129 pgs: 129 active+clean; 752 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.0 MiB/s wr, 829 op/s 2024-08-18T23:50:13.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:13 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1829476347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:50:13.723 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-18T23:50:13.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:13 smithi123 ceph-mon[21808]: pgmap v527: 129 pgs: 129 active+clean; 752 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.0 MiB/s wr, 829 op/s 2024-08-18T23:50:13.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:13 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1829476347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:50:14.111 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:50:14.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.828+0000 7fd3018ff640 1 -- 172.21.15.78:0/4305924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2fc11ae70 msgr2=0x7fd2fc11d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:14.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.828+0000 7fd3018ff640 1 --2- 172.21.15.78:0/4305924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2fc11ae70 0x7fd2fc11d260 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009900 tx=0x7fd2f402a3f0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.829+0000 7fd3018ff640 1 -- 172.21.15.78:0/4305924 shutdown_connections 2024-08-18T23:50:14.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.829+0000 7fd3018ff640 1 --2- 172.21.15.78:0/4305924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2fc11ae70 0x7fd2fc11d260 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.829+0000 7fd3018ff640 1 --2- 172.21.15.78:0/4305924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2fc109560 0x7fd2fc1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.829+0000 7fd3018ff640 1 --2- 172.21.15.78:0/4305924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2fc109180 0x7fd2fc10ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.829+0000 7fd3018ff640 1 -- 172.21.15.78:0/4305924 >> 172.21.15.78:0/4305924 conn(0x7fd2fc06de70 msgr2=0x7fd2fc06e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:14.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.835+0000 7fd3018ff640 1 -- 172.21.15.78:0/4305924 shutdown_connections 2024-08-18T23:50:14.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.835+0000 7fd3018ff640 1 -- 172.21.15.78:0/4305924 wait complete. 2024-08-18T23:50:14.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.836+0000 7fd3018ff640 1 Processor -- start 2024-08-18T23:50:14.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.837+0000 7fd3018ff640 1 -- start start 2024-08-18T23:50:14.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.837+0000 7fd3018ff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2fc109180 0x7fd2fc1183a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:14.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.837+0000 7fd3018ff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2fc109560 0x7fd2fc1188e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:14.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.837+0000 7fd3018ff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2fc11ae70 0x7fd2fc111610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:14.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.837+0000 7fd3018ff640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc120af0 con 0x7fd2fc109560 2024-08-18T23:50:14.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.837+0000 7fd3018ff640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc120970 con 0x7fd2fc109180 2024-08-18T23:50:14.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.837+0000 7fd3018ff640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc120c70 con 0x7fd2fc11ae70 2024-08-18T23:50:14.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.837+0000 7fd2fb7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2fc11ae70 0x7fd2fc111610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:14.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.838+0000 7fd2fb7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2fc11ae70 0x7fd2fc111610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:54276/0 (socket says 172.21.15.78:54276) 2024-08-18T23:50:14.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.838+0000 7fd2fb7fe640 1 -- 172.21.15.78:0/3297061947 learned_addr learned my addr 172.21.15.78:0/3297061947 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:14.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.838+0000 7fd2fb7fe640 1 -- 172.21.15.78:0/3297061947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2fc109180 msgr2=0x7fd2fc1183a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-18T23:50:14.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.838+0000 7fd2fb7fe640 1 --2- 172.21.15.78:0/3297061947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2fc109180 0x7fd2fc1183a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.838+0000 7fd2fb7fe640 1 -- 172.21.15.78:0/3297061947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2fc109560 msgr2=0x7fd2fc1188e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:14.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.838+0000 7fd2fb7fe640 1 --2- 172.21.15.78:0/3297061947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2fc109560 0x7fd2fc1188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.838+0000 7fd2fb7fe640 1 -- 172.21.15.78:0/3297061947 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc111ed0 con 0x7fd2fc11ae70 2024-08-18T23:50:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.839+0000 7fd2fb7fe640 1 --2- 172.21.15.78:0/3297061947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2fc11ae70 0x7fd2fc111610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd2f402a960 tx=0x7fd2f40317e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.839+0000 7fd3008fd640 1 -- 172.21.15.78:0/3297061947 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f403f030 con 0x7fd2fc11ae70 2024-08-18T23:50:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.839+0000 7fd3018ff640 1 -- 172.21.15.78:0/3297061947 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2fc112160 con 0x7fd2fc11ae70 2024-08-18T23:50:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.839+0000 7fd3018ff640 1 -- 172.21.15.78:0/3297061947 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2fc116150 con 0x7fd2fc11ae70 2024-08-18T23:50:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.840+0000 7fd3008fd640 1 -- 172.21.15.78:0/3297061947 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2f4003ad0 con 0x7fd2fc11ae70 2024-08-18T23:50:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.840+0000 7fd3008fd640 1 -- 172.21.15.78:0/3297061947 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4035070 con 0x7fd2fc11ae70 2024-08-18T23:50:14.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.846+0000 7fd3008fd640 1 -- 172.21.15.78:0/3297061947 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd2f4042030 con 0x7fd2fc11ae70 2024-08-18T23:50:14.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.851+0000 7fd3008fd640 1 --2- 172.21.15.78:0/3297061947 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2e80a2c40 0x7fd2e80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:14.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.851+0000 7fd3008fd640 1 -- 172.21.15.78:0/3297061947 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fd2f40f08b0 con 0x7fd2fc11ae70 2024-08-18T23:50:14.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.852+0000 7fd2faffd640 1 --2- 172.21.15.78:0/3297061947 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2e80a2c40 0x7fd2e80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:14.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.855+0000 7fd303cfd740 1 Processor -- start 2024-08-18T23:50:14.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.856+0000 7fd303cfd740 1 -- start start 2024-08-18T23:50:14.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.857+0000 7fd303cfd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e3399130 0x55e8e3399510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:14.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.857+0000 7fd303cfd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e3399a50 0x55e8e33a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:14.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.857+0000 7fd303cfd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e33a4310 0x55e8e33a6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:14.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.857+0000 7fd303cfd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e8e32fea70 con 0x55e8e33a4310 2024-08-18T23:50:14.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.857+0000 7fd303cfd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e8e3296780 con 0x55e8e3399130 2024-08-18T23:50:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.857+0000 7fd303cfd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e8e32faa30 con 0x55e8e3399a50 2024-08-18T23:50:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.857+0000 7fd2fa7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e3399a50 0x55e8e33a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.857+0000 7fd2fa7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e3399a50 0x55e8e33a3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:54290/0 (socket says 172.21.15.78:54290) 2024-08-18T23:50:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.857+0000 7fd2fa7fc640 1 -- 172.21.15.78:0/3944528415 learned_addr learned my addr 172.21.15.78:0/3944528415 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.857+0000 7fd2fa7fc640 1 -- 172.21.15.78:0/3944528415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e3399130 msgr2=0x55e8e3399510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T23:50:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.857+0000 7fd2fa7fc640 1 --2- 172.21.15.78:0/3944528415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e3399130 0x55e8e3399510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.858+0000 7fd2fb7fe640 1 --2- 172.21.15.78:0/3944528415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e33a4310 0x55e8e33a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.857+0000 7fd2fa7fc640 1 -- 172.21.15.78:0/3944528415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e33a4310 msgr2=0x55e8e33a6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.858+0000 7fd2fa7fc640 1 --2- 172.21.15.78:0/3944528415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e33a4310 0x55e8e33a6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.858+0000 7fd2fa7fc640 1 -- 172.21.15.78:0/3944528415 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e3070b90 con 0x55e8e3399a50 2024-08-18T23:50:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.858+0000 7fd2fb7fe640 1 --2- 172.21.15.78:0/3944528415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e33a4310 0x55e8e33a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:50:14.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.858+0000 7fd2fa7fc640 1 --2- 172.21.15.78:0/3944528415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e3399a50 0x55e8e33a3dd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40058c0 tx=0x7fd2e400bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae4bc60928409993 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:14.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.858+0000 7fd2faffd640 1 --2- 172.21.15.78:0/3297061947 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2e80a2c40 0x7fd2e80a5100 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00ab90 tx=0x7fd2ec009250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:14.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.865+0000 7fd2f1ffb640 1 -- 172.21.15.78:0/3944528415 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e400dca0 con 0x55e8e3399a50 2024-08-18T23:50:14.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.865+0000 7fd2f1ffb640 1 -- 172.21.15.78:0/3944528415 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2e400de40 con 0x55e8e3399a50 2024-08-18T23:50:14.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.865+0000 7fd2f1ffb640 1 -- 172.21.15.78:0/3944528415 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e4010460 con 0x55e8e3399a50 2024-08-18T23:50:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.866+0000 7fd303cfd740 1 -- 172.21.15.78:0/3944528415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e3399a50 msgr2=0x55e8e33a3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.866+0000 7fd303cfd740 1 --2- 172.21.15.78:0/3944528415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e3399a50 0x55e8e33a3dd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40058c0 tx=0x7fd2e400bf70 comp rx=0 tx=0).stop 2024-08-18T23:50:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.868+0000 7fd303cfd740 1 -- 172.21.15.78:0/3944528415 shutdown_connections 2024-08-18T23:50:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.868+0000 7fd303cfd740 1 --2- 172.21.15.78:0/3944528415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e33a4310 0x55e8e33a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.868+0000 7fd303cfd740 1 --2- 172.21.15.78:0/3944528415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e3399a50 0x55e8e33a3dd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.868+0000 7fd303cfd740 1 --2- 172.21.15.78:0/3944528415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e3399130 0x55e8e3399510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.868+0000 7fd303cfd740 1 -- 172.21.15.78:0/3944528415 >> 172.21.15.78:0/3944528415 conn(0x55e8e33956c0 msgr2=0x55e8e3395aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:14.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.870+0000 7fd303cfd740 1 -- 172.21.15.78:0/3944528415 shutdown_connections 2024-08-18T23:50:14.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.871+0000 7fd303cfd740 1 -- 172.21.15.78:0/3944528415 wait complete. 2024-08-18T23:50:14.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.872+0000 7fd303cfd740 1 -- start start 2024-08-18T23:50:14.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.872+0000 7fd303cfd740 1 Processor -- start 2024-08-18T23:50:14.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.877+0000 7fd303cfd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e3399130 0x55e8e33a1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:14.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.877+0000 7fd303cfd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e3399a50 0x55e8e33a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:14.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.877+0000 7fd303cfd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e33a4310 0x55e8e33a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:14.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.877+0000 7fd303cfd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e8e33a6ee0 con 0x55e8e3399130 2024-08-18T23:50:14.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.877+0000 7fd303cfd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e8e32b9990 con 0x55e8e33a4310 2024-08-18T23:50:14.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.877+0000 7fd303cfd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e8e33a7060 con 0x55e8e3399a50 2024-08-18T23:50:14.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.877+0000 7fd303cfd740 10 client.0 _ll_register_callbacks cb 0x55e8e300e090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:14.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.877+0000 7fd2fb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e33a4310 0x55e8e33a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:14.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.877+0000 7fd2fb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e33a4310 0x55e8e33a1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:58470/0 (socket says 172.21.15.78:58470) 2024-08-18T23:50:14.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.877+0000 7fd2fb7fe640 1 -- 172.21.15.78:0/1410301424 learned_addr learned my addr 172.21.15.78:0/1410301424 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:14.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.878+0000 7fd2faffd640 1 --2- 172.21.15.78:0/1410301424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e3399130 0x55e8e33a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:14.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.878+0000 7fd2fb7fe640 1 -- 172.21.15.78:0/1410301424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e3399a50 msgr2=0x55e8e33a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:14.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.878+0000 7fd2fb7fe640 1 --2- 172.21.15.78:0/1410301424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e3399a50 0x55e8e33a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.878+0000 7fd2fb7fe640 1 -- 172.21.15.78:0/1410301424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e3399130 msgr2=0x55e8e33a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:14.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.878+0000 7fd2fb7fe640 1 --2- 172.21.15.78:0/1410301424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e3399130 0x55e8e33a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.878+0000 7fd2fb7fe640 1 -- 172.21.15.78:0/1410301424 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e32b86c0 con 0x55e8e33a4310 2024-08-18T23:50:14.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.879+0000 7fd2fb7fe640 1 --2- 172.21.15.78:0/1410301424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e33a4310 0x55e8e33a1bf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4004650 tx=0x7fd2f40bd480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c781e5a5242cd2b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:14.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.880+0000 7fd2daffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:50:14.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.880+0000 7fd2daffd640 1 -- 172.21.15.78:0/1410301424 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f40c59f0 con 0x55e8e33a4310 2024-08-18T23:50:14.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.880+0000 7fd2daffd640 1 -- 172.21.15.78:0/1410301424 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fd2f4003c80 con 0x55e8e33a4310 2024-08-18T23:50:14.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.881+0000 7fd303cfd740 1 -- 172.21.15.78:0/1410301424 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e8e330c1a0 con 0x55e8e33a4310 2024-08-18T23:50:14.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.882+0000 7fd2daffd640 1 -- 172.21.15.78:0/1410301424 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd2f40da050 con 0x55e8e33a4310 2024-08-18T23:50:14.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.883+0000 7fd303cfd740 10 client.25748 fetch_fsmap learned FSMap version 28 2024-08-18T23:50:14.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.883+0000 7fd303cfd740 1 -- 172.21.15.78:0/1410301424 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e8e33b2b10 con 0x55e8e33a4310 2024-08-18T23:50:14.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.885+0000 7fd2daffd640 1 -- 172.21.15.78:0/1410301424 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e8e330c1a0 con 0x55e8e33a4310 2024-08-18T23:50:14.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.885+0000 7fd2daffd640 1 -- 172.21.15.78:0/1410301424 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd2f4038ce0 con 0x55e8e33a4310 2024-08-18T23:50:14.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.885+0000 7fd303cfd740 10 client.25748 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:50:14.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.885+0000 7fd303cfd740 10 client.25748 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:50:14.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.885+0000 7fd303cfd740 20 client.25748 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:14.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.885+0000 7fd303cfd740 1 --2- 172.21.15.78:0/1410301424 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55e8e33b40a0 0x55e8e33b6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:14.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.885+0000 7fd303cfd740 4 client.25748 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:50:14.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.885+0000 7fd303cfd740 1 -- 172.21.15.78:0/1410301424 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8e33064f0 con 0x55e8e33b40a0 2024-08-18T23:50:14.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.887+0000 7fd2faffd640 1 --2- 172.21.15.78:0/1410301424 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55e8e33b40a0 0x55e8e33b6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:14.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.892+0000 7fd2faffd640 1 --2- 172.21.15.78:0/1410301424 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55e8e33b40a0 0x55e8e33b6560 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98187179f924a2d2 server_cookie=c8f816ae8b553893 in_seq=0 out_seq=0 2024-08-18T23:50:14.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.892+0000 7fd2daffd640 10 client.25748 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:14.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.896+0000 7fd2daffd640 1 -- 172.21.15.78:0/1410301424 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8e33064f0 con 0x55e8e33b40a0 2024-08-18T23:50:14.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.896+0000 7fd2daffd640 10 client.25748 handle_command_reply: tid=0 2024-08-18T23:50:14.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.896+0000 7fd303cfd740 1 client.25748 shutdown 2024-08-18T23:50:14.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.905+0000 7fd303cfd740 1 -- 172.21.15.78:0/1410301424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e33a4310 msgr2=0x55e8e33a1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:14.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.905+0000 7fd303cfd740 1 --2- 172.21.15.78:0/1410301424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e33a4310 0x55e8e33a1bf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4004650 tx=0x7fd2f40bd480 comp rx=0 tx=0).stop 2024-08-18T23:50:14.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.905+0000 7fd303cfd740 1 -- 172.21.15.78:0/1410301424 shutdown_connections 2024-08-18T23:50:14.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.905+0000 7fd303cfd740 1 --2- 172.21.15.78:0/1410301424 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55e8e33b40a0 0x55e8e33b6560 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.905+0000 7fd303cfd740 1 --2- 172.21.15.78:0/1410301424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e33a4310 0x55e8e33a1bf0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.905+0000 7fd303cfd740 1 --2- 172.21.15.78:0/1410301424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e3399a50 0x55e8e33a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.905+0000 7fd303cfd740 1 --2- 172.21.15.78:0/1410301424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e3399130 0x55e8e33a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.905+0000 7fd2daffd640 0 client.25748 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:14.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.905+0000 7fd303cfd740 1 -- 172.21.15.78:0/1410301424 >> 172.21.15.78:0/1410301424 conn(0x55e8e33956c0 msgr2=0x55e8e339b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:14.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.905+0000 7fd303cfd740 1 -- 172.21.15.78:0/1410301424 shutdown_connections 2024-08-18T23:50:14.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.905+0000 7fd303cfd740 1 -- 172.21.15.78:0/1410301424 wait complete. 2024-08-18T23:50:14.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.905+0000 7fd303cfd740 20 client.25748 trim_cache size 0 max 16384 2024-08-18T23:50:14.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.949+0000 7fd303cfd740 1 Processor -- start 2024-08-18T23:50:14.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.953+0000 7fd303cfd740 1 -- start start 2024-08-18T23:50:14.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.953+0000 7fd303cfd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e3399130 0x55e8e339bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:14.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.953+0000 7fd303cfd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e3399a50 0x55e8e33945e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:14.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.953+0000 7fd303cfd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e33a4310 0x55e8e33bbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:14.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.953+0000 7fd303cfd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc012480 con 0x55e8e3399a50 2024-08-18T23:50:14.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.953+0000 7fd303cfd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc00fd50 con 0x55e8e3399130 2024-08-18T23:50:14.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.953+0000 7fd303cfd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc00dba0 con 0x55e8e33a4310 2024-08-18T23:50:14.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.953+0000 7fd2fa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e3399a50 0x55e8e33945e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:14.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.953+0000 7fd2fa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e3399a50 0x55e8e33945e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40730/0 (socket says 172.21.15.78:40730) 2024-08-18T23:50:14.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.953+0000 7fd2fa7fc640 1 -- 172.21.15.78:0/3306167764 learned_addr learned my addr 172.21.15.78:0/3306167764 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:14.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.954+0000 7fd2fb7fe640 1 --2- 172.21.15.78:0/3306167764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e33a4310 0x55e8e33bbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:14.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.954+0000 7fd2faffd640 1 --2- 172.21.15.78:0/3306167764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e3399130 0x55e8e339bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:14.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.954+0000 7fd2fa7fc640 1 -- 172.21.15.78:0/3306167764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e33a4310 msgr2=0x55e8e33bbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:14.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.954+0000 7fd2fa7fc640 1 --2- 172.21.15.78:0/3306167764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e33a4310 0x55e8e33bbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.954+0000 7fd2fa7fc640 1 -- 172.21.15.78:0/3306167764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e3399130 msgr2=0x55e8e339bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:14.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.954+0000 7fd2fa7fc640 1 --2- 172.21.15.78:0/3306167764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e3399130 0x55e8e339bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.954+0000 7fd2fa7fc640 1 -- 172.21.15.78:0/3306167764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e33a0120 con 0x55e8e3399a50 2024-08-18T23:50:14.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.954+0000 7fd2fa7fc640 1 --2- 172.21.15.78:0/3306167764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e3399a50 0x55e8e33945e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x55e8e3396520 tx=0x7fd2e400b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1718c02257798ad1 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:14.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.957+0000 7fd2f17fa640 1 -- 172.21.15.78:0/3306167764 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e4004200 con 0x55e8e3399a50 2024-08-18T23:50:14.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.958+0000 7fd2f17fa640 1 -- 172.21.15.78:0/3306167764 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2e40043a0 con 0x55e8e3399a50 2024-08-18T23:50:14.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.958+0000 7fd2f17fa640 1 -- 172.21.15.78:0/3306167764 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e4010930 con 0x55e8e3399a50 2024-08-18T23:50:14.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.959+0000 7fd303cfd740 1 -- 172.21.15.78:0/3306167764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e3399a50 msgr2=0x55e8e33945e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:14.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.959+0000 7fd303cfd740 1 --2- 172.21.15.78:0/3306167764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e3399a50 0x55e8e33945e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x55e8e3396520 tx=0x7fd2e400b7a0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.962+0000 7fd303cfd740 1 -- 172.21.15.78:0/3306167764 shutdown_connections 2024-08-18T23:50:14.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.962+0000 7fd303cfd740 1 --2- 172.21.15.78:0/3306167764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e33a4310 0x55e8e33bbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.962+0000 7fd303cfd740 1 --2- 172.21.15.78:0/3306167764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e3399a50 0x55e8e33945e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.962+0000 7fd303cfd740 1 --2- 172.21.15.78:0/3306167764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e3399130 0x55e8e339bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.962+0000 7fd303cfd740 1 -- 172.21.15.78:0/3306167764 >> 172.21.15.78:0/3306167764 conn(0x55e8e33956c0 msgr2=0x55e8e3393ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:14.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.968+0000 7fd303cfd740 1 -- 172.21.15.78:0/3306167764 shutdown_connections 2024-08-18T23:50:14.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.968+0000 7fd303cfd740 1 -- 172.21.15.78:0/3306167764 wait complete. 2024-08-18T23:50:14.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.968+0000 7fd303cfd740 1 -- start start 2024-08-18T23:50:14.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.969+0000 7fd303cfd740 1 Processor -- start 2024-08-18T23:50:14.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.970+0000 7fd303cfd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e3399130 0x55e8e33a8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:14.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.970+0000 7fd303cfd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e3399a50 0x55e8e33a8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:14.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.970+0000 7fd2faffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e3399130 0x55e8e33a8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:14.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.970+0000 7fd2faffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e3399130 0x55e8e33a8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:54302/0 (socket says 172.21.15.78:54302) 2024-08-18T23:50:14.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.970+0000 7fd303cfd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e33a4310 0x55e8e33a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:14.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.970+0000 7fd2faffd640 1 -- 172.21.15.78:0/2130302402 learned_addr learned my addr 172.21.15.78:0/2130302402 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:14.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.970+0000 7fd303cfd740 1 -- 172.21.15.78:0/2130302402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc00e210 con 0x55e8e33a4310 2024-08-18T23:50:14.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.970+0000 7fd303cfd740 1 -- 172.21.15.78:0/2130302402 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc00f680 con 0x55e8e3399a50 2024-08-18T23:50:14.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.970+0000 7fd303cfd740 1 -- 172.21.15.78:0/2130302402 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc00e8e0 con 0x55e8e3399130 2024-08-18T23:50:14.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.971+0000 7fd2fb7fe640 1 --2- 172.21.15.78:0/2130302402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e33a4310 0x55e8e33a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:14.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.971+0000 7fd2fa7fc640 1 --2- 172.21.15.78:0/2130302402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e3399a50 0x55e8e33a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:14.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.971+0000 7fd2fb7fe640 1 -- 172.21.15.78:0/2130302402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e3399130 msgr2=0x55e8e33a8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:14.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.971+0000 7fd2fb7fe640 1 --2- 172.21.15.78:0/2130302402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e3399130 0x55e8e33a8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.971+0000 7fd2fb7fe640 1 -- 172.21.15.78:0/2130302402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e3399a50 msgr2=0x55e8e33a8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:14.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.971+0000 7fd2fb7fe640 1 --2- 172.21.15.78:0/2130302402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e3399a50 0x55e8e33a8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:14.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.971+0000 7fd2fb7fe640 1 -- 172.21.15.78:0/2130302402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e341e050 con 0x55e8e33a4310 2024-08-18T23:50:14.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.972+0000 7fd2fb7fe640 1 --2- 172.21.15.78:0/2130302402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e33a4310 0x55e8e33a8d90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40b91e0 tx=0x7fd2f40bc610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8cdd3fd3f4830 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:14.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.972+0000 7fd303cfd740 10 client.0 _ll_register_callbacks cb 0x7fd2f4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:14.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.972+0000 7fd303cfd740 1 -- 172.21.15.78:0/2130302402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2e40043a0 con 0x55e8e33a4310 2024-08-18T23:50:14.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.973+0000 7fd2daffd640 10 client.16332 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:50:14.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.973+0000 7fd2daffd640 1 -- 172.21.15.78:0/2130302402 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f40b9020 con 0x55e8e33a4310 2024-08-18T23:50:14.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.973+0000 7fd2daffd640 1 -- 172.21.15.78:0/2130302402 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fd2f40c5560 con 0x55e8e33a4310 2024-08-18T23:50:14.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.974+0000 7fd2daffd640 1 -- 172.21.15.78:0/2130302402 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd2f40c59f0 con 0x55e8e33a4310 2024-08-18T23:50:14.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.975+0000 7fd303cfd740 10 client.16332 fetch_fsmap learned FSMap version 28 2024-08-18T23:50:14.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.975+0000 7fd303cfd740 1 -- 172.21.15.78:0/2130302402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55e8e341f1f0 con 0x55e8e33a4310 2024-08-18T23:50:14.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.978+0000 7fd2daffd640 1 -- 172.21.15.78:0/2130302402 <== mon.0 v2:172.21.15.78:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd2f40a2070 con 0x55e8e33a4310 2024-08-18T23:50:14.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.978+0000 7fd303cfd740 10 client.16332 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:50:14.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.978+0000 7fd303cfd740 10 client.16332 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:50:14.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.978+0000 7fd303cfd740 20 client.16332 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:14.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.978+0000 7fd303cfd740 1 --2- 172.21.15.78:0/2130302402 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55e8e341f6c0 0x55e8e3421b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:14.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.978+0000 7fd303cfd740 4 client.16332 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.2, "await_for": 45.0}] 2024-08-18T23:50:14.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.978+0000 7fd303cfd740 1 -- 172.21.15.78:0/2130302402 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.2, "await_for": 45.0}) -- 0x7fd2e40043a0 con 0x55e8e341f6c0 2024-08-18T23:50:14.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.979+0000 7fd2faffd640 1 --2- 172.21.15.78:0/2130302402 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55e8e341f6c0 0x55e8e3421b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:14.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.983+0000 7fd2faffd640 1 --2- 172.21.15.78:0/2130302402 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55e8e341f6c0 0x55e8e3421b20 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67fc06f9066d21e2 server_cookie=ebb6cb3aacd97254 in_seq=0 out_seq=0 2024-08-18T23:50:14.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:14.983+0000 7fd2daffd640 10 client.16332 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:15.005 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:50:15.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:15 smithi161 ceph-mon[22027]: pgmap v528: 129 pgs: 129 active+clean; 743 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 836 op/s 2024-08-18T23:50:15.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:15 smithi078 ceph-mon[18440]: pgmap v528: 129 pgs: 129 active+clean; 743 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 836 op/s 2024-08-18T23:50:15.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:15 smithi123 ceph-mon[21808]: pgmap v528: 129 pgs: 129 active+clean; 743 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 836 op/s 2024-08-18T23:50:15.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.945+0000 7fa2c77ee640 1 -- 172.21.15.78:0/1039476364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2c01007b0 msgr2=0x7fa2c0100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:15.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.945+0000 7fa2c77ee640 1 --2- 172.21.15.78:0/1039476364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2c01007b0 0x7fa2c0100c10 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009930 tx=0x7fa2b002a420 comp rx=0 tx=0).stop 2024-08-18T23:50:15.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.945+0000 7fa2c77ee640 1 -- 172.21.15.78:0/1039476364 shutdown_connections 2024-08-18T23:50:15.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.945+0000 7fa2c77ee640 1 --2- 172.21.15.78:0/1039476364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c0101150 0x7fa2c010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.945+0000 7fa2c77ee640 1 --2- 172.21.15.78:0/1039476364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2c01007b0 0x7fa2c0100c10 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.945+0000 7fa2c77ee640 1 --2- 172.21.15.78:0/1039476364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2c01067b0 0x7fa2c0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.945+0000 7fa2c77ee640 1 -- 172.21.15.78:0/1039476364 >> 172.21.15.78:0/1039476364 conn(0x7fa2c00fc4d0 msgr2=0x7fa2c00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:15.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.945+0000 7fa2c77ee640 1 -- 172.21.15.78:0/1039476364 shutdown_connections 2024-08-18T23:50:15.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.945+0000 7fa2c77ee640 1 -- 172.21.15.78:0/1039476364 wait complete. 2024-08-18T23:50:15.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.946+0000 7fa2c77ee640 1 Processor -- start 2024-08-18T23:50:15.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.946+0000 7fa2c77ee640 1 -- start start 2024-08-18T23:50:15.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.946+0000 7fa2c77ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2c01007b0 0x7fa2c0198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:15.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.946+0000 7fa2c77ee640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2c0101150 0x7fa2c0199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:15.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.946+0000 7fa2c77ee640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c01067b0 0x7fa2c01a4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:15.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.946+0000 7fa2c77ee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c0110bc0 con 0x7fa2c01007b0 2024-08-18T23:50:15.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.946+0000 7fa2c77ee640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c0110a40 con 0x7fa2c01067b0 2024-08-18T23:50:15.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.946+0000 7fa2c77ee640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c0110d40 con 0x7fa2c0101150 2024-08-18T23:50:15.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.946+0000 7fa2c558a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2c01007b0 0x7fa2c0198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:15.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.946+0000 7fa2c5d8b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c01067b0 0x7fa2c01a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:15.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.946+0000 7fa2c558a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2c01007b0 0x7fa2c0198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45864/0 (socket says 172.21.15.78:45864) 2024-08-18T23:50:15.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.946+0000 7fa2c558a640 1 -- 172.21.15.78:0/383833320 learned_addr learned my addr 172.21.15.78:0/383833320 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:15.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.947+0000 7fa2c5d8b640 1 -- 172.21.15.78:0/383833320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2c0101150 msgr2=0x7fa2c0199530 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-18T23:50:15.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.947+0000 7fa2c5d8b640 1 --2- 172.21.15.78:0/383833320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2c0101150 0x7fa2c0199530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.947+0000 7fa2c5d8b640 1 -- 172.21.15.78:0/383833320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2c01007b0 msgr2=0x7fa2c0198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:15.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.947+0000 7fa2c5d8b640 1 --2- 172.21.15.78:0/383833320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2c01007b0 0x7fa2c0198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.947+0000 7fa2c5d8b640 1 -- 172.21.15.78:0/383833320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c01a45b0 con 0x7fa2c01067b0 2024-08-18T23:50:15.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.947+0000 7fa2c558a640 1 --2- 172.21.15.78:0/383833320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2c01007b0 0x7fa2c0198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:50:15.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.947+0000 7fa2c5d8b640 1 --2- 172.21.15.78:0/383833320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c01067b0 0x7fa2c01a4070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800e970 tx=0x7fa2b800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:15.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.947+0000 7fa2b67fc640 1 -- 172.21.15.78:0/383833320 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b800ccc0 con 0x7fa2c01067b0 2024-08-18T23:50:15.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.947+0000 7fa2c77ee640 1 -- 172.21.15.78:0/383833320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c01a4780 con 0x7fa2c01067b0 2024-08-18T23:50:15.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.947+0000 7fa2b67fc640 1 -- 172.21.15.78:0/383833320 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2b8005020 con 0x7fa2c01067b0 2024-08-18T23:50:15.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.947+0000 7fa2b67fc640 1 -- 172.21.15.78:0/383833320 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8004a00 con 0x7fa2c01067b0 2024-08-18T23:50:15.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.947+0000 7fa2c77ee640 1 -- 172.21.15.78:0/383833320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c0197070 con 0x7fa2c01067b0 2024-08-18T23:50:15.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.950+0000 7fa2c9bec740 1 Processor -- start 2024-08-18T23:50:15.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.950+0000 7fa2b67fc640 1 -- 172.21.15.78:0/383833320 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa2b80051d0 con 0x7fa2c01067b0 2024-08-18T23:50:15.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.950+0000 7fa2c9bec740 1 -- start start 2024-08-18T23:50:15.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.951+0000 7fa2c9bec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f638f10 0x563f0f6392f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:15.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.951+0000 7fa2c9bec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f639830 0x563f0f64c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:15.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.951+0000 7fa2c9bec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f64c9e0 0x563f0f64edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:15.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.951+0000 7fa2c9bec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563f0f59ea40 con 0x563f0f639830 2024-08-18T23:50:15.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.951+0000 7fa2c9bec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563f0f5369d0 con 0x563f0f638f10 2024-08-18T23:50:15.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.951+0000 7fa2c9bec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563f0f59aa00 con 0x563f0f64c9e0 2024-08-18T23:50:15.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.951+0000 7fa2c4d89640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f638f10 0x563f0f6392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:15.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.951+0000 7fa2c4d89640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f638f10 0x563f0f6392f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:50418/0 (socket says 172.21.15.78:50418) 2024-08-18T23:50:15.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.951+0000 7fa2c4d89640 1 -- 172.21.15.78:0/956272133 learned_addr learned my addr 172.21.15.78:0/956272133 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:15.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.951+0000 7fa2c558a640 1 --2- 172.21.15.78:0/956272133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f639830 0x563f0f64c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:15.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.951+0000 7fa2c558a640 1 -- 172.21.15.78:0/956272133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f64c9e0 msgr2=0x563f0f64edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:15.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.951+0000 7fa2c4d89640 1 --2- 172.21.15.78:0/956272133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f64c9e0 0x563f0f64edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:15.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.952+0000 7fa2c558a640 1 --2- 172.21.15.78:0/956272133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f64c9e0 0x563f0f64edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.952+0000 7fa2c558a640 1 -- 172.21.15.78:0/956272133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f638f10 msgr2=0x563f0f6392f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:15.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.952+0000 7fa2c558a640 1 --2- 172.21.15.78:0/956272133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f638f10 0x563f0f6392f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.952+0000 7fa2c558a640 1 -- 172.21.15.78:0/956272133 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f0f53d650 con 0x563f0f639830 2024-08-18T23:50:15.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.952+0000 7fa2c4d89640 1 --2- 172.21.15.78:0/956272133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f64c9e0 0x563f0f64edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:50:15.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.952+0000 7fa2b67fc640 1 --2- 172.21.15.78:0/383833320 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa2ac0a2b80 0x7fa2ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:15.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.952+0000 7fa2c558a640 1 --2- 172.21.15.78:0/956272133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f639830 0x563f0f64c4a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc011780 tx=0x7fa2bc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c796213704681c1d server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:15.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.952+0000 7fa2c4d89640 1 --2- 172.21.15.78:0/383833320 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa2ac0a2b80 0x7fa2ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:15.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.952+0000 7fa2b67fc640 1 -- 172.21.15.78:0/383833320 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa2b80cf3b0 con 0x7fa2c01067b0 2024-08-18T23:50:15.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.952+0000 7fa2ab7fe640 1 -- 172.21.15.78:0/956272133 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc003b70 con 0x563f0f639830 2024-08-18T23:50:15.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.952+0000 7fa2ab7fe640 1 -- 172.21.15.78:0/956272133 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2bc00fe20 con 0x563f0f639830 2024-08-18T23:50:15.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.952+0000 7fa2ab7fe640 1 -- 172.21.15.78:0/956272133 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc0047f0 con 0x563f0f639830 2024-08-18T23:50:15.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.953+0000 7fa2c9bec740 1 -- 172.21.15.78:0/956272133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f639830 msgr2=0x563f0f64c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:15.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.953+0000 7fa2c4d89640 1 --2- 172.21.15.78:0/383833320 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa2ac0a2b80 0x7fa2ac0a5040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00096c0 tx=0x7fa2b0031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:15.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.953+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/956272133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f639830 0x563f0f64c4a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc011780 tx=0x7fa2bc011c40 comp rx=0 tx=0).stop 2024-08-18T23:50:15.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.954+0000 7fa2c9bec740 1 -- 172.21.15.78:0/956272133 shutdown_connections 2024-08-18T23:50:15.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.954+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/956272133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f64c9e0 0x563f0f64edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.954+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/956272133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f639830 0x563f0f64c4a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.954+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/956272133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f638f10 0x563f0f6392f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.954+0000 7fa2c9bec740 1 -- 172.21.15.78:0/956272133 >> 172.21.15.78:0/956272133 conn(0x563f0f635580 msgr2=0x563f0f635960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:15.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.954+0000 7fa2c9bec740 1 -- 172.21.15.78:0/956272133 shutdown_connections 2024-08-18T23:50:15.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.954+0000 7fa2c9bec740 1 -- 172.21.15.78:0/956272133 wait complete. 2024-08-18T23:50:15.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.954+0000 7fa2c9bec740 1 -- start start 2024-08-18T23:50:15.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.954+0000 7fa2c9bec740 1 Processor -- start 2024-08-18T23:50:15.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2c9bec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f638f10 0x563f0f649980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:15.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2c9bec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f639830 0x563f0f649ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:15.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2c9bec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f64c9e0 0x563f0f64a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:15.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2c4d89640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f639830 0x563f0f649ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:15.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2c4d89640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f639830 0x563f0f649ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:50434/0 (socket says 172.21.15.78:50434) 2024-08-18T23:50:15.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2c5d8b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f64c9e0 0x563f0f64a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:15.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2c5d8b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f64c9e0 0x563f0f64a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45886/0 (socket says 172.21.15.78:45886) 2024-08-18T23:50:15.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2c5d8b640 1 -- 172.21.15.78:0/3019343429 learned_addr learned my addr 172.21.15.78:0/3019343429 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:15.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2c9bec740 1 -- 172.21.15.78:0/3019343429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563f0f64f550 con 0x563f0f64c9e0 2024-08-18T23:50:15.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2c9bec740 1 -- 172.21.15.78:0/3019343429 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563f0f559b20 con 0x563f0f639830 2024-08-18T23:50:15.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2c9bec740 1 -- 172.21.15.78:0/3019343429 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563f0f64f6d0 con 0x563f0f638f10 2024-08-18T23:50:15.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2c558a640 1 --2- 172.21.15.78:0/3019343429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f638f10 0x563f0f649980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:15.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2c9bec740 10 client.0 _ll_register_callbacks cb 0x563f0f557bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:15.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2c5d8b640 1 -- 172.21.15.78:0/3019343429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f638f10 msgr2=0x563f0f649980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:15.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2c5d8b640 1 --2- 172.21.15.78:0/3019343429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f638f10 0x563f0f649980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2c5d8b640 1 -- 172.21.15.78:0/3019343429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f639830 msgr2=0x563f0f649ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:15.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2c5d8b640 1 --2- 172.21.15.78:0/3019343429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f639830 0x563f0f649ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2c5d8b640 1 -- 172.21.15.78:0/3019343429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f0f3103c0 con 0x563f0f64c9e0 2024-08-18T23:50:15.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2c5d8b640 1 --2- 172.21.15.78:0/3019343429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f64c9e0 0x563f0f64a400 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8098290 tx=0x7fa2b809c080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d7e67ac359f4ec server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:15.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.955+0000 7fa2a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:50:15.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.956+0000 7fa2a8ff9640 1 -- 172.21.15.78:0/3019343429 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b801c040 con 0x563f0f64c9e0 2024-08-18T23:50:15.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.956+0000 7fa2a8ff9640 1 -- 172.21.15.78:0/3019343429 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa2b80860d0 con 0x563f0f64c9e0 2024-08-18T23:50:15.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.956+0000 7fa2c9bec740 1 -- 172.21.15.78:0/3019343429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2bc00fe20 con 0x563f0f64c9e0 2024-08-18T23:50:15.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.956+0000 7fa2a8ff9640 1 -- 172.21.15.78:0/3019343429 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2b80862d0 con 0x563f0f64c9e0 2024-08-18T23:50:15.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.956+0000 7fa2c9bec740 10 client.16344 fetch_fsmap learned FSMap version 28 2024-08-18T23:50:15.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.956+0000 7fa2c9bec740 1 -- 172.21.15.78:0/3019343429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563f0f65b1d0 con 0x563f0f64c9e0 2024-08-18T23:50:15.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.956+0000 7fa2a8ff9640 1 -- 172.21.15.78:0/3019343429 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2b8084e50 con 0x563f0f64c9e0 2024-08-18T23:50:15.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.956+0000 7fa2a8ff9640 1 -- 172.21.15.78:0/3019343429 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa2b8004ba0 con 0x563f0f64c9e0 2024-08-18T23:50:15.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.956+0000 7fa2c9bec740 10 client.16344 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:50:15.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.956+0000 7fa2c9bec740 10 client.16344 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:50:15.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.956+0000 7fa2c9bec740 20 client.16344 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:15.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.956+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/3019343429 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x563f0f65c690 0x563f0f65eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:15.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.957+0000 7fa2c9bec740 4 client.16344 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:50:15.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.957+0000 7fa2c9bec740 1 -- 172.21.15.78:0/3019343429 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa2bc00fe20 con 0x563f0f65c690 2024-08-18T23:50:15.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.957+0000 7fa2c558a640 1 --2- 172.21.15.78:0/3019343429 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x563f0f65c690 0x563f0f65eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:15.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.957+0000 7fa2a8ff9640 10 client.16344 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:50:15.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.957+0000 7fa2c558a640 1 --2- 172.21.15.78:0/3019343429 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x563f0f65c690 0x563f0f65eae0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=88b2964de68bb44f server_cookie=9909d49c44a5f35 in_seq=0 out_seq=0 2024-08-18T23:50:15.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.958+0000 7fa2a8ff9640 1 -- 172.21.15.78:0/3019343429 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa2bc00fe20 con 0x563f0f65c690 2024-08-18T23:50:15.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.958+0000 7fa2a8ff9640 10 client.16344 handle_command_reply: tid=0 2024-08-18T23:50:15.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.958+0000 7fa2c9bec740 1 client.16344 shutdown 2024-08-18T23:50:15.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.958+0000 7fa2c9bec740 1 -- 172.21.15.78:0/3019343429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f64c9e0 msgr2=0x563f0f64a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:15.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.958+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/3019343429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f64c9e0 0x563f0f64a400 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8098290 tx=0x7fa2b809c080 comp rx=0 tx=0).stop 2024-08-18T23:50:15.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.958+0000 7fa2c9bec740 1 -- 172.21.15.78:0/3019343429 shutdown_connections 2024-08-18T23:50:15.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.958+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/3019343429 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x563f0f65c690 0x563f0f65eae0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.958+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/3019343429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f64c9e0 0x563f0f64a400 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.958+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/3019343429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f639830 0x563f0f649ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.958+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/3019343429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f638f10 0x563f0f649980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.958+0000 7fa2c9bec740 1 -- 172.21.15.78:0/3019343429 >> 172.21.15.78:0/3019343429 conn(0x563f0f635580 msgr2=0x563f0f63b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:15.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.958+0000 7fa2a8ff9640 0 client.16344 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:50:15.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.958+0000 7fa2c9bec740 1 -- 172.21.15.78:0/3019343429 shutdown_connections 2024-08-18T23:50:15.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.958+0000 7fa2c9bec740 1 -- 172.21.15.78:0/3019343429 wait complete. 2024-08-18T23:50:15.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.959+0000 7fa2c9bec740 20 client.16344 trim_cache size 0 max 16384 2024-08-18T23:50:15.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.981+0000 7fa2c9bec740 1 Processor -- start 2024-08-18T23:50:15.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.981+0000 7fa2c9bec740 1 -- start start 2024-08-18T23:50:15.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.981+0000 7fa2c9bec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f638f10 0x563f0f6301b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:15.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.981+0000 7fa2c9bec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f639830 0x563f0f644c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:15.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.981+0000 7fa2c9bec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f64c9e0 0x563f0f645160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:15.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.981+0000 7fa2c9bec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01af170 con 0x563f0f639830 2024-08-18T23:50:15.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.981+0000 7fa2c9bec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01aca40 con 0x563f0f638f10 2024-08-18T23:50:15.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.981+0000 7fa2c9bec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01aa890 con 0x563f0f64c9e0 2024-08-18T23:50:15.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.981+0000 7fa2c4d89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f639830 0x563f0f644c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:15.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.981+0000 7fa2c558a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f638f10 0x563f0f6301b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:15.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.981+0000 7fa2c4d89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f639830 0x563f0f644c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45902/0 (socket says 172.21.15.78:45902) 2024-08-18T23:50:15.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.981+0000 7fa2c4d89640 1 -- 172.21.15.78:0/4146679717 learned_addr learned my addr 172.21.15.78:0/4146679717 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:15.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.981+0000 7fa2c5d8b640 1 --2- 172.21.15.78:0/4146679717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f64c9e0 0x563f0f645160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:15.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.982+0000 7fa2c558a640 1 -- 172.21.15.78:0/4146679717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f64c9e0 msgr2=0x563f0f645160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:15.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.982+0000 7fa2c558a640 1 --2- 172.21.15.78:0/4146679717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f64c9e0 0x563f0f645160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.982+0000 7fa2c558a640 1 -- 172.21.15.78:0/4146679717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f639830 msgr2=0x563f0f644c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:15.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.982+0000 7fa2c558a640 1 --2- 172.21.15.78:0/4146679717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f639830 0x563f0f644c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.982+0000 7fa2c558a640 1 -- 172.21.15.78:0/4146679717 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f0f6366d0 con 0x563f0f638f10 2024-08-18T23:50:15.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.982+0000 7fa2c4d89640 1 --2- 172.21.15.78:0/4146679717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f639830 0x563f0f644c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:50:15.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.982+0000 7fa2c558a640 1 --2- 172.21.15.78:0/4146679717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f638f10 0x563f0f6301b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc004450 tx=0x7fa2bc004350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d2e198886649a8a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:15.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.982+0000 7fa2aaffd640 1 -- 172.21.15.78:0/4146679717 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc01f040 con 0x563f0f638f10 2024-08-18T23:50:15.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.982+0000 7fa2aaffd640 1 -- 172.21.15.78:0/4146679717 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2bc00fd80 con 0x563f0f638f10 2024-08-18T23:50:15.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.982+0000 7fa2aaffd640 1 -- 172.21.15.78:0/4146679717 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc005330 con 0x563f0f638f10 2024-08-18T23:50:15.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.983+0000 7fa2c9bec740 1 -- 172.21.15.78:0/4146679717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f638f10 msgr2=0x563f0f6301b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:15.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.983+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/4146679717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f638f10 0x563f0f6301b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc004450 tx=0x7fa2bc004350 comp rx=0 tx=0).stop 2024-08-18T23:50:15.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.983+0000 7fa2c9bec740 1 -- 172.21.15.78:0/4146679717 shutdown_connections 2024-08-18T23:50:15.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.983+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/4146679717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f64c9e0 0x563f0f645160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.983+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/4146679717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f639830 0x563f0f644c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.983+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/4146679717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f638f10 0x563f0f6301b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.983+0000 7fa2c9bec740 1 -- 172.21.15.78:0/4146679717 >> 172.21.15.78:0/4146679717 conn(0x563f0f635580 msgr2=0x563f0f635960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:15.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.983+0000 7fa2c9bec740 1 -- 172.21.15.78:0/4146679717 shutdown_connections 2024-08-18T23:50:15.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.983+0000 7fa2c9bec740 1 -- 172.21.15.78:0/4146679717 wait complete. 2024-08-18T23:50:15.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.983+0000 7fa2c9bec740 1 -- start start 2024-08-18T23:50:15.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.983+0000 7fa2c9bec740 1 Processor -- start 2024-08-18T23:50:15.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.984+0000 7fa2c9bec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f638f10 0x563f0f64f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:15.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.984+0000 7fa2c9bec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f639830 0x563f0f64fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:15.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.984+0000 7fa2c9bec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f64c9e0 0x563f0f6503d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:15.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.984+0000 7fa2c9bec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01aaf00 con 0x563f0f638f10 2024-08-18T23:50:15.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.984+0000 7fa2c9bec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01ac370 con 0x563f0f64c9e0 2024-08-18T23:50:15.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.984+0000 7fa2c9bec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01ab5d0 con 0x563f0f639830 2024-08-18T23:50:15.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.984+0000 7fa2c9bec740 10 client.0 _ll_register_callbacks cb 0x7fa2b8097be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:15.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.984+0000 7fa2c558a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f638f10 0x563f0f64f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:15.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.984+0000 7fa2c558a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f638f10 0x563f0f64f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45906/0 (socket says 172.21.15.78:45906) 2024-08-18T23:50:15.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.984+0000 7fa2c5d8b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f64c9e0 0x563f0f6503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:15.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.984+0000 7fa2c558a640 1 -- 172.21.15.78:0/574789286 learned_addr learned my addr 172.21.15.78:0/574789286 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:15.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.984+0000 7fa2c4d89640 1 --2- 172.21.15.78:0/574789286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f639830 0x563f0f64fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:15.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.984+0000 7fa2c5d8b640 1 -- 172.21.15.78:0/574789286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f639830 msgr2=0x563f0f64fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:15.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.984+0000 7fa2c5d8b640 1 --2- 172.21.15.78:0/574789286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f639830 0x563f0f64fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.984+0000 7fa2c5d8b640 1 -- 172.21.15.78:0/574789286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f638f10 msgr2=0x563f0f64f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:15.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.984+0000 7fa2c5d8b640 1 --2- 172.21.15.78:0/574789286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f638f10 0x563f0f64f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:15.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.985+0000 7fa2c5d8b640 1 -- 172.21.15.78:0/574789286 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f0f6c55e0 con 0x563f0f64c9e0 2024-08-18T23:50:15.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.985+0000 7fa2c558a640 1 --2- 172.21.15.78:0/574789286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f638f10 0x563f0f64f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:50:15.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.985+0000 7fa2c5d8b640 1 --2- 172.21.15.78:0/574789286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f64c9e0 0x563f0f6503d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa2b801a040 tx=0x7fa2b809aee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da09da05916d463b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:15.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.985+0000 7fa2a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:50:15.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.985+0000 7fa2a8ff9640 1 -- 172.21.15.78:0/574789286 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b801c070 con 0x563f0f64c9e0 2024-08-18T23:50:15.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.985+0000 7fa2c9bec740 1 -- 172.21.15.78:0/574789286 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2bc00fd80 con 0x563f0f64c9e0 2024-08-18T23:50:15.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.985+0000 7fa2a8ff9640 1 -- 172.21.15.78:0/574789286 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa2b80a40a0 con 0x563f0f64c9e0 2024-08-18T23:50:15.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.985+0000 7fa2a8ff9640 1 -- 172.21.15.78:0/574789286 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2b8081050 con 0x563f0f64c9e0 2024-08-18T23:50:15.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.985+0000 7fa2c9bec740 10 client.25772 fetch_fsmap learned FSMap version 28 2024-08-18T23:50:15.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.985+0000 7fa2c9bec740 1 -- 172.21.15.78:0/574789286 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563f0f6c68e0 con 0x563f0f64c9e0 2024-08-18T23:50:15.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.985+0000 7fa2a8ff9640 1 -- 172.21.15.78:0/574789286 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2bc00fd80 con 0x563f0f64c9e0 2024-08-18T23:50:15.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.986+0000 7fa2a8ff9640 1 -- 172.21.15.78:0/574789286 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa2b80b4ac0 con 0x563f0f64c9e0 2024-08-18T23:50:15.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.986+0000 7fa2c9bec740 10 client.25772 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:50:15.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.986+0000 7fa2c9bec740 10 client.25772 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:50:15.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.986+0000 7fa2c9bec740 20 client.25772 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:16.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.986+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/574789286 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x563f0f6c7000 0x563f0f6c9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:16.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.986+0000 7fa2c9bec740 4 client.25772 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:50:16.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.986+0000 7fa2c9bec740 1 -- 172.21.15.78:0/574789286 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f0f5abfb0 con 0x563f0f6c7000 2024-08-18T23:50:16.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.986+0000 7fa2c558a640 1 --2- 172.21.15.78:0/574789286 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x563f0f6c7000 0x563f0f6c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:16.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.986+0000 7fa2a8ff9640 10 client.25772 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:50:16.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.986+0000 7fa2c558a640 1 --2- 172.21.15.78:0/574789286 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x563f0f6c7000 0x563f0f6c9410 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a95c42a22ac6e3bd server_cookie=bc633b573c0fbc39 in_seq=0 out_seq=0 2024-08-18T23:50:16.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.986+0000 7fa2a8ff9640 1 -- 172.21.15.78:0/574789286 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f0f5abfb0 con 0x563f0f6c7000 2024-08-18T23:50:16.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.986+0000 7fa2a8ff9640 10 client.25772 handle_command_reply: tid=0 2024-08-18T23:50:16.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.986+0000 7fa2c9bec740 1 client.25772 shutdown 2024-08-18T23:50:16.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.987+0000 7fa2c9bec740 1 -- 172.21.15.78:0/574789286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f64c9e0 msgr2=0x563f0f6503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:16.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.987+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/574789286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f64c9e0 0x563f0f6503d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa2b801a040 tx=0x7fa2b809aee0 comp rx=0 tx=0).stop 2024-08-18T23:50:16.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.987+0000 7fa2c9bec740 1 -- 172.21.15.78:0/574789286 shutdown_connections 2024-08-18T23:50:16.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.987+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/574789286 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x563f0f6c7000 0x563f0f6c9410 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:16.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.987+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/574789286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f0f64c9e0 0x563f0f6503d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:16.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.987+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/574789286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f0f639830 0x563f0f64fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:16.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.987+0000 7fa2a8ff9640 0 client.25772 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:50:16.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.987+0000 7fa2c9bec740 1 --2- 172.21.15.78:0/574789286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563f0f638f10 0x563f0f64f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:16.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.987+0000 7fa2c9bec740 1 -- 172.21.15.78:0/574789286 >> 172.21.15.78:0/574789286 conn(0x563f0f635580 msgr2=0x563f0f6301b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:16.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.987+0000 7fa2c9bec740 1 -- 172.21.15.78:0/574789286 shutdown_connections 2024-08-18T23:50:16.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.987+0000 7fa2c9bec740 1 -- 172.21.15.78:0/574789286 wait complete. 2024-08-18T23:50:16.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.987+0000 7fa2c9bec740 20 client.25772 trim_cache size 0 max 16384 2024-08-18T23:50:16.005 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:50:16.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.987+0000 7fa2c77ee640 1 -- 172.21.15.78:0/383833320 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa2ac0a2b80 msgr2=0x7fa2ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:16.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.987+0000 7fa2c77ee640 1 --2- 172.21.15.78:0/383833320 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa2ac0a2b80 0x7fa2ac0a5040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00096c0 tx=0x7fa2b0031510 comp rx=0 tx=0).stop 2024-08-18T23:50:16.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.988+0000 7fa2c77ee640 1 -- 172.21.15.78:0/383833320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c01067b0 msgr2=0x7fa2c01a4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:16.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.988+0000 7fa2c77ee640 1 --2- 172.21.15.78:0/383833320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c01067b0 0x7fa2c01a4070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800e970 tx=0x7fa2b800ee30 comp rx=0 tx=0).stop 2024-08-18T23:50:16.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.988+0000 7fa2c77ee640 1 -- 172.21.15.78:0/383833320 shutdown_connections 2024-08-18T23:50:16.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.988+0000 7fa2c77ee640 1 --2- 172.21.15.78:0/383833320 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa2ac0a2b80 0x7fa2ac0a5040 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:16.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.988+0000 7fa2c77ee640 1 --2- 172.21.15.78:0/383833320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2c01067b0 0x7fa2c01a4070 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:16.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.988+0000 7fa2c77ee640 1 --2- 172.21.15.78:0/383833320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2c0101150 0x7fa2c0199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:16.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.988+0000 7fa2c77ee640 1 --2- 172.21.15.78:0/383833320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2c01007b0 0x7fa2c0198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:16.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.988+0000 7fa2c77ee640 1 -- 172.21.15.78:0/383833320 >> 172.21.15.78:0/383833320 conn(0x7fa2c00fc4d0 msgr2=0x7fa2c00fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:16.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.988+0000 7fa2c77ee640 1 -- 172.21.15.78:0/383833320 shutdown_connections 2024-08-18T23:50:16.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:15.988+0000 7fa2c77ee640 1 -- 172.21.15.78:0/383833320 wait complete. 2024-08-18T23:50:16.908 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-18T23:50:16.985 INFO:tasks.workunit.client.0.smithi078.stdout: 154 1422514 446 1018771 493 1832771 1453 2024-08-18T23:50:17.169 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:50:17.602 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:17 smithi078 ceph-mon[18440]: pgmap v529: 129 pgs: 129 active+clean; 733 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 883 op/s 2024-08-18T23:50:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:17 smithi161 ceph-mon[22027]: pgmap v529: 129 pgs: 129 active+clean; 733 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 883 op/s 2024-08-18T23:50:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.694+0000 7fa28a19e640 1 -- 172.21.15.78:0/1597867613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2841016d0 msgr2=0x7fa284101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.694+0000 7fa28a19e640 1 --2- 172.21.15.78:0/1597867613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2841016d0 0x7fa284101b30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa274009930 tx=0x7fa27402a420 comp rx=0 tx=0).stop 2024-08-18T23:50:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.695+0000 7fa28a19e640 1 -- 172.21.15.78:0/1597867613 shutdown_connections 2024-08-18T23:50:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.695+0000 7fa28a19e640 1 --2- 172.21.15.78:0/1597867613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa284102070 0x7fa28410e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.695+0000 7fa28a19e640 1 --2- 172.21.15.78:0/1597867613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2841016d0 0x7fa284101b30 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.695+0000 7fa28a19e640 1 --2- 172.21.15.78:0/1597867613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2841076d0 0x7fa284107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.695+0000 7fa28a19e640 1 -- 172.21.15.78:0/1597867613 >> 172.21.15.78:0/1597867613 conn(0x7fa2840fd430 msgr2=0x7fa2840ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.695+0000 7fa28a19e640 1 -- 172.21.15.78:0/1597867613 shutdown_connections 2024-08-18T23:50:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.695+0000 7fa28a19e640 1 -- 172.21.15.78:0/1597867613 wait complete. 2024-08-18T23:50:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.695+0000 7fa28a19e640 1 Processor -- start 2024-08-18T23:50:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.695+0000 7fa28a19e640 1 -- start start 2024-08-18T23:50:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.696+0000 7fa28a19e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2841016d0 0x7fa284197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.696+0000 7fa28a19e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa284102070 0x7fa284198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.696+0000 7fa28a19e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2841076d0 0x7fa28419c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.696+0000 7fa28a19e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa284111ad0 con 0x7fa284102070 2024-08-18T23:50:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.696+0000 7fa28a19e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa284111950 con 0x7fa2841016d0 2024-08-18T23:50:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.696+0000 7fa28a19e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa284111c50 con 0x7fa2841076d0 2024-08-18T23:50:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.696+0000 7fa28899b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa284102070 0x7fa284198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.696+0000 7fa28899b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa284102070 0x7fa284198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45938/0 (socket says 172.21.15.78:45938) 2024-08-18T23:50:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.696+0000 7fa28919c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2841016d0 0x7fa284197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.696+0000 7fa28899b640 1 -- 172.21.15.78:0/2125623221 learned_addr learned my addr 172.21.15.78:0/2125623221 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.696+0000 7fa28999d640 1 --2- 172.21.15.78:0/2125623221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2841076d0 0x7fa28419c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.696+0000 7fa28899b640 1 -- 172.21.15.78:0/2125623221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2841076d0 msgr2=0x7fa28419c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.696+0000 7fa28899b640 1 --2- 172.21.15.78:0/2125623221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2841076d0 0x7fa28419c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.696+0000 7fa28899b640 1 -- 172.21.15.78:0/2125623221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2841016d0 msgr2=0x7fa284197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.696+0000 7fa28899b640 1 --2- 172.21.15.78:0/2125623221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2841016d0 0x7fa284197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.696+0000 7fa28899b640 1 -- 172.21.15.78:0/2125623221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28419cf80 con 0x7fa284102070 2024-08-18T23:50:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.696+0000 7fa28899b640 1 --2- 172.21.15.78:0/2125623221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa284102070 0x7fa284198510 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa274009230 tx=0x7fa274004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.697+0000 7fa27a7fc640 1 -- 172.21.15.78:0/2125623221 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27403f030 con 0x7fa284102070 2024-08-18T23:50:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.697+0000 7fa27a7fc640 1 -- 172.21.15.78:0/2125623221 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa274003ad0 con 0x7fa284102070 2024-08-18T23:50:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.697+0000 7fa28a19e640 1 -- 172.21.15.78:0/2125623221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28419d210 con 0x7fa284102070 2024-08-18T23:50:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.697+0000 7fa27a7fc640 1 -- 172.21.15.78:0/2125623221 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274035070 con 0x7fa284102070 2024-08-18T23:50:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.697+0000 7fa28a19e640 1 -- 172.21.15.78:0/2125623221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2841a4b50 con 0x7fa284102070 2024-08-18T23:50:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.699+0000 7fa27a7fc640 1 -- 172.21.15.78:0/2125623221 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa274042030 con 0x7fa284102070 2024-08-18T23:50:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.699+0000 7fa28d7fe740 1 Processor -- start 2024-08-18T23:50:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.699+0000 7fa28d7fe740 1 -- start start 2024-08-18T23:50:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.699+0000 7fa28d7fe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf6af601c0 0x55bf6af605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa28d7fe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af60ae0 0x55bf6af6ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa28d7fe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af6b3a0 0x55bf6af6d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa28d7fe740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bf6aec5a40 con 0x55bf6af60ae0 2024-08-18T23:50:17.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa28d7fe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bf6ae5d9d0 con 0x55bf6af6b3a0 2024-08-18T23:50:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa28d7fe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bf6aec1a00 con 0x55bf6af601c0 2024-08-18T23:50:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa28919c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af60ae0 0x55bf6af6ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa28919c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af60ae0 0x55bf6af6ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45940/0 (socket says 172.21.15.78:45940) 2024-08-18T23:50:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa28919c640 1 -- 172.21.15.78:0/1589222039 learned_addr learned my addr 172.21.15.78:0/1589222039 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa27a7fc640 1 --2- 172.21.15.78:0/2125623221 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa2700a2b80 0x7fa2700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa28899b640 1 --2- 172.21.15.78:0/1589222039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af6b3a0 0x55bf6af6d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa28999d640 1 --2- 172.21.15.78:0/1589222039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf6af601c0 0x55bf6af605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa28999d640 1 --2- 172.21.15.78:0/2125623221 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa2700a2b80 0x7fa2700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa28919c640 1 -- 172.21.15.78:0/1589222039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf6af601c0 msgr2=0x55bf6af605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa28919c640 1 --2- 172.21.15.78:0/1589222039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf6af601c0 0x55bf6af605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa28919c640 1 -- 172.21.15.78:0/1589222039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af6b3a0 msgr2=0x55bf6af6d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa28919c640 1 --2- 172.21.15.78:0/1589222039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af6b3a0 0x55bf6af6d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa28919c640 1 -- 172.21.15.78:0/1589222039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf6ae64650 con 0x55bf6af60ae0 2024-08-18T23:50:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa27a7fc640 1 -- 172.21.15.78:0/2125623221 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa2740ef990 con 0x7fa284102070 2024-08-18T23:50:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa28899b640 1 --2- 172.21.15.78:0/1589222039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af6b3a0 0x55bf6af6d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:50:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa28919c640 1 --2- 172.21.15.78:0/1589222039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af60ae0 0x55bf6af6ae60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa28000bc90 tx=0x7fa2800107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a7cd3b88888abb server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:17.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa26f7fe640 1 -- 172.21.15.78:0/1589222039 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa280015850 con 0x55bf6af60ae0 2024-08-18T23:50:17.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa28999d640 1 --2- 172.21.15.78:0/2125623221 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa2700a2b80 0x7fa2700a5040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa27c005c40 tx=0x7fa27c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:17.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa26f7fe640 1 -- 172.21.15.78:0/1589222039 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa280020480 con 0x55bf6af60ae0 2024-08-18T23:50:17.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.700+0000 7fa26f7fe640 1 -- 172.21.15.78:0/1589222039 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28001e5e0 con 0x55bf6af60ae0 2024-08-18T23:50:17.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.701+0000 7fa28d7fe740 1 -- 172.21.15.78:0/1589222039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af60ae0 msgr2=0x55bf6af6ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:17.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.701+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/1589222039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af60ae0 0x55bf6af6ae60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa28000bc90 tx=0x7fa2800107a0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.701+0000 7fa28d7fe740 1 -- 172.21.15.78:0/1589222039 shutdown_connections 2024-08-18T23:50:17.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.701+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/1589222039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af6b3a0 0x55bf6af6d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.701+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/1589222039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af60ae0 0x55bf6af6ae60 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.701+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/1589222039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf6af601c0 0x55bf6af605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.701+0000 7fa28d7fe740 1 -- 172.21.15.78:0/1589222039 >> 172.21.15.78:0/1589222039 conn(0x55bf6af5c750 msgr2=0x55bf6af5cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:17.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.701+0000 7fa28d7fe740 1 -- 172.21.15.78:0/1589222039 shutdown_connections 2024-08-18T23:50:17.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.701+0000 7fa28d7fe740 1 -- 172.21.15.78:0/1589222039 wait complete. 2024-08-18T23:50:17.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.702+0000 7fa28d7fe740 1 -- start start 2024-08-18T23:50:17.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.702+0000 7fa28d7fe740 1 Processor -- start 2024-08-18T23:50:17.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.702+0000 7fa28d7fe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af601c0 0x55bf6af68260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:17.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.702+0000 7fa28d7fe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf6af60ae0 0x55bf6af687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:17.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.702+0000 7fa28d7fe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af6b3a0 0x55bf6af68ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:17.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.702+0000 7fa28d7fe740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bf6af6df70 con 0x55bf6af601c0 2024-08-18T23:50:17.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.702+0000 7fa28d7fe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bf6ae80b20 con 0x55bf6af6b3a0 2024-08-18T23:50:17.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.702+0000 7fa28d7fe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55bf6af6e0f0 con 0x55bf6af60ae0 2024-08-18T23:50:17.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.702+0000 7fa28d7fe740 10 client.0 _ll_register_callbacks cb 0x55bf6ae7ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:17.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.702+0000 7fa28919c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af601c0 0x55bf6af68260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:17.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.702+0000 7fa28999d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af6b3a0 0x55bf6af68ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.702+0000 7fa28919c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af601c0 0x55bf6af68260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45944/0 (socket says 172.21.15.78:45944) 2024-08-18T23:50:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.702+0000 7fa28919c640 1 -- 172.21.15.78:0/3234935272 learned_addr learned my addr 172.21.15.78:0/3234935272 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.702+0000 7fa28899b640 1 --2- 172.21.15.78:0/3234935272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf6af60ae0 0x55bf6af687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.703+0000 7fa28919c640 1 -- 172.21.15.78:0/3234935272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf6af60ae0 msgr2=0x55bf6af687a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.703+0000 7fa28919c640 1 --2- 172.21.15.78:0/3234935272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf6af60ae0 0x55bf6af687a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.703+0000 7fa28919c640 1 -- 172.21.15.78:0/3234935272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af6b3a0 msgr2=0x55bf6af68ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.703+0000 7fa28919c640 1 --2- 172.21.15.78:0/3234935272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af6b3a0 0x55bf6af68ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.703+0000 7fa28919c640 1 -- 172.21.15.78:0/3234935272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf6ac373c0 con 0x55bf6af601c0 2024-08-18T23:50:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.703+0000 7fa28999d640 1 --2- 172.21.15.78:0/3234935272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af6b3a0 0x55bf6af68ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:50:17.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.703+0000 7fa28919c640 1 --2- 172.21.15.78:0/3234935272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af601c0 0x55bf6af68260 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa280010ca0 tx=0x7fa28000bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61b97864286685e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.703+0000 7fa26cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:50:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.703+0000 7fa26cff9640 1 -- 172.21.15.78:0/3234935272 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa280004020 con 0x55bf6af601c0 2024-08-18T23:50:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.703+0000 7fa26cff9640 1 -- 172.21.15.78:0/3234935272 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa2800041c0 con 0x55bf6af601c0 2024-08-18T23:50:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.703+0000 7fa28d7fe740 1 -- 172.21.15.78:0/3234935272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa280020480 con 0x55bf6af601c0 2024-08-18T23:50:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.703+0000 7fa26cff9640 1 -- 172.21.15.78:0/3234935272 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2800043c0 con 0x55bf6af601c0 2024-08-18T23:50:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.703+0000 7fa28d7fe740 10 client.16362 fetch_fsmap learned FSMap version 28 2024-08-18T23:50:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.703+0000 7fa28d7fe740 1 -- 172.21.15.78:0/3234935272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bf6af79bb0 con 0x55bf6af601c0 2024-08-18T23:50:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.703+0000 7fa26cff9640 1 -- 172.21.15.78:0/3234935272 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa280020480 con 0x55bf6af601c0 2024-08-18T23:50:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.703+0000 7fa26cff9640 1 -- 172.21.15.78:0/3234935272 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa2800046c0 con 0x55bf6af601c0 2024-08-18T23:50:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.704+0000 7fa28d7fe740 10 client.16362 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:50:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.704+0000 7fa28d7fe740 10 client.16362 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:50:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.704+0000 7fa28d7fe740 20 client.16362 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.704+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/3234935272 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55bf6af7afc0 0x55bf6af7d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.704+0000 7fa28d7fe740 4 client.16362 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:50:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.704+0000 7fa28d7fe740 1 -- 172.21.15.78:0/3234935272 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf6aed2fb0 con 0x55bf6af7afc0 2024-08-18T23:50:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.704+0000 7fa28899b640 1 --2- 172.21.15.78:0/3234935272 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55bf6af7afc0 0x55bf6af7d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.704+0000 7fa28899b640 1 --2- 172.21.15.78:0/3234935272 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55bf6af7afc0 0x55bf6af7d3b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=70190ef339e66678 server_cookie=792752d5a4c33861 in_seq=0 out_seq=0 2024-08-18T23:50:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.704+0000 7fa26cff9640 10 client.16362 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:50:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.705+0000 7fa26cff9640 1 -- 172.21.15.78:0/3234935272 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf6aed2fb0 con 0x55bf6af7afc0 2024-08-18T23:50:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.705+0000 7fa26cff9640 10 client.16362 handle_command_reply: tid=0 2024-08-18T23:50:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.705+0000 7fa28d7fe740 1 client.16362 shutdown 2024-08-18T23:50:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.705+0000 7fa28d7fe740 1 -- 172.21.15.78:0/3234935272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af601c0 msgr2=0x55bf6af68260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.705+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/3234935272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af601c0 0x55bf6af68260 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa280010ca0 tx=0x7fa28000bd80 comp rx=0 tx=0).stop 2024-08-18T23:50:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.705+0000 7fa28d7fe740 1 -- 172.21.15.78:0/3234935272 shutdown_connections 2024-08-18T23:50:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.705+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/3234935272 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55bf6af7afc0 0x55bf6af7d3b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.705+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/3234935272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af6b3a0 0x55bf6af68ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.705+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/3234935272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf6af60ae0 0x55bf6af687a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.705+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/3234935272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af601c0 0x55bf6af68260 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.705+0000 7fa28d7fe740 1 -- 172.21.15.78:0/3234935272 >> 172.21.15.78:0/3234935272 conn(0x55bf6af5c750 msgr2=0x55bf6af62430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.705+0000 7fa26cff9640 0 client.16362 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:50:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.705+0000 7fa28d7fe740 1 -- 172.21.15.78:0/3234935272 shutdown_connections 2024-08-18T23:50:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.705+0000 7fa28d7fe740 1 -- 172.21.15.78:0/3234935272 wait complete. 2024-08-18T23:50:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.705+0000 7fa28d7fe740 20 client.16362 trim_cache size 0 max 16384 2024-08-18T23:50:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.718+0000 7fa28d7fe740 1 Processor -- start 2024-08-18T23:50:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.718+0000 7fa28d7fe740 1 -- start start 2024-08-18T23:50:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.718+0000 7fa28d7fe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af601c0 0x55bf6af62c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.718+0000 7fa28d7fe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af60ae0 0x55bf6af82b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.719+0000 7fa28d7fe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf6af6b3a0 0x55bf6af83060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.719+0000 7fa28d7fe740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2841aeea0 con 0x55bf6af60ae0 2024-08-18T23:50:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.719+0000 7fa28d7fe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2841ac770 con 0x55bf6af601c0 2024-08-18T23:50:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.719+0000 7fa28d7fe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa2841aa5c0 con 0x55bf6af6b3a0 2024-08-18T23:50:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.719+0000 7fa28899b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af60ae0 0x55bf6af82b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.719+0000 7fa28899b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af60ae0 0x55bf6af82b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45956/0 (socket says 172.21.15.78:45956) 2024-08-18T23:50:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.719+0000 7fa28899b640 1 -- 172.21.15.78:0/2460066259 learned_addr learned my addr 172.21.15.78:0/2460066259 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.719+0000 7fa28919c640 1 --2- 172.21.15.78:0/2460066259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af601c0 0x55bf6af62c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.719+0000 7fa28899b640 1 -- 172.21.15.78:0/2460066259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf6af6b3a0 msgr2=0x55bf6af83060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.719+0000 7fa28899b640 1 --2- 172.21.15.78:0/2460066259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf6af6b3a0 0x55bf6af83060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.719+0000 7fa28899b640 1 -- 172.21.15.78:0/2460066259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af601c0 msgr2=0x55bf6af62c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.719+0000 7fa28899b640 1 --2- 172.21.15.78:0/2460066259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af601c0 0x55bf6af62c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.719+0000 7fa28899b640 1 -- 172.21.15.78:0/2460066259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf6af6daf0 con 0x55bf6af60ae0 2024-08-18T23:50:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.719+0000 7fa28899b640 1 --2- 172.21.15.78:0/2460066259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af60ae0 0x55bf6af82b20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa2740ed470 tx=0x7fa274031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69d3ad2be8fc280 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.719+0000 7fa26effd640 1 -- 172.21.15.78:0/2460066259 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274049040 con 0x55bf6af60ae0 2024-08-18T23:50:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.719+0000 7fa26effd640 1 -- 172.21.15.78:0/2460066259 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2740bbf10 con 0x55bf6af60ae0 2024-08-18T23:50:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.719+0000 7fa26effd640 1 -- 172.21.15.78:0/2460066259 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2740b8580 con 0x55bf6af60ae0 2024-08-18T23:50:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.720+0000 7fa28d7fe740 1 -- 172.21.15.78:0/2460066259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af60ae0 msgr2=0x55bf6af82b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.720+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/2460066259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af60ae0 0x55bf6af82b20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa2740ed470 tx=0x7fa274031b20 comp rx=0 tx=0).stop 2024-08-18T23:50:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.720+0000 7fa28d7fe740 1 -- 172.21.15.78:0/2460066259 shutdown_connections 2024-08-18T23:50:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.720+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/2460066259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf6af6b3a0 0x55bf6af83060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.720+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/2460066259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af60ae0 0x55bf6af82b20 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.720+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/2460066259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af601c0 0x55bf6af62c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.720+0000 7fa28d7fe740 1 -- 172.21.15.78:0/2460066259 >> 172.21.15.78:0/2460066259 conn(0x55bf6af5c750 msgr2=0x55bf6af6a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.720+0000 7fa28d7fe740 1 -- 172.21.15.78:0/2460066259 shutdown_connections 2024-08-18T23:50:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.720+0000 7fa28d7fe740 1 -- 172.21.15.78:0/2460066259 wait complete. 2024-08-18T23:50:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.720+0000 7fa28d7fe740 1 -- start start 2024-08-18T23:50:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.720+0000 7fa28d7fe740 1 Processor -- start 2024-08-18T23:50:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa28d7fe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af601c0 0x55bf6af6e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa28d7fe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf6af60ae0 0x55bf6af6e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa28d7fe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af6b3a0 0x55bf6af6ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa28d7fe740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2841aac30 con 0x55bf6af601c0 2024-08-18T23:50:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa28d7fe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2841ac0a0 con 0x55bf6af6b3a0 2024-08-18T23:50:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa28d7fe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa2841ab300 con 0x55bf6af60ae0 2024-08-18T23:50:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa28d7fe740 10 client.0 _ll_register_callbacks cb 0x7fa28000bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa28919c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af601c0 0x55bf6af6e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa28919c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af601c0 0x55bf6af6e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45962/0 (socket says 172.21.15.78:45962) 2024-08-18T23:50:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa28919c640 1 -- 172.21.15.78:0/1496306129 learned_addr learned my addr 172.21.15.78:0/1496306129 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa28999d640 1 --2- 172.21.15.78:0/1496306129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af6b3a0 0x55bf6af6ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa28919c640 1 -- 172.21.15.78:0/1496306129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf6af60ae0 msgr2=0x55bf6af6e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa28919c640 1 --2- 172.21.15.78:0/1496306129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf6af60ae0 0x55bf6af6e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa28919c640 1 -- 172.21.15.78:0/1496306129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af6b3a0 msgr2=0x55bf6af6ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa28919c640 1 --2- 172.21.15.78:0/1496306129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af6b3a0 0x55bf6af6ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa28919c640 1 -- 172.21.15.78:0/1496306129 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf6afe3d80 con 0x55bf6af601c0 2024-08-18T23:50:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa28919c640 1 --2- 172.21.15.78:0/1496306129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af601c0 0x55bf6af6e360 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa280004c80 tx=0x7fa28000bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5099ec14104dd30e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa26cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:50:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa26cff9640 1 -- 172.21.15.78:0/1496306129 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa280004760 con 0x55bf6af601c0 2024-08-18T23:50:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa26cff9640 1 -- 172.21.15.78:0/1496306129 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa280015850 con 0x55bf6af601c0 2024-08-18T23:50:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.721+0000 7fa28d7fe740 1 -- 172.21.15.78:0/1496306129 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2740bbf10 con 0x55bf6af601c0 2024-08-18T23:50:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.722+0000 7fa26cff9640 1 -- 172.21.15.78:0/1496306129 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa280047050 con 0x55bf6af601c0 2024-08-18T23:50:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.722+0000 7fa28d7fe740 10 client.16368 fetch_fsmap learned FSMap version 28 2024-08-18T23:50:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.722+0000 7fa28d7fe740 1 -- 172.21.15.78:0/1496306129 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bf6afe5080 con 0x55bf6af601c0 2024-08-18T23:50:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.722+0000 7fa26cff9640 1 -- 172.21.15.78:0/1496306129 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2740bbf10 con 0x55bf6af601c0 2024-08-18T23:50:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.722+0000 7fa26cff9640 1 -- 172.21.15.78:0/1496306129 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa280005000 con 0x55bf6af601c0 2024-08-18T23:50:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.722+0000 7fa28d7fe740 10 client.16368 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:50:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.722+0000 7fa28d7fe740 10 client.16368 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:50:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.722+0000 7fa28d7fe740 20 client.16368 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.722+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/1496306129 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55bf6afe5730 0x55bf6afe7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.722+0000 7fa28d7fe740 4 client.16368 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:50:17.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.722+0000 7fa28d7fe740 1 -- 172.21.15.78:0/1496306129 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf6aecd4c0 con 0x55bf6afe5730 2024-08-18T23:50:17.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.722+0000 7fa28899b640 1 --2- 172.21.15.78:0/1496306129 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55bf6afe5730 0x55bf6afe7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:17.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.723+0000 7fa26cff9640 10 client.16368 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:50:17.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.723+0000 7fa28899b640 1 --2- 172.21.15.78:0/1496306129 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55bf6afe5730 0x55bf6afe7b50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=287c4bc34eb35c31 server_cookie=624d877716bcba7a in_seq=0 out_seq=0 2024-08-18T23:50:17.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.723+0000 7fa26cff9640 1 -- 172.21.15.78:0/1496306129 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf6aecd4c0 con 0x55bf6afe5730 2024-08-18T23:50:17.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.723+0000 7fa26cff9640 10 client.16368 handle_command_reply: tid=0 2024-08-18T23:50:17.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.723+0000 7fa28d7fe740 1 client.16368 shutdown 2024-08-18T23:50:17.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.723+0000 7fa28d7fe740 1 -- 172.21.15.78:0/1496306129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af601c0 msgr2=0x55bf6af6e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:17.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.723+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/1496306129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af601c0 0x55bf6af6e360 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa280004c80 tx=0x7fa28000bc50 comp rx=0 tx=0).stop 2024-08-18T23:50:17.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.723+0000 7fa28d7fe740 1 -- 172.21.15.78:0/1496306129 shutdown_connections 2024-08-18T23:50:17.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.723+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/1496306129 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55bf6afe5730 0x55bf6afe7b50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.723+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/1496306129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf6af6b3a0 0x55bf6af6ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.723+0000 7fa26cff9640 0 client.16368 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:50:17.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.723+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/1496306129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bf6af60ae0 0x55bf6af6e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.723+0000 7fa28d7fe740 1 --2- 172.21.15.78:0/1496306129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf6af601c0 0x55bf6af6e360 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.723+0000 7fa28d7fe740 1 -- 172.21.15.78:0/1496306129 >> 172.21.15.78:0/1496306129 conn(0x55bf6af5c750 msgr2=0x55bf6af761c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:17.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.723+0000 7fa28d7fe740 1 -- 172.21.15.78:0/1496306129 shutdown_connections 2024-08-18T23:50:17.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.723+0000 7fa28d7fe740 1 -- 172.21.15.78:0/1496306129 wait complete. 2024-08-18T23:50:17.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.723+0000 7fa28d7fe740 20 client.16368 trim_cache size 0 max 16384 2024-08-18T23:50:17.747 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:50:17.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.724+0000 7fa28a19e640 1 -- 172.21.15.78:0/2125623221 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa2700a2b80 msgr2=0x7fa2700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.724+0000 7fa28a19e640 1 --2- 172.21.15.78:0/2125623221 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa2700a2b80 0x7fa2700a5040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa27c005c40 tx=0x7fa27c009500 comp rx=0 tx=0).stop 2024-08-18T23:50:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.724+0000 7fa28a19e640 1 -- 172.21.15.78:0/2125623221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa284102070 msgr2=0x7fa284198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.724+0000 7fa28a19e640 1 --2- 172.21.15.78:0/2125623221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa284102070 0x7fa284198510 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa274009230 tx=0x7fa274004270 comp rx=0 tx=0).stop 2024-08-18T23:50:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.724+0000 7fa28a19e640 1 -- 172.21.15.78:0/2125623221 shutdown_connections 2024-08-18T23:50:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.724+0000 7fa28a19e640 1 --2- 172.21.15.78:0/2125623221 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa2700a2b80 0x7fa2700a5040 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.724+0000 7fa28a19e640 1 --2- 172.21.15.78:0/2125623221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2841076d0 0x7fa28419c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.724+0000 7fa28a19e640 1 --2- 172.21.15.78:0/2125623221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa284102070 0x7fa284198510 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.724+0000 7fa28a19e640 1 --2- 172.21.15.78:0/2125623221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2841016d0 0x7fa284197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.724+0000 7fa28a19e640 1 -- 172.21.15.78:0/2125623221 >> 172.21.15.78:0/2125623221 conn(0x7fa2840fd430 msgr2=0x7fa2840fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.724+0000 7fa28a19e640 1 -- 172.21.15.78:0/2125623221 shutdown_connections 2024-08-18T23:50:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:17.724+0000 7fa28a19e640 1 -- 172.21.15.78:0/2125623221 wait complete. 2024-08-18T23:50:17.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:17 smithi123 ceph-mon[21808]: pgmap v529: 129 pgs: 129 active+clean; 733 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 883 op/s 2024-08-18T23:50:19.469 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-18T23:50:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:19 smithi161 ceph-mon[22027]: pgmap v530: 129 pgs: 129 active+clean; 732 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 719 op/s 2024-08-18T23:50:19.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:19 smithi078 ceph-mon[18440]: pgmap v530: 129 pgs: 129 active+clean; 732 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 719 op/s 2024-08-18T23:50:19.745 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:50:19.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:19 smithi123 ceph-mon[21808]: pgmap v530: 129 pgs: 129 active+clean; 732 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 719 op/s 2024-08-18T23:50:20.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.236+0000 7f9003918640 1 -- 172.21.15.78:0/633014473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ffc100b50 msgr2=0x7f8ffc100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:20.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.236+0000 7f9003918640 1 --2- 172.21.15.78:0/633014473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ffc100b50 0x7f8ffc100f30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8009900 tx=0x7f8ff802a480 comp rx=0 tx=0).stop 2024-08-18T23:50:20.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.236+0000 7f9003918640 1 -- 172.21.15.78:0/633014473 shutdown_connections 2024-08-18T23:50:20.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.236+0000 7f9003918640 1 --2- 172.21.15.78:0/633014473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc102c20 0x7f8ffc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.236+0000 7f9003918640 1 --2- 172.21.15.78:0/633014473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ffc1022a0 0x7f8ffc1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.236+0000 7f9003918640 1 --2- 172.21.15.78:0/633014473 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ffc100b50 0x7f8ffc100f30 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.236+0000 7f9003918640 1 -- 172.21.15.78:0/633014473 >> 172.21.15.78:0/633014473 conn(0x7f8ffc0fc500 msgr2=0x7f8ffc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:20.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.236+0000 7f9003918640 1 -- 172.21.15.78:0/633014473 shutdown_connections 2024-08-18T23:50:20.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.236+0000 7f9003918640 1 -- 172.21.15.78:0/633014473 wait complete. 2024-08-18T23:50:20.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.236+0000 7f9003918640 1 Processor -- start 2024-08-18T23:50:20.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.236+0000 7f9003918640 1 -- start start 2024-08-18T23:50:20.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f9003918640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc100b50 0x7f8ffc198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f9003918640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ffc1022a0 0x7f8ffc198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f9003918640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ffc102c20 0x7f8ffc19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f9003918640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc110cb0 con 0x7f8ffc1022a0 2024-08-18T23:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f9003918640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc110b30 con 0x7f8ffc102c20 2024-08-18T23:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f9003918640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc110e30 con 0x7f8ffc100b50 2024-08-18T23:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f9000eb3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ffc1022a0 0x7f8ffc198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f9000eb3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ffc1022a0 0x7f8ffc198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45968/0 (socket says 172.21.15.78:45968) 2024-08-18T23:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f9000eb3640 1 -- 172.21.15.78:0/1103002623 learned_addr learned my addr 172.21.15.78:0/1103002623 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f9001eb5640 1 --2- 172.21.15.78:0/1103002623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ffc102c20 0x7f8ffc19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f90016b4640 1 --2- 172.21.15.78:0/1103002623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc100b50 0x7f8ffc198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f9000eb3640 1 -- 172.21.15.78:0/1103002623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc100b50 msgr2=0x7f8ffc198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f9000eb3640 1 --2- 172.21.15.78:0/1103002623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc100b50 0x7f8ffc198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f9000eb3640 1 -- 172.21.15.78:0/1103002623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ffc102c20 msgr2=0x7f8ffc19cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f9000eb3640 1 --2- 172.21.15.78:0/1103002623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ffc102c20 0x7f8ffc19cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f9000eb3640 1 -- 172.21.15.78:0/1103002623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ffc19d1e0 con 0x7f8ffc1022a0 2024-08-18T23:50:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f9000eb3640 1 --2- 172.21.15.78:0/1103002623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ffc1022a0 0x7f8ffc198770 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00e9f0 tx=0x7f8fec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f8ff27fc640 1 -- 172.21.15.78:0/1103002623 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec00cd80 con 0x7f8ffc1022a0 2024-08-18T23:50:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f8ff27fc640 1 -- 172.21.15.78:0/1103002623 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fec010910 con 0x7f8ffc1022a0 2024-08-18T23:50:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f9003918640 1 -- 172.21.15.78:0/1103002623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ffc19d4d0 con 0x7f8ffc1022a0 2024-08-18T23:50:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f8ff27fc640 1 -- 172.21.15.78:0/1103002623 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec018990 con 0x7f8ffc1022a0 2024-08-18T23:50:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.237+0000 7f9003918640 1 -- 172.21.15.78:0/1103002623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ffc1a4db0 con 0x7f8ffc1022a0 2024-08-18T23:50:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.240+0000 7f8ff27fc640 1 -- 172.21.15.78:0/1103002623 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8fec010ac0 con 0x7f8ffc1022a0 2024-08-18T23:50:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.240+0000 7f9005d16740 1 Processor -- start 2024-08-18T23:50:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.240+0000 7f9005d16740 1 -- start start 2024-08-18T23:50:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.240+0000 7f9005d16740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689586ff0 0x55a6895873d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.240+0000 7f9005d16740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a689587910 0x55a689591c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.240+0000 7f9005d16740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6895921d0 0x55a6895945c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.240+0000 7f9005d16740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a6894eca40 con 0x55a6895921d0 2024-08-18T23:50:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.240+0000 7f9005d16740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a6894849d0 con 0x55a689586ff0 2024-08-18T23:50:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.240+0000 7f9005d16740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a6894e8a00 con 0x55a689587910 2024-08-18T23:50:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.240+0000 7f9000eb3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6895921d0 0x55a6895945c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.241+0000 7f90016b4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a689587910 0x55a689591c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.241+0000 7f9001eb5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689586ff0 0x55a6895873d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.241+0000 7f8ff27fc640 1 --2- 172.21.15.78:0/1103002623 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8fe80a2b80 0x7f8fe80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.241+0000 7f90016b4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a689587910 0x55a689591c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:56990/0 (socket says 172.21.15.78:56990) 2024-08-18T23:50:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.241+0000 7f9000eb3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6895921d0 0x55a6895945c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45978/0 (socket says 172.21.15.78:45978) 2024-08-18T23:50:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.241+0000 7f90016b4640 1 -- 172.21.15.78:0/2663295239 learned_addr learned my addr 172.21.15.78:0/2663295239 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.241+0000 7f9001eb5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689586ff0 0x55a6895873d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:50546/0 (socket says 172.21.15.78:50546) 2024-08-18T23:50:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.241+0000 7f8ff27fc640 1 -- 172.21.15.78:0/1103002623 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f8fec0cf790 con 0x7f8ffc1022a0 2024-08-18T23:50:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.241+0000 7f9001eb5640 1 --2- 172.21.15.78:0/1103002623 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8fe80a2b80 0x7f8fe80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.241+0000 7f90016b4640 1 -- 172.21.15.78:0/2663295239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689586ff0 msgr2=0x55a6895873d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.241+0000 7f90016b4640 1 --2- 172.21.15.78:0/2663295239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689586ff0 0x55a6895873d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.241+0000 7f90016b4640 1 -- 172.21.15.78:0/2663295239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6895921d0 msgr2=0x55a6895945c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.241+0000 7f90016b4640 1 --2- 172.21.15.78:0/2663295239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6895921d0 0x55a6895945c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.241+0000 7f90016b4640 1 -- 172.21.15.78:0/2663295239 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a68948b650 con 0x55a689587910 2024-08-18T23:50:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.241+0000 7f90016b4640 1 --2- 172.21.15.78:0/2663295239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a689587910 0x55a689591c90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8005ab0 tx=0x7f8ff8009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f645b872dd98ffe server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.241+0000 7f9001eb5640 1 --2- 172.21.15.78:0/1103002623 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8fe80a2b80 0x7f8fe80a5040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40098b0 tx=0x7f8ff4004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.241+0000 7f8fe77fe640 1 -- 172.21.15.78:0/2663295239 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8035070 con 0x55a689587910 2024-08-18T23:50:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.241+0000 7f8fe77fe640 1 -- 172.21.15.78:0/2663295239 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ff8005520 con 0x55a689587910 2024-08-18T23:50:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.241+0000 7f8fe77fe640 1 -- 172.21.15.78:0/2663295239 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8004dd0 con 0x55a689587910 2024-08-18T23:50:20.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.242+0000 7f9005d16740 1 -- 172.21.15.78:0/2663295239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a689587910 msgr2=0x55a689591c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.242+0000 7f9005d16740 1 --2- 172.21.15.78:0/2663295239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a689587910 0x55a689591c90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8005ab0 tx=0x7f8ff8009610 comp rx=0 tx=0).stop 2024-08-18T23:50:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.242+0000 7f9005d16740 1 -- 172.21.15.78:0/2663295239 shutdown_connections 2024-08-18T23:50:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.242+0000 7f9005d16740 1 --2- 172.21.15.78:0/2663295239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6895921d0 0x55a6895945c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.242+0000 7f9005d16740 1 --2- 172.21.15.78:0/2663295239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a689587910 0x55a689591c90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.242+0000 7f9005d16740 1 --2- 172.21.15.78:0/2663295239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689586ff0 0x55a6895873d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.242+0000 7f9005d16740 1 -- 172.21.15.78:0/2663295239 >> 172.21.15.78:0/2663295239 conn(0x55a689583580 msgr2=0x55a689583960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.242+0000 7f9005d16740 1 -- 172.21.15.78:0/2663295239 shutdown_connections 2024-08-18T23:50:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.242+0000 7f9005d16740 1 -- 172.21.15.78:0/2663295239 wait complete. 2024-08-18T23:50:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.242+0000 7f9005d16740 1 -- start start 2024-08-18T23:50:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.243+0000 7f9005d16740 1 Processor -- start 2024-08-18T23:50:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.243+0000 7f9005d16740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689586ff0 0x55a689591c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.243+0000 7f9005d16740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a689587910 0x55a68958efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.243+0000 7f9005d16740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6895921d0 0x55a68958f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.243+0000 7f9005d16740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a689594da0 con 0x55a6895921d0 2024-08-18T23:50:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.243+0000 7f9005d16740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a6894a7b20 con 0x55a689586ff0 2024-08-18T23:50:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.243+0000 7f9005d16740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a689594f20 con 0x55a689587910 2024-08-18T23:50:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.243+0000 7f9005d16740 10 client.0 _ll_register_callbacks cb 0x55a6894a5bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.243+0000 7f9001eb5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6895921d0 0x55a68958f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.243+0000 7f9000eb3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a689587910 0x55a68958efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.243+0000 7f90016b4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689586ff0 0x55a689591c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.243+0000 7f9000eb3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a689587910 0x55a68958efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:56994/0 (socket says 172.21.15.78:56994) 2024-08-18T23:50:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.243+0000 7f9000eb3640 1 -- 172.21.15.78:0/2784666215 learned_addr learned my addr 172.21.15.78:0/2784666215 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.243+0000 7f9000eb3640 1 -- 172.21.15.78:0/2784666215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689586ff0 msgr2=0x55a689591c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.243+0000 7f9000eb3640 1 --2- 172.21.15.78:0/2784666215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689586ff0 0x55a689591c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.243+0000 7f9000eb3640 1 -- 172.21.15.78:0/2784666215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6895921d0 msgr2=0x55a68958f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.243+0000 7f9000eb3640 1 --2- 172.21.15.78:0/2784666215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6895921d0 0x55a68958f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.243+0000 7f9000eb3640 1 -- 172.21.15.78:0/2784666215 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a68925e3c0 con 0x55a689587910 2024-08-18T23:50:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.243+0000 7f90016b4640 1 --2- 172.21.15.78:0/2784666215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689586ff0 0x55a689591c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:50:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.244+0000 7f9000eb3640 1 --2- 172.21.15.78:0/2784666215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a689587910 0x55a68958efc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8fec098630 tx=0x7f8fec09c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1e1292b03299aab server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.244+0000 7f8fe4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:50:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.244+0000 7f8fe4ff9640 1 -- 172.21.15.78:0/2784666215 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec026040 con 0x55a689587910 2024-08-18T23:50:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.244+0000 7f9005d16740 1 -- 172.21.15.78:0/2784666215 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ff8005520 con 0x55a689587910 2024-08-18T23:50:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.244+0000 7f8fe4ff9640 1 -- 172.21.15.78:0/2784666215 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f8fec018d30 con 0x55a689587910 2024-08-18T23:50:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.244+0000 7f8fe4ff9640 1 -- 172.21.15.78:0/2784666215 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8fec0b4e60 con 0x55a689587910 2024-08-18T23:50:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.244+0000 7f9005d16740 10 client.25642 fetch_fsmap learned FSMap version 28 2024-08-18T23:50:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.244+0000 7f9005d16740 1 -- 172.21.15.78:0/2784666215 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a6895a0900 con 0x55a689587910 2024-08-18T23:50:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.244+0000 7f8fe4ff9640 1 -- 172.21.15.78:0/2784666215 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fec0b1020 con 0x55a689587910 2024-08-18T23:50:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.244+0000 7f8fe4ff9640 1 -- 172.21.15.78:0/2784666215 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8fec0979b0 con 0x55a689587910 2024-08-18T23:50:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.244+0000 7f9005d16740 10 client.25642 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:50:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.244+0000 7f9005d16740 10 client.25642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:50:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.244+0000 7f9005d16740 20 client.25642 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.244+0000 7f9005d16740 1 --2- 172.21.15.78:0/2784666215 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a6895a1d10 0x55a6895a4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.244+0000 7f9005d16740 4 client.25642 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:50:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.244+0000 7f9005d16740 1 -- 172.21.15.78:0/2784666215 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6894f9fb0 con 0x55a6895a1d10 2024-08-18T23:50:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.245+0000 7f90016b4640 1 --2- 172.21.15.78:0/2784666215 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a6895a1d10 0x55a6895a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.245+0000 7f90016b4640 1 --2- 172.21.15.78:0/2784666215 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a6895a1d10 0x55a6895a4180 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3edf9e709aedba6 server_cookie=baced8d658c33e50 in_seq=0 out_seq=0 2024-08-18T23:50:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.245+0000 7f8fe4ff9640 10 client.25642 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.259+0000 7f8fe4ff9640 1 -- 172.21.15.78:0/2784666215 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6894f9fb0 con 0x55a6895a1d10 2024-08-18T23:50:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.259+0000 7f8fe4ff9640 10 client.25642 handle_command_reply: tid=0 2024-08-18T23:50:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.259+0000 7f9005d16740 1 client.25642 shutdown 2024-08-18T23:50:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.260+0000 7f9005d16740 1 -- 172.21.15.78:0/2784666215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a689587910 msgr2=0x55a68958efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.260+0000 7f9005d16740 1 --2- 172.21.15.78:0/2784666215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a689587910 0x55a68958efc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8fec098630 tx=0x7f8fec09c360 comp rx=0 tx=0).stop 2024-08-18T23:50:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.260+0000 7f9005d16740 1 -- 172.21.15.78:0/2784666215 shutdown_connections 2024-08-18T23:50:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.260+0000 7f9005d16740 1 --2- 172.21.15.78:0/2784666215 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a6895a1d10 0x55a6895a4180 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.260+0000 7f9005d16740 1 --2- 172.21.15.78:0/2784666215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6895921d0 0x55a68958f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.260+0000 7f8fe4ff9640 0 client.25642 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.260+0000 7f9005d16740 1 --2- 172.21.15.78:0/2784666215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a689587910 0x55a68958efc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.260+0000 7f9005d16740 1 --2- 172.21.15.78:0/2784666215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689586ff0 0x55a689591c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.260+0000 7f9005d16740 1 -- 172.21.15.78:0/2784666215 >> 172.21.15.78:0/2784666215 conn(0x55a689583580 msgr2=0x55a689589260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.260+0000 7f9005d16740 1 -- 172.21.15.78:0/2784666215 shutdown_connections 2024-08-18T23:50:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.260+0000 7f9005d16740 1 -- 172.21.15.78:0/2784666215 wait complete. 2024-08-18T23:50:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.260+0000 7f9005d16740 20 client.25642 trim_cache size 0 max 16384 2024-08-18T23:50:20.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.273+0000 7f9005d16740 1 Processor -- start 2024-08-18T23:50:20.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.273+0000 7f9005d16740 1 -- start start 2024-08-18T23:50:20.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.273+0000 7f9005d16740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689586ff0 0x55a68958a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:20.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.273+0000 7f9005d16740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a689587910 0x55a68958a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:20.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.273+0000 7f9005d16740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6895921d0 0x55a6895a9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:20.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.273+0000 7f9005d16740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc1af0c0 con 0x55a689587910 2024-08-18T23:50:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.273+0000 7f9005d16740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc1ac990 con 0x55a689586ff0 2024-08-18T23:50:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.273+0000 7f9005d16740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc1aa7e0 con 0x55a6895921d0 2024-08-18T23:50:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.273+0000 7f9000eb3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a689587910 0x55a68958a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.273+0000 7f9000eb3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a689587910 0x55a68958a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45990/0 (socket says 172.21.15.78:45990) 2024-08-18T23:50:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.273+0000 7f9000eb3640 1 -- 172.21.15.78:0/4042100995 learned_addr learned my addr 172.21.15.78:0/4042100995 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.273+0000 7f90016b4640 1 --2- 172.21.15.78:0/4042100995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689586ff0 0x55a68958a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.273+0000 7f9000eb3640 1 -- 172.21.15.78:0/4042100995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6895921d0 msgr2=0x55a6895a9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.273+0000 7f9001eb5640 1 --2- 172.21.15.78:0/4042100995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6895921d0 0x55a6895a9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.273+0000 7f9000eb3640 1 --2- 172.21.15.78:0/4042100995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6895921d0 0x55a6895a9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.273+0000 7f9000eb3640 1 -- 172.21.15.78:0/4042100995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689586ff0 msgr2=0x55a68958a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.273+0000 7f9000eb3640 1 --2- 172.21.15.78:0/4042100995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689586ff0 0x55a68958a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.274+0000 7f9000eb3640 1 -- 172.21.15.78:0/4042100995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a68958d510 con 0x55a689587910 2024-08-18T23:50:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.274+0000 7f9001eb5640 1 --2- 172.21.15.78:0/4042100995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6895921d0 0x55a6895a9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:50:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.274+0000 7f9000eb3640 1 --2- 172.21.15.78:0/4042100995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a689587910 0x55a68958a7b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00cfd0 tx=0x7f8fec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3861ef3869e5239 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.274+0000 7f8fe6ffd640 1 -- 172.21.15.78:0/4042100995 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec09aea0 con 0x55a689587910 2024-08-18T23:50:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.274+0000 7f8fe6ffd640 1 -- 172.21.15.78:0/4042100995 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fec0b1070 con 0x55a689587910 2024-08-18T23:50:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.274+0000 7f8fe6ffd640 1 -- 172.21.15.78:0/4042100995 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec018b30 con 0x55a689587910 2024-08-18T23:50:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.275+0000 7f9005d16740 1 -- 172.21.15.78:0/4042100995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a689587910 msgr2=0x55a68958a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.275+0000 7f9005d16740 1 --2- 172.21.15.78:0/4042100995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a689587910 0x55a68958a7b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00cfd0 tx=0x7f8fec000bc0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.275+0000 7f9005d16740 1 -- 172.21.15.78:0/4042100995 shutdown_connections 2024-08-18T23:50:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.275+0000 7f9005d16740 1 --2- 172.21.15.78:0/4042100995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6895921d0 0x55a6895a9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.275+0000 7f9005d16740 1 --2- 172.21.15.78:0/4042100995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a689587910 0x55a68958a7b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.275+0000 7f9005d16740 1 --2- 172.21.15.78:0/4042100995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689586ff0 0x55a68958a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.275+0000 7f9005d16740 1 -- 172.21.15.78:0/4042100995 >> 172.21.15.78:0/4042100995 conn(0x55a689583580 msgr2=0x55a68957e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.275+0000 7f9005d16740 1 -- 172.21.15.78:0/4042100995 shutdown_connections 2024-08-18T23:50:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.275+0000 7f9005d16740 1 -- 172.21.15.78:0/4042100995 wait complete. 2024-08-18T23:50:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.275+0000 7f9005d16740 1 -- start start 2024-08-18T23:50:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.275+0000 7f9005d16740 1 Processor -- start 2024-08-18T23:50:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.275+0000 7f9005d16740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a689586ff0 0x55a689592b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.275+0000 7f9005d16740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689587910 0x55a689593090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f9005d16740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6895921d0 0x55a689599170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f9005d16740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc1aae50 con 0x55a6895921d0 2024-08-18T23:50:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f9005d16740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc1ac2c0 con 0x55a689587910 2024-08-18T23:50:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f9005d16740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc1ab520 con 0x55a689586ff0 2024-08-18T23:50:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f9005d16740 10 client.0 _ll_register_callbacks cb 0x7f8fec009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f9001eb5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6895921d0 0x55a689599170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f9000eb3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689587910 0x55a689593090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f9001eb5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6895921d0 0x55a689599170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45992/0 (socket says 172.21.15.78:45992) 2024-08-18T23:50:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f9001eb5640 1 -- 172.21.15.78:0/4129298886 learned_addr learned my addr 172.21.15.78:0/4129298886 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f90016b4640 1 --2- 172.21.15.78:0/4129298886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a689586ff0 0x55a689592b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f9000eb3640 1 -- 172.21.15.78:0/4129298886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a689586ff0 msgr2=0x55a689592b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f9000eb3640 1 --2- 172.21.15.78:0/4129298886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a689586ff0 0x55a689592b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f9000eb3640 1 -- 172.21.15.78:0/4129298886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6895921d0 msgr2=0x55a689599170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f9000eb3640 1 --2- 172.21.15.78:0/4129298886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6895921d0 0x55a689599170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f9000eb3640 1 -- 172.21.15.78:0/4129298886 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6895996b0 con 0x55a689587910 2024-08-18T23:50:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f9001eb5640 1 --2- 172.21.15.78:0/4129298886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6895921d0 0x55a689599170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:50:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f9000eb3640 1 --2- 172.21.15.78:0/4129298886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689587910 0x55a689593090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55a6894a7c70 tx=0x7f8fec09aec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a42e84174325cad server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f8fe4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:50:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f8fe4ff9640 1 -- 172.21.15.78:0/4129298886 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec0a48d0 con 0x55a689587910 2024-08-18T23:50:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f9005d16740 1 -- 172.21.15.78:0/4129298886 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fec0b1070 con 0x55a689587910 2024-08-18T23:50:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.276+0000 7f8fe4ff9640 1 -- 172.21.15.78:0/4129298886 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f8fec09b0b0 con 0x55a689587910 2024-08-18T23:50:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.277+0000 7f8fe4ff9640 1 -- 172.21.15.78:0/4129298886 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8fec09b2b0 con 0x55a689587910 2024-08-18T23:50:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.277+0000 7f9005d16740 10 client.25808 fetch_fsmap learned FSMap version 28 2024-08-18T23:50:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.277+0000 7f9005d16740 1 -- 172.21.15.78:0/4129298886 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a6895996b0 con 0x55a689587910 2024-08-18T23:50:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.277+0000 7f8fe4ff9640 1 -- 172.21.15.78:0/4129298886 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fec0b1070 con 0x55a689587910 2024-08-18T23:50:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.277+0000 7f8fe4ff9640 1 -- 172.21.15.78:0/4129298886 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8fec0a11a0 con 0x55a689587910 2024-08-18T23:50:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.277+0000 7f9005d16740 10 client.25808 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:50:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.277+0000 7f9005d16740 10 client.25808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:50:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.277+0000 7f9005d16740 20 client.25808 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.277+0000 7f9005d16740 1 --2- 172.21.15.78:0/4129298886 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a68960ca20 0x55a68960ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.277+0000 7f9005d16740 4 client.25808 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:50:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.277+0000 7f9005d16740 1 -- 172.21.15.78:0/4129298886 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a6894f44c0 con 0x55a68960ca20 2024-08-18T23:50:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.277+0000 7f90016b4640 1 --2- 172.21.15.78:0/4129298886 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a68960ca20 0x55a68960ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:20.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.278+0000 7f90016b4640 1 --2- 172.21.15.78:0/4129298886 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a68960ca20 0x55a68960ee10 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78fbb2a5452e18af server_cookie=146151d73103455a in_seq=0 out_seq=0 2024-08-18T23:50:20.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.278+0000 7f8fe4ff9640 10 client.25808 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:20.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.304+0000 7f8fe4ff9640 1 -- 172.21.15.78:0/4129298886 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a6894f44c0 con 0x55a68960ca20 2024-08-18T23:50:20.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.304+0000 7f8fe4ff9640 10 client.25808 handle_command_reply: tid=0 2024-08-18T23:50:20.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.304+0000 7f9005d16740 1 client.25808 shutdown 2024-08-18T23:50:20.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.304+0000 7f9005d16740 1 -- 172.21.15.78:0/4129298886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689587910 msgr2=0x55a689593090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:20.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.304+0000 7f9005d16740 1 --2- 172.21.15.78:0/4129298886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689587910 0x55a689593090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55a6894a7c70 tx=0x7f8fec09aec0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.304+0000 7f9005d16740 1 -- 172.21.15.78:0/4129298886 shutdown_connections 2024-08-18T23:50:20.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.304+0000 7f9005d16740 1 --2- 172.21.15.78:0/4129298886 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a68960ca20 0x55a68960ee10 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.304+0000 7f9005d16740 1 --2- 172.21.15.78:0/4129298886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a6895921d0 0x55a689599170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.304+0000 7f9005d16740 1 --2- 172.21.15.78:0/4129298886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a689587910 0x55a689593090 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.304+0000 7f8fe4ff9640 0 client.25808 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:20.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.304+0000 7f9005d16740 1 --2- 172.21.15.78:0/4129298886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a689586ff0 0x55a689592b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.304+0000 7f9005d16740 1 -- 172.21.15.78:0/4129298886 >> 172.21.15.78:0/4129298886 conn(0x55a689583580 msgr2=0x55a68958a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:20.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.304+0000 7f9005d16740 1 -- 172.21.15.78:0/4129298886 shutdown_connections 2024-08-18T23:50:20.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.304+0000 7f9005d16740 1 -- 172.21.15.78:0/4129298886 wait complete. 2024-08-18T23:50:20.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.304+0000 7f9005d16740 20 client.25808 trim_cache size 0 max 16384 2024-08-18T23:50:20.307 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:50:20.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.305+0000 7f9003918640 1 -- 172.21.15.78:0/1103002623 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8fe80a2b80 msgr2=0x7f8fe80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:20.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.305+0000 7f9003918640 1 --2- 172.21.15.78:0/1103002623 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8fe80a2b80 0x7f8fe80a5040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40098b0 tx=0x7f8ff4004ab0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.305+0000 7f9003918640 1 -- 172.21.15.78:0/1103002623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ffc1022a0 msgr2=0x7f8ffc198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:20.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.305+0000 7f9003918640 1 --2- 172.21.15.78:0/1103002623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ffc1022a0 0x7f8ffc198770 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00e9f0 tx=0x7f8fec00eeb0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.305+0000 7f9003918640 1 -- 172.21.15.78:0/1103002623 shutdown_connections 2024-08-18T23:50:20.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.305+0000 7f9003918640 1 --2- 172.21.15.78:0/1103002623 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8fe80a2b80 0x7f8fe80a5040 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.305+0000 7f9003918640 1 --2- 172.21.15.78:0/1103002623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ffc102c20 0x7f8ffc19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.305+0000 7f9003918640 1 --2- 172.21.15.78:0/1103002623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ffc1022a0 0x7f8ffc198770 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.305+0000 7f9003918640 1 --2- 172.21.15.78:0/1103002623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc100b50 0x7f8ffc198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:20.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.305+0000 7f9003918640 1 -- 172.21.15.78:0/1103002623 >> 172.21.15.78:0/1103002623 conn(0x7f8ffc0fc500 msgr2=0x7f8ffc069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:20.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.305+0000 7f9003918640 1 -- 172.21.15.78:0/1103002623 shutdown_connections 2024-08-18T23:50:20.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:20.305+0000 7f9003918640 1 -- 172.21.15.78:0/1103002623 wait complete. 2024-08-18T23:50:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:21 smithi161 ceph-mon[22027]: pgmap v531: 129 pgs: 129 active+clean; 727 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 553 op/s 2024-08-18T23:50:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:21 smithi078 ceph-mon[18440]: pgmap v531: 129 pgs: 129 active+clean; 727 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 553 op/s 2024-08-18T23:50:21.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:21 smithi123 ceph-mon[21808]: pgmap v531: 129 pgs: 129 active+clean; 727 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 553 op/s 2024-08-18T23:50:22.046 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-18T23:50:22.046 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-18T23:50:22.308 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:50:22.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.846+0000 7f6fb479a640 1 -- 172.21.15.78:0/1658989325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fac102250 msgr2=0x7f6fac10e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:22.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.846+0000 7f6fb479a640 1 --2- 172.21.15.78:0/1658989325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fac102250 0x7f6fac10e780 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009cd0 tx=0x7f6fa402a8e0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.847+0000 7f6fb479a640 1 -- 172.21.15.78:0/1658989325 shutdown_connections 2024-08-18T23:50:22.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.847+0000 7f6fb479a640 1 --2- 172.21.15.78:0/1658989325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fac102250 0x7f6fac10e780 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.847+0000 7f6fb479a640 1 --2- 172.21.15.78:0/1658989325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fac1018b0 0x7f6fac101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.847+0000 7f6fb479a640 1 --2- 172.21.15.78:0/1658989325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fac1078b0 0x7f6fac107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.847+0000 7f6fb479a640 1 -- 172.21.15.78:0/1658989325 >> 172.21.15.78:0/1658989325 conn(0x7f6fac0fd5f0 msgr2=0x7f6fac0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:22.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.847+0000 7f6fb479a640 1 -- 172.21.15.78:0/1658989325 shutdown_connections 2024-08-18T23:50:22.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.847+0000 7f6fb479a640 1 -- 172.21.15.78:0/1658989325 wait complete. 2024-08-18T23:50:22.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.847+0000 7f6fb479a640 1 Processor -- start 2024-08-18T23:50:22.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.847+0000 7f6fb479a640 1 -- start start 2024-08-18T23:50:22.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.847+0000 7f6fb479a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fac1018b0 0x7f6fac198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:22.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.847+0000 7f6fb479a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fac102250 0x7f6fac1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:22.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.847+0000 7f6fb479a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fac1078b0 0x7f6fac19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:22.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.847+0000 7f6fb479a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac111cb0 con 0x7f6fac1078b0 2024-08-18T23:50:22.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.847+0000 7f6fb479a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac111b30 con 0x7f6fac102250 2024-08-18T23:50:22.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.847+0000 7f6fb479a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac111e30 con 0x7f6fac1018b0 2024-08-18T23:50:22.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.848+0000 7f6fb1d35640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fac102250 0x7f6fac1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:22.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.848+0000 7f6fb2d37640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fac1078b0 0x7f6fac19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:22.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.848+0000 7f6fb1d35640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fac102250 0x7f6fac1986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:50584/0 (socket says 172.21.15.78:50584) 2024-08-18T23:50:22.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.848+0000 7f6fb1d35640 1 -- 172.21.15.78:0/2681899974 learned_addr learned my addr 172.21.15.78:0/2681899974 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:22.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.848+0000 7f6fb2536640 1 --2- 172.21.15.78:0/2681899974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fac1018b0 0x7f6fac198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:22.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.848+0000 7f6fb2d37640 1 -- 172.21.15.78:0/2681899974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fac1018b0 msgr2=0x7f6fac198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:22.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.848+0000 7f6fb2d37640 1 --2- 172.21.15.78:0/2681899974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fac1018b0 0x7f6fac198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.848+0000 7f6fb2d37640 1 -- 172.21.15.78:0/2681899974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fac102250 msgr2=0x7f6fac1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:22.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.848+0000 7f6fb2d37640 1 --2- 172.21.15.78:0/2681899974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fac102250 0x7f6fac1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.848+0000 7f6fb2d37640 1 -- 172.21.15.78:0/2681899974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fac19d120 con 0x7f6fac1078b0 2024-08-18T23:50:22.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.848+0000 7f6fb2d37640 1 --2- 172.21.15.78:0/2681899974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fac1078b0 0x7f6fac19ca40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6fa402ae00 tx=0x7f6fa4031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:22.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.848+0000 7f6fa37fe640 1 -- 172.21.15.78:0/2681899974 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4004040 con 0x7f6fac1078b0 2024-08-18T23:50:22.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.848+0000 7f6fa37fe640 1 -- 172.21.15.78:0/2681899974 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa40046c0 con 0x7f6fac1078b0 2024-08-18T23:50:22.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.848+0000 7f6fb479a640 1 -- 172.21.15.78:0/2681899974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fac19d3b0 con 0x7f6fac1078b0 2024-08-18T23:50:22.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.848+0000 7f6fa37fe640 1 -- 172.21.15.78:0/2681899974 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4030e40 con 0x7f6fac1078b0 2024-08-18T23:50:22.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.848+0000 7f6fb479a640 1 -- 172.21.15.78:0/2681899974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fac1a4cf0 con 0x7f6fac1078b0 2024-08-18T23:50:22.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.851+0000 7f6fb6b98740 1 Processor -- start 2024-08-18T23:50:22.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.851+0000 7f6fa37fe640 1 -- 172.21.15.78:0/2681899974 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6fa40305a0 con 0x7f6fac1078b0 2024-08-18T23:50:22.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.851+0000 7f6fb6b98740 1 -- start start 2024-08-18T23:50:22.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fb6b98740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767200 0x55634d7675e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:22.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fb6b98740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d767b20 0x55634d771ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:22.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fb6b98740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d7723e0 0x55634d7747d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:22.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fb6b98740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55634d72ae40 con 0x55634d767200 2024-08-18T23:50:22.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fb6b98740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55634d6cc7f0 con 0x55634d767b20 2024-08-18T23:50:22.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fb6b98740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55634d673fb0 con 0x55634d7723e0 2024-08-18T23:50:22.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fb1d35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767200 0x55634d7675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:22.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fb1d35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767200 0x55634d7675e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46030/0 (socket says 172.21.15.78:46030) 2024-08-18T23:50:22.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fb1d35640 1 -- 172.21.15.78:0/298499020 learned_addr learned my addr 172.21.15.78:0/298499020 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:22.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fb2536640 1 --2- 172.21.15.78:0/298499020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d767b20 0x55634d771ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:22.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fa37fe640 1 --2- 172.21.15.78:0/2681899974 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6f980a2b80 0x7f6f980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:22.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fb1d35640 1 --2- 172.21.15.78:0/298499020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d7723e0 0x55634d7747d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:22.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fb2536640 1 -- 172.21.15.78:0/298499020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d7723e0 msgr2=0x55634d7747d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:22.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fb2536640 1 --2- 172.21.15.78:0/298499020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d7723e0 0x55634d7747d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fb2536640 1 -- 172.21.15.78:0/298499020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767200 msgr2=0x55634d7675e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fb2536640 1 --2- 172.21.15.78:0/298499020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767200 0x55634d7675e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fb2536640 1 -- 172.21.15.78:0/298499020 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55634d43e6a0 con 0x55634d767b20 2024-08-18T23:50:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fb1d35640 1 --2- 172.21.15.78:0/298499020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767200 0x55634d7675e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:50:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fb2d37640 1 --2- 172.21.15.78:0/2681899974 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6f980a2b80 0x7f6f980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fa37fe640 1 -- 172.21.15.78:0/2681899974 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f6fa402e030 con 0x7f6fac1078b0 2024-08-18T23:50:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.852+0000 7f6fb2536640 1 --2- 172.21.15.78:0/298499020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d767b20 0x55634d771ea0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009af0 tx=0x7f6fa80113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6068eaa40e676a8b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.853+0000 7f6fa0ff9640 1 -- 172.21.15.78:0/298499020 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8011e10 con 0x55634d767b20 2024-08-18T23:50:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.853+0000 7f6fa0ff9640 1 -- 172.21.15.78:0/298499020 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa8016070 con 0x55634d767b20 2024-08-18T23:50:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.853+0000 7f6fa0ff9640 1 -- 172.21.15.78:0/298499020 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa80126b0 con 0x55634d767b20 2024-08-18T23:50:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.853+0000 7f6fb6b98740 1 -- 172.21.15.78:0/298499020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d767b20 msgr2=0x55634d771ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:22.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.853+0000 7f6fb2d37640 1 --2- 172.21.15.78:0/2681899974 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6f980a2b80 0x7f6f980a5040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40b9700 tx=0x7f6fa403d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.853+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/298499020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d767b20 0x55634d771ea0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009af0 tx=0x7f6fa80113e0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.853+0000 7f6fb6b98740 1 -- 172.21.15.78:0/298499020 shutdown_connections 2024-08-18T23:50:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.853+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/298499020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d7723e0 0x55634d7747d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.853+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/298499020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d767b20 0x55634d771ea0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.853+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/298499020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767200 0x55634d7675e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.853+0000 7f6fb6b98740 1 -- 172.21.15.78:0/298499020 >> 172.21.15.78:0/298499020 conn(0x55634d763790 msgr2=0x55634d763b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.854+0000 7f6fb6b98740 1 -- 172.21.15.78:0/298499020 shutdown_connections 2024-08-18T23:50:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.854+0000 7f6fb6b98740 1 -- 172.21.15.78:0/298499020 wait complete. 2024-08-18T23:50:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.854+0000 7f6fb6b98740 1 -- start start 2024-08-18T23:50:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.854+0000 7f6fb6b98740 1 Processor -- start 2024-08-18T23:50:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.854+0000 7f6fb6b98740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767200 0x55634d76f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.854+0000 7f6fb6b98740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d767b20 0x55634d76f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.854+0000 7f6fb6b98740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d7723e0 0x55634d76fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.854+0000 7f6fb6b98740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55634d687cb0 con 0x55634d767200 2024-08-18T23:50:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.854+0000 7f6fb6b98740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55634d6c87b0 con 0x55634d767b20 2024-08-18T23:50:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.854+0000 7f6fb6b98740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55634d774fb0 con 0x55634d7723e0 2024-08-18T23:50:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.854+0000 7f6fb2536640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d767b20 0x55634d76f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.854+0000 7f6fb1d35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767200 0x55634d76f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.855+0000 7f6fb2536640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d767b20 0x55634d76f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:50614/0 (socket says 172.21.15.78:50614) 2024-08-18T23:50:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.855+0000 7f6fb2536640 1 -- 172.21.15.78:0/1917196982 learned_addr learned my addr 172.21.15.78:0/1917196982 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.855+0000 7f6fb6b98740 10 client.0 _ll_register_callbacks cb 0x55634d3dbba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.855+0000 7f6fb1d35640 1 --2- 172.21.15.78:0/1917196982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d7723e0 0x55634d76fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.855+0000 7f6fb2536640 1 -- 172.21.15.78:0/1917196982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d7723e0 msgr2=0x55634d76fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.855+0000 7f6fb2536640 1 --2- 172.21.15.78:0/1917196982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d7723e0 0x55634d76fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.855+0000 7f6fb2536640 1 -- 172.21.15.78:0/1917196982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767200 msgr2=0x55634d76f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.855+0000 7f6fb2536640 1 --2- 172.21.15.78:0/1917196982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767200 0x55634d76f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.855+0000 7f6fb2536640 1 -- 172.21.15.78:0/1917196982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55634d686bc0 con 0x55634d767b20 2024-08-18T23:50:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.855+0000 7f6fb2536640 1 --2- 172.21.15.78:0/1917196982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d767b20 0x55634d76f830 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8003bb0 tx=0x7f6fa8005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34cc381b43bf03e8 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.855+0000 7f6f95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:50:22.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.856+0000 7f6f95ffb640 1 -- 172.21.15.78:0/1917196982 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa802b060 con 0x55634d767b20 2024-08-18T23:50:22.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.856+0000 7f6f95ffb640 1 -- 172.21.15.78:0/1917196982 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f6fa8004050 con 0x55634d767b20 2024-08-18T23:50:22.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.856+0000 7f6fb6b98740 1 -- 172.21.15.78:0/1917196982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fa8016070 con 0x55634d767b20 2024-08-18T23:50:22.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.858+0000 7f6f95ffb640 1 -- 172.21.15.78:0/1917196982 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6fa8004250 con 0x55634d767b20 2024-08-18T23:50:22.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.867+0000 7f6fb6b98740 10 client.25820 fetch_fsmap learned FSMap version 28 2024-08-18T23:50:22.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.867+0000 7f6fb6b98740 1 -- 172.21.15.78:0/1917196982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55634d785290 con 0x55634d767b20 2024-08-18T23:50:22.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.868+0000 7f6f95ffb640 1 -- 172.21.15.78:0/1917196982 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa80043f0 con 0x55634d767b20 2024-08-18T23:50:22.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.868+0000 7f6f95ffb640 1 -- 172.21.15.78:0/1917196982 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6fa8012450 con 0x55634d767b20 2024-08-18T23:50:22.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.868+0000 7f6fb6b98740 10 client.25820 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:50:22.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.868+0000 7f6fb6b98740 10 client.25820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:50:22.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.868+0000 7f6fb6b98740 20 client.25820 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:22.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.868+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/1917196982 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55634d786750 0x55634d788b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:22.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.868+0000 7f6fb6b98740 4 client.25820 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:50:22.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.868+0000 7f6fb6b98740 1 -- 172.21.15.78:0/1917196982 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6fa8016070 con 0x55634d786750 2024-08-18T23:50:22.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.871+0000 7f6fb1d35640 1 --2- 172.21.15.78:0/1917196982 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55634d786750 0x55634d788b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:22.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.872+0000 7f6fb1d35640 1 --2- 172.21.15.78:0/1917196982 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55634d786750 0x55634d788b40 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc275cc9a7df50a5 server_cookie=fa49bc1b149603ac in_seq=0 out_seq=0 2024-08-18T23:50:22.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.873+0000 7f6f95ffb640 10 client.25820 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:22.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.882+0000 7f6f95ffb640 1 -- 172.21.15.78:0/1917196982 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f6fa8016070 con 0x55634d786750 2024-08-18T23:50:22.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.882+0000 7f6f95ffb640 10 client.25820 handle_command_reply: tid=0 2024-08-18T23:50:22.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.882+0000 7f6fb6b98740 1 client.25820 shutdown 2024-08-18T23:50:22.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.884+0000 7f6fb6b98740 1 -- 172.21.15.78:0/1917196982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d767b20 msgr2=0x55634d76f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:22.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.884+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/1917196982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d767b20 0x55634d76f830 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8003bb0 tx=0x7f6fa8005850 comp rx=0 tx=0).stop 2024-08-18T23:50:22.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.884+0000 7f6fb6b98740 1 -- 172.21.15.78:0/1917196982 shutdown_connections 2024-08-18T23:50:22.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.884+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/1917196982 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55634d786750 0x55634d788b40 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.884+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/1917196982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d7723e0 0x55634d76fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.884+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/1917196982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d767b20 0x55634d76f830 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.884+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/1917196982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767200 0x55634d76f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.884+0000 7f6fb6b98740 1 -- 172.21.15.78:0/1917196982 >> 172.21.15.78:0/1917196982 conn(0x55634d763790 msgr2=0x55634d7628f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:22.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.885+0000 7f6f95ffb640 0 client.25820 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:22.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.885+0000 7f6fb6b98740 1 -- 172.21.15.78:0/1917196982 shutdown_connections 2024-08-18T23:50:22.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.885+0000 7f6fb6b98740 1 -- 172.21.15.78:0/1917196982 wait complete. 2024-08-18T23:50:22.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.885+0000 7f6fb6b98740 20 client.25820 trim_cache size 0 max 16384 2024-08-18T23:50:22.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.916+0000 7f6fb6b98740 1 Processor -- start 2024-08-18T23:50:22.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.916+0000 7f6fb6b98740 1 -- start start 2024-08-18T23:50:22.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.916+0000 7f6fb6b98740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d767200 0x55634d76fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:22.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.916+0000 7f6fb6b98740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767b20 0x55634d771260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:22.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.916+0000 7f6fb6b98740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d7723e0 0x55634d78e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:22.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.916+0000 7f6fb6b98740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac1af0e0 con 0x55634d767b20 2024-08-18T23:50:22.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.916+0000 7f6fb6b98740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac1ac9b0 con 0x55634d767200 2024-08-18T23:50:22.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.916+0000 7f6fb6b98740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac1aa800 con 0x55634d7723e0 2024-08-18T23:50:22.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.916+0000 7f6fb2536640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767b20 0x55634d771260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:22.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.916+0000 7f6fb2536640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767b20 0x55634d771260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46040/0 (socket says 172.21.15.78:46040) 2024-08-18T23:50:22.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.916+0000 7f6fb2536640 1 -- 172.21.15.78:0/4178396868 learned_addr learned my addr 172.21.15.78:0/4178396868 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:22.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.916+0000 7f6fb1d35640 1 --2- 172.21.15.78:0/4178396868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d767200 0x55634d76fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:22.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.916+0000 7f6fb1d35640 1 --2- 172.21.15.78:0/4178396868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d7723e0 0x55634d78e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:22.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.917+0000 7f6fb2536640 1 -- 172.21.15.78:0/4178396868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d7723e0 msgr2=0x55634d78e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:22.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.917+0000 7f6fb2536640 1 --2- 172.21.15.78:0/4178396868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d7723e0 0x55634d78e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.917+0000 7f6fb2536640 1 -- 172.21.15.78:0/4178396868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d767200 msgr2=0x55634d76fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:22.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.917+0000 7f6fb2536640 1 --2- 172.21.15.78:0/4178396868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d767200 0x55634d76fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.917+0000 7f6fb2536640 1 -- 172.21.15.78:0/4178396868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55634d76e2a0 con 0x55634d767b20 2024-08-18T23:50:22.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.917+0000 7f6fb2536640 1 --2- 172.21.15.78:0/4178396868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767b20 0x55634d771260 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80129c0 tx=0x7f6fa8011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78eedbf22f9a6fe4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:22.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.917+0000 7f6f97fff640 1 -- 172.21.15.78:0/4178396868 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa802b070 con 0x55634d767b20 2024-08-18T23:50:22.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.917+0000 7f6f97fff640 1 -- 172.21.15.78:0/4178396868 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa8016040 con 0x55634d767b20 2024-08-18T23:50:22.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.917+0000 7f6f97fff640 1 -- 172.21.15.78:0/4178396868 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8035080 con 0x55634d767b20 2024-08-18T23:50:22.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.918+0000 7f6fb6b98740 1 -- 172.21.15.78:0/4178396868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767b20 msgr2=0x55634d771260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:22.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.918+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/4178396868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767b20 0x55634d771260 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80129c0 tx=0x7f6fa8011a10 comp rx=0 tx=0).stop 2024-08-18T23:50:22.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.918+0000 7f6fb6b98740 1 -- 172.21.15.78:0/4178396868 shutdown_connections 2024-08-18T23:50:22.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.918+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/4178396868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d7723e0 0x55634d78e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.918+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/4178396868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767b20 0x55634d771260 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.918+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/4178396868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d767200 0x55634d76fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.918+0000 7f6fb6b98740 1 -- 172.21.15.78:0/4178396868 >> 172.21.15.78:0/4178396868 conn(0x55634d763790 msgr2=0x55634d770b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:22.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.918+0000 7f6fb6b98740 1 -- 172.21.15.78:0/4178396868 shutdown_connections 2024-08-18T23:50:22.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.918+0000 7f6fb6b98740 1 -- 172.21.15.78:0/4178396868 wait complete. 2024-08-18T23:50:22.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.918+0000 7f6fb6b98740 1 -- start start 2024-08-18T23:50:22.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.918+0000 7f6fb6b98740 1 Processor -- start 2024-08-18T23:50:22.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6fb6b98740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767200 0x55634d78e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:22.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6fb6b98740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d767b20 0x55634d78b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:22.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6fb6b98740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d7723e0 0x55634d78be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:22.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6fb6b98740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac1aae70 con 0x55634d767200 2024-08-18T23:50:22.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6fb6b98740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac1ac2e0 con 0x55634d7723e0 2024-08-18T23:50:22.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6fb6b98740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac1ab540 con 0x55634d767b20 2024-08-18T23:50:22.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6fb6b98740 10 client.0 _ll_register_callbacks cb 0x7f6f9c00a750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:22.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6fb1d35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767200 0x55634d78e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:22.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6fb1d35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767200 0x55634d78e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46048/0 (socket says 172.21.15.78:46048) 2024-08-18T23:50:22.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6fb1d35640 1 -- 172.21.15.78:0/3466409202 learned_addr learned my addr 172.21.15.78:0/3466409202 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:22.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6fb1d35640 1 --2- 172.21.15.78:0/3466409202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d7723e0 0x55634d78be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6fb2536640 1 --2- 172.21.15.78:0/3466409202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d767b20 0x55634d78b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6fb1d35640 1 -- 172.21.15.78:0/3466409202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d767b20 msgr2=0x55634d78b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6fb1d35640 1 --2- 172.21.15.78:0/3466409202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d767b20 0x55634d78b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6fb1d35640 1 -- 172.21.15.78:0/3466409202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d7723e0 msgr2=0x55634d78be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6fb1d35640 1 --2- 172.21.15.78:0/3466409202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d7723e0 0x55634d78be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6fb1d35640 1 -- 172.21.15.78:0/3466409202 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55634d76a1b0 con 0x55634d767200 2024-08-18T23:50:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6fb1d35640 1 --2- 172.21.15.78:0/3466409202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767200 0x55634d78e5f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x55634d6c8840 tx=0x7f6f9c011040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcdb84882c499bfe server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6f95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:50:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6f95ffb640 1 -- 172.21.15.78:0/3466409202 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c026030 con 0x55634d767200 2024-08-18T23:50:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6f95ffb640 1 -- 172.21.15.78:0/3466409202 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f6f9c004020 con 0x55634d767200 2024-08-18T23:50:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.919+0000 7f6fb6b98740 1 -- 172.21.15.78:0/3466409202 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fa8016040 con 0x55634d767200 2024-08-18T23:50:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.920+0000 7f6f95ffb640 1 -- 172.21.15.78:0/3466409202 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f9c014e60 con 0x55634d767200 2024-08-18T23:50:22.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.920+0000 7f6fb6b98740 10 client.16404 fetch_fsmap learned FSMap version 28 2024-08-18T23:50:22.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.920+0000 7f6fb6b98740 1 -- 172.21.15.78:0/3466409202 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55634d7f0d70 con 0x55634d767200 2024-08-18T23:50:22.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.920+0000 7f6f95ffb640 1 -- 172.21.15.78:0/3466409202 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa8016040 con 0x55634d767200 2024-08-18T23:50:22.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.920+0000 7f6f95ffb640 1 -- 172.21.15.78:0/3466409202 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6f9c02c030 con 0x55634d767200 2024-08-18T23:50:22.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.920+0000 7f6fb6b98740 10 client.16404 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:50:22.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.920+0000 7f6fb6b98740 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:50:22.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.920+0000 7f6fb6b98740 20 client.16404 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:22.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.920+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/3466409202 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55634d7f14d0 0x55634d7f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:22.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.920+0000 7f6fb6b98740 4 client.16404 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-18T23:50:22.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.920+0000 7f6fb6b98740 1 -- 172.21.15.78:0/3466409202 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55634d6d9d60 con 0x55634d7f14d0 2024-08-18T23:50:22.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.920+0000 7f6fb2536640 1 --2- 172.21.15.78:0/3466409202 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55634d7f14d0 0x55634d7f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:22.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.921+0000 7f6fb2536640 1 --2- 172.21.15.78:0/3466409202 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55634d7f14d0 0x55634d7f38f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16dab31484db9a85 server_cookie=1b40420bc25f9913 in_seq=0 out_seq=0 2024-08-18T23:50:22.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.921+0000 7f6f95ffb640 10 client.16404 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:22.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.922+0000 7f6f95ffb640 1 -- 172.21.15.78:0/3466409202 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55634d6d9d60 con 0x55634d7f14d0 2024-08-18T23:50:22.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.922+0000 7f6f95ffb640 10 client.16404 handle_command_reply: tid=0 2024-08-18T23:50:22.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.922+0000 7f6fb6b98740 1 client.16404 shutdown 2024-08-18T23:50:22.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.922+0000 7f6fb6b98740 1 -- 172.21.15.78:0/3466409202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767200 msgr2=0x55634d78e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:22.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.922+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/3466409202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767200 0x55634d78e5f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x55634d6c8840 tx=0x7f6f9c011040 comp rx=0 tx=0).stop 2024-08-18T23:50:22.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.922+0000 7f6fb6b98740 1 -- 172.21.15.78:0/3466409202 shutdown_connections 2024-08-18T23:50:22.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.922+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/3466409202 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55634d7f14d0 0x55634d7f38f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.922+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/3466409202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55634d7723e0 0x55634d78be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.922+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/3466409202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55634d767b20 0x55634d78b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.922+0000 7f6fb6b98740 1 --2- 172.21.15.78:0/3466409202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55634d767200 0x55634d78e5f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.922+0000 7f6fb6b98740 1 -- 172.21.15.78:0/3466409202 >> 172.21.15.78:0/3466409202 conn(0x55634d763790 msgr2=0x55634d77da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:22.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.922+0000 7f6f95ffb640 0 client.16404 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:22.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.922+0000 7f6fb6b98740 1 -- 172.21.15.78:0/3466409202 shutdown_connections 2024-08-18T23:50:22.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.922+0000 7f6fb6b98740 1 -- 172.21.15.78:0/3466409202 wait complete. 2024-08-18T23:50:22.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.922+0000 7f6fb6b98740 20 client.16404 trim_cache size 0 max 16384 2024-08-18T23:50:22.929 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:50:22.929 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-08-18T23:50:22.929 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "a712077c-3d46-4141-92b6-6b95ad6674cc", 2024-08-18T23:50:22.929 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-08-18T23:50:22.929 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:50:22.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.923+0000 7f6fb479a640 1 -- 172.21.15.78:0/2681899974 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6f980a2b80 msgr2=0x7f6f980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:22.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.923+0000 7f6fb479a640 1 --2- 172.21.15.78:0/2681899974 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6f980a2b80 0x7f6f980a5040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40b9700 tx=0x7f6fa403d040 comp rx=0 tx=0).stop 2024-08-18T23:50:22.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.923+0000 7f6fb479a640 1 -- 172.21.15.78:0/2681899974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fac1078b0 msgr2=0x7f6fac19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:22.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.923+0000 7f6fb479a640 1 --2- 172.21.15.78:0/2681899974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fac1078b0 0x7f6fac19ca40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6fa402ae00 tx=0x7f6fa4031e60 comp rx=0 tx=0).stop 2024-08-18T23:50:22.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.923+0000 7f6fb479a640 1 -- 172.21.15.78:0/2681899974 shutdown_connections 2024-08-18T23:50:22.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.923+0000 7f6fb479a640 1 --2- 172.21.15.78:0/2681899974 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6f980a2b80 0x7f6f980a5040 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.923+0000 7f6fb479a640 1 --2- 172.21.15.78:0/2681899974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fac1078b0 0x7f6fac19ca40 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.923+0000 7f6fb479a640 1 --2- 172.21.15.78:0/2681899974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fac102250 0x7f6fac1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.923+0000 7f6fb479a640 1 --2- 172.21.15.78:0/2681899974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fac1018b0 0x7f6fac198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:22.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.923+0000 7f6fb479a640 1 -- 172.21.15.78:0/2681899974 >> 172.21.15.78:0/2681899974 conn(0x7f6fac0fd5f0 msgr2=0x7f6fac0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:22.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.923+0000 7f6fb479a640 1 -- 172.21.15.78:0/2681899974 shutdown_connections 2024-08-18T23:50:22.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:22.923+0000 7f6fb479a640 1 -- 172.21.15.78:0/2681899974 wait complete. 2024-08-18T23:50:23.620 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:23 smithi078 ceph-mon[18440]: pgmap v532: 129 pgs: 129 active+clean; 728 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 407 op/s 2024-08-18T23:50:23.647 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:50:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:23 smithi161 ceph-mon[22027]: pgmap v532: 129 pgs: 129 active+clean; 728 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 407 op/s 2024-08-18T23:50:23.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:23 smithi123 ceph-mon[21808]: pgmap v532: 129 pgs: 129 active+clean; 728 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 407 op/s 2024-08-18T23:50:23.898 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:50:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:24 smithi161 ceph-mon[22027]: scrub queued for path: / 2024-08-18T23:50:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:24 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:24 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:50:24.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:24 smithi078 ceph-mon[18440]: scrub queued for path: / 2024-08-18T23:50:24.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:24 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:24.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:24 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:50:24.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:24 smithi123 ceph-mon[21808]: scrub queued for path: / 2024-08-18T23:50:24.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:24 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:24.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:24 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:50:25.362 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:25 smithi078 ceph-mon[18440]: pgmap v533: 129 pgs: 129 active+clean; 728 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 800 KiB/s wr, 297 op/s 2024-08-18T23:50:25.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.360+0000 7f38969e8640 1 -- 172.21.15.78:0/121884240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3890102c20 msgr2=0x7f389010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.360+0000 7f38969e8640 1 --2- 172.21.15.78:0/121884240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3890102c20 0x7f389010d770 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3888009ca0 tx=0x7f388802a950 comp rx=0 tx=0).stop 2024-08-18T23:50:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.360+0000 7f38969e8640 1 -- 172.21.15.78:0/121884240 shutdown_connections 2024-08-18T23:50:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.360+0000 7f38969e8640 1 --2- 172.21.15.78:0/121884240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3890102c20 0x7f389010d770 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.360+0000 7f38969e8640 1 --2- 172.21.15.78:0/121884240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38901022a0 0x7f38901026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.360+0000 7f38969e8640 1 --2- 172.21.15.78:0/121884240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3890100b50 0x7f3890100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.360+0000 7f38969e8640 1 -- 172.21.15.78:0/121884240 >> 172.21.15.78:0/121884240 conn(0x7f38900fc500 msgr2=0x7f38900fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.360+0000 7f38969e8640 1 -- 172.21.15.78:0/121884240 shutdown_connections 2024-08-18T23:50:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.360+0000 7f38969e8640 1 -- 172.21.15.78:0/121884240 wait complete. 2024-08-18T23:50:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.361+0000 7f38969e8640 1 Processor -- start 2024-08-18T23:50:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.361+0000 7f38969e8640 1 -- start start 2024-08-18T23:50:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.361+0000 7f38969e8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3890100b50 0x7f3890196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.361+0000 7f38969e8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38901022a0 0x7f3890197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:25.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.361+0000 7f38969e8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3890102c20 0x7f3890197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:25.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.361+0000 7f38969e8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3890110cb0 con 0x7f3890100b50 2024-08-18T23:50:25.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.361+0000 7f38969e8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3890110b30 con 0x7f3890102c20 2024-08-18T23:50:25.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.361+0000 7f38969e8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3890110e30 con 0x7f38901022a0 2024-08-18T23:50:25.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.361+0000 7f3894f85640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3890102c20 0x7f3890197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:25.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.361+0000 7f388ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3890100b50 0x7f3890196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:25.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.361+0000 7f388f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38901022a0 0x7f3890197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:25.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.361+0000 7f388ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3890100b50 0x7f3890196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46066/0 (socket says 172.21.15.78:46066) 2024-08-18T23:50:25.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.361+0000 7f3894f85640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3890102c20 0x7f3890197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:50660/0 (socket says 172.21.15.78:50660) 2024-08-18T23:50:25.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.361+0000 7f388ffff640 1 -- 172.21.15.78:0/3988354408 learned_addr learned my addr 172.21.15.78:0/3988354408 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:25.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.362+0000 7f3894f85640 1 -- 172.21.15.78:0/3988354408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38901022a0 msgr2=0x7f3890197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:25.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.362+0000 7f3894f85640 1 --2- 172.21.15.78:0/3988354408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38901022a0 0x7f3890197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.362+0000 7f3894f85640 1 -- 172.21.15.78:0/3988354408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3890100b50 msgr2=0x7f3890196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:25.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.362+0000 7f3894f85640 1 --2- 172.21.15.78:0/3988354408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3890100b50 0x7f3890196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.362+0000 7f3894f85640 1 -- 172.21.15.78:0/3988354408 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38901a6740 con 0x7f3890102c20 2024-08-18T23:50:25.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.362+0000 7f388ffff640 1 --2- 172.21.15.78:0/3988354408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3890100b50 0x7f3890196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:50:25.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.362+0000 7f3894f85640 1 --2- 172.21.15.78:0/3988354408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3890102c20 0x7f3890197950 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3888005cc0 tx=0x7f388802f420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:25.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.362+0000 7f388d7fa640 1 -- 172.21.15.78:0/3988354408 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888034040 con 0x7f3890102c20 2024-08-18T23:50:25.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.362+0000 7f388d7fa640 1 -- 172.21.15.78:0/3988354408 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3888030920 con 0x7f3890102c20 2024-08-18T23:50:25.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.362+0000 7f38969e8640 1 -- 172.21.15.78:0/3988354408 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38901a69d0 con 0x7f3890102c20 2024-08-18T23:50:25.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.362+0000 7f388d7fa640 1 -- 172.21.15.78:0/3988354408 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888004600 con 0x7f3890102c20 2024-08-18T23:50:25.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.362+0000 7f38969e8640 1 -- 172.21.15.78:0/3988354408 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38901a6eb0 con 0x7f3890102c20 2024-08-18T23:50:25.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.364+0000 7f388d7fa640 1 -- 172.21.15.78:0/3988354408 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3888037020 con 0x7f3890102c20 2024-08-18T23:50:25.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.365+0000 7f3898de6740 1 Processor -- start 2024-08-18T23:50:25.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.365+0000 7f3898de6740 1 -- start start 2024-08-18T23:50:25.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.365+0000 7f388d7fa640 1 --2- 172.21.15.78:0/3988354408 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f38780a2b80 0x7f38780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:25.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.365+0000 7f3898de6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700fec210 0x555700fec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:25.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f3898de6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fecb30 0x555700ff6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:25.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f3898de6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700ff73f0 0x555700ff97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:25.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f3898de6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555700f51980 con 0x555700ff73f0 2024-08-18T23:50:25.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f3898de6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555700ee9a70 con 0x555700fecb30 2024-08-18T23:50:25.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f3898de6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555700f4d940 con 0x555700fec210 2024-08-18T23:50:25.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f388f7fe640 1 --2- 172.21.15.78:0/3988354408 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f38780a2b80 0x7f38780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:25.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f388d7fa640 1 -- 172.21.15.78:0/3988354408 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f38880f0170 con 0x7f3890102c20 2024-08-18T23:50:25.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f388ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700fec210 0x555700fec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:25.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f388f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fecb30 0x555700ff6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:25.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f3894f85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700ff73f0 0x555700ff97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:25.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f388ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700fec210 0x555700fec5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:57092/0 (socket says 172.21.15.78:57092) 2024-08-18T23:50:25.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f388ffff640 1 -- 172.21.15.78:0/2417151893 learned_addr learned my addr 172.21.15.78:0/2417151893 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:25.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f388ffff640 1 -- 172.21.15.78:0/2417151893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fecb30 msgr2=0x555700ff6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:25.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f388ffff640 1 --2- 172.21.15.78:0/2417151893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fecb30 0x555700ff6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f388ffff640 1 -- 172.21.15.78:0/2417151893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700ff73f0 msgr2=0x555700ff97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:25.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f388ffff640 1 --2- 172.21.15.78:0/2417151893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700ff73f0 0x555700ff97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f388ffff640 1 -- 172.21.15.78:0/2417151893 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555700f4c6e0 con 0x555700fec210 2024-08-18T23:50:25.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f388f7fe640 1 --2- 172.21.15.78:0/3988354408 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f38780a2b80 0x7f38780a5040 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f389006a520 tx=0x7f3884009ae0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:25.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f388ffff640 1 --2- 172.21.15.78:0/2417151893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700fec210 0x555700fec5f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f388000f470 tx=0x7f388000f930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b2af151f3e76748 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:25.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f38767fc640 1 -- 172.21.15.78:0/2417151893 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388001d070 con 0x555700fec210 2024-08-18T23:50:25.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f38767fc640 1 -- 172.21.15.78:0/2417151893 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3880010e40 con 0x555700fec210 2024-08-18T23:50:25.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.366+0000 7f38767fc640 1 -- 172.21.15.78:0/2417151893 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880016340 con 0x555700fec210 2024-08-18T23:50:25.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.367+0000 7f3898de6740 1 -- 172.21.15.78:0/2417151893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700fec210 msgr2=0x555700fec5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:25.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.367+0000 7f3898de6740 1 --2- 172.21.15.78:0/2417151893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700fec210 0x555700fec5f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f388000f470 tx=0x7f388000f930 comp rx=0 tx=0).stop 2024-08-18T23:50:25.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.367+0000 7f3898de6740 1 -- 172.21.15.78:0/2417151893 shutdown_connections 2024-08-18T23:50:25.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.367+0000 7f3898de6740 1 --2- 172.21.15.78:0/2417151893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700ff73f0 0x555700ff97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.367+0000 7f3898de6740 1 --2- 172.21.15.78:0/2417151893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fecb30 0x555700ff6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.367+0000 7f3898de6740 1 --2- 172.21.15.78:0/2417151893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700fec210 0x555700fec5f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.367+0000 7f3898de6740 1 -- 172.21.15.78:0/2417151893 >> 172.21.15.78:0/2417151893 conn(0x555700fe87a0 msgr2=0x555700fe8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:25.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.367+0000 7f3898de6740 1 -- 172.21.15.78:0/2417151893 shutdown_connections 2024-08-18T23:50:25.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.367+0000 7f3898de6740 1 -- 172.21.15.78:0/2417151893 wait complete. 2024-08-18T23:50:25.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.367+0000 7f3898de6740 1 -- start start 2024-08-18T23:50:25.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.368+0000 7f3898de6740 1 Processor -- start 2024-08-18T23:50:25.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.368+0000 7f3898de6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fec210 0x555700ff42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:25.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.368+0000 7f3898de6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fecb30 0x555700ff4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.368+0000 7f3898de6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700ff73f0 0x555700ff4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.368+0000 7f3898de6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555700ff9fc0 con 0x555700fec210 2024-08-18T23:50:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.368+0000 7f3898de6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555700f0caa0 con 0x555700fecb30 2024-08-18T23:50:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.368+0000 7f3898de6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555700ffa140 con 0x555700ff73f0 2024-08-18T23:50:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.368+0000 7f3898de6740 10 client.0 _ll_register_callbacks cb 0x555700efbe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.368+0000 7f388ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fec210 0x555700ff42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.368+0000 7f388ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fec210 0x555700ff42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46084/0 (socket says 172.21.15.78:46084) 2024-08-18T23:50:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.368+0000 7f388ffff640 1 -- 172.21.15.78:0/3420814626 learned_addr learned my addr 172.21.15.78:0/3420814626 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.368+0000 7f388f7fe640 1 --2- 172.21.15.78:0/3420814626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fecb30 0x555700ff4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.368+0000 7f3894f85640 1 --2- 172.21.15.78:0/3420814626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700ff73f0 0x555700ff4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.368+0000 7f388ffff640 1 -- 172.21.15.78:0/3420814626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700ff73f0 msgr2=0x555700ff4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.368+0000 7f388ffff640 1 --2- 172.21.15.78:0/3420814626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700ff73f0 0x555700ff4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.368+0000 7f388ffff640 1 -- 172.21.15.78:0/3420814626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fecb30 msgr2=0x555700ff4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:25.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.368+0000 7f388ffff640 1 --2- 172.21.15.78:0/3420814626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fecb30 0x555700ff4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.368+0000 7f388ffff640 1 -- 172.21.15.78:0/3420814626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555700ef0650 con 0x555700fec210 2024-08-18T23:50:25.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.369+0000 7f388ffff640 1 --2- 172.21.15.78:0/3420814626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fec210 0x555700ff42d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3880006010 tx=0x7f38800186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d072eb678fb168b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:25.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.369+0000 7f386f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:50:25.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.369+0000 7f386f7fe640 1 -- 172.21.15.78:0/3420814626 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388001d050 con 0x555700fec210 2024-08-18T23:50:25.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.369+0000 7f386f7fe640 1 -- 172.21.15.78:0/3420814626 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f3880010970 con 0x555700fec210 2024-08-18T23:50:25.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.369+0000 7f3898de6740 1 -- 172.21.15.78:0/3420814626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555700f5eef0 con 0x555700fec210 2024-08-18T23:50:25.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.369+0000 7f386f7fe640 1 -- 172.21.15.78:0/3420814626 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3880031050 con 0x555700fec210 2024-08-18T23:50:25.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.369+0000 7f3898de6740 10 client.16416 fetch_fsmap learned FSMap version 28 2024-08-18T23:50:25.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.369+0000 7f3898de6740 1 -- 172.21.15.78:0/3420814626 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555700cc33c0 con 0x555700fec210 2024-08-18T23:50:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.369+0000 7f386f7fe640 1 -- 172.21.15.78:0/3420814626 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555700f5eef0 con 0x555700fec210 2024-08-18T23:50:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.369+0000 7f386f7fe640 1 -- 172.21.15.78:0/3420814626 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3880004860 con 0x555700fec210 2024-08-18T23:50:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.369+0000 7f3898de6740 10 client.16416 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:50:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.369+0000 7f3898de6740 10 client.16416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:50:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.369+0000 7f3898de6740 20 client.16416 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.369+0000 7f3898de6740 1 --2- 172.21.15.78:0/3420814626 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x555701006e50 0x555701009240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.369+0000 7f3898de6740 4 client.16416 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:50:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.369+0000 7f3898de6740 1 -- 172.21.15.78:0/3420814626 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555700f59400 con 0x555701006e50 2024-08-18T23:50:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.370+0000 7f388f7fe640 1 --2- 172.21.15.78:0/3420814626 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x555701006e50 0x555701009240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.370+0000 7f386f7fe640 10 client.16416 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.370+0000 7f388f7fe640 1 --2- 172.21.15.78:0/3420814626 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x555701006e50 0x555701009240 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=422c20efb5a0b3b server_cookie=4ac1b2983a62bb37 in_seq=0 out_seq=0 2024-08-18T23:50:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.373+0000 7f386f7fe640 1 -- 172.21.15.78:0/3420814626 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555700f59400 con 0x555701006e50 2024-08-18T23:50:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.373+0000 7f386f7fe640 10 client.16416 handle_command_reply: tid=0 2024-08-18T23:50:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.373+0000 7f3898de6740 1 client.16416 shutdown 2024-08-18T23:50:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.373+0000 7f3898de6740 1 -- 172.21.15.78:0/3420814626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fec210 msgr2=0x555700ff42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.373+0000 7f3898de6740 1 --2- 172.21.15.78:0/3420814626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fec210 0x555700ff42d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3880006010 tx=0x7f38800186f0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.373+0000 7f3898de6740 1 -- 172.21.15.78:0/3420814626 shutdown_connections 2024-08-18T23:50:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.373+0000 7f3898de6740 1 --2- 172.21.15.78:0/3420814626 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x555701006e50 0x555701009240 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.373+0000 7f3898de6740 1 --2- 172.21.15.78:0/3420814626 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700ff73f0 0x555700ff4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.373+0000 7f3898de6740 1 --2- 172.21.15.78:0/3420814626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700fecb30 0x555700ff4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.373+0000 7f386f7fe640 0 client.16416 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.373+0000 7f3898de6740 1 --2- 172.21.15.78:0/3420814626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fec210 0x555700ff42d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.373+0000 7f3898de6740 1 -- 172.21.15.78:0/3420814626 >> 172.21.15.78:0/3420814626 conn(0x555700fe87a0 msgr2=0x555700fee480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.373+0000 7f3898de6740 1 -- 172.21.15.78:0/3420814626 shutdown_connections 2024-08-18T23:50:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.373+0000 7f3898de6740 1 -- 172.21.15.78:0/3420814626 wait complete. 2024-08-18T23:50:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.373+0000 7f3898de6740 20 client.16416 trim_cache size 0 max 16384 2024-08-18T23:50:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.392+0000 7f3898de6740 1 Processor -- start 2024-08-18T23:50:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.392+0000 7f3898de6740 1 -- start start 2024-08-18T23:50:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.392+0000 7f3898de6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fec210 0x555700ff32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.392+0000 7f3898de6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700fecb30 0x55570100e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.392+0000 7f3898de6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700ff73f0 0x55570100ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.392+0000 7f3898de6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38901b34c0 con 0x555700fec210 2024-08-18T23:50:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.392+0000 7f3898de6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f38901b0d90 con 0x555700ff73f0 2024-08-18T23:50:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.392+0000 7f3898de6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f38901aebe0 con 0x555700fecb30 2024-08-18T23:50:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.392+0000 7f388ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fec210 0x555700ff32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.392+0000 7f388ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fec210 0x555700ff32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46090/0 (socket says 172.21.15.78:46090) 2024-08-18T23:50:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.392+0000 7f388ffff640 1 -- 172.21.15.78:0/674593886 learned_addr learned my addr 172.21.15.78:0/674593886 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.392+0000 7f3894f85640 1 --2- 172.21.15.78:0/674593886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700ff73f0 0x55570100ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:25.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.392+0000 7f388f7fe640 1 --2- 172.21.15.78:0/674593886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700fecb30 0x55570100e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:25.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.393+0000 7f3894f85640 1 -- 172.21.15.78:0/674593886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700fecb30 msgr2=0x55570100e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:25.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.393+0000 7f3894f85640 1 --2- 172.21.15.78:0/674593886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700fecb30 0x55570100e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.393+0000 7f3894f85640 1 -- 172.21.15.78:0/674593886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fec210 msgr2=0x555700ff32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:25.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.393+0000 7f3894f85640 1 --2- 172.21.15.78:0/674593886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fec210 0x555700ff32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.393+0000 7f3894f85640 1 -- 172.21.15.78:0/674593886 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555700fe9430 con 0x555700ff73f0 2024-08-18T23:50:25.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.393+0000 7f388ffff640 1 --2- 172.21.15.78:0/674593886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fec210 0x555700ff32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:50:25.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.393+0000 7f3894f85640 1 --2- 172.21.15.78:0/674593886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700ff73f0 0x55570100ee70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f38880bc2e0 tx=0x7f38880040d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf14ef8672a1e192 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:25.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.393+0000 7f3875ffb640 1 -- 172.21.15.78:0/674593886 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38880bc5e0 con 0x555700ff73f0 2024-08-18T23:50:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.393+0000 7f3875ffb640 1 -- 172.21.15.78:0/674593886 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38880c0990 con 0x555700ff73f0 2024-08-18T23:50:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.393+0000 7f3875ffb640 1 -- 172.21.15.78:0/674593886 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38880a8480 con 0x555700ff73f0 2024-08-18T23:50:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.394+0000 7f3898de6740 1 -- 172.21.15.78:0/674593886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700ff73f0 msgr2=0x55570100ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.394+0000 7f3898de6740 1 --2- 172.21.15.78:0/674593886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700ff73f0 0x55570100ee70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f38880bc2e0 tx=0x7f38880040d0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.394+0000 7f3898de6740 1 -- 172.21.15.78:0/674593886 shutdown_connections 2024-08-18T23:50:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.394+0000 7f3898de6740 1 --2- 172.21.15.78:0/674593886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700ff73f0 0x55570100ee70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.394+0000 7f3898de6740 1 --2- 172.21.15.78:0/674593886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700fecb30 0x55570100e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.394+0000 7f3898de6740 1 --2- 172.21.15.78:0/674593886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fec210 0x555700ff32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.394+0000 7f3898de6740 1 -- 172.21.15.78:0/674593886 >> 172.21.15.78:0/674593886 conn(0x555700fe87a0 msgr2=0x555700ff2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.394+0000 7f3898de6740 1 -- 172.21.15.78:0/674593886 shutdown_connections 2024-08-18T23:50:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.394+0000 7f3898de6740 1 -- 172.21.15.78:0/674593886 wait complete. 2024-08-18T23:50:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.394+0000 7f3898de6740 1 -- start start 2024-08-18T23:50:25.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.395+0000 7f3898de6740 1 Processor -- start 2024-08-18T23:50:25.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.395+0000 7f3898de6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700fec210 0x555700ffa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:25.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.395+0000 7f3898de6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fecb30 0x555700ffa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:25.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.395+0000 7f3898de6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700ff73f0 0x55570106f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:25.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.395+0000 7f3898de6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38901af250 con 0x555700fecb30 2024-08-18T23:50:25.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.395+0000 7f3898de6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f38901b06c0 con 0x555700ff73f0 2024-08-18T23:50:25.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.395+0000 7f3898de6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f38901af920 con 0x555700fec210 2024-08-18T23:50:25.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.395+0000 7f3898de6740 10 client.0 _ll_register_callbacks cb 0x7f388001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:25.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.395+0000 7f388f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fecb30 0x555700ffa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:25.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.395+0000 7f388f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fecb30 0x555700ffa9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46102/0 (socket says 172.21.15.78:46102) 2024-08-18T23:50:25.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.395+0000 7f388f7fe640 1 -- 172.21.15.78:0/3873637250 learned_addr learned my addr 172.21.15.78:0/3873637250 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:25.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.395+0000 7f388ffff640 1 --2- 172.21.15.78:0/3873637250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700fec210 0x555700ffa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:25.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.396+0000 7f3894f85640 1 --2- 172.21.15.78:0/3873637250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700ff73f0 0x55570106f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:25.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.396+0000 7f388f7fe640 1 -- 172.21.15.78:0/3873637250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700fec210 msgr2=0x555700ffa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:25.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.396+0000 7f388f7fe640 1 --2- 172.21.15.78:0/3873637250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700fec210 0x555700ffa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.396+0000 7f388f7fe640 1 -- 172.21.15.78:0/3873637250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700ff73f0 msgr2=0x55570106f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:25.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.396+0000 7f388f7fe640 1 --2- 172.21.15.78:0/3873637250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700ff73f0 0x55570106f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.396+0000 7f388f7fe640 1 -- 172.21.15.78:0/3873637250 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555700ffb2a0 con 0x555700fecb30 2024-08-18T23:50:25.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.396+0000 7f3894f85640 1 --2- 172.21.15.78:0/3873637250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700ff73f0 0x55570106f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:50:25.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.396+0000 7f388f7fe640 1 --2- 172.21.15.78:0/3873637250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fecb30 0x555700ffa9b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x555700fe9600 tx=0x7f3884005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fb57be3a9151e2 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:25.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.396+0000 7f386f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:50:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.396+0000 7f386f7fe640 1 -- 172.21.15.78:0/3873637250 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884027040 con 0x555700fecb30 2024-08-18T23:50:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.396+0000 7f386f7fe640 1 -- 172.21.15.78:0/3873637250 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f388401a070 con 0x555700fecb30 2024-08-18T23:50:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.396+0000 7f3898de6740 1 -- 172.21.15.78:0/3873637250 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38880c0990 con 0x555700fecb30 2024-08-18T23:50:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.396+0000 7f386f7fe640 1 -- 172.21.15.78:0/3873637250 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3884040050 con 0x555700fecb30 2024-08-18T23:50:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.396+0000 7f3898de6740 10 client.16422 fetch_fsmap learned FSMap version 28 2024-08-18T23:50:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.396+0000 7f3898de6740 1 -- 172.21.15.78:0/3873637250 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555701070fa0 con 0x555700fecb30 2024-08-18T23:50:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.396+0000 7f386f7fe640 1 -- 172.21.15.78:0/3873637250 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38880c0990 con 0x555700fecb30 2024-08-18T23:50:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.397+0000 7f386f7fe640 1 -- 172.21.15.78:0/3873637250 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f388401e490 con 0x555700fecb30 2024-08-18T23:50:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.397+0000 7f3898de6740 10 client.16422 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:50:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.397+0000 7f3898de6740 10 client.16422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:50:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.397+0000 7f3898de6740 20 client.16422 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:25.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.397+0000 7f3898de6740 1 --2- 172.21.15.78:0/3873637250 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5557010717b0 0x555701073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:25.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.397+0000 7f3898de6740 4 client.16422 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:50:25.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.397+0000 7f3898de6740 1 -- 172.21.15.78:0/3873637250 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555700ed6210 con 0x5557010717b0 2024-08-18T23:50:25.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.397+0000 7f388ffff640 1 --2- 172.21.15.78:0/3873637250 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5557010717b0 0x555701073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:25.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.397+0000 7f386f7fe640 10 client.16422 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:25.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.397+0000 7f388ffff640 1 --2- 172.21.15.78:0/3873637250 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5557010717b0 0x555701073bd0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74079cfcef8a9765 server_cookie=ac56b864316f36df in_seq=0 out_seq=0 2024-08-18T23:50:25.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.399+0000 7f386f7fe640 1 -- 172.21.15.78:0/3873637250 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x555700ed6210 con 0x5557010717b0 2024-08-18T23:50:25.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.399+0000 7f386f7fe640 10 client.16422 handle_command_reply: tid=0 2024-08-18T23:50:25.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.399+0000 7f3898de6740 1 client.16422 shutdown 2024-08-18T23:50:25.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.399+0000 7f3898de6740 1 -- 172.21.15.78:0/3873637250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fecb30 msgr2=0x555700ffa9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:25.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.399+0000 7f3898de6740 1 --2- 172.21.15.78:0/3873637250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fecb30 0x555700ffa9b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x555700fe9600 tx=0x7f3884005f90 comp rx=0 tx=0).stop 2024-08-18T23:50:25.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.399+0000 7f3898de6740 1 -- 172.21.15.78:0/3873637250 shutdown_connections 2024-08-18T23:50:25.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.399+0000 7f3898de6740 1 --2- 172.21.15.78:0/3873637250 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5557010717b0 0x555701073bd0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.399+0000 7f3898de6740 1 --2- 172.21.15.78:0/3873637250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555700ff73f0 0x55570106f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.399+0000 7f3898de6740 1 --2- 172.21.15.78:0/3873637250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555700fecb30 0x555700ffa9b0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.399+0000 7f386f7fe640 0 client.16422 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:25.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.399+0000 7f3898de6740 1 --2- 172.21.15.78:0/3873637250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555700fec210 0x555700ffa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.399+0000 7f3898de6740 1 -- 172.21.15.78:0/3873637250 >> 172.21.15.78:0/3873637250 conn(0x555700fe87a0 msgr2=0x555700ff33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:25.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.399+0000 7f3898de6740 1 -- 172.21.15.78:0/3873637250 shutdown_connections 2024-08-18T23:50:25.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.399+0000 7f3898de6740 1 -- 172.21.15.78:0/3873637250 wait complete. 2024-08-18T23:50:25.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.399+0000 7f3898de6740 20 client.16422 trim_cache size 0 max 16384 2024-08-18T23:50:25.417 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:50:25.417 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "scrub active (168 inodes in the stack)", 2024-08-18T23:50:25.417 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-08-18T23:50:25.417 INFO:teuthology.orchestra.run.smithi078.stdout: "a712077c-3d46-4141-92b6-6b95ad6674cc": { 2024-08-18T23:50:25.417 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-08-18T23:50:25.417 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "a712077c-3d46-4141-92b6-6b95ad6674cc", 2024-08-18T23:50:25.417 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-08-18T23:50:25.417 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:50:25.417 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:50:25.418 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:50:25.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.400+0000 7f38969e8640 1 -- 172.21.15.78:0/3988354408 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f38780a2b80 msgr2=0x7f38780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:25.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.400+0000 7f38969e8640 1 --2- 172.21.15.78:0/3988354408 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f38780a2b80 0x7f38780a5040 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f389006a520 tx=0x7f3884009ae0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.400+0000 7f38969e8640 1 -- 172.21.15.78:0/3988354408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3890102c20 msgr2=0x7f3890197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:25.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.400+0000 7f38969e8640 1 --2- 172.21.15.78:0/3988354408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3890102c20 0x7f3890197950 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3888005cc0 tx=0x7f388802f420 comp rx=0 tx=0).stop 2024-08-18T23:50:25.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.400+0000 7f38969e8640 1 -- 172.21.15.78:0/3988354408 shutdown_connections 2024-08-18T23:50:25.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.400+0000 7f38969e8640 1 --2- 172.21.15.78:0/3988354408 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f38780a2b80 0x7f38780a5040 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.400+0000 7f38969e8640 1 --2- 172.21.15.78:0/3988354408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3890102c20 0x7f3890197950 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.400+0000 7f38969e8640 1 --2- 172.21.15.78:0/3988354408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38901022a0 0x7f3890197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.400+0000 7f38969e8640 1 --2- 172.21.15.78:0/3988354408 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3890100b50 0x7f3890196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:25.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.400+0000 7f38969e8640 1 -- 172.21.15.78:0/3988354408 >> 172.21.15.78:0/3988354408 conn(0x7f38900fc500 msgr2=0x7f3890106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:25.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.400+0000 7f38969e8640 1 -- 172.21.15.78:0/3988354408 shutdown_connections 2024-08-18T23:50:25.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:25.400+0000 7f38969e8640 1 -- 172.21.15.78:0/3988354408 wait complete. 2024-08-18T23:50:25.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:25 smithi161 ceph-mon[22027]: pgmap v533: 129 pgs: 129 active+clean; 728 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 800 KiB/s wr, 297 op/s 2024-08-18T23:50:25.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:25 smithi123 ceph-mon[21808]: pgmap v533: 129 pgs: 129 active+clean; 728 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 800 KiB/s wr, 297 op/s 2024-08-18T23:50:26.111 INFO:tasks.cephfs.filesystem:scrub status for tag:a712077c-3d46-4141-92b6-6b95ad6674cc - {'path': '/', 'tag': 'a712077c-3d46-4141-92b6-6b95ad6674cc', 'options': 'recursive,force'} 2024-08-18T23:50:26.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:26 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:50:26.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:26 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:26 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:50:26.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:26 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:26 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:50:26.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:26 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:26 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:50:26.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:26 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:50:26.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:26 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:26 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:26 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:50:26.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:26 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:26 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:50:26.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:26 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:26 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:50:26.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:26 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:26 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:50:26.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:26 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:26 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:50:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:26 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:26 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:50:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:26 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:26 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:50:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:26 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:26 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:50:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:26 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:50:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:26 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:26 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:26 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:50:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:26 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:26 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:50:26.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:26 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:26 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:50:26.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:26 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:26 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:50:26.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:26 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:26 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:50:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:26 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:26 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:50:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:26 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:26 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:50:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:26 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:26 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:50:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:26 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:50:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:26 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:26 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:26 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:50:26.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:26 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:26 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:50:26.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:26 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:26 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:50:26.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:26 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:26 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:50:26.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:26 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:26.985 INFO:tasks.workunit.client.0.smithi078.stdout: 154 2465 0 1954 0 509 0 2024-08-18T23:50:27.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:27 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:50:27.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:27 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:27.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:27 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:50:27.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:27 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:27.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:27 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:50:27.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:27 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:27.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:27 smithi161 ceph-mon[22027]: pgmap v534: 129 pgs: 129 active+clean; 751 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 414 op/s 2024-08-18T23:50:27.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:27 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:50:27.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:27 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:27.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:27 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:50:27.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:27 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:27.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:27 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:50:27.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:27 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:27.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:27 smithi078 ceph-mon[18440]: pgmap v534: 129 pgs: 129 active+clean; 751 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 414 op/s 2024-08-18T23:50:27.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:27 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:50:27.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:27 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:27.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:27 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:50:27.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:27 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:27.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:27 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:50:27.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:27 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:27.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:27 smithi123 ceph-mon[21808]: pgmap v534: 129 pgs: 129 active+clean; 751 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 414 op/s 2024-08-18T23:50:29.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:29 smithi161 ceph-mon[22027]: pgmap v535: 129 pgs: 129 active+clean; 765 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 468 op/s 2024-08-18T23:50:29.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:29 smithi078 ceph-mon[18440]: pgmap v535: 129 pgs: 129 active+clean; 765 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 468 op/s 2024-08-18T23:50:29.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:29 smithi123 ceph-mon[21808]: pgmap v535: 129 pgs: 129 active+clean; 765 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 468 op/s 2024-08-18T23:50:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:31 smithi161 ceph-mon[22027]: pgmap v536: 129 pgs: 129 active+clean; 769 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 540 op/s 2024-08-18T23:50:31.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:31 smithi078 ceph-mon[18440]: pgmap v536: 129 pgs: 129 active+clean; 769 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 540 op/s 2024-08-18T23:50:31.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:31 smithi123 ceph-mon[21808]: pgmap v536: 129 pgs: 129 active+clean; 769 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 540 op/s 2024-08-18T23:50:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:33 smithi161 ceph-mon[22027]: pgmap v537: 129 pgs: 129 active+clean; 771 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 693 op/s 2024-08-18T23:50:33.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:33 smithi078 ceph-mon[18440]: pgmap v537: 129 pgs: 129 active+clean; 771 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 693 op/s 2024-08-18T23:50:33.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:33 smithi123 ceph-mon[21808]: pgmap v537: 129 pgs: 129 active+clean; 771 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 693 op/s 2024-08-18T23:50:35.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:35 smithi161 ceph-mon[22027]: pgmap v538: 129 pgs: 129 active+clean; 772 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 727 op/s 2024-08-18T23:50:35.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:35 smithi078 ceph-mon[18440]: pgmap v538: 129 pgs: 129 active+clean; 772 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 727 op/s 2024-08-18T23:50:35.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:35 smithi123 ceph-mon[21808]: pgmap v538: 129 pgs: 129 active+clean; 772 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 727 op/s 2024-08-18T23:50:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:37 smithi161 ceph-mon[22027]: pgmap v539: 129 pgs: 129 active+clean; 774 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 801 op/s 2024-08-18T23:50:37.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:37 smithi078 ceph-mon[18440]: pgmap v539: 129 pgs: 129 active+clean; 774 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 801 op/s 2024-08-18T23:50:37.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:37 smithi123 ceph-mon[21808]: pgmap v539: 129 pgs: 129 active+clean; 774 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 801 op/s 2024-08-18T23:50:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:39 smithi161 ceph-mon[22027]: pgmap v540: 129 pgs: 129 active+clean; 771 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 657 op/s 2024-08-18T23:50:39.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:39 smithi078 ceph-mon[18440]: pgmap v540: 129 pgs: 129 active+clean; 771 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 657 op/s 2024-08-18T23:50:39.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:39 smithi123 ceph-mon[21808]: pgmap v540: 129 pgs: 129 active+clean; 771 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 657 op/s 2024-08-18T23:50:41.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.235+0000 7fd2daffd640 1 -- 172.21.15.78:0/2130302402 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fd2e40043a0 con 0x55e8e341f6c0 2024-08-18T23:50:41.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.235+0000 7fd2daffd640 10 client.16332 handle_command_reply: tid=0 2024-08-18T23:50:41.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.235+0000 7fd303cfd740 1 client.16332 shutdown 2024-08-18T23:50:41.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.236+0000 7fd303cfd740 1 -- 172.21.15.78:0/2130302402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e33a4310 msgr2=0x55e8e33a8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:41.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.236+0000 7fd303cfd740 1 --2- 172.21.15.78:0/2130302402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e33a4310 0x55e8e33a8d90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40b91e0 tx=0x7fd2f40bc610 comp rx=0 tx=0).stop 2024-08-18T23:50:41.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.236+0000 7fd303cfd740 1 -- 172.21.15.78:0/2130302402 shutdown_connections 2024-08-18T23:50:41.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.236+0000 7fd303cfd740 1 --2- 172.21.15.78:0/2130302402 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55e8e341f6c0 0x55e8e3421b20 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:41.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.236+0000 7fd303cfd740 1 --2- 172.21.15.78:0/2130302402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8e33a4310 0x55e8e33a8d90 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:41.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.236+0000 7fd303cfd740 1 --2- 172.21.15.78:0/2130302402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8e3399a50 0x55e8e33a8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:41.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.236+0000 7fd303cfd740 1 --2- 172.21.15.78:0/2130302402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e8e3399130 0x55e8e33a8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:41.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.236+0000 7fd303cfd740 1 -- 172.21.15.78:0/2130302402 >> 172.21.15.78:0/2130302402 conn(0x55e8e33956c0 msgr2=0x55e8e33946f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:41.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.236+0000 7fd2daffd640 0 client.16332 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:41.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.236+0000 7fd303cfd740 1 -- 172.21.15.78:0/2130302402 shutdown_connections 2024-08-18T23:50:41.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.236+0000 7fd303cfd740 1 -- 172.21.15.78:0/2130302402 wait complete. 2024-08-18T23:50:41.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.236+0000 7fd303cfd740 20 client.16332 trim_cache size 0 max 16384 2024-08-18T23:50:41.243 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:50:41.243 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 28, 2024-08-18T23:50:41.244 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24452, 2024-08-18T23:50:41.244 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 18, 2024-08-18T23:50:41.244 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-08-18T23:50:41.244 INFO:teuthology.orchestra.run.smithi078.stdout: "48fb2815": { 2024-08-18T23:50:41.244 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 18, 2024-08-18T23:50:41.244 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-08-18T23:50:41.245 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:50:41.245 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-08-18T23:50:41.245 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:50:41.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:50:41.246 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-08-18T23:50:41.247 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 139.2, 2024-08-18T23:50:41.247 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-08-18T23:50:41.247 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-08-18T23:50:41.247 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-08-18T23:50:41.247 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:50:41.248 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-08-18T23:50:41.248 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:50:41.248 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:50:41.248 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:50:41.248 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:50:41.248 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:50:41.249 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:50:41.249 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:50:41.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.237+0000 7fd3018ff640 1 -- 172.21.15.78:0/3297061947 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2e80a2c40 msgr2=0x7fd2e80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:41.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.237+0000 7fd3018ff640 1 --2- 172.21.15.78:0/3297061947 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2e80a2c40 0x7fd2e80a5100 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00ab90 tx=0x7fd2ec009250 comp rx=0 tx=0).stop 2024-08-18T23:50:41.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.237+0000 7fd3018ff640 1 -- 172.21.15.78:0/3297061947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2fc11ae70 msgr2=0x7fd2fc111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.237+0000 7fd3018ff640 1 --2- 172.21.15.78:0/3297061947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2fc11ae70 0x7fd2fc111610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd2f402a960 tx=0x7fd2f40317e0 comp rx=0 tx=0).stop 2024-08-18T23:50:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.237+0000 7fd3018ff640 1 -- 172.21.15.78:0/3297061947 shutdown_connections 2024-08-18T23:50:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.237+0000 7fd3018ff640 1 --2- 172.21.15.78:0/3297061947 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2e80a2c40 0x7fd2e80a5100 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.237+0000 7fd3018ff640 1 --2- 172.21.15.78:0/3297061947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2fc11ae70 0x7fd2fc111610 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:41.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.237+0000 7fd3018ff640 1 --2- 172.21.15.78:0/3297061947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2fc109560 0x7fd2fc1188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:41.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.237+0000 7fd3018ff640 1 --2- 172.21.15.78:0/3297061947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2fc109180 0x7fd2fc1183a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:41.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.237+0000 7fd3018ff640 1 -- 172.21.15.78:0/3297061947 >> 172.21.15.78:0/3297061947 conn(0x7fd2fc06de70 msgr2=0x7fd2fc11bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:41.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.237+0000 7fd3018ff640 1 -- 172.21.15.78:0/3297061947 shutdown_connections 2024-08-18T23:50:41.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:41.237+0000 7fd3018ff640 1 -- 172.21.15.78:0/3297061947 wait complete. 2024-08-18T23:50:41.576 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:41 smithi078 ceph-mon[18440]: pgmap v541: 129 pgs: 129 active+clean; 765 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 601 op/s 2024-08-18T23:50:41.683 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:41 smithi161 ceph-mon[22027]: pgmap v541: 129 pgs: 129 active+clean; 765 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 601 op/s 2024-08-18T23:50:41.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:41 smithi123 ceph-mon[21808]: pgmap v541: 129 pgs: 129 active+clean; 765 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 601 op/s 2024-08-18T23:50:42.022 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '48fb2815', quiesce took 35.0 seconds. Will release after: -15.8 2024-08-18T23:50:42.023 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:50:42.294 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:50:42.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.835+0000 7f2f0a142640 1 -- 172.21.15.78:0/2219284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f04102a50 msgr2=0x7f2f04102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:42.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.835+0000 7f2f0a142640 1 --2- 172.21.15.78:0/2219284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f04102a50 0x7f2f04102e30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4009900 tx=0x7f2ef402a480 comp rx=0 tx=0).stop 2024-08-18T23:50:42.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.835+0000 7f2f0a142640 1 -- 172.21.15.78:0/2219284 shutdown_connections 2024-08-18T23:50:42.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.835+0000 7f2f0a142640 1 --2- 172.21.15.78:0/2219284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0410be80 0x7f2f0410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:42.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.835+0000 7f2f0a142640 1 --2- 172.21.15.78:0/2219284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f04103370 0x7f2f0410b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:42.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.835+0000 7f2f0a142640 1 --2- 172.21.15.78:0/2219284 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f04102a50 0x7f2f04102e30 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:42.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.835+0000 7f2f0a142640 1 -- 172.21.15.78:0/2219284 >> 172.21.15.78:0/2219284 conn(0x7f2f040fc4b0 msgr2=0x7f2f040fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:42.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.835+0000 7f2f0a142640 1 -- 172.21.15.78:0/2219284 shutdown_connections 2024-08-18T23:50:42.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.835+0000 7f2f0a142640 1 -- 172.21.15.78:0/2219284 wait complete. 2024-08-18T23:50:42.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.835+0000 7f2f0a142640 1 Processor -- start 2024-08-18T23:50:42.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.836+0000 7f2f0a142640 1 -- start start 2024-08-18T23:50:42.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.836+0000 7f2f0a142640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f04102a50 0x7f2f041006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:42.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.836+0000 7f2f0a142640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f04103370 0x7f2f04100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:42.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.836+0000 7f2f0a142640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0410be80 0x7f2f04101130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:42.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.836+0000 7f2f0a142640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f04111c80 con 0x7f2f04103370 2024-08-18T23:50:42.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.836+0000 7f2f0a142640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2f04111b00 con 0x7f2f0410be80 2024-08-18T23:50:42.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.836+0000 7f2f0a142640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2f04111e00 con 0x7f2f04102a50 2024-08-18T23:50:42.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.836+0000 7f2f02ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f04103370 0x7f2f04100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:42.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.836+0000 7f2f02ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f04103370 0x7f2f04100bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51464/0 (socket says 172.21.15.78:51464) 2024-08-18T23:50:42.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.836+0000 7f2f037fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f04102a50 0x7f2f041006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:42.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.836+0000 7f2f02ffd640 1 -- 172.21.15.78:0/2675354599 learned_addr learned my addr 172.21.15.78:0/2675354599 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:42.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.836+0000 7f2f03fff640 1 --2- 172.21.15.78:0/2675354599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0410be80 0x7f2f04101130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:42.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.836+0000 7f2f02ffd640 1 -- 172.21.15.78:0/2675354599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f04102a50 msgr2=0x7f2f041006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:42.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.836+0000 7f2f02ffd640 1 --2- 172.21.15.78:0/2675354599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f04102a50 0x7f2f041006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:42.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.836+0000 7f2f02ffd640 1 -- 172.21.15.78:0/2675354599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0410be80 msgr2=0x7f2f04101130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:42.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.836+0000 7f2f02ffd640 1 --2- 172.21.15.78:0/2675354599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0410be80 0x7f2f04101130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:42.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.836+0000 7f2f02ffd640 1 -- 172.21.15.78:0/2675354599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f041a6690 con 0x7f2f04103370 2024-08-18T23:50:42.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.836+0000 7f2f03fff640 1 --2- 172.21.15.78:0/2675354599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0410be80 0x7f2f04101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:50:42.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.836+0000 7f2f02ffd640 1 --2- 172.21.15.78:0/2675354599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f04103370 0x7f2f04100bf0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800e9f0 tx=0x7f2ef800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:42.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.837+0000 7f2f00ff9640 1 -- 172.21.15.78:0/2675354599 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef800ccc0 con 0x7f2f04103370 2024-08-18T23:50:42.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.837+0000 7f2f00ff9640 1 -- 172.21.15.78:0/2675354599 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ef800ce60 con 0x7f2f04103370 2024-08-18T23:50:42.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.837+0000 7f2f0a142640 1 -- 172.21.15.78:0/2675354599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f041a6980 con 0x7f2f04103370 2024-08-18T23:50:42.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.837+0000 7f2f00ff9640 1 -- 172.21.15.78:0/2675354599 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8010690 con 0x7f2f04103370 2024-08-18T23:50:42.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.837+0000 7f2f0a142640 1 -- 172.21.15.78:0/2675354599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f041a6ec0 con 0x7f2f04103370 2024-08-18T23:50:42.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.838+0000 7f2f0a142640 1 -- 172.21.15.78:0/2675354599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f04102e30 con 0x7f2f04103370 2024-08-18T23:50:42.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.839+0000 7f2f00ff9640 1 -- 172.21.15.78:0/2675354599 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2ef80046c0 con 0x7f2f04103370 2024-08-18T23:50:42.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.840+0000 7f2f00ff9640 1 --2- 172.21.15.78:0/2675354599 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2eec0a2c00 0x7f2eec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:42.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.840+0000 7f2f037fe640 1 --2- 172.21.15.78:0/2675354599 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2eec0a2c00 0x7f2eec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:42.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.840+0000 7f2f00ff9640 1 -- 172.21.15.78:0/2675354599 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2ef80cf8d0 con 0x7f2f04103370 2024-08-18T23:50:42.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.841+0000 7f2f037fe640 1 --2- 172.21.15.78:0/2675354599 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2eec0a2c00 0x7f2eec0a50c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4009230 tx=0x7f2ef40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:42.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:42.844+0000 7f2f00ff9640 1 -- 172.21.15.78:0/2675354599 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2ef8053df0 con 0x7f2f04103370 2024-08-18T23:50:43.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:43.068+0000 7f2f0a142640 1 -- 172.21.15.78:0/2675354599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f04101be0 con 0x7f2f04103370 2024-08-18T23:50:43.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:43.068+0000 7f2f00ff9640 1 -- 172.21.15.78:0/2675354599 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11933 (secure 0 0 0) 0x7f2f04101be0 con 0x7f2f04103370 2024-08-18T23:50:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:50:43.070 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"btime":"2024-08-18T23:48:43:326157+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:48:42.900232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:50:43.070 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-08-18T23:50:43.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:43.071+0000 7f2f0a142640 1 -- 172.21.15.78:0/2675354599 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2eec0a2c00 msgr2=0x7f2eec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:43.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:43.071+0000 7f2f0a142640 1 --2- 172.21.15.78:0/2675354599 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2eec0a2c00 0x7f2eec0a50c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4009230 tx=0x7f2ef40058d0 comp rx=0 tx=0).stop 2024-08-18T23:50:43.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:43.071+0000 7f2f0a142640 1 -- 172.21.15.78:0/2675354599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f04103370 msgr2=0x7f2f04100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:43.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:43.071+0000 7f2f0a142640 1 --2- 172.21.15.78:0/2675354599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f04103370 0x7f2f04100bf0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800e9f0 tx=0x7f2ef800eeb0 comp rx=0 tx=0).stop 2024-08-18T23:50:43.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:43.071+0000 7f2f0a142640 1 -- 172.21.15.78:0/2675354599 shutdown_connections 2024-08-18T23:50:43.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:43.071+0000 7f2f0a142640 1 --2- 172.21.15.78:0/2675354599 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2eec0a2c00 0x7f2eec0a50c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:43.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:43.071+0000 7f2f0a142640 1 --2- 172.21.15.78:0/2675354599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0410be80 0x7f2f04101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:43.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:43.071+0000 7f2f0a142640 1 --2- 172.21.15.78:0/2675354599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f04103370 0x7f2f04100bf0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:43.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:43.071+0000 7f2f0a142640 1 --2- 172.21.15.78:0/2675354599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f04102a50 0x7f2f041006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:43.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:43.071+0000 7f2f0a142640 1 -- 172.21.15.78:0/2675354599 >> 172.21.15.78:0/2675354599 conn(0x7f2f040fc4b0 msgr2=0x7f2f040fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:43.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:43.072+0000 7f2f0a142640 1 -- 172.21.15.78:0/2675354599 shutdown_connections 2024-08-18T23:50:43.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:43.072+0000 7f2f0a142640 1 -- 172.21.15.78:0/2675354599 wait complete. 2024-08-18T23:50:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:43 smithi161 ceph-mon[22027]: pgmap v542: 129 pgs: 129 active+clean; 771 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 535 op/s 2024-08-18T23:50:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:43 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2675354599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:50:43.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:43 smithi078 ceph-mon[18440]: pgmap v542: 129 pgs: 129 active+clean; 771 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 535 op/s 2024-08-18T23:50:43.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:43 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2675354599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:50:43.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:43 smithi123 ceph-mon[21808]: pgmap v542: 129 pgs: 129 active+clean; 771 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 535 op/s 2024-08-18T23:50:43.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:43 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2675354599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:50:43.900 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 48fb2815 --release --await' 2024-08-18T23:50:43.901 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 48fb2815 --release --await 2024-08-18T23:50:44.187 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:50:45.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.994+0000 7facae29c640 1 -- 172.21.15.78:0/3380253103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faca81078e0 msgr2=0x7faca8107cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:45.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.994+0000 7facae29c640 1 --2- 172.21.15.78:0/3380253103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faca81078e0 0x7faca8107cc0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fac98009930 tx=0x7fac9802a420 comp rx=0 tx=0).stop 2024-08-18T23:50:45.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.994+0000 7facae29c640 1 -- 172.21.15.78:0/3380253103 shutdown_connections 2024-08-18T23:50:45.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.994+0000 7facae29c640 1 --2- 172.21.15.78:0/3380253103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faca8102280 0x7faca810e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.994+0000 7facae29c640 1 --2- 172.21.15.78:0/3380253103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faca81018e0 0x7faca8101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.994+0000 7facae29c640 1 --2- 172.21.15.78:0/3380253103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faca81078e0 0x7faca8107cc0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.994+0000 7facae29c640 1 -- 172.21.15.78:0/3380253103 >> 172.21.15.78:0/3380253103 conn(0x7faca80fd600 msgr2=0x7faca80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:45.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.994+0000 7facae29c640 1 -- 172.21.15.78:0/3380253103 shutdown_connections 2024-08-18T23:50:45.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.994+0000 7facae29c640 1 -- 172.21.15.78:0/3380253103 wait complete. 2024-08-18T23:50:45.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.995+0000 7facae29c640 1 Processor -- start 2024-08-18T23:50:45.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.995+0000 7facae29c640 1 -- start start 2024-08-18T23:50:45.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.995+0000 7facae29c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faca81018e0 0x7faca81981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:45.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.995+0000 7facae29c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faca8102280 0x7faca8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:45.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.995+0000 7facae29c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faca81078e0 0x7faca819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:45.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.995+0000 7facae29c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faca8111ce0 con 0x7faca8102280 2024-08-18T23:50:45.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.995+0000 7facae29c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faca8111b60 con 0x7faca81078e0 2024-08-18T23:50:45.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.995+0000 7facae29c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faca8111e60 con 0x7faca81018e0 2024-08-18T23:50:45.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.995+0000 7fac9ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faca8102280 0x7faca8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:45.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.995+0000 7faca7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faca81018e0 0x7faca81981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:45.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.995+0000 7fac9ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faca8102280 0x7faca8198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51484/0 (socket says 172.21.15.78:51484) 2024-08-18T23:50:45.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.995+0000 7fac9ffff640 1 -- 172.21.15.78:0/98818292 learned_addr learned my addr 172.21.15.78:0/98818292 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:45.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.995+0000 7facac839640 1 --2- 172.21.15.78:0/98818292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faca81078e0 0x7faca819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:45.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.996+0000 7fac9ffff640 1 -- 172.21.15.78:0/98818292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faca81018e0 msgr2=0x7faca81981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:45.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.996+0000 7fac9ffff640 1 --2- 172.21.15.78:0/98818292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faca81018e0 0x7faca81981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.996+0000 7fac9ffff640 1 -- 172.21.15.78:0/98818292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faca81078e0 msgr2=0x7faca819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:45.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.996+0000 7fac9ffff640 1 --2- 172.21.15.78:0/98818292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faca81078e0 0x7faca819cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.996+0000 7fac9ffff640 1 -- 172.21.15.78:0/98818292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca819d190 con 0x7faca8102280 2024-08-18T23:50:45.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.996+0000 7fac9ffff640 1 --2- 172.21.15.78:0/98818292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faca8102280 0x7faca8198720 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fac9000e9f0 tx=0x7fac9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:45.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.996+0000 7faca5ffb640 1 -- 172.21.15.78:0/98818292 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9000cd80 con 0x7faca8102280 2024-08-18T23:50:45.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.996+0000 7facae29c640 1 -- 172.21.15.78:0/98818292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca819d480 con 0x7faca8102280 2024-08-18T23:50:45.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.996+0000 7faca5ffb640 1 -- 172.21.15.78:0/98818292 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac90010910 con 0x7faca8102280 2024-08-18T23:50:45.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.996+0000 7faca5ffb640 1 -- 172.21.15.78:0/98818292 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac900188f0 con 0x7faca8102280 2024-08-18T23:50:45.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.996+0000 7facae29c640 1 -- 172.21.15.78:0/98818292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca81a4d60 con 0x7faca8102280 2024-08-18T23:50:45.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.999+0000 7facb069a740 1 Processor -- start 2024-08-18T23:50:45.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:44.999+0000 7faca5ffb640 1 -- 172.21.15.78:0/98818292 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fac90010ac0 con 0x7faca8102280 2024-08-18T23:50:45.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.000+0000 7facb069a740 1 -- start start 2024-08-18T23:50:45.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.000+0000 7faca5ffb640 1 --2- 172.21.15.78:0/98818292 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fac940a2c00 0x7fac940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:45.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.000+0000 7facb069a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563384476bd0 0x563384476fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:45.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.000+0000 7facb069a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633844774f0 0x563384471520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:45.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.000+0000 7facb069a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384471b90 0x563384472010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:45.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.000+0000 7facb069a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5633843cdc60 con 0x563384471b90 2024-08-18T23:50:45.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.000+0000 7facb069a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563384365b90 con 0x563384476bd0 2024-08-18T23:50:45.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.000+0000 7facb069a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5633843c9c20 con 0x5633844774f0 2024-08-18T23:50:45.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.001+0000 7facac839640 1 --2- 172.21.15.78:0/98818292 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fac940a2c00 0x7fac940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:45.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.001+0000 7fac9ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633844774f0 0x563384471520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:45.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.001+0000 7faca7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563384476bd0 0x563384476fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:45.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.001+0000 7faca7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563384476bd0 0x563384476fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:45304/0 (socket says 172.21.15.78:45304) 2024-08-18T23:50:45.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.001+0000 7faca7fff640 1 -- 172.21.15.78:0/2846982322 learned_addr learned my addr 172.21.15.78:0/2846982322 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:45.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.001+0000 7faca5ffb640 1 -- 172.21.15.78:0/98818292 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fac900cf7c0 con 0x7faca8102280 2024-08-18T23:50:45.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.001+0000 7faca7fff640 1 -- 172.21.15.78:0/2846982322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633844774f0 msgr2=0x563384471520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:45.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.001+0000 7faca7fff640 1 --2- 172.21.15.78:0/2846982322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633844774f0 0x563384471520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.001+0000 7facac839640 1 --2- 172.21.15.78:0/2846982322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384471b90 0x563384472010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:45.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.001+0000 7faca7fff640 1 -- 172.21.15.78:0/2846982322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384471b90 msgr2=0x563384472010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:45.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.001+0000 7faca7fff640 1 --2- 172.21.15.78:0/2846982322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384471b90 0x563384472010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.001+0000 7faca7fff640 1 -- 172.21.15.78:0/2846982322 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633840dd580 con 0x563384476bd0 2024-08-18T23:50:45.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.001+0000 7fac9ffff640 1 --2- 172.21.15.78:0/2846982322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633844774f0 0x563384471520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:50:45.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.001+0000 7facac839640 1 --2- 172.21.15.78:0/2846982322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384471b90 0x563384472010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:50:45.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.001+0000 7faca7fff640 1 --2- 172.21.15.78:0/2846982322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563384476bd0 0x563384476fb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fac9802a940 tx=0x7fac98005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9123ecd1d6944b20 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:45.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.002+0000 7fac9e7fc640 1 -- 172.21.15.78:0/2846982322 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98040070 con 0x563384476bd0 2024-08-18T23:50:45.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.002+0000 7facac839640 1 --2- 172.21.15.78:0/98818292 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fac940a2c00 0x7fac940a50c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x563384465220 tx=0x7faca0009aa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:45.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.002+0000 7fac9e7fc640 1 -- 172.21.15.78:0/2846982322 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac98031870 con 0x563384476bd0 2024-08-18T23:50:45.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.002+0000 7fac9e7fc640 1 -- 172.21.15.78:0/2846982322 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98035040 con 0x563384476bd0 2024-08-18T23:50:45.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.002+0000 7facb069a740 1 -- 172.21.15.78:0/2846982322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563384476bd0 msgr2=0x563384476fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:45.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.002+0000 7facb069a740 1 --2- 172.21.15.78:0/2846982322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563384476bd0 0x563384476fb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fac9802a940 tx=0x7fac98005bd0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.003+0000 7facb069a740 1 -- 172.21.15.78:0/2846982322 shutdown_connections 2024-08-18T23:50:45.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.003+0000 7facb069a740 1 --2- 172.21.15.78:0/2846982322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384471b90 0x563384472010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.003+0000 7facb069a740 1 --2- 172.21.15.78:0/2846982322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633844774f0 0x563384471520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.003+0000 7facb069a740 1 --2- 172.21.15.78:0/2846982322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563384476bd0 0x563384476fb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.003+0000 7facb069a740 1 -- 172.21.15.78:0/2846982322 >> 172.21.15.78:0/2846982322 conn(0x5633844639d0 msgr2=0x563384463db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:45.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.003+0000 7facb069a740 1 -- 172.21.15.78:0/2846982322 shutdown_connections 2024-08-18T23:50:45.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.003+0000 7facb069a740 1 -- 172.21.15.78:0/2846982322 wait complete. 2024-08-18T23:50:45.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.003+0000 7facb069a740 1 -- start start 2024-08-18T23:50:45.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.003+0000 7facb069a740 1 Processor -- start 2024-08-18T23:50:45.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7facb069a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563384471b90 0x56338447a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:45.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7facb069a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 0x56338447ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:45.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7facb069a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633844774f0 0x56338447b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:45.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7facb069a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56338447deb0 con 0x563384476bd0 2024-08-18T23:50:45.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7facb069a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56338436b7c0 con 0x563384471b90 2024-08-18T23:50:45.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7facb069a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56338447e030 con 0x5633844774f0 2024-08-18T23:50:45.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7facb069a740 10 client.0 _ll_register_callbacks cb 0x563384374f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:45.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7fac9ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 0x56338447ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:45.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7faca7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563384471b90 0x56338447a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:45.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7fac9ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 0x56338447ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51500/0 (socket says 172.21.15.78:51500) 2024-08-18T23:50:45.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7fac9ffff640 1 -- 172.21.15.78:0/830464762 learned_addr learned my addr 172.21.15.78:0/830464762 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:45.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7facac839640 1 --2- 172.21.15.78:0/830464762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633844774f0 0x56338447b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:45.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7fac9ffff640 1 -- 172.21.15.78:0/830464762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633844774f0 msgr2=0x56338447b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:45.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7fac9ffff640 1 --2- 172.21.15.78:0/830464762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633844774f0 0x56338447b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7fac9ffff640 1 -- 172.21.15.78:0/830464762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563384471b90 msgr2=0x56338447a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:45.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7fac9ffff640 1 --2- 172.21.15.78:0/830464762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563384471b90 0x56338447a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7fac9ffff640 1 -- 172.21.15.78:0/830464762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633841a7960 con 0x563384476bd0 2024-08-18T23:50:45.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7faca7fff640 1 --2- 172.21.15.78:0/830464762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563384471b90 0x56338447a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:50:45.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7fac9ffff640 1 --2- 172.21.15.78:0/830464762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 0x56338447ae70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fac900a4370 tx=0x7fac90079410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f7b888350aac75 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:45.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7fac877fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:50:45.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.004+0000 7fac877fe640 1 -- 172.21.15.78:0/830464762 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac90026040 con 0x563384476bd0 2024-08-18T23:50:45.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.005+0000 7fac877fe640 1 -- 172.21.15.78:0/830464762 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fac90076560 con 0x563384476bd0 2024-08-18T23:50:45.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.005+0000 7facb069a740 1 -- 172.21.15.78:0/830464762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac98031870 con 0x563384476bd0 2024-08-18T23:50:45.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.005+0000 7fac877fe640 1 -- 172.21.15.78:0/830464762 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fac90076760 con 0x563384476bd0 2024-08-18T23:50:45.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.005+0000 7facb069a740 10 client.16437 fetch_fsmap learned FSMap version 28 2024-08-18T23:50:45.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.005+0000 7facb069a740 1 -- 172.21.15.78:0/830464762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563384489e70 con 0x563384476bd0 2024-08-18T23:50:45.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.005+0000 7fac877fe640 1 -- 172.21.15.78:0/830464762 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac98031870 con 0x563384476bd0 2024-08-18T23:50:45.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.005+0000 7fac877fe640 1 -- 172.21.15.78:0/830464762 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fac9009b230 con 0x563384476bd0 2024-08-18T23:50:45.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.005+0000 7facb069a740 10 client.16437 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:50:45.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.005+0000 7facb069a740 10 client.16437 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:50:45.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.005+0000 7facb069a740 20 client.16437 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:45.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.005+0000 7facb069a740 1 --2- 172.21.15.78:0/830464762 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56338448b390 0x56338448d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:45.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.005+0000 7facb069a740 4 client.16437 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:50:45.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.005+0000 7facb069a740 1 -- 172.21.15.78:0/830464762 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633843db1d0 con 0x56338448b390 2024-08-18T23:50:45.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.006+0000 7faca7fff640 1 --2- 172.21.15.78:0/830464762 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56338448b390 0x56338448d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:45.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.006+0000 7fac877fe640 10 client.16437 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:45.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.006+0000 7faca7fff640 1 --2- 172.21.15.78:0/830464762 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56338448b390 0x56338448d7b0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b7bd60e1055ca5c server_cookie=5b5c9f524e4ea9a9 in_seq=0 out_seq=0 2024-08-18T23:50:45.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.008+0000 7fac877fe640 1 -- 172.21.15.78:0/830464762 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633843db1d0 con 0x56338448b390 2024-08-18T23:50:45.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.008+0000 7fac877fe640 10 client.16437 handle_command_reply: tid=0 2024-08-18T23:50:45.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.008+0000 7facb069a740 1 client.16437 shutdown 2024-08-18T23:50:45.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.008+0000 7facb069a740 1 -- 172.21.15.78:0/830464762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 msgr2=0x56338447ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:45.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.008+0000 7facb069a740 1 --2- 172.21.15.78:0/830464762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 0x56338447ae70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fac900a4370 tx=0x7fac90079410 comp rx=0 tx=0).stop 2024-08-18T23:50:45.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.009+0000 7facb069a740 1 -- 172.21.15.78:0/830464762 shutdown_connections 2024-08-18T23:50:45.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.009+0000 7facb069a740 1 --2- 172.21.15.78:0/830464762 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x56338448b390 0x56338448d7b0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.009+0000 7facb069a740 1 --2- 172.21.15.78:0/830464762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5633844774f0 0x56338447b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.009+0000 7facb069a740 1 --2- 172.21.15.78:0/830464762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 0x56338447ae70 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.009+0000 7facb069a740 1 --2- 172.21.15.78:0/830464762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563384471b90 0x56338447a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.009+0000 7fac877fe640 0 client.16437 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:45.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.009+0000 7facb069a740 1 -- 172.21.15.78:0/830464762 >> 172.21.15.78:0/830464762 conn(0x5633844639d0 msgr2=0x563384463080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:45.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.009+0000 7facb069a740 1 -- 172.21.15.78:0/830464762 shutdown_connections 2024-08-18T23:50:45.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.009+0000 7facb069a740 1 -- 172.21.15.78:0/830464762 wait complete. 2024-08-18T23:50:45.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.009+0000 7facb069a740 20 client.16437 trim_cache size 0 max 16384 2024-08-18T23:50:45.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.026+0000 7facb069a740 1 Processor -- start 2024-08-18T23:50:45.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.026+0000 7facb069a740 1 -- start start 2024-08-18T23:50:45.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.027+0000 7facb069a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563384471b90 0x563384471520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:45.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.027+0000 7facb069a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 0x56338447a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:45.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.027+0000 7facb069a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633844774f0 0x56338447ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:45.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.027+0000 7facb069a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faca81af0b0 con 0x563384476bd0 2024-08-18T23:50:45.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.027+0000 7facb069a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faca81ac980 con 0x5633844774f0 2024-08-18T23:50:45.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.027+0000 7facb069a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faca81aa7d0 con 0x563384471b90 2024-08-18T23:50:45.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.027+0000 7fac9ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 0x56338447a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:45.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.027+0000 7faca7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563384471b90 0x563384471520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:45.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.027+0000 7fac9ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 0x56338447a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51516/0 (socket says 172.21.15.78:51516) 2024-08-18T23:50:45.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.027+0000 7fac9ffff640 1 -- 172.21.15.78:0/2806817533 learned_addr learned my addr 172.21.15.78:0/2806817533 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:45.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.027+0000 7facac839640 1 --2- 172.21.15.78:0/2806817533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633844774f0 0x56338447ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:45.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.027+0000 7fac9ffff640 1 -- 172.21.15.78:0/2806817533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563384471b90 msgr2=0x563384471520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:45.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.027+0000 7fac9ffff640 1 --2- 172.21.15.78:0/2806817533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563384471b90 0x563384471520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.027+0000 7fac9ffff640 1 -- 172.21.15.78:0/2806817533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633844774f0 msgr2=0x56338447ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:45.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.027+0000 7fac9ffff640 1 --2- 172.21.15.78:0/2806817533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633844774f0 0x56338447ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.027+0000 7fac9ffff640 1 -- 172.21.15.78:0/2806817533 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633844798e0 con 0x563384476bd0 2024-08-18T23:50:45.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.027+0000 7fac9ffff640 1 --2- 172.21.15.78:0/2806817533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 0x56338447a8f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c6a0 tx=0x7fac9009bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee167f8a55d8ed7 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:45.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.027+0000 7fac9dffb640 1 -- 172.21.15.78:0/2806817533 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9009ffe0 con 0x563384476bd0 2024-08-18T23:50:45.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.028+0000 7fac9dffb640 1 -- 172.21.15.78:0/2806817533 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac90076120 con 0x563384476bd0 2024-08-18T23:50:45.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.028+0000 7fac9dffb640 1 -- 172.21.15.78:0/2806817533 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9009b390 con 0x563384476bd0 2024-08-18T23:50:45.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.028+0000 7facb069a740 1 -- 172.21.15.78:0/2806817533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 msgr2=0x56338447a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:45.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.028+0000 7facb069a740 1 --2- 172.21.15.78:0/2806817533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 0x56338447a8f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c6a0 tx=0x7fac9009bbe0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.028+0000 7facb069a740 1 -- 172.21.15.78:0/2806817533 shutdown_connections 2024-08-18T23:50:45.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.028+0000 7facb069a740 1 --2- 172.21.15.78:0/2806817533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633844774f0 0x56338447ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.028+0000 7facb069a740 1 --2- 172.21.15.78:0/2806817533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 0x56338447a8f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.028+0000 7facb069a740 1 --2- 172.21.15.78:0/2806817533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563384471b90 0x563384471520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.028+0000 7facb069a740 1 -- 172.21.15.78:0/2806817533 >> 172.21.15.78:0/2806817533 conn(0x5633844639d0 msgr2=0x5633844762e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:45.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.029+0000 7facb069a740 1 -- 172.21.15.78:0/2806817533 shutdown_connections 2024-08-18T23:50:45.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.029+0000 7facb069a740 1 -- 172.21.15.78:0/2806817533 wait complete. 2024-08-18T23:50:45.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.029+0000 7facb069a740 1 -- start start 2024-08-18T23:50:45.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.029+0000 7facb069a740 1 Processor -- start 2024-08-18T23:50:45.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.029+0000 7facb069a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563384471b90 0x563384492b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:45.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.029+0000 7facb069a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 0x5633844930d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:45.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.029+0000 7facb069a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633844774f0 0x563384493610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:45.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.029+0000 7facb069a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faca81aae40 con 0x563384476bd0 2024-08-18T23:50:45.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.030+0000 7facb069a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faca81ac2b0 con 0x5633844774f0 2024-08-18T23:50:45.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.030+0000 7facb069a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faca81ab510 con 0x563384471b90 2024-08-18T23:50:45.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.030+0000 7facb069a740 10 client.0 _ll_register_callbacks cb 0x7fac90073d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:45.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.030+0000 7fac9ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 0x5633844930d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:45.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.030+0000 7facac839640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633844774f0 0x563384493610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:45.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.030+0000 7fac9ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 0x5633844930d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51520/0 (socket says 172.21.15.78:51520) 2024-08-18T23:50:45.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.030+0000 7fac9ffff640 1 -- 172.21.15.78:0/2883452117 learned_addr learned my addr 172.21.15.78:0/2883452117 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:45.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.030+0000 7faca7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563384471b90 0x563384492b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:45.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.030+0000 7fac9ffff640 1 -- 172.21.15.78:0/2883452117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563384471b90 msgr2=0x563384492b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:45.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.030+0000 7fac9ffff640 1 --2- 172.21.15.78:0/2883452117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563384471b90 0x563384492b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.030+0000 7fac9ffff640 1 -- 172.21.15.78:0/2883452117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633844774f0 msgr2=0x563384493610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:45.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.030+0000 7fac9ffff640 1 --2- 172.21.15.78:0/2883452117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633844774f0 0x563384493610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:45.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.030+0000 7fac9ffff640 1 -- 172.21.15.78:0/2883452117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633844f5050 con 0x563384476bd0 2024-08-18T23:50:45.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.030+0000 7fac9ffff640 1 --2- 172.21.15.78:0/2883452117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 0x5633844930d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fac9009bc70 tx=0x7fac90079600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c914d519ca04ca server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:45.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.030+0000 7fac877fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:50:45.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.030+0000 7fac877fe640 1 -- 172.21.15.78:0/2883452117 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac900b8070 con 0x563384476bd0 2024-08-18T23:50:45.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.030+0000 7fac877fe640 1 -- 172.21.15.78:0/2883452117 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fac9009b330 con 0x563384476bd0 2024-08-18T23:50:45.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.030+0000 7facb069a740 1 -- 172.21.15.78:0/2883452117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac90076120 con 0x563384476bd0 2024-08-18T23:50:45.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.031+0000 7fac877fe640 1 -- 172.21.15.78:0/2883452117 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fac900ad050 con 0x563384476bd0 2024-08-18T23:50:45.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.031+0000 7facb069a740 10 client.16443 fetch_fsmap learned FSMap version 28 2024-08-18T23:50:45.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.031+0000 7facb069a740 1 -- 172.21.15.78:0/2883452117 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5633844f5050 con 0x563384476bd0 2024-08-18T23:50:45.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.031+0000 7fac877fe640 1 -- 172.21.15.78:0/2883452117 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac90076120 con 0x563384476bd0 2024-08-18T23:50:45.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.031+0000 7fac877fe640 1 -- 172.21.15.78:0/2883452117 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fac900a4af0 con 0x563384476bd0 2024-08-18T23:50:45.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.031+0000 7facb069a740 10 client.16443 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:50:45.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.031+0000 7facb069a740 10 client.16443 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:50:45.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.031+0000 7facb069a740 20 client.16443 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:45.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.031+0000 7facb069a740 1 --2- 172.21.15.78:0/2883452117 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5633844f6580 0x5633844f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:45.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.031+0000 7facb069a740 4 client.16443 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "48fb2815", "release": true, "await": true}] 2024-08-18T23:50:45.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.031+0000 7facb069a740 1 -- 172.21.15.78:0/2883452117 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "quiesce db", "set_id": "48fb2815", "release": true, "await": true}) -- 0x5633843d56e0 con 0x5633844f6580 2024-08-18T23:50:45.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.031+0000 7faca7fff640 1 --2- 172.21.15.78:0/2883452117 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5633844f6580 0x5633844f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:45.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.032+0000 7faca7fff640 1 --2- 172.21.15.78:0/2883452117 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5633844f6580 0x5633844f89a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3b9e90715feb691 server_cookie=6ffe5eaf89fffc3 in_seq=0 out_seq=0 2024-08-18T23:50:45.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:45.032+0000 7fac877fe640 10 client.16443 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:45 smithi161 ceph-mon[22027]: pgmap v543: 129 pgs: 129 active+clean; 774 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-08-18T23:50:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:45 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:50:45.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:45 smithi078 ceph-mon[18440]: pgmap v543: 129 pgs: 129 active+clean; 774 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-08-18T23:50:45.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:45 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:50:45.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:45 smithi123 ceph-mon[21808]: pgmap v543: 129 pgs: 129 active+clean; 774 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-08-18T23:50:45.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:45 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:50:46.208 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:50:46.208 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 28, 2024-08-18T23:50:46.208 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24452, 2024-08-18T23:50:46.208 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 20, 2024-08-18T23:50:46.209 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-08-18T23:50:46.209 INFO:teuthology.orchestra.run.smithi078.stdout: "48fb2815": { 2024-08-18T23:50:46.209 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 20, 2024-08-18T23:50:46.209 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-08-18T23:50:46.209 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:50:46.209 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-08-18T23:50:46.209 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:50:46.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:50:46.209 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-08-18T23:50:46.209 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 139.2, 2024-08-18T23:50:46.209 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-08-18T23:50:46.209 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-08-18T23:50:46.209 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-08-18T23:50:46.210 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:50:46.210 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-08-18T23:50:46.210 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:50:46.210 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:50:46.210 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:50:46.210 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:50:46.210 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:50:46.210 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:50:46.210 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:50:46.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.206+0000 7fac877fe640 1 -- 172.21.15.78:0/2883452117 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5633843d56e0 con 0x5633844f6580 2024-08-18T23:50:46.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.206+0000 7fac877fe640 10 client.16443 handle_command_reply: tid=0 2024-08-18T23:50:46.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.206+0000 7facb069a740 1 client.16443 shutdown 2024-08-18T23:50:46.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.206+0000 7facb069a740 1 -- 172.21.15.78:0/2883452117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 msgr2=0x5633844930d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:46.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.206+0000 7facb069a740 1 --2- 172.21.15.78:0/2883452117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 0x5633844930d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fac9009bc70 tx=0x7fac90079600 comp rx=0 tx=0).stop 2024-08-18T23:50:46.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.206+0000 7facb069a740 1 -- 172.21.15.78:0/2883452117 shutdown_connections 2024-08-18T23:50:46.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.206+0000 7facb069a740 1 --2- 172.21.15.78:0/2883452117 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5633844f6580 0x5633844f89a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:46.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.206+0000 7facb069a740 1 --2- 172.21.15.78:0/2883452117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633844774f0 0x563384493610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:46.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.206+0000 7facb069a740 1 --2- 172.21.15.78:0/2883452117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563384476bd0 0x5633844930d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:46.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.206+0000 7fac877fe640 0 client.16443 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:46.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.206+0000 7facb069a740 1 --2- 172.21.15.78:0/2883452117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563384471b90 0x563384492b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:46.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.206+0000 7facb069a740 1 -- 172.21.15.78:0/2883452117 >> 172.21.15.78:0/2883452117 conn(0x5633844639d0 msgr2=0x563384462860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:46.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.206+0000 7facb069a740 1 -- 172.21.15.78:0/2883452117 shutdown_connections 2024-08-18T23:50:46.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.206+0000 7facb069a740 1 -- 172.21.15.78:0/2883452117 wait complete. 2024-08-18T23:50:46.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.206+0000 7facb069a740 20 client.16443 trim_cache size 0 max 16384 2024-08-18T23:50:46.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.207+0000 7facae29c640 1 -- 172.21.15.78:0/98818292 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fac940a2c00 msgr2=0x7fac940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:46.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.207+0000 7facae29c640 1 --2- 172.21.15.78:0/98818292 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fac940a2c00 0x7fac940a50c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x563384465220 tx=0x7faca0009aa0 comp rx=0 tx=0).stop 2024-08-18T23:50:46.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.207+0000 7facae29c640 1 -- 172.21.15.78:0/98818292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faca8102280 msgr2=0x7faca8198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:46.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.207+0000 7facae29c640 1 --2- 172.21.15.78:0/98818292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faca8102280 0x7faca8198720 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fac9000e9f0 tx=0x7fac9000eeb0 comp rx=0 tx=0).stop 2024-08-18T23:50:46.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.207+0000 7facae29c640 1 -- 172.21.15.78:0/98818292 shutdown_connections 2024-08-18T23:50:46.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.207+0000 7facae29c640 1 --2- 172.21.15.78:0/98818292 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fac940a2c00 0x7fac940a50c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:46.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.207+0000 7facae29c640 1 --2- 172.21.15.78:0/98818292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faca81078e0 0x7faca819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:46.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.207+0000 7facae29c640 1 --2- 172.21.15.78:0/98818292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faca8102280 0x7faca8198720 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:46.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.207+0000 7facae29c640 1 --2- 172.21.15.78:0/98818292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faca81018e0 0x7faca81981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:46.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.207+0000 7facae29c640 1 -- 172.21.15.78:0/98818292 >> 172.21.15.78:0/98818292 conn(0x7faca80fd600 msgr2=0x7faca80fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:46.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.207+0000 7facae29c640 1 -- 172.21.15.78:0/98818292 shutdown_connections 2024-08-18T23:50:46.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:46.207+0000 7facae29c640 1 -- 172.21.15.78:0/98818292 wait complete. 2024-08-18T23:50:46.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:46 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:46 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:46 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:46 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:46 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:46 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:46 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:46 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:46 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:46 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:46 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:46 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:46 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:50:46.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:46 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:50:46.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:46 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:50:46.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:46 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:46 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:46 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:46 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:46 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:46 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:46 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:46 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:46 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:46 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:46 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:46 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:46 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:50:46.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:46 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:50:46.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:46 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:50:46.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:46 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:46 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:46 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:46 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:46 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:46 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:46 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:46 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:46 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:46 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:46 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:46 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-18T23:50:46.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:46 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:50:46.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:46 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:50:46.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:46 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:50:47.140 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '48fb2815', total seconds elapsed: 40.1 2024-08-18T23:50:47.141 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.2 2024-08-18T23:50:47.141 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:50:47.215 INFO:tasks.workunit.client.0.smithi078.stdout: 2024-08-18T23:50:47.215 INFO:tasks.workunit.client.0.smithi078.stdout:Final score for writes: 154 2024-08-18T23:50:47.215 INFO:tasks.workunit.client.0.smithi078.stdout:Final score for reads : 488414 2024-08-18T23:50:47.215 INFO:tasks.workunit.client.0.smithi078.stdout: 2024-08-18T23:50:47.216 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-08-18T23:50:47.216 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-08-18T23:50:47.411 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:50:47.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:47 smithi078 ceph-mon[18440]: Adjusting osd_memory_target on smithi161 to 176.0M 2024-08-18T23:50:47.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:47 smithi078 ceph-mon[18440]: Adjusting osd_memory_target on smithi078 to 176.0M 2024-08-18T23:50:47.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:47 smithi078 ceph-mon[18440]: Unable to set osd_memory_target on smithi161 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-08-18T23:50:47.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:47 smithi078 ceph-mon[18440]: Unable to set osd_memory_target on smithi078 to 184572416: error parsing value: Value '184572416' is below minimum 939524096 2024-08-18T23:50:47.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:47 smithi078 ceph-mon[18440]: Adjusting osd_memory_target on smithi123 to 176.0M 2024-08-18T23:50:47.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:47 smithi078 ceph-mon[18440]: Unable to set osd_memory_target on smithi123 to 184593920: error parsing value: Value '184593920' is below minimum 939524096 2024-08-18T23:50:47.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:47 smithi078 ceph-mon[18440]: pgmap v544: 129 pgs: 129 active+clean; 776 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.5 MiB/s wr, 501 op/s 2024-08-18T23:50:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:47 smithi161 ceph-mon[22027]: Adjusting osd_memory_target on smithi161 to 176.0M 2024-08-18T23:50:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:47 smithi161 ceph-mon[22027]: Adjusting osd_memory_target on smithi078 to 176.0M 2024-08-18T23:50:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:47 smithi161 ceph-mon[22027]: Unable to set osd_memory_target on smithi161 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-08-18T23:50:47.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:47 smithi161 ceph-mon[22027]: Unable to set osd_memory_target on smithi078 to 184572416: error parsing value: Value '184572416' is below minimum 939524096 2024-08-18T23:50:47.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:47 smithi161 ceph-mon[22027]: Adjusting osd_memory_target on smithi123 to 176.0M 2024-08-18T23:50:47.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:47 smithi161 ceph-mon[22027]: Unable to set osd_memory_target on smithi123 to 184593920: error parsing value: Value '184593920' is below minimum 939524096 2024-08-18T23:50:47.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:47 smithi161 ceph-mon[22027]: pgmap v544: 129 pgs: 129 active+clean; 776 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.5 MiB/s wr, 501 op/s 2024-08-18T23:50:47.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:47 smithi123 ceph-mon[21808]: Adjusting osd_memory_target on smithi161 to 176.0M 2024-08-18T23:50:47.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:47 smithi123 ceph-mon[21808]: Adjusting osd_memory_target on smithi078 to 176.0M 2024-08-18T23:50:47.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:47 smithi123 ceph-mon[21808]: Unable to set osd_memory_target on smithi161 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-08-18T23:50:47.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:47 smithi123 ceph-mon[21808]: Unable to set osd_memory_target on smithi078 to 184572416: error parsing value: Value '184572416' is below minimum 939524096 2024-08-18T23:50:47.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:47 smithi123 ceph-mon[21808]: Adjusting osd_memory_target on smithi123 to 176.0M 2024-08-18T23:50:47.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:47 smithi123 ceph-mon[21808]: Unable to set osd_memory_target on smithi123 to 184593920: error parsing value: Value '184593920' is below minimum 939524096 2024-08-18T23:50:47.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:47 smithi123 ceph-mon[21808]: pgmap v544: 129 pgs: 129 active+clean; 776 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.5 MiB/s wr, 501 op/s 2024-08-18T23:50:47.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.976+0000 7fa65d1af640 1 -- 172.21.15.78:0/14564504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa658101150 msgr2=0x7fa65810d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:47.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.976+0000 7fa65d1af640 1 --2- 172.21.15.78:0/14564504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa658101150 0x7fa65810d680 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa650009cd0 tx=0x7fa65002a8e0 comp rx=0 tx=0).stop 2024-08-18T23:50:47.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.976+0000 7fa65d1af640 1 -- 172.21.15.78:0/14564504 shutdown_connections 2024-08-18T23:50:47.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.976+0000 7fa65d1af640 1 --2- 172.21.15.78:0/14564504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa658101150 0x7fa65810d680 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:47.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.976+0000 7fa65d1af640 1 --2- 172.21.15.78:0/14564504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6581007b0 0x7fa658100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:47.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.976+0000 7fa65d1af640 1 --2- 172.21.15.78:0/14564504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6581067b0 0x7fa658106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:47.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.976+0000 7fa65d1af640 1 -- 172.21.15.78:0/14564504 >> 172.21.15.78:0/14564504 conn(0x7fa6580fc4d0 msgr2=0x7fa6580fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:47.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.976+0000 7fa65d1af640 1 -- 172.21.15.78:0/14564504 shutdown_connections 2024-08-18T23:50:47.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.976+0000 7fa65d1af640 1 -- 172.21.15.78:0/14564504 wait complete. 2024-08-18T23:50:47.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.977+0000 7fa65d1af640 1 Processor -- start 2024-08-18T23:50:47.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa65d1af640 1 -- start start 2024-08-18T23:50:47.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa65d1af640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6581007b0 0x7fa6581980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:47.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa65d1af640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa658101150 0x7fa658198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:47.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa65d1af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6581067b0 0x7fa65819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:47.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa65d1af640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa658110bb0 con 0x7fa6581067b0 2024-08-18T23:50:47.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa65d1af640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa658110a30 con 0x7fa6581007b0 2024-08-18T23:50:47.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa65d1af640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa658110d30 con 0x7fa658101150 2024-08-18T23:50:47.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa65759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6581067b0 0x7fa65819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:47.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa656d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6581007b0 0x7fa6581980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:47.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa65759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6581067b0 0x7fa65819c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53052/0 (socket says 172.21.15.78:53052) 2024-08-18T23:50:47.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa65759e640 1 -- 172.21.15.78:0/4159853970 learned_addr learned my addr 172.21.15.78:0/4159853970 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:47.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa65659c640 1 --2- 172.21.15.78:0/4159853970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa658101150 0x7fa658198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:47.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa656d9d640 1 -- 172.21.15.78:0/4159853970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa658101150 msgr2=0x7fa658198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:47.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa656d9d640 1 --2- 172.21.15.78:0/4159853970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa658101150 0x7fa658198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:47.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa656d9d640 1 -- 172.21.15.78:0/4159853970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6581067b0 msgr2=0x7fa65819c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa656d9d640 1 --2- 172.21.15.78:0/4159853970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6581067b0 0x7fa65819c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa656d9d640 1 -- 172.21.15.78:0/4159853970 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65819d090 con 0x7fa6581007b0 2024-08-18T23:50:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa65659c640 1 --2- 172.21.15.78:0/4159853970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa658101150 0x7fa658198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:50:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa65759e640 1 --2- 172.21.15.78:0/4159853970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6581067b0 0x7fa65819c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:50:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.978+0000 7fa656d9d640 1 --2- 172.21.15.78:0/4159853970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6581007b0 0x7fa6581980e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa64800e9f0 tx=0x7fa64800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.979+0000 7fa63ffff640 1 -- 172.21.15.78:0/4159853970 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64800ccc0 con 0x7fa6581007b0 2024-08-18T23:50:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.979+0000 7fa65d1af640 1 -- 172.21.15.78:0/4159853970 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65819d380 con 0x7fa6581007b0 2024-08-18T23:50:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.979+0000 7fa63ffff640 1 -- 172.21.15.78:0/4159853970 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa648010910 con 0x7fa6581007b0 2024-08-18T23:50:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.979+0000 7fa63ffff640 1 -- 172.21.15.78:0/4159853970 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648018960 con 0x7fa6581007b0 2024-08-18T23:50:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.979+0000 7fa65d1af640 1 -- 172.21.15.78:0/4159853970 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6581a4c60 con 0x7fa6581007b0 2024-08-18T23:50:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.980+0000 7fa65d1af640 1 -- 172.21.15.78:0/4159853970 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa658100c10 con 0x7fa6581007b0 2024-08-18T23:50:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.981+0000 7fa63ffff640 1 -- 172.21.15.78:0/4159853970 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa648010ac0 con 0x7fa6581007b0 2024-08-18T23:50:47.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.981+0000 7fa63ffff640 1 --2- 172.21.15.78:0/4159853970 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa6400a2b80 0x7fa6400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:47.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.981+0000 7fa65659c640 1 --2- 172.21.15.78:0/4159853970 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa6400a2b80 0x7fa6400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:47.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.981+0000 7fa63ffff640 1 -- 172.21.15.78:0/4159853970 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa6480cfbb0 con 0x7fa6581007b0 2024-08-18T23:50:47.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.982+0000 7fa65659c640 1 --2- 172.21.15.78:0/4159853970 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa6400a2b80 0x7fa6400a5040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa658104eb0 tx=0x7fa64c00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:47.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:47.985+0000 7fa63ffff640 1 -- 172.21.15.78:0/4159853970 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa64809c720 con 0x7fa6581007b0 2024-08-18T23:50:48.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:48.216+0000 7fa65d1af640 1 -- 172.21.15.78:0/4159853970 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa658109d90 con 0x7fa6581007b0 2024-08-18T23:50:48.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:48.217+0000 7fa63ffff640 1 -- 172.21.15.78:0/4159853970 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11933 (secure 0 0 0) 0x7fa6480bc0e0 con 0x7fa6581007b0 2024-08-18T23:50:48.219 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:50:48.219 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"btime":"2024-08-18T23:48:43:326157+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:48:42.900232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:50:48.219 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-08-18T23:50:48.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:48.220+0000 7fa65d1af640 1 -- 172.21.15.78:0/4159853970 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa6400a2b80 msgr2=0x7fa6400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:48.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:48.220+0000 7fa65d1af640 1 --2- 172.21.15.78:0/4159853970 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa6400a2b80 0x7fa6400a5040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa658104eb0 tx=0x7fa64c00a5c0 comp rx=0 tx=0).stop 2024-08-18T23:50:48.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:48.220+0000 7fa65d1af640 1 -- 172.21.15.78:0/4159853970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6581007b0 msgr2=0x7fa6581980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:48.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:48.220+0000 7fa65d1af640 1 --2- 172.21.15.78:0/4159853970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6581007b0 0x7fa6581980e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa64800e9f0 tx=0x7fa64800eeb0 comp rx=0 tx=0).stop 2024-08-18T23:50:48.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:48.220+0000 7fa65d1af640 1 -- 172.21.15.78:0/4159853970 shutdown_connections 2024-08-18T23:50:48.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:48.220+0000 7fa65d1af640 1 --2- 172.21.15.78:0/4159853970 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa6400a2b80 0x7fa6400a5040 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:48.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:48.220+0000 7fa65d1af640 1 --2- 172.21.15.78:0/4159853970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6581067b0 0x7fa65819c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:48.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:48.220+0000 7fa65d1af640 1 --2- 172.21.15.78:0/4159853970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa658101150 0x7fa658198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:48.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:48.220+0000 7fa65d1af640 1 --2- 172.21.15.78:0/4159853970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6581007b0 0x7fa6581980e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:48.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:48.220+0000 7fa65d1af640 1 -- 172.21.15.78:0/4159853970 >> 172.21.15.78:0/4159853970 conn(0x7fa6580fc4d0 msgr2=0x7fa65806a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:48.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:48.221+0000 7fa65d1af640 1 -- 172.21.15.78:0/4159853970 shutdown_connections 2024-08-18T23:50:48.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:48.221+0000 7fa65d1af640 1 -- 172.21.15.78:0/4159853970 wait complete. 2024-08-18T23:50:48.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:48 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/4159853970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:50:48.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:48 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/4159853970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:50:48.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:48 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/4159853970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:50:48.933 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 139.2 --await-for 45.0' 2024-08-18T23:50:48.934 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 139.2 --await-for 45.0 2024-08-18T23:50:49.202 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:50:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:49 smithi161 ceph-mon[22027]: pgmap v545: 129 pgs: 129 active+clean; 779 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.3 MiB/s wr, 500 op/s 2024-08-18T23:50:49.685 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:49 smithi078 ceph-mon[18440]: pgmap v545: 129 pgs: 129 active+clean; 779 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.3 MiB/s wr, 500 op/s 2024-08-18T23:50:49.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:49 smithi123 ceph-mon[21808]: pgmap v545: 129 pgs: 129 active+clean; 779 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.3 MiB/s wr, 500 op/s 2024-08-18T23:50:49.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.824+0000 7f4cde322640 1 -- 172.21.15.78:0/703127549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd8102b00 msgr2=0x7f4cd8102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:49.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.824+0000 7f4cde322640 1 --2- 172.21.15.78:0/703127549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd8102b00 0x7f4cd8102ee0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8009930 tx=0x7f4cc802a420 comp rx=0 tx=0).stop 2024-08-18T23:50:49.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.824+0000 7f4cde322640 1 -- 172.21.15.78:0/703127549 shutdown_connections 2024-08-18T23:50:49.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.824+0000 7f4cde322640 1 --2- 172.21.15.78:0/703127549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cd8108030 0x7f4cd810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.824+0000 7f4cde322640 1 --2- 172.21.15.78:0/703127549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd81034b0 0x7f4cd81078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.824+0000 7f4cde322640 1 --2- 172.21.15.78:0/703127549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd8102b00 0x7f4cd8102ee0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.824+0000 7f4cde322640 1 -- 172.21.15.78:0/703127549 >> 172.21.15.78:0/703127549 conn(0x7f4cd80fc500 msgr2=0x7f4cd80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:49.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.824+0000 7f4cde322640 1 -- 172.21.15.78:0/703127549 shutdown_connections 2024-08-18T23:50:49.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.824+0000 7f4cde322640 1 -- 172.21.15.78:0/703127549 wait complete. 2024-08-18T23:50:49.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.825+0000 7f4cde322640 1 Processor -- start 2024-08-18T23:50:49.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.825+0000 7f4cde322640 1 -- start start 2024-08-18T23:50:49.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.825+0000 7f4cde322640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd8102b00 0x7f4cd8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:49.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.825+0000 7f4cde322640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cd81034b0 0x7f4cd8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:49.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.825+0000 7f4cde322640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd8108030 0x7f4cd819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:49.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.825+0000 7f4cde322640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd8111e00 con 0x7f4cd8108030 2024-08-18T23:50:49.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.825+0000 7f4cde322640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd8111c80 con 0x7f4cd81034b0 2024-08-18T23:50:49.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.825+0000 7f4cde322640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd8111f80 con 0x7f4cd8102b00 2024-08-18T23:50:49.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.825+0000 7f4cdc8bf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd8108030 0x7f4cd819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:49.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.825+0000 7f4cdc8bf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd8108030 0x7f4cd819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53080/0 (socket says 172.21.15.78:53080) 2024-08-18T23:50:49.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.825+0000 7f4cd77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cd81034b0 0x7f4cd8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:49.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.825+0000 7f4cdc8bf640 1 -- 172.21.15.78:0/2017631855 learned_addr learned my addr 172.21.15.78:0/2017631855 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:49.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.825+0000 7f4cd7fff640 1 --2- 172.21.15.78:0/2017631855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd8102b00 0x7f4cd8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:49.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.826+0000 7f4cdc8bf640 1 -- 172.21.15.78:0/2017631855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd8102b00 msgr2=0x7f4cd8198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:49.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.826+0000 7f4cdc8bf640 1 --2- 172.21.15.78:0/2017631855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd8102b00 0x7f4cd8198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.826+0000 7f4cdc8bf640 1 -- 172.21.15.78:0/2017631855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cd81034b0 msgr2=0x7f4cd8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:49.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.826+0000 7f4cdc8bf640 1 --2- 172.21.15.78:0/2017631855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cd81034b0 0x7f4cd8198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.826+0000 7f4cdc8bf640 1 -- 172.21.15.78:0/2017631855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd819d3b0 con 0x7f4cd8108030 2024-08-18T23:50:49.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.826+0000 7f4cd7fff640 1 --2- 172.21.15.78:0/2017631855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd8102b00 0x7f4cd8198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:50:49.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.826+0000 7f4cdc8bf640 1 --2- 172.21.15.78:0/2017631855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd8108030 0x7f4cd819ccd0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009fd0 tx=0x7f4cd000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:49.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.826+0000 7f4cd57fa640 1 -- 172.21.15.78:0/2017631855 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0017070 con 0x7f4cd8108030 2024-08-18T23:50:49.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.826+0000 7f4cde322640 1 -- 172.21.15.78:0/2017631855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd819d6a0 con 0x7f4cd8108030 2024-08-18T23:50:49.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.826+0000 7f4cd57fa640 1 -- 172.21.15.78:0/2017631855 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd0003e50 con 0x7f4cd8108030 2024-08-18T23:50:49.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.826+0000 7f4cde322640 1 -- 172.21.15.78:0/2017631855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd81a4f80 con 0x7f4cd8108030 2024-08-18T23:50:49.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.826+0000 7f4cd57fa640 1 -- 172.21.15.78:0/2017631855 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0020060 con 0x7f4cd8108030 2024-08-18T23:50:49.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.828+0000 7f4cd57fa640 1 -- 172.21.15.78:0/2017631855 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4cd0003970 con 0x7f4cd8108030 2024-08-18T23:50:49.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.829+0000 7f4ce0720740 1 Processor -- start 2024-08-18T23:50:49.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.829+0000 7f4ce0720740 1 -- start start 2024-08-18T23:50:49.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.829+0000 7f4cd57fa640 1 --2- 172.21.15.78:0/2017631855 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4cc40a2b80 0x7f4cc40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:49.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.829+0000 7f4ce0720740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 0x5605f6706510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:49.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.829+0000 7f4ce0720740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6706a50 0x5605f6710dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:49.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.829+0000 7f4ce0720740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 0x5605f6713700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:49.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.829+0000 7f4ce0720740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5605f666ba70 con 0x5605f6711310 2024-08-18T23:50:49.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.829+0000 7f4ce0720740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5605f6603780 con 0x5605f6706a50 2024-08-18T23:50:49.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.829+0000 7f4ce0720740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5605f6667a30 con 0x5605f6706130 2024-08-18T23:50:49.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.829+0000 7f4cd77fe640 1 --2- 172.21.15.78:0/2017631855 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4cc40a2b80 0x7f4cc40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:49.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.829+0000 7f4cd57fa640 1 -- 172.21.15.78:0/2017631855 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f4cd00cf010 con 0x7f4cd8108030 2024-08-18T23:50:49.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.829+0000 7f4cd77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6706a50 0x5605f6710dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:49.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.829+0000 7f4cd77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6706a50 0x5605f6710dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:56058/0 (socket says 172.21.15.78:56058) 2024-08-18T23:50:49.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.829+0000 7f4cd77fe640 1 -- 172.21.15.78:0/3531140259 learned_addr learned my addr 172.21.15.78:0/3531140259 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:49.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.829+0000 7f4cdc8bf640 1 --2- 172.21.15.78:0/3531140259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 0x5605f6713700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:49.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.829+0000 7f4cd7fff640 1 --2- 172.21.15.78:0/3531140259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 0x5605f6706510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:49.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.830+0000 7f4cdc8bf640 1 -- 172.21.15.78:0/3531140259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 msgr2=0x5605f6706510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:49.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.830+0000 7f4cdc8bf640 1 --2- 172.21.15.78:0/3531140259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 0x5605f6706510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.830+0000 7f4cdc8bf640 1 -- 172.21.15.78:0/3531140259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6706a50 msgr2=0x5605f6710dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:49.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.830+0000 7f4cdc8bf640 1 --2- 172.21.15.78:0/3531140259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6706a50 0x5605f6710dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.830+0000 7f4cdc8bf640 1 -- 172.21.15.78:0/3531140259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605f63ddb90 con 0x5605f6711310 2024-08-18T23:50:49.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.830+0000 7f4cd77fe640 1 --2- 172.21.15.78:0/2017631855 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4cc40a2b80 0x7f4cc40a5040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8069500 tx=0x7f4cc000aa90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:49.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.830+0000 7f4cdc8bf640 1 --2- 172.21.15.78:0/3531140259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 0x5605f6713700 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000b760 tx=0x7f4cd00978b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff7dcd413a6f55f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:49.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.830+0000 7f4cce7fc640 1 -- 172.21.15.78:0/3531140259 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd000c2d0 con 0x5605f6711310 2024-08-18T23:50:49.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.830+0000 7f4cce7fc640 1 -- 172.21.15.78:0/3531140259 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd0019cd0 con 0x5605f6711310 2024-08-18T23:50:49.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.830+0000 7f4cce7fc640 1 -- 172.21.15.78:0/3531140259 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd00040b0 con 0x5605f6711310 2024-08-18T23:50:49.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.831+0000 7f4ce0720740 1 -- 172.21.15.78:0/3531140259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 msgr2=0x5605f6713700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:49.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.831+0000 7f4ce0720740 1 --2- 172.21.15.78:0/3531140259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 0x5605f6713700 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000b760 tx=0x7f4cd00978b0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.831+0000 7f4ce0720740 1 -- 172.21.15.78:0/3531140259 shutdown_connections 2024-08-18T23:50:49.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.831+0000 7f4ce0720740 1 --2- 172.21.15.78:0/3531140259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 0x5605f6713700 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.831+0000 7f4ce0720740 1 --2- 172.21.15.78:0/3531140259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6706a50 0x5605f6710dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.831+0000 7f4ce0720740 1 --2- 172.21.15.78:0/3531140259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 0x5605f6706510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.831+0000 7f4ce0720740 1 -- 172.21.15.78:0/3531140259 >> 172.21.15.78:0/3531140259 conn(0x5605f67026c0 msgr2=0x5605f6702aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:49.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.831+0000 7f4ce0720740 1 -- 172.21.15.78:0/3531140259 shutdown_connections 2024-08-18T23:50:49.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.831+0000 7f4ce0720740 1 -- 172.21.15.78:0/3531140259 wait complete. 2024-08-18T23:50:49.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.831+0000 7f4ce0720740 1 -- start start 2024-08-18T23:50:49.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.832+0000 7f4ce0720740 1 Processor -- start 2024-08-18T23:50:49.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.832+0000 7f4ce0720740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 0x5605f670e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:49.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.832+0000 7f4ce0720740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6706a50 0x5605f670e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:49.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.832+0000 7f4ce0720740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 0x5605f670ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:49.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.832+0000 7f4ce0720740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5605f6713e80 con 0x5605f6711310 2024-08-18T23:50:49.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.832+0000 7f4ce0720740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5605f6626990 con 0x5605f6706a50 2024-08-18T23:50:49.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.832+0000 7f4ce0720740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5605f6714000 con 0x5605f6706130 2024-08-18T23:50:49.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.832+0000 7f4ce0720740 10 client.0 _ll_register_callbacks cb 0x5605f637b090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:49.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.832+0000 7f4cdc8bf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 0x5605f670ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:49.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.832+0000 7f4cd7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 0x5605f670e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:49.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.832+0000 7f4cd77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6706a50 0x5605f670e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:49.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.832+0000 7f4cdc8bf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 0x5605f670ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53096/0 (socket says 172.21.15.78:53096) 2024-08-18T23:50:49.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.832+0000 7f4cdc8bf640 1 -- 172.21.15.78:0/1526459630 learned_addr learned my addr 172.21.15.78:0/1526459630 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:49.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.832+0000 7f4cd77fe640 1 -- 172.21.15.78:0/1526459630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 msgr2=0x5605f670e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:49.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.832+0000 7f4cd77fe640 1 --2- 172.21.15.78:0/1526459630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 0x5605f670e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.832+0000 7f4cd77fe640 1 -- 172.21.15.78:0/1526459630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 msgr2=0x5605f670ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:49.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.832+0000 7f4cd77fe640 1 --2- 172.21.15.78:0/1526459630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 0x5605f670ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.832+0000 7f4cd77fe640 1 -- 172.21.15.78:0/1526459630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605f66256c0 con 0x5605f6706a50 2024-08-18T23:50:49.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.833+0000 7f4cdc8bf640 1 --2- 172.21.15.78:0/1526459630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 0x5605f670ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:50:49.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.833+0000 7f4cd77fe640 1 --2- 172.21.15.78:0/1526459630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6706a50 0x5605f670e6f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4cc001c000 tx=0x7f4cc000a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3232eb24910c5cd7 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:49.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.833+0000 7f4cb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:50:49.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.833+0000 7f4cb77fe640 1 -- 172.21.15.78:0/1526459630 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc001fdc0 con 0x5605f6706a50 2024-08-18T23:50:49.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.833+0000 7f4cb77fe640 1 -- 172.21.15.78:0/1526459630 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f4cc0004020 con 0x5605f6706a50 2024-08-18T23:50:49.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.833+0000 7f4ce0720740 1 -- 172.21.15.78:0/1526459630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cd0019cd0 con 0x5605f6706a50 2024-08-18T23:50:49.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.833+0000 7f4cb77fe640 1 -- 172.21.15.78:0/1526459630 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4cc0004220 con 0x5605f6706a50 2024-08-18T23:50:49.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.833+0000 7f4ce0720740 10 client.25883 fetch_fsmap learned FSMap version 28 2024-08-18T23:50:49.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.833+0000 7f4ce0720740 1 -- 172.21.15.78:0/1526459630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5605f671fb30 con 0x5605f6706a50 2024-08-18T23:50:49.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.833+0000 7f4cb77fe640 1 -- 172.21.15.78:0/1526459630 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd0019cd0 con 0x5605f6706a50 2024-08-18T23:50:49.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.834+0000 7f4cb77fe640 1 -- 172.21.15.78:0/1526459630 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4cc001e050 con 0x5605f6706a50 2024-08-18T23:50:49.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.834+0000 7f4ce0720740 10 client.25883 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:50:49.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.834+0000 7f4ce0720740 10 client.25883 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:50:49.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.834+0000 7f4ce0720740 20 client.25883 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:49.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.834+0000 7f4ce0720740 1 --2- 172.21.15.78:0/1526459630 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5605f6721090 0x5605f67234b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:49.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.834+0000 7f4ce0720740 4 client.25883 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:50:49.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.834+0000 7f4ce0720740 1 -- 172.21.15.78:0/1526459630 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605f66791a0 con 0x5605f6721090 2024-08-18T23:50:49.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.834+0000 7f4cd7fff640 1 --2- 172.21.15.78:0/1526459630 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5605f6721090 0x5605f67234b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:49.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.834+0000 7f4cb77fe640 10 client.25883 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:49.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.834+0000 7f4cd7fff640 1 --2- 172.21.15.78:0/1526459630 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5605f6721090 0x5605f67234b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6887f63cca4fd674 server_cookie=4f1c44385515f994 in_seq=0 out_seq=0 2024-08-18T23:50:49.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.836+0000 7f4cb77fe640 1 -- 172.21.15.78:0/1526459630 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605f66791a0 con 0x5605f6721090 2024-08-18T23:50:49.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.836+0000 7f4cb77fe640 10 client.25883 handle_command_reply: tid=0 2024-08-18T23:50:49.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.836+0000 7f4ce0720740 1 client.25883 shutdown 2024-08-18T23:50:49.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.836+0000 7f4ce0720740 1 -- 172.21.15.78:0/1526459630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6706a50 msgr2=0x5605f670e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:49.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.836+0000 7f4ce0720740 1 --2- 172.21.15.78:0/1526459630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6706a50 0x5605f670e6f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4cc001c000 tx=0x7f4cc000a250 comp rx=0 tx=0).stop 2024-08-18T23:50:49.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.836+0000 7f4ce0720740 1 -- 172.21.15.78:0/1526459630 shutdown_connections 2024-08-18T23:50:49.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.836+0000 7f4ce0720740 1 --2- 172.21.15.78:0/1526459630 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5605f6721090 0x5605f67234b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.836+0000 7f4ce0720740 1 --2- 172.21.15.78:0/1526459630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 0x5605f670ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.836+0000 7f4ce0720740 1 --2- 172.21.15.78:0/1526459630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6706a50 0x5605f670e6f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.836+0000 7f4ce0720740 1 --2- 172.21.15.78:0/1526459630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 0x5605f670e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.836+0000 7f4ce0720740 1 -- 172.21.15.78:0/1526459630 >> 172.21.15.78:0/1526459630 conn(0x5605f67026c0 msgr2=0x5605f67083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:49.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.836+0000 7f4cb77fe640 0 client.25883 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:49.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.836+0000 7f4ce0720740 1 -- 172.21.15.78:0/1526459630 shutdown_connections 2024-08-18T23:50:49.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.836+0000 7f4ce0720740 1 -- 172.21.15.78:0/1526459630 wait complete. 2024-08-18T23:50:49.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.837+0000 7f4ce0720740 20 client.25883 trim_cache size 0 max 16384 2024-08-18T23:50:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.854+0000 7f4ce0720740 1 Processor -- start 2024-08-18T23:50:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.854+0000 7f4ce0720740 1 -- start start 2024-08-18T23:50:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.854+0000 7f4ce0720740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 0x5605f67015e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.854+0000 7f4ce0720740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6706a50 0x5605f6728d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.854+0000 7f4ce0720740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6711310 0x5605f67292c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.854+0000 7f4ce0720740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81b1320 con 0x5605f6706a50 2024-08-18T23:50:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.854+0000 7f4ce0720740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81aebf0 con 0x5605f6711310 2024-08-18T23:50:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.854+0000 7f4ce0720740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81aca40 con 0x5605f6706130 2024-08-18T23:50:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.854+0000 7f4cd77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6706a50 0x5605f6728d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.854+0000 7f4cd77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6706a50 0x5605f6728d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53104/0 (socket says 172.21.15.78:53104) 2024-08-18T23:50:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.854+0000 7f4cd77fe640 1 -- 172.21.15.78:0/1369151111 learned_addr learned my addr 172.21.15.78:0/1369151111 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.854+0000 7f4cdc8bf640 1 --2- 172.21.15.78:0/1369151111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6711310 0x5605f67292c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.854+0000 7f4cd7fff640 1 --2- 172.21.15.78:0/1369151111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 0x5605f67015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.854+0000 7f4cd77fe640 1 -- 172.21.15.78:0/1369151111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 msgr2=0x5605f67015e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.854+0000 7f4cd77fe640 1 --2- 172.21.15.78:0/1369151111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 0x5605f67015e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.854+0000 7f4cd77fe640 1 -- 172.21.15.78:0/1369151111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6711310 msgr2=0x5605f67292c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.854+0000 7f4cd77fe640 1 --2- 172.21.15.78:0/1369151111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6711310 0x5605f67292c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.854+0000 7f4cd77fe640 1 -- 172.21.15.78:0/1369151111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605f6710d00 con 0x5605f6706a50 2024-08-18T23:50:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.855+0000 7f4cd77fe640 1 --2- 172.21.15.78:0/1369151111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6706a50 0x5605f6728d80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0029690 tx=0x7f4cc0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abaaf4211a5a31d1 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.855+0000 7f4ccdffb640 1 -- 172.21.15.78:0/1369151111 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc003b070 con 0x5605f6706a50 2024-08-18T23:50:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.855+0000 7f4ccdffb640 1 -- 172.21.15.78:0/1369151111 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cc000a5f0 con 0x5605f6706a50 2024-08-18T23:50:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.855+0000 7f4ccdffb640 1 -- 172.21.15.78:0/1369151111 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc001f070 con 0x5605f6706a50 2024-08-18T23:50:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.855+0000 7f4ce0720740 1 -- 172.21.15.78:0/1369151111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6706a50 msgr2=0x5605f6728d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.855+0000 7f4ce0720740 1 --2- 172.21.15.78:0/1369151111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6706a50 0x5605f6728d80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0029690 tx=0x7f4cc0005d50 comp rx=0 tx=0).stop 2024-08-18T23:50:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.855+0000 7f4ce0720740 1 -- 172.21.15.78:0/1369151111 shutdown_connections 2024-08-18T23:50:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.855+0000 7f4ce0720740 1 --2- 172.21.15.78:0/1369151111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6711310 0x5605f67292c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.855+0000 7f4ce0720740 1 --2- 172.21.15.78:0/1369151111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6706a50 0x5605f6728d80 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.855+0000 7f4ce0720740 1 --2- 172.21.15.78:0/1369151111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 0x5605f67015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.855+0000 7f4ce0720740 1 -- 172.21.15.78:0/1369151111 >> 172.21.15.78:0/1369151111 conn(0x5605f67026c0 msgr2=0x5605f6700ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.855+0000 7f4ce0720740 1 -- 172.21.15.78:0/1369151111 shutdown_connections 2024-08-18T23:50:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.855+0000 7f4ce0720740 1 -- 172.21.15.78:0/1369151111 wait complete. 2024-08-18T23:50:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.856+0000 7f4ce0720740 1 -- start start 2024-08-18T23:50:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.856+0000 7f4ce0720740 1 Processor -- start 2024-08-18T23:50:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.856+0000 7f4ce0720740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 0x5605f6719930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.856+0000 7f4ce0720740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6706a50 0x5605f6714c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.856+0000 7f4ce0720740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 0x5605f67151a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.856+0000 7f4ce0720740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81ad0b0 con 0x5605f6711310 2024-08-18T23:50:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.856+0000 7f4ce0720740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81ae520 con 0x5605f6706a50 2024-08-18T23:50:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.856+0000 7f4ce0720740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81ad780 con 0x5605f6706130 2024-08-18T23:50:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.856+0000 7f4ce0720740 10 client.0 _ll_register_callbacks cb 0x7f4cc000fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.856+0000 7f4cdc8bf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 0x5605f67151a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.856+0000 7f4cd77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6706a50 0x5605f6714c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.856+0000 7f4cdc8bf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 0x5605f67151a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53112/0 (socket says 172.21.15.78:53112) 2024-08-18T23:50:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.856+0000 7f4cdc8bf640 1 -- 172.21.15.78:0/33852276 learned_addr learned my addr 172.21.15.78:0/33852276 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.856+0000 7f4cd7fff640 1 --2- 172.21.15.78:0/33852276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 0x5605f6719930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.857+0000 7f4cdc8bf640 1 -- 172.21.15.78:0/33852276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 msgr2=0x5605f6719930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.857+0000 7f4cdc8bf640 1 --2- 172.21.15.78:0/33852276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 0x5605f6719930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.857+0000 7f4cdc8bf640 1 -- 172.21.15.78:0/33852276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6706a50 msgr2=0x5605f6714c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.857+0000 7f4cdc8bf640 1 --2- 172.21.15.78:0/33852276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6706a50 0x5605f6714c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.857+0000 7f4cdc8bf640 1 -- 172.21.15.78:0/33852276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605f67159d0 con 0x5605f6711310 2024-08-18T23:50:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.857+0000 7f4cdc8bf640 1 --2- 172.21.15.78:0/33852276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 0x5605f67151a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009a30 tx=0x7f4cd0019f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d513bf42b6311f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.857+0000 7f4cb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:50:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.857+0000 7f4cb77fe640 1 -- 172.21.15.78:0/33852276 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd009b590 con 0x5605f6711310 2024-08-18T23:50:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.857+0000 7f4cb77fe640 1 -- 172.21.15.78:0/33852276 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f4cd00b4240 con 0x5605f6711310 2024-08-18T23:50:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.857+0000 7f4ce0720740 1 -- 172.21.15.78:0/33852276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cc000a5f0 con 0x5605f6711310 2024-08-18T23:50:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.857+0000 7f4cb77fe640 1 -- 172.21.15.78:0/33852276 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4cd0003c60 con 0x5605f6711310 2024-08-18T23:50:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.857+0000 7f4ce0720740 10 client.16467 fetch_fsmap learned FSMap version 28 2024-08-18T23:50:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.857+0000 7f4ce0720740 1 -- 172.21.15.78:0/33852276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5605f678bcc0 con 0x5605f6711310 2024-08-18T23:50:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.857+0000 7f4cb77fe640 1 -- 172.21.15.78:0/33852276 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cc000a5f0 con 0x5605f6711310 2024-08-18T23:50:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.858+0000 7f4cb77fe640 1 -- 172.21.15.78:0/33852276 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4cd0082030 con 0x5605f6711310 2024-08-18T23:50:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.858+0000 7f4ce0720740 10 client.16467 fetch_fsmap finished waiting for FSMap version 28 2024-08-18T23:50:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.858+0000 7f4ce0720740 10 client.16467 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:50:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.858+0000 7f4ce0720740 20 client.16467 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.858+0000 7f4ce0720740 1 --2- 172.21.15.78:0/33852276 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5605f678c010 0x5605f678e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.858+0000 7f4ce0720740 4 client.16467 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.2, "await_for": 45.0}] 2024-08-18T23:50:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.858+0000 7f4ce0720740 1 -- 172.21.15.78:0/33852276 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.2, "await_for": 45.0}) -- 0x5605f66734f0 con 0x5605f678c010 2024-08-18T23:50:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.858+0000 7f4cd7fff640 1 --2- 172.21.15.78:0/33852276 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5605f678c010 0x5605f678e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.858+0000 7f4cd7fff640 1 --2- 172.21.15.78:0/33852276 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5605f678c010 0x5605f678e400 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b6ab24c861613fb server_cookie=be655bfae760d80b in_seq=0 out_seq=0 2024-08-18T23:50:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:49.858+0000 7f4cb77fe640 10 client.16467 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:51 smithi161 ceph-mon[22027]: pgmap v546: 129 pgs: 129 active+clean; 774 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 603 op/s 2024-08-18T23:50:51.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:51 smithi078 ceph-mon[18440]: pgmap v546: 129 pgs: 129 active+clean; 774 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 603 op/s 2024-08-18T23:50:51.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:51 smithi123 ceph-mon[21808]: pgmap v546: 129 pgs: 129 active+clean; 774 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 603 op/s 2024-08-18T23:50:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:52 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:52.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:52 smithi161 ceph-mon[22027]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:50:52.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:52 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:52.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:52 smithi078 ceph-mon[18440]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:50:52.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:52 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:50:52.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:52 smithi123 ceph-mon[21808]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:50:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:53 smithi161 ceph-mon[22027]: pgmap v547: 129 pgs: 129 active+clean; 768 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.3 MiB/s wr, 653 op/s 2024-08-18T23:50:53.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:53 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:50:53.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:53 smithi078 ceph-mon[18440]: pgmap v547: 129 pgs: 129 active+clean; 768 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.3 MiB/s wr, 653 op/s 2024-08-18T23:50:53.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:53 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:50:53.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:53 smithi123 ceph-mon[21808]: pgmap v547: 129 pgs: 129 active+clean; 768 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.3 MiB/s wr, 653 op/s 2024-08-18T23:50:53.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:53 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:50:54.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:54 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:50:54.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:54 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:50:54.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:54 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:50:54.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:54 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:50:54.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:54 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:50:54.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:54 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:50:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:55 smithi161 ceph-mon[22027]: pgmap v548: 129 pgs: 129 active+clean; 767 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.1 MiB/s wr, 653 op/s 2024-08-18T23:50:55.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:55 smithi078 ceph-mon[18440]: pgmap v548: 129 pgs: 129 active+clean; 767 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.1 MiB/s wr, 653 op/s 2024-08-18T23:50:55.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:55 smithi123 ceph-mon[21808]: pgmap v548: 129 pgs: 129 active+clean; 767 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.1 MiB/s wr, 653 op/s 2024-08-18T23:50:56.113 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:50:56.367 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:50:57.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.095+0000 7fa1d22ce640 1 -- 172.21.15.78:0/3552607461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1cc100b50 msgr2=0x7fa1cc100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:57.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.095+0000 7fa1d22ce640 1 --2- 172.21.15.78:0/3552607461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1cc100b50 0x7fa1cc100f30 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009900 tx=0x7fa1bc02a480 comp rx=0 tx=0).stop 2024-08-18T23:50:57.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.095+0000 7fa1d22ce640 1 -- 172.21.15.78:0/3552607461 shutdown_connections 2024-08-18T23:50:57.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.095+0000 7fa1d22ce640 1 --2- 172.21.15.78:0/3552607461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1cc102c20 0x7fa1cc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.095+0000 7fa1d22ce640 1 --2- 172.21.15.78:0/3552607461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1cc1022a0 0x7fa1cc1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.095+0000 7fa1d22ce640 1 --2- 172.21.15.78:0/3552607461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1cc100b50 0x7fa1cc100f30 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.095+0000 7fa1d22ce640 1 -- 172.21.15.78:0/3552607461 >> 172.21.15.78:0/3552607461 conn(0x7fa1cc0fc500 msgr2=0x7fa1cc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:57.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.095+0000 7fa1d22ce640 1 -- 172.21.15.78:0/3552607461 shutdown_connections 2024-08-18T23:50:57.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.095+0000 7fa1d22ce640 1 -- 172.21.15.78:0/3552607461 wait complete. 2024-08-18T23:50:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.095+0000 7fa1d22ce640 1 Processor -- start 2024-08-18T23:50:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.096+0000 7fa1d22ce640 1 -- start start 2024-08-18T23:50:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.096+0000 7fa1d22ce640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1cc100b50 0x7fa1cc0fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.096+0000 7fa1d22ce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1cc1022a0 0x7fa1cc0fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.096+0000 7fa1d22ce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1cc102c20 0x7fa1cc0fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.096+0000 7fa1d22ce640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc110c70 con 0x7fa1cc1022a0 2024-08-18T23:50:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.096+0000 7fa1d22ce640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc110af0 con 0x7fa1cc100b50 2024-08-18T23:50:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.096+0000 7fa1d22ce640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc110df0 con 0x7fa1cc102c20 2024-08-18T23:50:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.096+0000 7fa1c3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1cc1022a0 0x7fa1cc0fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.096+0000 7fa1c3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1cc1022a0 0x7fa1cc0fcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33030/0 (socket says 172.21.15.78:33030) 2024-08-18T23:50:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.096+0000 7fa1cbfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1cc100b50 0x7fa1cc0fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:57.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.096+0000 7fa1c3fff640 1 -- 172.21.15.78:0/3028440026 learned_addr learned my addr 172.21.15.78:0/3028440026 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.096+0000 7fa1d086b640 1 --2- 172.21.15.78:0/3028440026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1cc102c20 0x7fa1cc0fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.096+0000 7fa1c3fff640 1 -- 172.21.15.78:0/3028440026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1cc102c20 msgr2=0x7fa1cc0fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.096+0000 7fa1c3fff640 1 --2- 172.21.15.78:0/3028440026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1cc102c20 0x7fa1cc0fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.096+0000 7fa1c3fff640 1 -- 172.21.15.78:0/3028440026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1cc100b50 msgr2=0x7fa1cc0fc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.096+0000 7fa1c3fff640 1 --2- 172.21.15.78:0/3028440026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1cc100b50 0x7fa1cc0fc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.096+0000 7fa1c3fff640 1 -- 172.21.15.78:0/3028440026 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1cc1a66d0 con 0x7fa1cc1022a0 2024-08-18T23:50:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.096+0000 7fa1c3fff640 1 --2- 172.21.15.78:0/3028440026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1cc1022a0 0x7fa1cc0fcef0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400d950 tx=0x7fa1b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.097+0000 7fa1c9ffb640 1 -- 172.21.15.78:0/3028440026 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4017030 con 0x7fa1cc1022a0 2024-08-18T23:50:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.097+0000 7fa1d22ce640 1 -- 172.21.15.78:0/3028440026 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1cc1a69c0 con 0x7fa1cc1022a0 2024-08-18T23:50:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.097+0000 7fa1c9ffb640 1 -- 172.21.15.78:0/3028440026 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1b4004020 con 0x7fa1cc1022a0 2024-08-18T23:50:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.097+0000 7fa1c9ffb640 1 -- 172.21.15.78:0/3028440026 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b40047a0 con 0x7fa1cc1022a0 2024-08-18T23:50:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.097+0000 7fa1d22ce640 1 -- 172.21.15.78:0/3028440026 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1cc1a6f50 con 0x7fa1cc1022a0 2024-08-18T23:50:57.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.099+0000 7fa1c9ffb640 1 -- 172.21.15.78:0/3028440026 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa1b40041d0 con 0x7fa1cc1022a0 2024-08-18T23:50:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.099+0000 7fa1d46cc740 1 Processor -- start 2024-08-18T23:50:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.099+0000 7fa1d46cc740 1 -- start start 2024-08-18T23:50:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.099+0000 7fa1d46cc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 0x564fe0eb15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.099+0000 7fa1d46cc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0eb1b30 0x564fe0ebbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.099+0000 7fa1c9ffb640 1 --2- 172.21.15.78:0/3028440026 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa1b80a2b80 0x7fa1b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1d46cc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0ebc3f0 0x564fe0ebe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1d46cc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564fe0e16980 con 0x564fe0eb1210 2024-08-18T23:50:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1d46cc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564fe0daea70 con 0x564fe0ebc3f0 2024-08-18T23:50:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1d46cc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564fe0e12940 con 0x564fe0eb1b30 2024-08-18T23:50:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1d086b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 0x564fe0eb15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:57.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1d086b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 0x564fe0eb15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33040/0 (socket says 172.21.15.78:33040) 2024-08-18T23:50:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1d086b640 1 -- 172.21.15.78:0/190540888 learned_addr learned my addr 172.21.15.78:0/190540888 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1cbfff640 1 --2- 172.21.15.78:0/190540888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0eb1b30 0x564fe0ebbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1c3fff640 1 --2- 172.21.15.78:0/3028440026 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa1b80a2b80 0x7fa1b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1d086b640 1 --2- 172.21.15.78:0/190540888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0ebc3f0 0x564fe0ebe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1c9ffb640 1 -- 172.21.15.78:0/3028440026 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa1b40ced60 con 0x7fa1cc1022a0 2024-08-18T23:50:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1d086b640 1 -- 172.21.15.78:0/190540888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0eb1b30 msgr2=0x564fe0ebbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1d086b640 1 --2- 172.21.15.78:0/190540888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0eb1b30 0x564fe0ebbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1d086b640 1 -- 172.21.15.78:0/190540888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0ebc3f0 msgr2=0x564fe0ebe7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1d086b640 1 --2- 172.21.15.78:0/190540888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0ebc3f0 0x564fe0ebe7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1d086b640 1 -- 172.21.15.78:0/190540888 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe0e116e0 con 0x564fe0eb1210 2024-08-18T23:50:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1c3fff640 1 --2- 172.21.15.78:0/3028440026 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa1b80a2b80 0x7fa1b80a5040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009610 tx=0x7fa1b4018cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1d086b640 1 --2- 172.21.15.78:0/190540888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 0x564fe0eb15f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x564fe0eaeb20 tx=0x7fa1c4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d79e72ae6abd88 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:57.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1c27fc640 1 -- 172.21.15.78:0/190540888 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4013130 con 0x564fe0eb1210 2024-08-18T23:50:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1c27fc640 1 -- 172.21.15.78:0/190540888 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c400f940 con 0x564fe0eb1210 2024-08-18T23:50:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.100+0000 7fa1c27fc640 1 -- 172.21.15.78:0/190540888 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4005280 con 0x564fe0eb1210 2024-08-18T23:50:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.101+0000 7fa1d46cc740 1 -- 172.21.15.78:0/190540888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 msgr2=0x564fe0eb15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.101+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/190540888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 0x564fe0eb15f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x564fe0eaeb20 tx=0x7fa1c4005bd0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.101+0000 7fa1d46cc740 1 -- 172.21.15.78:0/190540888 shutdown_connections 2024-08-18T23:50:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.101+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/190540888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0ebc3f0 0x564fe0ebe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.101+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/190540888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0eb1b30 0x564fe0ebbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.101+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/190540888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 0x564fe0eb15f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.101+0000 7fa1d46cc740 1 -- 172.21.15.78:0/190540888 >> 172.21.15.78:0/190540888 conn(0x564fe0ead7a0 msgr2=0x564fe0eadb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.101+0000 7fa1d46cc740 1 -- 172.21.15.78:0/190540888 shutdown_connections 2024-08-18T23:50:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.101+0000 7fa1d46cc740 1 -- 172.21.15.78:0/190540888 wait complete. 2024-08-18T23:50:57.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.102+0000 7fa1d46cc740 1 -- start start 2024-08-18T23:50:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.102+0000 7fa1d46cc740 1 Processor -- start 2024-08-18T23:50:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.102+0000 7fa1d46cc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 0x564fe0eb9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.102+0000 7fa1d46cc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0eb1b30 0x564fe0eb9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.102+0000 7fa1d46cc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0ebc3f0 0x564fe0eb9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.102+0000 7fa1d46cc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564fe0ebefc0 con 0x564fe0eb1210 2024-08-18T23:50:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.102+0000 7fa1d46cc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564fe0dd1aa0 con 0x564fe0eb1b30 2024-08-18T23:50:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.102+0000 7fa1d46cc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564fe0ebf140 con 0x564fe0ebc3f0 2024-08-18T23:50:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.102+0000 7fa1d46cc740 10 client.0 _ll_register_callbacks cb 0x564fe0dc0e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.102+0000 7fa1d086b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 0x564fe0eb9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.102+0000 7fa1d086b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 0x564fe0eb9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33042/0 (socket says 172.21.15.78:33042) 2024-08-18T23:50:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.102+0000 7fa1d086b640 1 -- 172.21.15.78:0/527633624 learned_addr learned my addr 172.21.15.78:0/527633624 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.102+0000 7fa1cbfff640 1 --2- 172.21.15.78:0/527633624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0eb1b30 0x564fe0eb9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.102+0000 7fa1d086b640 1 --2- 172.21.15.78:0/527633624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0ebc3f0 0x564fe0eb9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:57.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.103+0000 7fa1d086b640 1 -- 172.21.15.78:0/527633624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0ebc3f0 msgr2=0x564fe0eb9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.103+0000 7fa1d086b640 1 --2- 172.21.15.78:0/527633624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0ebc3f0 0x564fe0eb9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.103+0000 7fa1d086b640 1 -- 172.21.15.78:0/527633624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0eb1b30 msgr2=0x564fe0eb9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.103+0000 7fa1d086b640 1 --2- 172.21.15.78:0/527633624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0eb1b30 0x564fe0eb9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.103+0000 7fa1d086b640 1 -- 172.21.15.78:0/527633624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fe0db5650 con 0x564fe0eb1210 2024-08-18T23:50:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.103+0000 7fa1d086b640 1 --2- 172.21.15.78:0/527633624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 0x564fe0eb9240 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4005d20 tx=0x7fa1c4009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a56d48d6bdf2c41 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.103+0000 7fa1ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:50:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.103+0000 7fa1ab7fe640 1 -- 172.21.15.78:0/527633624 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4029060 con 0x564fe0eb1210 2024-08-18T23:50:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.103+0000 7fa1ab7fe640 1 -- 172.21.15.78:0/527633624 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa1c4021650 con 0x564fe0eb1210 2024-08-18T23:50:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.103+0000 7fa1d46cc740 1 -- 172.21.15.78:0/527633624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1c400f940 con 0x564fe0eb1210 2024-08-18T23:50:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.103+0000 7fa1ab7fe640 1 -- 172.21.15.78:0/527633624 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa1c4021850 con 0x564fe0eb1210 2024-08-18T23:50:57.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.103+0000 7fa1d46cc740 10 client.16479 fetch_fsmap learned FSMap version 29 2024-08-18T23:50:57.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.103+0000 7fa1d46cc740 1 -- 172.21.15.78:0/527633624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564fe0b883c0 con 0x564fe0eb1210 2024-08-18T23:50:57.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.103+0000 7fa1ab7fe640 1 -- 172.21.15.78:0/527633624 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c400f940 con 0x564fe0eb1210 2024-08-18T23:50:57.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.103+0000 7fa1ab7fe640 1 -- 172.21.15.78:0/527633624 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fa1c4020ae0 con 0x564fe0eb1210 2024-08-18T23:50:57.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.103+0000 7fa1d46cc740 10 client.16479 fetch_fsmap finished waiting for FSMap version 29 2024-08-18T23:50:57.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.103+0000 7fa1d46cc740 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:50:57.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.103+0000 7fa1d46cc740 20 client.16479 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:57.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.103+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/527633624 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x564fe0ecbde0 0x564fe0ece1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:57.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.104+0000 7fa1d46cc740 4 client.16479 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:50:57.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.104+0000 7fa1d46cc740 1 -- 172.21.15.78:0/527633624 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fe0e23ef0 con 0x564fe0ecbde0 2024-08-18T23:50:57.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.104+0000 7fa1cbfff640 1 --2- 172.21.15.78:0/527633624 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x564fe0ecbde0 0x564fe0ece1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:57.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.104+0000 7fa1ab7fe640 10 client.16479 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:57.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.104+0000 7fa1cbfff640 1 --2- 172.21.15.78:0/527633624 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x564fe0ecbde0 0x564fe0ece1d0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72cf25a830279892 server_cookie=adf88725f70242a0 in_seq=0 out_seq=0 2024-08-18T23:50:57.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.106+0000 7fa1ab7fe640 1 -- 172.21.15.78:0/527633624 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564fe0e23ef0 con 0x564fe0ecbde0 2024-08-18T23:50:57.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.106+0000 7fa1ab7fe640 10 client.16479 handle_command_reply: tid=0 2024-08-18T23:50:57.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.106+0000 7fa1d46cc740 1 client.16479 shutdown 2024-08-18T23:50:57.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.106+0000 7fa1d46cc740 1 -- 172.21.15.78:0/527633624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 msgr2=0x564fe0eb9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:57.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.106+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/527633624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 0x564fe0eb9240 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4005d20 tx=0x7fa1c4009650 comp rx=0 tx=0).stop 2024-08-18T23:50:57.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.106+0000 7fa1d46cc740 1 -- 172.21.15.78:0/527633624 shutdown_connections 2024-08-18T23:50:57.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.106+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/527633624 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x564fe0ecbde0 0x564fe0ece1d0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.106+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/527633624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0ebc3f0 0x564fe0eb9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.106+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/527633624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0eb1b30 0x564fe0eb9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.106+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/527633624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 0x564fe0eb9240 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.106+0000 7fa1d46cc740 1 -- 172.21.15.78:0/527633624 >> 172.21.15.78:0/527633624 conn(0x564fe0ead7a0 msgr2=0x564fe0eb3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:57.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.106+0000 7fa1ab7fe640 0 client.16479 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:57.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.106+0000 7fa1d46cc740 1 -- 172.21.15.78:0/527633624 shutdown_connections 2024-08-18T23:50:57.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.106+0000 7fa1d46cc740 1 -- 172.21.15.78:0/527633624 wait complete. 2024-08-18T23:50:57.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.106+0000 7fa1d46cc740 20 client.16479 trim_cache size 0 max 16384 2024-08-18T23:50:57.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.123+0000 7fa1d46cc740 1 Processor -- start 2024-08-18T23:50:57.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.123+0000 7fa1d46cc740 1 -- start start 2024-08-18T23:50:57.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.123+0000 7fa1d46cc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0eb1210 0x564fe0ed3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:57.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.123+0000 7fa1d46cc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0eb1b30 0x564fe0ed3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:57.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.123+0000 7fa1d46cc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0ebc3f0 0x564fe0eac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:57.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.123+0000 7fa1d46cc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc1b3590 con 0x564fe0ebc3f0 2024-08-18T23:50:57.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.123+0000 7fa1d46cc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc1b0e30 con 0x564fe0eb1210 2024-08-18T23:50:57.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.123+0000 7fa1d46cc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc1aec80 con 0x564fe0eb1b30 2024-08-18T23:50:57.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.124+0000 7fa1d086b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0ebc3f0 0x564fe0eac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:57.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.124+0000 7fa1d086b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0ebc3f0 0x564fe0eac0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33050/0 (socket says 172.21.15.78:33050) 2024-08-18T23:50:57.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.124+0000 7fa1d086b640 1 -- 172.21.15.78:0/2681883342 learned_addr learned my addr 172.21.15.78:0/2681883342 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:57.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.124+0000 7fa1d086b640 1 --2- 172.21.15.78:0/2681883342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0eb1210 0x564fe0ed3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:57.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.124+0000 7fa1cbfff640 1 --2- 172.21.15.78:0/2681883342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0eb1b30 0x564fe0ed3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:57.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.124+0000 7fa1d086b640 1 -- 172.21.15.78:0/2681883342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0eb1b30 msgr2=0x564fe0ed3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:57.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.124+0000 7fa1d086b640 1 --2- 172.21.15.78:0/2681883342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0eb1b30 0x564fe0ed3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.124+0000 7fa1d086b640 1 -- 172.21.15.78:0/2681883342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0eb1210 msgr2=0x564fe0ed3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:57.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.124+0000 7fa1d086b640 1 --2- 172.21.15.78:0/2681883342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0eb1210 0x564fe0ed3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.124+0000 7fa1d086b640 1 -- 172.21.15.78:0/2681883342 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe0eb1960 con 0x564fe0ebc3f0 2024-08-18T23:50:57.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.124+0000 7fa1cbfff640 1 --2- 172.21.15.78:0/2681883342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0eb1b30 0x564fe0ed3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:50:57.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.124+0000 7fa1d086b640 1 --2- 172.21.15.78:0/2681883342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0ebc3f0 0x564fe0eac0e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4000f80 tx=0x7fa1c400fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91760d24f4dc1c72 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:57.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.124+0000 7fa1c1ffb640 1 -- 172.21.15.78:0/2681883342 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4029070 con 0x564fe0ebc3f0 2024-08-18T23:50:57.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.124+0000 7fa1c1ffb640 1 -- 172.21.15.78:0/2681883342 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c4034070 con 0x564fe0ebc3f0 2024-08-18T23:50:57.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.124+0000 7fa1c1ffb640 1 -- 172.21.15.78:0/2681883342 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4010930 con 0x564fe0ebc3f0 2024-08-18T23:50:57.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.125+0000 7fa1d46cc740 1 -- 172.21.15.78:0/2681883342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0ebc3f0 msgr2=0x564fe0eac0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:57.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.125+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/2681883342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0ebc3f0 0x564fe0eac0e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4000f80 tx=0x7fa1c400fe00 comp rx=0 tx=0).stop 2024-08-18T23:50:57.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.125+0000 7fa1d46cc740 1 -- 172.21.15.78:0/2681883342 shutdown_connections 2024-08-18T23:50:57.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.125+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/2681883342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0ebc3f0 0x564fe0eac0e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.125+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/2681883342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0eb1b30 0x564fe0ed3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.125+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/2681883342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0eb1210 0x564fe0ed3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.125+0000 7fa1d46cc740 1 -- 172.21.15.78:0/2681883342 >> 172.21.15.78:0/2681883342 conn(0x564fe0ead7a0 msgr2=0x564fe0ebb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:57.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.125+0000 7fa1d46cc740 1 -- 172.21.15.78:0/2681883342 shutdown_connections 2024-08-18T23:50:57.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.125+0000 7fa1d46cc740 1 -- 172.21.15.78:0/2681883342 wait complete. 2024-08-18T23:50:57.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.125+0000 7fa1d46cc740 1 -- start start 2024-08-18T23:50:57.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1d46cc740 1 Processor -- start 2024-08-18T23:50:57.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1d46cc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 0x564fe0ec4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:57.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1d46cc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0eb1b30 0x564fe0ec48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:57.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1d46cc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0ebc3f0 0x564fe0eb54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:57.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1d46cc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc1af2f0 con 0x564fe0eb1210 2024-08-18T23:50:57.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1d46cc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc1b0760 con 0x564fe0eb1b30 2024-08-18T23:50:57.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1d46cc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc1af9c0 con 0x564fe0ebc3f0 2024-08-18T23:50:57.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1d46cc740 10 client.0 _ll_register_callbacks cb 0x7fa1c4013560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:50:57.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1d086b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 0x564fe0ec4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:57.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1d086b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 0x564fe0ec4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33062/0 (socket says 172.21.15.78:33062) 2024-08-18T23:50:57.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1d086b640 1 -- 172.21.15.78:0/1546863605 learned_addr learned my addr 172.21.15.78:0/1546863605 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:57.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1cbfff640 1 --2- 172.21.15.78:0/1546863605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0eb1b30 0x564fe0ec48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:57.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1d086b640 1 --2- 172.21.15.78:0/1546863605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0ebc3f0 0x564fe0eb54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:57.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1d086b640 1 -- 172.21.15.78:0/1546863605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0ebc3f0 msgr2=0x564fe0eb54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:57.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1d086b640 1 --2- 172.21.15.78:0/1546863605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0ebc3f0 0x564fe0eb54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1d086b640 1 -- 172.21.15.78:0/1546863605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0eb1b30 msgr2=0x564fe0ec48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:50:57.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1d086b640 1 --2- 172.21.15.78:0/1546863605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0eb1b30 0x564fe0ec48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1d086b640 1 -- 172.21.15.78:0/1546863605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fe0eb70c0 con 0x564fe0eb1210 2024-08-18T23:50:57.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1d086b640 1 --2- 172.21.15.78:0/1546863605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 0x564fe0ec4390 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4010800 tx=0x7fa1c400fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483b61abc4264ec3 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:57.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.126+0000 7fa1ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:50:57.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.127+0000 7fa1ab7fe640 1 -- 172.21.15.78:0/1546863605 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4043060 con 0x564fe0eb1210 2024-08-18T23:50:57.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.127+0000 7fa1ab7fe640 1 -- 172.21.15.78:0/1546863605 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa1c4021bf0 con 0x564fe0eb1210 2024-08-18T23:50:57.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.127+0000 7fa1d46cc740 1 -- 172.21.15.78:0/1546863605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1c4034070 con 0x564fe0eb1210 2024-08-18T23:50:57.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.127+0000 7fa1ab7fe640 1 -- 172.21.15.78:0/1546863605 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa1c4029040 con 0x564fe0eb1210 2024-08-18T23:50:57.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.127+0000 7fa1d46cc740 10 client.16485 fetch_fsmap learned FSMap version 29 2024-08-18T23:50:57.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.127+0000 7fa1d46cc740 1 -- 172.21.15.78:0/1546863605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564fe0f35c40 con 0x564fe0eb1210 2024-08-18T23:50:57.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.127+0000 7fa1ab7fe640 1 -- 172.21.15.78:0/1546863605 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c4034070 con 0x564fe0eb1210 2024-08-18T23:50:57.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.127+0000 7fa1ab7fe640 1 -- 172.21.15.78:0/1546863605 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fa1c4020560 con 0x564fe0eb1210 2024-08-18T23:50:57.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.127+0000 7fa1d46cc740 10 client.16485 fetch_fsmap finished waiting for FSMap version 29 2024-08-18T23:50:57.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.127+0000 7fa1d46cc740 10 client.16485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:50:57.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.127+0000 7fa1d46cc740 20 client.16485 populate_metadata read hostname 'smithi078' 2024-08-18T23:50:57.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.127+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/1546863605 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x564fe0f36310 0x564fe0f38730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:57.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.127+0000 7fa1d46cc740 4 client.16485 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:50:57.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.127+0000 7fa1d46cc740 1 -- 172.21.15.78:0/1546863605 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564fe0e1e400 con 0x564fe0f36310 2024-08-18T23:50:57.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.128+0000 7fa1cbfff640 1 --2- 172.21.15.78:0/1546863605 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x564fe0f36310 0x564fe0f38730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:57.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.128+0000 7fa1cbfff640 1 --2- 172.21.15.78:0/1546863605 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x564fe0f36310 0x564fe0f38730 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=384f7a8872ae0204 server_cookie=14f4d437d75e5756 in_seq=0 out_seq=0 2024-08-18T23:50:57.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.128+0000 7fa1ab7fe640 10 client.16485 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:57.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.131+0000 7fa1ab7fe640 1 -- 172.21.15.78:0/1546863605 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564fe0e1e400 con 0x564fe0f36310 2024-08-18T23:50:57.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.131+0000 7fa1ab7fe640 10 client.16485 handle_command_reply: tid=0 2024-08-18T23:50:57.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.131+0000 7fa1d46cc740 1 client.16485 shutdown 2024-08-18T23:50:57.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.131+0000 7fa1d46cc740 1 -- 172.21.15.78:0/1546863605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 msgr2=0x564fe0ec4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:57.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.131+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/1546863605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 0x564fe0ec4390 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4010800 tx=0x7fa1c400fa60 comp rx=0 tx=0).stop 2024-08-18T23:50:57.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.131+0000 7fa1d46cc740 1 -- 172.21.15.78:0/1546863605 shutdown_connections 2024-08-18T23:50:57.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.131+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/1546863605 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x564fe0f36310 0x564fe0f38730 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.131+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/1546863605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fe0ebc3f0 0x564fe0eb54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.131+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/1546863605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fe0eb1b30 0x564fe0ec48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.131+0000 7fa1ab7fe640 0 client.16485 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:57.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.131+0000 7fa1d46cc740 1 --2- 172.21.15.78:0/1546863605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564fe0eb1210 0x564fe0ec4390 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.131+0000 7fa1d46cc740 1 -- 172.21.15.78:0/1546863605 >> 172.21.15.78:0/1546863605 conn(0x564fe0ead7a0 msgr2=0x564fe0eba7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:57.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.131+0000 7fa1d46cc740 1 -- 172.21.15.78:0/1546863605 shutdown_connections 2024-08-18T23:50:57.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.131+0000 7fa1d46cc740 1 -- 172.21.15.78:0/1546863605 wait complete. 2024-08-18T23:50:57.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.131+0000 7fa1d46cc740 20 client.16485 trim_cache size 0 max 16384 2024-08-18T23:50:57.147 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:50:57.147 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-08-18T23:50:57.147 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-08-18T23:50:57.147 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:50:57.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.132+0000 7fa1d22ce640 1 -- 172.21.15.78:0/3028440026 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa1b80a2b80 msgr2=0x7fa1b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:57.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.132+0000 7fa1d22ce640 1 --2- 172.21.15.78:0/3028440026 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa1b80a2b80 0x7fa1b80a5040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009610 tx=0x7fa1b4018cc0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.132+0000 7fa1d22ce640 1 -- 172.21.15.78:0/3028440026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1cc1022a0 msgr2=0x7fa1cc0fcef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:57.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.132+0000 7fa1d22ce640 1 --2- 172.21.15.78:0/3028440026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1cc1022a0 0x7fa1cc0fcef0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400d950 tx=0x7fa1b400de10 comp rx=0 tx=0).stop 2024-08-18T23:50:57.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.132+0000 7fa1d22ce640 1 -- 172.21.15.78:0/3028440026 shutdown_connections 2024-08-18T23:50:57.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.132+0000 7fa1d22ce640 1 --2- 172.21.15.78:0/3028440026 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa1b80a2b80 0x7fa1b80a5040 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.132+0000 7fa1d22ce640 1 --2- 172.21.15.78:0/3028440026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1cc102c20 0x7fa1cc0fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.132+0000 7fa1d22ce640 1 --2- 172.21.15.78:0/3028440026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1cc1022a0 0x7fa1cc0fcef0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.132+0000 7fa1d22ce640 1 --2- 172.21.15.78:0/3028440026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1cc100b50 0x7fa1cc0fc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:57.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.132+0000 7fa1d22ce640 1 -- 172.21.15.78:0/3028440026 >> 172.21.15.78:0/3028440026 conn(0x7fa1cc0fc500 msgr2=0x7fa1cc069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:57.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.132+0000 7fa1d22ce640 1 -- 172.21.15.78:0/3028440026 shutdown_connections 2024-08-18T23:50:57.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:57.132+0000 7fa1d22ce640 1 -- 172.21.15.78:0/3028440026 wait complete. 2024-08-18T23:50:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:57 smithi161 ceph-mon[22027]: pgmap v549: 129 pgs: 129 active+clean; 762 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 965 KiB/s wr, 682 op/s 2024-08-18T23:50:57.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:57 smithi078 ceph-mon[18440]: pgmap v549: 129 pgs: 129 active+clean; 762 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 965 KiB/s wr, 682 op/s 2024-08-18T23:50:57.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:57 smithi123 ceph-mon[21808]: pgmap v549: 129 pgs: 129 active+clean; 762 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 965 KiB/s wr, 682 op/s 2024-08-18T23:50:57.861 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-18T23:50:57.861 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:50:58.110 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:50:58.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.659+0000 7f7fb9637640 1 -- 172.21.15.78:0/2693976150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fb41007b0 msgr2=0x7f7fb4100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:58.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.659+0000 7f7fb9637640 1 --2- 172.21.15.78:0/2693976150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fb41007b0 0x7f7fb4100c10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8009930 tx=0x7f7fa802a420 comp rx=0 tx=0).stop 2024-08-18T23:50:58.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.659+0000 7f7fb9637640 1 -- 172.21.15.78:0/2693976150 shutdown_connections 2024-08-18T23:50:58.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.659+0000 7f7fb9637640 1 --2- 172.21.15.78:0/2693976150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fb4101150 0x7f7fb410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:58.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.659+0000 7f7fb9637640 1 --2- 172.21.15.78:0/2693976150 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fb41007b0 0x7f7fb4100c10 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:58.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.659+0000 7f7fb9637640 1 --2- 172.21.15.78:0/2693976150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fb41067b0 0x7f7fb4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:58.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.659+0000 7f7fb9637640 1 -- 172.21.15.78:0/2693976150 >> 172.21.15.78:0/2693976150 conn(0x7f7fb40fc4d0 msgr2=0x7f7fb40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:58.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.659+0000 7f7fb9637640 1 -- 172.21.15.78:0/2693976150 shutdown_connections 2024-08-18T23:50:58.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.659+0000 7f7fb9637640 1 -- 172.21.15.78:0/2693976150 wait complete. 2024-08-18T23:50:58.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.659+0000 7f7fb9637640 1 Processor -- start 2024-08-18T23:50:58.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.660+0000 7f7fb9637640 1 -- start start 2024-08-18T23:50:58.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.660+0000 7f7fb9637640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fb41007b0 0x7f7fb4198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:58.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.660+0000 7f7fb9637640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fb4101150 0x7f7fb4199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:58.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.660+0000 7f7fb9637640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fb41067b0 0x7f7fb41a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:58.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.660+0000 7f7fb9637640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb4110bb0 con 0x7f7fb41007b0 2024-08-18T23:50:58.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.660+0000 7f7fb9637640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb4110a30 con 0x7f7fb41067b0 2024-08-18T23:50:58.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.660+0000 7f7fb9637640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb4110d30 con 0x7f7fb4101150 2024-08-18T23:50:58.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.660+0000 7f7fb2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fb41007b0 0x7f7fb4198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:58.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.660+0000 7f7fb2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fb41007b0 0x7f7fb4198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33086/0 (socket says 172.21.15.78:33086) 2024-08-18T23:50:58.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.660+0000 7f7fb2ffd640 1 -- 172.21.15.78:0/1062632074 learned_addr learned my addr 172.21.15.78:0/1062632074 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:50:58.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.660+0000 7f7fb37fe640 1 --2- 172.21.15.78:0/1062632074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fb41067b0 0x7f7fb41a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:58.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.660+0000 7f7fb27fc640 1 --2- 172.21.15.78:0/1062632074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fb4101150 0x7f7fb4199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:58.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.660+0000 7f7fb2ffd640 1 -- 172.21.15.78:0/1062632074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fb4101150 msgr2=0x7f7fb4199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:58.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.660+0000 7f7fb2ffd640 1 --2- 172.21.15.78:0/1062632074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fb4101150 0x7f7fb4199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:58.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.660+0000 7f7fb2ffd640 1 -- 172.21.15.78:0/1062632074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fb41067b0 msgr2=0x7f7fb41a4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:58.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.660+0000 7f7fb2ffd640 1 --2- 172.21.15.78:0/1062632074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fb41067b0 0x7f7fb41a4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:58.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.660+0000 7f7fb2ffd640 1 -- 172.21.15.78:0/1062632074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb41a45a0 con 0x7f7fb41007b0 2024-08-18T23:50:58.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.661+0000 7f7fb2ffd640 1 --2- 172.21.15.78:0/1062632074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fb41007b0 0x7f7fb4198fe0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400e9f0 tx=0x7f7fa400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:58.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.661+0000 7f7f9bfff640 1 -- 172.21.15.78:0/1062632074 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa400cd80 con 0x7f7fb41007b0 2024-08-18T23:50:58.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.661+0000 7f7f9bfff640 1 -- 172.21.15.78:0/1062632074 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fa4005500 con 0x7f7fb41007b0 2024-08-18T23:50:58.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.661+0000 7f7f9bfff640 1 -- 172.21.15.78:0/1062632074 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4010690 con 0x7f7fb41007b0 2024-08-18T23:50:58.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.661+0000 7f7fb9637640 1 -- 172.21.15.78:0/1062632074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb41a4770 con 0x7f7fb41007b0 2024-08-18T23:50:58.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.661+0000 7f7fb9637640 1 -- 172.21.15.78:0/1062632074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb41a4c30 con 0x7f7fb41007b0 2024-08-18T23:50:58.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.662+0000 7f7fb9637640 1 -- 172.21.15.78:0/1062632074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fb4100c10 con 0x7f7fb41007b0 2024-08-18T23:50:58.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.663+0000 7f7f9bfff640 1 -- 172.21.15.78:0/1062632074 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7fa40046c0 con 0x7f7fb41007b0 2024-08-18T23:50:58.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.663+0000 7f7f9bfff640 1 --2- 172.21.15.78:0/1062632074 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7f9c0a2b80 0x7f7f9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:50:58.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.663+0000 7f7fb27fc640 1 --2- 172.21.15.78:0/1062632074 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7f9c0a2b80 0x7f7f9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:50:58.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.663+0000 7f7f9bfff640 1 -- 172.21.15.78:0/1062632074 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f7fa40cf770 con 0x7f7fb41007b0 2024-08-18T23:50:58.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.664+0000 7f7fb27fc640 1 --2- 172.21.15.78:0/1062632074 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7f9c0a2b80 0x7f7f9c0a5040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8009900 tx=0x7f7fa80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:50:58.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.668+0000 7f7f9bfff640 1 -- 172.21.15.78:0/1062632074 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7fa409c340 con 0x7f7fb41007b0 2024-08-18T23:50:58.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.883+0000 7f7fb9637640 1 -- 172.21.15.78:0/1062632074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fb4108d20 con 0x7f7fb41007b0 2024-08-18T23:50:58.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.884+0000 7f7f9bfff640 1 -- 172.21.15.78:0/1062632074 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11934 (secure 0 0 0) 0x7f7fa409bc40 con 0x7f7fb41007b0 2024-08-18T23:50:58.885 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:50:58.885 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"btime":"2024-08-18T23:50:51:349107+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:50:50.645203+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24428,24427]},"id":1}]} 2024-08-18T23:50:58.886 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2024-08-18T23:50:58.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.886+0000 7f7fb9637640 1 -- 172.21.15.78:0/1062632074 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7f9c0a2b80 msgr2=0x7f7f9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:58.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.887+0000 7f7fb9637640 1 --2- 172.21.15.78:0/1062632074 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7f9c0a2b80 0x7f7f9c0a5040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8009900 tx=0x7f7fa80058d0 comp rx=0 tx=0).stop 2024-08-18T23:50:58.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.887+0000 7f7fb9637640 1 -- 172.21.15.78:0/1062632074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fb41007b0 msgr2=0x7f7fb4198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:58.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.887+0000 7f7fb9637640 1 --2- 172.21.15.78:0/1062632074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fb41007b0 0x7f7fb4198fe0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400e9f0 tx=0x7f7fa400edf0 comp rx=0 tx=0).stop 2024-08-18T23:50:58.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.887+0000 7f7fb9637640 1 -- 172.21.15.78:0/1062632074 shutdown_connections 2024-08-18T23:50:58.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.887+0000 7f7fb9637640 1 --2- 172.21.15.78:0/1062632074 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7f9c0a2b80 0x7f7f9c0a5040 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:58.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.887+0000 7f7fb9637640 1 --2- 172.21.15.78:0/1062632074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fb41067b0 0x7f7fb41a4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:58.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.887+0000 7f7fb9637640 1 --2- 172.21.15.78:0/1062632074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fb4101150 0x7f7fb4199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.887+0000 7f7fb9637640 1 --2- 172.21.15.78:0/1062632074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fb41007b0 0x7f7fb4198fe0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.887+0000 7f7fb9637640 1 -- 172.21.15.78:0/1062632074 >> 172.21.15.78:0/1062632074 conn(0x7f7fb40fc4d0 msgr2=0x7f7fb40fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.887+0000 7f7fb9637640 1 -- 172.21.15.78:0/1062632074 shutdown_connections 2024-08-18T23:50:58.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:58.887+0000 7f7fb9637640 1 -- 172.21.15.78:0/1062632074 wait complete. 2024-08-18T23:50:59.565 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:59 smithi078 ceph-mon[18440]: pgmap v550: 129 pgs: 129 active+clean; 746 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 884 KiB/s wr, 518 op/s 2024-08-18T23:50:59.565 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:50:59 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1062632074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:50:59.594 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-18T23:50:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:59 smithi161 ceph-mon[22027]: pgmap v550: 129 pgs: 129 active+clean; 746 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 884 KiB/s wr, 518 op/s 2024-08-18T23:50:59.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:50:59 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1062632074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:50:59.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.693+0000 7f4cb77fe640 1 -- 172.21.15.78:0/33852276 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5605f66734f0 con 0x5605f678c010 2024-08-18T23:50:59.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.693+0000 7f4cb77fe640 10 client.16467 handle_command_reply: tid=0 2024-08-18T23:50:59.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.693+0000 7f4ce0720740 1 client.16467 shutdown 2024-08-18T23:50:59.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.693+0000 7f4ce0720740 1 -- 172.21.15.78:0/33852276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 msgr2=0x5605f67151a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:59.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.693+0000 7f4ce0720740 1 --2- 172.21.15.78:0/33852276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 0x5605f67151a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009a30 tx=0x7f4cd0019f90 comp rx=0 tx=0).stop 2024-08-18T23:50:59.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.694+0000 7f4ce0720740 1 -- 172.21.15.78:0/33852276 shutdown_connections 2024-08-18T23:50:59.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.694+0000 7f4ce0720740 1 --2- 172.21.15.78:0/33852276 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5605f678c010 0x5605f678e400 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:59.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.694+0000 7f4ce0720740 1 --2- 172.21.15.78:0/33852276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605f6711310 0x5605f67151a0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:59.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.694+0000 7f4ce0720740 1 --2- 172.21.15.78:0/33852276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5605f6706a50 0x5605f6714c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:59.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.694+0000 7f4ce0720740 1 --2- 172.21.15.78:0/33852276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f6706130 0x5605f6719930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:59.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.694+0000 7f4cb77fe640 0 client.16467 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:50:59.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.694+0000 7f4ce0720740 1 -- 172.21.15.78:0/33852276 >> 172.21.15.78:0/33852276 conn(0x5605f67026c0 msgr2=0x5605f670e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:59.696 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:50:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 29, 2024-08-18T23:50:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24452, 2024-08-18T23:50:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 22, 2024-08-18T23:50:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-08-18T23:50:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "10964c59": { 2024-08-18T23:50:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 22, 2024-08-18T23:50:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-08-18T23:50:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:50:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-08-18T23:50:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:50:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:50:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-08-18T23:50:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 139.2, 2024-08-18T23:50:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-08-18T23:50:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-08-18T23:50:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-08-18T23:50:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:50:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-08-18T23:50:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:50:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:50:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:50:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:50:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:50:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:50:59.698 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:50:59.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.694+0000 7f4ce0720740 1 -- 172.21.15.78:0/33852276 shutdown_connections 2024-08-18T23:50:59.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.694+0000 7f4ce0720740 1 -- 172.21.15.78:0/33852276 wait complete. 2024-08-18T23:50:59.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.694+0000 7f4ce0720740 20 client.16467 trim_cache size 0 max 16384 2024-08-18T23:50:59.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.695+0000 7f4cde322640 1 -- 172.21.15.78:0/2017631855 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4cc40a2b80 msgr2=0x7f4cc40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:59.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.695+0000 7f4cde322640 1 --2- 172.21.15.78:0/2017631855 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4cc40a2b80 0x7f4cc40a5040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8069500 tx=0x7f4cc000aa90 comp rx=0 tx=0).stop 2024-08-18T23:50:59.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.695+0000 7f4cde322640 1 -- 172.21.15.78:0/2017631855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd8108030 msgr2=0x7f4cd819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:50:59.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.695+0000 7f4cde322640 1 --2- 172.21.15.78:0/2017631855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd8108030 0x7f4cd819ccd0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009fd0 tx=0x7f4cd000b260 comp rx=0 tx=0).stop 2024-08-18T23:50:59.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.695+0000 7f4cde322640 1 -- 172.21.15.78:0/2017631855 shutdown_connections 2024-08-18T23:50:59.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.695+0000 7f4cde322640 1 --2- 172.21.15.78:0/2017631855 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4cc40a2b80 0x7f4cc40a5040 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:59.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.695+0000 7f4cde322640 1 --2- 172.21.15.78:0/2017631855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cd8108030 0x7f4cd819ccd0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:59.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.695+0000 7f4cde322640 1 --2- 172.21.15.78:0/2017631855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cd81034b0 0x7f4cd8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:59.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.695+0000 7f4cde322640 1 --2- 172.21.15.78:0/2017631855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd8102b00 0x7f4cd8198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:50:59.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.695+0000 7f4cde322640 1 -- 172.21.15.78:0/2017631855 >> 172.21.15.78:0/2017631855 conn(0x7f4cd80fc500 msgr2=0x7f4cd8105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:50:59.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.695+0000 7f4cde322640 1 -- 172.21.15.78:0/2017631855 shutdown_connections 2024-08-18T23:50:59.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:50:59.695+0000 7f4cde322640 1 -- 172.21.15.78:0/2017631855 wait complete. 2024-08-18T23:50:59.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:59 smithi123 ceph-mon[21808]: pgmap v550: 129 pgs: 129 active+clean; 746 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 884 KiB/s wr, 518 op/s 2024-08-18T23:50:59.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:50:59 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1062632074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:51:00.403 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '10964c59', quiesce took 13.3 seconds. Will release after: 5.899999999999999 2024-08-18T23:51:00.447 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:51:00.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.953+0000 7fb086e93640 1 -- 172.21.15.78:0/1029463213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb080103410 msgr2=0x7fb0801078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:00.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.953+0000 7fb086e93640 1 --2- 172.21.15.78:0/1029463213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb080103410 0x7fb0801078c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009930 tx=0x7fb06c02a420 comp rx=0 tx=0).stop 2024-08-18T23:51:00.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.953+0000 7fb086e93640 1 -- 172.21.15.78:0/1029463213 shutdown_connections 2024-08-18T23:51:00.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.953+0000 7fb086e93640 1 --2- 172.21.15.78:0/1029463213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb080107fd0 0x7fb08010e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.953+0000 7fb086e93640 1 --2- 172.21.15.78:0/1029463213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb080103410 0x7fb0801078c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.953+0000 7fb086e93640 1 --2- 172.21.15.78:0/1029463213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb080102a60 0x7fb080102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.953+0000 7fb086e93640 1 -- 172.21.15.78:0/1029463213 >> 172.21.15.78:0/1029463213 conn(0x7fb0800fc500 msgr2=0x7fb0800fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:00.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.953+0000 7fb086e93640 1 -- 172.21.15.78:0/1029463213 shutdown_connections 2024-08-18T23:51:00.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.953+0000 7fb086e93640 1 -- 172.21.15.78:0/1029463213 wait complete. 2024-08-18T23:51:00.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.954+0000 7fb086e93640 1 Processor -- start 2024-08-18T23:51:00.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.954+0000 7fb086e93640 1 -- start start 2024-08-18T23:51:00.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.954+0000 7fb086e93640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb080102a60 0x7fb080198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:00.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.954+0000 7fb086e93640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb080103410 0x7fb0801989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:00.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.954+0000 7fb086e93640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb080107fd0 0x7fb08019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:00.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.954+0000 7fb086e93640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb080111e70 con 0x7fb080102a60 2024-08-18T23:51:00.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.954+0000 7fb086e93640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb080111cf0 con 0x7fb080103410 2024-08-18T23:51:00.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.954+0000 7fb086e93640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb080111ff0 con 0x7fb080107fd0 2024-08-18T23:51:00.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.954+0000 7fb084c2f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb080102a60 0x7fb080198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:00.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.954+0000 7fb084c2f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb080102a60 0x7fb080198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33100/0 (socket says 172.21.15.78:33100) 2024-08-18T23:51:00.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.954+0000 7fb084c2f640 1 -- 172.21.15.78:0/3325859911 learned_addr learned my addr 172.21.15.78:0/3325859911 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:00.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.954+0000 7fb085430640 1 --2- 172.21.15.78:0/3325859911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb080107fd0 0x7fb08019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:00.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.954+0000 7fb084c2f640 1 -- 172.21.15.78:0/3325859911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb080107fd0 msgr2=0x7fb08019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:00.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.954+0000 7fb084c2f640 1 --2- 172.21.15.78:0/3325859911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb080107fd0 0x7fb08019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.954+0000 7fb084c2f640 1 -- 172.21.15.78:0/3325859911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb080103410 msgr2=0x7fb0801989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:00.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.954+0000 7fb084c2f640 1 --2- 172.21.15.78:0/3325859911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb080103410 0x7fb0801989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.954+0000 7fb084c2f640 1 -- 172.21.15.78:0/3325859911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08019d420 con 0x7fb080102a60 2024-08-18T23:51:00.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.954+0000 7fb085430640 1 --2- 172.21.15.78:0/3325859911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb080107fd0 0x7fb08019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:51:00.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.955+0000 7fb084c2f640 1 --2- 172.21.15.78:0/3325859911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb080102a60 0x7fb080198470 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb07400b4a0 tx=0x7fb07400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:00.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.955+0000 7fb07dffb640 1 -- 172.21.15.78:0/3325859911 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb074014070 con 0x7fb080102a60 2024-08-18T23:51:00.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.955+0000 7fb07dffb640 1 -- 172.21.15.78:0/3325859911 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb074010af0 con 0x7fb080102a60 2024-08-18T23:51:00.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.955+0000 7fb086e93640 1 -- 172.21.15.78:0/3325859911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08019d710 con 0x7fb080102a60 2024-08-18T23:51:00.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.955+0000 7fb07dffb640 1 -- 172.21.15.78:0/3325859911 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb074017d10 con 0x7fb080102a60 2024-08-18T23:51:00.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.955+0000 7fb086e93640 1 -- 172.21.15.78:0/3325859911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0801a4ff0 con 0x7fb080102a60 2024-08-18T23:51:00.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.957+0000 7fb07dffb640 1 -- 172.21.15.78:0/3325859911 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb074017070 con 0x7fb080102a60 2024-08-18T23:51:00.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb089291740 1 Processor -- start 2024-08-18T23:51:00.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb089291740 1 -- start start 2024-08-18T23:51:00.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb089291740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d4ff0 0x5565395d53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:00.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb089291740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d5910 0x5565395dfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:00.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb089291740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395e01d0 0x5565395e25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:00.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb089291740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55653953aa40 con 0x5565395d4ff0 2024-08-18T23:51:00.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb089291740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5565394d29d0 con 0x5565395e01d0 2024-08-18T23:51:00.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb089291740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556539536a00 con 0x5565395d5910 2024-08-18T23:51:00.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb085430640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d4ff0 0x5565395d53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:00.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb07dffb640 1 --2- 172.21.15.78:0/3325859911 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb0700a2b80 0x7fb0700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:00.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb085430640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d4ff0 0x5565395d53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33102/0 (socket says 172.21.15.78:33102) 2024-08-18T23:51:00.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb085430640 1 -- 172.21.15.78:0/2368838476 learned_addr learned my addr 172.21.15.78:0/2368838476 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:00.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb07ffff640 1 --2- 172.21.15.78:0/2368838476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395e01d0 0x5565395e25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:00.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb085430640 1 --2- 172.21.15.78:0/3325859911 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb0700a2b80 0x7fb0700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:00.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb07dffb640 1 -- 172.21.15.78:0/3325859911 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fb0740ce7f0 con 0x7fb080102a60 2024-08-18T23:51:00.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb084c2f640 1 --2- 172.21.15.78:0/2368838476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d5910 0x5565395dfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:00.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb085430640 1 -- 172.21.15.78:0/2368838476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d5910 msgr2=0x5565395dfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:00.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb085430640 1 --2- 172.21.15.78:0/2368838476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d5910 0x5565395dfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb085430640 1 -- 172.21.15.78:0/2368838476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395e01d0 msgr2=0x5565395e25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:00.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb085430640 1 --2- 172.21.15.78:0/2368838476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395e01d0 0x5565395e25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb085430640 1 -- 172.21.15.78:0/2368838476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565394d9650 con 0x5565395d4ff0 2024-08-18T23:51:00.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.958+0000 7fb084c2f640 1 --2- 172.21.15.78:0/2368838476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d5910 0x5565395dfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:51:00.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.959+0000 7fb085430640 1 --2- 172.21.15.78:0/2368838476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d4ff0 0x5565395d53d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb078010a30 tx=0x7fb078010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc432a786a295258 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:00.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.959+0000 7fb06affd640 1 -- 172.21.15.78:0/2368838476 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078020070 con 0x5565395d4ff0 2024-08-18T23:51:00.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.959+0000 7fb06affd640 1 -- 172.21.15.78:0/2368838476 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb07801bbd0 con 0x5565395d4ff0 2024-08-18T23:51:00.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.959+0000 7fb06affd640 1 -- 172.21.15.78:0/2368838476 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07800e570 con 0x5565395d4ff0 2024-08-18T23:51:00.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.959+0000 7fb085430640 1 --2- 172.21.15.78:0/3325859911 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb0700a2b80 0x7fb0700a5040 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb07801d000 tx=0x7fb078019000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:00.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.959+0000 7fb089291740 1 -- 172.21.15.78:0/2368838476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d4ff0 msgr2=0x5565395d53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:00.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.959+0000 7fb089291740 1 --2- 172.21.15.78:0/2368838476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d4ff0 0x5565395d53d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb078010a30 tx=0x7fb078010ef0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.959+0000 7fb089291740 1 -- 172.21.15.78:0/2368838476 shutdown_connections 2024-08-18T23:51:00.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.959+0000 7fb089291740 1 --2- 172.21.15.78:0/2368838476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395e01d0 0x5565395e25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.959+0000 7fb089291740 1 --2- 172.21.15.78:0/2368838476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d5910 0x5565395dfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.959+0000 7fb089291740 1 --2- 172.21.15.78:0/2368838476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d4ff0 0x5565395d53d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.959+0000 7fb089291740 1 -- 172.21.15.78:0/2368838476 >> 172.21.15.78:0/2368838476 conn(0x5565395d1580 msgr2=0x5565395d1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:00.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.960+0000 7fb089291740 1 -- 172.21.15.78:0/2368838476 shutdown_connections 2024-08-18T23:51:00.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.960+0000 7fb089291740 1 -- 172.21.15.78:0/2368838476 wait complete. 2024-08-18T23:51:00.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.960+0000 7fb089291740 1 -- start start 2024-08-18T23:51:00.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.960+0000 7fb089291740 1 Processor -- start 2024-08-18T23:51:00.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.960+0000 7fb089291740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d4ff0 0x5565395dd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:00.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.960+0000 7fb089291740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395d5910 0x5565395dd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:00.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.960+0000 7fb089291740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395e01d0 0x5565395ddab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:00.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.960+0000 7fb089291740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5565395e2da0 con 0x5565395e01d0 2024-08-18T23:51:00.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.960+0000 7fb089291740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5565394f5b20 con 0x5565395d5910 2024-08-18T23:51:00.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.960+0000 7fb089291740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5565395e2f20 con 0x5565395d4ff0 2024-08-18T23:51:00.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.960+0000 7fb089291740 10 client.0 _ll_register_callbacks cb 0x5565394f3bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:00.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.961+0000 7fb085430640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395e01d0 0x5565395ddab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:00.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.961+0000 7fb085430640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395e01d0 0x5565395ddab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33110/0 (socket says 172.21.15.78:33110) 2024-08-18T23:51:00.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.961+0000 7fb085430640 1 -- 172.21.15.78:0/660344729 learned_addr learned my addr 172.21.15.78:0/660344729 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:00.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.961+0000 7fb07ffff640 1 --2- 172.21.15.78:0/660344729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395d5910 0x5565395dd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:00.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.961+0000 7fb084c2f640 1 --2- 172.21.15.78:0/660344729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d4ff0 0x5565395dd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:00.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.961+0000 7fb085430640 1 -- 172.21.15.78:0/660344729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d4ff0 msgr2=0x5565395dd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:00.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.961+0000 7fb085430640 1 --2- 172.21.15.78:0/660344729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d4ff0 0x5565395dd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.961+0000 7fb085430640 1 -- 172.21.15.78:0/660344729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395d5910 msgr2=0x5565395dd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:00.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.961+0000 7fb085430640 1 --2- 172.21.15.78:0/660344729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395d5910 0x5565395dd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.961+0000 7fb085430640 1 -- 172.21.15.78:0/660344729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565392ac3c0 con 0x5565395e01d0 2024-08-18T23:51:00.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.961+0000 7fb085430640 1 --2- 172.21.15.78:0/660344729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395e01d0 0x5565395ddab0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb078004a80 tx=0x7fb07800da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd10b2a6b242eefe server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:00.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.961+0000 7fb05ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:51:00.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.961+0000 7fb05ffff640 1 -- 172.21.15.78:0/660344729 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078033060 con 0x5565395e01d0 2024-08-18T23:51:00.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.961+0000 7fb05ffff640 1 -- 172.21.15.78:0/660344729 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fb07800e350 con 0x5565395e01d0 2024-08-18T23:51:00.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.961+0000 7fb089291740 1 -- 172.21.15.78:0/660344729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb07801bbd0 con 0x5565395e01d0 2024-08-18T23:51:00.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.961+0000 7fb05ffff640 1 -- 172.21.15.78:0/660344729 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb078020050 con 0x5565395e01d0 2024-08-18T23:51:00.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.961+0000 7fb089291740 10 client.16503 fetch_fsmap learned FSMap version 29 2024-08-18T23:51:00.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.961+0000 7fb089291740 1 -- 172.21.15.78:0/660344729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5565395ee8a0 con 0x5565395e01d0 2024-08-18T23:51:00.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.962+0000 7fb05ffff640 1 -- 172.21.15.78:0/660344729 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb07801bbd0 con 0x5565395e01d0 2024-08-18T23:51:00.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.962+0000 7fb05ffff640 1 -- 172.21.15.78:0/660344729 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fb07800e8a0 con 0x5565395e01d0 2024-08-18T23:51:00.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.962+0000 7fb089291740 10 client.16503 fetch_fsmap finished waiting for FSMap version 29 2024-08-18T23:51:00.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.962+0000 7fb089291740 10 client.16503 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:51:00.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.962+0000 7fb089291740 20 client.16503 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:00.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.962+0000 7fb089291740 1 --2- 172.21.15.78:0/660344729 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5565395efd00 0x5565395f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:00.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.962+0000 7fb089291740 4 client.16503 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:51:00.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.962+0000 7fb089291740 1 -- 172.21.15.78:0/660344729 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556539547fb0 con 0x5565395efd00 2024-08-18T23:51:00.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.962+0000 7fb084c2f640 1 --2- 172.21.15.78:0/660344729 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5565395efd00 0x5565395f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:00.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.963+0000 7fb084c2f640 1 --2- 172.21.15.78:0/660344729 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5565395efd00 0x5565395f20f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb8e58a80e866623 server_cookie=28c00b578d51bae4 in_seq=0 out_seq=0 2024-08-18T23:51:00.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.963+0000 7fb05ffff640 10 client.16503 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:51:00.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.963+0000 7fb05ffff640 1 -- 172.21.15.78:0/660344729 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556539547fb0 con 0x5565395efd00 2024-08-18T23:51:00.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.963+0000 7fb05ffff640 10 client.16503 handle_command_reply: tid=0 2024-08-18T23:51:00.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.963+0000 7fb089291740 1 client.16503 shutdown 2024-08-18T23:51:00.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.964+0000 7fb089291740 1 -- 172.21.15.78:0/660344729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395e01d0 msgr2=0x5565395ddab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:00.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.964+0000 7fb089291740 1 --2- 172.21.15.78:0/660344729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395e01d0 0x5565395ddab0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb078004a80 tx=0x7fb07800da10 comp rx=0 tx=0).stop 2024-08-18T23:51:00.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.964+0000 7fb089291740 1 -- 172.21.15.78:0/660344729 shutdown_connections 2024-08-18T23:51:00.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.964+0000 7fb089291740 1 --2- 172.21.15.78:0/660344729 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5565395efd00 0x5565395f20f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.964+0000 7fb089291740 1 --2- 172.21.15.78:0/660344729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395e01d0 0x5565395ddab0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.964+0000 7fb089291740 1 --2- 172.21.15.78:0/660344729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395d5910 0x5565395dd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.964+0000 7fb05ffff640 0 client.16503 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:51:00.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.964+0000 7fb089291740 1 --2- 172.21.15.78:0/660344729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d4ff0 0x5565395dd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.964+0000 7fb089291740 1 -- 172.21.15.78:0/660344729 >> 172.21.15.78:0/660344729 conn(0x5565395d1580 msgr2=0x5565395d7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:00.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.964+0000 7fb089291740 1 -- 172.21.15.78:0/660344729 shutdown_connections 2024-08-18T23:51:00.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.964+0000 7fb089291740 1 -- 172.21.15.78:0/660344729 wait complete. 2024-08-18T23:51:00.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.964+0000 7fb089291740 20 client.16503 trim_cache size 0 max 16384 2024-08-18T23:51:00.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.977+0000 7fb089291740 1 Processor -- start 2024-08-18T23:51:00.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.977+0000 7fb089291740 1 -- start start 2024-08-18T23:51:00.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.977+0000 7fb089291740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d4ff0 0x5565395d7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:00.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.977+0000 7fb089291740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d5910 0x5565395f7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:00.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.977+0000 7fb089291740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395e01d0 0x5565395f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:00.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.977+0000 7fb089291740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801b1590 con 0x5565395d5910 2024-08-18T23:51:00.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.977+0000 7fb089291740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801aee30 con 0x5565395e01d0 2024-08-18T23:51:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.977+0000 7fb089291740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801acc80 con 0x5565395d4ff0 2024-08-18T23:51:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.978+0000 7fb07ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d5910 0x5565395f7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.978+0000 7fb07ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d5910 0x5565395f7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33120/0 (socket says 172.21.15.78:33120) 2024-08-18T23:51:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.978+0000 7fb07ffff640 1 -- 172.21.15.78:0/391282197 learned_addr learned my addr 172.21.15.78:0/391282197 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.978+0000 7fb084c2f640 1 --2- 172.21.15.78:0/391282197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d4ff0 0x5565395d7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.978+0000 7fb085430640 1 --2- 172.21.15.78:0/391282197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395e01d0 0x5565395f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.978+0000 7fb07ffff640 1 -- 172.21.15.78:0/391282197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d4ff0 msgr2=0x5565395d7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.978+0000 7fb07ffff640 1 --2- 172.21.15.78:0/391282197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d4ff0 0x5565395d7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.978+0000 7fb07ffff640 1 -- 172.21.15.78:0/391282197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395e01d0 msgr2=0x5565395f7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.978+0000 7fb07ffff640 1 --2- 172.21.15.78:0/391282197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395e01d0 0x5565395f7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.978+0000 7fb07ffff640 1 -- 172.21.15.78:0/391282197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565395d6500 con 0x5565395d5910 2024-08-18T23:51:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.978+0000 7fb085430640 1 --2- 172.21.15.78:0/391282197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395e01d0 0x5565395f7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:51:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.978+0000 7fb07ffff640 1 --2- 172.21.15.78:0/391282197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d5910 0x5565395f7860 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009900 tx=0x7fb06c0315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5ed229c4f5b1d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:00.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.978+0000 7fb06a7fc640 1 -- 172.21.15.78:0/391282197 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c035040 con 0x5565395d5910 2024-08-18T23:51:00.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.978+0000 7fb06a7fc640 1 -- 172.21.15.78:0/391282197 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb06c0319e0 con 0x5565395d5910 2024-08-18T23:51:00.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.978+0000 7fb06a7fc640 1 -- 172.21.15.78:0/391282197 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c03d450 con 0x5565395d5910 2024-08-18T23:51:00.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.979+0000 7fb089291740 1 -- 172.21.15.78:0/391282197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d5910 msgr2=0x5565395f7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:00.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.979+0000 7fb089291740 1 --2- 172.21.15.78:0/391282197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d5910 0x5565395f7860 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009900 tx=0x7fb06c0315b0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.979+0000 7fb089291740 1 -- 172.21.15.78:0/391282197 shutdown_connections 2024-08-18T23:51:00.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.979+0000 7fb089291740 1 --2- 172.21.15.78:0/391282197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395e01d0 0x5565395f7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.979+0000 7fb089291740 1 --2- 172.21.15.78:0/391282197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d5910 0x5565395f7860 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.979+0000 7fb089291740 1 --2- 172.21.15.78:0/391282197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d4ff0 0x5565395d7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.979+0000 7fb089291740 1 -- 172.21.15.78:0/391282197 >> 172.21.15.78:0/391282197 conn(0x5565395d1580 msgr2=0x5565395d6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:00.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.979+0000 7fb089291740 1 -- 172.21.15.78:0/391282197 shutdown_connections 2024-08-18T23:51:00.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.979+0000 7fb089291740 1 -- 172.21.15.78:0/391282197 wait complete. 2024-08-18T23:51:00.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.980+0000 7fb089291740 1 -- start start 2024-08-18T23:51:00.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.980+0000 7fb089291740 1 Processor -- start 2024-08-18T23:51:00.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.980+0000 7fb089291740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d4ff0 0x5565395e8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:00.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.980+0000 7fb089291740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d5910 0x5565395e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:00.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.980+0000 7fb089291740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395e01d0 0x5565395d92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:00.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.980+0000 7fb089291740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801ad2f0 con 0x5565395d5910 2024-08-18T23:51:00.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.980+0000 7fb089291740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801ae760 con 0x5565395e01d0 2024-08-18T23:51:00.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.980+0000 7fb089291740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801ad9c0 con 0x5565395d4ff0 2024-08-18T23:51:00.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.980+0000 7fb089291740 10 client.0 _ll_register_callbacks cb 0x7fb07801e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:00.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.980+0000 7fb07ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d5910 0x5565395e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:00.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.980+0000 7fb07ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d5910 0x5565395e86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33134/0 (socket says 172.21.15.78:33134) 2024-08-18T23:51:00.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.980+0000 7fb07ffff640 1 -- 172.21.15.78:0/2671744994 learned_addr learned my addr 172.21.15.78:0/2671744994 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:00.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.981+0000 7fb084c2f640 1 --2- 172.21.15.78:0/2671744994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d4ff0 0x5565395e8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:00.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.981+0000 7fb085430640 1 --2- 172.21.15.78:0/2671744994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395e01d0 0x5565395d92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:00.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.981+0000 7fb07ffff640 1 -- 172.21.15.78:0/2671744994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d4ff0 msgr2=0x5565395e8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:00.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.981+0000 7fb07ffff640 1 --2- 172.21.15.78:0/2671744994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d4ff0 0x5565395e8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.981+0000 7fb07ffff640 1 -- 172.21.15.78:0/2671744994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395e01d0 msgr2=0x5565395d92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:00.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.981+0000 7fb07ffff640 1 --2- 172.21.15.78:0/2671744994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395e01d0 0x5565395d92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:00.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.981+0000 7fb07ffff640 1 -- 172.21.15.78:0/2671744994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565395e8cf0 con 0x5565395d5910 2024-08-18T23:51:00.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.981+0000 7fb084c2f640 1 --2- 172.21.15.78:0/2671744994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d4ff0 0x5565395e8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:51:00.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.981+0000 7fb085430640 1 --2- 172.21.15.78:0/2671744994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395e01d0 0x5565395d92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:51:00.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.981+0000 7fb07ffff640 1 --2- 172.21.15.78:0/2671744994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d5910 0x5565395e86b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb06c006010 tx=0x7fb06c0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed601a93cb653068 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:00.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.981+0000 7fb05ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:51:00.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.981+0000 7fb05ffff640 1 -- 172.21.15.78:0/2671744994 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c053020 con 0x5565395d5910 2024-08-18T23:51:00.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.981+0000 7fb05ffff640 1 -- 172.21.15.78:0/2671744994 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fb06c003870 con 0x5565395d5910 2024-08-18T23:51:00.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.981+0000 7fb089291740 1 -- 172.21.15.78:0/2671744994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb06c0319e0 con 0x5565395d5910 2024-08-18T23:51:00.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.981+0000 7fb05ffff640 1 -- 172.21.15.78:0/2671744994 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb06c05e050 con 0x5565395d5910 2024-08-18T23:51:00.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.981+0000 7fb089291740 10 client.16509 fetch_fsmap learned FSMap version 29 2024-08-18T23:51:00.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.981+0000 7fb089291740 1 -- 172.21.15.78:0/2671744994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556539659d60 con 0x5565395d5910 2024-08-18T23:51:00.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.982+0000 7fb05ffff640 1 -- 172.21.15.78:0/2671744994 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb06c0319e0 con 0x5565395d5910 2024-08-18T23:51:00.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.982+0000 7fb05ffff640 1 -- 172.21.15.78:0/2671744994 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fb06c04e030 con 0x5565395d5910 2024-08-18T23:51:00.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.982+0000 7fb089291740 10 client.16509 fetch_fsmap finished waiting for FSMap version 29 2024-08-18T23:51:00.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.982+0000 7fb089291740 10 client.16509 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:51:00.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.982+0000 7fb089291740 20 client.16509 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:00.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.982+0000 7fb089291740 1 --2- 172.21.15.78:0/2671744994 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55653965a500 0x55653965c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:00.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.982+0000 7fb089291740 4 client.16509 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:51:00.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.982+0000 7fb089291740 1 -- 172.21.15.78:0/2671744994 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565395424c0 con 0x55653965a500 2024-08-18T23:51:00.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.982+0000 7fb084c2f640 1 --2- 172.21.15.78:0/2671744994 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55653965a500 0x55653965c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:00.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.983+0000 7fb05ffff640 10 client.16509 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:51:01.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.983+0000 7fb084c2f640 1 --2- 172.21.15.78:0/2671744994 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55653965a500 0x55653965c920 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd0ed04d3ae6038 server_cookie=fd771616b4bdfdc7 in_seq=0 out_seq=0 2024-08-18T23:51:01.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.983+0000 7fb05ffff640 1 -- 172.21.15.78:0/2671744994 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565395424c0 con 0x55653965a500 2024-08-18T23:51:01.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.983+0000 7fb05ffff640 10 client.16509 handle_command_reply: tid=0 2024-08-18T23:51:01.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.983+0000 7fb089291740 1 client.16509 shutdown 2024-08-18T23:51:01.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.983+0000 7fb089291740 1 -- 172.21.15.78:0/2671744994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d5910 msgr2=0x5565395e86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:01.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.983+0000 7fb089291740 1 --2- 172.21.15.78:0/2671744994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d5910 0x5565395e86b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb06c006010 tx=0x7fb06c0317e0 comp rx=0 tx=0).stop 2024-08-18T23:51:01.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.983+0000 7fb089291740 1 -- 172.21.15.78:0/2671744994 shutdown_connections 2024-08-18T23:51:01.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.983+0000 7fb089291740 1 --2- 172.21.15.78:0/2671744994 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55653965a500 0x55653965c920 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:01.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.983+0000 7fb089291740 1 --2- 172.21.15.78:0/2671744994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565395e01d0 0x5565395d92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:01.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.983+0000 7fb089291740 1 --2- 172.21.15.78:0/2671744994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565395d5910 0x5565395e86b0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:01.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.983+0000 7fb089291740 1 --2- 172.21.15.78:0/2671744994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5565395d4ff0 0x5565395e8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:01.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.983+0000 7fb089291740 1 -- 172.21.15.78:0/2671744994 >> 172.21.15.78:0/2671744994 conn(0x5565395d1580 msgr2=0x5565395d7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:01.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.983+0000 7fb05ffff640 0 client.16509 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:51:01.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.983+0000 7fb089291740 1 -- 172.21.15.78:0/2671744994 shutdown_connections 2024-08-18T23:51:01.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.983+0000 7fb089291740 1 -- 172.21.15.78:0/2671744994 wait complete. 2024-08-18T23:51:01.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.983+0000 7fb089291740 20 client.16509 trim_cache size 0 max 16384 2024-08-18T23:51:01.001 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:51:01.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.984+0000 7fb086e93640 1 -- 172.21.15.78:0/3325859911 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb0700a2b80 msgr2=0x7fb0700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:01.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.984+0000 7fb086e93640 1 --2- 172.21.15.78:0/3325859911 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb0700a2b80 0x7fb0700a5040 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb07801d000 tx=0x7fb078019000 comp rx=0 tx=0).stop 2024-08-18T23:51:01.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.984+0000 7fb086e93640 1 -- 172.21.15.78:0/3325859911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb080102a60 msgr2=0x7fb080198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:01.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.984+0000 7fb086e93640 1 --2- 172.21.15.78:0/3325859911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb080102a60 0x7fb080198470 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb07400b4a0 tx=0x7fb07400b960 comp rx=0 tx=0).stop 2024-08-18T23:51:01.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.984+0000 7fb086e93640 1 -- 172.21.15.78:0/3325859911 shutdown_connections 2024-08-18T23:51:01.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.984+0000 7fb086e93640 1 --2- 172.21.15.78:0/3325859911 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb0700a2b80 0x7fb0700a5040 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:01.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.984+0000 7fb086e93640 1 --2- 172.21.15.78:0/3325859911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb080107fd0 0x7fb08019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:01.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.984+0000 7fb086e93640 1 --2- 172.21.15.78:0/3325859911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb080103410 0x7fb0801989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:01.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.984+0000 7fb086e93640 1 --2- 172.21.15.78:0/3325859911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb080102a60 0x7fb080198470 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:01.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.984+0000 7fb086e93640 1 -- 172.21.15.78:0/3325859911 >> 172.21.15.78:0/3325859911 conn(0x7fb0800fc500 msgr2=0x7fb080105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:01.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.984+0000 7fb086e93640 1 -- 172.21.15.78:0/3325859911 shutdown_connections 2024-08-18T23:51:01.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:00.984+0000 7fb086e93640 1 -- 172.21.15.78:0/3325859911 wait complete. 2024-08-18T23:51:01.662 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:01 smithi078 ceph-mon[18440]: pgmap v551: 129 pgs: 129 active+clean; 744 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 879 KiB/s wr, 501 op/s 2024-08-18T23:51:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:01 smithi161 ceph-mon[22027]: pgmap v551: 129 pgs: 129 active+clean; 744 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 879 KiB/s wr, 501 op/s 2024-08-18T23:51:01.690 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-18T23:51:01.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:01 smithi123 ceph-mon[21808]: pgmap v551: 129 pgs: 129 active+clean; 744 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 879 KiB/s wr, 501 op/s 2024-08-18T23:51:01.939 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:51:02.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.556+0000 7f8d2bb62640 1 -- 172.21.15.78:0/1865285833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d24102260 msgr2=0x7f8d2410e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:02.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.556+0000 7f8d2bb62640 1 --2- 172.21.15.78:0/1865285833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d24102260 0x7f8d2410e790 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009930 tx=0x7f8d1c02a420 comp rx=0 tx=0).stop 2024-08-18T23:51:02.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.556+0000 7f8d2bb62640 1 -- 172.21.15.78:0/1865285833 shutdown_connections 2024-08-18T23:51:02.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.556+0000 7f8d2bb62640 1 --2- 172.21.15.78:0/1865285833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d24102260 0x7f8d2410e790 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.556+0000 7f8d2bb62640 1 --2- 172.21.15.78:0/1865285833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d241018c0 0x7f8d24101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.556+0000 7f8d2bb62640 1 --2- 172.21.15.78:0/1865285833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d241078c0 0x7f8d24107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.556+0000 7f8d2bb62640 1 -- 172.21.15.78:0/1865285833 >> 172.21.15.78:0/1865285833 conn(0x7f8d240fd620 msgr2=0x7f8d240ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:02.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.557+0000 7f8d2bb62640 1 -- 172.21.15.78:0/1865285833 shutdown_connections 2024-08-18T23:51:02.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.557+0000 7f8d2bb62640 1 -- 172.21.15.78:0/1865285833 wait complete. 2024-08-18T23:51:02.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.557+0000 7f8d2bb62640 1 Processor -- start 2024-08-18T23:51:02.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.557+0000 7f8d2bb62640 1 -- start start 2024-08-18T23:51:02.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.557+0000 7f8d2bb62640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d241018c0 0x7f8d24198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:02.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.557+0000 7f8d2bb62640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d24102260 0x7f8d241986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:02.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.557+0000 7f8d2bb62640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d241078c0 0x7f8d2419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:02.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.557+0000 7f8d2bb62640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d24111ce0 con 0x7f8d24102260 2024-08-18T23:51:02.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.557+0000 7f8d2bb62640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d24111b60 con 0x7f8d241018c0 2024-08-18T23:51:02.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.557+0000 7f8d2bb62640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8d24111e60 con 0x7f8d241078c0 2024-08-18T23:51:02.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.558+0000 7f8d290fd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d24102260 0x7f8d241986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:02.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.558+0000 7f8d290fd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d24102260 0x7f8d241986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33144/0 (socket says 172.21.15.78:33144) 2024-08-18T23:51:02.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.558+0000 7f8d290fd640 1 -- 172.21.15.78:0/1112691602 learned_addr learned my addr 172.21.15.78:0/1112691602 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:02.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.558+0000 7f8d298fe640 1 --2- 172.21.15.78:0/1112691602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d241018c0 0x7f8d24198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:02.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.558+0000 7f8d2a0ff640 1 --2- 172.21.15.78:0/1112691602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d241078c0 0x7f8d2419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:02.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.558+0000 7f8d290fd640 1 -- 172.21.15.78:0/1112691602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d241078c0 msgr2=0x7f8d2419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:02.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.558+0000 7f8d290fd640 1 --2- 172.21.15.78:0/1112691602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d241078c0 0x7f8d2419ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.558+0000 7f8d290fd640 1 -- 172.21.15.78:0/1112691602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d241018c0 msgr2=0x7f8d24198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:02.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.558+0000 7f8d290fd640 1 --2- 172.21.15.78:0/1112691602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d241018c0 0x7f8d24198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.558+0000 7f8d290fd640 1 -- 172.21.15.78:0/1112691602 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2419d110 con 0x7f8d24102260 2024-08-18T23:51:02.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.558+0000 7f8d290fd640 1 --2- 172.21.15.78:0/1112691602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d24102260 0x7f8d241986a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8d1800c920 tx=0x7f8d1800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:02.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.558+0000 7f8d22ffd640 1 -- 172.21.15.78:0/1112691602 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d18007c70 con 0x7f8d24102260 2024-08-18T23:51:02.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.558+0000 7f8d22ffd640 1 -- 172.21.15.78:0/1112691602 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d18007e10 con 0x7f8d24102260 2024-08-18T23:51:02.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.558+0000 7f8d2bb62640 1 -- 172.21.15.78:0/1112691602 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2419d400 con 0x7f8d24102260 2024-08-18T23:51:02.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.558+0000 7f8d22ffd640 1 -- 172.21.15.78:0/1112691602 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d18018890 con 0x7f8d24102260 2024-08-18T23:51:02.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.558+0000 7f8d2bb62640 1 -- 172.21.15.78:0/1112691602 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d241a4ce0 con 0x7f8d24102260 2024-08-18T23:51:02.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.560+0000 7f8d22ffd640 1 -- 172.21.15.78:0/1112691602 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8d18018a70 con 0x7f8d24102260 2024-08-18T23:51:02.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.561+0000 7f8d2df60740 1 Processor -- start 2024-08-18T23:51:02.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.561+0000 7f8d2df60740 1 -- start start 2024-08-18T23:51:02.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.561+0000 7f8d2df60740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159adff0 0x558f159ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:02.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.561+0000 7f8d2df60740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f159ae910 0x558f159b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:02.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.561+0000 7f8d22ffd640 1 --2- 172.21.15.78:0/1112691602 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8d0c0a2b80 0x7f8d0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:02.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.561+0000 7f8d2df60740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f159b91d0 0x558f159bb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:02.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.561+0000 7f8d2df60740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558f15913a40 con 0x558f159adff0 2024-08-18T23:51:02.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.561+0000 7f8d2df60740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558f158ab9d0 con 0x558f159b91d0 2024-08-18T23:51:02.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.561+0000 7f8d2df60740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558f1590fa00 con 0x558f159ae910 2024-08-18T23:51:02.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.561+0000 7f8d2a0ff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159adff0 0x558f159ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:02.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.561+0000 7f8d2a0ff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159adff0 0x558f159ae3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33160/0 (socket says 172.21.15.78:33160) 2024-08-18T23:51:02.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.561+0000 7f8d290fd640 1 --2- 172.21.15.78:0/1112691602 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8d0c0a2b80 0x7f8d0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:02.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.561+0000 7f8d2a0ff640 1 -- 172.21.15.78:0/447095070 learned_addr learned my addr 172.21.15.78:0/447095070 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:02.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.562+0000 7f8d298fe640 1 --2- 172.21.15.78:0/447095070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f159ae910 0x558f159b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:02.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.562+0000 7f8d22ffd640 1 -- 172.21.15.78:0/1112691602 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f8d180cf4d0 con 0x7f8d24102260 2024-08-18T23:51:02.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.562+0000 7f8d2a0ff640 1 --2- 172.21.15.78:0/447095070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f159b91d0 0x558f159bb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:02.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.562+0000 7f8d2a0ff640 1 -- 172.21.15.78:0/447095070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f159ae910 msgr2=0x558f159b8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:02.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.562+0000 7f8d2a0ff640 1 --2- 172.21.15.78:0/447095070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f159ae910 0x558f159b8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.562+0000 7f8d2a0ff640 1 -- 172.21.15.78:0/447095070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f159b91d0 msgr2=0x558f159bb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:02.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.562+0000 7f8d2a0ff640 1 --2- 172.21.15.78:0/447095070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f159b91d0 0x558f159bb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.562+0000 7f8d2a0ff640 1 -- 172.21.15.78:0/447095070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f158b2650 con 0x558f159adff0 2024-08-18T23:51:02.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.562+0000 7f8d290fd640 1 --2- 172.21.15.78:0/1112691602 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8d0c0a2b80 0x7f8d0c0a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8d1801b000 tx=0x7f8d1800ff30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:02.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.562+0000 7f8d2a0ff640 1 --2- 172.21.15.78:0/447095070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159adff0 0x558f159ae3d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c031550 tx=0x7f8d1c005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46285aa4ee44d21 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:02.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.562+0000 7f8d0bfff640 1 -- 172.21.15.78:0/447095070 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c044030 con 0x558f159adff0 2024-08-18T23:51:02.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.562+0000 7f8d0bfff640 1 -- 172.21.15.78:0/447095070 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d1c005020 con 0x558f159adff0 2024-08-18T23:51:02.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.562+0000 7f8d0bfff640 1 -- 172.21.15.78:0/447095070 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c035040 con 0x558f159adff0 2024-08-18T23:51:02.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.563+0000 7f8d2df60740 1 -- 172.21.15.78:0/447095070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159adff0 msgr2=0x558f159ae3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:02.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.563+0000 7f8d2df60740 1 --2- 172.21.15.78:0/447095070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159adff0 0x558f159ae3d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c031550 tx=0x7f8d1c005d90 comp rx=0 tx=0).stop 2024-08-18T23:51:02.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.563+0000 7f8d2df60740 1 -- 172.21.15.78:0/447095070 shutdown_connections 2024-08-18T23:51:02.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.563+0000 7f8d2df60740 1 --2- 172.21.15.78:0/447095070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f159b91d0 0x558f159bb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.563+0000 7f8d2df60740 1 --2- 172.21.15.78:0/447095070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f159ae910 0x558f159b8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.563+0000 7f8d2df60740 1 --2- 172.21.15.78:0/447095070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159adff0 0x558f159ae3d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.563+0000 7f8d2df60740 1 -- 172.21.15.78:0/447095070 >> 172.21.15.78:0/447095070 conn(0x558f159aa580 msgr2=0x558f159aa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:02.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.563+0000 7f8d2df60740 1 -- 172.21.15.78:0/447095070 shutdown_connections 2024-08-18T23:51:02.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.563+0000 7f8d2df60740 1 -- 172.21.15.78:0/447095070 wait complete. 2024-08-18T23:51:02.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.563+0000 7f8d2df60740 1 -- start start 2024-08-18T23:51:02.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.563+0000 7f8d2df60740 1 Processor -- start 2024-08-18T23:51:02.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.564+0000 7f8d2df60740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f159adff0 0x558f159b6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:02.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.564+0000 7f8d2df60740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f159ae910 0x558f159b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:02.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.564+0000 7f8d2df60740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159b91d0 0x558f159b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:02.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.564+0000 7f8d2df60740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558f159bbda0 con 0x558f159b91d0 2024-08-18T23:51:02.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.564+0000 7f8d2df60740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558f158ceb20 con 0x558f159adff0 2024-08-18T23:51:02.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.564+0000 7f8d2df60740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558f159bbf20 con 0x558f159ae910 2024-08-18T23:51:02.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.564+0000 7f8d2df60740 10 client.0 _ll_register_callbacks cb 0x558f158ccbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:02.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.564+0000 7f8d2a0ff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159b91d0 0x558f159b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:02.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.564+0000 7f8d2a0ff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159b91d0 0x558f159b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33170/0 (socket says 172.21.15.78:33170) 2024-08-18T23:51:02.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.564+0000 7f8d2a0ff640 1 -- 172.21.15.78:0/4000993351 learned_addr learned my addr 172.21.15.78:0/4000993351 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:02.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.564+0000 7f8d298fe640 1 --2- 172.21.15.78:0/4000993351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f159ae910 0x558f159b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:02.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.564+0000 7f8d2a0ff640 1 -- 172.21.15.78:0/4000993351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f159ae910 msgr2=0x558f159b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:02.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.564+0000 7f8d2a0ff640 1 --2- 172.21.15.78:0/4000993351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f159ae910 0x558f159b65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.564+0000 7f8d2a0ff640 1 -- 172.21.15.78:0/4000993351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f159adff0 msgr2=0x558f159b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:02.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.564+0000 7f8d2a0ff640 1 --2- 172.21.15.78:0/4000993351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f159adff0 0x558f159b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.564+0000 7f8d2a0ff640 1 -- 172.21.15.78:0/4000993351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f156853c0 con 0x558f159b91d0 2024-08-18T23:51:02.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.564+0000 7f8d2a0ff640 1 --2- 172.21.15.78:0/4000993351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159b91d0 0x558f159b6b00 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c004650 tx=0x7f8d1c009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56dd0f5e3afb486 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:02.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.564+0000 7f8d097fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:51:02.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.565+0000 7f8d097fa640 1 -- 172.21.15.78:0/4000993351 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c035070 con 0x558f159b91d0 2024-08-18T23:51:02.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.565+0000 7f8d097fa640 1 -- 172.21.15.78:0/4000993351 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f8d1c0049a0 con 0x558f159b91d0 2024-08-18T23:51:02.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.565+0000 7f8d2df60740 1 -- 172.21.15.78:0/4000993351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d1c005020 con 0x558f159b91d0 2024-08-18T23:51:02.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.565+0000 7f8d097fa640 1 -- 172.21.15.78:0/4000993351 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8d1c004ba0 con 0x558f159b91d0 2024-08-18T23:51:02.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.565+0000 7f8d2df60740 10 client.16521 fetch_fsmap learned FSMap version 29 2024-08-18T23:51:02.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.565+0000 7f8d2df60740 1 -- 172.21.15.78:0/4000993351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558f159c7970 con 0x558f159b91d0 2024-08-18T23:51:02.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.565+0000 7f8d097fa640 1 -- 172.21.15.78:0/4000993351 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d1c005020 con 0x558f159b91d0 2024-08-18T23:51:02.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.565+0000 7f8d097fa640 1 -- 172.21.15.78:0/4000993351 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f8d1c04a3f0 con 0x558f159b91d0 2024-08-18T23:51:02.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.565+0000 7f8d2df60740 10 client.16521 fetch_fsmap finished waiting for FSMap version 29 2024-08-18T23:51:02.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.565+0000 7f8d2df60740 10 client.16521 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:51:02.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.565+0000 7f8d2df60740 20 client.16521 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:02.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.565+0000 7f8d2df60740 1 --2- 172.21.15.78:0/4000993351 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x558f159c8cc0 0x558f159cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:02.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.565+0000 7f8d2df60740 4 client.16521 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:51:02.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.565+0000 7f8d2df60740 1 -- 172.21.15.78:0/4000993351 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f15920fb0 con 0x558f159c8cc0 2024-08-18T23:51:02.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.565+0000 7f8d298fe640 1 --2- 172.21.15.78:0/4000993351 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x558f159c8cc0 0x558f159cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:02.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.566+0000 7f8d298fe640 1 --2- 172.21.15.78:0/4000993351 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x558f159c8cc0 0x558f159cb0b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6f0d14e1cba3363 server_cookie=1cdf2d2e8ca27bdb in_seq=0 out_seq=0 2024-08-18T23:51:02.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.566+0000 7f8d097fa640 10 client.16521 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:51:02.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.567+0000 7f8d097fa640 1 -- 172.21.15.78:0/4000993351 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558f15920fb0 con 0x558f159c8cc0 2024-08-18T23:51:02.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.567+0000 7f8d097fa640 10 client.16521 handle_command_reply: tid=0 2024-08-18T23:51:02.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.567+0000 7f8d2df60740 1 client.16521 shutdown 2024-08-18T23:51:02.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.567+0000 7f8d2df60740 1 -- 172.21.15.78:0/4000993351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159b91d0 msgr2=0x558f159b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:02.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.567+0000 7f8d2df60740 1 --2- 172.21.15.78:0/4000993351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159b91d0 0x558f159b6b00 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c004650 tx=0x7f8d1c009670 comp rx=0 tx=0).stop 2024-08-18T23:51:02.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.567+0000 7f8d2df60740 1 -- 172.21.15.78:0/4000993351 shutdown_connections 2024-08-18T23:51:02.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.567+0000 7f8d2df60740 1 --2- 172.21.15.78:0/4000993351 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x558f159c8cc0 0x558f159cb0b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.567+0000 7f8d2df60740 1 --2- 172.21.15.78:0/4000993351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159b91d0 0x558f159b6b00 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.567+0000 7f8d2df60740 1 --2- 172.21.15.78:0/4000993351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f159ae910 0x558f159b65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.567+0000 7f8d2df60740 1 --2- 172.21.15.78:0/4000993351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f159adff0 0x558f159b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.567+0000 7f8d097fa640 0 client.16521 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:51:02.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.567+0000 7f8d2df60740 1 -- 172.21.15.78:0/4000993351 >> 172.21.15.78:0/4000993351 conn(0x558f159aa580 msgr2=0x558f159b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:02.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.567+0000 7f8d2df60740 1 -- 172.21.15.78:0/4000993351 shutdown_connections 2024-08-18T23:51:02.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.567+0000 7f8d2df60740 1 -- 172.21.15.78:0/4000993351 wait complete. 2024-08-18T23:51:02.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.567+0000 7f8d2df60740 20 client.16521 trim_cache size 0 max 16384 2024-08-18T23:51:02.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.586+0000 7f8d2df60740 1 Processor -- start 2024-08-18T23:51:02.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.586+0000 7f8d2df60740 1 -- start start 2024-08-18T23:51:02.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.586+0000 7f8d2df60740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f159adff0 0x558f159b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:02.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.586+0000 7f8d2df60740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159ae910 0x558f159b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:02.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.586+0000 7f8d2df60740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f159b91d0 0x558f159b3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:02.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.586+0000 7f8d2df60740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d241b12c0 con 0x558f159ae910 2024-08-18T23:51:02.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.586+0000 7f8d2df60740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d241aeb60 con 0x558f159b91d0 2024-08-18T23:51:02.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.586+0000 7f8d2df60740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8d241ac9b0 con 0x558f159adff0 2024-08-18T23:51:02.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.586+0000 7f8d298fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159ae910 0x558f159b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:02.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.586+0000 7f8d298fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159ae910 0x558f159b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33174/0 (socket says 172.21.15.78:33174) 2024-08-18T23:51:02.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.586+0000 7f8d298fe640 1 -- 172.21.15.78:0/3326256179 learned_addr learned my addr 172.21.15.78:0/3326256179 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:02.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.586+0000 7f8d298fe640 1 -- 172.21.15.78:0/3326256179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f159adff0 msgr2=0x558f159b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:02.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.586+0000 7f8d298fe640 1 --2- 172.21.15.78:0/3326256179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f159adff0 0x558f159b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.586+0000 7f8d2a0ff640 1 --2- 172.21.15.78:0/3326256179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f159b91d0 0x558f159b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:02.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.586+0000 7f8d298fe640 1 -- 172.21.15.78:0/3326256179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f159b91d0 msgr2=0x558f159b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:02.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.586+0000 7f8d298fe640 1 --2- 172.21.15.78:0/3326256179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f159b91d0 0x558f159b3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.586+0000 7f8d298fe640 1 -- 172.21.15.78:0/3326256179 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f159ab630 con 0x558f159ae910 2024-08-18T23:51:02.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.587+0000 7f8d298fe640 1 --2- 172.21.15.78:0/3326256179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159ae910 0x558f159b3300 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x558f159b76c0 tx=0x7f8d1400d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3a8af3f408bcc4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:02.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.587+0000 7f8d0b7fe640 1 -- 172.21.15.78:0/3326256179 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14026070 con 0x558f159ae910 2024-08-18T23:51:02.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.587+0000 7f8d0b7fe640 1 -- 172.21.15.78:0/3326256179 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d1400ae50 con 0x558f159ae910 2024-08-18T23:51:02.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.587+0000 7f8d0b7fe640 1 -- 172.21.15.78:0/3326256179 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14021e00 con 0x558f159ae910 2024-08-18T23:51:02.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.588+0000 7f8d2df60740 1 -- 172.21.15.78:0/3326256179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159ae910 msgr2=0x558f159b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:02.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.588+0000 7f8d2df60740 1 --2- 172.21.15.78:0/3326256179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159ae910 0x558f159b3300 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x558f159b76c0 tx=0x7f8d1400d040 comp rx=0 tx=0).stop 2024-08-18T23:51:02.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.588+0000 7f8d2df60740 1 -- 172.21.15.78:0/3326256179 shutdown_connections 2024-08-18T23:51:02.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.588+0000 7f8d2df60740 1 --2- 172.21.15.78:0/3326256179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f159b91d0 0x558f159b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.588+0000 7f8d2df60740 1 --2- 172.21.15.78:0/3326256179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159ae910 0x558f159b3300 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.588+0000 7f8d2df60740 1 --2- 172.21.15.78:0/3326256179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f159adff0 0x558f159b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.588+0000 7f8d2df60740 1 -- 172.21.15.78:0/3326256179 >> 172.21.15.78:0/3326256179 conn(0x558f159aa580 msgr2=0x558f159b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:02.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.588+0000 7f8d2df60740 1 -- 172.21.15.78:0/3326256179 shutdown_connections 2024-08-18T23:51:02.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.588+0000 7f8d2df60740 1 -- 172.21.15.78:0/3326256179 wait complete. 2024-08-18T23:51:02.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.588+0000 7f8d2df60740 1 -- start start 2024-08-18T23:51:02.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.588+0000 7f8d2df60740 1 Processor -- start 2024-08-18T23:51:02.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d2df60740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f159adff0 0x558f15a2b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:02.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d2df60740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f159ae910 0x558f15a2bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:02.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d2df60740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159b91d0 0x558f15a2c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:02.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d2df60740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d241ad020 con 0x558f159b91d0 2024-08-18T23:51:02.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d2df60740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d241ae490 con 0x558f159ae910 2024-08-18T23:51:02.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d2df60740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8d241ad6f0 con 0x558f159adff0 2024-08-18T23:51:02.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d2df60740 10 client.0 _ll_register_callbacks cb 0x7f8d14009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:02.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d2a0ff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159b91d0 0x558f15a2c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:02.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d2a0ff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159b91d0 0x558f15a2c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33190/0 (socket says 172.21.15.78:33190) 2024-08-18T23:51:02.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d2a0ff640 1 -- 172.21.15.78:0/2846069681 learned_addr learned my addr 172.21.15.78:0/2846069681 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:02.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d298fe640 1 --2- 172.21.15.78:0/2846069681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f159ae910 0x558f15a2bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:02.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d2a0ff640 1 --2- 172.21.15.78:0/2846069681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f159adff0 0x558f15a2b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:02.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d2a0ff640 1 -- 172.21.15.78:0/2846069681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f159adff0 msgr2=0x558f15a2b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:02.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d2a0ff640 1 --2- 172.21.15.78:0/2846069681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f159adff0 0x558f15a2b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d2a0ff640 1 -- 172.21.15.78:0/2846069681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f159ae910 msgr2=0x558f15a2bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:02.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d2a0ff640 1 --2- 172.21.15.78:0/2846069681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f159ae910 0x558f15a2bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d2a0ff640 1 -- 172.21.15.78:0/2846069681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f159cd350 con 0x558f159b91d0 2024-08-18T23:51:02.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d2a0ff640 1 --2- 172.21.15.78:0/2846069681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159b91d0 0x558f15a2c190 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c005ab0 tx=0x7f8d1c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e955ba805143a0e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:02.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d097fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:51:02.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d097fa640 1 -- 172.21.15.78:0/2846069681 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c044070 con 0x558f159b91d0 2024-08-18T23:51:02.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d097fa640 1 -- 172.21.15.78:0/2846069681 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f8d1c031750 con 0x558f159b91d0 2024-08-18T23:51:02.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.589+0000 7f8d2df60740 1 -- 172.21.15.78:0/2846069681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d1400ae50 con 0x558f159b91d0 2024-08-18T23:51:02.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.590+0000 7f8d097fa640 1 -- 172.21.15.78:0/2846069681 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8d1c035040 con 0x558f159b91d0 2024-08-18T23:51:02.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.590+0000 7f8d2df60740 10 client.16527 fetch_fsmap learned FSMap version 29 2024-08-18T23:51:02.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.590+0000 7f8d2df60740 1 -- 172.21.15.78:0/2846069681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558f159ce650 con 0x558f159b91d0 2024-08-18T23:51:02.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.590+0000 7f8d097fa640 1 -- 172.21.15.78:0/2846069681 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d1400ae50 con 0x558f159b91d0 2024-08-18T23:51:02.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.590+0000 7f8d097fa640 1 -- 172.21.15.78:0/2846069681 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f8d1c0035f0 con 0x558f159b91d0 2024-08-18T23:51:02.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.590+0000 7f8d2df60740 10 client.16527 fetch_fsmap finished waiting for FSMap version 29 2024-08-18T23:51:02.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.590+0000 7f8d2df60740 10 client.16527 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:51:02.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.590+0000 7f8d2df60740 20 client.16527 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:02.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.590+0000 7f8d2df60740 1 --2- 172.21.15.78:0/2846069681 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x558f15a37a00 0x558f15a39df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:02.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.590+0000 7f8d2df60740 4 client.16527 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:51:02.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.590+0000 7f8d2df60740 1 -- 172.21.15.78:0/2846069681 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f1591b4c0 con 0x558f15a37a00 2024-08-18T23:51:02.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.590+0000 7f8d298fe640 1 --2- 172.21.15.78:0/2846069681 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x558f15a37a00 0x558f15a39df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:02.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.591+0000 7f8d298fe640 1 --2- 172.21.15.78:0/2846069681 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x558f15a37a00 0x558f15a39df0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f5405fcad5baaed server_cookie=a7a7b01d459a788b in_seq=0 out_seq=0 2024-08-18T23:51:02.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.591+0000 7f8d097fa640 10 client.16527 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:51:02.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.591+0000 7f8d097fa640 1 -- 172.21.15.78:0/2846069681 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f1591b4c0 con 0x558f15a37a00 2024-08-18T23:51:02.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.591+0000 7f8d097fa640 10 client.16527 handle_command_reply: tid=0 2024-08-18T23:51:02.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.591+0000 7f8d2df60740 1 client.16527 shutdown 2024-08-18T23:51:02.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.591+0000 7f8d2df60740 1 -- 172.21.15.78:0/2846069681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159b91d0 msgr2=0x558f15a2c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:02.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.591+0000 7f8d2df60740 1 --2- 172.21.15.78:0/2846069681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159b91d0 0x558f15a2c190 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c005ab0 tx=0x7f8d1c001e50 comp rx=0 tx=0).stop 2024-08-18T23:51:02.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.591+0000 7f8d2df60740 1 -- 172.21.15.78:0/2846069681 shutdown_connections 2024-08-18T23:51:02.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.591+0000 7f8d2df60740 1 --2- 172.21.15.78:0/2846069681 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x558f15a37a00 0x558f15a39df0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.591+0000 7f8d2df60740 1 --2- 172.21.15.78:0/2846069681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558f159b91d0 0x558f15a2c190 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.591+0000 7f8d2df60740 1 --2- 172.21.15.78:0/2846069681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f159ae910 0x558f15a2bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.591+0000 7f8d2df60740 1 --2- 172.21.15.78:0/2846069681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f159adff0 0x558f15a2b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.591+0000 7f8d097fa640 0 client.16527 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:51:02.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.591+0000 7f8d2df60740 1 -- 172.21.15.78:0/2846069681 >> 172.21.15.78:0/2846069681 conn(0x558f159aa580 msgr2=0x558f159a8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:02.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.591+0000 7f8d2df60740 1 -- 172.21.15.78:0/2846069681 shutdown_connections 2024-08-18T23:51:02.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.591+0000 7f8d2df60740 1 -- 172.21.15.78:0/2846069681 wait complete. 2024-08-18T23:51:02.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.591+0000 7f8d2df60740 20 client.16527 trim_cache size 0 max 16384 2024-08-18T23:51:02.599 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:51:02.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.592+0000 7f8d2bb62640 1 -- 172.21.15.78:0/1112691602 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8d0c0a2b80 msgr2=0x7f8d0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:02.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.592+0000 7f8d2bb62640 1 --2- 172.21.15.78:0/1112691602 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8d0c0a2b80 0x7f8d0c0a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8d1801b000 tx=0x7f8d1800ff30 comp rx=0 tx=0).stop 2024-08-18T23:51:02.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.592+0000 7f8d2bb62640 1 -- 172.21.15.78:0/1112691602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d24102260 msgr2=0x7f8d241986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:02.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.592+0000 7f8d2bb62640 1 --2- 172.21.15.78:0/1112691602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d24102260 0x7f8d241986a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8d1800c920 tx=0x7f8d1800cde0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.592+0000 7f8d2bb62640 1 -- 172.21.15.78:0/1112691602 shutdown_connections 2024-08-18T23:51:02.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.592+0000 7f8d2bb62640 1 --2- 172.21.15.78:0/1112691602 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8d0c0a2b80 0x7f8d0c0a5040 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.592+0000 7f8d2bb62640 1 --2- 172.21.15.78:0/1112691602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d241078c0 0x7f8d2419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.592+0000 7f8d2bb62640 1 --2- 172.21.15.78:0/1112691602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d24102260 0x7f8d241986a0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.592+0000 7f8d2bb62640 1 --2- 172.21.15.78:0/1112691602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d241018c0 0x7f8d24198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:02.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.592+0000 7f8d2bb62640 1 -- 172.21.15.78:0/1112691602 >> 172.21.15.78:0/1112691602 conn(0x7f8d240fd620 msgr2=0x7f8d240fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:02.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.592+0000 7f8d2bb62640 1 -- 172.21.15.78:0/1112691602 shutdown_connections 2024-08-18T23:51:02.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:02.592+0000 7f8d2bb62640 1 -- 172.21.15.78:0/1112691602 wait complete. 2024-08-18T23:51:03.317 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-18T23:51:03.567 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:51:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:03 smithi161 ceph-mon[22027]: pgmap v552: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.0 MiB/s wr, 413 op/s 2024-08-18T23:51:03.689 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:03 smithi078 ceph-mon[18440]: pgmap v552: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.0 MiB/s wr, 413 op/s 2024-08-18T23:51:03.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:03 smithi123 ceph-mon[21808]: pgmap v552: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.0 MiB/s wr, 413 op/s 2024-08-18T23:51:04.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.098+0000 7f5788f5c640 1 -- 172.21.15.78:0/403161690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5784101180 msgr2=0x7f5784105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:04.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.098+0000 7f5788f5c640 1 --2- 172.21.15.78:0/403161690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5784101180 0x7f5784105650 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5778009900 tx=0x7f577802a470 comp rx=0 tx=0).stop 2024-08-18T23:51:04.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.099+0000 7f5788f5c640 1 -- 172.21.15.78:0/403161690 shutdown_connections 2024-08-18T23:51:04.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.099+0000 7f5788f5c640 1 --2- 172.21.15.78:0/403161690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5784105c70 0x7f5784106100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.099+0000 7f5788f5c640 1 --2- 172.21.15.78:0/403161690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5784101180 0x7f5784105650 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.099+0000 7f5788f5c640 1 --2- 172.21.15.78:0/403161690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5784100860 0x7f5784100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.099+0000 7f5788f5c640 1 -- 172.21.15.78:0/403161690 >> 172.21.15.78:0/403161690 conn(0x7f57840fc4d0 msgr2=0x7f57840fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:04.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.099+0000 7f5788f5c640 1 -- 172.21.15.78:0/403161690 shutdown_connections 2024-08-18T23:51:04.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.099+0000 7f5788f5c640 1 -- 172.21.15.78:0/403161690 wait complete. 2024-08-18T23:51:04.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.099+0000 7f5788f5c640 1 Processor -- start 2024-08-18T23:51:04.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.099+0000 7f5788f5c640 1 -- start start 2024-08-18T23:51:04.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f5788f5c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5784100860 0x7f57841a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:04.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f5788f5c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5784101180 0x7f57841a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:04.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f5788f5c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5784105c70 0x7f578419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:04.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f5788f5c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5784118270 con 0x7f5784105c70 2024-08-18T23:51:04.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f5788f5c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f57841180f0 con 0x7f5784100860 2024-08-18T23:51:04.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f5788f5c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f57841183f0 con 0x7f5784101180 2024-08-18T23:51:04.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f5782d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5784105c70 0x7f578419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:04.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f5782d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5784105c70 0x7f578419e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33200/0 (socket says 172.21.15.78:33200) 2024-08-18T23:51:04.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f5782d9d640 1 -- 172.21.15.78:0/291010180 learned_addr learned my addr 172.21.15.78:0/291010180 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:04.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f5781d9b640 1 --2- 172.21.15.78:0/291010180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5784101180 0x7f57841a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:04.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f578259c640 1 --2- 172.21.15.78:0/291010180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5784100860 0x7f57841a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:04.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f5782d9d640 1 -- 172.21.15.78:0/291010180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5784101180 msgr2=0x7f57841a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:04.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f5782d9d640 1 --2- 172.21.15.78:0/291010180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5784101180 0x7f57841a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f5782d9d640 1 -- 172.21.15.78:0/291010180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5784100860 msgr2=0x7f57841a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:04.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f5782d9d640 1 --2- 172.21.15.78:0/291010180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5784100860 0x7f57841a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f5782d9d640 1 -- 172.21.15.78:0/291010180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578419ed10 con 0x7f5784105c70 2024-08-18T23:51:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f5782d9d640 1 --2- 172.21.15.78:0/291010180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5784105c70 0x7f578419e5e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f577c00d750 tx=0x7f577c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f57737fe640 1 -- 172.21.15.78:0/291010180 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c017070 con 0x7f5784105c70 2024-08-18T23:51:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f5788f5c640 1 -- 172.21.15.78:0/291010180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578419f000 con 0x7f5784105c70 2024-08-18T23:51:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f57737fe640 1 -- 172.21.15.78:0/291010180 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f577c005500 con 0x7f5784105c70 2024-08-18T23:51:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f57737fe640 1 -- 172.21.15.78:0/291010180 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c004d90 con 0x7f5784105c70 2024-08-18T23:51:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.100+0000 7f5788f5c640 1 -- 172.21.15.78:0/291010180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57841ab2f0 con 0x7f5784105c70 2024-08-18T23:51:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.103+0000 7f57737fe640 1 -- 172.21.15.78:0/291010180 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f577c004f30 con 0x7f5784105c70 2024-08-18T23:51:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.103+0000 7f578b35a740 1 Processor -- start 2024-08-18T23:51:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.103+0000 7f578b35a740 1 -- start start 2024-08-18T23:51:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.103+0000 7f578b35a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cdecff0 0x55b94cded3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:04.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.103+0000 7f578b35a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cded910 0x55b94cdf7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:04.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.103+0000 7f578b35a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 0x55b94cdfa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:04.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.103+0000 7f578b35a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b94cd52a40 con 0x55b94cdf81d0 2024-08-18T23:51:04.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.103+0000 7f578b35a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b94ccea9d0 con 0x55b94cdecff0 2024-08-18T23:51:04.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.103+0000 7f578b35a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b94cd4ea00 con 0x55b94cded910 2024-08-18T23:51:04.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.103+0000 7f57737fe640 1 --2- 172.21.15.78:0/291010180 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f576c0a2b80 0x7f576c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:04.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.104+0000 7f5781d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 0x55b94cdfa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:04.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.104+0000 7f5781d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 0x55b94cdfa5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33210/0 (socket says 172.21.15.78:33210) 2024-08-18T23:51:04.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.104+0000 7f5781d9b640 1 -- 172.21.15.78:0/493029280 learned_addr learned my addr 172.21.15.78:0/493029280 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:04.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.104+0000 7f5782d9d640 1 --2- 172.21.15.78:0/291010180 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f576c0a2b80 0x7f576c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:04.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.104+0000 7f578259c640 1 --2- 172.21.15.78:0/493029280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cded910 0x55b94cdf7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:04.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.104+0000 7f57737fe640 1 -- 172.21.15.78:0/291010180 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f577c0ced40 con 0x7f5784105c70 2024-08-18T23:51:04.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.104+0000 7f5781d9b640 1 --2- 172.21.15.78:0/493029280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cdecff0 0x55b94cded3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:04.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.104+0000 7f5781d9b640 1 -- 172.21.15.78:0/493029280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cded910 msgr2=0x55b94cdf7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:04.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.104+0000 7f5781d9b640 1 --2- 172.21.15.78:0/493029280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cded910 0x55b94cdf7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.104+0000 7f5781d9b640 1 -- 172.21.15.78:0/493029280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cdecff0 msgr2=0x55b94cded3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:04.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.104+0000 7f5781d9b640 1 --2- 172.21.15.78:0/493029280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cdecff0 0x55b94cded3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.104+0000 7f5781d9b640 1 -- 172.21.15.78:0/493029280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b94ccf1650 con 0x55b94cdf81d0 2024-08-18T23:51:04.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.104+0000 7f578259c640 1 --2- 172.21.15.78:0/493029280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cded910 0x55b94cdf7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:51:04.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.104+0000 7f5782d9d640 1 --2- 172.21.15.78:0/291010180 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f576c0a2b80 0x7f576c0a5040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f577c09b1a0 tx=0x7f577c065e90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:04.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.104+0000 7f5781d9b640 1 --2- 172.21.15.78:0/493029280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 0x55b94cdfa5c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5778000c00 tx=0x7f57780314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d595f2b3792603 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:04.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.104+0000 7f5770ff9640 1 -- 172.21.15.78:0/493029280 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778035070 con 0x55b94cdf81d0 2024-08-18T23:51:04.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.104+0000 7f5770ff9640 1 -- 172.21.15.78:0/493029280 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5778005020 con 0x55b94cdf81d0 2024-08-18T23:51:04.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.104+0000 7f5770ff9640 1 -- 172.21.15.78:0/493029280 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778004a00 con 0x55b94cdf81d0 2024-08-18T23:51:04.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.105+0000 7f578b35a740 1 -- 172.21.15.78:0/493029280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 msgr2=0x55b94cdfa5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:04.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.105+0000 7f578b35a740 1 --2- 172.21.15.78:0/493029280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 0x55b94cdfa5c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5778000c00 tx=0x7f57780314e0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.105+0000 7f578b35a740 1 -- 172.21.15.78:0/493029280 shutdown_connections 2024-08-18T23:51:04.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.105+0000 7f578b35a740 1 --2- 172.21.15.78:0/493029280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 0x55b94cdfa5c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.105+0000 7f578b35a740 1 --2- 172.21.15.78:0/493029280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cded910 0x55b94cdf7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.105+0000 7f578b35a740 1 --2- 172.21.15.78:0/493029280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cdecff0 0x55b94cded3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.105+0000 7f578b35a740 1 -- 172.21.15.78:0/493029280 >> 172.21.15.78:0/493029280 conn(0x55b94cde9580 msgr2=0x55b94cde9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:04.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.105+0000 7f578b35a740 1 -- 172.21.15.78:0/493029280 shutdown_connections 2024-08-18T23:51:04.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.105+0000 7f578b35a740 1 -- 172.21.15.78:0/493029280 wait complete. 2024-08-18T23:51:04.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.106+0000 7f578b35a740 1 -- start start 2024-08-18T23:51:04.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.106+0000 7f578b35a740 1 Processor -- start 2024-08-18T23:51:04.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.106+0000 7f578b35a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cdecff0 0x55b94cdf5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:04.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.106+0000 7f578b35a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cded910 0x55b94cdf55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:04.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.106+0000 7f578b35a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 0x55b94cdf5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:04.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.106+0000 7f578b35a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b94cdfada0 con 0x55b94cdf81d0 2024-08-18T23:51:04.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.106+0000 7f578b35a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b94cd0db20 con 0x55b94cded910 2024-08-18T23:51:04.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.106+0000 7f578b35a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b94cdfaf20 con 0x55b94cdecff0 2024-08-18T23:51:04.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.106+0000 7f578b35a740 10 client.0 _ll_register_callbacks cb 0x55b94cd0bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.106+0000 7f5781d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 0x55b94cdf5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.106+0000 7f5781d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 0x55b94cdf5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33214/0 (socket says 172.21.15.78:33214) 2024-08-18T23:51:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.106+0000 7f5781d9b640 1 -- 172.21.15.78:0/880322340 learned_addr learned my addr 172.21.15.78:0/880322340 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.106+0000 7f578259c640 1 --2- 172.21.15.78:0/880322340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cded910 0x55b94cdf55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.106+0000 7f5781d9b640 1 --2- 172.21.15.78:0/880322340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cdecff0 0x55b94cdf5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.106+0000 7f5781d9b640 1 -- 172.21.15.78:0/880322340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cdecff0 msgr2=0x55b94cdf5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.106+0000 7f5781d9b640 1 --2- 172.21.15.78:0/880322340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cdecff0 0x55b94cdf5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.106+0000 7f5781d9b640 1 -- 172.21.15.78:0/880322340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cded910 msgr2=0x55b94cdf55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.106+0000 7f5781d9b640 1 --2- 172.21.15.78:0/880322340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cded910 0x55b94cdf55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.106+0000 7f5781d9b640 1 -- 172.21.15.78:0/880322340 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b94cac43c0 con 0x55b94cdf81d0 2024-08-18T23:51:04.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.107+0000 7f5781d9b640 1 --2- 172.21.15.78:0/880322340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 0x55b94cdf5ae0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5778001eb0 tx=0x7f5778031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d683d4757b3459 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:04.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.107+0000 7f5769ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:51:04.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.107+0000 7f5769ffb640 1 -- 172.21.15.78:0/880322340 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778035040 con 0x55b94cdf81d0 2024-08-18T23:51:04.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.107+0000 7f578b35a740 1 -- 172.21.15.78:0/880322340 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5778005020 con 0x55b94cdf81d0 2024-08-18T23:51:04.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.107+0000 7f5769ffb640 1 -- 172.21.15.78:0/880322340 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f5778005520 con 0x55b94cdf81d0 2024-08-18T23:51:04.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.107+0000 7f5769ffb640 1 -- 172.21.15.78:0/880322340 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f577805d050 con 0x55b94cdf81d0 2024-08-18T23:51:04.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.107+0000 7f578b35a740 10 client.16539 fetch_fsmap learned FSMap version 29 2024-08-18T23:51:04.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.107+0000 7f578b35a740 1 -- 172.21.15.78:0/880322340 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b94ce06980 con 0x55b94cdf81d0 2024-08-18T23:51:04.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.107+0000 7f5769ffb640 1 -- 172.21.15.78:0/880322340 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5778005020 con 0x55b94cdf81d0 2024-08-18T23:51:04.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.107+0000 7f5769ffb640 1 -- 172.21.15.78:0/880322340 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f577803d870 con 0x55b94cdf81d0 2024-08-18T23:51:04.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.107+0000 7f578b35a740 10 client.16539 fetch_fsmap finished waiting for FSMap version 29 2024-08-18T23:51:04.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.107+0000 7f578b35a740 10 client.16539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:51:04.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.107+0000 7f578b35a740 20 client.16539 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:04.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.107+0000 7f578b35a740 1 --2- 172.21.15.78:0/880322340 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b94ce07cd0 0x55b94ce0a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:04.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.107+0000 7f578b35a740 4 client.16539 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:51:04.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.107+0000 7f578b35a740 1 -- 172.21.15.78:0/880322340 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b94cd5ffb0 con 0x55b94ce07cd0 2024-08-18T23:51:04.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.108+0000 7f578259c640 1 --2- 172.21.15.78:0/880322340 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b94ce07cd0 0x55b94ce0a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:04.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.108+0000 7f578259c640 1 --2- 172.21.15.78:0/880322340 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b94ce07cd0 0x55b94ce0a0c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68c8f5261f8b02b6 server_cookie=b0e08a26879c6540 in_seq=0 out_seq=0 2024-08-18T23:51:04.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.108+0000 7f5769ffb640 10 client.16539 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:04.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.109+0000 7f5769ffb640 1 -- 172.21.15.78:0/880322340 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b94cd5ffb0 con 0x55b94ce07cd0 2024-08-18T23:51:04.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.109+0000 7f5769ffb640 10 client.16539 handle_command_reply: tid=0 2024-08-18T23:51:04.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.109+0000 7f578b35a740 1 client.16539 shutdown 2024-08-18T23:51:04.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.109+0000 7f578b35a740 1 -- 172.21.15.78:0/880322340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 msgr2=0x55b94cdf5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:04.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.109+0000 7f578b35a740 1 --2- 172.21.15.78:0/880322340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 0x55b94cdf5ae0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5778001eb0 tx=0x7f5778031400 comp rx=0 tx=0).stop 2024-08-18T23:51:04.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.109+0000 7f578b35a740 1 -- 172.21.15.78:0/880322340 shutdown_connections 2024-08-18T23:51:04.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.109+0000 7f578b35a740 1 --2- 172.21.15.78:0/880322340 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b94ce07cd0 0x55b94ce0a0c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.109+0000 7f578b35a740 1 --2- 172.21.15.78:0/880322340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 0x55b94cdf5ae0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.109+0000 7f578b35a740 1 --2- 172.21.15.78:0/880322340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cded910 0x55b94cdf55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.109+0000 7f578b35a740 1 --2- 172.21.15.78:0/880322340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cdecff0 0x55b94cdf5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.109+0000 7f5769ffb640 0 client.16539 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:04.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.109+0000 7f578b35a740 1 -- 172.21.15.78:0/880322340 >> 172.21.15.78:0/880322340 conn(0x55b94cde9580 msgr2=0x55b94cdef260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:04.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.109+0000 7f578b35a740 1 -- 172.21.15.78:0/880322340 shutdown_connections 2024-08-18T23:51:04.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.109+0000 7f578b35a740 1 -- 172.21.15.78:0/880322340 wait complete. 2024-08-18T23:51:04.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.109+0000 7f578b35a740 20 client.16539 trim_cache size 0 max 16384 2024-08-18T23:51:04.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.122+0000 7f578b35a740 1 Processor -- start 2024-08-18T23:51:04.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.122+0000 7f578b35a740 1 -- start start 2024-08-18T23:51:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f578b35a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cdecff0 0x55b94cde7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f578b35a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cded910 0x55b94cde82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f578b35a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cdf81d0 0x55b94cde8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f578b35a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f57841b7840 con 0x55b94cded910 2024-08-18T23:51:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f578b35a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f57841b50e0 con 0x55b94cdf81d0 2024-08-18T23:51:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f578b35a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f57841b2f30 con 0x55b94cdecff0 2024-08-18T23:51:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f578259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cded910 0x55b94cde82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f578259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cded910 0x55b94cde82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33230/0 (socket says 172.21.15.78:33230) 2024-08-18T23:51:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f578259c640 1 -- 172.21.15.78:0/1707533760 learned_addr learned my addr 172.21.15.78:0/1707533760 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f5781d9b640 1 --2- 172.21.15.78:0/1707533760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cdecff0 0x55b94cde7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f5781d9b640 1 --2- 172.21.15.78:0/1707533760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cdf81d0 0x55b94cde8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f578259c640 1 -- 172.21.15.78:0/1707533760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cdecff0 msgr2=0x55b94cde7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:04.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f578259c640 1 --2- 172.21.15.78:0/1707533760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cdecff0 0x55b94cde7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f578259c640 1 -- 172.21.15.78:0/1707533760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cdf81d0 msgr2=0x55b94cde8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f578259c640 1 --2- 172.21.15.78:0/1707533760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cdf81d0 0x55b94cde8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f578259c640 1 -- 172.21.15.78:0/1707533760 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b94cded740 con 0x55b94cded910 2024-08-18T23:51:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f5781d9b640 1 --2- 172.21.15.78:0/1707533760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cdecff0 0x55b94cde7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:51:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f578259c640 1 --2- 172.21.15.78:0/1707533760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cded910 0x55b94cde82d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x55b94cdf48f0 tx=0x7f57740128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a896ca3fe537be server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f576bfff640 1 -- 172.21.15.78:0/1707533760 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577401d6e0 con 0x55b94cded910 2024-08-18T23:51:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.123+0000 7f576bfff640 1 -- 172.21.15.78:0/1707533760 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f577401c3a0 con 0x55b94cded910 2024-08-18T23:51:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.124+0000 7f576bfff640 1 -- 172.21.15.78:0/1707533760 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774011640 con 0x55b94cded910 2024-08-18T23:51:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.124+0000 7f578b35a740 1 -- 172.21.15.78:0/1707533760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cded910 msgr2=0x55b94cde82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.124+0000 7f578b35a740 1 --2- 172.21.15.78:0/1707533760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cded910 0x55b94cde82d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x55b94cdf48f0 tx=0x7f57740128e0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.124+0000 7f578b35a740 1 -- 172.21.15.78:0/1707533760 shutdown_connections 2024-08-18T23:51:04.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.124+0000 7f578b35a740 1 --2- 172.21.15.78:0/1707533760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cdf81d0 0x55b94cde8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.124+0000 7f578b35a740 1 --2- 172.21.15.78:0/1707533760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cded910 0x55b94cde82d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.124+0000 7f578b35a740 1 --2- 172.21.15.78:0/1707533760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cdecff0 0x55b94cde7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.124+0000 7f578b35a740 1 -- 172.21.15.78:0/1707533760 >> 172.21.15.78:0/1707533760 conn(0x55b94cde9580 msgr2=0x55b94cdf03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:04.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.124+0000 7f578b35a740 1 -- 172.21.15.78:0/1707533760 shutdown_connections 2024-08-18T23:51:04.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.124+0000 7f578b35a740 1 -- 172.21.15.78:0/1707533760 wait complete. 2024-08-18T23:51:04.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.125+0000 7f578b35a740 1 -- start start 2024-08-18T23:51:04.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.125+0000 7f578b35a740 1 Processor -- start 2024-08-18T23:51:04.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.125+0000 7f578b35a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cdecff0 0x55b94ce0d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:04.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.125+0000 7f578b35a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cded910 0x55b94ce09ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:04.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.125+0000 7f578b35a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 0x55b94ce0a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:04.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.125+0000 7f578b35a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f57841b35a0 con 0x55b94cdf81d0 2024-08-18T23:51:04.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.125+0000 7f578b35a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f57841b4a10 con 0x55b94cdecff0 2024-08-18T23:51:04.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.125+0000 7f578b35a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f57841b3c70 con 0x55b94cded910 2024-08-18T23:51:04.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.125+0000 7f578b35a740 10 client.0 _ll_register_callbacks cb 0x7f5778032cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:04.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.125+0000 7f5781d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 0x55b94ce0a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:04.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.125+0000 7f5781d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 0x55b94ce0a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33244/0 (socket says 172.21.15.78:33244) 2024-08-18T23:51:04.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.125+0000 7f5781d9b640 1 -- 172.21.15.78:0/178775106 learned_addr learned my addr 172.21.15.78:0/178775106 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:04.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.125+0000 7f5781d9b640 1 --2- 172.21.15.78:0/178775106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cdecff0 0x55b94ce0d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:04.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.125+0000 7f578259c640 1 --2- 172.21.15.78:0/178775106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cded910 0x55b94ce09ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:04.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.125+0000 7f5781d9b640 1 -- 172.21.15.78:0/178775106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cded910 msgr2=0x55b94ce09ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:04.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.125+0000 7f5781d9b640 1 --2- 172.21.15.78:0/178775106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cded910 0x55b94ce09ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.125+0000 7f5781d9b640 1 -- 172.21.15.78:0/178775106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cdecff0 msgr2=0x55b94ce0d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:04.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.125+0000 7f5781d9b640 1 --2- 172.21.15.78:0/178775106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cdecff0 0x55b94ce0d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.125+0000 7f5781d9b640 1 -- 172.21.15.78:0/178775106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b94ce0a910 con 0x55b94cdf81d0 2024-08-18T23:51:04.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.126+0000 7f5781d9b640 1 --2- 172.21.15.78:0/178775106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 0x55b94ce0a220 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5778031850 tx=0x7f5778031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b778513d9ce33a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:04.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.126+0000 7f5769ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:51:04.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.126+0000 7f5769ffb640 1 -- 172.21.15.78:0/178775106 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778035040 con 0x55b94cdf81d0 2024-08-18T23:51:04.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.126+0000 7f5769ffb640 1 -- 172.21.15.78:0/178775106 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f5778004030 con 0x55b94cdf81d0 2024-08-18T23:51:04.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.126+0000 7f578b35a740 1 -- 172.21.15.78:0/178775106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f577401c3a0 con 0x55b94cdf81d0 2024-08-18T23:51:04.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.126+0000 7f5769ffb640 1 -- 172.21.15.78:0/178775106 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f577803c440 con 0x55b94cdf81d0 2024-08-18T23:51:04.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.126+0000 7f578b35a740 10 client.16545 fetch_fsmap learned FSMap version 29 2024-08-18T23:51:04.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.126+0000 7f578b35a740 1 -- 172.21.15.78:0/178775106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b94ce71430 con 0x55b94cdf81d0 2024-08-18T23:51:04.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.126+0000 7f5769ffb640 1 -- 172.21.15.78:0/178775106 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f577401c3a0 con 0x55b94cdf81d0 2024-08-18T23:51:04.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.126+0000 7f5769ffb640 1 -- 172.21.15.78:0/178775106 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f577803d450 con 0x55b94cdf81d0 2024-08-18T23:51:04.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.126+0000 7f578b35a740 10 client.16545 fetch_fsmap finished waiting for FSMap version 29 2024-08-18T23:51:04.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.126+0000 7f578b35a740 10 client.16545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:51:04.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.126+0000 7f578b35a740 20 client.16545 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:04.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.126+0000 7f578b35a740 1 --2- 172.21.15.78:0/178775106 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b94ce71a10 0x55b94ce73e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:04.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.126+0000 7f578b35a740 4 client.16545 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:51:04.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.126+0000 7f578b35a740 1 -- 172.21.15.78:0/178775106 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b94cd5a4c0 con 0x55b94ce71a10 2024-08-18T23:51:04.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.127+0000 7f578259c640 1 --2- 172.21.15.78:0/178775106 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b94ce71a10 0x55b94ce73e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:04.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.127+0000 7f578259c640 1 --2- 172.21.15.78:0/178775106 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b94ce71a10 0x55b94ce73e30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c30c09548a44487c server_cookie=8531638246349dbe in_seq=0 out_seq=0 2024-08-18T23:51:04.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.127+0000 7f5769ffb640 10 client.16545 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:04.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.127+0000 7f5769ffb640 1 -- 172.21.15.78:0/178775106 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b94cd5a4c0 con 0x55b94ce71a10 2024-08-18T23:51:04.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.127+0000 7f5769ffb640 10 client.16545 handle_command_reply: tid=0 2024-08-18T23:51:04.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.127+0000 7f578b35a740 1 client.16545 shutdown 2024-08-18T23:51:04.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.127+0000 7f578b35a740 1 -- 172.21.15.78:0/178775106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 msgr2=0x55b94ce0a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:04.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.127+0000 7f578b35a740 1 --2- 172.21.15.78:0/178775106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 0x55b94ce0a220 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5778031850 tx=0x7f5778031400 comp rx=0 tx=0).stop 2024-08-18T23:51:04.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.127+0000 7f578b35a740 1 -- 172.21.15.78:0/178775106 shutdown_connections 2024-08-18T23:51:04.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.127+0000 7f578b35a740 1 --2- 172.21.15.78:0/178775106 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b94ce71a10 0x55b94ce73e30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.127+0000 7f578b35a740 1 --2- 172.21.15.78:0/178775106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b94cdf81d0 0x55b94ce0a220 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.127+0000 7f578b35a740 1 --2- 172.21.15.78:0/178775106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b94cded910 0x55b94ce09ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.127+0000 7f578b35a740 1 --2- 172.21.15.78:0/178775106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b94cdecff0 0x55b94ce0d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.127+0000 7f578b35a740 1 -- 172.21.15.78:0/178775106 >> 172.21.15.78:0/178775106 conn(0x55b94cde9580 msgr2=0x55b94cdf5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:04.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.127+0000 7f5769ffb640 0 client.16545 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:04.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.128+0000 7f578b35a740 1 -- 172.21.15.78:0/178775106 shutdown_connections 2024-08-18T23:51:04.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.128+0000 7f578b35a740 1 -- 172.21.15.78:0/178775106 wait complete. 2024-08-18T23:51:04.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.128+0000 7f578b35a740 20 client.16545 trim_cache size 0 max 16384 2024-08-18T23:51:04.143 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:51:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.128+0000 7f5788f5c640 1 -- 172.21.15.78:0/291010180 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f576c0a2b80 msgr2=0x7f576c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.128+0000 7f5788f5c640 1 --2- 172.21.15.78:0/291010180 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f576c0a2b80 0x7f576c0a5040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f577c09b1a0 tx=0x7f577c065e90 comp rx=0 tx=0).stop 2024-08-18T23:51:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.128+0000 7f5788f5c640 1 -- 172.21.15.78:0/291010180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5784105c70 msgr2=0x7f578419e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.128+0000 7f5788f5c640 1 --2- 172.21.15.78:0/291010180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5784105c70 0x7f578419e5e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f577c00d750 tx=0x7f577c00dc10 comp rx=0 tx=0).stop 2024-08-18T23:51:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.128+0000 7f5788f5c640 1 -- 172.21.15.78:0/291010180 shutdown_connections 2024-08-18T23:51:04.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.128+0000 7f5788f5c640 1 --2- 172.21.15.78:0/291010180 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f576c0a2b80 0x7f576c0a5040 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.128+0000 7f5788f5c640 1 --2- 172.21.15.78:0/291010180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5784105c70 0x7f578419e5e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.128+0000 7f5788f5c640 1 --2- 172.21.15.78:0/291010180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5784101180 0x7f57841a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.128+0000 7f5788f5c640 1 --2- 172.21.15.78:0/291010180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5784100860 0x7f57841a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.128+0000 7f5788f5c640 1 -- 172.21.15.78:0/291010180 >> 172.21.15.78:0/291010180 conn(0x7f57840fc4d0 msgr2=0x7f578410f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.128+0000 7f5788f5c640 1 -- 172.21.15.78:0/291010180 shutdown_connections 2024-08-18T23:51:04.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:04.128+0000 7f5788f5c640 1 -- 172.21.15.78:0/291010180 wait complete. 2024-08-18T23:51:05.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:05 smithi161 ceph-mon[22027]: pgmap v553: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.0 MiB/s wr, 291 op/s 2024-08-18T23:51:05.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:05 smithi078 ceph-mon[18440]: pgmap v553: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.0 MiB/s wr, 291 op/s 2024-08-18T23:51:05.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:05 smithi123 ceph-mon[21808]: pgmap v553: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.0 MiB/s wr, 291 op/s 2024-08-18T23:51:05.894 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-18T23:51:05.894 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-18T23:51:06.154 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:51:06.304 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:51:06.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.686+0000 7f4985f31640 1 -- 172.21.15.78:0/271584010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4980100190 msgr2=0x7f4980100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:06.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.686+0000 7f4985f31640 1 --2- 172.21.15.78:0/271584010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4980100190 0x7f4980100570 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4970009900 tx=0x7f497002a410 comp rx=0 tx=0).stop 2024-08-18T23:51:06.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.686+0000 7f4985f31640 1 -- 172.21.15.78:0/271584010 shutdown_connections 2024-08-18T23:51:06.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.686+0000 7f4985f31640 1 --2- 172.21.15.78:0/271584010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49801052c0 0x7f498010ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.686+0000 7f4985f31640 1 --2- 172.21.15.78:0/271584010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4980100b40 0x7f4980104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.686+0000 7f4985f31640 1 --2- 172.21.15.78:0/271584010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4980100190 0x7f4980100570 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.686+0000 7f4985f31640 1 -- 172.21.15.78:0/271584010 >> 172.21.15.78:0/271584010 conn(0x7f49800fbc10 msgr2=0x7f49800fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:06.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.686+0000 7f4985f31640 1 -- 172.21.15.78:0/271584010 shutdown_connections 2024-08-18T23:51:06.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.686+0000 7f4985f31640 1 -- 172.21.15.78:0/271584010 wait complete. 2024-08-18T23:51:06.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.687+0000 7f4985f31640 1 Processor -- start 2024-08-18T23:51:06.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.687+0000 7f4985f31640 1 -- start start 2024-08-18T23:51:06.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.687+0000 7f4985f31640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4980100190 0x7f4980101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:06.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.687+0000 7f4985f31640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4980100b40 0x7f4980102170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:06.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.687+0000 7f4985f31640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49801052c0 0x7f49801026b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:06.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.687+0000 7f4985f31640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4980110be0 con 0x7f49801052c0 2024-08-18T23:51:06.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.687+0000 7f4985f31640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4980110a60 con 0x7f4980100190 2024-08-18T23:51:06.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.687+0000 7f4985f31640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4980110d60 con 0x7f4980100b40 2024-08-18T23:51:06.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.687+0000 7f497ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49801052c0 0x7f49801026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:06.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.687+0000 7f497ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49801052c0 0x7f49801026b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48880/0 (socket says 172.21.15.78:48880) 2024-08-18T23:51:06.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.687+0000 7f497ffff640 1 -- 172.21.15.78:0/3914896849 learned_addr learned my addr 172.21.15.78:0/3914896849 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:06.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.687+0000 7f497effd640 1 --2- 172.21.15.78:0/3914896849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4980100b40 0x7f4980102170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:06.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.687+0000 7f497f7fe640 1 --2- 172.21.15.78:0/3914896849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4980100190 0x7f4980101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:06.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.688+0000 7f497ffff640 1 -- 172.21.15.78:0/3914896849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4980100b40 msgr2=0x7f4980102170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:06.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.688+0000 7f497ffff640 1 --2- 172.21.15.78:0/3914896849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4980100b40 0x7f4980102170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.688+0000 7f497ffff640 1 -- 172.21.15.78:0/3914896849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4980100190 msgr2=0x7f4980101c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:06.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.688+0000 7f497ffff640 1 --2- 172.21.15.78:0/3914896849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4980100190 0x7f4980101c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.688+0000 7f497ffff640 1 -- 172.21.15.78:0/3914896849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49801a66f0 con 0x7f49801052c0 2024-08-18T23:51:06.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.688+0000 7f497f7fe640 1 --2- 172.21.15.78:0/3914896849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4980100190 0x7f4980101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:51:06.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.688+0000 7f497ffff640 1 --2- 172.21.15.78:0/3914896849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49801052c0 0x7f49801026b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f497800b970 tx=0x7f497800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:06.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.688+0000 7f497cff9640 1 -- 172.21.15.78:0/3914896849 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4978019070 con 0x7f49801052c0 2024-08-18T23:51:06.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.688+0000 7f4985f31640 1 -- 172.21.15.78:0/3914896849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49801a69e0 con 0x7f49801052c0 2024-08-18T23:51:06.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.688+0000 7f497cff9640 1 -- 172.21.15.78:0/3914896849 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f497800cb40 con 0x7f49801052c0 2024-08-18T23:51:06.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.688+0000 7f497cff9640 1 -- 172.21.15.78:0/3914896849 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4978007650 con 0x7f49801052c0 2024-08-18T23:51:06.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.688+0000 7f4985f31640 1 -- 172.21.15.78:0/3914896849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49801a6f20 con 0x7f49801052c0 2024-08-18T23:51:06.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.690+0000 7f497cff9640 1 -- 172.21.15.78:0/3914896849 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f497801c070 con 0x7f49801052c0 2024-08-18T23:51:06.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f498832f740 1 Processor -- start 2024-08-18T23:51:06.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f498832f740 1 -- start start 2024-08-18T23:51:06.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f498832f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d78714200 0x558d787145e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:06.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f498832f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d78714b20 0x558d7871eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:06.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f498832f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d7871f3e0 0x558d787217d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:06.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f498832f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558d786d7e40 con 0x558d7871f3e0 2024-08-18T23:51:06.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f498832f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558d786797f0 con 0x558d78714200 2024-08-18T23:51:06.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f498832f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558d78620fb0 con 0x558d78714b20 2024-08-18T23:51:06.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f497effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d7871f3e0 0x558d787217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:06.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f497cff9640 1 --2- 172.21.15.78:0/3914896849 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f49680a2b80 0x7f49680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:06.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f497effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d7871f3e0 0x558d787217d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48890/0 (socket says 172.21.15.78:48890) 2024-08-18T23:51:06.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f497effd640 1 -- 172.21.15.78:0/1296078906 learned_addr learned my addr 172.21.15.78:0/1296078906 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:06.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f497ffff640 1 --2- 172.21.15.78:0/3914896849 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f49680a2b80 0x7f49680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:06.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f497f7fe640 1 --2- 172.21.15.78:0/1296078906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d78714b20 0x558d7871eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:06.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f497effd640 1 --2- 172.21.15.78:0/1296078906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d78714200 0x558d787145e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:06.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f497cff9640 1 -- 172.21.15.78:0/3914896849 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f49780ce990 con 0x7f49801052c0 2024-08-18T23:51:06.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f497effd640 1 -- 172.21.15.78:0/1296078906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d78714b20 msgr2=0x558d7871eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:06.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f497effd640 1 --2- 172.21.15.78:0/1296078906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d78714b20 0x558d7871eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f497effd640 1 -- 172.21.15.78:0/1296078906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d78714200 msgr2=0x558d787145e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:06.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f497effd640 1 --2- 172.21.15.78:0/1296078906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d78714200 0x558d787145e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.691+0000 7f497effd640 1 -- 172.21.15.78:0/1296078906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d783eb6a0 con 0x558d7871f3e0 2024-08-18T23:51:06.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.692+0000 7f497effd640 1 --2- 172.21.15.78:0/1296078906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d7871f3e0 0x558d787217d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f49740125a0 tx=0x7f497400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4015d48a32135639 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:06.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.692+0000 7f497ffff640 1 --2- 172.21.15.78:0/3914896849 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f49680a2b80 0x7f49680a5040 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f49780cc470 tx=0x7f497806cf20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:06.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.692+0000 7f4965ffb640 1 -- 172.21.15.78:0/1296078906 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974004250 con 0x558d7871f3e0 2024-08-18T23:51:06.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.692+0000 7f4965ffb640 1 -- 172.21.15.78:0/1296078906 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49740043f0 con 0x558d7871f3e0 2024-08-18T23:51:06.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.692+0000 7f4965ffb640 1 -- 172.21.15.78:0/1296078906 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974004e10 con 0x558d7871f3e0 2024-08-18T23:51:06.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.692+0000 7f498832f740 1 -- 172.21.15.78:0/1296078906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d7871f3e0 msgr2=0x558d787217d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:06.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.692+0000 7f498832f740 1 --2- 172.21.15.78:0/1296078906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d7871f3e0 0x558d787217d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f49740125a0 tx=0x7f497400b870 comp rx=0 tx=0).stop 2024-08-18T23:51:06.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.692+0000 7f498832f740 1 -- 172.21.15.78:0/1296078906 shutdown_connections 2024-08-18T23:51:06.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.693+0000 7f498832f740 1 --2- 172.21.15.78:0/1296078906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d7871f3e0 0x558d787217d0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.693+0000 7f498832f740 1 --2- 172.21.15.78:0/1296078906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d78714b20 0x558d7871eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.693+0000 7f498832f740 1 --2- 172.21.15.78:0/1296078906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d78714200 0x558d787145e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.693+0000 7f498832f740 1 -- 172.21.15.78:0/1296078906 >> 172.21.15.78:0/1296078906 conn(0x558d78710790 msgr2=0x558d78710b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:06.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.693+0000 7f498832f740 1 -- 172.21.15.78:0/1296078906 shutdown_connections 2024-08-18T23:51:06.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.693+0000 7f498832f740 1 -- 172.21.15.78:0/1296078906 wait complete. 2024-08-18T23:51:06.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.693+0000 7f498832f740 1 -- start start 2024-08-18T23:51:06.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.693+0000 7f498832f740 1 Processor -- start 2024-08-18T23:51:06.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.693+0000 7f498832f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d78714200 0x558d7871c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:06.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.693+0000 7f498832f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d78714b20 0x558d7871c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:06.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.693+0000 7f498832f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d7871f3e0 0x558d7871cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:06.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.693+0000 7f498832f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558d78634cb0 con 0x558d78714200 2024-08-18T23:51:06.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.693+0000 7f498832f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558d786757b0 con 0x558d7871f3e0 2024-08-18T23:51:06.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.693+0000 7f498832f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558d78721fb0 con 0x558d78714b20 2024-08-18T23:51:06.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.693+0000 7f498832f740 10 client.0 _ll_register_callbacks cb 0x558d78388ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:06.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.694+0000 7f497effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d78714200 0x558d7871c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:06.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.694+0000 7f497effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d78714200 0x558d7871c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48896/0 (socket says 172.21.15.78:48896) 2024-08-18T23:51:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.694+0000 7f497effd640 1 -- 172.21.15.78:0/932423109 learned_addr learned my addr 172.21.15.78:0/932423109 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.694+0000 7f497f7fe640 1 --2- 172.21.15.78:0/932423109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d78714b20 0x558d7871c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.694+0000 7f497effd640 1 --2- 172.21.15.78:0/932423109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d7871f3e0 0x558d7871cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.694+0000 7f497effd640 1 -- 172.21.15.78:0/932423109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d78714b20 msgr2=0x558d7871c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.694+0000 7f497effd640 1 --2- 172.21.15.78:0/932423109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d78714b20 0x558d7871c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.694+0000 7f497effd640 1 -- 172.21.15.78:0/932423109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d7871f3e0 msgr2=0x558d7871cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.694+0000 7f497effd640 1 --2- 172.21.15.78:0/932423109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d7871f3e0 0x558d7871cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.694+0000 7f497effd640 1 -- 172.21.15.78:0/932423109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d78633bc0 con 0x558d78714200 2024-08-18T23:51:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.694+0000 7f497effd640 1 --2- 172.21.15.78:0/932423109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d78714200 0x558d7871c330 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4974012ef0 tx=0x7f4974004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aedf25c03cd4ea10 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.694+0000 7f495effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:51:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.694+0000 7f495effd640 1 -- 172.21.15.78:0/932423109 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974024070 con 0x558d78714200 2024-08-18T23:51:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.694+0000 7f495effd640 1 -- 172.21.15.78:0/932423109 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f4974005270 con 0x558d78714200 2024-08-18T23:51:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.694+0000 7f498832f740 1 -- 172.21.15.78:0/932423109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49740043f0 con 0x558d78714200 2024-08-18T23:51:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.694+0000 7f495effd640 1 -- 172.21.15.78:0/932423109 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4974037050 con 0x558d78714200 2024-08-18T23:51:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.694+0000 7f498832f740 10 client.16557 fetch_fsmap learned FSMap version 29 2024-08-18T23:51:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.694+0000 7f498832f740 1 -- 172.21.15.78:0/932423109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558d7872df00 con 0x558d78714200 2024-08-18T23:51:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.695+0000 7f495effd640 1 -- 172.21.15.78:0/932423109 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49740043f0 con 0x558d78714200 2024-08-18T23:51:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.695+0000 7f495effd640 1 -- 172.21.15.78:0/932423109 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f49740314e0 con 0x558d78714200 2024-08-18T23:51:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.695+0000 7f498832f740 10 client.16557 fetch_fsmap finished waiting for FSMap version 29 2024-08-18T23:51:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.695+0000 7f498832f740 10 client.16557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:51:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.695+0000 7f498832f740 20 client.16557 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.695+0000 7f498832f740 1 --2- 172.21.15.78:0/932423109 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558d7872f2b0 0x558d787316a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.695+0000 7f498832f740 4 client.16557 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:51:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.695+0000 7f498832f740 1 -- 172.21.15.78:0/932423109 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d78686d60 con 0x558d7872f2b0 2024-08-18T23:51:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.695+0000 7f497f7fe640 1 --2- 172.21.15.78:0/932423109 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558d7872f2b0 0x558d787316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.696+0000 7f497f7fe640 1 --2- 172.21.15.78:0/932423109 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558d7872f2b0 0x558d787316a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad00ede17c032ea server_cookie=eecdb93f4d46d1f2 in_seq=0 out_seq=0 2024-08-18T23:51:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.696+0000 7f495effd640 10 client.16557 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.696+0000 7f495effd640 1 -- 172.21.15.78:0/932423109 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d78686d60 con 0x558d7872f2b0 2024-08-18T23:51:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.696+0000 7f495effd640 10 client.16557 handle_command_reply: tid=0 2024-08-18T23:51:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.696+0000 7f498832f740 1 client.16557 shutdown 2024-08-18T23:51:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.696+0000 7f498832f740 1 -- 172.21.15.78:0/932423109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d78714200 msgr2=0x558d7871c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.696+0000 7f498832f740 1 --2- 172.21.15.78:0/932423109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d78714200 0x558d7871c330 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4974012ef0 tx=0x7f4974004ad0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.696+0000 7f498832f740 1 -- 172.21.15.78:0/932423109 shutdown_connections 2024-08-18T23:51:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.696+0000 7f498832f740 1 --2- 172.21.15.78:0/932423109 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558d7872f2b0 0x558d787316a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.696+0000 7f498832f740 1 --2- 172.21.15.78:0/932423109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d7871f3e0 0x558d7871cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.696+0000 7f498832f740 1 --2- 172.21.15.78:0/932423109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d78714b20 0x558d7871c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.696+0000 7f498832f740 1 --2- 172.21.15.78:0/932423109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d78714200 0x558d7871c330 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.696+0000 7f498832f740 1 -- 172.21.15.78:0/932423109 >> 172.21.15.78:0/932423109 conn(0x558d78710790 msgr2=0x558d78716470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.696+0000 7f495effd640 0 client.16557 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:06.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.697+0000 7f498832f740 1 -- 172.21.15.78:0/932423109 shutdown_connections 2024-08-18T23:51:06.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.697+0000 7f498832f740 1 -- 172.21.15.78:0/932423109 wait complete. 2024-08-18T23:51:06.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.697+0000 7f498832f740 20 client.16557 trim_cache size 0 max 16384 2024-08-18T23:51:06.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.714+0000 7f498832f740 1 Processor -- start 2024-08-18T23:51:06.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.714+0000 7f498832f740 1 -- start start 2024-08-18T23:51:06.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.715+0000 7f498832f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d78714200 0x558d7871bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:06.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.715+0000 7f498832f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d78714b20 0x558d78716360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:06.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.715+0000 7f498832f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d7871f3e0 0x558d78736e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:06.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.715+0000 7f498832f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f49801b35b0 con 0x558d78714b20 2024-08-18T23:51:06.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.715+0000 7f498832f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f49801b0e50 con 0x558d78714200 2024-08-18T23:51:06.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.715+0000 7f498832f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f49801aeca0 con 0x558d7871f3e0 2024-08-18T23:51:06.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.715+0000 7f497f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d78714b20 0x558d78716360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:06.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.715+0000 7f497f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d78714b20 0x558d78716360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48910/0 (socket says 172.21.15.78:48910) 2024-08-18T23:51:06.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.715+0000 7f497f7fe640 1 -- 172.21.15.78:0/4096593129 learned_addr learned my addr 172.21.15.78:0/4096593129 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:06.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.715+0000 7f497effd640 1 --2- 172.21.15.78:0/4096593129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d78714200 0x558d7871bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:06.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.715+0000 7f497effd640 1 --2- 172.21.15.78:0/4096593129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d7871f3e0 0x558d78736e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:06.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.715+0000 7f497f7fe640 1 -- 172.21.15.78:0/4096593129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d7871f3e0 msgr2=0x558d78736e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:06.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.715+0000 7f497f7fe640 1 --2- 172.21.15.78:0/4096593129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d7871f3e0 0x558d78736e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.715+0000 7f497f7fe640 1 -- 172.21.15.78:0/4096593129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d78714200 msgr2=0x558d7871bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:06.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.715+0000 7f497f7fe640 1 --2- 172.21.15.78:0/4096593129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d78714200 0x558d7871bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.715+0000 7f497f7fe640 1 -- 172.21.15.78:0/4096593129 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d78714030 con 0x558d78714b20 2024-08-18T23:51:06.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.715+0000 7f497f7fe640 1 --2- 172.21.15.78:0/4096593129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d78714b20 0x558d78716360 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4970032fd0 tx=0x7f49700314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ca2e49d024a60c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:06.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.715+0000 7f49657fa640 1 -- 172.21.15.78:0/4096593129 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4970049070 con 0x558d78714b20 2024-08-18T23:51:06.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.716+0000 7f49657fa640 1 -- 172.21.15.78:0/4096593129 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49700044b0 con 0x558d78714b20 2024-08-18T23:51:06.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.716+0000 7f49657fa640 1 -- 172.21.15.78:0/4096593129 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4970035040 con 0x558d78714b20 2024-08-18T23:51:06.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.716+0000 7f498832f740 1 -- 172.21.15.78:0/4096593129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d78714b20 msgr2=0x558d78716360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:06.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.716+0000 7f498832f740 1 --2- 172.21.15.78:0/4096593129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d78714b20 0x558d78716360 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4970032fd0 tx=0x7f49700314e0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.716+0000 7f498832f740 1 -- 172.21.15.78:0/4096593129 shutdown_connections 2024-08-18T23:51:06.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.716+0000 7f498832f740 1 --2- 172.21.15.78:0/4096593129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d7871f3e0 0x558d78736e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.716+0000 7f498832f740 1 --2- 172.21.15.78:0/4096593129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d78714b20 0x558d78716360 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.716+0000 7f498832f740 1 --2- 172.21.15.78:0/4096593129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d78714200 0x558d7871bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.716+0000 7f498832f740 1 -- 172.21.15.78:0/4096593129 >> 172.21.15.78:0/4096593129 conn(0x558d78710790 msgr2=0x558d78715c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:06.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.716+0000 7f498832f740 1 -- 172.21.15.78:0/4096593129 shutdown_connections 2024-08-18T23:51:06.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.716+0000 7f498832f740 1 -- 172.21.15.78:0/4096593129 wait complete. 2024-08-18T23:51:06.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.717+0000 7f498832f740 1 -- start start 2024-08-18T23:51:06.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.717+0000 7f498832f740 1 Processor -- start 2024-08-18T23:51:06.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.717+0000 7f498832f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d78714200 0x558d78727410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:06.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.717+0000 7f498832f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d78714b20 0x558d78727950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:06.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.717+0000 7f498832f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d7871f3e0 0x558d787189a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:06.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.717+0000 7f498832f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f49801af310 con 0x558d7871f3e0 2024-08-18T23:51:06.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.717+0000 7f498832f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f49801b0780 con 0x558d78714200 2024-08-18T23:51:06.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.717+0000 7f498832f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f49801af9e0 con 0x558d78714b20 2024-08-18T23:51:06.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.717+0000 7f498832f740 10 client.0 _ll_register_callbacks cb 0x7f49740123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:06.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.717+0000 7f497effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d7871f3e0 0x558d787189a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:06.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.718+0000 7f497effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d7871f3e0 0x558d787189a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48926/0 (socket says 172.21.15.78:48926) 2024-08-18T23:51:06.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.718+0000 7f497effd640 1 -- 172.21.15.78:0/4022641353 learned_addr learned my addr 172.21.15.78:0/4022641353 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:06.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.718+0000 7f497effd640 1 --2- 172.21.15.78:0/4022641353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d78714200 0x558d78727410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:06.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.718+0000 7f497f7fe640 1 --2- 172.21.15.78:0/4022641353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d78714b20 0x558d78727950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:06.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.718+0000 7f497effd640 1 -- 172.21.15.78:0/4022641353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d78714b20 msgr2=0x558d78727950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:06.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.718+0000 7f497effd640 1 --2- 172.21.15.78:0/4022641353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d78714b20 0x558d78727950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.718+0000 7f497effd640 1 -- 172.21.15.78:0/4022641353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d78714200 msgr2=0x558d78727410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:06.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.718+0000 7f497effd640 1 --2- 172.21.15.78:0/4022641353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d78714200 0x558d78727410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.718+0000 7f497effd640 1 -- 172.21.15.78:0/4022641353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d78719040 con 0x558d7871f3e0 2024-08-18T23:51:06.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.718+0000 7f497effd640 1 --2- 172.21.15.78:0/4022641353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d7871f3e0 0x558d787189a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4974006010 tx=0x7f497400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d317a7b22029ae server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:06.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.718+0000 7f495effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:51:06.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.718+0000 7f495effd640 1 -- 172.21.15.78:0/4022641353 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974024070 con 0x558d7871f3e0 2024-08-18T23:51:06.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.718+0000 7f495effd640 1 -- 172.21.15.78:0/4022641353 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f49740120d0 con 0x558d7871f3e0 2024-08-18T23:51:06.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.718+0000 7f498832f740 1 -- 172.21.15.78:0/4022641353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49700044b0 con 0x558d7871f3e0 2024-08-18T23:51:06.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.719+0000 7f495effd640 1 -- 172.21.15.78:0/4022641353 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4974027040 con 0x558d7871f3e0 2024-08-18T23:51:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.719+0000 7f498832f740 10 client.16563 fetch_fsmap learned FSMap version 29 2024-08-18T23:51:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.719+0000 7f498832f740 1 -- 172.21.15.78:0/4022641353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558d787997a0 con 0x558d7871f3e0 2024-08-18T23:51:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.719+0000 7f495effd640 1 -- 172.21.15.78:0/4022641353 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49700044b0 con 0x558d7871f3e0 2024-08-18T23:51:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.719+0000 7f495effd640 1 -- 172.21.15.78:0/4022641353 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f4974031380 con 0x558d7871f3e0 2024-08-18T23:51:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.719+0000 7f498832f740 10 client.16563 fetch_fsmap finished waiting for FSMap version 29 2024-08-18T23:51:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.719+0000 7f498832f740 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:51:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.719+0000 7f498832f740 20 client.16563 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.719+0000 7f498832f740 1 --2- 172.21.15.78:0/4022641353 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558d78799e60 0x558d7879c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.719+0000 7f498832f740 4 client.16563 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-18T23:51:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.719+0000 7f498832f740 1 -- 172.21.15.78:0/4022641353 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558d78681270 con 0x558d78799e60 2024-08-18T23:51:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.719+0000 7f497f7fe640 1 --2- 172.21.15.78:0/4022641353 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558d78799e60 0x558d7879c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:06.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.720+0000 7f497f7fe640 1 --2- 172.21.15.78:0/4022641353 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558d78799e60 0x558d7879c280 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70f4dc2ef3756eb2 server_cookie=9d5773903b6cc635 in_seq=0 out_seq=0 2024-08-18T23:51:06.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.720+0000 7f495effd640 10 client.16563 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:06.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.721+0000 7f495effd640 1 -- 172.21.15.78:0/4022641353 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558d78681270 con 0x558d78799e60 2024-08-18T23:51:06.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.721+0000 7f495effd640 10 client.16563 handle_command_reply: tid=0 2024-08-18T23:51:06.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.721+0000 7f498832f740 1 client.16563 shutdown 2024-08-18T23:51:06.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.721+0000 7f498832f740 1 -- 172.21.15.78:0/4022641353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d7871f3e0 msgr2=0x558d787189a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:06.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.721+0000 7f498832f740 1 --2- 172.21.15.78:0/4022641353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d7871f3e0 0x558d787189a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4974006010 tx=0x7f497400be10 comp rx=0 tx=0).stop 2024-08-18T23:51:06.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.722+0000 7f498832f740 1 -- 172.21.15.78:0/4022641353 shutdown_connections 2024-08-18T23:51:06.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.722+0000 7f498832f740 1 --2- 172.21.15.78:0/4022641353 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558d78799e60 0x558d7879c280 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.722+0000 7f498832f740 1 --2- 172.21.15.78:0/4022641353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558d7871f3e0 0x558d787189a0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.722+0000 7f498832f740 1 --2- 172.21.15.78:0/4022641353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558d78714b20 0x558d78727950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.722+0000 7f495effd640 0 client.16563 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:06.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.722+0000 7f498832f740 1 --2- 172.21.15.78:0/4022641353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558d78714200 0x558d78727410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.722+0000 7f498832f740 1 -- 172.21.15.78:0/4022641353 >> 172.21.15.78:0/4022641353 conn(0x558d78710790 msgr2=0x558d7870f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:06.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.722+0000 7f498832f740 1 -- 172.21.15.78:0/4022641353 shutdown_connections 2024-08-18T23:51:06.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.722+0000 7f498832f740 1 -- 172.21.15.78:0/4022641353 wait complete. 2024-08-18T23:51:06.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.722+0000 7f498832f740 20 client.16563 trim_cache size 0 max 16384 2024-08-18T23:51:06.728 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:51:06.728 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-08-18T23:51:06.728 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "8a99f9fd-4a29-47cd-8469-aa8bdf4c9f23", 2024-08-18T23:51:06.728 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-08-18T23:51:06.728 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:51:06.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.722+0000 7f4985f31640 1 -- 172.21.15.78:0/3914896849 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f49680a2b80 msgr2=0x7f49680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:06.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.722+0000 7f4985f31640 1 --2- 172.21.15.78:0/3914896849 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f49680a2b80 0x7f49680a5040 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f49780cc470 tx=0x7f497806cf20 comp rx=0 tx=0).stop 2024-08-18T23:51:06.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.722+0000 7f4985f31640 1 -- 172.21.15.78:0/3914896849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49801052c0 msgr2=0x7f49801026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:06.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.722+0000 7f4985f31640 1 --2- 172.21.15.78:0/3914896849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49801052c0 0x7f49801026b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f497800b970 tx=0x7f497800be30 comp rx=0 tx=0).stop 2024-08-18T23:51:06.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.722+0000 7f4985f31640 1 -- 172.21.15.78:0/3914896849 shutdown_connections 2024-08-18T23:51:06.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.722+0000 7f4985f31640 1 --2- 172.21.15.78:0/3914896849 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f49680a2b80 0x7f49680a5040 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.722+0000 7f4985f31640 1 --2- 172.21.15.78:0/3914896849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f49801052c0 0x7f49801026b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.723+0000 7f4985f31640 1 --2- 172.21.15.78:0/3914896849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4980100b40 0x7f4980102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.723+0000 7f4985f31640 1 --2- 172.21.15.78:0/3914896849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4980100190 0x7f4980101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:06.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.723+0000 7f4985f31640 1 -- 172.21.15.78:0/3914896849 >> 172.21.15.78:0/3914896849 conn(0x7f49800fbc10 msgr2=0x7f498010da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:06.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.723+0000 7f4985f31640 1 -- 172.21.15.78:0/3914896849 shutdown_connections 2024-08-18T23:51:06.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:06.723+0000 7f4985f31640 1 -- 172.21.15.78:0/3914896849 wait complete. 2024-08-18T23:51:06.986 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:51:07.480 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:51:07.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:07 smithi078 ceph-mon[18440]: pgmap v554: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 646 KiB/s wr, 230 op/s 2024-08-18T23:51:07.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:07 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:51:07.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:07 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:07.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:07 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:51:07.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:07 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:07.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:07 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:51:07.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:07 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:07.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:07 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:51:07.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:07 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:07.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:07 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:51:07.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:07 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:07 smithi161 ceph-mon[22027]: pgmap v554: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 646 KiB/s wr, 230 op/s 2024-08-18T23:51:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:07 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:51:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:07 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:07 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:51:07.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:07 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:07.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:07 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:51:07.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:07 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:07.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:07 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:51:07.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:07 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:07.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:07 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:51:07.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:07 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:07.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:07 smithi123 ceph-mon[21808]: pgmap v554: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 646 KiB/s wr, 230 op/s 2024-08-18T23:51:07.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:07 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:51:07.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:07 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:07.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:07 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:51:07.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:07 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:07.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:07 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:51:07.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:07 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:07.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:07 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:51:07.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:07 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:07.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:07 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:51:07.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:07 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:08.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.052+0000 7fdaacd88640 1 -- 172.21.15.78:0/3391686292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaa8102c20 msgr2=0x7fdaa810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:08.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.052+0000 7fdaacd88640 1 --2- 172.21.15.78:0/3391686292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaa8102c20 0x7fdaa810d770 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009ca0 tx=0x7fdaa002a950 comp rx=0 tx=0).stop 2024-08-18T23:51:08.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.052+0000 7fdaacd88640 1 -- 172.21.15.78:0/3391686292 shutdown_connections 2024-08-18T23:51:08.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.052+0000 7fdaacd88640 1 --2- 172.21.15.78:0/3391686292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaa8102c20 0x7fdaa810d770 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.052+0000 7fdaacd88640 1 --2- 172.21.15.78:0/3391686292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa81022a0 0x7fdaa81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.052+0000 7fdaacd88640 1 --2- 172.21.15.78:0/3391686292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaa8100b50 0x7fdaa8100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.052+0000 7fdaacd88640 1 -- 172.21.15.78:0/3391686292 >> 172.21.15.78:0/3391686292 conn(0x7fdaa80fc500 msgr2=0x7fdaa80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:08.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.052+0000 7fdaacd88640 1 -- 172.21.15.78:0/3391686292 shutdown_connections 2024-08-18T23:51:08.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.052+0000 7fdaacd88640 1 -- 172.21.15.78:0/3391686292 wait complete. 2024-08-18T23:51:08.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.053+0000 7fdaacd88640 1 Processor -- start 2024-08-18T23:51:08.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.053+0000 7fdaacd88640 1 -- start start 2024-08-18T23:51:08.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.053+0000 7fdaacd88640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaa8100b50 0x7fdaa8198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.053+0000 7fdaacd88640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaa81022a0 0x7fdaa8199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.053+0000 7fdaacd88640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa8102c20 0x7fdaa81a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.053+0000 7fdaacd88640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa8110cb0 con 0x7fdaa81022a0 2024-08-18T23:51:08.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.053+0000 7fdaacd88640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa8110b30 con 0x7fdaa8100b50 2024-08-18T23:51:08.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.053+0000 7fdaacd88640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa8110e30 con 0x7fdaa8102c20 2024-08-18T23:51:08.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.053+0000 7fda9ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaa81022a0 0x7fdaa8199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.053+0000 7fdaa659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaa8100b50 0x7fdaa8198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.053+0000 7fda9ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaa81022a0 0x7fdaa8199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48942/0 (socket says 172.21.15.78:48942) 2024-08-18T23:51:08.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.053+0000 7fda9ffff640 1 -- 172.21.15.78:0/2543311233 learned_addr learned my addr 172.21.15.78:0/2543311233 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:08.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.053+0000 7fdaa6d9d640 1 --2- 172.21.15.78:0/2543311233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa8102c20 0x7fdaa81a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.054+0000 7fdaa659c640 1 -- 172.21.15.78:0/2543311233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa8102c20 msgr2=0x7fdaa81a4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:08.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.054+0000 7fdaa659c640 1 --2- 172.21.15.78:0/2543311233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa8102c20 0x7fdaa81a4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.054+0000 7fdaa659c640 1 -- 172.21.15.78:0/2543311233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaa81022a0 msgr2=0x7fdaa8199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:08.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.054+0000 7fdaa659c640 1 --2- 172.21.15.78:0/2543311233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaa81022a0 0x7fdaa8199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.054+0000 7fdaa659c640 1 -- 172.21.15.78:0/2543311233 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa81a45a0 con 0x7fdaa8100b50 2024-08-18T23:51:08.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.054+0000 7fdaa659c640 1 --2- 172.21.15.78:0/2543311233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaa8100b50 0x7fdaa8198fe0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fda9800e9f0 tx=0x7fda9800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:08.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.054+0000 7fda9f7fe640 1 -- 172.21.15.78:0/2543311233 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9800ccc0 con 0x7fdaa8100b50 2024-08-18T23:51:08.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.054+0000 7fdaacd88640 1 -- 172.21.15.78:0/2543311233 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa81a4770 con 0x7fdaa8100b50 2024-08-18T23:51:08.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.054+0000 7fda9f7fe640 1 -- 172.21.15.78:0/2543311233 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda98010910 con 0x7fdaa8100b50 2024-08-18T23:51:08.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.054+0000 7fda9f7fe640 1 -- 172.21.15.78:0/2543311233 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98018960 con 0x7fdaa8100b50 2024-08-18T23:51:08.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.054+0000 7fdaacd88640 1 -- 172.21.15.78:0/2543311233 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa81a4d00 con 0x7fdaa8100b50 2024-08-18T23:51:08.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.055+0000 7fdaacd88640 1 -- 172.21.15.78:0/2543311233 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa8100f30 con 0x7fdaa8100b50 2024-08-18T23:51:08.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.057+0000 7fda9f7fe640 1 -- 172.21.15.78:0/2543311233 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fda98010ac0 con 0x7fdaa8100b50 2024-08-18T23:51:08.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.057+0000 7fda9f7fe640 1 --2- 172.21.15.78:0/2543311233 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fda940a2b80 0x7fda940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.057+0000 7fda9ffff640 1 --2- 172.21.15.78:0/2543311233 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fda940a2b80 0x7fda940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.057+0000 7fda9f7fe640 1 -- 172.21.15.78:0/2543311233 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fda980cfc00 con 0x7fdaa8100b50 2024-08-18T23:51:08.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.058+0000 7fda9ffff640 1 --2- 172.21.15.78:0/2543311233 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fda940a2b80 0x7fda940a5040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdaa806a830 tx=0x7fda90005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:08.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.064+0000 7fda9f7fe640 1 -- 172.21.15.78:0/2543311233 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fda9809c180 con 0x7fdaa8100b50 2024-08-18T23:51:08.110 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:51:08.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.346+0000 7fdaacd88640 1 -- 172.21.15.78:0/2543311233 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdaa81a4fc0 con 0x7fdaa8100b50 2024-08-18T23:51:08.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.347+0000 7fda9f7fe640 1 -- 172.21.15.78:0/2543311233 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11934 (secure 0 0 0) 0x7fda980a0480 con 0x7fdaa8100b50 2024-08-18T23:51:08.349 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:51:08.349 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"btime":"2024-08-18T23:50:51:349107+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:50:50.645203+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24428,24427]},"id":1}]} 2024-08-18T23:51:08.350 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2024-08-18T23:51:08.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.349+0000 7fdaacd88640 1 -- 172.21.15.78:0/2543311233 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fda940a2b80 msgr2=0x7fda940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:08.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.349+0000 7fdaacd88640 1 --2- 172.21.15.78:0/2543311233 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fda940a2b80 0x7fda940a5040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdaa806a830 tx=0x7fda90005f90 comp rx=0 tx=0).stop 2024-08-18T23:51:08.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.349+0000 7fdaacd88640 1 -- 172.21.15.78:0/2543311233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaa8100b50 msgr2=0x7fdaa8198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:08.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.349+0000 7fdaacd88640 1 --2- 172.21.15.78:0/2543311233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaa8100b50 0x7fdaa8198fe0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fda9800e9f0 tx=0x7fda9800eeb0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.349+0000 7fdaacd88640 1 -- 172.21.15.78:0/2543311233 shutdown_connections 2024-08-18T23:51:08.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.349+0000 7fdaacd88640 1 --2- 172.21.15.78:0/2543311233 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fda940a2b80 0x7fda940a5040 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.349+0000 7fdaacd88640 1 --2- 172.21.15.78:0/2543311233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa8102c20 0x7fdaa81a4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.349+0000 7fdaacd88640 1 --2- 172.21.15.78:0/2543311233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdaa81022a0 0x7fdaa8199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.349+0000 7fdaacd88640 1 --2- 172.21.15.78:0/2543311233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaa8100b50 0x7fdaa8198fe0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.349+0000 7fdaacd88640 1 -- 172.21.15.78:0/2543311233 >> 172.21.15.78:0/2543311233 conn(0x7fdaa80fc500 msgr2=0x7fdaa8106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:08.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.349+0000 7fdaacd88640 1 -- 172.21.15.78:0/2543311233 shutdown_connections 2024-08-18T23:51:08.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.349+0000 7fdaacd88640 1 -- 172.21.15.78:0/2543311233 wait complete. 2024-08-18T23:51:08.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:08 smithi161 ceph-mon[22027]: scrub queued for path: / 2024-08-18T23:51:08.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:08 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:08.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:08 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:51:08.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:08 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:51:08.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:08 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:08.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:08 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:51:08.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:08 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:08.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:08 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:51:08.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:08 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:08.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:08 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:51:08.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:08 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:08.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:08 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:51:08.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:08 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:08.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:08 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2543311233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:51:08.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:08 smithi078 ceph-mon[18440]: scrub queued for path: / 2024-08-18T23:51:08.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:08 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:08.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:08 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:51:08.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:08 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:51:08.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:08 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:08.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:08 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:51:08.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:08 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:08.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:08 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:51:08.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:08 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:08.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:08 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:51:08.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:08 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:08.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:08 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:51:08.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:08 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:08.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:08 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2543311233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:51:08.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:08 smithi123 ceph-mon[21808]: scrub queued for path: / 2024-08-18T23:51:08.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:08 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:08.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:08 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:51:08.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:08 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:51:08.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:08 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:08.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:08 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:51:08.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:08 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:08.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:08 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:51:08.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:08 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:08.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:08 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:51:08.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:08 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:08.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:08 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:51:08.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:08 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:08.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:08 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2543311233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:51:08.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.873+0000 7efcc5ae8640 1 -- 172.21.15.78:0/1711025588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc0102c20 msgr2=0x7efcc010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:08.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.873+0000 7efcc5ae8640 1 --2- 172.21.15.78:0/1711025588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc0102c20 0x7efcc010d770 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7efcb8009ca0 tx=0x7efcb802a950 comp rx=0 tx=0).stop 2024-08-18T23:51:08.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.873+0000 7efcc5ae8640 1 -- 172.21.15.78:0/1711025588 shutdown_connections 2024-08-18T23:51:08.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.873+0000 7efcc5ae8640 1 --2- 172.21.15.78:0/1711025588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc0102c20 0x7efcc010d770 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.873+0000 7efcc5ae8640 1 --2- 172.21.15.78:0/1711025588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcc01022a0 0x7efcc01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.873+0000 7efcc5ae8640 1 --2- 172.21.15.78:0/1711025588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcc0100b50 0x7efcc0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.873+0000 7efcc5ae8640 1 -- 172.21.15.78:0/1711025588 >> 172.21.15.78:0/1711025588 conn(0x7efcc00fc500 msgr2=0x7efcc00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:08.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.873+0000 7efcc5ae8640 1 -- 172.21.15.78:0/1711025588 shutdown_connections 2024-08-18T23:51:08.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.873+0000 7efcc5ae8640 1 -- 172.21.15.78:0/1711025588 wait complete. 2024-08-18T23:51:08.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.874+0000 7efcc5ae8640 1 Processor -- start 2024-08-18T23:51:08.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.874+0000 7efcc5ae8640 1 -- start start 2024-08-18T23:51:08.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.875+0000 7efcc5ae8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc0100b50 0x7efcc0198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.875+0000 7efcc5ae8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcc01022a0 0x7efcc01987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.875+0000 7efcc5ae8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcc0102c20 0x7efcc019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.875+0000 7efcc5ae8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efcc0110cb0 con 0x7efcc0100b50 2024-08-18T23:51:08.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.875+0000 7efcc5ae8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efcc0110b30 con 0x7efcc0102c20 2024-08-18T23:51:08.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.875+0000 7efcc5ae8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efcc0110e30 con 0x7efcc01022a0 2024-08-18T23:51:08.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.875+0000 7efcbf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc0100b50 0x7efcc0198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.875+0000 7efcbf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc0100b50 0x7efcc0198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48972/0 (socket says 172.21.15.78:48972) 2024-08-18T23:51:08.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.875+0000 7efcbf7fe640 1 -- 172.21.15.78:0/714296164 learned_addr learned my addr 172.21.15.78:0/714296164 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:08.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.875+0000 7efcbffff640 1 --2- 172.21.15.78:0/714296164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcc0102c20 0x7efcc019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.875+0000 7efcb7fff640 1 --2- 172.21.15.78:0/714296164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcc01022a0 0x7efcc01987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.875+0000 7efcbf7fe640 1 -- 172.21.15.78:0/714296164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcc01022a0 msgr2=0x7efcc01987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:08.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.875+0000 7efcbf7fe640 1 --2- 172.21.15.78:0/714296164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcc01022a0 0x7efcc01987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.875+0000 7efcbf7fe640 1 -- 172.21.15.78:0/714296164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcc0102c20 msgr2=0x7efcc019cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:08.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.875+0000 7efcbf7fe640 1 --2- 172.21.15.78:0/714296164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcc0102c20 0x7efcc019cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.875+0000 7efcbf7fe640 1 -- 172.21.15.78:0/714296164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc019d210 con 0x7efcc0100b50 2024-08-18T23:51:08.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.875+0000 7efcbf7fe640 1 --2- 172.21.15.78:0/714296164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc0100b50 0x7efcc0198260 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7efcb000e9f0 tx=0x7efcb000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:08.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.876+0000 7efcbd7fa640 1 -- 172.21.15.78:0/714296164 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb000cd80 con 0x7efcc0100b50 2024-08-18T23:51:08.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.876+0000 7efcc5ae8640 1 -- 172.21.15.78:0/714296164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc019d500 con 0x7efcc0100b50 2024-08-18T23:51:08.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.876+0000 7efcbd7fa640 1 -- 172.21.15.78:0/714296164 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb0010910 con 0x7efcc0100b50 2024-08-18T23:51:08.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.876+0000 7efcbd7fa640 1 -- 172.21.15.78:0/714296164 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0018990 con 0x7efcc0100b50 2024-08-18T23:51:08.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.876+0000 7efcc5ae8640 1 -- 172.21.15.78:0/714296164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc00fc8e0 con 0x7efcc0100b50 2024-08-18T23:51:08.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.879+0000 7efcbd7fa640 1 -- 172.21.15.78:0/714296164 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7efcb0010ac0 con 0x7efcc0100b50 2024-08-18T23:51:08.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.879+0000 7efcc7ee6740 1 Processor -- start 2024-08-18T23:51:08.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.879+0000 7efcc7ee6740 1 -- start start 2024-08-18T23:51:08.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.879+0000 7efcc7ee6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c4b210 0x559d40c4b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.879+0000 7efcc7ee6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 0x559d40c55eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.879+0000 7efcc7ee6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c563f0 0x559d40c587e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.879+0000 7efcc7ee6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559d40bb0980 con 0x559d40c4bb30 2024-08-18T23:51:08.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.879+0000 7efcc7ee6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559d40b48a70 con 0x559d40c563f0 2024-08-18T23:51:08.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.879+0000 7efcc7ee6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559d40bac940 con 0x559d40c4b210 2024-08-18T23:51:08.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.879+0000 7efcbf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 0x559d40c55eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.879+0000 7efcbffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c4b210 0x559d40c4b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.879+0000 7efcbffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c4b210 0x559d40c4b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:48896/0 (socket says 172.21.15.78:48896) 2024-08-18T23:51:08.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.879+0000 7efcbffff640 1 -- 172.21.15.78:0/1558920002 learned_addr learned my addr 172.21.15.78:0/1558920002 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:08.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.879+0000 7efcbf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 0x559d40c55eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48976/0 (socket says 172.21.15.78:48976) 2024-08-18T23:51:08.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.879+0000 7efcb7fff640 1 --2- 172.21.15.78:0/1558920002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c563f0 0x559d40c587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.880+0000 7efcbffff640 1 -- 172.21.15.78:0/1558920002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c563f0 msgr2=0x559d40c587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:08.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.880+0000 7efcbffff640 1 --2- 172.21.15.78:0/1558920002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c563f0 0x559d40c587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.880+0000 7efcbffff640 1 -- 172.21.15.78:0/1558920002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 msgr2=0x559d40c55eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:08.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.880+0000 7efcbffff640 1 --2- 172.21.15.78:0/1558920002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 0x559d40c55eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.880+0000 7efcbffff640 1 -- 172.21.15.78:0/1558920002 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d40bab6e0 con 0x559d40c4b210 2024-08-18T23:51:08.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.880+0000 7efcbf7fe640 1 --2- 172.21.15.78:0/1558920002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 0x559d40c55eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:51:08.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.880+0000 7efcbd7fa640 1 --2- 172.21.15.78:0/714296164 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efcac0a2b80 0x7efcac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.880+0000 7efcbffff640 1 --2- 172.21.15.78:0/1558920002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c4b210 0x559d40c4b5f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efcb8035000 tx=0x7efcb80093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf56e545c673e9b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:08.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.880+0000 7efcbd7fa640 1 -- 172.21.15.78:0/714296164 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7efcb00cf6e0 con 0x7efcc0100b50 2024-08-18T23:51:08.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.880+0000 7efcb5ffb640 1 -- 172.21.15.78:0/1558920002 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8048070 con 0x559d40c4b210 2024-08-18T23:51:08.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.880+0000 7efcb5ffb640 1 -- 172.21.15.78:0/1558920002 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb80098a0 con 0x559d40c4b210 2024-08-18T23:51:08.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.880+0000 7efcbffff640 1 --2- 172.21.15.78:0/714296164 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efcac0a2b80 0x7efcac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.881+0000 7efcb5ffb640 1 -- 172.21.15.78:0/1558920002 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8034040 con 0x559d40c4b210 2024-08-18T23:51:08.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.881+0000 7efcbffff640 1 --2- 172.21.15.78:0/714296164 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efcac0a2b80 0x7efcac0a5040 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efcb8003fd0 tx=0x7efcb8004530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:08.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.881+0000 7efcc7ee6740 1 -- 172.21.15.78:0/1558920002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c4b210 msgr2=0x559d40c4b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:08.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.881+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/1558920002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c4b210 0x559d40c4b5f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efcb8035000 tx=0x7efcb80093e0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.881+0000 7efcc7ee6740 1 -- 172.21.15.78:0/1558920002 shutdown_connections 2024-08-18T23:51:08.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.881+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/1558920002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c563f0 0x559d40c587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.881+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/1558920002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 0x559d40c55eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.881+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/1558920002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c4b210 0x559d40c4b5f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.881+0000 7efcc7ee6740 1 -- 172.21.15.78:0/1558920002 >> 172.21.15.78:0/1558920002 conn(0x559d40c477a0 msgr2=0x559d40c47b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:08.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.881+0000 7efcc7ee6740 1 -- 172.21.15.78:0/1558920002 shutdown_connections 2024-08-18T23:51:08.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.881+0000 7efcc7ee6740 1 -- 172.21.15.78:0/1558920002 wait complete. 2024-08-18T23:51:08.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.882+0000 7efcc7ee6740 1 -- start start 2024-08-18T23:51:08.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.882+0000 7efcc7ee6740 1 Processor -- start 2024-08-18T23:51:08.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.882+0000 7efcc7ee6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c4b210 0x559d40c532b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.882+0000 7efcc7ee6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c4bb30 0x559d40c537f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.882+0000 7efcc7ee6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c563f0 0x559d40c53d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.882+0000 7efcc7ee6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559d40c58fc0 con 0x559d40c563f0 2024-08-18T23:51:08.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.882+0000 7efcc7ee6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559d40b6baa0 con 0x559d40c4b210 2024-08-18T23:51:08.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.882+0000 7efcc7ee6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559d40c59140 con 0x559d40c4bb30 2024-08-18T23:51:08.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.882+0000 7efcc7ee6740 10 client.0 _ll_register_callbacks cb 0x559d40b5ae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:08.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.882+0000 7efcb7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c4bb30 0x559d40c537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.882+0000 7efcbf7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c4b210 0x559d40c532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.882+0000 7efcb7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c4bb30 0x559d40c537f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:48904/0 (socket says 172.21.15.78:48904) 2024-08-18T23:51:08.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.882+0000 7efcb7fff640 1 -- 172.21.15.78:0/2262185067 learned_addr learned my addr 172.21.15.78:0/2262185067 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:08.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.882+0000 7efcbf7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c4b210 0x559d40c532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:50510/0 (socket says 172.21.15.78:50510) 2024-08-18T23:51:08.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.883+0000 7efcb7fff640 1 -- 172.21.15.78:0/2262185067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c4b210 msgr2=0x559d40c532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:08.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.883+0000 7efcb7fff640 1 --2- 172.21.15.78:0/2262185067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c4b210 0x559d40c532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.883+0000 7efcb7fff640 1 -- 172.21.15.78:0/2262185067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c563f0 msgr2=0x559d40c53d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T23:51:08.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.883+0000 7efcb7fff640 1 --2- 172.21.15.78:0/2262185067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c563f0 0x559d40c53d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.883+0000 7efcb7fff640 1 -- 172.21.15.78:0/2262185067 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d40b4f650 con 0x559d40c4bb30 2024-08-18T23:51:08.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.883+0000 7efcb7fff640 1 --2- 172.21.15.78:0/2262185067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c4bb30 0x559d40c537f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efca8012700 tx=0x7efca8012b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51c7a01d4aa4bb83 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:08.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.883+0000 7efc9effd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:51:08.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.883+0000 7efc9effd640 1 -- 172.21.15.78:0/2262185067 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8005020 con 0x559d40c4bb30 2024-08-18T23:51:08.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.883+0000 7efcc7ee6740 1 -- 172.21.15.78:0/2262185067 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcb80098a0 con 0x559d40c4bb30 2024-08-18T23:51:08.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.883+0000 7efc9effd640 1 -- 172.21.15.78:0/2262185067 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7efca80051c0 con 0x559d40c4bb30 2024-08-18T23:51:08.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.884+0000 7efc9effd640 1 -- 172.21.15.78:0/2262185067 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efca8025050 con 0x559d40c4bb30 2024-08-18T23:51:08.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.884+0000 7efcc7ee6740 10 client.25804 fetch_fsmap learned FSMap version 29 2024-08-18T23:51:08.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.884+0000 7efcc7ee6740 1 -- 172.21.15.78:0/2262185067 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d409223c0 con 0x559d40c4bb30 2024-08-18T23:51:08.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.884+0000 7efc9effd640 1 -- 172.21.15.78:0/2262185067 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb80098a0 con 0x559d40c4bb30 2024-08-18T23:51:08.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.884+0000 7efc9effd640 1 -- 172.21.15.78:0/2262185067 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7efca800f760 con 0x559d40c4bb30 2024-08-18T23:51:08.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.884+0000 7efcc7ee6740 10 client.25804 fetch_fsmap finished waiting for FSMap version 29 2024-08-18T23:51:08.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.884+0000 7efcc7ee6740 10 client.25804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:51:08.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.884+0000 7efcc7ee6740 20 client.25804 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:08.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.884+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/2262185067 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x559d40c65fe0 0x559d40c68440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.884+0000 7efcc7ee6740 4 client.25804 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:51:08.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.884+0000 7efcc7ee6740 1 -- 172.21.15.78:0/2262185067 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d40bbdef0 con 0x559d40c65fe0 2024-08-18T23:51:08.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.884+0000 7efcbf7fe640 1 --2- 172.21.15.78:0/2262185067 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x559d40c65fe0 0x559d40c68440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.885+0000 7efc9effd640 10 client.25804 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:08.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.885+0000 7efcbf7fe640 1 --2- 172.21.15.78:0/2262185067 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x559d40c65fe0 0x559d40c68440 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d78a1acdf9bfb8b server_cookie=b677c69e09450c07 in_seq=0 out_seq=0 2024-08-18T23:51:08.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.886+0000 7efc9effd640 1 -- 172.21.15.78:0/2262185067 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d40bbdef0 con 0x559d40c65fe0 2024-08-18T23:51:08.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.886+0000 7efc9effd640 10 client.25804 handle_command_reply: tid=0 2024-08-18T23:51:08.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.886+0000 7efcc7ee6740 1 client.25804 shutdown 2024-08-18T23:51:08.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.886+0000 7efcc7ee6740 1 -- 172.21.15.78:0/2262185067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c4bb30 msgr2=0x559d40c537f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:08.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.886+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/2262185067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c4bb30 0x559d40c537f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efca8012700 tx=0x7efca8012b00 comp rx=0 tx=0).stop 2024-08-18T23:51:08.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.886+0000 7efcc7ee6740 1 -- 172.21.15.78:0/2262185067 shutdown_connections 2024-08-18T23:51:08.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.886+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/2262185067 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x559d40c65fe0 0x559d40c68440 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.886+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/2262185067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c563f0 0x559d40c53d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.886+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/2262185067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c4bb30 0x559d40c537f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.886+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/2262185067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c4b210 0x559d40c532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.886+0000 7efcc7ee6740 1 -- 172.21.15.78:0/2262185067 >> 172.21.15.78:0/2262185067 conn(0x559d40c477a0 msgr2=0x559d40c4d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:08.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.886+0000 7efc9effd640 0 client.25804 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:08.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.886+0000 7efcc7ee6740 1 -- 172.21.15.78:0/2262185067 shutdown_connections 2024-08-18T23:51:08.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.886+0000 7efcc7ee6740 1 -- 172.21.15.78:0/2262185067 wait complete. 2024-08-18T23:51:08.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.886+0000 7efcc7ee6740 20 client.25804 trim_cache size 0 max 16384 2024-08-18T23:51:08.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.907+0000 7efcc7ee6740 1 Processor -- start 2024-08-18T23:51:08.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.907+0000 7efcc7ee6740 1 -- start start 2024-08-18T23:51:08.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.907+0000 7efcc7ee6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c4b210 0x559d40c4dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.907+0000 7efcc7ee6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 0x559d40c52270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.907+0000 7efcc7ee6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c563f0 0x559d40c6dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.907+0000 7efcc7ee6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01b3490 con 0x559d40c4bb30 2024-08-18T23:51:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.907+0000 7efcc7ee6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01b0d30 con 0x559d40c4b210 2024-08-18T23:51:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.907+0000 7efcc7ee6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01aeb80 con 0x559d40c563f0 2024-08-18T23:51:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.907+0000 7efcb7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 0x559d40c52270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.907+0000 7efcb7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 0x559d40c52270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49004/0 (socket says 172.21.15.78:49004) 2024-08-18T23:51:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.907+0000 7efcb7fff640 1 -- 172.21.15.78:0/4089060521 learned_addr learned my addr 172.21.15.78:0/4089060521 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.907+0000 7efcbf7fe640 1 --2- 172.21.15.78:0/4089060521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c4b210 0x559d40c4dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.907+0000 7efcbffff640 1 --2- 172.21.15.78:0/4089060521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c563f0 0x559d40c6dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.908+0000 7efcb7fff640 1 -- 172.21.15.78:0/4089060521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c563f0 msgr2=0x559d40c6dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.908+0000 7efcb7fff640 1 --2- 172.21.15.78:0/4089060521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c563f0 0x559d40c6dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.908+0000 7efcb7fff640 1 -- 172.21.15.78:0/4089060521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c4b210 msgr2=0x559d40c4dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.908+0000 7efcb7fff640 1 --2- 172.21.15.78:0/4089060521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c4b210 0x559d40c4dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.908+0000 7efcb7fff640 1 -- 172.21.15.78:0/4089060521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d40c4b040 con 0x559d40c4bb30 2024-08-18T23:51:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.908+0000 7efcbf7fe640 1 --2- 172.21.15.78:0/4089060521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c4b210 0x559d40c4dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:51:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.908+0000 7efcb7fff640 1 --2- 172.21.15.78:0/4089060521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 0x559d40c52270 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7efca800fb00 tx=0x7efca8009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f4f7bf1d9a1940 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.908+0000 7efcb57fa640 1 -- 172.21.15.78:0/4089060521 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8022040 con 0x559d40c4bb30 2024-08-18T23:51:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.908+0000 7efcb57fa640 1 -- 172.21.15.78:0/4089060521 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca800fd70 con 0x559d40c4bb30 2024-08-18T23:51:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.908+0000 7efcb57fa640 1 -- 172.21.15.78:0/4089060521 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca80048e0 con 0x559d40c4bb30 2024-08-18T23:51:08.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.909+0000 7efcc7ee6740 1 -- 172.21.15.78:0/4089060521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 msgr2=0x559d40c52270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:08.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.909+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/4089060521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 0x559d40c52270 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7efca800fb00 tx=0x7efca8009620 comp rx=0 tx=0).stop 2024-08-18T23:51:08.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.909+0000 7efcc7ee6740 1 -- 172.21.15.78:0/4089060521 shutdown_connections 2024-08-18T23:51:08.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.909+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/4089060521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c563f0 0x559d40c6dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.909+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/4089060521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 0x559d40c52270 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.909+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/4089060521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c4b210 0x559d40c4dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.909+0000 7efcc7ee6740 1 -- 172.21.15.78:0/4089060521 >> 172.21.15.78:0/4089060521 conn(0x559d40c477a0 msgr2=0x559d40c552a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:08.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.909+0000 7efcc7ee6740 1 -- 172.21.15.78:0/4089060521 shutdown_connections 2024-08-18T23:51:08.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.909+0000 7efcc7ee6740 1 -- 172.21.15.78:0/4089060521 wait complete. 2024-08-18T23:51:08.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.910+0000 7efcc7ee6740 1 -- start start 2024-08-18T23:51:08.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.910+0000 7efcc7ee6740 1 Processor -- start 2024-08-18T23:51:08.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.910+0000 7efcc7ee6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c4b210 0x559d40c5e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.910+0000 7efcc7ee6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 0x559d40c4fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.910+0000 7efcc7ee6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c563f0 0x559d40c501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.910+0000 7efcc7ee6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01af1f0 con 0x559d40c4bb30 2024-08-18T23:51:08.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.910+0000 7efcc7ee6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01b0660 con 0x559d40c4b210 2024-08-18T23:51:08.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.910+0000 7efcc7ee6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01af8c0 con 0x559d40c563f0 2024-08-18T23:51:08.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.910+0000 7efcc7ee6740 10 client.0 _ll_register_callbacks cb 0x7efca8009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:08.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.911+0000 7efcbf7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c4b210 0x559d40c5e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.911+0000 7efcbf7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c4b210 0x559d40c5e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:50532/0 (socket says 172.21.15.78:50532) 2024-08-18T23:51:08.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.911+0000 7efcbf7fe640 1 -- 172.21.15.78:0/334917643 learned_addr learned my addr 172.21.15.78:0/334917643 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:08.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.911+0000 7efcbffff640 1 --2- 172.21.15.78:0/334917643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c563f0 0x559d40c501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.911+0000 7efcb7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 0x559d40c4fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.911+0000 7efcbf7fe640 1 -- 172.21.15.78:0/334917643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c563f0 msgr2=0x559d40c501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:08.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.911+0000 7efcbf7fe640 1 --2- 172.21.15.78:0/334917643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c563f0 0x559d40c501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.911+0000 7efcbf7fe640 1 -- 172.21.15.78:0/334917643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 msgr2=0x559d40c4fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:08.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.911+0000 7efcbf7fe640 1 --2- 172.21.15.78:0/334917643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 0x559d40c4fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.911+0000 7efcbf7fe640 1 -- 172.21.15.78:0/334917643 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d40c48910 con 0x559d40c4b210 2024-08-18T23:51:08.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.911+0000 7efcb7fff640 1 --2- 172.21.15.78:0/334917643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 0x559d40c4fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:51:08.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.911+0000 7efcbf7fe640 1 --2- 172.21.15.78:0/334917643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c4b210 0x559d40c5e7f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efcb009bbf0 tx=0x7efcb0018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8630b55236dfb4e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:08.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.911+0000 7efc9effd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:51:08.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.911+0000 7efc9effd640 1 -- 172.21.15.78:0/334917643 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb009ff60 con 0x559d40c4b210 2024-08-18T23:51:08.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.912+0000 7efcc7ee6740 1 -- 172.21.15.78:0/334917643 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efca800fd70 con 0x559d40c4b210 2024-08-18T23:51:08.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.912+0000 7efc9effd640 1 -- 172.21.15.78:0/334917643 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7efcb00cf3f0 con 0x559d40c4b210 2024-08-18T23:51:08.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.912+0000 7efc9effd640 1 -- 172.21.15.78:0/334917643 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efcb00a4820 con 0x559d40c4b210 2024-08-18T23:51:08.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.912+0000 7efcc7ee6740 10 client.25988 fetch_fsmap learned FSMap version 29 2024-08-18T23:51:08.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.912+0000 7efcc7ee6740 1 -- 172.21.15.78:0/334917643 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d40cd0870 con 0x559d40c4b210 2024-08-18T23:51:08.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.912+0000 7efc9effd640 1 -- 172.21.15.78:0/334917643 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb00982d0 con 0x559d40c4b210 2024-08-18T23:51:08.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.912+0000 7efc9effd640 1 -- 172.21.15.78:0/334917643 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7efcb0018c50 con 0x559d40c4b210 2024-08-18T23:51:08.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.913+0000 7efcc7ee6740 10 client.25988 fetch_fsmap finished waiting for FSMap version 29 2024-08-18T23:51:08.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.913+0000 7efcc7ee6740 10 client.25988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:51:08.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.913+0000 7efcc7ee6740 20 client.25988 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:08.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.913+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/334917643 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x559d40cd0ff0 0x559d40cd3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:08.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.913+0000 7efcc7ee6740 4 client.25988 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:51:08.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.913+0000 7efcc7ee6740 1 -- 172.21.15.78:0/334917643 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7efca800fd70 con 0x559d40cd0ff0 2024-08-18T23:51:08.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.913+0000 7efcb7fff640 1 --2- 172.21.15.78:0/334917643 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x559d40cd0ff0 0x559d40cd3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:08.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.913+0000 7efcb7fff640 1 --2- 172.21.15.78:0/334917643 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x559d40cd0ff0 0x559d40cd3410 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d8ccc29572e0986 server_cookie=eca9e1e957b98ee4 in_seq=0 out_seq=0 2024-08-18T23:51:08.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.914+0000 7efc9effd640 10 client.25988 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:08.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.914+0000 7efc9effd640 1 -- 172.21.15.78:0/334917643 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x7efca800fd70 con 0x559d40cd0ff0 2024-08-18T23:51:08.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.914+0000 7efc9effd640 10 client.25988 handle_command_reply: tid=0 2024-08-18T23:51:08.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.914+0000 7efcc7ee6740 1 client.25988 shutdown 2024-08-18T23:51:08.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.914+0000 7efcc7ee6740 1 -- 172.21.15.78:0/334917643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c4b210 msgr2=0x559d40c5e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:08.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.914+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/334917643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c4b210 0x559d40c5e7f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efcb009bbf0 tx=0x7efcb0018420 comp rx=0 tx=0).stop 2024-08-18T23:51:08.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.914+0000 7efcc7ee6740 1 -- 172.21.15.78:0/334917643 shutdown_connections 2024-08-18T23:51:08.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.914+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/334917643 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x559d40cd0ff0 0x559d40cd3410 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.914+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/334917643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d40c563f0 0x559d40c501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.914+0000 7efc9effd640 0 client.25988 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:08.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.914+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/334917643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559d40c4bb30 0x559d40c4fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.914+0000 7efcc7ee6740 1 --2- 172.21.15.78:0/334917643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d40c4b210 0x559d40c5e7f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.914+0000 7efcc7ee6740 1 -- 172.21.15.78:0/334917643 >> 172.21.15.78:0/334917643 conn(0x559d40c477a0 msgr2=0x559d40c46900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:08.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.915+0000 7efcc7ee6740 1 -- 172.21.15.78:0/334917643 shutdown_connections 2024-08-18T23:51:08.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.915+0000 7efcc7ee6740 1 -- 172.21.15.78:0/334917643 wait complete. 2024-08-18T23:51:08.922 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:51:08.922 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "scrub active (5149 inodes in the stack)", 2024-08-18T23:51:08.922 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-08-18T23:51:08.922 INFO:teuthology.orchestra.run.smithi078.stdout: "8a99f9fd-4a29-47cd-8469-aa8bdf4c9f23": { 2024-08-18T23:51:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-08-18T23:51:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "8a99f9fd-4a29-47cd-8469-aa8bdf4c9f23", 2024-08-18T23:51:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-08-18T23:51:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:51:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:51:08.923 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:51:08.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.915+0000 7efcc7ee6740 20 client.25988 trim_cache size 0 max 16384 2024-08-18T23:51:08.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.915+0000 7efcc5ae8640 1 -- 172.21.15.78:0/714296164 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efcac0a2b80 msgr2=0x7efcac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:08.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.915+0000 7efcc5ae8640 1 --2- 172.21.15.78:0/714296164 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efcac0a2b80 0x7efcac0a5040 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efcb8003fd0 tx=0x7efcb8004530 comp rx=0 tx=0).stop 2024-08-18T23:51:08.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.915+0000 7efcc5ae8640 1 -- 172.21.15.78:0/714296164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc0100b50 msgr2=0x7efcc0198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:08.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.915+0000 7efcc5ae8640 1 --2- 172.21.15.78:0/714296164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc0100b50 0x7efcc0198260 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7efcb000e9f0 tx=0x7efcb000eeb0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.916+0000 7efcc5ae8640 1 -- 172.21.15.78:0/714296164 shutdown_connections 2024-08-18T23:51:08.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.916+0000 7efcc5ae8640 1 --2- 172.21.15.78:0/714296164 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7efcac0a2b80 0x7efcac0a5040 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.916+0000 7efcc5ae8640 1 --2- 172.21.15.78:0/714296164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcc0102c20 0x7efcc019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.916+0000 7efcc5ae8640 1 --2- 172.21.15.78:0/714296164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcc01022a0 0x7efcc01987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.916+0000 7efcc5ae8640 1 --2- 172.21.15.78:0/714296164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efcc0100b50 0x7efcc0198260 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:08.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.916+0000 7efcc5ae8640 1 -- 172.21.15.78:0/714296164 >> 172.21.15.78:0/714296164 conn(0x7efcc00fc500 msgr2=0x7efcc0069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:08.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.916+0000 7efcc5ae8640 1 -- 172.21.15.78:0/714296164 shutdown_connections 2024-08-18T23:51:08.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:08.916+0000 7efcc5ae8640 1 -- 172.21.15.78:0/714296164 wait complete. 2024-08-18T23:51:09.449 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 10964c59 --release --await' 2024-08-18T23:51:09.450 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 10964c59 --release --await 2024-08-18T23:51:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:09 smithi161 ceph-mon[22027]: pgmap v555: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 634 KiB/s wr, 195 op/s 2024-08-18T23:51:09.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:09 smithi078 ceph-mon[18440]: pgmap v555: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 634 KiB/s wr, 195 op/s 2024-08-18T23:51:09.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:09 smithi123 ceph-mon[21808]: pgmap v555: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 634 KiB/s wr, 195 op/s 2024-08-18T23:51:10.142 INFO:tasks.cephfs.filesystem:scrub status for tag:8a99f9fd-4a29-47cd-8469-aa8bdf4c9f23 - {'path': '/', 'tag': '8a99f9fd-4a29-47cd-8469-aa8bdf4c9f23', 'options': 'recursive,force'} 2024-08-18T23:51:10.169 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:51:10.399 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:10 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:10.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.642+0000 7fa048337640 1 -- 172.21.15.78:0/3413383396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa040074350 msgr2=0x7fa040074730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:10.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.642+0000 7fa048337640 1 --2- 172.21.15.78:0/3413383396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa040074350 0x7fa040074730 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009900 tx=0x7fa03c02a480 comp rx=0 tx=0).stop 2024-08-18T23:51:10.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.642+0000 7fa048337640 1 -- 172.21.15.78:0/3413383396 shutdown_connections 2024-08-18T23:51:10.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.642+0000 7fa048337640 1 --2- 172.21.15.78:0/3413383396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa04010e0f0 0x7fa040110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.642+0000 7fa048337640 1 --2- 172.21.15.78:0/3413383396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa040074c70 0x7fa04010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.642+0000 7fa048337640 1 --2- 172.21.15.78:0/3413383396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa040074350 0x7fa040074730 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.642+0000 7fa048337640 1 -- 172.21.15.78:0/3413383396 >> 172.21.15.78:0/3413383396 conn(0x7fa0400fd5f0 msgr2=0x7fa0400ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:10.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.642+0000 7fa048337640 1 -- 172.21.15.78:0/3413383396 shutdown_connections 2024-08-18T23:51:10.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.642+0000 7fa048337640 1 -- 172.21.15.78:0/3413383396 wait complete. 2024-08-18T23:51:10.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.643+0000 7fa048337640 1 Processor -- start 2024-08-18T23:51:10.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.643+0000 7fa048337640 1 -- start start 2024-08-18T23:51:10.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.643+0000 7fa048337640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa040074350 0x7fa04019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:10.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.643+0000 7fa048337640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa040074c70 0x7fa04019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:10.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.643+0000 7fa048337640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa04010e0f0 0x7fa04019ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:10.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.643+0000 7fa048337640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa040113ea0 con 0x7fa04010e0f0 2024-08-18T23:51:10.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.643+0000 7fa048337640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa040113d20 con 0x7fa040074350 2024-08-18T23:51:10.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.643+0000 7fa048337640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa040114020 con 0x7fa040074c70 2024-08-18T23:51:10.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.643+0000 7fa0468d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa04010e0f0 0x7fa04019ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:10.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.643+0000 7fa0468d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa04010e0f0 0x7fa04019ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49026/0 (socket says 172.21.15.78:49026) 2024-08-18T23:51:10.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.643+0000 7fa0468d4640 1 -- 172.21.15.78:0/1427892200 learned_addr learned my addr 172.21.15.78:0/1427892200 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:10.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.643+0000 7fa0460d3640 1 --2- 172.21.15.78:0/1427892200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa040074350 0x7fa04019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:10.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.643+0000 7fa0458d2640 1 --2- 172.21.15.78:0/1427892200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa040074c70 0x7fa04019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:10.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.643+0000 7fa0468d4640 1 -- 172.21.15.78:0/1427892200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa040074c70 msgr2=0x7fa04019a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:10.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.643+0000 7fa0468d4640 1 --2- 172.21.15.78:0/1427892200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa040074c70 0x7fa04019a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.643+0000 7fa0468d4640 1 -- 172.21.15.78:0/1427892200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa040074350 msgr2=0x7fa04019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:10.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.643+0000 7fa0468d4640 1 --2- 172.21.15.78:0/1427892200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa040074350 0x7fa04019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.643+0000 7fa0468d4640 1 -- 172.21.15.78:0/1427892200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04019f390 con 0x7fa04010e0f0 2024-08-18T23:51:10.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.644+0000 7fa0468d4640 1 --2- 172.21.15.78:0/1427892200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa04010e0f0 0x7fa04019ecb0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa038009fd0 tx=0x7fa03800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:10.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.644+0000 7fa0377fe640 1 -- 172.21.15.78:0/1427892200 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa038017070 con 0x7fa04010e0f0 2024-08-18T23:51:10.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.644+0000 7fa0377fe640 1 -- 172.21.15.78:0/1427892200 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa038003e50 con 0x7fa04010e0f0 2024-08-18T23:51:10.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.644+0000 7fa0377fe640 1 -- 172.21.15.78:0/1427892200 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0380106c0 con 0x7fa04010e0f0 2024-08-18T23:51:10.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.644+0000 7fa048337640 1 -- 172.21.15.78:0/1427892200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04019f680 con 0x7fa04010e0f0 2024-08-18T23:51:10.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.644+0000 7fa048337640 1 -- 172.21.15.78:0/1427892200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0401a6f60 con 0x7fa04010e0f0 2024-08-18T23:51:10.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.646+0000 7fa0377fe640 1 -- 172.21.15.78:0/1427892200 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa038003970 con 0x7fa04010e0f0 2024-08-18T23:51:10.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.646+0000 7fa04a735740 1 Processor -- start 2024-08-18T23:51:10.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa04a735740 1 -- start start 2024-08-18T23:51:10.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa04a735740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa83799990 0x55aa83799d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:10.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa04a735740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa8379a2b0 0x55aa837a4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:10.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa04a735740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa837a4b70 0x55aa837a6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:10.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa04a735740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55aa836ffc60 con 0x55aa8379a2b0 2024-08-18T23:51:10.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa04a735740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aa83697b90 con 0x55aa83799990 2024-08-18T23:51:10.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa04a735740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55aa836fbc20 con 0x55aa837a4b70 2024-08-18T23:51:10.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa0460d3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa8379a2b0 0x55aa837a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:10.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa0460d3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa8379a2b0 0x55aa837a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49042/0 (socket says 172.21.15.78:49042) 2024-08-18T23:51:10.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa0460d3640 1 -- 172.21.15.78:0/2733691718 learned_addr learned my addr 172.21.15.78:0/2733691718 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:10.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa0377fe640 1 --2- 172.21.15.78:0/1427892200 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa02c0a2b80 0x7fa02c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:10.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa0458d2640 1 --2- 172.21.15.78:0/2733691718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa83799990 0x55aa83799d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:10.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa0458d2640 1 --2- 172.21.15.78:0/2733691718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa837a4b70 0x55aa837a6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:10.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa0468d4640 1 --2- 172.21.15.78:0/1427892200 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa02c0a2b80 0x7fa02c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa0460d3640 1 -- 172.21.15.78:0/2733691718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa837a4b70 msgr2=0x55aa837a6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa0460d3640 1 --2- 172.21.15.78:0/2733691718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa837a4b70 0x55aa837a6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa0377fe640 1 -- 172.21.15.78:0/1427892200 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa0380cf160 con 0x7fa04010e0f0 2024-08-18T23:51:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa0460d3640 1 -- 172.21.15.78:0/2733691718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa83799990 msgr2=0x55aa83799d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa0460d3640 1 --2- 172.21.15.78:0/2733691718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa83799990 0x55aa83799d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa0460d3640 1 -- 172.21.15.78:0/2733691718 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa8340f580 con 0x55aa8379a2b0 2024-08-18T23:51:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.647+0000 7fa0458d2640 1 --2- 172.21.15.78:0/2733691718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa837a4b70 0x55aa837a6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:51:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.648+0000 7fa0460d3640 1 --2- 172.21.15.78:0/2733691718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa8379a2b0 0x55aa837a4630 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009900 tx=0x7fa03c031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38b3c1de6062427 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.648+0000 7fa0468d4640 1 --2- 172.21.15.78:0/1427892200 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa02c0a2b80 0x7fa02c0a5040 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa03800b760 tx=0x7fa03801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.648+0000 7fa034dea640 1 -- 172.21.15.78:0/2733691718 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c040070 con 0x55aa8379a2b0 2024-08-18T23:51:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.648+0000 7fa034dea640 1 -- 172.21.15.78:0/2733691718 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa03c005020 con 0x55aa8379a2b0 2024-08-18T23:51:10.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.648+0000 7fa034dea640 1 -- 172.21.15.78:0/2733691718 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c035040 con 0x55aa8379a2b0 2024-08-18T23:51:10.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.649+0000 7fa04a735740 1 -- 172.21.15.78:0/2733691718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa8379a2b0 msgr2=0x55aa837a4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:10.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.649+0000 7fa04a735740 1 --2- 172.21.15.78:0/2733691718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa8379a2b0 0x55aa837a4630 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009900 tx=0x7fa03c031630 comp rx=0 tx=0).stop 2024-08-18T23:51:10.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.649+0000 7fa04a735740 1 -- 172.21.15.78:0/2733691718 shutdown_connections 2024-08-18T23:51:10.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.649+0000 7fa04a735740 1 --2- 172.21.15.78:0/2733691718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa837a4b70 0x55aa837a6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.649+0000 7fa04a735740 1 --2- 172.21.15.78:0/2733691718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa8379a2b0 0x55aa837a4630 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.649+0000 7fa04a735740 1 --2- 172.21.15.78:0/2733691718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa83799990 0x55aa83799d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.649+0000 7fa04a735740 1 -- 172.21.15.78:0/2733691718 >> 172.21.15.78:0/2733691718 conn(0x55aa83795f20 msgr2=0x55aa83796300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:10.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.649+0000 7fa04a735740 1 -- 172.21.15.78:0/2733691718 shutdown_connections 2024-08-18T23:51:10.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.649+0000 7fa04a735740 1 -- 172.21.15.78:0/2733691718 wait complete. 2024-08-18T23:51:10.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.649+0000 7fa04a735740 1 -- start start 2024-08-18T23:51:10.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.649+0000 7fa04a735740 1 Processor -- start 2024-08-18T23:51:10.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa04a735740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa83799990 0x55aa837a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:10.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa04a735740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa8379a2b0 0x55aa837a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:10.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa04a735740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa837a4b70 0x55aa837a2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:10.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa04a735740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55aa837a7740 con 0x55aa8379a2b0 2024-08-18T23:51:10.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa04a735740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aa8369d7c0 con 0x55aa837a4b70 2024-08-18T23:51:10.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa04a735740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55aa837a78c0 con 0x55aa83799990 2024-08-18T23:51:10.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa04a735740 10 client.0 _ll_register_callbacks cb 0x55aa836a6f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:10.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa0460d3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa8379a2b0 0x55aa837a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:10.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa0460d3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa8379a2b0 0x55aa837a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49044/0 (socket says 172.21.15.78:49044) 2024-08-18T23:51:10.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa0460d3640 1 -- 172.21.15.78:0/1729219754 learned_addr learned my addr 172.21.15.78:0/1729219754 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:10.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa0458d2640 1 --2- 172.21.15.78:0/1729219754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa83799990 0x55aa837a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:10.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa0458d2640 1 --2- 172.21.15.78:0/1729219754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa837a4b70 0x55aa837a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa0460d3640 1 -- 172.21.15.78:0/1729219754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa83799990 msgr2=0x55aa837a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa0460d3640 1 --2- 172.21.15.78:0/1729219754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa83799990 0x55aa837a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa0460d3640 1 -- 172.21.15.78:0/1729219754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa837a4b70 msgr2=0x55aa837a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa0460d3640 1 --2- 172.21.15.78:0/1729219754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa837a4b70 0x55aa837a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa0460d3640 1 -- 172.21.15.78:0/1729219754 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa834d9960 con 0x55aa8379a2b0 2024-08-18T23:51:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa0458d2640 1 --2- 172.21.15.78:0/1729219754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa837a4b70 0x55aa837a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:51:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa0460d3640 1 --2- 172.21.15.78:0/1729219754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa8379a2b0 0x55aa837a1f10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa03c02a9a0 tx=0x7fa03c005480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262df9be90175707 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa029ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:51:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa029ffb640 1 -- 172.21.15.78:0/1729219754 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c04c020 con 0x55aa8379a2b0 2024-08-18T23:51:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa04a735740 1 -- 172.21.15.78:0/1729219754 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa03c005020 con 0x55aa8379a2b0 2024-08-18T23:51:10.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.650+0000 7fa029ffb640 1 -- 172.21.15.78:0/1729219754 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa03c004ab0 con 0x55aa8379a2b0 2024-08-18T23:51:10.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.651+0000 7fa029ffb640 1 -- 172.21.15.78:0/1729219754 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa03c057050 con 0x55aa8379a2b0 2024-08-18T23:51:10.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.651+0000 7fa04a735740 10 client.16596 fetch_fsmap learned FSMap version 29 2024-08-18T23:51:10.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.651+0000 7fa04a735740 1 -- 172.21.15.78:0/1729219754 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aa837b3700 con 0x55aa8379a2b0 2024-08-18T23:51:10.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.651+0000 7fa029ffb640 1 -- 172.21.15.78:0/1729219754 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa03c005020 con 0x55aa8379a2b0 2024-08-18T23:51:10.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.651+0000 7fa029ffb640 1 -- 172.21.15.78:0/1729219754 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fa03c044400 con 0x55aa8379a2b0 2024-08-18T23:51:10.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.651+0000 7fa04a735740 10 client.16596 fetch_fsmap finished waiting for FSMap version 29 2024-08-18T23:51:10.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.651+0000 7fa04a735740 10 client.16596 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:51:10.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.651+0000 7fa04a735740 20 client.16596 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:10.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.651+0000 7fa04a735740 1 --2- 172.21.15.78:0/1729219754 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55aa837b4a50 0x55aa837b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:10.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.651+0000 7fa04a735740 4 client.16596 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:51:10.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.651+0000 7fa04a735740 1 -- 172.21.15.78:0/1729219754 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa8370d1d0 con 0x55aa837b4a50 2024-08-18T23:51:10.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.651+0000 7fa0458d2640 1 --2- 172.21.15.78:0/1729219754 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55aa837b4a50 0x55aa837b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:10.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.652+0000 7fa029ffb640 10 client.16596 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:10.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.652+0000 7fa0458d2640 1 --2- 172.21.15.78:0/1729219754 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55aa837b4a50 0x55aa837b6e40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58f4e1a506420a8a server_cookie=979bc0b506a4ba60 in_seq=0 out_seq=0 2024-08-18T23:51:10.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.652+0000 7fa029ffb640 1 -- 172.21.15.78:0/1729219754 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aa8370d1d0 con 0x55aa837b4a50 2024-08-18T23:51:10.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.652+0000 7fa029ffb640 10 client.16596 handle_command_reply: tid=0 2024-08-18T23:51:10.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.652+0000 7fa04a735740 1 client.16596 shutdown 2024-08-18T23:51:10.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.653+0000 7fa04a735740 1 -- 172.21.15.78:0/1729219754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa8379a2b0 msgr2=0x55aa837a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:10.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.653+0000 7fa04a735740 1 --2- 172.21.15.78:0/1729219754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa8379a2b0 0x55aa837a1f10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa03c02a9a0 tx=0x7fa03c005480 comp rx=0 tx=0).stop 2024-08-18T23:51:10.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.653+0000 7fa04a735740 1 -- 172.21.15.78:0/1729219754 shutdown_connections 2024-08-18T23:51:10.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.653+0000 7fa04a735740 1 --2- 172.21.15.78:0/1729219754 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55aa837b4a50 0x55aa837b6e40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.653+0000 7fa04a735740 1 --2- 172.21.15.78:0/1729219754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa837a4b70 0x55aa837a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.653+0000 7fa04a735740 1 --2- 172.21.15.78:0/1729219754 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa8379a2b0 0x55aa837a1f10 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.653+0000 7fa029ffb640 0 client.16596 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:10.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.653+0000 7fa04a735740 1 --2- 172.21.15.78:0/1729219754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa83799990 0x55aa837a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.653+0000 7fa04a735740 1 -- 172.21.15.78:0/1729219754 >> 172.21.15.78:0/1729219754 conn(0x55aa83795f20 msgr2=0x55aa83795080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:10.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.653+0000 7fa04a735740 1 -- 172.21.15.78:0/1729219754 shutdown_connections 2024-08-18T23:51:10.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.653+0000 7fa04a735740 1 -- 172.21.15.78:0/1729219754 wait complete. 2024-08-18T23:51:10.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.653+0000 7fa04a735740 20 client.16596 trim_cache size 0 max 16384 2024-08-18T23:51:10.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.670+0000 7fa04a735740 1 Processor -- start 2024-08-18T23:51:10.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.670+0000 7fa04a735740 1 -- start start 2024-08-18T23:51:10.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.670+0000 7fa04a735740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa83799990 0x55aa837a0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:10.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.670+0000 7fa04a735740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa8379a2b0 0x55aa837a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:10.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.670+0000 7fa04a735740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa837a4b70 0x55aa837bc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:10.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.670+0000 7fa04a735740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401b3490 con 0x55aa837a4b70 2024-08-18T23:51:10.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.670+0000 7fa04a735740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401b0d30 con 0x55aa83799990 2024-08-18T23:51:10.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.670+0000 7fa04a735740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401aeb80 con 0x55aa8379a2b0 2024-08-18T23:51:10.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.670+0000 7fa0458d2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa837a4b70 0x55aa837bc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:10.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.670+0000 7fa0458d2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa837a4b70 0x55aa837bc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49048/0 (socket says 172.21.15.78:49048) 2024-08-18T23:51:10.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.670+0000 7fa0458d2640 1 -- 172.21.15.78:0/2266728169 learned_addr learned my addr 172.21.15.78:0/2266728169 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:10.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.671+0000 7fa0460d3640 1 --2- 172.21.15.78:0/2266728169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa8379a2b0 0x55aa837a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:10.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.671+0000 7fa0458d2640 1 --2- 172.21.15.78:0/2266728169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa83799990 0x55aa837a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:10.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.671+0000 7fa0458d2640 1 -- 172.21.15.78:0/2266728169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa8379a2b0 msgr2=0x55aa837a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:10.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.671+0000 7fa0458d2640 1 --2- 172.21.15.78:0/2266728169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa8379a2b0 0x55aa837a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.671+0000 7fa0458d2640 1 -- 172.21.15.78:0/2266728169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa83799990 msgr2=0x55aa837a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:10.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.671+0000 7fa0458d2640 1 --2- 172.21.15.78:0/2266728169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa83799990 0x55aa837a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.671+0000 7fa0458d2640 1 -- 172.21.15.78:0/2266728169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa8379c800 con 0x55aa837a4b70 2024-08-18T23:51:10.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.671+0000 7fa0458d2640 1 --2- 172.21.15.78:0/2266728169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa837a4b70 0x55aa837bc4f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa04010f550 tx=0x7fa030014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185021c28870a450 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:10.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.671+0000 7fa02bfff640 1 -- 172.21.15.78:0/2266728169 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030025070 con 0x55aa837a4b70 2024-08-18T23:51:10.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.671+0000 7fa02bfff640 1 -- 172.21.15.78:0/2266728169 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa030023030 con 0x55aa837a4b70 2024-08-18T23:51:10.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.671+0000 7fa02bfff640 1 -- 172.21.15.78:0/2266728169 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030004a00 con 0x55aa837a4b70 2024-08-18T23:51:10.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.672+0000 7fa04a735740 1 -- 172.21.15.78:0/2266728169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa837a4b70 msgr2=0x55aa837bc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:10.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.672+0000 7fa04a735740 1 --2- 172.21.15.78:0/2266728169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa837a4b70 0x55aa837bc4f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa04010f550 tx=0x7fa030014040 comp rx=0 tx=0).stop 2024-08-18T23:51:10.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.672+0000 7fa04a735740 1 -- 172.21.15.78:0/2266728169 shutdown_connections 2024-08-18T23:51:10.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.672+0000 7fa04a735740 1 --2- 172.21.15.78:0/2266728169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa837a4b70 0x55aa837bc4f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.672+0000 7fa04a735740 1 --2- 172.21.15.78:0/2266728169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa8379a2b0 0x55aa837a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.672+0000 7fa04a735740 1 --2- 172.21.15.78:0/2266728169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa83799990 0x55aa837a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.672+0000 7fa04a735740 1 -- 172.21.15.78:0/2266728169 >> 172.21.15.78:0/2266728169 conn(0x55aa83795f20 msgr2=0x55aa837a3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.672+0000 7fa04a735740 1 -- 172.21.15.78:0/2266728169 shutdown_connections 2024-08-18T23:51:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.672+0000 7fa04a735740 1 -- 172.21.15.78:0/2266728169 wait complete. 2024-08-18T23:51:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.672+0000 7fa04a735740 1 -- start start 2024-08-18T23:51:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.672+0000 7fa04a735740 1 Processor -- start 2024-08-18T23:51:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa04a735740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa83799990 0x55aa837ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:10.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa04a735740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa8379a2b0 0x55aa8379e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:10.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa04a735740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa837a4b70 0x55aa8379eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:10.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa04a735740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401af1f0 con 0x55aa83799990 2024-08-18T23:51:10.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa04a735740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401b0660 con 0x55aa837a4b70 2024-08-18T23:51:10.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa04a735740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa0401af8c0 con 0x55aa8379a2b0 2024-08-18T23:51:10.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa04a735740 10 client.0 _ll_register_callbacks cb 0x7fa03c02aba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:10.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa0458d2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa83799990 0x55aa837ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:10.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa0458d2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa83799990 0x55aa837ad2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49054/0 (socket says 172.21.15.78:49054) 2024-08-18T23:51:10.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa0458d2640 1 -- 172.21.15.78:0/1953672197 learned_addr learned my addr 172.21.15.78:0/1953672197 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:10.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa0460d3640 1 --2- 172.21.15.78:0/1953672197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa8379a2b0 0x55aa8379e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:10.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa0458d2640 1 -- 172.21.15.78:0/1953672197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa8379a2b0 msgr2=0x55aa8379e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:10.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa0458d2640 1 --2- 172.21.15.78:0/1953672197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa8379a2b0 0x55aa8379e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa0458d2640 1 -- 172.21.15.78:0/1953672197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa837a4b70 msgr2=0x55aa8379eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:10.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa0458d2640 1 --2- 172.21.15.78:0/1953672197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa837a4b70 0x55aa8379eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:10.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa0458d2640 1 -- 172.21.15.78:0/1953672197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa8379f4a0 con 0x55aa83799990 2024-08-18T23:51:10.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa0460d3640 1 --2- 172.21.15.78:0/1953672197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa8379a2b0 0x55aa8379e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:51:10.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa0458d2640 1 --2- 172.21.15.78:0/1953672197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa83799990 0x55aa837ad2f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa030010cd0 tx=0x7fa030011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a1b264090a8c06 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:10.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa029ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:51:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa029ffb640 1 -- 172.21.15.78:0/1953672197 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030025070 con 0x55aa83799990 2024-08-18T23:51:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa029ffb640 1 -- 172.21.15.78:0/1953672197 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa030005230 con 0x55aa83799990 2024-08-18T23:51:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.673+0000 7fa04a735740 1 -- 172.21.15.78:0/1953672197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa030023030 con 0x55aa83799990 2024-08-18T23:51:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.674+0000 7fa029ffb640 1 -- 172.21.15.78:0/1953672197 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa030005430 con 0x55aa83799990 2024-08-18T23:51:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.674+0000 7fa04a735740 10 client.16602 fetch_fsmap learned FSMap version 29 2024-08-18T23:51:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.674+0000 7fa04a735740 1 -- 172.21.15.78:0/1953672197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aa8381f500 con 0x55aa83799990 2024-08-18T23:51:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.674+0000 7fa029ffb640 1 -- 172.21.15.78:0/1953672197 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa030023030 con 0x55aa83799990 2024-08-18T23:51:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.674+0000 7fa029ffb640 1 -- 172.21.15.78:0/1953672197 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fa030031070 con 0x55aa83799990 2024-08-18T23:51:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.674+0000 7fa04a735740 10 client.16602 fetch_fsmap finished waiting for FSMap version 29 2024-08-18T23:51:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.674+0000 7fa04a735740 10 client.16602 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:51:10.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.674+0000 7fa04a735740 20 client.16602 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:10.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.674+0000 7fa04a735740 1 --2- 172.21.15.78:0/1953672197 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55aa8381fc60 0x55aa83822080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:10.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.674+0000 7fa04a735740 4 client.16602 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "10964c59", "release": true, "await": true}] 2024-08-18T23:51:10.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.674+0000 7fa04a735740 1 -- 172.21.15.78:0/1953672197 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "quiesce db", "set_id": "10964c59", "release": true, "await": true}) -- 0x55aa837076e0 con 0x55aa8381fc60 2024-08-18T23:51:10.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.674+0000 7fa0460d3640 1 --2- 172.21.15.78:0/1953672197 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55aa8381fc60 0x55aa83822080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:10.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.675+0000 7fa0460d3640 1 --2- 172.21.15.78:0/1953672197 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55aa8381fc60 0x55aa83822080 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f714aa6eff35ac4 server_cookie=9f1e28da6da3141 in_seq=0 out_seq=0 2024-08-18T23:51:10.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:10.675+0000 7fa029ffb640 10 client.16602 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:10.684 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:10 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:10.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:10 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:11.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.665+0000 7fa029ffb640 1 -- 172.21.15.78:0/1953672197 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55aa837076e0 con 0x55aa8381fc60 2024-08-18T23:51:11.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.665+0000 7fa029ffb640 10 client.16602 handle_command_reply: tid=0 2024-08-18T23:51:11.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.665+0000 7fa04a735740 1 client.16602 shutdown 2024-08-18T23:51:11.667 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:11 smithi078 ceph-mon[18440]: pgmap v556: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 540 KiB/s wr, 489 op/s 2024-08-18T23:51:11.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.665+0000 7fa04a735740 1 -- 172.21.15.78:0/1953672197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa83799990 msgr2=0x55aa837ad2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:11.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.665+0000 7fa04a735740 1 --2- 172.21.15.78:0/1953672197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa83799990 0x55aa837ad2f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa030010cd0 tx=0x7fa030011e00 comp rx=0 tx=0).stop 2024-08-18T23:51:11.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.665+0000 7fa04a735740 1 -- 172.21.15.78:0/1953672197 shutdown_connections 2024-08-18T23:51:11.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.665+0000 7fa04a735740 1 --2- 172.21.15.78:0/1953672197 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55aa8381fc60 0x55aa83822080 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:11.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.665+0000 7fa04a735740 1 --2- 172.21.15.78:0/1953672197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa837a4b70 0x55aa8379eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:11.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.665+0000 7fa04a735740 1 --2- 172.21.15.78:0/1953672197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa8379a2b0 0x55aa8379e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:11.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.665+0000 7fa04a735740 1 --2- 172.21.15.78:0/1953672197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aa83799990 0x55aa837ad2f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:11.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.665+0000 7fa04a735740 1 -- 172.21.15.78:0/1953672197 >> 172.21.15.78:0/1953672197 conn(0x55aa83795f20 msgr2=0x55aa83794860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:11.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.665+0000 7fa029ffb640 0 client.16602 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:11.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.665+0000 7fa04a735740 1 -- 172.21.15.78:0/1953672197 shutdown_connections 2024-08-18T23:51:11.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.665+0000 7fa04a735740 1 -- 172.21.15.78:0/1953672197 wait complete. 2024-08-18T23:51:11.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.665+0000 7fa04a735740 20 client.16602 trim_cache size 0 max 16384 2024-08-18T23:51:11.669 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:51:11.669 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 29, 2024-08-18T23:51:11.669 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24452, 2024-08-18T23:51:11.669 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 24, 2024-08-18T23:51:11.669 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-08-18T23:51:11.669 INFO:teuthology.orchestra.run.smithi078.stdout: "10964c59": { 2024-08-18T23:51:11.669 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 24, 2024-08-18T23:51:11.669 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-08-18T23:51:11.669 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:51:11.669 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-08-18T23:51:11.669 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:51:11.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:51:11.670 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-08-18T23:51:11.670 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 139.2, 2024-08-18T23:51:11.670 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-08-18T23:51:11.670 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-08-18T23:51:11.670 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-08-18T23:51:11.670 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:51:11.670 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-08-18T23:51:11.670 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:51:11.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:51:11.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:51:11.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:51:11.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:51:11.671 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:51:11.671 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:51:11.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.666+0000 7fa048337640 1 -- 172.21.15.78:0/1427892200 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa02c0a2b80 msgr2=0x7fa02c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:11.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.666+0000 7fa048337640 1 --2- 172.21.15.78:0/1427892200 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa02c0a2b80 0x7fa02c0a5040 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa03800b760 tx=0x7fa03801e000 comp rx=0 tx=0).stop 2024-08-18T23:51:11.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.666+0000 7fa048337640 1 -- 172.21.15.78:0/1427892200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa04010e0f0 msgr2=0x7fa04019ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:11.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.666+0000 7fa048337640 1 --2- 172.21.15.78:0/1427892200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa04010e0f0 0x7fa04019ecb0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa038009fd0 tx=0x7fa03800b260 comp rx=0 tx=0).stop 2024-08-18T23:51:11.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.666+0000 7fa048337640 1 -- 172.21.15.78:0/1427892200 shutdown_connections 2024-08-18T23:51:11.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.666+0000 7fa048337640 1 --2- 172.21.15.78:0/1427892200 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fa02c0a2b80 0x7fa02c0a5040 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:11.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.666+0000 7fa048337640 1 --2- 172.21.15.78:0/1427892200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa04010e0f0 0x7fa04019ecb0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:11.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.666+0000 7fa048337640 1 --2- 172.21.15.78:0/1427892200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa040074c70 0x7fa04019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:11.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.666+0000 7fa048337640 1 --2- 172.21.15.78:0/1427892200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa040074350 0x7fa04019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:11.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.666+0000 7fa048337640 1 -- 172.21.15.78:0/1427892200 >> 172.21.15.78:0/1427892200 conn(0x7fa0400fd5f0 msgr2=0x7fa0400fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:11.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.666+0000 7fa048337640 1 -- 172.21.15.78:0/1427892200 shutdown_connections 2024-08-18T23:51:11.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:11.666+0000 7fa048337640 1 -- 172.21.15.78:0/1427892200 wait complete. 2024-08-18T23:51:11.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:11 smithi161 ceph-mon[22027]: pgmap v556: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 540 KiB/s wr, 489 op/s 2024-08-18T23:51:11.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:11 smithi123 ceph-mon[21808]: pgmap v556: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 540 KiB/s wr, 489 op/s 2024-08-18T23:51:12.335 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '10964c59', total seconds elapsed: 25.2 2024-08-18T23:51:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:13 smithi161 ceph-mon[22027]: pgmap v557: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 256 KiB/s wr, 718 op/s 2024-08-18T23:51:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:13 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:51:13.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:13 smithi078 ceph-mon[18440]: pgmap v557: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 256 KiB/s wr, 718 op/s 2024-08-18T23:51:13.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:13 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:51:13.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:13 smithi123 ceph-mon[21808]: pgmap v557: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 256 KiB/s wr, 718 op/s 2024-08-18T23:51:13.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:13 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:51:14.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:14 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:51:14.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:14 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:51:14.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:14 smithi161 ceph-mon[22027]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:51:14.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:14 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:51:14.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:14 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:51:14.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:14 smithi078 ceph-mon[18440]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:51:14.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:14 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:51:14.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:14 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:51:14.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:14 smithi123 ceph-mon[21808]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:51:15.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:15 smithi078 ceph-mon[18440]: pgmap v558: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 62 KiB/s wr, 772 op/s 2024-08-18T23:51:15.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:15 smithi161 ceph-mon[22027]: pgmap v558: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 62 KiB/s wr, 772 op/s 2024-08-18T23:51:15.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:15 smithi123 ceph-mon[21808]: pgmap v558: 129 pgs: 129 active+clean; 745 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 62 KiB/s wr, 772 op/s 2024-08-18T23:51:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:17 smithi161 ceph-mon[22027]: pgmap v559: 129 pgs: 129 active+clean; 738 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 276 KiB/s wr, 853 op/s 2024-08-18T23:51:17.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:17 smithi078 ceph-mon[18440]: pgmap v559: 129 pgs: 129 active+clean; 738 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 276 KiB/s wr, 853 op/s 2024-08-18T23:51:17.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:17 smithi123 ceph-mon[21808]: pgmap v559: 129 pgs: 129 active+clean; 738 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 276 KiB/s wr, 853 op/s 2024-08-18T23:51:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:19 smithi161 ceph-mon[22027]: pgmap v560: 129 pgs: 129 active+clean; 733 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 683 KiB/s wr, 959 op/s 2024-08-18T23:51:19.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:19 smithi078 ceph-mon[18440]: pgmap v560: 129 pgs: 129 active+clean; 733 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 683 KiB/s wr, 959 op/s 2024-08-18T23:51:19.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:19 smithi123 ceph-mon[21808]: pgmap v560: 129 pgs: 129 active+clean; 733 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 683 KiB/s wr, 959 op/s 2024-08-18T23:51:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:21 smithi161 ceph-mon[22027]: pgmap v561: 129 pgs: 129 active+clean; 718 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 937 KiB/s wr, 1.09k op/s 2024-08-18T23:51:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:21 smithi078 ceph-mon[18440]: pgmap v561: 129 pgs: 129 active+clean; 718 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 937 KiB/s wr, 1.09k op/s 2024-08-18T23:51:21.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:21 smithi123 ceph-mon[21808]: pgmap v561: 129 pgs: 129 active+clean; 718 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 937 KiB/s wr, 1.09k op/s 2024-08-18T23:51:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:23 smithi161 ceph-mon[22027]: pgmap v562: 129 pgs: 129 active+clean; 704 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 975 KiB/s wr, 933 op/s 2024-08-18T23:51:23.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:23 smithi078 ceph-mon[18440]: pgmap v562: 129 pgs: 129 active+clean; 704 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 975 KiB/s wr, 933 op/s 2024-08-18T23:51:23.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:23 smithi123 ceph-mon[21808]: pgmap v562: 129 pgs: 129 active+clean; 704 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 975 KiB/s wr, 933 op/s 2024-08-18T23:51:24.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:24 smithi161 ceph-mon[22027]: pgmap v563: 129 pgs: 129 active+clean; 700 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 699 op/s 2024-08-18T23:51:24.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:24 smithi078 ceph-mon[18440]: pgmap v563: 129 pgs: 129 active+clean; 700 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 699 op/s 2024-08-18T23:51:24.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:24 smithi123 ceph-mon[21808]: pgmap v563: 129 pgs: 129 active+clean; 700 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 699 op/s 2024-08-18T23:51:27.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:27 smithi161 ceph-mon[22027]: pgmap v564: 129 pgs: 129 active+clean; 683 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 806 op/s 2024-08-18T23:51:27.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:27 smithi078 ceph-mon[18440]: pgmap v564: 129 pgs: 129 active+clean; 683 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 806 op/s 2024-08-18T23:51:27.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:27 smithi123 ceph-mon[21808]: pgmap v564: 129 pgs: 129 active+clean; 683 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 806 op/s 2024-08-18T23:51:29.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:29 smithi161 ceph-mon[22027]: pgmap v565: 129 pgs: 129 active+clean; 678 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 819 op/s 2024-08-18T23:51:29.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:29 smithi078 ceph-mon[18440]: pgmap v565: 129 pgs: 129 active+clean; 678 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 819 op/s 2024-08-18T23:51:29.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:29 smithi123 ceph-mon[21808]: pgmap v565: 129 pgs: 129 active+clean; 678 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 819 op/s 2024-08-18T23:51:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:31 smithi161 ceph-mon[22027]: pgmap v566: 129 pgs: 129 active+clean; 656 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 847 op/s 2024-08-18T23:51:31.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:31 smithi078 ceph-mon[18440]: pgmap v566: 129 pgs: 129 active+clean; 656 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 847 op/s 2024-08-18T23:51:31.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:31 smithi123 ceph-mon[21808]: pgmap v566: 129 pgs: 129 active+clean; 656 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 847 op/s 2024-08-18T23:51:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:33 smithi161 ceph-mon[22027]: pgmap v567: 129 pgs: 129 active+clean; 630 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 819 op/s 2024-08-18T23:51:33.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:33 smithi078 ceph-mon[18440]: pgmap v567: 129 pgs: 129 active+clean; 630 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 819 op/s 2024-08-18T23:51:33.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:33 smithi123 ceph-mon[21808]: pgmap v567: 129 pgs: 129 active+clean; 630 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 819 op/s 2024-08-18T23:51:35.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:35 smithi161 ceph-mon[22027]: pgmap v568: 129 pgs: 129 active+clean; 622 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 737 op/s 2024-08-18T23:51:35.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:35 smithi078 ceph-mon[18440]: pgmap v568: 129 pgs: 129 active+clean; 622 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 737 op/s 2024-08-18T23:51:35.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:35 smithi123 ceph-mon[21808]: pgmap v568: 129 pgs: 129 active+clean; 622 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 737 op/s 2024-08-18T23:51:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:37 smithi161 ceph-mon[22027]: pgmap v569: 129 pgs: 129 active+clean; 604 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 858 op/s 2024-08-18T23:51:37.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:37 smithi078 ceph-mon[18440]: pgmap v569: 129 pgs: 129 active+clean; 604 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 858 op/s 2024-08-18T23:51:37.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:37 smithi123 ceph-mon[21808]: pgmap v569: 129 pgs: 129 active+clean; 604 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 858 op/s 2024-08-18T23:51:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:39 smithi161 ceph-mon[22027]: pgmap v570: 129 pgs: 129 active+clean; 597 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 764 op/s 2024-08-18T23:51:39.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:39 smithi078 ceph-mon[18440]: pgmap v570: 129 pgs: 129 active+clean; 597 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 764 op/s 2024-08-18T23:51:39.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:39 smithi123 ceph-mon[21808]: pgmap v570: 129 pgs: 129 active+clean; 597 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 764 op/s 2024-08-18T23:51:40.143 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:51:40.415 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:51:40.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.973+0000 7f4b831be640 1 -- 172.21.15.78:0/3512249666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b7c103410 msgr2=0x7f4b7c1078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:40.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.973+0000 7f4b831be640 1 --2- 172.21.15.78:0/3512249666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b7c103410 0x7f4b7c1078c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009930 tx=0x7f4b6802a420 comp rx=0 tx=0).stop 2024-08-18T23:51:40.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.973+0000 7f4b831be640 1 -- 172.21.15.78:0/3512249666 shutdown_connections 2024-08-18T23:51:40.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.973+0000 7f4b831be640 1 --2- 172.21.15.78:0/3512249666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b7c107fd0 0x7f4b7c10e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:40.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.974+0000 7f4b831be640 1 --2- 172.21.15.78:0/3512249666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b7c103410 0x7f4b7c1078c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:40.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.974+0000 7f4b831be640 1 --2- 172.21.15.78:0/3512249666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b7c102a60 0x7f4b7c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:40.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.974+0000 7f4b831be640 1 -- 172.21.15.78:0/3512249666 >> 172.21.15.78:0/3512249666 conn(0x7f4b7c0fc500 msgr2=0x7f4b7c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:40.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.974+0000 7f4b831be640 1 -- 172.21.15.78:0/3512249666 shutdown_connections 2024-08-18T23:51:40.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.974+0000 7f4b831be640 1 -- 172.21.15.78:0/3512249666 wait complete. 2024-08-18T23:51:40.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.974+0000 7f4b831be640 1 Processor -- start 2024-08-18T23:51:40.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.974+0000 7f4b831be640 1 -- start start 2024-08-18T23:51:40.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.974+0000 7f4b831be640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b7c102a60 0x7f4b7c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:40.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.974+0000 7f4b831be640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b7c103410 0x7f4b7c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:40.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.974+0000 7f4b831be640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b7c107fd0 0x7f4b7c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:40.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.974+0000 7f4b831be640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c111dd0 con 0x7f4b7c102a60 2024-08-18T23:51:40.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.974+0000 7f4b831be640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c111c50 con 0x7f4b7c103410 2024-08-18T23:51:40.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.974+0000 7f4b831be640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c111f50 con 0x7f4b7c107fd0 2024-08-18T23:51:40.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.974+0000 7f4b80f5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b7c102a60 0x7f4b7c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:40.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.974+0000 7f4b80f5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b7c102a60 0x7f4b7c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59488/0 (socket says 172.21.15.78:59488) 2024-08-18T23:51:40.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.974+0000 7f4b80f5a640 1 -- 172.21.15.78:0/987908253 learned_addr learned my addr 172.21.15.78:0/987908253 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:40.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.974+0000 7f4b7bfff640 1 --2- 172.21.15.78:0/987908253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b7c103410 0x7f4b7c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:40.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.975+0000 7f4b8175b640 1 --2- 172.21.15.78:0/987908253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b7c107fd0 0x7f4b7c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:40.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.975+0000 7f4b80f5a640 1 -- 172.21.15.78:0/987908253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b7c107fd0 msgr2=0x7f4b7c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:40.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.975+0000 7f4b80f5a640 1 --2- 172.21.15.78:0/987908253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b7c107fd0 0x7f4b7c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:40.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.975+0000 7f4b80f5a640 1 -- 172.21.15.78:0/987908253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b7c103410 msgr2=0x7f4b7c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:40.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.975+0000 7f4b80f5a640 1 --2- 172.21.15.78:0/987908253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b7c103410 0x7f4b7c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:40.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.975+0000 7f4b80f5a640 1 -- 172.21.15.78:0/987908253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7c19d320 con 0x7f4b7c102a60 2024-08-18T23:51:40.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.975+0000 7f4b8175b640 1 --2- 172.21.15.78:0/987908253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b7c107fd0 0x7f4b7c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:51:40.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.975+0000 7f4b80f5a640 1 --2- 172.21.15.78:0/987908253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b7c102a60 0x7f4b7c198370 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000e9f0 tx=0x7f4b7000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:40.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.975+0000 7f4b79ffb640 1 -- 172.21.15.78:0/987908253 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7000cd80 con 0x7f4b7c102a60 2024-08-18T23:51:40.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.975+0000 7f4b831be640 1 -- 172.21.15.78:0/987908253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b7c19d610 con 0x7f4b7c102a60 2024-08-18T23:51:40.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.975+0000 7f4b79ffb640 1 -- 172.21.15.78:0/987908253 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b70010910 con 0x7f4b7c102a60 2024-08-18T23:51:40.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.975+0000 7f4b79ffb640 1 -- 172.21.15.78:0/987908253 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b700188f0 con 0x7f4b7c102a60 2024-08-18T23:51:40.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.975+0000 7f4b831be640 1 -- 172.21.15.78:0/987908253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b7c1a4ef0 con 0x7f4b7c102a60 2024-08-18T23:51:40.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.977+0000 7f4b79ffb640 1 -- 172.21.15.78:0/987908253 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4b70010ac0 con 0x7f4b7c102a60 2024-08-18T23:51:40.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.978+0000 7f4b855bc740 1 Processor -- start 2024-08-18T23:51:40.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.978+0000 7f4b855bc740 1 -- start start 2024-08-18T23:51:40.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.978+0000 7f4b79ffb640 1 --2- 172.21.15.78:0/987908253 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4b6c0a2b80 0x7f4b6c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:40.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.978+0000 7f4b855bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 0x556d7f2085f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:40.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.978+0000 7f4b855bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208b30 0x556d7f212eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:40.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.978+0000 7f4b855bc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f2133f0 0x556d7f2157e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:40.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.978+0000 7f4b855bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556d7f16d980 con 0x556d7f208210 2024-08-18T23:51:40.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.978+0000 7f4b855bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556d7f105a70 con 0x556d7f208b30 2024-08-18T23:51:40.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.978+0000 7f4b855bc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556d7f169940 con 0x556d7f2133f0 2024-08-18T23:51:40.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.978+0000 7f4b8175b640 1 --2- 172.21.15.78:0/987908253 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4b6c0a2b80 0x7f4b6c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:40.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.978+0000 7f4b80f5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 0x556d7f2085f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:40.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.978+0000 7f4b80f5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 0x556d7f2085f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59498/0 (socket says 172.21.15.78:59498) 2024-08-18T23:51:40.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.978+0000 7f4b80f5a640 1 -- 172.21.15.78:0/3467294910 learned_addr learned my addr 172.21.15.78:0/3467294910 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:40.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.978+0000 7f4b79ffb640 1 -- 172.21.15.78:0/987908253 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f4b700cf890 con 0x7f4b7c102a60 2024-08-18T23:51:40.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.978+0000 7f4b7bfff640 1 --2- 172.21.15.78:0/3467294910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208b30 0x556d7f212eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:40.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.979+0000 7f4b8175b640 1 --2- 172.21.15.78:0/3467294910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f2133f0 0x556d7f2157e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:40.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.979+0000 7f4b80f5a640 1 -- 172.21.15.78:0/3467294910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f2133f0 msgr2=0x556d7f2157e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:40.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.979+0000 7f4b80f5a640 1 --2- 172.21.15.78:0/3467294910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f2133f0 0x556d7f2157e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:40.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.979+0000 7f4b80f5a640 1 -- 172.21.15.78:0/3467294910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208b30 msgr2=0x556d7f212eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:40.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.979+0000 7f4b80f5a640 1 --2- 172.21.15.78:0/3467294910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208b30 0x556d7f212eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:40.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.979+0000 7f4b80f5a640 1 -- 172.21.15.78:0/3467294910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d7f1686e0 con 0x556d7f208210 2024-08-18T23:51:40.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.979+0000 7f4b7bfff640 1 --2- 172.21.15.78:0/3467294910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208b30 0x556d7f212eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:51:40.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.979+0000 7f4b8175b640 1 --2- 172.21.15.78:0/3467294910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f2133f0 0x556d7f2157e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:51:40.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.979+0000 7f4b8175b640 1 --2- 172.21.15.78:0/987908253 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4b6c0a2b80 0x7f4b6c0a5040 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4b740094d0 tx=0x7f4b74013b20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:40.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.979+0000 7f4b80f5a640 1 --2- 172.21.15.78:0/3467294910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 0x556d7f2085f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4b70098160 tx=0x7f4b700980d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7264af4f96d21910 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:40.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.979+0000 7f4b66ffd640 1 -- 172.21.15.78:0/3467294910 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70026040 con 0x556d7f208210 2024-08-18T23:51:40.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.979+0000 7f4b66ffd640 1 -- 172.21.15.78:0/3467294910 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b7009bd00 con 0x556d7f208210 2024-08-18T23:51:40.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.979+0000 7f4b66ffd640 1 -- 172.21.15.78:0/3467294910 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70098370 con 0x556d7f208210 2024-08-18T23:51:40.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.980+0000 7f4b855bc740 1 -- 172.21.15.78:0/3467294910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 msgr2=0x556d7f2085f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:40.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.980+0000 7f4b855bc740 1 --2- 172.21.15.78:0/3467294910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 0x556d7f2085f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4b70098160 tx=0x7f4b700980d0 comp rx=0 tx=0).stop 2024-08-18T23:51:40.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.980+0000 7f4b855bc740 1 -- 172.21.15.78:0/3467294910 shutdown_connections 2024-08-18T23:51:40.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.980+0000 7f4b855bc740 1 --2- 172.21.15.78:0/3467294910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f2133f0 0x556d7f2157e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:40.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.980+0000 7f4b855bc740 1 --2- 172.21.15.78:0/3467294910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208b30 0x556d7f212eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:40.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.980+0000 7f4b855bc740 1 --2- 172.21.15.78:0/3467294910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 0x556d7f2085f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:40.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.980+0000 7f4b855bc740 1 -- 172.21.15.78:0/3467294910 >> 172.21.15.78:0/3467294910 conn(0x556d7f2047a0 msgr2=0x556d7f204b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:40.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.980+0000 7f4b855bc740 1 -- 172.21.15.78:0/3467294910 shutdown_connections 2024-08-18T23:51:40.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.980+0000 7f4b855bc740 1 -- 172.21.15.78:0/3467294910 wait complete. 2024-08-18T23:51:40.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.980+0000 7f4b855bc740 1 -- start start 2024-08-18T23:51:40.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.980+0000 7f4b855bc740 1 Processor -- start 2024-08-18T23:51:40.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.981+0000 7f4b855bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208210 0x556d7f2102e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:40.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.981+0000 7f4b855bc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f208b30 0x556d7f210820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:40.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.981+0000 7f4b855bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f2133f0 0x556d7f210d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:40.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.981+0000 7f4b855bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556d7f215fc0 con 0x556d7f2133f0 2024-08-18T23:51:40.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.981+0000 7f4b855bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556d7f128aa0 con 0x556d7f208210 2024-08-18T23:51:40.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.981+0000 7f4b855bc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556d7f216140 con 0x556d7f208b30 2024-08-18T23:51:40.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.981+0000 7f4b855bc740 10 client.0 _ll_register_callbacks cb 0x556d7f117e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:40.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.981+0000 7f4b80f5a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208210 0x556d7f2102e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:40.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.981+0000 7f4b80f5a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208210 0x556d7f2102e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:59464/0 (socket says 172.21.15.78:59464) 2024-08-18T23:51:40.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.981+0000 7f4b80f5a640 1 -- 172.21.15.78:0/1360508856 learned_addr learned my addr 172.21.15.78:0/1360508856 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:40.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.981+0000 7f4b8175b640 1 --2- 172.21.15.78:0/1360508856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f2133f0 0x556d7f210d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:40.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.981+0000 7f4b7bfff640 1 --2- 172.21.15.78:0/1360508856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f208b30 0x556d7f210820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:40.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.981+0000 7f4b80f5a640 1 -- 172.21.15.78:0/1360508856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f208b30 msgr2=0x556d7f210820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:40.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.981+0000 7f4b80f5a640 1 --2- 172.21.15.78:0/1360508856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f208b30 0x556d7f210820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:40.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.981+0000 7f4b80f5a640 1 -- 172.21.15.78:0/1360508856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f2133f0 msgr2=0x556d7f210d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:40.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.981+0000 7f4b80f5a640 1 --2- 172.21.15.78:0/1360508856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f2133f0 0x556d7f210d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:40.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.981+0000 7f4b80f5a640 1 -- 172.21.15.78:0/1360508856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d7f10c650 con 0x556d7f208210 2024-08-18T23:51:40.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.981+0000 7f4b80f5a640 1 --2- 172.21.15.78:0/1360508856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208210 0x556d7f2102e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000cf60 tx=0x7f4b700a4770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d47a337af3ae3cfe server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:40.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.981+0000 7f4b5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:51:40.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.982+0000 7f4b5bfff640 1 -- 172.21.15.78:0/1360508856 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b700a00b0 con 0x556d7f208210 2024-08-18T23:51:40.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.982+0000 7f4b855bc740 1 -- 172.21.15.78:0/1360508856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b7009bd00 con 0x556d7f208210 2024-08-18T23:51:40.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.982+0000 7f4b5bfff640 1 -- 172.21.15.78:0/1360508856 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f4b7009b150 con 0x556d7f208210 2024-08-18T23:51:40.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.982+0000 7f4b5bfff640 1 -- 172.21.15.78:0/1360508856 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4b7009b350 con 0x556d7f208210 2024-08-18T23:51:40.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.982+0000 7f4b855bc740 10 client.26015 fetch_fsmap learned FSMap version 30 2024-08-18T23:51:40.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.982+0000 7f4b855bc740 1 -- 172.21.15.78:0/1360508856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556d7eedf3c0 con 0x556d7f208210 2024-08-18T23:51:40.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.982+0000 7f4b5bfff640 1 -- 172.21.15.78:0/1360508856 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b7009bd00 con 0x556d7f208210 2024-08-18T23:51:40.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.982+0000 7f4b5bfff640 1 -- 172.21.15.78:0/1360508856 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4b7009abc0 con 0x556d7f208210 2024-08-18T23:51:40.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.983+0000 7f4b855bc740 10 client.26015 fetch_fsmap finished waiting for FSMap version 30 2024-08-18T23:51:40.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.983+0000 7f4b855bc740 10 client.26015 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:51:40.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.983+0000 7f4b855bc740 20 client.26015 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:40.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.983+0000 7f4b855bc740 1 --2- 172.21.15.78:0/1360508856 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x556d7f222d50 0x556d7f225140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:40.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.983+0000 7f4b855bc740 4 client.26015 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:51:40.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.983+0000 7f4b855bc740 1 -- 172.21.15.78:0/1360508856 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d7f17aef0 con 0x556d7f222d50 2024-08-18T23:51:40.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.983+0000 7f4b7bfff640 1 --2- 172.21.15.78:0/1360508856 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x556d7f222d50 0x556d7f225140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:40.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.983+0000 7f4b5bfff640 10 client.26015 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:40.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.983+0000 7f4b7bfff640 1 --2- 172.21.15.78:0/1360508856 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x556d7f222d50 0x556d7f225140 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13d36a476a7fa528 server_cookie=ed462180f96e77cf in_seq=0 out_seq=0 2024-08-18T23:51:40.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.984+0000 7f4b5bfff640 1 -- 172.21.15.78:0/1360508856 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d7f17aef0 con 0x556d7f222d50 2024-08-18T23:51:40.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.984+0000 7f4b5bfff640 10 client.26015 handle_command_reply: tid=0 2024-08-18T23:51:41.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.984+0000 7f4b855bc740 1 client.26015 shutdown 2024-08-18T23:51:41.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.985+0000 7f4b855bc740 1 -- 172.21.15.78:0/1360508856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208210 msgr2=0x556d7f2102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:41.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.985+0000 7f4b855bc740 1 --2- 172.21.15.78:0/1360508856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208210 0x556d7f2102e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000cf60 tx=0x7f4b700a4770 comp rx=0 tx=0).stop 2024-08-18T23:51:41.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.985+0000 7f4b855bc740 1 -- 172.21.15.78:0/1360508856 shutdown_connections 2024-08-18T23:51:41.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.985+0000 7f4b855bc740 1 --2- 172.21.15.78:0/1360508856 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x556d7f222d50 0x556d7f225140 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.985+0000 7f4b855bc740 1 --2- 172.21.15.78:0/1360508856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f2133f0 0x556d7f210d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.985+0000 7f4b855bc740 1 --2- 172.21.15.78:0/1360508856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f208b30 0x556d7f210820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.985+0000 7f4b5bfff640 0 client.26015 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:41.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.985+0000 7f4b855bc740 1 --2- 172.21.15.78:0/1360508856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208210 0x556d7f2102e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.985+0000 7f4b855bc740 1 -- 172.21.15.78:0/1360508856 >> 172.21.15.78:0/1360508856 conn(0x556d7f2047a0 msgr2=0x556d7f20a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:41.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.985+0000 7f4b855bc740 1 -- 172.21.15.78:0/1360508856 shutdown_connections 2024-08-18T23:51:41.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.985+0000 7f4b855bc740 1 -- 172.21.15.78:0/1360508856 wait complete. 2024-08-18T23:51:41.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:40.985+0000 7f4b855bc740 20 client.26015 trim_cache size 0 max 16384 2024-08-18T23:51:41.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.002+0000 7f4b855bc740 1 Processor -- start 2024-08-18T23:51:41.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.002+0000 7f4b855bc740 1 -- start start 2024-08-18T23:51:41.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.002+0000 7f4b855bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 0x556d7f212500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:41.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.002+0000 7f4b855bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208b30 0x556d7f20fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:41.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.002+0000 7f4b855bc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f2133f0 0x556d7f228160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:41.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.002+0000 7f4b855bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c1af060 con 0x556d7f208210 2024-08-18T23:51:41.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.002+0000 7f4b855bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c1ac8d0 con 0x556d7f208b30 2024-08-18T23:51:41.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.002+0000 7f4b855bc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c1aa7f0 con 0x556d7f2133f0 2024-08-18T23:51:41.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.002+0000 7f4b80f5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 0x556d7f212500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:41.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.002+0000 7f4b80f5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 0x556d7f212500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59522/0 (socket says 172.21.15.78:59522) 2024-08-18T23:51:41.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.002+0000 7f4b80f5a640 1 -- 172.21.15.78:0/2635326283 learned_addr learned my addr 172.21.15.78:0/2635326283 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:41.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.003+0000 7f4b7bfff640 1 --2- 172.21.15.78:0/2635326283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208b30 0x556d7f20fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:41.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.003+0000 7f4b8175b640 1 --2- 172.21.15.78:0/2635326283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f2133f0 0x556d7f228160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:41.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.003+0000 7f4b80f5a640 1 -- 172.21.15.78:0/2635326283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f2133f0 msgr2=0x556d7f228160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:41.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.003+0000 7f4b80f5a640 1 --2- 172.21.15.78:0/2635326283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f2133f0 0x556d7f228160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.003+0000 7f4b80f5a640 1 -- 172.21.15.78:0/2635326283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208b30 msgr2=0x556d7f20fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:41.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.003+0000 7f4b80f5a640 1 --2- 172.21.15.78:0/2635326283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208b30 0x556d7f20fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.003+0000 7f4b80f5a640 1 -- 172.21.15.78:0/2635326283 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d7f20ffd0 con 0x556d7f208210 2024-08-18T23:51:41.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.003+0000 7f4b7bfff640 1 --2- 172.21.15.78:0/2635326283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208b30 0x556d7f20fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:51:41.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.003+0000 7f4b80f5a640 1 --2- 172.21.15.78:0/2635326283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 0x556d7f212500 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4b70098160 tx=0x7f4b700980d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d958276af16ff8 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:41.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.003+0000 7f4b667fc640 1 -- 172.21.15.78:0/2635326283 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b700a00b0 con 0x556d7f208210 2024-08-18T23:51:41.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.003+0000 7f4b667fc640 1 -- 172.21.15.78:0/2635326283 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b70084070 con 0x556d7f208210 2024-08-18T23:51:41.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.003+0000 7f4b667fc640 1 -- 172.21.15.78:0/2635326283 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b700a4bc0 con 0x556d7f208210 2024-08-18T23:51:41.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.004+0000 7f4b855bc740 1 -- 172.21.15.78:0/2635326283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 msgr2=0x556d7f212500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:41.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.004+0000 7f4b855bc740 1 --2- 172.21.15.78:0/2635326283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 0x556d7f212500 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4b70098160 tx=0x7f4b700980d0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.004+0000 7f4b855bc740 1 -- 172.21.15.78:0/2635326283 shutdown_connections 2024-08-18T23:51:41.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.004+0000 7f4b855bc740 1 --2- 172.21.15.78:0/2635326283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f2133f0 0x556d7f228160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.004+0000 7f4b855bc740 1 --2- 172.21.15.78:0/2635326283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208b30 0x556d7f20fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.004+0000 7f4b855bc740 1 --2- 172.21.15.78:0/2635326283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 0x556d7f212500 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.004+0000 7f4b855bc740 1 -- 172.21.15.78:0/2635326283 >> 172.21.15.78:0/2635326283 conn(0x556d7f2047a0 msgr2=0x556d7f21ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:41.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.004+0000 7f4b855bc740 1 -- 172.21.15.78:0/2635326283 shutdown_connections 2024-08-18T23:51:41.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.004+0000 7f4b855bc740 1 -- 172.21.15.78:0/2635326283 wait complete. 2024-08-18T23:51:41.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.004+0000 7f4b855bc740 1 -- start start 2024-08-18T23:51:41.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.004+0000 7f4b855bc740 1 Processor -- start 2024-08-18T23:51:41.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.005+0000 7f4b855bc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 0x556d7f225e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:41.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.005+0000 7f4b855bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208b30 0x556d7f2263d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:41.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.005+0000 7f4b855bc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f2133f0 0x556d7f226910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:41.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.005+0000 7f4b855bc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c1aadb0 con 0x556d7f208210 2024-08-18T23:51:41.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.005+0000 7f4b855bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c1ac200 con 0x556d7f208b30 2024-08-18T23:51:41.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.005+0000 7f4b855bc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c1ab460 con 0x556d7f2133f0 2024-08-18T23:51:41.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.005+0000 7f4b855bc740 10 client.0 _ll_register_callbacks cb 0x7f4b740093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:41.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.005+0000 7f4b80f5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 0x556d7f225e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:41.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.005+0000 7f4b7bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208b30 0x556d7f2263d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:41.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.005+0000 7f4b80f5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 0x556d7f225e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59532/0 (socket says 172.21.15.78:59532) 2024-08-18T23:51:41.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.005+0000 7f4b80f5a640 1 -- 172.21.15.78:0/1763458778 learned_addr learned my addr 172.21.15.78:0/1763458778 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:41.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.005+0000 7f4b8175b640 1 --2- 172.21.15.78:0/1763458778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f2133f0 0x556d7f226910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:41.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.005+0000 7f4b80f5a640 1 -- 172.21.15.78:0/1763458778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f2133f0 msgr2=0x556d7f226910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:41.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.005+0000 7f4b80f5a640 1 --2- 172.21.15.78:0/1763458778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f2133f0 0x556d7f226910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.005+0000 7f4b80f5a640 1 -- 172.21.15.78:0/1763458778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208b30 msgr2=0x556d7f2263d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:41.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.005+0000 7f4b80f5a640 1 --2- 172.21.15.78:0/1763458778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208b30 0x556d7f2263d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.005+0000 7f4b80f5a640 1 -- 172.21.15.78:0/1763458778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d7f28c690 con 0x556d7f208210 2024-08-18T23:51:41.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.005+0000 7f4b80f5a640 1 --2- 172.21.15.78:0/1763458778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 0x556d7f225e90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4b7009bd40 tx=0x7f4b7009be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df487cb0a2c613b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:41.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.005+0000 7f4b5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:51:41.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.006+0000 7f4b5bfff640 1 -- 172.21.15.78:0/1763458778 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b700a00b0 con 0x556d7f208210 2024-08-18T23:51:41.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.006+0000 7f4b5bfff640 1 -- 172.21.15.78:0/1763458778 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f4b700a4bc0 con 0x556d7f208210 2024-08-18T23:51:41.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.006+0000 7f4b855bc740 1 -- 172.21.15.78:0/1763458778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b70084070 con 0x556d7f208210 2024-08-18T23:51:41.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.006+0000 7f4b5bfff640 1 -- 172.21.15.78:0/1763458778 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4b700a4770 con 0x556d7f208210 2024-08-18T23:51:41.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.006+0000 7f4b855bc740 10 client.16620 fetch_fsmap learned FSMap version 30 2024-08-18T23:51:41.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.006+0000 7f4b855bc740 1 -- 172.21.15.78:0/1763458778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556d7f28c690 con 0x556d7f208210 2024-08-18T23:51:41.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.006+0000 7f4b5bfff640 1 -- 172.21.15.78:0/1763458778 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b70084070 con 0x556d7f208210 2024-08-18T23:51:41.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.006+0000 7f4b5bfff640 1 -- 172.21.15.78:0/1763458778 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4b70018bb0 con 0x556d7f208210 2024-08-18T23:51:41.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.006+0000 7f4b855bc740 10 client.16620 fetch_fsmap finished waiting for FSMap version 30 2024-08-18T23:51:41.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.006+0000 7f4b855bc740 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:51:41.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.006+0000 7f4b855bc740 20 client.16620 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:41.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.006+0000 7f4b855bc740 1 --2- 172.21.15.78:0/1763458778 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x556d7f28dc20 0x556d7f290010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:41.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.006+0000 7f4b855bc740 4 client.16620 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:51:41.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.006+0000 7f4b855bc740 1 -- 172.21.15.78:0/1763458778 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556d7f175400 con 0x556d7f28dc20 2024-08-18T23:51:41.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.006+0000 7f4b7bfff640 1 --2- 172.21.15.78:0/1763458778 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x556d7f28dc20 0x556d7f290010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:41.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.007+0000 7f4b7bfff640 1 --2- 172.21.15.78:0/1763458778 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x556d7f28dc20 0x556d7f290010 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be6c4962f4ddb022 server_cookie=59675cd23c61eea7 in_seq=0 out_seq=0 2024-08-18T23:51:41.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.007+0000 7f4b5bfff640 10 client.16620 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:41.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.007+0000 7f4b5bfff640 1 -- 172.21.15.78:0/1763458778 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556d7f175400 con 0x556d7f28dc20 2024-08-18T23:51:41.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.007+0000 7f4b5bfff640 10 client.16620 handle_command_reply: tid=0 2024-08-18T23:51:41.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.007+0000 7f4b855bc740 1 client.16620 shutdown 2024-08-18T23:51:41.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.007+0000 7f4b855bc740 1 -- 172.21.15.78:0/1763458778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 msgr2=0x556d7f225e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:41.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.007+0000 7f4b855bc740 1 --2- 172.21.15.78:0/1763458778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 0x556d7f225e90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4b7009bd40 tx=0x7f4b7009be40 comp rx=0 tx=0).stop 2024-08-18T23:51:41.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.007+0000 7f4b855bc740 1 -- 172.21.15.78:0/1763458778 shutdown_connections 2024-08-18T23:51:41.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.007+0000 7f4b855bc740 1 --2- 172.21.15.78:0/1763458778 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x556d7f28dc20 0x556d7f290010 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.007+0000 7f4b5bfff640 0 client.16620 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:41.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.007+0000 7f4b855bc740 1 --2- 172.21.15.78:0/1763458778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556d7f2133f0 0x556d7f226910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.007+0000 7f4b855bc740 1 --2- 172.21.15.78:0/1763458778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7f208b30 0x556d7f2263d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.007+0000 7f4b855bc740 1 --2- 172.21.15.78:0/1763458778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556d7f208210 0x556d7f225e90 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.007+0000 7f4b855bc740 1 -- 172.21.15.78:0/1763458778 >> 172.21.15.78:0/1763458778 conn(0x556d7f2047a0 msgr2=0x556d7f2030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:41.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.008+0000 7f4b855bc740 1 -- 172.21.15.78:0/1763458778 shutdown_connections 2024-08-18T23:51:41.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.008+0000 7f4b855bc740 1 -- 172.21.15.78:0/1763458778 wait complete. 2024-08-18T23:51:41.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.008+0000 7f4b855bc740 20 client.16620 trim_cache size 0 max 16384 2024-08-18T23:51:41.016 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:51:41.016 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-08-18T23:51:41.016 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-08-18T23:51:41.016 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:51:41.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.008+0000 7f4b831be640 1 -- 172.21.15.78:0/987908253 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4b6c0a2b80 msgr2=0x7f4b6c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:41.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.008+0000 7f4b831be640 1 --2- 172.21.15.78:0/987908253 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4b6c0a2b80 0x7f4b6c0a5040 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4b740094d0 tx=0x7f4b74013b20 comp rx=0 tx=0).stop 2024-08-18T23:51:41.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.008+0000 7f4b831be640 1 -- 172.21.15.78:0/987908253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b7c102a60 msgr2=0x7f4b7c198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:41.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.008+0000 7f4b831be640 1 --2- 172.21.15.78:0/987908253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b7c102a60 0x7f4b7c198370 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000e9f0 tx=0x7f4b7000eeb0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.008+0000 7f4b831be640 1 -- 172.21.15.78:0/987908253 shutdown_connections 2024-08-18T23:51:41.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.008+0000 7f4b831be640 1 --2- 172.21.15.78:0/987908253 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f4b6c0a2b80 0x7f4b6c0a5040 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.008+0000 7f4b831be640 1 --2- 172.21.15.78:0/987908253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b7c107fd0 0x7f4b7c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.008+0000 7f4b831be640 1 --2- 172.21.15.78:0/987908253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b7c103410 0x7f4b7c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.008+0000 7f4b831be640 1 --2- 172.21.15.78:0/987908253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b7c102a60 0x7f4b7c198370 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:41.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.009+0000 7f4b831be640 1 -- 172.21.15.78:0/987908253 >> 172.21.15.78:0/987908253 conn(0x7f4b7c0fc500 msgr2=0x7f4b7c1060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:41.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.009+0000 7f4b831be640 1 -- 172.21.15.78:0/987908253 shutdown_connections 2024-08-18T23:51:41.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:41.009+0000 7f4b831be640 1 -- 172.21.15.78:0/987908253 wait complete. 2024-08-18T23:51:41.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:41 smithi161 ceph-mon[22027]: pgmap v571: 129 pgs: 129 active+clean; 581 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 766 op/s 2024-08-18T23:51:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:41 smithi078 ceph-mon[18440]: pgmap v571: 129 pgs: 129 active+clean; 581 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 766 op/s 2024-08-18T23:51:41.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:41 smithi123 ceph-mon[21808]: pgmap v571: 129 pgs: 129 active+clean; 581 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 766 op/s 2024-08-18T23:51:41.859 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-18T23:51:41.859 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:51:42.121 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:51:42.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.672+0000 7f33d3d8f640 1 -- 172.21.15.78:0/654555634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33cc10e170 msgr2=0x7f33cc110560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:42.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.672+0000 7f33d3d8f640 1 --2- 172.21.15.78:0/654555634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33cc10e170 0x7f33cc110560 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009ca0 tx=0x7f33c402a950 comp rx=0 tx=0).stop 2024-08-18T23:51:42.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.672+0000 7f33d3d8f640 1 -- 172.21.15.78:0/654555634 shutdown_connections 2024-08-18T23:51:42.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.672+0000 7f33d3d8f640 1 --2- 172.21.15.78:0/654555634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33cc10e170 0x7f33cc110560 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:42.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.672+0000 7f33d3d8f640 1 --2- 172.21.15.78:0/654555634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33cc074b60 0x7f33cc10dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:42.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.672+0000 7f33d3d8f640 1 --2- 172.21.15.78:0/654555634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33cc074240 0x7f33cc074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:42.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.672+0000 7f33d3d8f640 1 -- 172.21.15.78:0/654555634 >> 172.21.15.78:0/654555634 conn(0x7f33cc0fd5f0 msgr2=0x7f33cc0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:42.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.673+0000 7f33d3d8f640 1 -- 172.21.15.78:0/654555634 shutdown_connections 2024-08-18T23:51:42.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.673+0000 7f33d3d8f640 1 -- 172.21.15.78:0/654555634 wait complete. 2024-08-18T23:51:42.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.673+0000 7f33d3d8f640 1 Processor -- start 2024-08-18T23:51:42.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.673+0000 7f33d3d8f640 1 -- start start 2024-08-18T23:51:42.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.673+0000 7f33d3d8f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33cc074240 0x7f33cc19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:42.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.673+0000 7f33d3d8f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33cc074b60 0x7f33cc19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:42.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.673+0000 7f33d3d8f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33cc10e170 0x7f33cc19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:42.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.673+0000 7f33d3d8f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc113ea0 con 0x7f33cc074b60 2024-08-18T23:51:42.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.673+0000 7f33d3d8f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc113d20 con 0x7f33cc10e170 2024-08-18T23:51:42.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.673+0000 7f33d3d8f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc114020 con 0x7f33cc074240 2024-08-18T23:51:42.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.673+0000 7f33d132a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33cc074b60 0x7f33cc19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:42.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.673+0000 7f33d132a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33cc074b60 0x7f33cc19a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59562/0 (socket says 172.21.15.78:59562) 2024-08-18T23:51:42.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.673+0000 7f33d132a640 1 -- 172.21.15.78:0/455123952 learned_addr learned my addr 172.21.15.78:0/455123952 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:42.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.673+0000 7f33d1b2b640 1 --2- 172.21.15.78:0/455123952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33cc074240 0x7f33cc19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:42.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.674+0000 7f33d232c640 1 --2- 172.21.15.78:0/455123952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33cc10e170 0x7f33cc19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:42.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.674+0000 7f33d132a640 1 -- 172.21.15.78:0/455123952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33cc074240 msgr2=0x7f33cc19a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:42.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.674+0000 7f33d132a640 1 --2- 172.21.15.78:0/455123952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33cc074240 0x7f33cc19a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:42.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.674+0000 7f33d132a640 1 -- 172.21.15.78:0/455123952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33cc10e170 msgr2=0x7f33cc19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:42.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.674+0000 7f33d132a640 1 --2- 172.21.15.78:0/455123952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33cc10e170 0x7f33cc19ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:42.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.674+0000 7f33d132a640 1 -- 172.21.15.78:0/455123952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc19f340 con 0x7f33cc074b60 2024-08-18T23:51:42.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.674+0000 7f33d132a640 1 --2- 172.21.15.78:0/455123952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33cc074b60 0x7f33cc19a8d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00e9f0 tx=0x7f33bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:42.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.674+0000 7f33c2ffd640 1 -- 172.21.15.78:0/455123952 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc00cd80 con 0x7f33cc074b60 2024-08-18T23:51:42.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.674+0000 7f33c2ffd640 1 -- 172.21.15.78:0/455123952 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33bc005500 con 0x7f33cc074b60 2024-08-18T23:51:42.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.674+0000 7f33c2ffd640 1 -- 172.21.15.78:0/455123952 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc010690 con 0x7f33cc074b60 2024-08-18T23:51:42.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.674+0000 7f33d3d8f640 1 -- 172.21.15.78:0/455123952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33cc19f630 con 0x7f33cc074b60 2024-08-18T23:51:42.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.674+0000 7f33d3d8f640 1 -- 172.21.15.78:0/455123952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33cc1a6f10 con 0x7f33cc074b60 2024-08-18T23:51:42.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.675+0000 7f33d3d8f640 1 -- 172.21.15.78:0/455123952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33cc074620 con 0x7f33cc074b60 2024-08-18T23:51:42.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.676+0000 7f33c2ffd640 1 -- 172.21.15.78:0/455123952 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f33bc0046c0 con 0x7f33cc074b60 2024-08-18T23:51:42.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.677+0000 7f33c2ffd640 1 --2- 172.21.15.78:0/455123952 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f33b80a2b80 0x7f33b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:42.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.677+0000 7f33d1b2b640 1 --2- 172.21.15.78:0/455123952 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f33b80a2b80 0x7f33b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:42.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.677+0000 7f33c2ffd640 1 -- 172.21.15.78:0/455123952 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f33bc0cf880 con 0x7f33cc074b60 2024-08-18T23:51:42.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.677+0000 7f33d1b2b640 1 --2- 172.21.15.78:0/455123952 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f33b80a2b80 0x7f33b80a5040 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009bd0 tx=0x7f33c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:42.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.681+0000 7f33c2ffd640 1 -- 172.21.15.78:0/455123952 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f33bc09c3f0 con 0x7f33cc074b60 2024-08-18T23:51:42.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.916+0000 7f33d3d8f640 1 -- 172.21.15.78:0/455123952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33cc108d60 con 0x7f33cc074b60 2024-08-18T23:51:42.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.917+0000 7f33c2ffd640 1 -- 172.21.15.78:0/455123952 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11935 (secure 0 0 0) 0x7f33bc09bcf0 con 0x7f33cc074b60 2024-08-18T23:51:42.919 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:51:42.920 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":30,"btime":"2024-08-18T23:51:13:400845+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:51:13.279724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:51:42.920 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 30 2024-08-18T23:51:42.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.919+0000 7f33d3d8f640 1 -- 172.21.15.78:0/455123952 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f33b80a2b80 msgr2=0x7f33b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.919+0000 7f33d3d8f640 1 --2- 172.21.15.78:0/455123952 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f33b80a2b80 0x7f33b80a5040 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009bd0 tx=0x7f33c8005f90 comp rx=0 tx=0).stop 2024-08-18T23:51:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.919+0000 7f33d3d8f640 1 -- 172.21.15.78:0/455123952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33cc074b60 msgr2=0x7f33cc19a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.919+0000 7f33d3d8f640 1 --2- 172.21.15.78:0/455123952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33cc074b60 0x7f33cc19a8d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00e9f0 tx=0x7f33bc00edf0 comp rx=0 tx=0).stop 2024-08-18T23:51:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.919+0000 7f33d3d8f640 1 -- 172.21.15.78:0/455123952 shutdown_connections 2024-08-18T23:51:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.919+0000 7f33d3d8f640 1 --2- 172.21.15.78:0/455123952 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f33b80a2b80 0x7f33b80a5040 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.919+0000 7f33d3d8f640 1 --2- 172.21.15.78:0/455123952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33cc10e170 0x7f33cc19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.919+0000 7f33d3d8f640 1 --2- 172.21.15.78:0/455123952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33cc074b60 0x7f33cc19a8d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.919+0000 7f33d3d8f640 1 --2- 172.21.15.78:0/455123952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33cc074240 0x7f33cc19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.919+0000 7f33d3d8f640 1 -- 172.21.15.78:0/455123952 >> 172.21.15.78:0/455123952 conn(0x7f33cc0fd5f0 msgr2=0x7f33cc0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.919+0000 7f33d3d8f640 1 -- 172.21.15.78:0/455123952 shutdown_connections 2024-08-18T23:51:42.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:42.919+0000 7f33d3d8f640 1 -- 172.21.15.78:0/455123952 wait complete. 2024-08-18T23:51:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:43 smithi161 ceph-mon[22027]: pgmap v572: 129 pgs: 129 active+clean; 573 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 770 op/s 2024-08-18T23:51:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:43 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/455123952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:51:43.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:43 smithi078 ceph-mon[18440]: pgmap v572: 129 pgs: 129 active+clean; 573 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 770 op/s 2024-08-18T23:51:43.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:43 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/455123952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:51:43.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:43 smithi123 ceph-mon[21808]: pgmap v572: 129 pgs: 129 active+clean; 573 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 770 op/s 2024-08-18T23:51:43.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:43 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/455123952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:51:45.540 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-18T23:51:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:45 smithi161 ceph-mon[22027]: pgmap v573: 129 pgs: 129 active+clean; 565 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 732 op/s 2024-08-18T23:51:45.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:45 smithi078 ceph-mon[18440]: pgmap v573: 129 pgs: 129 active+clean; 565 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 732 op/s 2024-08-18T23:51:45.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:45 smithi123 ceph-mon[21808]: pgmap v573: 129 pgs: 129 active+clean; 565 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 732 op/s 2024-08-18T23:51:45.843 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:51:46.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.402+0000 7f6ac201e640 1 -- 172.21.15.78:0/2093692954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6abc1014f0 msgr2=0x7f6abc1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:46.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.402+0000 7f6ac201e640 1 --2- 172.21.15.78:0/2093692954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6abc1014f0 0x7f6abc1018d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00d910 tx=0x7f6aac02c380 comp rx=0 tx=0).stop 2024-08-18T23:51:46.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.402+0000 7f6ac201e640 1 -- 172.21.15.78:0/2093692954 shutdown_connections 2024-08-18T23:51:46.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.402+0000 7f6ac201e640 1 --2- 172.21.15.78:0/2093692954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6abc105b10 0x7f6abc10d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.402+0000 7f6ac201e640 1 --2- 172.21.15.78:0/2093692954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6abc101ea0 0x7f6abc1053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.402+0000 7f6ac201e640 1 --2- 172.21.15.78:0/2093692954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6abc1014f0 0x7f6abc1018d0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.402+0000 7f6ac201e640 1 -- 172.21.15.78:0/2093692954 >> 172.21.15.78:0/2093692954 conn(0x7f6abc0fb7c0 msgr2=0x7f6abc0fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:46.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.405+0000 7f6ac201e640 1 -- 172.21.15.78:0/2093692954 shutdown_connections 2024-08-18T23:51:46.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.405+0000 7f6ac201e640 1 -- 172.21.15.78:0/2093692954 wait complete. 2024-08-18T23:51:46.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.405+0000 7f6ac201e640 1 Processor -- start 2024-08-18T23:51:46.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.406+0000 7f6ac201e640 1 -- start start 2024-08-18T23:51:46.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.406+0000 7f6ac201e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6abc1014f0 0x7f6abc19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:46.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.406+0000 7f6ac201e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6abc101ea0 0x7f6abc195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:46.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.406+0000 7f6ac201e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6abc105b10 0x7f6abc196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:46.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.406+0000 7f6ac201e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc10fab0 con 0x7f6abc101ea0 2024-08-18T23:51:46.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.406+0000 7f6ac201e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc10f930 con 0x7f6abc105b10 2024-08-18T23:51:46.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.406+0000 7f6ac201e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc10fc30 con 0x7f6abc1014f0 2024-08-18T23:51:46.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.406+0000 7f6abb7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6abc1014f0 0x7f6abc19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:46.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.407+0000 7f6abbfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6abc105b10 0x7f6abc196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:46.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.407+0000 7f6abb7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6abc1014f0 0x7f6abc19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:43026/0 (socket says 172.21.15.78:43026) 2024-08-18T23:51:46.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.407+0000 7f6abb7fe640 1 -- 172.21.15.78:0/1659966935 learned_addr learned my addr 172.21.15.78:0/1659966935 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:46.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.407+0000 7f6ab3fff640 1 --2- 172.21.15.78:0/1659966935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6abc101ea0 0x7f6abc195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:46.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.407+0000 7f6abb7fe640 1 -- 172.21.15.78:0/1659966935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6abc105b10 msgr2=0x7f6abc196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:46.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.407+0000 7f6abb7fe640 1 --2- 172.21.15.78:0/1659966935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6abc105b10 0x7f6abc196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.407+0000 7f6abb7fe640 1 -- 172.21.15.78:0/1659966935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6abc101ea0 msgr2=0x7f6abc195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.407+0000 7f6abb7fe640 1 --2- 172.21.15.78:0/1659966935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6abc101ea0 0x7f6abc195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.407+0000 7f6abb7fe640 1 -- 172.21.15.78:0/1659966935 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6abc1968f0 con 0x7f6abc1014f0 2024-08-18T23:51:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.408+0000 7f6abb7fe640 1 --2- 172.21.15.78:0/1659966935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6abc1014f0 0x7f6abc19cd30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6aac000c00 tx=0x7f6aac0349c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.408+0000 7f6ab97fa640 1 -- 172.21.15.78:0/1659966935 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac034d40 con 0x7f6abc1014f0 2024-08-18T23:51:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.408+0000 7f6ac201e640 1 -- 172.21.15.78:0/1659966935 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6abc196b80 con 0x7f6abc1014f0 2024-08-18T23:51:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.408+0000 7f6ac201e640 1 -- 172.21.15.78:0/1659966935 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6abc19aae0 con 0x7f6abc1014f0 2024-08-18T23:51:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.408+0000 7f6ab97fa640 1 -- 172.21.15.78:0/1659966935 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aac0035f0 con 0x7f6abc1014f0 2024-08-18T23:51:46.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.408+0000 7f6ab97fa640 1 -- 172.21.15.78:0/1659966935 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac03c420 con 0x7f6abc1014f0 2024-08-18T23:51:46.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.413+0000 7f6ac441c740 1 Processor -- start 2024-08-18T23:51:46.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.413+0000 7f6ac441c740 1 -- start start 2024-08-18T23:51:46.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.413+0000 7f6ac441c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfce5ff0 0x55aacfce63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:46.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.413+0000 7f6ac441c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfce6910 0x55aacfcf0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:46.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.413+0000 7f6ac441c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfcf11d0 0x55aacfcf35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:46.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.413+0000 7f6ac441c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55aacfc4ba40 con 0x55aacfce6910 2024-08-18T23:51:46.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.413+0000 7f6ac441c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aacfbe39d0 con 0x55aacfce5ff0 2024-08-18T23:51:46.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.413+0000 7f6ac441c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55aacfc47a00 con 0x55aacfcf11d0 2024-08-18T23:51:46.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.414+0000 7f6ab3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfcf11d0 0x55aacfcf35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:46.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.414+0000 7f6abbfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfce5ff0 0x55aacfce63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:46.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.414+0000 7f6abbfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfce5ff0 0x55aacfce63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:42552/0 (socket says 172.21.15.78:42552) 2024-08-18T23:51:46.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.414+0000 7f6abbfff640 1 -- 172.21.15.78:0/2938131676 learned_addr learned my addr 172.21.15.78:0/2938131676 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:46.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.416+0000 7f6abb7fe640 1 --2- 172.21.15.78:0/2938131676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfce6910 0x55aacfcf0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:46.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.416+0000 7f6ab97fa640 1 -- 172.21.15.78:0/1659966935 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6aac043020 con 0x7f6abc1014f0 2024-08-18T23:51:46.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.416+0000 7f6abbfff640 1 -- 172.21.15.78:0/2938131676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfcf11d0 msgr2=0x55aacfcf35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:46.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.416+0000 7f6abbfff640 1 --2- 172.21.15.78:0/2938131676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfcf11d0 0x55aacfcf35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.416+0000 7f6abbfff640 1 -- 172.21.15.78:0/2938131676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfce6910 msgr2=0x55aacfcf0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:46.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.416+0000 7f6abbfff640 1 --2- 172.21.15.78:0/2938131676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfce6910 0x55aacfcf0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.416+0000 7f6abbfff640 1 -- 172.21.15.78:0/2938131676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aacfbea650 con 0x55aacfce5ff0 2024-08-18T23:51:46.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.417+0000 7f6abbfff640 1 --2- 172.21.15.78:0/2938131676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfce5ff0 0x55aacfce63d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009f20 tx=0x7f6ab400f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aecfd035dfa0921d server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:46.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.418+0000 7f6ab97fa640 1 --2- 172.21.15.78:0/1659966935 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6aa80a2cd0 0x7f6aa80a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:46.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.418+0000 7f6ab97fa640 1 -- 172.21.15.78:0/1659966935 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f6aac0eeb70 con 0x7f6abc1014f0 2024-08-18T23:51:46.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.419+0000 7f6ab1ffb640 1 -- 172.21.15.78:0/2938131676 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4015070 con 0x55aacfce5ff0 2024-08-18T23:51:46.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.419+0000 7f6ab1ffb640 1 -- 172.21.15.78:0/2938131676 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ab401ab40 con 0x55aacfce5ff0 2024-08-18T23:51:46.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.419+0000 7f6ab1ffb640 1 -- 172.21.15.78:0/2938131676 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4011ca0 con 0x55aacfce5ff0 2024-08-18T23:51:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.421+0000 7f6abb7fe640 1 --2- 172.21.15.78:0/1659966935 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6aa80a2cd0 0x7f6aa80a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:46.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.422+0000 7f6abb7fe640 1 --2- 172.21.15.78:0/1659966935 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6aa80a2cd0 0x7f6aa80a5190 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6aac006010 tx=0x7f6aac034f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.423+0000 7f6ac441c740 1 -- 172.21.15.78:0/2938131676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfce5ff0 msgr2=0x55aacfce63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.423+0000 7f6ac441c740 1 --2- 172.21.15.78:0/2938131676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfce5ff0 0x55aacfce63d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009f20 tx=0x7f6ab400f7f0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.423+0000 7f6ac441c740 1 -- 172.21.15.78:0/2938131676 shutdown_connections 2024-08-18T23:51:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.423+0000 7f6ac441c740 1 --2- 172.21.15.78:0/2938131676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfcf11d0 0x55aacfcf35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.423+0000 7f6ac441c740 1 --2- 172.21.15.78:0/2938131676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfce6910 0x55aacfcf0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.423+0000 7f6ac441c740 1 --2- 172.21.15.78:0/2938131676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfce5ff0 0x55aacfce63d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.423+0000 7f6ac441c740 1 -- 172.21.15.78:0/2938131676 >> 172.21.15.78:0/2938131676 conn(0x55aacfce2580 msgr2=0x55aacfce2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.424+0000 7f6ac441c740 1 -- 172.21.15.78:0/2938131676 shutdown_connections 2024-08-18T23:51:46.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.424+0000 7f6ac441c740 1 -- 172.21.15.78:0/2938131676 wait complete. 2024-08-18T23:51:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.424+0000 7f6ac441c740 1 -- start start 2024-08-18T23:51:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.425+0000 7f6ac441c740 1 Processor -- start 2024-08-18T23:51:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.425+0000 7f6ac441c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfce5ff0 0x55aacfcee0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.425+0000 7f6ac441c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfce6910 0x55aacfcee5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.425+0000 7f6ac441c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfcf11d0 0x55aacfceeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.426+0000 7f6ac441c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55aacfcf3da0 con 0x55aacfcf11d0 2024-08-18T23:51:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.426+0000 7f6abbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfcf11d0 0x55aacfceeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.426+0000 7f6abbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfcf11d0 0x55aacfceeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44808/0 (socket says 172.21.15.78:44808) 2024-08-18T23:51:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.426+0000 7f6abbfff640 1 -- 172.21.15.78:0/251123445 learned_addr learned my addr 172.21.15.78:0/251123445 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.426+0000 7f6ab3fff640 1 --2- 172.21.15.78:0/251123445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfce6910 0x55aacfcee5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.426+0000 7f6ac441c740 1 -- 172.21.15.78:0/251123445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aacfc06b20 con 0x55aacfce5ff0 2024-08-18T23:51:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.426+0000 7f6ac441c740 1 -- 172.21.15.78:0/251123445 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55aacfcf3f20 con 0x55aacfce6910 2024-08-18T23:51:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.426+0000 7f6abbfff640 1 --2- 172.21.15.78:0/251123445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfce5ff0 0x55aacfcee0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.426+0000 7f6abbfff640 1 -- 172.21.15.78:0/251123445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfce6910 msgr2=0x55aacfcee5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.426+0000 7f6abbfff640 1 --2- 172.21.15.78:0/251123445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfce6910 0x55aacfcee5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.426+0000 7f6abbfff640 1 -- 172.21.15.78:0/251123445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfce5ff0 msgr2=0x55aacfcee0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.426+0000 7f6abbfff640 1 --2- 172.21.15.78:0/251123445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfce5ff0 0x55aacfcee0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.426+0000 7f6abbfff640 1 -- 172.21.15.78:0/251123445 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aacf9bd3c0 con 0x55aacfcf11d0 2024-08-18T23:51:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.427+0000 7f6ac441c740 10 client.0 _ll_register_callbacks cb 0x55aacfc04bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.427+0000 7f6ac441c740 1 -- 172.21.15.78:0/251123445 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ab401ab40 con 0x55aacfcf11d0 2024-08-18T23:51:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.427+0000 7f6abbfff640 1 --2- 172.21.15.78:0/251123445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfcf11d0 0x55aacfceeb30 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4011730 tx=0x7f6ab4011880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124fbd2b18da03dd server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:46.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.428+0000 7f6a9affd640 10 client.16638 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:51:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.432+0000 7f6a9affd640 1 -- 172.21.15.78:0/251123445 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4015040 con 0x55aacfcf11d0 2024-08-18T23:51:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.432+0000 7f6a9affd640 1 -- 172.21.15.78:0/251123445 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f6ab4012ab0 con 0x55aacfcf11d0 2024-08-18T23:51:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.432+0000 7f6a9affd640 1 -- 172.21.15.78:0/251123445 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6ab4012cb0 con 0x55aacfcf11d0 2024-08-18T23:51:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.432+0000 7f6ac441c740 10 client.16638 fetch_fsmap learned FSMap version 30 2024-08-18T23:51:46.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.432+0000 7f6ac441c740 1 -- 172.21.15.78:0/251123445 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55aacfcff800 con 0x55aacfcf11d0 2024-08-18T23:51:46.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.434+0000 7f6a9affd640 1 -- 172.21.15.78:0/251123445 <== mon.0 v2:172.21.15.78:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6ab4003970 con 0x55aacfcf11d0 2024-08-18T23:51:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.434+0000 7f6ac441c740 10 client.16638 fetch_fsmap finished waiting for FSMap version 30 2024-08-18T23:51:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.434+0000 7f6ac441c740 10 client.16638 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:51:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.434+0000 7f6ac441c740 20 client.16638 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.434+0000 7f6ac441c740 1 --2- 172.21.15.78:0/251123445 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55aacfd01050 0x55aacfd03510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:46.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.434+0000 7f6ac441c740 4 client.16638 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:51:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.434+0000 7f6ac441c740 1 -- 172.21.15.78:0/251123445 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aacfc58fb0 con 0x55aacfd01050 2024-08-18T23:51:46.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.435+0000 7f6ab3fff640 1 --2- 172.21.15.78:0/251123445 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55aacfd01050 0x55aacfd03510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:46.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.438+0000 7f6ab3fff640 1 --2- 172.21.15.78:0/251123445 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55aacfd01050 0x55aacfd03510 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c71efffc70145 server_cookie=10398d97a417c696 in_seq=0 out_seq=0 2024-08-18T23:51:46.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.438+0000 7f6a9affd640 10 client.16638 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:51:46.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.439+0000 7f6a9affd640 1 -- 172.21.15.78:0/251123445 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aacfc58fb0 con 0x55aacfd01050 2024-08-18T23:51:46.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.439+0000 7f6a9affd640 10 client.16638 handle_command_reply: tid=0 2024-08-18T23:51:46.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.439+0000 7f6ac441c740 1 client.16638 shutdown 2024-08-18T23:51:46.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.440+0000 7f6ac441c740 1 -- 172.21.15.78:0/251123445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfcf11d0 msgr2=0x55aacfceeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:46.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.440+0000 7f6ac441c740 1 --2- 172.21.15.78:0/251123445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfcf11d0 0x55aacfceeb30 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4011730 tx=0x7f6ab4011880 comp rx=0 tx=0).stop 2024-08-18T23:51:46.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.440+0000 7f6ac441c740 1 -- 172.21.15.78:0/251123445 shutdown_connections 2024-08-18T23:51:46.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.440+0000 7f6ac441c740 1 --2- 172.21.15.78:0/251123445 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55aacfd01050 0x55aacfd03510 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.440+0000 7f6ac441c740 1 --2- 172.21.15.78:0/251123445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfcf11d0 0x55aacfceeb30 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.440+0000 7f6ac441c740 1 --2- 172.21.15.78:0/251123445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfce6910 0x55aacfcee5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.440+0000 7f6ac441c740 1 --2- 172.21.15.78:0/251123445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfce5ff0 0x55aacfcee0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.440+0000 7f6ac441c740 1 -- 172.21.15.78:0/251123445 >> 172.21.15.78:0/251123445 conn(0x55aacfce2580 msgr2=0x55aacfce82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:46.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.440+0000 7f6a9affd640 0 client.16638 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:51:46.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.445+0000 7f6ac441c740 1 -- 172.21.15.78:0/251123445 shutdown_connections 2024-08-18T23:51:46.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.445+0000 7f6ac441c740 1 -- 172.21.15.78:0/251123445 wait complete. 2024-08-18T23:51:46.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.445+0000 7f6ac441c740 20 client.16638 trim_cache size 0 max 16384 2024-08-18T23:51:46.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.472+0000 7f6ac441c740 1 Processor -- start 2024-08-18T23:51:46.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.472+0000 7f6ac441c740 1 -- start start 2024-08-18T23:51:46.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.472+0000 7f6ac441c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfce5ff0 0x55aacfce1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:46.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.475+0000 7f6ac441c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfce6910 0x55aacfd08e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:46.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.475+0000 7f6abbfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfce5ff0 0x55aacfce1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:46.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.475+0000 7f6abbfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfce5ff0 0x55aacfce1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:42578/0 (socket says 172.21.15.78:42578) 2024-08-18T23:51:46.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.475+0000 7f6abbfff640 1 -- 172.21.15.78:0/2225479779 learned_addr learned my addr 172.21.15.78:0/2225479779 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:46.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.476+0000 7f6ac441c740 1 --2- 172.21.15.78:0/2225479779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfcf11d0 0x55aacfd09360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:46.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.476+0000 7f6ac441c740 1 -- 172.21.15.78:0/2225479779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c012570 con 0x55aacfcf11d0 2024-08-18T23:51:46.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.476+0000 7f6ac441c740 1 -- 172.21.15.78:0/2225479779 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c00fde0 con 0x55aacfce5ff0 2024-08-18T23:51:46.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.476+0000 7f6ac441c740 1 -- 172.21.15.78:0/2225479779 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c00dc30 con 0x55aacfce6910 2024-08-18T23:51:46.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.478+0000 7f6abbfff640 1 --2- 172.21.15.78:0/2225479779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfcf11d0 0x55aacfd09360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:46.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.478+0000 7f6abbfff640 1 -- 172.21.15.78:0/2225479779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfce6910 msgr2=0x55aacfd08e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-18T23:51:46.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.478+0000 7f6abbfff640 1 --2- 172.21.15.78:0/2225479779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfce6910 0x55aacfd08e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.478+0000 7f6abbfff640 1 -- 172.21.15.78:0/2225479779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfcf11d0 msgr2=0x55aacfd09360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:46.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.478+0000 7f6abbfff640 1 --2- 172.21.15.78:0/2225479779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfcf11d0 0x55aacfd09360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.478+0000 7f6abbfff640 1 -- 172.21.15.78:0/2225479779 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aacfcfd6b0 con 0x55aacfce5ff0 2024-08-18T23:51:46.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.479+0000 7f6abbfff640 1 --2- 172.21.15.78:0/2225479779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfce5ff0 0x55aacfce1da0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55aacfce3880 tx=0x7f6ab401e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa6a4161a22e86ad server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:46.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.480+0000 7f6ab17fa640 1 -- 172.21.15.78:0/2225479779 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4015070 con 0x55aacfce5ff0 2024-08-18T23:51:46.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.480+0000 7f6ab17fa640 1 -- 172.21.15.78:0/2225479779 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ab4012e50 con 0x55aacfce5ff0 2024-08-18T23:51:46.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.480+0000 7f6ab17fa640 1 -- 172.21.15.78:0/2225479779 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab40050b0 con 0x55aacfce5ff0 2024-08-18T23:51:46.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.481+0000 7f6ac441c740 1 -- 172.21.15.78:0/2225479779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfce5ff0 msgr2=0x55aacfce1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:46.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.481+0000 7f6ac441c740 1 --2- 172.21.15.78:0/2225479779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfce5ff0 0x55aacfce1da0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55aacfce3880 tx=0x7f6ab401e040 comp rx=0 tx=0).stop 2024-08-18T23:51:46.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.481+0000 7f6ac441c740 1 -- 172.21.15.78:0/2225479779 shutdown_connections 2024-08-18T23:51:46.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.481+0000 7f6ac441c740 1 --2- 172.21.15.78:0/2225479779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfcf11d0 0x55aacfd09360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.482+0000 7f6ac441c740 1 --2- 172.21.15.78:0/2225479779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfce6910 0x55aacfd08e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.482+0000 7f6ac441c740 1 --2- 172.21.15.78:0/2225479779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfce5ff0 0x55aacfce1da0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.482+0000 7f6ac441c740 1 -- 172.21.15.78:0/2225479779 >> 172.21.15.78:0/2225479779 conn(0x55aacfce2580 msgr2=0x55aacfce8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:46.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.483+0000 7f6ac441c740 1 -- 172.21.15.78:0/2225479779 shutdown_connections 2024-08-18T23:51:46.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.483+0000 7f6ac441c740 1 -- 172.21.15.78:0/2225479779 wait complete. 2024-08-18T23:51:46.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.484+0000 7f6ac441c740 1 -- start start 2024-08-18T23:51:46.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.484+0000 7f6ac441c740 1 Processor -- start 2024-08-18T23:51:46.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.485+0000 7f6ac441c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfce5ff0 0x55aacfcf96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:46.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.485+0000 7f6ac441c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfce6910 0x55aacfceb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:46.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.485+0000 7f6abbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfce5ff0 0x55aacfcf96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:46.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.485+0000 7f6abbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfce5ff0 0x55aacfcf96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44834/0 (socket says 172.21.15.78:44834) 2024-08-18T23:51:46.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.485+0000 7f6abbfff640 1 -- 172.21.15.78:0/1871213818 learned_addr learned my addr 172.21.15.78:0/1871213818 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:46.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.485+0000 7f6ac441c740 1 --2- 172.21.15.78:0/1871213818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfcf11d0 0x55aacfceb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:46.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.486+0000 7f6ab3fff640 1 --2- 172.21.15.78:0/1871213818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfce6910 0x55aacfceb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:46.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.486+0000 7f6ac441c740 1 -- 172.21.15.78:0/1871213818 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c00e2a0 con 0x55aacfce5ff0 2024-08-18T23:51:46.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.486+0000 7f6ac441c740 1 -- 172.21.15.78:0/1871213818 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c00f710 con 0x55aacfcf11d0 2024-08-18T23:51:46.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.486+0000 7f6ac441c740 1 -- 172.21.15.78:0/1871213818 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c00e970 con 0x55aacfce6910 2024-08-18T23:51:46.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.486+0000 7f6ac441c740 10 client.0 _ll_register_callbacks cb 0x7f6ab4009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:46.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.490+0000 7f6abbfff640 1 --2- 172.21.15.78:0/1871213818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfcf11d0 0x55aacfceb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:46.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.490+0000 7f6ab3fff640 1 -- 172.21.15.78:0/1871213818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfcf11d0 msgr2=0x55aacfceb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:46.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.490+0000 7f6ab3fff640 1 --2- 172.21.15.78:0/1871213818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfcf11d0 0x55aacfceb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.490+0000 7f6ab3fff640 1 -- 172.21.15.78:0/1871213818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfce5ff0 msgr2=0x55aacfcf96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:46.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.490+0000 7f6ab3fff640 1 --2- 172.21.15.78:0/1871213818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfce5ff0 0x55aacfcf96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.490+0000 7f6ab3fff640 1 -- 172.21.15.78:0/1871213818 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aacfcebec0 con 0x55aacfce6910 2024-08-18T23:51:46.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.490+0000 7f6ab3fff640 1 --2- 172.21.15.78:0/1871213818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfce6910 0x55aacfceb410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55aacfcef6f0 tx=0x7f6aa4014840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21f00d0c26118f94 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:46.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.490+0000 7f6a9affd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:51:46.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.491+0000 7f6a9affd640 1 -- 172.21.15.78:0/1871213818 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4023070 con 0x55aacfce6910 2024-08-18T23:51:46.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.491+0000 7f6a9affd640 1 -- 172.21.15.78:0/1871213818 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f6aa4012650 con 0x55aacfce6910 2024-08-18T23:51:46.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.491+0000 7f6ac441c740 1 -- 172.21.15.78:0/1871213818 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ab4012e50 con 0x55aacfce6910 2024-08-18T23:51:46.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.491+0000 7f6a9affd640 1 -- 172.21.15.78:0/1871213818 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6aa4012870 con 0x55aacfce6910 2024-08-18T23:51:46.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.491+0000 7f6ac441c740 10 client.25858 fetch_fsmap learned FSMap version 30 2024-08-18T23:51:46.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.491+0000 7f6ac441c740 1 -- 172.21.15.78:0/1871213818 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aacfd6adf0 con 0x55aacfce6910 2024-08-18T23:51:46.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.491+0000 7f6a9affd640 1 -- 172.21.15.78:0/1871213818 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa4028440 con 0x55aacfce6910 2024-08-18T23:51:46.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.493+0000 7f6a9affd640 1 -- 172.21.15.78:0/1871213818 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6aa4026020 con 0x55aacfce6910 2024-08-18T23:51:46.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.493+0000 7f6ac441c740 10 client.25858 fetch_fsmap finished waiting for FSMap version 30 2024-08-18T23:51:46.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.493+0000 7f6ac441c740 10 client.25858 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:51:46.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.493+0000 7f6ac441c740 20 client.25858 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:46.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.493+0000 7f6ac441c740 1 --2- 172.21.15.78:0/1871213818 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55aacfd6b420 0x55aacfd6d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:46.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.493+0000 7f6ac441c740 4 client.25858 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:51:46.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.493+0000 7f6ac441c740 1 -- 172.21.15.78:0/1871213818 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6ab4012e50 con 0x55aacfd6b420 2024-08-18T23:51:46.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.493+0000 7f6abbfff640 1 --2- 172.21.15.78:0/1871213818 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55aacfd6b420 0x55aacfd6d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:46.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.494+0000 7f6abbfff640 1 --2- 172.21.15.78:0/1871213818 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55aacfd6b420 0x55aacfd6d840 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6664e4d9ba4724a5 server_cookie=efd5983420a4632d in_seq=0 out_seq=0 2024-08-18T23:51:46.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.494+0000 7f6a9affd640 10 client.25858 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:51:46.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.496+0000 7f6a9affd640 1 -- 172.21.15.78:0/1871213818 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6ab4012e50 con 0x55aacfd6b420 2024-08-18T23:51:46.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.496+0000 7f6a9affd640 10 client.25858 handle_command_reply: tid=0 2024-08-18T23:51:46.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.496+0000 7f6ac441c740 1 client.25858 shutdown 2024-08-18T23:51:46.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.502+0000 7f6ac441c740 1 -- 172.21.15.78:0/1871213818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfce6910 msgr2=0x55aacfceb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:46.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.502+0000 7f6ac441c740 1 --2- 172.21.15.78:0/1871213818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfce6910 0x55aacfceb410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55aacfcef6f0 tx=0x7f6aa4014840 comp rx=0 tx=0).stop 2024-08-18T23:51:46.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.503+0000 7f6ac441c740 1 -- 172.21.15.78:0/1871213818 shutdown_connections 2024-08-18T23:51:46.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.503+0000 7f6ac441c740 1 --2- 172.21.15.78:0/1871213818 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x55aacfd6b420 0x55aacfd6d840 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.503+0000 7f6a9affd640 0 client.25858 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:51:46.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.503+0000 7f6ac441c740 1 --2- 172.21.15.78:0/1871213818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aacfcf11d0 0x55aacfceb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.503+0000 7f6ac441c740 1 --2- 172.21.15.78:0/1871213818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aacfce6910 0x55aacfceb410 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.503+0000 7f6ac441c740 1 --2- 172.21.15.78:0/1871213818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aacfce5ff0 0x55aacfcf96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.503+0000 7f6ac441c740 1 -- 172.21.15.78:0/1871213818 >> 172.21.15.78:0/1871213818 conn(0x55aacfce2580 msgr2=0x55aacfce79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:46.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.503+0000 7f6ac441c740 1 -- 172.21.15.78:0/1871213818 shutdown_connections 2024-08-18T23:51:46.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.503+0000 7f6ac441c740 1 -- 172.21.15.78:0/1871213818 wait complete. 2024-08-18T23:51:46.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.504+0000 7f6ac441c740 20 client.25858 trim_cache size 0 max 16384 2024-08-18T23:51:46.507 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:51:46.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.506+0000 7f6ac201e640 1 -- 172.21.15.78:0/1659966935 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6aa80a2cd0 msgr2=0x7f6aa80a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:46.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.506+0000 7f6ac201e640 1 --2- 172.21.15.78:0/1659966935 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6aa80a2cd0 0x7f6aa80a5190 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6aac006010 tx=0x7f6aac034f90 comp rx=0 tx=0).stop 2024-08-18T23:51:46.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.506+0000 7f6ac201e640 1 -- 172.21.15.78:0/1659966935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6abc1014f0 msgr2=0x7f6abc19cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:46.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.506+0000 7f6ac201e640 1 --2- 172.21.15.78:0/1659966935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6abc1014f0 0x7f6abc19cd30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6aac000c00 tx=0x7f6aac0349c0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.506+0000 7f6ac201e640 1 -- 172.21.15.78:0/1659966935 shutdown_connections 2024-08-18T23:51:46.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.506+0000 7f6ac201e640 1 --2- 172.21.15.78:0/1659966935 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6aa80a2cd0 0x7f6aa80a5190 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.506+0000 7f6ac201e640 1 --2- 172.21.15.78:0/1659966935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6abc105b10 0x7f6abc196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.506+0000 7f6ac201e640 1 --2- 172.21.15.78:0/1659966935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6abc101ea0 0x7f6abc195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.506+0000 7f6ac201e640 1 --2- 172.21.15.78:0/1659966935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6abc1014f0 0x7f6abc19cd30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:46.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.506+0000 7f6ac201e640 1 -- 172.21.15.78:0/1659966935 >> 172.21.15.78:0/1659966935 conn(0x7f6abc0fb7c0 msgr2=0x7f6abc0fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:46.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.507+0000 7f6ac201e640 1 -- 172.21.15.78:0/1659966935 shutdown_connections 2024-08-18T23:51:46.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:46.507+0000 7f6ac201e640 1 -- 172.21.15.78:0/1659966935 wait complete. 2024-08-18T23:51:46.527 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:46 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:51:46.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:46 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:51:46.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:46 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:51:47.558 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-18T23:51:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:47 smithi161 ceph-mon[22027]: pgmap v574: 129 pgs: 129 active+clean; 552 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 828 op/s 2024-08-18T23:51:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:47 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:51:47.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:47 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:51:47.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:47 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:51:47.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:47 smithi078 ceph-mon[18440]: pgmap v574: 129 pgs: 129 active+clean; 552 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 828 op/s 2024-08-18T23:51:47.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:47 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:51:47.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:47 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:51:47.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:47 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:51:47.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:47 smithi123 ceph-mon[21808]: pgmap v574: 129 pgs: 129 active+clean; 552 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 828 op/s 2024-08-18T23:51:47.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:47 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:51:47.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:47 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:51:47.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:47 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:51:47.882 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:51:48.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.422+0000 7f8d53491640 1 -- 172.21.15.78:0/3059096865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d440a4a80 msgr2=0x7f8d440a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:48.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.422+0000 7f8d53491640 1 --2- 172.21.15.78:0/3059096865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d440a4a80 0x7f8d440a4e60 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c051230 tx=0x7f8d4c08e2a0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.422+0000 7f8d53491640 1 -- 172.21.15.78:0/3059096865 shutdown_connections 2024-08-18T23:51:48.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.422+0000 7f8d53491640 1 --2- 172.21.15.78:0/3059096865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d440b2df0 0x7f8d440b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.422+0000 7f8d53491640 1 --2- 172.21.15.78:0/3059096865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d440a53a0 0x7f8d440b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.422+0000 7f8d53491640 1 --2- 172.21.15.78:0/3059096865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d440a4a80 0x7f8d440a4e60 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.422+0000 7f8d53491640 1 -- 172.21.15.78:0/3059096865 >> 172.21.15.78:0/3059096865 conn(0x7f8d4401a0e0 msgr2=0x7f8d4401a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:48.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.423+0000 7f8d53491640 1 -- 172.21.15.78:0/3059096865 shutdown_connections 2024-08-18T23:51:48.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.423+0000 7f8d53491640 1 -- 172.21.15.78:0/3059096865 wait complete. 2024-08-18T23:51:48.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.423+0000 7f8d53491640 1 Processor -- start 2024-08-18T23:51:48.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.424+0000 7f8d53491640 1 -- start start 2024-08-18T23:51:48.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.424+0000 7f8d53491640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d440a4a80 0x7f8d440a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:48.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.424+0000 7f8d53491640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d440b2df0 0x7f8d440a89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:48.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.424+0000 7f8d5248f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d440a4a80 0x7f8d440a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:48.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.424+0000 7f8d5248f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d440a4a80 0x7f8d440a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44856/0 (socket says 172.21.15.78:44856) 2024-08-18T23:51:48.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.425+0000 7f8d53491640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d440a8f20 0x7f8d4414f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:48.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.425+0000 7f8d53491640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d440b8bc0 con 0x7f8d440a4a80 2024-08-18T23:51:48.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.425+0000 7f8d53491640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d440b8a40 con 0x7f8d440b2df0 2024-08-18T23:51:48.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.425+0000 7f8d53491640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8d440b8d40 con 0x7f8d440a8f20 2024-08-18T23:51:48.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.426+0000 7f8d5248f640 1 -- 172.21.15.78:0/2083190636 learned_addr learned my addr 172.21.15.78:0/2083190636 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:48.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.427+0000 7f8d51c8e640 1 --2- 172.21.15.78:0/2083190636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d440b2df0 0x7f8d440a89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:48.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.428+0000 7f8d52c90640 1 --2- 172.21.15.78:0/2083190636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d440a8f20 0x7f8d4414f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:48.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.429+0000 7f8d5248f640 1 -- 172.21.15.78:0/2083190636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d440a8f20 msgr2=0x7f8d4414f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:48.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.429+0000 7f8d5248f640 1 --2- 172.21.15.78:0/2083190636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d440a8f20 0x7f8d4414f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.429+0000 7f8d5248f640 1 -- 172.21.15.78:0/2083190636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d440b2df0 msgr2=0x7f8d440a89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:48.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.429+0000 7f8d5248f640 1 --2- 172.21.15.78:0/2083190636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d440b2df0 0x7f8d440a89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.429+0000 7f8d5248f640 1 -- 172.21.15.78:0/2083190636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4414fc60 con 0x7f8d440a4a80 2024-08-18T23:51:48.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.429+0000 7f8d5248f640 1 --2- 172.21.15.78:0/2083190636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d440a4a80 0x7f8d440a84a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c08e890 tx=0x7f8d4c097a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:48.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.432+0000 7f8d437fe640 1 -- 172.21.15.78:0/2083190636 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c0a5070 con 0x7f8d440a4a80 2024-08-18T23:51:48.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.432+0000 7f8d437fe640 1 -- 172.21.15.78:0/2083190636 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d4c09d070 con 0x7f8d440a4a80 2024-08-18T23:51:48.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.432+0000 7f8d53491640 1 -- 172.21.15.78:0/2083190636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4414fef0 con 0x7f8d440a4a80 2024-08-18T23:51:48.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.432+0000 7f8d53491640 1 -- 172.21.15.78:0/2083190636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d441502e0 con 0x7f8d440a4a80 2024-08-18T23:51:48.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.433+0000 7f8d437fe640 1 -- 172.21.15.78:0/2083190636 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c0957d0 con 0x7f8d440a4a80 2024-08-18T23:51:48.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.436+0000 7f8d56af1740 1 Processor -- start 2024-08-18T23:51:48.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.437+0000 7f8d56af1740 1 -- start start 2024-08-18T23:51:48.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.437+0000 7f8d56af1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 0x55a7569865b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:48.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.437+0000 7f8d56af1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a756986af0 0x55a756990e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:48.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.437+0000 7f8d56af1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7569913b0 0x55a7569937a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:48.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.437+0000 7f8d56af1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a7568eba40 con 0x55a7569913b0 2024-08-18T23:51:48.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.437+0000 7f8d56af1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a7568839d0 con 0x55a7569861d0 2024-08-18T23:51:48.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.437+0000 7f8d56af1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a7568e7a00 con 0x55a756986af0 2024-08-18T23:51:48.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.437+0000 7f8d52c90640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 0x55a7569865b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:48.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.437+0000 7f8d52c90640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 0x55a7569865b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:42624/0 (socket says 172.21.15.78:42624) 2024-08-18T23:51:48.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.437+0000 7f8d52c90640 1 -- 172.21.15.78:0/1178058502 learned_addr learned my addr 172.21.15.78:0/1178058502 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:48.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.437+0000 7f8d51c8e640 1 --2- 172.21.15.78:0/1178058502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7569913b0 0x55a7569937a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:48.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.438+0000 7f8d52c90640 1 -- 172.21.15.78:0/1178058502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a756986af0 msgr2=0x55a756990e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T23:51:48.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.438+0000 7f8d52c90640 1 --2- 172.21.15.78:0/1178058502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a756986af0 0x55a756990e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.438+0000 7f8d52c90640 1 -- 172.21.15.78:0/1178058502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7569913b0 msgr2=0x55a7569937a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:48.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.438+0000 7f8d52c90640 1 --2- 172.21.15.78:0/1178058502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7569913b0 0x55a7569937a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.438+0000 7f8d52c90640 1 -- 172.21.15.78:0/1178058502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a75688a650 con 0x55a7569861d0 2024-08-18T23:51:48.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.438+0000 7f8d51c8e640 1 --2- 172.21.15.78:0/1178058502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7569913b0 0x55a7569937a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:51:48.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.438+0000 7f8d52c90640 1 --2- 172.21.15.78:0/1178058502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 0x55a7569865b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800a8a0 tx=0x7f8d4800ad60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfc9a571dff69efe server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:48.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.439+0000 7f8d437fe640 1 -- 172.21.15.78:0/2083190636 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8d4c09f070 con 0x7f8d440a4a80 2024-08-18T23:51:48.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.440+0000 7f8d40ff9640 1 -- 172.21.15.78:0/1178058502 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48015070 con 0x55a7569861d0 2024-08-18T23:51:48.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.440+0000 7f8d40ff9640 1 -- 172.21.15.78:0/1178058502 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d4800cb40 con 0x55a7569861d0 2024-08-18T23:51:48.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.440+0000 7f8d40ff9640 1 -- 172.21.15.78:0/1178058502 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d480185c0 con 0x55a7569861d0 2024-08-18T23:51:48.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.440+0000 7f8d437fe640 1 --2- 172.21.15.78:0/2083190636 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8d380a2da0 0x7f8d380a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:48.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.441+0000 7f8d437fe640 1 -- 172.21.15.78:0/2083190636 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f8d4c152ce0 con 0x7f8d440a4a80 2024-08-18T23:51:48.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.443+0000 7f8d56af1740 1 -- 172.21.15.78:0/1178058502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 msgr2=0x55a7569865b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:48.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.443+0000 7f8d56af1740 1 --2- 172.21.15.78:0/1178058502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 0x55a7569865b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800a8a0 tx=0x7f8d4800ad60 comp rx=0 tx=0).stop 2024-08-18T23:51:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.445+0000 7f8d56af1740 1 -- 172.21.15.78:0/1178058502 shutdown_connections 2024-08-18T23:51:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.445+0000 7f8d56af1740 1 --2- 172.21.15.78:0/1178058502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7569913b0 0x55a7569937a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.445+0000 7f8d56af1740 1 --2- 172.21.15.78:0/1178058502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a756986af0 0x55a756990e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.445+0000 7f8d56af1740 1 --2- 172.21.15.78:0/1178058502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 0x55a7569865b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.445+0000 7f8d56af1740 1 -- 172.21.15.78:0/1178058502 >> 172.21.15.78:0/1178058502 conn(0x55a756982760 msgr2=0x55a756982b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.445+0000 7f8d5248f640 1 --2- 172.21.15.78:0/2083190636 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8d380a2da0 0x7f8d380a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:48.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.445+0000 7f8d56af1740 1 -- 172.21.15.78:0/1178058502 shutdown_connections 2024-08-18T23:51:48.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.445+0000 7f8d56af1740 1 -- 172.21.15.78:0/1178058502 wait complete. 2024-08-18T23:51:48.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.446+0000 7f8d56af1740 1 -- start start 2024-08-18T23:51:48.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.446+0000 7f8d56af1740 1 Processor -- start 2024-08-18T23:51:48.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.446+0000 7f8d56af1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 0x55a75698e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:48.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.446+0000 7f8d56af1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a756986af0 0x55a75698e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:48.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.446+0000 7f8d56af1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7569913b0 0x55a75698ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:48.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.446+0000 7f8d52c90640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 0x55a75698e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:48.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.446+0000 7f8d52c90640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 0x55a75698e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:42626/0 (socket says 172.21.15.78:42626) 2024-08-18T23:51:48.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.447+0000 7f8d51c8e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a756986af0 0x55a75698e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:48.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.447+0000 7f8d51c8e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a756986af0 0x55a75698e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44888/0 (socket says 172.21.15.78:44888) 2024-08-18T23:51:48.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.447+0000 7f8d56af1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a756993f80 con 0x55a756986af0 2024-08-18T23:51:48.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.447+0000 7f8d51c8e640 1 -- 172.21.15.78:0/3557628051 learned_addr learned my addr 172.21.15.78:0/3557628051 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:48.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.447+0000 7f8d56af1740 1 -- 172.21.15.78:0/3557628051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a7568a6b20 con 0x55a7569861d0 2024-08-18T23:51:48.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.447+0000 7f8d56af1740 1 -- 172.21.15.78:0/3557628051 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a756994100 con 0x55a7569913b0 2024-08-18T23:51:48.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.447+0000 7f8d56af1740 10 client.0 _ll_register_callbacks cb 0x55a7568a4bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:48.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.447+0000 7f8d52c90640 1 --2- 172.21.15.78:0/3557628051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7569913b0 0x55a75698ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:48.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.447+0000 7f8d52c90640 1 -- 172.21.15.78:0/3557628051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7569913b0 msgr2=0x55a75698ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:48.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.447+0000 7f8d52c90640 1 --2- 172.21.15.78:0/3557628051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7569913b0 0x55a75698ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.447+0000 7f8d52c90640 1 -- 172.21.15.78:0/3557628051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a756986af0 msgr2=0x55a75698e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:48.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.447+0000 7f8d52c90640 1 --2- 172.21.15.78:0/3557628051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a756986af0 0x55a75698e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.447+0000 7f8d52c90640 1 -- 172.21.15.78:0/3557628051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a75665d3c0 con 0x55a7569861d0 2024-08-18T23:51:48.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.448+0000 7f8d5248f640 1 --2- 172.21.15.78:0/2083190636 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8d380a2da0 0x7f8d380a5260 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c067700 tx=0x7f8d4c08e2a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:48.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.448+0000 7f8d52c90640 1 --2- 172.21.15.78:0/3557628051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 0x55a75698e2d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8d48004700 tx=0x7f8d48011c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ed0a896ee96c47 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:48.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.448+0000 7f8d35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:51:48.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.448+0000 7f8d35ffb640 1 -- 172.21.15.78:0/3557628051 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48015040 con 0x55a7569861d0 2024-08-18T23:51:48.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.448+0000 7f8d56af1740 1 -- 172.21.15.78:0/3557628051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d4800cb40 con 0x55a7569861d0 2024-08-18T23:51:48.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.448+0000 7f8d35ffb640 1 -- 172.21.15.78:0/3557628051 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f8d4801a660 con 0x55a7569861d0 2024-08-18T23:51:48.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.449+0000 7f8d35ffb640 1 -- 172.21.15.78:0/3557628051 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d4801a860 con 0x55a7569861d0 2024-08-18T23:51:48.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.449+0000 7f8d56af1740 10 client.26054 fetch_fsmap learned FSMap version 30 2024-08-18T23:51:48.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.449+0000 7f8d56af1740 1 -- 172.21.15.78:0/3557628051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a75699fc50 con 0x55a7569861d0 2024-08-18T23:51:48.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.449+0000 7f8d35ffb640 1 -- 172.21.15.78:0/3557628051 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d48029420 con 0x55a7569861d0 2024-08-18T23:51:48.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.450+0000 7f8d35ffb640 1 -- 172.21.15.78:0/3557628051 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8d480283c0 con 0x55a7569861d0 2024-08-18T23:51:48.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.450+0000 7f8d56af1740 10 client.26054 fetch_fsmap finished waiting for FSMap version 30 2024-08-18T23:51:48.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.450+0000 7f8d56af1740 10 client.26054 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:51:48.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.450+0000 7f8d56af1740 20 client.26054 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:48.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.450+0000 7f8d56af1740 1 --2- 172.21.15.78:0/3557628051 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55a7569a1380 0x55a7569a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:48.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.450+0000 7f8d56af1740 4 client.26054 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:51:48.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.450+0000 7f8d56af1740 1 -- 172.21.15.78:0/3557628051 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8d4800cb40 con 0x55a7569a1380 2024-08-18T23:51:48.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.450+0000 7f8d51c8e640 1 --2- 172.21.15.78:0/3557628051 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55a7569a1380 0x55a7569a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:48.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.455+0000 7f8d35ffb640 10 client.26054 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:51:48.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.455+0000 7f8d51c8e640 1 --2- 172.21.15.78:0/3557628051 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55a7569a1380 0x55a7569a37a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=699b79a916345a6c server_cookie=d5b23bdafa29c9fe in_seq=0 out_seq=0 2024-08-18T23:51:48.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.458+0000 7f8d35ffb640 1 -- 172.21.15.78:0/3557628051 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8d4800cb40 con 0x55a7569a1380 2024-08-18T23:51:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.458+0000 7f8d35ffb640 10 client.26054 handle_command_reply: tid=0 2024-08-18T23:51:48.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.458+0000 7f8d56af1740 1 client.26054 shutdown 2024-08-18T23:51:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.459+0000 7f8d56af1740 1 -- 172.21.15.78:0/3557628051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 msgr2=0x55a75698e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.459+0000 7f8d56af1740 1 --2- 172.21.15.78:0/3557628051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 0x55a75698e2d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8d48004700 tx=0x7f8d48011c60 comp rx=0 tx=0).stop 2024-08-18T23:51:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.459+0000 7f8d56af1740 1 -- 172.21.15.78:0/3557628051 shutdown_connections 2024-08-18T23:51:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.459+0000 7f8d56af1740 1 --2- 172.21.15.78:0/3557628051 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55a7569a1380 0x55a7569a37a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.459+0000 7f8d56af1740 1 --2- 172.21.15.78:0/3557628051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7569913b0 0x55a75698ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.459+0000 7f8d56af1740 1 --2- 172.21.15.78:0/3557628051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a756986af0 0x55a75698e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.459+0000 7f8d56af1740 1 --2- 172.21.15.78:0/3557628051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 0x55a75698e2d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.459+0000 7f8d56af1740 1 -- 172.21.15.78:0/3557628051 >> 172.21.15.78:0/3557628051 conn(0x55a756982760 msgr2=0x55a756988440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:48.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.459+0000 7f8d35ffb640 0 client.26054 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:51:48.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.459+0000 7f8d56af1740 1 -- 172.21.15.78:0/3557628051 shutdown_connections 2024-08-18T23:51:48.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.460+0000 7f8d56af1740 1 -- 172.21.15.78:0/3557628051 wait complete. 2024-08-18T23:51:48.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.461+0000 7f8d56af1740 20 client.26054 trim_cache size 0 max 16384 2024-08-18T23:51:48.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.483+0000 7f8d56af1740 1 Processor -- start 2024-08-18T23:51:48.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.484+0000 7f8d56af1740 1 -- start start 2024-08-18T23:51:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.484+0000 7f8d56af1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 0x55a756988c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.484+0000 7f8d56af1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a756986af0 0x55a75698eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.484+0000 7f8d56af1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7569913b0 0x55a7569a9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.484+0000 7f8d56af1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d30012530 con 0x55a756986af0 2024-08-18T23:51:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.484+0000 7f8d56af1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3000fda0 con 0x55a7569861d0 2024-08-18T23:51:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.484+0000 7f8d56af1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3000dbf0 con 0x55a7569913b0 2024-08-18T23:51:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.484+0000 7f8d51c8e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a756986af0 0x55a75698eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.484+0000 7f8d51c8e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a756986af0 0x55a75698eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44898/0 (socket says 172.21.15.78:44898) 2024-08-18T23:51:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.484+0000 7f8d51c8e640 1 -- 172.21.15.78:0/3513837198 learned_addr learned my addr 172.21.15.78:0/3513837198 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.484+0000 7f8d52c90640 1 --2- 172.21.15.78:0/3513837198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 0x55a756988c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:48.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.485+0000 7f8d52c90640 1 --2- 172.21.15.78:0/3513837198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7569913b0 0x55a7569a9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.485+0000 7f8d52c90640 1 -- 172.21.15.78:0/3513837198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7569913b0 msgr2=0x55a7569a9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.485+0000 7f8d52c90640 1 --2- 172.21.15.78:0/3513837198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7569913b0 0x55a7569a9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.485+0000 7f8d52c90640 1 -- 172.21.15.78:0/3513837198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a756986af0 msgr2=0x55a75698eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.485+0000 7f8d52c90640 1 --2- 172.21.15.78:0/3513837198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a756986af0 0x55a75698eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.485+0000 7f8d52c90640 1 -- 172.21.15.78:0/3513837198 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a756980ce0 con 0x55a7569861d0 2024-08-18T23:51:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.485+0000 7f8d51c8e640 1 --2- 172.21.15.78:0/3513837198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a756986af0 0x55a75698eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:51:48.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.485+0000 7f8d52c90640 1 --2- 172.21.15.78:0/3513837198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 0x55a756988c30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800b010 tx=0x7f8d48013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d69615b0547f031f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.486+0000 7f8d37fff640 1 -- 172.21.15.78:0/3513837198 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4800cb40 con 0x55a7569861d0 2024-08-18T23:51:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.486+0000 7f8d37fff640 1 -- 172.21.15.78:0/3513837198 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d48006ae0 con 0x55a7569861d0 2024-08-18T23:51:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.486+0000 7f8d37fff640 1 -- 172.21.15.78:0/3513837198 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48029820 con 0x55a7569861d0 2024-08-18T23:51:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.487+0000 7f8d56af1740 1 -- 172.21.15.78:0/3513837198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 msgr2=0x55a756988c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.487+0000 7f8d56af1740 1 --2- 172.21.15.78:0/3513837198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 0x55a756988c30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800b010 tx=0x7f8d48013040 comp rx=0 tx=0).stop 2024-08-18T23:51:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.487+0000 7f8d56af1740 1 -- 172.21.15.78:0/3513837198 shutdown_connections 2024-08-18T23:51:48.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.487+0000 7f8d56af1740 1 --2- 172.21.15.78:0/3513837198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7569913b0 0x55a7569a9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.487+0000 7f8d56af1740 1 --2- 172.21.15.78:0/3513837198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a756986af0 0x55a75698eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.487+0000 7f8d56af1740 1 --2- 172.21.15.78:0/3513837198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7569861d0 0x55a756988c30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.487+0000 7f8d56af1740 1 -- 172.21.15.78:0/3513837198 >> 172.21.15.78:0/3513837198 conn(0x55a756982760 msgr2=0x55a756989590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:48.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.488+0000 7f8d56af1740 1 -- 172.21.15.78:0/3513837198 shutdown_connections 2024-08-18T23:51:48.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.488+0000 7f8d56af1740 1 -- 172.21.15.78:0/3513837198 wait complete. 2024-08-18T23:51:48.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.488+0000 7f8d56af1740 1 -- start start 2024-08-18T23:51:48.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.489+0000 7f8d56af1740 1 Processor -- start 2024-08-18T23:51:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.489+0000 7f8d56af1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7569861d0 0x55a7569993e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.490+0000 7f8d56af1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a756986af0 0x55a756999920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.490+0000 7f8d56af1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7569913b0 0x55a75698a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.490+0000 7f8d56af1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3000e260 con 0x55a7569861d0 2024-08-18T23:51:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.490+0000 7f8d56af1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3000f6d0 con 0x55a756986af0 2024-08-18T23:51:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.490+0000 7f8d56af1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3000e930 con 0x55a7569913b0 2024-08-18T23:51:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.490+0000 7f8d52c90640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7569913b0 0x55a75698a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.490+0000 7f8d52c90640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7569913b0 0x55a75698a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:43126/0 (socket says 172.21.15.78:43126) 2024-08-18T23:51:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.490+0000 7f8d52c90640 1 -- 172.21.15.78:0/1721429193 learned_addr learned my addr 172.21.15.78:0/1721429193 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:48.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.491+0000 7f8d56af1740 10 client.0 _ll_register_callbacks cb 0x7f8d4800a6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.491+0000 7f8d51c8e640 1 --2- 172.21.15.78:0/1721429193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a756986af0 0x55a756999920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.491+0000 7f8d52c90640 1 --2- 172.21.15.78:0/1721429193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7569861d0 0x55a7569993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.491+0000 7f8d52c90640 1 -- 172.21.15.78:0/1721429193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a756986af0 msgr2=0x55a756999920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.491+0000 7f8d52c90640 1 --2- 172.21.15.78:0/1721429193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a756986af0 0x55a756999920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.491+0000 7f8d52c90640 1 -- 172.21.15.78:0/1721429193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7569861d0 msgr2=0x55a7569993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.491+0000 7f8d52c90640 1 --2- 172.21.15.78:0/1721429193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7569861d0 0x55a7569993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.491+0000 7f8d52c90640 1 -- 172.21.15.78:0/1721429193 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a75698aee0 con 0x55a7569913b0 2024-08-18T23:51:48.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.494+0000 7f8d52c90640 1 --2- 172.21.15.78:0/1721429193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7569913b0 0x55a75698a920 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55a7569a9b10 tx=0x7f8d48004660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d5107bb70639b9c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:48.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.495+0000 7f8d35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:51:48.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.495+0000 7f8d35ffb640 1 -- 172.21.15.78:0/1721429193 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d48015040 con 0x55a7569913b0 2024-08-18T23:51:48.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.495+0000 7f8d35ffb640 1 -- 172.21.15.78:0/1721429193 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f8d4800a3b0 con 0x55a7569913b0 2024-08-18T23:51:48.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.495+0000 7f8d56af1740 1 -- 172.21.15.78:0/1721429193 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d48006ae0 con 0x55a7569913b0 2024-08-18T23:51:48.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.495+0000 7f8d35ffb640 1 -- 172.21.15.78:0/1721429193 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d4801a660 con 0x55a7569913b0 2024-08-18T23:51:48.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.496+0000 7f8d56af1740 10 client.25870 fetch_fsmap learned FSMap version 30 2024-08-18T23:51:48.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.496+0000 7f8d56af1740 1 -- 172.21.15.78:0/1721429193 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a756a0b8e0 con 0x55a7569913b0 2024-08-18T23:51:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.496+0000 7f8d35ffb640 1 -- 172.21.15.78:0/1721429193 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d48006ae0 con 0x55a7569913b0 2024-08-18T23:51:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.497+0000 7f8d35ffb640 1 -- 172.21.15.78:0/1721429193 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8d48042050 con 0x55a7569913b0 2024-08-18T23:51:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.497+0000 7f8d56af1740 10 client.25870 fetch_fsmap finished waiting for FSMap version 30 2024-08-18T23:51:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.497+0000 7f8d56af1740 10 client.25870 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:51:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.497+0000 7f8d56af1740 20 client.25870 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.497+0000 7f8d56af1740 1 --2- 172.21.15.78:0/1721429193 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55a756a0bd70 0x55a756a0e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.497+0000 7f8d56af1740 4 client.25870 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:51:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.497+0000 7f8d56af1740 1 -- 172.21.15.78:0/1721429193 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7568f8fb0 con 0x55a756a0bd70 2024-08-18T23:51:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.498+0000 7f8d51c8e640 1 --2- 172.21.15.78:0/1721429193 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55a756a0bd70 0x55a756a0e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.503+0000 7f8d51c8e640 1 --2- 172.21.15.78:0/1721429193 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55a756a0bd70 0x55a756a0e190 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bda3a3ca8e210965 server_cookie=95093823cc670ad3 in_seq=0 out_seq=0 2024-08-18T23:51:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.503+0000 7f8d35ffb640 10 client.25870 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:51:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.503+0000 7f8d35ffb640 1 -- 172.21.15.78:0/1721429193 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7568f8fb0 con 0x55a756a0bd70 2024-08-18T23:51:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.503+0000 7f8d35ffb640 10 client.25870 handle_command_reply: tid=0 2024-08-18T23:51:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.503+0000 7f8d56af1740 1 client.25870 shutdown 2024-08-18T23:51:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.504+0000 7f8d56af1740 1 -- 172.21.15.78:0/1721429193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7569913b0 msgr2=0x55a75698a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.504+0000 7f8d56af1740 1 --2- 172.21.15.78:0/1721429193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7569913b0 0x55a75698a920 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55a7569a9b10 tx=0x7f8d48004660 comp rx=0 tx=0).stop 2024-08-18T23:51:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.504+0000 7f8d56af1740 1 -- 172.21.15.78:0/1721429193 shutdown_connections 2024-08-18T23:51:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.504+0000 7f8d56af1740 1 --2- 172.21.15.78:0/1721429193 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55a756a0bd70 0x55a756a0e190 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.504+0000 7f8d56af1740 1 --2- 172.21.15.78:0/1721429193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a7569913b0 0x55a75698a920 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.504+0000 7f8d56af1740 1 --2- 172.21.15.78:0/1721429193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a756986af0 0x55a756999920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.504+0000 7f8d56af1740 1 --2- 172.21.15.78:0/1721429193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7569861d0 0x55a7569993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.504+0000 7f8d56af1740 1 -- 172.21.15.78:0/1721429193 >> 172.21.15.78:0/1721429193 conn(0x55a756982760 msgr2=0x55a7569810a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.504+0000 7f8d35ffb640 0 client.25870 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:51:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.504+0000 7f8d56af1740 1 -- 172.21.15.78:0/1721429193 shutdown_connections 2024-08-18T23:51:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.504+0000 7f8d56af1740 1 -- 172.21.15.78:0/1721429193 wait complete. 2024-08-18T23:51:48.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.505+0000 7f8d56af1740 20 client.25870 trim_cache size 0 max 16384 2024-08-18T23:51:48.507 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:51:48.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.508+0000 7f8d53491640 1 -- 172.21.15.78:0/2083190636 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8d380a2da0 msgr2=0x7f8d380a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:48.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.508+0000 7f8d53491640 1 --2- 172.21.15.78:0/2083190636 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8d380a2da0 0x7f8d380a5260 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c067700 tx=0x7f8d4c08e2a0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.508+0000 7f8d53491640 1 -- 172.21.15.78:0/2083190636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d440a4a80 msgr2=0x7f8d440a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:48.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.508+0000 7f8d53491640 1 --2- 172.21.15.78:0/2083190636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d440a4a80 0x7f8d440a84a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c08e890 tx=0x7f8d4c097a10 comp rx=0 tx=0).stop 2024-08-18T23:51:48.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.508+0000 7f8d53491640 1 -- 172.21.15.78:0/2083190636 shutdown_connections 2024-08-18T23:51:48.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.508+0000 7f8d53491640 1 --2- 172.21.15.78:0/2083190636 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8d380a2da0 0x7f8d380a5260 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.508+0000 7f8d53491640 1 --2- 172.21.15.78:0/2083190636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d440a8f20 0x7f8d4414f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.508+0000 7f8d53491640 1 --2- 172.21.15.78:0/2083190636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d440b2df0 0x7f8d440a89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.508+0000 7f8d53491640 1 --2- 172.21.15.78:0/2083190636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d440a4a80 0x7f8d440a84a0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:48.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.508+0000 7f8d53491640 1 -- 172.21.15.78:0/2083190636 >> 172.21.15.78:0/2083190636 conn(0x7f8d4401a0e0 msgr2=0x7f8d440b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:48.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.509+0000 7f8d53491640 1 -- 172.21.15.78:0/2083190636 shutdown_connections 2024-08-18T23:51:48.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:48.509+0000 7f8d53491640 1 -- 172.21.15.78:0/2083190636 wait complete. 2024-08-18T23:51:49.360 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-18T23:51:49.615 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:51:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:49 smithi161 ceph-mon[22027]: pgmap v575: 129 pgs: 129 active+clean; 543 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 759 op/s 2024-08-18T23:51:49.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:49 smithi078 ceph-mon[18440]: pgmap v575: 129 pgs: 129 active+clean; 543 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 759 op/s 2024-08-18T23:51:49.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:49 smithi123 ceph-mon[21808]: pgmap v575: 129 pgs: 129 active+clean; 543 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 759 op/s 2024-08-18T23:51:50.398 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:51:50.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.203+0000 7f284f886640 1 -- 172.21.15.78:0/2955925637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2848106810 msgr2=0x7f2848106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:50.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.203+0000 7f284f886640 1 --2- 172.21.15.78:0/2955925637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2848106810 0x7f2848106bf0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2838009cd0 tx=0x7f283802a8e0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.203+0000 7f284f886640 1 -- 172.21.15.78:0/2955925637 shutdown_connections 2024-08-18T23:51:50.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.203+0000 7f284f886640 1 --2- 172.21.15.78:0/2955925637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2848101900 0x7f284810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.203+0000 7f284f886640 1 --2- 172.21.15.78:0/2955925637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2848100f40 0x7f28481013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.203+0000 7f284f886640 1 --2- 172.21.15.78:0/2955925637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2848106810 0x7f2848106bf0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.203+0000 7f284f886640 1 -- 172.21.15.78:0/2955925637 >> 172.21.15.78:0/2955925637 conn(0x7f2848076f60 msgr2=0x7f28480fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:50.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.204+0000 7f284f886640 1 -- 172.21.15.78:0/2955925637 shutdown_connections 2024-08-18T23:51:50.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.204+0000 7f284f886640 1 -- 172.21.15.78:0/2955925637 wait complete. 2024-08-18T23:51:50.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.204+0000 7f284f886640 1 Processor -- start 2024-08-18T23:51:50.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.204+0000 7f284f886640 1 -- start start 2024-08-18T23:51:50.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.204+0000 7f284f886640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2848100f40 0x7f2848198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:50.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.204+0000 7f284f886640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2848101900 0x7f2848199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:50.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.204+0000 7f284f886640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2848106810 0x7f28481a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:50.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.205+0000 7f284f886640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2848110b30 con 0x7f2848101900 2024-08-18T23:51:50.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.205+0000 7f284f886640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28481109b0 con 0x7f2848100f40 2024-08-18T23:51:50.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.205+0000 7f284f886640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2848110cb0 con 0x7f2848106810 2024-08-18T23:51:50.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.205+0000 7f284ce21640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2848101900 0x7f2848199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:50.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.205+0000 7f284ce21640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2848101900 0x7f2848199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44914/0 (socket says 172.21.15.78:44914) 2024-08-18T23:51:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.205+0000 7f284ce21640 1 -- 172.21.15.78:0/39267358 learned_addr learned my addr 172.21.15.78:0/39267358 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.205+0000 7f284d622640 1 --2- 172.21.15.78:0/39267358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2848100f40 0x7f2848198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.205+0000 7f284de23640 1 --2- 172.21.15.78:0/39267358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2848106810 0x7f28481a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.205+0000 7f284ce21640 1 -- 172.21.15.78:0/39267358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2848106810 msgr2=0x7f28481a3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.205+0000 7f284ce21640 1 --2- 172.21.15.78:0/39267358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2848106810 0x7f28481a3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.205+0000 7f284ce21640 1 -- 172.21.15.78:0/39267358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2848100f40 msgr2=0x7f2848198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.205+0000 7f284ce21640 1 --2- 172.21.15.78:0/39267358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2848100f40 0x7f2848198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.205+0000 7f284ce21640 1 -- 172.21.15.78:0/39267358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28481a44a0 con 0x7f2848101900 2024-08-18T23:51:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.205+0000 7f284ce21640 1 --2- 172.21.15.78:0/39267358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2848101900 0x7f2848199420 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f283c00d950 tx=0x7f283c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.205+0000 7f28467fc640 1 -- 172.21.15.78:0/39267358 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c017030 con 0x7f2848101900 2024-08-18T23:51:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.205+0000 7f284f886640 1 -- 172.21.15.78:0/39267358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28481a4670 con 0x7f2848101900 2024-08-18T23:51:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.205+0000 7f28467fc640 1 -- 172.21.15.78:0/39267358 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f283c004020 con 0x7f2848101900 2024-08-18T23:51:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.205+0000 7f28467fc640 1 -- 172.21.15.78:0/39267358 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c0047a0 con 0x7f2848101900 2024-08-18T23:51:50.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.205+0000 7f284f886640 1 -- 172.21.15.78:0/39267358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28481a4bb0 con 0x7f2848101900 2024-08-18T23:51:50.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.208+0000 7f28467fc640 1 -- 172.21.15.78:0/39267358 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f283c0041d0 con 0x7f2848101900 2024-08-18T23:51:50.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.208+0000 7f2851c84740 1 Processor -- start 2024-08-18T23:51:50.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.208+0000 7f2851c84740 1 -- start start 2024-08-18T23:51:50.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.209+0000 7f2851c84740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c48e90 0x55cc22c49270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:50.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.209+0000 7f2851c84740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc22c497b0 0x55cc22c580a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:50.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.209+0000 7f2851c84740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c585e0 0x55cc22c5a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:50.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.209+0000 7f2851c84740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cc22baea40 con 0x55cc22c48e90 2024-08-18T23:51:50.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.209+0000 7f2851c84740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cc22b469d0 con 0x55cc22c585e0 2024-08-18T23:51:50.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.209+0000 7f2851c84740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cc22baaa00 con 0x55cc22c497b0 2024-08-18T23:51:50.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.209+0000 7f284de23640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c48e90 0x55cc22c49270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:50.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.209+0000 7f284de23640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c48e90 0x55cc22c49270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44924/0 (socket says 172.21.15.78:44924) 2024-08-18T23:51:50.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.209+0000 7f284de23640 1 -- 172.21.15.78:0/3868563345 learned_addr learned my addr 172.21.15.78:0/3868563345 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:50.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.209+0000 7f28467fc640 1 --2- 172.21.15.78:0/39267358 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f28300a2b80 0x7f28300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:50.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.209+0000 7f284ce21640 1 --2- 172.21.15.78:0/3868563345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c585e0 0x55cc22c5a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:50.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.209+0000 7f284d622640 1 --2- 172.21.15.78:0/3868563345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc22c497b0 0x55cc22c580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:50.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.209+0000 7f284de23640 1 --2- 172.21.15.78:0/39267358 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f28300a2b80 0x7f28300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:50.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.209+0000 7f284de23640 1 -- 172.21.15.78:0/3868563345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc22c497b0 msgr2=0x55cc22c580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:50.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.209+0000 7f284de23640 1 --2- 172.21.15.78:0/3868563345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc22c497b0 0x55cc22c580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.209+0000 7f28467fc640 1 -- 172.21.15.78:0/39267358 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f283c0ced40 con 0x7f2848101900 2024-08-18T23:51:50.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.209+0000 7f284de23640 1 -- 172.21.15.78:0/3868563345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c585e0 msgr2=0x55cc22c5a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:50.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.209+0000 7f284de23640 1 --2- 172.21.15.78:0/3868563345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c585e0 0x55cc22c5a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.209+0000 7f284de23640 1 -- 172.21.15.78:0/3868563345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc22b4d650 con 0x55cc22c48e90 2024-08-18T23:51:50.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.210+0000 7f284de23640 1 --2- 172.21.15.78:0/3868563345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c48e90 0x55cc22c49270 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f284000bd20 tx=0x7f28400039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e280bfb35de919 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:50.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.210+0000 7f282f7fe640 1 -- 172.21.15.78:0/3868563345 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284000b6e0 con 0x55cc22c48e90 2024-08-18T23:51:50.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.210+0000 7f282f7fe640 1 -- 172.21.15.78:0/3868563345 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28400054b0 con 0x55cc22c48e90 2024-08-18T23:51:50.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.210+0000 7f282f7fe640 1 -- 172.21.15.78:0/3868563345 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284001ed50 con 0x55cc22c48e90 2024-08-18T23:51:50.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.210+0000 7f284de23640 1 --2- 172.21.15.78:0/39267358 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f28300a2b80 0x7f28300a5040 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2840021000 tx=0x7f2840020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:50.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.211+0000 7f2851c84740 1 -- 172.21.15.78:0/3868563345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c48e90 msgr2=0x55cc22c49270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:50.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.211+0000 7f2851c84740 1 --2- 172.21.15.78:0/3868563345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c48e90 0x55cc22c49270 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f284000bd20 tx=0x7f28400039c0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.211+0000 7f2851c84740 1 -- 172.21.15.78:0/3868563345 shutdown_connections 2024-08-18T23:51:50.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.211+0000 7f2851c84740 1 --2- 172.21.15.78:0/3868563345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c585e0 0x55cc22c5a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.211+0000 7f2851c84740 1 --2- 172.21.15.78:0/3868563345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc22c497b0 0x55cc22c580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.211+0000 7f2851c84740 1 --2- 172.21.15.78:0/3868563345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c48e90 0x55cc22c49270 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.211+0000 7f2851c84740 1 -- 172.21.15.78:0/3868563345 >> 172.21.15.78:0/3868563345 conn(0x55cc22c45580 msgr2=0x55cc22c45960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:50.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.211+0000 7f2851c84740 1 -- 172.21.15.78:0/3868563345 shutdown_connections 2024-08-18T23:51:50.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.211+0000 7f2851c84740 1 -- 172.21.15.78:0/3868563345 wait complete. 2024-08-18T23:51:50.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.211+0000 7f2851c84740 1 -- start start 2024-08-18T23:51:50.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.211+0000 7f2851c84740 1 Processor -- start 2024-08-18T23:51:50.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.211+0000 7f2851c84740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c48e90 0x55cc22c554f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:50.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.211+0000 7f2851c84740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c497b0 0x55cc22c55a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:50.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.211+0000 7f2851c84740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc22c585e0 0x55cc22c55f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:50.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.211+0000 7f2851c84740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cc22c5b1b0 con 0x55cc22c497b0 2024-08-18T23:51:50.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.211+0000 7f2851c84740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cc22b69b20 con 0x55cc22c48e90 2024-08-18T23:51:50.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.211+0000 7f2851c84740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cc22c5b330 con 0x55cc22c585e0 2024-08-18T23:51:50.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f2851c84740 10 client.0 _ll_register_callbacks cb 0x55cc22b67bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:50.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f284ce21640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c497b0 0x55cc22c55a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:50.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f284d622640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c48e90 0x55cc22c554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:50.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f284ce21640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c497b0 0x55cc22c55a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44934/0 (socket says 172.21.15.78:44934) 2024-08-18T23:51:50.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f284ce21640 1 -- 172.21.15.78:0/3572031138 learned_addr learned my addr 172.21.15.78:0/3572031138 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:50.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f284de23640 1 --2- 172.21.15.78:0/3572031138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc22c585e0 0x55cc22c55f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:50.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f284ce21640 1 -- 172.21.15.78:0/3572031138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc22c585e0 msgr2=0x55cc22c55f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:50.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f284ce21640 1 --2- 172.21.15.78:0/3572031138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc22c585e0 0x55cc22c55f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f284ce21640 1 -- 172.21.15.78:0/3572031138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c48e90 msgr2=0x55cc22c554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:50.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f284ce21640 1 --2- 172.21.15.78:0/3572031138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c48e90 0x55cc22c554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f284ce21640 1 -- 172.21.15.78:0/3572031138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc229203c0 con 0x55cc22c497b0 2024-08-18T23:51:50.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f284d622640 1 --2- 172.21.15.78:0/3572031138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c48e90 0x55cc22c554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:51:50.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f284ce21640 1 --2- 172.21.15.78:0/3572031138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c497b0 0x55cc22c55a30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f283c000c00 tx=0x7f283c005200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4fe2946e0cf1fa server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:50.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f282cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:51:50.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f282cff9640 1 -- 172.21.15.78:0/3572031138 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c09f5c0 con 0x55cc22c497b0 2024-08-18T23:51:50.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f282cff9640 1 -- 172.21.15.78:0/3572031138 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f283c09f140 con 0x55cc22c497b0 2024-08-18T23:51:50.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f2851c84740 1 -- 172.21.15.78:0/3572031138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28400054b0 con 0x55cc22c497b0 2024-08-18T23:51:50.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f282cff9640 1 -- 172.21.15.78:0/3572031138 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f283c09b910 con 0x55cc22c497b0 2024-08-18T23:51:50.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f2851c84740 10 client.16674 fetch_fsmap learned FSMap version 30 2024-08-18T23:51:50.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.212+0000 7f2851c84740 1 -- 172.21.15.78:0/3572031138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cc22c66e80 con 0x55cc22c497b0 2024-08-18T23:51:50.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.213+0000 7f282cff9640 1 -- 172.21.15.78:0/3572031138 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28400054b0 con 0x55cc22c497b0 2024-08-18T23:51:50.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.213+0000 7f282cff9640 1 -- 172.21.15.78:0/3572031138 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f283c0a39f0 con 0x55cc22c497b0 2024-08-18T23:51:50.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.213+0000 7f2851c84740 10 client.16674 fetch_fsmap finished waiting for FSMap version 30 2024-08-18T23:51:50.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.213+0000 7f2851c84740 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:51:50.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.213+0000 7f2851c84740 20 client.16674 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:50.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.213+0000 7f2851c84740 1 --2- 172.21.15.78:0/3572031138 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cc22c68270 0x55cc22c6a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:50.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.213+0000 7f2851c84740 4 client.16674 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:51:50.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.213+0000 7f2851c84740 1 -- 172.21.15.78:0/3572031138 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc22bbbfb0 con 0x55cc22c68270 2024-08-18T23:51:50.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.213+0000 7f284d622640 1 --2- 172.21.15.78:0/3572031138 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cc22c68270 0x55cc22c6a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:50.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.213+0000 7f282cff9640 10 client.16674 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:50.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.213+0000 7f284d622640 1 --2- 172.21.15.78:0/3572031138 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cc22c68270 0x55cc22c6a6c0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f158a0141d8c4b7a server_cookie=a71c6413d56de27 in_seq=0 out_seq=0 2024-08-18T23:51:50.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.215+0000 7f282cff9640 1 -- 172.21.15.78:0/3572031138 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc22bbbfb0 con 0x55cc22c68270 2024-08-18T23:51:50.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.215+0000 7f282cff9640 10 client.16674 handle_command_reply: tid=0 2024-08-18T23:51:50.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.215+0000 7f2851c84740 1 client.16674 shutdown 2024-08-18T23:51:50.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.215+0000 7f2851c84740 1 -- 172.21.15.78:0/3572031138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c497b0 msgr2=0x55cc22c55a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:50.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.215+0000 7f2851c84740 1 --2- 172.21.15.78:0/3572031138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c497b0 0x55cc22c55a30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f283c000c00 tx=0x7f283c005200 comp rx=0 tx=0).stop 2024-08-18T23:51:50.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.215+0000 7f2851c84740 1 -- 172.21.15.78:0/3572031138 shutdown_connections 2024-08-18T23:51:50.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.215+0000 7f2851c84740 1 --2- 172.21.15.78:0/3572031138 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cc22c68270 0x55cc22c6a6c0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.215+0000 7f2851c84740 1 --2- 172.21.15.78:0/3572031138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc22c585e0 0x55cc22c55f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.215+0000 7f2851c84740 1 --2- 172.21.15.78:0/3572031138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c497b0 0x55cc22c55a30 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.215+0000 7f2851c84740 1 --2- 172.21.15.78:0/3572031138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c48e90 0x55cc22c554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.215+0000 7f2851c84740 1 -- 172.21.15.78:0/3572031138 >> 172.21.15.78:0/3572031138 conn(0x55cc22c45580 msgr2=0x55cc22c4b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:50.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.215+0000 7f282cff9640 0 client.16674 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:50.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.216+0000 7f2851c84740 1 -- 172.21.15.78:0/3572031138 shutdown_connections 2024-08-18T23:51:50.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.216+0000 7f2851c84740 1 -- 172.21.15.78:0/3572031138 wait complete. 2024-08-18T23:51:50.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.216+0000 7f2851c84740 20 client.16674 trim_cache size 0 max 16384 2024-08-18T23:51:50.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.228+0000 7f2851c84740 1 Processor -- start 2024-08-18T23:51:50.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.228+0000 7f2851c84740 1 -- start start 2024-08-18T23:51:50.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.229+0000 7f2851c84740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c48e90 0x55cc22c6ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:50.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.229+0000 7f2851c84740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c497b0 0x55cc22c70510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:50.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.229+0000 7f2851c84740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc22c585e0 0x55cc22c4aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:50.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.229+0000 7f2851c84740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f28481af160 con 0x55cc22c48e90 2024-08-18T23:51:50.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.229+0000 7f2851c84740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28481ac9d0 con 0x55cc22c497b0 2024-08-18T23:51:50.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.229+0000 7f2851c84740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f28481aa820 con 0x55cc22c585e0 2024-08-18T23:51:50.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.229+0000 7f284d622640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c48e90 0x55cc22c6ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:50.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.229+0000 7f284d622640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c48e90 0x55cc22c6ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44944/0 (socket says 172.21.15.78:44944) 2024-08-18T23:51:50.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.229+0000 7f284d622640 1 -- 172.21.15.78:0/1598168684 learned_addr learned my addr 172.21.15.78:0/1598168684 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:50.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.229+0000 7f284ce21640 1 --2- 172.21.15.78:0/1598168684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c497b0 0x55cc22c70510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:50.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.229+0000 7f284d622640 1 -- 172.21.15.78:0/1598168684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc22c585e0 msgr2=0x55cc22c4aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:50.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.229+0000 7f284d622640 1 --2- 172.21.15.78:0/1598168684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc22c585e0 0x55cc22c4aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.229+0000 7f284d622640 1 -- 172.21.15.78:0/1598168684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c497b0 msgr2=0x55cc22c70510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:50.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.229+0000 7f284d622640 1 --2- 172.21.15.78:0/1598168684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c497b0 0x55cc22c70510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.229+0000 7f284d622640 1 -- 172.21.15.78:0/1598168684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc22c50270 con 0x55cc22c48e90 2024-08-18T23:51:50.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.229+0000 7f284d622640 1 --2- 172.21.15.78:0/1598168684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c48e90 0x55cc22c6ffd0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2838006010 tx=0x7f2838005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45041a9b0cce4f32 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:50.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.229+0000 7f282effd640 1 -- 172.21.15.78:0/1598168684 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838003970 con 0x55cc22c48e90 2024-08-18T23:51:50.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.229+0000 7f282effd640 1 -- 172.21.15.78:0/1598168684 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f283804d070 con 0x55cc22c48e90 2024-08-18T23:51:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.229+0000 7f282effd640 1 -- 172.21.15.78:0/1598168684 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838030800 con 0x55cc22c48e90 2024-08-18T23:51:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.230+0000 7f2851c84740 1 -- 172.21.15.78:0/1598168684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c48e90 msgr2=0x55cc22c6ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.230+0000 7f2851c84740 1 --2- 172.21.15.78:0/1598168684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c48e90 0x55cc22c6ffd0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2838006010 tx=0x7f2838005ef0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.230+0000 7f2851c84740 1 -- 172.21.15.78:0/1598168684 shutdown_connections 2024-08-18T23:51:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.230+0000 7f2851c84740 1 --2- 172.21.15.78:0/1598168684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc22c585e0 0x55cc22c4aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.230+0000 7f2851c84740 1 --2- 172.21.15.78:0/1598168684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c497b0 0x55cc22c70510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.230+0000 7f2851c84740 1 --2- 172.21.15.78:0/1598168684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c48e90 0x55cc22c6ffd0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.230+0000 7f2851c84740 1 -- 172.21.15.78:0/1598168684 >> 172.21.15.78:0/1598168684 conn(0x55cc22c45580 msgr2=0x55cc22c53dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.230+0000 7f2851c84740 1 -- 172.21.15.78:0/1598168684 shutdown_connections 2024-08-18T23:51:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.230+0000 7f2851c84740 1 -- 172.21.15.78:0/1598168684 wait complete. 2024-08-18T23:51:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.230+0000 7f2851c84740 1 -- start start 2024-08-18T23:51:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.230+0000 7f2851c84740 1 Processor -- start 2024-08-18T23:51:50.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f2851c84740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc22c48e90 0x55cc22c5b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f2851c84740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c497b0 0x55cc22c5bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f2851c84740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c585e0 0x55cc22c5c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f2851c84740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f28481aae90 con 0x55cc22c497b0 2024-08-18T23:51:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f2851c84740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28481ac300 con 0x55cc22c585e0 2024-08-18T23:51:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f2851c84740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f28481ab560 con 0x55cc22c48e90 2024-08-18T23:51:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f2851c84740 10 client.0 _ll_register_callbacks cb 0x7f283c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f284ce21640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c497b0 0x55cc22c5bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f284ce21640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c497b0 0x55cc22c5bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44954/0 (socket says 172.21.15.78:44954) 2024-08-18T23:51:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f284ce21640 1 -- 172.21.15.78:0/228911008 learned_addr learned my addr 172.21.15.78:0/228911008 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f284de23640 1 --2- 172.21.15.78:0/228911008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c585e0 0x55cc22c5c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f284d622640 1 --2- 172.21.15.78:0/228911008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc22c48e90 0x55cc22c5b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:50.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f284ce21640 1 -- 172.21.15.78:0/228911008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc22c48e90 msgr2=0x55cc22c5b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f284ce21640 1 --2- 172.21.15.78:0/228911008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc22c48e90 0x55cc22c5b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f284ce21640 1 -- 172.21.15.78:0/228911008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c585e0 msgr2=0x55cc22c5c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f284ce21640 1 --2- 172.21.15.78:0/228911008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c585e0 0x55cc22c5c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f284ce21640 1 -- 172.21.15.78:0/228911008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc22cd10f0 con 0x55cc22c497b0 2024-08-18T23:51:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f284ce21640 1 --2- 172.21.15.78:0/228911008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c497b0 0x55cc22c5bac0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f283c09b1a0 tx=0x7f283c097ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c22fc65c945e878 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f282cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:51:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f282cff9640 1 -- 172.21.15.78:0/228911008 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c09f5c0 con 0x55cc22c497b0 2024-08-18T23:51:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f282cff9640 1 -- 172.21.15.78:0/228911008 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f283c09f350 con 0x55cc22c497b0 2024-08-18T23:51:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.231+0000 7f2851c84740 1 -- 172.21.15.78:0/228911008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f283804d070 con 0x55cc22c497b0 2024-08-18T23:51:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.232+0000 7f282cff9640 1 -- 172.21.15.78:0/228911008 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f283c09b210 con 0x55cc22c497b0 2024-08-18T23:51:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.232+0000 7f2851c84740 10 client.16680 fetch_fsmap learned FSMap version 30 2024-08-18T23:51:50.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.232+0000 7f2851c84740 1 -- 172.21.15.78:0/228911008 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cc22cd23d0 con 0x55cc22c497b0 2024-08-18T23:51:50.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.232+0000 7f282cff9640 1 -- 172.21.15.78:0/228911008 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f283804d070 con 0x55cc22c497b0 2024-08-18T23:51:50.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.232+0000 7f282cff9640 1 -- 172.21.15.78:0/228911008 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f283c09a0d0 con 0x55cc22c497b0 2024-08-18T23:51:50.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.232+0000 7f2851c84740 10 client.16680 fetch_fsmap finished waiting for FSMap version 30 2024-08-18T23:51:50.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.232+0000 7f2851c84740 10 client.16680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:51:50.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.232+0000 7f2851c84740 20 client.16680 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:50.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.232+0000 7f2851c84740 1 --2- 172.21.15.78:0/228911008 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cc22cd2a90 0x55cc22cd4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:50.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.232+0000 7f2851c84740 4 client.16680 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:51:50.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.232+0000 7f2851c84740 1 -- 172.21.15.78:0/228911008 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cc22bb64c0 con 0x55cc22cd2a90 2024-08-18T23:51:50.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.232+0000 7f284d622640 1 --2- 172.21.15.78:0/228911008 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cc22cd2a90 0x55cc22cd4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:50.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.233+0000 7f282cff9640 10 client.16680 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:50.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.233+0000 7f284d622640 1 --2- 172.21.15.78:0/228911008 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cc22cd2a90 0x55cc22cd4eb0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bedbb60599b3bc1 server_cookie=dc5be0f3b6af550e in_seq=0 out_seq=0 2024-08-18T23:51:50.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.236+0000 7f282cff9640 1 -- 172.21.15.78:0/228911008 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cc22bb64c0 con 0x55cc22cd2a90 2024-08-18T23:51:50.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.236+0000 7f282cff9640 10 client.16680 handle_command_reply: tid=0 2024-08-18T23:51:50.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.236+0000 7f2851c84740 1 client.16680 shutdown 2024-08-18T23:51:50.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.236+0000 7f2851c84740 1 -- 172.21.15.78:0/228911008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c497b0 msgr2=0x55cc22c5bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:50.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.236+0000 7f2851c84740 1 --2- 172.21.15.78:0/228911008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c497b0 0x55cc22c5bac0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f283c09b1a0 tx=0x7f283c097ab0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.236+0000 7f2851c84740 1 -- 172.21.15.78:0/228911008 shutdown_connections 2024-08-18T23:51:50.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.236+0000 7f2851c84740 1 --2- 172.21.15.78:0/228911008 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cc22cd2a90 0x55cc22cd4eb0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.237+0000 7f2851c84740 1 --2- 172.21.15.78:0/228911008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cc22c585e0 0x55cc22c5c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.237+0000 7f2851c84740 1 --2- 172.21.15.78:0/228911008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cc22c497b0 0x55cc22c5bac0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.237+0000 7f2851c84740 1 --2- 172.21.15.78:0/228911008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc22c48e90 0x55cc22c5b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.237+0000 7f2851c84740 1 -- 172.21.15.78:0/228911008 >> 172.21.15.78:0/228911008 conn(0x55cc22c45580 msgr2=0x55cc22c554b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:50.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.237+0000 7f282cff9640 0 client.16680 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:50.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.237+0000 7f2851c84740 1 -- 172.21.15.78:0/228911008 shutdown_connections 2024-08-18T23:51:50.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.237+0000 7f2851c84740 1 -- 172.21.15.78:0/228911008 wait complete. 2024-08-18T23:51:50.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.237+0000 7f2851c84740 20 client.16680 trim_cache size 0 max 16384 2024-08-18T23:51:50.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.237+0000 7f284f886640 1 -- 172.21.15.78:0/39267358 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f28300a2b80 msgr2=0x7f28300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:50.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.237+0000 7f284f886640 1 --2- 172.21.15.78:0/39267358 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f28300a2b80 0x7f28300a5040 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2840021000 tx=0x7f2840020000 comp rx=0 tx=0).stop 2024-08-18T23:51:50.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.237+0000 7f284f886640 1 -- 172.21.15.78:0/39267358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2848101900 msgr2=0x7f2848199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:50.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.237+0000 7f284f886640 1 --2- 172.21.15.78:0/39267358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2848101900 0x7f2848199420 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f283c00d950 tx=0x7f283c00de10 comp rx=0 tx=0).stop 2024-08-18T23:51:50.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.237+0000 7f284f886640 1 -- 172.21.15.78:0/39267358 shutdown_connections 2024-08-18T23:51:50.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.237+0000 7f284f886640 1 --2- 172.21.15.78:0/39267358 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f28300a2b80 0x7f28300a5040 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.237+0000 7f284f886640 1 --2- 172.21.15.78:0/39267358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2848106810 0x7f28481a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.237+0000 7f284f886640 1 --2- 172.21.15.78:0/39267358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2848101900 0x7f2848199420 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.237+0000 7f284f886640 1 --2- 172.21.15.78:0/39267358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2848100f40 0x7f2848198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:50.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.237+0000 7f284f886640 1 -- 172.21.15.78:0/39267358 >> 172.21.15.78:0/39267358 conn(0x7f2848076f60 msgr2=0x7f28480fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:50.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.238+0000 7f284f886640 1 -- 172.21.15.78:0/39267358 shutdown_connections 2024-08-18T23:51:50.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:50.238+0000 7f284f886640 1 -- 172.21.15.78:0/39267358 wait complete. 2024-08-18T23:51:51.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:51 smithi078 ceph-mon[18440]: pgmap v576: 129 pgs: 129 active+clean; 536 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 794 op/s 2024-08-18T23:51:51.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:51 smithi161 ceph-mon[22027]: pgmap v576: 129 pgs: 129 active+clean; 536 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 794 op/s 2024-08-18T23:51:51.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:51 smithi123 ceph-mon[21808]: pgmap v576: 129 pgs: 129 active+clean; 536 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 794 op/s 2024-08-18T23:51:52.229 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-18T23:51:52.229 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-18T23:51:52.610 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:51:53.456 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:53 smithi078 ceph-mon[18440]: pgmap v577: 129 pgs: 129 active+clean; 508 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 796 op/s 2024-08-18T23:51:53.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.580+0000 7f8af66ee640 1 -- 172.21.15.78:0/1590784353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8af01022a0 msgr2=0x7f8af01026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:53.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.580+0000 7f8af66ee640 1 --2- 172.21.15.78:0/1590784353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8af01022a0 0x7f8af01026e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009900 tx=0x7f8ad802a480 comp rx=0 tx=0).stop 2024-08-18T23:51:53.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.580+0000 7f8af66ee640 1 -- 172.21.15.78:0/1590784353 shutdown_connections 2024-08-18T23:51:53.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.580+0000 7f8af66ee640 1 --2- 172.21.15.78:0/1590784353 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8af0102c20 0x7f8af010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.580+0000 7f8af66ee640 1 --2- 172.21.15.78:0/1590784353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8af01022a0 0x7f8af01026e0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.580+0000 7f8af66ee640 1 --2- 172.21.15.78:0/1590784353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8af0100b50 0x7f8af0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.580+0000 7f8af66ee640 1 -- 172.21.15.78:0/1590784353 >> 172.21.15.78:0/1590784353 conn(0x7f8af00fc500 msgr2=0x7f8af00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:53.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.580+0000 7f8af66ee640 1 -- 172.21.15.78:0/1590784353 shutdown_connections 2024-08-18T23:51:53.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.580+0000 7f8af66ee640 1 -- 172.21.15.78:0/1590784353 wait complete. 2024-08-18T23:51:53.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.581+0000 7f8af66ee640 1 Processor -- start 2024-08-18T23:51:53.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.581+0000 7f8af66ee640 1 -- start start 2024-08-18T23:51:53.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.581+0000 7f8af66ee640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8af0100b50 0x7f8af01970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:53.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.581+0000 7f8af66ee640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8af01022a0 0x7f8af0197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.581+0000 7f8af66ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8af0102c20 0x7f8af019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.581+0000 7f8af66ee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0110d00 con 0x7f8af0102c20 2024-08-18T23:51:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.581+0000 7f8af66ee640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0110b80 con 0x7f8af0100b50 2024-08-18T23:51:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.581+0000 7f8af66ee640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0110e80 con 0x7f8af01022a0 2024-08-18T23:51:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.581+0000 7f8aeffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8af0100b50 0x7f8af01970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.581+0000 7f8aeffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8af0100b50 0x7f8af01970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:42718/0 (socket says 172.21.15.78:42718) 2024-08-18T23:51:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.581+0000 7f8aeffff640 1 -- 172.21.15.78:0/864316700 learned_addr learned my addr 172.21.15.78:0/864316700 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.581+0000 7f8af4c8b640 1 --2- 172.21.15.78:0/864316700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8af0102c20 0x7f8af019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.581+0000 7f8ae7fff640 1 --2- 172.21.15.78:0/864316700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8af01022a0 0x7f8af0197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.581+0000 7f8aeffff640 1 -- 172.21.15.78:0/864316700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8af01022a0 msgr2=0x7f8af0197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.581+0000 7f8aeffff640 1 --2- 172.21.15.78:0/864316700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8af01022a0 0x7f8af0197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.581+0000 7f8aeffff640 1 -- 172.21.15.78:0/864316700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8af0102c20 msgr2=0x7f8af019b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:53.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.581+0000 7f8aeffff640 1 --2- 172.21.15.78:0/864316700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8af0102c20 0x7f8af019b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.581+0000 7f8aeffff640 1 -- 172.21.15.78:0/864316700 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af019c0b0 con 0x7f8af0100b50 2024-08-18T23:51:53.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.582+0000 7f8aeffff640 1 --2- 172.21.15.78:0/864316700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8af0100b50 0x7f8af01970e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0009f20 tx=0x7f8ae000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:53.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.582+0000 7f8aedffb640 1 -- 172.21.15.78:0/864316700 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0020070 con 0x7f8af0100b50 2024-08-18T23:51:53.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.582+0000 7f8af66ee640 1 -- 172.21.15.78:0/864316700 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af019c3a0 con 0x7f8af0100b50 2024-08-18T23:51:53.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.582+0000 7f8aedffb640 1 -- 172.21.15.78:0/864316700 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ae000cd80 con 0x7f8af0100b50 2024-08-18T23:51:53.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.582+0000 7f8af66ee640 1 -- 172.21.15.78:0/864316700 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af0069ca0 con 0x7f8af0100b50 2024-08-18T23:51:53.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.582+0000 7f8aedffb640 1 -- 172.21.15.78:0/864316700 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae001bbb0 con 0x7f8af0100b50 2024-08-18T23:51:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.584+0000 7f8aedffb640 1 -- 172.21.15.78:0/864316700 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8ae0023070 con 0x7f8af0100b50 2024-08-18T23:51:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.584+0000 7f8af8aec740 1 Processor -- start 2024-08-18T23:51:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.584+0000 7f8af8aec740 1 -- start start 2024-08-18T23:51:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.584+0000 7f8af8aec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a9a200 0x5560d4a9a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:53.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.584+0000 7f8af8aec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560d4a9ab20 0x5560d4aa4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:53.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8af8aec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4aa53e0 0x5560d4aa77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:53.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8af8aec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5560d4a5de40 con 0x5560d4aa53e0 2024-08-18T23:51:53.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8af8aec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5560d49ff7f0 con 0x5560d4a9a200 2024-08-18T23:51:53.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8af8aec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5560d49a6fb0 con 0x5560d4a9ab20 2024-08-18T23:51:53.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8ae7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4aa53e0 0x5560d4aa77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:53.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8ae7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4aa53e0 0x5560d4aa77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44982/0 (socket says 172.21.15.78:44982) 2024-08-18T23:51:53.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8af4c8b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a9a200 0x5560d4a9a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:53.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8ae7fff640 1 -- 172.21.15.78:0/4163031489 learned_addr learned my addr 172.21.15.78:0/4163031489 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:53.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8aeffff640 1 --2- 172.21.15.78:0/4163031489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560d4a9ab20 0x5560d4aa4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:53.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8ae7fff640 1 -- 172.21.15.78:0/4163031489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560d4a9ab20 msgr2=0x5560d4aa4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:53.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8ae7fff640 1 --2- 172.21.15.78:0/4163031489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560d4a9ab20 0x5560d4aa4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8ae7fff640 1 -- 172.21.15.78:0/4163031489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a9a200 msgr2=0x5560d4a9a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:53.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8ae7fff640 1 --2- 172.21.15.78:0/4163031489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a9a200 0x5560d4a9a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8ae7fff640 1 -- 172.21.15.78:0/4163031489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560d47716a0 con 0x5560d4aa53e0 2024-08-18T23:51:53.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8aedffb640 1 --2- 172.21.15.78:0/864316700 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8adc0a2c00 0x7f8adc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:53.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8ae7fff640 1 --2- 172.21.15.78:0/4163031489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4aa53e0 0x5560d4aa77d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009230 tx=0x7f8ad8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6748cd411c1549fa server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:53.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8af4c8b640 1 --2- 172.21.15.78:0/864316700 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8adc0a2c00 0x7f8adc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:53.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8ae67fc640 1 -- 172.21.15.78:0/4163031489 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8035040 con 0x5560d4aa53e0 2024-08-18T23:51:53.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8aedffb640 1 -- 172.21.15.78:0/864316700 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f8ae00d33b0 con 0x7f8af0100b50 2024-08-18T23:51:53.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8ae67fc640 1 -- 172.21.15.78:0/4163031489 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ad8005520 con 0x5560d4aa53e0 2024-08-18T23:51:53.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.585+0000 7f8ae67fc640 1 -- 172.21.15.78:0/4163031489 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8003850 con 0x5560d4aa53e0 2024-08-18T23:51:53.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.586+0000 7f8af8aec740 1 -- 172.21.15.78:0/4163031489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4aa53e0 msgr2=0x5560d4aa77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:53.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.586+0000 7f8af4c8b640 1 --2- 172.21.15.78:0/864316700 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8adc0a2c00 0x7f8adc0a50c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8005fd0 tx=0x7f8ae80096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:53.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.586+0000 7f8af8aec740 1 --2- 172.21.15.78:0/4163031489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4aa53e0 0x5560d4aa77d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009230 tx=0x7f8ad8004040 comp rx=0 tx=0).stop 2024-08-18T23:51:53.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.586+0000 7f8af8aec740 1 -- 172.21.15.78:0/4163031489 shutdown_connections 2024-08-18T23:51:53.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.586+0000 7f8af8aec740 1 --2- 172.21.15.78:0/4163031489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4aa53e0 0x5560d4aa77d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.586+0000 7f8af8aec740 1 --2- 172.21.15.78:0/4163031489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560d4a9ab20 0x5560d4aa4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.586+0000 7f8af8aec740 1 --2- 172.21.15.78:0/4163031489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a9a200 0x5560d4a9a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.586+0000 7f8af8aec740 1 -- 172.21.15.78:0/4163031489 >> 172.21.15.78:0/4163031489 conn(0x5560d4a96790 msgr2=0x5560d4a96b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:53.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.586+0000 7f8af8aec740 1 -- 172.21.15.78:0/4163031489 shutdown_connections 2024-08-18T23:51:53.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.586+0000 7f8af8aec740 1 -- 172.21.15.78:0/4163031489 wait complete. 2024-08-18T23:51:53.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.587+0000 7f8af8aec740 1 -- start start 2024-08-18T23:51:53.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.587+0000 7f8af8aec740 1 Processor -- start 2024-08-18T23:51:53.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.587+0000 7f8af8aec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9a200 0x5560d4aa2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:53.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.587+0000 7f8af8aec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a9ab20 0x5560d4aa27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:53.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.587+0000 7f8af8aec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560d4aa53e0 0x5560d4aa2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:53.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.587+0000 7f8af8aec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5560d49bacb0 con 0x5560d4a9a200 2024-08-18T23:51:53.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.587+0000 7f8af8aec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5560d49fb7b0 con 0x5560d4a9ab20 2024-08-18T23:51:53.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.587+0000 7f8af8aec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5560d4aa7fb0 con 0x5560d4aa53e0 2024-08-18T23:51:53.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.587+0000 7f8aeffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9a200 0x5560d4aa2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:53.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.587+0000 7f8af8aec740 10 client.0 _ll_register_callbacks cb 0x5560d470eba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:53.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.587+0000 7f8aeffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9a200 0x5560d4aa2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44998/0 (socket says 172.21.15.78:44998) 2024-08-18T23:51:53.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.587+0000 7f8aeffff640 1 -- 172.21.15.78:0/1093222398 learned_addr learned my addr 172.21.15.78:0/1093222398 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:53.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.587+0000 7f8ae7fff640 1 --2- 172.21.15.78:0/1093222398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a9ab20 0x5560d4aa27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:53.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.588+0000 7f8aeffff640 1 -- 172.21.15.78:0/1093222398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560d4aa53e0 msgr2=0x5560d4aa2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:53.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.588+0000 7f8aeffff640 1 --2- 172.21.15.78:0/1093222398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560d4aa53e0 0x5560d4aa2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.588+0000 7f8aeffff640 1 -- 172.21.15.78:0/1093222398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a9ab20 msgr2=0x5560d4aa27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:53.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.588+0000 7f8aeffff640 1 --2- 172.21.15.78:0/1093222398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a9ab20 0x5560d4aa27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.588+0000 7f8aeffff640 1 -- 172.21.15.78:0/1093222398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560d49b9bc0 con 0x5560d4a9a200 2024-08-18T23:51:53.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.588+0000 7f8aeffff640 1 --2- 172.21.15.78:0/1093222398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9a200 0x5560d4aa2270 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000c6a0 tx=0x7f8ae009f550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5371e60f67d21b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:53.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.588+0000 7f8acf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:51:53.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.588+0000 7f8acf7fe640 1 -- 172.21.15.78:0/1093222398 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae00a3c50 con 0x5560d4a9a200 2024-08-18T23:51:53.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.588+0000 7f8acf7fe640 1 -- 172.21.15.78:0/1093222398 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f8ae001b440 con 0x5560d4a9a200 2024-08-18T23:51:53.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.588+0000 7f8af8aec740 1 -- 172.21.15.78:0/1093222398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ad8005520 con 0x5560d4a9a200 2024-08-18T23:51:53.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.588+0000 7f8acf7fe640 1 -- 172.21.15.78:0/1093222398 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ae00a8510 con 0x5560d4a9a200 2024-08-18T23:51:53.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.588+0000 7f8af8aec740 10 client.16692 fetch_fsmap learned FSMap version 30 2024-08-18T23:51:53.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.588+0000 7f8af8aec740 1 -- 172.21.15.78:0/1093222398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5560d4abc700 con 0x5560d4a9a200 2024-08-18T23:51:53.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.588+0000 7f8acf7fe640 1 -- 172.21.15.78:0/1093222398 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ad8005520 con 0x5560d4a9a200 2024-08-18T23:51:53.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.589+0000 7f8acf7fe640 1 -- 172.21.15.78:0/1093222398 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8ae00a8080 con 0x5560d4a9a200 2024-08-18T23:51:53.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.589+0000 7f8af8aec740 10 client.16692 fetch_fsmap finished waiting for FSMap version 30 2024-08-18T23:51:53.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.589+0000 7f8af8aec740 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:51:53.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.589+0000 7f8af8aec740 20 client.16692 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:53.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.589+0000 7f8af8aec740 1 --2- 172.21.15.78:0/1093222398 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5560d4abdbc0 0x5560d4abffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:53.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.589+0000 7f8af8aec740 4 client.16692 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:51:53.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.589+0000 7f8af8aec740 1 -- 172.21.15.78:0/1093222398 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560d4a0cd60 con 0x5560d4abdbc0 2024-08-18T23:51:53.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.589+0000 7f8ae7fff640 1 --2- 172.21.15.78:0/1093222398 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5560d4abdbc0 0x5560d4abffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:53.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.589+0000 7f8acf7fe640 10 client.16692 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:53.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.589+0000 7f8ae7fff640 1 --2- 172.21.15.78:0/1093222398 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5560d4abdbc0 0x5560d4abffb0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7de4c8e6772dd14 server_cookie=1d891cb8b355586d in_seq=0 out_seq=0 2024-08-18T23:51:53.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.590+0000 7f8acf7fe640 1 -- 172.21.15.78:0/1093222398 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560d4a0cd60 con 0x5560d4abdbc0 2024-08-18T23:51:53.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.590+0000 7f8acf7fe640 10 client.16692 handle_command_reply: tid=0 2024-08-18T23:51:53.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.590+0000 7f8af8aec740 1 client.16692 shutdown 2024-08-18T23:51:53.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.590+0000 7f8af8aec740 1 -- 172.21.15.78:0/1093222398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9a200 msgr2=0x5560d4aa2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:53.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.590+0000 7f8af8aec740 1 --2- 172.21.15.78:0/1093222398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9a200 0x5560d4aa2270 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000c6a0 tx=0x7f8ae009f550 comp rx=0 tx=0).stop 2024-08-18T23:51:53.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.591+0000 7f8af8aec740 1 -- 172.21.15.78:0/1093222398 shutdown_connections 2024-08-18T23:51:53.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.591+0000 7f8af8aec740 1 --2- 172.21.15.78:0/1093222398 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5560d4abdbc0 0x5560d4abffb0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.591+0000 7f8af8aec740 1 --2- 172.21.15.78:0/1093222398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560d4aa53e0 0x5560d4aa2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.591+0000 7f8af8aec740 1 --2- 172.21.15.78:0/1093222398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a9ab20 0x5560d4aa27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.591+0000 7f8acf7fe640 0 client.16692 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:53.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.591+0000 7f8af8aec740 1 --2- 172.21.15.78:0/1093222398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9a200 0x5560d4aa2270 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.591+0000 7f8af8aec740 1 -- 172.21.15.78:0/1093222398 >> 172.21.15.78:0/1093222398 conn(0x5560d4a96790 msgr2=0x5560d4a9c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:53.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.591+0000 7f8af8aec740 1 -- 172.21.15.78:0/1093222398 shutdown_connections 2024-08-18T23:51:53.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.591+0000 7f8af8aec740 1 -- 172.21.15.78:0/1093222398 wait complete. 2024-08-18T23:51:53.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.591+0000 7f8af8aec740 20 client.16692 trim_cache size 0 max 16384 2024-08-18T23:51:53.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.609+0000 7f8af8aec740 1 Processor -- start 2024-08-18T23:51:53.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.609+0000 7f8af8aec740 1 -- start start 2024-08-18T23:51:53.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.609+0000 7f8af8aec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560d4a96790 0x5560d4aa11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:53.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.609+0000 7f8af8aec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a9a200 0x5560d4ac5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:53.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.609+0000 7f8af8aec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9ab20 0x5560d4ac5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:53.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.609+0000 7f8af8aec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad00124c0 con 0x5560d4a9ab20 2024-08-18T23:51:53.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.609+0000 7f8af8aec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad000fd30 con 0x5560d4a9a200 2024-08-18T23:51:53.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.609+0000 7f8af8aec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad000db80 con 0x5560d4a96790 2024-08-18T23:51:53.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.609+0000 7f8af4c8b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9ab20 0x5560d4ac5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:53.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.609+0000 7f8af4c8b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9ab20 0x5560d4ac5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45012/0 (socket says 172.21.15.78:45012) 2024-08-18T23:51:53.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.609+0000 7f8af4c8b640 1 -- 172.21.15.78:0/1584810543 learned_addr learned my addr 172.21.15.78:0/1584810543 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:53.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.609+0000 7f8ae7fff640 1 --2- 172.21.15.78:0/1584810543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a9a200 0x5560d4ac5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:53.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.609+0000 7f8aeffff640 1 --2- 172.21.15.78:0/1584810543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560d4a96790 0x5560d4aa11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:53.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.609+0000 7f8af4c8b640 1 -- 172.21.15.78:0/1584810543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560d4a96790 msgr2=0x5560d4aa11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:53.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.609+0000 7f8af4c8b640 1 --2- 172.21.15.78:0/1584810543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560d4a96790 0x5560d4aa11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.609+0000 7f8af4c8b640 1 -- 172.21.15.78:0/1584810543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a9a200 msgr2=0x5560d4ac5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:53.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.609+0000 7f8af4c8b640 1 --2- 172.21.15.78:0/1584810543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a9a200 0x5560d4ac5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.609+0000 7f8af4c8b640 1 -- 172.21.15.78:0/1584810543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560d4ac0320 con 0x5560d4a9ab20 2024-08-18T23:51:53.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.609+0000 7f8af4c8b640 1 --2- 172.21.15.78:0/1584810543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9ab20 0x5560d4ac5d50 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009e30 tx=0x7f8ae8014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3fb20638838974 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:53.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.610+0000 7f8ae5ffb640 1 -- 172.21.15.78:0/1584810543 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8004d20 con 0x5560d4a9ab20 2024-08-18T23:51:53.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.610+0000 7f8ae5ffb640 1 -- 172.21.15.78:0/1584810543 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ae8005500 con 0x5560d4a9ab20 2024-08-18T23:51:53.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.610+0000 7f8ae5ffb640 1 -- 172.21.15.78:0/1584810543 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae801ba10 con 0x5560d4a9ab20 2024-08-18T23:51:53.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.610+0000 7f8af8aec740 1 -- 172.21.15.78:0/1584810543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9ab20 msgr2=0x5560d4ac5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:53.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.610+0000 7f8af8aec740 1 --2- 172.21.15.78:0/1584810543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9ab20 0x5560d4ac5d50 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009e30 tx=0x7f8ae8014040 comp rx=0 tx=0).stop 2024-08-18T23:51:53.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.610+0000 7f8af8aec740 1 -- 172.21.15.78:0/1584810543 shutdown_connections 2024-08-18T23:51:53.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.610+0000 7f8af8aec740 1 --2- 172.21.15.78:0/1584810543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9ab20 0x5560d4ac5d50 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.610+0000 7f8af8aec740 1 --2- 172.21.15.78:0/1584810543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a9a200 0x5560d4ac5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.610+0000 7f8af8aec740 1 --2- 172.21.15.78:0/1584810543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560d4a96790 0x5560d4aa11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.610+0000 7f8af8aec740 1 -- 172.21.15.78:0/1584810543 >> 172.21.15.78:0/1584810543 conn(0x5560d4a9dc60 msgr2=0x5560d4a958f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:53.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.610+0000 7f8af8aec740 1 -- 172.21.15.78:0/1584810543 shutdown_connections 2024-08-18T23:51:53.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.610+0000 7f8af8aec740 1 -- 172.21.15.78:0/1584810543 wait complete. 2024-08-18T23:51:53.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.611+0000 7f8af8aec740 1 -- start start 2024-08-18T23:51:53.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.611+0000 7f8af8aec740 1 Processor -- start 2024-08-18T23:51:53.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.611+0000 7f8af8aec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a96790 0x5560d4ab6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:53.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.611+0000 7f8af8aec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9a200 0x5560d4ab1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:53.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.611+0000 7f8af8aec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560d4a9ab20 0x5560d4ab1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:53.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.611+0000 7f8af8aec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad000e1f0 con 0x5560d4a9a200 2024-08-18T23:51:53.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.611+0000 7f8af8aec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad000f660 con 0x5560d4a96790 2024-08-18T23:51:53.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.611+0000 7f8af8aec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad000e8c0 con 0x5560d4a9ab20 2024-08-18T23:51:53.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.611+0000 7f8af8aec740 10 client.0 _ll_register_callbacks cb 0x7f8ae0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:53.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.611+0000 7f8ae7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9a200 0x5560d4ab1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:53.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.611+0000 7f8ae7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9a200 0x5560d4ab1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45028/0 (socket says 172.21.15.78:45028) 2024-08-18T23:51:53.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.611+0000 7f8ae7fff640 1 -- 172.21.15.78:0/919055639 learned_addr learned my addr 172.21.15.78:0/919055639 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:53.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.611+0000 7f8aeffff640 1 --2- 172.21.15.78:0/919055639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a96790 0x5560d4ab6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:53.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.611+0000 7f8af4c8b640 1 --2- 172.21.15.78:0/919055639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560d4a9ab20 0x5560d4ab1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:53.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.612+0000 7f8ae7fff640 1 -- 172.21.15.78:0/919055639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560d4a9ab20 msgr2=0x5560d4ab1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:53.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.612+0000 7f8ae7fff640 1 --2- 172.21.15.78:0/919055639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560d4a9ab20 0x5560d4ab1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.612+0000 7f8ae7fff640 1 -- 172.21.15.78:0/919055639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a96790 msgr2=0x5560d4ab6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:53.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.612+0000 7f8ae7fff640 1 --2- 172.21.15.78:0/919055639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a96790 0x5560d4ab6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.612+0000 7f8ae7fff640 1 -- 172.21.15.78:0/919055639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560d4ab2520 con 0x5560d4a9a200 2024-08-18T23:51:53.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.612+0000 7f8ae7fff640 1 --2- 172.21.15.78:0/919055639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9a200 0x5560d4ab1700 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8ad802aa70 tx=0x7f8ad8004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80197db712b54ad7 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:53.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.612+0000 7f8acf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:51:53.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.612+0000 7f8acf7fe640 1 -- 172.21.15.78:0/919055639 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8035040 con 0x5560d4a9a200 2024-08-18T23:51:53.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.612+0000 7f8acf7fe640 1 -- 172.21.15.78:0/919055639 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f8ad8004980 con 0x5560d4a9a200 2024-08-18T23:51:53.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.612+0000 7f8af8aec740 1 -- 172.21.15.78:0/919055639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ae8005500 con 0x5560d4a9a200 2024-08-18T23:51:53.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.612+0000 7f8acf7fe640 1 -- 172.21.15.78:0/919055639 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ad8061050 con 0x5560d4a9a200 2024-08-18T23:51:53.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.612+0000 7f8af8aec740 10 client.16698 fetch_fsmap learned FSMap version 30 2024-08-18T23:51:53.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.612+0000 7f8af8aec740 1 -- 172.21.15.78:0/919055639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5560d4b28990 con 0x5560d4a9a200 2024-08-18T23:51:53.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.612+0000 7f8acf7fe640 1 -- 172.21.15.78:0/919055639 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ae8005500 con 0x5560d4a9a200 2024-08-18T23:51:53.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.612+0000 7f8acf7fe640 1 -- 172.21.15.78:0/919055639 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8ad8004fc0 con 0x5560d4a9a200 2024-08-18T23:51:53.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.613+0000 7f8af8aec740 10 client.16698 fetch_fsmap finished waiting for FSMap version 30 2024-08-18T23:51:53.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.613+0000 7f8af8aec740 10 client.16698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:51:53.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.613+0000 7f8af8aec740 20 client.16698 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:53.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.613+0000 7f8af8aec740 1 --2- 172.21.15.78:0/919055639 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5560d4aa53e0 0x5560d4b2ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:53.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.613+0000 7f8af8aec740 4 client.16698 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-18T23:51:53.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.613+0000 7f8af8aec740 1 -- 172.21.15.78:0/919055639 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5560d4a07270 con 0x5560d4aa53e0 2024-08-18T23:51:53.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.613+0000 7f8aeffff640 1 --2- 172.21.15.78:0/919055639 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5560d4aa53e0 0x5560d4b2ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:53.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.613+0000 7f8acf7fe640 10 client.16698 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:53.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.613+0000 7f8aeffff640 1 --2- 172.21.15.78:0/919055639 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5560d4aa53e0 0x5560d4b2ad60 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297ca886c846383b server_cookie=d86700c3c32df31f in_seq=0 out_seq=0 2024-08-18T23:51:53.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.650+0000 7f8acf7fe640 1 -- 172.21.15.78:0/919055639 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5560d4a07270 con 0x5560d4aa53e0 2024-08-18T23:51:53.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.650+0000 7f8acf7fe640 10 client.16698 handle_command_reply: tid=0 2024-08-18T23:51:53.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.650+0000 7f8af8aec740 1 client.16698 shutdown 2024-08-18T23:51:53.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.651+0000 7f8af8aec740 1 -- 172.21.15.78:0/919055639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9a200 msgr2=0x5560d4ab1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:53.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.651+0000 7f8af8aec740 1 --2- 172.21.15.78:0/919055639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9a200 0x5560d4ab1700 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8ad802aa70 tx=0x7f8ad8004ce0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.652+0000 7f8af8aec740 1 -- 172.21.15.78:0/919055639 shutdown_connections 2024-08-18T23:51:53.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.652+0000 7f8af8aec740 1 --2- 172.21.15.78:0/919055639 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5560d4aa53e0 0x5560d4b2ad60 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.652+0000 7f8af8aec740 1 --2- 172.21.15.78:0/919055639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5560d4a9ab20 0x5560d4ab1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.652+0000 7f8af8aec740 1 --2- 172.21.15.78:0/919055639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560d4a9a200 0x5560d4ab1700 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.652+0000 7f8af8aec740 1 --2- 172.21.15.78:0/919055639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560d4a96790 0x5560d4ab6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.652+0000 7f8af8aec740 1 -- 172.21.15.78:0/919055639 >> 172.21.15.78:0/919055639 conn(0x5560d4a9dc60 msgr2=0x5560d4a950d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:53.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.652+0000 7f8acf7fe640 0 client.16698 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:53.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.654+0000 7f8af8aec740 1 -- 172.21.15.78:0/919055639 shutdown_connections 2024-08-18T23:51:53.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.655+0000 7f8af8aec740 1 -- 172.21.15.78:0/919055639 wait complete. 2024-08-18T23:51:53.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.655+0000 7f8af8aec740 20 client.16698 trim_cache size 0 max 16384 2024-08-18T23:51:53.657 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:51:53.657 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-08-18T23:51:53.657 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "43b8eb93-5b02-4ed2-bf19-71187efd877f", 2024-08-18T23:51:53.657 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-08-18T23:51:53.657 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:51:53.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.656+0000 7f8af66ee640 1 -- 172.21.15.78:0/864316700 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8adc0a2c00 msgr2=0x7f8adc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:53.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.656+0000 7f8af66ee640 1 --2- 172.21.15.78:0/864316700 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8adc0a2c00 0x7f8adc0a50c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8005fd0 tx=0x7f8ae80096a0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.656+0000 7f8af66ee640 1 -- 172.21.15.78:0/864316700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8af0100b50 msgr2=0x7f8af01970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:53.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.656+0000 7f8af66ee640 1 --2- 172.21.15.78:0/864316700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8af0100b50 0x7f8af01970e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0009f20 tx=0x7f8ae000ee80 comp rx=0 tx=0).stop 2024-08-18T23:51:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.657+0000 7f8af66ee640 1 -- 172.21.15.78:0/864316700 shutdown_connections 2024-08-18T23:51:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.657+0000 7f8af66ee640 1 --2- 172.21.15.78:0/864316700 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8adc0a2c00 0x7f8adc0a50c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.657+0000 7f8af66ee640 1 --2- 172.21.15.78:0/864316700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8af0102c20 0x7f8af019b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.657+0000 7f8af66ee640 1 --2- 172.21.15.78:0/864316700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8af01022a0 0x7f8af0197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.657+0000 7f8af66ee640 1 --2- 172.21.15.78:0/864316700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8af0100b50 0x7f8af01970e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:53.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.657+0000 7f8af66ee640 1 -- 172.21.15.78:0/864316700 >> 172.21.15.78:0/864316700 conn(0x7f8af00fc500 msgr2=0x7f8af00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:53.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.658+0000 7f8af66ee640 1 -- 172.21.15.78:0/864316700 shutdown_connections 2024-08-18T23:51:53.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:53.658+0000 7f8af66ee640 1 -- 172.21.15.78:0/864316700 wait complete. 2024-08-18T23:51:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:53 smithi161 ceph-mon[22027]: pgmap v577: 129 pgs: 129 active+clean; 508 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 796 op/s 2024-08-18T23:51:53.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:53 smithi123 ceph-mon[21808]: pgmap v577: 129 pgs: 129 active+clean; 508 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 796 op/s 2024-08-18T23:51:54.405 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:51:54.443 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:54 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:51:54.444 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:54 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:54.444 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:54 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:51:54.444 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:54 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:54.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:54 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:51:54.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:54 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:54.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:54 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:51:54.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:54 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:54.710 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:51:54.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:54 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:51:54.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:54 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:54.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:54 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:51:54.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:54 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.198+0000 7fd281ad9640 1 -- 172.21.15.78:0/964587740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd27c1007c0 msgr2=0x7fd27c100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.198+0000 7fd281ad9640 1 --2- 172.21.15.78:0/964587740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd27c1007c0 0x7fd27c100c20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd270009930 tx=0x7fd27002a420 comp rx=0 tx=0).stop 2024-08-18T23:51:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.199+0000 7fd281ad9640 1 -- 172.21.15.78:0/964587740 shutdown_connections 2024-08-18T23:51:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.199+0000 7fd281ad9640 1 --2- 172.21.15.78:0/964587740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd27c101160 0x7fd27c10d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.199+0000 7fd281ad9640 1 --2- 172.21.15.78:0/964587740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd27c1007c0 0x7fd27c100c20 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.199+0000 7fd281ad9640 1 --2- 172.21.15.78:0/964587740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd27c1067c0 0x7fd27c106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.199+0000 7fd281ad9640 1 -- 172.21.15.78:0/964587740 >> 172.21.15.78:0/964587740 conn(0x7fd27c0fc500 msgr2=0x7fd27c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.199+0000 7fd281ad9640 1 -- 172.21.15.78:0/964587740 shutdown_connections 2024-08-18T23:51:55.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.199+0000 7fd281ad9640 1 -- 172.21.15.78:0/964587740 wait complete. 2024-08-18T23:51:55.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.199+0000 7fd281ad9640 1 Processor -- start 2024-08-18T23:51:55.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.200+0000 7fd281ad9640 1 -- start start 2024-08-18T23:51:55.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.200+0000 7fd281ad9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd27c1007c0 0x7fd27c199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:55.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.200+0000 7fd281ad9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd27c101160 0x7fd27c199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:55.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.200+0000 7fd281ad9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd27c1067c0 0x7fd27c1a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:55.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.200+0000 7fd281ad9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c110bc0 con 0x7fd27c1007c0 2024-08-18T23:51:55.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.200+0000 7fd281ad9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c110a40 con 0x7fd27c101160 2024-08-18T23:51:55.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.200+0000 7fd281ad9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c110d40 con 0x7fd27c1067c0 2024-08-18T23:51:55.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.200+0000 7fd27b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd27c1007c0 0x7fd27c199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:55.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.200+0000 7fd27b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd27c1007c0 0x7fd27c199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45046/0 (socket says 172.21.15.78:45046) 2024-08-18T23:51:55.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.200+0000 7fd27affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd27c101160 0x7fd27c199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:55.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.200+0000 7fd27b7fe640 1 -- 172.21.15.78:0/671031989 learned_addr learned my addr 172.21.15.78:0/671031989 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:55.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.200+0000 7fd27bfff640 1 --2- 172.21.15.78:0/671031989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd27c1067c0 0x7fd27c1a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:55.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.200+0000 7fd27b7fe640 1 -- 172.21.15.78:0/671031989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd27c1067c0 msgr2=0x7fd27c1a4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:55.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.200+0000 7fd27b7fe640 1 --2- 172.21.15.78:0/671031989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd27c1067c0 0x7fd27c1a4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.200+0000 7fd27b7fe640 1 -- 172.21.15.78:0/671031989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd27c101160 msgr2=0x7fd27c199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:55.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.200+0000 7fd27b7fe640 1 --2- 172.21.15.78:0/671031989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd27c101160 0x7fd27c199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.200+0000 7fd27b7fe640 1 -- 172.21.15.78:0/671031989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27c1a45c0 con 0x7fd27c1007c0 2024-08-18T23:51:55.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.200+0000 7fd27b7fe640 1 --2- 172.21.15.78:0/671031989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd27c1007c0 0x7fd27c199000 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00d950 tx=0x7fd26c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:55.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.201+0000 7fd278ff9640 1 -- 172.21.15.78:0/671031989 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c017030 con 0x7fd27c1007c0 2024-08-18T23:51:55.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.201+0000 7fd278ff9640 1 -- 172.21.15.78:0/671031989 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd26c004020 con 0x7fd27c1007c0 2024-08-18T23:51:55.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.201+0000 7fd278ff9640 1 -- 172.21.15.78:0/671031989 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c0047a0 con 0x7fd27c1007c0 2024-08-18T23:51:55.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.201+0000 7fd281ad9640 1 -- 172.21.15.78:0/671031989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd27c1a4790 con 0x7fd27c1007c0 2024-08-18T23:51:55.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.201+0000 7fd281ad9640 1 -- 172.21.15.78:0/671031989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd27c1a4c50 con 0x7fd27c1007c0 2024-08-18T23:51:55.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.203+0000 7fd278ff9640 1 -- 172.21.15.78:0/671031989 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd26c0041d0 con 0x7fd27c1007c0 2024-08-18T23:51:55.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.203+0000 7fd283ed7740 1 Processor -- start 2024-08-18T23:51:55.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.203+0000 7fd283ed7740 1 -- start start 2024-08-18T23:51:55.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.204+0000 7fd283ed7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55882903e210 0x55882903e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:55.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.204+0000 7fd283ed7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55882903eb30 0x558829048eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:55.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.204+0000 7fd283ed7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588290493f0 0x55882904b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:55.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.204+0000 7fd27bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55882903e210 0x55882903e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:55.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.204+0000 7fd283ed7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558828fa3980 con 0x55882903e210 2024-08-18T23:51:55.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.204+0000 7fd283ed7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558828f3ba70 con 0x5588290493f0 2024-08-18T23:51:55.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.204+0000 7fd283ed7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558828f9f940 con 0x55882903eb30 2024-08-18T23:51:55.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.204+0000 7fd27bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55882903e210 0x55882903e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45048/0 (socket says 172.21.15.78:45048) 2024-08-18T23:51:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.204+0000 7fd27bfff640 1 -- 172.21.15.78:0/2353230908 learned_addr learned my addr 172.21.15.78:0/2353230908 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.204+0000 7fd27b7fe640 1 --2- 172.21.15.78:0/2353230908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55882903eb30 0x558829048eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.204+0000 7fd278ff9640 1 --2- 172.21.15.78:0/671031989 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2640a2b80 0x7fd2640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.204+0000 7fd27affd640 1 --2- 172.21.15.78:0/2353230908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588290493f0 0x55882904b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.204+0000 7fd27bfff640 1 --2- 172.21.15.78:0/671031989 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2640a2b80 0x7fd2640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.204+0000 7fd278ff9640 1 -- 172.21.15.78:0/671031989 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fd26c0ced00 con 0x7fd27c1007c0 2024-08-18T23:51:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.205+0000 7fd27b7fe640 1 -- 172.21.15.78:0/2353230908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588290493f0 msgr2=0x55882904b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.205+0000 7fd27b7fe640 1 --2- 172.21.15.78:0/2353230908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588290493f0 0x55882904b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.205+0000 7fd27b7fe640 1 -- 172.21.15.78:0/2353230908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55882903e210 msgr2=0x55882903e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.205+0000 7fd27b7fe640 1 --2- 172.21.15.78:0/2353230908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55882903e210 0x55882903e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.205+0000 7fd27b7fe640 1 -- 172.21.15.78:0/2353230908 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558828f9e6e0 con 0x55882903eb30 2024-08-18T23:51:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.205+0000 7fd27affd640 1 --2- 172.21.15.78:0/2353230908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588290493f0 0x55882904b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:51:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.205+0000 7fd27b7fe640 1 --2- 172.21.15.78:0/2353230908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55882903eb30 0x558829048eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd26c09ae70 tx=0x7fd26c004540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c826befbaefc4be server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:55.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.205+0000 7fd27bfff640 1 --2- 172.21.15.78:0/671031989 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2640a2b80 0x7fd2640a5040 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd274005f40 tx=0x7fd274009c20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:55.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.205+0000 7fd261ffb640 1 -- 172.21.15.78:0/2353230908 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c09af40 con 0x55882903eb30 2024-08-18T23:51:55.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.205+0000 7fd261ffb640 1 -- 172.21.15.78:0/2353230908 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd26c09b870 con 0x55882903eb30 2024-08-18T23:51:55.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.205+0000 7fd261ffb640 1 -- 172.21.15.78:0/2353230908 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c09aad0 con 0x55882903eb30 2024-08-18T23:51:55.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.206+0000 7fd283ed7740 1 -- 172.21.15.78:0/2353230908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55882903eb30 msgr2=0x558829048eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:55.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.206+0000 7fd283ed7740 1 --2- 172.21.15.78:0/2353230908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55882903eb30 0x558829048eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd26c09ae70 tx=0x7fd26c004540 comp rx=0 tx=0).stop 2024-08-18T23:51:55.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.206+0000 7fd283ed7740 1 -- 172.21.15.78:0/2353230908 shutdown_connections 2024-08-18T23:51:55.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.206+0000 7fd283ed7740 1 --2- 172.21.15.78:0/2353230908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588290493f0 0x55882904b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.206+0000 7fd283ed7740 1 --2- 172.21.15.78:0/2353230908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55882903eb30 0x558829048eb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.206+0000 7fd283ed7740 1 --2- 172.21.15.78:0/2353230908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55882903e210 0x55882903e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.206+0000 7fd283ed7740 1 -- 172.21.15.78:0/2353230908 >> 172.21.15.78:0/2353230908 conn(0x55882903a7a0 msgr2=0x55882903ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:55.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.206+0000 7fd283ed7740 1 -- 172.21.15.78:0/2353230908 shutdown_connections 2024-08-18T23:51:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.206+0000 7fd283ed7740 1 -- 172.21.15.78:0/2353230908 wait complete. 2024-08-18T23:51:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.206+0000 7fd283ed7740 1 -- start start 2024-08-18T23:51:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd283ed7740 1 Processor -- start 2024-08-18T23:51:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd283ed7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55882903e210 0x558829046420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd283ed7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55882903eb30 0x558829046960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd283ed7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588290493f0 0x558829046ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd283ed7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55882904bfc0 con 0x5588290493f0 2024-08-18T23:51:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd283ed7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558828f5eaa0 con 0x55882903eb30 2024-08-18T23:51:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd283ed7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55882904c140 con 0x55882903e210 2024-08-18T23:51:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd283ed7740 10 client.0 _ll_register_callbacks cb 0x558828f4de60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd27bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588290493f0 0x558829046ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd27affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55882903eb30 0x558829046960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:55.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd27bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588290493f0 0x558829046ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45052/0 (socket says 172.21.15.78:45052) 2024-08-18T23:51:55.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd27affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55882903eb30 0x558829046960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:42774/0 (socket says 172.21.15.78:42774) 2024-08-18T23:51:55.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd27bfff640 1 -- 172.21.15.78:0/1847214074 learned_addr learned my addr 172.21.15.78:0/1847214074 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:55.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd27b7fe640 1 --2- 172.21.15.78:0/1847214074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55882903e210 0x558829046420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:55.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd27bfff640 1 -- 172.21.15.78:0/1847214074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55882903e210 msgr2=0x558829046420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:55.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd27bfff640 1 --2- 172.21.15.78:0/1847214074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55882903e210 0x558829046420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd27bfff640 1 -- 172.21.15.78:0/1847214074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55882903eb30 msgr2=0x558829046960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:55.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd27bfff640 1 --2- 172.21.15.78:0/1847214074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55882903eb30 0x558829046960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd27bfff640 1 -- 172.21.15.78:0/1847214074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558828f42650 con 0x5588290493f0 2024-08-18T23:51:55.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.207+0000 7fd27b7fe640 1 --2- 172.21.15.78:0/1847214074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55882903e210 0x558829046420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:51:55.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.208+0000 7fd27bfff640 1 --2- 172.21.15.78:0/1847214074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588290493f0 0x558829046ea0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd274003bd0 tx=0x7fd274009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633285b7870c4f6 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:55.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.208+0000 7fd25affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:51:55.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.208+0000 7fd25affd640 1 -- 172.21.15.78:0/1847214074 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274020450 con 0x5588290493f0 2024-08-18T23:51:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.208+0000 7fd25affd640 1 -- 172.21.15.78:0/1847214074 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fd2740205f0 con 0x5588290493f0 2024-08-18T23:51:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.208+0000 7fd283ed7740 1 -- 172.21.15.78:0/1847214074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd26c09b870 con 0x5588290493f0 2024-08-18T23:51:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.208+0000 7fd25affd640 1 -- 172.21.15.78:0/1847214074 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd2740207f0 con 0x5588290493f0 2024-08-18T23:51:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.208+0000 7fd283ed7740 10 client.16710 fetch_fsmap learned FSMap version 30 2024-08-18T23:51:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.208+0000 7fd283ed7740 1 -- 172.21.15.78:0/1847214074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558828d153c0 con 0x5588290493f0 2024-08-18T23:51:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.209+0000 7fd25affd640 1 -- 172.21.15.78:0/1847214074 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd26c09b870 con 0x5588290493f0 2024-08-18T23:51:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.209+0000 7fd25affd640 1 -- 172.21.15.78:0/1847214074 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd27400f040 con 0x5588290493f0 2024-08-18T23:51:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.209+0000 7fd283ed7740 10 client.16710 fetch_fsmap finished waiting for FSMap version 30 2024-08-18T23:51:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.209+0000 7fd283ed7740 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:51:55.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.209+0000 7fd283ed7740 20 client.16710 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.209+0000 7fd283ed7740 1 --2- 172.21.15.78:0/1847214074 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558829059050 0x55882905b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.209+0000 7fd283ed7740 4 client.16710 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:51:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.209+0000 7fd283ed7740 1 -- 172.21.15.78:0/1847214074 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558828fb0ef0 con 0x558829059050 2024-08-18T23:51:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.209+0000 7fd27b7fe640 1 --2- 172.21.15.78:0/1847214074 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558829059050 0x55882905b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.209+0000 7fd25affd640 10 client.16710 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.209+0000 7fd27b7fe640 1 --2- 172.21.15.78:0/1847214074 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558829059050 0x55882905b4a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=621310671ae91f9e server_cookie=b7097e6fc474a551 in_seq=0 out_seq=0 2024-08-18T23:51:55.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.224+0000 7fd25affd640 1 -- 172.21.15.78:0/1847214074 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558828fb0ef0 con 0x558829059050 2024-08-18T23:51:55.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.224+0000 7fd25affd640 10 client.16710 handle_command_reply: tid=0 2024-08-18T23:51:55.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.224+0000 7fd283ed7740 1 client.16710 shutdown 2024-08-18T23:51:55.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.224+0000 7fd283ed7740 1 -- 172.21.15.78:0/1847214074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588290493f0 msgr2=0x558829046ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:55.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.224+0000 7fd283ed7740 1 --2- 172.21.15.78:0/1847214074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588290493f0 0x558829046ea0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd274003bd0 tx=0x7fd274009a70 comp rx=0 tx=0).stop 2024-08-18T23:51:55.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.224+0000 7fd283ed7740 1 -- 172.21.15.78:0/1847214074 shutdown_connections 2024-08-18T23:51:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.225+0000 7fd283ed7740 1 --2- 172.21.15.78:0/1847214074 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558829059050 0x55882905b4a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.225+0000 7fd25affd640 0 client.16710 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.225+0000 7fd283ed7740 1 --2- 172.21.15.78:0/1847214074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588290493f0 0x558829046ea0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.225+0000 7fd283ed7740 1 --2- 172.21.15.78:0/1847214074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55882903eb30 0x558829046960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.225+0000 7fd283ed7740 1 --2- 172.21.15.78:0/1847214074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55882903e210 0x558829046420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.225+0000 7fd283ed7740 1 -- 172.21.15.78:0/1847214074 >> 172.21.15.78:0/1847214074 conn(0x55882903a7a0 msgr2=0x5588290404d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.225+0000 7fd283ed7740 1 -- 172.21.15.78:0/1847214074 shutdown_connections 2024-08-18T23:51:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.225+0000 7fd283ed7740 1 -- 172.21.15.78:0/1847214074 wait complete. 2024-08-18T23:51:55.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.225+0000 7fd283ed7740 20 client.16710 trim_cache size 0 max 16384 2024-08-18T23:51:55.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.244+0000 7fd283ed7740 1 Processor -- start 2024-08-18T23:51:55.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.244+0000 7fd283ed7740 1 -- start start 2024-08-18T23:51:55.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.244+0000 7fd283ed7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55882903e210 0x558829060d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:55.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.244+0000 7fd283ed7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55882903eb30 0x5588290612b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:55.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.244+0000 7fd283ed7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588290493f0 0x558829043700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:55.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.244+0000 7fd283ed7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c1b13a0 con 0x55882903eb30 2024-08-18T23:51:55.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.244+0000 7fd283ed7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c1aec10 con 0x55882903e210 2024-08-18T23:51:55.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.244+0000 7fd283ed7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c1aca60 con 0x5588290493f0 2024-08-18T23:51:55.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.244+0000 7fd27affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55882903eb30 0x5588290612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:55.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.244+0000 7fd27affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55882903eb30 0x5588290612b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45062/0 (socket says 172.21.15.78:45062) 2024-08-18T23:51:55.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.244+0000 7fd27affd640 1 -- 172.21.15.78:0/1247883606 learned_addr learned my addr 172.21.15.78:0/1247883606 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:55.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.244+0000 7fd27b7fe640 1 --2- 172.21.15.78:0/1247883606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55882903e210 0x558829060d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:55.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.245+0000 7fd27bfff640 1 --2- 172.21.15.78:0/1247883606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588290493f0 0x558829043700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:55.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.245+0000 7fd27affd640 1 -- 172.21.15.78:0/1247883606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588290493f0 msgr2=0x558829043700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:55.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.245+0000 7fd27affd640 1 --2- 172.21.15.78:0/1247883606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588290493f0 0x558829043700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.245+0000 7fd27affd640 1 -- 172.21.15.78:0/1247883606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55882903e210 msgr2=0x558829060d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:55.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.245+0000 7fd27affd640 1 --2- 172.21.15.78:0/1247883606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55882903e210 0x558829060d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.245+0000 7fd27affd640 1 -- 172.21.15.78:0/1247883606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55882903b430 con 0x55882903eb30 2024-08-18T23:51:55.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.245+0000 7fd27b7fe640 1 --2- 172.21.15.78:0/1247883606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55882903e210 0x558829060d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:51:55.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.245+0000 7fd27bfff640 1 --2- 172.21.15.78:0/1247883606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588290493f0 0x558829043700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:51:55.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.245+0000 7fd27affd640 1 --2- 172.21.15.78:0/1247883606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55882903eb30 0x5588290612b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd270000c00 tx=0x7fd270031880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a980eacff76f0b2f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:55.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.245+0000 7fd2617fa640 1 -- 172.21.15.78:0/1247883606 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd270035040 con 0x55882903eb30 2024-08-18T23:51:55.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.245+0000 7fd2617fa640 1 -- 172.21.15.78:0/1247883606 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd270031970 con 0x55882903eb30 2024-08-18T23:51:55.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.245+0000 7fd2617fa640 1 -- 172.21.15.78:0/1247883606 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd270004a00 con 0x55882903eb30 2024-08-18T23:51:55.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.246+0000 7fd283ed7740 1 -- 172.21.15.78:0/1247883606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55882903eb30 msgr2=0x5588290612b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:55.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.246+0000 7fd283ed7740 1 --2- 172.21.15.78:0/1247883606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55882903eb30 0x5588290612b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd270000c00 tx=0x7fd270031880 comp rx=0 tx=0).stop 2024-08-18T23:51:55.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.246+0000 7fd283ed7740 1 -- 172.21.15.78:0/1247883606 shutdown_connections 2024-08-18T23:51:55.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.246+0000 7fd283ed7740 1 --2- 172.21.15.78:0/1247883606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588290493f0 0x558829043700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.246+0000 7fd283ed7740 1 --2- 172.21.15.78:0/1247883606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55882903eb30 0x5588290612b0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.246+0000 7fd283ed7740 1 --2- 172.21.15.78:0/1247883606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55882903e210 0x558829060d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.246+0000 7fd283ed7740 1 -- 172.21.15.78:0/1247883606 >> 172.21.15.78:0/1247883606 conn(0x55882903a7a0 msgr2=0x558829048390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:55.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.246+0000 7fd283ed7740 1 -- 172.21.15.78:0/1247883606 shutdown_connections 2024-08-18T23:51:55.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.246+0000 7fd283ed7740 1 -- 172.21.15.78:0/1247883606 wait complete. 2024-08-18T23:51:55.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.246+0000 7fd283ed7740 1 -- start start 2024-08-18T23:51:55.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.246+0000 7fd283ed7740 1 Processor -- start 2024-08-18T23:51:55.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.247+0000 7fd283ed7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55882903e210 0x558829051570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:55.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.247+0000 7fd283ed7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55882903eb30 0x558829051ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:55.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.247+0000 7fd283ed7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588290493f0 0x55882904e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:55.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.247+0000 7fd283ed7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c1ad0d0 con 0x5588290493f0 2024-08-18T23:51:55.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.247+0000 7fd283ed7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c1ae540 con 0x55882903e210 2024-08-18T23:51:55.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.247+0000 7fd283ed7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c1ad7a0 con 0x55882903eb30 2024-08-18T23:51:55.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.247+0000 7fd283ed7740 10 client.0 _ll_register_callbacks cb 0x7fd274012ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:51:55.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.247+0000 7fd27bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588290493f0 0x55882904e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:55.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.247+0000 7fd27b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55882903e210 0x558829051570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:55.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.247+0000 7fd27b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55882903e210 0x558829051570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:42786/0 (socket says 172.21.15.78:42786) 2024-08-18T23:51:55.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.247+0000 7fd27b7fe640 1 -- 172.21.15.78:0/1422319782 learned_addr learned my addr 172.21.15.78:0/1422319782 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:51:55.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.247+0000 7fd27affd640 1 --2- 172.21.15.78:0/1422319782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55882903eb30 0x558829051ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:55.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.247+0000 7fd27b7fe640 1 -- 172.21.15.78:0/1422319782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55882903eb30 msgr2=0x558829051ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:55.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.247+0000 7fd27b7fe640 1 --2- 172.21.15.78:0/1422319782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55882903eb30 0x558829051ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.247+0000 7fd27b7fe640 1 -- 172.21.15.78:0/1422319782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588290493f0 msgr2=0x55882904e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:51:55.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.247+0000 7fd27b7fe640 1 --2- 172.21.15.78:0/1422319782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588290493f0 0x55882904e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.247+0000 7fd27b7fe640 1 -- 172.21.15.78:0/1422319782 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55882904ea70 con 0x55882903e210 2024-08-18T23:51:55.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.247+0000 7fd27b7fe640 1 --2- 172.21.15.78:0/1422319782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55882903e210 0x558829051570 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0975d0 tx=0x7fd26c005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2c44d86206cbf4b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:51:55.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.247+0000 7fd25affd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:51:55.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.248+0000 7fd25affd640 1 -- 172.21.15.78:0/1422319782 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c0b4070 con 0x55882903e210 2024-08-18T23:51:55.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.248+0000 7fd25affd640 1 -- 172.21.15.78:0/1422319782 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fd26c0977e0 con 0x55882903e210 2024-08-18T23:51:55.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.248+0000 7fd283ed7740 1 -- 172.21.15.78:0/1422319782 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd270031970 con 0x55882903e210 2024-08-18T23:51:55.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.248+0000 7fd25affd640 1 -- 172.21.15.78:0/1422319782 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd26c09acb0 con 0x55882903e210 2024-08-18T23:51:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.248+0000 7fd283ed7740 10 client.26114 fetch_fsmap learned FSMap version 30 2024-08-18T23:51:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.248+0000 7fd283ed7740 1 -- 172.21.15.78:0/1422319782 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5588290c4760 con 0x55882903e210 2024-08-18T23:51:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.248+0000 7fd25affd640 1 -- 172.21.15.78:0/1422319782 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd270031970 con 0x55882903e210 2024-08-18T23:51:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.248+0000 7fd25affd640 1 -- 172.21.15.78:0/1422319782 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd26c09f170 con 0x55882903e210 2024-08-18T23:51:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.248+0000 7fd283ed7740 10 client.26114 fetch_fsmap finished waiting for FSMap version 30 2024-08-18T23:51:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.248+0000 7fd283ed7740 10 client.26114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:51:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.248+0000 7fd283ed7740 20 client.26114 populate_metadata read hostname 'smithi078' 2024-08-18T23:51:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.248+0000 7fd283ed7740 1 --2- 172.21.15.78:0/1422319782 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558829059050 0x5588290c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:51:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.248+0000 7fd283ed7740 4 client.26114 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:51:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.248+0000 7fd283ed7740 1 -- 172.21.15.78:0/1422319782 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558828fab400 con 0x558829059050 2024-08-18T23:51:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.248+0000 7fd27affd640 1 --2- 172.21.15.78:0/1422319782 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558829059050 0x5588290c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:51:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.249+0000 7fd25affd640 10 client.26114 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.249+0000 7fd27affd640 1 --2- 172.21.15.78:0/1422319782 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558829059050 0x5588290c6aa0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ac544bf8f353126 server_cookie=373be5e89df72aa9 in_seq=0 out_seq=0 2024-08-18T23:51:55.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.251+0000 7fd25affd640 1 -- 172.21.15.78:0/1422319782 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x558828fab400 con 0x558829059050 2024-08-18T23:51:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.251+0000 7fd25affd640 10 client.26114 handle_command_reply: tid=0 2024-08-18T23:51:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.251+0000 7fd283ed7740 1 client.26114 shutdown 2024-08-18T23:51:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.251+0000 7fd283ed7740 1 -- 172.21.15.78:0/1422319782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55882903e210 msgr2=0x558829051570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.251+0000 7fd283ed7740 1 --2- 172.21.15.78:0/1422319782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55882903e210 0x558829051570 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0975d0 tx=0x7fd26c005bf0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.251+0000 7fd283ed7740 1 -- 172.21.15.78:0/1422319782 shutdown_connections 2024-08-18T23:51:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.251+0000 7fd283ed7740 1 --2- 172.21.15.78:0/1422319782 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x558829059050 0x5588290c6aa0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.251+0000 7fd283ed7740 1 --2- 172.21.15.78:0/1422319782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5588290493f0 0x55882904e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.251+0000 7fd25affd640 0 client.26114 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:51:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.251+0000 7fd283ed7740 1 --2- 172.21.15.78:0/1422319782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55882903eb30 0x558829051ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.251+0000 7fd283ed7740 1 --2- 172.21.15.78:0/1422319782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55882903e210 0x558829051570 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.251+0000 7fd283ed7740 1 -- 172.21.15.78:0/1422319782 >> 172.21.15.78:0/1422319782 conn(0x55882903a7a0 msgr2=0x55882903fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.251+0000 7fd283ed7740 1 -- 172.21.15.78:0/1422319782 shutdown_connections 2024-08-18T23:51:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.251+0000 7fd283ed7740 1 -- 172.21.15.78:0/1422319782 wait complete. 2024-08-18T23:51:55.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.251+0000 7fd283ed7740 20 client.26114 trim_cache size 0 max 16384 2024-08-18T23:51:55.257 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:51:55.257 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "scrub active (8659 inodes in the stack)", 2024-08-18T23:51:55.257 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-08-18T23:51:55.257 INFO:teuthology.orchestra.run.smithi078.stdout: "43b8eb93-5b02-4ed2-bf19-71187efd877f": { 2024-08-18T23:51:55.258 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-08-18T23:51:55.258 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "43b8eb93-5b02-4ed2-bf19-71187efd877f", 2024-08-18T23:51:55.258 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-08-18T23:51:55.258 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:51:55.258 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:51:55.258 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:51:55.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.252+0000 7fd281ad9640 1 -- 172.21.15.78:0/671031989 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2640a2b80 msgr2=0x7fd2640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:55.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.252+0000 7fd281ad9640 1 --2- 172.21.15.78:0/671031989 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2640a2b80 0x7fd2640a5040 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd274005f40 tx=0x7fd274009c20 comp rx=0 tx=0).stop 2024-08-18T23:51:55.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.252+0000 7fd281ad9640 1 -- 172.21.15.78:0/671031989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd27c1007c0 msgr2=0x7fd27c199000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:51:55.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.252+0000 7fd281ad9640 1 --2- 172.21.15.78:0/671031989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd27c1007c0 0x7fd27c199000 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00d950 tx=0x7fd26c00de10 comp rx=0 tx=0).stop 2024-08-18T23:51:55.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.252+0000 7fd281ad9640 1 -- 172.21.15.78:0/671031989 shutdown_connections 2024-08-18T23:51:55.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.252+0000 7fd281ad9640 1 --2- 172.21.15.78:0/671031989 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2640a2b80 0x7fd2640a5040 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.252+0000 7fd281ad9640 1 --2- 172.21.15.78:0/671031989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd27c1067c0 0x7fd27c1a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.252+0000 7fd281ad9640 1 --2- 172.21.15.78:0/671031989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd27c101160 0x7fd27c199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.252+0000 7fd281ad9640 1 --2- 172.21.15.78:0/671031989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd27c1007c0 0x7fd27c199000 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:51:55.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.252+0000 7fd281ad9640 1 -- 172.21.15.78:0/671031989 >> 172.21.15.78:0/671031989 conn(0x7fd27c0fc500 msgr2=0x7fd27c0fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:51:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.252+0000 7fd281ad9640 1 -- 172.21.15.78:0/671031989 shutdown_connections 2024-08-18T23:51:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:51:55.252+0000 7fd281ad9640 1 -- 172.21.15.78:0/671031989 wait complete. 2024-08-18T23:51:55.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:55 smithi161 ceph-mon[22027]: scrub queued for path: / 2024-08-18T23:51:55.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:55 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:55.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:55 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:51:55.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:55 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:51:55.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:55 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:55.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:55 smithi161 ceph-mon[22027]: pgmap v578: 129 pgs: 129 active+clean; 485 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.2 MiB/s wr, 732 op/s 2024-08-18T23:51:55.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:55 smithi078 ceph-mon[18440]: scrub queued for path: / 2024-08-18T23:51:55.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:55 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:55.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:55 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:51:55.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:55 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:51:55.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:55 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:55.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:55 smithi078 ceph-mon[18440]: pgmap v578: 129 pgs: 129 active+clean; 485 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.2 MiB/s wr, 732 op/s 2024-08-18T23:51:55.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:55 smithi123 ceph-mon[21808]: scrub queued for path: / 2024-08-18T23:51:55.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:55 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:55.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:55 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:51:55.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:55 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:51:55.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:55 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:51:55.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:55 smithi123 ceph-mon[21808]: pgmap v578: 129 pgs: 129 active+clean; 485 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.2 MiB/s wr, 732 op/s 2024-08-18T23:51:56.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:56 smithi123 ceph-mon[21808]: pgmap v579: 129 pgs: 129 active+clean; 471 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 844 op/s 2024-08-18T23:51:56.814 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:56 smithi078 ceph-mon[18440]: pgmap v579: 129 pgs: 129 active+clean; 471 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 844 op/s 2024-08-18T23:51:56.864 INFO:tasks.cephfs.filesystem:scrub status for tag:43b8eb93-5b02-4ed2-bf19-71187efd877f - {'path': '/', 'tag': '43b8eb93-5b02-4ed2-bf19-71187efd877f', 'options': 'recursive,force'} 2024-08-18T23:51:56.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:56 smithi161 ceph-mon[22027]: pgmap v579: 129 pgs: 129 active+clean; 471 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 844 op/s 2024-08-18T23:51:59.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:59 smithi123 ceph-mon[21808]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:51:59.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:51:59 smithi123 ceph-mon[21808]: pgmap v580: 129 pgs: 129 active+clean; 468 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 834 op/s 2024-08-18T23:51:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:59 smithi161 ceph-mon[22027]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:51:59.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:51:59 smithi161 ceph-mon[22027]: pgmap v580: 129 pgs: 129 active+clean; 468 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 834 op/s 2024-08-18T23:51:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:59 smithi078 ceph-mon[18440]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:51:59.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:51:59 smithi078 ceph-mon[18440]: pgmap v580: 129 pgs: 129 active+clean; 468 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 834 op/s 2024-08-18T23:52:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:01 smithi161 ceph-mon[22027]: pgmap v581: 129 pgs: 129 active+clean; 459 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 925 op/s 2024-08-18T23:52:01.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:01 smithi078 ceph-mon[18440]: pgmap v581: 129 pgs: 129 active+clean; 459 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 925 op/s 2024-08-18T23:52:01.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:01 smithi123 ceph-mon[21808]: pgmap v581: 129 pgs: 129 active+clean; 459 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 925 op/s 2024-08-18T23:52:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:03 smithi161 ceph-mon[22027]: pgmap v582: 129 pgs: 129 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 971 op/s 2024-08-18T23:52:03.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:03 smithi078 ceph-mon[18440]: pgmap v582: 129 pgs: 129 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 971 op/s 2024-08-18T23:52:03.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:03 smithi123 ceph-mon[21808]: pgmap v582: 129 pgs: 129 active+clean; 449 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 971 op/s 2024-08-18T23:52:04.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:04 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:52:04.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:04 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:52:04.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:04 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:52:05.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:05 smithi161 ceph-mon[22027]: pgmap v583: 129 pgs: 129 active+clean; 442 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 997 op/s 2024-08-18T23:52:05.698 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:05 smithi123 ceph-mon[21808]: pgmap v583: 129 pgs: 129 active+clean; 442 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 997 op/s 2024-08-18T23:52:05.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:05 smithi078 ceph-mon[18440]: pgmap v583: 129 pgs: 129 active+clean; 442 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 997 op/s 2024-08-18T23:52:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:07 smithi161 ceph-mon[22027]: pgmap v584: 129 pgs: 129 active+clean; 434 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 1.18k op/s 2024-08-18T23:52:07.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:07 smithi078 ceph-mon[18440]: pgmap v584: 129 pgs: 129 active+clean; 434 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 1.18k op/s 2024-08-18T23:52:07.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:07 smithi123 ceph-mon[21808]: pgmap v584: 129 pgs: 129 active+clean; 434 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 1.18k op/s 2024-08-18T23:52:08.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:08 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:52:08.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:08 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:52:08.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:08 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:52:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:09 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:52:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:09 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:52:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:09 smithi161 ceph-mon[22027]: pgmap v585: 129 pgs: 129 active+clean; 423 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 1.09k op/s 2024-08-18T23:52:09.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:09 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:52:09.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:09 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:52:09.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:09 smithi078 ceph-mon[18440]: pgmap v585: 129 pgs: 129 active+clean; 423 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 1.09k op/s 2024-08-18T23:52:09.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:09 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:52:09.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:09 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:52:09.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:09 smithi123 ceph-mon[21808]: pgmap v585: 129 pgs: 129 active+clean; 423 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 1.09k op/s 2024-08-18T23:52:11.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:11 smithi161 ceph-mon[22027]: pgmap v586: 129 pgs: 129 active+clean; 410 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 1.07k op/s 2024-08-18T23:52:11.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:11 smithi078 ceph-mon[18440]: pgmap v586: 129 pgs: 129 active+clean; 410 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 1.07k op/s 2024-08-18T23:52:11.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:11 smithi123 ceph-mon[21808]: pgmap v586: 129 pgs: 129 active+clean; 410 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 1.07k op/s 2024-08-18T23:52:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:13 smithi161 ceph-mon[22027]: pgmap v587: 129 pgs: 129 active+clean; 384 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 968 op/s 2024-08-18T23:52:13.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:13 smithi078 ceph-mon[18440]: pgmap v587: 129 pgs: 129 active+clean; 384 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 968 op/s 2024-08-18T23:52:13.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:13 smithi123 ceph-mon[21808]: pgmap v587: 129 pgs: 129 active+clean; 384 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 968 op/s 2024-08-18T23:52:15.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:15 smithi161 ceph-mon[22027]: pgmap v588: 129 pgs: 129 active+clean; 364 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 837 op/s 2024-08-18T23:52:15.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:15 smithi078 ceph-mon[18440]: pgmap v588: 129 pgs: 129 active+clean; 364 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 837 op/s 2024-08-18T23:52:15.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:15 smithi123 ceph-mon[21808]: pgmap v588: 129 pgs: 129 active+clean; 364 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 837 op/s 2024-08-18T23:52:17.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:17 smithi161 ceph-mon[22027]: pgmap v589: 129 pgs: 129 active+clean; 347 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 806 op/s 2024-08-18T23:52:17.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:17 smithi078 ceph-mon[18440]: pgmap v589: 129 pgs: 129 active+clean; 347 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 806 op/s 2024-08-18T23:52:17.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:17 smithi123 ceph-mon[21808]: pgmap v589: 129 pgs: 129 active+clean; 347 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 806 op/s 2024-08-18T23:52:19.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:19 smithi161 ceph-mon[22027]: pgmap v590: 129 pgs: 129 active+clean; 341 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1006 KiB/s wr, 625 op/s 2024-08-18T23:52:19.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:19 smithi078 ceph-mon[18440]: pgmap v590: 129 pgs: 129 active+clean; 341 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1006 KiB/s wr, 625 op/s 2024-08-18T23:52:19.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:19 smithi123 ceph-mon[21808]: pgmap v590: 129 pgs: 129 active+clean; 341 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1006 KiB/s wr, 625 op/s 2024-08-18T23:52:21.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:21 smithi161 ceph-mon[22027]: pgmap v591: 129 pgs: 129 active+clean; 335 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 614 op/s 2024-08-18T23:52:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:21 smithi078 ceph-mon[18440]: pgmap v591: 129 pgs: 129 active+clean; 335 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 614 op/s 2024-08-18T23:52:21.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:21 smithi123 ceph-mon[21808]: pgmap v591: 129 pgs: 129 active+clean; 335 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 614 op/s 2024-08-18T23:52:23.650 INFO:tasks.quiescer.fs.[cephfs]:Total duration (47.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-18T23:52:23.650 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.75 2024-08-18T23:52:23.651 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:52:23.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:23 smithi161 ceph-mon[22027]: pgmap v592: 129 pgs: 129 active+clean; 319 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 584 op/s 2024-08-18T23:52:23.689 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:23 smithi078 ceph-mon[18440]: pgmap v592: 129 pgs: 129 active+clean; 319 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 584 op/s 2024-08-18T23:52:23.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:23 smithi123 ceph-mon[21808]: pgmap v592: 129 pgs: 129 active+clean; 319 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 584 op/s 2024-08-18T23:52:24.086 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.623+0000 7f247f821640 1 -- 172.21.15.78:0/1964812423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f247811ae70 msgr2=0x7f247811d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.623+0000 7f247f821640 1 --2- 172.21.15.78:0/1964812423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f247811ae70 0x7f247811d260 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2470009ca0 tx=0x7f247002a8f0 comp rx=0 tx=0).stop 2024-08-18T23:52:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.624+0000 7f247f821640 1 -- 172.21.15.78:0/1964812423 shutdown_connections 2024-08-18T23:52:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.624+0000 7f247f821640 1 --2- 172.21.15.78:0/1964812423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f247811ae70 0x7f247811d260 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.624+0000 7f247f821640 1 --2- 172.21.15.78:0/1964812423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2478109560 0x7f24781099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.624+0000 7f247f821640 1 --2- 172.21.15.78:0/1964812423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2478109180 0x7f247810ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:24.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.624+0000 7f247f821640 1 -- 172.21.15.78:0/1964812423 >> 172.21.15.78:0/1964812423 conn(0x7f247806de70 msgr2=0x7f247806e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:24.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.624+0000 7f247f821640 1 -- 172.21.15.78:0/1964812423 shutdown_connections 2024-08-18T23:52:24.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.625+0000 7f247f821640 1 -- 172.21.15.78:0/1964812423 wait complete. 2024-08-18T23:52:24.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.625+0000 7f247f821640 1 Processor -- start 2024-08-18T23:52:24.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.625+0000 7f247f821640 1 -- start start 2024-08-18T23:52:24.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.626+0000 7f247f821640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2478109180 0x7f24781a6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:24.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.626+0000 7f247f821640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2478109560 0x7f24781a74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:24.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.626+0000 7f247f821640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f247811ae70 0x7f24781ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:24.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.626+0000 7f247f821640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2478120b20 con 0x7f2478109560 2024-08-18T23:52:24.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.626+0000 7f247f821640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24781209a0 con 0x7f2478109180 2024-08-18T23:52:24.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.626+0000 7f247f821640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2478120ca0 con 0x7f247811ae70 2024-08-18T23:52:24.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.626+0000 7f247ddbe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f247811ae70 0x7f24781ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:24.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.626+0000 7f247ddbe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f247811ae70 0x7f24781ab870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:37000/0 (socket says 172.21.15.78:37000) 2024-08-18T23:52:24.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.626+0000 7f247ddbe640 1 -- 172.21.15.78:0/3006663906 learned_addr learned my addr 172.21.15.78:0/3006663906 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:24.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.626+0000 7f247cdbc640 1 --2- 172.21.15.78:0/3006663906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2478109560 0x7f24781a74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:24.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.627+0000 7f247cdbc640 1 -- 172.21.15.78:0/3006663906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f247811ae70 msgr2=0x7f24781ab870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:24.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.627+0000 7f247cdbc640 1 --2- 172.21.15.78:0/3006663906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f247811ae70 0x7f24781ab870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:24.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.627+0000 7f247cdbc640 1 -- 172.21.15.78:0/3006663906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2478109180 msgr2=0x7f24781a6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:24.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.627+0000 7f247cdbc640 1 --2- 172.21.15.78:0/3006663906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2478109180 0x7f24781a6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:24.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.627+0000 7f247cdbc640 1 -- 172.21.15.78:0/3006663906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24781abf50 con 0x7f2478109560 2024-08-18T23:52:24.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.627+0000 7f247ddbe640 1 --2- 172.21.15.78:0/3006663906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f247811ae70 0x7f24781ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:52:24.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.628+0000 7f247cdbc640 1 --2- 172.21.15.78:0/3006663906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2478109560 0x7f24781a74e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f246800c970 tx=0x7f246800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:24.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.631+0000 7f246e7fc640 1 -- 172.21.15.78:0/3006663906 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2468007bb0 con 0x7f2478109560 2024-08-18T23:52:24.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.631+0000 7f247f821640 1 -- 172.21.15.78:0/3006663906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24781ac240 con 0x7f2478109560 2024-08-18T23:52:24.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.631+0000 7f247f821640 1 -- 172.21.15.78:0/3006663906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24781b3b20 con 0x7f2478109560 2024-08-18T23:52:24.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.631+0000 7f246e7fc640 1 -- 172.21.15.78:0/3006663906 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2468007d50 con 0x7f2478109560 2024-08-18T23:52:24.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.631+0000 7f246e7fc640 1 -- 172.21.15.78:0/3006663906 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246800f450 con 0x7f2478109560 2024-08-18T23:52:24.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.636+0000 7f247f821640 1 -- 172.21.15.78:0/3006663906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245c005290 con 0x7f2478109560 2024-08-18T23:52:24.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.641+0000 7f246e7fc640 1 -- 172.21.15.78:0/3006663906 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f24680046c0 con 0x7f2478109560 2024-08-18T23:52:24.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.642+0000 7f246e7fc640 1 --2- 172.21.15.78:0/3006663906 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f24640a2d50 0x7f24640a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:24.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.643+0000 7f247d5bd640 1 --2- 172.21.15.78:0/3006663906 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f24640a2d50 0x7f24640a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:24.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.643+0000 7f246e7fc640 1 -- 172.21.15.78:0/3006663906 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f24680ceda0 con 0x7f2478109560 2024-08-18T23:52:24.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.644+0000 7f247d5bd640 1 --2- 172.21.15.78:0/3006663906 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f24640a2d50 0x7f24640a5210 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f247400a8f0 tx=0x7f2474005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:24.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.648+0000 7f246e7fc640 1 -- 172.21.15.78:0/3006663906 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2468014030 con 0x7f2478109560 2024-08-18T23:52:24.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.967+0000 7f247f821640 1 -- 172.21.15.78:0/3006663906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f245c005580 con 0x7f2478109560 2024-08-18T23:52:24.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.968+0000 7f246e7fc640 1 -- 172.21.15.78:0/3006663906 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11933 (secure 0 0 0) 0x7f246809b910 con 0x7f2478109560 2024-08-18T23:52:24.969 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:52:24.970 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":31,"btime":"2024-08-18T23:51:58:009010+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:51:57.958854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24428,24427]},"id":1}]} 2024-08-18T23:52:24.970 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 31 2024-08-18T23:52:24.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.970+0000 7f247f821640 1 -- 172.21.15.78:0/3006663906 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f24640a2d50 msgr2=0x7f24640a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:24.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.970+0000 7f247f821640 1 --2- 172.21.15.78:0/3006663906 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f24640a2d50 0x7f24640a5210 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f247400a8f0 tx=0x7f2474005bf0 comp rx=0 tx=0).stop 2024-08-18T23:52:24.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.970+0000 7f247f821640 1 -- 172.21.15.78:0/3006663906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2478109560 msgr2=0x7f24781a74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:24.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.970+0000 7f247f821640 1 --2- 172.21.15.78:0/3006663906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2478109560 0x7f24781a74e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f246800c970 tx=0x7f246800ce30 comp rx=0 tx=0).stop 2024-08-18T23:52:24.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.970+0000 7f247f821640 1 -- 172.21.15.78:0/3006663906 shutdown_connections 2024-08-18T23:52:24.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.970+0000 7f247f821640 1 --2- 172.21.15.78:0/3006663906 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f24640a2d50 0x7f24640a5210 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:24.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.970+0000 7f247f821640 1 --2- 172.21.15.78:0/3006663906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f247811ae70 0x7f24781ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:24.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.970+0000 7f247f821640 1 --2- 172.21.15.78:0/3006663906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2478109560 0x7f24781a74e0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:24.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.970+0000 7f247f821640 1 --2- 172.21.15.78:0/3006663906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2478109180 0x7f24781a6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:24.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.970+0000 7f247f821640 1 -- 172.21.15.78:0/3006663906 >> 172.21.15.78:0/3006663906 conn(0x7f247806de70 msgr2=0x7f247811ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:24.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.970+0000 7f247f821640 1 -- 172.21.15.78:0/3006663906 shutdown_connections 2024-08-18T23:52:24.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:24.970+0000 7f247f821640 1 -- 172.21.15.78:0/3006663906 wait complete. 2024-08-18T23:52:25.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:25 smithi161 ceph-mon[22027]: pgmap v593: 129 pgs: 129 active+clean; 314 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 571 op/s 2024-08-18T23:52:25.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:25 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3006663906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:52:25.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:25 smithi078 ceph-mon[18440]: pgmap v593: 129 pgs: 129 active+clean; 314 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 571 op/s 2024-08-18T23:52:25.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:25 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3006663906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:52:25.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:25 smithi123 ceph-mon[21808]: pgmap v593: 129 pgs: 129 active+clean; 314 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 571 op/s 2024-08-18T23:52:25.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:25 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3006663906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:52:26.315 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 143.75 --await-for 45.0' 2024-08-18T23:52:26.315 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 143.75 --await-for 45.0 2024-08-18T23:52:26.639 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:26.865 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:52:27.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.163+0000 7f2fb3061640 1 -- 172.21.15.78:0/2318482641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac1078c0 msgr2=0x7f2fac107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:27.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.163+0000 7f2fb3061640 1 --2- 172.21.15.78:0/2318482641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac1078c0 0x7f2fac107ca0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009930 tx=0x7f2fa002a420 comp rx=0 tx=0).stop 2024-08-18T23:52:27.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.163+0000 7f2fb3061640 1 -- 172.21.15.78:0/2318482641 shutdown_connections 2024-08-18T23:52:27.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.163+0000 7f2fb3061640 1 --2- 172.21.15.78:0/2318482641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fac102260 0x7f2fac10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.163+0000 7f2fb3061640 1 --2- 172.21.15.78:0/2318482641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fac1018c0 0x7f2fac101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.163+0000 7f2fb3061640 1 --2- 172.21.15.78:0/2318482641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac1078c0 0x7f2fac107ca0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.163+0000 7f2fb3061640 1 -- 172.21.15.78:0/2318482641 >> 172.21.15.78:0/2318482641 conn(0x7f2fac0fd620 msgr2=0x7f2fac0ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:27.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.163+0000 7f2fb3061640 1 -- 172.21.15.78:0/2318482641 shutdown_connections 2024-08-18T23:52:27.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.163+0000 7f2fb3061640 1 -- 172.21.15.78:0/2318482641 wait complete. 2024-08-18T23:52:27.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.164+0000 7f2fb3061640 1 Processor -- start 2024-08-18T23:52:27.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.164+0000 7f2fb3061640 1 -- start start 2024-08-18T23:52:27.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.164+0000 7f2fb3061640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac1018c0 0x7f2fac1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:27.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.164+0000 7f2fb3061640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fac102260 0x7f2fac198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:27.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.164+0000 7f2fb3061640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fac1078c0 0x7f2fac19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:27.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.164+0000 7f2fb3061640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac111cc0 con 0x7f2fac1018c0 2024-08-18T23:52:27.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.164+0000 7f2fb3061640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac111b40 con 0x7f2fac1078c0 2024-08-18T23:52:27.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.164+0000 7f2fb3061640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac111e40 con 0x7f2fac102260 2024-08-18T23:52:27.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.164+0000 7f2fb0dfd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac1018c0 0x7f2fac1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:27.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.164+0000 7f2fb0dfd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac1018c0 0x7f2fac1981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59076/0 (socket says 172.21.15.78:59076) 2024-08-18T23:52:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.164+0000 7f2fb0dfd640 1 -- 172.21.15.78:0/3628346692 learned_addr learned my addr 172.21.15.78:0/3628346692 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.164+0000 7f2fb15fe640 1 --2- 172.21.15.78:0/3628346692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fac1078c0 0x7f2fac19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.165+0000 7f2fabfff640 1 --2- 172.21.15.78:0/3628346692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fac102260 0x7f2fac198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.165+0000 7f2fb15fe640 1 -- 172.21.15.78:0/3628346692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fac102260 msgr2=0x7f2fac198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.165+0000 7f2fb15fe640 1 --2- 172.21.15.78:0/3628346692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fac102260 0x7f2fac198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.165+0000 7f2fb15fe640 1 -- 172.21.15.78:0/3628346692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac1018c0 msgr2=0x7f2fac1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:27.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.165+0000 7f2fb15fe640 1 --2- 172.21.15.78:0/3628346692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac1018c0 0x7f2fac1981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.165+0000 7f2fb15fe640 1 -- 172.21.15.78:0/3628346692 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fac19d170 con 0x7f2fac1078c0 2024-08-18T23:52:27.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.165+0000 7f2fb0dfd640 1 --2- 172.21.15.78:0/3628346692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac1018c0 0x7f2fac1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:52:27.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.165+0000 7f2fb15fe640 1 --2- 172.21.15.78:0/3628346692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fac1078c0 0x7f2fac19ca90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400ca30 tx=0x7f2fa400cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:27.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.165+0000 7f2fa9ffb640 1 -- 172.21.15.78:0/3628346692 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4007d90 con 0x7f2fac1078c0 2024-08-18T23:52:27.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.165+0000 7f2fb3061640 1 -- 172.21.15.78:0/3628346692 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fac19d460 con 0x7f2fac1078c0 2024-08-18T23:52:27.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.165+0000 7f2fa9ffb640 1 -- 172.21.15.78:0/3628346692 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa4005020 con 0x7f2fac1078c0 2024-08-18T23:52:27.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.165+0000 7f2fa9ffb640 1 -- 172.21.15.78:0/3628346692 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa400f780 con 0x7f2fac1078c0 2024-08-18T23:52:27.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.165+0000 7f2fb3061640 1 -- 172.21.15.78:0/3628346692 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fac1a4d40 con 0x7f2fac1078c0 2024-08-18T23:52:27.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.167+0000 7f2fa9ffb640 1 -- 172.21.15.78:0/3628346692 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2fa40051d0 con 0x7f2fac1078c0 2024-08-18T23:52:27.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fb545f740 1 Processor -- start 2024-08-18T23:52:27.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fb545f740 1 -- start start 2024-08-18T23:52:27.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fb545f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c22c0 0x55b56a6c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:27.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fb545f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c2be0 0x55b56a6ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:27.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fb545f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6cd4a0 0x55b56a6cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:27.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fb545f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b56a627810 con 0x55b56a6c2be0 2024-08-18T23:52:27.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fb545f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b56a5cec70 con 0x55b56a6cd4a0 2024-08-18T23:52:27.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fb545f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b56a6237d0 con 0x55b56a6c22c0 2024-08-18T23:52:27.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fb0dfd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c2be0 0x55b56a6ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:27.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fb0dfd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c2be0 0x55b56a6ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59086/0 (socket says 172.21.15.78:59086) 2024-08-18T23:52:27.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fb0dfd640 1 -- 172.21.15.78:0/3245603721 learned_addr learned my addr 172.21.15.78:0/3245603721 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:27.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fa9ffb640 1 --2- 172.21.15.78:0/3628346692 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2f9c0a2b80 0x7f2f9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fabfff640 1 --2- 172.21.15.78:0/3245603721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6cd4a0 0x55b56a6cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fabfff640 1 --2- 172.21.15.78:0/3245603721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c22c0 0x55b56a6c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fb15fe640 1 --2- 172.21.15.78:0/3628346692 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2f9c0a2b80 0x7f2f9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fb0dfd640 1 -- 172.21.15.78:0/3245603721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c22c0 msgr2=0x55b56a6c26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fb0dfd640 1 --2- 172.21.15.78:0/3245603721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c22c0 0x55b56a6c26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fb0dfd640 1 -- 172.21.15.78:0/3245603721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6cd4a0 msgr2=0x55b56a6cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fb0dfd640 1 --2- 172.21.15.78:0/3245603721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6cd4a0 0x55b56a6cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fa9ffb640 1 -- 172.21.15.78:0/3628346692 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2fa40cf640 con 0x7f2fac1078c0 2024-08-18T23:52:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fb0dfd640 1 -- 172.21.15.78:0/3245603721 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b56a399ba0 con 0x55b56a6c2be0 2024-08-18T23:52:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.168+0000 7f2fabfff640 1 --2- 172.21.15.78:0/3245603721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c22c0 0x55b56a6c26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.169+0000 7f2fb0dfd640 1 --2- 172.21.15.78:0/3245603721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c2be0 0x55b56a6ccf60 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0006010 tx=0x7f2fa0031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d2a6f2085467a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.169+0000 7f2fb15fe640 1 --2- 172.21.15.78:0/3628346692 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2f9c0a2b80 0x7f2f9c0a5040 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40ccd50 tx=0x7f2fa4086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.169+0000 7f2f96ffd640 1 -- 172.21.15.78:0/3245603721 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0035040 con 0x55b56a6c2be0 2024-08-18T23:52:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.169+0000 7f2f96ffd640 1 -- 172.21.15.78:0/3245603721 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa0031a00 con 0x55b56a6c2be0 2024-08-18T23:52:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.169+0000 7f2f96ffd640 1 -- 172.21.15.78:0/3245603721 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0004ab0 con 0x55b56a6c2be0 2024-08-18T23:52:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.169+0000 7f2fb545f740 1 -- 172.21.15.78:0/3245603721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c2be0 msgr2=0x55b56a6ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.169+0000 7f2fb545f740 1 --2- 172.21.15.78:0/3245603721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c2be0 0x55b56a6ccf60 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0006010 tx=0x7f2fa0031660 comp rx=0 tx=0).stop 2024-08-18T23:52:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.170+0000 7f2fb545f740 1 -- 172.21.15.78:0/3245603721 shutdown_connections 2024-08-18T23:52:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.170+0000 7f2fb545f740 1 --2- 172.21.15.78:0/3245603721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6cd4a0 0x55b56a6cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.170+0000 7f2fb545f740 1 --2- 172.21.15.78:0/3245603721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c2be0 0x55b56a6ccf60 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.170+0000 7f2fb545f740 1 --2- 172.21.15.78:0/3245603721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c22c0 0x55b56a6c26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.170+0000 7f2fb545f740 1 -- 172.21.15.78:0/3245603721 >> 172.21.15.78:0/3245603721 conn(0x55b56a6be850 msgr2=0x55b56a6bec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.170+0000 7f2fb545f740 1 -- 172.21.15.78:0/3245603721 shutdown_connections 2024-08-18T23:52:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.170+0000 7f2fb545f740 1 -- 172.21.15.78:0/3245603721 wait complete. 2024-08-18T23:52:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.170+0000 7f2fb545f740 1 -- start start 2024-08-18T23:52:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.170+0000 7f2fb545f740 1 Processor -- start 2024-08-18T23:52:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.170+0000 7f2fb545f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c22c0 0x55b56a6ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.170+0000 7f2fb545f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c2be0 0x55b56a6caae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.170+0000 7f2fb545f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6cd4a0 0x55b56a6cb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.170+0000 7f2fb545f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b56a6cffb0 con 0x55b56a6c2be0 2024-08-18T23:52:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.170+0000 7f2fb545f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b56a5e29a0 con 0x55b56a6cd4a0 2024-08-18T23:52:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.170+0000 7f2fb545f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b56a6d0130 con 0x55b56a6c22c0 2024-08-18T23:52:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.170+0000 7f2fb545f740 10 client.0 _ll_register_callbacks cb 0x55b56a5d0c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:27.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6cd4a0 0x55b56a6cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:27.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2fb0dfd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c2be0 0x55b56a6caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:27.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6cd4a0 0x55b56a6cb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:54278/0 (socket says 172.21.15.78:54278) 2024-08-18T23:52:27.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2fabfff640 1 -- 172.21.15.78:0/1390605260 learned_addr learned my addr 172.21.15.78:0/1390605260 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:27.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2fb0dfd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c2be0 0x55b56a6caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59096/0 (socket says 172.21.15.78:59096) 2024-08-18T23:52:27.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2fabfff640 1 --2- 172.21.15.78:0/1390605260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c22c0 0x55b56a6ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2fb0dfd640 1 -- 172.21.15.78:0/1390605260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c22c0 msgr2=0x55b56a6ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2fb0dfd640 1 --2- 172.21.15.78:0/1390605260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c22c0 0x55b56a6ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2fb0dfd640 1 -- 172.21.15.78:0/1390605260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6cd4a0 msgr2=0x55b56a6cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2fb0dfd640 1 --2- 172.21.15.78:0/1390605260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6cd4a0 0x55b56a6cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2fb0dfd640 1 -- 172.21.15.78:0/1390605260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b56a5ee220 con 0x55b56a6c2be0 2024-08-18T23:52:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2fabfff640 1 --2- 172.21.15.78:0/1390605260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6cd4a0 0x55b56a6cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:52:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2fb0dfd640 1 --2- 172.21.15.78:0/1390605260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c2be0 0x55b56a6caae0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009a50 tx=0x7f2fa0031470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62efd42cd2e7c817 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2f8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2f8bfff640 1 -- 172.21.15.78:0/1390605260 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa004e020 con 0x55b56a6c2be0 2024-08-18T23:52:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2f8bfff640 1 -- 172.21.15.78:0/1390605260 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2fa00035f0 con 0x55b56a6c2be0 2024-08-18T23:52:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2fb545f740 1 -- 172.21.15.78:0/1390605260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fa0031a00 con 0x55b56a6c2be0 2024-08-18T23:52:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2f8bfff640 1 -- 172.21.15.78:0/1390605260 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2fa0059050 con 0x55b56a6c2be0 2024-08-18T23:52:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2fb545f740 10 client.16734 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.171+0000 7f2fb545f740 1 -- 172.21.15.78:0/1390605260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b56a6dbd70 con 0x55b56a6c2be0 2024-08-18T23:52:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.172+0000 7f2f8bfff640 1 -- 172.21.15.78:0/1390605260 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa0031a00 con 0x55b56a6c2be0 2024-08-18T23:52:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.172+0000 7f2f8bfff640 1 -- 172.21.15.78:0/1390605260 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f2fa00467f0 con 0x55b56a6c2be0 2024-08-18T23:52:27.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.172+0000 7f2fb545f740 10 client.16734 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.172+0000 7f2fb545f740 10 client.16734 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:52:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.172+0000 7f2fb545f740 20 client.16734 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.172+0000 7f2fb545f740 1 --2- 172.21.15.78:0/1390605260 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b56a6dd0c0 0x55b56a6df4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.172+0000 7f2fb545f740 4 client.16734 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:52:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.172+0000 7f2fb545f740 1 -- 172.21.15.78:0/1390605260 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b56a634f40 con 0x55b56a6dd0c0 2024-08-18T23:52:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.172+0000 7f2fabfff640 1 --2- 172.21.15.78:0/1390605260 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b56a6dd0c0 0x55b56a6df4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.172+0000 7f2fabfff640 1 --2- 172.21.15.78:0/1390605260 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b56a6dd0c0 0x55b56a6df4b0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff9833adc474d29c server_cookie=a3e57390ca790e93 in_seq=0 out_seq=0 2024-08-18T23:52:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.172+0000 7f2f8bfff640 10 client.16734 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.173+0000 7f2f8bfff640 1 -- 172.21.15.78:0/1390605260 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b56a634f40 con 0x55b56a6dd0c0 2024-08-18T23:52:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.173+0000 7f2f8bfff640 10 client.16734 handle_command_reply: tid=0 2024-08-18T23:52:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.173+0000 7f2fb545f740 1 client.16734 shutdown 2024-08-18T23:52:27.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.173+0000 7f2fb545f740 1 -- 172.21.15.78:0/1390605260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c2be0 msgr2=0x55b56a6caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.173+0000 7f2fb545f740 1 --2- 172.21.15.78:0/1390605260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c2be0 0x55b56a6caae0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009a50 tx=0x7f2fa0031470 comp rx=0 tx=0).stop 2024-08-18T23:52:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.173+0000 7f2fb545f740 1 -- 172.21.15.78:0/1390605260 shutdown_connections 2024-08-18T23:52:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.173+0000 7f2fb545f740 1 --2- 172.21.15.78:0/1390605260 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b56a6dd0c0 0x55b56a6df4b0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.173+0000 7f2fb545f740 1 --2- 172.21.15.78:0/1390605260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6cd4a0 0x55b56a6cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.173+0000 7f2fb545f740 1 --2- 172.21.15.78:0/1390605260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c2be0 0x55b56a6caae0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.173+0000 7f2f8bfff640 0 client.16734 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:27.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.173+0000 7f2fb545f740 1 --2- 172.21.15.78:0/1390605260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c22c0 0x55b56a6ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.173+0000 7f2fb545f740 1 -- 172.21.15.78:0/1390605260 >> 172.21.15.78:0/1390605260 conn(0x55b56a6be850 msgr2=0x55b56a6bdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:27.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.173+0000 7f2fb545f740 1 -- 172.21.15.78:0/1390605260 shutdown_connections 2024-08-18T23:52:27.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.174+0000 7f2fb545f740 1 -- 172.21.15.78:0/1390605260 wait complete. 2024-08-18T23:52:27.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.174+0000 7f2fb545f740 20 client.16734 trim_cache size 0 max 16384 2024-08-18T23:52:27.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.190+0000 7f2fb545f740 1 Processor -- start 2024-08-18T23:52:27.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.190+0000 7f2fb545f740 1 -- start start 2024-08-18T23:52:27.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.191+0000 7f2fb545f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c22c0 0x55b56a6bda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:27.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.191+0000 7f2fb545f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c2be0 0x55b56a6e4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:27.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.191+0000 7f2fb545f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6cd4a0 0x55b56a6e5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:27.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.191+0000 7f2fb545f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1b13f0 con 0x55b56a6c22c0 2024-08-18T23:52:27.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.191+0000 7f2fb545f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1aec60 con 0x55b56a6cd4a0 2024-08-18T23:52:27.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.191+0000 7f2fb545f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1acab0 con 0x55b56a6c2be0 2024-08-18T23:52:27.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.191+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c22c0 0x55b56a6bda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:27.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.191+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c22c0 0x55b56a6bda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59098/0 (socket says 172.21.15.78:59098) 2024-08-18T23:52:27.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.191+0000 7f2fabfff640 1 -- 172.21.15.78:0/561764525 learned_addr learned my addr 172.21.15.78:0/561764525 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:27.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.191+0000 7f2fb0dfd640 1 --2- 172.21.15.78:0/561764525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c2be0 0x55b56a6e4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:27.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.191+0000 7f2fabfff640 1 -- 172.21.15.78:0/561764525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c2be0 msgr2=0x55b56a6e4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:27.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.191+0000 7f2fabfff640 1 --2- 172.21.15.78:0/561764525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c2be0 0x55b56a6e4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.191+0000 7f2fabfff640 1 -- 172.21.15.78:0/561764525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6cd4a0 msgr2=0x55b56a6e5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:27.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.191+0000 7f2fabfff640 1 --2- 172.21.15.78:0/561764525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6cd4a0 0x55b56a6e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.191+0000 7f2fabfff640 1 -- 172.21.15.78:0/561764525 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b56a6c3150 con 0x55b56a6c22c0 2024-08-18T23:52:27.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.191+0000 7f2fabfff640 1 --2- 172.21.15.78:0/561764525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c22c0 0x55b56a6bda20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2f98005940 tx=0x7f2f98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce9b8941504cebe server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:27.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.191+0000 7f2f967fc640 1 -- 172.21.15.78:0/561764525 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f980035f0 con 0x55b56a6c22c0 2024-08-18T23:52:27.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.191+0000 7f2f967fc640 1 -- 172.21.15.78:0/561764525 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f98003c70 con 0x55b56a6c22c0 2024-08-18T23:52:27.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.191+0000 7f2f967fc640 1 -- 172.21.15.78:0/561764525 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f98022bb0 con 0x55b56a6c22c0 2024-08-18T23:52:27.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.192+0000 7f2fb545f740 1 -- 172.21.15.78:0/561764525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c22c0 msgr2=0x55b56a6bda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:27.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.192+0000 7f2fb545f740 1 --2- 172.21.15.78:0/561764525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c22c0 0x55b56a6bda20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2f98005940 tx=0x7f2f98004730 comp rx=0 tx=0).stop 2024-08-18T23:52:27.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.192+0000 7f2fb545f740 1 -- 172.21.15.78:0/561764525 shutdown_connections 2024-08-18T23:52:27.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.192+0000 7f2fb545f740 1 --2- 172.21.15.78:0/561764525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6cd4a0 0x55b56a6e5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.192+0000 7f2fb545f740 1 --2- 172.21.15.78:0/561764525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c2be0 0x55b56a6e4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.192+0000 7f2fb545f740 1 --2- 172.21.15.78:0/561764525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6c22c0 0x55b56a6bda20 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.192+0000 7f2fb545f740 1 -- 172.21.15.78:0/561764525 >> 172.21.15.78:0/561764525 conn(0x55b56a6be850 msgr2=0x55b56a6bd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:27.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.192+0000 7f2fb545f740 1 -- 172.21.15.78:0/561764525 shutdown_connections 2024-08-18T23:52:27.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.192+0000 7f2fb545f740 1 -- 172.21.15.78:0/561764525 wait complete. 2024-08-18T23:52:27.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.192+0000 7f2fb545f740 1 -- start start 2024-08-18T23:52:27.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.193+0000 7f2fb545f740 1 Processor -- start 2024-08-18T23:52:27.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.193+0000 7f2fb545f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6c22c0 0x55b56a6c7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:27.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.193+0000 7f2fb545f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c2be0 0x55b56a6c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:27.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.193+0000 7f2fb545f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6cd4a0 0x55b56a6c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:27.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.193+0000 7f2fb545f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1ad120 con 0x55b56a6cd4a0 2024-08-18T23:52:27.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.193+0000 7f2fb545f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1ae590 con 0x55b56a6c22c0 2024-08-18T23:52:27.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.193+0000 7f2fb545f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1ad7f0 con 0x55b56a6c2be0 2024-08-18T23:52:27.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.193+0000 7f2fb545f740 10 client.0 _ll_register_callbacks cb 0x7f2fa0040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:27.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.193+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6cd4a0 0x55b56a6c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:27.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.193+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6c22c0 0x55b56a6c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:27.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.193+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6c22c0 0x55b56a6c7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.78:54298/0 (socket says 172.21.15.78:54298) 2024-08-18T23:52:27.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.193+0000 7f2fabfff640 1 -- 172.21.15.78:0/1237562553 learned_addr learned my addr 172.21.15.78:0/1237562553 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:27.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.193+0000 7f2fb0dfd640 1 --2- 172.21.15.78:0/1237562553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c2be0 0x55b56a6c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:27.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.194+0000 7f2fabfff640 1 -- 172.21.15.78:0/1237562553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c2be0 msgr2=0x55b56a6c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:27.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.194+0000 7f2fabfff640 1 --2- 172.21.15.78:0/1237562553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c2be0 0x55b56a6c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.194+0000 7f2fabfff640 1 -- 172.21.15.78:0/1237562553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6c22c0 msgr2=0x55b56a6c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:27.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.194+0000 7f2fabfff640 1 --2- 172.21.15.78:0/1237562553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6c22c0 0x55b56a6c7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.194+0000 7f2fabfff640 1 -- 172.21.15.78:0/1237562553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b56a746b60 con 0x55b56a6cd4a0 2024-08-18T23:52:27.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.194+0000 7f2fb0dfd640 1 --2- 172.21.15.78:0/1237562553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c2be0 0x55b56a6c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:52:27.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.194+0000 7f2fabfff640 1 --2- 172.21.15.78:0/1237562553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6cd4a0 0x55b56a6c81b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2f980096e0 tx=0x7f2f98022790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592a8082860f1ae server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:27.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.194+0000 7f2f8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:27.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.194+0000 7f2f8bfff640 1 -- 172.21.15.78:0/1237562553 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f98022590 con 0x55b56a6cd4a0 2024-08-18T23:52:27.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.194+0000 7f2f8bfff640 1 -- 172.21.15.78:0/1237562553 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2f98003800 con 0x55b56a6cd4a0 2024-08-18T23:52:27.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.194+0000 7f2fb545f740 1 -- 172.21.15.78:0/1237562553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f98003c70 con 0x55b56a6cd4a0 2024-08-18T23:52:27.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.194+0000 7f2f8bfff640 1 -- 172.21.15.78:0/1237562553 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2f98003a00 con 0x55b56a6cd4a0 2024-08-18T23:52:27.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.194+0000 7f2fb545f740 10 client.16740 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:27.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.194+0000 7f2fb545f740 1 -- 172.21.15.78:0/1237562553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b56a747e20 con 0x55b56a6cd4a0 2024-08-18T23:52:27.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.194+0000 7f2f8bfff640 1 -- 172.21.15.78:0/1237562553 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f98003c70 con 0x55b56a6cd4a0 2024-08-18T23:52:27.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.194+0000 7f2f8bfff640 1 -- 172.21.15.78:0/1237562553 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f2f9803b470 con 0x55b56a6cd4a0 2024-08-18T23:52:27.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.194+0000 7f2fb545f740 10 client.16740 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:27.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.194+0000 7f2fb545f740 10 client.16740 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:52:27.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.194+0000 7f2fb545f740 20 client.16740 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:27.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.195+0000 7f2fb545f740 1 --2- 172.21.15.78:0/1237562553 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b56a748430 0x55b56a74a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:27.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.195+0000 7f2fb545f740 4 client.16740 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.75, "await_for": 45.0}] 2024-08-18T23:52:27.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.195+0000 7f2fb545f740 1 -- 172.21.15.78:0/1237562553 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.75, "await_for": 45.0}) -- 0x55b56a62f290 con 0x55b56a748430 2024-08-18T23:52:27.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.195+0000 7f2fb0dfd640 1 --2- 172.21.15.78:0/1237562553 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b56a748430 0x55b56a74a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:27.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.195+0000 7f2fb0dfd640 1 --2- 172.21.15.78:0/1237562553 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b56a748430 0x55b56a74a850 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=855d2a48b993a6cc server_cookie=222584ed11de59d5 in_seq=0 out_seq=0 2024-08-18T23:52:27.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.195+0000 7f2f8bfff640 10 client.16740 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:27.240 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:27.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:27 smithi161 ceph-mon[22027]: pgmap v594: 129 pgs: 129 active+clean; 295 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 633 op/s 2024-08-18T23:52:27.709 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:27 smithi078 ceph-mon[18440]: pgmap v594: 129 pgs: 129 active+clean; 295 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 633 op/s 2024-08-18T23:52:27.793 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:52:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 31, 2024-08-18T23:52:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24452, 2024-08-18T23:52:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 26, 2024-08-18T23:52:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-08-18T23:52:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "71c1b92": { 2024-08-18T23:52:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 26, 2024-08-18T23:52:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-08-18T23:52:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:52:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-08-18T23:52:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:52:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:52:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-08-18T23:52:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 143.8, 2024-08-18T23:52:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-08-18T23:52:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-08-18T23:52:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-08-18T23:52:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:52:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-08-18T23:52:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:52:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:52:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:52:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:52:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:52:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:52:27.795 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:52:27.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.790+0000 7f2f8bfff640 1 -- 172.21.15.78:0/1237562553 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55b56a62f290 con 0x55b56a748430 2024-08-18T23:52:27.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.790+0000 7f2f8bfff640 10 client.16740 handle_command_reply: tid=0 2024-08-18T23:52:27.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.790+0000 7f2fb545f740 1 client.16740 shutdown 2024-08-18T23:52:27.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.790+0000 7f2fb545f740 1 -- 172.21.15.78:0/1237562553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6cd4a0 msgr2=0x55b56a6c81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:27.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.790+0000 7f2fb545f740 1 --2- 172.21.15.78:0/1237562553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6cd4a0 0x55b56a6c81b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2f980096e0 tx=0x7f2f98022790 comp rx=0 tx=0).stop 2024-08-18T23:52:27.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.790+0000 7f2fb545f740 1 -- 172.21.15.78:0/1237562553 shutdown_connections 2024-08-18T23:52:27.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.790+0000 7f2fb545f740 1 --2- 172.21.15.78:0/1237562553 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55b56a748430 0x55b56a74a850 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.791+0000 7f2fb545f740 1 --2- 172.21.15.78:0/1237562553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b56a6cd4a0 0x55b56a6c81b0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.791+0000 7f2f8bfff640 0 client.16740 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:27.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.791+0000 7f2fb545f740 1 --2- 172.21.15.78:0/1237562553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b56a6c2be0 0x55b56a6c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.791+0000 7f2fb545f740 1 --2- 172.21.15.78:0/1237562553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b56a6c22c0 0x55b56a6c7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.791+0000 7f2fb545f740 1 -- 172.21.15.78:0/1237562553 >> 172.21.15.78:0/1237562553 conn(0x55b56a6be850 msgr2=0x55b56a6bdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:27.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.791+0000 7f2fb545f740 1 -- 172.21.15.78:0/1237562553 shutdown_connections 2024-08-18T23:52:27.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.791+0000 7f2fb545f740 1 -- 172.21.15.78:0/1237562553 wait complete. 2024-08-18T23:52:27.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.791+0000 7f2fb545f740 20 client.16740 trim_cache size 0 max 16384 2024-08-18T23:52:27.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.792+0000 7f2fb3061640 1 -- 172.21.15.78:0/3628346692 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2f9c0a2b80 msgr2=0x7f2f9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:27.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.792+0000 7f2fb3061640 1 --2- 172.21.15.78:0/3628346692 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2f9c0a2b80 0x7f2f9c0a5040 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40ccd50 tx=0x7f2fa4086040 comp rx=0 tx=0).stop 2024-08-18T23:52:27.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.792+0000 7f2fb3061640 1 -- 172.21.15.78:0/3628346692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fac1078c0 msgr2=0x7f2fac19ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:27.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.792+0000 7f2fb3061640 1 --2- 172.21.15.78:0/3628346692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fac1078c0 0x7f2fac19ca90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400ca30 tx=0x7f2fa400cef0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.792+0000 7f2fb3061640 1 -- 172.21.15.78:0/3628346692 shutdown_connections 2024-08-18T23:52:27.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.792+0000 7f2fb3061640 1 --2- 172.21.15.78:0/3628346692 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2f9c0a2b80 0x7f2f9c0a5040 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.792+0000 7f2fb3061640 1 --2- 172.21.15.78:0/3628346692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fac1078c0 0x7f2fac19ca90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.792+0000 7f2fb3061640 1 --2- 172.21.15.78:0/3628346692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fac102260 0x7f2fac198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.792+0000 7f2fb3061640 1 --2- 172.21.15.78:0/3628346692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac1018c0 0x7f2fac1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:27.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.792+0000 7f2fb3061640 1 -- 172.21.15.78:0/3628346692 >> 172.21.15.78:0/3628346692 conn(0x7f2fac0fd620 msgr2=0x7f2fac0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:27.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.793+0000 7f2fb3061640 1 -- 172.21.15.78:0/3628346692 shutdown_connections 2024-08-18T23:52:27.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.793+0000 7f2fb3061640 1 -- 172.21.15.78:0/3628346692 wait complete. 2024-08-18T23:52:27.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:27 smithi123 ceph-mon[21808]: pgmap v594: 129 pgs: 129 active+clean; 295 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 633 op/s 2024-08-18T23:52:28.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.997+0000 7f68c2fe5640 1 -- 172.21.15.78:0/3312627312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68bc10c020 msgr2=0x7f68bc10e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:28.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.997+0000 7f68c2fe5640 1 --2- 172.21.15.78:0/3312627312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68bc10c020 0x7f68bc10e4c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009ca0 tx=0x7f68b402a8c0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.997+0000 7f68c2fe5640 1 -- 172.21.15.78:0/3312627312 shutdown_connections 2024-08-18T23:52:28.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.997+0000 7f68c2fe5640 1 --2- 172.21.15.78:0/3312627312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68bc10c020 0x7f68bc10e4c0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.997+0000 7f68c2fe5640 1 --2- 172.21.15.78:0/3312627312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68bc1034d0 0x7f68bc10b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.997+0000 7f68c2fe5640 1 --2- 172.21.15.78:0/3312627312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68bc102b20 0x7f68bc102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.997+0000 7f68c2fe5640 1 -- 172.21.15.78:0/3312627312 >> 172.21.15.78:0/3312627312 conn(0x7f68bc0fc500 msgr2=0x7f68bc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:28.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.997+0000 7f68c2fe5640 1 -- 172.21.15.78:0/3312627312 shutdown_connections 2024-08-18T23:52:28.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.997+0000 7f68c2fe5640 1 -- 172.21.15.78:0/3312627312 wait complete. 2024-08-18T23:52:28.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.998+0000 7f68c2fe5640 1 Processor -- start 2024-08-18T23:52:28.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.998+0000 7f68c2fe5640 1 -- start start 2024-08-18T23:52:28.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.998+0000 7f68c2fe5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68bc102b20 0x7f68bc1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:28.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.998+0000 7f68c2fe5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68bc1034d0 0x7f68bc198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:28.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.998+0000 7f68c2fe5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68bc10c020 0x7f68bc19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:28.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.998+0000 7f68c2fe5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc111e20 con 0x7f68bc1034d0 2024-08-18T23:52:28.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.998+0000 7f68c2fe5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc111ca0 con 0x7f68bc102b20 2024-08-18T23:52:28.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.998+0000 7f68c2fe5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc111fa0 con 0x7f68bc10c020 2024-08-18T23:52:28.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.998+0000 7f68bbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68bc1034d0 0x7f68bc198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:28.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.998+0000 7f68bbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68bc1034d0 0x7f68bc198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59128/0 (socket says 172.21.15.78:59128) 2024-08-18T23:52:28.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.998+0000 7f68bbfff640 1 -- 172.21.15.78:0/1851549515 learned_addr learned my addr 172.21.15.78:0/1851549515 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:28.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.998+0000 7f68c0d81640 1 --2- 172.21.15.78:0/1851549515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68bc102b20 0x7f68bc1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:28.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.998+0000 7f68c1582640 1 --2- 172.21.15.78:0/1851549515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68bc10c020 0x7f68bc19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:28.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.998+0000 7f68bbfff640 1 -- 172.21.15.78:0/1851549515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68bc10c020 msgr2=0x7f68bc19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:28.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.998+0000 7f68bbfff640 1 --2- 172.21.15.78:0/1851549515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68bc10c020 0x7f68bc19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.998+0000 7f68bbfff640 1 -- 172.21.15.78:0/1851549515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68bc102b20 msgr2=0x7f68bc1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:28.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.998+0000 7f68bbfff640 1 --2- 172.21.15.78:0/1851549515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68bc102b20 0x7f68bc1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.998+0000 7f68bbfff640 1 -- 172.21.15.78:0/1851549515 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68bc19d270 con 0x7f68bc1034d0 2024-08-18T23:52:28.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.999+0000 7f68bbfff640 1 --2- 172.21.15.78:0/1851549515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68bc1034d0 0x7f68bc198800 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f68a800e9f0 tx=0x7f68a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:28.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.999+0000 7f68b9ffb640 1 -- 172.21.15.78:0/1851549515 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a800ccc0 con 0x7f68bc1034d0 2024-08-18T23:52:28.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.999+0000 7f68c2fe5640 1 -- 172.21.15.78:0/1851549515 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68bc19d560 con 0x7f68bc1034d0 2024-08-18T23:52:28.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.999+0000 7f68b9ffb640 1 -- 172.21.15.78:0/1851549515 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a8010910 con 0x7f68bc1034d0 2024-08-18T23:52:28.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.999+0000 7f68b9ffb640 1 -- 172.21.15.78:0/1851549515 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8018920 con 0x7f68bc1034d0 2024-08-18T23:52:28.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:27.999+0000 7f68c2fe5640 1 -- 172.21.15.78:0/1851549515 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68bc1a4e40 con 0x7f68bc1034d0 2024-08-18T23:52:28.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.001+0000 7f68b9ffb640 1 -- 172.21.15.78:0/1851549515 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f68a8010ac0 con 0x7f68bc1034d0 2024-08-18T23:52:28.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.002+0000 7f68b9ffb640 1 --2- 172.21.15.78:0/1851549515 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f68ac0a2b80 0x7f68ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:28.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.002+0000 7f68b9ffb640 1 -- 172.21.15.78:0/1851549515 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f68a8099c10 con 0x7f68bc1034d0 2024-08-18T23:52:28.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.002+0000 7f68c53e3740 1 Processor -- start 2024-08-18T23:52:28.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.002+0000 7f68c0d81640 1 --2- 172.21.15.78:0/1851549515 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f68ac0a2b80 0x7f68ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:28.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.002+0000 7f68c53e3740 1 -- start start 2024-08-18T23:52:28.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.002+0000 7f68c53e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 0x55d125b1e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:28.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.002+0000 7f68c0d81640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 0x55d125b1e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:28.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.002+0000 7f68c0d81640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 0x55d125b1e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59140/0 (socket says 172.21.15.78:59140) 2024-08-18T23:52:28.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.002+0000 7f68c0d81640 1 -- 172.21.15.78:0/4018321126 learned_addr learned my addr 172.21.15.78:0/4018321126 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:28.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.003+0000 7f68c53e3740 1 --2- 172.21.15.78:0/4018321126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b1eb30 0x55d125b28eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:28.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.003+0000 7f68c53e3740 1 --2- 172.21.15.78:0/4018321126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b293f0 0x55d125b2b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:28.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.003+0000 7f68c53e3740 1 -- 172.21.15.78:0/4018321126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d125a83980 con 0x55d125b1e210 2024-08-18T23:52:28.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.003+0000 7f68c53e3740 1 -- 172.21.15.78:0/4018321126 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d125a1ba70 con 0x55d125b293f0 2024-08-18T23:52:28.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.003+0000 7f68c53e3740 1 -- 172.21.15.78:0/4018321126 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d125a7f940 con 0x55d125b1eb30 2024-08-18T23:52:28.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.003+0000 7f68c0d81640 1 --2- 172.21.15.78:0/1851549515 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f68ac0a2b80 0x7f68ac0a5040 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f68b00095f0 tx=0x7f68b0011810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:28.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.003+0000 7f68c1582640 1 --2- 172.21.15.78:0/4018321126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b293f0 0x55d125b2b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:28.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.003+0000 7f68bbfff640 1 --2- 172.21.15.78:0/4018321126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b1eb30 0x55d125b28eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:28.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.003+0000 7f68c0d81640 1 -- 172.21.15.78:0/4018321126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b1eb30 msgr2=0x55d125b28eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:28.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.003+0000 7f68c0d81640 1 --2- 172.21.15.78:0/4018321126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b1eb30 0x55d125b28eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.003+0000 7f68c0d81640 1 -- 172.21.15.78:0/4018321126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b293f0 msgr2=0x55d125b2b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:28.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.003+0000 7f68c0d81640 1 --2- 172.21.15.78:0/4018321126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b293f0 0x55d125b2b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.003+0000 7f68c0d81640 1 -- 172.21.15.78:0/4018321126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d125a7e6e0 con 0x55d125b1e210 2024-08-18T23:52:28.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.003+0000 7f68bbfff640 1 --2- 172.21.15.78:0/4018321126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b1eb30 0x55d125b28eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:52:28.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.003+0000 7f68c0d81640 1 --2- 172.21.15.78:0/4018321126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 0x55d125b1e5f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f68b001d4f0 tx=0x7f68b001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230a75fdfb67bcaa server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:28.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.003+0000 7f68a6ffd640 1 -- 172.21.15.78:0/4018321126 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0017040 con 0x55d125b1e210 2024-08-18T23:52:28.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.003+0000 7f68a6ffd640 1 -- 172.21.15.78:0/4018321126 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b0019050 con 0x55d125b1e210 2024-08-18T23:52:28.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.003+0000 7f68a6ffd640 1 -- 172.21.15.78:0/4018321126 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b000e8b0 con 0x55d125b1e210 2024-08-18T23:52:28.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.004+0000 7f68c53e3740 1 -- 172.21.15.78:0/4018321126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 msgr2=0x55d125b1e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:28.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.004+0000 7f68c53e3740 1 --2- 172.21.15.78:0/4018321126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 0x55d125b1e5f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f68b001d4f0 tx=0x7f68b001c000 comp rx=0 tx=0).stop 2024-08-18T23:52:28.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.004+0000 7f68c53e3740 1 -- 172.21.15.78:0/4018321126 shutdown_connections 2024-08-18T23:52:28.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.004+0000 7f68c53e3740 1 --2- 172.21.15.78:0/4018321126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b293f0 0x55d125b2b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.004+0000 7f68c53e3740 1 --2- 172.21.15.78:0/4018321126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b1eb30 0x55d125b28eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.004+0000 7f68c53e3740 1 --2- 172.21.15.78:0/4018321126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 0x55d125b1e5f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.004+0000 7f68c53e3740 1 -- 172.21.15.78:0/4018321126 >> 172.21.15.78:0/4018321126 conn(0x55d125b1a7a0 msgr2=0x55d125b1ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:28.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.004+0000 7f68c53e3740 1 -- 172.21.15.78:0/4018321126 shutdown_connections 2024-08-18T23:52:28.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.004+0000 7f68c53e3740 1 -- 172.21.15.78:0/4018321126 wait complete. 2024-08-18T23:52:28.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.004+0000 7f68c53e3740 1 -- start start 2024-08-18T23:52:28.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.004+0000 7f68c53e3740 1 Processor -- start 2024-08-18T23:52:28.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.005+0000 7f68c53e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b1e210 0x55d125b263c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:28.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.005+0000 7f68c53e3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b1eb30 0x55d125b26900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:28.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.005+0000 7f68c53e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b293f0 0x55d125b26e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:28.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.005+0000 7f68c53e3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d125b303d0 con 0x55d125b293f0 2024-08-18T23:52:28.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.005+0000 7f68c53e3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d125a3eaa0 con 0x55d125b1eb30 2024-08-18T23:52:28.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.005+0000 7f68c53e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d125b30550 con 0x55d125b1e210 2024-08-18T23:52:28.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.005+0000 7f68c0d81640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b1e210 0x55d125b263c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:28.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.005+0000 7f68c0d81640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b1e210 0x55d125b263c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:41672/0 (socket says 172.21.15.78:41672) 2024-08-18T23:52:28.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.005+0000 7f68c0d81640 1 -- 172.21.15.78:0/3876425860 learned_addr learned my addr 172.21.15.78:0/3876425860 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:28.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.005+0000 7f68bbfff640 1 --2- 172.21.15.78:0/3876425860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b1eb30 0x55d125b26900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:28.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.005+0000 7f68c53e3740 10 client.0 _ll_register_callbacks cb 0x55d125a2de60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:28.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.005+0000 7f68c1582640 1 --2- 172.21.15.78:0/3876425860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b293f0 0x55d125b26e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:28.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.005+0000 7f68bbfff640 1 -- 172.21.15.78:0/3876425860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b1e210 msgr2=0x55d125b263c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:28.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.005+0000 7f68bbfff640 1 --2- 172.21.15.78:0/3876425860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b1e210 0x55d125b263c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.005+0000 7f68bbfff640 1 -- 172.21.15.78:0/3876425860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b293f0 msgr2=0x55d125b26e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:28.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.005+0000 7f68bbfff640 1 --2- 172.21.15.78:0/3876425860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b293f0 0x55d125b26e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.005+0000 7f68bbfff640 1 -- 172.21.15.78:0/3876425860 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d125a22650 con 0x55d125b1eb30 2024-08-18T23:52:28.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.005+0000 7f68c0d81640 1 --2- 172.21.15.78:0/3876425860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b1e210 0x55d125b263c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:52:28.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.006+0000 7f68bbfff640 1 --2- 172.21.15.78:0/3876425860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b1eb30 0x55d125b26900 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f68a800c6a0 tx=0x7f68a8011f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=992c8c7512c2e17b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:28.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.006+0000 7f689bfff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-08-18T23:52:28.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.006+0000 7f689bfff640 1 -- 172.21.15.78:0/3876425860 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a809b1c0 con 0x55d125b1eb30 2024-08-18T23:52:28.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.006+0000 7f68c53e3740 1 -- 172.21.15.78:0/3876425860 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b0019050 con 0x55d125b1eb30 2024-08-18T23:52:28.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.006+0000 7f689bfff640 1 -- 172.21.15.78:0/3876425860 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f68a8010430 con 0x55d125b1eb30 2024-08-18T23:52:28.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.006+0000 7f689bfff640 1 -- 172.21.15.78:0/3876425860 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f68a8010630 con 0x55d125b1eb30 2024-08-18T23:52:28.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.006+0000 7f68c53e3740 10 client.26141 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:28.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.006+0000 7f68c53e3740 1 -- 172.21.15.78:0/3876425860 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d1257f53c0 con 0x55d125b1eb30 2024-08-18T23:52:28.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.006+0000 7f689bfff640 1 -- 172.21.15.78:0/3876425860 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b0019050 con 0x55d125b1eb30 2024-08-18T23:52:28.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.006+0000 7f689bfff640 1 -- 172.21.15.78:0/3876425860 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f68a809aa30 con 0x55d125b1eb30 2024-08-18T23:52:28.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.006+0000 7f68c53e3740 10 client.26141 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:28.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.006+0000 7f68c53e3740 10 client.26141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:52:28.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.006+0000 7f68c53e3740 20 client.26141 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:28.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.006+0000 7f68c53e3740 1 --2- 172.21.15.78:0/3876425860 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55d125b3d2b0 0x55d125b3f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:28.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.006+0000 7f68c53e3740 4 client.26141 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:52:28.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.006+0000 7f68c53e3740 1 -- 172.21.15.78:0/3876425860 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d125a90ef0 con 0x55d125b3d2b0 2024-08-18T23:52:28.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.006+0000 7f68c0d81640 1 --2- 172.21.15.78:0/3876425860 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55d125b3d2b0 0x55d125b3f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:28.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.007+0000 7f689bfff640 10 client.26141 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:28.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.007+0000 7f68c0d81640 1 --2- 172.21.15.78:0/3876425860 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55d125b3d2b0 0x55d125b3f6d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f16bc4b56ed386bd server_cookie=c70cb92a8c71a273 in_seq=0 out_seq=0 2024-08-18T23:52:28.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.007+0000 7f689bfff640 1 -- 172.21.15.78:0/3876425860 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d125a90ef0 con 0x55d125b3d2b0 2024-08-18T23:52:28.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.007+0000 7f689bfff640 10 client.26141 handle_command_reply: tid=0 2024-08-18T23:52:28.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.007+0000 7f68c53e3740 1 client.26141 shutdown 2024-08-18T23:52:28.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.008+0000 7f68c53e3740 1 -- 172.21.15.78:0/3876425860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b1eb30 msgr2=0x55d125b26900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:28.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.008+0000 7f68c53e3740 1 --2- 172.21.15.78:0/3876425860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b1eb30 0x55d125b26900 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f68a800c6a0 tx=0x7f68a8011f70 comp rx=0 tx=0).stop 2024-08-18T23:52:28.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.008+0000 7f68c53e3740 1 -- 172.21.15.78:0/3876425860 shutdown_connections 2024-08-18T23:52:28.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.008+0000 7f68c53e3740 1 --2- 172.21.15.78:0/3876425860 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55d125b3d2b0 0x55d125b3f6d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.008+0000 7f68c53e3740 1 --2- 172.21.15.78:0/3876425860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b293f0 0x55d125b26e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.008+0000 7f689bfff640 0 client.26141 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:28.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.008+0000 7f68c53e3740 1 --2- 172.21.15.78:0/3876425860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b1eb30 0x55d125b26900 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.008+0000 7f68c53e3740 1 --2- 172.21.15.78:0/3876425860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b1e210 0x55d125b263c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.008+0000 7f68c53e3740 1 -- 172.21.15.78:0/3876425860 >> 172.21.15.78:0/3876425860 conn(0x55d125b1a7a0 msgr2=0x55d125b20480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:28.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.008+0000 7f68c53e3740 1 -- 172.21.15.78:0/3876425860 shutdown_connections 2024-08-18T23:52:28.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.008+0000 7f68c53e3740 1 -- 172.21.15.78:0/3876425860 wait complete. 2024-08-18T23:52:28.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.008+0000 7f68c53e3740 20 client.26141 trim_cache size 0 max 16384 2024-08-18T23:52:28.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.025+0000 7f68c53e3740 1 Processor -- start 2024-08-18T23:52:28.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.025+0000 7f68c53e3740 1 -- start start 2024-08-18T23:52:28.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.025+0000 7f68c53e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 0x55d125b1a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:28.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.025+0000 7f68c53e3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b1eb30 0x55d125b44f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:28.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.025+0000 7f68c53e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b293f0 0x55d125b454c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:28.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.025+0000 7f68c53e3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc1af1a0 con 0x55d125b1e210 2024-08-18T23:52:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.025+0000 7f68c53e3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc1aca10 con 0x55d125b1eb30 2024-08-18T23:52:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.025+0000 7f68c53e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc1aa860 con 0x55d125b293f0 2024-08-18T23:52:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.025+0000 7f68c0d81640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 0x55d125b1a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.025+0000 7f68c0d81640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 0x55d125b1a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59156/0 (socket says 172.21.15.78:59156) 2024-08-18T23:52:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.025+0000 7f68c0d81640 1 -- 172.21.15.78:0/1334063040 learned_addr learned my addr 172.21.15.78:0/1334063040 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.025+0000 7f68bbfff640 1 --2- 172.21.15.78:0/1334063040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b1eb30 0x55d125b44f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:28.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.025+0000 7f68c0d81640 1 -- 172.21.15.78:0/1334063040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b293f0 msgr2=0x55d125b454c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:28.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.025+0000 7f68c0d81640 1 --2- 172.21.15.78:0/1334063040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b293f0 0x55d125b454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.025+0000 7f68c0d81640 1 -- 172.21.15.78:0/1334063040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b1eb30 msgr2=0x55d125b44f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:28.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.025+0000 7f68c0d81640 1 --2- 172.21.15.78:0/1334063040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b1eb30 0x55d125b44f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.025+0000 7f68c0d81640 1 -- 172.21.15.78:0/1334063040 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d125b14c50 con 0x55d125b1e210 2024-08-18T23:52:28.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.025+0000 7f68c0d81640 1 --2- 172.21.15.78:0/1334063040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 0x55d125b1a0f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f68b000ff80 tx=0x7f68b000e450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f0a3503d24e00b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:28.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.026+0000 7f68a67fc640 1 -- 172.21.15.78:0/1334063040 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0037030 con 0x55d125b1e210 2024-08-18T23:52:28.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.026+0000 7f68a67fc640 1 -- 172.21.15.78:0/1334063040 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b000e9a0 con 0x55d125b1e210 2024-08-18T23:52:28.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.026+0000 7f68a67fc640 1 -- 172.21.15.78:0/1334063040 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0017040 con 0x55d125b1e210 2024-08-18T23:52:28.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.026+0000 7f68c53e3740 1 -- 172.21.15.78:0/1334063040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 msgr2=0x55d125b1a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:28.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.026+0000 7f68c53e3740 1 --2- 172.21.15.78:0/1334063040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 0x55d125b1a0f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f68b000ff80 tx=0x7f68b000e450 comp rx=0 tx=0).stop 2024-08-18T23:52:28.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.026+0000 7f68c53e3740 1 -- 172.21.15.78:0/1334063040 shutdown_connections 2024-08-18T23:52:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.026+0000 7f68c53e3740 1 --2- 172.21.15.78:0/1334063040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b293f0 0x55d125b454c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.026+0000 7f68c53e3740 1 --2- 172.21.15.78:0/1334063040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b1eb30 0x55d125b44f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.026+0000 7f68c53e3740 1 --2- 172.21.15.78:0/1334063040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 0x55d125b1a0f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.026+0000 7f68c53e3740 1 -- 172.21.15.78:0/1334063040 >> 172.21.15.78:0/1334063040 conn(0x55d125b1a7a0 msgr2=0x55d125b28330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.026+0000 7f68c53e3740 1 -- 172.21.15.78:0/1334063040 shutdown_connections 2024-08-18T23:52:28.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.026+0000 7f68c53e3740 1 -- 172.21.15.78:0/1334063040 wait complete. 2024-08-18T23:52:28.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.027+0000 7f68c53e3740 1 -- start start 2024-08-18T23:52:28.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.027+0000 7f68c53e3740 1 Processor -- start 2024-08-18T23:52:28.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.027+0000 7f68c53e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 0x55d125b35eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:28.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.027+0000 7f68c53e3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b1eb30 0x55d125b230b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:28.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.027+0000 7f68c53e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b293f0 0x55d125b235f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:28.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.027+0000 7f68c53e3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc1aaed0 con 0x55d125b1e210 2024-08-18T23:52:28.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.027+0000 7f68c53e3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc1ac340 con 0x55d125b1eb30 2024-08-18T23:52:28.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.027+0000 7f68c53e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc1ab5a0 con 0x55d125b293f0 2024-08-18T23:52:28.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.027+0000 7f68c0d81640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 0x55d125b35eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:28.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.028+0000 7f68c0d81640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 0x55d125b35eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59162/0 (socket says 172.21.15.78:59162) 2024-08-18T23:52:28.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.028+0000 7f68c0d81640 1 -- 172.21.15.78:0/339037803 learned_addr learned my addr 172.21.15.78:0/339037803 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:28.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.028+0000 7f68c53e3740 10 client.0 _ll_register_callbacks cb 0x7f68a8097e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:28.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.028+0000 7f68bbfff640 1 --2- 172.21.15.78:0/339037803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b1eb30 0x55d125b230b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:28.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.028+0000 7f68c1582640 1 --2- 172.21.15.78:0/339037803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b293f0 0x55d125b235f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:28.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.028+0000 7f68c0d81640 1 -- 172.21.15.78:0/339037803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b293f0 msgr2=0x55d125b235f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:28.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.028+0000 7f68c0d81640 1 --2- 172.21.15.78:0/339037803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b293f0 0x55d125b235f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.028+0000 7f68c0d81640 1 -- 172.21.15.78:0/339037803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b1eb30 msgr2=0x55d125b230b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:28.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.028+0000 7f68c0d81640 1 --2- 172.21.15.78:0/339037803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b1eb30 0x55d125b230b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.028+0000 7f68c0d81640 1 -- 172.21.15.78:0/339037803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d125b23df0 con 0x55d125b1e210 2024-08-18T23:52:28.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.028+0000 7f68c0d81640 1 --2- 172.21.15.78:0/339037803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 0x55d125b35eb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f68b0011d10 tx=0x7f68b001da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0b5248a4644752 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:28.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.028+0000 7f689bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:28.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.028+0000 7f689bfff640 1 -- 172.21.15.78:0/339037803 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b000e6c0 con 0x55d125b1e210 2024-08-18T23:52:28.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.028+0000 7f68c53e3740 1 -- 172.21.15.78:0/339037803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b000e9a0 con 0x55d125b1e210 2024-08-18T23:52:28.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.028+0000 7f689bfff640 1 -- 172.21.15.78:0/339037803 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f68b000f3c0 con 0x55d125b1e210 2024-08-18T23:52:28.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.028+0000 7f689bfff640 1 -- 172.21.15.78:0/339037803 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f68b000f600 con 0x55d125b1e210 2024-08-18T23:52:28.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.028+0000 7f68c53e3740 10 client.16758 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:28.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.028+0000 7f68c53e3740 1 -- 172.21.15.78:0/339037803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d125ba7ee0 con 0x55d125b1e210 2024-08-18T23:52:28.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.029+0000 7f689bfff640 1 -- 172.21.15.78:0/339037803 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b000e9a0 con 0x55d125b1e210 2024-08-18T23:52:28.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.029+0000 7f689bfff640 1 -- 172.21.15.78:0/339037803 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f68b0024020 con 0x55d125b1e210 2024-08-18T23:52:28.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.029+0000 7f68c53e3740 10 client.16758 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:28.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.029+0000 7f68c53e3740 10 client.16758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:52:28.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.029+0000 7f68c53e3740 20 client.16758 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:28.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.029+0000 7f68c53e3740 1 --2- 172.21.15.78:0/339037803 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55d125ba83a0 0x55d125baa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:28.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.029+0000 7f68c53e3740 4 client.16758 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:52:28.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.029+0000 7f68c53e3740 1 -- 172.21.15.78:0/339037803 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d125a8b400 con 0x55d125ba83a0 2024-08-18T23:52:28.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.029+0000 7f68bbfff640 1 --2- 172.21.15.78:0/339037803 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55d125ba83a0 0x55d125baa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:28.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.029+0000 7f689bfff640 10 client.16758 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:28.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.029+0000 7f68bbfff640 1 --2- 172.21.15.78:0/339037803 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55d125ba83a0 0x55d125baa790 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c94dc0c4fbfcd4 server_cookie=8d908f1ee50898b7 in_seq=0 out_seq=0 2024-08-18T23:52:28.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.029+0000 7f689bfff640 1 -- 172.21.15.78:0/339037803 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d125a8b400 con 0x55d125ba83a0 2024-08-18T23:52:28.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.029+0000 7f689bfff640 10 client.16758 handle_command_reply: tid=0 2024-08-18T23:52:28.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.029+0000 7f68c53e3740 1 client.16758 shutdown 2024-08-18T23:52:28.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c53e3740 1 -- 172.21.15.78:0/339037803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 msgr2=0x55d125b35eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:28.347 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:52:28.347 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-08-18T23:52:28.348 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-08-18T23:52:28.348 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:52:28.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c53e3740 1 --2- 172.21.15.78:0/339037803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 0x55d125b35eb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f68b0011d10 tx=0x7f68b001da20 comp rx=0 tx=0).stop 2024-08-18T23:52:28.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c53e3740 1 -- 172.21.15.78:0/339037803 shutdown_connections 2024-08-18T23:52:28.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c53e3740 1 --2- 172.21.15.78:0/339037803 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55d125ba83a0 0x55d125baa790 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c53e3740 1 --2- 172.21.15.78:0/339037803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d125b293f0 0x55d125b235f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c53e3740 1 --2- 172.21.15.78:0/339037803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d125b1eb30 0x55d125b230b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f689bfff640 0 client.16758 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:28.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c53e3740 1 --2- 172.21.15.78:0/339037803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d125b1e210 0x55d125b35eb0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c53e3740 1 -- 172.21.15.78:0/339037803 >> 172.21.15.78:0/339037803 conn(0x55d125b1a7a0 msgr2=0x55d125b1fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:28.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c53e3740 1 -- 172.21.15.78:0/339037803 shutdown_connections 2024-08-18T23:52:28.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c53e3740 1 -- 172.21.15.78:0/339037803 wait complete. 2024-08-18T23:52:28.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c53e3740 20 client.16758 trim_cache size 0 max 16384 2024-08-18T23:52:28.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c2fe5640 1 -- 172.21.15.78:0/1851549515 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f68ac0a2b80 msgr2=0x7f68ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:28.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c2fe5640 1 --2- 172.21.15.78:0/1851549515 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f68ac0a2b80 0x7f68ac0a5040 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f68b00095f0 tx=0x7f68b0011810 comp rx=0 tx=0).stop 2024-08-18T23:52:28.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c2fe5640 1 -- 172.21.15.78:0/1851549515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68bc1034d0 msgr2=0x7f68bc198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:28.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c2fe5640 1 --2- 172.21.15.78:0/1851549515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68bc1034d0 0x7f68bc198800 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f68a800e9f0 tx=0x7f68a800eeb0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c2fe5640 1 -- 172.21.15.78:0/1851549515 shutdown_connections 2024-08-18T23:52:28.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c2fe5640 1 --2- 172.21.15.78:0/1851549515 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f68ac0a2b80 0x7f68ac0a5040 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c2fe5640 1 --2- 172.21.15.78:0/1851549515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68bc10c020 0x7f68bc19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c2fe5640 1 --2- 172.21.15.78:0/1851549515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f68bc1034d0 0x7f68bc198800 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c2fe5640 1 --2- 172.21.15.78:0/1851549515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68bc102b20 0x7f68bc1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:28.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.030+0000 7f68c2fe5640 1 -- 172.21.15.78:0/1851549515 >> 172.21.15.78:0/1851549515 conn(0x7f68bc0fc500 msgr2=0x7f68bc0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:28.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.031+0000 7f68c2fe5640 1 -- 172.21.15.78:0/1851549515 shutdown_connections 2024-08-18T23:52:28.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:28.031+0000 7f68c2fe5640 1 -- 172.21.15.78:0/1851549515 wait complete. 2024-08-18T23:52:29.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:29 smithi078 ceph-mon[18440]: pgmap v595: 129 pgs: 129 active+clean; 288 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 590 op/s 2024-08-18T23:52:29.511 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-18T23:52:29.511 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:52:29.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:29 smithi161 ceph-mon[22027]: pgmap v595: 129 pgs: 129 active+clean; 288 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 590 op/s 2024-08-18T23:52:29.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:29 smithi123 ceph-mon[21808]: pgmap v595: 129 pgs: 129 active+clean; 288 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 590 op/s 2024-08-18T23:52:29.976 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '71c1b92', quiesce took 6.3 seconds. Will release after: 17.45 2024-08-18T23:52:30.098 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:30.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.564+0000 7fd2aca8c640 1 -- 172.21.15.78:0/1206771100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a8101150 msgr2=0x7fd2a810d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:30.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.564+0000 7fd2aca8c640 1 --2- 172.21.15.78:0/1206771100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a8101150 0x7fd2a810d680 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009cd0 tx=0x7fd2a002a8e0 comp rx=0 tx=0).stop 2024-08-18T23:52:30.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.564+0000 7fd2aca8c640 1 -- 172.21.15.78:0/1206771100 shutdown_connections 2024-08-18T23:52:30.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.564+0000 7fd2aca8c640 1 --2- 172.21.15.78:0/1206771100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a8101150 0x7fd2a810d680 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:30.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.564+0000 7fd2aca8c640 1 --2- 172.21.15.78:0/1206771100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a81007b0 0x7fd2a8100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:30.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.564+0000 7fd2aca8c640 1 --2- 172.21.15.78:0/1206771100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a81067b0 0x7fd2a8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:30.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.564+0000 7fd2aca8c640 1 -- 172.21.15.78:0/1206771100 >> 172.21.15.78:0/1206771100 conn(0x7fd2a80fc4d0 msgr2=0x7fd2a80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:30.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.564+0000 7fd2aca8c640 1 -- 172.21.15.78:0/1206771100 shutdown_connections 2024-08-18T23:52:30.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.564+0000 7fd2aca8c640 1 -- 172.21.15.78:0/1206771100 wait complete. 2024-08-18T23:52:30.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.565+0000 7fd2aca8c640 1 Processor -- start 2024-08-18T23:52:30.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.565+0000 7fd2aca8c640 1 -- start start 2024-08-18T23:52:30.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.565+0000 7fd2aca8c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a81007b0 0x7fd2a8199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:30.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.565+0000 7fd2aca8c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a8101150 0x7fd2a8199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:30.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.565+0000 7fd2aca8c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a81067b0 0x7fd2a81a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:30.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.565+0000 7fd2aca8c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8110bf0 con 0x7fd2a81007b0 2024-08-18T23:52:30.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.565+0000 7fd2aca8c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8110a70 con 0x7fd2a81067b0 2024-08-18T23:52:30.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.565+0000 7fd2aca8c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8110d70 con 0x7fd2a8101150 2024-08-18T23:52:30.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.565+0000 7fd2a659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a81007b0 0x7fd2a8199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:30.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.565+0000 7fd2a659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a81007b0 0x7fd2a8199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59186/0 (socket says 172.21.15.78:59186) 2024-08-18T23:52:30.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.565+0000 7fd2a659c640 1 -- 172.21.15.78:0/2371961920 learned_addr learned my addr 172.21.15.78:0/2371961920 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:30.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.566+0000 7fd2a6d9d640 1 --2- 172.21.15.78:0/2371961920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a81067b0 0x7fd2a81a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:30.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.566+0000 7fd2a5d9b640 1 --2- 172.21.15.78:0/2371961920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a8101150 0x7fd2a8199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:30.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.566+0000 7fd2a659c640 1 -- 172.21.15.78:0/2371961920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a8101150 msgr2=0x7fd2a8199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:30.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.566+0000 7fd2a659c640 1 --2- 172.21.15.78:0/2371961920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a8101150 0x7fd2a8199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:30.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.566+0000 7fd2a659c640 1 -- 172.21.15.78:0/2371961920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a81067b0 msgr2=0x7fd2a81a40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:30.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.566+0000 7fd2a659c640 1 --2- 172.21.15.78:0/2371961920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a81067b0 0x7fd2a81a40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:30.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.566+0000 7fd2a659c640 1 -- 172.21.15.78:0/2371961920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a81a4600 con 0x7fd2a81007b0 2024-08-18T23:52:30.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.566+0000 7fd2a5d9b640 1 --2- 172.21.15.78:0/2371961920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a8101150 0x7fd2a8199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:52:30.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.566+0000 7fd2a6d9d640 1 --2- 172.21.15.78:0/2371961920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a81067b0 0x7fd2a81a40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:30.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.566+0000 7fd2a659c640 1 --2- 172.21.15.78:0/2371961920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a81007b0 0x7fd2a8199040 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd29800b4a0 tx=0x7fd29800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:30.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.566+0000 7fd2977fe640 1 -- 172.21.15.78:0/2371961920 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298014070 con 0x7fd2a81007b0 2024-08-18T23:52:30.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.566+0000 7fd2977fe640 1 -- 172.21.15.78:0/2371961920 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd298010af0 con 0x7fd2a81007b0 2024-08-18T23:52:30.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.566+0000 7fd2977fe640 1 -- 172.21.15.78:0/2371961920 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298017d10 con 0x7fd2a81007b0 2024-08-18T23:52:30.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.566+0000 7fd2aca8c640 1 -- 172.21.15.78:0/2371961920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a81a47d0 con 0x7fd2a81007b0 2024-08-18T23:52:30.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.566+0000 7fd2aca8c640 1 -- 172.21.15.78:0/2371961920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a81a4ce0 con 0x7fd2a81007b0 2024-08-18T23:52:30.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.567+0000 7fd2aca8c640 1 -- 172.21.15.78:0/2371961920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a8100c10 con 0x7fd2a81007b0 2024-08-18T23:52:30.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.569+0000 7fd2977fe640 1 -- 172.21.15.78:0/2371961920 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd298017070 con 0x7fd2a81007b0 2024-08-18T23:52:30.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.569+0000 7fd2977fe640 1 --2- 172.21.15.78:0/2371961920 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2900a2b80 0x7fd2900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:30.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.570+0000 7fd2a5d9b640 1 --2- 172.21.15.78:0/2371961920 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2900a2b80 0x7fd2900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:30.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.570+0000 7fd2977fe640 1 -- 172.21.15.78:0/2371961920 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fd2980cebe0 con 0x7fd2a81007b0 2024-08-18T23:52:30.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.570+0000 7fd2a5d9b640 1 --2- 172.21.15.78:0/2371961920 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2900a2b80 0x7fd2900a5040 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd2a806b010 tx=0x7fd29c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:30.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.574+0000 7fd2977fe640 1 -- 172.21.15.78:0/2371961920 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd29809b7b0 con 0x7fd2a81007b0 2024-08-18T23:52:30.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.776+0000 7fd2aca8c640 1 -- 172.21.15.78:0/2371961920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2a806aab0 con 0x7fd2a81007b0 2024-08-18T23:52:30.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.777+0000 7fd2977fe640 1 -- 172.21.15.78:0/2371961920 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11933 (secure 0 0 0) 0x7fd2a806aab0 con 0x7fd2a81007b0 2024-08-18T23:52:30.779 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:52:30.779 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":31,"btime":"2024-08-18T23:51:58:009010+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:51:57.958854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24428,24427]},"id":1}]} 2024-08-18T23:52:30.779 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 31 2024-08-18T23:52:30.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.779+0000 7fd2aca8c640 1 -- 172.21.15.78:0/2371961920 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2900a2b80 msgr2=0x7fd2900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:30.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.779+0000 7fd2aca8c640 1 --2- 172.21.15.78:0/2371961920 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2900a2b80 0x7fd2900a5040 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd2a806b010 tx=0x7fd29c005e50 comp rx=0 tx=0).stop 2024-08-18T23:52:30.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.779+0000 7fd2aca8c640 1 -- 172.21.15.78:0/2371961920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a81007b0 msgr2=0x7fd2a8199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:30.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.779+0000 7fd2aca8c640 1 --2- 172.21.15.78:0/2371961920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a81007b0 0x7fd2a8199040 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd29800b4a0 tx=0x7fd29800b960 comp rx=0 tx=0).stop 2024-08-18T23:52:30.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.779+0000 7fd2aca8c640 1 -- 172.21.15.78:0/2371961920 shutdown_connections 2024-08-18T23:52:30.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.779+0000 7fd2aca8c640 1 --2- 172.21.15.78:0/2371961920 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fd2900a2b80 0x7fd2900a5040 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:30.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.779+0000 7fd2aca8c640 1 --2- 172.21.15.78:0/2371961920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a81067b0 0x7fd2a81a40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:30.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.779+0000 7fd2aca8c640 1 --2- 172.21.15.78:0/2371961920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a8101150 0x7fd2a8199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:30.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.779+0000 7fd2aca8c640 1 --2- 172.21.15.78:0/2371961920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2a81007b0 0x7fd2a8199040 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:30.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.779+0000 7fd2aca8c640 1 -- 172.21.15.78:0/2371961920 >> 172.21.15.78:0/2371961920 conn(0x7fd2a80fc4d0 msgr2=0x7fd2a80fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:30.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.780+0000 7fd2aca8c640 1 -- 172.21.15.78:0/2371961920 shutdown_connections 2024-08-18T23:52:30.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:30.780+0000 7fd2aca8c640 1 -- 172.21.15.78:0/2371961920 wait complete. 2024-08-18T23:52:31.444 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-18T23:52:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:31 smithi161 ceph-mon[22027]: pgmap v596: 129 pgs: 129 active+clean; 277 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 603 op/s 2024-08-18T23:52:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:31 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2371961920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:52:31.693 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:31.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:31 smithi078 ceph-mon[18440]: pgmap v596: 129 pgs: 129 active+clean; 277 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 603 op/s 2024-08-18T23:52:31.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:31 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2371961920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:52:31.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:31 smithi123 ceph-mon[21808]: pgmap v596: 129 pgs: 129 active+clean; 277 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 603 op/s 2024-08-18T23:52:31.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:31 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2371961920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:52:32.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.174+0000 7f9f8e1b8640 1 -- 172.21.15.78:0/2130640601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f88100790 msgr2=0x7f9f88100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:32.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.174+0000 7f9f8e1b8640 1 --2- 172.21.15.78:0/2130640601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f88100790 0x7f9f88100bf0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009930 tx=0x7f9f7c02a420 comp rx=0 tx=0).stop 2024-08-18T23:52:32.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.174+0000 7f9f8e1b8640 1 -- 172.21.15.78:0/2130640601 shutdown_connections 2024-08-18T23:52:32.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.174+0000 7f9f8e1b8640 1 --2- 172.21.15.78:0/2130640601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f88101130 0x7f9f8810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.174+0000 7f9f8e1b8640 1 --2- 172.21.15.78:0/2130640601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f88100790 0x7f9f88100bf0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.174+0000 7f9f8e1b8640 1 --2- 172.21.15.78:0/2130640601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f88106790 0x7f9f88106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.174+0000 7f9f8e1b8640 1 -- 172.21.15.78:0/2130640601 >> 172.21.15.78:0/2130640601 conn(0x7f9f880fc4b0 msgr2=0x7f9f880fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:32.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.174+0000 7f9f8e1b8640 1 -- 172.21.15.78:0/2130640601 shutdown_connections 2024-08-18T23:52:32.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.174+0000 7f9f8e1b8640 1 -- 172.21.15.78:0/2130640601 wait complete. 2024-08-18T23:52:32.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.175+0000 7f9f8e1b8640 1 Processor -- start 2024-08-18T23:52:32.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.175+0000 7f9f8e1b8640 1 -- start start 2024-08-18T23:52:32.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.175+0000 7f9f8e1b8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f88100790 0x7f9f88197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:32.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.175+0000 7f9f8e1b8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f88101130 0x7f9f881975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:32.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.175+0000 7f9f8e1b8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f88106790 0x7f9f8819b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:32.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.175+0000 7f9f8e1b8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f88110b90 con 0x7f9f88106790 2024-08-18T23:52:32.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.175+0000 7f9f8e1b8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9f88110a10 con 0x7f9f88100790 2024-08-18T23:52:32.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.175+0000 7f9f8e1b8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9f88110d10 con 0x7f9f88101130 2024-08-18T23:52:32.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.175+0000 7f9f87fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f88106790 0x7f9f8819b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:32.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.175+0000 7f9f87fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f88106790 0x7f9f8819b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59206/0 (socket says 172.21.15.78:59206) 2024-08-18T23:52:32.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.175+0000 7f9f87fff640 1 -- 172.21.15.78:0/1097765692 learned_addr learned my addr 172.21.15.78:0/1097765692 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:32.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.175+0000 7f9f86ffd640 1 --2- 172.21.15.78:0/1097765692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f88101130 0x7f9f881975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:32.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.175+0000 7f9f877fe640 1 --2- 172.21.15.78:0/1097765692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f88100790 0x7f9f88197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:32.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.175+0000 7f9f87fff640 1 -- 172.21.15.78:0/1097765692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f88101130 msgr2=0x7f9f881975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:32.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.175+0000 7f9f87fff640 1 --2- 172.21.15.78:0/1097765692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f88101130 0x7f9f881975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.175+0000 7f9f87fff640 1 -- 172.21.15.78:0/1097765692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f88100790 msgr2=0x7f9f88197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:32.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.175+0000 7f9f87fff640 1 --2- 172.21.15.78:0/1097765692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f88100790 0x7f9f88197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.175+0000 7f9f87fff640 1 -- 172.21.15.78:0/1097765692 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8819c020 con 0x7f9f88106790 2024-08-18T23:52:32.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.175+0000 7f9f877fe640 1 --2- 172.21.15.78:0/1097765692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f88100790 0x7f9f88197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:52:32.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.176+0000 7f9f87fff640 1 --2- 172.21.15.78:0/1097765692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f88106790 0x7f9f8819b940 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000cce0 tx=0x7f9f8000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:32.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.176+0000 7f9f84ff9640 1 -- 172.21.15.78:0/1097765692 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8000f450 con 0x7f9f88106790 2024-08-18T23:52:32.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.176+0000 7f9f8e1b8640 1 -- 172.21.15.78:0/1097765692 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8819c310 con 0x7f9f88106790 2024-08-18T23:52:32.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.176+0000 7f9f84ff9640 1 -- 172.21.15.78:0/1097765692 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f8000fad0 con 0x7f9f88106790 2024-08-18T23:52:32.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.176+0000 7f9f84ff9640 1 -- 172.21.15.78:0/1097765692 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80018c10 con 0x7f9f88106790 2024-08-18T23:52:32.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.176+0000 7f9f8e1b8640 1 -- 172.21.15.78:0/1097765692 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f881a3bf0 con 0x7f9f88106790 2024-08-18T23:52:32.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.178+0000 7f9f84ff9640 1 -- 172.21.15.78:0/1097765692 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9f8000f5f0 con 0x7f9f88106790 2024-08-18T23:52:32.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.178+0000 7f9f905b6740 1 Processor -- start 2024-08-18T23:52:32.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.178+0000 7f9f905b6740 1 -- start start 2024-08-18T23:52:32.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.178+0000 7f9f905b6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f8ff0 0x558c2e2f93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:32.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.178+0000 7f9f905b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f9910 0x558c2e303c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:32.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.178+0000 7f9f905b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e3041d0 0x558c2e3065c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:32.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.178+0000 7f9f905b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c2e25ea40 con 0x558c2e2f9910 2024-08-18T23:52:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.178+0000 7f9f905b6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558c2e1f69d0 con 0x558c2e3041d0 2024-08-18T23:52:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.178+0000 7f9f905b6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558c2e25aa00 con 0x558c2e2f8ff0 2024-08-18T23:52:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.179+0000 7f9f877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f9910 0x558c2e303c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.179+0000 7f9f84ff9640 1 --2- 172.21.15.78:0/1097765692 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9f700a2b80 0x7f9f700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.179+0000 7f9f877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f9910 0x558c2e303c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59218/0 (socket says 172.21.15.78:59218) 2024-08-18T23:52:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.179+0000 7f9f877fe640 1 -- 172.21.15.78:0/3528609223 learned_addr learned my addr 172.21.15.78:0/3528609223 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.179+0000 7f9f86ffd640 1 --2- 172.21.15.78:0/3528609223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f8ff0 0x558c2e2f93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.179+0000 7f9f87fff640 1 --2- 172.21.15.78:0/1097765692 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9f700a2b80 0x7f9f700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.179+0000 7f9f86ffd640 1 --2- 172.21.15.78:0/3528609223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e3041d0 0x558c2e3065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.179+0000 7f9f84ff9640 1 -- 172.21.15.78:0/1097765692 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f9f800cfb60 con 0x7f9f88106790 2024-08-18T23:52:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.179+0000 7f9f877fe640 1 -- 172.21.15.78:0/3528609223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f8ff0 msgr2=0x558c2e2f93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.179+0000 7f9f877fe640 1 --2- 172.21.15.78:0/3528609223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f8ff0 0x558c2e2f93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.179+0000 7f9f877fe640 1 -- 172.21.15.78:0/3528609223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e3041d0 msgr2=0x558c2e3065c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.179+0000 7f9f877fe640 1 --2- 172.21.15.78:0/3528609223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e3041d0 0x558c2e3065c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.179+0000 7f9f877fe640 1 -- 172.21.15.78:0/3528609223 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c2e1fd650 con 0x558c2e2f9910 2024-08-18T23:52:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.179+0000 7f9f86ffd640 1 --2- 172.21.15.78:0/3528609223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f8ff0 0x558c2e2f93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.179+0000 7f9f877fe640 1 --2- 172.21.15.78:0/3528609223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f9910 0x558c2e303c90 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009bd0 tx=0x7f9f7800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549b4ad3ec2f7aaa server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.179+0000 7f9f87fff640 1 --2- 172.21.15.78:0/1097765692 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9f700a2b80 0x7f9f700a5040 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9f8009bd30 tx=0x7f9f800670a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.179+0000 7f9f6dffb640 1 -- 172.21.15.78:0/3528609223 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78016070 con 0x558c2e2f9910 2024-08-18T23:52:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.179+0000 7f9f6dffb640 1 -- 172.21.15.78:0/3528609223 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f78005500 con 0x558c2e2f9910 2024-08-18T23:52:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.179+0000 7f9f6dffb640 1 -- 172.21.15.78:0/3528609223 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f780116d0 con 0x558c2e2f9910 2024-08-18T23:52:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.180+0000 7f9f905b6740 1 -- 172.21.15.78:0/3528609223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f9910 msgr2=0x558c2e303c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.180+0000 7f9f905b6740 1 --2- 172.21.15.78:0/3528609223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f9910 0x558c2e303c90 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009bd0 tx=0x7f9f7800e6c0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.180+0000 7f9f905b6740 1 -- 172.21.15.78:0/3528609223 shutdown_connections 2024-08-18T23:52:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.180+0000 7f9f905b6740 1 --2- 172.21.15.78:0/3528609223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e3041d0 0x558c2e3065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.180+0000 7f9f905b6740 1 --2- 172.21.15.78:0/3528609223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f9910 0x558c2e303c90 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.180+0000 7f9f905b6740 1 --2- 172.21.15.78:0/3528609223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f8ff0 0x558c2e2f93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.180+0000 7f9f905b6740 1 -- 172.21.15.78:0/3528609223 >> 172.21.15.78:0/3528609223 conn(0x558c2e2f5580 msgr2=0x558c2e2f5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:32.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.180+0000 7f9f905b6740 1 -- 172.21.15.78:0/3528609223 shutdown_connections 2024-08-18T23:52:32.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.180+0000 7f9f905b6740 1 -- 172.21.15.78:0/3528609223 wait complete. 2024-08-18T23:52:32.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.180+0000 7f9f905b6740 1 -- start start 2024-08-18T23:52:32.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f905b6740 1 Processor -- start 2024-08-18T23:52:32.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f905b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f8ff0 0x558c2e303c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:32.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f905b6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f9910 0x558c2e300fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:32.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f905b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e3041d0 0x558c2e301770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:32.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f905b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c2e306da0 con 0x558c2e2f8ff0 2024-08-18T23:52:32.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f905b6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558c2e219b20 con 0x558c2e3041d0 2024-08-18T23:52:32.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f905b6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558c2e306f20 con 0x558c2e2f9910 2024-08-18T23:52:32.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f905b6740 10 client.0 _ll_register_callbacks cb 0x558c2e217bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:32.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f86ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f8ff0 0x558c2e303c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:32.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f86ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f8ff0 0x558c2e303c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59234/0 (socket says 172.21.15.78:59234) 2024-08-18T23:52:32.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f86ffd640 1 -- 172.21.15.78:0/1622629409 learned_addr learned my addr 172.21.15.78:0/1622629409 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:32.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f86ffd640 1 --2- 172.21.15.78:0/1622629409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e3041d0 0x558c2e301770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:32.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f877fe640 1 --2- 172.21.15.78:0/1622629409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f9910 0x558c2e300fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:32.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f86ffd640 1 -- 172.21.15.78:0/1622629409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f9910 msgr2=0x558c2e300fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:32.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f86ffd640 1 --2- 172.21.15.78:0/1622629409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f9910 0x558c2e300fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f86ffd640 1 -- 172.21.15.78:0/1622629409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e3041d0 msgr2=0x558c2e301770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:32.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f86ffd640 1 --2- 172.21.15.78:0/1622629409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e3041d0 0x558c2e301770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f86ffd640 1 -- 172.21.15.78:0/1622629409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c2dfd03c0 con 0x558c2e2f8ff0 2024-08-18T23:52:32.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f86ffd640 1 --2- 172.21.15.78:0/1622629409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f8ff0 0x558c2e303c90 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c02a940 tx=0x7f9f7c005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa517f1a0fd37d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:32.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.181+0000 7f9f66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:32.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.182+0000 7f9f66ffd640 1 -- 172.21.15.78:0/1622629409 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c035040 con 0x558c2e2f8ff0 2024-08-18T23:52:32.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.182+0000 7f9f66ffd640 1 -- 172.21.15.78:0/1622629409 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f9f7c003800 con 0x558c2e2f8ff0 2024-08-18T23:52:32.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.182+0000 7f9f905b6740 1 -- 172.21.15.78:0/1622629409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f78005500 con 0x558c2e2f8ff0 2024-08-18T23:52:32.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.182+0000 7f9f66ffd640 1 -- 172.21.15.78:0/1622629409 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9f7c003a00 con 0x558c2e2f8ff0 2024-08-18T23:52:32.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.182+0000 7f9f905b6740 10 client.16776 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:32.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.182+0000 7f9f905b6740 1 -- 172.21.15.78:0/1622629409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558c2e3128d0 con 0x558c2e2f8ff0 2024-08-18T23:52:32.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.182+0000 7f9f66ffd640 1 -- 172.21.15.78:0/1622629409 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f78005500 con 0x558c2e2f8ff0 2024-08-18T23:52:32.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.182+0000 7f9f66ffd640 1 -- 172.21.15.78:0/1622629409 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f9f7c04a420 con 0x558c2e2f8ff0 2024-08-18T23:52:32.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.182+0000 7f9f905b6740 10 client.16776 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:32.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.182+0000 7f9f905b6740 10 client.16776 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:52:32.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.182+0000 7f9f905b6740 20 client.16776 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:32.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.182+0000 7f9f905b6740 1 --2- 172.21.15.78:0/1622629409 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x558c2e313d90 0x558c2e316180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:32.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.182+0000 7f9f905b6740 4 client.16776 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:52:32.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.182+0000 7f9f905b6740 1 -- 172.21.15.78:0/1622629409 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c2e26bfb0 con 0x558c2e313d90 2024-08-18T23:52:32.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.183+0000 7f9f877fe640 1 --2- 172.21.15.78:0/1622629409 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x558c2e313d90 0x558c2e316180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:32.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.183+0000 7f9f66ffd640 10 client.16776 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:52:32.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.183+0000 7f9f877fe640 1 --2- 172.21.15.78:0/1622629409 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x558c2e313d90 0x558c2e316180 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4cd217c9b6e85224 server_cookie=a4a78422510c4de9 in_seq=0 out_seq=0 2024-08-18T23:52:32.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.184+0000 7f9f66ffd640 1 -- 172.21.15.78:0/1622629409 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c2e26bfb0 con 0x558c2e313d90 2024-08-18T23:52:32.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.184+0000 7f9f66ffd640 10 client.16776 handle_command_reply: tid=0 2024-08-18T23:52:32.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.184+0000 7f9f905b6740 1 client.16776 shutdown 2024-08-18T23:52:32.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.184+0000 7f9f905b6740 1 -- 172.21.15.78:0/1622629409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f8ff0 msgr2=0x558c2e303c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:32.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.184+0000 7f9f905b6740 1 --2- 172.21.15.78:0/1622629409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f8ff0 0x558c2e303c90 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c02a940 tx=0x7f9f7c005a40 comp rx=0 tx=0).stop 2024-08-18T23:52:32.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.184+0000 7f9f905b6740 1 -- 172.21.15.78:0/1622629409 shutdown_connections 2024-08-18T23:52:32.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.184+0000 7f9f905b6740 1 --2- 172.21.15.78:0/1622629409 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x558c2e313d90 0x558c2e316180 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.184+0000 7f9f905b6740 1 --2- 172.21.15.78:0/1622629409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e3041d0 0x558c2e301770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.184+0000 7f9f905b6740 1 --2- 172.21.15.78:0/1622629409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f9910 0x558c2e300fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.184+0000 7f9f905b6740 1 --2- 172.21.15.78:0/1622629409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f8ff0 0x558c2e303c90 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.184+0000 7f9f905b6740 1 -- 172.21.15.78:0/1622629409 >> 172.21.15.78:0/1622629409 conn(0x558c2e2f5580 msgr2=0x558c2e2fb260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:32.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.184+0000 7f9f66ffd640 0 client.16776 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:52:32.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.184+0000 7f9f905b6740 1 -- 172.21.15.78:0/1622629409 shutdown_connections 2024-08-18T23:52:32.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.184+0000 7f9f905b6740 1 -- 172.21.15.78:0/1622629409 wait complete. 2024-08-18T23:52:32.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.184+0000 7f9f905b6740 20 client.16776 trim_cache size 0 max 16384 2024-08-18T23:52:32.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.197+0000 7f9f905b6740 1 Processor -- start 2024-08-18T23:52:32.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.197+0000 7f9f905b6740 1 -- start start 2024-08-18T23:52:32.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f905b6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f8ff0 0x558c2e2ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:32.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f905b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e2f9910 0x558c2e2ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:32.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f905b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e3041d0 0x558c2e31ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:32.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f905b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881af0e0 con 0x558c2e3041d0 2024-08-18T23:52:32.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f905b6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881ac950 con 0x558c2e2f9910 2024-08-18T23:52:32.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f905b6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881aa7a0 con 0x558c2e2f8ff0 2024-08-18T23:52:32.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f86ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e3041d0 0x558c2e31ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:32.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f86ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e3041d0 0x558c2e31ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59250/0 (socket says 172.21.15.78:59250) 2024-08-18T23:52:32.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f86ffd640 1 -- 172.21.15.78:0/534720122 learned_addr learned my addr 172.21.15.78:0/534720122 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:32.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f86ffd640 1 --2- 172.21.15.78:0/534720122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f8ff0 0x558c2e2ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:32.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f877fe640 1 --2- 172.21.15.78:0/534720122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e2f9910 0x558c2e2ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:32.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f86ffd640 1 -- 172.21.15.78:0/534720122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f8ff0 msgr2=0x558c2e2ff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:32.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f86ffd640 1 --2- 172.21.15.78:0/534720122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f8ff0 0x558c2e2ff8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f86ffd640 1 -- 172.21.15.78:0/534720122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e2f9910 msgr2=0x558c2e2ffdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:32.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f86ffd640 1 --2- 172.21.15.78:0/534720122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e2f9910 0x558c2e2ffdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f86ffd640 1 -- 172.21.15.78:0/534720122 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c2e301ae0 con 0x558c2e3041d0 2024-08-18T23:52:32.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f86ffd640 1 --2- 172.21.15.78:0/534720122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e3041d0 0x558c2e31ba10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c04a9b0 tx=0x7f9f7c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756b90fbfc4a9d88 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:32.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f6d7fa640 1 -- 172.21.15.78:0/534720122 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c035070 con 0x558c2e3041d0 2024-08-18T23:52:32.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f6d7fa640 1 -- 172.21.15.78:0/534720122 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f7c04a420 con 0x558c2e3041d0 2024-08-18T23:52:32.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.198+0000 7f9f6d7fa640 1 -- 172.21.15.78:0/534720122 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c0053a0 con 0x558c2e3041d0 2024-08-18T23:52:32.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.199+0000 7f9f905b6740 1 -- 172.21.15.78:0/534720122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e3041d0 msgr2=0x558c2e31ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:32.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.199+0000 7f9f905b6740 1 --2- 172.21.15.78:0/534720122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e3041d0 0x558c2e31ba10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c04a9b0 tx=0x7f9f7c02a420 comp rx=0 tx=0).stop 2024-08-18T23:52:32.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.199+0000 7f9f905b6740 1 -- 172.21.15.78:0/534720122 shutdown_connections 2024-08-18T23:52:32.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.199+0000 7f9f905b6740 1 --2- 172.21.15.78:0/534720122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e3041d0 0x558c2e31ba10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.199+0000 7f9f905b6740 1 --2- 172.21.15.78:0/534720122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e2f9910 0x558c2e2ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.199+0000 7f9f905b6740 1 --2- 172.21.15.78:0/534720122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f8ff0 0x558c2e2ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.199+0000 7f9f905b6740 1 -- 172.21.15.78:0/534720122 >> 172.21.15.78:0/534720122 conn(0x558c2e2f5580 msgr2=0x558c2e2fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:32.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.199+0000 7f9f905b6740 1 -- 172.21.15.78:0/534720122 shutdown_connections 2024-08-18T23:52:32.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.199+0000 7f9f905b6740 1 -- 172.21.15.78:0/534720122 wait complete. 2024-08-18T23:52:32.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.199+0000 7f9f905b6740 1 -- start start 2024-08-18T23:52:32.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.200+0000 7f9f905b6740 1 Processor -- start 2024-08-18T23:52:32.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.200+0000 7f9f905b6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f8ff0 0x558c2e30c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:32.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.200+0000 7f9f905b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f9910 0x558c2e304df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:32.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.200+0000 7f9f905b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e3041d0 0x558c2e305330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:32.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.200+0000 7f9f905b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881aae10 con 0x558c2e2f9910 2024-08-18T23:52:32.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.200+0000 7f9f905b6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881ac280 con 0x558c2e3041d0 2024-08-18T23:52:32.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.200+0000 7f9f905b6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881ab4e0 con 0x558c2e2f8ff0 2024-08-18T23:52:32.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.200+0000 7f9f905b6740 10 client.0 _ll_register_callbacks cb 0x7f9f7c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:32.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.200+0000 7f9f877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f9910 0x558c2e304df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:32.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.200+0000 7f9f877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f9910 0x558c2e304df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59264/0 (socket says 172.21.15.78:59264) 2024-08-18T23:52:32.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.200+0000 7f9f877fe640 1 -- 172.21.15.78:0/1147498323 learned_addr learned my addr 172.21.15.78:0/1147498323 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:32.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.200+0000 7f9f86ffd640 1 --2- 172.21.15.78:0/1147498323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f8ff0 0x558c2e30c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:32.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.200+0000 7f9f877fe640 1 -- 172.21.15.78:0/1147498323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f8ff0 msgr2=0x558c2e30c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:32.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.200+0000 7f9f86ffd640 1 --2- 172.21.15.78:0/1147498323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e3041d0 0x558c2e305330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:32.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.200+0000 7f9f877fe640 1 --2- 172.21.15.78:0/1147498323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f8ff0 0x558c2e30c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.200+0000 7f9f877fe640 1 -- 172.21.15.78:0/1147498323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e3041d0 msgr2=0x558c2e305330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:32.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.200+0000 7f9f877fe640 1 --2- 172.21.15.78:0/1147498323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e3041d0 0x558c2e305330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.200+0000 7f9f877fe640 1 -- 172.21.15.78:0/1147498323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c2e305970 con 0x558c2e2f9910 2024-08-18T23:52:32.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.200+0000 7f9f86ffd640 1 --2- 172.21.15.78:0/1147498323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e3041d0 0x558c2e305330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:52:32.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.201+0000 7f9f877fe640 1 --2- 172.21.15.78:0/1147498323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f9910 0x558c2e304df0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f9f78004c00 tx=0x7f9f78009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b579e13e5380dee8 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:32.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.201+0000 7f9f66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:32.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.201+0000 7f9f66ffd640 1 -- 172.21.15.78:0/1147498323 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78016040 con 0x558c2e2f9910 2024-08-18T23:52:32.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.201+0000 7f9f905b6740 1 -- 172.21.15.78:0/1147498323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f7c04a420 con 0x558c2e2f9910 2024-08-18T23:52:32.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.201+0000 7f9f66ffd640 1 -- 172.21.15.78:0/1147498323 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f9f78004880 con 0x558c2e2f9910 2024-08-18T23:52:32.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.201+0000 7f9f66ffd640 1 -- 172.21.15.78:0/1147498323 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9f78034050 con 0x558c2e2f9910 2024-08-18T23:52:32.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.201+0000 7f9f905b6740 10 client.16782 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:32.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.201+0000 7f9f905b6740 1 -- 172.21.15.78:0/1147498323 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558c2e30afe0 con 0x558c2e2f9910 2024-08-18T23:52:32.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.201+0000 7f9f66ffd640 1 -- 172.21.15.78:0/1147498323 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f7c04a420 con 0x558c2e2f9910 2024-08-18T23:52:32.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.201+0000 7f9f66ffd640 1 -- 172.21.15.78:0/1147498323 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f9f7802e020 con 0x558c2e2f9910 2024-08-18T23:52:32.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.201+0000 7f9f905b6740 10 client.16782 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:32.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.201+0000 7f9f905b6740 10 client.16782 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:52:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.201+0000 7f9f905b6740 20 client.16782 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.201+0000 7f9f905b6740 1 --2- 172.21.15.78:0/1147498323 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x558c2e37ed40 0x558c2e381130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.201+0000 7f9f905b6740 4 client.16782 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:52:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.201+0000 7f9f905b6740 1 -- 172.21.15.78:0/1147498323 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558c2e2664c0 con 0x558c2e37ed40 2024-08-18T23:52:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.202+0000 7f9f86ffd640 1 --2- 172.21.15.78:0/1147498323 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x558c2e37ed40 0x558c2e381130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.202+0000 7f9f66ffd640 10 client.16782 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:52:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.202+0000 7f9f86ffd640 1 --2- 172.21.15.78:0/1147498323 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x558c2e37ed40 0x558c2e381130 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7c3f90cf717bd22c server_cookie=925c75261eaeb0a8 in_seq=0 out_seq=0 2024-08-18T23:52:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.202+0000 7f9f66ffd640 1 -- 172.21.15.78:0/1147498323 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558c2e2664c0 con 0x558c2e37ed40 2024-08-18T23:52:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.202+0000 7f9f66ffd640 10 client.16782 handle_command_reply: tid=0 2024-08-18T23:52:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.202+0000 7f9f905b6740 1 client.16782 shutdown 2024-08-18T23:52:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.202+0000 7f9f905b6740 1 -- 172.21.15.78:0/1147498323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f9910 msgr2=0x558c2e304df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.202+0000 7f9f905b6740 1 --2- 172.21.15.78:0/1147498323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f9910 0x558c2e304df0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f9f78004c00 tx=0x7f9f78009380 comp rx=0 tx=0).stop 2024-08-18T23:52:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.202+0000 7f9f905b6740 1 -- 172.21.15.78:0/1147498323 shutdown_connections 2024-08-18T23:52:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.202+0000 7f9f905b6740 1 --2- 172.21.15.78:0/1147498323 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x558c2e37ed40 0x558c2e381130 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.202+0000 7f9f905b6740 1 --2- 172.21.15.78:0/1147498323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c2e3041d0 0x558c2e305330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.203+0000 7f9f905b6740 1 --2- 172.21.15.78:0/1147498323 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c2e2f9910 0x558c2e304df0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.203+0000 7f9f905b6740 1 --2- 172.21.15.78:0/1147498323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558c2e2f8ff0 0x558c2e30c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.203+0000 7f9f905b6740 1 -- 172.21.15.78:0/1147498323 >> 172.21.15.78:0/1147498323 conn(0x558c2e2f5580 msgr2=0x558c2e2ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.203+0000 7f9f66ffd640 0 client.16782 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:52:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.203+0000 7f9f905b6740 1 -- 172.21.15.78:0/1147498323 shutdown_connections 2024-08-18T23:52:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.203+0000 7f9f905b6740 1 -- 172.21.15.78:0/1147498323 wait complete. 2024-08-18T23:52:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.203+0000 7f9f905b6740 20 client.16782 trim_cache size 0 max 16384 2024-08-18T23:52:32.215 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:52:32.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.203+0000 7f9f8e1b8640 1 -- 172.21.15.78:0/1097765692 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9f700a2b80 msgr2=0x7f9f700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:32.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.203+0000 7f9f8e1b8640 1 --2- 172.21.15.78:0/1097765692 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9f700a2b80 0x7f9f700a5040 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9f8009bd30 tx=0x7f9f800670a0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.203+0000 7f9f8e1b8640 1 -- 172.21.15.78:0/1097765692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f88106790 msgr2=0x7f9f8819b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:32.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.203+0000 7f9f8e1b8640 1 --2- 172.21.15.78:0/1097765692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f88106790 0x7f9f8819b940 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000cce0 tx=0x7f9f8000a620 comp rx=0 tx=0).stop 2024-08-18T23:52:32.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.203+0000 7f9f8e1b8640 1 -- 172.21.15.78:0/1097765692 shutdown_connections 2024-08-18T23:52:32.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.203+0000 7f9f8e1b8640 1 --2- 172.21.15.78:0/1097765692 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f9f700a2b80 0x7f9f700a5040 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.203+0000 7f9f8e1b8640 1 --2- 172.21.15.78:0/1097765692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f88106790 0x7f9f8819b940 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.203+0000 7f9f8e1b8640 1 --2- 172.21.15.78:0/1097765692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f88101130 0x7f9f881975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.203+0000 7f9f8e1b8640 1 --2- 172.21.15.78:0/1097765692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f88100790 0x7f9f88197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:32.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.203+0000 7f9f8e1b8640 1 -- 172.21.15.78:0/1097765692 >> 172.21.15.78:0/1097765692 conn(0x7f9f880fc4b0 msgr2=0x7f9f880fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:32.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.203+0000 7f9f8e1b8640 1 -- 172.21.15.78:0/1097765692 shutdown_connections 2024-08-18T23:52:32.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:32.203+0000 7f9f8e1b8640 1 -- 172.21.15.78:0/1097765692 wait complete. 2024-08-18T23:52:32.930 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-18T23:52:33.180 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:33.604 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:33 smithi078 ceph-mon[18440]: pgmap v597: 129 pgs: 129 active+clean; 255 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 823 KiB/s wr, 532 op/s 2024-08-18T23:52:33.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:33 smithi161 ceph-mon[22027]: pgmap v597: 129 pgs: 129 active+clean; 255 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 823 KiB/s wr, 532 op/s 2024-08-18T23:52:33.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.683+0000 7f94bd91b640 1 -- 172.21.15.78:0/3160011978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b8103490 msgr2=0x7f94b81078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:33.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.683+0000 7f94bd91b640 1 --2- 172.21.15.78:0/3160011978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b8103490 0x7f94b81078d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009930 tx=0x7f94a002a420 comp rx=0 tx=0).stop 2024-08-18T23:52:33.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.683+0000 7f94bd91b640 1 -- 172.21.15.78:0/3160011978 shutdown_connections 2024-08-18T23:52:33.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.683+0000 7f94bd91b640 1 --2- 172.21.15.78:0/3160011978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94b8108010 0x7f94b810e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.683+0000 7f94bd91b640 1 --2- 172.21.15.78:0/3160011978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b8103490 0x7f94b81078d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.683+0000 7f94bd91b640 1 --2- 172.21.15.78:0/3160011978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94b8102ae0 0x7f94b8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.683+0000 7f94bd91b640 1 -- 172.21.15.78:0/3160011978 >> 172.21.15.78:0/3160011978 conn(0x7f94b80fc500 msgr2=0x7f94b80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:33.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.683+0000 7f94bd91b640 1 -- 172.21.15.78:0/3160011978 shutdown_connections 2024-08-18T23:52:33.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.683+0000 7f94bd91b640 1 -- 172.21.15.78:0/3160011978 wait complete. 2024-08-18T23:52:33.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.684+0000 7f94bd91b640 1 Processor -- start 2024-08-18T23:52:33.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.684+0000 7f94bd91b640 1 -- start start 2024-08-18T23:52:33.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.684+0000 7f94bd91b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94b8102ae0 0x7f94b8100740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:33.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.684+0000 7f94bd91b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94b8103490 0x7f94b8100c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:33.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.684+0000 7f94bd91b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b8108010 0x7f94b81011c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:33.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.684+0000 7f94bd91b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94b8111de0 con 0x7f94b8108010 2024-08-18T23:52:33.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.684+0000 7f94bd91b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94b8111c60 con 0x7f94b8102ae0 2024-08-18T23:52:33.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.684+0000 7f94bd91b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f94b8111f60 con 0x7f94b8103490 2024-08-18T23:52:33.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.685+0000 7f94b77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b8108010 0x7f94b81011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:33.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.685+0000 7f94b77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b8108010 0x7f94b81011c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59290/0 (socket says 172.21.15.78:59290) 2024-08-18T23:52:33.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.685+0000 7f94b77fe640 1 -- 172.21.15.78:0/3719466750 learned_addr learned my addr 172.21.15.78:0/3719466750 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:33.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.685+0000 7f94b67fc640 1 --2- 172.21.15.78:0/3719466750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94b8103490 0x7f94b8100c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:33.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.685+0000 7f94b6ffd640 1 --2- 172.21.15.78:0/3719466750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94b8102ae0 0x7f94b8100740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:33.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.685+0000 7f94b77fe640 1 -- 172.21.15.78:0/3719466750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94b8103490 msgr2=0x7f94b8100c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:33.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.685+0000 7f94b77fe640 1 --2- 172.21.15.78:0/3719466750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94b8103490 0x7f94b8100c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.685+0000 7f94b77fe640 1 -- 172.21.15.78:0/3719466750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94b8102ae0 msgr2=0x7f94b8100740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:33.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.685+0000 7f94b77fe640 1 --2- 172.21.15.78:0/3719466750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94b8102ae0 0x7f94b8100740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.685+0000 7f94b77fe640 1 -- 172.21.15.78:0/3719466750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b81a6750 con 0x7f94b8108010 2024-08-18T23:52:33.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.685+0000 7f94b6ffd640 1 --2- 172.21.15.78:0/3719466750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94b8102ae0 0x7f94b8100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:33.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.685+0000 7f94b77fe640 1 --2- 172.21.15.78:0/3719466750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b8108010 0x7f94b81011c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f94b000e970 tx=0x7f94b000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:33.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.685+0000 7f94bc919640 1 -- 172.21.15.78:0/3719466750 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b000ccc0 con 0x7f94b8108010 2024-08-18T23:52:33.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.685+0000 7f94bd91b640 1 -- 172.21.15.78:0/3719466750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b81a6a40 con 0x7f94b8108010 2024-08-18T23:52:33.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.685+0000 7f94bc919640 1 -- 172.21.15.78:0/3719466750 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94b000ce60 con 0x7f94b8108010 2024-08-18T23:52:33.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.685+0000 7f94bc919640 1 -- 172.21.15.78:0/3719466750 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0010450 con 0x7f94b8108010 2024-08-18T23:52:33.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.685+0000 7f94bd91b640 1 -- 172.21.15.78:0/3719466750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b81a6f50 con 0x7f94b8108010 2024-08-18T23:52:33.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.687+0000 7f94bc919640 1 -- 172.21.15.78:0/3719466750 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f94b0005020 con 0x7f94b8108010 2024-08-18T23:52:33.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.688+0000 7f94bfd19740 1 Processor -- start 2024-08-18T23:52:33.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.688+0000 7f94bfd19740 1 -- start start 2024-08-18T23:52:33.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.688+0000 7f94bfd19740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4069e2ff0 0x55f4069e33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:33.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.688+0000 7f94bfd19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e3910 0x55f4069edc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:33.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.688+0000 7f94bfd19740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069ee1d0 0x55f4069f05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:33.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.688+0000 7f94bfd19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f406948a40 con 0x55f4069e3910 2024-08-18T23:52:33.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.688+0000 7f94bfd19740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f4068e09d0 con 0x55f4069ee1d0 2024-08-18T23:52:33.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.688+0000 7f94bfd19740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f406944a00 con 0x55f4069e2ff0 2024-08-18T23:52:33.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.688+0000 7f94bc919640 1 --2- 172.21.15.78:0/3719466750 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94a40a2b80 0x7f94a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:33.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.688+0000 7f94b6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e3910 0x55f4069edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:33.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.688+0000 7f94b6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e3910 0x55f4069edc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59298/0 (socket says 172.21.15.78:59298) 2024-08-18T23:52:33.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.688+0000 7f94b6ffd640 1 -- 172.21.15.78:0/1698234578 learned_addr learned my addr 172.21.15.78:0/1698234578 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:33.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.688+0000 7f94b67fc640 1 --2- 172.21.15.78:0/1698234578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069ee1d0 0x55f4069f05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:33.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.688+0000 7f94b77fe640 1 --2- 172.21.15.78:0/3719466750 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94a40a2b80 0x7f94a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:33.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.688+0000 7f94b67fc640 1 --2- 172.21.15.78:0/1698234578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4069e2ff0 0x55f4069e33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:33.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.688+0000 7f94bc919640 1 -- 172.21.15.78:0/3719466750 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f94b00cf890 con 0x7f94b8108010 2024-08-18T23:52:33.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.689+0000 7f94b6ffd640 1 -- 172.21.15.78:0/1698234578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4069e2ff0 msgr2=0x55f4069e33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.689+0000 7f94b6ffd640 1 --2- 172.21.15.78:0/1698234578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4069e2ff0 0x55f4069e33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.689+0000 7f94b6ffd640 1 -- 172.21.15.78:0/1698234578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069ee1d0 msgr2=0x55f4069f05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.689+0000 7f94b6ffd640 1 --2- 172.21.15.78:0/1698234578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069ee1d0 0x55f4069f05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.689+0000 7f94b6ffd640 1 -- 172.21.15.78:0/1698234578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4068e7650 con 0x55f4069e3910 2024-08-18T23:52:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.689+0000 7f94b67fc640 1 --2- 172.21.15.78:0/1698234578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069ee1d0 0x55f4069f05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.689+0000 7f94b6ffd640 1 --2- 172.21.15.78:0/1698234578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e3910 0x55f4069edc90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f94a8007f70 tx=0x7f94a800e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae238d3b078ec3de server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.689+0000 7f94b77fe640 1 --2- 172.21.15.78:0/3719466750 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94a40a2b80 0x7f94a40a5040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f94b00986d0 tx=0x7f94b0066d70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.689+0000 7f94adffb640 1 -- 172.21.15.78:0/1698234578 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8014070 con 0x55f4069e3910 2024-08-18T23:52:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.689+0000 7f94adffb640 1 -- 172.21.15.78:0/1698234578 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a8004d20 con 0x55f4069e3910 2024-08-18T23:52:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.689+0000 7f94adffb640 1 -- 172.21.15.78:0/1698234578 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8005540 con 0x55f4069e3910 2024-08-18T23:52:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.690+0000 7f94bfd19740 1 -- 172.21.15.78:0/1698234578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e3910 msgr2=0x55f4069edc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.690+0000 7f94bfd19740 1 --2- 172.21.15.78:0/1698234578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e3910 0x55f4069edc90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f94a8007f70 tx=0x7f94a800e7e0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.690+0000 7f94bfd19740 1 -- 172.21.15.78:0/1698234578 shutdown_connections 2024-08-18T23:52:33.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.690+0000 7f94bfd19740 1 --2- 172.21.15.78:0/1698234578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069ee1d0 0x55f4069f05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.690+0000 7f94bfd19740 1 --2- 172.21.15.78:0/1698234578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e3910 0x55f4069edc90 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.690+0000 7f94bfd19740 1 --2- 172.21.15.78:0/1698234578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4069e2ff0 0x55f4069e33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.690+0000 7f94bfd19740 1 -- 172.21.15.78:0/1698234578 >> 172.21.15.78:0/1698234578 conn(0x55f4069df580 msgr2=0x55f4069df960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:33.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.690+0000 7f94bfd19740 1 -- 172.21.15.78:0/1698234578 shutdown_connections 2024-08-18T23:52:33.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.690+0000 7f94bfd19740 1 -- 172.21.15.78:0/1698234578 wait complete. 2024-08-18T23:52:33.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.690+0000 7f94bfd19740 1 -- start start 2024-08-18T23:52:33.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.690+0000 7f94bfd19740 1 Processor -- start 2024-08-18T23:52:33.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f94bfd19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e2ff0 0x55f4069eb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:33.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f94bfd19740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069e3910 0x55f4069eb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:33.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f94bfd19740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4069ee1d0 0x55f4069ebab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:33.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f94bfd19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f4069f0da0 con 0x55f4069e2ff0 2024-08-18T23:52:33.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f94bfd19740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f406903b20 con 0x55f4069e3910 2024-08-18T23:52:33.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f94bfd19740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f4069f0f20 con 0x55f4069ee1d0 2024-08-18T23:52:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f94bfd19740 10 client.0 _ll_register_callbacks cb 0x55f406901bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f94b67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e2ff0 0x55f4069eb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f94b67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e2ff0 0x55f4069eb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59314/0 (socket says 172.21.15.78:59314) 2024-08-18T23:52:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f94b67fc640 1 -- 172.21.15.78:0/4115401299 learned_addr learned my addr 172.21.15.78:0/4115401299 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f94b6ffd640 1 --2- 172.21.15.78:0/4115401299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069e3910 0x55f4069eb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f94b67fc640 1 -- 172.21.15.78:0/4115401299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4069ee1d0 msgr2=0x55f4069ebab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-18T23:52:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f94b67fc640 1 --2- 172.21.15.78:0/4115401299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4069ee1d0 0x55f4069ebab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f94b67fc640 1 -- 172.21.15.78:0/4115401299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069e3910 msgr2=0x55f4069eb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f94b67fc640 1 --2- 172.21.15.78:0/4115401299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069e3910 0x55f4069eb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f94b67fc640 1 -- 172.21.15.78:0/4115401299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4066ba3c0 con 0x55f4069e2ff0 2024-08-18T23:52:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f94b67fc640 1 --2- 172.21.15.78:0/4115401299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e2ff0 0x55f4069eb030 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f94b8069500 tx=0x7f94a0003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12e02d362c220a2 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f9496ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:33.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f9496ffd640 1 -- 172.21.15.78:0/4115401299 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0035030 con 0x55f4069e2ff0 2024-08-18T23:52:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f9496ffd640 1 -- 172.21.15.78:0/4115401299 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f94a0003b10 con 0x55f4069e2ff0 2024-08-18T23:52:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.691+0000 7f94bfd19740 1 -- 172.21.15.78:0/4115401299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94a8004d20 con 0x55f4069e2ff0 2024-08-18T23:52:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.692+0000 7f9496ffd640 1 -- 172.21.15.78:0/4115401299 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f94a0051050 con 0x55f4069e2ff0 2024-08-18T23:52:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.692+0000 7f94bfd19740 10 client.16794 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.692+0000 7f94bfd19740 1 -- 172.21.15.78:0/4115401299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f4069fc930 con 0x55f4069e2ff0 2024-08-18T23:52:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.692+0000 7f9496ffd640 1 -- 172.21.15.78:0/4115401299 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a8004d20 con 0x55f4069e2ff0 2024-08-18T23:52:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.692+0000 7f9496ffd640 1 -- 172.21.15.78:0/4115401299 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f94a003c470 con 0x55f4069e2ff0 2024-08-18T23:52:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.692+0000 7f94bfd19740 10 client.16794 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.692+0000 7f94bfd19740 10 client.16794 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:52:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.692+0000 7f94bfd19740 20 client.16794 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.692+0000 7f94bfd19740 1 --2- 172.21.15.78:0/4115401299 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55f4069fddf0 0x55f406a001e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.692+0000 7f94bfd19740 4 client.16794 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:52:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.692+0000 7f94bfd19740 1 -- 172.21.15.78:0/4115401299 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f406955fb0 con 0x55f4069fddf0 2024-08-18T23:52:33.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.692+0000 7f94b6ffd640 1 --2- 172.21.15.78:0/4115401299 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55f4069fddf0 0x55f406a001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:33.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.693+0000 7f9496ffd640 10 client.16794 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:52:33.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.693+0000 7f94b6ffd640 1 --2- 172.21.15.78:0/4115401299 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55f4069fddf0 0x55f406a001e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0cfd795976927fb server_cookie=ab646a03ed496e27 in_seq=0 out_seq=0 2024-08-18T23:52:33.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.693+0000 7f9496ffd640 1 -- 172.21.15.78:0/4115401299 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f406955fb0 con 0x55f4069fddf0 2024-08-18T23:52:33.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.693+0000 7f9496ffd640 10 client.16794 handle_command_reply: tid=0 2024-08-18T23:52:33.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.694+0000 7f94bfd19740 1 client.16794 shutdown 2024-08-18T23:52:33.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.694+0000 7f94bfd19740 1 -- 172.21.15.78:0/4115401299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e2ff0 msgr2=0x55f4069eb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:33.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.694+0000 7f94bfd19740 1 --2- 172.21.15.78:0/4115401299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e2ff0 0x55f4069eb030 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f94b8069500 tx=0x7f94a0003820 comp rx=0 tx=0).stop 2024-08-18T23:52:33.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.694+0000 7f94bfd19740 1 -- 172.21.15.78:0/4115401299 shutdown_connections 2024-08-18T23:52:33.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.694+0000 7f94bfd19740 1 --2- 172.21.15.78:0/4115401299 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55f4069fddf0 0x55f406a001e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.694+0000 7f94bfd19740 1 --2- 172.21.15.78:0/4115401299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4069ee1d0 0x55f4069ebab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.694+0000 7f94bfd19740 1 --2- 172.21.15.78:0/4115401299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069e3910 0x55f4069eb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.694+0000 7f9496ffd640 0 client.16794 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:52:33.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.694+0000 7f94bfd19740 1 --2- 172.21.15.78:0/4115401299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e2ff0 0x55f4069eb030 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.694+0000 7f94bfd19740 1 -- 172.21.15.78:0/4115401299 >> 172.21.15.78:0/4115401299 conn(0x55f4069df580 msgr2=0x55f4069e5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:33.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.694+0000 7f94bfd19740 1 -- 172.21.15.78:0/4115401299 shutdown_connections 2024-08-18T23:52:33.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.694+0000 7f94bfd19740 1 -- 172.21.15.78:0/4115401299 wait complete. 2024-08-18T23:52:33.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.694+0000 7f94bfd19740 20 client.16794 trim_cache size 0 max 16384 2024-08-18T23:52:33.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.707+0000 7f94bfd19740 1 Processor -- start 2024-08-18T23:52:33.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.707+0000 7f94bfd19740 1 -- start start 2024-08-18T23:52:33.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.707+0000 7f94bfd19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e2ff0 0x55f4069e5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:33.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.707+0000 7f94bfd19740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069e3910 0x55f406a05a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:33.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.707+0000 7f94bfd19740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4069ee1d0 0x55f406a05fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:33.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.707+0000 7f94bfd19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94b81b3590 con 0x55f4069e2ff0 2024-08-18T23:52:33.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.707+0000 7f94bfd19740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94b81b0e00 con 0x55f4069e3910 2024-08-18T23:52:33.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.707+0000 7f94bfd19740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f94b81aec50 con 0x55f4069ee1d0 2024-08-18T23:52:33.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.707+0000 7f94b67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e2ff0 0x55f4069e5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:33.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.707+0000 7f94b67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e2ff0 0x55f4069e5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59330/0 (socket says 172.21.15.78:59330) 2024-08-18T23:52:33.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.707+0000 7f94b67fc640 1 -- 172.21.15.78:0/437047330 learned_addr learned my addr 172.21.15.78:0/437047330 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:33.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.707+0000 7f94b67fc640 1 -- 172.21.15.78:0/437047330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4069ee1d0 msgr2=0x55f406a05fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:33.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.707+0000 7f94b67fc640 1 --2- 172.21.15.78:0/437047330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4069ee1d0 0x55f406a05fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.707+0000 7f94b67fc640 1 -- 172.21.15.78:0/437047330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069e3910 msgr2=0x55f406a05a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:33.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.707+0000 7f94b6ffd640 1 --2- 172.21.15.78:0/437047330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069e3910 0x55f406a05a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:33.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.707+0000 7f94b67fc640 1 --2- 172.21.15.78:0/437047330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069e3910 0x55f406a05a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.707+0000 7f94b67fc640 1 -- 172.21.15.78:0/437047330 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4069e9580 con 0x55f4069e2ff0 2024-08-18T23:52:33.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.708+0000 7f94b67fc640 1 --2- 172.21.15.78:0/437047330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e2ff0 0x55f4069e5a50 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f94a00096a0 tx=0x7f94a0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ba91b4807f3cae server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:33.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.708+0000 7f94ad7fa640 1 -- 172.21.15.78:0/437047330 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0035070 con 0x55f4069e2ff0 2024-08-18T23:52:33.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.708+0000 7f94ad7fa640 1 -- 172.21.15.78:0/437047330 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a0043070 con 0x55f4069e2ff0 2024-08-18T23:52:33.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.708+0000 7f94ad7fa640 1 -- 172.21.15.78:0/437047330 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0005290 con 0x55f4069e2ff0 2024-08-18T23:52:33.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.708+0000 7f94bfd19740 1 -- 172.21.15.78:0/437047330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e2ff0 msgr2=0x55f4069e5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:33.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.708+0000 7f94bfd19740 1 --2- 172.21.15.78:0/437047330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e2ff0 0x55f4069e5a50 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f94a00096a0 tx=0x7f94a0005d50 comp rx=0 tx=0).stop 2024-08-18T23:52:33.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.708+0000 7f94bfd19740 1 -- 172.21.15.78:0/437047330 shutdown_connections 2024-08-18T23:52:33.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.708+0000 7f94bfd19740 1 --2- 172.21.15.78:0/437047330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4069ee1d0 0x55f406a05fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.708+0000 7f94bfd19740 1 --2- 172.21.15.78:0/437047330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069e3910 0x55f406a05a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.708+0000 7f94bfd19740 1 --2- 172.21.15.78:0/437047330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069e2ff0 0x55f4069e5a50 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.708+0000 7f94bfd19740 1 -- 172.21.15.78:0/437047330 >> 172.21.15.78:0/437047330 conn(0x55f4069df580 msgr2=0x55f4069e9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:33.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.708+0000 7f94bfd19740 1 -- 172.21.15.78:0/437047330 shutdown_connections 2024-08-18T23:52:33.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.708+0000 7f94bfd19740 1 -- 172.21.15.78:0/437047330 wait complete. 2024-08-18T23:52:33.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.709+0000 7f94bfd19740 1 -- start start 2024-08-18T23:52:33.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.709+0000 7f94bfd19740 1 Processor -- start 2024-08-18T23:52:33.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.709+0000 7f94bfd19740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069e2ff0 0x55f4069f6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:33.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.709+0000 7f94bfd19740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4069e3910 0x55f4069f6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:33.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.709+0000 7f94bfd19740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069ee1d0 0x55f4069f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:33.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.709+0000 7f94bfd19740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94b81af2c0 con 0x55f4069ee1d0 2024-08-18T23:52:33.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.709+0000 7f94bfd19740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94b81b0730 con 0x55f4069e2ff0 2024-08-18T23:52:33.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.709+0000 7f94bfd19740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f94b81af990 con 0x55f4069e3910 2024-08-18T23:52:33.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.709+0000 7f94bfd19740 10 client.0 _ll_register_callbacks cb 0x7f94a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:33.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.709+0000 7f94b67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069ee1d0 0x55f4069f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:33.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.709+0000 7f94b67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069ee1d0 0x55f4069f16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59332/0 (socket says 172.21.15.78:59332) 2024-08-18T23:52:33.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.709+0000 7f94b67fc640 1 -- 172.21.15.78:0/3388951586 learned_addr learned my addr 172.21.15.78:0/3388951586 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:33.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.710+0000 7f94b6ffd640 1 --2- 172.21.15.78:0/3388951586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4069e3910 0x55f4069f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:33.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.710+0000 7f94b67fc640 1 --2- 172.21.15.78:0/3388951586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069e2ff0 0x55f4069f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:33.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.710+0000 7f94b67fc640 1 -- 172.21.15.78:0/3388951586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4069e3910 msgr2=0x55f4069f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:33.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.710+0000 7f94b67fc640 1 --2- 172.21.15.78:0/3388951586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4069e3910 0x55f4069f6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.710+0000 7f94b67fc640 1 -- 172.21.15.78:0/3388951586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069e2ff0 msgr2=0x55f4069f6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:33.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.710+0000 7f94b67fc640 1 --2- 172.21.15.78:0/3388951586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069e2ff0 0x55f4069f6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.710+0000 7f94b67fc640 1 -- 172.21.15.78:0/3388951586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4069f1ed0 con 0x55f4069ee1d0 2024-08-18T23:52:33.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.710+0000 7f94b6ffd640 1 --2- 172.21.15.78:0/3388951586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4069e3910 0x55f4069f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:33.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.710+0000 7f94b67fc640 1 --2- 172.21.15.78:0/3388951586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069ee1d0 0x55f4069f16d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f94a003c7c0 tx=0x7f94a003cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e8cacbed137497 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:33.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.710+0000 7f9496ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:33.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.710+0000 7f9496ffd640 1 -- 172.21.15.78:0/3388951586 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0035070 con 0x55f4069ee1d0 2024-08-18T23:52:33.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.710+0000 7f9496ffd640 1 -- 172.21.15.78:0/3388951586 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f94a0031540 con 0x55f4069ee1d0 2024-08-18T23:52:33.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.710+0000 7f94bfd19740 1 -- 172.21.15.78:0/3388951586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94a0043070 con 0x55f4069ee1d0 2024-08-18T23:52:33.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.710+0000 7f9496ffd640 1 -- 172.21.15.78:0/3388951586 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f94a0031740 con 0x55f4069ee1d0 2024-08-18T23:52:33.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.710+0000 7f94bfd19740 10 client.16800 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:33.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.710+0000 7f94bfd19740 1 -- 172.21.15.78:0/3388951586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f406a68540 con 0x55f4069ee1d0 2024-08-18T23:52:33.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.710+0000 7f9496ffd640 1 -- 172.21.15.78:0/3388951586 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a0043070 con 0x55f4069ee1d0 2024-08-18T23:52:33.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.711+0000 7f9496ffd640 1 -- 172.21.15.78:0/3388951586 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f94a0054030 con 0x55f4069ee1d0 2024-08-18T23:52:33.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.711+0000 7f94bfd19740 10 client.16800 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:33.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.711+0000 7f94bfd19740 10 client.16800 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:52:33.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.711+0000 7f94bfd19740 20 client.16800 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:33.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.711+0000 7f94bfd19740 1 --2- 172.21.15.78:0/3388951586 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55f4069fddf0 0x55f406a6abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:33.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.711+0000 7f94bfd19740 4 client.16800 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:52:33.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.711+0000 7f94bfd19740 1 -- 172.21.15.78:0/3388951586 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4069504c0 con 0x55f4069fddf0 2024-08-18T23:52:33.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.711+0000 7f94b6ffd640 1 --2- 172.21.15.78:0/3388951586 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55f4069fddf0 0x55f406a6abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:33.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.711+0000 7f94b6ffd640 1 --2- 172.21.15.78:0/3388951586 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55f4069fddf0 0x55f406a6abe0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=762b9f2c7b1189d2 server_cookie=4c94814da1195e0d in_seq=0 out_seq=0 2024-08-18T23:52:33.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.711+0000 7f9496ffd640 10 client.16800 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:52:33.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.711+0000 7f9496ffd640 1 -- 172.21.15.78:0/3388951586 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4069504c0 con 0x55f4069fddf0 2024-08-18T23:52:33.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.711+0000 7f9496ffd640 10 client.16800 handle_command_reply: tid=0 2024-08-18T23:52:33.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.711+0000 7f94bfd19740 1 client.16800 shutdown 2024-08-18T23:52:33.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bfd19740 1 -- 172.21.15.78:0/3388951586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069ee1d0 msgr2=0x55f4069f16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:33.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bfd19740 1 --2- 172.21.15.78:0/3388951586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069ee1d0 0x55f4069f16d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f94a003c7c0 tx=0x7f94a003cba0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bfd19740 1 -- 172.21.15.78:0/3388951586 shutdown_connections 2024-08-18T23:52:33.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bfd19740 1 --2- 172.21.15.78:0/3388951586 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55f4069fddf0 0x55f406a6abe0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bfd19740 1 --2- 172.21.15.78:0/3388951586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4069ee1d0 0x55f4069f16d0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bfd19740 1 --2- 172.21.15.78:0/3388951586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4069e3910 0x55f4069f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bfd19740 1 --2- 172.21.15.78:0/3388951586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4069e2ff0 0x55f4069f6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bfd19740 1 -- 172.21.15.78:0/3388951586 >> 172.21.15.78:0/3388951586 conn(0x55f4069df580 msgr2=0x55f4069f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:33.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f9496ffd640 0 client.16800 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:52:33.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bfd19740 1 -- 172.21.15.78:0/3388951586 shutdown_connections 2024-08-18T23:52:33.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bfd19740 1 -- 172.21.15.78:0/3388951586 wait complete. 2024-08-18T23:52:33.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bfd19740 20 client.16800 trim_cache size 0 max 16384 2024-08-18T23:52:33.725 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:52:33.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bd91b640 1 -- 172.21.15.78:0/3719466750 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94a40a2b80 msgr2=0x7f94a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:33.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bd91b640 1 --2- 172.21.15.78:0/3719466750 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94a40a2b80 0x7f94a40a5040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f94b00986d0 tx=0x7f94b0066d70 comp rx=0 tx=0).stop 2024-08-18T23:52:33.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bd91b640 1 -- 172.21.15.78:0/3719466750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b8108010 msgr2=0x7f94b81011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:33.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bd91b640 1 --2- 172.21.15.78:0/3719466750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b8108010 0x7f94b81011c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f94b000e970 tx=0x7f94b000ed70 comp rx=0 tx=0).stop 2024-08-18T23:52:33.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bd91b640 1 -- 172.21.15.78:0/3719466750 shutdown_connections 2024-08-18T23:52:33.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bd91b640 1 --2- 172.21.15.78:0/3719466750 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f94a40a2b80 0x7f94a40a5040 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bd91b640 1 --2- 172.21.15.78:0/3719466750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b8108010 0x7f94b81011c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bd91b640 1 --2- 172.21.15.78:0/3719466750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94b8103490 0x7f94b8100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bd91b640 1 --2- 172.21.15.78:0/3719466750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94b8102ae0 0x7f94b8100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:33.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bd91b640 1 -- 172.21.15.78:0/3719466750 >> 172.21.15.78:0/3719466750 conn(0x7f94b80fc500 msgr2=0x7f94b8105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:33.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.712+0000 7f94bd91b640 1 -- 172.21.15.78:0/3719466750 shutdown_connections 2024-08-18T23:52:33.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:33.713+0000 7f94bd91b640 1 -- 172.21.15.78:0/3719466750 wait complete. 2024-08-18T23:52:33.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:33 smithi123 ceph-mon[21808]: pgmap v597: 129 pgs: 129 active+clean; 255 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 823 KiB/s wr, 532 op/s 2024-08-18T23:52:34.404 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-18T23:52:34.658 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:35.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.134+0000 7fe0d5f28640 1 -- 172.21.15.78:0/202615691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0d0102a60 msgr2=0x7fe0d0102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:35.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.134+0000 7fe0d5f28640 1 --2- 172.21.15.78:0/202615691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0d0102a60 0x7fe0d0102e40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009ca0 tx=0x7fe0c002a950 comp rx=0 tx=0).stop 2024-08-18T23:52:35.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.134+0000 7fe0d5f28640 1 -- 172.21.15.78:0/202615691 shutdown_connections 2024-08-18T23:52:35.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.134+0000 7fe0d5f28640 1 --2- 172.21.15.78:0/202615691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0d010be90 0x7fe0d010e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.134+0000 7fe0d5f28640 1 --2- 172.21.15.78:0/202615691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0d0103380 0x7fe0d010b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.134+0000 7fe0d5f28640 1 --2- 172.21.15.78:0/202615691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0d0102a60 0x7fe0d0102e40 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.135+0000 7fe0d5f28640 1 -- 172.21.15.78:0/202615691 >> 172.21.15.78:0/202615691 conn(0x7fe0d00fc500 msgr2=0x7fe0d00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.135+0000 7fe0d5f28640 1 -- 172.21.15.78:0/202615691 shutdown_connections 2024-08-18T23:52:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.135+0000 7fe0d5f28640 1 -- 172.21.15.78:0/202615691 wait complete. 2024-08-18T23:52:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.135+0000 7fe0d5f28640 1 Processor -- start 2024-08-18T23:52:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.135+0000 7fe0d5f28640 1 -- start start 2024-08-18T23:52:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.135+0000 7fe0d5f28640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0d0102a60 0x7fe0d01982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.135+0000 7fe0d5f28640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0d0103380 0x7fe0d01987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0d5f28640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0d010be90 0x7fe0d019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0d5f28640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d0111cd0 con 0x7fe0d0103380 2024-08-18T23:52:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0d5f28640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d0111b50 con 0x7fe0d010be90 2024-08-18T23:52:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0d5f28640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d0111e50 con 0x7fe0d0102a60 2024-08-18T23:52:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0ceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0d0103380 0x7fe0d01987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:35.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0ceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0d0103380 0x7fe0d01987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59358/0 (socket says 172.21.15.78:59358) 2024-08-18T23:52:35.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0ceffd640 1 -- 172.21.15.78:0/4179416281 learned_addr learned my addr 172.21.15.78:0/4179416281 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:35.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0cf7fe640 1 --2- 172.21.15.78:0/4179416281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0d0102a60 0x7fe0d01982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:35.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0cffff640 1 --2- 172.21.15.78:0/4179416281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0d010be90 0x7fe0d019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:35.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0ceffd640 1 -- 172.21.15.78:0/4179416281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0d0102a60 msgr2=0x7fe0d01982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:35.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0ceffd640 1 --2- 172.21.15.78:0/4179416281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0d0102a60 0x7fe0d01982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0ceffd640 1 -- 172.21.15.78:0/4179416281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0d010be90 msgr2=0x7fe0d019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:35.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0ceffd640 1 --2- 172.21.15.78:0/4179416281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0d010be90 0x7fe0d019cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0ceffd640 1 -- 172.21.15.78:0/4179416281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d019d260 con 0x7fe0d0103380 2024-08-18T23:52:35.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0ceffd640 1 --2- 172.21.15.78:0/4179416281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0d0103380 0x7fe0d01987f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800d740 tx=0x7fe0b800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:35.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0ccff9640 1 -- 172.21.15.78:0/4179416281 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8017070 con 0x7fe0d0103380 2024-08-18T23:52:35.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0d5f28640 1 -- 172.21.15.78:0/4179416281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d019d550 con 0x7fe0d0103380 2024-08-18T23:52:35.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0ccff9640 1 -- 172.21.15.78:0/4179416281 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0b800de50 con 0x7fe0d0103380 2024-08-18T23:52:35.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0ccff9640 1 -- 172.21.15.78:0/4179416281 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b800f620 con 0x7fe0d0103380 2024-08-18T23:52:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.136+0000 7fe0d5f28640 1 -- 172.21.15.78:0/4179416281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d01a4e30 con 0x7fe0d0103380 2024-08-18T23:52:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.139+0000 7fe0ccff9640 1 -- 172.21.15.78:0/4179416281 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe0b8009c30 con 0x7fe0d0103380 2024-08-18T23:52:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.139+0000 7fe0d8326740 1 Processor -- start 2024-08-18T23:52:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.139+0000 7fe0d8326740 1 -- start start 2024-08-18T23:52:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.139+0000 7fe0d8326740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 0x563fc3c433d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.139+0000 7fe0d8326740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c43910 0x563fc3c4dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.139+0000 7fe0d8326740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 0x563fc3c505c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.139+0000 7fe0d8326740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563fc3ba8a40 con 0x563fc3c4e1d0 2024-08-18T23:52:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.139+0000 7fe0d8326740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563fc3b409d0 con 0x563fc3c42ff0 2024-08-18T23:52:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.139+0000 7fe0d8326740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563fc3ba4a00 con 0x563fc3c43910 2024-08-18T23:52:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.139+0000 7fe0ceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 0x563fc3c505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.139+0000 7fe0ceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 0x563fc3c505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59374/0 (socket says 172.21.15.78:59374) 2024-08-18T23:52:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.139+0000 7fe0ceffd640 1 -- 172.21.15.78:0/507312435 learned_addr learned my addr 172.21.15.78:0/507312435 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:35.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.139+0000 7fe0cffff640 1 --2- 172.21.15.78:0/507312435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 0x563fc3c433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:35.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.139+0000 7fe0cf7fe640 1 --2- 172.21.15.78:0/507312435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c43910 0x563fc3c4dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:35.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.139+0000 7fe0ccff9640 1 --2- 172.21.15.78:0/4179416281 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe0bc0a2b80 0x7fe0bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:35.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.140+0000 7fe0ceffd640 1 -- 172.21.15.78:0/507312435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c43910 msgr2=0x563fc3c4dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:35.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.140+0000 7fe0ceffd640 1 --2- 172.21.15.78:0/507312435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c43910 0x563fc3c4dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.140+0000 7fe0ceffd640 1 -- 172.21.15.78:0/507312435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 msgr2=0x563fc3c433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:35.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.140+0000 7fe0ceffd640 1 --2- 172.21.15.78:0/507312435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 0x563fc3c433d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.140+0000 7fe0ceffd640 1 -- 172.21.15.78:0/507312435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc3b47650 con 0x563fc3c4e1d0 2024-08-18T23:52:35.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.140+0000 7fe0cffff640 1 --2- 172.21.15.78:0/507312435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 0x563fc3c433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:52:35.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.140+0000 7fe0cffff640 1 --2- 172.21.15.78:0/4179416281 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe0bc0a2b80 0x7fe0bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:35.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.140+0000 7fe0ccff9640 1 -- 172.21.15.78:0/4179416281 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fe0b80cefd0 con 0x7fe0d0103380 2024-08-18T23:52:35.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.140+0000 7fe0ceffd640 1 --2- 172.21.15.78:0/507312435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 0x563fc3c505c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe0b80cc680 tx=0x7fe0b8097870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be034ec09e5d787 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:35.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.140+0000 7fe0c5ffb640 1 -- 172.21.15.78:0/507312435 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8020070 con 0x563fc3c4e1d0 2024-08-18T23:52:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.140+0000 7fe0cffff640 1 --2- 172.21.15.78:0/4179416281 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe0bc0a2b80 0x7fe0bc0a5040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8006fd0 tx=0x7fe0c80047d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.140+0000 7fe0c5ffb640 1 -- 172.21.15.78:0/507312435 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0b80a2bd0 con 0x563fc3c4e1d0 2024-08-18T23:52:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.140+0000 7fe0c5ffb640 1 -- 172.21.15.78:0/507312435 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b809f850 con 0x563fc3c4e1d0 2024-08-18T23:52:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.141+0000 7fe0d8326740 1 -- 172.21.15.78:0/507312435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 msgr2=0x563fc3c505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.141+0000 7fe0d8326740 1 --2- 172.21.15.78:0/507312435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 0x563fc3c505c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe0b80cc680 tx=0x7fe0b8097870 comp rx=0 tx=0).stop 2024-08-18T23:52:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.141+0000 7fe0d8326740 1 -- 172.21.15.78:0/507312435 shutdown_connections 2024-08-18T23:52:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.141+0000 7fe0d8326740 1 --2- 172.21.15.78:0/507312435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 0x563fc3c505c0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.141+0000 7fe0d8326740 1 --2- 172.21.15.78:0/507312435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c43910 0x563fc3c4dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.141+0000 7fe0d8326740 1 --2- 172.21.15.78:0/507312435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 0x563fc3c433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.141+0000 7fe0d8326740 1 -- 172.21.15.78:0/507312435 >> 172.21.15.78:0/507312435 conn(0x563fc3c3f580 msgr2=0x563fc3c3f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.141+0000 7fe0d8326740 1 -- 172.21.15.78:0/507312435 shutdown_connections 2024-08-18T23:52:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.141+0000 7fe0d8326740 1 -- 172.21.15.78:0/507312435 wait complete. 2024-08-18T23:52:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.141+0000 7fe0d8326740 1 -- start start 2024-08-18T23:52:35.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0d8326740 1 Processor -- start 2024-08-18T23:52:35.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0d8326740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 0x563fc3c4b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:35.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0d8326740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c43910 0x563fc3c4b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:35.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0d8326740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c4e1d0 0x563fc3c4bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:35.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0d8326740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563fc3c50d40 con 0x563fc3c43910 2024-08-18T23:52:35.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0d8326740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563fc3b63b20 con 0x563fc3c42ff0 2024-08-18T23:52:35.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0d8326740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563fc3c50ec0 con 0x563fc3c4e1d0 2024-08-18T23:52:35.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0d8326740 10 client.0 _ll_register_callbacks cb 0x563fc3b61bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:35.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0ceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c43910 0x563fc3c4b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:35.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0ceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c43910 0x563fc3c4b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59390/0 (socket says 172.21.15.78:59390) 2024-08-18T23:52:35.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0ceffd640 1 -- 172.21.15.78:0/2889801379 learned_addr learned my addr 172.21.15.78:0/2889801379 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:35.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0cf7fe640 1 --2- 172.21.15.78:0/2889801379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 0x563fc3c4b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:35.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0cffff640 1 --2- 172.21.15.78:0/2889801379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c4e1d0 0x563fc3c4bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0ceffd640 1 -- 172.21.15.78:0/2889801379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c4e1d0 msgr2=0x563fc3c4bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0ceffd640 1 --2- 172.21.15.78:0/2889801379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c4e1d0 0x563fc3c4bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0ceffd640 1 -- 172.21.15.78:0/2889801379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 msgr2=0x563fc3c4b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0ceffd640 1 --2- 172.21.15.78:0/2889801379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 0x563fc3c4b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0ceffd640 1 -- 172.21.15.78:0/2889801379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc391a3c0 con 0x563fc3c43910 2024-08-18T23:52:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0ceffd640 1 --2- 172.21.15.78:0/2889801379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c43910 0x563fc3c4b5c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8012040 tx=0x7fe0b809b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f3211f043e2ae4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.142+0000 7fe0aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.143+0000 7fe0aeffd640 1 -- 172.21.15.78:0/2889801379 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8020040 con 0x563fc3c43910 2024-08-18T23:52:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.143+0000 7fe0aeffd640 1 -- 172.21.15.78:0/2889801379 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fe0b80a3c80 con 0x563fc3c43910 2024-08-18T23:52:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.143+0000 7fe0d8326740 1 -- 172.21.15.78:0/2889801379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563fc3bb5fb0 con 0x563fc3c43910 2024-08-18T23:52:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.143+0000 7fe0aeffd640 1 -- 172.21.15.78:0/2889801379 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe0b80d12f0 con 0x563fc3c43910 2024-08-18T23:52:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.143+0000 7fe0d8326740 10 client.16812 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:35.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.143+0000 7fe0d8326740 1 -- 172.21.15.78:0/2889801379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563fc3c5c910 con 0x563fc3c43910 2024-08-18T23:52:35.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.143+0000 7fe0aeffd640 1 -- 172.21.15.78:0/2889801379 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563fc3bb5fb0 con 0x563fc3c43910 2024-08-18T23:52:35.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.143+0000 7fe0aeffd640 1 -- 172.21.15.78:0/2889801379 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fe0b809a8f0 con 0x563fc3c43910 2024-08-18T23:52:35.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.143+0000 7fe0d8326740 10 client.16812 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:35.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.143+0000 7fe0d8326740 10 client.16812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:52:35.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.143+0000 7fe0d8326740 20 client.16812 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:35.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.143+0000 7fe0d8326740 1 --2- 172.21.15.78:0/2889801379 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563fc3c5dc60 0x563fc3c60050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:35.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.143+0000 7fe0d8326740 4 client.16812 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:52:35.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.143+0000 7fe0d8326740 1 -- 172.21.15.78:0/2889801379 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fc3bb04c0 con 0x563fc3c5dc60 2024-08-18T23:52:35.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.143+0000 7fe0cf7fe640 1 --2- 172.21.15.78:0/2889801379 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563fc3c5dc60 0x563fc3c60050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:35.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.144+0000 7fe0aeffd640 10 client.16812 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:35.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.144+0000 7fe0cf7fe640 1 --2- 172.21.15.78:0/2889801379 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563fc3c5dc60 0x563fc3c60050 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27d0445f95314e3e server_cookie=cc5591695a286715 in_seq=0 out_seq=0 2024-08-18T23:52:35.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.144+0000 7fe0aeffd640 1 -- 172.21.15.78:0/2889801379 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563fc3bb04c0 con 0x563fc3c5dc60 2024-08-18T23:52:35.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.144+0000 7fe0aeffd640 10 client.16812 handle_command_reply: tid=0 2024-08-18T23:52:35.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.145+0000 7fe0d8326740 1 client.16812 shutdown 2024-08-18T23:52:35.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.145+0000 7fe0d8326740 1 -- 172.21.15.78:0/2889801379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c43910 msgr2=0x563fc3c4b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:35.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.145+0000 7fe0d8326740 1 --2- 172.21.15.78:0/2889801379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c43910 0x563fc3c4b5c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8012040 tx=0x7fe0b809b090 comp rx=0 tx=0).stop 2024-08-18T23:52:35.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.145+0000 7fe0d8326740 1 -- 172.21.15.78:0/2889801379 shutdown_connections 2024-08-18T23:52:35.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.145+0000 7fe0d8326740 1 --2- 172.21.15.78:0/2889801379 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563fc3c5dc60 0x563fc3c60050 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.145+0000 7fe0d8326740 1 --2- 172.21.15.78:0/2889801379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c4e1d0 0x563fc3c4bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.145+0000 7fe0d8326740 1 --2- 172.21.15.78:0/2889801379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c43910 0x563fc3c4b5c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.145+0000 7fe0aeffd640 0 client.16812 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:35.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.145+0000 7fe0d8326740 1 --2- 172.21.15.78:0/2889801379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 0x563fc3c4b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.145+0000 7fe0d8326740 1 -- 172.21.15.78:0/2889801379 >> 172.21.15.78:0/2889801379 conn(0x563fc3c3f580 msgr2=0x563fc3c452b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:35.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.145+0000 7fe0d8326740 1 -- 172.21.15.78:0/2889801379 shutdown_connections 2024-08-18T23:52:35.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.145+0000 7fe0d8326740 1 -- 172.21.15.78:0/2889801379 wait complete. 2024-08-18T23:52:35.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.145+0000 7fe0d8326740 20 client.16812 trim_cache size 0 max 16384 2024-08-18T23:52:35.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0d8326740 1 Processor -- start 2024-08-18T23:52:35.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0d8326740 1 -- start start 2024-08-18T23:52:35.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0d8326740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 0x563fc3c5a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:35.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0d8326740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c43910 0x563fc3c655f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:35.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0d8326740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 0x563fc3c65b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:35.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0d8326740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d01b13d0 con 0x563fc3c4e1d0 2024-08-18T23:52:35.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0d8326740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d01aec40 con 0x563fc3c42ff0 2024-08-18T23:52:35.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0d8326740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d01aca90 con 0x563fc3c43910 2024-08-18T23:52:35.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 0x563fc3c65b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:35.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 0x563fc3c65b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59398/0 (socket says 172.21.15.78:59398) 2024-08-18T23:52:35.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0cffff640 1 -- 172.21.15.78:0/641015913 learned_addr learned my addr 172.21.15.78:0/641015913 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:35.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0cf7fe640 1 --2- 172.21.15.78:0/641015913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 0x563fc3c5a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:35.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0ceffd640 1 --2- 172.21.15.78:0/641015913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c43910 0x563fc3c655f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:35.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0cffff640 1 -- 172.21.15.78:0/641015913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c43910 msgr2=0x563fc3c655f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:35.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0cffff640 1 --2- 172.21.15.78:0/641015913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c43910 0x563fc3c655f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0cffff640 1 -- 172.21.15.78:0/641015913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 msgr2=0x563fc3c5a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:35.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0cffff640 1 --2- 172.21.15.78:0/641015913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 0x563fc3c5a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0cffff640 1 -- 172.21.15.78:0/641015913 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc3c46cc0 con 0x563fc3c4e1d0 2024-08-18T23:52:35.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0ceffd640 1 --2- 172.21.15.78:0/641015913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c43910 0x563fc3c655f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:52:35.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.158+0000 7fe0cffff640 1 --2- 172.21.15.78:0/641015913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 0x563fc3c65b30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8010b90 tx=0x7fe0c8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38bbe0328a096397 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:35.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.159+0000 7fe0c57fa640 1 -- 172.21.15.78:0/641015913 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8025070 con 0x563fc3c4e1d0 2024-08-18T23:52:35.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.159+0000 7fe0c57fa640 1 -- 172.21.15.78:0/641015913 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0c8014b80 con 0x563fc3c4e1d0 2024-08-18T23:52:35.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.159+0000 7fe0c57fa640 1 -- 172.21.15.78:0/641015913 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8020620 con 0x563fc3c4e1d0 2024-08-18T23:52:35.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.159+0000 7fe0d8326740 1 -- 172.21.15.78:0/641015913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 msgr2=0x563fc3c65b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:35.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.159+0000 7fe0d8326740 1 --2- 172.21.15.78:0/641015913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 0x563fc3c65b30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8010b90 tx=0x7fe0c8013000 comp rx=0 tx=0).stop 2024-08-18T23:52:35.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.159+0000 7fe0d8326740 1 -- 172.21.15.78:0/641015913 shutdown_connections 2024-08-18T23:52:35.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.159+0000 7fe0d8326740 1 --2- 172.21.15.78:0/641015913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 0x563fc3c65b30 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.159+0000 7fe0d8326740 1 --2- 172.21.15.78:0/641015913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c43910 0x563fc3c655f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.159+0000 7fe0d8326740 1 --2- 172.21.15.78:0/641015913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 0x563fc3c5a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.159+0000 7fe0d8326740 1 -- 172.21.15.78:0/641015913 >> 172.21.15.78:0/641015913 conn(0x563fc3c3f580 msgr2=0x563fc3c49990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:35.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.159+0000 7fe0d8326740 1 -- 172.21.15.78:0/641015913 shutdown_connections 2024-08-18T23:52:35.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.159+0000 7fe0d8326740 1 -- 172.21.15.78:0/641015913 wait complete. 2024-08-18T23:52:35.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.160+0000 7fe0d8326740 1 -- start start 2024-08-18T23:52:35.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.160+0000 7fe0d8326740 1 Processor -- start 2024-08-18T23:52:35.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.160+0000 7fe0d8326740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 0x563fc3c56110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:35.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.160+0000 7fe0d8326740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c43910 0x563fc3c56650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:35.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.160+0000 7fe0d8326740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 0x563fc3c474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:35.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.160+0000 7fe0d8326740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d01ad100 con 0x563fc3c4e1d0 2024-08-18T23:52:35.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.160+0000 7fe0d8326740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d01ae570 con 0x563fc3c42ff0 2024-08-18T23:52:35.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.160+0000 7fe0d8326740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d01ad7d0 con 0x563fc3c43910 2024-08-18T23:52:35.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.160+0000 7fe0d8326740 10 client.0 _ll_register_callbacks cb 0x7fe0c800b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:35.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.160+0000 7fe0cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 0x563fc3c474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:35.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.160+0000 7fe0cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 0x563fc3c474f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59414/0 (socket says 172.21.15.78:59414) 2024-08-18T23:52:35.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.160+0000 7fe0cffff640 1 -- 172.21.15.78:0/1543832592 learned_addr learned my addr 172.21.15.78:0/1543832592 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:35.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.160+0000 7fe0cf7fe640 1 --2- 172.21.15.78:0/1543832592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 0x563fc3c56110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:35.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.160+0000 7fe0ceffd640 1 --2- 172.21.15.78:0/1543832592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c43910 0x563fc3c56650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:35.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.160+0000 7fe0cffff640 1 -- 172.21.15.78:0/1543832592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c43910 msgr2=0x563fc3c56650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:35.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.160+0000 7fe0cffff640 1 --2- 172.21.15.78:0/1543832592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c43910 0x563fc3c56650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.161+0000 7fe0cffff640 1 -- 172.21.15.78:0/1543832592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 msgr2=0x563fc3c56110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:35.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.161+0000 7fe0cffff640 1 --2- 172.21.15.78:0/1543832592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 0x563fc3c56110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.161+0000 7fe0cffff640 1 -- 172.21.15.78:0/1543832592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc3c47a30 con 0x563fc3c4e1d0 2024-08-18T23:52:35.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.161+0000 7fe0cffff640 1 --2- 172.21.15.78:0/1543832592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 0x563fc3c474f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80105c0 tx=0x7fe0c8021ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f54260addcc6b08 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:35.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.161+0000 7fe0aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:35.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.161+0000 7fe0aeffd640 1 -- 172.21.15.78:0/1543832592 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8025070 con 0x563fc3c4e1d0 2024-08-18T23:52:35.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.161+0000 7fe0aeffd640 1 -- 172.21.15.78:0/1543832592 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fe0c8033650 con 0x563fc3c4e1d0 2024-08-18T23:52:35.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.161+0000 7fe0d8326740 1 -- 172.21.15.78:0/1543832592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0c8014b80 con 0x563fc3c4e1d0 2024-08-18T23:52:35.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.161+0000 7fe0aeffd640 1 -- 172.21.15.78:0/1543832592 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe0c80059c0 con 0x563fc3c4e1d0 2024-08-18T23:52:35.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.161+0000 7fe0d8326740 10 client.16818 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:35.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.161+0000 7fe0d8326740 1 -- 172.21.15.78:0/1543832592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563fc3cc7cb0 con 0x563fc3c4e1d0 2024-08-18T23:52:35.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.161+0000 7fe0aeffd640 1 -- 172.21.15.78:0/1543832592 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0c8014b80 con 0x563fc3c4e1d0 2024-08-18T23:52:35.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.161+0000 7fe0aeffd640 1 -- 172.21.15.78:0/1543832592 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fe0c8005cb0 con 0x563fc3c4e1d0 2024-08-18T23:52:35.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.161+0000 7fe0d8326740 10 client.16818 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:35.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.161+0000 7fe0d8326740 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:52:35.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.161+0000 7fe0d8326740 20 client.16818 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:35.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.161+0000 7fe0d8326740 1 --2- 172.21.15.78:0/1543832592 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563fc3cc8310 0x563fc3cca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:35.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.161+0000 7fe0d8326740 4 client.16818 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:52:35.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.161+0000 7fe0d8326740 1 -- 172.21.15.78:0/1543832592 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563fc3b2d170 con 0x563fc3cc8310 2024-08-18T23:52:35.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.162+0000 7fe0cf7fe640 1 --2- 172.21.15.78:0/1543832592 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563fc3cc8310 0x563fc3cca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:35.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.162+0000 7fe0aeffd640 10 client.16818 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:35.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.162+0000 7fe0cf7fe640 1 --2- 172.21.15.78:0/1543832592 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563fc3cc8310 0x563fc3cca700 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15b4aec6bbaab271 server_cookie=72a167fe3ae108e5 in_seq=0 out_seq=0 2024-08-18T23:52:35.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.163+0000 7fe0aeffd640 1 -- 172.21.15.78:0/1543832592 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563fc3b2d170 con 0x563fc3cc8310 2024-08-18T23:52:35.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.163+0000 7fe0aeffd640 10 client.16818 handle_command_reply: tid=0 2024-08-18T23:52:35.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.163+0000 7fe0d8326740 1 client.16818 shutdown 2024-08-18T23:52:35.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.163+0000 7fe0d8326740 1 -- 172.21.15.78:0/1543832592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 msgr2=0x563fc3c474f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:35.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.163+0000 7fe0d8326740 1 --2- 172.21.15.78:0/1543832592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 0x563fc3c474f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80105c0 tx=0x7fe0c8021ad0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.163+0000 7fe0d8326740 1 -- 172.21.15.78:0/1543832592 shutdown_connections 2024-08-18T23:52:35.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.163+0000 7fe0d8326740 1 --2- 172.21.15.78:0/1543832592 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563fc3cc8310 0x563fc3cca700 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.163+0000 7fe0d8326740 1 --2- 172.21.15.78:0/1543832592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563fc3c4e1d0 0x563fc3c474f0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.163+0000 7fe0aeffd640 0 client.16818 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:35.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.163+0000 7fe0d8326740 1 --2- 172.21.15.78:0/1543832592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563fc3c43910 0x563fc3c56650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.163+0000 7fe0d8326740 1 --2- 172.21.15.78:0/1543832592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fc3c42ff0 0x563fc3c56110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.163+0000 7fe0d8326740 1 -- 172.21.15.78:0/1543832592 >> 172.21.15.78:0/1543832592 conn(0x563fc3c3f580 msgr2=0x563fc3c49990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:35.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.163+0000 7fe0d8326740 1 -- 172.21.15.78:0/1543832592 shutdown_connections 2024-08-18T23:52:35.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.163+0000 7fe0d8326740 1 -- 172.21.15.78:0/1543832592 wait complete. 2024-08-18T23:52:35.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.164+0000 7fe0d8326740 20 client.16818 trim_cache size 0 max 16384 2024-08-18T23:52:35.182 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:52:35.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.164+0000 7fe0d5f28640 1 -- 172.21.15.78:0/4179416281 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe0bc0a2b80 msgr2=0x7fe0bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:35.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.164+0000 7fe0d5f28640 1 --2- 172.21.15.78:0/4179416281 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe0bc0a2b80 0x7fe0bc0a5040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8006fd0 tx=0x7fe0c80047d0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.164+0000 7fe0d5f28640 1 -- 172.21.15.78:0/4179416281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0d0103380 msgr2=0x7fe0d01987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:35.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.164+0000 7fe0d5f28640 1 --2- 172.21.15.78:0/4179416281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0d0103380 0x7fe0d01987f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800d740 tx=0x7fe0b800dc00 comp rx=0 tx=0).stop 2024-08-18T23:52:35.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.164+0000 7fe0d5f28640 1 -- 172.21.15.78:0/4179416281 shutdown_connections 2024-08-18T23:52:35.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.164+0000 7fe0d5f28640 1 --2- 172.21.15.78:0/4179416281 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe0bc0a2b80 0x7fe0bc0a5040 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.164+0000 7fe0d5f28640 1 --2- 172.21.15.78:0/4179416281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0d010be90 0x7fe0d019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.164+0000 7fe0d5f28640 1 --2- 172.21.15.78:0/4179416281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0d0103380 0x7fe0d01987f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.164+0000 7fe0d5f28640 1 --2- 172.21.15.78:0/4179416281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0d0102a60 0x7fe0d01982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:35.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.164+0000 7fe0d5f28640 1 -- 172.21.15.78:0/4179416281 >> 172.21.15.78:0/4179416281 conn(0x7fe0d00fc500 msgr2=0x7fe0d00fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:35.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.164+0000 7fe0d5f28640 1 -- 172.21.15.78:0/4179416281 shutdown_connections 2024-08-18T23:52:35.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:35.164+0000 7fe0d5f28640 1 -- 172.21.15.78:0/4179416281 wait complete. 2024-08-18T23:52:35.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:35 smithi161 ceph-mon[22027]: pgmap v598: 129 pgs: 129 active+clean; 253 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 827 KiB/s wr, 442 op/s 2024-08-18T23:52:35.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:35 smithi078 ceph-mon[18440]: pgmap v598: 129 pgs: 129 active+clean; 253 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 827 KiB/s wr, 442 op/s 2024-08-18T23:52:35.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:35 smithi123 ceph-mon[21808]: pgmap v598: 129 pgs: 129 active+clean; 253 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 827 KiB/s wr, 442 op/s 2024-08-18T23:52:36.881 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-18T23:52:36.881 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-18T23:52:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:37.523 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:37 smithi078 ceph-mon[18440]: pgmap v599: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 524 KiB/s wr, 354 op/s 2024-08-18T23:52:37.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.605+0000 7f09aff37640 1 -- 172.21.15.78:0/720780908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09a810bea0 msgr2=0x7f09a810e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:37.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.605+0000 7f09aff37640 1 --2- 172.21.15.78:0/720780908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09a810bea0 0x7f09a810e340 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009ca0 tx=0x7f09a002a950 comp rx=0 tx=0).stop 2024-08-18T23:52:37.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.605+0000 7f09aff37640 1 -- 172.21.15.78:0/720780908 shutdown_connections 2024-08-18T23:52:37.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.605+0000 7f09aff37640 1 --2- 172.21.15.78:0/720780908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09a810bea0 0x7f09a810e340 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.605+0000 7f09aff37640 1 --2- 172.21.15.78:0/720780908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09a8103390 0x7f09a810b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.605+0000 7f09aff37640 1 --2- 172.21.15.78:0/720780908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09a8102a70 0x7f09a8102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.605+0000 7f09aff37640 1 -- 172.21.15.78:0/720780908 >> 172.21.15.78:0/720780908 conn(0x7f09a80fc4d0 msgr2=0x7f09a80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:37.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.605+0000 7f09aff37640 1 -- 172.21.15.78:0/720780908 shutdown_connections 2024-08-18T23:52:37.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.605+0000 7f09aff37640 1 -- 172.21.15.78:0/720780908 wait complete. 2024-08-18T23:52:37.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.605+0000 7f09aff37640 1 Processor -- start 2024-08-18T23:52:37.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.605+0000 7f09aff37640 1 -- start start 2024-08-18T23:52:37.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09aff37640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09a8102a70 0x7f09a8198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:37.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09aff37640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09a8103390 0x7f09a8198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:37.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09aff37640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09a810bea0 0x7f09a819c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:37.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09aff37640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8111ce0 con 0x7f09a810bea0 2024-08-18T23:52:37.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09aff37640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8111b60 con 0x7f09a8103390 2024-08-18T23:52:37.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09aff37640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8111e60 con 0x7f09a8102a70 2024-08-18T23:52:37.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09ae4d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09a810bea0 0x7f09a819c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:37.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09ae4d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09a810bea0 0x7f09a819c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52080/0 (socket says 172.21.15.78:52080) 2024-08-18T23:52:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09ae4d4640 1 -- 172.21.15.78:0/1428753901 learned_addr learned my addr 172.21.15.78:0/1428753901 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09adcd3640 1 --2- 172.21.15.78:0/1428753901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09a8102a70 0x7f09a8198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09ad4d2640 1 --2- 172.21.15.78:0/1428753901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09a8103390 0x7f09a8198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09ae4d4640 1 -- 172.21.15.78:0/1428753901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09a8102a70 msgr2=0x7f09a8198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09ae4d4640 1 --2- 172.21.15.78:0/1428753901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09a8102a70 0x7f09a8198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09ae4d4640 1 -- 172.21.15.78:0/1428753901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09a8103390 msgr2=0x7f09a8198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09ae4d4640 1 --2- 172.21.15.78:0/1428753901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09a8103390 0x7f09a8198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09ae4d4640 1 -- 172.21.15.78:0/1428753901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a819d0b0 con 0x7f09a810bea0 2024-08-18T23:52:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09adcd3640 1 --2- 172.21.15.78:0/1428753901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09a8102a70 0x7f09a8198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09ae4d4640 1 --2- 172.21.15.78:0/1428753901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09a810bea0 0x7f09a819c9d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f09a0006010 tx=0x7f09a002f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09a6ffd640 1 -- 172.21.15.78:0/1428753901 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0034040 con 0x7f09a810bea0 2024-08-18T23:52:37.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09a6ffd640 1 -- 172.21.15.78:0/1428753901 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a0003e00 con 0x7f09a810bea0 2024-08-18T23:52:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.606+0000 7f09aff37640 1 -- 172.21.15.78:0/1428753901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a819d340 con 0x7f09a810bea0 2024-08-18T23:52:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.607+0000 7f09a6ffd640 1 -- 172.21.15.78:0/1428753901 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a00456e0 con 0x7f09a810bea0 2024-08-18T23:52:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.607+0000 7f09aff37640 1 -- 172.21.15.78:0/1428753901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a81a4c80 con 0x7f09a810bea0 2024-08-18T23:52:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.609+0000 7f09a6ffd640 1 -- 172.21.15.78:0/1428753901 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f09a0037020 con 0x7f09a810bea0 2024-08-18T23:52:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.609+0000 7f09b2335740 1 Processor -- start 2024-08-18T23:52:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.609+0000 7f09b2335740 1 -- start start 2024-08-18T23:52:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.609+0000 7f09b2335740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56424509f0c0 0x56424509f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.609+0000 7f09b2335740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 0x5642450ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09b2335740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642450ae7f0 0x5642450b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09b2335740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564245062e40 con 0x56424509f9e0 2024-08-18T23:52:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09b2335740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5642450047f0 con 0x56424509f0c0 2024-08-18T23:52:37.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09b2335740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564244fabfb0 con 0x5642450ae7f0 2024-08-18T23:52:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09adcd3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 0x5642450ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09a6ffd640 1 --2- 172.21.15.78:0/1428753901 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f09900a2b80 0x7f09900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09adcd3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 0x5642450ae2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52090/0 (socket says 172.21.15.78:52090) 2024-08-18T23:52:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09adcd3640 1 -- 172.21.15.78:0/760905711 learned_addr learned my addr 172.21.15.78:0/760905711 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09ae4d4640 1 --2- 172.21.15.78:0/1428753901 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f09900a2b80 0x7f09900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09ad4d2640 1 --2- 172.21.15.78:0/760905711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642450ae7f0 0x5642450b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09ad4d2640 1 --2- 172.21.15.78:0/760905711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56424509f0c0 0x56424509f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09adcd3640 1 -- 172.21.15.78:0/760905711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642450ae7f0 msgr2=0x5642450b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09adcd3640 1 --2- 172.21.15.78:0/760905711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642450ae7f0 0x5642450b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09adcd3640 1 -- 172.21.15.78:0/760905711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56424509f0c0 msgr2=0x56424509f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09adcd3640 1 --2- 172.21.15.78:0/760905711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56424509f0c0 0x56424509f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09a6ffd640 1 -- 172.21.15.78:0/1428753901 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f09a00f0500 con 0x7f09a810bea0 2024-08-18T23:52:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09adcd3640 1 -- 172.21.15.78:0/760905711 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564244d766a0 con 0x56424509f9e0 2024-08-18T23:52:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09ad4d2640 1 --2- 172.21.15.78:0/760905711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56424509f0c0 0x56424509f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:52:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09adcd3640 1 --2- 172.21.15.78:0/760905711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 0x5642450ae2b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0998008c60 tx=0x7f099800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9357dab38c517511 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f09ae4d4640 1 --2- 172.21.15.78:0/1428753901 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f09900a2b80 0x7f09900a5040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f09a00b9340 tx=0x7f09a00879e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f098ffff640 1 -- 172.21.15.78:0/760905711 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998014070 con 0x56424509f9e0 2024-08-18T23:52:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f098ffff640 1 -- 172.21.15.78:0/760905711 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09980047a0 con 0x56424509f9e0 2024-08-18T23:52:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.610+0000 7f098ffff640 1 -- 172.21.15.78:0/760905711 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998010760 con 0x56424509f9e0 2024-08-18T23:52:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.611+0000 7f09b2335740 1 -- 172.21.15.78:0/760905711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 msgr2=0x5642450ae2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.611+0000 7f09b2335740 1 --2- 172.21.15.78:0/760905711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 0x5642450ae2b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0998008c60 tx=0x7f099800e720 comp rx=0 tx=0).stop 2024-08-18T23:52:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.611+0000 7f09b2335740 1 -- 172.21.15.78:0/760905711 shutdown_connections 2024-08-18T23:52:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.611+0000 7f09b2335740 1 --2- 172.21.15.78:0/760905711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5642450ae7f0 0x5642450b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.611+0000 7f09b2335740 1 --2- 172.21.15.78:0/760905711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 0x5642450ae2b0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.611+0000 7f09b2335740 1 --2- 172.21.15.78:0/760905711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56424509f0c0 0x56424509f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.611+0000 7f09b2335740 1 -- 172.21.15.78:0/760905711 >> 172.21.15.78:0/760905711 conn(0x56424509b790 msgr2=0x56424509bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:37.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.611+0000 7f09b2335740 1 -- 172.21.15.78:0/760905711 shutdown_connections 2024-08-18T23:52:37.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.611+0000 7f09b2335740 1 -- 172.21.15.78:0/760905711 wait complete. 2024-08-18T23:52:37.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.612+0000 7f09b2335740 1 -- start start 2024-08-18T23:52:37.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.612+0000 7f09b2335740 1 Processor -- start 2024-08-18T23:52:37.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.612+0000 7f09b2335740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56424509f0c0 0x5642450ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:37.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.612+0000 7f09b2335740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 0x5642450abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:37.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.612+0000 7f09b2335740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642450ae7f0 0x5642450ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:37.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.612+0000 7f09b2335740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564244fbfcb0 con 0x56424509f9e0 2024-08-18T23:52:37.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.612+0000 7f09b2335740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5642450007b0 con 0x5642450ae7f0 2024-08-18T23:52:37.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.612+0000 7f09b2335740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5642450b13c0 con 0x56424509f0c0 2024-08-18T23:52:37.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.612+0000 7f09b2335740 10 client.0 _ll_register_callbacks cb 0x564244d13ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:37.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.612+0000 7f09adcd3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 0x5642450abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:37.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.612+0000 7f09adcd3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 0x5642450abca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52098/0 (socket says 172.21.15.78:52098) 2024-08-18T23:52:37.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.612+0000 7f09adcd3640 1 -- 172.21.15.78:0/209007391 learned_addr learned my addr 172.21.15.78:0/209007391 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:37.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.612+0000 7f09ad4d2640 1 --2- 172.21.15.78:0/209007391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642450ae7f0 0x5642450ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:37.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.612+0000 7f09adcd3640 1 -- 172.21.15.78:0/209007391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56424509f0c0 msgr2=0x5642450ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:37.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.612+0000 7f09adcd3640 1 --2- 172.21.15.78:0/209007391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56424509f0c0 0x5642450ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.612+0000 7f09adcd3640 1 -- 172.21.15.78:0/209007391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642450ae7f0 msgr2=0x5642450ac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:37.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.612+0000 7f09adcd3640 1 --2- 172.21.15.78:0/209007391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642450ae7f0 0x5642450ac1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.612+0000 7f09adcd3640 1 -- 172.21.15.78:0/209007391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564244fbebc0 con 0x56424509f9e0 2024-08-18T23:52:37.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.612+0000 7f09ad4d2640 1 --2- 172.21.15.78:0/209007391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642450ae7f0 0x5642450ac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:52:37.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.613+0000 7f09adcd3640 1 --2- 172.21.15.78:0/209007391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 0x5642450abca0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0998005170 tx=0x7f09980051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b951e2f278f214e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:37.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.613+0000 7f098d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:37.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.613+0000 7f098d7fa640 1 -- 172.21.15.78:0/209007391 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998027060 con 0x56424509f9e0 2024-08-18T23:52:37.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.613+0000 7f09b2335740 1 -- 172.21.15.78:0/209007391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09980047a0 con 0x56424509f9e0 2024-08-18T23:52:37.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.613+0000 7f098d7fa640 1 -- 172.21.15.78:0/209007391 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f0998005550 con 0x56424509f9e0 2024-08-18T23:52:37.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.613+0000 7f098d7fa640 1 -- 172.21.15.78:0/209007391 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f099802e050 con 0x56424509f9e0 2024-08-18T23:52:37.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.613+0000 7f09b2335740 10 client.16830 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:37.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.613+0000 7f09b2335740 1 -- 172.21.15.78:0/209007391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5642450bd2a0 con 0x56424509f9e0 2024-08-18T23:52:37.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.613+0000 7f098d7fa640 1 -- 172.21.15.78:0/209007391 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09980047a0 con 0x56424509f9e0 2024-08-18T23:52:37.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.613+0000 7f098d7fa640 1 -- 172.21.15.78:0/209007391 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f0998018510 con 0x56424509f9e0 2024-08-18T23:52:37.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.613+0000 7f09b2335740 10 client.16830 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:37.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.613+0000 7f09b2335740 10 client.16830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:52:37.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.613+0000 7f09b2335740 20 client.16830 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:37.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.613+0000 7f09b2335740 1 --2- 172.21.15.78:0/209007391 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5642450be6a0 0x5642450c0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:37.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.613+0000 7f09b2335740 4 client.16830 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:52:37.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.613+0000 7f09b2335740 1 -- 172.21.15.78:0/209007391 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564245011d60 con 0x5642450be6a0 2024-08-18T23:52:37.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.614+0000 7f09ad4d2640 1 --2- 172.21.15.78:0/209007391 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5642450be6a0 0x5642450c0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:37.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.614+0000 7f098d7fa640 10 client.16830 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:37.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.614+0000 7f09ad4d2640 1 --2- 172.21.15.78:0/209007391 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5642450be6a0 0x5642450c0a90 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddccd1356c464df7 server_cookie=2c066066fd2d433b in_seq=0 out_seq=0 2024-08-18T23:52:37.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.615+0000 7f098d7fa640 1 -- 172.21.15.78:0/209007391 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564245011d60 con 0x5642450be6a0 2024-08-18T23:52:37.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.615+0000 7f098d7fa640 10 client.16830 handle_command_reply: tid=0 2024-08-18T23:52:37.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.615+0000 7f09b2335740 1 client.16830 shutdown 2024-08-18T23:52:37.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.615+0000 7f09b2335740 1 -- 172.21.15.78:0/209007391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 msgr2=0x5642450abca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:37.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.615+0000 7f09b2335740 1 --2- 172.21.15.78:0/209007391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 0x5642450abca0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0998005170 tx=0x7f09980051a0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.615+0000 7f09b2335740 1 -- 172.21.15.78:0/209007391 shutdown_connections 2024-08-18T23:52:37.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.615+0000 7f09b2335740 1 --2- 172.21.15.78:0/209007391 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5642450be6a0 0x5642450c0a90 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.615+0000 7f09b2335740 1 --2- 172.21.15.78:0/209007391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642450ae7f0 0x5642450ac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.615+0000 7f098d7fa640 0 client.16830 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:37.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.615+0000 7f09b2335740 1 --2- 172.21.15.78:0/209007391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 0x5642450abca0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.615+0000 7f09b2335740 1 --2- 172.21.15.78:0/209007391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56424509f0c0 0x5642450ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.615+0000 7f09b2335740 1 -- 172.21.15.78:0/209007391 >> 172.21.15.78:0/209007391 conn(0x56424509b790 msgr2=0x5642450a14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:37.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.615+0000 7f09b2335740 1 -- 172.21.15.78:0/209007391 shutdown_connections 2024-08-18T23:52:37.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.615+0000 7f09b2335740 1 -- 172.21.15.78:0/209007391 wait complete. 2024-08-18T23:52:37.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.615+0000 7f09b2335740 20 client.16830 trim_cache size 0 max 16384 2024-08-18T23:52:37.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.632+0000 7f09b2335740 1 Processor -- start 2024-08-18T23:52:37.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.632+0000 7f09b2335740 1 -- start start 2024-08-18T23:52:37.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.632+0000 7f09b2335740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56424509f0c0 0x56424509a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:37.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.633+0000 7f09b2335740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56424509f9e0 0x5642450c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:37.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.633+0000 7f09b2335740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642450ae7f0 0x5642450c6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:37.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.633+0000 7f09b2335740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81af060 con 0x5642450ae7f0 2024-08-18T23:52:37.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.633+0000 7f09b2335740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81ac8d0 con 0x56424509f9e0 2024-08-18T23:52:37.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.633+0000 7f09b2335740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81aa720 con 0x56424509f0c0 2024-08-18T23:52:37.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.633+0000 7f09ad4d2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642450ae7f0 0x5642450c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:37.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.633+0000 7f09ad4d2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642450ae7f0 0x5642450c6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52102/0 (socket says 172.21.15.78:52102) 2024-08-18T23:52:37.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.633+0000 7f09ad4d2640 1 -- 172.21.15.78:0/507365789 learned_addr learned my addr 172.21.15.78:0/507365789 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:37.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.633+0000 7f09adcd3640 1 --2- 172.21.15.78:0/507365789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56424509f9e0 0x5642450c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:37.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.633+0000 7f09ad4d2640 1 --2- 172.21.15.78:0/507365789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56424509f0c0 0x56424509a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:37.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.633+0000 7f09ad4d2640 1 -- 172.21.15.78:0/507365789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56424509f0c0 msgr2=0x56424509a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:37.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.633+0000 7f09ad4d2640 1 --2- 172.21.15.78:0/507365789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56424509f0c0 0x56424509a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.633+0000 7f09ad4d2640 1 -- 172.21.15.78:0/507365789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56424509f9e0 msgr2=0x5642450c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:37.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.633+0000 7f09ad4d2640 1 --2- 172.21.15.78:0/507365789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56424509f9e0 0x5642450c60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.633+0000 7f09ad4d2640 1 -- 172.21.15.78:0/507365789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642450aa710 con 0x5642450ae7f0 2024-08-18T23:52:37.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.633+0000 7f09ad4d2640 1 --2- 172.21.15.78:0/507365789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642450ae7f0 0x5642450c6630 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x5642450c1060 tx=0x7f099c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701ef52ac0699aaf server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:37.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.633+0000 7f098f7fe640 1 -- 172.21.15.78:0/507365789 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c0148e0 con 0x5642450ae7f0 2024-08-18T23:52:37.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.633+0000 7f098f7fe640 1 -- 172.21.15.78:0/507365789 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f099c01f030 con 0x5642450ae7f0 2024-08-18T23:52:37.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.633+0000 7f098f7fe640 1 -- 172.21.15.78:0/507365789 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c023c90 con 0x5642450ae7f0 2024-08-18T23:52:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.634+0000 7f09b2335740 1 -- 172.21.15.78:0/507365789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642450ae7f0 msgr2=0x5642450c6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.634+0000 7f09b2335740 1 --2- 172.21.15.78:0/507365789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642450ae7f0 0x5642450c6630 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x5642450c1060 tx=0x7f099c0047d0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.634+0000 7f09b2335740 1 -- 172.21.15.78:0/507365789 shutdown_connections 2024-08-18T23:52:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.634+0000 7f09b2335740 1 --2- 172.21.15.78:0/507365789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5642450ae7f0 0x5642450c6630 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.634+0000 7f09b2335740 1 --2- 172.21.15.78:0/507365789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56424509f9e0 0x5642450c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.634+0000 7f09b2335740 1 --2- 172.21.15.78:0/507365789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56424509f0c0 0x56424509a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.634+0000 7f09b2335740 1 -- 172.21.15.78:0/507365789 >> 172.21.15.78:0/507365789 conn(0x56424509b790 msgr2=0x56424509a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.634+0000 7f09b2335740 1 -- 172.21.15.78:0/507365789 shutdown_connections 2024-08-18T23:52:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.634+0000 7f09b2335740 1 -- 172.21.15.78:0/507365789 wait complete. 2024-08-18T23:52:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.634+0000 7f09b2335740 1 -- start start 2024-08-18T23:52:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.634+0000 7f09b2335740 1 Processor -- start 2024-08-18T23:52:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f09b2335740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56424509f0c0 0x5642450b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f09b2335740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 0x5642450b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:37.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f09b2335740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642450ae7f0 0x5642450b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f09b2335740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81aad90 con 0x56424509f9e0 2024-08-18T23:52:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f09b2335740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81ac200 con 0x5642450ae7f0 2024-08-18T23:52:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f09b2335740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81ab460 con 0x56424509f0c0 2024-08-18T23:52:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f09b2335740 10 client.0 _ll_register_callbacks cb 0x7f0998008ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f09adcd3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 0x5642450b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f09adcd3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 0x5642450b1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52104/0 (socket says 172.21.15.78:52104) 2024-08-18T23:52:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f09adcd3640 1 -- 172.21.15.78:0/4202349770 learned_addr learned my addr 172.21.15.78:0/4202349770 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f09ad4d2640 1 --2- 172.21.15.78:0/4202349770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642450ae7f0 0x5642450b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f09adcd3640 1 -- 172.21.15.78:0/4202349770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56424509f0c0 msgr2=0x5642450b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f09adcd3640 1 --2- 172.21.15.78:0/4202349770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56424509f0c0 0x5642450b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f09adcd3640 1 -- 172.21.15.78:0/4202349770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642450ae7f0 msgr2=0x5642450b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f09adcd3640 1 --2- 172.21.15.78:0/4202349770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642450ae7f0 0x5642450b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f09adcd3640 1 -- 172.21.15.78:0/4202349770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642450b2990 con 0x56424509f9e0 2024-08-18T23:52:37.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f09ad4d2640 1 --2- 172.21.15.78:0/4202349770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642450ae7f0 0x5642450b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:52:37.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f09adcd3640 1 --2- 172.21.15.78:0/4202349770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 0x5642450b1e90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0998018910 tx=0x7f099800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f230927cdd58fc server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:37.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f098d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:37.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f098d7fa640 1 -- 172.21.15.78:0/4202349770 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998027070 con 0x56424509f9e0 2024-08-18T23:52:37.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.635+0000 7f098d7fa640 1 -- 172.21.15.78:0/4202349770 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f0998018b00 con 0x56424509f9e0 2024-08-18T23:52:37.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.636+0000 7f09b2335740 1 -- 172.21.15.78:0/4202349770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f099c01f030 con 0x56424509f9e0 2024-08-18T23:52:37.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.636+0000 7f098d7fa640 1 -- 172.21.15.78:0/4202349770 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0998014070 con 0x56424509f9e0 2024-08-18T23:52:37.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.636+0000 7f09b2335740 10 client.16836 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:37.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.636+0000 7f09b2335740 1 -- 172.21.15.78:0/4202349770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564245128ed0 con 0x56424509f9e0 2024-08-18T23:52:37.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.636+0000 7f098d7fa640 1 -- 172.21.15.78:0/4202349770 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f099c01f030 con 0x56424509f9e0 2024-08-18T23:52:37.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.636+0000 7f098d7fa640 1 -- 172.21.15.78:0/4202349770 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f0998018410 con 0x56424509f9e0 2024-08-18T23:52:37.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.636+0000 7f09b2335740 10 client.16836 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:37.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.636+0000 7f09b2335740 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:52:37.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.636+0000 7f09b2335740 20 client.16836 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:37.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.636+0000 7f09b2335740 1 --2- 172.21.15.78:0/4202349770 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x564245129620 0x56424512ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:37.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.636+0000 7f09b2335740 4 client.16836 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-18T23:52:37.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.636+0000 7f09b2335740 1 -- 172.21.15.78:0/4202349770 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56424500c270 con 0x564245129620 2024-08-18T23:52:37.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.636+0000 7f09ad4d2640 1 --2- 172.21.15.78:0/4202349770 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x564245129620 0x56424512ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:37.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.637+0000 7f098d7fa640 10 client.16836 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:37.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.637+0000 7f09ad4d2640 1 --2- 172.21.15.78:0/4202349770 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x564245129620 0x56424512ba60 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff370116eb78ed92 server_cookie=25c98d060f6d7fcb in_seq=0 out_seq=0 2024-08-18T23:52:37.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.638+0000 7f098d7fa640 1 -- 172.21.15.78:0/4202349770 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56424500c270 con 0x564245129620 2024-08-18T23:52:37.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.638+0000 7f098d7fa640 10 client.16836 handle_command_reply: tid=0 2024-08-18T23:52:37.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.638+0000 7f09b2335740 1 client.16836 shutdown 2024-08-18T23:52:37.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.638+0000 7f09b2335740 1 -- 172.21.15.78:0/4202349770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 msgr2=0x5642450b1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:37.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.638+0000 7f09b2335740 1 --2- 172.21.15.78:0/4202349770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 0x5642450b1e90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0998018910 tx=0x7f099800ed30 comp rx=0 tx=0).stop 2024-08-18T23:52:37.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.638+0000 7f09b2335740 1 -- 172.21.15.78:0/4202349770 shutdown_connections 2024-08-18T23:52:37.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.638+0000 7f09b2335740 1 --2- 172.21.15.78:0/4202349770 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x564245129620 0x56424512ba60 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.638+0000 7f09b2335740 1 --2- 172.21.15.78:0/4202349770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642450ae7f0 0x5642450b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.638+0000 7f09b2335740 1 --2- 172.21.15.78:0/4202349770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56424509f9e0 0x5642450b1e90 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.638+0000 7f098d7fa640 0 client.16836 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:37.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.638+0000 7f09b2335740 1 --2- 172.21.15.78:0/4202349770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56424509f0c0 0x5642450b6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.638+0000 7f09b2335740 1 -- 172.21.15.78:0/4202349770 >> 172.21.15.78:0/4202349770 conn(0x56424509b790 msgr2=0x5642450a1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:37.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.638+0000 7f09b2335740 1 -- 172.21.15.78:0/4202349770 shutdown_connections 2024-08-18T23:52:37.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.638+0000 7f09b2335740 1 -- 172.21.15.78:0/4202349770 wait complete. 2024-08-18T23:52:37.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.639+0000 7f09b2335740 20 client.16836 trim_cache size 0 max 16384 2024-08-18T23:52:37.649 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:52:37.649 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-08-18T23:52:37.649 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "ec19d7bc-67c5-49e3-96bb-c7800fce10cb", 2024-08-18T23:52:37.649 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-08-18T23:52:37.649 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:52:37.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.639+0000 7f09aff37640 1 -- 172.21.15.78:0/1428753901 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f09900a2b80 msgr2=0x7f09900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:37.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.639+0000 7f09aff37640 1 --2- 172.21.15.78:0/1428753901 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f09900a2b80 0x7f09900a5040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f09a00b9340 tx=0x7f09a00879e0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.639+0000 7f09aff37640 1 -- 172.21.15.78:0/1428753901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09a810bea0 msgr2=0x7f09a819c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:37.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.639+0000 7f09aff37640 1 --2- 172.21.15.78:0/1428753901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09a810bea0 0x7f09a819c9d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f09a0006010 tx=0x7f09a002f710 comp rx=0 tx=0).stop 2024-08-18T23:52:37.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.639+0000 7f09aff37640 1 -- 172.21.15.78:0/1428753901 shutdown_connections 2024-08-18T23:52:37.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.639+0000 7f09aff37640 1 --2- 172.21.15.78:0/1428753901 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f09900a2b80 0x7f09900a5040 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.639+0000 7f09aff37640 1 --2- 172.21.15.78:0/1428753901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09a810bea0 0x7f09a819c9d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.639+0000 7f09aff37640 1 --2- 172.21.15.78:0/1428753901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09a8103390 0x7f09a8198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.639+0000 7f09aff37640 1 --2- 172.21.15.78:0/1428753901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09a8102a70 0x7f09a8198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:37.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.639+0000 7f09aff37640 1 -- 172.21.15.78:0/1428753901 >> 172.21.15.78:0/1428753901 conn(0x7f09a80fc4d0 msgr2=0x7f09a80fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:37.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.639+0000 7f09aff37640 1 -- 172.21.15.78:0/1428753901 shutdown_connections 2024-08-18T23:52:37.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:37.639+0000 7f09aff37640 1 -- 172.21.15.78:0/1428753901 wait complete. 2024-08-18T23:52:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:37 smithi161 ceph-mon[22027]: pgmap v599: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 524 KiB/s wr, 354 op/s 2024-08-18T23:52:37.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:37 smithi123 ceph-mon[21808]: pgmap v599: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 524 KiB/s wr, 354 op/s 2024-08-18T23:52:38.523 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:52:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.261+0000 7f2ba48b4640 1 -- 172.21.15.78:0/714295764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba0100fc0 msgr2=0x7f2ba0107780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.261+0000 7f2ba48b4640 1 --2- 172.21.15.78:0/714295764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba0100fc0 0x7f2ba0107780 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009900 tx=0x7f2b9402a3f0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.261+0000 7f2ba48b4640 1 -- 172.21.15.78:0/714295764 shutdown_connections 2024-08-18T23:52:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.261+0000 7f2ba48b4640 1 --2- 172.21.15.78:0/714295764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ba0107eb0 0x7f2ba010e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.261+0000 7f2ba48b4640 1 --2- 172.21.15.78:0/714295764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba0100fc0 0x7f2ba0107780 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.261+0000 7f2ba48b4640 1 --2- 172.21.15.78:0/714295764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ba0100610 0x7f2ba01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.261+0000 7f2ba48b4640 1 -- 172.21.15.78:0/714295764 >> 172.21.15.78:0/714295764 conn(0x7f2ba0076f40 msgr2=0x7f2ba0077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.261+0000 7f2ba48b4640 1 -- 172.21.15.78:0/714295764 shutdown_connections 2024-08-18T23:52:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.261+0000 7f2ba48b4640 1 -- 172.21.15.78:0/714295764 wait complete. 2024-08-18T23:52:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.262+0000 7f2ba48b4640 1 Processor -- start 2024-08-18T23:52:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.262+0000 7f2ba48b4640 1 -- start start 2024-08-18T23:52:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.262+0000 7f2ba48b4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ba0100610 0x7f2ba0198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.262+0000 7f2ba48b4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba0100fc0 0x7f2ba0198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.262+0000 7f2ba48b4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ba0107eb0 0x7f2ba019c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.262+0000 7f2ba48b4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba0111da0 con 0x7f2ba0100fc0 2024-08-18T23:52:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.262+0000 7f2ba48b4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba0111c20 con 0x7f2ba0100610 2024-08-18T23:52:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.262+0000 7f2ba48b4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba0111f20 con 0x7f2ba0107eb0 2024-08-18T23:52:39.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.262+0000 7f2b9dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba0100fc0 0x7f2ba0198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.262+0000 7f2b9dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba0100fc0 0x7f2ba0198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52118/0 (socket says 172.21.15.78:52118) 2024-08-18T23:52:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.262+0000 7f2b9dd9b640 1 -- 172.21.15.78:0/1262954200 learned_addr learned my addr 172.21.15.78:0/1262954200 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.262+0000 7f2b9ed9d640 1 --2- 172.21.15.78:0/1262954200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ba0107eb0 0x7f2ba019c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.263+0000 7f2b9e59c640 1 --2- 172.21.15.78:0/1262954200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ba0100610 0x7f2ba0198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.263+0000 7f2b9dd9b640 1 -- 172.21.15.78:0/1262954200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ba0107eb0 msgr2=0x7f2ba019c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.263+0000 7f2b9dd9b640 1 --2- 172.21.15.78:0/1262954200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ba0107eb0 0x7f2ba019c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.263+0000 7f2b9dd9b640 1 -- 172.21.15.78:0/1262954200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ba0100610 msgr2=0x7f2ba0198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.263+0000 7f2b9dd9b640 1 --2- 172.21.15.78:0/1262954200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ba0100610 0x7f2ba0198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.263+0000 7f2b9dd9b640 1 -- 172.21.15.78:0/1262954200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba019d0b0 con 0x7f2ba0100fc0 2024-08-18T23:52:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.263+0000 7f2b9dd9b640 1 --2- 172.21.15.78:0/1262954200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba0100fc0 0x7f2ba0198640 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2b94006010 tx=0x7f2b940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.263+0000 7f2b8f7fe640 1 -- 172.21.15.78:0/1262954200 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9403f030 con 0x7f2ba0100fc0 2024-08-18T23:52:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.263+0000 7f2b8f7fe640 1 -- 172.21.15.78:0/1262954200 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b94003ce0 con 0x7f2ba0100fc0 2024-08-18T23:52:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.263+0000 7f2ba48b4640 1 -- 172.21.15.78:0/1262954200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba019d340 con 0x7f2ba0100fc0 2024-08-18T23:52:39.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.263+0000 7f2b8f7fe640 1 -- 172.21.15.78:0/1262954200 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94035070 con 0x7f2ba0100fc0 2024-08-18T23:52:39.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.263+0000 7f2ba48b4640 1 -- 172.21.15.78:0/1262954200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba01a4c80 con 0x7f2ba0100fc0 2024-08-18T23:52:39.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.265+0000 7f2b8f7fe640 1 -- 172.21.15.78:0/1262954200 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2b94042030 con 0x7f2ba0100fc0 2024-08-18T23:52:39.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.266+0000 7f2ba6cb2740 1 Processor -- start 2024-08-18T23:52:39.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.266+0000 7f2ba6cb2740 1 -- start start 2024-08-18T23:52:39.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.266+0000 7f2ba6cb2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd154370d0 0x55cd154374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:39.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.266+0000 7f2ba6cb2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 0x55cd154462c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:39.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.266+0000 7f2ba6cb2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd15446800 0x55cd15448bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:39.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.266+0000 7f2ba6cb2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cd1539c980 con 0x55cd154379f0 2024-08-18T23:52:39.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.266+0000 7f2ba6cb2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cd15334a70 con 0x55cd15446800 2024-08-18T23:52:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.266+0000 7f2ba6cb2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cd15398940 con 0x55cd154370d0 2024-08-18T23:52:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.266+0000 7f2b9e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 0x55cd154462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.266+0000 7f2b9e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 0x55cd154462c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52126/0 (socket says 172.21.15.78:52126) 2024-08-18T23:52:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.266+0000 7f2b9e59c640 1 -- 172.21.15.78:0/1490399746 learned_addr learned my addr 172.21.15.78:0/1490399746 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.266+0000 7f2b9dd9b640 1 --2- 172.21.15.78:0/1490399746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd15446800 0x55cd15448bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.266+0000 7f2b8f7fe640 1 --2- 172.21.15.78:0/1262954200 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2b880a2b80 0x7f2b880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.267+0000 7f2b9ed9d640 1 --2- 172.21.15.78:0/1490399746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd154370d0 0x55cd154374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.267+0000 7f2b9ed9d640 1 --2- 172.21.15.78:0/1262954200 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2b880a2b80 0x7f2b880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.267+0000 7f2b9e59c640 1 -- 172.21.15.78:0/1490399746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd154370d0 msgr2=0x55cd154374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.267+0000 7f2b9e59c640 1 --2- 172.21.15.78:0/1490399746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd154370d0 0x55cd154374b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.267+0000 7f2b9e59c640 1 -- 172.21.15.78:0/1490399746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd15446800 msgr2=0x55cd15448bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.267+0000 7f2b9e59c640 1 --2- 172.21.15.78:0/1490399746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd15446800 0x55cd15448bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.267+0000 7f2b9e59c640 1 -- 172.21.15.78:0/1490399746 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd153976e0 con 0x55cd154379f0 2024-08-18T23:52:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.267+0000 7f2b9ed9d640 1 --2- 172.21.15.78:0/1490399746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd154370d0 0x55cd154374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.267+0000 7f2b8f7fe640 1 -- 172.21.15.78:0/1262954200 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2b940efa70 con 0x7f2ba0100fc0 2024-08-18T23:52:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.267+0000 7f2b9e59c640 1 --2- 172.21.15.78:0/1490399746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 0x55cd154462c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000cc60 tx=0x7f2b9000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218cb5ebb3c4be49 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.267+0000 7f2b9ed9d640 1 --2- 172.21.15.78:0/1262954200 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2b880a2b80 0x7f2b880a5040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2b98013600 tx=0x7f2b98004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.267+0000 7f2b8cff9640 1 -- 172.21.15.78:0/1490399746 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b90014070 con 0x55cd154379f0 2024-08-18T23:52:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.267+0000 7f2b8cff9640 1 -- 172.21.15.78:0/1490399746 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b900047a0 con 0x55cd154379f0 2024-08-18T23:52:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.267+0000 7f2b8cff9640 1 -- 172.21.15.78:0/1490399746 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b90010720 con 0x55cd154379f0 2024-08-18T23:52:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.268+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/1490399746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 msgr2=0x55cd154462c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.268+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/1490399746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 0x55cd154462c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000cc60 tx=0x7f2b9000e720 comp rx=0 tx=0).stop 2024-08-18T23:52:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.268+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/1490399746 shutdown_connections 2024-08-18T23:52:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.268+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/1490399746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd15446800 0x55cd15448bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.268+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/1490399746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 0x55cd154462c0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.268+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/1490399746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd154370d0 0x55cd154374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.268+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/1490399746 >> 172.21.15.78:0/1490399746 conn(0x55cd154337a0 msgr2=0x55cd15433b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.268+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/1490399746 shutdown_connections 2024-08-18T23:52:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.268+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/1490399746 wait complete. 2024-08-18T23:52:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.268+0000 7f2ba6cb2740 1 -- start start 2024-08-18T23:52:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.268+0000 7f2ba6cb2740 1 Processor -- start 2024-08-18T23:52:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2ba6cb2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd154370d0 0x55cd15443740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2ba6cb2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd154379f0 0x55cd15443c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2ba6cb2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd15446800 0x55cd154441c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2ba6cb2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cd154493d0 con 0x55cd15446800 2024-08-18T23:52:39.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2ba6cb2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cd15357aa0 con 0x55cd154379f0 2024-08-18T23:52:39.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2ba6cb2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cd15449550 con 0x55cd154370d0 2024-08-18T23:52:39.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2ba6cb2740 10 client.0 _ll_register_callbacks cb 0x55cd15346e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:39.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2b9ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd15446800 0x55cd154441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:39.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2b9ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd15446800 0x55cd154441c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52136/0 (socket says 172.21.15.78:52136) 2024-08-18T23:52:39.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2b9ed9d640 1 -- 172.21.15.78:0/2166617131 learned_addr learned my addr 172.21.15.78:0/2166617131 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:39.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2b9dd9b640 1 --2- 172.21.15.78:0/2166617131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd154379f0 0x55cd15443c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:39.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2b9e59c640 1 --2- 172.21.15.78:0/2166617131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd154370d0 0x55cd15443740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:39.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2b9ed9d640 1 -- 172.21.15.78:0/2166617131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd154370d0 msgr2=0x55cd15443740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:39.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2b9ed9d640 1 --2- 172.21.15.78:0/2166617131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd154370d0 0x55cd15443740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2b9ed9d640 1 -- 172.21.15.78:0/2166617131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd154379f0 msgr2=0x55cd15443c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:39.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2b9ed9d640 1 --2- 172.21.15.78:0/2166617131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd154379f0 0x55cd15443c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2b9ed9d640 1 -- 172.21.15.78:0/2166617131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd1533b650 con 0x55cd15446800 2024-08-18T23:52:39.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2b9e59c640 1 --2- 172.21.15.78:0/2166617131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd154370d0 0x55cd15443740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:39.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2b9ed9d640 1 --2- 172.21.15.78:0/2166617131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd15446800 0x55cd154441c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800ef30 tx=0x7f2b98004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ab285e51226b01 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:39.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2b85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:39.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2b85ffb640 1 -- 172.21.15.78:0/2166617131 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b98022070 con 0x55cd15446800 2024-08-18T23:52:39.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.269+0000 7f2b85ffb640 1 -- 172.21.15.78:0/2166617131 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2b98010c80 con 0x55cd15446800 2024-08-18T23:52:39.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.270+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/2166617131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b900047a0 con 0x55cd15446800 2024-08-18T23:52:39.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.270+0000 7f2b85ffb640 1 -- 172.21.15.78:0/2166617131 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2b9802e050 con 0x55cd15446800 2024-08-18T23:52:39.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.270+0000 7f2ba6cb2740 10 client.16848 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:39.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.270+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/2166617131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cd1510e3c0 con 0x55cd15446800 2024-08-18T23:52:39.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.270+0000 7f2b85ffb640 1 -- 172.21.15.78:0/2166617131 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b900047a0 con 0x55cd15446800 2024-08-18T23:52:39.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.270+0000 7f2b85ffb640 1 -- 172.21.15.78:0/2166617131 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f2b9801d040 con 0x55cd15446800 2024-08-18T23:52:39.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.270+0000 7f2ba6cb2740 10 client.16848 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:39.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.270+0000 7f2ba6cb2740 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:52:39.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.270+0000 7f2ba6cb2740 20 client.16848 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:39.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.270+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/2166617131 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cd15456360 0x55cd154587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:39.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.270+0000 7f2ba6cb2740 4 client.16848 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:52:39.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.270+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/2166617131 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd153a9ef0 con 0x55cd15456360 2024-08-18T23:52:39.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.270+0000 7f2b9e59c640 1 --2- 172.21.15.78:0/2166617131 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cd15456360 0x55cd154587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:39.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.271+0000 7f2b9e59c640 1 --2- 172.21.15.78:0/2166617131 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cd15456360 0x55cd154587d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fea48617cc42941 server_cookie=7f27a1df094bf3c8 in_seq=0 out_seq=0 2024-08-18T23:52:39.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.271+0000 7f2b85ffb640 10 client.16848 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.271+0000 7f2b85ffb640 1 -- 172.21.15.78:0/2166617131 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd153a9ef0 con 0x55cd15456360 2024-08-18T23:52:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.271+0000 7f2b85ffb640 10 client.16848 handle_command_reply: tid=0 2024-08-18T23:52:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.272+0000 7f2ba6cb2740 1 client.16848 shutdown 2024-08-18T23:52:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.272+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/2166617131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd15446800 msgr2=0x55cd154441c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.272+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/2166617131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd15446800 0x55cd154441c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800ef30 tx=0x7f2b98004ab0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.272+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/2166617131 shutdown_connections 2024-08-18T23:52:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.272+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/2166617131 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cd15456360 0x55cd154587d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.272+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/2166617131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd15446800 0x55cd154441c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.272+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/2166617131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd154379f0 0x55cd15443c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.272+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/2166617131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd154370d0 0x55cd15443740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.272+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/2166617131 >> 172.21.15.78:0/2166617131 conn(0x55cd154337a0 msgr2=0x55cd15439500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:39.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.272+0000 7f2b85ffb640 0 client.16848 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:39.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.272+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/2166617131 shutdown_connections 2024-08-18T23:52:39.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.272+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/2166617131 wait complete. 2024-08-18T23:52:39.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.272+0000 7f2ba6cb2740 20 client.16848 trim_cache size 0 max 16384 2024-08-18T23:52:39.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.288+0000 7f2ba6cb2740 1 Processor -- start 2024-08-18T23:52:39.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.289+0000 7f2ba6cb2740 1 -- start start 2024-08-18T23:52:39.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.289+0000 7f2ba6cb2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd154370d0 0x55cd1545e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:39.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.289+0000 7f2ba6cb2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 0x55cd1545e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:39.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.289+0000 7f2ba6cb2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd15446800 0x55cd154409a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:39.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.289+0000 7f2ba6cb2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba01aefd0 con 0x55cd154379f0 2024-08-18T23:52:39.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.289+0000 7f2ba6cb2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba01ac840 con 0x55cd15446800 2024-08-18T23:52:39.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.289+0000 7f2ba6cb2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba01aa690 con 0x55cd154370d0 2024-08-18T23:52:39.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.289+0000 7f2b9dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 0x55cd1545e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:39.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.289+0000 7f2b9dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 0x55cd1545e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52140/0 (socket says 172.21.15.78:52140) 2024-08-18T23:52:39.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.289+0000 7f2b9dd9b640 1 -- 172.21.15.78:0/1920140793 learned_addr learned my addr 172.21.15.78:0/1920140793 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:39.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.289+0000 7f2b9e59c640 1 --2- 172.21.15.78:0/1920140793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd154370d0 0x55cd1545e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:39.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.289+0000 7f2b9dd9b640 1 -- 172.21.15.78:0/1920140793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd154370d0 msgr2=0x55cd1545e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:39.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.289+0000 7f2b9dd9b640 1 --2- 172.21.15.78:0/1920140793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd154370d0 0x55cd1545e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.289+0000 7f2b9dd9b640 1 -- 172.21.15.78:0/1920140793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd15446800 msgr2=0x55cd154409a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:39.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.289+0000 7f2b9dd9b640 1 --2- 172.21.15.78:0/1920140793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd15446800 0x55cd154409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.289+0000 7f2b9dd9b640 1 -- 172.21.15.78:0/1920140793 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd15441c90 con 0x55cd154379f0 2024-08-18T23:52:39.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.289+0000 7f2b9dd9b640 1 --2- 172.21.15.78:0/1920140793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 0x55cd1545e5e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f2b940ed550 tx=0x7f2b940dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32aa5e29d3aa689 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:39.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.289+0000 7f2b87fff640 1 -- 172.21.15.78:0/1920140793 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94049040 con 0x55cd154379f0 2024-08-18T23:52:39.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.290+0000 7f2b87fff640 1 -- 172.21.15.78:0/1920140793 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b940bbf40 con 0x55cd154379f0 2024-08-18T23:52:39.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.290+0000 7f2b87fff640 1 -- 172.21.15.78:0/1920140793 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9403b460 con 0x55cd154379f0 2024-08-18T23:52:39.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.290+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/1920140793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 msgr2=0x55cd1545e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:39.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.290+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/1920140793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 0x55cd1545e5e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f2b940ed550 tx=0x7f2b940dc040 comp rx=0 tx=0).stop 2024-08-18T23:52:39.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.290+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/1920140793 shutdown_connections 2024-08-18T23:52:39.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.290+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/1920140793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd15446800 0x55cd154409a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.290+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/1920140793 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 0x55cd1545e5e0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.290+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/1920140793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd154370d0 0x55cd1545e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.290+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/1920140793 >> 172.21.15.78:0/1920140793 conn(0x55cd154337a0 msgr2=0x55cd154456b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:39.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.290+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/1920140793 shutdown_connections 2024-08-18T23:52:39.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.290+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/1920140793 wait complete. 2024-08-18T23:52:39.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.290+0000 7f2ba6cb2740 1 -- start start 2024-08-18T23:52:39.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.290+0000 7f2ba6cb2740 1 Processor -- start 2024-08-18T23:52:39.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2ba6cb2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd154370d0 0x55cd1544ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:39.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2ba6cb2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 0x55cd1544f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2ba6cb2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd15446800 0x55cd1544a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2ba6cb2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba01aad00 con 0x55cd154379f0 2024-08-18T23:52:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2ba6cb2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba01ac170 con 0x55cd154370d0 2024-08-18T23:52:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2ba6cb2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba01ab3d0 con 0x55cd15446800 2024-08-18T23:52:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2ba6cb2740 10 client.0 _ll_register_callbacks cb 0x7f2b98007c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2b9dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 0x55cd1544f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2b9dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 0x55cd1544f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52142/0 (socket says 172.21.15.78:52142) 2024-08-18T23:52:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2b9dd9b640 1 -- 172.21.15.78:0/2428203074 learned_addr learned my addr 172.21.15.78:0/2428203074 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2b9ed9d640 1 --2- 172.21.15.78:0/2428203074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd15446800 0x55cd1544a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2b9e59c640 1 --2- 172.21.15.78:0/2428203074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd154370d0 0x55cd1544ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2b9dd9b640 1 -- 172.21.15.78:0/2428203074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd15446800 msgr2=0x55cd1544a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2b9dd9b640 1 --2- 172.21.15.78:0/2428203074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd15446800 0x55cd1544a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2b9dd9b640 1 -- 172.21.15.78:0/2428203074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd154370d0 msgr2=0x55cd1544ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2b9dd9b640 1 --2- 172.21.15.78:0/2428203074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd154370d0 0x55cd1544ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2b9dd9b640 1 -- 172.21.15.78:0/2428203074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd1544a980 con 0x55cd154379f0 2024-08-18T23:52:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2b9ed9d640 1 --2- 172.21.15.78:0/2428203074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd15446800 0x55cd1544a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2b9dd9b640 1 --2- 172.21.15.78:0/2428203074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 0x55cd1544f110 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2b94032c00 tx=0x7f2b940c4c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834787270cd8587a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2b85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2b85ffb640 1 -- 172.21.15.78:0/2428203074 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b940bb8a0 con 0x55cd154379f0 2024-08-18T23:52:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2b85ffb640 1 -- 172.21.15.78:0/2428203074 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2b940b85b0 con 0x55cd154379f0 2024-08-18T23:52:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.291+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/2428203074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cd153a4400 con 0x55cd154379f0 2024-08-18T23:52:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.292+0000 7f2b85ffb640 1 -- 172.21.15.78:0/2428203074 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2b940bc640 con 0x55cd154379f0 2024-08-18T23:52:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.292+0000 7f2ba6cb2740 10 client.16854 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.292+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/2428203074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cd154c0d30 con 0x55cd154379f0 2024-08-18T23:52:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.292+0000 7f2b85ffb640 1 -- 172.21.15.78:0/2428203074 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cd153a4400 con 0x55cd154379f0 2024-08-18T23:52:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.292+0000 7f2b85ffb640 1 -- 172.21.15.78:0/2428203074 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f2b940c4720 con 0x55cd154379f0 2024-08-18T23:52:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.292+0000 7f2ba6cb2740 10 client.16854 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.292+0000 7f2ba6cb2740 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:52:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.292+0000 7f2ba6cb2740 20 client.16854 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.292+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/2428203074 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cd154c1340 0x55cd154c3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.292+0000 7f2ba6cb2740 4 client.16854 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:52:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.292+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/2428203074 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cd15321210 con 0x55cd154c1340 2024-08-18T23:52:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.292+0000 7f2b9e59c640 1 --2- 172.21.15.78:0/2428203074 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cd154c1340 0x55cd154c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.293+0000 7f2b9e59c640 1 --2- 172.21.15.78:0/2428203074 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cd154c1340 0x55cd154c3760 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71bd197d24b6120b server_cookie=c1fe40542e6cebc2 in_seq=0 out_seq=0 2024-08-18T23:52:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.293+0000 7f2b85ffb640 10 client.16854 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.293+0000 7f2b85ffb640 1 -- 172.21.15.78:0/2428203074 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cd15321210 con 0x55cd154c1340 2024-08-18T23:52:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.293+0000 7f2b85ffb640 10 client.16854 handle_command_reply: tid=0 2024-08-18T23:52:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.293+0000 7f2ba6cb2740 1 client.16854 shutdown 2024-08-18T23:52:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.293+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/2428203074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 msgr2=0x55cd1544f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:39.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.293+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/2428203074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 0x55cd1544f110 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2b94032c00 tx=0x7f2b940c4c20 comp rx=0 tx=0).stop 2024-08-18T23:52:39.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.293+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/2428203074 shutdown_connections 2024-08-18T23:52:39.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.293+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/2428203074 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55cd154c1340 0x55cd154c3760 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.293+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/2428203074 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd15446800 0x55cd1544a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.293+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/2428203074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd154379f0 0x55cd1544f110 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.293+0000 7f2ba6cb2740 1 --2- 172.21.15.78:0/2428203074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cd154370d0 0x55cd1544ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.293+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/2428203074 >> 172.21.15.78:0/2428203074 conn(0x55cd154337a0 msgr2=0x55cd15443700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:39.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.293+0000 7f2b85ffb640 0 client.16854 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:39.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.293+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/2428203074 shutdown_connections 2024-08-18T23:52:39.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.293+0000 7f2ba6cb2740 1 -- 172.21.15.78:0/2428203074 wait complete. 2024-08-18T23:52:39.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.293+0000 7f2ba6cb2740 20 client.16854 trim_cache size 0 max 16384 2024-08-18T23:52:39.300 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:52:39.301 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-08-18T23:52:39.301 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-08-18T23:52:39.301 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:52:39.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.294+0000 7f2ba48b4640 1 -- 172.21.15.78:0/1262954200 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2b880a2b80 msgr2=0x7f2b880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:39.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.294+0000 7f2ba48b4640 1 --2- 172.21.15.78:0/1262954200 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2b880a2b80 0x7f2b880a5040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2b98013600 tx=0x7f2b98004a40 comp rx=0 tx=0).stop 2024-08-18T23:52:39.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.294+0000 7f2ba48b4640 1 -- 172.21.15.78:0/1262954200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba0100fc0 msgr2=0x7f2ba0198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:39.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.294+0000 7f2ba48b4640 1 --2- 172.21.15.78:0/1262954200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba0100fc0 0x7f2ba0198640 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2b94006010 tx=0x7f2b940042f0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.294+0000 7f2ba48b4640 1 -- 172.21.15.78:0/1262954200 shutdown_connections 2024-08-18T23:52:39.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.294+0000 7f2ba48b4640 1 --2- 172.21.15.78:0/1262954200 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2b880a2b80 0x7f2b880a5040 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.294+0000 7f2ba48b4640 1 --2- 172.21.15.78:0/1262954200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ba0107eb0 0x7f2ba019c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.294+0000 7f2ba48b4640 1 --2- 172.21.15.78:0/1262954200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ba0100fc0 0x7f2ba0198640 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.294+0000 7f2ba48b4640 1 --2- 172.21.15.78:0/1262954200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ba0100610 0x7f2ba0198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:39.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.294+0000 7f2ba48b4640 1 -- 172.21.15.78:0/1262954200 >> 172.21.15.78:0/1262954200 conn(0x7f2ba0076f40 msgr2=0x7f2ba00ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:39.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.294+0000 7f2ba48b4640 1 -- 172.21.15.78:0/1262954200 shutdown_connections 2024-08-18T23:52:39.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:39.294+0000 7f2ba48b4640 1 -- 172.21.15.78:0/1262954200 wait complete. 2024-08-18T23:52:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:39 smithi161 ceph-mon[22027]: scrub queued for path: / 2024-08-18T23:52:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:39 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:52:39.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:39 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:52:39.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:39 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:52:39.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:39 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:52:39.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:39 smithi161 ceph-mon[22027]: pgmap v600: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 419 KiB/s wr, 226 op/s 2024-08-18T23:52:39.721 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:39 smithi078 ceph-mon[18440]: scrub queued for path: / 2024-08-18T23:52:39.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:39 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:52:39.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:39 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:52:39.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:39 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:52:39.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:39 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:52:39.722 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:39 smithi078 ceph-mon[18440]: pgmap v600: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 419 KiB/s wr, 226 op/s 2024-08-18T23:52:39.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:39 smithi123 ceph-mon[21808]: scrub queued for path: / 2024-08-18T23:52:39.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:39 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:52:39.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:39 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:52:39.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:39 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:52:39.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:39 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:52:39.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:39 smithi123 ceph-mon[21808]: pgmap v600: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 419 KiB/s wr, 226 op/s 2024-08-18T23:52:40.049 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-18T23:52:40.049 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:52:40.297 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:40.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.777+0000 7f406c3ce640 1 -- 172.21.15.78:0/3594237479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4064102b20 msgr2=0x7f4064102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:40.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.777+0000 7f406c3ce640 1 --2- 172.21.15.78:0/3594237479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4064102b20 0x7f4064102f00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f405c009900 tx=0x7f405c02a410 comp rx=0 tx=0).stop 2024-08-18T23:52:40.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.777+0000 7f406c3ce640 1 -- 172.21.15.78:0/3594237479 shutdown_connections 2024-08-18T23:52:40.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.777+0000 7f406c3ce640 1 --2- 172.21.15.78:0/3594237479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f406410c020 0x7f406410e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:40.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.777+0000 7f406c3ce640 1 --2- 172.21.15.78:0/3594237479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40641034d0 0x7f406410b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:40.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.777+0000 7f406c3ce640 1 --2- 172.21.15.78:0/3594237479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4064102b20 0x7f4064102f00 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:40.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.777+0000 7f406c3ce640 1 -- 172.21.15.78:0/3594237479 >> 172.21.15.78:0/3594237479 conn(0x7f40640fc500 msgr2=0x7f40640fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:40.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.777+0000 7f406c3ce640 1 -- 172.21.15.78:0/3594237479 shutdown_connections 2024-08-18T23:52:40.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.777+0000 7f406c3ce640 1 -- 172.21.15.78:0/3594237479 wait complete. 2024-08-18T23:52:40.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f406c3ce640 1 Processor -- start 2024-08-18T23:52:40.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f406c3ce640 1 -- start start 2024-08-18T23:52:40.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f406c3ce640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4064102b20 0x7f4064198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:40.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f406c3ce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40641034d0 0x7f4064198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:40.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f406c3ce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f406410c020 0x7f406419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:40.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f406c3ce640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4064111e40 con 0x7f40641034d0 2024-08-18T23:52:40.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f406c3ce640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4064111cc0 con 0x7f4064102b20 2024-08-18T23:52:40.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f406c3ce640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4064111fc0 con 0x7f406410c020 2024-08-18T23:52:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f4069969640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40641034d0 0x7f4064198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f4069969640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40641034d0 0x7f4064198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52160/0 (socket says 172.21.15.78:52160) 2024-08-18T23:52:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f4069969640 1 -- 172.21.15.78:0/2358888396 learned_addr learned my addr 172.21.15.78:0/2358888396 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f406a16a640 1 --2- 172.21.15.78:0/2358888396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4064102b20 0x7f4064198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f406a96b640 1 --2- 172.21.15.78:0/2358888396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f406410c020 0x7f406419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f4069969640 1 -- 172.21.15.78:0/2358888396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f406410c020 msgr2=0x7f406419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f4069969640 1 --2- 172.21.15.78:0/2358888396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f406410c020 0x7f406419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f4069969640 1 -- 172.21.15.78:0/2358888396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4064102b20 msgr2=0x7f4064198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f4069969640 1 --2- 172.21.15.78:0/2358888396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4064102b20 0x7f4064198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f4069969640 1 -- 172.21.15.78:0/2358888396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406419d2d0 con 0x7f40641034d0 2024-08-18T23:52:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f406a96b640 1 --2- 172.21.15.78:0/2358888396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f406410c020 0x7f406419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:52:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f406a16a640 1 --2- 172.21.15.78:0/2358888396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4064102b20 0x7f4064198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:40.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.778+0000 7f4069969640 1 --2- 172.21.15.78:0/2358888396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40641034d0 0x7f4064198860 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f405800cce0 tx=0x7f405800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.779+0000 7f40637fe640 1 -- 172.21.15.78:0/2358888396 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058004b20 con 0x7f40641034d0 2024-08-18T23:52:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.779+0000 7f40637fe640 1 -- 172.21.15.78:0/2358888396 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4058004cc0 con 0x7f40641034d0 2024-08-18T23:52:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.779+0000 7f406c3ce640 1 -- 172.21.15.78:0/2358888396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406419d5c0 con 0x7f40641034d0 2024-08-18T23:52:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.779+0000 7f40637fe640 1 -- 172.21.15.78:0/2358888396 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405800f780 con 0x7f40641034d0 2024-08-18T23:52:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.779+0000 7f406c3ce640 1 -- 172.21.15.78:0/2358888396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40641a4ea0 con 0x7f40641034d0 2024-08-18T23:52:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.780+0000 7f406c3ce640 1 -- 172.21.15.78:0/2358888396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f406406a4c0 con 0x7f40641034d0 2024-08-18T23:52:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.781+0000 7f40637fe640 1 -- 172.21.15.78:0/2358888396 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f405800f920 con 0x7f40641034d0 2024-08-18T23:52:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.781+0000 7f40637fe640 1 --2- 172.21.15.78:0/2358888396 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f404c0a2b80 0x7f404c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.782+0000 7f406a16a640 1 --2- 172.21.15.78:0/2358888396 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f404c0a2b80 0x7f404c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:40.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.782+0000 7f40637fe640 1 -- 172.21.15.78:0/2358888396 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f40580cf800 con 0x7f40641034d0 2024-08-18T23:52:40.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.782+0000 7f406a16a640 1 --2- 172.21.15.78:0/2358888396 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f404c0a2b80 0x7f404c0a5040 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f405c005ab0 tx=0x7f405c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:40.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.785+0000 7f40637fe640 1 -- 172.21.15.78:0/2358888396 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4058014030 con 0x7f40641034d0 2024-08-18T23:52:40.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.985+0000 7f406c3ce640 1 -- 172.21.15.78:0/2358888396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4064102f00 con 0x7f40641034d0 2024-08-18T23:52:40.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.985+0000 7f40637fe640 1 -- 172.21.15.78:0/2358888396 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11933 (secure 0 0 0) 0x7f4064102f00 con 0x7f40641034d0 2024-08-18T23:52:40.987 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:52:40.987 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":31,"btime":"2024-08-18T23:51:58:009010+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:51:57.958854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24428,24427]},"id":1}]} 2024-08-18T23:52:40.987 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 31 2024-08-18T23:52:40.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.987+0000 7f406c3ce640 1 -- 172.21.15.78:0/2358888396 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f404c0a2b80 msgr2=0x7f404c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:40.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.987+0000 7f406c3ce640 1 --2- 172.21.15.78:0/2358888396 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f404c0a2b80 0x7f404c0a5040 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f405c005ab0 tx=0x7f405c005a20 comp rx=0 tx=0).stop 2024-08-18T23:52:40.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.987+0000 7f406c3ce640 1 -- 172.21.15.78:0/2358888396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40641034d0 msgr2=0x7f4064198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:40.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.987+0000 7f406c3ce640 1 --2- 172.21.15.78:0/2358888396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40641034d0 0x7f4064198860 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f405800cce0 tx=0x7f405800a620 comp rx=0 tx=0).stop 2024-08-18T23:52:40.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.987+0000 7f406c3ce640 1 -- 172.21.15.78:0/2358888396 shutdown_connections 2024-08-18T23:52:40.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.987+0000 7f406c3ce640 1 --2- 172.21.15.78:0/2358888396 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f404c0a2b80 0x7f404c0a5040 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:40.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.987+0000 7f406c3ce640 1 --2- 172.21.15.78:0/2358888396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f406410c020 0x7f406419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:40.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.987+0000 7f406c3ce640 1 --2- 172.21.15.78:0/2358888396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40641034d0 0x7f4064198860 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:40.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.987+0000 7f406c3ce640 1 --2- 172.21.15.78:0/2358888396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4064102b20 0x7f4064198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:40.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.987+0000 7f406c3ce640 1 -- 172.21.15.78:0/2358888396 >> 172.21.15.78:0/2358888396 conn(0x7f40640fc500 msgr2=0x7f40640fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:40.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.988+0000 7f406c3ce640 1 -- 172.21.15.78:0/2358888396 shutdown_connections 2024-08-18T23:52:40.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:40.988+0000 7f406c3ce640 1 -- 172.21.15.78:0/2358888396 wait complete. 2024-08-18T23:52:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:41 smithi078 ceph-mon[18440]: pgmap v601: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 419 KiB/s wr, 154 op/s 2024-08-18T23:52:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:41 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2358888396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:52:41.777 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-18T23:52:41.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:41 smithi123 ceph-mon[21808]: pgmap v601: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 419 KiB/s wr, 154 op/s 2024-08-18T23:52:41.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:41 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2358888396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:52:41.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:41 smithi161 ceph-mon[22027]: pgmap v601: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 419 KiB/s wr, 154 op/s 2024-08-18T23:52:41.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:41 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2358888396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:52:42.024 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:42.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.490+0000 7fc7a79dc640 1 -- 172.21.15.78:0/2175517336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7a01034b0 msgr2=0x7fc7a01078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:42.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.491+0000 7fc7a79dc640 1 --2- 172.21.15.78:0/2175517336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7a01034b0 0x7fc7a01078f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc794009930 tx=0x7fc79402a420 comp rx=0 tx=0).stop 2024-08-18T23:52:42.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.491+0000 7fc7a79dc640 1 -- 172.21.15.78:0/2175517336 shutdown_connections 2024-08-18T23:52:42.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.491+0000 7fc7a79dc640 1 --2- 172.21.15.78:0/2175517336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7a0108030 0x7fc7a010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.491+0000 7fc7a79dc640 1 --2- 172.21.15.78:0/2175517336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7a01034b0 0x7fc7a01078f0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.491+0000 7fc7a79dc640 1 --2- 172.21.15.78:0/2175517336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7a0102b00 0x7fc7a0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.491+0000 7fc7a79dc640 1 -- 172.21.15.78:0/2175517336 >> 172.21.15.78:0/2175517336 conn(0x7fc7a00fc500 msgr2=0x7fc7a00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.491+0000 7fc7a79dc640 1 -- 172.21.15.78:0/2175517336 shutdown_connections 2024-08-18T23:52:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.491+0000 7fc7a79dc640 1 -- 172.21.15.78:0/2175517336 wait complete. 2024-08-18T23:52:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.491+0000 7fc7a79dc640 1 Processor -- start 2024-08-18T23:52:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.491+0000 7fc7a79dc640 1 -- start start 2024-08-18T23:52:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.491+0000 7fc7a79dc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7a0102b00 0x7fc7a01982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.491+0000 7fc7a79dc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7a01034b0 0x7fc7a0198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc7a79dc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7a0108030 0x7fc7a019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc7a79dc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a0111e00 con 0x7fc7a0108030 2024-08-18T23:52:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc7a79dc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a0111c80 con 0x7fc7a01034b0 2024-08-18T23:52:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc7a79dc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a0111f80 con 0x7fc7a0102b00 2024-08-18T23:52:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc7a5f79640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7a0108030 0x7fc7a019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc7a5f79640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7a0108030 0x7fc7a019cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52180/0 (socket says 172.21.15.78:52180) 2024-08-18T23:52:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc7a5f79640 1 -- 172.21.15.78:0/2617081325 learned_addr learned my addr 172.21.15.78:0/2617081325 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:42.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc7a5778640 1 --2- 172.21.15.78:0/2617081325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7a0102b00 0x7fc7a01982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc7a4f77640 1 --2- 172.21.15.78:0/2617081325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7a01034b0 0x7fc7a0198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc7a5f79640 1 -- 172.21.15.78:0/2617081325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7a0102b00 msgr2=0x7fc7a01982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc7a5f79640 1 --2- 172.21.15.78:0/2617081325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7a0102b00 0x7fc7a01982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc7a5f79640 1 -- 172.21.15.78:0/2617081325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7a01034b0 msgr2=0x7fc7a0198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc7a5f79640 1 --2- 172.21.15.78:0/2617081325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7a01034b0 0x7fc7a0198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc7a5f79640 1 -- 172.21.15.78:0/2617081325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a019d270 con 0x7fc7a0108030 2024-08-18T23:52:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc7a4f77640 1 --2- 172.21.15.78:0/2617081325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7a01034b0 0x7fc7a0198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:52:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc7a5f79640 1 --2- 172.21.15.78:0/2617081325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7a0108030 0x7fc7a019cb90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc79000eaa0 tx=0x7fc79000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc79e7fc640 1 -- 172.21.15.78:0/2617081325 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79000cdb0 con 0x7fc7a0108030 2024-08-18T23:52:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc79e7fc640 1 -- 172.21.15.78:0/2617081325 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc790005020 con 0x7fc7a0108030 2024-08-18T23:52:42.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc7a79dc640 1 -- 172.21.15.78:0/2617081325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a019d560 con 0x7fc7a0108030 2024-08-18T23:52:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc79e7fc640 1 -- 172.21.15.78:0/2617081325 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7900116b0 con 0x7fc7a0108030 2024-08-18T23:52:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.492+0000 7fc7a79dc640 1 -- 172.21.15.78:0/2617081325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a01a4e40 con 0x7fc7a0108030 2024-08-18T23:52:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.494+0000 7fc79e7fc640 1 -- 172.21.15.78:0/2617081325 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc790005560 con 0x7fc7a0108030 2024-08-18T23:52:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.495+0000 7fc7a9dda740 1 Processor -- start 2024-08-18T23:52:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.495+0000 7fc7a9dda740 1 -- start start 2024-08-18T23:52:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.495+0000 7fc7a9dda740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1a992b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.495+0000 7fc7a9dda740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653d1a997f0 0x5653d1aa80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.495+0000 7fc7a9dda740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1aa85e0 0x5653d1aaa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:42.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.495+0000 7fc7a9dda740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5653d19fea40 con 0x5653d1a98ed0 2024-08-18T23:52:42.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.495+0000 7fc7a9dda740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5653d19969d0 con 0x5653d1a997f0 2024-08-18T23:52:42.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.495+0000 7fc7a9dda740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5653d19faa00 con 0x5653d1aa85e0 2024-08-18T23:52:42.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.495+0000 7fc79e7fc640 1 --2- 172.21.15.78:0/2617081325 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc7880a2c00 0x7fc7880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:42.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.495+0000 7fc7a4f77640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1a992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:42.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.495+0000 7fc7a4f77640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1a992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52186/0 (socket says 172.21.15.78:52186) 2024-08-18T23:52:42.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.495+0000 7fc7a4f77640 1 -- 172.21.15.78:0/1308063967 learned_addr learned my addr 172.21.15.78:0/1308063967 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:42.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.495+0000 7fc7a5f79640 1 --2- 172.21.15.78:0/2617081325 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc7880a2c00 0x7fc7880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:42.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.496+0000 7fc7a5778640 1 --2- 172.21.15.78:0/1308063967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653d1a997f0 0x5653d1aa80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:42.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.496+0000 7fc7a4f77640 1 --2- 172.21.15.78:0/1308063967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1aa85e0 0x5653d1aaa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:42.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.496+0000 7fc79e7fc640 1 -- 172.21.15.78:0/2617081325 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc7900cf5c0 con 0x7fc7a0108030 2024-08-18T23:52:42.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.496+0000 7fc7a4f77640 1 -- 172.21.15.78:0/1308063967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1aa85e0 msgr2=0x5653d1aaa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:42.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.496+0000 7fc7a4f77640 1 --2- 172.21.15.78:0/1308063967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1aa85e0 0x5653d1aaa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.496+0000 7fc7a4f77640 1 -- 172.21.15.78:0/1308063967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653d1a997f0 msgr2=0x5653d1aa80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:42.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.496+0000 7fc7a4f77640 1 --2- 172.21.15.78:0/1308063967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653d1a997f0 0x5653d1aa80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.496+0000 7fc7a4f77640 1 -- 172.21.15.78:0/1308063967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653d199d650 con 0x5653d1a98ed0 2024-08-18T23:52:42.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.496+0000 7fc7a4f77640 1 --2- 172.21.15.78:0/1308063967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1a992b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc794006010 tx=0x7fc794005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2441ed9337a18dfb server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:42.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.496+0000 7fc7a5f79640 1 --2- 172.21.15.78:0/2617081325 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc7880a2c00 0x7fc7880a50c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc7900ccd80 tx=0x7fc7900054d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:42.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.496+0000 7fc7877fe640 1 -- 172.21.15.78:0/1308063967 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc794035050 con 0x5653d1a98ed0 2024-08-18T23:52:42.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.496+0000 7fc7877fe640 1 -- 172.21.15.78:0/1308063967 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc794031980 con 0x5653d1a98ed0 2024-08-18T23:52:42.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.496+0000 7fc7877fe640 1 -- 172.21.15.78:0/1308063967 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7940046c0 con 0x5653d1a98ed0 2024-08-18T23:52:42.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.497+0000 7fc7a9dda740 1 -- 172.21.15.78:0/1308063967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 msgr2=0x5653d1a992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:42.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.497+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/1308063967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1a992b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc794006010 tx=0x7fc794005bb0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.497+0000 7fc7a9dda740 1 -- 172.21.15.78:0/1308063967 shutdown_connections 2024-08-18T23:52:42.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.497+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/1308063967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1aa85e0 0x5653d1aaa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.497+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/1308063967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653d1a997f0 0x5653d1aa80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.497+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/1308063967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1a992b0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.497+0000 7fc7a9dda740 1 -- 172.21.15.78:0/1308063967 >> 172.21.15.78:0/1308063967 conn(0x5653d1a95580 msgr2=0x5653d1a95960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:42.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.497+0000 7fc7a9dda740 1 -- 172.21.15.78:0/1308063967 shutdown_connections 2024-08-18T23:52:42.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.497+0000 7fc7a9dda740 1 -- 172.21.15.78:0/1308063967 wait complete. 2024-08-18T23:52:42.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.497+0000 7fc7a9dda740 1 -- start start 2024-08-18T23:52:42.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.497+0000 7fc7a9dda740 1 Processor -- start 2024-08-18T23:52:42.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc7a9dda740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1aa5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:42.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc7a9dda740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1a997f0 0x5653d1aa5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:42.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc7a9dda740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653d1aa85e0 0x5653d1aa5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:42.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc7a9dda740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5653d1aab1b0 con 0x5653d1a98ed0 2024-08-18T23:52:42.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc7a9dda740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5653d19b9b20 con 0x5653d1aa85e0 2024-08-18T23:52:42.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc7a9dda740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5653d1aab330 con 0x5653d1a997f0 2024-08-18T23:52:42.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc7a9dda740 10 client.0 _ll_register_callbacks cb 0x5653d19b7bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:42.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc7a4f77640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1aa5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:42.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc7a4f77640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1aa5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52190/0 (socket says 172.21.15.78:52190) 2024-08-18T23:52:42.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc7a4f77640 1 -- 172.21.15.78:0/1125766020 learned_addr learned my addr 172.21.15.78:0/1125766020 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:42.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc7a4f77640 1 --2- 172.21.15.78:0/1125766020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653d1aa85e0 0x5653d1aa5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:42.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc7a5778640 1 --2- 172.21.15.78:0/1125766020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1a997f0 0x5653d1aa5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:42.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc7a4f77640 1 -- 172.21.15.78:0/1125766020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1a997f0 msgr2=0x5653d1aa5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:42.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc7a4f77640 1 --2- 172.21.15.78:0/1125766020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1a997f0 0x5653d1aa5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc7a4f77640 1 -- 172.21.15.78:0/1125766020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653d1aa85e0 msgr2=0x5653d1aa5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:42.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc7a4f77640 1 --2- 172.21.15.78:0/1125766020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653d1aa85e0 0x5653d1aa5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc7a4f77640 1 -- 172.21.15.78:0/1125766020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653d17703c0 con 0x5653d1a98ed0 2024-08-18T23:52:42.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc7a5778640 1 --2- 172.21.15.78:0/1125766020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1a997f0 0x5653d1aa5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:52:42.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc7a4f77640 1 --2- 172.21.15.78:0/1125766020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1aa5500 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc794031550 tx=0x7fc794031b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13a00a2091a6b85 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:42.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc784ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:42.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.498+0000 7fc784ff9640 1 -- 172.21.15.78:0/1125766020 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc794052060 con 0x5653d1a98ed0 2024-08-18T23:52:42.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.499+0000 7fc784ff9640 1 -- 172.21.15.78:0/1125766020 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc794003a00 con 0x5653d1a98ed0 2024-08-18T23:52:42.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.499+0000 7fc7a9dda740 1 -- 172.21.15.78:0/1125766020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc794031980 con 0x5653d1a98ed0 2024-08-18T23:52:42.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.499+0000 7fc784ff9640 1 -- 172.21.15.78:0/1125766020 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc794059050 con 0x5653d1a98ed0 2024-08-18T23:52:42.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.499+0000 7fc7a9dda740 10 client.16872 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:42.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.499+0000 7fc7a9dda740 1 -- 172.21.15.78:0/1125766020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5653d1ab6e30 con 0x5653d1a98ed0 2024-08-18T23:52:42.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.499+0000 7fc784ff9640 1 -- 172.21.15.78:0/1125766020 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc794031980 con 0x5653d1a98ed0 2024-08-18T23:52:42.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.499+0000 7fc784ff9640 1 -- 172.21.15.78:0/1125766020 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fc794039410 con 0x5653d1a98ed0 2024-08-18T23:52:42.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.499+0000 7fc7a9dda740 10 client.16872 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:42.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.499+0000 7fc7a9dda740 10 client.16872 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:52:42.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.499+0000 7fc7a9dda740 20 client.16872 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:42.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.499+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/1125766020 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5653d1ab8180 0x5653d1aba570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:42.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.499+0000 7fc7a9dda740 4 client.16872 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:52:42.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.499+0000 7fc7a9dda740 1 -- 172.21.15.78:0/1125766020 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653d1a0bfb0 con 0x5653d1ab8180 2024-08-18T23:52:42.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.499+0000 7fc7a5778640 1 --2- 172.21.15.78:0/1125766020 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5653d1ab8180 0x5653d1aba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:42.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.500+0000 7fc7a5778640 1 --2- 172.21.15.78:0/1125766020 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5653d1ab8180 0x5653d1aba570 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c02736f51c789769 server_cookie=6e590fddf8dee99b in_seq=0 out_seq=0 2024-08-18T23:52:42.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.500+0000 7fc784ff9640 10 client.16872 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:52:42.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.501+0000 7fc784ff9640 1 -- 172.21.15.78:0/1125766020 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653d1a0bfb0 con 0x5653d1ab8180 2024-08-18T23:52:42.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.501+0000 7fc784ff9640 10 client.16872 handle_command_reply: tid=0 2024-08-18T23:52:42.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.501+0000 7fc7a9dda740 1 client.16872 shutdown 2024-08-18T23:52:42.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.501+0000 7fc7a9dda740 1 -- 172.21.15.78:0/1125766020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 msgr2=0x5653d1aa5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:42.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.501+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/1125766020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1aa5500 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc794031550 tx=0x7fc794031b30 comp rx=0 tx=0).stop 2024-08-18T23:52:42.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.501+0000 7fc7a9dda740 1 -- 172.21.15.78:0/1125766020 shutdown_connections 2024-08-18T23:52:42.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.501+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/1125766020 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5653d1ab8180 0x5653d1aba570 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.501+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/1125766020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653d1aa85e0 0x5653d1aa5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.501+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/1125766020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1a997f0 0x5653d1aa5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.501+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/1125766020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1aa5500 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.501+0000 7fc784ff9640 0 client.16872 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:52:42.542 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:52:42.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.501+0000 7fc7a9dda740 1 -- 172.21.15.78:0/1125766020 >> 172.21.15.78:0/1125766020 conn(0x5653d1a95580 msgr2=0x5653d1a9b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:42.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.501+0000 7fc7a9dda740 1 -- 172.21.15.78:0/1125766020 shutdown_connections 2024-08-18T23:52:42.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.501+0000 7fc7a9dda740 1 -- 172.21.15.78:0/1125766020 wait complete. 2024-08-18T23:52:42.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.501+0000 7fc7a9dda740 20 client.16872 trim_cache size 0 max 16384 2024-08-18T23:52:42.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.514+0000 7fc7a9dda740 1 Processor -- start 2024-08-18T23:52:42.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.514+0000 7fc7a9dda740 1 -- start start 2024-08-18T23:52:42.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.514+0000 7fc7a9dda740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1aa2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:42.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.514+0000 7fc7a9dda740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653d1a997f0 0x5653d1aa2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:42.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.514+0000 7fc7a9dda740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1aa85e0 0x5653d1ab35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:42.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.514+0000 7fc7a9dda740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a01af090 con 0x5653d1a98ed0 2024-08-18T23:52:42.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.514+0000 7fc7a9dda740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a01ac900 con 0x5653d1a997f0 2024-08-18T23:52:42.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.514+0000 7fc7a9dda740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a01aa750 con 0x5653d1aa85e0 2024-08-18T23:52:42.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.514+0000 7fc7a4f77640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1aa2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:42.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.514+0000 7fc7a4f77640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1aa2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52192/0 (socket says 172.21.15.78:52192) 2024-08-18T23:52:42.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.514+0000 7fc7a4f77640 1 -- 172.21.15.78:0/1984791124 learned_addr learned my addr 172.21.15.78:0/1984791124 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:42.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.514+0000 7fc7a5778640 1 --2- 172.21.15.78:0/1984791124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653d1a997f0 0x5653d1aa2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:42.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.514+0000 7fc7a4f77640 1 -- 172.21.15.78:0/1984791124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1aa85e0 msgr2=0x5653d1ab35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:42.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.514+0000 7fc7a4f77640 1 --2- 172.21.15.78:0/1984791124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1aa85e0 0x5653d1ab35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.514+0000 7fc7a4f77640 1 -- 172.21.15.78:0/1984791124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653d1a997f0 msgr2=0x5653d1aa2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:42.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.514+0000 7fc7a4f77640 1 --2- 172.21.15.78:0/1984791124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653d1a997f0 0x5653d1aa2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.514+0000 7fc7a4f77640 1 -- 172.21.15.78:0/1984791124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653d1a953b0 con 0x5653d1a98ed0 2024-08-18T23:52:42.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.515+0000 7fc7a4f77640 1 --2- 172.21.15.78:0/1984791124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1aa2760 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc794032f50 tx=0x7fc794000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4223df6979ed94f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:42.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.515+0000 7fc786ffd640 1 -- 172.21.15.78:0/1984791124 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc794062070 con 0x5653d1a98ed0 2024-08-18T23:52:42.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.515+0000 7fc786ffd640 1 -- 172.21.15.78:0/1984791124 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc794004410 con 0x5653d1a98ed0 2024-08-18T23:52:42.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.515+0000 7fc786ffd640 1 -- 172.21.15.78:0/1984791124 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc794052040 con 0x5653d1a98ed0 2024-08-18T23:52:42.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.515+0000 7fc7a9dda740 1 -- 172.21.15.78:0/1984791124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 msgr2=0x5653d1aa2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:42.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.515+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/1984791124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1aa2760 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc794032f50 tx=0x7fc794000bc0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.515+0000 7fc7a9dda740 1 -- 172.21.15.78:0/1984791124 shutdown_connections 2024-08-18T23:52:42.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.515+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/1984791124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1aa85e0 0x5653d1ab35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.515+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/1984791124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653d1a997f0 0x5653d1aa2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.515+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/1984791124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1aa2760 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.515+0000 7fc7a9dda740 1 -- 172.21.15.78:0/1984791124 >> 172.21.15.78:0/1984791124 conn(0x5653d1a95580 msgr2=0x5653d1aa0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:42.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.515+0000 7fc7a9dda740 1 -- 172.21.15.78:0/1984791124 shutdown_connections 2024-08-18T23:52:42.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.515+0000 7fc7a9dda740 1 -- 172.21.15.78:0/1984791124 wait complete. 2024-08-18T23:52:42.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.516+0000 7fc7a9dda740 1 -- start start 2024-08-18T23:52:42.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.516+0000 7fc7a9dda740 1 Processor -- start 2024-08-18T23:52:42.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.516+0000 7fc7a9dda740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1aba2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:42.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.516+0000 7fc7a9dda740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1a997f0 0x5653d1aba820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:42.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.516+0000 7fc7a9dda740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653d1aa85e0 0x5653d1b20ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:42.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.516+0000 7fc7a9dda740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a01aadc0 con 0x5653d1a98ed0 2024-08-18T23:52:42.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.516+0000 7fc7a9dda740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a01ac230 con 0x5653d1aa85e0 2024-08-18T23:52:42.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.516+0000 7fc7a9dda740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a01ab490 con 0x5653d1a997f0 2024-08-18T23:52:42.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.516+0000 7fc7a9dda740 10 client.0 _ll_register_callbacks cb 0x7fc79800b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:42.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.516+0000 7fc7a4f77640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1aba2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:42.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.516+0000 7fc7a4f77640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1aba2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52206/0 (socket says 172.21.15.78:52206) 2024-08-18T23:52:42.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.516+0000 7fc7a4f77640 1 -- 172.21.15.78:0/967709599 learned_addr learned my addr 172.21.15.78:0/967709599 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:42.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.517+0000 7fc7a5778640 1 --2- 172.21.15.78:0/967709599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1a997f0 0x5653d1aba820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:42.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.517+0000 7fc7a4f77640 1 -- 172.21.15.78:0/967709599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1a997f0 msgr2=0x5653d1aba820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:42.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.517+0000 7fc7a4f77640 1 --2- 172.21.15.78:0/967709599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1a997f0 0x5653d1aba820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.517+0000 7fc7a4f77640 1 -- 172.21.15.78:0/967709599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653d1aa85e0 msgr2=0x5653d1b20ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:42.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.517+0000 7fc7a4f77640 1 --2- 172.21.15.78:0/967709599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653d1aa85e0 0x5653d1b20ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.517+0000 7fc7a4f77640 1 -- 172.21.15.78:0/967709599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653d1aa0270 con 0x5653d1a98ed0 2024-08-18T23:52:42.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.517+0000 7fc7a4f77640 1 --2- 172.21.15.78:0/967709599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1aba2e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc794039a70 tx=0x7fc794003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce5955155c63d5a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:42.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.517+0000 7fc784ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:42.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.517+0000 7fc784ff9640 1 -- 172.21.15.78:0/967709599 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79404e070 con 0x5653d1a98ed0 2024-08-18T23:52:42.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.517+0000 7fc784ff9640 1 -- 172.21.15.78:0/967709599 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc794031800 con 0x5653d1a98ed0 2024-08-18T23:52:42.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.517+0000 7fc7a9dda740 1 -- 172.21.15.78:0/967709599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc794004410 con 0x5653d1a98ed0 2024-08-18T23:52:42.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.517+0000 7fc784ff9640 1 -- 172.21.15.78:0/967709599 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc794070050 con 0x5653d1a98ed0 2024-08-18T23:52:42.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.517+0000 7fc7a9dda740 10 client.16878 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:42.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.517+0000 7fc7a9dda740 1 -- 172.21.15.78:0/967709599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5653d1a9a2b0 con 0x5653d1a98ed0 2024-08-18T23:52:42.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.517+0000 7fc784ff9640 1 -- 172.21.15.78:0/967709599 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc794004410 con 0x5653d1a98ed0 2024-08-18T23:52:42.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.518+0000 7fc784ff9640 1 -- 172.21.15.78:0/967709599 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fc794065020 con 0x5653d1a98ed0 2024-08-18T23:52:42.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.518+0000 7fc7a9dda740 10 client.16878 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:42.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.518+0000 7fc7a9dda740 10 client.16878 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:52:42.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.518+0000 7fc7a9dda740 20 client.16878 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:42.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.518+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/967709599 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5653d1b223b0 0x5653d1b247d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:42.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.518+0000 7fc7a9dda740 4 client.16878 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:52:42.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.518+0000 7fc7a9dda740 1 -- 172.21.15.78:0/967709599 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653d1a064c0 con 0x5653d1b223b0 2024-08-18T23:52:42.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.518+0000 7fc7a5778640 1 --2- 172.21.15.78:0/967709599 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5653d1b223b0 0x5653d1b247d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:42.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.518+0000 7fc784ff9640 10 client.16878 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:52:42.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.518+0000 7fc7a5778640 1 --2- 172.21.15.78:0/967709599 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5653d1b223b0 0x5653d1b247d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=59813d7901d21262 server_cookie=fd05ef852d4c62c4 in_seq=0 out_seq=0 2024-08-18T23:52:42.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.518+0000 7fc784ff9640 1 -- 172.21.15.78:0/967709599 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653d1a064c0 con 0x5653d1b223b0 2024-08-18T23:52:42.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.518+0000 7fc784ff9640 10 client.16878 handle_command_reply: tid=0 2024-08-18T23:52:42.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.518+0000 7fc7a9dda740 1 client.16878 shutdown 2024-08-18T23:52:42.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.519+0000 7fc7a9dda740 1 -- 172.21.15.78:0/967709599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 msgr2=0x5653d1aba2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:42.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.519+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/967709599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1aba2e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc794039a70 tx=0x7fc794003f90 comp rx=0 tx=0).stop 2024-08-18T23:52:42.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.519+0000 7fc7a9dda740 1 -- 172.21.15.78:0/967709599 shutdown_connections 2024-08-18T23:52:42.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.519+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/967709599 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x5653d1b223b0 0x5653d1b247d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.519+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/967709599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653d1aa85e0 0x5653d1b20ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.519+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/967709599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653d1a997f0 0x5653d1aba820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.519+0000 7fc7a9dda740 1 --2- 172.21.15.78:0/967709599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5653d1a98ed0 0x5653d1aba2e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.519+0000 7fc7a9dda740 1 -- 172.21.15.78:0/967709599 >> 172.21.15.78:0/967709599 conn(0x5653d1a95580 msgr2=0x5653d1aa6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:42.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.519+0000 7fc784ff9640 0 client.16878 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:52:42.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.519+0000 7fc7a9dda740 1 -- 172.21.15.78:0/967709599 shutdown_connections 2024-08-18T23:52:42.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.519+0000 7fc7a9dda740 1 -- 172.21.15.78:0/967709599 wait complete. 2024-08-18T23:52:42.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.519+0000 7fc7a9dda740 20 client.16878 trim_cache size 0 max 16384 2024-08-18T23:52:42.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.519+0000 7fc7a79dc640 1 -- 172.21.15.78:0/2617081325 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc7880a2c00 msgr2=0x7fc7880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:42.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.519+0000 7fc7a79dc640 1 --2- 172.21.15.78:0/2617081325 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc7880a2c00 0x7fc7880a50c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc7900ccd80 tx=0x7fc7900054d0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.519+0000 7fc7a79dc640 1 -- 172.21.15.78:0/2617081325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7a0108030 msgr2=0x7fc7a019cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:42.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.519+0000 7fc7a79dc640 1 --2- 172.21.15.78:0/2617081325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7a0108030 0x7fc7a019cb90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc79000eaa0 tx=0x7fc79000ef60 comp rx=0 tx=0).stop 2024-08-18T23:52:42.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.520+0000 7fc7a79dc640 1 -- 172.21.15.78:0/2617081325 shutdown_connections 2024-08-18T23:52:42.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.520+0000 7fc7a79dc640 1 --2- 172.21.15.78:0/2617081325 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc7880a2c00 0x7fc7880a50c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.520+0000 7fc7a79dc640 1 --2- 172.21.15.78:0/2617081325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7a0108030 0x7fc7a019cb90 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.520+0000 7fc7a79dc640 1 --2- 172.21.15.78:0/2617081325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7a01034b0 0x7fc7a0198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.520+0000 7fc7a79dc640 1 --2- 172.21.15.78:0/2617081325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7a0102b00 0x7fc7a01982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:42.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.520+0000 7fc7a79dc640 1 -- 172.21.15.78:0/2617081325 >> 172.21.15.78:0/2617081325 conn(0x7fc7a00fc500 msgr2=0x7fc7a01060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:42.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.520+0000 7fc7a79dc640 1 -- 172.21.15.78:0/2617081325 shutdown_connections 2024-08-18T23:52:42.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:42.520+0000 7fc7a79dc640 1 -- 172.21.15.78:0/2617081325 wait complete. 2024-08-18T23:52:43.295 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-18T23:52:43.548 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:43.699 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:43 smithi078 ceph-mon[18440]: pgmap v602: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 KiB/s wr, 67 op/s 2024-08-18T23:52:43.730 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:43 smithi161 ceph-mon[22027]: pgmap v602: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 KiB/s wr, 67 op/s 2024-08-18T23:52:43.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:43 smithi123 ceph-mon[21808]: pgmap v602: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 KiB/s wr, 67 op/s 2024-08-18T23:52:44.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.064+0000 7f7d24efd640 1 -- 172.21.15.78:0/2731517768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d20108030 msgr2=0x7f7d2010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:44.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.064+0000 7f7d24efd640 1 --2- 172.21.15.78:0/2731517768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d20108030 0x7f7d2010e9a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009cd0 tx=0x7f7d1802a8d0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.064+0000 7f7d24efd640 1 -- 172.21.15.78:0/2731517768 shutdown_connections 2024-08-18T23:52:44.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.064+0000 7f7d24efd640 1 --2- 172.21.15.78:0/2731517768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d20108030 0x7f7d2010e9a0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.064+0000 7f7d24efd640 1 --2- 172.21.15.78:0/2731517768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d201034b0 0x7f7d201078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.064+0000 7f7d24efd640 1 --2- 172.21.15.78:0/2731517768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d20102b00 0x7f7d20102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.064+0000 7f7d24efd640 1 -- 172.21.15.78:0/2731517768 >> 172.21.15.78:0/2731517768 conn(0x7f7d200fc500 msgr2=0x7f7d200fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:44.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.064+0000 7f7d24efd640 1 -- 172.21.15.78:0/2731517768 shutdown_connections 2024-08-18T23:52:44.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.064+0000 7f7d24efd640 1 -- 172.21.15.78:0/2731517768 wait complete. 2024-08-18T23:52:44.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.064+0000 7f7d24efd640 1 Processor -- start 2024-08-18T23:52:44.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d24efd640 1 -- start start 2024-08-18T23:52:44.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d24efd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d20102b00 0x7f7d20198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d24efd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d201034b0 0x7f7d201987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d24efd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d20108030 0x7f7d2019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d24efd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7d20111e00 con 0x7f7d20108030 2024-08-18T23:52:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d24efd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d20111c80 con 0x7f7d20102b00 2024-08-18T23:52:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d24efd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7d20111f80 con 0x7f7d201034b0 2024-08-18T23:52:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d1ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d20108030 0x7f7d2019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d1ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d20108030 0x7f7d2019cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52224/0 (socket says 172.21.15.78:52224) 2024-08-18T23:52:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d1ed9d640 1 -- 172.21.15.78:0/2322316452 learned_addr learned my addr 172.21.15.78:0/2322316452 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d1e59c640 1 --2- 172.21.15.78:0/2322316452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d20102b00 0x7f7d20198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d1dd9b640 1 --2- 172.21.15.78:0/2322316452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d201034b0 0x7f7d201987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d1ed9d640 1 -- 172.21.15.78:0/2322316452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d201034b0 msgr2=0x7f7d201987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:44.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d1ed9d640 1 --2- 172.21.15.78:0/2322316452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d201034b0 0x7f7d201987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d1ed9d640 1 -- 172.21.15.78:0/2322316452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d20102b00 msgr2=0x7f7d20198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d1ed9d640 1 --2- 172.21.15.78:0/2322316452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d20102b00 0x7f7d20198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d1ed9d640 1 -- 172.21.15.78:0/2322316452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2019d210 con 0x7f7d20108030 2024-08-18T23:52:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d1e59c640 1 --2- 172.21.15.78:0/2322316452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d20102b00 0x7f7d20198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d1ed9d640 1 --2- 172.21.15.78:0/2322316452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d20108030 0x7f7d2019cb30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009ca0 tx=0x7f7d1802f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.065+0000 7f7d0f7fe640 1 -- 172.21.15.78:0/2322316452 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18031860 con 0x7f7d20108030 2024-08-18T23:52:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.066+0000 7f7d0f7fe640 1 -- 172.21.15.78:0/2322316452 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d18003e50 con 0x7f7d20108030 2024-08-18T23:52:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.066+0000 7f7d0f7fe640 1 -- 172.21.15.78:0/2322316452 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d180307f0 con 0x7f7d20108030 2024-08-18T23:52:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.066+0000 7f7d24efd640 1 -- 172.21.15.78:0/2322316452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2019d4a0 con 0x7f7d20108030 2024-08-18T23:52:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.066+0000 7f7d24efd640 1 -- 172.21.15.78:0/2322316452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d201a4de0 con 0x7f7d20108030 2024-08-18T23:52:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.069+0000 7f7d0f7fe640 1 -- 172.21.15.78:0/2322316452 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7d180041d0 con 0x7f7d20108030 2024-08-18T23:52:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.069+0000 7f7d272fb740 1 Processor -- start 2024-08-18T23:52:44.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.069+0000 7f7d272fb740 1 -- start start 2024-08-18T23:52:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.069+0000 7f7d272fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d457bafff0 0x55d457bb03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.069+0000 7f7d272fb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bb0910 0x55d457bbac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.069+0000 7f7d272fb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bbb1d0 0x55d457bbd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.069+0000 7f7d272fb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d457b15a40 con 0x55d457bbb1d0 2024-08-18T23:52:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.069+0000 7f7d272fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d457aad9d0 con 0x55d457bafff0 2024-08-18T23:52:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.069+0000 7f7d272fb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d457b11a00 con 0x55d457bb0910 2024-08-18T23:52:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.069+0000 7f7d1dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bbb1d0 0x55d457bbd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.069+0000 7f7d1dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bbb1d0 0x55d457bbd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52240/0 (socket says 172.21.15.78:52240) 2024-08-18T23:52:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.069+0000 7f7d1dd9b640 1 -- 172.21.15.78:0/349162757 learned_addr learned my addr 172.21.15.78:0/349162757 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.069+0000 7f7d1dd9b640 1 --2- 172.21.15.78:0/349162757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d457bafff0 0x55d457bb03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.069+0000 7f7d1e59c640 1 --2- 172.21.15.78:0/349162757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bb0910 0x55d457bbac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.069+0000 7f7d0f7fe640 1 --2- 172.21.15.78:0/2322316452 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7d080a2b80 0x7f7d080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:44.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.070+0000 7f7d1dd9b640 1 -- 172.21.15.78:0/349162757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bb0910 msgr2=0x55d457bbac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.070+0000 7f7d1dd9b640 1 --2- 172.21.15.78:0/349162757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bb0910 0x55d457bbac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.070+0000 7f7d1dd9b640 1 -- 172.21.15.78:0/349162757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d457bafff0 msgr2=0x55d457bb03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.070+0000 7f7d1dd9b640 1 --2- 172.21.15.78:0/349162757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d457bafff0 0x55d457bb03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.070+0000 7f7d1dd9b640 1 -- 172.21.15.78:0/349162757 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d457ab4650 con 0x55d457bbb1d0 2024-08-18T23:52:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.070+0000 7f7d1ed9d640 1 --2- 172.21.15.78:0/2322316452 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7d080a2b80 0x7f7d080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.070+0000 7f7d0f7fe640 1 -- 172.21.15.78:0/2322316452 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f7d180f04b0 con 0x7f7d20108030 2024-08-18T23:52:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.070+0000 7f7d1dd9b640 1 --2- 172.21.15.78:0/349162757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bbb1d0 0x55d457bbd5c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7d14014990 tx=0x7f7d1400c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b892117f993017de server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.070+0000 7f7d0cff9640 1 -- 172.21.15.78:0/349162757 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d140142d0 con 0x55d457bbb1d0 2024-08-18T23:52:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.070+0000 7f7d0cff9640 1 -- 172.21.15.78:0/349162757 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d1400fc40 con 0x55d457bbb1d0 2024-08-18T23:52:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.070+0000 7f7d0cff9640 1 -- 172.21.15.78:0/349162757 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14012b00 con 0x55d457bbb1d0 2024-08-18T23:52:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.071+0000 7f7d272fb740 1 -- 172.21.15.78:0/349162757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bbb1d0 msgr2=0x55d457bbd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.071+0000 7f7d1ed9d640 1 --2- 172.21.15.78:0/2322316452 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7d080a2b80 0x7f7d080a5040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7d180b8de0 tx=0x7f7d180b1e60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:44.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.071+0000 7f7d272fb740 1 --2- 172.21.15.78:0/349162757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bbb1d0 0x55d457bbd5c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7d14014990 tx=0x7f7d1400c3f0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.071+0000 7f7d272fb740 1 -- 172.21.15.78:0/349162757 shutdown_connections 2024-08-18T23:52:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.071+0000 7f7d272fb740 1 --2- 172.21.15.78:0/349162757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bbb1d0 0x55d457bbd5c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.071+0000 7f7d272fb740 1 --2- 172.21.15.78:0/349162757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bb0910 0x55d457bbac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.071+0000 7f7d272fb740 1 --2- 172.21.15.78:0/349162757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d457bafff0 0x55d457bb03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.071+0000 7f7d272fb740 1 -- 172.21.15.78:0/349162757 >> 172.21.15.78:0/349162757 conn(0x55d457bac580 msgr2=0x55d457bac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.071+0000 7f7d272fb740 1 -- 172.21.15.78:0/349162757 shutdown_connections 2024-08-18T23:52:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.071+0000 7f7d272fb740 1 -- 172.21.15.78:0/349162757 wait complete. 2024-08-18T23:52:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.071+0000 7f7d272fb740 1 -- start start 2024-08-18T23:52:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.071+0000 7f7d272fb740 1 Processor -- start 2024-08-18T23:52:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.071+0000 7f7d272fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d457bafff0 0x55d457bb8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.071+0000 7f7d272fb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bb0910 0x55d457bb85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.071+0000 7f7d272fb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bbb1d0 0x55d457bb8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:44.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d272fb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d457bbdda0 con 0x55d457bb0910 2024-08-18T23:52:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d272fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d457ad0b20 con 0x55d457bafff0 2024-08-18T23:52:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d272fb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d457bbdf20 con 0x55d457bbb1d0 2024-08-18T23:52:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d1e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bb0910 0x55d457bb85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d1e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bb0910 0x55d457bb85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52244/0 (socket says 172.21.15.78:52244) 2024-08-18T23:52:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d1e59c640 1 -- 172.21.15.78:0/1505478804 learned_addr learned my addr 172.21.15.78:0/1505478804 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d272fb740 10 client.0 _ll_register_callbacks cb 0x55d457acebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d1dd9b640 1 --2- 172.21.15.78:0/1505478804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bbb1d0 0x55d457bb8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d1dd9b640 1 --2- 172.21.15.78:0/1505478804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d457bafff0 0x55d457bb8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d1e59c640 1 -- 172.21.15.78:0/1505478804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bbb1d0 msgr2=0x55d457bb8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d1e59c640 1 --2- 172.21.15.78:0/1505478804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bbb1d0 0x55d457bb8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d1e59c640 1 -- 172.21.15.78:0/1505478804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d457bafff0 msgr2=0x55d457bb8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d1e59c640 1 --2- 172.21.15.78:0/1505478804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d457bafff0 0x55d457bb8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d1e59c640 1 -- 172.21.15.78:0/1505478804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4578873c0 con 0x55d457bb0910 2024-08-18T23:52:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d1dd9b640 1 --2- 172.21.15.78:0/1505478804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d457bafff0 0x55d457bb8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:52:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d1e59c640 1 --2- 172.21.15.78:0/1505478804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bb0910 0x55d457bb85d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7d10011a60 tx=0x7f7d10011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11a886c057e1a32 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d05ffb640 1 -- 172.21.15.78:0/1505478804 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1000fdb0 con 0x55d457bb0910 2024-08-18T23:52:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d05ffb640 1 -- 172.21.15.78:0/1505478804 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f7d1000e3b0 con 0x55d457bb0910 2024-08-18T23:52:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d272fb740 1 -- 172.21.15.78:0/1505478804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d1400fc40 con 0x55d457bb0910 2024-08-18T23:52:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d05ffb640 1 -- 172.21.15.78:0/1505478804 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7d1000e5b0 con 0x55d457bb0910 2024-08-18T23:52:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d272fb740 10 client.16890 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.072+0000 7f7d272fb740 1 -- 172.21.15.78:0/1505478804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d457bc99c0 con 0x55d457bb0910 2024-08-18T23:52:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.073+0000 7f7d05ffb640 1 -- 172.21.15.78:0/1505478804 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d1400fc40 con 0x55d457bb0910 2024-08-18T23:52:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.073+0000 7f7d05ffb640 1 -- 172.21.15.78:0/1505478804 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f7d1001b940 con 0x55d457bb0910 2024-08-18T23:52:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.073+0000 7f7d272fb740 10 client.16890 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.073+0000 7f7d272fb740 10 client.16890 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:52:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.073+0000 7f7d272fb740 20 client.16890 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.073+0000 7f7d272fb740 1 --2- 172.21.15.78:0/1505478804 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55d457bcae80 0x55d457bcd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.073+0000 7f7d272fb740 4 client.16890 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:52:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.073+0000 7f7d272fb740 1 -- 172.21.15.78:0/1505478804 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d457b22fb0 con 0x55d457bcae80 2024-08-18T23:52:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.073+0000 7f7d1dd9b640 1 --2- 172.21.15.78:0/1505478804 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55d457bcae80 0x55d457bcd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.074+0000 7f7d05ffb640 10 client.16890 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:52:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.074+0000 7f7d1dd9b640 1 --2- 172.21.15.78:0/1505478804 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55d457bcae80 0x55d457bcd310 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=780c19a9e4a32c86 server_cookie=6a74cb1689b0ed72 in_seq=0 out_seq=0 2024-08-18T23:52:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.074+0000 7f7d05ffb640 1 -- 172.21.15.78:0/1505478804 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d457b22fb0 con 0x55d457bcae80 2024-08-18T23:52:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.074+0000 7f7d05ffb640 10 client.16890 handle_command_reply: tid=0 2024-08-18T23:52:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.074+0000 7f7d272fb740 1 client.16890 shutdown 2024-08-18T23:52:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.074+0000 7f7d272fb740 1 -- 172.21.15.78:0/1505478804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bb0910 msgr2=0x55d457bb85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.074+0000 7f7d272fb740 1 --2- 172.21.15.78:0/1505478804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bb0910 0x55d457bb85d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7d10011a60 tx=0x7f7d10011f20 comp rx=0 tx=0).stop 2024-08-18T23:52:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.074+0000 7f7d272fb740 1 -- 172.21.15.78:0/1505478804 shutdown_connections 2024-08-18T23:52:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.074+0000 7f7d272fb740 1 --2- 172.21.15.78:0/1505478804 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55d457bcae80 0x55d457bcd310 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.074+0000 7f7d272fb740 1 --2- 172.21.15.78:0/1505478804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bbb1d0 0x55d457bb8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.074+0000 7f7d272fb740 1 --2- 172.21.15.78:0/1505478804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bb0910 0x55d457bb85d0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.074+0000 7f7d272fb740 1 --2- 172.21.15.78:0/1505478804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d457bafff0 0x55d457bb8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.074+0000 7f7d272fb740 1 -- 172.21.15.78:0/1505478804 >> 172.21.15.78:0/1505478804 conn(0x55d457bac580 msgr2=0x55d457bb2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.074+0000 7f7d05ffb640 0 client.16890 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:52:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.074+0000 7f7d272fb740 1 -- 172.21.15.78:0/1505478804 shutdown_connections 2024-08-18T23:52:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.074+0000 7f7d272fb740 1 -- 172.21.15.78:0/1505478804 wait complete. 2024-08-18T23:52:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.075+0000 7f7d272fb740 20 client.16890 trim_cache size 0 max 16384 2024-08-18T23:52:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.087+0000 7f7d272fb740 1 Processor -- start 2024-08-18T23:52:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d272fb740 1 -- start start 2024-08-18T23:52:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d272fb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bafff0 0x55d457bac960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d272fb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bb0910 0x55d457bd2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d272fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d457bbb1d0 0x55d457bd3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d272fb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7d201b1300 con 0x55d457bafff0 2024-08-18T23:52:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d272fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d201aeb70 con 0x55d457bbb1d0 2024-08-18T23:52:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d272fb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7d201ac9c0 con 0x55d457bb0910 2024-08-18T23:52:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d1dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bafff0 0x55d457bac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d1dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bafff0 0x55d457bac960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52250/0 (socket says 172.21.15.78:52250) 2024-08-18T23:52:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d1dd9b640 1 -- 172.21.15.78:0/2573029137 learned_addr learned my addr 172.21.15.78:0/2573029137 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d1e59c640 1 --2- 172.21.15.78:0/2573029137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bb0910 0x55d457bd2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d1dd9b640 1 -- 172.21.15.78:0/2573029137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bb0910 msgr2=0x55d457bd2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d1dd9b640 1 --2- 172.21.15.78:0/2573029137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bb0910 0x55d457bd2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d1dd9b640 1 -- 172.21.15.78:0/2573029137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d457bbb1d0 msgr2=0x55d457bd3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d1dd9b640 1 --2- 172.21.15.78:0/2573029137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d457bbb1d0 0x55d457bd3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d1dd9b640 1 -- 172.21.15.78:0/2573029137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d457bb0740 con 0x55d457bafff0 2024-08-18T23:52:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d1dd9b640 1 --2- 172.21.15.78:0/2573029137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bafff0 0x55d457bac960 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7d14000c00 tx=0x7f7d14005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a5071ffc5b23dc server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d07fff640 1 -- 172.21.15.78:0/2573029137 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14012450 con 0x55d457bafff0 2024-08-18T23:52:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d07fff640 1 -- 172.21.15.78:0/2573029137 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d140125f0 con 0x55d457bafff0 2024-08-18T23:52:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.088+0000 7f7d07fff640 1 -- 172.21.15.78:0/2573029137 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14012ab0 con 0x55d457bafff0 2024-08-18T23:52:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.089+0000 7f7d272fb740 1 -- 172.21.15.78:0/2573029137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bafff0 msgr2=0x55d457bac960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.089+0000 7f7d272fb740 1 --2- 172.21.15.78:0/2573029137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bafff0 0x55d457bac960 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7d14000c00 tx=0x7f7d14005850 comp rx=0 tx=0).stop 2024-08-18T23:52:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.089+0000 7f7d272fb740 1 -- 172.21.15.78:0/2573029137 shutdown_connections 2024-08-18T23:52:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.089+0000 7f7d272fb740 1 --2- 172.21.15.78:0/2573029137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d457bbb1d0 0x55d457bd3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.089+0000 7f7d272fb740 1 --2- 172.21.15.78:0/2573029137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bb0910 0x55d457bd2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.089+0000 7f7d272fb740 1 --2- 172.21.15.78:0/2573029137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bafff0 0x55d457bac960 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.089+0000 7f7d272fb740 1 -- 172.21.15.78:0/2573029137 >> 172.21.15.78:0/2573029137 conn(0x55d457bac580 msgr2=0x55d457bb7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.089+0000 7f7d272fb740 1 -- 172.21.15.78:0/2573029137 shutdown_connections 2024-08-18T23:52:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.089+0000 7f7d272fb740 1 -- 172.21.15.78:0/2573029137 wait complete. 2024-08-18T23:52:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.089+0000 7f7d272fb740 1 -- start start 2024-08-18T23:52:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.090+0000 7f7d272fb740 1 Processor -- start 2024-08-18T23:52:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.090+0000 7f7d272fb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bafff0 0x55d457bc3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.090+0000 7f7d272fb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bb0910 0x55d457bc36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.090+0000 7f7d272fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d457bbb1d0 0x55d457bb4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.090+0000 7f7d272fb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7d201ad030 con 0x55d457bb0910 2024-08-18T23:52:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.090+0000 7f7d272fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d201ae4a0 con 0x55d457bbb1d0 2024-08-18T23:52:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.090+0000 7f7d272fb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7d201ad700 con 0x55d457bafff0 2024-08-18T23:52:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.090+0000 7f7d272fb740 10 client.0 _ll_register_callbacks cb 0x7f7d1000c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:44.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.090+0000 7f7d1e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bb0910 0x55d457bc36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.090+0000 7f7d1e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bb0910 0x55d457bc36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52262/0 (socket says 172.21.15.78:52262) 2024-08-18T23:52:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.090+0000 7f7d1e59c640 1 -- 172.21.15.78:0/1650201053 learned_addr learned my addr 172.21.15.78:0/1650201053 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.090+0000 7f7d1dd9b640 1 --2- 172.21.15.78:0/1650201053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bafff0 0x55d457bc3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.090+0000 7f7d1e59c640 1 -- 172.21.15.78:0/1650201053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bafff0 msgr2=0x55d457bc3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.090+0000 7f7d1e59c640 1 --2- 172.21.15.78:0/1650201053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bafff0 0x55d457bc3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.090+0000 7f7d1e59c640 1 -- 172.21.15.78:0/1650201053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d457bbb1d0 msgr2=0x55d457bb4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.090+0000 7f7d1e59c640 1 --2- 172.21.15.78:0/1650201053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d457bbb1d0 0x55d457bb4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.090+0000 7f7d1e59c640 1 -- 172.21.15.78:0/1650201053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d457bb4cc0 con 0x55d457bb0910 2024-08-18T23:52:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.090+0000 7f7d1e59c640 1 --2- 172.21.15.78:0/1650201053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bb0910 0x55d457bc36b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7d100058c0 tx=0x7f7d1000fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382856945ff7c154 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.090+0000 7f7d05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.091+0000 7f7d05ffb640 1 -- 172.21.15.78:0/1650201053 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1002d070 con 0x55d457bb0910 2024-08-18T23:52:44.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.091+0000 7f7d05ffb640 1 -- 172.21.15.78:0/1650201053 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f7d10005250 con 0x55d457bb0910 2024-08-18T23:52:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.091+0000 7f7d272fb740 1 -- 172.21.15.78:0/1650201053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d140125f0 con 0x55d457bb0910 2024-08-18T23:52:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.091+0000 7f7d05ffb640 1 -- 172.21.15.78:0/1650201053 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7d1003a050 con 0x55d457bb0910 2024-08-18T23:52:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.091+0000 7f7d272fb740 10 client.16896 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.091+0000 7f7d272fb740 1 -- 172.21.15.78:0/1650201053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d457c35530 con 0x55d457bb0910 2024-08-18T23:52:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.091+0000 7f7d05ffb640 1 -- 172.21.15.78:0/1650201053 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d140125f0 con 0x55d457bb0910 2024-08-18T23:52:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.091+0000 7f7d05ffb640 1 -- 172.21.15.78:0/1650201053 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f7d1000e750 con 0x55d457bb0910 2024-08-18T23:52:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.091+0000 7f7d272fb740 10 client.16896 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.091+0000 7f7d272fb740 10 client.16896 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:52:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.091+0000 7f7d272fb740 20 client.16896 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.091+0000 7f7d272fb740 1 --2- 172.21.15.78:0/1650201053 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55d457c35cb0 0x55d457c380d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.091+0000 7f7d272fb740 4 client.16896 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:52:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.091+0000 7f7d272fb740 1 -- 172.21.15.78:0/1650201053 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d457b1d4c0 con 0x55d457c35cb0 2024-08-18T23:52:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.092+0000 7f7d1dd9b640 1 --2- 172.21.15.78:0/1650201053 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55d457c35cb0 0x55d457c380d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.092+0000 7f7d05ffb640 10 client.16896 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:52:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.092+0000 7f7d1dd9b640 1 --2- 172.21.15.78:0/1650201053 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55d457c35cb0 0x55d457c380d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=98f1b5cd65e91e54 server_cookie=54c44f3d754cbdd3 in_seq=0 out_seq=0 2024-08-18T23:52:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.092+0000 7f7d05ffb640 1 -- 172.21.15.78:0/1650201053 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d457b1d4c0 con 0x55d457c35cb0 2024-08-18T23:52:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.092+0000 7f7d05ffb640 10 client.16896 handle_command_reply: tid=0 2024-08-18T23:52:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.092+0000 7f7d272fb740 1 client.16896 shutdown 2024-08-18T23:52:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.092+0000 7f7d272fb740 1 -- 172.21.15.78:0/1650201053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bb0910 msgr2=0x55d457bc36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.092+0000 7f7d272fb740 1 --2- 172.21.15.78:0/1650201053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bb0910 0x55d457bc36b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7d100058c0 tx=0x7f7d1000fe60 comp rx=0 tx=0).stop 2024-08-18T23:52:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.092+0000 7f7d272fb740 1 -- 172.21.15.78:0/1650201053 shutdown_connections 2024-08-18T23:52:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.092+0000 7f7d272fb740 1 --2- 172.21.15.78:0/1650201053 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55d457c35cb0 0x55d457c380d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.092+0000 7f7d272fb740 1 --2- 172.21.15.78:0/1650201053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d457bbb1d0 0x55d457bb4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.092+0000 7f7d272fb740 1 --2- 172.21.15.78:0/1650201053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d457bb0910 0x55d457bc36b0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.092+0000 7f7d272fb740 1 --2- 172.21.15.78:0/1650201053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d457bafff0 0x55d457bc3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.092+0000 7f7d272fb740 1 -- 172.21.15.78:0/1650201053 >> 172.21.15.78:0/1650201053 conn(0x55d457bac580 msgr2=0x55d457bb1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.092+0000 7f7d05ffb640 0 client.16896 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:52:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.092+0000 7f7d272fb740 1 -- 172.21.15.78:0/1650201053 shutdown_connections 2024-08-18T23:52:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.092+0000 7f7d272fb740 1 -- 172.21.15.78:0/1650201053 wait complete. 2024-08-18T23:52:44.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.092+0000 7f7d272fb740 20 client.16896 trim_cache size 0 max 16384 2024-08-18T23:52:44.107 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:52:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.093+0000 7f7d24efd640 1 -- 172.21.15.78:0/2322316452 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7d080a2b80 msgr2=0x7f7d080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.093+0000 7f7d24efd640 1 --2- 172.21.15.78:0/2322316452 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7d080a2b80 0x7f7d080a5040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7d180b8de0 tx=0x7f7d180b1e60 comp rx=0 tx=0).stop 2024-08-18T23:52:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.093+0000 7f7d24efd640 1 -- 172.21.15.78:0/2322316452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d20108030 msgr2=0x7f7d2019cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.093+0000 7f7d24efd640 1 --2- 172.21.15.78:0/2322316452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d20108030 0x7f7d2019cb30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009ca0 tx=0x7f7d1802f420 comp rx=0 tx=0).stop 2024-08-18T23:52:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.093+0000 7f7d24efd640 1 -- 172.21.15.78:0/2322316452 shutdown_connections 2024-08-18T23:52:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.093+0000 7f7d24efd640 1 --2- 172.21.15.78:0/2322316452 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7d080a2b80 0x7f7d080a5040 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.093+0000 7f7d24efd640 1 --2- 172.21.15.78:0/2322316452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7d20108030 0x7f7d2019cb30 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.093+0000 7f7d24efd640 1 --2- 172.21.15.78:0/2322316452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d201034b0 0x7f7d201987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.093+0000 7f7d24efd640 1 --2- 172.21.15.78:0/2322316452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d20102b00 0x7f7d20198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:44.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.093+0000 7f7d24efd640 1 -- 172.21.15.78:0/2322316452 >> 172.21.15.78:0/2322316452 conn(0x7f7d200fc500 msgr2=0x7f7d201060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:44.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.093+0000 7f7d24efd640 1 -- 172.21.15.78:0/2322316452 shutdown_connections 2024-08-18T23:52:44.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:44.093+0000 7f7d24efd640 1 -- 172.21.15.78:0/2322316452 wait complete. 2024-08-18T23:52:44.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:44 smithi078 ceph-mon[18440]: pgmap v603: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 KiB/s wr, 27 op/s 2024-08-18T23:52:44.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:44 smithi123 ceph-mon[21808]: pgmap v603: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 KiB/s wr, 27 op/s 2024-08-18T23:52:44.820 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-18T23:52:44.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:44 smithi161 ceph-mon[22027]: pgmap v603: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 KiB/s wr, 27 op/s 2024-08-18T23:52:45.077 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:45.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.572+0000 7f638b0ee640 1 -- 172.21.15.78:0/2879973344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63841018b0 msgr2=0x7f6384101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:45.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.572+0000 7f638b0ee640 1 --2- 172.21.15.78:0/2879973344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63841018b0 0x7f6384101d10 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f636c009930 tx=0x7f636c02a420 comp rx=0 tx=0).stop 2024-08-18T23:52:45.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.573+0000 7f638b0ee640 1 -- 172.21.15.78:0/2879973344 shutdown_connections 2024-08-18T23:52:45.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.573+0000 7f638b0ee640 1 --2- 172.21.15.78:0/2879973344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6384102250 0x7f638410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.573+0000 7f638b0ee640 1 --2- 172.21.15.78:0/2879973344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63841018b0 0x7f6384101d10 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.573+0000 7f638b0ee640 1 --2- 172.21.15.78:0/2879973344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63841078b0 0x7f6384107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.573+0000 7f638b0ee640 1 -- 172.21.15.78:0/2879973344 >> 172.21.15.78:0/2879973344 conn(0x7f63840fd5f0 msgr2=0x7f63840ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:45.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.573+0000 7f638b0ee640 1 -- 172.21.15.78:0/2879973344 shutdown_connections 2024-08-18T23:52:45.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.573+0000 7f638b0ee640 1 -- 172.21.15.78:0/2879973344 wait complete. 2024-08-18T23:52:45.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.573+0000 7f638b0ee640 1 Processor -- start 2024-08-18T23:52:45.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.573+0000 7f638b0ee640 1 -- start start 2024-08-18T23:52:45.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.573+0000 7f638b0ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63841018b0 0x7f6384198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:45.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.573+0000 7f638b0ee640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6384102250 0x7f6384198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:45.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.573+0000 7f638b0ee640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63841078b0 0x7f638419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:45.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.573+0000 7f638b0ee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6384111d00 con 0x7f63841018b0 2024-08-18T23:52:45.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.573+0000 7f638b0ee640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6384111b80 con 0x7f6384102250 2024-08-18T23:52:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.573+0000 7f638b0ee640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6384111e80 con 0x7f63841078b0 2024-08-18T23:52:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.574+0000 7f6388e8a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63841018b0 0x7f6384198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.574+0000 7f6388e8a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63841018b0 0x7f6384198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52272/0 (socket says 172.21.15.78:52272) 2024-08-18T23:52:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.574+0000 7f6388e8a640 1 -- 172.21.15.78:0/465982507 learned_addr learned my addr 172.21.15.78:0/465982507 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.574+0000 7f638968b640 1 --2- 172.21.15.78:0/465982507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63841078b0 0x7f638419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.574+0000 7f6383fff640 1 --2- 172.21.15.78:0/465982507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6384102250 0x7f6384198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.574+0000 7f6388e8a640 1 -- 172.21.15.78:0/465982507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63841078b0 msgr2=0x7f638419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.574+0000 7f6388e8a640 1 --2- 172.21.15.78:0/465982507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63841078b0 0x7f638419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.574+0000 7f6388e8a640 1 -- 172.21.15.78:0/465982507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6384102250 msgr2=0x7f6384198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.574+0000 7f6388e8a640 1 --2- 172.21.15.78:0/465982507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6384102250 0x7f6384198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.574+0000 7f6388e8a640 1 -- 172.21.15.78:0/465982507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638419d1d0 con 0x7f63841018b0 2024-08-18T23:52:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.574+0000 7f638968b640 1 --2- 172.21.15.78:0/465982507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63841078b0 0x7f638419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:52:45.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.574+0000 7f6388e8a640 1 --2- 172.21.15.78:0/465982507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63841018b0 0x7f6384198220 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f637400d770 tx=0x7f637400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:45.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.574+0000 7f6381ffb640 1 -- 172.21.15.78:0/465982507 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6374017070 con 0x7f63841018b0 2024-08-18T23:52:45.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.574+0000 7f6381ffb640 1 -- 172.21.15.78:0/465982507 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6374005500 con 0x7f63841018b0 2024-08-18T23:52:45.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.574+0000 7f638b0ee640 1 -- 172.21.15.78:0/465982507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f638419d4c0 con 0x7f63841018b0 2024-08-18T23:52:45.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.574+0000 7f6381ffb640 1 -- 172.21.15.78:0/465982507 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6374004e00 con 0x7f63841018b0 2024-08-18T23:52:45.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.574+0000 7f638b0ee640 1 -- 172.21.15.78:0/465982507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63841a4da0 con 0x7f63841018b0 2024-08-18T23:52:45.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.577+0000 7f6381ffb640 1 -- 172.21.15.78:0/465982507 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6374004fa0 con 0x7f63841018b0 2024-08-18T23:52:45.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.577+0000 7f638d4ec740 1 Processor -- start 2024-08-18T23:52:45.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.577+0000 7f638d4ec740 1 -- start start 2024-08-18T23:52:45.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.577+0000 7f6381ffb640 1 --2- 172.21.15.78:0/465982507 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f63700a2b80 0x7f63700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:45.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.577+0000 7f638d4ec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9aff0 0x563b5bb9b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:45.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.577+0000 7f638d4ec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bb9b910 0x563b5bba5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:45.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.577+0000 7f638d4ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bba61d0 0x563b5bba85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:45.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.577+0000 7f638d4ec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563b5bb00a40 con 0x563b5bb9aff0 2024-08-18T23:52:45.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.577+0000 7f638d4ec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563b5ba989d0 con 0x563b5bb9b910 2024-08-18T23:52:45.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.577+0000 7f638d4ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563b5bafca00 con 0x563b5bba61d0 2024-08-18T23:52:45.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.577+0000 7f638968b640 1 --2- 172.21.15.78:0/465982507 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f63700a2b80 0x7f63700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:45.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.577+0000 7f6381ffb640 1 -- 172.21.15.78:0/465982507 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f63740ceea0 con 0x7f63841018b0 2024-08-18T23:52:45.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.577+0000 7f6388e8a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9aff0 0x563b5bb9b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:45.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.577+0000 7f6388e8a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9aff0 0x563b5bb9b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52278/0 (socket says 172.21.15.78:52278) 2024-08-18T23:52:45.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.577+0000 7f6388e8a640 1 -- 172.21.15.78:0/3467782549 learned_addr learned my addr 172.21.15.78:0/3467782549 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:45.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.578+0000 7f6383fff640 1 --2- 172.21.15.78:0/3467782549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bb9b910 0x563b5bba5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:45.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.578+0000 7f638968b640 1 --2- 172.21.15.78:0/3467782549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bba61d0 0x563b5bba85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:45.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.578+0000 7f6388e8a640 1 -- 172.21.15.78:0/3467782549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bba61d0 msgr2=0x563b5bba85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:45.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.578+0000 7f6388e8a640 1 --2- 172.21.15.78:0/3467782549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bba61d0 0x563b5bba85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.578+0000 7f6388e8a640 1 -- 172.21.15.78:0/3467782549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bb9b910 msgr2=0x563b5bba5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:45.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.578+0000 7f6388e8a640 1 --2- 172.21.15.78:0/3467782549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bb9b910 0x563b5bba5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.578+0000 7f6388e8a640 1 -- 172.21.15.78:0/3467782549 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b5ba9f650 con 0x563b5bb9aff0 2024-08-18T23:52:45.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.578+0000 7f638968b640 1 --2- 172.21.15.78:0/3467782549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bba61d0 0x563b5bba85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:52:45.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.578+0000 7f6388e8a640 1 --2- 172.21.15.78:0/3467782549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9aff0 0x563b5bb9b3d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f637409b300 tx=0x7f6374009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd611c100faa9dc server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:45.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.578+0000 7f638968b640 1 --2- 172.21.15.78:0/465982507 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f63700a2b80 0x7f63700a5040 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x563b5bb98900 tx=0x7f637c008b20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:45.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.578+0000 7f637affd640 1 -- 172.21.15.78:0/3467782549 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6374020070 con 0x563b5bb9aff0 2024-08-18T23:52:45.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.578+0000 7f637affd640 1 -- 172.21.15.78:0/3467782549 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63740046c0 con 0x563b5bb9aff0 2024-08-18T23:52:45.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.578+0000 7f637affd640 1 -- 172.21.15.78:0/3467782549 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637409f720 con 0x563b5bb9aff0 2024-08-18T23:52:45.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.579+0000 7f638d4ec740 1 -- 172.21.15.78:0/3467782549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9aff0 msgr2=0x563b5bb9b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:45.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.579+0000 7f638d4ec740 1 --2- 172.21.15.78:0/3467782549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9aff0 0x563b5bb9b3d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f637409b300 tx=0x7f6374009c30 comp rx=0 tx=0).stop 2024-08-18T23:52:45.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.579+0000 7f638d4ec740 1 -- 172.21.15.78:0/3467782549 shutdown_connections 2024-08-18T23:52:45.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.579+0000 7f638d4ec740 1 --2- 172.21.15.78:0/3467782549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bba61d0 0x563b5bba85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.579+0000 7f638d4ec740 1 --2- 172.21.15.78:0/3467782549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bb9b910 0x563b5bba5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.579+0000 7f638d4ec740 1 --2- 172.21.15.78:0/3467782549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9aff0 0x563b5bb9b3d0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.579+0000 7f638d4ec740 1 -- 172.21.15.78:0/3467782549 >> 172.21.15.78:0/3467782549 conn(0x563b5bb97580 msgr2=0x563b5bb97960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.579+0000 7f638d4ec740 1 -- 172.21.15.78:0/3467782549 shutdown_connections 2024-08-18T23:52:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.579+0000 7f638d4ec740 1 -- 172.21.15.78:0/3467782549 wait complete. 2024-08-18T23:52:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.579+0000 7f638d4ec740 1 -- start start 2024-08-18T23:52:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.579+0000 7f638d4ec740 1 Processor -- start 2024-08-18T23:52:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f638d4ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bb9aff0 0x563b5bba3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f638d4ec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bb9b910 0x563b5bba35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f638d4ec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bba61d0 0x563b5bba3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f638d4ec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563b5bba8da0 con 0x563b5bba61d0 2024-08-18T23:52:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f638d4ec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563b5babbb20 con 0x563b5bb9b910 2024-08-18T23:52:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f638d4ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563b5bba8f20 con 0x563b5bb9aff0 2024-08-18T23:52:45.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f638d4ec740 10 client.0 _ll_register_callbacks cb 0x563b5bab9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f638968b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bba61d0 0x563b5bba3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f638968b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bba61d0 0x563b5bba3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52282/0 (socket says 172.21.15.78:52282) 2024-08-18T23:52:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f638968b640 1 -- 172.21.15.78:0/3615085535 learned_addr learned my addr 172.21.15.78:0/3615085535 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f6388e8a640 1 --2- 172.21.15.78:0/3615085535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bb9aff0 0x563b5bba3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f6383fff640 1 --2- 172.21.15.78:0/3615085535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bb9b910 0x563b5bba35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f638968b640 1 -- 172.21.15.78:0/3615085535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bb9aff0 msgr2=0x563b5bba3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f638968b640 1 --2- 172.21.15.78:0/3615085535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bb9aff0 0x563b5bba3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f638968b640 1 -- 172.21.15.78:0/3615085535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bb9b910 msgr2=0x563b5bba35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f638968b640 1 --2- 172.21.15.78:0/3615085535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bb9b910 0x563b5bba35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f638968b640 1 -- 172.21.15.78:0/3615085535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b5b8723c0 con 0x563b5bba61d0 2024-08-18T23:52:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f638968b640 1 --2- 172.21.15.78:0/3615085535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bba61d0 0x563b5bba3b00 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f637c012e20 tx=0x7f637c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f57c197577992f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:45.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f6363fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.580+0000 7f6363fff640 1 -- 172.21.15.78:0/3615085535 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c021070 con 0x563b5bba61d0 2024-08-18T23:52:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.581+0000 7f638d4ec740 1 -- 172.21.15.78:0/3615085535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563b5bb0dfb0 con 0x563b5bba61d0 2024-08-18T23:52:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.581+0000 7f6363fff640 1 -- 172.21.15.78:0/3615085535 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f637c01d400 con 0x563b5bba61d0 2024-08-18T23:52:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.581+0000 7f6363fff640 1 -- 172.21.15.78:0/3615085535 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f637c02d050 con 0x563b5bba61d0 2024-08-18T23:52:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.581+0000 7f638d4ec740 10 client.16908 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.581+0000 7f638d4ec740 1 -- 172.21.15.78:0/3615085535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563b5bbb4940 con 0x563b5bba61d0 2024-08-18T23:52:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.581+0000 7f6363fff640 1 -- 172.21.15.78:0/3615085535 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563b5bb0dfb0 con 0x563b5bba61d0 2024-08-18T23:52:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.581+0000 7f6363fff640 1 -- 172.21.15.78:0/3615085535 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f637c01c040 con 0x563b5bba61d0 2024-08-18T23:52:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.581+0000 7f638d4ec740 10 client.16908 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.581+0000 7f638d4ec740 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:52:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.581+0000 7f638d4ec740 20 client.16908 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.581+0000 7f638d4ec740 1 --2- 172.21.15.78:0/3615085535 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563b5bbb5d50 0x563b5bbb81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:45.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.581+0000 7f638d4ec740 4 client.16908 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:52:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.581+0000 7f638d4ec740 1 -- 172.21.15.78:0/3615085535 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b5bb084c0 con 0x563b5bbb5d50 2024-08-18T23:52:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.581+0000 7f6388e8a640 1 --2- 172.21.15.78:0/3615085535 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563b5bbb5d50 0x563b5bbb81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.582+0000 7f6363fff640 10 client.16908 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.582+0000 7f6388e8a640 1 --2- 172.21.15.78:0/3615085535 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563b5bbb5d50 0x563b5bbb81b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6e36ec81f721cc8 server_cookie=f5f46723a9710ad5 in_seq=0 out_seq=0 2024-08-18T23:52:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.582+0000 7f6363fff640 1 -- 172.21.15.78:0/3615085535 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b5bb084c0 con 0x563b5bbb5d50 2024-08-18T23:52:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.582+0000 7f6363fff640 10 client.16908 handle_command_reply: tid=0 2024-08-18T23:52:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.583+0000 7f638d4ec740 1 client.16908 shutdown 2024-08-18T23:52:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.583+0000 7f638d4ec740 1 -- 172.21.15.78:0/3615085535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bba61d0 msgr2=0x563b5bba3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.583+0000 7f638d4ec740 1 --2- 172.21.15.78:0/3615085535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bba61d0 0x563b5bba3b00 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f637c012e20 tx=0x7f637c004a40 comp rx=0 tx=0).stop 2024-08-18T23:52:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.583+0000 7f638d4ec740 1 -- 172.21.15.78:0/3615085535 shutdown_connections 2024-08-18T23:52:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.583+0000 7f638d4ec740 1 --2- 172.21.15.78:0/3615085535 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563b5bbb5d50 0x563b5bbb81b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.583+0000 7f638d4ec740 1 --2- 172.21.15.78:0/3615085535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bba61d0 0x563b5bba3b00 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.583+0000 7f638d4ec740 1 --2- 172.21.15.78:0/3615085535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bb9b910 0x563b5bba35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.583+0000 7f6363fff640 0 client.16908 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:45.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.583+0000 7f638d4ec740 1 --2- 172.21.15.78:0/3615085535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bb9aff0 0x563b5bba3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.583+0000 7f638d4ec740 1 -- 172.21.15.78:0/3615085535 >> 172.21.15.78:0/3615085535 conn(0x563b5bb97580 msgr2=0x563b5bb9d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:45.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.583+0000 7f638d4ec740 1 -- 172.21.15.78:0/3615085535 shutdown_connections 2024-08-18T23:52:45.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.583+0000 7f638d4ec740 1 -- 172.21.15.78:0/3615085535 wait complete. 2024-08-18T23:52:45.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.583+0000 7f638d4ec740 20 client.16908 trim_cache size 0 max 16384 2024-08-18T23:52:45.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.596+0000 7f638d4ec740 1 Processor -- start 2024-08-18T23:52:45.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.596+0000 7f638d4ec740 1 -- start start 2024-08-18T23:52:45.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.596+0000 7f638d4ec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bb9aff0 0x563b5bb9da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:45.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.596+0000 7f638d4ec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9b910 0x563b5bbbdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:45.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.596+0000 7f638d4ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bba61d0 0x563b5bbbe010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:45.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.596+0000 7f638d4ec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63841b1400 con 0x563b5bb9b910 2024-08-18T23:52:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.596+0000 7f638d4ec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f63841aec70 con 0x563b5bb9aff0 2024-08-18T23:52:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.596+0000 7f638d4ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f63841acac0 con 0x563b5bba61d0 2024-08-18T23:52:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.596+0000 7f6383fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9b910 0x563b5bbbdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.596+0000 7f6383fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9b910 0x563b5bbbdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52286/0 (socket says 172.21.15.78:52286) 2024-08-18T23:52:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.596+0000 7f6383fff640 1 -- 172.21.15.78:0/219471201 learned_addr learned my addr 172.21.15.78:0/219471201 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.596+0000 7f6388e8a640 1 --2- 172.21.15.78:0/219471201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bb9aff0 0x563b5bb9da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.596+0000 7f638968b640 1 --2- 172.21.15.78:0/219471201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bba61d0 0x563b5bbbe010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.597+0000 7f6383fff640 1 -- 172.21.15.78:0/219471201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bba61d0 msgr2=0x563b5bbbe010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.597+0000 7f6383fff640 1 --2- 172.21.15.78:0/219471201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bba61d0 0x563b5bbbe010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.597+0000 7f6383fff640 1 -- 172.21.15.78:0/219471201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bb9aff0 msgr2=0x563b5bb9da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.597+0000 7f6383fff640 1 --2- 172.21.15.78:0/219471201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bb9aff0 0x563b5bb9da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.597+0000 7f6383fff640 1 -- 172.21.15.78:0/219471201 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b5bb97020 con 0x563b5bb9b910 2024-08-18T23:52:45.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.597+0000 7f6383fff640 1 --2- 172.21.15.78:0/219471201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9b910 0x563b5bbbdad0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f636c009930 tx=0x7f636c004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5266ca956b626ad6 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.597+0000 7f637a7fc640 1 -- 172.21.15.78:0/219471201 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c035040 con 0x563b5bb9b910 2024-08-18T23:52:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.597+0000 7f637a7fc640 1 -- 172.21.15.78:0/219471201 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f636c005520 con 0x563b5bb9b910 2024-08-18T23:52:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.597+0000 7f637a7fc640 1 -- 172.21.15.78:0/219471201 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c003850 con 0x563b5bb9b910 2024-08-18T23:52:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.598+0000 7f638d4ec740 1 -- 172.21.15.78:0/219471201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9b910 msgr2=0x563b5bbbdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.598+0000 7f638d4ec740 1 --2- 172.21.15.78:0/219471201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9b910 0x563b5bbbdad0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f636c009930 tx=0x7f636c004310 comp rx=0 tx=0).stop 2024-08-18T23:52:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.598+0000 7f638d4ec740 1 -- 172.21.15.78:0/219471201 shutdown_connections 2024-08-18T23:52:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.598+0000 7f638d4ec740 1 --2- 172.21.15.78:0/219471201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bba61d0 0x563b5bbbe010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.598+0000 7f638d4ec740 1 --2- 172.21.15.78:0/219471201 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9b910 0x563b5bbbdad0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.598+0000 7f638d4ec740 1 --2- 172.21.15.78:0/219471201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bb9aff0 0x563b5bb9da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.598+0000 7f638d4ec740 1 -- 172.21.15.78:0/219471201 >> 172.21.15.78:0/219471201 conn(0x563b5bb97580 msgr2=0x563b5bba2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.598+0000 7f638d4ec740 1 -- 172.21.15.78:0/219471201 shutdown_connections 2024-08-18T23:52:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.598+0000 7f638d4ec740 1 -- 172.21.15.78:0/219471201 wait complete. 2024-08-18T23:52:45.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.598+0000 7f638d4ec740 1 -- start start 2024-08-18T23:52:45.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.598+0000 7f638d4ec740 1 Processor -- start 2024-08-18T23:52:45.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f638d4ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bb9aff0 0x563b5bba96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:45.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f638d4ec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9b910 0x563b5bba9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:45.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f638d4ec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bba61d0 0x563b5bc1ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:45.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f638d4ec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f63841ad130 con 0x563b5bb9b910 2024-08-18T23:52:45.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f638d4ec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f63841ae5a0 con 0x563b5bba61d0 2024-08-18T23:52:45.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f638d4ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f63841ad800 con 0x563b5bb9aff0 2024-08-18T23:52:45.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f638d4ec740 10 client.0 _ll_register_callbacks cb 0x7f637c0089e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:45.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f6383fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9b910 0x563b5bba9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:45.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f6383fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9b910 0x563b5bba9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52290/0 (socket says 172.21.15.78:52290) 2024-08-18T23:52:45.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f6383fff640 1 -- 172.21.15.78:0/4221225698 learned_addr learned my addr 172.21.15.78:0/4221225698 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:45.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f638968b640 1 --2- 172.21.15.78:0/4221225698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bba61d0 0x563b5bc1ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:45.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f6388e8a640 1 --2- 172.21.15.78:0/4221225698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bb9aff0 0x563b5bba96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:45.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f6383fff640 1 -- 172.21.15.78:0/4221225698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bb9aff0 msgr2=0x563b5bba96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:45.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f6383fff640 1 --2- 172.21.15.78:0/4221225698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bb9aff0 0x563b5bba96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f6383fff640 1 -- 172.21.15.78:0/4221225698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bba61d0 msgr2=0x563b5bc1ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:45.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f6383fff640 1 --2- 172.21.15.78:0/4221225698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bba61d0 0x563b5bc1ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f6383fff640 1 -- 172.21.15.78:0/4221225698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b5bc1f000 con 0x563b5bb9b910 2024-08-18T23:52:45.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f638968b640 1 --2- 172.21.15.78:0/4221225698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bba61d0 0x563b5bc1ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:45.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f6383fff640 1 --2- 172.21.15.78:0/4221225698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9b910 0x563b5bba9c30 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f636c02a940 tx=0x7f636c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75925840df7886a2 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:45.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f6363fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:45.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f6363fff640 1 -- 172.21.15.78:0/4221225698 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c04f060 con 0x563b5bb9b910 2024-08-18T23:52:45.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f6363fff640 1 -- 172.21.15.78:0/4221225698 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f636c0317c0 con 0x563b5bb9b910 2024-08-18T23:52:45.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f638d4ec740 1 -- 172.21.15.78:0/4221225698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f636c005520 con 0x563b5bb9b910 2024-08-18T23:52:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.599+0000 7f6363fff640 1 -- 172.21.15.78:0/4221225698 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f636c056050 con 0x563b5bb9b910 2024-08-18T23:52:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.600+0000 7f638d4ec740 10 client.16914 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.600+0000 7f638d4ec740 1 -- 172.21.15.78:0/4221225698 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563b5bc20300 con 0x563b5bb9b910 2024-08-18T23:52:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.600+0000 7f6363fff640 1 -- 172.21.15.78:0/4221225698 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f636c005520 con 0x563b5bb9b910 2024-08-18T23:52:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.600+0000 7f6363fff640 1 -- 172.21.15.78:0/4221225698 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f636c044020 con 0x563b5bb9b910 2024-08-18T23:52:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.600+0000 7f638d4ec740 10 client.16914 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.600+0000 7f638d4ec740 10 client.16914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:52:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.600+0000 7f638d4ec740 20 client.16914 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.600+0000 7f638d4ec740 1 --2- 172.21.15.78:0/4221225698 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563b5bc20a30 0x563b5bc22e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.600+0000 7f638d4ec740 4 client.16914 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:52:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.600+0000 7f638d4ec740 1 -- 172.21.15.78:0/4221225698 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b5ba85170 con 0x563b5bc20a30 2024-08-18T23:52:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.600+0000 7f6388e8a640 1 --2- 172.21.15.78:0/4221225698 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563b5bc20a30 0x563b5bc22e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.601+0000 7f6363fff640 10 client.16914 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.601+0000 7f6388e8a640 1 --2- 172.21.15.78:0/4221225698 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563b5bc20a30 0x563b5bc22e20 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a18d5fb40ade99f server_cookie=b9fe11154631bcb0 in_seq=0 out_seq=0 2024-08-18T23:52:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.601+0000 7f6363fff640 1 -- 172.21.15.78:0/4221225698 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b5ba85170 con 0x563b5bc20a30 2024-08-18T23:52:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.601+0000 7f6363fff640 10 client.16914 handle_command_reply: tid=0 2024-08-18T23:52:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.601+0000 7f638d4ec740 1 client.16914 shutdown 2024-08-18T23:52:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.601+0000 7f638d4ec740 1 -- 172.21.15.78:0/4221225698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9b910 msgr2=0x563b5bba9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.601+0000 7f638d4ec740 1 --2- 172.21.15.78:0/4221225698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9b910 0x563b5bba9c30 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f636c02a940 tx=0x7f636c02a420 comp rx=0 tx=0).stop 2024-08-18T23:52:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.601+0000 7f638d4ec740 1 -- 172.21.15.78:0/4221225698 shutdown_connections 2024-08-18T23:52:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.601+0000 7f638d4ec740 1 --2- 172.21.15.78:0/4221225698 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x563b5bc20a30 0x563b5bc22e20 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.601+0000 7f638d4ec740 1 --2- 172.21.15.78:0/4221225698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b5bba61d0 0x563b5bc1ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.601+0000 7f638d4ec740 1 --2- 172.21.15.78:0/4221225698 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563b5bb9b910 0x563b5bba9c30 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.601+0000 7f6363fff640 0 client.16914 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.601+0000 7f638d4ec740 1 --2- 172.21.15.78:0/4221225698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b5bb9aff0 0x563b5bba96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.601+0000 7f638d4ec740 1 -- 172.21.15.78:0/4221225698 >> 172.21.15.78:0/4221225698 conn(0x563b5bb97580 msgr2=0x563b5bba3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:45.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.601+0000 7f638d4ec740 1 -- 172.21.15.78:0/4221225698 shutdown_connections 2024-08-18T23:52:45.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.601+0000 7f638d4ec740 1 -- 172.21.15.78:0/4221225698 wait complete. 2024-08-18T23:52:45.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.601+0000 7f638d4ec740 20 client.16914 trim_cache size 0 max 16384 2024-08-18T23:52:45.620 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:52:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.602+0000 7f638b0ee640 1 -- 172.21.15.78:0/465982507 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f63700a2b80 msgr2=0x7f63700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.602+0000 7f638b0ee640 1 --2- 172.21.15.78:0/465982507 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f63700a2b80 0x7f63700a5040 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x563b5bb98900 tx=0x7f637c008b20 comp rx=0 tx=0).stop 2024-08-18T23:52:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.602+0000 7f638b0ee640 1 -- 172.21.15.78:0/465982507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63841018b0 msgr2=0x7f6384198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.602+0000 7f638b0ee640 1 --2- 172.21.15.78:0/465982507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63841018b0 0x7f6384198220 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f637400d770 tx=0x7f637400dc30 comp rx=0 tx=0).stop 2024-08-18T23:52:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.602+0000 7f638b0ee640 1 -- 172.21.15.78:0/465982507 shutdown_connections 2024-08-18T23:52:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.602+0000 7f638b0ee640 1 --2- 172.21.15.78:0/465982507 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f63700a2b80 0x7f63700a5040 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.602+0000 7f638b0ee640 1 --2- 172.21.15.78:0/465982507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63841078b0 0x7f638419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.602+0000 7f638b0ee640 1 --2- 172.21.15.78:0/465982507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6384102250 0x7f6384198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.602+0000 7f638b0ee640 1 --2- 172.21.15.78:0/465982507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f63841018b0 0x7f6384198220 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.602+0000 7f638b0ee640 1 -- 172.21.15.78:0/465982507 >> 172.21.15.78:0/465982507 conn(0x7f63840fd5f0 msgr2=0x7f63840fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.602+0000 7f638b0ee640 1 -- 172.21.15.78:0/465982507 shutdown_connections 2024-08-18T23:52:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:45.602+0000 7f638b0ee640 1 -- 172.21.15.78:0/465982507 wait complete. 2024-08-18T23:52:47.384 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-18T23:52:47.384 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-18T23:52:47.427 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:52:47.641 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:47 smithi161 ceph-mon[22027]: pgmap v604: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2024-08-18T23:52:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:47 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:52:47.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:47 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:52:47.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:47 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:52:47.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:47 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:52:47.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:47 smithi078 ceph-mon[18440]: pgmap v604: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2024-08-18T23:52:47.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:47 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:52:47.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:47 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:52:47.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:47 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:52:47.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:47 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:52:47.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:47 smithi123 ceph-mon[21808]: pgmap v604: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2024-08-18T23:52:47.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:47 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:52:47.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:47 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:52:47.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:47 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:52:47.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:47 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:52:48.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.130+0000 7f85084df640 1 -- 172.21.15.78:0/2555188466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85001022a0 msgr2=0x7f85001026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:48.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.130+0000 7f85084df640 1 --2- 172.21.15.78:0/2555188466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85001022a0 0x7f85001026e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009900 tx=0x7f84f002a480 comp rx=0 tx=0).stop 2024-08-18T23:52:48.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.130+0000 7f85084df640 1 -- 172.21.15.78:0/2555188466 shutdown_connections 2024-08-18T23:52:48.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.130+0000 7f85084df640 1 --2- 172.21.15.78:0/2555188466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8500102c20 0x7f850010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.130+0000 7f85084df640 1 --2- 172.21.15.78:0/2555188466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85001022a0 0x7f85001026e0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.130+0000 7f85084df640 1 --2- 172.21.15.78:0/2555188466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8500100b50 0x7f8500100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.130+0000 7f85084df640 1 -- 172.21.15.78:0/2555188466 >> 172.21.15.78:0/2555188466 conn(0x7f85000fc500 msgr2=0x7f85000fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:48.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.130+0000 7f85084df640 1 -- 172.21.15.78:0/2555188466 shutdown_connections 2024-08-18T23:52:48.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.130+0000 7f85084df640 1 -- 172.21.15.78:0/2555188466 wait complete. 2024-08-18T23:52:48.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f85084df640 1 Processor -- start 2024-08-18T23:52:48.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f85084df640 1 -- start start 2024-08-18T23:52:48.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f85084df640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8500100b50 0x7f850010e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:48.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f85084df640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85001022a0 0x7f850010e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:48.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f85084df640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8500102c20 0x7f850010ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:48.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f85084df640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8500111dd0 con 0x7f8500102c20 2024-08-18T23:52:48.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f85084df640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8500111c50 con 0x7f85001022a0 2024-08-18T23:52:48.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f85084df640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8500111f50 con 0x7f8500100b50 2024-08-18T23:52:48.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f8506a7c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8500102c20 0x7f850010ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:48.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f8506a7c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8500102c20 0x7f850010ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55676/0 (socket says 172.21.15.78:55676) 2024-08-18T23:52:48.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f8506a7c640 1 -- 172.21.15.78:0/1539114528 learned_addr learned my addr 172.21.15.78:0/1539114528 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:48.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f8505a7a640 1 --2- 172.21.15.78:0/1539114528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85001022a0 0x7f850010e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:48.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f850627b640 1 --2- 172.21.15.78:0/1539114528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8500100b50 0x7f850010e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:48.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f8506a7c640 1 -- 172.21.15.78:0/1539114528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8500100b50 msgr2=0x7f850010e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:48.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f8506a7c640 1 --2- 172.21.15.78:0/1539114528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8500100b50 0x7f850010e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f8506a7c640 1 -- 172.21.15.78:0/1539114528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85001022a0 msgr2=0x7f850010e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:48.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f8506a7c640 1 --2- 172.21.15.78:0/1539114528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85001022a0 0x7f850010e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f8506a7c640 1 -- 172.21.15.78:0/1539114528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85001a6610 con 0x7f8500102c20 2024-08-18T23:52:48.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f850627b640 1 --2- 172.21.15.78:0/1539114528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8500100b50 0x7f850010e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:48.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.131+0000 7f8506a7c640 1 --2- 172.21.15.78:0/1539114528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8500102c20 0x7f850010ec80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f84f800eaa0 tx=0x7f84f800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:48.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.132+0000 7f84f77fe640 1 -- 172.21.15.78:0/1539114528 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f800cdb0 con 0x7f8500102c20 2024-08-18T23:52:48.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.132+0000 7f85084df640 1 -- 172.21.15.78:0/1539114528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85001a6900 con 0x7f8500102c20 2024-08-18T23:52:48.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.132+0000 7f84f77fe640 1 -- 172.21.15.78:0/1539114528 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f8005020 con 0x7f8500102c20 2024-08-18T23:52:48.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.132+0000 7f84f77fe640 1 -- 172.21.15.78:0/1539114528 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f80116b0 con 0x7f8500102c20 2024-08-18T23:52:48.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.132+0000 7f85084df640 1 -- 172.21.15.78:0/1539114528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85001a6e10 con 0x7f8500102c20 2024-08-18T23:52:48.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.134+0000 7f84f77fe640 1 -- 172.21.15.78:0/1539114528 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f84f8005560 con 0x7f8500102c20 2024-08-18T23:52:48.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.134+0000 7f850a8dd740 1 Processor -- start 2024-08-18T23:52:48.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f850a8dd740 1 -- start start 2024-08-18T23:52:48.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f850a8dd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581c9961200 0x5581c99615e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:48.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f850a8dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961b20 0x5581c996bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:48.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f850a8dd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c996c3e0 0x5581c996e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:48.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f850a8dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5581c9924e40 con 0x5581c9961b20 2024-08-18T23:52:48.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f84f77fe640 1 --2- 172.21.15.78:0/1539114528 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f84ec0a2b80 0x7f84ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:48.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f850a8dd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5581c98c67f0 con 0x5581c996c3e0 2024-08-18T23:52:48.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f850a8dd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5581c986dfb0 con 0x5581c9961200 2024-08-18T23:52:48.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f850627b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961b20 0x5581c996bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:48.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f850627b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961b20 0x5581c996bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55678/0 (socket says 172.21.15.78:55678) 2024-08-18T23:52:48.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f850627b640 1 -- 172.21.15.78:0/3280529676 learned_addr learned my addr 172.21.15.78:0/3280529676 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:48.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f8506a7c640 1 --2- 172.21.15.78:0/1539114528 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f84ec0a2b80 0x7f84ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:48.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f84f77fe640 1 -- 172.21.15.78:0/1539114528 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f84f80cf5e0 con 0x7f8500102c20 2024-08-18T23:52:48.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f8505a7a640 1 --2- 172.21.15.78:0/3280529676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c996c3e0 0x5581c996e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:48.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f8505a7a640 1 --2- 172.21.15.78:0/3280529676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581c9961200 0x5581c99615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:48.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f850627b640 1 -- 172.21.15.78:0/3280529676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581c9961200 msgr2=0x5581c99615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:48.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f850627b640 1 --2- 172.21.15.78:0/3280529676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581c9961200 0x5581c99615e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f850627b640 1 -- 172.21.15.78:0/3280529676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c996c3e0 msgr2=0x5581c996e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f850627b640 1 --2- 172.21.15.78:0/3280529676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c996c3e0 0x5581c996e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f850627b640 1 -- 172.21.15.78:0/3280529676 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581c96386a0 con 0x5581c9961b20 2024-08-18T23:52:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f8505a7a640 1 --2- 172.21.15.78:0/3280529676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c996c3e0 0x5581c996e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:52:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.135+0000 7f850627b640 1 --2- 172.21.15.78:0/3280529676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961b20 0x5581c996bea0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f84fc008c60 tx=0x7f84fc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37d6513ccfbdd35 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.136+0000 7f8506a7c640 1 --2- 172.21.15.78:0/1539114528 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f84ec0a2b80 0x7f84ec0a5040 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f84f809ba40 tx=0x7f84f8090f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.136+0000 7f84f4ff9640 1 -- 172.21.15.78:0/3280529676 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc014070 con 0x5581c9961b20 2024-08-18T23:52:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.136+0000 7f84f4ff9640 1 -- 172.21.15.78:0/3280529676 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84fc0047a0 con 0x5581c9961b20 2024-08-18T23:52:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.136+0000 7f84f4ff9640 1 -- 172.21.15.78:0/3280529676 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc010740 con 0x5581c9961b20 2024-08-18T23:52:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.136+0000 7f850a8dd740 1 -- 172.21.15.78:0/3280529676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961b20 msgr2=0x5581c996bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.136+0000 7f850a8dd740 1 --2- 172.21.15.78:0/3280529676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961b20 0x5581c996bea0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f84fc008c60 tx=0x7f84fc00e720 comp rx=0 tx=0).stop 2024-08-18T23:52:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.136+0000 7f850a8dd740 1 -- 172.21.15.78:0/3280529676 shutdown_connections 2024-08-18T23:52:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.136+0000 7f850a8dd740 1 --2- 172.21.15.78:0/3280529676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c996c3e0 0x5581c996e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.136+0000 7f850a8dd740 1 --2- 172.21.15.78:0/3280529676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961b20 0x5581c996bea0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.136+0000 7f850a8dd740 1 --2- 172.21.15.78:0/3280529676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581c9961200 0x5581c99615e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.136+0000 7f850a8dd740 1 -- 172.21.15.78:0/3280529676 >> 172.21.15.78:0/3280529676 conn(0x5581c995d790 msgr2=0x5581c995db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.137+0000 7f850a8dd740 1 -- 172.21.15.78:0/3280529676 shutdown_connections 2024-08-18T23:52:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.137+0000 7f850a8dd740 1 -- 172.21.15.78:0/3280529676 wait complete. 2024-08-18T23:52:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.137+0000 7f850a8dd740 1 -- start start 2024-08-18T23:52:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.137+0000 7f850a8dd740 1 Processor -- start 2024-08-18T23:52:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.137+0000 7f850a8dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961200 0x5581c9969260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.137+0000 7f850a8dd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c9961b20 0x5581c99697a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.137+0000 7f850a8dd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581c996c3e0 0x5581c9969ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.137+0000 7f850a8dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5581c9881cb0 con 0x5581c9961200 2024-08-18T23:52:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.137+0000 7f850a8dd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5581c98c27b0 con 0x5581c9961b20 2024-08-18T23:52:48.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.137+0000 7f850a8dd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5581c996efb0 con 0x5581c996c3e0 2024-08-18T23:52:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.137+0000 7f850a8dd740 10 client.0 _ll_register_callbacks cb 0x5581c95d5ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.137+0000 7f8505a7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961200 0x5581c9969260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.137+0000 7f8505a7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961200 0x5581c9969260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55680/0 (socket says 172.21.15.78:55680) 2024-08-18T23:52:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.137+0000 7f8505a7a640 1 -- 172.21.15.78:0/3778509053 learned_addr learned my addr 172.21.15.78:0/3778509053 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.137+0000 7f850627b640 1 --2- 172.21.15.78:0/3778509053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c9961b20 0x5581c99697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.138+0000 7f8505a7a640 1 -- 172.21.15.78:0/3778509053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581c996c3e0 msgr2=0x5581c9969ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.138+0000 7f8505a7a640 1 --2- 172.21.15.78:0/3778509053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581c996c3e0 0x5581c9969ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.138+0000 7f8505a7a640 1 -- 172.21.15.78:0/3778509053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c9961b20 msgr2=0x5581c99697a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.138+0000 7f8505a7a640 1 --2- 172.21.15.78:0/3778509053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c9961b20 0x5581c99697a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.138+0000 7f8505a7a640 1 -- 172.21.15.78:0/3778509053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581c9880bc0 con 0x5581c9961200 2024-08-18T23:52:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.138+0000 7f8505a7a640 1 --2- 172.21.15.78:0/3778509053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961200 0x5581c9969260 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009900 tx=0x7f84f00044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1207b4a127de864 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.138+0000 7f84e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.138+0000 7f84e9ffb640 1 -- 172.21.15.78:0/3778509053 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0035040 con 0x5581c9961200 2024-08-18T23:52:48.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.138+0000 7f84e9ffb640 1 -- 172.21.15.78:0/3778509053 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f84f0003ac0 con 0x5581c9961200 2024-08-18T23:52:48.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.138+0000 7f850a8dd740 1 -- 172.21.15.78:0/3778509053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84fc0047a0 con 0x5581c9961200 2024-08-18T23:52:48.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.138+0000 7f84e9ffb640 1 -- 172.21.15.78:0/3778509053 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f84f0003cc0 con 0x5581c9961200 2024-08-18T23:52:48.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.138+0000 7f850a8dd740 10 client.16926 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:48.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.138+0000 7f850a8dd740 1 -- 172.21.15.78:0/3778509053 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5581c997ad70 con 0x5581c9961200 2024-08-18T23:52:48.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.138+0000 7f84e9ffb640 1 -- 172.21.15.78:0/3778509053 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84fc0047a0 con 0x5581c9961200 2024-08-18T23:52:48.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.139+0000 7f84e9ffb640 1 -- 172.21.15.78:0/3778509053 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f84f003c540 con 0x5581c9961200 2024-08-18T23:52:48.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.139+0000 7f850a8dd740 10 client.16926 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:48.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.139+0000 7f850a8dd740 10 client.16926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:52:48.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.139+0000 7f850a8dd740 20 client.16926 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:48.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.139+0000 7f850a8dd740 1 --2- 172.21.15.78:0/3778509053 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5581c997c230 0x5581c997e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:48.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.139+0000 7f850a8dd740 4 client.16926 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:52:48.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.139+0000 7f850a8dd740 1 -- 172.21.15.78:0/3778509053 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581c98d3d60 con 0x5581c997c230 2024-08-18T23:52:48.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.139+0000 7f850627b640 1 --2- 172.21.15.78:0/3778509053 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5581c997c230 0x5581c997e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:48.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.139+0000 7f84e9ffb640 10 client.16926 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:48.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.139+0000 7f850627b640 1 --2- 172.21.15.78:0/3778509053 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5581c997c230 0x5581c997e670 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63e7de9207ecbaed server_cookie=b19d91d1792d1e70 in_seq=0 out_seq=0 2024-08-18T23:52:48.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.140+0000 7f84e9ffb640 1 -- 172.21.15.78:0/3778509053 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581c98d3d60 con 0x5581c997c230 2024-08-18T23:52:48.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.140+0000 7f84e9ffb640 10 client.16926 handle_command_reply: tid=0 2024-08-18T23:52:48.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.140+0000 7f850a8dd740 1 client.16926 shutdown 2024-08-18T23:52:48.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.140+0000 7f850a8dd740 1 -- 172.21.15.78:0/3778509053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961200 msgr2=0x5581c9969260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:48.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.140+0000 7f850a8dd740 1 --2- 172.21.15.78:0/3778509053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961200 0x5581c9969260 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009900 tx=0x7f84f00044a0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.140+0000 7f850a8dd740 1 -- 172.21.15.78:0/3778509053 shutdown_connections 2024-08-18T23:52:48.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.140+0000 7f850a8dd740 1 --2- 172.21.15.78:0/3778509053 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5581c997c230 0x5581c997e670 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.140+0000 7f850a8dd740 1 --2- 172.21.15.78:0/3778509053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581c996c3e0 0x5581c9969ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.140+0000 7f84e9ffb640 0 client.16926 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:48.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.140+0000 7f850a8dd740 1 --2- 172.21.15.78:0/3778509053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c9961b20 0x5581c99697a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.140+0000 7f850a8dd740 1 --2- 172.21.15.78:0/3778509053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961200 0x5581c9969260 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.140+0000 7f850a8dd740 1 -- 172.21.15.78:0/3778509053 >> 172.21.15.78:0/3778509053 conn(0x5581c995d790 msgr2=0x5581c9963470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:48.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.140+0000 7f850a8dd740 1 -- 172.21.15.78:0/3778509053 shutdown_connections 2024-08-18T23:52:48.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.140+0000 7f850a8dd740 1 -- 172.21.15.78:0/3778509053 wait complete. 2024-08-18T23:52:48.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.140+0000 7f850a8dd740 20 client.16926 trim_cache size 0 max 16384 2024-08-18T23:52:48.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.157+0000 7f850a8dd740 1 Processor -- start 2024-08-18T23:52:48.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.157+0000 7f850a8dd740 1 -- start start 2024-08-18T23:52:48.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.158+0000 7f850a8dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961200 0x5581c9963c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:48.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.158+0000 7f850a8dd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c9961b20 0x5581c996b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:48.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.158+0000 7f850a8dd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581c996c3e0 0x5581c9983f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:48.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.158+0000 7f850a8dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85001b3450 con 0x5581c9961200 2024-08-18T23:52:48.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.158+0000 7f850a8dd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f85001b0cc0 con 0x5581c9961b20 2024-08-18T23:52:48.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.158+0000 7f850a8dd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f85001aeb10 con 0x5581c996c3e0 2024-08-18T23:52:48.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.158+0000 7f8505a7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961200 0x5581c9963c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:48.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.158+0000 7f8505a7a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961200 0x5581c9963c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55686/0 (socket says 172.21.15.78:55686) 2024-08-18T23:52:48.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.158+0000 7f8505a7a640 1 -- 172.21.15.78:0/1783357129 learned_addr learned my addr 172.21.15.78:0/1783357129 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:48.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.158+0000 7f850627b640 1 --2- 172.21.15.78:0/1783357129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c9961b20 0x5581c996b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:48.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.158+0000 7f8505a7a640 1 -- 172.21.15.78:0/1783357129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581c996c3e0 msgr2=0x5581c9983f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:48.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.158+0000 7f8505a7a640 1 --2- 172.21.15.78:0/1783357129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581c996c3e0 0x5581c9983f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.158+0000 7f8505a7a640 1 -- 172.21.15.78:0/1783357129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c9961b20 msgr2=0x5581c996b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:48.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.158+0000 7f8505a7a640 1 --2- 172.21.15.78:0/1783357129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c9961b20 0x5581c996b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.158+0000 7f8505a7a640 1 -- 172.21.15.78:0/1783357129 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581c9961950 con 0x5581c9961200 2024-08-18T23:52:48.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.158+0000 7f8505a7a640 1 --2- 172.21.15.78:0/1783357129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961200 0x5581c9963c60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f84f002a9a0 tx=0x7f84f00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c005e9a308474793 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:48.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.159+0000 7f84ebfff640 1 -- 172.21.15.78:0/1783357129 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0035040 con 0x5581c9961200 2024-08-18T23:52:48.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.159+0000 7f84ebfff640 1 -- 172.21.15.78:0/1783357129 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f0059070 con 0x5581c9961200 2024-08-18T23:52:48.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.159+0000 7f84ebfff640 1 -- 172.21.15.78:0/1783357129 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f003c6a0 con 0x5581c9961200 2024-08-18T23:52:48.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.159+0000 7f850a8dd740 1 -- 172.21.15.78:0/1783357129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961200 msgr2=0x5581c9963c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:48.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.159+0000 7f850a8dd740 1 --2- 172.21.15.78:0/1783357129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961200 0x5581c9963c60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f84f002a9a0 tx=0x7f84f00040c0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.159+0000 7f850a8dd740 1 -- 172.21.15.78:0/1783357129 shutdown_connections 2024-08-18T23:52:48.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.159+0000 7f850a8dd740 1 --2- 172.21.15.78:0/1783357129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581c996c3e0 0x5581c9983f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.159+0000 7f850a8dd740 1 --2- 172.21.15.78:0/1783357129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c9961b20 0x5581c996b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.159+0000 7f850a8dd740 1 --2- 172.21.15.78:0/1783357129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961200 0x5581c9963c60 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.159+0000 7f850a8dd740 1 -- 172.21.15.78:0/1783357129 >> 172.21.15.78:0/1783357129 conn(0x5581c995d790 msgr2=0x5581c996aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:48.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.160+0000 7f850a8dd740 1 -- 172.21.15.78:0/1783357129 shutdown_connections 2024-08-18T23:52:48.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.160+0000 7f850a8dd740 1 -- 172.21.15.78:0/1783357129 wait complete. 2024-08-18T23:52:48.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.160+0000 7f850a8dd740 1 -- start start 2024-08-18T23:52:48.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.160+0000 7f850a8dd740 1 Processor -- start 2024-08-18T23:52:48.171 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:48.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.160+0000 7f850a8dd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581c9961200 0x5581c9966330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:48.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.160+0000 7f850a8dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961b20 0x5581c9966870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:48.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.160+0000 7f850a8dd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c996c3e0 0x5581c9966db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:48.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.160+0000 7f850a8dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85001af180 con 0x5581c9961b20 2024-08-18T23:52:48.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.160+0000 7f850a8dd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f85001b05f0 con 0x5581c996c3e0 2024-08-18T23:52:48.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.160+0000 7f850a8dd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f85001af850 con 0x5581c9961200 2024-08-18T23:52:48.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.160+0000 7f850a8dd740 10 client.0 _ll_register_callbacks cb 0x7f84f0004130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:48.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.160+0000 7f850627b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961b20 0x5581c9966870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:48.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.160+0000 7f850627b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961b20 0x5581c9966870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55694/0 (socket says 172.21.15.78:55694) 2024-08-18T23:52:48.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.160+0000 7f850627b640 1 -- 172.21.15.78:0/1530671815 learned_addr learned my addr 172.21.15.78:0/1530671815 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:48.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.161+0000 7f8505a7a640 1 --2- 172.21.15.78:0/1530671815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581c9961200 0x5581c9966330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:48.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.161+0000 7f8505a7a640 1 --2- 172.21.15.78:0/1530671815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c996c3e0 0x5581c9966db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:48.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.161+0000 7f850627b640 1 -- 172.21.15.78:0/1530671815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581c9961200 msgr2=0x5581c9966330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:48.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.161+0000 7f850627b640 1 --2- 172.21.15.78:0/1530671815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581c9961200 0x5581c9966330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.161+0000 7f850627b640 1 -- 172.21.15.78:0/1530671815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c996c3e0 msgr2=0x5581c9966db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:48.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.161+0000 7f850627b640 1 --2- 172.21.15.78:0/1530671815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c996c3e0 0x5581c9966db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.161+0000 7f850627b640 1 -- 172.21.15.78:0/1530671815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581c99e5fc0 con 0x5581c9961b20 2024-08-18T23:52:48.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.161+0000 7f850627b640 1 --2- 172.21.15.78:0/1530671815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961b20 0x5581c9966870 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f84fc008c60 tx=0x7f84fc008430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6b537822be2fbb server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:48.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.161+0000 7f84e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:48.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.161+0000 7f84e9ffb640 1 -- 172.21.15.78:0/1530671815 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc014040 con 0x5581c9961b20 2024-08-18T23:52:48.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.161+0000 7f84e9ffb640 1 -- 172.21.15.78:0/1530671815 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f84fc004550 con 0x5581c9961b20 2024-08-18T23:52:48.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.161+0000 7f850a8dd740 1 -- 172.21.15.78:0/1530671815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84f0059070 con 0x5581c9961b20 2024-08-18T23:52:48.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.161+0000 7f84e9ffb640 1 -- 172.21.15.78:0/1530671815 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f84fc004750 con 0x5581c9961b20 2024-08-18T23:52:48.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.161+0000 7f850a8dd740 10 client.16932 fetch_fsmap learned FSMap version 31 2024-08-18T23:52:48.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.161+0000 7f850a8dd740 1 -- 172.21.15.78:0/1530671815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5581c99e7370 con 0x5581c9961b20 2024-08-18T23:52:48.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.161+0000 7f84e9ffb640 1 -- 172.21.15.78:0/1530671815 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f0059070 con 0x5581c9961b20 2024-08-18T23:52:48.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.162+0000 7f84e9ffb640 1 -- 172.21.15.78:0/1530671815 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f84fc025820 con 0x5581c9961b20 2024-08-18T23:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.162+0000 7f850a8dd740 10 client.16932 fetch_fsmap finished waiting for FSMap version 31 2024-08-18T23:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.162+0000 7f850a8dd740 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.162+0000 7f850a8dd740 20 client.16932 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.162+0000 7f850a8dd740 1 --2- 172.21.15.78:0/1530671815 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5581c99e7a60 0x5581c99e9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.162+0000 7f850a8dd740 4 client.16932 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-18T23:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.162+0000 7f850a8dd740 1 -- 172.21.15.78:0/1530671815 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5581c98ce270 con 0x5581c99e7a60 2024-08-18T23:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.162+0000 7f8505a7a640 1 --2- 172.21.15.78:0/1530671815 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5581c99e7a60 0x5581c99e9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.162+0000 7f84e9ffb640 10 client.16932 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.162+0000 7f8505a7a640 1 --2- 172.21.15.78:0/1530671815 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5581c99e7a60 0x5581c99e9e80 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70b84f3be5d45736 server_cookie=25f9c799cb5abb5 in_seq=0 out_seq=0 2024-08-18T23:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.163+0000 7f84e9ffb640 1 -- 172.21.15.78:0/1530671815 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5581c98ce270 con 0x5581c99e7a60 2024-08-18T23:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.163+0000 7f84e9ffb640 10 client.16932 handle_command_reply: tid=0 2024-08-18T23:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.163+0000 7f850a8dd740 1 client.16932 shutdown 2024-08-18T23:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.163+0000 7f850a8dd740 1 -- 172.21.15.78:0/1530671815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961b20 msgr2=0x5581c9966870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:48.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.163+0000 7f850a8dd740 1 --2- 172.21.15.78:0/1530671815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961b20 0x5581c9966870 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f84fc008c60 tx=0x7f84fc008430 comp rx=0 tx=0).stop 2024-08-18T23:52:48.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.164+0000 7f850a8dd740 1 -- 172.21.15.78:0/1530671815 shutdown_connections 2024-08-18T23:52:48.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.164+0000 7f850a8dd740 1 --2- 172.21.15.78:0/1530671815 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x5581c99e7a60 0x5581c99e9e80 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.164+0000 7f850a8dd740 1 --2- 172.21.15.78:0/1530671815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581c996c3e0 0x5581c9966db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.164+0000 7f850a8dd740 1 --2- 172.21.15.78:0/1530671815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581c9961b20 0x5581c9966870 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.164+0000 7f84e9ffb640 0 client.16932 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:48.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.164+0000 7f850a8dd740 1 --2- 172.21.15.78:0/1530671815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5581c9961200 0x5581c9966330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.164+0000 7f850a8dd740 1 -- 172.21.15.78:0/1530671815 >> 172.21.15.78:0/1530671815 conn(0x5581c995d790 msgr2=0x5581c995c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:48.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.164+0000 7f850a8dd740 1 -- 172.21.15.78:0/1530671815 shutdown_connections 2024-08-18T23:52:48.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.164+0000 7f850a8dd740 1 -- 172.21.15.78:0/1530671815 wait complete. 2024-08-18T23:52:48.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.164+0000 7f850a8dd740 20 client.16932 trim_cache size 0 max 16384 2024-08-18T23:52:48.178 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:52:48.178 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-08-18T23:52:48.178 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "2354aaf0-b85a-41d5-8c99-2a1b31f64a89", 2024-08-18T23:52:48.178 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-08-18T23:52:48.178 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:52:48.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.164+0000 7f85084df640 1 -- 172.21.15.78:0/1539114528 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f84ec0a2b80 msgr2=0x7f84ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:48.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.164+0000 7f85084df640 1 --2- 172.21.15.78:0/1539114528 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f84ec0a2b80 0x7f84ec0a5040 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f84f809ba40 tx=0x7f84f8090f90 comp rx=0 tx=0).stop 2024-08-18T23:52:48.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.164+0000 7f85084df640 1 -- 172.21.15.78:0/1539114528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8500102c20 msgr2=0x7f850010ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:48.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.164+0000 7f85084df640 1 --2- 172.21.15.78:0/1539114528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8500102c20 0x7f850010ec80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f84f800eaa0 tx=0x7f84f800ef60 comp rx=0 tx=0).stop 2024-08-18T23:52:48.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.165+0000 7f85084df640 1 -- 172.21.15.78:0/1539114528 shutdown_connections 2024-08-18T23:52:48.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.165+0000 7f85084df640 1 --2- 172.21.15.78:0/1539114528 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f84ec0a2b80 0x7f84ec0a5040 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.165+0000 7f85084df640 1 --2- 172.21.15.78:0/1539114528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8500102c20 0x7f850010ec80 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.165+0000 7f85084df640 1 --2- 172.21.15.78:0/1539114528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85001022a0 0x7f850010e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.165+0000 7f85084df640 1 --2- 172.21.15.78:0/1539114528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8500100b50 0x7f850010e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:48.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.165+0000 7f85084df640 1 -- 172.21.15.78:0/1539114528 >> 172.21.15.78:0/1539114528 conn(0x7f85000fc500 msgr2=0x7f85000fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:48.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.165+0000 7f85084df640 1 -- 172.21.15.78:0/1539114528 shutdown_connections 2024-08-18T23:52:48.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:48.165+0000 7f85084df640 1 -- 172.21.15.78:0/1539114528 wait complete. 2024-08-18T23:52:49.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.183+0000 7f0d06387640 1 -- 172.21.15.78:0/1787854240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d001022a0 msgr2=0x7f0d001026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:49.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.183+0000 7f0d06387640 1 --2- 172.21.15.78:0/1787854240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d001022a0 0x7f0d001026e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009900 tx=0x7f0cf402a480 comp rx=0 tx=0).stop 2024-08-18T23:52:49.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.183+0000 7f0d06387640 1 -- 172.21.15.78:0/1787854240 shutdown_connections 2024-08-18T23:52:49.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.183+0000 7f0d06387640 1 --2- 172.21.15.78:0/1787854240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d00102c20 0x7f0d0010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:49.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.183+0000 7f0d06387640 1 --2- 172.21.15.78:0/1787854240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d001022a0 0x7f0d001026e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:49.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.183+0000 7f0d06387640 1 --2- 172.21.15.78:0/1787854240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d00100b50 0x7f0d00100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:49.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.183+0000 7f0d06387640 1 -- 172.21.15.78:0/1787854240 >> 172.21.15.78:0/1787854240 conn(0x7f0d000fc500 msgr2=0x7f0d000fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:49.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.183+0000 7f0d06387640 1 -- 172.21.15.78:0/1787854240 shutdown_connections 2024-08-18T23:52:49.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.183+0000 7f0d06387640 1 -- 172.21.15.78:0/1787854240 wait complete. 2024-08-18T23:52:49.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.183+0000 7f0d06387640 1 Processor -- start 2024-08-18T23:52:49.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.183+0000 7f0d06387640 1 -- start start 2024-08-18T23:52:49.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.184+0000 7f0d06387640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d00100b50 0x7f0d001990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:49.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.184+0000 7f0d06387640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d001022a0 0x7f0d00199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:49.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.184+0000 7f0d06387640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d00102c20 0x7f0d001a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:49.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.184+0000 7f0d06387640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d00110cb0 con 0x7f0d001022a0 2024-08-18T23:52:49.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.184+0000 7f0d06387640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d00110b30 con 0x7f0d00100b50 2024-08-18T23:52:49.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.184+0000 7f0d06387640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0d00110e30 con 0x7f0d00102c20 2024-08-18T23:52:49.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.184+0000 7f0cff7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d001022a0 0x7f0d00199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:49.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.184+0000 7f0cff7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d001022a0 0x7f0d00199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55714/0 (socket says 172.21.15.78:55714) 2024-08-18T23:52:49.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.184+0000 7f0cff7fe640 1 -- 172.21.15.78:0/1067091167 learned_addr learned my addr 172.21.15.78:0/1067091167 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:49.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.184+0000 7f0cfffff640 1 --2- 172.21.15.78:0/1067091167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d00100b50 0x7f0d001990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:49.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.184+0000 7f0d04924640 1 --2- 172.21.15.78:0/1067091167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d00102c20 0x7f0d001a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:49.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.184+0000 7f0cff7fe640 1 -- 172.21.15.78:0/1067091167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d00102c20 msgr2=0x7f0d001a4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:49.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.184+0000 7f0cff7fe640 1 --2- 172.21.15.78:0/1067091167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d00102c20 0x7f0d001a4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:49.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.184+0000 7f0cff7fe640 1 -- 172.21.15.78:0/1067091167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d00100b50 msgr2=0x7f0d001990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:49.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.184+0000 7f0cff7fe640 1 --2- 172.21.15.78:0/1067091167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d00100b50 0x7f0d001990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:49.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.184+0000 7f0cff7fe640 1 -- 172.21.15.78:0/1067091167 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d001a4680 con 0x7f0d001022a0 2024-08-18T23:52:49.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.184+0000 7f0d04924640 1 --2- 172.21.15.78:0/1067091167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d00102c20 0x7f0d001a4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:49.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.184+0000 7f0cff7fe640 1 --2- 172.21.15.78:0/1067091167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d001022a0 0x7f0d00199600 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4006010 tx=0x7f0cf4003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:49.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.185+0000 7f0cfd7fa640 1 -- 172.21.15.78:0/1067091167 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf403f070 con 0x7f0d001022a0 2024-08-18T23:52:49.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.185+0000 7f0d06387640 1 -- 172.21.15.78:0/1067091167 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d001a4850 con 0x7f0d001022a0 2024-08-18T23:52:49.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.185+0000 7f0cfd7fa640 1 -- 172.21.15.78:0/1067091167 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf4031e30 con 0x7f0d001022a0 2024-08-18T23:52:49.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.185+0000 7f0cfd7fa640 1 -- 172.21.15.78:0/1067091167 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4035030 con 0x7f0d001022a0 2024-08-18T23:52:49.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.185+0000 7f0d06387640 1 -- 172.21.15.78:0/1067091167 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d001a4d00 con 0x7f0d001022a0 2024-08-18T23:52:49.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.186+0000 7f0d06387640 1 -- 172.21.15.78:0/1067091167 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d00100f30 con 0x7f0d001022a0 2024-08-18T23:52:49.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.187+0000 7f0cfd7fa640 1 -- 172.21.15.78:0/1067091167 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0cf4042070 con 0x7f0d001022a0 2024-08-18T23:52:49.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.187+0000 7f0cfd7fa640 1 --2- 172.21.15.78:0/1067091167 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0ce80a2b80 0x7f0ce80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:49.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.187+0000 7f0cfffff640 1 --2- 172.21.15.78:0/1067091167 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0ce80a2b80 0x7f0ce80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:49.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.187+0000 7f0cfd7fa640 1 -- 172.21.15.78:0/1067091167 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f0cf40efd20 con 0x7f0d001022a0 2024-08-18T23:52:49.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.188+0000 7f0cfffff640 1 --2- 172.21.15.78:0/1067091167 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0ce80a2b80 0x7f0ce80a5040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00058c0 tx=0x7f0cf000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:49.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.192+0000 7f0cfd7fa640 1 -- 172.21.15.78:0/1067091167 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0cf40bc8f0 con 0x7f0d001022a0 2024-08-18T23:52:49.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.407+0000 7f0d06387640 1 -- 172.21.15.78:0/1067091167 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d00108dc0 con 0x7f0d001022a0 2024-08-18T23:52:49.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.407+0000 7f0cfd7fa640 1 -- 172.21.15.78:0/1067091167 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11932 (secure 0 0 0) 0x7f0cf40bc1f0 con 0x7f0d001022a0 2024-08-18T23:52:49.410 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:52:49.410 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":32,"btime":"2024-08-18T23:52:48:372666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:52:47.960244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:52:49.410 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 32 2024-08-18T23:52:49.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.409+0000 7f0d06387640 1 -- 172.21.15.78:0/1067091167 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0ce80a2b80 msgr2=0x7f0ce80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:49.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.409+0000 7f0d06387640 1 --2- 172.21.15.78:0/1067091167 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0ce80a2b80 0x7f0ce80a5040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00058c0 tx=0x7f0cf000a250 comp rx=0 tx=0).stop 2024-08-18T23:52:49.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.409+0000 7f0d06387640 1 -- 172.21.15.78:0/1067091167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d001022a0 msgr2=0x7f0d00199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:49.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.409+0000 7f0d06387640 1 --2- 172.21.15.78:0/1067091167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d001022a0 0x7f0d00199600 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4006010 tx=0x7f0cf4003c80 comp rx=0 tx=0).stop 2024-08-18T23:52:49.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.409+0000 7f0d06387640 1 -- 172.21.15.78:0/1067091167 shutdown_connections 2024-08-18T23:52:49.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.410+0000 7f0d06387640 1 --2- 172.21.15.78:0/1067091167 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0ce80a2b80 0x7f0ce80a5040 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:49.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.410+0000 7f0d06387640 1 --2- 172.21.15.78:0/1067091167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d00102c20 0x7f0d001a4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:49.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.410+0000 7f0d06387640 1 --2- 172.21.15.78:0/1067091167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d001022a0 0x7f0d00199600 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:49.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.410+0000 7f0d06387640 1 --2- 172.21.15.78:0/1067091167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d00100b50 0x7f0d001990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:49.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.410+0000 7f0d06387640 1 -- 172.21.15.78:0/1067091167 >> 172.21.15.78:0/1067091167 conn(0x7f0d000fc500 msgr2=0x7f0d00105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:49.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.410+0000 7f0d06387640 1 -- 172.21.15.78:0/1067091167 shutdown_connections 2024-08-18T23:52:49.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:49.410+0000 7f0d06387640 1 -- 172.21.15.78:0/1067091167 wait complete. 2024-08-18T23:52:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:49 smithi161 ceph-mon[22027]: scrub queued for path: / 2024-08-18T23:52:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:49 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:52:49.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:49 smithi161 ceph-mon[22027]: scrub summary: active paths [/] 2024-08-18T23:52:49.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:49 smithi161 ceph-mon[22027]: scrub summary: idle+waiting paths [/] 2024-08-18T23:52:49.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:49 smithi161 ceph-mon[22027]: pgmap v605: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-08-18T23:52:49.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:49 smithi161 ceph-mon[22027]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:52:49.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:49 smithi078 ceph-mon[18440]: scrub queued for path: / 2024-08-18T23:52:49.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:49 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:52:49.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:49 smithi078 ceph-mon[18440]: scrub summary: active paths [/] 2024-08-18T23:52:49.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:49 smithi078 ceph-mon[18440]: scrub summary: idle+waiting paths [/] 2024-08-18T23:52:49.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:49 smithi078 ceph-mon[18440]: pgmap v605: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-08-18T23:52:49.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:49 smithi078 ceph-mon[18440]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:52:49.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:49 smithi123 ceph-mon[21808]: scrub queued for path: / 2024-08-18T23:52:49.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:49 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:52:49.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:49 smithi123 ceph-mon[21808]: scrub summary: active paths [/] 2024-08-18T23:52:49.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:49 smithi123 ceph-mon[21808]: scrub summary: idle+waiting paths [/] 2024-08-18T23:52:49.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:49 smithi123 ceph-mon[21808]: pgmap v605: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-08-18T23:52:49.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:49 smithi123 ceph-mon[21808]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-18T23:52:50.141 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:52:50.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:50 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1067091167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:52:50.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:50 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1067091167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:52:50.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:50 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1067091167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:52:50.934 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 71c1b92 --release --await' 2024-08-18T23:52:50.934 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 71c1b92 --release --await 2024-08-18T23:52:50.954 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:51.491 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:51 smithi078 ceph-mon[18440]: pgmap v606: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.6 KiB/s wr, 19 op/s 2024-08-18T23:52:51.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:51 smithi161 ceph-mon[22027]: pgmap v606: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.6 KiB/s wr, 19 op/s 2024-08-18T23:52:51.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.461+0000 7f6a1719e640 1 -- 172.21.15.78:0/1118766557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a10100b50 msgr2=0x7f6a10100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:51.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.461+0000 7f6a1719e640 1 --2- 172.21.15.78:0/1118766557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a10100b50 0x7f6a10100f30 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6a00009ca0 tx=0x7f6a0002a950 comp rx=0 tx=0).stop 2024-08-18T23:52:51.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.461+0000 7f6a1719e640 1 -- 172.21.15.78:0/1118766557 shutdown_connections 2024-08-18T23:52:51.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.461+0000 7f6a1719e640 1 --2- 172.21.15.78:0/1118766557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a10102c20 0x7f6a1010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.461+0000 7f6a1719e640 1 --2- 172.21.15.78:0/1118766557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a101022a0 0x7f6a101026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.461+0000 7f6a1719e640 1 --2- 172.21.15.78:0/1118766557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a10100b50 0x7f6a10100f30 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.461+0000 7f6a1719e640 1 -- 172.21.15.78:0/1118766557 >> 172.21.15.78:0/1118766557 conn(0x7f6a100fc500 msgr2=0x7f6a100fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:51.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.461+0000 7f6a1719e640 1 -- 172.21.15.78:0/1118766557 shutdown_connections 2024-08-18T23:52:51.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.461+0000 7f6a1719e640 1 -- 172.21.15.78:0/1118766557 wait complete. 2024-08-18T23:52:51.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.461+0000 7f6a1719e640 1 Processor -- start 2024-08-18T23:52:51.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.462+0000 7f6a1719e640 1 -- start start 2024-08-18T23:52:51.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.462+0000 7f6a1719e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a10100b50 0x7f6a101971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:51.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.462+0000 7f6a1719e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a101022a0 0x7f6a101976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:51.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.462+0000 7f6a1719e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a10102c20 0x7f6a1019ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:51.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.462+0000 7f6a1719e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10110c70 con 0x7f6a10102c20 2024-08-18T23:52:51.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.462+0000 7f6a1719e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10110af0 con 0x7f6a101022a0 2024-08-18T23:52:51.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.462+0000 7f6a1719e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10110df0 con 0x7f6a10100b50 2024-08-18T23:52:51.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.462+0000 7f6a1573b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a10102c20 0x7f6a1019ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:51.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.462+0000 7f6a1573b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a10102c20 0x7f6a1019ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55736/0 (socket says 172.21.15.78:55736) 2024-08-18T23:52:51.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.462+0000 7f6a1573b640 1 -- 172.21.15.78:0/1065575034 learned_addr learned my addr 172.21.15.78:0/1065575034 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:51.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.462+0000 7f6a14f3a640 1 --2- 172.21.15.78:0/1065575034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a10100b50 0x7f6a101971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:51.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.462+0000 7f6a0ffff640 1 --2- 172.21.15.78:0/1065575034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a101022a0 0x7f6a101976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:51.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.462+0000 7f6a1573b640 1 -- 172.21.15.78:0/1065575034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a10100b50 msgr2=0x7f6a101971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:51.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.462+0000 7f6a1573b640 1 --2- 172.21.15.78:0/1065575034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a10100b50 0x7f6a101971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.462+0000 7f6a1573b640 1 -- 172.21.15.78:0/1065575034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a101022a0 msgr2=0x7f6a101976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:51.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.462+0000 7f6a1573b640 1 --2- 172.21.15.78:0/1065575034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a101022a0 0x7f6a101976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.462+0000 7f6a1573b640 1 -- 172.21.15.78:0/1065575034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a1019c160 con 0x7f6a10102c20 2024-08-18T23:52:51.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.462+0000 7f6a1573b640 1 --2- 172.21.15.78:0/1065575034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a10102c20 0x7f6a1019ba80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f6a08009860 tx=0x7f6a08009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:51.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.462+0000 7f6a0dffb640 1 -- 172.21.15.78:0/1065575034 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08005020 con 0x7f6a10102c20 2024-08-18T23:52:51.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.463+0000 7f6a1719e640 1 -- 172.21.15.78:0/1065575034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a1019c450 con 0x7f6a10102c20 2024-08-18T23:52:51.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.463+0000 7f6a1719e640 1 -- 172.21.15.78:0/1065575034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a10069ca0 con 0x7f6a10102c20 2024-08-18T23:52:51.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.463+0000 7f6a0dffb640 1 -- 172.21.15.78:0/1065575034 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a080047a0 con 0x7f6a10102c20 2024-08-18T23:52:51.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.463+0000 7f6a0dffb640 1 -- 172.21.15.78:0/1065575034 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a080079a0 con 0x7f6a10102c20 2024-08-18T23:52:51.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.465+0000 7f6a0dffb640 1 -- 172.21.15.78:0/1065575034 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6a08007c20 con 0x7f6a10102c20 2024-08-18T23:52:51.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.465+0000 7f6a1959c740 1 Processor -- start 2024-08-18T23:52:51.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.465+0000 7f6a1959c740 1 -- start start 2024-08-18T23:52:51.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a1959c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557267e8b210 0x557267e8b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:51.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a1959c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8bb30 0x557267e95eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:51.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a1959c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e963f0 0x557267e987e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:51.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a1959c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557267df0980 con 0x557267e8bb30 2024-08-18T23:52:51.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a1959c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557267d88a70 con 0x557267e8b210 2024-08-18T23:52:51.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a1959c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557267dec940 con 0x557267e963f0 2024-08-18T23:52:51.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a0dffb640 1 --2- 172.21.15.78:0/1065575034 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f69fc0a2b80 0x7f69fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:51.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a14f3a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8bb30 0x557267e95eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:51.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a14f3a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8bb30 0x557267e95eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55748/0 (socket says 172.21.15.78:55748) 2024-08-18T23:52:51.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a14f3a640 1 -- 172.21.15.78:0/280063067 learned_addr learned my addr 172.21.15.78:0/280063067 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:51.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a0ffff640 1 --2- 172.21.15.78:0/280063067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557267e8b210 0x557267e8b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:51.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a1573b640 1 --2- 172.21.15.78:0/1065575034 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f69fc0a2b80 0x7f69fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:51.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a0ffff640 1 --2- 172.21.15.78:0/280063067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e963f0 0x557267e987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:51.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a0dffb640 1 -- 172.21.15.78:0/1065575034 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f6a080d0670 con 0x7f6a10102c20 2024-08-18T23:52:51.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a14f3a640 1 -- 172.21.15.78:0/280063067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e963f0 msgr2=0x557267e987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:51.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a14f3a640 1 --2- 172.21.15.78:0/280063067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e963f0 0x557267e987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a14f3a640 1 -- 172.21.15.78:0/280063067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557267e8b210 msgr2=0x557267e8b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:51.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a14f3a640 1 --2- 172.21.15.78:0/280063067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557267e8b210 0x557267e8b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a14f3a640 1 -- 172.21.15.78:0/280063067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557267deb6e0 con 0x557267e8bb30 2024-08-18T23:52:51.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a0ffff640 1 --2- 172.21.15.78:0/280063067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e963f0 0x557267e987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:51.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a14f3a640 1 --2- 172.21.15.78:0/280063067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8bb30 0x557267e95eb0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6a00035000 tx=0x7f6a00009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ae77fdb5d8c410 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:51.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.466+0000 7f6a1573b640 1 --2- 172.21.15.78:0/1065575034 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f69fc0a2b80 0x7f69fc0a5040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6a08098fa0 tx=0x7f6a0806fc00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:51.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.467+0000 7f6a06ffd640 1 -- 172.21.15.78:0/280063067 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00048070 con 0x557267e8bb30 2024-08-18T23:52:51.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.467+0000 7f6a06ffd640 1 -- 172.21.15.78:0/280063067 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a0002fa70 con 0x557267e8bb30 2024-08-18T23:52:51.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.467+0000 7f6a06ffd640 1 -- 172.21.15.78:0/280063067 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00034040 con 0x557267e8bb30 2024-08-18T23:52:51.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.467+0000 7f6a1959c740 1 -- 172.21.15.78:0/280063067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8bb30 msgr2=0x557267e95eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:51.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.467+0000 7f6a1959c740 1 --2- 172.21.15.78:0/280063067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8bb30 0x557267e95eb0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6a00035000 tx=0x7f6a00009960 comp rx=0 tx=0).stop 2024-08-18T23:52:51.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.467+0000 7f6a1959c740 1 -- 172.21.15.78:0/280063067 shutdown_connections 2024-08-18T23:52:51.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.467+0000 7f6a1959c740 1 --2- 172.21.15.78:0/280063067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e963f0 0x557267e987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.467+0000 7f6a1959c740 1 --2- 172.21.15.78:0/280063067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8bb30 0x557267e95eb0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.467+0000 7f6a1959c740 1 --2- 172.21.15.78:0/280063067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557267e8b210 0x557267e8b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.467+0000 7f6a1959c740 1 -- 172.21.15.78:0/280063067 >> 172.21.15.78:0/280063067 conn(0x557267e877a0 msgr2=0x557267e87b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:51.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.467+0000 7f6a1959c740 1 -- 172.21.15.78:0/280063067 shutdown_connections 2024-08-18T23:52:51.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.467+0000 7f6a1959c740 1 -- 172.21.15.78:0/280063067 wait complete. 2024-08-18T23:52:51.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.468+0000 7f6a1959c740 1 -- start start 2024-08-18T23:52:51.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.468+0000 7f6a1959c740 1 Processor -- start 2024-08-18T23:52:51.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.468+0000 7f6a1959c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e8b210 0x557267e93240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:51.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.468+0000 7f6a1959c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557267e8bb30 0x557267e93780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:51.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.468+0000 7f6a1959c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e963f0 0x557267e93cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:51.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.468+0000 7f6a1959c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557267e98fc0 con 0x557267e963f0 2024-08-18T23:52:51.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.468+0000 7f6a1959c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557267dabaa0 con 0x557267e8bb30 2024-08-18T23:52:51.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.468+0000 7f6a1959c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557267e99140 con 0x557267e8b210 2024-08-18T23:52:51.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.468+0000 7f6a1959c740 10 client.0 _ll_register_callbacks cb 0x557267d9ae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:51.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.468+0000 7f6a0ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e963f0 0x557267e93cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:51.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.468+0000 7f6a0ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e963f0 0x557267e93cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55754/0 (socket says 172.21.15.78:55754) 2024-08-18T23:52:51.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.468+0000 7f6a0ffff640 1 -- 172.21.15.78:0/1810184981 learned_addr learned my addr 172.21.15.78:0/1810184981 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:51.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.469+0000 7f6a0ffff640 1 --2- 172.21.15.78:0/1810184981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e8b210 0x557267e93240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:51.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.469+0000 7f6a14f3a640 1 --2- 172.21.15.78:0/1810184981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557267e8bb30 0x557267e93780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:51.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.469+0000 7f6a0ffff640 1 -- 172.21.15.78:0/1810184981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e8b210 msgr2=0x557267e93240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:51.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.469+0000 7f6a0ffff640 1 --2- 172.21.15.78:0/1810184981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e8b210 0x557267e93240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.469+0000 7f6a0ffff640 1 -- 172.21.15.78:0/1810184981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557267e8bb30 msgr2=0x557267e93780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:51.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.469+0000 7f6a0ffff640 1 --2- 172.21.15.78:0/1810184981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557267e8bb30 0x557267e93780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.469+0000 7f6a0ffff640 1 -- 172.21.15.78:0/1810184981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557267d8f650 con 0x557267e963f0 2024-08-18T23:52:51.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.469+0000 7f6a0ffff640 1 --2- 172.21.15.78:0/1810184981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e963f0 0x557267e93cc0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f69f800ef30 tx=0x7f69f8015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a7f997ff4aafbf server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:51.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.469+0000 7f69effff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:51.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.469+0000 7f69effff640 1 -- 172.21.15.78:0/1810184981 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8004020 con 0x557267e963f0 2024-08-18T23:52:51.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.469+0000 7f69effff640 1 -- 172.21.15.78:0/1810184981 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f69f80041c0 con 0x557267e963f0 2024-08-18T23:52:51.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.469+0000 7f6a1959c740 1 -- 172.21.15.78:0/1810184981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a0002fa70 con 0x557267e963f0 2024-08-18T23:52:51.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.469+0000 7f69effff640 1 -- 172.21.15.78:0/1810184981 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f69f8025050 con 0x557267e963f0 2024-08-18T23:52:51.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.469+0000 7f6a1959c740 10 client.16950 fetch_fsmap learned FSMap version 32 2024-08-18T23:52:51.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.469+0000 7f6a1959c740 1 -- 172.21.15.78:0/1810184981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557267b623c0 con 0x557267e963f0 2024-08-18T23:52:51.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.469+0000 7f69effff640 1 -- 172.21.15.78:0/1810184981 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a0002fa70 con 0x557267e963f0 2024-08-18T23:52:51.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.470+0000 7f69effff640 1 -- 172.21.15.78:0/1810184981 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f69f8010430 con 0x557267e963f0 2024-08-18T23:52:51.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.470+0000 7f6a1959c740 10 client.16950 fetch_fsmap finished waiting for FSMap version 32 2024-08-18T23:52:51.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.470+0000 7f6a1959c740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:52:51.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.470+0000 7f6a1959c740 20 client.16950 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:51.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.470+0000 7f6a1959c740 1 --2- 172.21.15.78:0/1810184981 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x557267ea5f80 0x557267ea83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:51.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.470+0000 7f6a1959c740 4 client.16950 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:52:51.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.470+0000 7f6a1959c740 1 -- 172.21.15.78:0/1810184981 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557267dfdef0 con 0x557267ea5f80 2024-08-18T23:52:51.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.470+0000 7f6a14f3a640 1 --2- 172.21.15.78:0/1810184981 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x557267ea5f80 0x557267ea83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:51.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.470+0000 7f69effff640 10 client.16950 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:51.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.470+0000 7f6a14f3a640 1 --2- 172.21.15.78:0/1810184981 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x557267ea5f80 0x557267ea83e0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b26edbd6ce8d555e server_cookie=5d2eb8a8d8014208 in_seq=0 out_seq=0 2024-08-18T23:52:51.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.471+0000 7f69effff640 1 -- 172.21.15.78:0/1810184981 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557267dfdef0 con 0x557267ea5f80 2024-08-18T23:52:51.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.471+0000 7f69effff640 10 client.16950 handle_command_reply: tid=0 2024-08-18T23:52:51.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.471+0000 7f6a1959c740 1 client.16950 shutdown 2024-08-18T23:52:51.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.472+0000 7f6a1959c740 1 -- 172.21.15.78:0/1810184981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e963f0 msgr2=0x557267e93cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:51.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.472+0000 7f6a1959c740 1 --2- 172.21.15.78:0/1810184981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e963f0 0x557267e93cc0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f69f800ef30 tx=0x7f69f8015040 comp rx=0 tx=0).stop 2024-08-18T23:52:51.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.472+0000 7f6a1959c740 1 -- 172.21.15.78:0/1810184981 shutdown_connections 2024-08-18T23:52:51.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.472+0000 7f6a1959c740 1 --2- 172.21.15.78:0/1810184981 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x557267ea5f80 0x557267ea83e0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.472+0000 7f6a1959c740 1 --2- 172.21.15.78:0/1810184981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e963f0 0x557267e93cc0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.472+0000 7f6a1959c740 1 --2- 172.21.15.78:0/1810184981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557267e8bb30 0x557267e93780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.472+0000 7f69effff640 0 client.16950 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:51.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.472+0000 7f6a1959c740 1 --2- 172.21.15.78:0/1810184981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e8b210 0x557267e93240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.472+0000 7f6a1959c740 1 -- 172.21.15.78:0/1810184981 >> 172.21.15.78:0/1810184981 conn(0x557267e877a0 msgr2=0x557267e86900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:51.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.472+0000 7f6a1959c740 1 -- 172.21.15.78:0/1810184981 shutdown_connections 2024-08-18T23:52:51.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.472+0000 7f6a1959c740 1 -- 172.21.15.78:0/1810184981 wait complete. 2024-08-18T23:52:51.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.472+0000 7f6a1959c740 20 client.16950 trim_cache size 0 max 16384 2024-08-18T23:52:51.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.489+0000 7f6a1959c740 1 Processor -- start 2024-08-18T23:52:51.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.489+0000 7f6a1959c740 1 -- start start 2024-08-18T23:52:51.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.489+0000 7f6a1959c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8b210 0x557267eadcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:51.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.489+0000 7f6a1959c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e8bb30 0x557267eae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:51.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.489+0000 7f6a1959c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557267e963f0 0x557267e8ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:51.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.489+0000 7f6a1959c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69f0012470 con 0x557267e8b210 2024-08-18T23:52:51.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.489+0000 7f6a1959c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69f000fce0 con 0x557267e963f0 2024-08-18T23:52:51.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.489+0000 7f6a1959c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f69f000db30 con 0x557267e8bb30 2024-08-18T23:52:51.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.489+0000 7f6a0ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8b210 0x557267eadcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:51.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.489+0000 7f6a0ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8b210 0x557267eadcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55760/0 (socket says 172.21.15.78:55760) 2024-08-18T23:52:51.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.489+0000 7f6a0ffff640 1 -- 172.21.15.78:0/2770048736 learned_addr learned my addr 172.21.15.78:0/2770048736 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:51.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.489+0000 7f6a14f3a640 1 --2- 172.21.15.78:0/2770048736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e8bb30 0x557267eae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:51.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.489+0000 7f6a0ffff640 1 -- 172.21.15.78:0/2770048736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e8bb30 msgr2=0x557267eae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:51.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.489+0000 7f6a0ffff640 1 --2- 172.21.15.78:0/2770048736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e8bb30 0x557267eae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.489+0000 7f6a0ffff640 1 -- 172.21.15.78:0/2770048736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557267e963f0 msgr2=0x557267e8ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:51.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.489+0000 7f6a0ffff640 1 --2- 172.21.15.78:0/2770048736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557267e963f0 0x557267e8ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.489+0000 7f6a0ffff640 1 -- 172.21.15.78:0/2770048736 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557267e8b960 con 0x557267e8b210 2024-08-18T23:52:51.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.489+0000 7f6a0ffff640 1 --2- 172.21.15.78:0/2770048736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8b210 0x557267eadcb0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f69f8000c00 tx=0x7f69f8005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca563e2e9ed6e3f server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:51.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.490+0000 7f6a067fc640 1 -- 172.21.15.78:0/2770048736 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f802d030 con 0x557267e8b210 2024-08-18T23:52:51.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.490+0000 7f6a067fc640 1 -- 172.21.15.78:0/2770048736 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69f8010430 con 0x557267e8b210 2024-08-18T23:52:51.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.490+0000 7f6a067fc640 1 -- 172.21.15.78:0/2770048736 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f80298d0 con 0x557267e8b210 2024-08-18T23:52:51.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.490+0000 7f6a1959c740 1 -- 172.21.15.78:0/2770048736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8b210 msgr2=0x557267eadcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:51.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.490+0000 7f6a1959c740 1 --2- 172.21.15.78:0/2770048736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8b210 0x557267eadcb0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f69f8000c00 tx=0x7f69f8005a10 comp rx=0 tx=0).stop 2024-08-18T23:52:51.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.490+0000 7f6a1959c740 1 -- 172.21.15.78:0/2770048736 shutdown_connections 2024-08-18T23:52:51.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.490+0000 7f6a1959c740 1 --2- 172.21.15.78:0/2770048736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557267e963f0 0x557267e8ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.490+0000 7f6a1959c740 1 --2- 172.21.15.78:0/2770048736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e8bb30 0x557267eae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.490+0000 7f6a1959c740 1 --2- 172.21.15.78:0/2770048736 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8b210 0x557267eadcb0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.490+0000 7f6a1959c740 1 -- 172.21.15.78:0/2770048736 >> 172.21.15.78:0/2770048736 conn(0x557267e877a0 msgr2=0x557267e92330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:51.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.490+0000 7f6a1959c740 1 -- 172.21.15.78:0/2770048736 shutdown_connections 2024-08-18T23:52:51.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.490+0000 7f6a1959c740 1 -- 172.21.15.78:0/2770048736 wait complete. 2024-08-18T23:52:51.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.491+0000 7f6a1959c740 1 -- start start 2024-08-18T23:52:51.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.491+0000 7f6a1959c740 1 Processor -- start 2024-08-18T23:52:51.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.491+0000 7f6a1959c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e8b210 0x557267e9e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:51.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.491+0000 7f6a1959c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8bb30 0x557267e9ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:51.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.491+0000 7f6a1959c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557267e963f0 0x557267e8fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:51.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.491+0000 7f6a1959c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69f000e1a0 con 0x557267e8bb30 2024-08-18T23:52:51.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.491+0000 7f6a1959c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69f000f610 con 0x557267e963f0 2024-08-18T23:52:51.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.491+0000 7f6a1959c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f69f000e870 con 0x557267e8b210 2024-08-18T23:52:51.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.491+0000 7f6a1959c740 10 client.0 _ll_register_callbacks cb 0x7f69f800e810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:51.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.491+0000 7f6a14f3a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8bb30 0x557267e9ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:51.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.491+0000 7f6a14f3a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8bb30 0x557267e9ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55772/0 (socket says 172.21.15.78:55772) 2024-08-18T23:52:51.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.491+0000 7f6a14f3a640 1 -- 172.21.15.78:0/451178245 learned_addr learned my addr 172.21.15.78:0/451178245 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:51.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f6a14f3a640 1 -- 172.21.15.78:0/451178245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e8b210 msgr2=0x557267e9e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:51.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f6a14f3a640 1 --2- 172.21.15.78:0/451178245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e8b210 0x557267e9e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f6a14f3a640 1 -- 172.21.15.78:0/451178245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557267e963f0 msgr2=0x557267e8fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:51.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f6a14f3a640 1 --2- 172.21.15.78:0/451178245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557267e963f0 0x557267e8fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f6a14f3a640 1 -- 172.21.15.78:0/451178245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557267e90290 con 0x557267e8bb30 2024-08-18T23:52:51.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f6a14f3a640 1 --2- 172.21.15.78:0/451178245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8bb30 0x557267e9ebb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6a00003e20 tx=0x7f6a00003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c19a43fbe80daf server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:51.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f69effff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:51.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f69effff640 1 -- 172.21.15.78:0/451178245 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00048040 con 0x557267e8bb30 2024-08-18T23:52:51.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f69effff640 1 -- 172.21.15.78:0/451178245 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f6a00003af0 con 0x557267e8bb30 2024-08-18T23:52:51.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f6a1959c740 1 -- 172.21.15.78:0/451178245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69f8010430 con 0x557267e8bb30 2024-08-18T23:52:51.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f69effff640 1 -- 172.21.15.78:0/451178245 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a00034040 con 0x557267e8bb30 2024-08-18T23:52:51.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f6a1959c740 10 client.16956 fetch_fsmap learned FSMap version 32 2024-08-18T23:52:51.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f6a1959c740 1 -- 172.21.15.78:0/451178245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557267f10790 con 0x557267e8bb30 2024-08-18T23:52:51.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f69effff640 1 -- 172.21.15.78:0/451178245 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69f8010430 con 0x557267e8bb30 2024-08-18T23:52:51.714 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:52:51.714 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-18T23:52:51.714 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-08-18T23:52:51.714 INFO:teuthology.orchestra.run.smithi078.stdout: "2354aaf0-b85a-41d5-8c99-2a1b31f64a89": { 2024-08-18T23:52:51.714 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-08-18T23:52:51.715 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "2354aaf0-b85a-41d5-8c99-2a1b31f64a89", 2024-08-18T23:52:51.715 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-08-18T23:52:51.715 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:52:51.715 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:52:51.715 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:52:51.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f69effff640 1 -- 172.21.15.78:0/451178245 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6a00004230 con 0x557267e8bb30 2024-08-18T23:52:51.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f6a1959c740 10 client.16956 fetch_fsmap finished waiting for FSMap version 32 2024-08-18T23:52:51.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f6a1959c740 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:52:51.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f6a1959c740 20 client.16956 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:51.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f6a1959c740 1 --2- 172.21.15.78:0/451178245 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x557267ea5f80 0x557267f12b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:51.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f6a1959c740 4 client.16956 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:52:51.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.492+0000 7f6a1959c740 1 -- 172.21.15.78:0/451178245 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557267df8400 con 0x557267ea5f80 2024-08-18T23:52:51.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.493+0000 7f6a0ffff640 1 --2- 172.21.15.78:0/451178245 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x557267ea5f80 0x557267f12b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:51.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.493+0000 7f6a0ffff640 1 --2- 172.21.15.78:0/451178245 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x557267ea5f80 0x557267f12b30 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76038595ea6220a8 server_cookie=c9532e0e429fd90b in_seq=0 out_seq=0 2024-08-18T23:52:51.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.493+0000 7f69effff640 10 client.16956 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:51.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.493+0000 7f69effff640 1 -- 172.21.15.78:0/451178245 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557267df8400 con 0x557267ea5f80 2024-08-18T23:52:51.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.493+0000 7f69effff640 10 client.16956 handle_command_reply: tid=0 2024-08-18T23:52:51.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.493+0000 7f6a1959c740 1 client.16956 shutdown 2024-08-18T23:52:51.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.493+0000 7f6a1959c740 1 -- 172.21.15.78:0/451178245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8bb30 msgr2=0x557267e9ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:51.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.493+0000 7f6a1959c740 1 --2- 172.21.15.78:0/451178245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8bb30 0x557267e9ebb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6a00003e20 tx=0x7f6a00003e50 comp rx=0 tx=0).stop 2024-08-18T23:52:51.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.493+0000 7f6a1959c740 1 -- 172.21.15.78:0/451178245 shutdown_connections 2024-08-18T23:52:51.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.493+0000 7f6a1959c740 1 --2- 172.21.15.78:0/451178245 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x557267ea5f80 0x557267f12b30 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.493+0000 7f6a1959c740 1 --2- 172.21.15.78:0/451178245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557267e963f0 0x557267e8fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.493+0000 7f6a1959c740 1 --2- 172.21.15.78:0/451178245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557267e8bb30 0x557267e9ebb0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.493+0000 7f69effff640 0 client.16956 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:51.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.493+0000 7f6a1959c740 1 --2- 172.21.15.78:0/451178245 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557267e8b210 0x557267e9e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.493+0000 7f6a1959c740 1 -- 172.21.15.78:0/451178245 >> 172.21.15.78:0/451178245 conn(0x557267e877a0 msgr2=0x557267e93200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:51.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.494+0000 7f6a1959c740 1 -- 172.21.15.78:0/451178245 shutdown_connections 2024-08-18T23:52:51.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.494+0000 7f6a1959c740 1 -- 172.21.15.78:0/451178245 wait complete. 2024-08-18T23:52:51.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.494+0000 7f6a1959c740 20 client.16956 trim_cache size 0 max 16384 2024-08-18T23:52:51.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.494+0000 7f6a1719e640 1 -- 172.21.15.78:0/1065575034 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f69fc0a2b80 msgr2=0x7f69fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:51.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.494+0000 7f6a1719e640 1 --2- 172.21.15.78:0/1065575034 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f69fc0a2b80 0x7f69fc0a5040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6a08098fa0 tx=0x7f6a0806fc00 comp rx=0 tx=0).stop 2024-08-18T23:52:51.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.494+0000 7f6a1719e640 1 -- 172.21.15.78:0/1065575034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a10102c20 msgr2=0x7f6a1019ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:51.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.494+0000 7f6a1719e640 1 --2- 172.21.15.78:0/1065575034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a10102c20 0x7f6a1019ba80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f6a08009860 tx=0x7f6a08009d20 comp rx=0 tx=0).stop 2024-08-18T23:52:51.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.494+0000 7f6a1719e640 1 -- 172.21.15.78:0/1065575034 shutdown_connections 2024-08-18T23:52:51.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.494+0000 7f6a1719e640 1 --2- 172.21.15.78:0/1065575034 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f69fc0a2b80 0x7f69fc0a5040 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.494+0000 7f6a1719e640 1 --2- 172.21.15.78:0/1065575034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a10102c20 0x7f6a1019ba80 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.494+0000 7f6a1719e640 1 --2- 172.21.15.78:0/1065575034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a101022a0 0x7f6a101976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.494+0000 7f6a1719e640 1 --2- 172.21.15.78:0/1065575034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a10100b50 0x7f6a101971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:51.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.494+0000 7f6a1719e640 1 -- 172.21.15.78:0/1065575034 >> 172.21.15.78:0/1065575034 conn(0x7f6a100fc500 msgr2=0x7f6a100fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:51.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.494+0000 7f6a1719e640 1 -- 172.21.15.78:0/1065575034 shutdown_connections 2024-08-18T23:52:51.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:51.494+0000 7f6a1719e640 1 -- 172.21.15.78:0/1065575034 wait complete. 2024-08-18T23:52:51.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:51 smithi123 ceph-mon[21808]: pgmap v606: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.6 KiB/s wr, 19 op/s 2024-08-18T23:52:51.996 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:52.827 INFO:tasks.cephfs.filesystem:scrub status for tag:2354aaf0-b85a-41d5-8c99-2a1b31f64a89 - {'path': '/', 'tag': '2354aaf0-b85a-41d5-8c99-2a1b31f64a89', 'options': 'recursive,force'} 2024-08-18T23:52:53.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.026+0000 7fe3d5f82640 1 -- 172.21.15.78:0/271789450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3d01068e0 msgr2=0x7fe3d0106cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:53.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.026+0000 7fe3d5f82640 1 --2- 172.21.15.78:0/271789450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3d01068e0 0x7fe3d0106cc0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009900 tx=0x7fe3c002a410 comp rx=0 tx=0).stop 2024-08-18T23:52:53.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.026+0000 7fe3d5f82640 1 -- 172.21.15.78:0/271789450 shutdown_connections 2024-08-18T23:52:53.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.026+0000 7fe3d5f82640 1 --2- 172.21.15.78:0/271789450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3d01006a0 0x7fe3d010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.026+0000 7fe3d5f82640 1 --2- 172.21.15.78:0/271789450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3d00ffd00 0x7fe3d0100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.026+0000 7fe3d5f82640 1 --2- 172.21.15.78:0/271789450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3d01068e0 0x7fe3d0106cc0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.026+0000 7fe3d5f82640 1 -- 172.21.15.78:0/271789450 >> 172.21.15.78:0/271789450 conn(0x7fe3d00fbc10 msgr2=0x7fe3d00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:53.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.026+0000 7fe3d5f82640 1 -- 172.21.15.78:0/271789450 shutdown_connections 2024-08-18T23:52:53.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.026+0000 7fe3d5f82640 1 -- 172.21.15.78:0/271789450 wait complete. 2024-08-18T23:52:53.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.027+0000 7fe3d5f82640 1 Processor -- start 2024-08-18T23:52:53.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.027+0000 7fe3d5f82640 1 -- start start 2024-08-18T23:52:53.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.027+0000 7fe3d5f82640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3d00ffd00 0x7fe3d0198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:53.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.027+0000 7fe3d5f82640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3d01006a0 0x7fe3d0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:53.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.027+0000 7fe3d5f82640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3d01068e0 0x7fe3d019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:53.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.027+0000 7fe3d5f82640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d0111db0 con 0x7fe3d01068e0 2024-08-18T23:52:53.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.027+0000 7fe3d5f82640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d0111c30 con 0x7fe3d00ffd00 2024-08-18T23:52:53.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.027+0000 7fe3d5f82640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d0111f30 con 0x7fe3d01006a0 2024-08-18T23:52:53.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.028+0000 7fe3cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3d01068e0 0x7fe3d019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:53.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.028+0000 7fe3cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3d01068e0 0x7fe3d019cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55800/0 (socket says 172.21.15.78:55800) 2024-08-18T23:52:53.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.028+0000 7fe3cffff640 1 -- 172.21.15.78:0/243936288 learned_addr learned my addr 172.21.15.78:0/243936288 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:53.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.028+0000 7fe3cf7fe640 1 --2- 172.21.15.78:0/243936288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3d00ffd00 0x7fe3d0198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:53.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.028+0000 7fe3c6ffd640 1 --2- 172.21.15.78:0/243936288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3d01006a0 0x7fe3d0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:53.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.028+0000 7fe3cffff640 1 -- 172.21.15.78:0/243936288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3d01006a0 msgr2=0x7fe3d0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:53.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.028+0000 7fe3cffff640 1 --2- 172.21.15.78:0/243936288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3d01006a0 0x7fe3d0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.028+0000 7fe3cffff640 1 -- 172.21.15.78:0/243936288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3d00ffd00 msgr2=0x7fe3d0198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:53.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.028+0000 7fe3cffff640 1 --2- 172.21.15.78:0/243936288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3d00ffd00 0x7fe3d0198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.028+0000 7fe3cffff640 1 -- 172.21.15.78:0/243936288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d019d1c0 con 0x7fe3d01068e0 2024-08-18T23:52:53.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.028+0000 7fe3cffff640 1 --2- 172.21.15.78:0/243936288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3d01068e0 0x7fe3d019cae0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800e9d0 tx=0x7fe3c800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:53.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.028+0000 7fe3cd7fa640 1 -- 172.21.15.78:0/243936288 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c800cd80 con 0x7fe3d01068e0 2024-08-18T23:52:53.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.028+0000 7fe3cd7fa640 1 -- 172.21.15.78:0/243936288 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c8005020 con 0x7fe3d01068e0 2024-08-18T23:52:53.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.028+0000 7fe3d5f82640 1 -- 172.21.15.78:0/243936288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d019d4b0 con 0x7fe3d01068e0 2024-08-18T23:52:53.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.028+0000 7fe3cd7fa640 1 -- 172.21.15.78:0/243936288 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80116d0 con 0x7fe3d01068e0 2024-08-18T23:52:53.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.028+0000 7fe3d5f82640 1 -- 172.21.15.78:0/243936288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d01a4d90 con 0x7fe3d01068e0 2024-08-18T23:52:53.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.030+0000 7fe3cd7fa640 1 -- 172.21.15.78:0/243936288 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe3c80051d0 con 0x7fe3d01068e0 2024-08-18T23:52:53.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.031+0000 7fe3cd7fa640 1 --2- 172.21.15.78:0/243936288 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe3bc0a2b80 0x7fe3bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:53.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.031+0000 7fe3d8380740 1 Processor -- start 2024-08-18T23:52:53.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.031+0000 7fe3d8380740 1 -- start start 2024-08-18T23:52:53.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.031+0000 7fe3cf7fe640 1 --2- 172.21.15.78:0/243936288 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe3bc0a2b80 0x7fe3bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:53.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.031+0000 7fe3cd7fa640 1 -- 172.21.15.78:0/243936288 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fe3c80cfa70 con 0x7fe3d01068e0 2024-08-18T23:52:53.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.031+0000 7fe3d8380740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d2990 0x55f1333d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:53.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.031+0000 7fe3d8380740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333d32b0 0x55f1333dd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:53.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.031+0000 7fe3d8380740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333ddb70 0x55f1333dff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:53.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.031+0000 7fe3d8380740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f133338c60 con 0x55f1333d32b0 2024-08-18T23:52:53.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.031+0000 7fe3d8380740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f1332d0b90 con 0x55f1333d2990 2024-08-18T23:52:53.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.031+0000 7fe3d8380740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f133334c20 con 0x55f1333ddb70 2024-08-18T23:52:53.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.031+0000 7fe3c6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333d32b0 0x55f1333dd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:53.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.032+0000 7fe3c6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333d32b0 0x55f1333dd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55816/0 (socket says 172.21.15.78:55816) 2024-08-18T23:52:53.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.032+0000 7fe3c6ffd640 1 -- 172.21.15.78:0/377175927 learned_addr learned my addr 172.21.15.78:0/377175927 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:53.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.032+0000 7fe3cf7fe640 1 --2- 172.21.15.78:0/377175927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d2990 0x55f1333d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:53.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.032+0000 7fe3cffff640 1 --2- 172.21.15.78:0/377175927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333ddb70 0x55f1333dff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:53.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.032+0000 7fe3c6ffd640 1 -- 172.21.15.78:0/377175927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333ddb70 msgr2=0x55f1333dff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:53.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.032+0000 7fe3c6ffd640 1 --2- 172.21.15.78:0/377175927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333ddb70 0x55f1333dff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.032+0000 7fe3c6ffd640 1 -- 172.21.15.78:0/377175927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d2990 msgr2=0x55f1333d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:53.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.032+0000 7fe3c6ffd640 1 --2- 172.21.15.78:0/377175927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d2990 0x55f1333d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.032+0000 7fe3c6ffd640 1 -- 172.21.15.78:0/377175927 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f133048580 con 0x55f1333d32b0 2024-08-18T23:52:53.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.032+0000 7fe3cf7fe640 1 --2- 172.21.15.78:0/243936288 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe3bc0a2b80 0x7fe3bc0a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0005ab0 tx=0x7fe3c0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:53.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.032+0000 7fe3c6ffd640 1 --2- 172.21.15.78:0/377175927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333d32b0 0x55f1333dd630 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800bc90 tx=0x7fe3b800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb2f92cb7419044 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:53.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.032+0000 7fe3c5ffb640 1 -- 172.21.15.78:0/377175927 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8015070 con 0x55f1333d32b0 2024-08-18T23:52:53.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.032+0000 7fe3c5ffb640 1 -- 172.21.15.78:0/377175927 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3b801a940 con 0x55f1333d32b0 2024-08-18T23:52:53.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.032+0000 7fe3c5ffb640 1 -- 172.21.15.78:0/377175927 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b80118c0 con 0x55f1333d32b0 2024-08-18T23:52:53.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.033+0000 7fe3d8380740 1 -- 172.21.15.78:0/377175927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333d32b0 msgr2=0x55f1333dd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:53.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.033+0000 7fe3d8380740 1 --2- 172.21.15.78:0/377175927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333d32b0 0x55f1333dd630 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800bc90 tx=0x7fe3b800f730 comp rx=0 tx=0).stop 2024-08-18T23:52:53.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.033+0000 7fe3d8380740 1 -- 172.21.15.78:0/377175927 shutdown_connections 2024-08-18T23:52:53.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.033+0000 7fe3d8380740 1 --2- 172.21.15.78:0/377175927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333ddb70 0x55f1333dff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.033+0000 7fe3d8380740 1 --2- 172.21.15.78:0/377175927 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333d32b0 0x55f1333dd630 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.033+0000 7fe3d8380740 1 --2- 172.21.15.78:0/377175927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d2990 0x55f1333d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.033+0000 7fe3d8380740 1 -- 172.21.15.78:0/377175927 >> 172.21.15.78:0/377175927 conn(0x55f1333cef20 msgr2=0x55f1333cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:53.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.033+0000 7fe3d8380740 1 -- 172.21.15.78:0/377175927 shutdown_connections 2024-08-18T23:52:53.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.033+0000 7fe3d8380740 1 -- 172.21.15.78:0/377175927 wait complete. 2024-08-18T23:52:53.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.033+0000 7fe3d8380740 1 -- start start 2024-08-18T23:52:53.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.033+0000 7fe3d8380740 1 Processor -- start 2024-08-18T23:52:53.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3d8380740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333d2990 0x55f1333daa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:53.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3d8380740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d32b0 0x55f1333daf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:53.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3d8380740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333ddb70 0x55f1333db4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:53.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3d8380740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f1333e0740 con 0x55f1333ddb70 2024-08-18T23:52:53.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3d8380740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f1332d67c0 con 0x55f1333d32b0 2024-08-18T23:52:53.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3d8380740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f1333e08c0 con 0x55f1333d2990 2024-08-18T23:52:53.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3d8380740 10 client.0 _ll_register_callbacks cb 0x55f1332dff50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:53.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333ddb70 0x55f1333db4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:53.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333ddb70 0x55f1333db4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55822/0 (socket says 172.21.15.78:55822) 2024-08-18T23:52:53.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3cffff640 1 -- 172.21.15.78:0/683955582 learned_addr learned my addr 172.21.15.78:0/683955582 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:53.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3c6ffd640 1 --2- 172.21.15.78:0/683955582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d32b0 0x55f1333daf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:53.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3cf7fe640 1 --2- 172.21.15.78:0/683955582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333d2990 0x55f1333daa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:53.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3cffff640 1 -- 172.21.15.78:0/683955582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333d2990 msgr2=0x55f1333daa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:53.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3cffff640 1 --2- 172.21.15.78:0/683955582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333d2990 0x55f1333daa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3cffff640 1 -- 172.21.15.78:0/683955582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d32b0 msgr2=0x55f1333daf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:53.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3cffff640 1 --2- 172.21.15.78:0/683955582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d32b0 0x55f1333daf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3cffff640 1 -- 172.21.15.78:0/683955582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f133112960 con 0x55f1333ddb70 2024-08-18T23:52:53.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3c6ffd640 1 --2- 172.21.15.78:0/683955582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d32b0 0x55f1333daf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3cffff640 1 --2- 172.21.15.78:0/683955582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333ddb70 0x55f1333db4d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009fd0 tx=0x7fe3c809c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61433fca8716bc0c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3aeffd640 1 -- 172.21.15.78:0/683955582 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c801f040 con 0x55f1333ddb70 2024-08-18T23:52:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3aeffd640 1 -- 172.21.15.78:0/683955582 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fe3c80c8060 con 0x55f1333ddb70 2024-08-18T23:52:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.034+0000 7fe3d8380740 1 -- 172.21.15.78:0/683955582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3b801a940 con 0x55f1333ddb70 2024-08-18T23:52:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.035+0000 7fe3aeffd640 1 -- 172.21.15.78:0/683955582 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe3c80a4bb0 con 0x55f1333ddb70 2024-08-18T23:52:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.035+0000 7fe3d8380740 10 client.16968 fetch_fsmap learned FSMap version 32 2024-08-18T23:52:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.035+0000 7fe3d8380740 1 -- 172.21.15.78:0/683955582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f1333ec6f0 con 0x55f1333ddb70 2024-08-18T23:52:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.035+0000 7fe3aeffd640 1 -- 172.21.15.78:0/683955582 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3b801a940 con 0x55f1333ddb70 2024-08-18T23:52:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.035+0000 7fe3aeffd640 1 -- 172.21.15.78:0/683955582 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe3c800f040 con 0x55f1333ddb70 2024-08-18T23:52:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.035+0000 7fe3d8380740 10 client.16968 fetch_fsmap finished waiting for FSMap version 32 2024-08-18T23:52:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.035+0000 7fe3d8380740 10 client.16968 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:52:53.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.035+0000 7fe3d8380740 20 client.16968 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:53.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.035+0000 7fe3d8380740 1 --2- 172.21.15.78:0/683955582 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f1333edb00 0x55f1333eff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:53.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.035+0000 7fe3d8380740 4 client.16968 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:52:53.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.035+0000 7fe3d8380740 1 -- 172.21.15.78:0/683955582 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1333461d0 con 0x55f1333edb00 2024-08-18T23:52:53.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.035+0000 7fe3cf7fe640 1 --2- 172.21.15.78:0/683955582 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f1333edb00 0x55f1333eff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:53.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.036+0000 7fe3aeffd640 10 client.16968 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:53.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.036+0000 7fe3cf7fe640 1 --2- 172.21.15.78:0/683955582 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f1333edb00 0x55f1333eff50 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80c9bdc4766eb80 server_cookie=f22b196c0e7b67c2 in_seq=0 out_seq=0 2024-08-18T23:52:53.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.036+0000 7fe3aeffd640 1 -- 172.21.15.78:0/683955582 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1333461d0 con 0x55f1333edb00 2024-08-18T23:52:53.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.036+0000 7fe3aeffd640 10 client.16968 handle_command_reply: tid=0 2024-08-18T23:52:53.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.037+0000 7fe3d8380740 1 client.16968 shutdown 2024-08-18T23:52:53.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.037+0000 7fe3d8380740 1 -- 172.21.15.78:0/683955582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333ddb70 msgr2=0x55f1333db4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:53.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.037+0000 7fe3d8380740 1 --2- 172.21.15.78:0/683955582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333ddb70 0x55f1333db4d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009fd0 tx=0x7fe3c809c700 comp rx=0 tx=0).stop 2024-08-18T23:52:53.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.037+0000 7fe3d8380740 1 -- 172.21.15.78:0/683955582 shutdown_connections 2024-08-18T23:52:53.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.037+0000 7fe3d8380740 1 --2- 172.21.15.78:0/683955582 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f1333edb00 0x55f1333eff50 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.037+0000 7fe3d8380740 1 --2- 172.21.15.78:0/683955582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333ddb70 0x55f1333db4d0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.037+0000 7fe3d8380740 1 --2- 172.21.15.78:0/683955582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d32b0 0x55f1333daf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.037+0000 7fe3aeffd640 0 client.16968 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:53.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.037+0000 7fe3d8380740 1 --2- 172.21.15.78:0/683955582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333d2990 0x55f1333daa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.037+0000 7fe3d8380740 1 -- 172.21.15.78:0/683955582 >> 172.21.15.78:0/683955582 conn(0x55f1333cef20 msgr2=0x55f1333ce080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:53.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.037+0000 7fe3d8380740 1 -- 172.21.15.78:0/683955582 shutdown_connections 2024-08-18T23:52:53.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.037+0000 7fe3d8380740 1 -- 172.21.15.78:0/683955582 wait complete. 2024-08-18T23:52:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.037+0000 7fe3d8380740 20 client.16968 trim_cache size 0 max 16384 2024-08-18T23:52:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.054+0000 7fe3d8380740 1 Processor -- start 2024-08-18T23:52:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.054+0000 7fe3d8380740 1 -- start start 2024-08-18T23:52:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.054+0000 7fe3d8380740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333d2990 0x55f1333d5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.054+0000 7fe3d8380740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d32b0 0x55f1333f5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.054+0000 7fe3d8380740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333ddb70 0x55f1333f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.054+0000 7fe3d8380740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01b1290 con 0x55f1333ddb70 2024-08-18T23:52:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.054+0000 7fe3d8380740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01aeb00 con 0x55f1333d32b0 2024-08-18T23:52:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.054+0000 7fe3d8380740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01ac950 con 0x55f1333d2990 2024-08-18T23:52:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.054+0000 7fe3cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333ddb70 0x55f1333f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.054+0000 7fe3cffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333ddb70 0x55f1333f5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55836/0 (socket says 172.21.15.78:55836) 2024-08-18T23:52:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.054+0000 7fe3cffff640 1 -- 172.21.15.78:0/222809101 learned_addr learned my addr 172.21.15.78:0/222809101 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.054+0000 7fe3c6ffd640 1 --2- 172.21.15.78:0/222809101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d32b0 0x55f1333f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.054+0000 7fe3cf7fe640 1 --2- 172.21.15.78:0/222809101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333d2990 0x55f1333d5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.054+0000 7fe3cffff640 1 -- 172.21.15.78:0/222809101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333d2990 msgr2=0x55f1333d5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.054+0000 7fe3cffff640 1 --2- 172.21.15.78:0/222809101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333d2990 0x55f1333d5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.054+0000 7fe3cffff640 1 -- 172.21.15.78:0/222809101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d32b0 msgr2=0x55f1333f5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.054+0000 7fe3cffff640 1 --2- 172.21.15.78:0/222809101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d32b0 0x55f1333f5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.054+0000 7fe3cffff640 1 -- 172.21.15.78:0/222809101 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1333f05a0 con 0x55f1333ddb70 2024-08-18T23:52:53.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.054+0000 7fe3cffff640 1 --2- 172.21.15.78:0/222809101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333ddb70 0x55f1333f5d80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80cd550 tx=0x7fe3c809ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05fbb25b873af91 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:53.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.055+0000 7fe3c57fa640 1 -- 172.21.15.78:0/222809101 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c801f050 con 0x55f1333ddb70 2024-08-18T23:52:53.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.055+0000 7fe3c57fa640 1 -- 172.21.15.78:0/222809101 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c80bad00 con 0x55f1333ddb70 2024-08-18T23:52:53.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.055+0000 7fe3c57fa640 1 -- 172.21.15.78:0/222809101 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8096ab0 con 0x55f1333ddb70 2024-08-18T23:52:53.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.055+0000 7fe3d8380740 1 -- 172.21.15.78:0/222809101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333ddb70 msgr2=0x55f1333f5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:53.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.055+0000 7fe3d8380740 1 --2- 172.21.15.78:0/222809101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333ddb70 0x55f1333f5d80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80cd550 tx=0x7fe3c809ba80 comp rx=0 tx=0).stop 2024-08-18T23:52:53.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.056+0000 7fe3d8380740 1 -- 172.21.15.78:0/222809101 shutdown_connections 2024-08-18T23:52:53.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.056+0000 7fe3d8380740 1 --2- 172.21.15.78:0/222809101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333ddb70 0x55f1333f5d80 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.056+0000 7fe3d8380740 1 --2- 172.21.15.78:0/222809101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d32b0 0x55f1333f5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.056+0000 7fe3d8380740 1 --2- 172.21.15.78:0/222809101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333d2990 0x55f1333d5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.056+0000 7fe3d8380740 1 -- 172.21.15.78:0/222809101 >> 172.21.15.78:0/222809101 conn(0x55f1333cef20 msgr2=0x55f1333dcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:53.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.056+0000 7fe3d8380740 1 -- 172.21.15.78:0/222809101 shutdown_connections 2024-08-18T23:52:53.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.056+0000 7fe3d8380740 1 -- 172.21.15.78:0/222809101 wait complete. 2024-08-18T23:52:53.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.056+0000 7fe3d8380740 1 -- start start 2024-08-18T23:52:53.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.056+0000 7fe3d8380740 1 Processor -- start 2024-08-18T23:52:53.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.056+0000 7fe3d8380740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333d2990 0x55f1333e6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:53.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.056+0000 7fe3d8380740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d32b0 0x55f1333e1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:53.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.056+0000 7fe3d8380740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333ddb70 0x55f1333e1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:53.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.056+0000 7fe3d8380740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01acfc0 con 0x55f1333d2990 2024-08-18T23:52:53.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.056+0000 7fe3d8380740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01ae430 con 0x55f1333d32b0 2024-08-18T23:52:53.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.056+0000 7fe3d8380740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01ad690 con 0x55f1333ddb70 2024-08-18T23:52:53.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.056+0000 7fe3d8380740 10 client.0 _ll_register_callbacks cb 0x7fe3c809b8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:53.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.057+0000 7fe3cf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333d2990 0x55f1333e6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:53.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.057+0000 7fe3cf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333d2990 0x55f1333e6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55852/0 (socket says 172.21.15.78:55852) 2024-08-18T23:52:53.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.057+0000 7fe3cf7fe640 1 -- 172.21.15.78:0/891536122 learned_addr learned my addr 172.21.15.78:0/891536122 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:53.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.057+0000 7fe3cffff640 1 --2- 172.21.15.78:0/891536122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333ddb70 0x55f1333e1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:53.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.057+0000 7fe3c6ffd640 1 --2- 172.21.15.78:0/891536122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d32b0 0x55f1333e1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:53.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.057+0000 7fe3cf7fe640 1 -- 172.21.15.78:0/891536122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333ddb70 msgr2=0x55f1333e1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:53.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.057+0000 7fe3cf7fe640 1 --2- 172.21.15.78:0/891536122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333ddb70 0x55f1333e1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.057+0000 7fe3cf7fe640 1 -- 172.21.15.78:0/891536122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d32b0 msgr2=0x55f1333e1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:53.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.057+0000 7fe3cf7fe640 1 --2- 172.21.15.78:0/891536122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d32b0 0x55f1333e1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.057+0000 7fe3cf7fe640 1 -- 172.21.15.78:0/891536122 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f13345bc20 con 0x55f1333d2990 2024-08-18T23:52:53.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.057+0000 7fe3cf7fe640 1 --2- 172.21.15.78:0/891536122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333d2990 0x55f1333e6400 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe3c002a930 tx=0x7fe3c002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81b9b549879cd3 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:53.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.057+0000 7fe3aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:53.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.057+0000 7fe3aeffd640 1 -- 172.21.15.78:0/891536122 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0055060 con 0x55f1333d2990 2024-08-18T23:52:53.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.057+0000 7fe3aeffd640 1 -- 172.21.15.78:0/891536122 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fe3c0004450 con 0x55f1333d2990 2024-08-18T23:52:53.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.057+0000 7fe3d8380740 1 -- 172.21.15.78:0/891536122 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c80bad00 con 0x55f1333d2990 2024-08-18T23:52:53.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.057+0000 7fe3aeffd640 1 -- 172.21.15.78:0/891536122 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe3c005c050 con 0x55f1333d2990 2024-08-18T23:52:53.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.057+0000 7fe3d8380740 10 client.16974 fetch_fsmap learned FSMap version 32 2024-08-18T23:52:53.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.057+0000 7fe3d8380740 1 -- 172.21.15.78:0/891536122 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f13345ce60 con 0x55f1333d2990 2024-08-18T23:52:53.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.058+0000 7fe3aeffd640 1 -- 172.21.15.78:0/891536122 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c80bad00 con 0x55f1333d2990 2024-08-18T23:52:53.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.058+0000 7fe3aeffd640 1 -- 172.21.15.78:0/891536122 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe3c00035f0 con 0x55f1333d2990 2024-08-18T23:52:53.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.058+0000 7fe3d8380740 10 client.16974 fetch_fsmap finished waiting for FSMap version 32 2024-08-18T23:52:53.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.058+0000 7fe3d8380740 10 client.16974 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:52:53.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.058+0000 7fe3d8380740 20 client.16974 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:53.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.058+0000 7fe3d8380740 1 --2- 172.21.15.78:0/891536122 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f13345d590 0x55f13345f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:53.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.058+0000 7fe3d8380740 4 client.16974 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "71c1b92", "release": true, "await": true}] 2024-08-18T23:52:53.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.058+0000 7fe3d8380740 1 -- 172.21.15.78:0/891536122 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "quiesce db", "set_id": "71c1b92", "release": true, "await": true}) -- 0x55f1333406e0 con 0x55f13345d590 2024-08-18T23:52:53.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.058+0000 7fe3c6ffd640 1 --2- 172.21.15.78:0/891536122 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f13345d590 0x55f13345f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:53.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.059+0000 7fe3aeffd640 10 client.16974 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:53.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.059+0000 7fe3c6ffd640 1 --2- 172.21.15.78:0/891536122 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f13345d590 0x55f13345f9b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a6b63e5572a1010 server_cookie=8ed18352fec5cd08 in_seq=0 out_seq=0 2024-08-18T23:52:53.230 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:52:53.230 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 32, 2024-08-18T23:52:53.230 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24452, 2024-08-18T23:52:53.230 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 28, 2024-08-18T23:52:53.230 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-08-18T23:52:53.230 INFO:teuthology.orchestra.run.smithi078.stdout: "71c1b92": { 2024-08-18T23:52:53.230 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 28, 2024-08-18T23:52:53.230 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-08-18T23:52:53.230 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:52:53.230 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-08-18T23:52:53.231 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:52:53.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:52:53.231 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-08-18T23:52:53.231 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 143.8, 2024-08-18T23:52:53.231 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-08-18T23:52:53.231 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-08-18T23:52:53.231 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-08-18T23:52:53.231 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:52:53.231 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-08-18T23:52:53.231 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:52:53.231 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:52:53.232 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:52:53.232 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:52:53.232 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:52:53.232 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:52:53.232 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:52:53.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.067+0000 7fe3aeffd640 1 -- 172.21.15.78:0/891536122 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55f1333406e0 con 0x55f13345d590 2024-08-18T23:52:53.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.067+0000 7fe3aeffd640 10 client.16974 handle_command_reply: tid=0 2024-08-18T23:52:53.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.067+0000 7fe3d8380740 1 client.16974 shutdown 2024-08-18T23:52:53.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d8380740 1 -- 172.21.15.78:0/891536122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333d2990 msgr2=0x55f1333e6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:53.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d8380740 1 --2- 172.21.15.78:0/891536122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333d2990 0x55f1333e6400 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe3c002a930 tx=0x7fe3c002a410 comp rx=0 tx=0).stop 2024-08-18T23:52:53.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d8380740 1 -- 172.21.15.78:0/891536122 shutdown_connections 2024-08-18T23:52:53.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d8380740 1 --2- 172.21.15.78:0/891536122 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f13345d590 0x55f13345f9b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d8380740 1 --2- 172.21.15.78:0/891536122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1333ddb70 0x55f1333e1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d8380740 1 --2- 172.21.15.78:0/891536122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1333d32b0 0x55f1333e1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d8380740 1 --2- 172.21.15.78:0/891536122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1333d2990 0x55f1333e6400 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d8380740 1 -- 172.21.15.78:0/891536122 >> 172.21.15.78:0/891536122 conn(0x55f1333cef20 msgr2=0x55f1333ce080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:53.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3aeffd640 0 client.16974 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:53.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d8380740 1 -- 172.21.15.78:0/891536122 shutdown_connections 2024-08-18T23:52:53.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d8380740 1 -- 172.21.15.78:0/891536122 wait complete. 2024-08-18T23:52:53.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d8380740 20 client.16974 trim_cache size 0 max 16384 2024-08-18T23:52:53.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d5f82640 1 -- 172.21.15.78:0/243936288 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe3bc0a2b80 msgr2=0x7fe3bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:53.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d5f82640 1 --2- 172.21.15.78:0/243936288 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe3bc0a2b80 0x7fe3bc0a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0005ab0 tx=0x7fe3c0005a20 comp rx=0 tx=0).stop 2024-08-18T23:52:53.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d5f82640 1 -- 172.21.15.78:0/243936288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3d01068e0 msgr2=0x7fe3d019cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:53.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d5f82640 1 --2- 172.21.15.78:0/243936288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3d01068e0 0x7fe3d019cae0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800e9d0 tx=0x7fe3c800ee90 comp rx=0 tx=0).stop 2024-08-18T23:52:53.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d5f82640 1 -- 172.21.15.78:0/243936288 shutdown_connections 2024-08-18T23:52:53.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d5f82640 1 --2- 172.21.15.78:0/243936288 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fe3bc0a2b80 0x7fe3bc0a5040 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d5f82640 1 --2- 172.21.15.78:0/243936288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3d01068e0 0x7fe3d019cae0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d5f82640 1 --2- 172.21.15.78:0/243936288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3d01006a0 0x7fe3d0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d5f82640 1 --2- 172.21.15.78:0/243936288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3d00ffd00 0x7fe3d0198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:53.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.068+0000 7fe3d5f82640 1 -- 172.21.15.78:0/243936288 >> 172.21.15.78:0/243936288 conn(0x7fe3d00fbc10 msgr2=0x7fe3d00fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:53.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.069+0000 7fe3d5f82640 1 -- 172.21.15.78:0/243936288 shutdown_connections 2024-08-18T23:52:53.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:53.069+0000 7fe3d5f82640 1 -- 172.21.15.78:0/243936288 wait complete. 2024-08-18T23:52:53.398 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-08-18T23:52:53.398 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-08-18T23:52:53.662 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:53 smithi078 ceph-mon[18440]: pgmap v607: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-08-18T23:52:53.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:53 smithi161 ceph-mon[22027]: pgmap v607: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-08-18T23:52:53.701 DEBUG:teuthology.parallel:result is None 2024-08-18T23:52:53.701 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-08-18T23:52:53.736 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-08-18T23:52:53.736 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-08-18T23:52:53.749 INFO:tasks.quiescer:joining Quiescers 2024-08-18T23:52:53.749 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-08-18T23:52:53.749 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:52:53.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:53 smithi123 ceph-mon[21808]: pgmap v607: 129 pgs: 129 active+clean; 245 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-08-18T23:52:54.052 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '71c1b92', total seconds elapsed: 30.4 2024-08-18T23:52:54.052 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.75 2024-08-18T23:52:54.053 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:52:54.094 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:54.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.596+0000 7f33ad847640 1 -- 172.21.15.78:0/1292220976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a8102a70 msgr2=0x7f33a8102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:54.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.596+0000 7f33ad847640 1 --2- 172.21.15.78:0/1292220976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a8102a70 0x7f33a8102e50 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3398009900 tx=0x7f339802a480 comp rx=0 tx=0).stop 2024-08-18T23:52:54.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.596+0000 7f33ad847640 1 -- 172.21.15.78:0/1292220976 shutdown_connections 2024-08-18T23:52:54.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.596+0000 7f33ad847640 1 --2- 172.21.15.78:0/1292220976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33a810bea0 0x7f33a810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:54.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.596+0000 7f33ad847640 1 --2- 172.21.15.78:0/1292220976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a8103390 0x7f33a810b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.596+0000 7f33ad847640 1 --2- 172.21.15.78:0/1292220976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a8102a70 0x7f33a8102e50 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.596+0000 7f33ad847640 1 -- 172.21.15.78:0/1292220976 >> 172.21.15.78:0/1292220976 conn(0x7f33a80fc4d0 msgr2=0x7f33a80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.596+0000 7f33ad847640 1 -- 172.21.15.78:0/1292220976 shutdown_connections 2024-08-18T23:52:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.596+0000 7f33ad847640 1 -- 172.21.15.78:0/1292220976 wait complete. 2024-08-18T23:52:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.597+0000 7f33ad847640 1 Processor -- start 2024-08-18T23:52:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.597+0000 7f33ad847640 1 -- start start 2024-08-18T23:52:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.597+0000 7f33ad847640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33a8102a70 0x7f33a819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.597+0000 7f33ad847640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a8103390 0x7f33a819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.597+0000 7f33ad847640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a810bea0 0x7f33a819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.597+0000 7f33ad847640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33a81023b0 con 0x7f33a810bea0 2024-08-18T23:52:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.597+0000 7f33ad847640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33a8102230 con 0x7f33a8103390 2024-08-18T23:52:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.597+0000 7f33ad847640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f33a8102530 con 0x7f33a8102a70 2024-08-18T23:52:54.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.598+0000 7f33a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a810bea0 0x7f33a819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:54.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.598+0000 7f33a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a810bea0 0x7f33a819ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55874/0 (socket says 172.21.15.78:55874) 2024-08-18T23:52:54.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.598+0000 7f33a77fe640 1 -- 172.21.15.78:0/824694642 learned_addr learned my addr 172.21.15.78:0/824694642 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:54.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.598+0000 7f33a6ffd640 1 --2- 172.21.15.78:0/824694642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33a8102a70 0x7f33a819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:54.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.598+0000 7f339e7fc640 1 --2- 172.21.15.78:0/824694642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a8103390 0x7f33a819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:54.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.598+0000 7f33a77fe640 1 -- 172.21.15.78:0/824694642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33a8102a70 msgr2=0x7f33a819a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:54.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.598+0000 7f33a77fe640 1 --2- 172.21.15.78:0/824694642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33a8102a70 0x7f33a819a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:54.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.598+0000 7f33a77fe640 1 -- 172.21.15.78:0/824694642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a8103390 msgr2=0x7f33a819a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:54.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.598+0000 7f33a77fe640 1 --2- 172.21.15.78:0/824694642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a8103390 0x7f33a819a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:54.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.598+0000 7f33a77fe640 1 -- 172.21.15.78:0/824694642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a819f2d0 con 0x7f33a810bea0 2024-08-18T23:52:54.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.598+0000 7f33a6ffd640 1 --2- 172.21.15.78:0/824694642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33a8102a70 0x7f33a819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:54.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.598+0000 7f339e7fc640 1 --2- 172.21.15.78:0/824694642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a8103390 0x7f33a819a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:52:54.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.598+0000 7f33a77fe640 1 --2- 172.21.15.78:0/824694642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a810bea0 0x7f33a819ebf0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f33a000eaa0 tx=0x7f33a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:54.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.598+0000 7f33ac845640 1 -- 172.21.15.78:0/824694642 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a000cdb0 con 0x7f33a810bea0 2024-08-18T23:52:54.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.598+0000 7f33ad847640 1 -- 172.21.15.78:0/824694642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a819f5c0 con 0x7f33a810bea0 2024-08-18T23:52:54.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.598+0000 7f33ac845640 1 -- 172.21.15.78:0/824694642 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33a0005020 con 0x7f33a810bea0 2024-08-18T23:52:54.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.598+0000 7f33ac845640 1 -- 172.21.15.78:0/824694642 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a00116d0 con 0x7f33a810bea0 2024-08-18T23:52:54.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.598+0000 7f33ad847640 1 -- 172.21.15.78:0/824694642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a81a6ea0 con 0x7f33a810bea0 2024-08-18T23:52:54.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.599+0000 7f33ad847640 1 -- 172.21.15.78:0/824694642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a8102e50 con 0x7f33a810bea0 2024-08-18T23:52:54.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.600+0000 7f33ac845640 1 -- 172.21.15.78:0/824694642 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f33a00054d0 con 0x7f33a810bea0 2024-08-18T23:52:54.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.601+0000 7f33ac845640 1 --2- 172.21.15.78:0/824694642 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f33940a2b80 0x7f33940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:54.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.601+0000 7f33a6ffd640 1 --2- 172.21.15.78:0/824694642 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f33940a2b80 0x7f33940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:54.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.601+0000 7f33ac845640 1 -- 172.21.15.78:0/824694642 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f33a00cf680 con 0x7f33a810bea0 2024-08-18T23:52:54.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.602+0000 7f33a6ffd640 1 --2- 172.21.15.78:0/824694642 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f33940a2b80 0x7f33940a5040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3398005ab0 tx=0x7f3398005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:54.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.605+0000 7f33ac845640 1 -- 172.21.15.78:0/824694642 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f33a0016030 con 0x7f33a810bea0 2024-08-18T23:52:54.647 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:54 smithi078 ceph-mon[18440]: scrub summary: idle 2024-08-18T23:52:54.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:54 smithi161 ceph-mon[22027]: scrub summary: idle 2024-08-18T23:52:54.773 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:54.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:54 smithi123 ceph-mon[21808]: scrub summary: idle 2024-08-18T23:52:54.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.834+0000 7f33ad847640 1 -- 172.21.15.78:0/824694642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33a81069b0 con 0x7f33a810bea0 2024-08-18T23:52:54.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.835+0000 7f33ac845640 1 -- 172.21.15.78:0/824694642 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11932 (secure 0 0 0) 0x7f33a009c250 con 0x7f33a810bea0 2024-08-18T23:52:54.836 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:52:54.837 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":32,"btime":"2024-08-18T23:52:48:372666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:52:47.960244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:52:54.837 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 32 2024-08-18T23:52:54.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.837+0000 7f33ad847640 1 -- 172.21.15.78:0/824694642 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f33940a2b80 msgr2=0x7f33940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:54.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.837+0000 7f33ad847640 1 --2- 172.21.15.78:0/824694642 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f33940a2b80 0x7f33940a5040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3398005ab0 tx=0x7f3398005a20 comp rx=0 tx=0).stop 2024-08-18T23:52:54.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.837+0000 7f33ad847640 1 -- 172.21.15.78:0/824694642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a810bea0 msgr2=0x7f33a819ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:54.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.837+0000 7f33ad847640 1 --2- 172.21.15.78:0/824694642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a810bea0 0x7f33a819ebf0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f33a000eaa0 tx=0x7f33a000ef60 comp rx=0 tx=0).stop 2024-08-18T23:52:54.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.837+0000 7f33ad847640 1 -- 172.21.15.78:0/824694642 shutdown_connections 2024-08-18T23:52:54.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.837+0000 7f33ad847640 1 --2- 172.21.15.78:0/824694642 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f33940a2b80 0x7f33940a5040 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:54.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.837+0000 7f33ad847640 1 --2- 172.21.15.78:0/824694642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33a810bea0 0x7f33a819ebf0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:54.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.837+0000 7f33ad847640 1 --2- 172.21.15.78:0/824694642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a8103390 0x7f33a819a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:54.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.837+0000 7f33ad847640 1 --2- 172.21.15.78:0/824694642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33a8102a70 0x7f33a819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:54.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.837+0000 7f33ad847640 1 -- 172.21.15.78:0/824694642 >> 172.21.15.78:0/824694642 conn(0x7f33a80fc4d0 msgr2=0x7f33a80fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:54.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.837+0000 7f33ad847640 1 -- 172.21.15.78:0/824694642 shutdown_connections 2024-08-18T23:52:54.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:54.837+0000 7f33ad847640 1 -- 172.21.15.78:0/824694642 wait complete. 2024-08-18T23:52:55.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.496+0000 7fc420762640 1 -- 172.21.15.78:0/728316607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4181022a0 msgr2=0x7fc4181026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:55.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.496+0000 7fc420762640 1 --2- 172.21.15.78:0/728316607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4181022a0 0x7fc4181026e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc408009900 tx=0x7fc40802a480 comp rx=0 tx=0).stop 2024-08-18T23:52:55.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.496+0000 7fc420762640 1 -- 172.21.15.78:0/728316607 shutdown_connections 2024-08-18T23:52:55.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.496+0000 7fc420762640 1 --2- 172.21.15.78:0/728316607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc418102c20 0x7fc41810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:55.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.496+0000 7fc420762640 1 --2- 172.21.15.78:0/728316607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4181022a0 0x7fc4181026e0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:55.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.496+0000 7fc420762640 1 --2- 172.21.15.78:0/728316607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc418100b50 0x7fc418100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:55.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.496+0000 7fc420762640 1 -- 172.21.15.78:0/728316607 >> 172.21.15.78:0/728316607 conn(0x7fc4180fc500 msgr2=0x7fc4180fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:55.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.496+0000 7fc420762640 1 -- 172.21.15.78:0/728316607 shutdown_connections 2024-08-18T23:52:55.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.496+0000 7fc420762640 1 -- 172.21.15.78:0/728316607 wait complete. 2024-08-18T23:52:55.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.497+0000 7fc420762640 1 Processor -- start 2024-08-18T23:52:55.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.497+0000 7fc420762640 1 -- start start 2024-08-18T23:52:55.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.497+0000 7fc420762640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc418100b50 0x7fc41819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:55.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.497+0000 7fc420762640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4181022a0 0x7fc41819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:55.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.497+0000 7fc420762640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc418102c20 0x7fc41819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:55.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.497+0000 7fc420762640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc418110cd0 con 0x7fc418100b50 2024-08-18T23:52:55.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.497+0000 7fc420762640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc418110b50 con 0x7fc418102c20 2024-08-18T23:52:55.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.497+0000 7fc420762640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc418110e50 con 0x7fc4181022a0 2024-08-18T23:52:55.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.497+0000 7fc41e4fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc418100b50 0x7fc41819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:55.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.497+0000 7fc41e4fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc418100b50 0x7fc41819a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55886/0 (socket says 172.21.15.78:55886) 2024-08-18T23:52:55.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.497+0000 7fc41e4fe640 1 -- 172.21.15.78:0/2487105095 learned_addr learned my addr 172.21.15.78:0/2487105095 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:55.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.497+0000 7fc41ecff640 1 --2- 172.21.15.78:0/2487105095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc418102c20 0x7fc41819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:55.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.497+0000 7fc41dcfd640 1 --2- 172.21.15.78:0/2487105095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4181022a0 0x7fc41819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:55.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.497+0000 7fc41e4fe640 1 -- 172.21.15.78:0/2487105095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4181022a0 msgr2=0x7fc41819a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:55.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.497+0000 7fc41e4fe640 1 --2- 172.21.15.78:0/2487105095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4181022a0 0x7fc41819a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:55.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.497+0000 7fc41e4fe640 1 -- 172.21.15.78:0/2487105095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc418102c20 msgr2=0x7fc41819ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:55.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.497+0000 7fc41e4fe640 1 --2- 172.21.15.78:0/2487105095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc418102c20 0x7fc41819ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:55.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.497+0000 7fc41e4fe640 1 -- 172.21.15.78:0/2487105095 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41819f370 con 0x7fc418100b50 2024-08-18T23:52:55.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.497+0000 7fc41ecff640 1 --2- 172.21.15.78:0/2487105095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc418102c20 0x7fc41819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:55.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.498+0000 7fc41e4fe640 1 --2- 172.21.15.78:0/2487105095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc418100b50 0x7fc41819a3c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc41400d950 tx=0x7fc41400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:55.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.498+0000 7fc40f7fe640 1 -- 172.21.15.78:0/2487105095 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414017070 con 0x7fc418100b50 2024-08-18T23:52:55.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.498+0000 7fc40f7fe640 1 -- 172.21.15.78:0/2487105095 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc414009c30 con 0x7fc418100b50 2024-08-18T23:52:55.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.498+0000 7fc420762640 1 -- 172.21.15.78:0/2487105095 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41819f660 con 0x7fc418100b50 2024-08-18T23:52:55.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.498+0000 7fc40f7fe640 1 -- 172.21.15.78:0/2487105095 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41400f6b0 con 0x7fc418100b50 2024-08-18T23:52:55.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.498+0000 7fc420762640 1 -- 172.21.15.78:0/2487105095 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4181a6f40 con 0x7fc418100b50 2024-08-18T23:52:55.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.499+0000 7fc420762640 1 -- 172.21.15.78:0/2487105095 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc418100f30 con 0x7fc418100b50 2024-08-18T23:52:55.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.501+0000 7fc40f7fe640 1 -- 172.21.15.78:0/2487105095 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc4140109d0 con 0x7fc418100b50 2024-08-18T23:52:55.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.501+0000 7fc40f7fe640 1 --2- 172.21.15.78:0/2487105095 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc4040a2b80 0x7fc4040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:55.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.501+0000 7fc41dcfd640 1 --2- 172.21.15.78:0/2487105095 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc4040a2b80 0x7fc4040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:55.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.501+0000 7fc40f7fe640 1 -- 172.21.15.78:0/2487105095 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc4140cf090 con 0x7fc418100b50 2024-08-18T23:52:55.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.502+0000 7fc41dcfd640 1 --2- 172.21.15.78:0/2487105095 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc4040a2b80 0x7fc4040a5040 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc418104910 tx=0x7fc408005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:55.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.504+0000 7fc40f7fe640 1 -- 172.21.15.78:0/2487105095 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc414009de0 con 0x7fc418100b50 2024-08-18T23:52:55.668 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:55 smithi078 ceph-mon[18440]: pgmap v608: 129 pgs: 129 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-08-18T23:52:55.668 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:55 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/824694642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:52:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:55 smithi161 ceph-mon[22027]: pgmap v608: 129 pgs: 129 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-08-18T23:52:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:55 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/824694642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:52:55.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.717+0000 7fc420762640 1 -- 172.21.15.78:0/2487105095 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4181a7110 con 0x7fc418100b50 2024-08-18T23:52:55.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.718+0000 7fc40f7fe640 1 -- 172.21.15.78:0/2487105095 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11932 (secure 0 0 0) 0x7fc41409bc60 con 0x7fc418100b50 2024-08-18T23:52:55.719 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:52:55.720 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":32,"btime":"2024-08-18T23:52:48:372666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:52:47.960244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:52:55.720 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 32 2024-08-18T23:52:55.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.720+0000 7fc420762640 1 -- 172.21.15.78:0/2487105095 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc4040a2b80 msgr2=0x7fc4040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:55.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.720+0000 7fc420762640 1 --2- 172.21.15.78:0/2487105095 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc4040a2b80 0x7fc4040a5040 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc418104910 tx=0x7fc408005a20 comp rx=0 tx=0).stop 2024-08-18T23:52:55.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.720+0000 7fc420762640 1 -- 172.21.15.78:0/2487105095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc418100b50 msgr2=0x7fc41819a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:55.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.720+0000 7fc420762640 1 --2- 172.21.15.78:0/2487105095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc418100b50 0x7fc41819a3c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc41400d950 tx=0x7fc41400de10 comp rx=0 tx=0).stop 2024-08-18T23:52:55.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.720+0000 7fc420762640 1 -- 172.21.15.78:0/2487105095 shutdown_connections 2024-08-18T23:52:55.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.720+0000 7fc420762640 1 --2- 172.21.15.78:0/2487105095 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fc4040a2b80 0x7fc4040a5040 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:55.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.720+0000 7fc420762640 1 --2- 172.21.15.78:0/2487105095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc418102c20 0x7fc41819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:55.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.720+0000 7fc420762640 1 --2- 172.21.15.78:0/2487105095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4181022a0 0x7fc41819a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:55.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.720+0000 7fc420762640 1 --2- 172.21.15.78:0/2487105095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc418100b50 0x7fc41819a3c0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:55.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.720+0000 7fc420762640 1 -- 172.21.15.78:0/2487105095 >> 172.21.15.78:0/2487105095 conn(0x7fc4180fc500 msgr2=0x7fc41806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:55.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.720+0000 7fc420762640 1 -- 172.21.15.78:0/2487105095 shutdown_connections 2024-08-18T23:52:55.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:55.720+0000 7fc420762640 1 -- 172.21.15.78:0/2487105095 wait complete. 2024-08-18T23:52:55.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:55 smithi123 ceph-mon[21808]: pgmap v608: 129 pgs: 129 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-08-18T23:52:55.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:55 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/824694642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:52:56.163 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --cancel --all' 2024-08-18T23:52:56.164 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --cancel --all 2024-08-18T23:52:56.507 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:56 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2487105095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:52:56.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:56 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2487105095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:52:56.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:56 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2487105095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:52:56.873 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 143.75 --await-for 45.0' 2024-08-18T23:52:56.873 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 143.75 --await-for 45.0 2024-08-18T23:52:56.889 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:57.439 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:57 smithi078 ceph-mon[18440]: pgmap v609: 129 pgs: 129 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-08-18T23:52:57.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.429+0000 7f2c8c41d640 1 -- 172.21.15.78:0/3442145181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c841022a0 msgr2=0x7f2c841026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:57.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.429+0000 7f2c8c41d640 1 --2- 172.21.15.78:0/3442145181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c841022a0 0x7f2c841026e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2c78009900 tx=0x7f2c7802a480 comp rx=0 tx=0).stop 2024-08-18T23:52:57.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.430+0000 7f2c8c41d640 1 -- 172.21.15.78:0/3442145181 shutdown_connections 2024-08-18T23:52:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.430+0000 7f2c8c41d640 1 --2- 172.21.15.78:0/3442145181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c84102c20 0x7f2c8410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.430+0000 7f2c8c41d640 1 --2- 172.21.15.78:0/3442145181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c841022a0 0x7f2c841026e0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.430+0000 7f2c8c41d640 1 --2- 172.21.15.78:0/3442145181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c84100b50 0x7f2c84100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.430+0000 7f2c8c41d640 1 -- 172.21.15.78:0/3442145181 >> 172.21.15.78:0/3442145181 conn(0x7f2c840fc500 msgr2=0x7f2c840fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.430+0000 7f2c8c41d640 1 -- 172.21.15.78:0/3442145181 shutdown_connections 2024-08-18T23:52:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.430+0000 7f2c8c41d640 1 -- 172.21.15.78:0/3442145181 wait complete. 2024-08-18T23:52:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.430+0000 7f2c8c41d640 1 Processor -- start 2024-08-18T23:52:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.430+0000 7f2c8c41d640 1 -- start start 2024-08-18T23:52:57.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.430+0000 7f2c8c41d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c84100b50 0x7f2c841983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.430+0000 7f2c8c41d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c841022a0 0x7f2c84198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.430+0000 7f2c8c41d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c84102c20 0x7f2c8419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.430+0000 7f2c8c41d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c84110d50 con 0x7f2c841022a0 2024-08-18T23:52:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.430+0000 7f2c8c41d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c84110bd0 con 0x7f2c84102c20 2024-08-18T23:52:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.430+0000 7f2c8c41d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c84110ed0 con 0x7f2c84100b50 2024-08-18T23:52:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.431+0000 7f2c899b8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c841022a0 0x7f2c84198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.431+0000 7f2c899b8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c841022a0 0x7f2c84198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39914/0 (socket says 172.21.15.78:39914) 2024-08-18T23:52:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.431+0000 7f2c899b8640 1 -- 172.21.15.78:0/2694403670 learned_addr learned my addr 172.21.15.78:0/2694403670 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.431+0000 7f2c8a1b9640 1 --2- 172.21.15.78:0/2694403670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c84100b50 0x7f2c841983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.431+0000 7f2c8a9ba640 1 --2- 172.21.15.78:0/2694403670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c84102c20 0x7f2c8419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.431+0000 7f2c899b8640 1 -- 172.21.15.78:0/2694403670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c84100b50 msgr2=0x7f2c841983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:57.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.431+0000 7f2c899b8640 1 --2- 172.21.15.78:0/2694403670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c84100b50 0x7f2c841983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.431+0000 7f2c899b8640 1 -- 172.21.15.78:0/2694403670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c84102c20 msgr2=0x7f2c8419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.431+0000 7f2c899b8640 1 --2- 172.21.15.78:0/2694403670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c84102c20 0x7f2c8419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.431+0000 7f2c899b8640 1 -- 172.21.15.78:0/2694403670 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8419d380 con 0x7f2c841022a0 2024-08-18T23:52:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.431+0000 7f2c899b8640 1 --2- 172.21.15.78:0/2694403670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c841022a0 0x7f2c84198910 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2c78006010 tx=0x7f2c78003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.431+0000 7f2c837fe640 1 -- 172.21.15.78:0/2694403670 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7803f070 con 0x7f2c841022a0 2024-08-18T23:52:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.431+0000 7f2c837fe640 1 -- 172.21.15.78:0/2694403670 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c78031e30 con 0x7f2c841022a0 2024-08-18T23:52:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.431+0000 7f2c8c41d640 1 -- 172.21.15.78:0/2694403670 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8419d610 con 0x7f2c841022a0 2024-08-18T23:52:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.431+0000 7f2c837fe640 1 -- 172.21.15.78:0/2694403670 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78035030 con 0x7f2c841022a0 2024-08-18T23:52:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.431+0000 7f2c8c41d640 1 -- 172.21.15.78:0/2694403670 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c841a4f50 con 0x7f2c841022a0 2024-08-18T23:52:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.433+0000 7f2c837fe640 1 -- 172.21.15.78:0/2694403670 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2c78042070 con 0x7f2c841022a0 2024-08-18T23:52:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.434+0000 7f2c8e81b740 1 Processor -- start 2024-08-18T23:52:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.434+0000 7f2c8e81b740 1 -- start start 2024-08-18T23:52:57.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.434+0000 7f2c8e81b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea06a0 0x55a922ea0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.434+0000 7f2c8e81b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922ea0fc0 0x55a922eab340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.434+0000 7f2c8e81b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922eab880 0x55a922eadc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.434+0000 7f2c8e81b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a922e018c0 con 0x55a922ea0fc0 2024-08-18T23:52:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.434+0000 7f2c8e81b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a922db39e0 con 0x55a922eab880 2024-08-18T23:52:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.434+0000 7f2c8e81b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a922e0f540 con 0x55a922ea06a0 2024-08-18T23:52:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.434+0000 7f2c8a1b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922ea0fc0 0x55a922eab340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.434+0000 7f2c8a1b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922ea0fc0 0x55a922eab340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39918/0 (socket says 172.21.15.78:39918) 2024-08-18T23:52:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.434+0000 7f2c8a1b9640 1 -- 172.21.15.78:0/715215427 learned_addr learned my addr 172.21.15.78:0/715215427 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.434+0000 7f2c837fe640 1 --2- 172.21.15.78:0/2694403670 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2c6c0a2c00 0x7f2c6c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.434+0000 7f2c8a9ba640 1 --2- 172.21.15.78:0/715215427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea06a0 0x55a922ea0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.434+0000 7f2c8a1b9640 1 -- 172.21.15.78:0/715215427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea06a0 msgr2=0x55a922ea0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.434+0000 7f2c899b8640 1 --2- 172.21.15.78:0/715215427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922eab880 0x55a922eadc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:57.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.434+0000 7f2c8a1b9640 1 --2- 172.21.15.78:0/715215427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea06a0 0x55a922ea0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.434+0000 7f2c8a1b9640 1 -- 172.21.15.78:0/715215427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922eab880 msgr2=0x55a922eadc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:57.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.435+0000 7f2c8a1b9640 1 --2- 172.21.15.78:0/715215427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922eab880 0x55a922eadc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.435+0000 7f2c8a1b9640 1 -- 172.21.15.78:0/715215427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a922d9d640 con 0x55a922ea0fc0 2024-08-18T23:52:57.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.435+0000 7f2c8a9ba640 1 --2- 172.21.15.78:0/715215427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea06a0 0x55a922ea0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:57.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.435+0000 7f2c899b8640 1 --2- 172.21.15.78:0/715215427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922eab880 0x55a922eadc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:52:57.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.435+0000 7f2c8a9ba640 1 --2- 172.21.15.78:0/2694403670 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2c6c0a2c00 0x7f2c6c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:57.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.435+0000 7f2c837fe640 1 -- 172.21.15.78:0/2694403670 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2c780efd50 con 0x7f2c841022a0 2024-08-18T23:52:57.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.435+0000 7f2c8a1b9640 1 --2- 172.21.15.78:0/715215427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922ea0fc0 0x55a922eab340 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009fa0 tx=0x7f2c7400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226fcff92bc2437 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:57.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.435+0000 7f2c80dea640 1 -- 172.21.15.78:0/715215427 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7401a070 con 0x55a922ea0fc0 2024-08-18T23:52:57.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.435+0000 7f2c80dea640 1 -- 172.21.15.78:0/715215427 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c740158f0 con 0x55a922ea0fc0 2024-08-18T23:52:57.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.435+0000 7f2c80dea640 1 -- 172.21.15.78:0/715215427 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7400a910 con 0x55a922ea0fc0 2024-08-18T23:52:57.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.436+0000 7f2c8e81b740 1 -- 172.21.15.78:0/715215427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922ea0fc0 msgr2=0x55a922eab340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:57.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.436+0000 7f2c8a9ba640 1 --2- 172.21.15.78:0/2694403670 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2c6c0a2c00 0x7f2c6c0a50c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00b940 tx=0x7f2c7c0047d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:57.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.436+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/715215427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922ea0fc0 0x55a922eab340 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009fa0 tx=0x7f2c7400c8b0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.436+0000 7f2c8e81b740 1 -- 172.21.15.78:0/715215427 shutdown_connections 2024-08-18T23:52:57.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.436+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/715215427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922eab880 0x55a922eadc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.436+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/715215427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922ea0fc0 0x55a922eab340 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.436+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/715215427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea06a0 0x55a922ea0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.436+0000 7f2c8e81b740 1 -- 172.21.15.78:0/715215427 >> 172.21.15.78:0/715215427 conn(0x55a922e9cc30 msgr2=0x55a922e9d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:57.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.436+0000 7f2c8e81b740 1 -- 172.21.15.78:0/715215427 shutdown_connections 2024-08-18T23:52:57.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.436+0000 7f2c8e81b740 1 -- 172.21.15.78:0/715215427 wait complete. 2024-08-18T23:52:57.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.436+0000 7f2c8e81b740 1 -- start start 2024-08-18T23:52:57.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.436+0000 7f2c8e81b740 1 Processor -- start 2024-08-18T23:52:57.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.436+0000 7f2c8e81b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922ea06a0 0x55a922eab110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:57.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.436+0000 7f2c8e81b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea0fc0 0x55a922ea83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:57.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.436+0000 7f2c8e81b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922eab880 0x55a922ea8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:57.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.436+0000 7f2c8e81b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a922e04c10 con 0x55a922eab880 2024-08-18T23:52:57.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.436+0000 7f2c8e81b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a922dc3f00 con 0x55a922ea06a0 2024-08-18T23:52:57.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.436+0000 7f2c8e81b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a922e09f30 con 0x55a922ea0fc0 2024-08-18T23:52:57.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.437+0000 7f2c8e81b740 10 client.0 _ll_register_callbacks cb 0x55a922b14b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:57.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.437+0000 7f2c8a9ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922eab880 0x55a922ea8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:57.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.437+0000 7f2c8a9ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922eab880 0x55a922ea8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39920/0 (socket says 172.21.15.78:39920) 2024-08-18T23:52:57.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.437+0000 7f2c8a9ba640 1 -- 172.21.15.78:0/2961737854 learned_addr learned my addr 172.21.15.78:0/2961737854 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:57.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.437+0000 7f2c8a1b9640 1 --2- 172.21.15.78:0/2961737854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922ea06a0 0x55a922eab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:57.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.437+0000 7f2c899b8640 1 --2- 172.21.15.78:0/2961737854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea0fc0 0x55a922ea83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:57.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.437+0000 7f2c8a9ba640 1 -- 172.21.15.78:0/2961737854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea0fc0 msgr2=0x55a922ea83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:57.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.437+0000 7f2c8a9ba640 1 --2- 172.21.15.78:0/2961737854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea0fc0 0x55a922ea83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.437+0000 7f2c8a9ba640 1 -- 172.21.15.78:0/2961737854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922ea06a0 msgr2=0x55a922eab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:57.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.437+0000 7f2c8a9ba640 1 --2- 172.21.15.78:0/2961737854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922ea06a0 0x55a922eab110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.437+0000 7f2c8a9ba640 1 -- 172.21.15.78:0/2961737854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a922b77620 con 0x55a922eab880 2024-08-18T23:52:57.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.437+0000 7f2c8a9ba640 1 --2- 172.21.15.78:0/2961737854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922eab880 0x55a922ea8900 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00b7c0 tx=0x7f2c7c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6438ceb90a361b86 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:57.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.437+0000 7f2c69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:57.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.437+0000 7f2c69ffb640 1 -- 172.21.15.78:0/2961737854 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c00f040 con 0x55a922eab880 2024-08-18T23:52:57.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.437+0000 7f2c69ffb640 1 -- 172.21.15.78:0/2961737854 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2c7c0137f0 con 0x55a922eab880 2024-08-18T23:52:57.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.437+0000 7f2c8e81b740 1 -- 172.21.15.78:0/2961737854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c740158f0 con 0x55a922eab880 2024-08-18T23:52:57.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.437+0000 7f2c69ffb640 1 -- 172.21.15.78:0/2961737854 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2c7c02d050 con 0x55a922eab880 2024-08-18T23:52:57.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.437+0000 7f2c8e81b740 10 client.16998 fetch_fsmap learned FSMap version 32 2024-08-18T23:52:57.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.437+0000 7f2c8e81b740 1 -- 172.21.15.78:0/2961737854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a922eb9df0 con 0x55a922eab880 2024-08-18T23:52:57.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.438+0000 7f2c69ffb640 1 -- 172.21.15.78:0/2961737854 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c740158f0 con 0x55a922eab880 2024-08-18T23:52:57.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.438+0000 7f2c69ffb640 1 -- 172.21.15.78:0/2961737854 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f2c7c028030 con 0x55a922eab880 2024-08-18T23:52:57.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.438+0000 7f2c8e81b740 10 client.16998 fetch_fsmap finished waiting for FSMap version 32 2024-08-18T23:52:57.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.438+0000 7f2c8e81b740 10 client.16998 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:52:57.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.438+0000 7f2c8e81b740 20 client.16998 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:57.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.438+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/2961737854 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a922ebb300 0x55a922ebd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:57.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.438+0000 7f2c8e81b740 4 client.16998 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:52:57.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.438+0000 7f2c8e81b740 1 -- 172.21.15.78:0/2961737854 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a922e12e70 con 0x55a922ebb300 2024-08-18T23:52:57.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.438+0000 7f2c8a1b9640 1 --2- 172.21.15.78:0/2961737854 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a922ebb300 0x55a922ebd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:57.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.439+0000 7f2c8a1b9640 1 --2- 172.21.15.78:0/2961737854 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a922ebb300 0x55a922ebd760 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90604cd503299435 server_cookie=d2960533240a023d in_seq=0 out_seq=0 2024-08-18T23:52:57.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.439+0000 7f2c69ffb640 10 client.16998 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:57.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.439+0000 7f2c69ffb640 1 -- 172.21.15.78:0/2961737854 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a922e12e70 con 0x55a922ebb300 2024-08-18T23:52:57.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.439+0000 7f2c69ffb640 10 client.16998 handle_command_reply: tid=0 2024-08-18T23:52:57.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.439+0000 7f2c8e81b740 1 client.16998 shutdown 2024-08-18T23:52:57.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.439+0000 7f2c8e81b740 1 -- 172.21.15.78:0/2961737854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922eab880 msgr2=0x55a922ea8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:57.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.439+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/2961737854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922eab880 0x55a922ea8900 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00b7c0 tx=0x7f2c7c008040 comp rx=0 tx=0).stop 2024-08-18T23:52:57.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.440+0000 7f2c8e81b740 1 -- 172.21.15.78:0/2961737854 shutdown_connections 2024-08-18T23:52:57.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.440+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/2961737854 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a922ebb300 0x55a922ebd760 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.440+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/2961737854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922eab880 0x55a922ea8900 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.440+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/2961737854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea0fc0 0x55a922ea83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.440+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/2961737854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922ea06a0 0x55a922eab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.440+0000 7f2c69ffb640 0 client.16998 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:57.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.440+0000 7f2c8e81b740 1 -- 172.21.15.78:0/2961737854 >> 172.21.15.78:0/2961737854 conn(0x55a922e9cc30 msgr2=0x55a922ea2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:57.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.440+0000 7f2c8e81b740 1 -- 172.21.15.78:0/2961737854 shutdown_connections 2024-08-18T23:52:57.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.440+0000 7f2c8e81b740 1 -- 172.21.15.78:0/2961737854 wait complete. 2024-08-18T23:52:57.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.440+0000 7f2c8e81b740 20 client.16998 trim_cache size 0 max 16384 2024-08-18T23:52:57.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.456+0000 7f2c8e81b740 1 Processor -- start 2024-08-18T23:52:57.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.456+0000 7f2c8e81b740 1 -- start start 2024-08-18T23:52:57.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.457+0000 7f2c8e81b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922ea06a0 0x55a922ec3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:57.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.457+0000 7f2c8e81b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea0fc0 0x55a922ec3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:57.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.457+0000 7f2c8e81b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922eab880 0x55a922eb3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:57.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.457+0000 7f2c8e81b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841aeb80 con 0x55a922ea06a0 2024-08-18T23:52:57.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.457+0000 7f2c8e81b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841ac9d0 con 0x55a922eab880 2024-08-18T23:52:57.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.457+0000 7f2c8e81b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841ae4b0 con 0x55a922ea0fc0 2024-08-18T23:52:57.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.457+0000 7f2c8a1b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922ea06a0 0x55a922ec3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:57.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.457+0000 7f2c8a1b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922ea06a0 0x55a922ec3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39934/0 (socket says 172.21.15.78:39934) 2024-08-18T23:52:57.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.457+0000 7f2c8a1b9640 1 -- 172.21.15.78:0/3973750493 learned_addr learned my addr 172.21.15.78:0/3973750493 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:57.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.457+0000 7f2c899b8640 1 --2- 172.21.15.78:0/3973750493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea0fc0 0x55a922ec3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:57.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.457+0000 7f2c8a1b9640 1 -- 172.21.15.78:0/3973750493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea0fc0 msgr2=0x55a922ec3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:57.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.457+0000 7f2c8a1b9640 1 --2- 172.21.15.78:0/3973750493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea0fc0 0x55a922ec3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.457+0000 7f2c8a1b9640 1 -- 172.21.15.78:0/3973750493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922eab880 msgr2=0x55a922eb3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:57.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.457+0000 7f2c8a1b9640 1 --2- 172.21.15.78:0/3973750493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922eab880 0x55a922eb3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.457+0000 7f2c8a1b9640 1 -- 172.21.15.78:0/3973750493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a922ea6910 con 0x55a922ea06a0 2024-08-18T23:52:57.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.457+0000 7f2c8a1b9640 1 --2- 172.21.15.78:0/3973750493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922ea06a0 0x55a922ec3030 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f2c74004b90 tx=0x7f2c74009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75c32358e5ba36b server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:57.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.458+0000 7f2c6bfff640 1 -- 172.21.15.78:0/3973750493 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74030070 con 0x55a922ea06a0 2024-08-18T23:52:57.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.458+0000 7f2c6bfff640 1 -- 172.21.15.78:0/3973750493 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7400ade0 con 0x55a922ea06a0 2024-08-18T23:52:57.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.458+0000 7f2c6bfff640 1 -- 172.21.15.78:0/3973750493 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7401a040 con 0x55a922ea06a0 2024-08-18T23:52:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.458+0000 7f2c8e81b740 1 -- 172.21.15.78:0/3973750493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922ea06a0 msgr2=0x55a922ec3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.458+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/3973750493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922ea06a0 0x55a922ec3030 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f2c74004b90 tx=0x7f2c74009b30 comp rx=0 tx=0).stop 2024-08-18T23:52:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.458+0000 7f2c8e81b740 1 -- 172.21.15.78:0/3973750493 shutdown_connections 2024-08-18T23:52:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.459+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/3973750493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922eab880 0x55a922eb3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.459+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/3973750493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea0fc0 0x55a922ec3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.459+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/3973750493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922ea06a0 0x55a922ec3030 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.459+0000 7f2c8e81b740 1 -- 172.21.15.78:0/3973750493 >> 172.21.15.78:0/3973750493 conn(0x55a922e9cc30 msgr2=0x55a922ea74b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.459+0000 7f2c8e81b740 1 -- 172.21.15.78:0/3973750493 shutdown_connections 2024-08-18T23:52:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.459+0000 7f2c8e81b740 1 -- 172.21.15.78:0/3973750493 wait complete. 2024-08-18T23:52:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.459+0000 7f2c8e81b740 1 -- start start 2024-08-18T23:52:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.459+0000 7f2c8e81b740 1 Processor -- start 2024-08-18T23:52:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.459+0000 7f2c8e81b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea06a0 0x55a922eb1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.459+0000 7f2c8e81b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922ea0fc0 0x55a922eb20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.459+0000 7f2c8e81b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922eab880 0x55a922f1f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.459+0000 7f2c8e81b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841ad710 con 0x55a922eab880 2024-08-18T23:52:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.459+0000 7f2c8e81b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841ad040 con 0x55a922ea0fc0 2024-08-18T23:52:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.459+0000 7f2c8e81b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841adde0 con 0x55a922ea06a0 2024-08-18T23:52:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.459+0000 7f2c8e81b740 10 client.0 _ll_register_callbacks cb 0x7f2c7c017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.460+0000 7f2c8a1b9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea06a0 0x55a922eb1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.460+0000 7f2c8a1b9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea06a0 0x55a922eb1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.78:54188/0 (socket says 172.21.15.78:54188) 2024-08-18T23:52:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.460+0000 7f2c8a1b9640 1 -- 172.21.15.78:0/1210141052 learned_addr learned my addr 172.21.15.78:0/1210141052 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.460+0000 7f2c8a9ba640 1 --2- 172.21.15.78:0/1210141052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922eab880 0x55a922f1f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.460+0000 7f2c899b8640 1 --2- 172.21.15.78:0/1210141052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922ea0fc0 0x55a922eb20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.460+0000 7f2c8a1b9640 1 -- 172.21.15.78:0/1210141052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922ea0fc0 msgr2=0x55a922eb20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.460+0000 7f2c8a1b9640 1 --2- 172.21.15.78:0/1210141052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922ea0fc0 0x55a922eb20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.460+0000 7f2c8a1b9640 1 -- 172.21.15.78:0/1210141052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922eab880 msgr2=0x55a922f1f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.460+0000 7f2c8a1b9640 1 --2- 172.21.15.78:0/1210141052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922eab880 0x55a922f1f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.460+0000 7f2c8a1b9640 1 -- 172.21.15.78:0/1210141052 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a922f1f610 con 0x55a922ea06a0 2024-08-18T23:52:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.460+0000 7f2c8a9ba640 1 --2- 172.21.15.78:0/1210141052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922eab880 0x55a922f1f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:52:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.460+0000 7f2c899b8640 1 --2- 172.21.15.78:0/1210141052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922ea0fc0 0x55a922eb20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.460+0000 7f2c8a1b9640 1 --2- 172.21.15.78:0/1210141052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea06a0 0x55a922eb1b60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009870 tx=0x7f2c740098a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2914d84b0ab5941e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.460+0000 7f2c69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-18T23:52:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.461+0000 7f2c69ffb640 1 -- 172.21.15.78:0/1210141052 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74030030 con 0x55a922ea06a0 2024-08-18T23:52:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.461+0000 7f2c8e81b740 1 -- 172.21.15.78:0/1210141052 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c7400ade0 con 0x55a922ea06a0 2024-08-18T23:52:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.461+0000 7f2c69ffb640 1 -- 172.21.15.78:0/1210141052 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2c74012070 con 0x55a922ea06a0 2024-08-18T23:52:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.461+0000 7f2c69ffb640 1 -- 172.21.15.78:0/1210141052 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2c7403f050 con 0x55a922ea06a0 2024-08-18T23:52:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.461+0000 7f2c8e81b740 10 client.26116 fetch_fsmap learned FSMap version 32 2024-08-18T23:52:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.461+0000 7f2c8e81b740 1 -- 172.21.15.78:0/1210141052 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a922f269a0 con 0x55a922ea06a0 2024-08-18T23:52:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.461+0000 7f2c69ffb640 1 -- 172.21.15.78:0/1210141052 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7400ade0 con 0x55a922ea06a0 2024-08-18T23:52:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.461+0000 7f2c69ffb640 1 -- 172.21.15.78:0/1210141052 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f2c74015c90 con 0x55a922ea06a0 2024-08-18T23:52:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.461+0000 7f2c8e81b740 10 client.26116 fetch_fsmap finished waiting for FSMap version 32 2024-08-18T23:52:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.461+0000 7f2c8e81b740 10 client.26116 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:52:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.461+0000 7f2c8e81b740 20 client.26116 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.461+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/1210141052 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a922ebb300 0x55a922f28e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.461+0000 7f2c8e81b740 4 client.26116 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-08-18T23:52:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.461+0000 7f2c8e81b740 1 -- 172.21.15.78:0/1210141052 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55a922e0d380 con 0x55a922ebb300 2024-08-18T23:52:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.462+0000 7f2c899b8640 1 --2- 172.21.15.78:0/1210141052 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a922ebb300 0x55a922f28e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.462+0000 7f2c69ffb640 10 client.26116 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.462+0000 7f2c899b8640 1 --2- 172.21.15.78:0/1210141052 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a922ebb300 0x55a922f28e90 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8168a879f6792fe7 server_cookie=7dd32c896ed7058e in_seq=0 out_seq=0 2024-08-18T23:52:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.462+0000 7f2c69ffb640 1 -- 172.21.15.78:0/1210141052 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55a922e0d380 con 0x55a922ebb300 2024-08-18T23:52:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.462+0000 7f2c69ffb640 10 client.26116 handle_command_reply: tid=0 2024-08-18T23:52:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.462+0000 7f2c8e81b740 1 client.26116 shutdown 2024-08-18T23:52:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.463+0000 7f2c8e81b740 1 -- 172.21.15.78:0/1210141052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea06a0 msgr2=0x55a922eb1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.463+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/1210141052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea06a0 0x55a922eb1b60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009870 tx=0x7f2c740098a0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.463+0000 7f2c8e81b740 1 -- 172.21.15.78:0/1210141052 shutdown_connections 2024-08-18T23:52:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.463+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/1210141052 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55a922ebb300 0x55a922f28e90 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.463+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/1210141052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a922eab880 0x55a922f1f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.463+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/1210141052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a922ea0fc0 0x55a922eb20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.463+0000 7f2c69ffb640 0 client.26116 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.463+0000 7f2c8e81b740 1 --2- 172.21.15.78:0/1210141052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a922ea06a0 0x55a922eb1b60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.463+0000 7f2c8e81b740 1 -- 172.21.15.78:0/1210141052 >> 172.21.15.78:0/1210141052 conn(0x55a922e9cc30 msgr2=0x55a922e9b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.463+0000 7f2c8e81b740 1 -- 172.21.15.78:0/1210141052 shutdown_connections 2024-08-18T23:52:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.463+0000 7f2c8e81b740 1 -- 172.21.15.78:0/1210141052 wait complete. 2024-08-18T23:52:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.463+0000 7f2c8e81b740 20 client.26116 trim_cache size 0 max 16384 2024-08-18T23:52:57.480 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:52:57.480 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 32, 2024-08-18T23:52:57.480 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24452, 2024-08-18T23:52:57.480 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 28, 2024-08-18T23:52:57.480 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": {} 2024-08-18T23:52:57.480 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:52:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.464+0000 7f2c8c41d640 1 -- 172.21.15.78:0/2694403670 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2c6c0a2c00 msgr2=0x7f2c6c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.464+0000 7f2c8c41d640 1 --2- 172.21.15.78:0/2694403670 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2c6c0a2c00 0x7f2c6c0a50c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00b940 tx=0x7f2c7c0047d0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.464+0000 7f2c8c41d640 1 -- 172.21.15.78:0/2694403670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c841022a0 msgr2=0x7f2c84198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.464+0000 7f2c8c41d640 1 --2- 172.21.15.78:0/2694403670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c841022a0 0x7f2c84198910 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2c78006010 tx=0x7f2c78003c80 comp rx=0 tx=0).stop 2024-08-18T23:52:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.464+0000 7f2c8c41d640 1 -- 172.21.15.78:0/2694403670 shutdown_connections 2024-08-18T23:52:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.464+0000 7f2c8c41d640 1 --2- 172.21.15.78:0/2694403670 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2c6c0a2c00 0x7f2c6c0a50c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.464+0000 7f2c8c41d640 1 --2- 172.21.15.78:0/2694403670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c84102c20 0x7f2c8419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.464+0000 7f2c8c41d640 1 --2- 172.21.15.78:0/2694403670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c841022a0 0x7f2c84198910 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.464+0000 7f2c8c41d640 1 --2- 172.21.15.78:0/2694403670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c84100b50 0x7f2c841983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.464+0000 7f2c8c41d640 1 -- 172.21.15.78:0/2694403670 >> 172.21.15.78:0/2694403670 conn(0x7f2c840fc500 msgr2=0x7f2c8406a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.464+0000 7f2c8c41d640 1 -- 172.21.15.78:0/2694403670 shutdown_connections 2024-08-18T23:52:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:57.464+0000 7f2c8c41d640 1 -- 172.21.15.78:0/2694403670 wait complete. 2024-08-18T23:52:57.689 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:57 smithi161 ceph-mon[22027]: pgmap v609: 129 pgs: 129 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-08-18T23:52:57.745 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:52:57.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:57 smithi123 ceph-mon[21808]: pgmap v609: 129 pgs: 129 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-08-18T23:52:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.973+0000 7f2d3d86a640 1 -- 172.21.15.78:0/3699069651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d381018c0 msgr2=0x7f2d38101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.973+0000 7f2d3d86a640 1 --2- 172.21.15.78:0/3699069651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d381018c0 0x7f2d38101d20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009930 tx=0x7f2d2c02a420 comp rx=0 tx=0).stop 2024-08-18T23:52:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.974+0000 7f2d3d86a640 1 -- 172.21.15.78:0/3699069651 shutdown_connections 2024-08-18T23:52:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.974+0000 7f2d3d86a640 1 --2- 172.21.15.78:0/3699069651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d38102260 0x7f2d3810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.974+0000 7f2d3d86a640 1 --2- 172.21.15.78:0/3699069651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d381018c0 0x7f2d38101d20 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.974+0000 7f2d3d86a640 1 --2- 172.21.15.78:0/3699069651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d381078c0 0x7f2d38107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.974+0000 7f2d3d86a640 1 -- 172.21.15.78:0/3699069651 >> 172.21.15.78:0/3699069651 conn(0x7f2d380fd620 msgr2=0x7f2d380ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:58.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.974+0000 7f2d3d86a640 1 -- 172.21.15.78:0/3699069651 shutdown_connections 2024-08-18T23:52:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.974+0000 7f2d3d86a640 1 -- 172.21.15.78:0/3699069651 wait complete. 2024-08-18T23:52:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.974+0000 7f2d3d86a640 1 Processor -- start 2024-08-18T23:52:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.974+0000 7f2d3d86a640 1 -- start start 2024-08-18T23:52:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d3d86a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d381018c0 0x7f2d381981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d3d86a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d38102260 0x7f2d38198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d3d86a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d381078c0 0x7f2d3819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d3d86a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2d38111c80 con 0x7f2d381018c0 2024-08-18T23:52:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d3d86a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d38111b00 con 0x7f2d381078c0 2024-08-18T23:52:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d3d86a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2d38111e00 con 0x7f2d38102260 2024-08-18T23:52:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d36ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d381018c0 0x7f2d381981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d36ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d381018c0 0x7f2d381981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39954/0 (socket says 172.21.15.78:39954) 2024-08-18T23:52:58.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d36ffd640 1 -- 172.21.15.78:0/1495355242 learned_addr learned my addr 172.21.15.78:0/1495355242 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d367fc640 1 --2- 172.21.15.78:0/1495355242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d38102260 0x7f2d38198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d377fe640 1 --2- 172.21.15.78:0/1495355242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d381078c0 0x7f2d3819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d36ffd640 1 -- 172.21.15.78:0/1495355242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d38102260 msgr2=0x7f2d38198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d36ffd640 1 --2- 172.21.15.78:0/1495355242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d38102260 0x7f2d38198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d36ffd640 1 -- 172.21.15.78:0/1495355242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d381078c0 msgr2=0x7f2d3819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d36ffd640 1 --2- 172.21.15.78:0/1495355242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d381078c0 0x7f2d3819cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d36ffd640 1 -- 172.21.15.78:0/1495355242 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3819d190 con 0x7f2d381018c0 2024-08-18T23:52:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d36ffd640 1 --2- 172.21.15.78:0/1495355242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d381018c0 0x7f2d381981e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800eab0 tx=0x7f2d2800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d3c868640 1 -- 172.21.15.78:0/1495355242 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2800ce40 con 0x7f2d381018c0 2024-08-18T23:52:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d3d86a640 1 -- 172.21.15.78:0/1495355242 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d3819d480 con 0x7f2d381018c0 2024-08-18T23:52:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d3c868640 1 -- 172.21.15.78:0/1495355242 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d28011950 con 0x7f2d381018c0 2024-08-18T23:52:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d3c868640 1 -- 172.21.15.78:0/1495355242 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28010ae0 con 0x7f2d381018c0 2024-08-18T23:52:58.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.975+0000 7f2d3d86a640 1 -- 172.21.15.78:0/1495355242 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d381a4d60 con 0x7f2d381018c0 2024-08-18T23:52:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.978+0000 7f2d3c868640 1 -- 172.21.15.78:0/1495355242 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2d28011b00 con 0x7f2d381018c0 2024-08-18T23:52:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.978+0000 7f2d3fc68740 1 Processor -- start 2024-08-18T23:52:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.978+0000 7f2d3fc68740 1 -- start start 2024-08-18T23:52:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d3fc68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c01583b2c0 0x55c01583b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d3fc68740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583bbe0 0x55c015845f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d3fc68740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0158464a0 0x55c015848890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d3fc68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c0157a0810 con 0x55c01583b2c0 2024-08-18T23:52:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d3fc68740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c015747c70 con 0x55c01583bbe0 2024-08-18T23:52:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d3fc68740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c01579c7d0 con 0x55c0158464a0 2024-08-18T23:52:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d377fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c01583b2c0 0x55c01583b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d377fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c01583b2c0 0x55c01583b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39968/0 (socket says 172.21.15.78:39968) 2024-08-18T23:52:58.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d377fe640 1 -- 172.21.15.78:0/271133135 learned_addr learned my addr 172.21.15.78:0/271133135 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d367fc640 1 --2- 172.21.15.78:0/271133135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0158464a0 0x55c015848890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d3c868640 1 --2- 172.21.15.78:0/1495355242 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2d200a2b80 0x7f2d200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d36ffd640 1 --2- 172.21.15.78:0/271133135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583bbe0 0x55c015845f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d377fe640 1 -- 172.21.15.78:0/271133135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0158464a0 msgr2=0x55c015848890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d377fe640 1 --2- 172.21.15.78:0/271133135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0158464a0 0x55c015848890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d377fe640 1 -- 172.21.15.78:0/271133135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583bbe0 msgr2=0x55c015845f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d377fe640 1 --2- 172.21.15.78:0/271133135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583bbe0 0x55c015845f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d377fe640 1 -- 172.21.15.78:0/271133135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c015512ba0 con 0x55c01583b2c0 2024-08-18T23:52:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d36ffd640 1 --2- 172.21.15.78:0/271133135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583bbe0 0x55c015845f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d377fe640 1 --2- 172.21.15.78:0/1495355242 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2d200a2b80 0x7f2d200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d3c868640 1 -- 172.21.15.78:0/1495355242 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2d280cf980 con 0x7f2d381018c0 2024-08-18T23:52:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.979+0000 7f2d377fe640 1 --2- 172.21.15.78:0/271133135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c01583b2c0 0x55c01583b6a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000b880 tx=0x7f2d3000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9606fb03f07a0097 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:58.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.980+0000 7f2d1dffb640 1 -- 172.21.15.78:0/271133135 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30024070 con 0x55c01583b2c0 2024-08-18T23:52:58.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.980+0000 7f2d1dffb640 1 -- 172.21.15.78:0/271133135 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d30003b70 con 0x55c01583b2c0 2024-08-18T23:52:58.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.980+0000 7f2d1dffb640 1 -- 172.21.15.78:0/271133135 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30011860 con 0x55c01583b2c0 2024-08-18T23:52:58.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.980+0000 7f2d3fc68740 1 -- 172.21.15.78:0/271133135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c01583b2c0 msgr2=0x55c01583b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:58.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.980+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/271133135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c01583b2c0 0x55c01583b6a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000b880 tx=0x7f2d3000bc80 comp rx=0 tx=0).stop 2024-08-18T23:52:58.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.980+0000 7f2d377fe640 1 --2- 172.21.15.78:0/1495355242 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2d200a2b80 0x7f2d200a5040 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2d30021000 tx=0x7f2d3001e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:58.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.980+0000 7f2d3fc68740 1 -- 172.21.15.78:0/271133135 shutdown_connections 2024-08-18T23:52:58.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.980+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/271133135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0158464a0 0x55c015848890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:58.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.980+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/271133135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583bbe0 0x55c015845f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:58.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.980+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/271133135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c01583b2c0 0x55c01583b6a0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:58.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.980+0000 7f2d3fc68740 1 -- 172.21.15.78:0/271133135 >> 172.21.15.78:0/271133135 conn(0x55c015837850 msgr2=0x55c015837c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:58.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.980+0000 7f2d3fc68740 1 -- 172.21.15.78:0/271133135 shutdown_connections 2024-08-18T23:52:58.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.980+0000 7f2d3fc68740 1 -- 172.21.15.78:0/271133135 wait complete. 2024-08-18T23:52:58.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.981+0000 7f2d3fc68740 1 -- start start 2024-08-18T23:52:58.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.981+0000 7f2d3fc68740 1 Processor -- start 2024-08-18T23:52:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.981+0000 7f2d3fc68740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583b2c0 0x55c015843510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.981+0000 7f2d3fc68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c01583bbe0 0x55c015843a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.981+0000 7f2d3fc68740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0158464a0 0x55c015843f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.981+0000 7f2d3fc68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c015848fb0 con 0x55c01583bbe0 2024-08-18T23:52:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.981+0000 7f2d3fc68740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c01575b9a0 con 0x55c01583b2c0 2024-08-18T23:52:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.981+0000 7f2d3fc68740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c015849130 con 0x55c0158464a0 2024-08-18T23:52:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.981+0000 7f2d3fc68740 10 client.0 _ll_register_callbacks cb 0x55c015749c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.981+0000 7f2d367fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c01583bbe0 0x55c015843a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.981+0000 7f2d367fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c01583bbe0 0x55c015843a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39976/0 (socket says 172.21.15.78:39976) 2024-08-18T23:52:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.981+0000 7f2d367fc640 1 -- 172.21.15.78:0/2830842576 learned_addr learned my addr 172.21.15.78:0/2830842576 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.982+0000 7f2d377fe640 1 --2- 172.21.15.78:0/2830842576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0158464a0 0x55c015843f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.982+0000 7f2d36ffd640 1 --2- 172.21.15.78:0/2830842576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583b2c0 0x55c015843510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:58.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.982+0000 7f2d367fc640 1 -- 172.21.15.78:0/2830842576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0158464a0 msgr2=0x55c015843f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.982+0000 7f2d367fc640 1 --2- 172.21.15.78:0/2830842576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0158464a0 0x55c015843f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.982+0000 7f2d367fc640 1 -- 172.21.15.78:0/2830842576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583b2c0 msgr2=0x55c015843510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.982+0000 7f2d367fc640 1 --2- 172.21.15.78:0/2830842576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583b2c0 0x55c015843510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.982+0000 7f2d367fc640 1 -- 172.21.15.78:0/2830842576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c015767220 con 0x55c01583bbe0 2024-08-18T23:52:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.982+0000 7f2d36ffd640 1 --2- 172.21.15.78:0/2830842576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583b2c0 0x55c015843510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:52:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.982+0000 7f2d367fc640 1 --2- 172.21.15.78:0/2830842576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c01583bbe0 0x55c015843a50 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c000c00 tx=0x7f2d2c031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d076aca0fca8b68 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.982+0000 7f2d16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.982+0000 7f2d16ffd640 1 -- 172.21.15.78:0/2830842576 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c035040 con 0x55c01583bbe0 2024-08-18T23:52:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.982+0000 7f2d16ffd640 1 -- 172.21.15.78:0/2830842576 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2d2c004240 con 0x55c01583bbe0 2024-08-18T23:52:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.982+0000 7f2d3fc68740 1 -- 172.21.15.78:0/2830842576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d30003b70 con 0x55c01583bbe0 2024-08-18T23:52:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.982+0000 7f2d16ffd640 1 -- 172.21.15.78:0/2830842576 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2d2c004440 con 0x55c01583bbe0 2024-08-18T23:52:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.982+0000 7f2d3fc68740 10 client.17016 fetch_fsmap learned FSMap version 32 2024-08-18T23:52:58.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.982+0000 7f2d3fc68740 1 -- 172.21.15.78:0/2830842576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c015854cd0 con 0x55c01583bbe0 2024-08-18T23:52:58.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.982+0000 7f2d16ffd640 1 -- 172.21.15.78:0/2830842576 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d30003b70 con 0x55c01583bbe0 2024-08-18T23:52:58.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.983+0000 7f2d16ffd640 1 -- 172.21.15.78:0/2830842576 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f2d2c0035f0 con 0x55c01583bbe0 2024-08-18T23:52:58.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.983+0000 7f2d3fc68740 10 client.17016 fetch_fsmap finished waiting for FSMap version 32 2024-08-18T23:52:58.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.983+0000 7f2d3fc68740 10 client.17016 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:52:58.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.983+0000 7f2d3fc68740 20 client.17016 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:58.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.983+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/2830842576 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55c015856190 0x55c015858610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:58.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.983+0000 7f2d3fc68740 4 client.17016 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:52:58.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.983+0000 7f2d3fc68740 1 -- 172.21.15.78:0/2830842576 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0157adf40 con 0x55c015856190 2024-08-18T23:52:58.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.983+0000 7f2d36ffd640 1 --2- 172.21.15.78:0/2830842576 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55c015856190 0x55c015858610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:58.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.983+0000 7f2d16ffd640 10 client.17016 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:58.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.983+0000 7f2d36ffd640 1 --2- 172.21.15.78:0/2830842576 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55c015856190 0x55c015858610 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef9b3898b0d82d7e server_cookie=4115c1e6728407d7 in_seq=0 out_seq=0 2024-08-18T23:52:58.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.984+0000 7f2d16ffd640 1 -- 172.21.15.78:0/2830842576 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0157adf40 con 0x55c015856190 2024-08-18T23:52:58.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.984+0000 7f2d16ffd640 10 client.17016 handle_command_reply: tid=0 2024-08-18T23:52:58.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.984+0000 7f2d3fc68740 1 client.17016 shutdown 2024-08-18T23:52:58.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.984+0000 7f2d3fc68740 1 -- 172.21.15.78:0/2830842576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c01583bbe0 msgr2=0x55c015843a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:58.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.984+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/2830842576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c01583bbe0 0x55c015843a50 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c000c00 tx=0x7f2d2c031920 comp rx=0 tx=0).stop 2024-08-18T23:52:58.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.984+0000 7f2d3fc68740 1 -- 172.21.15.78:0/2830842576 shutdown_connections 2024-08-18T23:52:58.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.984+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/2830842576 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55c015856190 0x55c015858610 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:58.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.984+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/2830842576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c0158464a0 0x55c015843f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:58.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.984+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/2830842576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c01583bbe0 0x55c015843a50 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:58.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.984+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/2830842576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583b2c0 0x55c015843510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:58.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.984+0000 7f2d3fc68740 1 -- 172.21.15.78:0/2830842576 >> 172.21.15.78:0/2830842576 conn(0x55c015837850 msgr2=0x55c015836b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:58.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.984+0000 7f2d16ffd640 0 client.17016 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:58.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.984+0000 7f2d3fc68740 1 -- 172.21.15.78:0/2830842576 shutdown_connections 2024-08-18T23:52:58.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.984+0000 7f2d3fc68740 1 -- 172.21.15.78:0/2830842576 wait complete. 2024-08-18T23:52:58.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:58.985+0000 7f2d3fc68740 20 client.17016 trim_cache size 0 max 16384 2024-08-18T23:52:59.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.002+0000 7f2d3fc68740 1 Processor -- start 2024-08-18T23:52:59.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.002+0000 7f2d3fc68740 1 -- start start 2024-08-18T23:52:59.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.002+0000 7f2d3fc68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c015837850 0x55c01585dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:59.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.002+0000 7f2d3fc68740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c01583b2c0 0x55c01585e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:59.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.002+0000 7f2d3fc68740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583bbe0 0x55c015840770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:59.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.002+0000 7f2d3fc68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381b1320 con 0x55c015837850 2024-08-18T23:52:59.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.002+0000 7f2d3fc68740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381aeb90 con 0x55c01583bbe0 2024-08-18T23:52:59.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.002+0000 7f2d3fc68740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381ac9e0 con 0x55c01583b2c0 2024-08-18T23:52:59.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.002+0000 7f2d36ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c015837850 0x55c01585dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:59.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.002+0000 7f2d36ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c015837850 0x55c01585dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39988/0 (socket says 172.21.15.78:39988) 2024-08-18T23:52:59.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.002+0000 7f2d36ffd640 1 -- 172.21.15.78:0/404297948 learned_addr learned my addr 172.21.15.78:0/404297948 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:59.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.002+0000 7f2d377fe640 1 --2- 172.21.15.78:0/404297948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583bbe0 0x55c015840770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:59.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.002+0000 7f2d367fc640 1 --2- 172.21.15.78:0/404297948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c01583b2c0 0x55c01585e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:59.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.003+0000 7f2d36ffd640 1 -- 172.21.15.78:0/404297948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c01583b2c0 msgr2=0x55c01585e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:59.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.003+0000 7f2d36ffd640 1 --2- 172.21.15.78:0/404297948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c01583b2c0 0x55c01585e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:59.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.003+0000 7f2d36ffd640 1 -- 172.21.15.78:0/404297948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583bbe0 msgr2=0x55c015840770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:59.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.003+0000 7f2d36ffd640 1 --2- 172.21.15.78:0/404297948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583bbe0 0x55c015840770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:59.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.003+0000 7f2d36ffd640 1 -- 172.21.15.78:0/404297948 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0158384e0 con 0x55c015837850 2024-08-18T23:52:59.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.003+0000 7f2d367fc640 1 --2- 172.21.15.78:0/404297948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c01583b2c0 0x55c01585e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:59.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.003+0000 7f2d36ffd640 1 --2- 172.21.15.78:0/404297948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c015837850 0x55c01585dee0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2d280cfd30 tx=0x7f2d280094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa5909cfedfec56 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:59.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.003+0000 7f2d1d7fa640 1 -- 172.21.15.78:0/404297948 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2809be50 con 0x55c015837850 2024-08-18T23:52:59.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.003+0000 7f2d1d7fa640 1 -- 172.21.15.78:0/404297948 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d2809e040 con 0x55c015837850 2024-08-18T23:52:59.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.003+0000 7f2d1d7fa640 1 -- 172.21.15.78:0/404297948 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2809c0b0 con 0x55c015837850 2024-08-18T23:52:59.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.004+0000 7f2d3fc68740 1 -- 172.21.15.78:0/404297948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c015837850 msgr2=0x55c01585dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.004+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/404297948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c015837850 0x55c01585dee0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2d280cfd30 tx=0x7f2d280094a0 comp rx=0 tx=0).stop 2024-08-18T23:52:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.004+0000 7f2d3fc68740 1 -- 172.21.15.78:0/404297948 shutdown_connections 2024-08-18T23:52:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.004+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/404297948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583bbe0 0x55c015840770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.004+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/404297948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c01583b2c0 0x55c01585e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.004+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/404297948 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c015837850 0x55c01585dee0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.004+0000 7f2d3fc68740 1 -- 172.21.15.78:0/404297948 >> 172.21.15.78:0/404297948 conn(0x55c01583ef00 msgr2=0x55c015836b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.004+0000 7f2d3fc68740 1 -- 172.21.15.78:0/404297948 shutdown_connections 2024-08-18T23:52:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.004+0000 7f2d3fc68740 1 -- 172.21.15.78:0/404297948 wait complete. 2024-08-18T23:52:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.004+0000 7f2d3fc68740 1 -- start start 2024-08-18T23:52:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.004+0000 7f2d3fc68740 1 Processor -- start 2024-08-18T23:52:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.005+0000 7f2d3fc68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c015837850 0x55c0158bc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.005+0000 7f2d3fc68740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583b2c0 0x55c0158bccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.005+0000 7f2d3fc68740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c01583bbe0 0x55c0158bad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:59.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.005+0000 7f2d3fc68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381ad050 con 0x55c015837850 2024-08-18T23:52:59.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.005+0000 7f2d3fc68740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381ae4c0 con 0x55c01583b2c0 2024-08-18T23:52:59.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.005+0000 7f2d3fc68740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381ad720 con 0x55c01583bbe0 2024-08-18T23:52:59.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.005+0000 7f2d3fc68740 10 client.0 _ll_register_callbacks cb 0x7f2d2c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:52:59.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.005+0000 7f2d36ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c015837850 0x55c0158bc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:59.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.005+0000 7f2d36ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c015837850 0x55c0158bc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39996/0 (socket says 172.21.15.78:39996) 2024-08-18T23:52:59.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.005+0000 7f2d36ffd640 1 -- 172.21.15.78:0/3758354404 learned_addr learned my addr 172.21.15.78:0/3758354404 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:52:59.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.005+0000 7f2d367fc640 1 --2- 172.21.15.78:0/3758354404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583b2c0 0x55c0158bccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:59.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.005+0000 7f2d36ffd640 1 -- 172.21.15.78:0/3758354404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c01583bbe0 msgr2=0x55c0158bad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:59.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.005+0000 7f2d36ffd640 1 --2- 172.21.15.78:0/3758354404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c01583bbe0 0x55c0158bad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:59.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.005+0000 7f2d36ffd640 1 -- 172.21.15.78:0/3758354404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583b2c0 msgr2=0x55c0158bccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:52:59.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.005+0000 7f2d36ffd640 1 --2- 172.21.15.78:0/3758354404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583b2c0 0x55c0158bccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:59.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.005+0000 7f2d36ffd640 1 -- 172.21.15.78:0/3758354404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0158bb580 con 0x55c015837850 2024-08-18T23:52:59.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.005+0000 7f2d367fc640 1 --2- 172.21.15.78:0/3758354404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583b2c0 0x55c0158bccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:52:59.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.005+0000 7f2d36ffd640 1 --2- 172.21.15.78:0/3758354404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c015837850 0x55c0158bc780 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800c700 tx=0x7f2d280a0200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cb6cf29b8ff9c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:52:59.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.005+0000 7f2d16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:52:59.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.006+0000 7f2d16ffd640 1 -- 172.21.15.78:0/3758354404 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d280b9020 con 0x55c015837850 2024-08-18T23:52:59.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.006+0000 7f2d16ffd640 1 -- 172.21.15.78:0/3758354404 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2d2809c0b0 con 0x55c015837850 2024-08-18T23:52:59.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.006+0000 7f2d3fc68740 1 -- 172.21.15.78:0/3758354404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d2809e040 con 0x55c015837850 2024-08-18T23:52:59.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.006+0000 7f2d16ffd640 1 -- 172.21.15.78:0/3758354404 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2d28099ef0 con 0x55c015837850 2024-08-18T23:52:59.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.006+0000 7f2d3fc68740 10 client.17022 fetch_fsmap learned FSMap version 32 2024-08-18T23:52:59.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.006+0000 7f2d3fc68740 1 -- 172.21.15.78:0/3758354404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c0158ca6d0 con 0x55c015837850 2024-08-18T23:52:59.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.006+0000 7f2d16ffd640 1 -- 172.21.15.78:0/3758354404 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d2809e040 con 0x55c015837850 2024-08-18T23:52:59.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.006+0000 7f2d16ffd640 1 -- 172.21.15.78:0/3758354404 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f2d2800f040 con 0x55c015837850 2024-08-18T23:52:59.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.006+0000 7f2d3fc68740 10 client.17022 fetch_fsmap finished waiting for FSMap version 32 2024-08-18T23:52:59.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.006+0000 7f2d3fc68740 10 client.17022 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-18T23:52:59.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.006+0000 7f2d3fc68740 20 client.17022 populate_metadata read hostname 'smithi078' 2024-08-18T23:52:59.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.006+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/3758354404 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55c0158464a0 0x55c0158cce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:52:59.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.006+0000 7f2d3fc68740 4 client.17022 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.75, "await_for": 45.0}] 2024-08-18T23:52:59.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.006+0000 7f2d3fc68740 1 -- 172.21.15.78:0/3758354404 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.75, "await_for": 45.0}) -- 0x55c0157a8290 con 0x55c0158464a0 2024-08-18T23:52:59.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.006+0000 7f2d367fc640 1 --2- 172.21.15.78:0/3758354404 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55c0158464a0 0x55c0158cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:52:59.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.007+0000 7f2d16ffd640 10 client.17022 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:59.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.007+0000 7f2d367fc640 1 --2- 172.21.15.78:0/3758354404 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55c0158464a0 0x55c0158cce10 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97a355ec6a3142f3 server_cookie=57bef41294196573 in_seq=0 out_seq=0 2024-08-18T23:52:59.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.016+0000 7f2d16ffd640 1 -- 172.21.15.78:0/3758354404 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c0157a8290 con 0x55c0158464a0 2024-08-18T23:52:59.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.016+0000 7f2d16ffd640 10 client.17022 handle_command_reply: tid=0 2024-08-18T23:52:59.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.016+0000 7f2d3fc68740 1 client.17022 shutdown 2024-08-18T23:52:59.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.016+0000 7f2d3fc68740 1 -- 172.21.15.78:0/3758354404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c015837850 msgr2=0x55c0158bc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:59.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.016+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/3758354404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c015837850 0x55c0158bc780 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800c700 tx=0x7f2d280a0200 comp rx=0 tx=0).stop 2024-08-18T23:52:59.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.016+0000 7f2d3fc68740 1 -- 172.21.15.78:0/3758354404 shutdown_connections 2024-08-18T23:52:59.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.016+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/3758354404 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55c0158464a0 0x55c0158cce10 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:59.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.016+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/3758354404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c01583bbe0 0x55c0158bad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:59.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.016+0000 7f2d16ffd640 0 client.17022 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:52:59.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.016+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/3758354404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c01583b2c0 0x55c0158bccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:59.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.016+0000 7f2d3fc68740 1 --2- 172.21.15.78:0/3758354404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c015837850 0x55c0158bc780 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:59.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.016+0000 7f2d3fc68740 1 -- 172.21.15.78:0/3758354404 >> 172.21.15.78:0/3758354404 conn(0x55c01583ef00 msgr2=0x55c01583cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:59.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.016+0000 7f2d3fc68740 1 -- 172.21.15.78:0/3758354404 shutdown_connections 2024-08-18T23:52:59.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.016+0000 7f2d3fc68740 1 -- 172.21.15.78:0/3758354404 wait complete. 2024-08-18T23:52:59.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.016+0000 7f2d3fc68740 20 client.17022 trim_cache size 0 max 16384 2024-08-18T23:52:59.019 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:52:59.019 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 32, 2024-08-18T23:52:59.019 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24452, 2024-08-18T23:52:59.019 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 30, 2024-08-18T23:52:59.019 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-08-18T23:52:59.019 INFO:teuthology.orchestra.run.smithi078.stdout: "788a429f": { 2024-08-18T23:52:59.019 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 30, 2024-08-18T23:52:59.019 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-08-18T23:52:59.019 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:52:59.019 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-08-18T23:52:59.019 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:52:59.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:52:59.020 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-08-18T23:52:59.020 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 143.8, 2024-08-18T23:52:59.020 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-08-18T23:52:59.020 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-08-18T23:52:59.020 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-08-18T23:52:59.021 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-08-18T23:52:59.021 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-08-18T23:52:59.021 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-08-18T23:52:59.021 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:52:59.021 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:52:59.021 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:52:59.021 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:52:59.021 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:52:59.021 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:52:59.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.017+0000 7f2d3d86a640 1 -- 172.21.15.78:0/1495355242 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2d200a2b80 msgr2=0x7f2d200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:59.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.017+0000 7f2d3d86a640 1 --2- 172.21.15.78:0/1495355242 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2d200a2b80 0x7f2d200a5040 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2d30021000 tx=0x7f2d3001e000 comp rx=0 tx=0).stop 2024-08-18T23:52:59.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.017+0000 7f2d3d86a640 1 -- 172.21.15.78:0/1495355242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d381018c0 msgr2=0x7f2d381981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:52:59.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.017+0000 7f2d3d86a640 1 --2- 172.21.15.78:0/1495355242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d381018c0 0x7f2d381981e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800eab0 tx=0x7f2d2800ef70 comp rx=0 tx=0).stop 2024-08-18T23:52:59.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.017+0000 7f2d3d86a640 1 -- 172.21.15.78:0/1495355242 shutdown_connections 2024-08-18T23:52:59.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.017+0000 7f2d3d86a640 1 --2- 172.21.15.78:0/1495355242 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2d200a2b80 0x7f2d200a5040 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:59.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.017+0000 7f2d3d86a640 1 --2- 172.21.15.78:0/1495355242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d381078c0 0x7f2d3819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:59.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.017+0000 7f2d3d86a640 1 --2- 172.21.15.78:0/1495355242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d38102260 0x7f2d38198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:59.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.017+0000 7f2d3d86a640 1 --2- 172.21.15.78:0/1495355242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d381018c0 0x7f2d381981e0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:52:59.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.017+0000 7f2d3d86a640 1 -- 172.21.15.78:0/1495355242 >> 172.21.15.78:0/1495355242 conn(0x7f2d380fd620 msgr2=0x7f2d380fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:52:59.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.017+0000 7f2d3d86a640 1 -- 172.21.15.78:0/1495355242 shutdown_connections 2024-08-18T23:52:59.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:52:59.017+0000 7f2d3d86a640 1 -- 172.21.15.78:0/1495355242 wait complete. 2024-08-18T23:52:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:52:59 smithi161 ceph-mon[22027]: pgmap v610: 129 pgs: 129 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-08-18T23:52:59.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:52:59 smithi078 ceph-mon[18440]: pgmap v610: 129 pgs: 129 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-08-18T23:52:59.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:52:59 smithi123 ceph-mon[21808]: pgmap v610: 129 pgs: 129 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-08-18T23:52:59.869 INFO:tasks.quiescer:done joining Quiescers 2024-08-18T23:52:59.869 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-08-18T23:52:59.880 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-08-18T23:53:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:01 smithi161 ceph-mon[22027]: pgmap v611: 129 pgs: 129 active+clean; 244 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-08-18T23:53:01.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:01 smithi078 ceph-mon[18440]: pgmap v611: 129 pgs: 129 active+clean; 244 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-08-18T23:53:01.797 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:01 smithi123 ceph-mon[21808]: pgmap v611: 129 pgs: 129 active+clean; 244 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 46 KiB/s wr, 24 op/s 2024-08-18T23:53:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:03 smithi161 ceph-mon[22027]: pgmap v612: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-08-18T23:53:03.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:03 smithi078 ceph-mon[18440]: pgmap v612: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-08-18T23:53:03.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:03 smithi123 ceph-mon[21808]: pgmap v612: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-08-18T23:53:05.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:05 smithi161 ceph-mon[22027]: pgmap v613: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 26 op/s 2024-08-18T23:53:05.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:05 smithi078 ceph-mon[18440]: pgmap v613: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 26 op/s 2024-08-18T23:53:05.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:05 smithi123 ceph-mon[21808]: pgmap v613: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 26 op/s 2024-08-18T23:53:07.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:07 smithi161 ceph-mon[22027]: pgmap v614: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 KiB/s wr, 25 op/s 2024-08-18T23:53:07.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:07 smithi078 ceph-mon[18440]: pgmap v614: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 KiB/s wr, 25 op/s 2024-08-18T23:53:07.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:07 smithi123 ceph-mon[21808]: pgmap v614: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 KiB/s wr, 25 op/s 2024-08-18T23:53:09.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:09 smithi161 ceph-mon[22027]: pgmap v615: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.3 KiB/s wr, 24 op/s 2024-08-18T23:53:09.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:09 smithi078 ceph-mon[18440]: pgmap v615: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.3 KiB/s wr, 24 op/s 2024-08-18T23:53:09.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:09 smithi123 ceph-mon[21808]: pgmap v615: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.3 KiB/s wr, 24 op/s 2024-08-18T23:53:11.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:11 smithi161 ceph-mon[22027]: pgmap v616: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.3 KiB/s wr, 26 op/s 2024-08-18T23:53:11.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:11 smithi078 ceph-mon[18440]: pgmap v616: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.3 KiB/s wr, 26 op/s 2024-08-18T23:53:11.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:11 smithi123 ceph-mon[21808]: pgmap v616: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.3 KiB/s wr, 26 op/s 2024-08-18T23:53:13.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:13 smithi161 ceph-mon[22027]: pgmap v617: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 16 op/s 2024-08-18T23:53:13.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:13 smithi078 ceph-mon[18440]: pgmap v617: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 16 op/s 2024-08-18T23:53:13.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:13 smithi123 ceph-mon[21808]: pgmap v617: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 16 op/s 2024-08-18T23:53:15.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:15 smithi161 ceph-mon[22027]: pgmap v618: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-08-18T23:53:15.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:15 smithi078 ceph-mon[18440]: pgmap v618: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-08-18T23:53:15.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:15 smithi123 ceph-mon[21808]: pgmap v618: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-08-18T23:53:17.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:17 smithi078 ceph-mon[18440]: pgmap v619: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-08-18T23:53:17.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:17 smithi123 ceph-mon[21808]: pgmap v619: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-08-18T23:53:17.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:17 smithi161 ceph-mon[22027]: pgmap v619: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 6 op/s 2024-08-18T23:53:19.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:19 smithi078 ceph-mon[18440]: pgmap v620: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-18T23:53:19.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:19 smithi123 ceph-mon[21808]: pgmap v620: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-18T23:53:19.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:19 smithi161 ceph-mon[22027]: pgmap v620: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-18T23:53:21.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:21 smithi078 ceph-mon[18440]: pgmap v621: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-08-18T23:53:21.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:21 smithi123 ceph-mon[21808]: pgmap v621: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-08-18T23:53:21.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:21 smithi161 ceph-mon[22027]: pgmap v621: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-08-18T23:53:22.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:22 smithi123 ceph-mon[21808]: pgmap v622: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-18T23:53:22.828 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-18T23:53:22.859 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:22 smithi078 ceph-mon[18440]: pgmap v622: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-18T23:53:22.927 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:22 smithi161 ceph-mon[22027]: pgmap v622: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-18T23:53:23.105 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:53:23.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.599+0000 7f1069394640 1 -- 172.21.15.78:0/209092352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1064100b50 msgr2=0x7f1064100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:23.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.599+0000 7f1069394640 1 --2- 172.21.15.78:0/209092352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1064100b50 0x7f1064100f30 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f1054009ca0 tx=0x7f105402a950 comp rx=0 tx=0).stop 2024-08-18T23:53:23.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.599+0000 7f1069394640 1 -- 172.21.15.78:0/209092352 shutdown_connections 2024-08-18T23:53:23.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.599+0000 7f1069394640 1 --2- 172.21.15.78:0/209092352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1064102c20 0x7f106410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.599+0000 7f1069394640 1 --2- 172.21.15.78:0/209092352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10641022a0 0x7f10641026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.599+0000 7f1069394640 1 --2- 172.21.15.78:0/209092352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1064100b50 0x7f1064100f30 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.599+0000 7f1069394640 1 -- 172.21.15.78:0/209092352 >> 172.21.15.78:0/209092352 conn(0x7f10640fc500 msgr2=0x7f10640fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.600+0000 7f1069394640 1 -- 172.21.15.78:0/209092352 shutdown_connections 2024-08-18T23:53:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.600+0000 7f1069394640 1 -- 172.21.15.78:0/209092352 wait complete. 2024-08-18T23:53:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.600+0000 7f1069394640 1 Processor -- start 2024-08-18T23:53:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.600+0000 7f1069394640 1 -- start start 2024-08-18T23:53:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.600+0000 7f1069394640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1064100b50 0x7f10640fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f1069394640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10641022a0 0x7f10640fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f1069394640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1064102c20 0x7f10640fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f1069394640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1064110cf0 con 0x7f1064102c20 2024-08-18T23:53:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f1069394640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1064110b70 con 0x7f10641022a0 2024-08-18T23:53:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f1069394640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1064110e70 con 0x7f1064100b50 2024-08-18T23:53:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f10637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1064102c20 0x7f10640fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f10637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1064102c20 0x7f10640fd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37186/0 (socket says 172.21.15.78:37186) 2024-08-18T23:53:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f10637fe640 1 -- 172.21.15.78:0/1052839640 learned_addr learned my addr 172.21.15.78:0/1052839640 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f1062ffd640 1 --2- 172.21.15.78:0/1052839640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1064100b50 0x7f10640fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f10627fc640 1 --2- 172.21.15.78:0/1052839640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10641022a0 0x7f10640fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f10637fe640 1 -- 172.21.15.78:0/1052839640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1064100b50 msgr2=0x7f10640fc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f10637fe640 1 --2- 172.21.15.78:0/1052839640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1064100b50 0x7f10640fc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f10637fe640 1 -- 172.21.15.78:0/1052839640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10641022a0 msgr2=0x7f10640fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f10637fe640 1 --2- 172.21.15.78:0/1052839640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10641022a0 0x7f10640fcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f10637fe640 1 -- 172.21.15.78:0/1052839640 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10641a6710 con 0x7f1064102c20 2024-08-18T23:53:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f10637fe640 1 --2- 172.21.15.78:0/1052839640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1064102c20 0x7f10640fd430 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f105c00d750 tx=0x7f105c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f105bfff640 1 -- 172.21.15.78:0/1052839640 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c017070 con 0x7f1064102c20 2024-08-18T23:53:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f1069394640 1 -- 172.21.15.78:0/1052839640 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10641a6a00 con 0x7f1064102c20 2024-08-18T23:53:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f105bfff640 1 -- 172.21.15.78:0/1052839640 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f105c005500 con 0x7f1064102c20 2024-08-18T23:53:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f105bfff640 1 -- 172.21.15.78:0/1052839640 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c004d90 con 0x7f1064102c20 2024-08-18T23:53:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.601+0000 7f1069394640 1 -- 172.21.15.78:0/1052839640 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10641a6f40 con 0x7f1064102c20 2024-08-18T23:53:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.604+0000 7f105bfff640 1 -- 172.21.15.78:0/1052839640 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f105c004f30 con 0x7f1064102c20 2024-08-18T23:53:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.604+0000 7f106b792740 1 Processor -- start 2024-08-18T23:53:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.604+0000 7f106b792740 1 -- start start 2024-08-18T23:53:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.604+0000 7f106b792740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7a0210 0x55dffc7a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.604+0000 7f106b792740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0b30 0x55dffc7aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.604+0000 7f106b792740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7ab3f0 0x55dffc7ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.604+0000 7f106b792740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dffc705980 con 0x55dffc7a0b30 2024-08-18T23:53:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.604+0000 7f106b792740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dffc69da70 con 0x55dffc7a0210 2024-08-18T23:53:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.604+0000 7f106b792740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55dffc701940 con 0x55dffc7ab3f0 2024-08-18T23:53:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.604+0000 7f105bfff640 1 --2- 172.21.15.78:0/1052839640 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f10500a2b80 0x7f10500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.605+0000 7f1062ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0b30 0x55dffc7aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.605+0000 7f1062ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0b30 0x55dffc7aaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37200/0 (socket says 172.21.15.78:37200) 2024-08-18T23:53:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.605+0000 7f1062ffd640 1 -- 172.21.15.78:0/1447625347 learned_addr learned my addr 172.21.15.78:0/1447625347 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.605+0000 7f10627fc640 1 --2- 172.21.15.78:0/1447625347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7a0210 0x55dffc7a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.605+0000 7f10637fe640 1 --2- 172.21.15.78:0/1052839640 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f10500a2b80 0x7f10500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.605+0000 7f10627fc640 1 --2- 172.21.15.78:0/1447625347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7ab3f0 0x55dffc7ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.605+0000 7f105bfff640 1 -- 172.21.15.78:0/1052839640 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f105c0ceda0 con 0x7f1064102c20 2024-08-18T23:53:23.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.605+0000 7f1062ffd640 1 -- 172.21.15.78:0/1447625347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7ab3f0 msgr2=0x55dffc7ad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.605+0000 7f1062ffd640 1 --2- 172.21.15.78:0/1447625347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7ab3f0 0x55dffc7ad7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.605+0000 7f1062ffd640 1 -- 172.21.15.78:0/1447625347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7a0210 msgr2=0x55dffc7a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.605+0000 7f1062ffd640 1 --2- 172.21.15.78:0/1447625347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7a0210 0x55dffc7a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.605+0000 7f1062ffd640 1 -- 172.21.15.78:0/1447625347 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dffc7006e0 con 0x55dffc7a0b30 2024-08-18T23:53:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.605+0000 7f1062ffd640 1 --2- 172.21.15.78:0/1447625347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0b30 0x55dffc7aaeb0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f1054035000 tx=0x7f1054009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc124c06fa69519 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.605+0000 7f10637fe640 1 --2- 172.21.15.78:0/1052839640 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f10500a2b80 0x7f10500a5040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f105c09b200 tx=0x7f105c06b120 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:23.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.605+0000 7f10597fa640 1 -- 172.21.15.78:0/1447625347 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1054048070 con 0x55dffc7a0b30 2024-08-18T23:53:23.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.605+0000 7f10597fa640 1 -- 172.21.15.78:0/1447625347 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f105402fa70 con 0x55dffc7a0b30 2024-08-18T23:53:23.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.605+0000 7f10597fa640 1 -- 172.21.15.78:0/1447625347 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1054034040 con 0x55dffc7a0b30 2024-08-18T23:53:23.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.606+0000 7f106b792740 1 -- 172.21.15.78:0/1447625347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0b30 msgr2=0x55dffc7aaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:23.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.606+0000 7f106b792740 1 --2- 172.21.15.78:0/1447625347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0b30 0x55dffc7aaeb0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f1054035000 tx=0x7f1054009960 comp rx=0 tx=0).stop 2024-08-18T23:53:23.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.606+0000 7f106b792740 1 -- 172.21.15.78:0/1447625347 shutdown_connections 2024-08-18T23:53:23.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.606+0000 7f106b792740 1 --2- 172.21.15.78:0/1447625347 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7ab3f0 0x55dffc7ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.606+0000 7f106b792740 1 --2- 172.21.15.78:0/1447625347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0b30 0x55dffc7aaeb0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.606+0000 7f106b792740 1 --2- 172.21.15.78:0/1447625347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7a0210 0x55dffc7a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.606+0000 7f106b792740 1 -- 172.21.15.78:0/1447625347 >> 172.21.15.78:0/1447625347 conn(0x55dffc79c7a0 msgr2=0x55dffc79cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:23.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.606+0000 7f106b792740 1 -- 172.21.15.78:0/1447625347 shutdown_connections 2024-08-18T23:53:23.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.606+0000 7f106b792740 1 -- 172.21.15.78:0/1447625347 wait complete. 2024-08-18T23:53:23.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.606+0000 7f106b792740 1 -- start start 2024-08-18T23:53:23.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f106b792740 1 Processor -- start 2024-08-18T23:53:23.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f106b792740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7a0210 0x55dffc7a82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:23.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f106b792740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7a0b30 0x55dffc7a8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:23.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f106b792740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7ab3f0 0x55dffc7a8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:23.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f106b792740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55dffc7adfc0 con 0x55dffc7ab3f0 2024-08-18T23:53:23.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f106b792740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dffc6c0aa0 con 0x55dffc7a0b30 2024-08-18T23:53:23.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f106b792740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55dffc7ae140 con 0x55dffc7a0210 2024-08-18T23:53:23.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f106b792740 10 client.0 _ll_register_callbacks cb 0x55dffc6afe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:53:23.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f10627fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7ab3f0 0x55dffc7a8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:23.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f10627fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7ab3f0 0x55dffc7a8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37214/0 (socket says 172.21.15.78:37214) 2024-08-18T23:53:23.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f10627fc640 1 -- 172.21.15.78:0/710218236 learned_addr learned my addr 172.21.15.78:0/710218236 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:23.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f1062ffd640 1 --2- 172.21.15.78:0/710218236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7a0b30 0x55dffc7a8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f10627fc640 1 --2- 172.21.15.78:0/710218236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7a0210 0x55dffc7a82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f10627fc640 1 -- 172.21.15.78:0/710218236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7a0210 msgr2=0x55dffc7a82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f10627fc640 1 --2- 172.21.15.78:0/710218236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7a0210 0x55dffc7a82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f10627fc640 1 -- 172.21.15.78:0/710218236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7a0b30 msgr2=0x55dffc7a8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f10627fc640 1 --2- 172.21.15.78:0/710218236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7a0b30 0x55dffc7a8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f10627fc640 1 -- 172.21.15.78:0/710218236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dffc6a4650 con 0x55dffc7ab3f0 2024-08-18T23:53:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f10627fc640 1 --2- 172.21.15.78:0/710218236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7ab3f0 0x55dffc7a8d50 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f104c016410 tx=0x7f104c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738577f330f7c3fb server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:23.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.607+0000 7f104a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:53:23.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.608+0000 7f104a7fc640 1 -- 172.21.15.78:0/710218236 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104c014560 con 0x55dffc7ab3f0 2024-08-18T23:53:23.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.608+0000 7f106b792740 1 -- 172.21.15.78:0/710218236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f105402fa70 con 0x55dffc7ab3f0 2024-08-18T23:53:23.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.608+0000 7f104a7fc640 1 -- 172.21.15.78:0/710218236 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f104c014700 con 0x55dffc7ab3f0 2024-08-18T23:53:23.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.608+0000 7f104a7fc640 1 -- 172.21.15.78:0/710218236 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f104c026050 con 0x55dffc7ab3f0 2024-08-18T23:53:23.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.608+0000 7f106b792740 10 client.17034 fetch_fsmap learned FSMap version 32 2024-08-18T23:53:23.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.608+0000 7f106b792740 1 -- 172.21.15.78:0/710218236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dffc4773c0 con 0x55dffc7ab3f0 2024-08-18T23:53:23.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.608+0000 7f104a7fc640 1 -- 172.21.15.78:0/710218236 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f105402fa70 con 0x55dffc7ab3f0 2024-08-18T23:53:23.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.608+0000 7f104a7fc640 1 -- 172.21.15.78:0/710218236 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f104c015040 con 0x55dffc7ab3f0 2024-08-18T23:53:23.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.608+0000 7f106b792740 10 client.17034 fetch_fsmap finished waiting for FSMap version 32 2024-08-18T23:53:23.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.608+0000 7f106b792740 10 client.17034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:53:23.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.608+0000 7f106b792740 20 client.17034 populate_metadata read hostname 'smithi078' 2024-08-18T23:53:23.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.608+0000 7f106b792740 1 --2- 172.21.15.78:0/710218236 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55dffc7baee0 0x55dffc7bd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:23.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.608+0000 7f106b792740 4 client.17034 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:53:23.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.608+0000 7f106b792740 1 -- 172.21.15.78:0/710218236 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dffc712ef0 con 0x55dffc7baee0 2024-08-18T23:53:23.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.608+0000 7f1062ffd640 1 --2- 172.21.15.78:0/710218236 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55dffc7baee0 0x55dffc7bd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:23.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.609+0000 7f104a7fc640 10 client.17034 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:53:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.609+0000 7f1062ffd640 1 --2- 172.21.15.78:0/710218236 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55dffc7baee0 0x55dffc7bd310 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4237f011c42eed8 server_cookie=f808931715c31a34 in_seq=0 out_seq=0 2024-08-18T23:53:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.609+0000 7f104a7fc640 1 -- 172.21.15.78:0/710218236 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dffc712ef0 con 0x55dffc7baee0 2024-08-18T23:53:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.610+0000 7f104a7fc640 10 client.17034 handle_command_reply: tid=0 2024-08-18T23:53:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.610+0000 7f106b792740 1 client.17034 shutdown 2024-08-18T23:53:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.610+0000 7f106b792740 1 -- 172.21.15.78:0/710218236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7ab3f0 msgr2=0x55dffc7a8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.610+0000 7f106b792740 1 --2- 172.21.15.78:0/710218236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7ab3f0 0x55dffc7a8d50 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f104c016410 tx=0x7f104c0046c0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.610+0000 7f106b792740 1 -- 172.21.15.78:0/710218236 shutdown_connections 2024-08-18T23:53:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.610+0000 7f106b792740 1 --2- 172.21.15.78:0/710218236 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55dffc7baee0 0x55dffc7bd310 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.610+0000 7f106b792740 1 --2- 172.21.15.78:0/710218236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7ab3f0 0x55dffc7a8d50 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.610+0000 7f106b792740 1 --2- 172.21.15.78:0/710218236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7a0b30 0x55dffc7a8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.610+0000 7f106b792740 1 --2- 172.21.15.78:0/710218236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7a0210 0x55dffc7a82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.610+0000 7f106b792740 1 -- 172.21.15.78:0/710218236 >> 172.21.15.78:0/710218236 conn(0x55dffc79c7a0 msgr2=0x55dffc79b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.610+0000 7f104a7fc640 0 client.17034 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:53:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.610+0000 7f106b792740 1 -- 172.21.15.78:0/710218236 shutdown_connections 2024-08-18T23:53:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.610+0000 7f106b792740 1 -- 172.21.15.78:0/710218236 wait complete. 2024-08-18T23:53:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.610+0000 7f106b792740 20 client.17034 trim_cache size 0 max 16384 2024-08-18T23:53:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.627+0000 7f106b792740 1 Processor -- start 2024-08-18T23:53:23.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.627+0000 7f106b792740 1 -- start start 2024-08-18T23:53:23.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.627+0000 7f106b792740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0210 0x55dffc7c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:23.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.627+0000 7f106b792740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7a0b30 0x55dffc7c3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:23.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.627+0000 7f106b792740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7ab3f0 0x55dffc79b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:23.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.627+0000 7f106b792740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10641b35f0 con 0x55dffc7a0210 2024-08-18T23:53:23.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.627+0000 7f106b792740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10641b0e60 con 0x55dffc7a0b30 2024-08-18T23:53:23.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.627+0000 7f106b792740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f10641aecb0 con 0x55dffc7ab3f0 2024-08-18T23:53:23.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.627+0000 7f10627fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0210 0x55dffc7c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:23.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.627+0000 7f10627fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0210 0x55dffc7c2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37218/0 (socket says 172.21.15.78:37218) 2024-08-18T23:53:23.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.627+0000 7f10627fc640 1 -- 172.21.15.78:0/614100504 learned_addr learned my addr 172.21.15.78:0/614100504 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:23.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.627+0000 7f1062ffd640 1 --2- 172.21.15.78:0/614100504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7a0b30 0x55dffc7c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:23.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.627+0000 7f10627fc640 1 -- 172.21.15.78:0/614100504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7ab3f0 msgr2=0x55dffc79b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:23.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.627+0000 7f10627fc640 1 --2- 172.21.15.78:0/614100504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7ab3f0 0x55dffc79b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.627+0000 7f10627fc640 1 -- 172.21.15.78:0/614100504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7a0b30 msgr2=0x55dffc7c3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:23.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.627+0000 7f10627fc640 1 --2- 172.21.15.78:0/614100504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7a0b30 0x55dffc7c3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.627+0000 7f10627fc640 1 -- 172.21.15.78:0/614100504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dffc79c240 con 0x55dffc7a0210 2024-08-18T23:53:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.627+0000 7f1062ffd640 1 --2- 172.21.15.78:0/614100504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7a0b30 0x55dffc7c3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:53:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.628+0000 7f10627fc640 1 --2- 172.21.15.78:0/614100504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0210 0x55dffc7c2be0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f104c018bb0 tx=0x7f104c018cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ac8976ae3a16e2 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.628+0000 7f1058ff9640 1 -- 172.21.15.78:0/614100504 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104c006ca0 con 0x55dffc7a0210 2024-08-18T23:53:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.628+0000 7f1058ff9640 1 -- 172.21.15.78:0/614100504 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f104c01d070 con 0x55dffc7a0210 2024-08-18T23:53:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.628+0000 7f1058ff9640 1 -- 172.21.15.78:0/614100504 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104c0147c0 con 0x55dffc7a0210 2024-08-18T23:53:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.628+0000 7f106b792740 1 -- 172.21.15.78:0/614100504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0210 msgr2=0x55dffc7c2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.628+0000 7f106b792740 1 --2- 172.21.15.78:0/614100504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0210 0x55dffc7c2be0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f104c018bb0 tx=0x7f104c018cd0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f106b792740 1 -- 172.21.15.78:0/614100504 shutdown_connections 2024-08-18T23:53:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f106b792740 1 --2- 172.21.15.78:0/614100504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7ab3f0 0x55dffc79b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f106b792740 1 --2- 172.21.15.78:0/614100504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7a0b30 0x55dffc7c3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f106b792740 1 --2- 172.21.15.78:0/614100504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0210 0x55dffc7c2be0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f106b792740 1 -- 172.21.15.78:0/614100504 >> 172.21.15.78:0/614100504 conn(0x55dffc79c7a0 msgr2=0x55dffc7a6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f106b792740 1 -- 172.21.15.78:0/614100504 shutdown_connections 2024-08-18T23:53:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f106b792740 1 -- 172.21.15.78:0/614100504 wait complete. 2024-08-18T23:53:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f106b792740 1 -- start start 2024-08-18T23:53:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f106b792740 1 Processor -- start 2024-08-18T23:53:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f106b792740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7a0210 0x55dffc7b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f106b792740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0b30 0x55dffc7b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f106b792740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7ab3f0 0x55dffc7aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f106b792740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10641af320 con 0x55dffc7a0b30 2024-08-18T23:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f106b792740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10641b0790 con 0x55dffc7ab3f0 2024-08-18T23:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f106b792740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f10641af9f0 con 0x55dffc7a0210 2024-08-18T23:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f106b792740 10 client.0 _ll_register_callbacks cb 0x7f104c00aca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f1062ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0b30 0x55dffc7b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f1062ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0b30 0x55dffc7b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37234/0 (socket says 172.21.15.78:37234) 2024-08-18T23:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f1062ffd640 1 -- 172.21.15.78:0/1827449452 learned_addr learned my addr 172.21.15.78:0/1827449452 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f10627fc640 1 --2- 172.21.15.78:0/1827449452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7a0210 0x55dffc7b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.629+0000 7f10627fc640 1 --2- 172.21.15.78:0/1827449452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7ab3f0 0x55dffc7aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.630+0000 7f1062ffd640 1 -- 172.21.15.78:0/1827449452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7a0210 msgr2=0x55dffc7b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.630+0000 7f1062ffd640 1 --2- 172.21.15.78:0/1827449452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7a0210 0x55dffc7b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.630+0000 7f1062ffd640 1 -- 172.21.15.78:0/1827449452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7ab3f0 msgr2=0x55dffc7aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.630+0000 7f1062ffd640 1 --2- 172.21.15.78:0/1827449452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7ab3f0 0x55dffc7aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.630+0000 7f1062ffd640 1 -- 172.21.15.78:0/1827449452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dffc7af2d0 con 0x55dffc7a0b30 2024-08-18T23:53:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.630+0000 7f10627fc640 1 --2- 172.21.15.78:0/1827449452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7a0210 0x55dffc7b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:53:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.630+0000 7f1062ffd640 1 --2- 172.21.15.78:0/1827449452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0b30 0x55dffc7b3be0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1054046040 tx=0x7f1054009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5795c19cf35eaed2 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.630+0000 7f104a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:53:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.630+0000 7f104a7fc640 1 -- 172.21.15.78:0/1827449452 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1054048040 con 0x55dffc7a0b30 2024-08-18T23:53:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.630+0000 7f104a7fc640 1 -- 172.21.15.78:0/1827449452 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f105402fb30 con 0x55dffc7a0b30 2024-08-18T23:53:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.630+0000 7f106b792740 1 -- 172.21.15.78:0/1827449452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f104c01d070 con 0x55dffc7a0b30 2024-08-18T23:53:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.630+0000 7f104a7fc640 1 -- 172.21.15.78:0/1827449452 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f105405c050 con 0x55dffc7a0b30 2024-08-18T23:53:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.630+0000 7f106b792740 10 client.17040 fetch_fsmap learned FSMap version 32 2024-08-18T23:53:23.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.630+0000 7f106b792740 1 -- 172.21.15.78:0/1827449452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dffc8256d0 con 0x55dffc7a0b30 2024-08-18T23:53:23.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.630+0000 7f104a7fc640 1 -- 172.21.15.78:0/1827449452 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f104c01d070 con 0x55dffc7a0b30 2024-08-18T23:53:23.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.631+0000 7f104a7fc640 1 -- 172.21.15.78:0/1827449452 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1054004380 con 0x55dffc7a0b30 2024-08-18T23:53:23.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.631+0000 7f106b792740 10 client.17040 fetch_fsmap finished waiting for FSMap version 32 2024-08-18T23:53:23.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.631+0000 7f106b792740 10 client.17040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:53:23.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.631+0000 7f106b792740 20 client.17040 populate_metadata read hostname 'smithi078' 2024-08-18T23:53:23.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.631+0000 7f106b792740 1 --2- 172.21.15.78:0/1827449452 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55dffc8259d0 0x55dffc827df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:23.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.631+0000 7f106b792740 4 client.17040 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-18T23:53:23.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.631+0000 7f106b792740 1 -- 172.21.15.78:0/1827449452 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dffc70d400 con 0x55dffc8259d0 2024-08-18T23:53:23.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.631+0000 7f10627fc640 1 --2- 172.21.15.78:0/1827449452 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55dffc8259d0 0x55dffc827df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:23.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.631+0000 7f104a7fc640 10 client.17040 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:53:23.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.631+0000 7f10627fc640 1 --2- 172.21.15.78:0/1827449452 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55dffc8259d0 0x55dffc827df0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a9ae44f139b1ad6 server_cookie=56e5b2aea8d69352 in_seq=0 out_seq=0 2024-08-18T23:53:23.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.631+0000 7f104a7fc640 1 -- 172.21.15.78:0/1827449452 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dffc70d400 con 0x55dffc8259d0 2024-08-18T23:53:23.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.631+0000 7f104a7fc640 10 client.17040 handle_command_reply: tid=0 2024-08-18T23:53:23.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.631+0000 7f106b792740 1 client.17040 shutdown 2024-08-18T23:53:23.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f106b792740 1 -- 172.21.15.78:0/1827449452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0b30 msgr2=0x55dffc7b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:23.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f106b792740 1 --2- 172.21.15.78:0/1827449452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0b30 0x55dffc7b3be0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1054046040 tx=0x7f1054009bd0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f106b792740 1 -- 172.21.15.78:0/1827449452 shutdown_connections 2024-08-18T23:53:23.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f106b792740 1 --2- 172.21.15.78:0/1827449452 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55dffc8259d0 0x55dffc827df0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f106b792740 1 --2- 172.21.15.78:0/1827449452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dffc7ab3f0 0x55dffc7aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f106b792740 1 --2- 172.21.15.78:0/1827449452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55dffc7a0b30 0x55dffc7b3be0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f104a7fc640 0 client.17040 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:53:23.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f106b792740 1 --2- 172.21.15.78:0/1827449452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dffc7a0210 0x55dffc7b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f106b792740 1 -- 172.21.15.78:0/1827449452 >> 172.21.15.78:0/1827449452 conn(0x55dffc79c7a0 msgr2=0x55dffc7a8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:23.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f106b792740 1 -- 172.21.15.78:0/1827449452 shutdown_connections 2024-08-18T23:53:23.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f106b792740 1 -- 172.21.15.78:0/1827449452 wait complete. 2024-08-18T23:53:23.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f106b792740 20 client.17040 trim_cache size 0 max 16384 2024-08-18T23:53:23.663 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:53:23.663 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-08-18T23:53:23.663 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-08-18T23:53:23.664 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:53:23.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f1069394640 1 -- 172.21.15.78:0/1052839640 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f10500a2b80 msgr2=0x7f10500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:23.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f1069394640 1 --2- 172.21.15.78:0/1052839640 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f10500a2b80 0x7f10500a5040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f105c09b200 tx=0x7f105c06b120 comp rx=0 tx=0).stop 2024-08-18T23:53:23.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f1069394640 1 -- 172.21.15.78:0/1052839640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1064102c20 msgr2=0x7f10640fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:23.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f1069394640 1 --2- 172.21.15.78:0/1052839640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1064102c20 0x7f10640fd430 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f105c00d750 tx=0x7f105c00dc10 comp rx=0 tx=0).stop 2024-08-18T23:53:23.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f1069394640 1 -- 172.21.15.78:0/1052839640 shutdown_connections 2024-08-18T23:53:23.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f1069394640 1 --2- 172.21.15.78:0/1052839640 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f10500a2b80 0x7f10500a5040 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f1069394640 1 --2- 172.21.15.78:0/1052839640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1064102c20 0x7f10640fd430 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f1069394640 1 --2- 172.21.15.78:0/1052839640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10641022a0 0x7f10640fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f1069394640 1 --2- 172.21.15.78:0/1052839640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1064100b50 0x7f10640fc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:23.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.632+0000 7f1069394640 1 -- 172.21.15.78:0/1052839640 >> 172.21.15.78:0/1052839640 conn(0x7f10640fc500 msgr2=0x7f1064069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:23.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.633+0000 7f1069394640 1 -- 172.21.15.78:0/1052839640 shutdown_connections 2024-08-18T23:53:23.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:23.633+0000 7f1069394640 1 -- 172.21.15.78:0/1052839640 wait complete. 2024-08-18T23:53:24.493 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-18T23:53:24.493 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:53:24.788 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:53:25.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.291+0000 7f3a5f3ac640 1 -- 172.21.15.78:0/4186851300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a58102c20 msgr2=0x7f3a5810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:25.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.291+0000 7f3a5f3ac640 1 --2- 172.21.15.78:0/4186851300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a58102c20 0x7f3a5810d770 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3a50009900 tx=0x7f3a5002a480 comp rx=0 tx=0).stop 2024-08-18T23:53:25.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.291+0000 7f3a5f3ac640 1 -- 172.21.15.78:0/4186851300 shutdown_connections 2024-08-18T23:53:25.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.291+0000 7f3a5f3ac640 1 --2- 172.21.15.78:0/4186851300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a58102c20 0x7f3a5810d770 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:25.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.291+0000 7f3a5f3ac640 1 --2- 172.21.15.78:0/4186851300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a581022a0 0x7f3a581026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:25.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.291+0000 7f3a5f3ac640 1 --2- 172.21.15.78:0/4186851300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a58100b50 0x7f3a58100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:25.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.291+0000 7f3a5f3ac640 1 -- 172.21.15.78:0/4186851300 >> 172.21.15.78:0/4186851300 conn(0x7f3a580fc500 msgr2=0x7f3a580fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:25.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.291+0000 7f3a5f3ac640 1 -- 172.21.15.78:0/4186851300 shutdown_connections 2024-08-18T23:53:25.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.291+0000 7f3a5f3ac640 1 -- 172.21.15.78:0/4186851300 wait complete. 2024-08-18T23:53:25.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.291+0000 7f3a5f3ac640 1 Processor -- start 2024-08-18T23:53:25.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.291+0000 7f3a5f3ac640 1 -- start start 2024-08-18T23:53:25.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.291+0000 7f3a5f3ac640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a58100b50 0x7f3a581981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:25.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.291+0000 7f3a5f3ac640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a581022a0 0x7f3a581986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:25.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.291+0000 7f3a5f3ac640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a58102c20 0x7f3a5819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:25.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.291+0000 7f3a5f3ac640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3a58111dd0 con 0x7f3a58102c20 2024-08-18T23:53:25.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.292+0000 7f3a5f3ac640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a58111c50 con 0x7f3a581022a0 2024-08-18T23:53:25.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.292+0000 7f3a5f3ac640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3a58111f50 con 0x7f3a58100b50 2024-08-18T23:53:25.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.292+0000 7f3a5d949640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a58102c20 0x7f3a5819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:25.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.292+0000 7f3a5d949640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a58102c20 0x7f3a5819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37244/0 (socket says 172.21.15.78:37244) 2024-08-18T23:53:25.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.292+0000 7f3a5d949640 1 -- 172.21.15.78:0/970511982 learned_addr learned my addr 172.21.15.78:0/970511982 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:25.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.292+0000 7f3a5d148640 1 --2- 172.21.15.78:0/970511982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a58100b50 0x7f3a581981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:25.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.292+0000 7f3a5c947640 1 --2- 172.21.15.78:0/970511982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a581022a0 0x7f3a581986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:25.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.292+0000 7f3a5d949640 1 -- 172.21.15.78:0/970511982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a58100b50 msgr2=0x7f3a581981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:25.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.292+0000 7f3a5d949640 1 --2- 172.21.15.78:0/970511982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a58100b50 0x7f3a581981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:25.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.292+0000 7f3a5d949640 1 -- 172.21.15.78:0/970511982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a581022a0 msgr2=0x7f3a581986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:25.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.292+0000 7f3a5d949640 1 --2- 172.21.15.78:0/970511982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a581022a0 0x7f3a581986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:25.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.292+0000 7f3a5d949640 1 -- 172.21.15.78:0/970511982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5819d160 con 0x7f3a58102c20 2024-08-18T23:53:25.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.292+0000 7f3a5d949640 1 --2- 172.21.15.78:0/970511982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a58102c20 0x7f3a5819ca80 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3a50004770 tx=0x7f3a50003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:25.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.292+0000 7f3a567fc640 1 -- 172.21.15.78:0/970511982 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5003f070 con 0x7f3a58102c20 2024-08-18T23:53:25.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.292+0000 7f3a5f3ac640 1 -- 172.21.15.78:0/970511982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5819d3f0 con 0x7f3a58102c20 2024-08-18T23:53:25.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.292+0000 7f3a567fc640 1 -- 172.21.15.78:0/970511982 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a50031c30 con 0x7f3a58102c20 2024-08-18T23:53:25.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.292+0000 7f3a567fc640 1 -- 172.21.15.78:0/970511982 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a50035030 con 0x7f3a58102c20 2024-08-18T23:53:25.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.292+0000 7f3a5f3ac640 1 -- 172.21.15.78:0/970511982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a581a4d30 con 0x7f3a58102c20 2024-08-18T23:53:25.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.293+0000 7f3a5f3ac640 1 -- 172.21.15.78:0/970511982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a58100f30 con 0x7f3a58102c20 2024-08-18T23:53:25.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.295+0000 7f3a567fc640 1 -- 172.21.15.78:0/970511982 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3a50042070 con 0x7f3a58102c20 2024-08-18T23:53:25.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.296+0000 7f3a567fc640 1 --2- 172.21.15.78:0/970511982 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3a400a2b80 0x7f3a400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:25.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.296+0000 7f3a5d148640 1 --2- 172.21.15.78:0/970511982 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3a400a2b80 0x7f3a400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:25.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.296+0000 7f3a567fc640 1 -- 172.21.15.78:0/970511982 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f3a500f03a0 con 0x7f3a58102c20 2024-08-18T23:53:25.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.296+0000 7f3a5d148640 1 --2- 172.21.15.78:0/970511982 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3a400a2b80 0x7f3a400a5040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3a48005fd0 tx=0x7f3a48009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:25.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.299+0000 7f3a567fc640 1 -- 172.21.15.78:0/970511982 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3a500bcf70 con 0x7f3a58102c20 2024-08-18T23:53:25.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.503+0000 7f3a5f3ac640 1 -- 172.21.15.78:0/970511982 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a581a4f00 con 0x7f3a58102c20 2024-08-18T23:53:25.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.503+0000 7f3a567fc640 1 -- 172.21.15.78:0/970511982 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11932 (secure 0 0 0) 0x7f3a500bc870 con 0x7f3a58102c20 2024-08-18T23:53:25.505 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:53:25.505 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":32,"btime":"2024-08-18T23:52:48:372666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:52:47.960244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:53:25.505 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 32 2024-08-18T23:53:25.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.505+0000 7f3a5f3ac640 1 -- 172.21.15.78:0/970511982 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3a400a2b80 msgr2=0x7f3a400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:25.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.505+0000 7f3a5f3ac640 1 --2- 172.21.15.78:0/970511982 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3a400a2b80 0x7f3a400a5040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3a48005fd0 tx=0x7f3a48009680 comp rx=0 tx=0).stop 2024-08-18T23:53:25.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.505+0000 7f3a5f3ac640 1 -- 172.21.15.78:0/970511982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a58102c20 msgr2=0x7f3a5819ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:25.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.505+0000 7f3a5f3ac640 1 --2- 172.21.15.78:0/970511982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a58102c20 0x7f3a5819ca80 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3a50004770 tx=0x7f3a50003c10 comp rx=0 tx=0).stop 2024-08-18T23:53:25.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.505+0000 7f3a5f3ac640 1 -- 172.21.15.78:0/970511982 shutdown_connections 2024-08-18T23:53:25.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.505+0000 7f3a5f3ac640 1 --2- 172.21.15.78:0/970511982 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f3a400a2b80 0x7f3a400a5040 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:25.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.505+0000 7f3a5f3ac640 1 --2- 172.21.15.78:0/970511982 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a58102c20 0x7f3a5819ca80 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:25.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.505+0000 7f3a5f3ac640 1 --2- 172.21.15.78:0/970511982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a581022a0 0x7f3a581986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:25.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.505+0000 7f3a5f3ac640 1 --2- 172.21.15.78:0/970511982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a58100b50 0x7f3a581981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:25.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.505+0000 7f3a5f3ac640 1 -- 172.21.15.78:0/970511982 >> 172.21.15.78:0/970511982 conn(0x7f3a580fc500 msgr2=0x7f3a58106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:25.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.506+0000 7f3a5f3ac640 1 -- 172.21.15.78:0/970511982 shutdown_connections 2024-08-18T23:53:25.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:25.506+0000 7f3a5f3ac640 1 -- 172.21.15.78:0/970511982 wait complete. 2024-08-18T23:53:25.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:25 smithi161 ceph-mon[22027]: pgmap v623: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:53:25.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:25 smithi078 ceph-mon[18440]: pgmap v623: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:53:25.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:25 smithi123 ceph-mon[21808]: pgmap v623: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:53:26.179 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-18T23:53:26.442 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:53:26.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:26 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/970511982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:53:26.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:26 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/970511982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:53:26.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:26 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/970511982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:53:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.993+0000 7f1cef394640 1 -- 172.21.15.78:0/3130979160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ce810bea0 msgr2=0x7f1ce810e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.993+0000 7f1cef394640 1 --2- 172.21.15.78:0/3130979160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ce810bea0 0x7f1ce810e340 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0009ca0 tx=0x7f1ce002a950 comp rx=0 tx=0).stop 2024-08-18T23:53:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.993+0000 7f1cef394640 1 -- 172.21.15.78:0/3130979160 shutdown_connections 2024-08-18T23:53:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.993+0000 7f1cef394640 1 --2- 172.21.15.78:0/3130979160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ce810bea0 0x7f1ce810e340 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.993+0000 7f1cef394640 1 --2- 172.21.15.78:0/3130979160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ce8103390 0x7f1ce810b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.993+0000 7f1cef394640 1 --2- 172.21.15.78:0/3130979160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce8102a70 0x7f1ce8102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.993+0000 7f1cef394640 1 -- 172.21.15.78:0/3130979160 >> 172.21.15.78:0/3130979160 conn(0x7f1ce80fc4d0 msgr2=0x7f1ce80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.993+0000 7f1cef394640 1 -- 172.21.15.78:0/3130979160 shutdown_connections 2024-08-18T23:53:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.993+0000 7f1cef394640 1 -- 172.21.15.78:0/3130979160 wait complete. 2024-08-18T23:53:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.994+0000 7f1cef394640 1 Processor -- start 2024-08-18T23:53:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.994+0000 7f1cef394640 1 -- start start 2024-08-18T23:53:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.994+0000 7f1cef394640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ce8102a70 0x7f1ce81006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.994+0000 7f1cef394640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ce8103390 0x7f1ce8100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.994+0000 7f1cef394640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce810bea0 0x7f1ce8101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.994+0000 7f1cef394640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce8111c60 con 0x7f1ce8102a70 2024-08-18T23:53:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.994+0000 7f1cef394640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce8111ae0 con 0x7f1ce810bea0 2024-08-18T23:53:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.994+0000 7f1cef394640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce8111de0 con 0x7f1ce8103390 2024-08-18T23:53:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.994+0000 7f1ced130640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ce8102a70 0x7f1ce81006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.994+0000 7f1ced130640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ce8102a70 0x7f1ce81006d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46096/0 (socket says 172.21.15.78:46096) 2024-08-18T23:53:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.994+0000 7f1ced130640 1 -- 172.21.15.78:0/4054555077 learned_addr learned my addr 172.21.15.78:0/4054555077 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.994+0000 7f1cec92f640 1 --2- 172.21.15.78:0/4054555077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ce8103390 0x7f1ce8100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.994+0000 7f1ced130640 1 -- 172.21.15.78:0/4054555077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ce8103390 msgr2=0x7f1ce8100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.994+0000 7f1ced130640 1 --2- 172.21.15.78:0/4054555077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ce8103390 0x7f1ce8100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.994+0000 7f1ced130640 1 -- 172.21.15.78:0/4054555077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce810bea0 msgr2=0x7f1ce8101150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.994+0000 7f1ced130640 1 --2- 172.21.15.78:0/4054555077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce810bea0 0x7f1ce8101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.994+0000 7f1ced130640 1 -- 172.21.15.78:0/4054555077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce81a6520 con 0x7f1ce8102a70 2024-08-18T23:53:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.994+0000 7f1cec92f640 1 --2- 172.21.15.78:0/4054555077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ce8103390 0x7f1ce8100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:53:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.995+0000 7f1ced130640 1 --2- 172.21.15.78:0/4054555077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ce8102a70 0x7f1ce81006d0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400c920 tx=0x7f1ce400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.995+0000 7f1cda7fc640 1 -- 172.21.15.78:0/4054555077 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4007c70 con 0x7f1ce8102a70 2024-08-18T23:53:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.995+0000 7f1cef394640 1 -- 172.21.15.78:0/4054555077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce81a6810 con 0x7f1ce8102a70 2024-08-18T23:53:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.995+0000 7f1cda7fc640 1 -- 172.21.15.78:0/4054555077 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce4007e10 con 0x7f1ce8102a70 2024-08-18T23:53:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.995+0000 7f1cda7fc640 1 -- 172.21.15.78:0/4054555077 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4018890 con 0x7f1ce8102a70 2024-08-18T23:53:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.995+0000 7f1cef394640 1 -- 172.21.15.78:0/4054555077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce81a6d50 con 0x7f1ce8102a70 2024-08-18T23:53:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.997+0000 7f1cda7fc640 1 -- 172.21.15.78:0/4054555077 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1ce4018b10 con 0x7f1ce8102a70 2024-08-18T23:53:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.997+0000 7f1cf1792740 1 Processor -- start 2024-08-18T23:53:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.997+0000 7f1cf1792740 1 -- start start 2024-08-18T23:53:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1cf1792740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564087002ff0 0x5640870033d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1cf1792740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087003910 0x56408700dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1cf1792740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 0x5640870105c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1cda7fc640 1 --2- 172.21.15.78:0/4054555077 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1cd40a2b80 0x7f1cd40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1cf1792740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564086f68a40 con 0x56408700e1d0 2024-08-18T23:53:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1cf1792740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564086f009d0 con 0x564087003910 2024-08-18T23:53:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1cf1792740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564086f64a00 con 0x564087002ff0 2024-08-18T23:53:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1cec92f640 1 --2- 172.21.15.78:0/4054555077 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1cd40a2b80 0x7f1cd40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1ced931640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 0x5640870105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1ced931640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 0x5640870105c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46098/0 (socket says 172.21.15.78:46098) 2024-08-18T23:53:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1ced931640 1 -- 172.21.15.78:0/2800997402 learned_addr learned my addr 172.21.15.78:0/2800997402 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1cda7fc640 1 -- 172.21.15.78:0/4054555077 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f1ce40cf620 con 0x7f1ce8102a70 2024-08-18T23:53:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1ced130640 1 --2- 172.21.15.78:0/2800997402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087003910 0x56408700dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1ced931640 1 --2- 172.21.15.78:0/2800997402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564087002ff0 0x5640870033d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1ced130640 1 -- 172.21.15.78:0/2800997402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564087002ff0 msgr2=0x5640870033d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1ced130640 1 --2- 172.21.15.78:0/2800997402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564087002ff0 0x5640870033d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1ced130640 1 -- 172.21.15.78:0/2800997402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 msgr2=0x5640870105c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:27.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1ced130640 1 --2- 172.21.15.78:0/2800997402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 0x5640870105c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1ced130640 1 -- 172.21.15.78:0/2800997402 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564086f07650 con 0x564087003910 2024-08-18T23:53:27.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1ced931640 1 --2- 172.21.15.78:0/2800997402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 0x5640870105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:53:27.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.998+0000 7f1cec92f640 1 --2- 172.21.15.78:0/4054555077 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1cd40a2b80 0x7f1cd40a5040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8102160 tx=0x7f1cdc00a5e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:27.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.999+0000 7f1ced130640 1 --2- 172.21.15.78:0/2800997402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087003910 0x56408700dc90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4097ef0 tx=0x7f1ce4097e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaa4a2e692f6ccf3 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:27.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.999+0000 7f1cd37fe640 1 -- 172.21.15.78:0/2800997402 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce409ba90 con 0x564087003910 2024-08-18T23:53:27.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.999+0000 7f1cd37fe640 1 -- 172.21.15.78:0/2800997402 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce409fe40 con 0x564087003910 2024-08-18T23:53:27.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.999+0000 7f1cd37fe640 1 -- 172.21.15.78:0/2800997402 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce400fb30 con 0x564087003910 2024-08-18T23:53:27.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.999+0000 7f1cf1792740 1 -- 172.21.15.78:0/2800997402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087003910 msgr2=0x56408700dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:27.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:26.999+0000 7f1cf1792740 1 --2- 172.21.15.78:0/2800997402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087003910 0x56408700dc90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4097ef0 tx=0x7f1ce4097e60 comp rx=0 tx=0).stop 2024-08-18T23:53:27.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.000+0000 7f1cf1792740 1 -- 172.21.15.78:0/2800997402 shutdown_connections 2024-08-18T23:53:27.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.000+0000 7f1cf1792740 1 --2- 172.21.15.78:0/2800997402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 0x5640870105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.000+0000 7f1cf1792740 1 --2- 172.21.15.78:0/2800997402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087003910 0x56408700dc90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.000+0000 7f1cf1792740 1 --2- 172.21.15.78:0/2800997402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564087002ff0 0x5640870033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.000+0000 7f1cf1792740 1 -- 172.21.15.78:0/2800997402 >> 172.21.15.78:0/2800997402 conn(0x564086fff580 msgr2=0x564086fff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:27.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.000+0000 7f1cf1792740 1 -- 172.21.15.78:0/2800997402 shutdown_connections 2024-08-18T23:53:27.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.000+0000 7f1cf1792740 1 -- 172.21.15.78:0/2800997402 wait complete. 2024-08-18T23:53:27.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.000+0000 7f1cf1792740 1 -- start start 2024-08-18T23:53:27.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.000+0000 7f1cf1792740 1 Processor -- start 2024-08-18T23:53:27.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.000+0000 7f1cf1792740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087002ff0 0x56408700b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:27.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1cf1792740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564087003910 0x56408700b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:27.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1cf1792740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 0x56408700bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:27.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1cf1792740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564087010da0 con 0x56408700e1d0 2024-08-18T23:53:27.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1cf1792740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564086f23b20 con 0x564087002ff0 2024-08-18T23:53:27.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1cf1792740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564087010f20 con 0x564087003910 2024-08-18T23:53:27.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1cf1792740 10 client.0 _ll_register_callbacks cb 0x564086f21bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:53:27.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1ced931640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 0x56408700bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:27.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1ced130640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087002ff0 0x56408700b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:27.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1ced931640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 0x56408700bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46104/0 (socket says 172.21.15.78:46104) 2024-08-18T23:53:27.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1ced931640 1 -- 172.21.15.78:0/298167467 learned_addr learned my addr 172.21.15.78:0/298167467 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:27.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1cec92f640 1 --2- 172.21.15.78:0/298167467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564087003910 0x56408700b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:27.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1ced931640 1 -- 172.21.15.78:0/298167467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564087003910 msgr2=0x56408700b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:27.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1ced931640 1 --2- 172.21.15.78:0/298167467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564087003910 0x56408700b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1ced931640 1 -- 172.21.15.78:0/298167467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087002ff0 msgr2=0x56408700b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:27.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1ced931640 1 --2- 172.21.15.78:0/298167467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087002ff0 0x56408700b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1ced931640 1 -- 172.21.15.78:0/298167467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564086cda3c0 con 0x56408700e1d0 2024-08-18T23:53:27.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1cec92f640 1 --2- 172.21.15.78:0/298167467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564087003910 0x56408700b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:53:27.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1ced931640 1 --2- 172.21.15.78:0/298167467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 0x56408700bb60 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1ce002f6a0 tx=0x7f1ce002f6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baec5c2e7b813dde server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:27.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1cd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:53:27.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1cd0ff9640 1 -- 172.21.15.78:0/298167467 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0034040 con 0x56408700e1d0 2024-08-18T23:53:27.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1cd0ff9640 1 -- 172.21.15.78:0/298167467 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f1ce0003d80 con 0x56408700e1d0 2024-08-18T23:53:27.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1cf1792740 1 -- 172.21.15.78:0/298167467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ce409fe40 con 0x56408700e1d0 2024-08-18T23:53:27.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1cd0ff9640 1 -- 172.21.15.78:0/298167467 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1ce0052050 con 0x56408700e1d0 2024-08-18T23:53:27.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1cf1792740 10 client.17058 fetch_fsmap learned FSMap version 32 2024-08-18T23:53:27.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.001+0000 7f1cf1792740 1 -- 172.21.15.78:0/298167467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56408701c9d0 con 0x56408700e1d0 2024-08-18T23:53:27.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.002+0000 7f1cd0ff9640 1 -- 172.21.15.78:0/298167467 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce409fe40 con 0x56408700e1d0 2024-08-18T23:53:27.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.002+0000 7f1cd0ff9640 1 -- 172.21.15.78:0/298167467 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1ce00307a0 con 0x56408700e1d0 2024-08-18T23:53:27.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.002+0000 7f1cf1792740 10 client.17058 fetch_fsmap finished waiting for FSMap version 32 2024-08-18T23:53:27.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.002+0000 7f1cf1792740 10 client.17058 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:53:27.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.002+0000 7f1cf1792740 20 client.17058 populate_metadata read hostname 'smithi078' 2024-08-18T23:53:27.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.002+0000 7f1cf1792740 1 --2- 172.21.15.78:0/298167467 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x56408701dea0 0x5640870202e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:27.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.002+0000 7f1cf1792740 4 client.17058 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:53:27.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.002+0000 7f1cf1792740 1 -- 172.21.15.78:0/298167467 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564086f75fb0 con 0x56408701dea0 2024-08-18T23:53:27.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.002+0000 7f1ced130640 1 --2- 172.21.15.78:0/298167467 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x56408701dea0 0x5640870202e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:27.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.003+0000 7f1ced130640 1 --2- 172.21.15.78:0/298167467 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x56408701dea0 0x5640870202e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d7a6c78eae70eee server_cookie=9285f8a168f6e506 in_seq=0 out_seq=0 2024-08-18T23:53:27.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.003+0000 7f1cd0ff9640 10 client.17058 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:53:27.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.003+0000 7f1cd0ff9640 1 -- 172.21.15.78:0/298167467 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564086f75fb0 con 0x56408701dea0 2024-08-18T23:53:27.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.003+0000 7f1cd0ff9640 10 client.17058 handle_command_reply: tid=0 2024-08-18T23:53:27.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.003+0000 7f1cf1792740 1 client.17058 shutdown 2024-08-18T23:53:27.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.004+0000 7f1cf1792740 1 -- 172.21.15.78:0/298167467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 msgr2=0x56408700bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:27.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.004+0000 7f1cf1792740 1 --2- 172.21.15.78:0/298167467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 0x56408700bb60 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1ce002f6a0 tx=0x7f1ce002f6d0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.004+0000 7f1cf1792740 1 -- 172.21.15.78:0/298167467 shutdown_connections 2024-08-18T23:53:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.004+0000 7f1cf1792740 1 --2- 172.21.15.78:0/298167467 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x56408701dea0 0x5640870202e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.004+0000 7f1cf1792740 1 --2- 172.21.15.78:0/298167467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 0x56408700bb60 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.004+0000 7f1cf1792740 1 --2- 172.21.15.78:0/298167467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564087003910 0x56408700b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.004+0000 7f1cf1792740 1 --2- 172.21.15.78:0/298167467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087002ff0 0x56408700b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.004+0000 7f1cf1792740 1 -- 172.21.15.78:0/298167467 >> 172.21.15.78:0/298167467 conn(0x564086fff580 msgr2=0x5640870052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.004+0000 7f1cd0ff9640 0 client.17058 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:53:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.004+0000 7f1cf1792740 1 -- 172.21.15.78:0/298167467 shutdown_connections 2024-08-18T23:53:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.004+0000 7f1cf1792740 1 -- 172.21.15.78:0/298167467 wait complete. 2024-08-18T23:53:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.004+0000 7f1cf1792740 20 client.17058 trim_cache size 0 max 16384 2024-08-18T23:53:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.017+0000 7f1cf1792740 1 Processor -- start 2024-08-18T23:53:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.017+0000 7f1cf1792740 1 -- start start 2024-08-18T23:53:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.017+0000 7f1cf1792740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564087002ff0 0x5640870099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.017+0000 7f1cf1792740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087003910 0x564087009ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.017+0000 7f1cf1792740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56408700e1d0 0x564087025bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.017+0000 7f1cf1792740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce81b3490 con 0x564087002ff0 2024-08-18T23:53:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.017+0000 7f1cf1792740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce81b0d00 con 0x564087003910 2024-08-18T23:53:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.017+0000 7f1cf1792740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce81aeb50 con 0x56408700e1d0 2024-08-18T23:53:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.017+0000 7f1ced130640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564087002ff0 0x5640870099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.017+0000 7f1ced130640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564087002ff0 0x5640870099b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46118/0 (socket says 172.21.15.78:46118) 2024-08-18T23:53:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.017+0000 7f1ced130640 1 -- 172.21.15.78:0/2622147993 learned_addr learned my addr 172.21.15.78:0/2622147993 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.017+0000 7f1cec92f640 1 --2- 172.21.15.78:0/2622147993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087003910 0x564087009ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.017+0000 7f1ced130640 1 -- 172.21.15.78:0/2622147993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56408700e1d0 msgr2=0x564087025bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.017+0000 7f1ced130640 1 --2- 172.21.15.78:0/2622147993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56408700e1d0 0x564087025bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.017+0000 7f1ced130640 1 -- 172.21.15.78:0/2622147993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087003910 msgr2=0x564087009ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.017+0000 7f1ced130640 1 --2- 172.21.15.78:0/2622147993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087003910 0x564087009ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.017+0000 7f1ced130640 1 -- 172.21.15.78:0/2622147993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564087010900 con 0x564087002ff0 2024-08-18T23:53:27.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.017+0000 7f1cec92f640 1 --2- 172.21.15.78:0/2622147993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087003910 0x564087009ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:53:27.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.017+0000 7f1ced130640 1 --2- 172.21.15.78:0/2622147993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564087002ff0 0x5640870099b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1ce409b790 tx=0x7f1ce400ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70646b695fffc487 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:27.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.018+0000 7f1cd2ffd640 1 -- 172.21.15.78:0/2622147993 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce409ba90 con 0x564087002ff0 2024-08-18T23:53:27.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.018+0000 7f1cd2ffd640 1 -- 172.21.15.78:0/2622147993 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce409fe40 con 0x564087002ff0 2024-08-18T23:53:27.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.018+0000 7f1cd2ffd640 1 -- 172.21.15.78:0/2622147993 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40a1210 con 0x564087002ff0 2024-08-18T23:53:27.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.018+0000 7f1cf1792740 1 -- 172.21.15.78:0/2622147993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564087002ff0 msgr2=0x5640870099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:27.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.018+0000 7f1cf1792740 1 --2- 172.21.15.78:0/2622147993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564087002ff0 0x5640870099b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1ce409b790 tx=0x7f1ce400ff30 comp rx=0 tx=0).stop 2024-08-18T23:53:27.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.018+0000 7f1cf1792740 1 -- 172.21.15.78:0/2622147993 shutdown_connections 2024-08-18T23:53:27.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.018+0000 7f1cf1792740 1 --2- 172.21.15.78:0/2622147993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56408700e1d0 0x564087025bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.018+0000 7f1cf1792740 1 --2- 172.21.15.78:0/2622147993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087003910 0x564087009ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.018+0000 7f1cf1792740 1 --2- 172.21.15.78:0/2622147993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564087002ff0 0x5640870099b0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.018+0000 7f1cf1792740 1 -- 172.21.15.78:0/2622147993 >> 172.21.15.78:0/2622147993 conn(0x564086fff580 msgr2=0x564087006370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.018+0000 7f1cf1792740 1 -- 172.21.15.78:0/2622147993 shutdown_connections 2024-08-18T23:53:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.019+0000 7f1cf1792740 1 -- 172.21.15.78:0/2622147993 wait complete. 2024-08-18T23:53:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.019+0000 7f1cf1792740 1 -- start start 2024-08-18T23:53:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.019+0000 7f1cf1792740 1 Processor -- start 2024-08-18T23:53:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1cf1792740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564087002ff0 0x564087011130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1cf1792740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087003910 0x564087011670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1cf1792740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 0x564087011bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1cf1792740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce81af1c0 con 0x56408700e1d0 2024-08-18T23:53:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1cf1792740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce81b0630 con 0x564087003910 2024-08-18T23:53:27.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1cf1792740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce81af890 con 0x564087002ff0 2024-08-18T23:53:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1cf1792740 10 client.0 _ll_register_callbacks cb 0x7f1ce002f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:53:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1ced931640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 0x564087011bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1ced931640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 0x564087011bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46122/0 (socket says 172.21.15.78:46122) 2024-08-18T23:53:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1ced931640 1 -- 172.21.15.78:0/3350488526 learned_addr learned my addr 172.21.15.78:0/3350488526 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1ced130640 1 --2- 172.21.15.78:0/3350488526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564087002ff0 0x564087011130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1cec92f640 1 --2- 172.21.15.78:0/3350488526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087003910 0x564087011670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1ced931640 1 -- 172.21.15.78:0/3350488526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564087002ff0 msgr2=0x564087011130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1ced931640 1 --2- 172.21.15.78:0/3350488526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564087002ff0 0x564087011130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1ced931640 1 -- 172.21.15.78:0/3350488526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087003910 msgr2=0x564087011670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1ced931640 1 --2- 172.21.15.78:0/3350488526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087003910 0x564087011670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1ced931640 1 -- 172.21.15.78:0/3350488526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564087086b70 con 0x56408700e1d0 2024-08-18T23:53:27.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1cec92f640 1 --2- 172.21.15.78:0/3350488526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087003910 0x564087011670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:53:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1ced931640 1 --2- 172.21.15.78:0/3350488526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 0x564087011bb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f1ce002ae70 tx=0x7f1ce00093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6747bf2f119402c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1cd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:53:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1cd0ff9640 1 -- 172.21.15.78:0/3350488526 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0034030 con 0x56408700e1d0 2024-08-18T23:53:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1cd0ff9640 1 -- 172.21.15.78:0/3350488526 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f1ce0004b40 con 0x56408700e1d0 2024-08-18T23:53:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.020+0000 7f1cf1792740 1 -- 172.21.15.78:0/3350488526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ce409fe40 con 0x56408700e1d0 2024-08-18T23:53:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.021+0000 7f1cd0ff9640 1 -- 172.21.15.78:0/3350488526 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1ce005e050 con 0x56408700e1d0 2024-08-18T23:53:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.021+0000 7f1cf1792740 10 client.17064 fetch_fsmap learned FSMap version 32 2024-08-18T23:53:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.021+0000 7f1cf1792740 1 -- 172.21.15.78:0/3350488526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564087087d20 con 0x56408700e1d0 2024-08-18T23:53:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.021+0000 7f1cd0ff9640 1 -- 172.21.15.78:0/3350488526 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce409fe40 con 0x56408700e1d0 2024-08-18T23:53:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.021+0000 7f1cd0ff9640 1 -- 172.21.15.78:0/3350488526 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1ce003a830 con 0x56408700e1d0 2024-08-18T23:53:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.021+0000 7f1cf1792740 10 client.17064 fetch_fsmap finished waiting for FSMap version 32 2024-08-18T23:53:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.021+0000 7f1cf1792740 10 client.17064 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-18T23:53:27.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.021+0000 7f1cf1792740 20 client.17064 populate_metadata read hostname 'smithi078' 2024-08-18T23:53:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.021+0000 7f1cf1792740 1 --2- 172.21.15.78:0/3350488526 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x564087088410 0x56408708a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.021+0000 7f1cf1792740 4 client.17064 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:53:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.021+0000 7f1cf1792740 1 -- 172.21.15.78:0/3350488526 --> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564086f704c0 con 0x564087088410 2024-08-18T23:53:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.021+0000 7f1ced130640 1 --2- 172.21.15.78:0/3350488526 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x564087088410 0x56408708a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.022+0000 7f1ced130640 1 --2- 172.21.15.78:0/3350488526 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x564087088410 0x56408708a830 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f11e0abe124c04e server_cookie=3485bbef9dd873b in_seq=0 out_seq=0 2024-08-18T23:53:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.022+0000 7f1cd0ff9640 10 client.17064 ms_handle_connect on v2:172.21.15.161:6838/2209088174 2024-08-18T23:53:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.022+0000 7f1cd0ff9640 1 -- 172.21.15.78:0/3350488526 <== mds.2 v2:172.21.15.161:6838/2209088174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564086f704c0 con 0x564087088410 2024-08-18T23:53:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.022+0000 7f1cd0ff9640 10 client.17064 handle_command_reply: tid=0 2024-08-18T23:53:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.022+0000 7f1cf1792740 1 client.17064 shutdown 2024-08-18T23:53:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.022+0000 7f1cf1792740 1 -- 172.21.15.78:0/3350488526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 msgr2=0x564087011bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.022+0000 7f1cf1792740 1 --2- 172.21.15.78:0/3350488526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 0x564087011bb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f1ce002ae70 tx=0x7f1ce00093e0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.022+0000 7f1cf1792740 1 -- 172.21.15.78:0/3350488526 shutdown_connections 2024-08-18T23:53:27.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.022+0000 7f1cf1792740 1 --2- 172.21.15.78:0/3350488526 >> [v2:172.21.15.161:6838/2209088174,v1:172.21.15.161:6839/2209088174] conn(0x564087088410 0x56408708a830 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.022+0000 7f1cf1792740 1 --2- 172.21.15.78:0/3350488526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56408700e1d0 0x564087011bb0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.022+0000 7f1cf1792740 1 --2- 172.21.15.78:0/3350488526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564087003910 0x564087011670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.022+0000 7f1cf1792740 1 --2- 172.21.15.78:0/3350488526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564087002ff0 0x564087011130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.022+0000 7f1cf1792740 1 -- 172.21.15.78:0/3350488526 >> 172.21.15.78:0/3350488526 conn(0x564086fff580 msgr2=0x564086ffdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.022+0000 7f1cd0ff9640 0 client.17064 ms_handle_reset on v2:172.21.15.161:6838/2209088174 2024-08-18T23:53:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.022+0000 7f1cf1792740 1 -- 172.21.15.78:0/3350488526 shutdown_connections 2024-08-18T23:53:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.022+0000 7f1cf1792740 1 -- 172.21.15.78:0/3350488526 wait complete. 2024-08-18T23:53:27.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.022+0000 7f1cf1792740 20 client.17064 trim_cache size 0 max 16384 2024-08-18T23:53:27.041 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:53:27.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.023+0000 7f1cef394640 1 -- 172.21.15.78:0/4054555077 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1cd40a2b80 msgr2=0x7f1cd40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:27.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.023+0000 7f1cef394640 1 --2- 172.21.15.78:0/4054555077 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1cd40a2b80 0x7f1cd40a5040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8102160 tx=0x7f1cdc00a5e0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.023+0000 7f1cef394640 1 -- 172.21.15.78:0/4054555077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ce8102a70 msgr2=0x7f1ce81006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:27.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.023+0000 7f1cef394640 1 --2- 172.21.15.78:0/4054555077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ce8102a70 0x7f1ce81006d0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400c920 tx=0x7f1ce400cde0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.023+0000 7f1cef394640 1 -- 172.21.15.78:0/4054555077 shutdown_connections 2024-08-18T23:53:27.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.023+0000 7f1cef394640 1 --2- 172.21.15.78:0/4054555077 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f1cd40a2b80 0x7f1cd40a5040 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.023+0000 7f1cef394640 1 --2- 172.21.15.78:0/4054555077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce810bea0 0x7f1ce8101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.023+0000 7f1cef394640 1 --2- 172.21.15.78:0/4054555077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ce8103390 0x7f1ce8100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.023+0000 7f1cef394640 1 --2- 172.21.15.78:0/4054555077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ce8102a70 0x7f1ce81006d0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:27.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.023+0000 7f1cef394640 1 -- 172.21.15.78:0/4054555077 >> 172.21.15.78:0/4054555077 conn(0x7f1ce80fc4d0 msgr2=0x7f1ce80fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:27.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.023+0000 7f1cef394640 1 -- 172.21.15.78:0/4054555077 shutdown_connections 2024-08-18T23:53:27.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:27.023+0000 7f1cef394640 1 -- 172.21.15.78:0/4054555077 wait complete. 2024-08-18T23:53:27.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:27 smithi161 ceph-mon[22027]: pgmap v624: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-18T23:53:27.707 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:27 smithi078 ceph-mon[18440]: pgmap v624: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-18T23:53:27.734 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-18T23:53:27.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:27 smithi123 ceph-mon[21808]: pgmap v624: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-18T23:53:27.986 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:53:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.484+0000 7ff0a5046640 1 -- 172.21.15.78:0/1215613041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a01067b0 msgr2=0x7ff0a0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.484+0000 7ff0a5046640 1 --2- 172.21.15.78:0/1215613041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a01067b0 0x7ff0a0106b90 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff090009930 tx=0x7ff09002a420 comp rx=0 tx=0).stop 2024-08-18T23:53:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.484+0000 7ff0a5046640 1 -- 172.21.15.78:0/1215613041 shutdown_connections 2024-08-18T23:53:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.484+0000 7ff0a5046640 1 --2- 172.21.15.78:0/1215613041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0a0101150 0x7ff0a010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.484+0000 7ff0a5046640 1 --2- 172.21.15.78:0/1215613041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0a01007b0 0x7ff0a0100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.484+0000 7ff0a5046640 1 --2- 172.21.15.78:0/1215613041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a01067b0 0x7ff0a0106b90 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.484+0000 7ff0a5046640 1 -- 172.21.15.78:0/1215613041 >> 172.21.15.78:0/1215613041 conn(0x7ff0a00fc4d0 msgr2=0x7ff0a00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.484+0000 7ff0a5046640 1 -- 172.21.15.78:0/1215613041 shutdown_connections 2024-08-18T23:53:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.484+0000 7ff0a5046640 1 -- 172.21.15.78:0/1215613041 wait complete. 2024-08-18T23:53:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.485+0000 7ff0a5046640 1 Processor -- start 2024-08-18T23:53:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.485+0000 7ff0a5046640 1 -- start start 2024-08-18T23:53:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.485+0000 7ff0a5046640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0a01007b0 0x7ff0a01971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.485+0000 7ff0a5046640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a0101150 0x7ff0a0197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.485+0000 7ff0a5046640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0a01067b0 0x7ff0a019ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.485+0000 7ff0a5046640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a0110c50 con 0x7ff0a0101150 2024-08-18T23:53:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.485+0000 7ff0a5046640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a0110ad0 con 0x7ff0a01007b0 2024-08-18T23:53:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.485+0000 7ff0a5046640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a0110dd0 con 0x7ff0a01067b0 2024-08-18T23:53:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.485+0000 7ff09e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a0101150 0x7ff0a0197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.485+0000 7ff09e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a0101150 0x7ff0a0197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46138/0 (socket says 172.21.15.78:46138) 2024-08-18T23:53:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.485+0000 7ff09e59c640 1 -- 172.21.15.78:0/1474075627 learned_addr learned my addr 172.21.15.78:0/1474075627 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.486+0000 7ff09f59e640 1 --2- 172.21.15.78:0/1474075627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0a01067b0 0x7ff0a019ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.486+0000 7ff09e59c640 1 -- 172.21.15.78:0/1474075627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0a01067b0 msgr2=0x7ff0a019ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.486+0000 7ff09ed9d640 1 --2- 172.21.15.78:0/1474075627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0a01007b0 0x7ff0a01971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.486+0000 7ff09e59c640 1 --2- 172.21.15.78:0/1474075627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0a01067b0 0x7ff0a019ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.486+0000 7ff09e59c640 1 -- 172.21.15.78:0/1474075627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0a01007b0 msgr2=0x7ff0a01971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.486+0000 7ff09e59c640 1 --2- 172.21.15.78:0/1474075627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0a01007b0 0x7ff0a01971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.486+0000 7ff09e59c640 1 -- 172.21.15.78:0/1474075627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a019c170 con 0x7ff0a0101150 2024-08-18T23:53:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.486+0000 7ff09ed9d640 1 --2- 172.21.15.78:0/1474075627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0a01007b0 0x7ff0a01971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:53:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.486+0000 7ff09e59c640 1 --2- 172.21.15.78:0/1474075627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a0101150 0x7ff0a0197700 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff09400d990 tx=0x7ff09400de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.486+0000 7ff087fff640 1 -- 172.21.15.78:0/1474075627 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09401b070 con 0x7ff0a0101150 2024-08-18T23:53:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.486+0000 7ff087fff640 1 -- 172.21.15.78:0/1474075627 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff09400edf0 con 0x7ff0a0101150 2024-08-18T23:53:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.486+0000 7ff0a5046640 1 -- 172.21.15.78:0/1474075627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a019c460 con 0x7ff0a0101150 2024-08-18T23:53:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.486+0000 7ff087fff640 1 -- 172.21.15.78:0/1474075627 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094014870 con 0x7ff0a0101150 2024-08-18T23:53:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.486+0000 7ff0a5046640 1 -- 172.21.15.78:0/1474075627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a006a4c0 con 0x7ff0a0101150 2024-08-18T23:53:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.488+0000 7ff087fff640 1 -- 172.21.15.78:0/1474075627 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff09401e070 con 0x7ff0a0101150 2024-08-18T23:53:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.488+0000 7ff0a7444740 1 Processor -- start 2024-08-18T23:53:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff0a7444740 1 -- start start 2024-08-18T23:53:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff0a7444740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca703ff0 0x55d6ca7043d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff0a7444740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca704910 0x55d6ca70ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff0a7444740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca70f1d0 0x55d6ca7115c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff0a7444740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d6ca669a40 con 0x55d6ca704910 2024-08-18T23:53:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff0a7444740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d6ca6019d0 con 0x55d6ca703ff0 2024-08-18T23:53:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff0a7444740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d6ca665a00 con 0x55d6ca70f1d0 2024-08-18T23:53:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff09ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca704910 0x55d6ca70ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff09ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca704910 0x55d6ca70ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46150/0 (socket says 172.21.15.78:46150) 2024-08-18T23:53:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff087fff640 1 --2- 172.21.15.78:0/1474075627 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff0880a2b80 0x7ff0880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff09ed9d640 1 -- 172.21.15.78:0/2337549990 learned_addr learned my addr 172.21.15.78:0/2337549990 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff09f59e640 1 --2- 172.21.15.78:0/2337549990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca703ff0 0x55d6ca7043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff09f59e640 1 --2- 172.21.15.78:0/1474075627 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff0880a2b80 0x7ff0880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff09e59c640 1 --2- 172.21.15.78:0/2337549990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca70f1d0 0x55d6ca7115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff09ed9d640 1 -- 172.21.15.78:0/2337549990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca70f1d0 msgr2=0x55d6ca7115c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff087fff640 1 -- 172.21.15.78:0/1474075627 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7ff0940d0bb0 con 0x7ff0a0101150 2024-08-18T23:53:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff09ed9d640 1 --2- 172.21.15.78:0/2337549990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca70f1d0 0x55d6ca7115c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff09ed9d640 1 -- 172.21.15.78:0/2337549990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca703ff0 msgr2=0x55d6ca7043d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff09ed9d640 1 --2- 172.21.15.78:0/2337549990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca703ff0 0x55d6ca7043d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff09ed9d640 1 -- 172.21.15.78:0/2337549990 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6ca608650 con 0x55d6ca704910 2024-08-18T23:53:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.489+0000 7ff09e59c640 1 --2- 172.21.15.78:0/2337549990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca70f1d0 0x55d6ca7115c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:53:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.490+0000 7ff09f59e640 1 --2- 172.21.15.78:0/1474075627 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff0880a2b80 0x7ff0880a5040 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff0980070b0 tx=0x7ff0980093d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.490+0000 7ff09ed9d640 1 --2- 172.21.15.78:0/2337549990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca704910 0x55d6ca70ec90 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff090005ab0 tx=0x7ff090009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86794f8f72c9c376 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.490+0000 7ff0857fa640 1 -- 172.21.15.78:0/2337549990 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09003f070 con 0x55d6ca704910 2024-08-18T23:53:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.490+0000 7ff0857fa640 1 -- 172.21.15.78:0/2337549990 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0900317b0 con 0x55d6ca704910 2024-08-18T23:53:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.490+0000 7ff0857fa640 1 -- 172.21.15.78:0/2337549990 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090035070 con 0x55d6ca704910 2024-08-18T23:53:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.490+0000 7ff0a7444740 1 -- 172.21.15.78:0/2337549990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca704910 msgr2=0x55d6ca70ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.490+0000 7ff0a7444740 1 --2- 172.21.15.78:0/2337549990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca704910 0x55d6ca70ec90 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff090005ab0 tx=0x7ff090009680 comp rx=0 tx=0).stop 2024-08-18T23:53:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff0a7444740 1 -- 172.21.15.78:0/2337549990 shutdown_connections 2024-08-18T23:53:28.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff0a7444740 1 --2- 172.21.15.78:0/2337549990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca70f1d0 0x55d6ca7115c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff0a7444740 1 --2- 172.21.15.78:0/2337549990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca704910 0x55d6ca70ec90 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff0a7444740 1 --2- 172.21.15.78:0/2337549990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca703ff0 0x55d6ca7043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff0a7444740 1 -- 172.21.15.78:0/2337549990 >> 172.21.15.78:0/2337549990 conn(0x55d6ca700580 msgr2=0x55d6ca700960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff0a7444740 1 -- 172.21.15.78:0/2337549990 shutdown_connections 2024-08-18T23:53:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff0a7444740 1 -- 172.21.15.78:0/2337549990 wait complete. 2024-08-18T23:53:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff0a7444740 1 -- start start 2024-08-18T23:53:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff0a7444740 1 Processor -- start 2024-08-18T23:53:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff0a7444740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 0x55d6ca70ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff0a7444740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca704910 0x55d6ca70bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff0a7444740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca70f1d0 0x55d6ca70c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff0a7444740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d6ca711da0 con 0x55d6ca703ff0 2024-08-18T23:53:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff0a7444740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d6ca624b20 con 0x55d6ca704910 2024-08-18T23:53:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff0a7444740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d6ca711f20 con 0x55d6ca70f1d0 2024-08-18T23:53:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff0a7444740 10 client.0 _ll_register_callbacks cb 0x55d6ca622bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:53:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff09ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 0x55d6ca70ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff09ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 0x55d6ca70ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46164/0 (socket says 172.21.15.78:46164) 2024-08-18T23:53:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff09ed9d640 1 -- 172.21.15.78:0/62895837 learned_addr learned my addr 172.21.15.78:0/62895837 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.491+0000 7ff09e59c640 1 --2- 172.21.15.78:0/62895837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca704910 0x55d6ca70bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.492+0000 7ff09f59e640 1 --2- 172.21.15.78:0/62895837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca70f1d0 0x55d6ca70c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.492+0000 7ff09ed9d640 1 -- 172.21.15.78:0/62895837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca70f1d0 msgr2=0x55d6ca70c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.492+0000 7ff09ed9d640 1 --2- 172.21.15.78:0/62895837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca70f1d0 0x55d6ca70c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.492+0000 7ff09ed9d640 1 -- 172.21.15.78:0/62895837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca704910 msgr2=0x55d6ca70bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.492+0000 7ff09ed9d640 1 --2- 172.21.15.78:0/62895837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca704910 0x55d6ca70bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.492+0000 7ff09ed9d640 1 -- 172.21.15.78:0/62895837 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6ca3db3c0 con 0x55d6ca703ff0 2024-08-18T23:53:28.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.492+0000 7ff09e59c640 1 --2- 172.21.15.78:0/62895837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca704910 0x55d6ca70bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:53:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.492+0000 7ff09ed9d640 1 --2- 172.21.15.78:0/62895837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 0x55d6ca70ec80 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff090031550 tx=0x7ff090005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6d15c55fae1207 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.492+0000 7ff082ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:53:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.492+0000 7ff082ff5640 1 -- 172.21.15.78:0/62895837 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09003f050 con 0x55d6ca703ff0 2024-08-18T23:53:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.492+0000 7ff082ff5640 1 -- 172.21.15.78:0/62895837 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7ff090004040 con 0x55d6ca703ff0 2024-08-18T23:53:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.492+0000 7ff0a7444740 1 -- 172.21.15.78:0/62895837 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0900317b0 con 0x55d6ca703ff0 2024-08-18T23:53:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.492+0000 7ff082ff5640 1 -- 172.21.15.78:0/62895837 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff090056050 con 0x55d6ca703ff0 2024-08-18T23:53:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.492+0000 7ff0a7444740 10 client.17076 fetch_fsmap learned FSMap version 32 2024-08-18T23:53:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.492+0000 7ff0a7444740 1 -- 172.21.15.78:0/62895837 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d6ca71d840 con 0x55d6ca703ff0 2024-08-18T23:53:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.492+0000 7ff082ff5640 1 -- 172.21.15.78:0/62895837 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0900317b0 con 0x55d6ca703ff0 2024-08-18T23:53:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.493+0000 7ff082ff5640 1 -- 172.21.15.78:0/62895837 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff090004750 con 0x55d6ca703ff0 2024-08-18T23:53:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.493+0000 7ff0a7444740 10 client.17076 fetch_fsmap finished waiting for FSMap version 32 2024-08-18T23:53:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.493+0000 7ff0a7444740 10 client.17076 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:53:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.493+0000 7ff0a7444740 20 client.17076 populate_metadata read hostname 'smithi078' 2024-08-18T23:53:28.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.493+0000 7ff0a7444740 1 --2- 172.21.15.78:0/62895837 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55d6ca71eb90 0x55d6ca720f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:28.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.493+0000 7ff0a7444740 4 client.17076 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:53:28.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.493+0000 7ff0a7444740 1 -- 172.21.15.78:0/62895837 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6ca676fb0 con 0x55d6ca71eb90 2024-08-18T23:53:28.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.493+0000 7ff09e59c640 1 --2- 172.21.15.78:0/62895837 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55d6ca71eb90 0x55d6ca720f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:28.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.493+0000 7ff09e59c640 1 --2- 172.21.15.78:0/62895837 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55d6ca71eb90 0x55d6ca720f80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=463a28300f617c82 server_cookie=361735630307d266 in_seq=0 out_seq=0 2024-08-18T23:53:28.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.493+0000 7ff082ff5640 10 client.17076 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:53:28.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.494+0000 7ff082ff5640 1 -- 172.21.15.78:0/62895837 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d6ca676fb0 con 0x55d6ca71eb90 2024-08-18T23:53:28.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.494+0000 7ff082ff5640 10 client.17076 handle_command_reply: tid=0 2024-08-18T23:53:28.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.494+0000 7ff0a7444740 1 client.17076 shutdown 2024-08-18T23:53:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.494+0000 7ff0a7444740 1 -- 172.21.15.78:0/62895837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 msgr2=0x55d6ca70ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.494+0000 7ff0a7444740 1 --2- 172.21.15.78:0/62895837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 0x55d6ca70ec80 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff090031550 tx=0x7ff090005240 comp rx=0 tx=0).stop 2024-08-18T23:53:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.495+0000 7ff0a7444740 1 -- 172.21.15.78:0/62895837 shutdown_connections 2024-08-18T23:53:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.495+0000 7ff0a7444740 1 --2- 172.21.15.78:0/62895837 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55d6ca71eb90 0x55d6ca720f80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.495+0000 7ff0a7444740 1 --2- 172.21.15.78:0/62895837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca70f1d0 0x55d6ca70c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.495+0000 7ff0a7444740 1 --2- 172.21.15.78:0/62895837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca704910 0x55d6ca70bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.495+0000 7ff0a7444740 1 --2- 172.21.15.78:0/62895837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 0x55d6ca70ec80 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.495+0000 7ff082ff5640 0 client.17076 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:53:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.495+0000 7ff0a7444740 1 -- 172.21.15.78:0/62895837 >> 172.21.15.78:0/62895837 conn(0x55d6ca700580 msgr2=0x55d6ca7062b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.495+0000 7ff0a7444740 1 -- 172.21.15.78:0/62895837 shutdown_connections 2024-08-18T23:53:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.495+0000 7ff0a7444740 1 -- 172.21.15.78:0/62895837 wait complete. 2024-08-18T23:53:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.495+0000 7ff0a7444740 20 client.17076 trim_cache size 0 max 16384 2024-08-18T23:53:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.507+0000 7ff0a7444740 1 Processor -- start 2024-08-18T23:53:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.507+0000 7ff0a7444740 1 -- start start 2024-08-18T23:53:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.507+0000 7ff0a7444740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 0x55d6ca707310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.507+0000 7ff0a7444740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca704910 0x55d6ca707850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.507+0000 7ff0a7444740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca70f1d0 0x55d6ca726520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.508+0000 7ff0a7444740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a01af1f0 con 0x55d6ca703ff0 2024-08-18T23:53:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.508+0000 7ff0a7444740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a01aca60 con 0x55d6ca70f1d0 2024-08-18T23:53:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.508+0000 7ff0a7444740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a01aa8b0 con 0x55d6ca704910 2024-08-18T23:53:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.508+0000 7ff09ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 0x55d6ca707310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.508+0000 7ff09ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 0x55d6ca707310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46168/0 (socket says 172.21.15.78:46168) 2024-08-18T23:53:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.508+0000 7ff09ed9d640 1 -- 172.21.15.78:0/865617763 learned_addr learned my addr 172.21.15.78:0/865617763 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.508+0000 7ff09e59c640 1 --2- 172.21.15.78:0/865617763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca704910 0x55d6ca707850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:28.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.508+0000 7ff09f59e640 1 --2- 172.21.15.78:0/865617763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca70f1d0 0x55d6ca726520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:28.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.508+0000 7ff09ed9d640 1 -- 172.21.15.78:0/865617763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca704910 msgr2=0x55d6ca707850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:28.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.508+0000 7ff09ed9d640 1 --2- 172.21.15.78:0/865617763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca704910 0x55d6ca707850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.508+0000 7ff09ed9d640 1 -- 172.21.15.78:0/865617763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca70f1d0 msgr2=0x55d6ca726520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:28.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.508+0000 7ff09ed9d640 1 --2- 172.21.15.78:0/865617763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca70f1d0 0x55d6ca726520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.508+0000 7ff09ed9d640 1 -- 172.21.15.78:0/865617763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6ca701270 con 0x55d6ca703ff0 2024-08-18T23:53:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.508+0000 7ff09ed9d640 1 --2- 172.21.15.78:0/865617763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 0x55d6ca707310 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7ff090000c00 tx=0x7ff090005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfaf1bde03e68dd server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.508+0000 7ff084ff9640 1 -- 172.21.15.78:0/865617763 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09003f070 con 0x55d6ca703ff0 2024-08-18T23:53:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.508+0000 7ff084ff9640 1 -- 172.21.15.78:0/865617763 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff090048070 con 0x55d6ca703ff0 2024-08-18T23:53:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.508+0000 7ff084ff9640 1 -- 172.21.15.78:0/865617763 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090035040 con 0x55d6ca703ff0 2024-08-18T23:53:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.509+0000 7ff0a7444740 1 -- 172.21.15.78:0/865617763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 msgr2=0x55d6ca707310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.509+0000 7ff0a7444740 1 --2- 172.21.15.78:0/865617763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 0x55d6ca707310 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7ff090000c00 tx=0x7ff090005fd0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.509+0000 7ff0a7444740 1 -- 172.21.15.78:0/865617763 shutdown_connections 2024-08-18T23:53:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.509+0000 7ff0a7444740 1 --2- 172.21.15.78:0/865617763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca70f1d0 0x55d6ca726520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.509+0000 7ff0a7444740 1 --2- 172.21.15.78:0/865617763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca704910 0x55d6ca707850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.509+0000 7ff0a7444740 1 --2- 172.21.15.78:0/865617763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 0x55d6ca707310 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.509+0000 7ff0a7444740 1 -- 172.21.15.78:0/865617763 >> 172.21.15.78:0/865617763 conn(0x55d6ca700580 msgr2=0x55d6ca6fb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.509+0000 7ff0a7444740 1 -- 172.21.15.78:0/865617763 shutdown_connections 2024-08-18T23:53:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.509+0000 7ff0a7444740 1 -- 172.21.15.78:0/865617763 wait complete. 2024-08-18T23:53:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.509+0000 7ff0a7444740 1 -- start start 2024-08-18T23:53:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.509+0000 7ff0a7444740 1 Processor -- start 2024-08-18T23:53:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.509+0000 7ff0a7444740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 0x55d6ca7124f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.509+0000 7ff0a7444740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca704910 0x55d6ca712a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.509+0000 7ff0a7444740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca70f1d0 0x55d6ca7874e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.509+0000 7ff0a7444740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a01aaf20 con 0x55d6ca703ff0 2024-08-18T23:53:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.509+0000 7ff0a7444740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a01ac390 con 0x55d6ca70f1d0 2024-08-18T23:53:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.509+0000 7ff0a7444740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a01ab5f0 con 0x55d6ca704910 2024-08-18T23:53:28.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.509+0000 7ff0a7444740 10 client.0 _ll_register_callbacks cb 0x7ff098009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:53:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff09ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 0x55d6ca7124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff09ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 0x55d6ca7124f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46182/0 (socket says 172.21.15.78:46182) 2024-08-18T23:53:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff09ed9d640 1 -- 172.21.15.78:0/425520166 learned_addr learned my addr 172.21.15.78:0/425520166 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff09e59c640 1 --2- 172.21.15.78:0/425520166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca704910 0x55d6ca712a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff09f59e640 1 --2- 172.21.15.78:0/425520166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca70f1d0 0x55d6ca7874e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff09ed9d640 1 -- 172.21.15.78:0/425520166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca704910 msgr2=0x55d6ca712a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff09ed9d640 1 --2- 172.21.15.78:0/425520166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca704910 0x55d6ca712a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff09ed9d640 1 -- 172.21.15.78:0/425520166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca70f1d0 msgr2=0x55d6ca7874e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff09ed9d640 1 --2- 172.21.15.78:0/425520166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca70f1d0 0x55d6ca7874e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff09ed9d640 1 -- 172.21.15.78:0/425520166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6ca787a20 con 0x55d6ca703ff0 2024-08-18T23:53:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff09f59e640 1 --2- 172.21.15.78:0/425520166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca70f1d0 0x55d6ca7874e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:53:28.541 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:53:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff09ed9d640 1 --2- 172.21.15.78:0/425520166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 0x55d6ca7124f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff090031550 tx=0x7ff0900050f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ff9eeeb83a5cb7 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff082ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:53:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff082ff5640 1 -- 172.21.15.78:0/425520166 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09005f060 con 0x55d6ca703ff0 2024-08-18T23:53:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff082ff5640 1 -- 172.21.15.78:0/425520166 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7ff090004440 con 0x55d6ca703ff0 2024-08-18T23:53:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff0a7444740 1 -- 172.21.15.78:0/425520166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff090048070 con 0x55d6ca703ff0 2024-08-18T23:53:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff082ff5640 1 -- 172.21.15.78:0/425520166 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff090066050 con 0x55d6ca703ff0 2024-08-18T23:53:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff0a7444740 10 client.17082 fetch_fsmap learned FSMap version 32 2024-08-18T23:53:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff0a7444740 1 -- 172.21.15.78:0/425520166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d6ca788c80 con 0x55d6ca703ff0 2024-08-18T23:53:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.510+0000 7ff082ff5640 1 -- 172.21.15.78:0/425520166 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff090048070 con 0x55d6ca703ff0 2024-08-18T23:53:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff082ff5640 1 -- 172.21.15.78:0/425520166 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff090059070 con 0x55d6ca703ff0 2024-08-18T23:53:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff0a7444740 10 client.17082 fetch_fsmap finished waiting for FSMap version 32 2024-08-18T23:53:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff0a7444740 10 client.17082 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-18T23:53:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff0a7444740 20 client.17082 populate_metadata read hostname 'smithi078' 2024-08-18T23:53:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff0a7444740 1 --2- 172.21.15.78:0/425520166 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55d6ca789310 0x55d6ca78b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff0a7444740 4 client.17082 mds_command: new command op to 24428 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:53:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff0a7444740 1 -- 172.21.15.78:0/425520166 --> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d6ca6714c0 con 0x55d6ca789310 2024-08-18T23:53:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff09e59c640 1 --2- 172.21.15.78:0/425520166 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55d6ca789310 0x55d6ca78b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff082ff5640 10 client.17082 ms_handle_connect on v2:172.21.15.123:6836/1639958554 2024-08-18T23:53:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff09e59c640 1 --2- 172.21.15.78:0/425520166 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55d6ca789310 0x55d6ca78b730 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c3676990670ff01 server_cookie=44e678065a0b1101 in_seq=0 out_seq=0 2024-08-18T23:53:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff082ff5640 1 -- 172.21.15.78:0/425520166 <== mds.1 v2:172.21.15.123:6836/1639958554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d6ca6714c0 con 0x55d6ca789310 2024-08-18T23:53:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff082ff5640 10 client.17082 handle_command_reply: tid=0 2024-08-18T23:53:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff0a7444740 1 client.17082 shutdown 2024-08-18T23:53:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff0a7444740 1 -- 172.21.15.78:0/425520166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 msgr2=0x55d6ca7124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff0a7444740 1 --2- 172.21.15.78:0/425520166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 0x55d6ca7124f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff090031550 tx=0x7ff0900050f0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff0a7444740 1 -- 172.21.15.78:0/425520166 shutdown_connections 2024-08-18T23:53:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff0a7444740 1 --2- 172.21.15.78:0/425520166 >> [v2:172.21.15.123:6836/1639958554,v1:172.21.15.123:6837/1639958554] conn(0x55d6ca789310 0x55d6ca78b730 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff0a7444740 1 --2- 172.21.15.78:0/425520166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6ca70f1d0 0x55d6ca7874e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff082ff5640 0 client.17082 ms_handle_reset on v2:172.21.15.123:6836/1639958554 2024-08-18T23:53:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff0a7444740 1 --2- 172.21.15.78:0/425520166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d6ca704910 0x55d6ca712a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff0a7444740 1 --2- 172.21.15.78:0/425520166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d6ca703ff0 0x55d6ca7124f0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.511+0000 7ff0a7444740 1 -- 172.21.15.78:0/425520166 >> 172.21.15.78:0/425520166 conn(0x55d6ca700580 msgr2=0x55d6ca70a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.512+0000 7ff0a7444740 1 -- 172.21.15.78:0/425520166 shutdown_connections 2024-08-18T23:53:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.512+0000 7ff0a7444740 1 -- 172.21.15.78:0/425520166 wait complete. 2024-08-18T23:53:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.512+0000 7ff0a7444740 20 client.17082 trim_cache size 0 max 16384 2024-08-18T23:53:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.512+0000 7ff0a5046640 1 -- 172.21.15.78:0/1474075627 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff0880a2b80 msgr2=0x7ff0880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.512+0000 7ff0a5046640 1 --2- 172.21.15.78:0/1474075627 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff0880a2b80 0x7ff0880a5040 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff0980070b0 tx=0x7ff0980093d0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.512+0000 7ff0a5046640 1 -- 172.21.15.78:0/1474075627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a0101150 msgr2=0x7ff0a0197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.512+0000 7ff0a5046640 1 --2- 172.21.15.78:0/1474075627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a0101150 0x7ff0a0197700 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff09400d990 tx=0x7ff09400de50 comp rx=0 tx=0).stop 2024-08-18T23:53:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.512+0000 7ff0a5046640 1 -- 172.21.15.78:0/1474075627 shutdown_connections 2024-08-18T23:53:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.512+0000 7ff0a5046640 1 --2- 172.21.15.78:0/1474075627 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff0880a2b80 0x7ff0880a5040 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.512+0000 7ff0a5046640 1 --2- 172.21.15.78:0/1474075627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0a01067b0 0x7ff0a019ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.512+0000 7ff0a5046640 1 --2- 172.21.15.78:0/1474075627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0a0101150 0x7ff0a0197700 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.512+0000 7ff0a5046640 1 --2- 172.21.15.78:0/1474075627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0a01007b0 0x7ff0a01971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.512+0000 7ff0a5046640 1 -- 172.21.15.78:0/1474075627 >> 172.21.15.78:0/1474075627 conn(0x7ff0a00fc4d0 msgr2=0x7ff0a00fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:28.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.512+0000 7ff0a5046640 1 -- 172.21.15.78:0/1474075627 shutdown_connections 2024-08-18T23:53:28.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:28.512+0000 7ff0a5046640 1 -- 172.21.15.78:0/1474075627 wait complete. 2024-08-18T23:53:29.214 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-18T23:53:29.466 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:53:29.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:29 smithi078 ceph-mon[18440]: pgmap v625: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:53:29.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:29 smithi161 ceph-mon[22027]: pgmap v625: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:53:29.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:29 smithi123 ceph-mon[21808]: pgmap v625: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:53:29.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.941+0000 7f921c3a9640 1 -- 172.21.15.78:0/2198044039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92141067b0 msgr2=0x7f9214106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:29.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.941+0000 7f921c3a9640 1 --2- 172.21.15.78:0/2198044039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92141067b0 0x7f9214106b90 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f9210009930 tx=0x7f921002a420 comp rx=0 tx=0).stop 2024-08-18T23:53:29.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.941+0000 7f921c3a9640 1 -- 172.21.15.78:0/2198044039 shutdown_connections 2024-08-18T23:53:29.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.941+0000 7f921c3a9640 1 --2- 172.21.15.78:0/2198044039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9214101150 0x7f921410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.941+0000 7f921c3a9640 1 --2- 172.21.15.78:0/2198044039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92141007b0 0x7f9214100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.941+0000 7f921c3a9640 1 --2- 172.21.15.78:0/2198044039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92141067b0 0x7f9214106b90 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.941+0000 7f921c3a9640 1 -- 172.21.15.78:0/2198044039 >> 172.21.15.78:0/2198044039 conn(0x7f92140fc4d0 msgr2=0x7f92140fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:29.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.941+0000 7f921c3a9640 1 -- 172.21.15.78:0/2198044039 shutdown_connections 2024-08-18T23:53:29.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.941+0000 7f921c3a9640 1 -- 172.21.15.78:0/2198044039 wait complete. 2024-08-18T23:53:29.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.942+0000 7f921c3a9640 1 Processor -- start 2024-08-18T23:53:29.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.942+0000 7f921c3a9640 1 -- start start 2024-08-18T23:53:29.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.942+0000 7f921c3a9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92141007b0 0x7f9214198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:29.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.942+0000 7f921c3a9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9214101150 0x7f9214198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:29.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.942+0000 7f921c3a9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92141067b0 0x7f921419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:29.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.942+0000 7f921c3a9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9214111cd0 con 0x7f92141007b0 2024-08-18T23:53:29.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.942+0000 7f921c3a9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9214111b50 con 0x7f92141067b0 2024-08-18T23:53:29.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.942+0000 7f921c3a9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9214111e50 con 0x7f9214101150 2024-08-18T23:53:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.942+0000 7f921a145640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92141007b0 0x7f9214198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.942+0000 7f921a145640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92141007b0 0x7f9214198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46192/0 (socket says 172.21.15.78:46192) 2024-08-18T23:53:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.942+0000 7f921a145640 1 -- 172.21.15.78:0/2143682455 learned_addr learned my addr 172.21.15.78:0/2143682455 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.942+0000 7f9219944640 1 --2- 172.21.15.78:0/2143682455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9214101150 0x7f9214198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.942+0000 7f921a946640 1 --2- 172.21.15.78:0/2143682455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92141067b0 0x7f921419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.943+0000 7f921a145640 1 -- 172.21.15.78:0/2143682455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9214101150 msgr2=0x7f9214198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.943+0000 7f921a145640 1 --2- 172.21.15.78:0/2143682455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9214101150 0x7f9214198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.943+0000 7f921a145640 1 -- 172.21.15.78:0/2143682455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92141067b0 msgr2=0x7f921419ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.943+0000 7f921a145640 1 --2- 172.21.15.78:0/2143682455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92141067b0 0x7f921419ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.943+0000 7f921a145640 1 -- 172.21.15.78:0/2143682455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921419d0e0 con 0x7f92141007b0 2024-08-18T23:53:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.943+0000 7f921a946640 1 --2- 172.21.15.78:0/2143682455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92141067b0 0x7f921419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:53:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.943+0000 7f921a145640 1 --2- 172.21.15.78:0/2143682455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92141007b0 0x7f9214198130 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9210009230 tx=0x7f9210004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.943+0000 7f920b7fe640 1 -- 172.21.15.78:0/2143682455 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921003f030 con 0x7f92141007b0 2024-08-18T23:53:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.943+0000 7f920b7fe640 1 -- 172.21.15.78:0/2143682455 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9210003ad0 con 0x7f92141007b0 2024-08-18T23:53:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.943+0000 7f921c3a9640 1 -- 172.21.15.78:0/2143682455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f921419d370 con 0x7f92141007b0 2024-08-18T23:53:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.943+0000 7f920b7fe640 1 -- 172.21.15.78:0/2143682455 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210035070 con 0x7f92141007b0 2024-08-18T23:53:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.943+0000 7f921c3a9640 1 -- 172.21.15.78:0/2143682455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92141a4cb0 con 0x7f92141007b0 2024-08-18T23:53:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.945+0000 7f920b7fe640 1 -- 172.21.15.78:0/2143682455 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9210042030 con 0x7f92141007b0 2024-08-18T23:53:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.946+0000 7f921e7a7740 1 Processor -- start 2024-08-18T23:53:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.946+0000 7f921e7a7740 1 -- start start 2024-08-18T23:53:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.946+0000 7f921e7a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f140428ff0 0x55f1404293d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.946+0000 7f921e7a7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f140429910 0x55f140433c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.946+0000 7f921e7a7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1404341d0 0x55f1404365c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.946+0000 7f921e7a7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f14038ea40 con 0x55f1404341d0 2024-08-18T23:53:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.946+0000 7f921e7a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f1403269d0 con 0x55f140428ff0 2024-08-18T23:53:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.946+0000 7f921e7a7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f14038aa00 con 0x55f140429910 2024-08-18T23:53:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.946+0000 7f9219944640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1404341d0 0x55f1404365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.946+0000 7f9219944640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1404341d0 0x55f1404365c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46202/0 (socket says 172.21.15.78:46202) 2024-08-18T23:53:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.946+0000 7f9219944640 1 -- 172.21.15.78:0/669437879 learned_addr learned my addr 172.21.15.78:0/669437879 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.946+0000 7f920b7fe640 1 --2- 172.21.15.78:0/2143682455 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f92000a2b80 0x7f92000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.947+0000 7f921a946640 1 --2- 172.21.15.78:0/669437879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f140428ff0 0x55f1404293d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.947+0000 7f921a145640 1 --2- 172.21.15.78:0/669437879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f140429910 0x55f140433c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.947+0000 7f921a946640 1 --2- 172.21.15.78:0/2143682455 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f92000a2b80 0x7f92000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.947+0000 7f9219944640 1 -- 172.21.15.78:0/669437879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f140429910 msgr2=0x55f140433c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.947+0000 7f9219944640 1 --2- 172.21.15.78:0/669437879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f140429910 0x55f140433c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.947+0000 7f9219944640 1 -- 172.21.15.78:0/669437879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f140428ff0 msgr2=0x55f1404293d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.947+0000 7f9219944640 1 --2- 172.21.15.78:0/669437879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f140428ff0 0x55f1404293d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.947+0000 7f9219944640 1 -- 172.21.15.78:0/669437879 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f14032d650 con 0x55f1404341d0 2024-08-18T23:53:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.947+0000 7f921a946640 1 --2- 172.21.15.78:0/669437879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f140428ff0 0x55f1404293d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:53:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.947+0000 7f920b7fe640 1 -- 172.21.15.78:0/2143682455 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f92100ef910 con 0x7f92141007b0 2024-08-18T23:53:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.947+0000 7f9219944640 1 --2- 172.21.15.78:0/669437879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1404341d0 0x55f1404365c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f920400aca0 tx=0x7f920400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed235aa5a3b1a3c server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.947+0000 7f9208dea640 1 -- 172.21.15.78:0/669437879 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920401d070 con 0x55f1404341d0 2024-08-18T23:53:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.947+0000 7f921a946640 1 --2- 172.21.15.78:0/2143682455 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f92000a2b80 0x7f92000a5040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f920c009fd0 tx=0x7f920c009280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.947+0000 7f9208dea640 1 -- 172.21.15.78:0/669437879 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9204018b20 con 0x55f1404341d0 2024-08-18T23:53:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.947+0000 7f9208dea640 1 -- 172.21.15.78:0/669437879 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204010c20 con 0x55f1404341d0 2024-08-18T23:53:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.948+0000 7f921e7a7740 1 -- 172.21.15.78:0/669437879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1404341d0 msgr2=0x55f1404365c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.948+0000 7f921e7a7740 1 --2- 172.21.15.78:0/669437879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1404341d0 0x55f1404365c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f920400aca0 tx=0x7f920400f790 comp rx=0 tx=0).stop 2024-08-18T23:53:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.948+0000 7f921e7a7740 1 -- 172.21.15.78:0/669437879 shutdown_connections 2024-08-18T23:53:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.948+0000 7f921e7a7740 1 --2- 172.21.15.78:0/669437879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1404341d0 0x55f1404365c0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.948+0000 7f921e7a7740 1 --2- 172.21.15.78:0/669437879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f140429910 0x55f140433c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.948+0000 7f921e7a7740 1 --2- 172.21.15.78:0/669437879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f140428ff0 0x55f1404293d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.948+0000 7f921e7a7740 1 -- 172.21.15.78:0/669437879 >> 172.21.15.78:0/669437879 conn(0x55f140425580 msgr2=0x55f140425960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.948+0000 7f921e7a7740 1 -- 172.21.15.78:0/669437879 shutdown_connections 2024-08-18T23:53:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.948+0000 7f921e7a7740 1 -- 172.21.15.78:0/669437879 wait complete. 2024-08-18T23:53:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.948+0000 7f921e7a7740 1 -- start start 2024-08-18T23:53:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.948+0000 7f921e7a7740 1 Processor -- start 2024-08-18T23:53:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f921e7a7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f140428ff0 0x55f140431050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f921e7a7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f140429910 0x55f140431590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f921e7a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1404341d0 0x55f140431ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f921e7a7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f140436da0 con 0x55f140429910 2024-08-18T23:53:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f921e7a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f140349b20 con 0x55f1404341d0 2024-08-18T23:53:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f921e7a7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f140436f20 con 0x55f140428ff0 2024-08-18T23:53:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f921e7a7740 10 client.0 _ll_register_callbacks cb 0x55f140347bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:53:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f9219944640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f140429910 0x55f140431590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f9219944640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f140429910 0x55f140431590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46216/0 (socket says 172.21.15.78:46216) 2024-08-18T23:53:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f9219944640 1 -- 172.21.15.78:0/1589447202 learned_addr learned my addr 172.21.15.78:0/1589447202 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f921a946640 1 --2- 172.21.15.78:0/1589447202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1404341d0 0x55f140431ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f921a145640 1 --2- 172.21.15.78:0/1589447202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f140428ff0 0x55f140431050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f9219944640 1 -- 172.21.15.78:0/1589447202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f140428ff0 msgr2=0x55f140431050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f9219944640 1 --2- 172.21.15.78:0/1589447202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f140428ff0 0x55f140431050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f9219944640 1 -- 172.21.15.78:0/1589447202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1404341d0 msgr2=0x55f140431ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f9219944640 1 --2- 172.21.15.78:0/1589447202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1404341d0 0x55f140431ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f9219944640 1 -- 172.21.15.78:0/1589447202 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1401003c0 con 0x55f140429910 2024-08-18T23:53:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f9219944640 1 --2- 172.21.15.78:0/1589447202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f140429910 0x55f140431590 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f92040107e0 tx=0x7f9204010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473c08134f7d5e28 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f91fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:53:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f91fdffb640 1 -- 172.21.15.78:0/1589447202 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920401d050 con 0x55f140429910 2024-08-18T23:53:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f91fdffb640 1 -- 172.21.15.78:0/1589447202 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f9204010d30 con 0x55f140429910 2024-08-18T23:53:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.949+0000 7f921e7a7740 1 -- 172.21.15.78:0/1589447202 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9204018b20 con 0x55f140429910 2024-08-18T23:53:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.950+0000 7f91fdffb640 1 -- 172.21.15.78:0/1589447202 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9204032050 con 0x55f140429910 2024-08-18T23:53:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.950+0000 7f921e7a7740 10 client.17094 fetch_fsmap learned FSMap version 32 2024-08-18T23:53:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.950+0000 7f921e7a7740 1 -- 172.21.15.78:0/1589447202 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f140442950 con 0x55f140429910 2024-08-18T23:53:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.950+0000 7f91fdffb640 1 -- 172.21.15.78:0/1589447202 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9204018b20 con 0x55f140429910 2024-08-18T23:53:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.950+0000 7f91fdffb640 1 -- 172.21.15.78:0/1589447202 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f92040046c0 con 0x55f140429910 2024-08-18T23:53:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.950+0000 7f921e7a7740 10 client.17094 fetch_fsmap finished waiting for FSMap version 32 2024-08-18T23:53:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.950+0000 7f921e7a7740 10 client.17094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:53:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.950+0000 7f921e7a7740 20 client.17094 populate_metadata read hostname 'smithi078' 2024-08-18T23:53:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.950+0000 7f921e7a7740 1 --2- 172.21.15.78:0/1589447202 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f140443db0 0x55f1404461a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.950+0000 7f921e7a7740 4 client.17094 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-18T23:53:29.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.950+0000 7f921e7a7740 1 -- 172.21.15.78:0/1589447202 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f14039bfb0 con 0x55f140443db0 2024-08-18T23:53:29.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.950+0000 7f921a145640 1 --2- 172.21.15.78:0/1589447202 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f140443db0 0x55f1404461a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:29.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.951+0000 7f921a145640 1 --2- 172.21.15.78:0/1589447202 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f140443db0 0x55f1404461a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c706ebf412e6dde server_cookie=5af35bc8d251c6fe in_seq=0 out_seq=0 2024-08-18T23:53:29.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.951+0000 7f91fdffb640 10 client.17094 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:53:29.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.951+0000 7f91fdffb640 1 -- 172.21.15.78:0/1589447202 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f14039bfb0 con 0x55f140443db0 2024-08-18T23:53:29.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.951+0000 7f91fdffb640 10 client.17094 handle_command_reply: tid=0 2024-08-18T23:53:29.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.951+0000 7f921e7a7740 1 client.17094 shutdown 2024-08-18T23:53:29.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.952+0000 7f921e7a7740 1 -- 172.21.15.78:0/1589447202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f140429910 msgr2=0x55f140431590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:29.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.952+0000 7f921e7a7740 1 --2- 172.21.15.78:0/1589447202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f140429910 0x55f140431590 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f92040107e0 tx=0x7f9204010660 comp rx=0 tx=0).stop 2024-08-18T23:53:29.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.952+0000 7f921e7a7740 1 -- 172.21.15.78:0/1589447202 shutdown_connections 2024-08-18T23:53:29.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.952+0000 7f921e7a7740 1 --2- 172.21.15.78:0/1589447202 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f140443db0 0x55f1404461a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.952+0000 7f921e7a7740 1 --2- 172.21.15.78:0/1589447202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1404341d0 0x55f140431ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.952+0000 7f91fdffb640 0 client.17094 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:53:29.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.952+0000 7f921e7a7740 1 --2- 172.21.15.78:0/1589447202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f140429910 0x55f140431590 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.952+0000 7f921e7a7740 1 --2- 172.21.15.78:0/1589447202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f140428ff0 0x55f140431050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.952+0000 7f921e7a7740 1 -- 172.21.15.78:0/1589447202 >> 172.21.15.78:0/1589447202 conn(0x55f140425580 msgr2=0x55f14042b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:29.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.952+0000 7f921e7a7740 1 -- 172.21.15.78:0/1589447202 shutdown_connections 2024-08-18T23:53:29.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.952+0000 7f921e7a7740 1 -- 172.21.15.78:0/1589447202 wait complete. 2024-08-18T23:53:29.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.952+0000 7f921e7a7740 20 client.17094 trim_cache size 0 max 16384 2024-08-18T23:53:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.965+0000 7f921e7a7740 1 Processor -- start 2024-08-18T23:53:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.965+0000 7f921e7a7740 1 -- start start 2024-08-18T23:53:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.965+0000 7f921e7a7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f140428ff0 0x55f140430910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.965+0000 7f921e7a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f140429910 0x55f14042b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.965+0000 7f921e7a7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1404341d0 0x55f14044b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.965+0000 7f921e7a7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f92141b1320 con 0x55f140428ff0 2024-08-18T23:53:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.965+0000 7f921e7a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f92141aeb90 con 0x55f140429910 2024-08-18T23:53:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.965+0000 7f921e7a7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f92141ac9e0 con 0x55f1404341d0 2024-08-18T23:53:29.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.965+0000 7f921a145640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f140428ff0 0x55f140430910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:29.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.965+0000 7f921a145640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f140428ff0 0x55f140430910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46218/0 (socket says 172.21.15.78:46218) 2024-08-18T23:53:29.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.965+0000 7f921a145640 1 -- 172.21.15.78:0/1814890475 learned_addr learned my addr 172.21.15.78:0/1814890475 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:29.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.965+0000 7f9219944640 1 --2- 172.21.15.78:0/1814890475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f140429910 0x55f14042b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:29.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.966+0000 7f921a145640 1 -- 172.21.15.78:0/1814890475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1404341d0 msgr2=0x55f14044b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:29.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.966+0000 7f921a145640 1 --2- 172.21.15.78:0/1814890475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1404341d0 0x55f14044b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.966+0000 7f921a145640 1 -- 172.21.15.78:0/1814890475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f140429910 msgr2=0x55f14042b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:29.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.966+0000 7f921a145640 1 --2- 172.21.15.78:0/1814890475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f140429910 0x55f14042b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.966+0000 7f921a145640 1 -- 172.21.15.78:0/1814890475 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f140440000 con 0x55f140428ff0 2024-08-18T23:53:29.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.966+0000 7f9219944640 1 --2- 172.21.15.78:0/1814890475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f140429910 0x55f14042b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:53:29.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.966+0000 7f921a145640 1 --2- 172.21.15.78:0/1814890475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f140428ff0 0x55f140430910 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f9210009900 tx=0x7f921008ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fb8f37ab03207e server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:29.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.966+0000 7f91fffff640 1 -- 172.21.15.78:0/1814890475 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210049040 con 0x55f140428ff0 2024-08-18T23:53:29.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.966+0000 7f91fffff640 1 -- 172.21.15.78:0/1814890475 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92100bc480 con 0x55f140428ff0 2024-08-18T23:53:29.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.966+0000 7f91fffff640 1 -- 172.21.15.78:0/1814890475 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210003c80 con 0x55f140428ff0 2024-08-18T23:53:29.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.966+0000 7f921e7a7740 1 -- 172.21.15.78:0/1814890475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f140428ff0 msgr2=0x55f140430910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:29.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.966+0000 7f921e7a7740 1 --2- 172.21.15.78:0/1814890475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f140428ff0 0x55f140430910 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f9210009900 tx=0x7f921008ee40 comp rx=0 tx=0).stop 2024-08-18T23:53:29.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.967+0000 7f921e7a7740 1 -- 172.21.15.78:0/1814890475 shutdown_connections 2024-08-18T23:53:29.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.967+0000 7f921e7a7740 1 --2- 172.21.15.78:0/1814890475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f1404341d0 0x55f14044b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.967+0000 7f921e7a7740 1 --2- 172.21.15.78:0/1814890475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f140429910 0x55f14042b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.967+0000 7f921e7a7740 1 --2- 172.21.15.78:0/1814890475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f140428ff0 0x55f140430910 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.967+0000 7f921e7a7740 1 -- 172.21.15.78:0/1814890475 >> 172.21.15.78:0/1814890475 conn(0x55f140425580 msgr2=0x55f14042aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:29.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.967+0000 7f921e7a7740 1 -- 172.21.15.78:0/1814890475 shutdown_connections 2024-08-18T23:53:29.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.967+0000 7f921e7a7740 1 -- 172.21.15.78:0/1814890475 wait complete. 2024-08-18T23:53:29.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.967+0000 7f921e7a7740 1 -- start start 2024-08-18T23:53:29.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.967+0000 7f921e7a7740 1 Processor -- start 2024-08-18T23:53:29.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.967+0000 7f921e7a7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f140428ff0 0x55f14043c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:29.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.967+0000 7f921e7a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f140429910 0x55f14043c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:29.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.967+0000 7f921e7a7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1404341d0 0x55f14042d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:29.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.967+0000 7f921e7a7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f92141ad050 con 0x55f1404341d0 2024-08-18T23:53:29.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.967+0000 7f921e7a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f92141ae4c0 con 0x55f140429910 2024-08-18T23:53:29.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.967+0000 7f921e7a7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f92141ad720 con 0x55f140428ff0 2024-08-18T23:53:29.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.967+0000 7f921e7a7740 10 client.0 _ll_register_callbacks cb 0x7f920401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-18T23:53:29.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.967+0000 7f921a946640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1404341d0 0x55f14042d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:29.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.968+0000 7f921a946640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1404341d0 0x55f14042d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46220/0 (socket says 172.21.15.78:46220) 2024-08-18T23:53:29.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.968+0000 7f921a946640 1 -- 172.21.15.78:0/3816061401 learned_addr learned my addr 172.21.15.78:0/3816061401 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:29.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.968+0000 7f921a145640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f140428ff0 0x55f14043c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:29.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.968+0000 7f9219944640 1 --2- 172.21.15.78:0/3816061401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f140429910 0x55f14043c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:29.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.968+0000 7f921a946640 1 -- 172.21.15.78:0/3816061401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f140428ff0 msgr2=0x55f14043c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:29.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.968+0000 7f921a946640 1 --2- 172.21.15.78:0/3816061401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f140428ff0 0x55f14043c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.968+0000 7f921a946640 1 -- 172.21.15.78:0/3816061401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f140429910 msgr2=0x55f14043c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-18T23:53:29.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.968+0000 7f921a946640 1 --2- 172.21.15.78:0/3816061401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f140429910 0x55f14043c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.968+0000 7f921a946640 1 -- 172.21.15.78:0/3816061401 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f14042d980 con 0x55f1404341d0 2024-08-18T23:53:29.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.968+0000 7f9219944640 1 --2- 172.21.15.78:0/3816061401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f140429910 0x55f14043c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-18T23:53:29.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.968+0000 7f921a946640 1 --2- 172.21.15.78:0/3816061401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1404341d0 0x55f14042d440 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f920c010940 tx=0x7f920c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f391541fc61af3a server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:29.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.968+0000 7f91fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-08-18T23:53:29.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.968+0000 7f91fdffb640 1 -- 172.21.15.78:0/3816061401 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c004b20 con 0x55f1404341d0 2024-08-18T23:53:29.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.968+0000 7f91fdffb640 1 -- 172.21.15.78:0/3816061401 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f920c004cc0 con 0x55f1404341d0 2024-08-18T23:53:29.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.968+0000 7f921e7a7740 1 -- 172.21.15.78:0/3816061401 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92100bc480 con 0x55f1404341d0 2024-08-18T23:53:29.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.968+0000 7f91fdffb640 1 -- 172.21.15.78:0/3816061401 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f920c020b10 con 0x55f1404341d0 2024-08-18T23:53:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.968+0000 7f921e7a7740 10 client.17100 fetch_fsmap learned FSMap version 32 2024-08-18T23:53:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.968+0000 7f921e7a7740 1 -- 172.21.15.78:0/3816061401 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f1404ade80 con 0x55f1404341d0 2024-08-18T23:53:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.969+0000 7f91fdffb640 1 -- 172.21.15.78:0/3816061401 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92100bc480 con 0x55f1404341d0 2024-08-18T23:53:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.969+0000 7f91fdffb640 1 -- 172.21.15.78:0/3816061401 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f920c027070 con 0x55f1404341d0 2024-08-18T23:53:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.969+0000 7f921e7a7740 10 client.17100 fetch_fsmap finished waiting for FSMap version 32 2024-08-18T23:53:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.969+0000 7f921e7a7740 10 client.17100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-18T23:53:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.969+0000 7f921e7a7740 20 client.17100 populate_metadata read hostname 'smithi078' 2024-08-18T23:53:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.969+0000 7f921e7a7740 1 --2- 172.21.15.78:0/3816061401 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f1404ae680 0x55f1404b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.969+0000 7f921e7a7740 4 client.17100 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-18T23:53:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.969+0000 7f921e7a7740 1 -- 172.21.15.78:0/3816061401 --> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f1403964c0 con 0x55f1404ae680 2024-08-18T23:53:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.969+0000 7f921a145640 1 --2- 172.21.15.78:0/3816061401 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f1404ae680 0x55f1404b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.969+0000 7f921a145640 1 --2- 172.21.15.78:0/3816061401 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f1404ae680 0x55f1404b0aa0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5af282f663557e8 server_cookie=948256631fe4057e in_seq=0 out_seq=0 2024-08-18T23:53:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.969+0000 7f91fdffb640 10 client.17100 ms_handle_connect on v2:172.21.15.123:6838/2656330956 2024-08-18T23:53:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f91fdffb640 1 -- 172.21.15.78:0/3816061401 <== mds.0 v2:172.21.15.123:6838/2656330956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1403964c0 con 0x55f1404ae680 2024-08-18T23:53:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f91fdffb640 10 client.17100 handle_command_reply: tid=0 2024-08-18T23:53:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921e7a7740 1 client.17100 shutdown 2024-08-18T23:53:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921e7a7740 1 -- 172.21.15.78:0/3816061401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1404341d0 msgr2=0x55f14042d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921e7a7740 1 --2- 172.21.15.78:0/3816061401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1404341d0 0x55f14042d440 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f920c010940 tx=0x7f920c009b20 comp rx=0 tx=0).stop 2024-08-18T23:53:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921e7a7740 1 -- 172.21.15.78:0/3816061401 shutdown_connections 2024-08-18T23:53:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921e7a7740 1 --2- 172.21.15.78:0/3816061401 >> [v2:172.21.15.123:6838/2656330956,v1:172.21.15.123:6839/2656330956] conn(0x55f1404ae680 0x55f1404b0aa0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921e7a7740 1 --2- 172.21.15.78:0/3816061401 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f1404341d0 0x55f14042d440 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921e7a7740 1 --2- 172.21.15.78:0/3816061401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f140429910 0x55f14043c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921e7a7740 1 --2- 172.21.15.78:0/3816061401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f140428ff0 0x55f14043c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921e7a7740 1 -- 172.21.15.78:0/3816061401 >> 172.21.15.78:0/3816061401 conn(0x55f140425580 msgr2=0x55f14043f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:29.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f91fdffb640 0 client.17100 ms_handle_reset on v2:172.21.15.123:6838/2656330956 2024-08-18T23:53:29.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921e7a7740 1 -- 172.21.15.78:0/3816061401 shutdown_connections 2024-08-18T23:53:29.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921e7a7740 1 -- 172.21.15.78:0/3816061401 wait complete. 2024-08-18T23:53:29.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921e7a7740 20 client.17100 trim_cache size 0 max 16384 2024-08-18T23:53:29.986 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-08-18T23:53:29.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921c3a9640 1 -- 172.21.15.78:0/2143682455 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f92000a2b80 msgr2=0x7f92000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:29.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921c3a9640 1 --2- 172.21.15.78:0/2143682455 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f92000a2b80 0x7f92000a5040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f920c009fd0 tx=0x7f920c009280 comp rx=0 tx=0).stop 2024-08-18T23:53:29.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921c3a9640 1 -- 172.21.15.78:0/2143682455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92141007b0 msgr2=0x7f9214198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:29.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921c3a9640 1 --2- 172.21.15.78:0/2143682455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92141007b0 0x7f9214198130 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9210009230 tx=0x7f9210004270 comp rx=0 tx=0).stop 2024-08-18T23:53:29.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921c3a9640 1 -- 172.21.15.78:0/2143682455 shutdown_connections 2024-08-18T23:53:29.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921c3a9640 1 --2- 172.21.15.78:0/2143682455 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f92000a2b80 0x7f92000a5040 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921c3a9640 1 --2- 172.21.15.78:0/2143682455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92141067b0 0x7f921419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921c3a9640 1 --2- 172.21.15.78:0/2143682455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9214101150 0x7f9214198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921c3a9640 1 --2- 172.21.15.78:0/2143682455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92141007b0 0x7f9214198130 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:29.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.970+0000 7f921c3a9640 1 -- 172.21.15.78:0/2143682455 >> 172.21.15.78:0/2143682455 conn(0x7f92140fc4d0 msgr2=0x7f92140fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:29.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.971+0000 7f921c3a9640 1 -- 172.21.15.78:0/2143682455 shutdown_connections 2024-08-18T23:53:29.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:29.971+0000 7f921c3a9640 1 -- 172.21.15.78:0/2143682455 wait complete. 2024-08-18T23:53:30.727 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_78215f5a1cf5eaf321d506a66539942e12851f34/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-08-18T23:53:30.728 INFO:tasks.fwd_scrub:done joining 2024-08-18T23:53:30.728 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-08-18T23:53:30.738 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-18T23:53:30.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.838+0000 7f6aadd81640 1 -- 172.21.15.78:0/998438033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aa8103410 msgr2=0x7f6aa8109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:30.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.838+0000 7f6aadd81640 1 --2- 172.21.15.78:0/998438033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aa8103410 0x7f6aa8109ca0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009ca0 tx=0x7f6aa002a8c0 comp rx=0 tx=0).stop 2024-08-18T23:53:30.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.838+0000 7f6aadd81640 1 -- 172.21.15.78:0/998438033 shutdown_connections 2024-08-18T23:53:30.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.838+0000 7f6aadd81640 1 --2- 172.21.15.78:0/998438033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aa8103410 0x7f6aa8109ca0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:30.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.838+0000 7f6aadd81640 1 --2- 172.21.15.78:0/998438033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa8102a50 0x7f6aa8102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:30.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.838+0000 7f6aadd81640 1 --2- 172.21.15.78:0/998438033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa8101860 0x7f6aa8101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:30.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.838+0000 7f6aadd81640 1 -- 172.21.15.78:0/998438033 >> 172.21.15.78:0/998438033 conn(0x7f6aa80fcfe0 msgr2=0x7f6aa80ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:30.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.838+0000 7f6aadd81640 1 -- 172.21.15.78:0/998438033 shutdown_connections 2024-08-18T23:53:30.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.838+0000 7f6aadd81640 1 -- 172.21.15.78:0/998438033 wait complete. 2024-08-18T23:53:30.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6aadd81640 1 Processor -- start 2024-08-18T23:53:30.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6aadd81640 1 -- start start 2024-08-18T23:53:30.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6aadd81640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa8101860 0x7f6aa8197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:30.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6aadd81640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aa8102a50 0x7f6aa81980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:30.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6aadd81640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa8103410 0x7f6aa819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:30.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6aadd81640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa810d4e0 con 0x7f6aa8102a50 2024-08-18T23:53:30.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6aadd81640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa810d360 con 0x7f6aa8101860 2024-08-18T23:53:30.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6aadd81640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa810d660 con 0x7f6aa8103410 2024-08-18T23:53:30.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6a9ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aa8102a50 0x7f6aa81980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:30.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6a9ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aa8102a50 0x7f6aa81980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46234/0 (socket says 172.21.15.78:46234) 2024-08-18T23:53:30.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6a9ffff640 1 -- 172.21.15.78:0/751456616 learned_addr learned my addr 172.21.15.78:0/751456616 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:30.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6aa7fff640 1 --2- 172.21.15.78:0/751456616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa8103410 0x7f6aa819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:30.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6aa77fe640 1 --2- 172.21.15.78:0/751456616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa8101860 0x7f6aa8197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:30.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6a9ffff640 1 -- 172.21.15.78:0/751456616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa8103410 msgr2=0x7f6aa819f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:30.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6a9ffff640 1 --2- 172.21.15.78:0/751456616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa8103410 0x7f6aa819f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:30.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6a9ffff640 1 -- 172.21.15.78:0/751456616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa8101860 msgr2=0x7f6aa8197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:30.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6a9ffff640 1 --2- 172.21.15.78:0/751456616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa8101860 0x7f6aa8197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:30.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6a9ffff640 1 -- 172.21.15.78:0/751456616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa819f780 con 0x7f6aa8102a50 2024-08-18T23:53:30.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6aa7fff640 1 --2- 172.21.15.78:0/751456616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa8103410 0x7f6aa819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:53:30.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.839+0000 7f6aa77fe640 1 --2- 172.21.15.78:0/751456616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa8101860 0x7f6aa8197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:53:30.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.840+0000 7f6a9ffff640 1 --2- 172.21.15.78:0/751456616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aa8102a50 0x7f6aa81980a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000d950 tx=0x7f6a9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:30.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.840+0000 7f6aa57fa640 1 -- 172.21.15.78:0/751456616 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90017030 con 0x7f6aa8102a50 2024-08-18T23:53:30.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.840+0000 7f6aa57fa640 1 -- 172.21.15.78:0/751456616 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a90004020 con 0x7f6aa8102a50 2024-08-18T23:53:30.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.840+0000 7f6aadd81640 1 -- 172.21.15.78:0/751456616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa819fa70 con 0x7f6aa8102a50 2024-08-18T23:53:30.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.840+0000 7f6aa57fa640 1 -- 172.21.15.78:0/751456616 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a900047a0 con 0x7f6aa8102a50 2024-08-18T23:53:30.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.840+0000 7f6aadd81640 1 -- 172.21.15.78:0/751456616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa819ffc0 con 0x7f6aa8102a50 2024-08-18T23:53:30.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.841+0000 7f6aadd81640 1 -- 172.21.15.78:0/751456616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa8108ba0 con 0x7f6aa8102a50 2024-08-18T23:53:30.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.842+0000 7f6aa57fa640 1 -- 172.21.15.78:0/751456616 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6a900041d0 con 0x7f6aa8102a50 2024-08-18T23:53:30.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.842+0000 7f6aa57fa640 1 --2- 172.21.15.78:0/751456616 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6a940a2b90 0x7f6a940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:30.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.843+0000 7f6aa77fe640 1 --2- 172.21.15.78:0/751456616 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6a940a2b90 0x7f6a940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:30.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.843+0000 7f6aa57fa640 1 -- 172.21.15.78:0/751456616 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f6a900cee80 con 0x7f6aa8102a50 2024-08-18T23:53:30.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.843+0000 7f6aa77fe640 1 --2- 172.21.15.78:0/751456616 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6a940a2b90 0x7f6a940a5050 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6aa81028b0 tx=0x7f6a98005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:30.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:30.847+0000 7f6aa57fa640 1 -- 172.21.15.78:0/751456616 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6a9009ba50 con 0x7f6aa8102a50 2024-08-18T23:53:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.015+0000 7f6aadd81640 1 -- 172.21.15.78:0/751456616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6aa8062b20 con 0x7f6aa8102a50 2024-08-18T23:53:31.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.016+0000 7f6aa57fa640 1 -- 172.21.15.78:0/751456616 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24747 (secure 0 0 0) 0x7f6a9009b350 con 0x7f6aa8102a50 2024-08-18T23:53:31.018 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:53:31.018 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":92,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","created":"2024-08-18T23:34:33.867246+0000","modified":"2024-08-18T23:43:24.291494+0000","last_up_change":"2024-08-18T23:40:42.200381+0000","last_in_change":"2024-08-18T23:40:22.425813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T23:37:22.338211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T23:41:28.920802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T23:41:31.026094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T23:41:37.403238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4de1c78-6ebc-49a9-b9c5-85d43535fab7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6803","nonce":3639645311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6805","nonce":3639645311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6809","nonce":3639645311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6807","nonce":3639645311}]},"public_addr":"172.21.15.78:6803/3639645311","cluster_addr":"172.21.15.78:6805/3639645311","heartbeat_back_addr":"172.21.15.78:6809/3639645311","heartbeat_front_addr":"172.21.15.78:6807/3639645311","state":["exists","up"]},{"osd":1,"uuid":"4f8e0c3d-472b-4230-ad23-2b0a34861251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6801","nonce":4199446368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6803","nonce":4199446368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6807","nonce":4199446368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6805","nonce":4199446368}]},"public_addr":"172.21.15.123:6801/4199446368","cluster_addr":"172.21.15.123:6803/4199446368","heartbeat_back_addr":"172.21.15.123:6807/4199446368","heartbeat_front_addr":"172.21.15.123:6805/4199446368","state":["exists","up"]},{"osd":2,"uuid":"27ef3314-bb6f-413d-be05-e10a105dec9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6801","nonce":2582810057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6803","nonce":2582810057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6807","nonce":2582810057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6805","nonce":2582810057}]},"public_addr":"172.21.15.161:6801/2582810057","cluster_addr":"172.21.15.161:6803/2582810057","heartbeat_back_addr":"172.21.15.161:6807/2582810057","heartbeat_front_addr":"172.21.15.161:6805/2582810057","state":["exists","up"]},{"osd":3,"uuid":"e9e52662-5ef9-4d70-bdd7-2b92165822ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6811","nonce":2697347521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6813","nonce":2697347521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6817","nonce":2697347521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6815","nonce":2697347521}]},"public_addr":"172.21.15.78:6811/2697347521","cluster_addr":"172.21.15.78:6813/2697347521","heartbeat_back_addr":"172.21.15.78:6817/2697347521","heartbeat_front_addr":"172.21.15.78:6815/2697347521","state":["exists","up"]},{"osd":4,"uuid":"5ed26775-79e3-4b24-9557-69fcf5e8e824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6809","nonce":1281511569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6811","nonce":1281511569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6815","nonce":1281511569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6813","nonce":1281511569}]},"public_addr":"172.21.15.123:6809/1281511569","cluster_addr":"172.21.15.123:6811/1281511569","heartbeat_back_addr":"172.21.15.123:6815/1281511569","heartbeat_front_addr":"172.21.15.123:6813/1281511569","state":["exists","up"]},{"osd":5,"uuid":"a9793b1d-5613-48a4-938e-adb718f4fa4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6809","nonce":2751796117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6811","nonce":2751796117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6815","nonce":2751796117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6813","nonce":2751796117}]},"public_addr":"172.21.15.161:6809/2751796117","cluster_addr":"172.21.15.161:6811/2751796117","heartbeat_back_addr":"172.21.15.161:6815/2751796117","heartbeat_front_addr":"172.21.15.161:6813/2751796117","state":["exists","up"]},{"osd":6,"uuid":"5dbc5e9c-a14f-4b71-910a-1e6d4827bb32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6819","nonce":95383223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6821","nonce":95383223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6825","nonce":95383223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6823","nonce":95383223}]},"public_addr":"172.21.15.78:6819/95383223","cluster_addr":"172.21.15.78:6821/95383223","heartbeat_back_addr":"172.21.15.78:6825/95383223","heartbeat_front_addr":"172.21.15.78:6823/95383223","state":["exists","up"]},{"osd":7,"uuid":"26fb7b69-131f-42ee-b745-6fe8b4f65391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6817","nonce":3440305156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6819","nonce":3440305156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6823","nonce":3440305156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6821","nonce":3440305156}]},"public_addr":"172.21.15.123:6817/3440305156","cluster_addr":"172.21.15.123:6819/3440305156","heartbeat_back_addr":"172.21.15.123:6823/3440305156","heartbeat_front_addr":"172.21.15.123:6821/3440305156","state":["exists","up"]},{"osd":8,"uuid":"e267bc50-0c29-4d7b-b368-b1621e29b2df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6817","nonce":1485008778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6819","nonce":1485008778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6823","nonce":1485008778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6821","nonce":1485008778}]},"public_addr":"172.21.15.161:6817/1485008778","cluster_addr":"172.21.15.161:6819/1485008778","heartbeat_back_addr":"172.21.15.161:6823/1485008778","heartbeat_front_addr":"172.21.15.161:6821/1485008778","state":["exists","up"]},{"osd":9,"uuid":"b7654818-b767-4fe6-8211-346297895f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6827","nonce":2884475661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6829","nonce":2884475661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6833","nonce":2884475661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6831","nonce":2884475661}]},"public_addr":"172.21.15.78:6827/2884475661","cluster_addr":"172.21.15.78:6829/2884475661","heartbeat_back_addr":"172.21.15.78:6833/2884475661","heartbeat_front_addr":"172.21.15.78:6831/2884475661","state":["exists","up"]},{"osd":10,"uuid":"59aa32ff-510e-4b52-8170-08a34a0b6b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6825","nonce":122226286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6827","nonce":122226286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6831","nonce":122226286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6829","nonce":122226286}]},"public_addr":"172.21.15.123:6825/122226286","cluster_addr":"172.21.15.123:6827/122226286","heartbeat_back_addr":"172.21.15.123:6831/122226286","heartbeat_front_addr":"172.21.15.123:6829/122226286","state":["exists","up"]},{"osd":11,"uuid":"67bacab4-d078-4c5c-b4ff-9f6a8a00f507","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6825","nonce":2037560985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6827","nonce":2037560985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6831","nonce":2037560985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6829","nonce":2037560985}]},"public_addr":"172.21.15.161:6825/2037560985","cluster_addr":"172.21.15.161:6827/2037560985","heartbeat_back_addr":"172.21.15.161:6831/2037560985","heartbeat_front_addr":"172.21.15.161:6829/2037560985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:36.826044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:57.688421+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:18.297424+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:40.355215+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:02.167902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:23.968759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:46.254083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:08.522038+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:31.028119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:54.119807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:16.847034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:39.607886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6800/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3040499240":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6801/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:6800/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/757409165":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/2562072965":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/636184437":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/1570708294":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3137806628":"2024-08-19T23:34:59.021165+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T23:53:31.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.018+0000 7f6aadd81640 1 -- 172.21.15.78:0/751456616 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6a940a2b90 msgr2=0x7f6a940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:31.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.018+0000 7f6aadd81640 1 --2- 172.21.15.78:0/751456616 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6a940a2b90 0x7f6a940a5050 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6aa81028b0 tx=0x7f6a98005f90 comp rx=0 tx=0).stop 2024-08-18T23:53:31.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.018+0000 7f6aadd81640 1 -- 172.21.15.78:0/751456616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aa8102a50 msgr2=0x7f6aa81980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:31.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.018+0000 7f6aadd81640 1 --2- 172.21.15.78:0/751456616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aa8102a50 0x7f6aa81980a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000d950 tx=0x7f6a9000de10 comp rx=0 tx=0).stop 2024-08-18T23:53:31.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.018+0000 7f6aadd81640 1 -- 172.21.15.78:0/751456616 shutdown_connections 2024-08-18T23:53:31.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.018+0000 7f6aadd81640 1 --2- 172.21.15.78:0/751456616 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f6a940a2b90 0x7f6a940a5050 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.018+0000 7f6aadd81640 1 --2- 172.21.15.78:0/751456616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa8103410 0x7f6aa819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.018+0000 7f6aadd81640 1 --2- 172.21.15.78:0/751456616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6aa8102a50 0x7f6aa81980a0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.018+0000 7f6aadd81640 1 --2- 172.21.15.78:0/751456616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa8101860 0x7f6aa8197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.018+0000 7f6aadd81640 1 -- 172.21.15.78:0/751456616 >> 172.21.15.78:0/751456616 conn(0x7f6aa80fcfe0 msgr2=0x7f6aa8105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:31.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.019+0000 7f6aadd81640 1 -- 172.21.15.78:0/751456616 shutdown_connections 2024-08-18T23:53:31.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.019+0000 7f6aadd81640 1 -- 172.21.15.78:0/751456616 wait complete. 2024-08-18T23:53:31.031 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-18T23:37:22.338211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-18T23:41:28.920802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-18T23:41:31.026094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-08-18T23:41:37.403238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-18T23:53:31.032 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-18T23:53:31.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.133+0000 7f5d3b079640 1 -- 172.21.15.78:0/31436221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d34102a40 msgr2=0x7f5d34102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:31.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.133+0000 7f5d3b079640 1 --2- 172.21.15.78:0/31436221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d34102a40 0x7f5d34102ec0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009900 tx=0x7f5d2402a410 comp rx=0 tx=0).stop 2024-08-18T23:53:31.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.133+0000 7f5d3b079640 1 -- 172.21.15.78:0/31436221 shutdown_connections 2024-08-18T23:53:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.133+0000 7f5d3b079640 1 --2- 172.21.15.78:0/31436221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d34103400 0x7f5d34109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.133+0000 7f5d3b079640 1 --2- 172.21.15.78:0/31436221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d34102a40 0x7f5d34102ec0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.133+0000 7f5d3b079640 1 --2- 172.21.15.78:0/31436221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d34101850 0x7f5d34101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.133+0000 7f5d3b079640 1 -- 172.21.15.78:0/31436221 >> 172.21.15.78:0/31436221 conn(0x7f5d340fcfd0 msgr2=0x7f5d340ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.133+0000 7f5d3b079640 1 -- 172.21.15.78:0/31436221 shutdown_connections 2024-08-18T23:53:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.133+0000 7f5d3b079640 1 -- 172.21.15.78:0/31436221 wait complete. 2024-08-18T23:53:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d3b079640 1 Processor -- start 2024-08-18T23:53:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d3b079640 1 -- start start 2024-08-18T23:53:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d3b079640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d34101850 0x7f5d34197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d3b079640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d34102a40 0x7f5d34198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d3b079640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d34103400 0x7f5d3419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:31.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d3b079640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3410d4b0 con 0x7f5d34101850 2024-08-18T23:53:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d3b079640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3410d330 con 0x7f5d34103400 2024-08-18T23:53:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d3b079640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3410d630 con 0x7f5d34102a40 2024-08-18T23:53:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d38e15640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d34101850 0x7f5d34197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d38e15640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d34101850 0x7f5d34197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46258/0 (socket says 172.21.15.78:46258) 2024-08-18T23:53:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d38e15640 1 -- 172.21.15.78:0/2880303965 learned_addr learned my addr 172.21.15.78:0/2880303965 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d2bfff640 1 --2- 172.21.15.78:0/2880303965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d34102a40 0x7f5d34198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d39616640 1 --2- 172.21.15.78:0/2880303965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d34103400 0x7f5d3419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d38e15640 1 -- 172.21.15.78:0/2880303965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d34102a40 msgr2=0x7f5d34198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d38e15640 1 --2- 172.21.15.78:0/2880303965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d34102a40 0x7f5d34198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d38e15640 1 -- 172.21.15.78:0/2880303965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d34103400 msgr2=0x7f5d3419f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d38e15640 1 --2- 172.21.15.78:0/2880303965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d34103400 0x7f5d3419f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d38e15640 1 -- 172.21.15.78:0/2880303965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3419f7e0 con 0x7f5d34101850 2024-08-18T23:53:31.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d38e15640 1 --2- 172.21.15.78:0/2880303965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d34101850 0x7f5d34197bc0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000ea80 tx=0x7f5d3000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:31.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.134+0000 7f5d29ffb640 1 -- 172.21.15.78:0/2880303965 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3000ce40 con 0x7f5d34101850 2024-08-18T23:53:31.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.135+0000 7f5d29ffb640 1 -- 172.21.15.78:0/2880303965 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d30011950 con 0x7f5d34101850 2024-08-18T23:53:31.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.135+0000 7f5d3b079640 1 -- 172.21.15.78:0/2880303965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3419fad0 con 0x7f5d34101850 2024-08-18T23:53:31.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.135+0000 7f5d29ffb640 1 -- 172.21.15.78:0/2880303965 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3001ab80 con 0x7f5d34101850 2024-08-18T23:53:31.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.135+0000 7f5d3b079640 1 -- 172.21.15.78:0/2880303965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3419fff0 con 0x7f5d34101850 2024-08-18T23:53:31.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.136+0000 7f5d3b079640 1 -- 172.21.15.78:0/2880303965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d34108ce0 con 0x7f5d34101850 2024-08-18T23:53:31.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.137+0000 7f5d29ffb640 1 -- 172.21.15.78:0/2880303965 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5d30011b00 con 0x7f5d34101850 2024-08-18T23:53:31.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.137+0000 7f5d29ffb640 1 --2- 172.21.15.78:0/2880303965 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f5d1c0a2b90 0x7f5d1c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:31.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.137+0000 7f5d2bfff640 1 --2- 172.21.15.78:0/2880303965 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f5d1c0a2b90 0x7f5d1c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:31.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.138+0000 7f5d29ffb640 1 -- 172.21.15.78:0/2880303965 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f5d300cfb30 con 0x7f5d34101850 2024-08-18T23:53:31.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.138+0000 7f5d2bfff640 1 --2- 172.21.15.78:0/2880303965 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f5d1c0a2b90 0x7f5d1c0a5050 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5d24000c00 tx=0x7f5d240315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:31.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.141+0000 7f5d29ffb640 1 -- 172.21.15.78:0/2880303965 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5d30016030 con 0x7f5d34101850 2024-08-18T23:53:31.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.313+0000 7f5d3b079640 1 -- 172.21.15.78:0/2880303965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5d34062b20 con 0x7f5d34101850 2024-08-18T23:53:31.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.314+0000 7f5d29ffb640 1 -- 172.21.15.78:0/2880303965 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f5d3009c700 con 0x7f5d34101850 2024-08-18T23:53:31.315 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-08-18T23:53:31.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.316+0000 7f5d3b079640 1 -- 172.21.15.78:0/2880303965 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f5d1c0a2b90 msgr2=0x7f5d1c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:31.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.316+0000 7f5d3b079640 1 --2- 172.21.15.78:0/2880303965 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f5d1c0a2b90 0x7f5d1c0a5050 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5d24000c00 tx=0x7f5d240315d0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.316+0000 7f5d3b079640 1 -- 172.21.15.78:0/2880303965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d34101850 msgr2=0x7f5d34197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:31.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.316+0000 7f5d3b079640 1 --2- 172.21.15.78:0/2880303965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d34101850 0x7f5d34197bc0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000ea80 tx=0x7f5d3000ef40 comp rx=0 tx=0).stop 2024-08-18T23:53:31.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.316+0000 7f5d3b079640 1 -- 172.21.15.78:0/2880303965 shutdown_connections 2024-08-18T23:53:31.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.316+0000 7f5d3b079640 1 --2- 172.21.15.78:0/2880303965 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f5d1c0a2b90 0x7f5d1c0a5050 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.316+0000 7f5d3b079640 1 --2- 172.21.15.78:0/2880303965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d34103400 0x7f5d3419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.316+0000 7f5d3b079640 1 --2- 172.21.15.78:0/2880303965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d34102a40 0x7f5d34198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.316+0000 7f5d3b079640 1 --2- 172.21.15.78:0/2880303965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d34101850 0x7f5d34197bc0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.316+0000 7f5d3b079640 1 -- 172.21.15.78:0/2880303965 >> 172.21.15.78:0/2880303965 conn(0x7f5d340fcfd0 msgr2=0x7f5d34105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:31.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.316+0000 7f5d3b079640 1 -- 172.21.15.78:0/2880303965 shutdown_connections 2024-08-18T23:53:31.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.316+0000 7f5d3b079640 1 -- 172.21.15.78:0/2880303965 wait complete. 2024-08-18T23:53:31.328 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-18T23:53:31.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40df0d2640 1 -- 172.21.15.78:0/2892051256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40d8102310 msgr2=0x7f40d8108ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:31.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40df0d2640 1 --2- 172.21.15.78:0/2892051256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40d8102310 0x7f40d8108ba0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009ca0 tx=0x7f40d002a8c0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40df0d2640 1 -- 172.21.15.78:0/2892051256 shutdown_connections 2024-08-18T23:53:31.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40df0d2640 1 --2- 172.21.15.78:0/2892051256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40d8102310 0x7f40d8108ba0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40df0d2640 1 --2- 172.21.15.78:0/2892051256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d8101950 0x7f40d8101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40df0d2640 1 --2- 172.21.15.78:0/2892051256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40d8100760 0x7f40d8100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40df0d2640 1 -- 172.21.15.78:0/2892051256 >> 172.21.15.78:0/2892051256 conn(0x7f40d80fbec0 msgr2=0x7f40d80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:31.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40df0d2640 1 -- 172.21.15.78:0/2892051256 shutdown_connections 2024-08-18T23:53:31.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40df0d2640 1 -- 172.21.15.78:0/2892051256 wait complete. 2024-08-18T23:53:31.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40df0d2640 1 Processor -- start 2024-08-18T23:53:31.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40df0d2640 1 -- start start 2024-08-18T23:53:31.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40df0d2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40d8100760 0x7f40d8197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:31.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40df0d2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d8101950 0x7f40d8198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:31.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40df0d2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40d8102310 0x7f40d819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:31.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40df0d2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40d810d570 con 0x7f40d8102310 2024-08-18T23:53:31.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40df0d2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f40d810d3f0 con 0x7f40d8100760 2024-08-18T23:53:31.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40df0d2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f40d810d6f0 con 0x7f40d8101950 2024-08-18T23:53:31.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40dd66f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40d8102310 0x7f40d819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:31.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40dd66f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40d8102310 0x7f40d819f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46276/0 (socket says 172.21.15.78:46276) 2024-08-18T23:53:31.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.431+0000 7f40dd66f640 1 -- 172.21.15.78:0/3399938812 learned_addr learned my addr 172.21.15.78:0/3399938812 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:31.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.432+0000 7f40dce6e640 1 --2- 172.21.15.78:0/3399938812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40d8100760 0x7f40d8197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:31.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.432+0000 7f40d7fff640 1 --2- 172.21.15.78:0/3399938812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d8101950 0x7f40d8198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.432+0000 7f40dd66f640 1 -- 172.21.15.78:0/3399938812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d8101950 msgr2=0x7f40d8198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.432+0000 7f40dd66f640 1 --2- 172.21.15.78:0/3399938812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d8101950 0x7f40d8198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.432+0000 7f40dd66f640 1 -- 172.21.15.78:0/3399938812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40d8100760 msgr2=0x7f40d8197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.432+0000 7f40dd66f640 1 --2- 172.21.15.78:0/3399938812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40d8100760 0x7f40d8197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.432+0000 7f40dd66f640 1 -- 172.21.15.78:0/3399938812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d819f760 con 0x7f40d8102310 2024-08-18T23:53:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.432+0000 7f40dce6e640 1 --2- 172.21.15.78:0/3399938812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40d8100760 0x7f40d8197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:53:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.432+0000 7f40dd66f640 1 --2- 172.21.15.78:0/3399938812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40d8102310 0x7f40d819f0f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f40d00098d0 tx=0x7f40d002f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.432+0000 7f40d5ffb640 1 -- 172.21.15.78:0/3399938812 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d002fc30 con 0x7f40d8102310 2024-08-18T23:53:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.432+0000 7f40df0d2640 1 -- 172.21.15.78:0/3399938812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d819f9f0 con 0x7f40d8102310 2024-08-18T23:53:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.432+0000 7f40d5ffb640 1 -- 172.21.15.78:0/3399938812 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40d0003e50 con 0x7f40d8102310 2024-08-18T23:53:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.432+0000 7f40d5ffb640 1 -- 172.21.15.78:0/3399938812 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d00305c0 con 0x7f40d8102310 2024-08-18T23:53:31.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.432+0000 7f40df0d2640 1 -- 172.21.15.78:0/3399938812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d819ffc0 con 0x7f40d8102310 2024-08-18T23:53:31.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.433+0000 7f40df0d2640 1 -- 172.21.15.78:0/3399938812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40d8107a80 con 0x7f40d8102310 2024-08-18T23:53:31.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.435+0000 7f40d5ffb640 1 -- 172.21.15.78:0/3399938812 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f40d0003970 con 0x7f40d8102310 2024-08-18T23:53:31.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.436+0000 7f40d5ffb640 1 --2- 172.21.15.78:0/3399938812 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f40c80a2b90 0x7f40c80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.436+0000 7f40d5ffb640 1 -- 172.21.15.78:0/3399938812 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f40d00f09e0 con 0x7f40d8102310 2024-08-18T23:53:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.436+0000 7f40dce6e640 1 --2- 172.21.15.78:0/3399938812 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f40c80a2b90 0x7f40c80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.436+0000 7f40dce6e640 1 --2- 172.21.15.78:0/3399938812 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f40c80a2b90 0x7f40c80a5050 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f40d81017b0 tx=0x7f40cc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:31.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.439+0000 7f40d5ffb640 1 -- 172.21.15.78:0/3399938812 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f40d00bd5b0 con 0x7f40d8102310 2024-08-18T23:53:31.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.607+0000 7f40df0d2640 1 -- 172.21.15.78:0/3399938812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f40d8062b20 con 0x7f40d8102310 2024-08-18T23:53:31.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.608+0000 7f40d5ffb640 1 -- 172.21.15.78:0/3399938812 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f40d00bceb0 con 0x7f40d8102310 2024-08-18T23:53:31.609 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-08-18T23:53:31.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.609+0000 7f40df0d2640 1 -- 172.21.15.78:0/3399938812 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f40c80a2b90 msgr2=0x7f40c80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:31.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.609+0000 7f40df0d2640 1 --2- 172.21.15.78:0/3399938812 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f40c80a2b90 0x7f40c80a5050 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f40d81017b0 tx=0x7f40cc008040 comp rx=0 tx=0).stop 2024-08-18T23:53:31.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.609+0000 7f40df0d2640 1 -- 172.21.15.78:0/3399938812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40d8102310 msgr2=0x7f40d819f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:31.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.609+0000 7f40df0d2640 1 --2- 172.21.15.78:0/3399938812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40d8102310 0x7f40d819f0f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f40d00098d0 tx=0x7f40d002f840 comp rx=0 tx=0).stop 2024-08-18T23:53:31.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.610+0000 7f40df0d2640 1 -- 172.21.15.78:0/3399938812 shutdown_connections 2024-08-18T23:53:31.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.610+0000 7f40df0d2640 1 --2- 172.21.15.78:0/3399938812 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f40c80a2b90 0x7f40c80a5050 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.610+0000 7f40df0d2640 1 --2- 172.21.15.78:0/3399938812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40d8102310 0x7f40d819f0f0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.610+0000 7f40df0d2640 1 --2- 172.21.15.78:0/3399938812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40d8101950 0x7f40d8198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.610+0000 7f40df0d2640 1 --2- 172.21.15.78:0/3399938812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40d8100760 0x7f40d8197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.610+0000 7f40df0d2640 1 -- 172.21.15.78:0/3399938812 >> 172.21.15.78:0/3399938812 conn(0x7f40d80fbec0 msgr2=0x7f40d8104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:31.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.610+0000 7f40df0d2640 1 -- 172.21.15.78:0/3399938812 shutdown_connections 2024-08-18T23:53:31.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.610+0000 7f40df0d2640 1 -- 172.21.15.78:0/3399938812 wait complete. 2024-08-18T23:53:31.621 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:31 smithi078 ceph-mon[18440]: pgmap v626: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-18T23:53:31.621 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:31 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/751456616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:53:31.621 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:31 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2880303965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-18T23:53:31.623 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-18T23:53:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:31 smithi161 ceph-mon[22027]: pgmap v626: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-18T23:53:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:31 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/751456616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:53:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:31 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2880303965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-18T23:53:31.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.726+0000 7f66b338a640 1 -- 172.21.15.78:0/201408436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac101920 msgr2=0x7f66ac101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:31.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.726+0000 7f66b338a640 1 --2- 172.21.15.78:0/201408436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac101920 0x7f66ac101da0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009900 tx=0x7f66a002a410 comp rx=0 tx=0).stop 2024-08-18T23:53:31.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.726+0000 7f66b338a640 1 -- 172.21.15.78:0/201408436 shutdown_connections 2024-08-18T23:53:31.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.726+0000 7f66b338a640 1 --2- 172.21.15.78:0/201408436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66ac1022e0 0x7f66ac108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.726+0000 7f66b338a640 1 --2- 172.21.15.78:0/201408436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac101920 0x7f66ac101da0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.726+0000 7f66b338a640 1 --2- 172.21.15.78:0/201408436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ac100730 0x7f66ac100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.726+0000 7f66b338a640 1 -- 172.21.15.78:0/201408436 >> 172.21.15.78:0/201408436 conn(0x7f66ac0fbeb0 msgr2=0x7f66ac0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:31.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.726+0000 7f66b338a640 1 -- 172.21.15.78:0/201408436 shutdown_connections 2024-08-18T23:53:31.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.726+0000 7f66b338a640 1 -- 172.21.15.78:0/201408436 wait complete. 2024-08-18T23:53:31.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.726+0000 7f66b338a640 1 Processor -- start 2024-08-18T23:53:31.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.726+0000 7f66b338a640 1 -- start start 2024-08-18T23:53:31.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f66b338a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ac100730 0x7f66ac1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:31.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f66b338a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac101920 0x7f66ac196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:31.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f66b338a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66ac1022e0 0x7f66ac19dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:31.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f66b338a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac10c390 con 0x7f66ac101920 2024-08-18T23:53:31.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f66b338a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac10c210 con 0x7f66ac1022e0 2024-08-18T23:53:31.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f66b338a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac10c510 con 0x7f66ac100730 2024-08-18T23:53:31.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f66b0925640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac101920 0x7f66ac196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:31.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f66b0925640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac101920 0x7f66ac196f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46298/0 (socket says 172.21.15.78:46298) 2024-08-18T23:53:31.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f66b0925640 1 -- 172.21.15.78:0/1131584900 learned_addr learned my addr 172.21.15.78:0/1131584900 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:31.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f66b1927640 1 --2- 172.21.15.78:0/1131584900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66ac1022e0 0x7f66ac19dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:31.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f66b1126640 1 --2- 172.21.15.78:0/1131584900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ac100730 0x7f66ac1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:31.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f66b0925640 1 -- 172.21.15.78:0/1131584900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ac100730 msgr2=0x7f66ac1969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:31.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f66b0925640 1 --2- 172.21.15.78:0/1131584900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ac100730 0x7f66ac1969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f66b0925640 1 -- 172.21.15.78:0/1131584900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66ac1022e0 msgr2=0x7f66ac19dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:31.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f66b0925640 1 --2- 172.21.15.78:0/1131584900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66ac1022e0 0x7f66ac19dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f66b0925640 1 -- 172.21.15.78:0/1131584900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ac19e610 con 0x7f66ac101920 2024-08-18T23:53:31.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f66b0925640 1 --2- 172.21.15.78:0/1131584900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac101920 0x7f66ac196f30 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009230 tx=0x7f66a0004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:31.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f669e7fc640 1 -- 172.21.15.78:0/1131584900 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a003f030 con 0x7f66ac101920 2024-08-18T23:53:31.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f669e7fc640 1 -- 172.21.15.78:0/1131584900 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66a0003ce0 con 0x7f66ac101920 2024-08-18T23:53:31.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f66b338a640 1 -- 172.21.15.78:0/1131584900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66ac19e8a0 con 0x7f66ac101920 2024-08-18T23:53:31.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f669e7fc640 1 -- 172.21.15.78:0/1131584900 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0035070 con 0x7f66ac101920 2024-08-18T23:53:31.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.727+0000 7f66b338a640 1 -- 172.21.15.78:0/1131584900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66ac19ed90 con 0x7f66ac101920 2024-08-18T23:53:31.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.728+0000 7f66b338a640 1 -- 172.21.15.78:0/1131584900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66ac107b40 con 0x7f66ac101920 2024-08-18T23:53:31.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.729+0000 7f669e7fc640 1 -- 172.21.15.78:0/1131584900 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f66a0042030 con 0x7f66ac101920 2024-08-18T23:53:31.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.730+0000 7f669e7fc640 1 --2- 172.21.15.78:0/1131584900 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f66980a2b90 0x7f66980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:31.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.730+0000 7f669e7fc640 1 -- 172.21.15.78:0/1131584900 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f66a00efed0 con 0x7f66ac101920 2024-08-18T23:53:31.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.730+0000 7f66b1126640 1 --2- 172.21.15.78:0/1131584900 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f66980a2b90 0x7f66980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:31.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.730+0000 7f66b1126640 1 --2- 172.21.15.78:0/1131584900 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f66980a2b90 0x7f66980a5050 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f66a80058c0 tx=0x7f66a8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:31.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.734+0000 7f669e7fc640 1 -- 172.21.15.78:0/1131584900 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f66a00bca40 con 0x7f66ac101920 2024-08-18T23:53:31.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:31 smithi123 ceph-mon[21808]: pgmap v626: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-18T23:53:31.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:31 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/751456616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:53:31.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:31 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2880303965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-18T23:53:31.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.902+0000 7f66b338a640 1 -- 172.21.15.78:0/1131584900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f66ac062b20 con 0x7f66ac101920 2024-08-18T23:53:31.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.902+0000 7f669e7fc640 1 -- 172.21.15.78:0/1131584900 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f66a00bc340 con 0x7f66ac101920 2024-08-18T23:53:31.903 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-08-18T23:53:31.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.904+0000 7f66b338a640 1 -- 172.21.15.78:0/1131584900 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f66980a2b90 msgr2=0x7f66980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.904+0000 7f66b338a640 1 --2- 172.21.15.78:0/1131584900 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f66980a2b90 0x7f66980a5050 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f66a80058c0 tx=0x7f66a8005850 comp rx=0 tx=0).stop 2024-08-18T23:53:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.904+0000 7f66b338a640 1 -- 172.21.15.78:0/1131584900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac101920 msgr2=0x7f66ac196f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.904+0000 7f66b338a640 1 --2- 172.21.15.78:0/1131584900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac101920 0x7f66ac196f30 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009230 tx=0x7f66a0004360 comp rx=0 tx=0).stop 2024-08-18T23:53:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.904+0000 7f66b338a640 1 -- 172.21.15.78:0/1131584900 shutdown_connections 2024-08-18T23:53:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.904+0000 7f66b338a640 1 --2- 172.21.15.78:0/1131584900 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f66980a2b90 0x7f66980a5050 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.904+0000 7f66b338a640 1 --2- 172.21.15.78:0/1131584900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66ac1022e0 0x7f66ac19dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.904+0000 7f66b338a640 1 --2- 172.21.15.78:0/1131584900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66ac101920 0x7f66ac196f30 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.904+0000 7f66b338a640 1 --2- 172.21.15.78:0/1131584900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ac100730 0x7f66ac1969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.904+0000 7f66b338a640 1 -- 172.21.15.78:0/1131584900 >> 172.21.15.78:0/1131584900 conn(0x7f66ac0fbeb0 msgr2=0x7f66ac104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.904+0000 7f66b338a640 1 -- 172.21.15.78:0/1131584900 shutdown_connections 2024-08-18T23:53:31.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:31.904+0000 7f66b338a640 1 -- 172.21.15.78:0/1131584900 wait complete. 2024-08-18T23:53:31.916 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-08-18T23:53:32.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.018+0000 7f0acb237640 1 -- 172.21.15.78:0/367308855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac4102a30 msgr2=0x7f0ac4102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:32.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.018+0000 7f0acb237640 1 --2- 172.21.15.78:0/367308855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac4102a30 0x7f0ac4102eb0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009900 tx=0x7f0ab002a410 comp rx=0 tx=0).stop 2024-08-18T23:53:32.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.018+0000 7f0acb237640 1 -- 172.21.15.78:0/367308855 shutdown_connections 2024-08-18T23:53:32.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.018+0000 7f0acb237640 1 --2- 172.21.15.78:0/367308855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ac41033f0 0x7f0ac4109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:32.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.018+0000 7f0acb237640 1 --2- 172.21.15.78:0/367308855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac4102a30 0x7f0ac4102eb0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:32.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.018+0000 7f0acb237640 1 --2- 172.21.15.78:0/367308855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac4101840 0x7f0ac4101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:32.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.018+0000 7f0acb237640 1 -- 172.21.15.78:0/367308855 >> 172.21.15.78:0/367308855 conn(0x7f0ac40fcfe0 msgr2=0x7f0ac40ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:32.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.018+0000 7f0acb237640 1 -- 172.21.15.78:0/367308855 shutdown_connections 2024-08-18T23:53:32.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.018+0000 7f0acb237640 1 -- 172.21.15.78:0/367308855 wait complete. 2024-08-18T23:53:32.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.018+0000 7f0acb237640 1 Processor -- start 2024-08-18T23:53:32.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.018+0000 7f0acb237640 1 -- start start 2024-08-18T23:53:32.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.018+0000 7f0acb237640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac4101840 0x7f0ac4075cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:32.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.018+0000 7f0acb237640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ac4102a30 0x7f0ac4074310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:32.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.018+0000 7f0acb237640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac41033f0 0x7f0ac4074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:32.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.018+0000 7f0acb237640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac410d4c0 con 0x7f0ac41033f0 2024-08-18T23:53:32.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.018+0000 7f0acb237640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac410d340 con 0x7f0ac4101840 2024-08-18T23:53:32.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.018+0000 7f0acb237640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac410d640 con 0x7f0ac4102a30 2024-08-18T23:53:32.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.019+0000 7f0ac97d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac41033f0 0x7f0ac4074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:32.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.019+0000 7f0ac97d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac41033f0 0x7f0ac4074850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46326/0 (socket says 172.21.15.78:46326) 2024-08-18T23:53:32.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.019+0000 7f0ac97d4640 1 -- 172.21.15.78:0/2038685969 learned_addr learned my addr 172.21.15.78:0/2038685969 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:32.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.019+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ac4102a30 0x7f0ac4074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:32.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.019+0000 7f0ac8fd3640 1 --2- 172.21.15.78:0/2038685969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac4101840 0x7f0ac4075cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:32.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.019+0000 7f0ac97d4640 1 -- 172.21.15.78:0/2038685969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ac4102a30 msgr2=0x7f0ac4074310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:32.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.019+0000 7f0ac97d4640 1 --2- 172.21.15.78:0/2038685969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ac4102a30 0x7f0ac4074310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:32.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.019+0000 7f0ac97d4640 1 -- 172.21.15.78:0/2038685969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac4101840 msgr2=0x7f0ac4075cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:32.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.019+0000 7f0ac97d4640 1 --2- 172.21.15.78:0/2038685969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac4101840 0x7f0ac4075cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:32.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.019+0000 7f0ac97d4640 1 -- 172.21.15.78:0/2038685969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac40750e0 con 0x7f0ac41033f0 2024-08-18T23:53:32.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.019+0000 7f0ac97d4640 1 --2- 172.21.15.78:0/2038685969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac41033f0 0x7f0ac4074850 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00cad0 tx=0x7f0abc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:32.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.019+0000 7f0ac1ffb640 1 -- 172.21.15.78:0/2038685969 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc00ae20 con 0x7f0ac41033f0 2024-08-18T23:53:32.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.019+0000 7f0ac1ffb640 1 -- 172.21.15.78:0/2038685969 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0abc005020 con 0x7f0ac41033f0 2024-08-18T23:53:32.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.019+0000 7f0ac1ffb640 1 -- 172.21.15.78:0/2038685969 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc00f6f0 con 0x7f0ac41033f0 2024-08-18T23:53:32.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.019+0000 7f0acb237640 1 -- 172.21.15.78:0/2038685969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac41a1c70 con 0x7f0ac41033f0 2024-08-18T23:53:32.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.019+0000 7f0acb237640 1 -- 172.21.15.78:0/2038685969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac41a2190 con 0x7f0ac41033f0 2024-08-18T23:53:32.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.020+0000 7f0acb237640 1 -- 172.21.15.78:0/2038685969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac4101c70 con 0x7f0ac41033f0 2024-08-18T23:53:32.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.021+0000 7f0ac1ffb640 1 -- 172.21.15.78:0/2038685969 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0abc0051d0 con 0x7f0ac41033f0 2024-08-18T23:53:32.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.022+0000 7f0ac1ffb640 1 --2- 172.21.15.78:0/2038685969 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0ab40a2b90 0x7f0ab40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:32.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.022+0000 7f0ac8fd3640 1 --2- 172.21.15.78:0/2038685969 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0ab40a2b90 0x7f0ab40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:32.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.023+0000 7f0ac1ffb640 1 -- 172.21.15.78:0/2038685969 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f0abc0cf770 con 0x7f0ac41033f0 2024-08-18T23:53:32.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.023+0000 7f0ac8fd3640 1 --2- 172.21.15.78:0/2038685969 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0ab40a2b90 0x7f0ab40a5050 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8005fd0 tx=0x7f0ab8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:32.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.027+0000 7f0ac1ffb640 1 -- 172.21.15.78:0/2038685969 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0ac4101c70 con 0x7f0ac41033f0 2024-08-18T23:53:32.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.195+0000 7f0acb237640 1 -- 172.21.15.78:0/2038685969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0ac4062b50 con 0x7f0ac41033f0 2024-08-18T23:53:32.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.196+0000 7f0ac1ffb640 1 -- 172.21.15.78:0/2038685969 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f0abc014030 con 0x7f0ac41033f0 2024-08-18T23:53:32.197 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-08-18T23:53:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.198+0000 7f0acb237640 1 -- 172.21.15.78:0/2038685969 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0ab40a2b90 msgr2=0x7f0ab40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.198+0000 7f0acb237640 1 --2- 172.21.15.78:0/2038685969 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0ab40a2b90 0x7f0ab40a5050 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8005fd0 tx=0x7f0ab8005850 comp rx=0 tx=0).stop 2024-08-18T23:53:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.198+0000 7f0acb237640 1 -- 172.21.15.78:0/2038685969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac41033f0 msgr2=0x7f0ac4074850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.198+0000 7f0acb237640 1 --2- 172.21.15.78:0/2038685969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac41033f0 0x7f0ac4074850 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00cad0 tx=0x7f0abc00cf90 comp rx=0 tx=0).stop 2024-08-18T23:53:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.198+0000 7f0acb237640 1 -- 172.21.15.78:0/2038685969 shutdown_connections 2024-08-18T23:53:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.198+0000 7f0acb237640 1 --2- 172.21.15.78:0/2038685969 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f0ab40a2b90 0x7f0ab40a5050 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:32.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.198+0000 7f0acb237640 1 --2- 172.21.15.78:0/2038685969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac41033f0 0x7f0ac4074850 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.198+0000 7f0acb237640 1 --2- 172.21.15.78:0/2038685969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ac4102a30 0x7f0ac4074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.198+0000 7f0acb237640 1 --2- 172.21.15.78:0/2038685969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ac4101840 0x7f0ac4075cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.198+0000 7f0acb237640 1 -- 172.21.15.78:0/2038685969 >> 172.21.15.78:0/2038685969 conn(0x7f0ac40fcfe0 msgr2=0x7f0ac4105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.198+0000 7f0acb237640 1 -- 172.21.15.78:0/2038685969 shutdown_connections 2024-08-18T23:53:32.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.198+0000 7f0acb237640 1 -- 172.21.15.78:0/2038685969 wait complete. 2024-08-18T23:53:32.211 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-08-18T23:53:32.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.309+0000 7f7e819f8640 1 -- 172.21.15.78:0/163611491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e7c102310 msgr2=0x7f7e7c108ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:32.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.309+0000 7f7e819f8640 1 --2- 172.21.15.78:0/163611491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e7c102310 0x7f7e7c108ba0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7e74009ca0 tx=0x7f7e7402a8c0 comp rx=0 tx=0).stop 2024-08-18T23:53:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.309+0000 7f7e819f8640 1 -- 172.21.15.78:0/163611491 shutdown_connections 2024-08-18T23:53:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.309+0000 7f7e819f8640 1 --2- 172.21.15.78:0/163611491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e7c102310 0x7f7e7c108ba0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.309+0000 7f7e819f8640 1 --2- 172.21.15.78:0/163611491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e7c101950 0x7f7e7c101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.309+0000 7f7e819f8640 1 --2- 172.21.15.78:0/163611491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e7c100760 0x7f7e7c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.309+0000 7f7e819f8640 1 -- 172.21.15.78:0/163611491 >> 172.21.15.78:0/163611491 conn(0x7f7e7c0fbec0 msgr2=0x7f7e7c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.309+0000 7f7e819f8640 1 -- 172.21.15.78:0/163611491 shutdown_connections 2024-08-18T23:53:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.309+0000 7f7e819f8640 1 -- 172.21.15.78:0/163611491 wait complete. 2024-08-18T23:53:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.309+0000 7f7e819f8640 1 Processor -- start 2024-08-18T23:53:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.309+0000 7f7e819f8640 1 -- start start 2024-08-18T23:53:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e819f8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e7c100760 0x7f7e7c10b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e819f8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e7c101950 0x7f7e7c109720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e819f8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e7c102310 0x7f7e7c109c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:32.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e819f8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c10d530 con 0x7f7e7c102310 2024-08-18T23:53:32.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e819f8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c10d3b0 con 0x7f7e7c101950 2024-08-18T23:53:32.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e819f8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c10d6b0 con 0x7f7e7c100760 2024-08-18T23:53:32.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e7b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e7c102310 0x7f7e7c109c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:32.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e7b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e7c102310 0x7f7e7c109c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46342/0 (socket says 172.21.15.78:46342) 2024-08-18T23:53:32.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e7b7fe640 1 -- 172.21.15.78:0/1751941981 learned_addr learned my addr 172.21.15.78:0/1751941981 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:32.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e7affd640 1 --2- 172.21.15.78:0/1751941981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e7c100760 0x7f7e7c10b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:32.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e7a7fc640 1 --2- 172.21.15.78:0/1751941981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e7c101950 0x7f7e7c109720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:32.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e7b7fe640 1 -- 172.21.15.78:0/1751941981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e7c100760 msgr2=0x7f7e7c10b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:32.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e7b7fe640 1 --2- 172.21.15.78:0/1751941981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e7c100760 0x7f7e7c10b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:32.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e7b7fe640 1 -- 172.21.15.78:0/1751941981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e7c101950 msgr2=0x7f7e7c109720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:32.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e7b7fe640 1 --2- 172.21.15.78:0/1751941981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e7c101950 0x7f7e7c109720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:32.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e7b7fe640 1 -- 172.21.15.78:0/1751941981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7c10a4f0 con 0x7f7e7c102310 2024-08-18T23:53:32.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e7a7fc640 1 --2- 172.21.15.78:0/1751941981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e7c101950 0x7f7e7c109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:53:32.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e7b7fe640 1 --2- 172.21.15.78:0/1751941981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e7c102310 0x7f7e7c109c60 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f7e74031ee0 tx=0x7f7e74031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:32.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e809f6640 1 -- 172.21.15.78:0/1751941981 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74003c30 con 0x7f7e7c102310 2024-08-18T23:53:32.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e819f8640 1 -- 172.21.15.78:0/1751941981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e7c1a1c40 con 0x7f7e7c102310 2024-08-18T23:53:32.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e809f6640 1 -- 172.21.15.78:0/1751941981 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e74003dd0 con 0x7f7e7c102310 2024-08-18T23:53:32.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e809f6640 1 -- 172.21.15.78:0/1751941981 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7402fc10 con 0x7f7e7c102310 2024-08-18T23:53:32.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.310+0000 7f7e819f8640 1 -- 172.21.15.78:0/1751941981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e7c1a2190 con 0x7f7e7c102310 2024-08-18T23:53:32.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.311+0000 7f7e819f8640 1 -- 172.21.15.78:0/1751941981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e7c107ad0 con 0x7f7e7c102310 2024-08-18T23:53:32.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.313+0000 7f7e809f6640 1 -- 172.21.15.78:0/1751941981 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7e740046c0 con 0x7f7e7c102310 2024-08-18T23:53:32.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.313+0000 7f7e809f6640 1 --2- 172.21.15.78:0/1751941981 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7e640a2b90 0x7f7e640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:32.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.313+0000 7f7e7affd640 1 --2- 172.21.15.78:0/1751941981 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7e640a2b90 0x7f7e640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:32.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.313+0000 7f7e809f6640 1 -- 172.21.15.78:0/1751941981 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f7e740f09f0 con 0x7f7e7c102310 2024-08-18T23:53:32.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.314+0000 7f7e7affd640 1 --2- 172.21.15.78:0/1751941981 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7e640a2b90 0x7f7e640a5050 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c005d00 tx=0x7f7e6c005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:32.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.316+0000 7f7e809f6640 1 -- 172.21.15.78:0/1751941981 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7e740bd650 con 0x7f7e7c102310 2024-08-18T23:53:32.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:32 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3399938812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-18T23:53:32.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:32 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1131584900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-18T23:53:32.470 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:32 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2038685969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-08-18T23:53:32.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.532+0000 7f7e819f8640 1 -- 172.21.15.78:0/1751941981 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f7e7c062b20 con 0x7f7e7c102310 2024-08-18T23:53:32.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.539+0000 7f7e809f6640 1 -- 172.21.15.78:0/1751941981 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740371 (secure 0 0 0) 0x7f7e740bcf50 con 0x7f7e7c102310 2024-08-18T23:53:32.550 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:53:32.550 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:53:32.551 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 20, 2024-08-18T23:53:32.551 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.551 INFO:teuthology.orchestra.run.smithi078.stdout: "active_gid": 14150, 2024-08-18T23:53:32.551 INFO:teuthology.orchestra.run.smithi078.stdout: "active_name": "x", 2024-08-18T23:53:32.551 INFO:teuthology.orchestra.run.smithi078.stdout: "active_addrs": { 2024-08-18T23:53:32.551 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-08-18T23:53:32.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.551 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-08-18T23:53:32.551 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:6800", 2024-08-18T23:53:32.551 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 4000744943 2024-08-18T23:53:32.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.552 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2024-08-18T23:53:32.552 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:6801", 2024-08-18T23:53:32.552 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 4000744943 2024-08-18T23:53:32.552 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.552 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-08-18T23:53:32.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.552 INFO:teuthology.orchestra.run.smithi078.stdout: "active_addr": "172.21.15.78:6801/4000744943", 2024-08-18T23:53:32.552 INFO:teuthology.orchestra.run.smithi078.stdout: "active_change": "2024-08-18T23:35:22.036194+0000", 2024-08-18T23:53:32.552 INFO:teuthology.orchestra.run.smithi078.stdout: "active_mgr_features": 4540701547738038271, 2024-08-18T23:53:32.552 INFO:teuthology.orchestra.run.smithi078.stdout: "available": true, 2024-08-18T23:53:32.552 INFO:teuthology.orchestra.run.smithi078.stdout: "standbys": [ 2024-08-18T23:53:32.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.553 INFO:teuthology.orchestra.run.smithi078.stdout: "gid": 24109, 2024-08-18T23:53:32.553 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "z", 2024-08-18T23:53:32.553 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr_features": 4540701547738038271, 2024-08-18T23:53:32.553 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-08-18T23:53:32.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.553 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-08-18T23:53:32.553 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.553 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.553 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.553 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-08-18T23:53:32.553 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-08-18T23:53:32.553 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:32.554 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.554 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.554 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-08-18T23:53:32.554 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.554 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.554 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.554 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-08-18T23:53:32.554 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.554 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.554 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.554 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.555 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.555 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.555 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.555 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.555 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.555 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.555 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.555 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.555 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.555 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.555 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.555 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.555 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.556 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.556 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.556 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.556 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.556 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.556 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.556 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.556 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.556 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.556 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.556 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.556 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.557 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.557 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.557 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.557 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.557 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.557 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.557 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.557 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.557 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.557 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.557 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.558 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.558 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.558 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.558 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.558 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.558 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.558 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.558 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.558 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.558 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.558 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.558 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.559 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.559 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.559 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.559 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.559 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.559 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.559 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.559 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.559 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.559 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.559 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.560 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.560 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.560 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.560 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.560 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.560 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.560 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-08-18T23:53:32.560 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-08-18T23:53:32.560 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.560 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.560 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.560 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.561 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.561 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.561 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.561 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to", 2024-08-18T23:53:32.561 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.561 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.561 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.561 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-08-18T23:53:32.561 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-08-18T23:53:32.561 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.561 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.561 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.562 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-08-18T23:53:32.562 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.562 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.562 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.562 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-08-18T23:53:32.562 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.562 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.562 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.562 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-08-18T23:53:32.562 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-08-18T23:53:32.562 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.562 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.563 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.563 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.563 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.563 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.563 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.563 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-08-18T23:53:32.563 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.563 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.563 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.563 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-08-18T23:53:32.563 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-08-18T23:53:32.564 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.564 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.564 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.564 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.564 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.564 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.564 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.564 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-08-18T23:53:32.564 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.564 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.564 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.564 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-08-18T23:53:32.564 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-08-18T23:53:32.565 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.565 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.565 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.565 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-08-18T23:53:32.565 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.565 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.565 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.565 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-08-18T23:53:32.565 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.565 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.565 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.565 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-08-18T23:53:32.566 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-08-18T23:53:32.566 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.566 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.566 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.566 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.566 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.566 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.566 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.566 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-08-18T23:53:32.566 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.566 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.566 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.567 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-08-18T23:53:32.567 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-08-18T23:53:32.567 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.567 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.567 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.567 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.567 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.567 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.567 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.567 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-18T23:53:32.567 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.568 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.568 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.568 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-08-18T23:53:32.568 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-08-18T23:53:32.568 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.568 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.568 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.568 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.568 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.568 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.568 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.568 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-08-18T23:53:32.569 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.569 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.569 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.569 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.569 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.569 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-08-18T23:53:32.569 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.569 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.569 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.569 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-08-18T23:53:32.569 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-08-18T23:53:32.569 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.570 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.570 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.570 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.570 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.570 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.570 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.570 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-08-18T23:53:32.570 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.570 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.570 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.570 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-08-18T23:53:32.571 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-08-18T23:53:32.571 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.571 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.571 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.571 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-08-18T23:53:32.571 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.571 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.571 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.571 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-08-18T23:53:32.571 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-18T23:53:32.571 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.571 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.572 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-08-18T23:53:32.572 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-08-18T23:53:32.572 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-08-18T23:53:32.572 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.572 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.572 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-08-18T23:53:32.572 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-08-18T23:53:32.572 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-08-18T23:53:32.572 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.572 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-18T23:53:32.572 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-18T23:53:32.572 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.572 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.573 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-08-18T23:53:32.573 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-08-18T23:53:32.573 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-08-18T23:53:32.573 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.573 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.573 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-08-18T23:53:32.573 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-08-18T23:53:32.573 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-08-18T23:53:32.573 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.573 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-18T23:53:32.573 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.574 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.574 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.574 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-08-18T23:53:32.574 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-08-18T23:53:32.574 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.574 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.574 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.574 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-08-18T23:53:32.574 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.574 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.574 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.574 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-18T23:53:32.575 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-18T23:53:32.575 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.575 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.575 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-08-18T23:53:32.575 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-08-18T23:53:32.575 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:32.575 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.575 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.575 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-08-18T23:53:32.575 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-08-18T23:53:32.575 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-08-18T23:53:32.575 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.575 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-08-18T23:53:32.576 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-18T23:53:32.576 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.576 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.576 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-08-18T23:53:32.576 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-08-18T23:53:32.576 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.576 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.576 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.576 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-08-18T23:53:32.576 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.576 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.576 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.577 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-08-18T23:53:32.577 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-18T23:53:32.577 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.577 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.577 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-08-18T23:53:32.577 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-08-18T23:53:32.577 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-08-18T23:53:32.577 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.577 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.577 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-08-18T23:53:32.577 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-08-18T23:53:32.577 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-08-18T23:53:32.578 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.578 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-18T23:53:32.578 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-18T23:53:32.578 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.578 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.578 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.578 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.578 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.578 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.578 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.578 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.578 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.580 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.581 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.582 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.582 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.582 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.582 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.582 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.582 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.582 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.582 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.582 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.582 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.582 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.582 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.583 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.583 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.583 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.583 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.583 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.583 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.583 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.583 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.583 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.583 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.583 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.583 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-08-18T23:53:32.584 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-08-18T23:53:32.584 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:32.584 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.584 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.584 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-08-18T23:53:32.584 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.584 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.584 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.584 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-18T23:53:32.584 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.584 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.584 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.585 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-08-18T23:53:32.585 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-08-18T23:53:32.585 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.585 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.585 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.585 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-08-18T23:53:32.585 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.585 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.585 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.585 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-08-18T23:53:32.585 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-08-18T23:53:32.585 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-08-18T23:53:32.585 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-08-18T23:53:32.586 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-08-18T23:53:32.586 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.586 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-08-18T23:53:32.586 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.586 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.586 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.586 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-08-18T23:53:32.586 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-08-18T23:53:32.586 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.586 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.586 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.586 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.587 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.587 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.587 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.587 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-18T23:53:32.587 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.587 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.587 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.587 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-08-18T23:53:32.587 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-08-18T23:53:32.587 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:32.587 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.587 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.588 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-08-18T23:53:32.588 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.588 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.588 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.588 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-18T23:53:32.588 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.588 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.588 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.588 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-08-18T23:53:32.588 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-08-18T23:53:32.588 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.588 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.589 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.589 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-08-18T23:53:32.589 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-08-18T23:53:32.589 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.589 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.589 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-18T23:53:32.589 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-18T23:53:32.589 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.589 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.589 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-08-18T23:53:32.589 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-08-18T23:53:32.589 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-08-18T23:53:32.590 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.590 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.590 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:32.590 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.590 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.590 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.590 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-18T23:53:32.590 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.590 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.590 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.590 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.590 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.591 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.591 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-08-18T23:53:32.591 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.591 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.591 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.591 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-08-18T23:53:32.591 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-08-18T23:53:32.591 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:32.591 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.591 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.591 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-08-18T23:53:32.591 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.592 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.592 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.592 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-18T23:53:32.592 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.592 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.592 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.592 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-08-18T23:53:32.592 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-08-18T23:53:32.592 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:32.592 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.592 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.592 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-08-18T23:53:32.593 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.593 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.593 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.593 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-18T23:53:32.593 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.593 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.593 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.593 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-08-18T23:53:32.593 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-08-18T23:53:32.593 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.593 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.593 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.594 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-08-18T23:53:32.594 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.594 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.594 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.594 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-18T23:53:32.594 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.594 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.594 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.594 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-08-18T23:53:32.595 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-08-18T23:53:32.595 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.595 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.595 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.595 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.595 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.595 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.595 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.595 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-18T23:53:32.595 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-18T23:53:32.595 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.595 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.596 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-08-18T23:53:32.596 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-08-18T23:53:32.596 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:32.596 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.596 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.596 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-08-18T23:53:32.596 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.596 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.596 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.596 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-18T23:53:32.596 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.596 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.597 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.597 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-08-18T23:53:32.597 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-08-18T23:53:32.597 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:32.597 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.597 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.597 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-08-18T23:53:32.597 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.597 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.597 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.597 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-18T23:53:32.598 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.598 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.598 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.598 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-08-18T23:53:32.598 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-08-18T23:53:32.598 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.598 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.598 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.598 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.598 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.599 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.599 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.599 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-08-18T23:53:32.599 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-08-18T23:53:32.599 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-08-18T23:53:32.599 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.599 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-18T23:53:32.599 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.599 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.599 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.599 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-08-18T23:53:32.599 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-08-18T23:53:32.600 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.600 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.600 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.600 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.600 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.600 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.600 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.600 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-18T23:53:32.600 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.600 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.600 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.600 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.600 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-08-18T23:53:32.601 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-08-18T23:53:32.601 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.601 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.601 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.601 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.601 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.601 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.601 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.601 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-18T23:53:32.601 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.601 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.601 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.601 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-08-18T23:53:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-08-18T23:53:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-18T23:53:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.602 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-08-18T23:53:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-08-18T23:53:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-08-18T23:53:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-08-18T23:53:32.603 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-08-18T23:53:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-08-18T23:53:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-18T23:53:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.604 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-08-18T23:53:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-08-18T23:53:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-08-18T23:53:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-18T23:53:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.605 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-08-18T23:53:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-08-18T23:53:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-08-18T23:53:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-08-18T23:53:32.606 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-08-18T23:53:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-08-18T23:53:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-08-18T23:53:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.607 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-18T23:53:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-08-18T23:53:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-08-18T23:53:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-08-18T23:53:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.608 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-18T23:53:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-08-18T23:53:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-08-18T23:53:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-08-18T23:53:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.609 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.610 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.610 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-18T23:53:32.610 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.610 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.610 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.610 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-08-18T23:53:32.610 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.610 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.610 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.610 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.610 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-08-18T23:53:32.610 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-08-18T23:53:32.611 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.611 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.611 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.611 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-18T23:53:32.611 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.611 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.611 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.611 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-08-18T23:53:32.611 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.611 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.611 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.611 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.611 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-08-18T23:53:32.612 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-08-18T23:53:32.612 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.612 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.612 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.612 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-18T23:53:32.612 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.612 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.612 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.612 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-08-18T23:53:32.612 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.612 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.612 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.613 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.613 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-08-18T23:53:32.613 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-08-18T23:53:32.613 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.613 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.613 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.613 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-18T23:53:32.613 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.613 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.613 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.613 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-08-18T23:53:32.613 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.613 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.614 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.614 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.614 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-08-18T23:53:32.614 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-08-18T23:53:32.614 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.614 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.614 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.614 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-08-18T23:53:32.614 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.614 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.614 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.614 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-08-18T23:53:32.614 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.615 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.615 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.615 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.615 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-08-18T23:53:32.615 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-08-18T23:53:32.615 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.615 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.615 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.618 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-18T23:53:32.618 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.618 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.618 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.618 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-08-18T23:53:32.618 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.619 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.619 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.619 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.619 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-08-18T23:53:32.619 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-08-18T23:53:32.619 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.619 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.619 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.619 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-08-18T23:53:32.619 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.619 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.619 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.619 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-08-18T23:53:32.620 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.620 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.620 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.620 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.620 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-08-18T23:53:32.620 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-08-18T23:53:32.620 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.620 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.620 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.620 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-18T23:53:32.620 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.620 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.620 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.621 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-08-18T23:53:32.621 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.621 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.621 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.621 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.621 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-08-18T23:53:32.621 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-08-18T23:53:32.621 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.621 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.621 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.621 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-18T23:53:32.621 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.621 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.622 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.622 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-08-18T23:53:32.622 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.622 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.622 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.622 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.622 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-08-18T23:53:32.622 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-08-18T23:53:32.622 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.622 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.622 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.622 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-18T23:53:32.622 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.623 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.623 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.623 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-08-18T23:53:32.623 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.623 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.623 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.623 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.623 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-08-18T23:53:32.623 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-08-18T23:53:32.623 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.623 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.623 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.623 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.624 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.624 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.624 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.624 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-08-18T23:53:32.624 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.624 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.624 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.624 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.624 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-08-18T23:53:32.624 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-08-18T23:53:32.624 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:32.625 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.625 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.625 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-08-18T23:53:32.625 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.625 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.625 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.625 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-18T23:53:32.625 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.625 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.625 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.626 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.626 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-08-18T23:53:32.626 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-08-18T23:53:32.626 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.626 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.626 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.626 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-08-18T23:53:32.626 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.626 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.626 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.626 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-18T23:53:32.626 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.626 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.627 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.627 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.627 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-08-18T23:53:32.627 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-08-18T23:53:32.627 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.627 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.627 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.627 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-08-18T23:53:32.627 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.627 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.627 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.627 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-18T23:53:32.627 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.628 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.628 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.628 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.628 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-08-18T23:53:32.628 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-08-18T23:53:32.628 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:32.628 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.628 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.628 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-08-18T23:53:32.628 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.628 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.628 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.628 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-08-18T23:53:32.629 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.629 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.629 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.629 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.629 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-08-18T23:53:32.629 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-08-18T23:53:32.629 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.629 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.629 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.629 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.629 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.629 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.630 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.630 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-18T23:53:32.630 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.630 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.630 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.630 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.630 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-08-18T23:53:32.630 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-08-18T23:53:32.630 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:32.630 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.630 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.630 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-08-18T23:53:32.631 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.631 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.631 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.631 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-08-18T23:53:32.631 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.631 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.631 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.631 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.631 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-08-18T23:53:32.631 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-08-18T23:53:32.631 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.631 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.631 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.632 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-18T23:53:32.632 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.632 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.632 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.632 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-18T23:53:32.632 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.632 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.632 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.632 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.632 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-08-18T23:53:32.632 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-08-18T23:53:32.632 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:32.632 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.633 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.633 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-08-18T23:53:32.633 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.633 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.633 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.633 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-08-18T23:53:32.633 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.633 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.633 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.633 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.633 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-08-18T23:53:32.633 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-08-18T23:53:32.633 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.634 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.634 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.634 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.634 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.634 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.634 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.634 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-18T23:53:32.634 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.634 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.634 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.634 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.634 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-08-18T23:53:32.635 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-08-18T23:53:32.635 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.635 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.635 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.635 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.635 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.635 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.635 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.635 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-18T23:53:32.635 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.635 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.635 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.635 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.636 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.636 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.636 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.636 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.636 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.636 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.636 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.636 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.636 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.636 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.636 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.636 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.636 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.637 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.637 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.637 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.637 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.637 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.637 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.637 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.637 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-08-18T23:53:32.637 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-08-18T23:53:32.637 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.637 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.637 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.638 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.638 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.638 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.638 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.638 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-18T23:53:32.638 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.638 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.638 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.638 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.638 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.638 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.639 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.639 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.639 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.639 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.639 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.639 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.639 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.639 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-18T23:53:32.639 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.639 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.639 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.639 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.640 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.640 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.640 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.640 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.640 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.640 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.640 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.640 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.640 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.640 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.640 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.640 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.640 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.641 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.641 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.641 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.641 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.641 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.641 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.641 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.641 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.641 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.641 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.641 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.641 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.641 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.642 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.642 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.642 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.642 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.642 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.642 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.642 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.642 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.642 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-18T23:53:32.642 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-18T23:53:32.642 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.642 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.643 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.643 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.643 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.643 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.643 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.643 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-18T23:53:32.643 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.643 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.643 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.643 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-18T23:53:32.644 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-18T23:53:32.644 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.644 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.644 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.644 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-08-18T23:53:32.645 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.645 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.645 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.645 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-18T23:53:32.645 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.645 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.646 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.646 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.646 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-08-18T23:53:32.646 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-08-18T23:53:32.646 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.647 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.647 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.647 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:32.647 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.647 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.648 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.648 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-08-18T23:53:32.648 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.648 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.648 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.648 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.649 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-08-18T23:53:32.649 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-08-18T23:53:32.649 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.649 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.649 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.649 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:32.649 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.649 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.649 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.649 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-18T23:53:32.649 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.650 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.650 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.650 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-08-18T23:53:32.650 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-08-18T23:53:32.650 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.650 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.650 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.650 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.650 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.650 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.650 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.651 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-08-18T23:53:32.651 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.651 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.651 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.651 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-08-18T23:53:32.651 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-08-18T23:53:32.651 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.651 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.651 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.651 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-08-18T23:53:32.652 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.652 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.652 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.652 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-08-18T23:53:32.652 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-08-18T23:53:32.653 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.653 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-08-18T23:53:32.653 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.653 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.653 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.653 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-08-18T23:53:32.653 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-08-18T23:53:32.654 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.654 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.654 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.654 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-08-18T23:53:32.654 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.654 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.654 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.654 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-18T23:53:32.654 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.654 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.654 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.654 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-08-18T23:53:32.655 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-08-18T23:53:32.655 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.655 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.655 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.655 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-18T23:53:32.655 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.655 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.655 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.655 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-18T23:53:32.655 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.655 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.655 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.656 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-08-18T23:53:32.656 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-08-18T23:53:32.656 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.656 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.656 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.656 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.656 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.656 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.656 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.656 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-18T23:53:32.656 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.657 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.657 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.657 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-08-18T23:53:32.657 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-08-18T23:53:32.657 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.657 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.657 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.657 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.657 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.657 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.657 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.657 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-18T23:53:32.658 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.658 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.658 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.658 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-08-18T23:53:32.658 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-08-18T23:53:32.658 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.658 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.658 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.658 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.658 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.658 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.659 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.659 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-18T23:53:32.659 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.659 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.659 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.659 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-08-18T23:53:32.659 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-08-18T23:53:32.659 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.659 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.659 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.659 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.659 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.660 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.660 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.660 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-18T23:53:32.660 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.660 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.660 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.660 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-08-18T23:53:32.660 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-08-18T23:53:32.660 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.660 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.660 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.661 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.661 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.661 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.661 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.661 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-18T23:53:32.661 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.661 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.661 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.661 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-08-18T23:53:32.661 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-08-18T23:53:32.661 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.661 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.662 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.662 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-08-18T23:53:32.662 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.662 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.662 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.662 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-08-18T23:53:32.662 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.662 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.662 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.662 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-08-18T23:53:32.662 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-08-18T23:53:32.662 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.663 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.663 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.663 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.663 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.663 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.663 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.663 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-18T23:53:32.663 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.663 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.663 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.663 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-08-18T23:53:32.664 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-08-18T23:53:32.664 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.664 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.664 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.664 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.664 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.664 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.664 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.664 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-18T23:53:32.664 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.664 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.664 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.664 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-08-18T23:53:32.665 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-08-18T23:53:32.665 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.665 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.665 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.665 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.665 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.665 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.665 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.665 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-18T23:53:32.665 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.665 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.665 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.666 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-08-18T23:53:32.666 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-08-18T23:53:32.666 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.666 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.666 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.666 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.666 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.666 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.666 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.666 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-08-18T23:53:32.666 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.666 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.666 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.667 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-08-18T23:53:32.667 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-08-18T23:53:32.667 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.667 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.667 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.667 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.667 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.667 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.667 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.667 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-18T23:53:32.667 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.667 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.668 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.668 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-08-18T23:53:32.668 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-08-18T23:53:32.668 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.668 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.668 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.668 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.668 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.668 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.668 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.668 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-18T23:53:32.668 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.669 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.669 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.669 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.669 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.669 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-08-18T23:53:32.669 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.669 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.669 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.669 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.669 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.670 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.670 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.670 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.670 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.670 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.670 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.670 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.670 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.670 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.670 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.670 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.670 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.670 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.671 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.671 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.671 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.671 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.671 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.671 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.671 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.671 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.671 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.671 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.671 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.671 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.672 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.672 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.672 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.672 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.672 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.672 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.672 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.672 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.672 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.672 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.672 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.672 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.673 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.673 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.673 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.673 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.673 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.673 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.673 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.673 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.673 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.673 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.673 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.673 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.673 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.674 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-08-18T23:53:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-08-18T23:53:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-08-18T23:53:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.675 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-18T23:53:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-08-18T23:53:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-08-18T23:53:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-08-18T23:53:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.676 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-18T23:53:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-08-18T23:53:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.677 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-18T23:53:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-18T23:53:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.678 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-08-18T23:53:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-18T23:53:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.679 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-18T23:53:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-18T23:53:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.680 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-08-18T23:53:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-08-18T23:53:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.681 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-18T23:53:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-18T23:53:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-08-18T23:53:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-18T23:53:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.683 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-18T23:53:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-18T23:53:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.684 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-18T23:53:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-18T23:53:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.685 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.686 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.686 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.686 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.686 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.686 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.686 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.686 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.686 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.686 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.686 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-18T23:53:32.686 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-18T23:53:32.686 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.687 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.687 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.687 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.687 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.687 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.687 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.687 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.687 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.687 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.687 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.687 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-18T23:53:32.688 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-18T23:53:32.688 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.688 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.688 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.688 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.688 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.688 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.688 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.688 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.688 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.688 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.688 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.689 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-18T23:53:32.689 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-18T23:53:32.689 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.689 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.689 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.689 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.689 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.689 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.689 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.689 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.689 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.689 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.689 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.690 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-18T23:53:32.690 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-18T23:53:32.690 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.690 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.690 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.690 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.690 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.690 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.690 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.690 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.690 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.691 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.691 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.691 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-18T23:53:32.691 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-18T23:53:32.691 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.691 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.691 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.691 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.691 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.691 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.691 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.691 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.692 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.692 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.692 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.692 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-18T23:53:32.692 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-18T23:53:32.692 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.692 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.692 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.692 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.692 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.692 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.692 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.693 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.693 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.693 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.693 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.693 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-08-18T23:53:32.693 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-18T23:53:32.693 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.693 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.693 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.693 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-08-18T23:53:32.693 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.694 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.694 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.694 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.694 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.694 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.694 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.694 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-18T23:53:32.694 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-18T23:53:32.694 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.694 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.694 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.694 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.695 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.695 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.695 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.695 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.695 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.695 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.695 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.696 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-08-18T23:53:32.696 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-08-18T23:53:32.696 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.696 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.696 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.696 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.696 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.697 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.697 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.697 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.697 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.697 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.697 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.698 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-08-18T23:53:32.698 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-18T23:53:32.698 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.698 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.698 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.699 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-08-18T23:53:32.699 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.699 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.699 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.699 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.699 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.699 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.699 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.700 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-18T23:53:32.700 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-18T23:53:32.700 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.700 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.700 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.700 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.700 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.700 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.700 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.700 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.700 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.700 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.701 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.701 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-18T23:53:32.701 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-18T23:53:32.701 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.701 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.701 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.701 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.701 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.701 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.701 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.702 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.702 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.702 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.702 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.702 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-18T23:53:32.702 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-18T23:53:32.702 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.702 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.702 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.702 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.702 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.703 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.703 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.703 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.703 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.703 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.703 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.703 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-18T23:53:32.703 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-18T23:53:32.703 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.703 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.703 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.703 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.704 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.704 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.704 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.704 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.704 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.704 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.704 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.704 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-08-18T23:53:32.704 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-18T23:53:32.704 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.704 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.704 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.705 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.705 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.705 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.705 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.705 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.705 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.705 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.705 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.705 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-18T23:53:32.705 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-18T23:53:32.705 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.706 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.706 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.706 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.706 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.706 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.706 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.706 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.706 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.706 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.706 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.706 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-18T23:53:32.706 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-18T23:53:32.707 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.707 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.707 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.707 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.707 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.707 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.707 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.707 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.707 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.707 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.707 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.707 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-18T23:53:32.708 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-18T23:53:32.708 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.708 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.708 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.708 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.708 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.708 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.708 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.708 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.708 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.708 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.708 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.709 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-18T23:53:32.709 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-18T23:53:32.709 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.709 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.709 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.709 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.709 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.709 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.709 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.709 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.709 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.709 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.709 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.710 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-18T23:53:32.710 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-18T23:53:32.710 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.710 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.710 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.710 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.710 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.710 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.710 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.710 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.710 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.711 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.711 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.711 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-18T23:53:32.711 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-18T23:53:32.711 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.711 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.711 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.711 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.711 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.711 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.711 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.711 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.712 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.712 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.712 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.712 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-18T23:53:32.712 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-18T23:53:32.712 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.712 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.712 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.712 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.712 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.712 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.712 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.713 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.713 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.713 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.713 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.713 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-18T23:53:32.713 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-18T23:53:32.713 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.713 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.713 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.713 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.713 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.713 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.713 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.714 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.714 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.714 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.714 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.714 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-08-18T23:53:32.714 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-18T23:53:32.714 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.714 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.714 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.714 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.714 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.715 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.715 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.715 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.715 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.715 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.715 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.715 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-18T23:53:32.715 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-18T23:53:32.715 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.715 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.715 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.716 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-18T23:53:32.716 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.716 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.716 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.716 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.716 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.716 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.716 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.716 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-18T23:53:32.716 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-18T23:53:32.716 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.716 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.717 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.717 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:32.717 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.717 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.717 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.717 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.717 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.717 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.717 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.717 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-18T23:53:32.717 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-18T23:53:32.717 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.718 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.718 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.718 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-08-18T23:53:32.718 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.718 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.718 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.718 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.718 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.719 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.719 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.719 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-18T23:53:32.719 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-18T23:53:32.719 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.719 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.719 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.719 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-08-18T23:53:32.719 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.719 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.719 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.720 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.720 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.720 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.720 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.720 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-08-18T23:53:32.720 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-18T23:53:32.720 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.720 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.720 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.720 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.720 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.720 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.721 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.721 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.721 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.721 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.721 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.721 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-18T23:53:32.721 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-18T23:53:32.721 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.721 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.721 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.721 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-08-18T23:53:32.722 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.722 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.722 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.722 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.722 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.722 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.722 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.722 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-08-18T23:53:32.722 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-18T23:53:32.722 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.722 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.722 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.723 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.723 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.723 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.723 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.723 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.723 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.723 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.723 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.723 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-08-18T23:53:32.723 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-18T23:53:32.723 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.723 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.724 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.724 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.724 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.724 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.724 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.724 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.724 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.724 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.724 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.724 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-18T23:53:32.724 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-18T23:53:32.725 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.725 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.725 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.725 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-08-18T23:53:32.725 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.725 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.725 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.725 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.725 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.725 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.725 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.726 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-18T23:53:32.726 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-18T23:53:32.726 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.726 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.726 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.726 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:32.726 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.726 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.726 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.726 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.726 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.726 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.727 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.727 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-18T23:53:32.727 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-18T23:53:32.727 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.727 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.727 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.727 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-08-18T23:53:32.727 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.727 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.727 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.727 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.728 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.728 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.728 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.728 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-08-18T23:53:32.728 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-08-18T23:53:32.728 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.728 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.728 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.728 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.728 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.728 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.728 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-08-18T23:53:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-08-18T23:53:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.729 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-08-18T23:53:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-08-18T23:53:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.730 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-08-18T23:53:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-08-18T23:53:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-08-18T23:53:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.731 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-08-18T23:53:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-08-18T23:53:32.732 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-08-18T23:53:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.733 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.734 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.735 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.736 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.737 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.738 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-08-18T23:53:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-08-18T23:53:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.739 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-08-18T23:53:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-08-18T23:53:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-08-18T23:53:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.740 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-08-18T23:53:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-08-18T23:53:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.741 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-08-18T23:53:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-08-18T23:53:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-08-18T23:53:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.742 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-08-18T23:53:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-08-18T23:53:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-08-18T23:53:32.743 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-08-18T23:53:32.744 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-08-18T23:53:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-08-18T23:53:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-08-18T23:53:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-08-18T23:53:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-08-18T23:53:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-08-18T23:53:32.746 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-08-18T23:53:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-08-18T23:53:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-08-18T23:53:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-08-18T23:53:32.747 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-08-18T23:53:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-08-18T23:53:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-08-18T23:53:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.748 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-08-18T23:53:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.749 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-08-18T23:53:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-08-18T23:53:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-08-18T23:53:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.750 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.751 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.752 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.753 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.754 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.755 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-08-18T23:53:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-08-18T23:53:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-08-18T23:53:32.756 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-18T23:53:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-08-18T23:53:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-08-18T23:53:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.757 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-08-18T23:53:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-18T23:53:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-08-18T23:53:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-08-18T23:53:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.758 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-08-18T23:53:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-08-18T23:53:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-08-18T23:53:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-08-18T23:53:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:32.759 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-08-18T23:53:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-18T23:53:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.760 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-08-18T23:53:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-08-18T23:53:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-18T23:53:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.761 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-08-18T23:53:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-08-18T23:53:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-08-18T23:53:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-08-18T23:53:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.762 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-08-18T23:53:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-08-18T23:53:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-08-18T23:53:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-18T23:53:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.763 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-08-18T23:53:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.764 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.765 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.766 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.767 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.768 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.769 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-08-18T23:53:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-08-18T23:53:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-08-18T23:53:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.770 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-08-18T23:53:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-08-18T23:53:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-08-18T23:53:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.771 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-08-18T23:53:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-08-18T23:53:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-08-18T23:53:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.772 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-08-18T23:53:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-08-18T23:53:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-08-18T23:53:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.773 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-08-18T23:53:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-08-18T23:53:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-08-18T23:53:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-18T23:53:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-08-18T23:53:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-08-18T23:53:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-08-18T23:53:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-18T23:53:32.775 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-08-18T23:53:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-08-18T23:53:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.776 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-08-18T23:53:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-08-18T23:53:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-08-18T23:53:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-08-18T23:53:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-08-18T23:53:32.777 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-18T23:53:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.778 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.779 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.779 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.779 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.779 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.779 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.779 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.779 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.779 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.779 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.779 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.779 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.779 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.779 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.780 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.780 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.780 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.780 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.780 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.780 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.780 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.780 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.780 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.780 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.780 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.780 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.780 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.781 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.781 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.781 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.781 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.781 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.781 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.781 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.781 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.781 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.781 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.781 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.781 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.782 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.782 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.782 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.782 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.782 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.782 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.782 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.782 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.782 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.782 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.782 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.783 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.783 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.783 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.783 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.783 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.783 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.783 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.783 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.783 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.783 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.783 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.783 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-08-18T23:53:32.784 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-08-18T23:53:32.784 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.784 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.784 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.784 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.784 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.784 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.784 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.784 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-08-18T23:53:32.784 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.784 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.784 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.785 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-08-18T23:53:32.785 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-08-18T23:53:32.785 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.785 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.785 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.785 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-08-18T23:53:32.785 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.785 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.785 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.785 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-08-18T23:53:32.785 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.785 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.785 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.786 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-08-18T23:53:32.786 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-08-18T23:53:32.786 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.786 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.786 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.786 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-08-18T23:53:32.786 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.786 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.786 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.786 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-18T23:53:32.786 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.786 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.787 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.787 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-08-18T23:53:32.787 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-08-18T23:53:32.787 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.787 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.787 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.787 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-08-18T23:53:32.787 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-08-18T23:53:32.787 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-08-18T23:53:32.787 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.787 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-18T23:53:32.787 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.788 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.788 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.788 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-08-18T23:53:32.788 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-08-18T23:53:32.788 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.788 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.788 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.788 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.788 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.788 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.788 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.789 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-08-18T23:53:32.789 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.789 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.789 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.789 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-08-18T23:53:32.789 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-08-18T23:53:32.789 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.789 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.789 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.789 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-08-18T23:53:32.790 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.790 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.790 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.790 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-18T23:53:32.790 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.790 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.790 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.790 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.790 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.790 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.790 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-08-18T23:53:32.790 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.791 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.791 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.791 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.791 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.791 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.791 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.791 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.791 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.791 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.791 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.791 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.791 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.791 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.792 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.792 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.792 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.792 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.792 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.792 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.792 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.792 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.792 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.792 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.792 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.792 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.793 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.793 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.793 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.793 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.793 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.793 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.793 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.793 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.793 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.793 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.793 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.793 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.793 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.794 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.794 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.794 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.794 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.794 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.794 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.794 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.794 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.794 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.794 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.794 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.794 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.795 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.795 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.795 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.795 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.795 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.795 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.795 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.795 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.795 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.795 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.795 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.795 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.796 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.796 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.796 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.796 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.796 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.796 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.796 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.796 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.796 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.796 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.796 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.796 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-08-18T23:53:32.796 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.797 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.797 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.797 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.797 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.797 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.797 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.797 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.797 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.797 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.797 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.797 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.797 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.797 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.797 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.798 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.798 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.798 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.798 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.798 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.798 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.798 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.798 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.798 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.798 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.798 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.799 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.799 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.799 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.799 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.799 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.799 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.799 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.799 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.799 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.799 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.799 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.799 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.800 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.800 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.800 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.800 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.800 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.800 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.800 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.800 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.800 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.800 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.800 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.800 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.800 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.801 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.801 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.801 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.801 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.801 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.801 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.801 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.801 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.801 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.801 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.801 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.801 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.802 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.802 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.802 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.802 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.802 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.802 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.802 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.802 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.802 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.802 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "k8sevents", 2024-08-18T23:53:32.802 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.803 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.803 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.803 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_event_retention_days": { 2024-08-18T23:53:32.803 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ceph_event_retention_days", 2024-08-18T23:53:32.803 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.803 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.803 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.803 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7", 2024-08-18T23:53:32.803 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.803 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.803 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.803 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-18T23:53:32.803 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.804 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.804 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.804 INFO:teuthology.orchestra.run.smithi078.stdout: "config_check_secs": { 2024-08-18T23:53:32.804 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_check_secs", 2024-08-18T23:53:32.804 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.804 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.804 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.804 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:32.804 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "10", 2024-08-18T23:53:32.804 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.804 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.805 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-18T23:53:32.805 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.805 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.805 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.805 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.805 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.805 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.805 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.805 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.805 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.805 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.806 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.806 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.806 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.806 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.806 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.806 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.806 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.806 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.806 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.806 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.806 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.806 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.806 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.807 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.807 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.807 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.807 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.807 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.807 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.807 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.807 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.807 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.807 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.807 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.807 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.808 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.808 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.808 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.808 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.808 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.808 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.808 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.808 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.808 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.808 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.808 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.808 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.808 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.809 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.809 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.809 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.809 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.809 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.809 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.809 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.809 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.809 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.809 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.809 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.809 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.810 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.810 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.810 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.810 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.810 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.810 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.810 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.810 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.810 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.810 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.810 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.810 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.811 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.811 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.811 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-08-18T23:53:32.811 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.811 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.811 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.811 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-08-18T23:53:32.811 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-08-18T23:53:32.811 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.811 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.811 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.811 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-08-18T23:53:32.811 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.811 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.812 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.812 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-08-18T23:53:32.812 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-18T23:53:32.812 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.812 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.812 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.812 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.812 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.812 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.812 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.812 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.812 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.812 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.813 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.813 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.813 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.813 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.813 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.813 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.813 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.813 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.813 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.813 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.813 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.813 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.814 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.814 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.814 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.814 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.814 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.814 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.814 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.814 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.814 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.814 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.814 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.814 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.814 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.815 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.815 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.815 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.815 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.815 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.815 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.815 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.815 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.815 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.815 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.815 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.816 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.816 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.816 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.816 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.816 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.816 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.816 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.816 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.816 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.816 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.816 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.816 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.817 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.817 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.817 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.817 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.817 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.817 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.817 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.817 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.817 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.817 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.817 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.817 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.817 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-08-18T23:53:32.818 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-08-18T23:53:32.818 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.818 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.818 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.818 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.818 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.818 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.818 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.818 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-08-18T23:53:32.818 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-18T23:53:32.818 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.819 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.819 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-08-18T23:53:32.819 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-08-18T23:53:32.819 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.819 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.819 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.819 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-08-18T23:53:32.819 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.820 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.820 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.820 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-08-18T23:53:32.820 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.820 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.820 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.820 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-08-18T23:53:32.820 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-08-18T23:53:32.820 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.820 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.820 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.821 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-08-18T23:53:32.821 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.821 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.821 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.821 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-08-18T23:53:32.821 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.821 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.821 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.821 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-08-18T23:53:32.821 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-08-18T23:53:32.821 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.821 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.822 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.822 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.822 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.822 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.822 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.822 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-08-18T23:53:32.822 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.822 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.822 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.822 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-08-18T23:53:32.822 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-08-18T23:53:32.823 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.823 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.823 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.823 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-08-18T23:53:32.823 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.823 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.823 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.823 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-18T23:53:32.823 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-18T23:53:32.823 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.823 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.823 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.824 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.824 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.824 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mds_autoscaler", 2024-08-18T23:53:32.824 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.824 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.824 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.824 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.824 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.824 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.824 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.824 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.824 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.825 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.825 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.825 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.825 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.825 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.825 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.825 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.825 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.825 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.825 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.825 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.825 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.826 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.826 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.826 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.826 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.826 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.826 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.826 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.826 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.826 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.826 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.826 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.826 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.827 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.827 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.827 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.827 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.827 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.827 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.827 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.827 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.827 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.827 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.827 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.827 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.828 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.828 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.828 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.828 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.828 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.828 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.828 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.828 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.828 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.828 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.828 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.828 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.829 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.829 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.829 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.829 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.829 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.829 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.829 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.829 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.829 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.829 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.829 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.829 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.829 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.830 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.830 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.830 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-08-18T23:53:32.830 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.830 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.830 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.830 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.830 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.830 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.830 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.830 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.830 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.831 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.831 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.831 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.831 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.831 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.831 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.831 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.831 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.831 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.831 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.831 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.831 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.831 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.832 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.832 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.832 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.832 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.832 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.832 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.832 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.832 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.832 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.832 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.832 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.833 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.833 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.833 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.833 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.833 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.833 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.833 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.833 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.833 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.833 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.833 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.833 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.833 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.834 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.834 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.834 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.834 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.834 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.834 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.834 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.834 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.834 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.834 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.834 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.835 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.835 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.835 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.835 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.835 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.835 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.835 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.835 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.835 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.835 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.835 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.835 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-08-18T23:53:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.837 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.837 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.837 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.837 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.837 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.837 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.837 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.837 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.837 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.837 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.837 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.837 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.838 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.838 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.838 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.838 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.838 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.838 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.838 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.838 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.838 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.838 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.838 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.838 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.839 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.839 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.839 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.839 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.839 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.839 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.839 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.839 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.839 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.839 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.839 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.839 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.839 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.840 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.840 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.840 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.840 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.840 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.840 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.840 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.840 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.840 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.840 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.840 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.841 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.841 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.841 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.841 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.841 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.841 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.841 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.841 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.841 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.841 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.841 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.841 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.841 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.842 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.842 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.842 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.842 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-08-18T23:53:32.842 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.842 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.842 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.842 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-08-18T23:53:32.842 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-08-18T23:53:32.842 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.842 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.843 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.843 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.843 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.843 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.843 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.843 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-18T23:53:32.843 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.843 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.843 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.843 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.843 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.843 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.843 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.844 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.844 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.844 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.844 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.844 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.844 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.844 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.844 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.844 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.844 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.844 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.844 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.844 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.845 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.845 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.845 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.845 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.845 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.845 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.845 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.845 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.845 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.845 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.845 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.845 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.846 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.846 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.846 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.846 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.846 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.846 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.846 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.846 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.846 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.846 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.846 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.847 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.847 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.847 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.847 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.847 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.847 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.847 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.847 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.847 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.847 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.847 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.847 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.847 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.848 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.848 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.848 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.848 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.848 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.848 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.848 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.848 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.848 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.848 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.848 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.848 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.849 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.849 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-08-18T23:53:32.849 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-08-18T23:53:32.849 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.849 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.849 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.849 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.849 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.849 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.849 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.849 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-08-18T23:53:32.850 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-08-18T23:53:32.850 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-08-18T23:53:32.850 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.850 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-08-18T23:53:32.850 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.850 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.850 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.850 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.850 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.850 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.851 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-08-18T23:53:32.851 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.851 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.851 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.851 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.851 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.851 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.851 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.851 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.851 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.851 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.851 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.851 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.852 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.852 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.852 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.852 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.852 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.852 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.852 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.852 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.852 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.852 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.852 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.852 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.853 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.853 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.853 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.853 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.853 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.853 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.853 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.853 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.853 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.853 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.853 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.853 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.854 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.854 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.854 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.854 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.854 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.854 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.854 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.854 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.854 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.854 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.854 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.854 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.855 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.855 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.855 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.855 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.855 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.855 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.855 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.855 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.855 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.855 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.855 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.855 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.856 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.856 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.856 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.856 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.856 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.856 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.856 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.856 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.856 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.856 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.856 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.857 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.857 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-08-18T23:53:32.857 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.857 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.857 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.857 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.857 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.857 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.857 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.857 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.857 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.857 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.857 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.858 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.858 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.858 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.858 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.858 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.858 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.858 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.858 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.858 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.858 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.858 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.858 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.859 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.859 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.859 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.859 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.859 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.859 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.859 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.859 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.859 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.859 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.859 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.859 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.859 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.860 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.860 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.860 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.860 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.860 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.860 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.860 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.860 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.860 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.860 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.860 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.861 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.861 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.861 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.861 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.861 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.861 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.861 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.861 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.861 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.861 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.861 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.861 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.862 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.862 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.862 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.862 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.862 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.862 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.862 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.862 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.862 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.862 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.862 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.862 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.863 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.863 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-08-18T23:53:32.863 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.863 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.863 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.863 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.863 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.863 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.863 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.863 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.863 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.863 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.863 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.864 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.864 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.864 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.864 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.864 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.864 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.864 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.864 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.864 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.864 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.864 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.864 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.865 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.865 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.865 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.865 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.865 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.865 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.865 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.865 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.865 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.865 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.865 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.865 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.865 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.866 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.866 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.866 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.866 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.866 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.866 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.866 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.866 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.866 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.866 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.867 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.867 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.867 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.867 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.867 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.867 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.867 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.867 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.867 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.867 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.867 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.867 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.868 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.868 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.868 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.868 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.868 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.868 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.868 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.868 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.868 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.868 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.868 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.868 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-08-18T23:53:32.869 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-08-18T23:53:32.869 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:32.869 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.869 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.869 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-08-18T23:53:32.869 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.869 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.869 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.869 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.869 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.869 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.869 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.870 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-08-18T23:53:32.870 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-08-18T23:53:32.870 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:32.870 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.870 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.870 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-08-18T23:53:32.870 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-08-18T23:53:32.870 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.870 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.870 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-08-18T23:53:32.870 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-18T23:53:32.870 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.871 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.871 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.871 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.871 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.871 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-08-18T23:53:32.871 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.871 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.871 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.871 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-08-18T23:53:32.871 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-08-18T23:53:32.871 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.872 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.872 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.872 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.872 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.872 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.872 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.872 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-18T23:53:32.872 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.872 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.872 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.872 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-08-18T23:53:32.872 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-08-18T23:53:32.873 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.873 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.873 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.873 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.873 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.873 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.873 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.873 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.873 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.873 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.873 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.873 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.873 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.874 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.874 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.874 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.874 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.874 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.874 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.874 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.874 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.874 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.874 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.874 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.874 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.875 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.875 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.875 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.875 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.875 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.875 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.875 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.875 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.875 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.875 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.875 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.875 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.875 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.876 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.876 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.876 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.876 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.876 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.876 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.876 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.876 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.876 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.876 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.876 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.877 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.877 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.877 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.877 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.877 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.877 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.877 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.877 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.877 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.877 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.877 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.877 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.877 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.878 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.878 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.878 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.878 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.878 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.878 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.878 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.878 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.878 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.878 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.878 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.878 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.879 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.879 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.879 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.879 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-08-18T23:53:32.879 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-08-18T23:53:32.879 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.879 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.879 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.879 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-08-18T23:53:32.879 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.879 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.879 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.880 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-08-18T23:53:32.880 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.880 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.880 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.880 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-08-18T23:53:32.880 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-08-18T23:53:32.880 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:32.880 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.880 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.880 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-08-18T23:53:32.880 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.880 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.881 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.881 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-08-18T23:53:32.881 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.881 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.881 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.881 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.881 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.881 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.881 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-08-18T23:53:32.881 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.881 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.882 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.882 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-08-18T23:53:32.882 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-08-18T23:53:32.882 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.882 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.882 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.882 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.882 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.882 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.882 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.882 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.882 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.883 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.883 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.883 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-08-18T23:53:32.883 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-08-18T23:53:32.883 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.883 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.883 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.883 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.883 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.883 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.883 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.883 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-18T23:53:32.884 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-18T23:53:32.884 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.884 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.884 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.884 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.884 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.884 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.884 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.884 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.884 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.884 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.885 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.885 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.885 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.885 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.885 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.885 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.885 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.885 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.885 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.885 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.885 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.885 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.886 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.886 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.886 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.886 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.886 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.886 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.886 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.886 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.886 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.886 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.886 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.886 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.887 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.887 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.887 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.887 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.887 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.887 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.887 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.887 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.887 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.887 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.887 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.887 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.888 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.888 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.888 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.888 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.888 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.888 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.888 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.888 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.888 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.888 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.888 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.888 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.889 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.889 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.889 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.889 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.889 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.889 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.889 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.889 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.889 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.889 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.889 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-08-18T23:53:32.890 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-08-18T23:53:32.890 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.890 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.890 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.890 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.890 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.890 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.890 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.890 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.890 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.890 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.890 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.891 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-18T23:53:32.891 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-18T23:53:32.891 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.891 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.891 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.891 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-08-18T23:53:32.891 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.891 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.891 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.891 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.891 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.891 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.891 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.892 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-08-18T23:53:32.892 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-08-18T23:53:32.892 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:32.892 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.892 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.892 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-08-18T23:53:32.892 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.892 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.892 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.892 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.892 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.892 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.893 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.893 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-08-18T23:53:32.893 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-08-18T23:53:32.893 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.893 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.893 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.893 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-08-18T23:53:32.893 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.893 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.893 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.893 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-18T23:53:32.893 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.893 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.894 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.894 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-08-18T23:53:32.894 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-08-18T23:53:32.894 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.894 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.894 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.894 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-08-18T23:53:32.894 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.894 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.894 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.894 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-18T23:53:32.895 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.895 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.895 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.895 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-08-18T23:53:32.895 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-08-18T23:53:32.895 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.895 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.895 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.895 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-08-18T23:53:32.895 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.895 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.896 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.896 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.896 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.896 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.896 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.896 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-08-18T23:53:32.896 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-08-18T23:53:32.896 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.896 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.896 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.896 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-08-18T23:53:32.896 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.897 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.897 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.897 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-08-18T23:53:32.897 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-08-18T23:53:32.897 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.897 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.897 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.897 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.897 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.897 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-08-18T23:53:32.897 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-08-18T23:53:32.897 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.897 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.898 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.898 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-08-18T23:53:32.898 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-08-18T23:53:32.898 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-08-18T23:53:32.898 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.898 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.898 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.898 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.898 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.898 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.898 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.899 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-08-18T23:53:32.899 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.899 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.899 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.899 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.899 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.899 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.899 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.899 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.899 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.899 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.900 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.900 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.900 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.900 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.900 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.900 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.900 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.900 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.900 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.900 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.900 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.900 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.900 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.901 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.901 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.901 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.901 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.901 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.901 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.901 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.901 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.901 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.901 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.901 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.901 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.902 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.902 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.902 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.902 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.902 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.902 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.902 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.902 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.902 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.902 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.902 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.902 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.903 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.903 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.903 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.903 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.903 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.903 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.903 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.903 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.903 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.903 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.903 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.903 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.904 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.904 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.904 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.904 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.904 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.904 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.904 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.904 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.904 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.904 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.904 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-08-18T23:53:32.904 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-08-18T23:53:32.904 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.905 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.905 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.905 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:32.905 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.905 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.905 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.905 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.905 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.905 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.905 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.905 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-08-18T23:53:32.905 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-08-18T23:53:32.906 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.906 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.906 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.906 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.906 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.906 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.906 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.906 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.906 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.906 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.906 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.907 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-08-18T23:53:32.907 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-08-18T23:53:32.907 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.907 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.907 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.907 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.907 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.907 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.907 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.907 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.907 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.907 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.907 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.908 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.908 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.908 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-08-18T23:53:32.908 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.908 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.908 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.908 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-08-18T23:53:32.908 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-08-18T23:53:32.908 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.908 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.909 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.909 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.909 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.909 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.909 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.909 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.909 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.909 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.909 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.909 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-08-18T23:53:32.909 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-08-18T23:53:32.909 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.910 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.910 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.910 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.910 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.910 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.910 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.910 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.910 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.910 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.911 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.911 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.911 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.911 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.911 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.911 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.911 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.911 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.911 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.911 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.911 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.912 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.912 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.912 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.912 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.912 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.912 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.912 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.912 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.912 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.912 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.912 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.912 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.913 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.913 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.913 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.913 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.913 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.913 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.913 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.913 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.913 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.913 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.913 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.913 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.914 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.914 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.914 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.914 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.914 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.914 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.914 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.914 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.914 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.914 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.914 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.914 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.915 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.915 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.915 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.915 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.915 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.915 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.915 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.915 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.915 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.915 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.915 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.916 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.916 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.916 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.916 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.916 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.916 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.916 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.916 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.916 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.916 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-08-18T23:53:32.916 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-08-18T23:53:32.916 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.917 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.917 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.917 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.917 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.917 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.917 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.917 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.917 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.917 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.917 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.917 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-08-18T23:53:32.918 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-08-18T23:53:32.918 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.918 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.918 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.918 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.918 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.918 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.918 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.918 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.918 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.918 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.918 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.918 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.919 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.919 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-08-18T23:53:32.919 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.919 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.919 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.919 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.919 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.919 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.919 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.920 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.920 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.920 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.920 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.920 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.920 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.920 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.920 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.920 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.920 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.920 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.920 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.920 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.921 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.921 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.921 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.921 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.921 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.921 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.921 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.921 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.921 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.921 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.921 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.921 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.922 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.922 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.922 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.922 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.922 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.922 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.922 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.922 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.922 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.922 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.922 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.923 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.923 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.923 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.923 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.923 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.923 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.923 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.923 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.923 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.923 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.923 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.923 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.923 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.924 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.924 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.924 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.924 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.924 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.924 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.924 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.924 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.924 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.924 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.924 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.924 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.924 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.925 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.925 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.925 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rook", 2024-08-18T23:53:32.925 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.925 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.925 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.925 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.925 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.925 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.925 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.926 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.926 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.926 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.926 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.926 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.926 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.926 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.926 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.926 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.926 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.926 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.926 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.927 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.927 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.927 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.927 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.927 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.927 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.927 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.927 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.927 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.927 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.927 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.928 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.928 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.928 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.928 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.928 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.928 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.928 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.928 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.928 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.928 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.928 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.928 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.929 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.929 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.929 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.929 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.929 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.929 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.929 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.929 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.929 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.929 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.929 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.929 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.930 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.930 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.930 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.930 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.930 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.930 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.930 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.930 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.930 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.930 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.930 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.931 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.931 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.931 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.931 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.931 INFO:teuthology.orchestra.run.smithi078.stdout: "storage_class": { 2024-08-18T23:53:32.931 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "storage_class", 2024-08-18T23:53:32.931 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.931 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.931 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.931 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "local", 2024-08-18T23:53:32.931 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.931 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.932 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.932 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-18T23:53:32.932 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.932 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.932 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.932 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.932 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.932 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-08-18T23:53:32.932 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.932 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.932 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.933 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.933 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.933 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.933 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.933 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.933 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.933 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.933 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.933 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.933 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.933 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.933 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.934 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.934 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.934 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.934 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.934 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.934 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.934 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.934 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.934 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.934 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.934 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.934 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.935 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.935 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.935 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.935 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.935 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.935 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.935 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.935 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.935 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.935 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.935 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.936 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.936 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.936 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.936 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.936 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.936 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.936 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.936 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.936 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.936 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.936 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.936 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.936 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.936 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.937 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.937 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.937 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.937 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.937 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.937 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.937 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.937 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.937 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.937 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.938 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.938 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.938 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.938 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.938 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.938 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.938 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.938 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-08-18T23:53:32.938 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-08-18T23:53:32.938 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.938 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.938 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.939 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.939 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.939 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.939 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.939 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.939 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.939 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.939 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.939 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-08-18T23:53:32.939 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-08-18T23:53:32.939 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.939 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.940 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.940 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-08-18T23:53:32.940 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.940 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.940 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.940 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.940 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.940 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.940 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.940 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-08-18T23:53:32.940 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-08-18T23:53:32.940 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.941 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.941 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.941 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.941 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.941 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.941 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.941 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.941 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.941 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.941 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.941 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-08-18T23:53:32.941 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-08-18T23:53:32.942 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.942 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.942 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.942 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.942 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.942 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.942 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.942 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.942 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.942 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.942 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.943 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-08-18T23:53:32.943 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-08-18T23:53:32.943 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:32.943 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.943 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.943 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.943 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.943 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.943 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.943 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.943 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.943 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.944 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.944 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-08-18T23:53:32.944 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-08-18T23:53:32.944 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.944 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.944 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.944 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.944 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.944 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.944 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.944 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.945 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.945 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.945 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.945 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-08-18T23:53:32.945 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-08-18T23:53:32.945 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.945 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.945 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.945 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.945 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.945 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.945 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.945 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.946 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.946 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.946 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.946 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-08-18T23:53:32.946 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-08-18T23:53:32.946 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.946 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.946 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.946 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.946 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.946 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.947 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.947 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.947 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.947 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.947 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.947 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-08-18T23:53:32.947 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-08-18T23:53:32.947 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.947 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.947 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.947 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.947 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-08-18T23:53:32.948 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-08-18T23:53:32.948 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.948 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.948 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.948 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.948 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.948 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-08-18T23:53:32.948 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-08-18T23:53:32.948 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.948 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.948 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.948 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.949 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.949 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.949 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.949 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.949 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.949 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.949 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.949 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-08-18T23:53:32.949 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-08-18T23:53:32.949 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.949 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.950 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.950 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.950 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.950 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.950 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.950 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.950 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.950 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.950 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.950 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-08-18T23:53:32.950 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-08-18T23:53:32.950 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.951 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.951 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.951 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.951 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.951 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.951 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.951 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.951 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.951 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.951 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.951 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.951 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.952 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-08-18T23:53:32.952 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.952 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.952 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.952 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-08-18T23:53:32.952 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-08-18T23:53:32.952 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.952 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.952 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.952 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.953 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.953 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.953 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.953 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-08-18T23:53:32.953 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.953 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.953 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.953 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-08-18T23:53:32.953 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-08-18T23:53:32.953 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.953 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.954 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.954 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.954 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.954 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.954 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.954 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-08-18T23:53:32.954 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.954 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.954 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.954 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.954 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.955 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.955 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.955 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.955 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.955 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.955 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.955 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.955 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.955 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.955 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.955 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.955 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.955 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.956 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.956 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.956 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.956 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.956 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.956 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.956 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.956 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.956 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.956 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.956 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.956 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.956 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.957 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.957 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.957 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.957 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.957 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.957 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.957 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.957 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.957 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.957 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.957 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.957 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.958 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.958 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.958 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.958 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.958 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.958 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.958 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.958 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.958 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.958 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.958 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.958 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.959 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.959 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.959 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.959 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.959 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.959 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.959 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.959 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.959 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.959 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.959 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.960 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.960 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.960 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.960 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.960 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.960 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-08-18T23:53:32.960 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.960 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.960 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.960 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.960 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.961 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.961 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.961 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.961 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.961 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.961 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.961 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.961 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.961 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.961 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.961 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.961 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.961 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.962 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.962 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.962 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.962 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.962 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.962 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.962 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.962 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.962 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.962 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.962 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.963 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.963 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.963 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.963 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.963 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.963 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.963 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.963 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.963 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.963 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.963 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.963 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.964 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.964 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.964 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.964 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.964 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.964 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.964 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.964 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.964 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.964 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.964 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.964 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.964 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.965 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.965 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.965 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.965 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.965 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.965 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.965 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.965 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.965 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.965 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.965 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.965 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.966 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.966 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.966 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.966 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.966 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.966 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-08-18T23:53:32.966 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.966 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.966 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.966 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.966 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.966 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.967 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.967 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.967 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.967 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.967 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.967 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.967 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.967 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.967 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.967 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.967 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.967 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.967 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-08-18T23:53:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-08-18T23:53:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-08-18T23:53:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-18T23:53:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-08-18T23:53:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-08-18T23:53:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-08-18T23:53:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.979 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.979 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.979 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.979 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.979 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.979 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.979 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.979 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.979 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.979 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.979 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.980 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.980 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.980 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.980 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.980 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.980 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.980 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:32.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:32.980 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-08-18T23:53:32.980 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:32.980 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:32.980 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-08-18T23:53:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-08-18T23:53:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-18T23:53:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.982 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-08-18T23:53:32.982 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-08-18T23:53:32.982 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.982 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.982 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.982 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.982 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.982 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.982 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.982 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-18T23:53:32.982 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.982 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.983 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.983 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-08-18T23:53:32.983 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-08-18T23:53:32.983 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.983 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.983 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.983 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:32.983 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.983 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.983 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.983 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-18T23:53:32.983 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.984 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.984 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.984 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-08-18T23:53:32.984 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-08-18T23:53:32.984 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.984 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.984 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.984 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.984 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.984 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.984 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.985 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-18T23:53:32.985 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.985 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.985 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.985 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-08-18T23:53:32.985 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-08-18T23:53:32.985 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.985 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.985 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.985 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.985 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.985 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.986 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.986 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-18T23:53:32.986 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.986 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.986 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.986 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-08-18T23:53:32.986 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-08-18T23:53:32.986 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.986 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.986 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.986 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.986 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.986 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.987 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.987 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.987 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.987 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.987 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.987 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-08-18T23:53:32.987 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-08-18T23:53:32.987 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.987 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.987 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.987 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.988 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.988 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.988 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.988 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.988 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.988 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.988 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.988 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-08-18T23:53:32.988 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-08-18T23:53:32.988 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.988 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.988 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.989 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-18T23:53:32.989 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.989 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.989 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.989 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.989 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.989 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.989 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.989 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-08-18T23:53:32.989 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-08-18T23:53:32.989 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.989 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.990 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.990 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.990 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.990 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.990 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.990 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.990 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.990 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.990 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.990 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-08-18T23:53:32.990 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-08-18T23:53:32.991 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.991 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.991 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.991 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-08-18T23:53:32.991 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-08-18T23:53:32.991 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.991 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.991 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.991 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.991 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.991 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.991 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.992 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-08-18T23:53:32.992 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-08-18T23:53:32.992 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:32.992 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.992 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.992 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-08-18T23:53:32.992 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.992 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.992 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.992 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.992 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.992 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.992 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.992 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.993 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-08-18T23:53:32.993 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-08-18T23:53:32.993 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.993 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.993 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.993 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.993 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.993 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.993 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.993 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.993 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.993 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.993 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.994 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-08-18T23:53:32.994 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-08-18T23:53:32.994 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.994 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.994 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:32.994 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.994 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.994 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.994 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.994 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.994 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.995 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.995 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.995 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:32.995 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:32.995 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.995 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.995 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.995 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:32.995 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.995 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.995 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.996 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.996 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.996 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.996 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.996 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.996 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.996 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.996 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.996 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.996 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.996 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.997 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:32.997 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:32.997 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:32.997 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.997 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.997 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:32.997 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.997 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.997 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:32.997 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.997 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.997 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.997 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.998 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:32.998 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:32.998 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:32.998 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:32.998 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:32.998 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:32.998 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:32.998 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:32.998 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:32.998 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:32.998 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:32.998 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:32.999 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:32.999 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:32.999 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:32.999 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:32.999 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:32.999 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:32.999 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:32.999 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:32.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:32.999 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:32.999 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:32.999 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.000 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.000 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.000 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.000 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.000 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.000 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.000 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.000 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.000 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.000 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.000 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-08-18T23:53:33.000 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-08-18T23:53:33.000 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.001 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.001 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.001 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.001 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.001 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.001 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.001 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.001 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.001 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.001 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.001 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.001 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-08-18T23:53:33.001 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-08-18T23:53:33.002 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.002 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.002 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.002 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.002 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.002 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.002 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.002 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.002 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.002 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.002 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.003 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-08-18T23:53:33.003 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-08-18T23:53:33.003 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.003 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.003 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.003 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-18T23:53:33.003 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.003 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.003 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.003 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.003 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.004 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.004 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.004 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.004 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.004 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.004 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-08-18T23:53:33.004 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.004 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.004 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.004 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.004 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.004 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.004 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.005 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.005 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.005 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.005 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.005 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.005 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.005 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.005 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.005 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.005 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.005 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.005 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.006 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.006 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.006 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.006 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.006 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.006 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.006 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.006 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.006 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.006 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.006 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.006 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.007 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.007 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.007 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.007 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.007 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.007 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.007 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.007 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.007 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.007 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.007 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.007 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.008 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.008 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.008 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.008 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.008 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.008 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.008 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.008 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.008 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.008 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.008 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.008 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.009 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.009 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.009 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.009 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.009 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.009 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.009 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.009 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.009 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.009 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.009 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.009 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.010 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.010 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.010 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.010 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.010 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.010 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-08-18T23:53:33.010 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.010 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.010 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.010 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.010 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.011 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.011 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.011 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.011 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.011 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.011 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.011 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.011 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.011 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.011 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.011 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.011 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.011 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.011 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.012 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.012 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.012 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.012 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.012 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.012 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.012 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.012 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.012 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.012 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.012 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.012 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.013 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.013 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.013 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.013 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.013 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.013 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.013 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.013 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.013 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.013 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.013 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.013 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.014 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.014 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.014 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.014 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.014 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.014 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.014 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.014 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.014 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.014 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.014 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.014 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.015 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.015 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.015 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.015 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.015 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.015 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.015 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.015 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.015 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.015 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.015 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.015 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.016 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.016 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.016 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-08-18T23:53:33.016 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-08-18T23:53:33.016 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.016 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.016 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.016 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-08-18T23:53:33.016 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.016 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.016 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.017 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-18T23:53:33.017 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.017 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.017 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.017 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-08-18T23:53:33.017 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-08-18T23:53:33.017 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.017 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.017 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.017 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.017 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.017 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.017 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.018 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-08-18T23:53:33.018 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.018 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.018 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.018 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-08-18T23:53:33.018 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-08-18T23:53:33.018 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.018 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.018 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.018 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-08-18T23:53:33.018 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.019 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.019 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.019 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-18T23:53:33.019 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.019 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.019 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.019 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-08-18T23:53:33.019 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-08-18T23:53:33.019 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.019 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.019 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.019 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.020 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.020 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.020 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.020 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-18T23:53:33.020 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.020 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.020 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.020 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.020 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.020 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.020 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-08-18T23:53:33.020 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.021 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.021 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.021 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-08-18T23:53:33.021 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-08-18T23:53:33.021 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-08-18T23:53:33.021 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.021 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.021 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-08-18T23:53:33.021 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.021 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.021 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.021 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.021 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.022 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.022 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.022 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-08-18T23:53:33.022 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-08-18T23:53:33.022 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.022 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.022 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.022 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.022 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.022 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.022 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.023 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.023 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.023 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.023 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.023 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-08-18T23:53:33.023 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-08-18T23:53:33.023 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.023 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.023 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.023 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-08-18T23:53:33.023 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.024 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.024 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.024 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.024 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.024 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.024 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.024 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.024 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.024 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.024 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.024 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.026 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.026 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.026 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.026 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.026 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.026 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.026 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.026 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.026 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.026 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.026 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.027 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.027 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.027 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.027 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.027 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.027 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.027 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.027 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.027 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.027 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.027 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.027 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.027 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.028 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.028 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.028 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.028 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.028 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.028 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.028 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.028 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.028 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.028 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.028 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.028 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.028 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.029 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.029 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.029 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.029 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.029 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.029 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.029 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.029 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.029 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.029 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.029 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.029 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.030 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.030 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.030 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.030 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.030 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.030 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.030 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.030 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.030 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.030 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.030 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.030 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.030 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.031 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-08-18T23:53:33.031 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-08-18T23:53:33.031 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.031 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.031 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.031 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.031 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.031 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.031 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.031 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.031 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.031 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.032 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.032 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-08-18T23:53:33.032 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-08-18T23:53:33.032 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.032 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.032 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.032 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-08-18T23:53:33.032 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.032 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.032 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.032 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.033 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.033 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.033 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.033 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-08-18T23:53:33.033 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-08-18T23:53:33.033 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.033 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.033 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.033 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-18T23:53:33.033 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.033 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.033 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.034 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.034 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.034 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.034 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.034 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.034 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.034 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.034 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-08-18T23:53:33.034 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.034 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.034 INFO:teuthology.orchestra.run.smithi078.stdout: "gid": 24113, 2024-08-18T23:53:33.034 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "y", 2024-08-18T23:53:33.035 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr_features": 4540701547738038271, 2024-08-18T23:53:33.035 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-08-18T23:53:33.035 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.035 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-08-18T23:53:33.035 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.035 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.035 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.035 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-08-18T23:53:33.035 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-08-18T23:53:33.035 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.035 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.035 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.035 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-08-18T23:53:33.036 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.036 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.036 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.036 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-08-18T23:53:33.036 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.036 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.036 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.036 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.036 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.036 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.036 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.036 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.037 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.037 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.037 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.037 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.037 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.037 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.037 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.037 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.037 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.037 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.037 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.037 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.037 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.038 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.038 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.038 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.038 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.038 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.038 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.038 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.038 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.038 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.038 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.038 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.039 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.039 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.039 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.039 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.039 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.039 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.039 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.039 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.039 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.039 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.039 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.039 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.040 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.040 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.040 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.040 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.040 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.040 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.040 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.040 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.040 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.040 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.040 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.040 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.041 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.041 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.041 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.041 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.041 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.041 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.041 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.041 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.041 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.041 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.041 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.041 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.041 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.042 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-08-18T23:53:33.042 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-08-18T23:53:33.042 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.042 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.042 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.042 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.042 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.042 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.042 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.042 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to", 2024-08-18T23:53:33.042 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.043 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.043 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.043 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-08-18T23:53:33.043 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-08-18T23:53:33.043 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.043 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.043 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.043 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-08-18T23:53:33.043 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.043 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.043 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.043 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-08-18T23:53:33.044 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.044 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.044 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.044 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-08-18T23:53:33.044 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-08-18T23:53:33.044 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.044 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.044 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.044 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.044 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.044 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.044 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.044 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-08-18T23:53:33.045 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.045 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.045 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.045 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-08-18T23:53:33.045 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-08-18T23:53:33.045 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.045 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.045 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.045 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.045 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.045 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.045 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.046 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-08-18T23:53:33.046 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.046 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.046 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.046 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-08-18T23:53:33.046 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-08-18T23:53:33.046 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.046 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.046 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.046 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-08-18T23:53:33.046 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.046 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.047 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.047 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-08-18T23:53:33.047 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.047 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.047 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.047 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-08-18T23:53:33.047 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-08-18T23:53:33.047 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.047 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.047 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.047 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.047 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-08-18T23:53:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-08-18T23:53:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-08-18T23:53:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.048 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-18T23:53:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-08-18T23:53:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-08-18T23:53:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.049 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-08-18T23:53:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.050 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-08-18T23:53:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-08-18T23:53:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-08-18T23:53:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.051 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-08-18T23:53:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-08-18T23:53:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-08-18T23:53:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-08-18T23:53:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.052 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-08-18T23:53:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-18T23:53:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-08-18T23:53:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-08-18T23:53:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-08-18T23:53:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-08-18T23:53:33.053 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-08-18T23:53:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-08-18T23:53:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-18T23:53:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-18T23:53:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-08-18T23:53:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-08-18T23:53:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-08-18T23:53:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.054 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-08-18T23:53:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-08-18T23:53:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-08-18T23:53:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-18T23:53:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-08-18T23:53:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-08-18T23:53:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.055 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.056 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.056 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-08-18T23:53:33.056 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.056 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.056 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.056 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-18T23:53:33.056 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-18T23:53:33.056 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.056 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.056 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-08-18T23:53:33.056 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-08-18T23:53:33.056 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:33.056 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.057 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.057 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-08-18T23:53:33.057 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-08-18T23:53:33.057 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-08-18T23:53:33.057 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.057 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-08-18T23:53:33.057 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-18T23:53:33.057 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.057 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.057 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-08-18T23:53:33.057 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-08-18T23:53:33.057 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.058 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.058 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.058 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-08-18T23:53:33.058 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.058 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.058 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.058 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-08-18T23:53:33.058 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-18T23:53:33.058 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.058 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.058 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-08-18T23:53:33.058 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-08-18T23:53:33.059 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-08-18T23:53:33.059 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.059 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.059 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-08-18T23:53:33.059 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-08-18T23:53:33.059 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-08-18T23:53:33.059 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.059 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-18T23:53:33.059 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-18T23:53:33.059 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.059 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.059 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.060 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.060 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.060 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.060 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.060 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.060 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.060 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.060 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.060 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.060 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.060 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.060 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.061 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.061 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.061 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.061 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.061 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.061 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.061 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.061 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.061 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.061 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.061 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.061 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.061 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.061 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.062 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.062 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.062 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.062 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.062 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.062 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.062 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.062 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.062 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.062 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.062 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.062 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.063 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.063 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.063 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.063 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.063 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.063 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.063 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.063 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.063 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.063 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.063 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.063 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.063 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.064 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.064 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.064 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.064 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.064 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.064 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.064 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.064 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.064 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.064 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.064 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.065 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.065 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.065 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.065 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.065 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.065 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.065 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.065 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.065 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-08-18T23:53:33.065 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-08-18T23:53:33.065 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:33.065 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.065 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.066 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-08-18T23:53:33.066 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.066 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.066 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.066 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-18T23:53:33.066 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.066 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.066 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.066 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.066 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-08-18T23:53:33.066 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-08-18T23:53:33.066 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.067 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.067 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.067 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-08-18T23:53:33.067 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.067 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.067 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.067 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-08-18T23:53:33.067 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-08-18T23:53:33.067 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-08-18T23:53:33.067 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-08-18T23:53:33.067 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-08-18T23:53:33.067 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.068 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-08-18T23:53:33.068 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.068 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.068 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.068 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-08-18T23:53:33.068 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-08-18T23:53:33.068 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.068 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.068 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.068 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.068 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.068 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.069 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.069 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-18T23:53:33.069 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.069 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.069 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.069 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-08-18T23:53:33.069 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-08-18T23:53:33.069 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.069 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.069 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.069 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-08-18T23:53:33.069 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.070 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.070 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.070 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-18T23:53:33.070 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.070 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.070 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.070 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.070 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-08-18T23:53:33.070 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-08-18T23:53:33.070 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.070 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.070 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.071 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-08-18T23:53:33.071 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-08-18T23:53:33.071 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.071 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.071 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-18T23:53:33.071 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-18T23:53:33.071 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.071 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.071 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-08-18T23:53:33.071 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-08-18T23:53:33.071 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-08-18T23:53:33.071 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.072 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.072 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:33.072 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.072 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.072 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.072 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-18T23:53:33.072 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.072 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.072 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.072 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.072 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.072 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.072 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-08-18T23:53:33.073 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.073 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.073 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.073 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-08-18T23:53:33.073 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-08-18T23:53:33.073 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:33.073 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.073 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.073 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-08-18T23:53:33.073 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.073 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.073 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.073 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-18T23:53:33.074 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.074 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.074 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.074 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-08-18T23:53:33.074 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-08-18T23:53:33.074 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.074 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.074 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.074 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-08-18T23:53:33.074 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.074 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.075 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.075 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-18T23:53:33.075 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.075 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.075 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.075 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.075 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-08-18T23:53:33.075 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-08-18T23:53:33.075 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.075 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.075 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.075 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-08-18T23:53:33.075 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.076 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.076 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.076 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-18T23:53:33.076 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.076 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.076 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.076 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-08-18T23:53:33.076 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-08-18T23:53:33.076 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.076 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.076 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.076 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.077 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.077 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.077 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.077 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-18T23:53:33.077 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-18T23:53:33.077 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.077 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.077 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-08-18T23:53:33.077 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-08-18T23:53:33.077 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.077 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.078 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.078 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-08-18T23:53:33.078 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.078 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.078 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.078 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-18T23:53:33.078 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.078 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.078 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.078 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-08-18T23:53:33.078 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-08-18T23:53:33.078 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:33.079 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.079 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.079 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-08-18T23:53:33.079 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.079 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.079 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.079 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-18T23:53:33.079 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.079 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.079 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.079 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-08-18T23:53:33.079 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-08-18T23:53:33.080 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.080 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.080 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.080 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.080 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.080 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.080 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.080 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-08-18T23:53:33.080 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-08-18T23:53:33.080 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-08-18T23:53:33.080 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.080 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-18T23:53:33.081 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.081 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.081 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.081 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-08-18T23:53:33.081 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-08-18T23:53:33.081 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.081 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.081 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.081 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.081 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.081 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.081 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.082 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-18T23:53:33.082 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.082 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.082 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.082 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-08-18T23:53:33.082 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-08-18T23:53:33.082 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.082 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.082 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.082 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.083 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.083 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.083 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.083 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-18T23:53:33.083 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.083 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.083 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.083 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-08-18T23:53:33.083 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-08-18T23:53:33.083 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.083 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.083 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.084 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.084 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.084 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.084 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.084 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-18T23:53:33.084 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.084 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.084 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.084 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-08-18T23:53:33.084 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-08-18T23:53:33.084 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.085 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.085 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.085 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-08-18T23:53:33.085 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.085 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.085 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.085 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-08-18T23:53:33.085 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.085 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.085 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.085 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-08-18T23:53:33.085 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-08-18T23:53:33.086 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.086 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.086 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.086 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-18T23:53:33.086 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.086 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.086 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.086 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-08-18T23:53:33.086 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.086 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.086 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.086 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-08-18T23:53:33.087 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-08-18T23:53:33.087 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.087 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.087 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.087 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-18T23:53:33.087 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.087 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.087 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.087 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-08-18T23:53:33.087 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.087 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.087 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.088 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-08-18T23:53:33.088 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-08-18T23:53:33.088 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.088 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.088 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.088 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-08-18T23:53:33.088 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.088 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.088 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.088 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-08-18T23:53:33.088 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.088 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.088 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.089 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-08-18T23:53:33.089 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-08-18T23:53:33.089 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.089 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.089 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.089 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-18T23:53:33.089 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.089 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.089 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.089 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-08-18T23:53:33.089 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.090 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.090 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.090 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-08-18T23:53:33.090 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-08-18T23:53:33.090 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.090 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.090 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.090 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-18T23:53:33.090 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.090 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.090 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.091 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-08-18T23:53:33.091 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.091 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.091 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.091 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-08-18T23:53:33.091 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-08-18T23:53:33.091 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.091 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.091 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.091 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-18T23:53:33.091 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.091 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.092 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.092 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-08-18T23:53:33.092 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.092 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.092 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.092 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-08-18T23:53:33.092 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-08-18T23:53:33.092 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.092 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.092 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.092 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-18T23:53:33.092 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.093 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.093 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.093 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-08-18T23:53:33.093 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.093 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.093 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.093 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-08-18T23:53:33.093 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-08-18T23:53:33.093 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.093 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.093 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.093 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-18T23:53:33.094 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.094 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.094 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.094 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-08-18T23:53:33.094 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.094 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.094 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.094 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-08-18T23:53:33.094 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-08-18T23:53:33.094 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.094 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.095 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.095 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-18T23:53:33.095 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.095 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.095 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.095 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-08-18T23:53:33.095 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.095 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.095 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.095 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-08-18T23:53:33.095 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-08-18T23:53:33.096 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.096 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.096 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.096 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-08-18T23:53:33.096 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.096 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.096 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.096 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-08-18T23:53:33.096 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.096 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.096 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.096 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-08-18T23:53:33.097 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-08-18T23:53:33.097 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.097 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.097 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.097 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-18T23:53:33.097 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.097 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.097 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.097 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-08-18T23:53:33.097 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.097 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.097 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.097 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.098 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-08-18T23:53:33.098 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-08-18T23:53:33.098 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.098 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.098 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.098 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-08-18T23:53:33.098 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.098 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.098 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.098 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-08-18T23:53:33.098 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.098 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.099 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.099 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-08-18T23:53:33.099 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-08-18T23:53:33.099 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.099 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.099 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.099 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-18T23:53:33.099 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.099 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.099 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.099 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-08-18T23:53:33.099 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.099 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.100 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.100 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-08-18T23:53:33.100 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-08-18T23:53:33.100 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.100 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.100 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.100 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-18T23:53:33.100 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.100 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.100 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.100 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-08-18T23:53:33.100 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.101 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.101 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.101 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-08-18T23:53:33.101 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-08-18T23:53:33.101 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.101 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.101 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.101 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-18T23:53:33.101 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.101 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.101 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.101 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-08-18T23:53:33.102 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.102 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.102 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.102 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-08-18T23:53:33.102 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-08-18T23:53:33.102 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.102 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.102 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.102 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.102 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.102 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.102 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.103 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-08-18T23:53:33.103 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.103 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.103 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.103 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-08-18T23:53:33.103 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-08-18T23:53:33.103 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.103 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.103 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.103 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-08-18T23:53:33.103 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.104 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.104 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.104 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-18T23:53:33.104 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.104 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.104 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.104 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-08-18T23:53:33.104 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-08-18T23:53:33.104 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.104 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.104 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.104 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-08-18T23:53:33.105 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.105 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.105 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.105 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-18T23:53:33.105 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.105 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.105 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.105 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-08-18T23:53:33.105 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-08-18T23:53:33.105 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.105 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.105 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.105 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-08-18T23:53:33.106 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.106 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.106 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.106 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-18T23:53:33.106 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.106 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.106 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.106 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-08-18T23:53:33.106 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-08-18T23:53:33.106 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.106 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.107 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.107 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-08-18T23:53:33.107 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.107 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.107 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.107 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-08-18T23:53:33.107 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.107 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.107 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.107 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-08-18T23:53:33.107 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-08-18T23:53:33.107 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.107 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.108 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.108 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.108 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.108 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.108 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.108 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-18T23:53:33.108 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.108 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.108 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.108 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-08-18T23:53:33.108 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-08-18T23:53:33.109 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.109 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.109 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.109 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-08-18T23:53:33.109 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.109 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.109 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.109 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-08-18T23:53:33.109 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.109 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.109 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.110 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-08-18T23:53:33.110 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-08-18T23:53:33.110 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.110 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.110 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.110 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-18T23:53:33.110 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.110 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.110 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.110 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-18T23:53:33.110 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.110 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.110 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.111 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-08-18T23:53:33.111 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-08-18T23:53:33.111 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.111 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.111 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.111 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-08-18T23:53:33.111 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.111 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.111 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.111 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-08-18T23:53:33.111 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.112 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.112 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.112 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-08-18T23:53:33.112 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-08-18T23:53:33.112 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.112 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.112 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.112 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.112 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.112 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.112 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.112 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-18T23:53:33.113 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.113 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.113 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.113 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-08-18T23:53:33.113 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-08-18T23:53:33.113 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.113 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.113 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.113 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.113 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.113 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.113 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.114 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-18T23:53:33.114 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.114 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.114 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.114 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.114 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.114 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.114 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.114 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.114 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.114 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.114 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.115 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.115 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.115 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.115 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.115 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.115 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.115 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.115 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.115 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.115 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.115 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.115 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.116 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-08-18T23:53:33.116 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-08-18T23:53:33.116 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.116 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.116 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.116 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.116 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.116 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.116 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.116 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-18T23:53:33.116 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.116 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.116 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.117 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.117 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.117 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.117 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.117 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.117 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.117 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.117 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.117 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.117 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-18T23:53:33.117 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.118 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.118 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.118 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.118 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.118 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.118 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.118 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.118 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.118 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.118 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.118 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.119 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.119 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.119 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.119 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.119 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.119 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.119 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.119 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.119 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.119 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.119 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.119 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.120 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.120 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.120 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.120 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.120 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.120 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.120 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.120 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.120 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.120 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.120 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.120 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.121 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-18T23:53:33.121 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-18T23:53:33.121 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.121 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.121 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.121 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.121 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.121 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.121 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.121 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-18T23:53:33.121 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.121 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.122 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.122 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-18T23:53:33.122 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-18T23:53:33.122 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.122 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.122 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.122 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-08-18T23:53:33.122 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.122 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.122 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.122 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-18T23:53:33.122 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.123 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.123 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.123 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-08-18T23:53:33.123 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-08-18T23:53:33.123 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.123 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.123 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.123 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:33.123 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.123 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.123 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.123 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-08-18T23:53:33.123 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.124 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.124 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.124 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-08-18T23:53:33.124 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-08-18T23:53:33.124 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.124 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.124 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.124 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:33.124 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.124 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.124 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.124 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-18T23:53:33.124 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-08-18T23:53:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-08-18T23:53:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-08-18T23:53:33.126 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.126 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.126 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.126 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-08-18T23:53:33.126 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-08-18T23:53:33.126 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.126 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.126 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.126 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-08-18T23:53:33.126 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.126 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.126 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.127 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-08-18T23:53:33.127 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-08-18T23:53:33.127 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.127 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-08-18T23:53:33.127 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.127 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.127 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.127 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-08-18T23:53:33.127 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-08-18T23:53:33.127 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.127 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.127 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.128 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-08-18T23:53:33.128 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.128 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.128 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.128 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-18T23:53:33.128 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.128 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.128 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.128 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-08-18T23:53:33.128 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-08-18T23:53:33.128 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.129 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.129 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.129 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-18T23:53:33.129 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.129 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.129 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.129 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-18T23:53:33.129 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.129 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.129 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.129 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-08-18T23:53:33.129 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-08-18T23:53:33.129 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.130 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.130 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.130 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.130 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.130 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.130 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.130 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-18T23:53:33.130 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.130 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.130 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.131 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-08-18T23:53:33.131 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-08-18T23:53:33.131 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.131 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.131 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.131 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.131 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.131 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.131 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.131 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-18T23:53:33.131 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.131 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.131 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.132 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-08-18T23:53:33.132 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-08-18T23:53:33.132 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.132 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.132 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.132 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.132 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.132 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.132 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.132 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-18T23:53:33.132 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.132 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.133 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.133 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-08-18T23:53:33.133 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-08-18T23:53:33.133 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.133 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.133 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.133 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.133 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.133 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.133 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.133 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-18T23:53:33.133 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.134 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.134 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.134 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-08-18T23:53:33.134 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-08-18T23:53:33.134 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.134 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.134 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.134 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.134 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.134 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.134 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.134 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-18T23:53:33.135 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.135 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.135 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.135 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-08-18T23:53:33.135 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-08-18T23:53:33.135 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.135 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.135 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.135 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-08-18T23:53:33.135 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.135 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.135 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.136 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-08-18T23:53:33.136 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.136 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.136 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.136 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-08-18T23:53:33.136 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-08-18T23:53:33.136 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.136 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.136 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.136 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.136 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.136 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.136 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.137 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-18T23:53:33.137 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.137 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.137 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.137 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-08-18T23:53:33.137 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-08-18T23:53:33.137 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.137 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.137 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.137 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.137 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.137 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.137 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.138 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-18T23:53:33.138 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.138 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.138 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.138 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-08-18T23:53:33.138 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-08-18T23:53:33.138 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.138 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.138 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.138 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.138 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.139 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.139 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.139 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-18T23:53:33.139 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.139 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.139 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.139 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-08-18T23:53:33.139 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-08-18T23:53:33.139 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.139 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.139 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.139 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.140 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.140 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.140 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.140 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-08-18T23:53:33.140 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.140 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.140 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.140 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-08-18T23:53:33.140 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-08-18T23:53:33.140 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.140 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.140 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.141 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.141 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.141 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.141 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.141 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-18T23:53:33.141 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.141 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.141 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.141 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-08-18T23:53:33.141 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-08-18T23:53:33.141 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.141 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.142 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.142 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.142 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.142 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.142 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.142 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-18T23:53:33.142 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.142 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.142 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.142 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.142 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-08-18T23:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-08-18T23:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-08-18T23:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-08-18T23:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-18T23:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-08-18T23:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-08-18T23:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-08-18T23:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-18T23:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-08-18T23:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-18T23:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-18T23:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-08-18T23:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-18T23:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-18T23:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-18T23:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-08-18T23:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-08-18T23:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-18T23:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-18T23:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-08-18T23:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-18T23:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-18T23:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-18T23:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-18T23:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-18T23:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-18T23:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-18T23:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-18T23:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-18T23:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-18T23:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-18T23:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-18T23:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-18T23:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-18T23:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-18T23:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-18T23:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-18T23:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-08-18T23:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-18T23:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-08-18T23:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-18T23:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-18T23:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-08-18T23:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-08-18T23:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-08-18T23:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-18T23:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-08-18T23:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-18T23:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-18T23:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-18T23:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-18T23:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-18T23:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-18T23:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-18T23:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-18T23:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-08-18T23:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-18T23:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-18T23:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-18T23:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-18T23:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-18T23:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-18T23:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-18T23:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-18T23:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-18T23:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-18T23:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-18T23:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-18T23:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-18T23:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-18T23:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-18T23:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-18T23:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-18T23:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-08-18T23:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-18T23:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-18T23:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-18T23:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-18T23:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-18T23:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-18T23:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-18T23:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-18T23:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-08-18T23:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-18T23:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-18T23:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-08-18T23:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-08-18T23:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-18T23:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-18T23:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-18T23:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-08-18T23:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-08-18T23:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-18T23:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-08-18T23:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-18T23:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-18T23:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-18T23:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-08-18T23:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-18T23:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-18T23:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-18T23:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-18T23:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-08-18T23:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-08-18T23:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-08-18T23:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-08-18T23:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-08-18T23:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-08-18T23:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-08-18T23:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-08-18T23:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-08-18T23:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-08-18T23:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-08-18T23:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-08-18T23:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-08-18T23:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-08-18T23:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-08-18T23:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-08-18T23:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-08-18T23:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-08-18T23:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-08-18T23:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-08-18T23:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-08-18T23:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-08-18T23:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-08-18T23:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-08-18T23:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-08-18T23:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-08-18T23:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-08-18T23:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-08-18T23:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-08-18T23:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-08-18T23:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-08-18T23:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-08-18T23:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-08-18T23:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-08-18T23:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-08-18T23:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.218 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.218 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-08-18T23:53:33.218 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-08-18T23:53:33.218 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-08-18T23:53:33.218 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.218 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.218 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.218 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.218 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.218 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.218 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-08-18T23:53:33.218 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-08-18T23:53:33.218 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.219 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.219 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.219 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.219 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.219 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.219 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.219 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.219 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.219 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.219 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.219 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.219 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-08-18T23:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-08-18T23:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-08-18T23:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-08-18T23:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-08-18T23:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-08-18T23:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-08-18T23:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-18T23:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-08-18T23:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-08-18T23:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-08-18T23:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.230 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.230 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-18T23:53:33.230 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.230 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.230 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-08-18T23:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-08-18T23:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-08-18T23:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-08-18T23:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-08-18T23:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-08-18T23:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-08-18T23:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-18T23:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-08-18T23:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-08-18T23:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-18T23:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-08-18T23:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-08-18T23:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-08-18T23:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-08-18T23:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-08-18T23:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-08-18T23:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-08-18T23:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-18T23:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-08-18T23:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-08-18T23:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-08-18T23:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-08-18T23:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-08-18T23:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-08-18T23:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-08-18T23:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-08-18T23:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-08-18T23:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-08-18T23:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-08-18T23:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-08-18T23:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-08-18T23:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-08-18T23:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-08-18T23:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-08-18T23:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-18T23:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-08-18T23:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-08-18T23:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-08-18T23:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-18T23:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-08-18T23:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-08-18T23:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-08-18T23:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-08-18T23:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-08-18T23:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-08-18T23:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-08-18T23:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-18T23:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-08-18T23:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-08-18T23:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-08-18T23:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-08-18T23:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-08-18T23:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-08-18T23:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-08-18T23:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-08-18T23:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-08-18T23:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-08-18T23:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-18T23:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-08-18T23:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-08-18T23:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-08-18T23:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-08-18T23:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-08-18T23:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-18T23:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-08-18T23:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-08-18T23:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-08-18T23:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-08-18T23:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-08-18T23:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-08-18T23:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-18T23:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-08-18T23:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-08-18T23:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "k8sevents", 2024-08-18T23:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_event_retention_days": { 2024-08-18T23:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ceph_event_retention_days", 2024-08-18T23:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7", 2024-08-18T23:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-18T23:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "config_check_secs": { 2024-08-18T23:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_check_secs", 2024-08-18T23:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "10", 2024-08-18T23:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-18T23:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-08-18T23:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-08-18T23:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-08-18T23:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-08-18T23:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-08-18T23:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-18T23:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-08-18T23:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-08-18T23:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-08-18T23:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-18T23:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-08-18T23:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-08-18T23:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-08-18T23:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-08-18T23:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-08-18T23:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-08-18T23:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-08-18T23:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-08-18T23:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-08-18T23:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-08-18T23:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-08-18T23:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-08-18T23:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-08-18T23:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-08-18T23:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-18T23:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-18T23:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mds_autoscaler", 2024-08-18T23:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-08-18T23:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-08-18T23:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-08-18T23:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-08-18T23:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-08-18T23:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-18T23:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-08-18T23:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-08-18T23:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-08-18T23:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-08-18T23:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-08-18T23:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-08-18T23:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-08-18T23:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-08-18T23:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-08-18T23:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-08-18T23:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-08-18T23:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-08-18T23:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-08-18T23:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-08-18T23:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-08-18T23:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-08-18T23:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-08-18T23:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-18T23:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-08-18T23:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-08-18T23:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-08-18T23:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-18T23:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-08-18T23:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-08-18T23:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-08-18T23:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-08-18T23:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-08-18T23:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-08-18T23:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-08-18T23:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-08-18T23:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-08-18T23:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-08-18T23:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-08-18T23:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-08-18T23:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-08-18T23:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-08-18T23:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-08-18T23:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-18T23:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-18T23:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-08-18T23:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-08-18T23:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-18T23:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-18T23:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-08-18T23:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-08-18T23:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-08-18T23:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-08-18T23:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-08-18T23:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-08-18T23:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-08-18T23:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-18T23:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-08-18T23:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-08-18T23:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-08-18T23:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-18T23:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-08-18T23:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-08-18T23:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-08-18T23:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-08-18T23:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-08-18T23:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-08-18T23:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-08-18T23:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-08-18T23:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-08-18T23:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-08-18T23:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-08-18T23:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-08-18T23:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-08-18T23:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-08-18T23:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-08-18T23:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-08-18T23:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-08-18T23:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-08-18T23:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-08-18T23:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-08-18T23:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-08-18T23:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-08-18T23:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-08-18T23:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-08-18T23:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-08-18T23:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-08-18T23:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-08-18T23:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-08-18T23:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-08-18T23:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-08-18T23:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rook", 2024-08-18T23:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "storage_class": { 2024-08-18T23:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "storage_class", 2024-08-18T23:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "local", 2024-08-18T23:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-18T23:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-08-18T23:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-08-18T23:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-08-18T23:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-08-18T23:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-08-18T23:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-08-18T23:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-08-18T23:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-08-18T23:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-08-18T23:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-08-18T23:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-08-18T23:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-08-18T23:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-08-18T23:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-08-18T23:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-08-18T23:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-08-18T23:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-08-18T23:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-08-18T23:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-08-18T23:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-08-18T23:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-08-18T23:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-08-18T23:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-08-18T23:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-08-18T23:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-08-18T23:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-08-18T23:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-08-18T23:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-08-18T23:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-08-18T23:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-08-18T23:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-08-18T23:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-08-18T23:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-08-18T23:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-08-18T23:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-08-18T23:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-08-18T23:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-08-18T23:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-08-18T23:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-08-18T23:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-08-18T23:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-18T23:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-08-18T23:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-08-18T23:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-08-18T23:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-08-18T23:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-08-18T23:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-08-18T23:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-18T23:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-08-18T23:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-08-18T23:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-18T23:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-08-18T23:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-08-18T23:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-18T23:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-08-18T23:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-08-18T23:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-18T23:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-08-18T23:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-08-18T23:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-18T23:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-08-18T23:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-08-18T23:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-08-18T23:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-08-18T23:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-08-18T23:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-08-18T23:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-18T23:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-08-18T23:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-08-18T23:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-08-18T23:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-08-18T23:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-08-18T23:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-08-18T23:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-08-18T23:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-08-18T23:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-08-18T23:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-08-18T23:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-08-18T23:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-08-18T23:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-08-18T23:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-08-18T23:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-08-18T23:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-08-18T23:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-08-18T23:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-08-18T23:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-08-18T23:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-18T23:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-08-18T23:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-08-18T23:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-08-18T23:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-08-18T23:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-08-18T23:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-18T23:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-08-18T23:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-08-18T23:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-08-18T23:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-08-18T23:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-08-18T23:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-08-18T23:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-18T23:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-08-18T23:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-08-18T23:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-18T23:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-08-18T23:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-08-18T23:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-08-18T23:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-08-18T23:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-08-18T23:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-08-18T23:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-08-18T23:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-08-18T23:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-08-18T23:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-08-18T23:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-08-18T23:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-08-18T23:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-08-18T23:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-08-18T23:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-08-18T23:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-08-18T23:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-08-18T23:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-18T23:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-08-18T23:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "modules": [ 2024-08-18T23:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-08-18T23:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "dashboard", 2024-08-18T23:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "iostat", 2024-08-18T23:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "nfs", 2024-08-18T23:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "restful" 2024-08-18T23:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-08-18T23:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-08-18T23:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-08-18T23:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-08-18T23:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-08-18T23:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-08-18T23:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-08-18T23:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-08-18T23:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to", 2024-08-18T23:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-08-18T23:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-08-18T23:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-08-18T23:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-08-18T23:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-08-18T23:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-08-18T23:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-08-18T23:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-08-18T23:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-08-18T23:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-08-18T23:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-08-18T23:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-08-18T23:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-08-18T23:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-08-18T23:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-08-18T23:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-08-18T23:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-08-18T23:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-08-18T23:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-08-18T23:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-18T23:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-08-18T23:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-08-18T23:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-08-18T23:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-08-18T23:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-08-18T23:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-08-18T23:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-08-18T23:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-08-18T23:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-08-18T23:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-08-18T23:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-08-18T23:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-18T23:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-08-18T23:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-08-18T23:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-08-18T23:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-08-18T23:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-08-18T23:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-08-18T23:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-18T23:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-18T23:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-08-18T23:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-08-18T23:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-08-18T23:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-08-18T23:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-08-18T23:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-08-18T23:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-18T23:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-08-18T23:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-08-18T23:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-08-18T23:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-18T23:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-18T23:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-08-18T23:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-08-18T23:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-08-18T23:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-08-18T23:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-08-18T23:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-08-18T23:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-18T23:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-08-18T23:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-08-18T23:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-08-18T23:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-08-18T23:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-18T23:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-08-18T23:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-08-18T23:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-08-18T23:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-08-18T23:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-08-18T23:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-08-18T23:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-18T23:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-18T23:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-08-18T23:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-08-18T23:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-08-18T23:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-18T23:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-08-18T23:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-08-18T23:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-08-18T23:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-08-18T23:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-08-18T23:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-08-18T23:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-08-18T23:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-08-18T23:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-08-18T23:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-08-18T23:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-08-18T23:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-18T23:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-08-18T23:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-08-18T23:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-08-18T23:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-18T23:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-08-18T23:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-08-18T23:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-08-18T23:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-08-18T23:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-18T23:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-18T23:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-08-18T23:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-08-18T23:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-08-18T23:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-18T23:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-08-18T23:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-08-18T23:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-08-18T23:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-08-18T23:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-18T23:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-08-18T23:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-08-18T23:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-08-18T23:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-18T23:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-08-18T23:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-08-18T23:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-08-18T23:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-18T23:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-08-18T23:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-08-18T23:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-18T23:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-18T23:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-08-18T23:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-08-18T23:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-08-18T23:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-18T23:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-08-18T23:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-08-18T23:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-08-18T23:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-18T23:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-08-18T23:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-08-18T23:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-08-18T23:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-08-18T23:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-08-18T23:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-18T23:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-08-18T23:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-08-18T23:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-18T23:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-08-18T23:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-08-18T23:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-18T23:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-08-18T23:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-08-18T23:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-18T23:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-08-18T23:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-08-18T23:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-08-18T23:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-08-18T23:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-08-18T23:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-08-18T23:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-18T23:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-08-18T23:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-08-18T23:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-08-18T23:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-18T23:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-08-18T23:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-08-18T23:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-08-18T23:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-08-18T23:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-08-18T23:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-08-18T23:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-08-18T23:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-18T23:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-08-18T23:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-08-18T23:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-08-18T23:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-18T23:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-08-18T23:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-08-18T23:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-08-18T23:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-18T23:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-08-18T23:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-08-18T23:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-08-18T23:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-18T23:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-08-18T23:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-08-18T23:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-08-18T23:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-18T23:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-08-18T23:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-08-18T23:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-08-18T23:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-18T23:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-08-18T23:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-08-18T23:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-08-18T23:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-08-18T23:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-08-18T23:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-08-18T23:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-08-18T23:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-18T23:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-08-18T23:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-08-18T23:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-08-18T23:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-08-18T23:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-08-18T23:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-08-18T23:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-08-18T23:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-18T23:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-08-18T23:53:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-08-18T23:53:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-08-18T23:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-18T23:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-08-18T23:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-08-18T23:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-08-18T23:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-18T23:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-08-18T23:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-08-18T23:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-08-18T23:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-08-18T23:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-08-18T23:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-08-18T23:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-08-18T23:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-18T23:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-08-18T23:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-08-18T23:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-08-18T23:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-18T23:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-08-18T23:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-08-18T23:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-08-18T23:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-18T23:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-08-18T23:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-08-18T23:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-08-18T23:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-08-18T23:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-08-18T23:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-08-18T23:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-18T23:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-08-18T23:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-08-18T23:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-08-18T23:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-08-18T23:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-08-18T23:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-08-18T23:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-18T23:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-18T23:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-08-18T23:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-08-18T23:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-08-18T23:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-08-18T23:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-08-18T23:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-08-18T23:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-18T23:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-08-18T23:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-08-18T23:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-18T23:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-08-18T23:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-08-18T23:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-18T23:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-18T23:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-18T23:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-18T23:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-18T23:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-18T23:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-18T23:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-08-18T23:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-18T23:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-08-18T23:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-08-18T23:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-08-18T23:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-08-18T23:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-08-18T23:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-18T23:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-08-18T23:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-08-18T23:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-08-18T23:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-08-18T23:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-08-18T23:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-08-18T23:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-08-18T23:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-08-18T23:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-08-18T23:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-08-18T23:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-08-18T23:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-08-18T23:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-18T23:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-08-18T23:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-08-18T23:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-18T23:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-18T23:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-08-18T23:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-08-18T23:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-18T23:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-08-18T23:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-08-18T23:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-18T23:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-08-18T23:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-08-18T23:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-18T23:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-08-18T23:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-08-18T23:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-18T23:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-08-18T23:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-08-18T23:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-18T23:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-08-18T23:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-08-18T23:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-08-18T23:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-08-18T23:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-08-18T23:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-08-18T23:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-18T23:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-08-18T23:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-08-18T23:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-18T23:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-08-18T23:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-08-18T23:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-18T23:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-08-18T23:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-08-18T23:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-08-18T23:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-08-18T23:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-08-18T23:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-18T23:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-08-18T23:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-08-18T23:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-18T23:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-08-18T23:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-08-18T23:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-08-18T23:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-08-18T23:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-18T23:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-08-18T23:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-08-18T23:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-08-18T23:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-18T23:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-08-18T23:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-18T23:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-18T23:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-08-18T23:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-18T23:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-18T23:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-18T23:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-08-18T23:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-08-18T23:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.635 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.635 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-18T23:53:33.635 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-18T23:53:33.635 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.635 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-08-18T23:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-18T23:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-18T23:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-18T23:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-18T23:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-18T23:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-18T23:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-18T23:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-18T23:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-18T23:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-18T23:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-18T23:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-18T23:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-18T23:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-18T23:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-18T23:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-18T23:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-18T23:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-08-18T23:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-18T23:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-08-18T23:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-18T23:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-18T23:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-08-18T23:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-08-18T23:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-08-18T23:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-18T23:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-08-18T23:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-18T23:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-18T23:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-18T23:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-18T23:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-18T23:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-18T23:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-18T23:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-18T23:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-08-18T23:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-18T23:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-18T23:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-18T23:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-18T23:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-18T23:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-18T23:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-18T23:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-18T23:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-18T23:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-18T23:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-18T23:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-18T23:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-18T23:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-18T23:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-18T23:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-18T23:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-18T23:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-08-18T23:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-18T23:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-18T23:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-18T23:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-18T23:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-18T23:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-18T23:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-18T23:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-18T23:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-08-18T23:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-18T23:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-18T23:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-08-18T23:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-08-18T23:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-18T23:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-18T23:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-18T23:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-08-18T23:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-08-18T23:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-18T23:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-08-18T23:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-18T23:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-18T23:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-18T23:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-08-18T23:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-18T23:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-18T23:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-18T23:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-18T23:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-08-18T23:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-08-18T23:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-08-18T23:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-08-18T23:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-08-18T23:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-08-18T23:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-08-18T23:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-08-18T23:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-08-18T23:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-08-18T23:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-08-18T23:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-08-18T23:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-08-18T23:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-08-18T23:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-08-18T23:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-08-18T23:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-08-18T23:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-08-18T23:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-08-18T23:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-08-18T23:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-08-18T23:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-08-18T23:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-08-18T23:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-08-18T23:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-08-18T23:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-08-18T23:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-08-18T23:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-08-18T23:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-08-18T23:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.697 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.697 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.697 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.697 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.697 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.697 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-08-18T23:53:33.697 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-08-18T23:53:33.697 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.697 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.697 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.697 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-08-18T23:53:33.697 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.698 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.698 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.698 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.698 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-08-18T23:53:33.698 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.698 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.698 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.698 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.698 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.698 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-08-18T23:53:33.698 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-08-18T23:53:33.699 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.699 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.699 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.699 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-08-18T23:53:33.699 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-08-18T23:53:33.699 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-08-18T23:53:33.699 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.699 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.699 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.699 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.699 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.699 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-08-18T23:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-08-18T23:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-08-18T23:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-08-18T23:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-08-18T23:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-08-18T23:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-08-18T23:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-08-18T23:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-08-18T23:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-18T23:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-08-18T23:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-08-18T23:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-08-18T23:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-18T23:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-08-18T23:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-08-18T23:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-08-18T23:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-08-18T23:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-08-18T23:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-08-18T23:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-08-18T23:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-18T23:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-08-18T23:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-08-18T23:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-18T23:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-08-18T23:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-08-18T23:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-08-18T23:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-08-18T23:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-08-18T23:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-08-18T23:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-08-18T23:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-18T23:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-08-18T23:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-08-18T23:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-08-18T23:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-08-18T23:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-08-18T23:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-08-18T23:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-08-18T23:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-08-18T23:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-08-18T23:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-08-18T23:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-08-18T23:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-08-18T23:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-08-18T23:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-08-18T23:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-08-18T23:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-08-18T23:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-18T23:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-08-18T23:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-08-18T23:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-08-18T23:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-18T23:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-08-18T23:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-08-18T23:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-08-18T23:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-08-18T23:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-08-18T23:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-08-18T23:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-08-18T23:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-18T23:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-08-18T23:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-08-18T23:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-08-18T23:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-08-18T23:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-08-18T23:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-08-18T23:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-08-18T23:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-08-18T23:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-08-18T23:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-08-18T23:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-18T23:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-08-18T23:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-08-18T23:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-08-18T23:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-08-18T23:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-08-18T23:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-18T23:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-08-18T23:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-08-18T23:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-08-18T23:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-08-18T23:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-08-18T23:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-08-18T23:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-18T23:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-08-18T23:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-08-18T23:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "k8sevents", 2024-08-18T23:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_event_retention_days": { 2024-08-18T23:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ceph_event_retention_days", 2024-08-18T23:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7", 2024-08-18T23:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-18T23:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "config_check_secs": { 2024-08-18T23:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_check_secs", 2024-08-18T23:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "10", 2024-08-18T23:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-18T23:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-08-18T23:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-08-18T23:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-08-18T23:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-08-18T23:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-08-18T23:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-18T23:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-08-18T23:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-08-18T23:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-08-18T23:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-18T23:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-08-18T23:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-08-18T23:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-08-18T23:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-08-18T23:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-08-18T23:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-08-18T23:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-08-18T23:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-08-18T23:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-08-18T23:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-08-18T23:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-08-18T23:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-08-18T23:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-08-18T23:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-08-18T23:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-18T23:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-18T23:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mds_autoscaler", 2024-08-18T23:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-08-18T23:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-08-18T23:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-08-18T23:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-08-18T23:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-08-18T23:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-18T23:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-08-18T23:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-08-18T23:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-08-18T23:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-08-18T23:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-08-18T23:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-08-18T23:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-08-18T23:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-08-18T23:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-08-18T23:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-08-18T23:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-08-18T23:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-08-18T23:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-08-18T23:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-08-18T23:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-08-18T23:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-08-18T23:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-08-18T23:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-18T23:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-08-18T23:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-08-18T23:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-08-18T23:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-18T23:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-08-18T23:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-08-18T23:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-08-18T23:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-08-18T23:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-08-18T23:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-08-18T23:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-08-18T23:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-08-18T23:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-08-18T23:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-08-18T23:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-08-18T23:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-08-18T23:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-08-18T23:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-08-18T23:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-08-18T23:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-18T23:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-18T23:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-08-18T23:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-08-18T23:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-18T23:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-18T23:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-08-18T23:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-08-18T23:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-08-18T23:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-08-18T23:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-08-18T23:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-08-18T23:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-08-18T23:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-18T23:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-08-18T23:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-08-18T23:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-08-18T23:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-18T23:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-08-18T23:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-08-18T23:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-08-18T23:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-08-18T23:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-08-18T23:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-08-18T23:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-08-18T23:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-08-18T23:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-08-18T23:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-08-18T23:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-08-18T23:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-08-18T23:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-08-18T23:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-08-18T23:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-08-18T23:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-08-18T23:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-08-18T23:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-08-18T23:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-08-18T23:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-08-18T23:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-08-18T23:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-08-18T23:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-08-18T23:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-08-18T23:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-08-18T23:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-08-18T23:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-08-18T23:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-08-18T23:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-08-18T23:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-08-18T23:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-08-18T23:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rook", 2024-08-18T23:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "storage_class": { 2024-08-18T23:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "storage_class", 2024-08-18T23:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "local", 2024-08-18T23:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-18T23:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-08-18T23:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-08-18T23:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-08-18T23:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-08-18T23:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-08-18T23:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-08-18T23:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-08-18T23:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-08-18T23:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-08-18T23:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-08-18T23:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-08-18T23:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-08-18T23:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-08-18T23:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-08-18T23:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-08-18T23:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-08-18T23:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-08-18T23:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-08-18T23:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-08-18T23:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-08-18T23:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-08-18T23:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-08-18T23:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-08-18T23:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-08-18T23:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-08-18T23:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-08-18T23:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-08-18T23:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-08-18T23:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-08-18T23:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-08-18T23:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-08-18T23:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-08-18T23:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-08-18T23:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-08-18T23:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-08-18T23:53:33.902 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.902 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.902 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.902 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.902 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.902 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.902 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.902 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-08-18T23:53:33.902 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.902 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.902 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.902 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.903 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.903 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.903 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.903 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.903 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.903 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.903 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.903 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.903 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.903 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.903 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.903 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.903 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-08-18T23:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-08-18T23:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-08-18T23:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-08-18T23:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-08-18T23:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-18T23:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-08-18T23:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-08-18T23:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-08-18T23:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-08-18T23:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-08-18T23:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-08-18T23:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-18T23:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-08-18T23:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-08-18T23:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-18T23:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-08-18T23:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-08-18T23:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-18T23:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-08-18T23:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-08-18T23:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-18T23:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-08-18T23:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-08-18T23:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-18T23:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-08-18T23:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-08-18T23:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-08-18T23:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-08-18T23:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-08-18T23:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-08-18T23:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-18T23:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-08-18T23:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-08-18T23:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-08-18T23:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-08-18T23:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-08-18T23:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-08-18T23:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-08-18T23:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-08-18T23:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-08-18T23:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-08-18T23:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-08-18T23:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-08-18T23:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-08-18T23:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-08-18T23:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-08-18T23:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-08-18T23:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-08-18T23:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-08-18T23:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-08-18T23:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-18T23:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-08-18T23:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-08-18T23:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-08-18T23:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-08-18T23:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-08-18T23:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-18T23:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-08-18T23:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-08-18T23:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-08-18T23:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-08-18T23:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-08-18T23:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-08-18T23:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-18T23:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-08-18T23:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-08-18T23:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-08-18T23:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-18T23:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-08-18T23:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-08-18T23:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-08-18T23:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-08-18T23:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-08-18T23:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-08-18T23:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-08-18T23:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-08-18T23:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-08-18T23:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-08-18T23:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-08-18T23:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-08-18T23:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-08-18T23:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-08-18T23:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-08-18T23:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-08-18T23:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-08-18T23:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-08-18T23:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-08-18T23:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-08-18T23:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-08-18T23:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-08-18T23:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-08-18T23:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-08-18T23:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-08-18T23:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-08-18T23:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-08-18T23:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-08-18T23:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-08-18T23:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-08-18T23:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-08-18T23:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-08-18T23:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-08-18T23:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-08-18T23:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-08-18T23:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-08-18T23:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-08-18T23:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-08-18T23:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-08-18T23:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-08-18T23:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-08-18T23:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-08-18T23:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-08-18T23:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-08-18T23:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-08-18T23:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-18T23:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-08-18T23:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-08-18T23:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-08-18T23:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-08-18T23:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-08-18T23:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-08-18T23:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-08-18T23:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "services": { 2024-08-18T23:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "dashboard": "https://172.21.15.78:8443/" 2024-08-18T23:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "always_on_modules": { 2024-08-18T23:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "octopus": [ 2024-08-18T23:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-08-18T23:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-08-18T23:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-08-18T23:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-08-18T23:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-08-18T23:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-08-18T23:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-08-18T23:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-08-18T23:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-08-18T23:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-08-18T23:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "pacific": [ 2024-08-18T23:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-08-18T23:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-08-18T23:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-08-18T23:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-08-18T23:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-08-18T23:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-08-18T23:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-08-18T23:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-08-18T23:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-08-18T23:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-08-18T23:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "quincy": [ 2024-08-18T23:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-08-18T23:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-08-18T23:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-08-18T23:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-08-18T23:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-08-18T23:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-08-18T23:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-08-18T23:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-08-18T23:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-08-18T23:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-08-18T23:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "reef": [ 2024-08-18T23:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-08-18T23:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-08-18T23:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-08-18T23:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-08-18T23:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-08-18T23:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-08-18T23:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-08-18T23:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-08-18T23:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-08-18T23:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-08-18T23:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-08-18T23:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "squid": [ 2024-08-18T23:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-08-18T23:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-08-18T23:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-08-18T23:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-08-18T23:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-08-18T23:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-08-18T23:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-08-18T23:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-08-18T23:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-08-18T23:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-08-18T23:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-08-18T23:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: "last_failure_osd_epoch": 3, 2024-08-18T23:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: "active_clients": [ 2024-08-18T23:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-08-18T23:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-08-18T23:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-08-18T23:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-08-18T23:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 2397561295 2024-08-18T23:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-08-18T23:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "libcephsqlite", 2024-08-18T23:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-08-18T23:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-08-18T23:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-08-18T23:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 1917247417 2024-08-18T23:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-08-18T23:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-08-18T23:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-08-18T23:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-08-18T23:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-08-18T23:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 2404266161 2024-08-18T23:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-08-18T23:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-08-18T23:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-08-18T23:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-08-18T23:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-08-18T23:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-08-18T23:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 2712900765 2024-08-18T23:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-08-18T23:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-08-18T23:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:53:33.991 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:32 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3399938812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-18T23:53:33.991 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:32 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1131584900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-18T23:53:33.991 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:32 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2038685969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-08-18T23:53:33.991 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:33 smithi123 ceph-mon[21808]: pgmap v627: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-18T23:53:33.991 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:33 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1751941981' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-08-18T23:53:33.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:32 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3399938812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-18T23:53:33.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:32 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1131584900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-18T23:53:33.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:32 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2038685969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-08-18T23:53:33.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:33 smithi161 ceph-mon[22027]: pgmap v627: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-18T23:53:33.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:33 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1751941981' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-08-18T23:53:33.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.544+0000 7f7e819f8640 1 -- 172.21.15.78:0/1751941981 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7e640a2b90 msgr2=0x7f7e640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:33.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.544+0000 7f7e819f8640 1 --2- 172.21.15.78:0/1751941981 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7e640a2b90 0x7f7e640a5050 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c005d00 tx=0x7f7e6c005c50 comp rx=0 tx=0).stop 2024-08-18T23:53:33.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.544+0000 7f7e819f8640 1 -- 172.21.15.78:0/1751941981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e7c102310 msgr2=0x7f7e7c109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:33.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.544+0000 7f7e819f8640 1 --2- 172.21.15.78:0/1751941981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e7c102310 0x7f7e7c109c60 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f7e74031ee0 tx=0x7f7e74031f10 comp rx=0 tx=0).stop 2024-08-18T23:53:33.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.544+0000 7f7e819f8640 1 -- 172.21.15.78:0/1751941981 shutdown_connections 2024-08-18T23:53:33.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.544+0000 7f7e819f8640 1 --2- 172.21.15.78:0/1751941981 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7e640a2b90 0x7f7e640a5050 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:33.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.544+0000 7f7e819f8640 1 --2- 172.21.15.78:0/1751941981 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e7c102310 0x7f7e7c109c60 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:33.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.544+0000 7f7e819f8640 1 --2- 172.21.15.78:0/1751941981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e7c101950 0x7f7e7c109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:33.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.544+0000 7f7e819f8640 1 --2- 172.21.15.78:0/1751941981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e7c100760 0x7f7e7c10b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:33.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.544+0000 7f7e819f8640 1 -- 172.21.15.78:0/1751941981 >> 172.21.15.78:0/1751941981 conn(0x7f7e7c0fbec0 msgr2=0x7f7e7c104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:33.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.545+0000 7f7e819f8640 1 -- 172.21.15.78:0/1751941981 shutdown_connections 2024-08-18T23:53:33.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:32.545+0000 7f7e819f8640 1 -- 172.21.15.78:0/1751941981 wait complete. 2024-08-18T23:53:33.999 DEBUG:tasks.check_counter:Getting stats from a 2024-08-18T23:53:34.000 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph daemon mds.a perf dump 2024-08-18T23:53:34.000 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:33 smithi078 ceph-mon[18440]: pgmap v627: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-18T23:53:34.000 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:33 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1751941981' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-08-18T23:53:34.271 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:53:34.811 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:53:34.812 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-08-18T23:53:34.812 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 4, 2024-08-18T23:53:34.812 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 4, 2024-08-18T23:53:34.812 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 3024, 2024-08-18T23:53:34.812 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 448, 2024-08-18T23:53:34.812 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-08-18T23:53:34.812 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 0, 2024-08-18T23:53:34.812 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.003047064, 2024-08-18T23:53:34.812 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000033961, 2024-08-18T23:53:34.812 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.002158404, 2024-08-18T23:53:34.812 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-18T23:53:34.812 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:34.813 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3, 2024-08-18T23:53:34.813 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.003919000, 2024-08-18T23:53:34.813 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001306333 2024-08-18T23:53:34.813 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:34.813 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:34.813 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-08-18T23:53:34.813 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-08-18T23:53:34.813 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-08-18T23:53:34.813 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:34.813 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-08-18T23:53:34.813 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 448 2024-08-18T23:53:34.813 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:34.814 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-08-18T23:53:34.814 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1, 2024-08-18T23:53:34.814 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 150, 2024-08-18T23:53:34.814 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 160, 2024-08-18T23:53:34.814 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 104624, 2024-08-18T23:53:34.814 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 3, 2024-08-18T23:53:34.814 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 2, 2024-08-18T23:53:34.814 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.011035717, 2024-08-18T23:53:34.814 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.008538722, 2024-08-18T23:53:34.814 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.001060918, 2024-08-18T23:53:34.814 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-18T23:53:34.814 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:34.815 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 148, 2024-08-18T23:53:34.815 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.006456787, 2024-08-18T23:53:34.815 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000043626 2024-08-18T23:53:34.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:34.815 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:34.815 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-08-18T23:53:34.815 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-08-18T23:53:34.815 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-08-18T23:53:34.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:34.815 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-18T23:53:34.815 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 104624 2024-08-18T23:53:34.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:34.815 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-08-18T23:53:34.816 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 203, 2024-08-18T23:53:34.816 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 198, 2024-08-18T23:53:34.816 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 749296, 2024-08-18T23:53:34.816 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 121504, 2024-08-18T23:53:34.816 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 2, 2024-08-18T23:53:34.816 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-08-18T23:53:34.816 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.036007800, 2024-08-18T23:53:34.816 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.013392829, 2024-08-18T23:53:34.816 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.027778216, 2024-08-18T23:53:34.816 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-18T23:53:34.816 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:34.816 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 197, 2024-08-18T23:53:34.816 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.009983167, 2024-08-18T23:53:34.817 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000050675 2024-08-18T23:53:34.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:34.817 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:34.817 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-08-18T23:53:34.817 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-08-18T23:53:34.817 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-08-18T23:53:34.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:34.817 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 749296, 2024-08-18T23:53:34.817 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 121504 2024-08-18T23:53:34.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:34.817 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-08-18T23:53:34.817 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 0, 2024-08-18T23:53:34.817 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-08-18T23:53:34.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:34.818 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-08-18T23:53:34.818 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-08-18T23:53:34.818 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-08-18T23:53:34.818 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-08-18T23:53:34.818 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-08-18T23:53:34.818 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-08-18T23:53:34.818 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-08-18T23:53:34.818 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-18T23:53:34.818 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-08-18T23:53:34.818 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-18T23:53:34.818 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-08-18T23:53:34.819 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-08-18T23:53:34.819 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-08-18T23:53:34.819 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-18T23:53:34.819 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-08-18T23:53:34.819 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-08-18T23:53:34.819 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-08-18T23:53:34.819 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-08-18T23:53:34.819 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-08-18T23:53:34.819 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-18T23:53:34.819 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-08-18T23:53:34.819 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-18T23:53:34.819 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-08-18T23:53:34.819 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-08-18T23:53:34.820 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-08-18T23:53:34.820 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-08-18T23:53:34.820 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-08-18T23:53:34.820 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-18T23:53:34.820 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-08-18T23:53:34.820 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-08-18T23:53:34.820 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-08-18T23:53:34.820 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-08-18T23:53:34.820 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-08-18T23:53:34.820 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-08-18T23:53:34.820 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-08-18T23:53:34.820 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-08-18T23:53:34.820 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-08-18T23:53:34.820 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 148967, 2024-08-18T23:53:34.821 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 29, 2024-08-18T23:53:34.821 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-08-18T23:53:34.821 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-08-18T23:53:34.821 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-08-18T23:53:34.821 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-08-18T23:53:34.821 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-08-18T23:53:34.821 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-08-18T23:53:34.821 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-08-18T23:53:34.821 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-08-18T23:53:34.821 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 0, 2024-08-18T23:53:34.821 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 0, 2024-08-18T23:53:34.821 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-08-18T23:53:34.821 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-08-18T23:53:34.822 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-08-18T23:53:34.822 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-08-18T23:53:34.822 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 928, 2024-08-18T23:53:34.822 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2, 2024-08-18T23:53:34.822 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-08-18T23:53:34.822 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-08-18T23:53:34.822 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-08-18T23:53:34.822 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-08-18T23:53:34.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:34.822 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-18T23:53:34.822 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-08-18T23:53:34.822 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-08-18T23:53:34.822 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-08-18T23:53:34.823 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 204, 2024-08-18T23:53:34.823 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 720648, 2024-08-18T23:53:34.823 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:34.823 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 204, 2024-08-18T23:53:34.823 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-08-18T23:53:34.823 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-08-18T23:53:34.823 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 204, 2024-08-18T23:53:34.823 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 720648, 2024-08-18T23:53:34.823 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-08-18T23:53:34.823 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-08-18T23:53:34.823 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-08-18T23:53:34.823 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-08-18T23:53:34.823 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:34.824 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:34.824 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:53:35.501 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-08-18T23:53:35.501 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-08-18T23:53:35.501 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-08-18T23:53:35.501 DEBUG:tasks.check_counter:Getting stats from d 2024-08-18T23:53:35.502 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph daemon mds.d perf dump 2024-08-18T23:53:35.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:35 smithi161 ceph-mon[22027]: pgmap v628: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:53:35.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:35 smithi078 ceph-mon[18440]: pgmap v628: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:53:35.752 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:53:35.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:35 smithi123 ceph-mon[21808]: pgmap v628: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:53:36.277 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:53:36.278 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-08-18T23:53:36.278 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 4, 2024-08-18T23:53:36.278 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 4, 2024-08-18T23:53:36.278 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 3024, 2024-08-18T23:53:36.278 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 448, 2024-08-18T23:53:36.278 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-08-18T23:53:36.278 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 0, 2024-08-18T23:53:36.278 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.003111528, 2024-08-18T23:53:36.278 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000039043, 2024-08-18T23:53:36.278 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.002256360, 2024-08-18T23:53:36.279 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-18T23:53:36.279 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:36.279 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3, 2024-08-18T23:53:36.279 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.003954825, 2024-08-18T23:53:36.279 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.001318275 2024-08-18T23:53:36.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:36.279 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:36.279 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-08-18T23:53:36.279 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-08-18T23:53:36.279 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-08-18T23:53:36.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:36.279 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-08-18T23:53:36.279 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 448 2024-08-18T23:53:36.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:36.280 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-08-18T23:53:36.280 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 208, 2024-08-18T23:53:36.280 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 203, 2024-08-18T23:53:36.280 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 750656, 2024-08-18T23:53:36.280 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 124704, 2024-08-18T23:53:36.280 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 2, 2024-08-18T23:53:36.280 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-08-18T23:53:36.280 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.030720941, 2024-08-18T23:53:36.280 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.013129965, 2024-08-18T23:53:36.280 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.023047740, 2024-08-18T23:53:36.280 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-18T23:53:36.280 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:36.281 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 202, 2024-08-18T23:53:36.281 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.021878389, 2024-08-18T23:53:36.281 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000108308 2024-08-18T23:53:36.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:36.281 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:36.281 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-08-18T23:53:36.281 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-08-18T23:53:36.281 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-08-18T23:53:36.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:36.281 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 750656, 2024-08-18T23:53:36.281 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 124704 2024-08-18T23:53:36.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:36.281 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-08-18T23:53:36.282 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1, 2024-08-18T23:53:36.282 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 155, 2024-08-18T23:53:36.282 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 160, 2024-08-18T23:53:36.282 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 105504, 2024-08-18T23:53:36.282 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 3, 2024-08-18T23:53:36.282 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 2, 2024-08-18T23:53:36.282 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.011339346, 2024-08-18T23:53:36.282 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.008506629, 2024-08-18T23:53:36.282 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.001298846, 2024-08-18T23:53:36.282 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-18T23:53:36.282 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:36.282 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 153, 2024-08-18T23:53:36.282 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.008175442, 2024-08-18T23:53:36.283 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000053434 2024-08-18T23:53:36.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:36.283 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:36.283 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-08-18T23:53:36.283 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-08-18T23:53:36.283 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-08-18T23:53:36.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:36.283 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-18T23:53:36.283 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 105504 2024-08-18T23:53:36.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:36.283 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-08-18T23:53:36.283 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 0, 2024-08-18T23:53:36.283 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-08-18T23:53:36.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:36.284 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-08-18T23:53:36.284 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-08-18T23:53:36.284 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-08-18T23:53:36.284 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-08-18T23:53:36.284 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-08-18T23:53:36.284 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-08-18T23:53:36.284 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-08-18T23:53:36.284 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-18T23:53:36.284 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-08-18T23:53:36.284 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-18T23:53:36.284 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-08-18T23:53:36.284 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-08-18T23:53:36.285 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-08-18T23:53:36.285 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-18T23:53:36.285 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-08-18T23:53:36.285 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-08-18T23:53:36.285 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-08-18T23:53:36.285 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-08-18T23:53:36.285 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-08-18T23:53:36.285 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-18T23:53:36.285 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-08-18T23:53:36.285 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-18T23:53:36.285 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-08-18T23:53:36.285 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-08-18T23:53:36.285 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-08-18T23:53:36.286 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-08-18T23:53:36.286 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-08-18T23:53:36.286 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-18T23:53:36.286 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-08-18T23:53:36.286 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-08-18T23:53:36.286 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-08-18T23:53:36.286 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-08-18T23:53:36.286 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-08-18T23:53:36.286 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-08-18T23:53:36.286 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-08-18T23:53:36.286 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-08-18T23:53:36.286 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-08-18T23:53:36.287 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 148984, 2024-08-18T23:53:36.287 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 30, 2024-08-18T23:53:36.287 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-08-18T23:53:36.287 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-08-18T23:53:36.287 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-08-18T23:53:36.287 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-08-18T23:53:36.287 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-08-18T23:53:36.287 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-08-18T23:53:36.287 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-08-18T23:53:36.287 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-08-18T23:53:36.287 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 0, 2024-08-18T23:53:36.287 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 0, 2024-08-18T23:53:36.287 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-08-18T23:53:36.287 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-08-18T23:53:36.288 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-08-18T23:53:36.288 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-08-18T23:53:36.288 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 928, 2024-08-18T23:53:36.288 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2, 2024-08-18T23:53:36.288 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-08-18T23:53:36.288 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-08-18T23:53:36.288 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-08-18T23:53:36.288 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-08-18T23:53:36.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:36.288 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-18T23:53:36.288 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-08-18T23:53:36.288 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-08-18T23:53:36.288 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-08-18T23:53:36.288 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 209, 2024-08-18T23:53:36.289 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 721298, 2024-08-18T23:53:36.289 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:36.289 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 209, 2024-08-18T23:53:36.289 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-08-18T23:53:36.289 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-08-18T23:53:36.289 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 209, 2024-08-18T23:53:36.289 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 721298, 2024-08-18T23:53:36.289 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-08-18T23:53:36.289 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-08-18T23:53:36.289 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-08-18T23:53:36.289 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-08-18T23:53:36.289 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:36.289 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:36.290 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:53:37.120 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-08-18T23:53:37.120 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-08-18T23:53:37.120 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-08-18T23:53:37.120 DEBUG:tasks.check_counter:Getting stats from g 2024-08-18T23:53:37.121 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph daemon mds.g perf dump 2024-08-18T23:53:37.373 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:53:37.672 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:37 smithi078 ceph-mon[18440]: pgmap v629: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-18T23:53:37.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:37 smithi161 ceph-mon[22027]: pgmap v629: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-18T23:53:37.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:37 smithi123 ceph-mon[21808]: pgmap v629: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-18T23:53:37.912 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:53:37.912 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-08-18T23:53:37.912 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 209, 2024-08-18T23:53:37.912 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 202, 2024-08-18T23:53:37.912 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 752864, 2024-08-18T23:53:37.912 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 123232, 2024-08-18T23:53:37.912 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-08-18T23:53:37.912 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 0, 2024-08-18T23:53:37.912 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.033627114, 2024-08-18T23:53:37.912 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.013205737, 2024-08-18T23:53:37.913 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.025079132, 2024-08-18T23:53:37.913 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-18T23:53:37.913 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:37.913 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 202, 2024-08-18T23:53:37.913 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.014138510, 2024-08-18T23:53:37.913 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000069992 2024-08-18T23:53:37.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:37.913 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:37.913 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-08-18T23:53:37.913 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-08-18T23:53:37.913 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-08-18T23:53:37.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:37.913 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 752864, 2024-08-18T23:53:37.913 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 123232 2024-08-18T23:53:37.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:37.914 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-08-18T23:53:37.914 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1, 2024-08-18T23:53:37.914 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 152, 2024-08-18T23:53:37.914 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 160, 2024-08-18T23:53:37.914 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 104976, 2024-08-18T23:53:37.914 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 3, 2024-08-18T23:53:37.914 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 2, 2024-08-18T23:53:37.914 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.012144054, 2024-08-18T23:53:37.914 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.009743971, 2024-08-18T23:53:37.914 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.000938185, 2024-08-18T23:53:37.914 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-18T23:53:37.915 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:37.915 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 150, 2024-08-18T23:53:37.915 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.012424206, 2024-08-18T23:53:37.915 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000082828 2024-08-18T23:53:37.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:37.915 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:37.915 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-08-18T23:53:37.915 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-08-18T23:53:37.915 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-08-18T23:53:37.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:37.915 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-18T23:53:37.915 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 104976 2024-08-18T23:53:37.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:37.916 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-08-18T23:53:37.916 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 0, 2024-08-18T23:53:37.916 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 2, 2024-08-18T23:53:37.916 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 0, 2024-08-18T23:53:37.916 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 0, 2024-08-18T23:53:37.916 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 2, 2024-08-18T23:53:37.916 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-08-18T23:53:37.916 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.000177282, 2024-08-18T23:53:37.916 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000000000, 2024-08-18T23:53:37.916 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.000121871, 2024-08-18T23:53:37.916 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-18T23:53:37.916 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:37.916 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-08-18T23:53:37.917 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-08-18T23:53:37.917 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-08-18T23:53:37.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:37.917 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:37.917 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-08-18T23:53:37.917 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-08-18T23:53:37.917 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-08-18T23:53:37.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:37.917 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-18T23:53:37.917 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2024-08-18T23:53:37.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:37.917 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-08-18T23:53:37.917 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 0, 2024-08-18T23:53:37.917 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-08-18T23:53:37.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:37.918 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-08-18T23:53:37.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-08-18T23:53:37.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-08-18T23:53:37.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-08-18T23:53:37.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-08-18T23:53:37.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-08-18T23:53:37.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-08-18T23:53:37.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-18T23:53:37.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-08-18T23:53:37.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-18T23:53:37.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-08-18T23:53:37.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-08-18T23:53:37.918 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-08-18T23:53:37.919 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-18T23:53:37.919 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-08-18T23:53:37.919 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-08-18T23:53:37.919 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-08-18T23:53:37.919 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-08-18T23:53:37.919 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-08-18T23:53:37.919 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-18T23:53:37.919 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-08-18T23:53:37.919 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-18T23:53:37.919 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-08-18T23:53:37.919 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-08-18T23:53:37.919 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-08-18T23:53:37.919 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-08-18T23:53:37.920 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-08-18T23:53:37.920 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-18T23:53:37.920 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-08-18T23:53:37.920 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-08-18T23:53:37.920 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-08-18T23:53:37.920 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-08-18T23:53:37.920 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-08-18T23:53:37.920 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-08-18T23:53:37.920 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-08-18T23:53:37.920 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-08-18T23:53:37.920 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-08-18T23:53:37.920 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 148923, 2024-08-18T23:53:37.920 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 29, 2024-08-18T23:53:37.920 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-08-18T23:53:37.921 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-08-18T23:53:37.921 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-08-18T23:53:37.921 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-08-18T23:53:37.921 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-08-18T23:53:37.921 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-08-18T23:53:37.921 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-08-18T23:53:37.921 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-08-18T23:53:37.921 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 0, 2024-08-18T23:53:37.921 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 0, 2024-08-18T23:53:37.921 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-08-18T23:53:37.921 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-08-18T23:53:37.921 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-08-18T23:53:37.921 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-08-18T23:53:37.921 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 928, 2024-08-18T23:53:37.922 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2, 2024-08-18T23:53:37.922 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-08-18T23:53:37.922 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-08-18T23:53:37.922 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-08-18T23:53:37.922 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-08-18T23:53:37.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:37.922 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-18T23:53:37.922 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-08-18T23:53:37.922 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-08-18T23:53:37.922 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-08-18T23:53:37.922 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 206, 2024-08-18T23:53:37.922 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 720908, 2024-08-18T23:53:37.922 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:37.923 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 206, 2024-08-18T23:53:37.923 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-08-18T23:53:37.923 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-08-18T23:53:37.923 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 206, 2024-08-18T23:53:37.923 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 720908, 2024-08-18T23:53:37.923 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-08-18T23:53:37.923 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-08-18T23:53:37.923 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-08-18T23:53:37.923 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-08-18T23:53:37.923 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:37.923 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:37.923 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:53:38.718 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-08-18T23:53:38.718 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-08-18T23:53:38.718 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-08-18T23:53:38.718 DEBUG:tasks.check_counter:Getting stats from j 2024-08-18T23:53:38.718 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph daemon mds.j perf dump 2024-08-18T23:53:38.968 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:53:39.446 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-08-18T23:53:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-08-18T23:53:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 210, 2024-08-18T23:53:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 205, 2024-08-18T23:53:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 751200, 2024-08-18T23:53:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 125984, 2024-08-18T23:53:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-08-18T23:53:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 0, 2024-08-18T23:53:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.032033041, 2024-08-18T23:53:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.013627802, 2024-08-18T23:53:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.023040733, 2024-08-18T23:53:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-18T23:53:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 204, 2024-08-18T23:53:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.014761560, 2024-08-18T23:53:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000072360 2024-08-18T23:53:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-08-18T23:53:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-08-18T23:53:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-08-18T23:53:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 751200, 2024-08-18T23:53:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 125984 2024-08-18T23:53:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-08-18T23:53:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1, 2024-08-18T23:53:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 156, 2024-08-18T23:53:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 160, 2024-08-18T23:53:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 105680, 2024-08-18T23:53:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 3, 2024-08-18T23:53:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 2, 2024-08-18T23:53:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.011707530, 2024-08-18T23:53:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.009162787, 2024-08-18T23:53:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.001112226, 2024-08-18T23:53:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-18T23:53:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 154, 2024-08-18T23:53:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.007133915, 2024-08-18T23:53:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000046324 2024-08-18T23:53:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-08-18T23:53:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-08-18T23:53:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-08-18T23:53:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-18T23:53:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 105680 2024-08-18T23:53:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-08-18T23:53:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 4, 2024-08-18T23:53:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 4, 2024-08-18T23:53:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 3024, 2024-08-18T23:53:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 448, 2024-08-18T23:53:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 2, 2024-08-18T23:53:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-08-18T23:53:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.003361744, 2024-08-18T23:53:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000045255, 2024-08-18T23:53:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.003295549, 2024-08-18T23:53:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-18T23:53:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3, 2024-08-18T23:53:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.006147627, 2024-08-18T23:53:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.002049209 2024-08-18T23:53:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-08-18T23:53:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-08-18T23:53:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-08-18T23:53:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-08-18T23:53:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 448 2024-08-18T23:53:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-08-18T23:53:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 0, 2024-08-18T23:53:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-08-18T23:53:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-08-18T23:53:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-08-18T23:53:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-08-18T23:53:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-08-18T23:53:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-08-18T23:53:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-08-18T23:53:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-08-18T23:53:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-18T23:53:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-08-18T23:53:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-18T23:53:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-08-18T23:53:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-08-18T23:53:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-08-18T23:53:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-18T23:53:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-08-18T23:53:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-08-18T23:53:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-08-18T23:53:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-08-18T23:53:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-08-18T23:53:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-18T23:53:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-08-18T23:53:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-18T23:53:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-08-18T23:53:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-08-18T23:53:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-08-18T23:53:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-08-18T23:53:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-08-18T23:53:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-18T23:53:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-08-18T23:53:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-08-18T23:53:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-08-18T23:53:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-08-18T23:53:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-08-18T23:53:39.455 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-08-18T23:53:39.455 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-08-18T23:53:39.455 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-08-18T23:53:39.455 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-08-18T23:53:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 148967, 2024-08-18T23:53:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 29, 2024-08-18T23:53:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-08-18T23:53:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-08-18T23:53:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-08-18T23:53:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-08-18T23:53:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-08-18T23:53:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-08-18T23:53:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-08-18T23:53:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-08-18T23:53:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 0, 2024-08-18T23:53:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 0, 2024-08-18T23:53:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-08-18T23:53:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-08-18T23:53:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-08-18T23:53:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-08-18T23:53:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 928, 2024-08-18T23:53:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2, 2024-08-18T23:53:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-08-18T23:53:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-08-18T23:53:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-08-18T23:53:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-08-18T23:53:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-08-18T23:53:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-18T23:53:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-08-18T23:53:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-08-18T23:53:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-08-18T23:53:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 211, 2024-08-18T23:53:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 721558, 2024-08-18T23:53:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 211, 2024-08-18T23:53:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-08-18T23:53:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-08-18T23:53:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 211, 2024-08-18T23:53:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 721558, 2024-08-18T23:53:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-08-18T23:53:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-08-18T23:53:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-08-18T23:53:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-08-18T23:53:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-08-18T23:53:39.461 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-08-18T23:53:39.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:39 smithi161 ceph-mon[22027]: pgmap v630: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:53:39.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:39 smithi078 ceph-mon[18440]: pgmap v630: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:53:39.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:39 smithi123 ceph-mon[21808]: pgmap v630: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:53:40.236 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-08-18T23:53:40.236 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-08-18T23:53:40.236 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-08-18T23:53:40.236 DEBUG:tasks.check_counter:Getting stats from b 2024-08-18T23:53:40.236 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph daemon mds.b perf dump 2024-08-18T23:53:40.512 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.b/config 2024-08-18T23:53:41.434 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-08-18T23:53:41.434 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-08-18T23:53:41.434 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 92711, 2024-08-18T23:53:41.434 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 87238, 2024-08-18T23:53:41.435 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 28355312, 2024-08-18T23:53:41.435 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 323151485, 2024-08-18T23:53:41.435 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 106, 2024-08-18T23:53:41.435 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 5, 2024-08-18T23:53:41.435 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 7.347853447, 2024-08-18T23:53:41.435 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 3.187415397, 2024-08-18T23:53:41.435 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 1.831103980, 2024-08-18T23:53:41.435 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 2.129473148, 2024-08-18T23:53:41.435 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:41.435 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 87240, 2024-08-18T23:53:41.435 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 28.756816192, 2024-08-18T23:53:41.435 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000329628 2024-08-18T23:53:41.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.436 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:41.436 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 28852, 2024-08-18T23:53:41.436 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.004381070, 2024-08-18T23:53:41.436 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000151 2024-08-18T23:53:41.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.436 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 43664, 2024-08-18T23:53:41.436 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 2304 2024-08-18T23:53:41.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.436 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-08-18T23:53:41.436 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 503988, 2024-08-18T23:53:41.436 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 727525, 2024-08-18T23:53:41.436 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 166745505, 2024-08-18T23:53:41.437 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 571066030, 2024-08-18T23:53:41.437 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 47, 2024-08-18T23:53:41.437 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2024-08-18T23:53:41.437 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 24.300407242, 2024-08-18T23:53:41.437 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 10.229416130, 2024-08-18T23:53:41.437 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 31.705870597, 2024-08-18T23:53:41.437 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 2.380928658, 2024-08-18T23:53:41.437 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:41.437 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 727523, 2024-08-18T23:53:41.437 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 71.542509820, 2024-08-18T23:53:41.437 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000098337 2024-08-18T23:53:41.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.437 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:41.438 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 927712, 2024-08-18T23:53:41.438 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.174967074, 2024-08-18T23:53:41.438 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000188 2024-08-18T23:53:41.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.438 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 49040, 2024-08-18T23:53:41.438 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 383568 2024-08-18T23:53:41.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.438 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-08-18T23:53:41.438 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 109750, 2024-08-18T23:53:41.439 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 105976, 2024-08-18T23:53:41.439 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 35186323, 2024-08-18T23:53:41.439 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 145420394, 2024-08-18T23:53:41.439 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 11, 2024-08-18T23:53:41.439 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 7, 2024-08-18T23:53:41.439 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 7.958595659, 2024-08-18T23:53:41.439 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 3.269569735, 2024-08-18T23:53:41.439 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 1.883024967, 2024-08-18T23:53:41.439 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 2.490404808, 2024-08-18T23:53:41.439 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:41.439 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 105972, 2024-08-18T23:53:41.440 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 42.527838773, 2024-08-18T23:53:41.440 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000401312 2024-08-18T23:53:41.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.440 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:41.440 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 22187, 2024-08-18T23:53:41.440 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.003487033, 2024-08-18T23:53:41.440 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000157 2024-08-18T23:53:41.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.440 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 836016, 2024-08-18T23:53:41.440 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 146944 2024-08-18T23:53:41.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.441 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-08-18T23:53:41.441 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-08-18T23:53:41.441 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-08-18T23:53:41.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.441 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-08-18T23:53:41.441 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-08-18T23:53:41.441 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-08-18T23:53:41.441 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 57469, 2024-08-18T23:53:41.441 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 264.896099836, 2024-08-18T23:53:41.441 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.004609373 2024-08-18T23:53:41.441 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:41.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.441 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-08-18T23:53:41.442 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-08-18T23:53:41.442 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-08-18T23:53:41.442 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 70481, 2024-08-18T23:53:41.442 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 29.786547787, 2024-08-18T23:53:41.442 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000422618 2024-08-18T23:53:41.442 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:41.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.442 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-08-18T23:53:41.442 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 257274, 2024-08-18T23:53:41.442 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 254244, 2024-08-18T23:53:41.442 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-08-18T23:53:41.442 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 254244, 2024-08-18T23:53:41.443 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 11321.435234984, 2024-08-18T23:53:41.443 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.044529803 2024-08-18T23:53:41.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.443 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-08-18T23:53:41.443 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 3030, 2024-08-18T23:53:41.443 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 25, 2024-08-18T23:53:41.443 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 2, 2024-08-18T23:53:41.443 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 1861, 2024-08-18T23:53:41.443 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 136, 2024-08-18T23:53:41.443 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 18, 2024-08-18T23:53:41.443 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 1468, 2024-08-18T23:53:41.444 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 1010, 2024-08-18T23:53:41.444 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 444, 2024-08-18T23:53:41.444 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 14, 2024-08-18T23:53:41.444 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 25, 2024-08-18T23:53:41.444 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 3821, 2024-08-18T23:53:41.444 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 3, 2024-08-18T23:53:41.444 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 3, 2024-08-18T23:53:41.444 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 4, 2024-08-18T23:53:41.444 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 428263, 2024-08-18T23:53:41.444 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 361302, 2024-08-18T23:53:41.444 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 3010, 2024-08-18T23:53:41.444 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 17, 2024-08-18T23:53:41.444 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-08-18T23:53:41.445 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-08-18T23:53:41.445 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 1168, 2024-08-18T23:53:41.445 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-08-18T23:53:41.445 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-08-18T23:53:41.445 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 15, 2024-08-18T23:53:41.445 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 2333, 2024-08-18T23:53:41.445 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 113, 2024-08-18T23:53:41.445 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 194, 2024-08-18T23:53:41.445 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-08-18T23:53:41.445 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 111275, 2024-08-18T23:53:41.445 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-08-18T23:53:41.445 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 4, 2024-08-18T23:53:41.445 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 234, 2024-08-18T23:53:41.446 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-08-18T23:53:41.446 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 111275, 2024-08-18T23:53:41.446 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-08-18T23:53:41.446 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-08-18T23:53:41.446 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-08-18T23:53:41.446 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 882, 2024-08-18T23:53:41.446 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 12, 2024-08-18T23:53:41.446 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 870, 2024-08-18T23:53:41.446 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 110393, 2024-08-18T23:53:41.446 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 34, 2024-08-18T23:53:41.446 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 114230, 2024-08-18T23:53:41.446 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 124956, 2024-08-18T23:53:41.446 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 6, 2024-08-18T23:53:41.447 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-18T23:53:41.447 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-18T23:53:41.447 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 162109, 2024-08-18T23:53:41.447 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 43848, 2024-08-18T23:53:41.447 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 10155, 2024-08-18T23:53:41.447 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 88712 2024-08-18T23:53:41.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.447 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-08-18T23:53:41.447 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 6, 2024-08-18T23:53:41.447 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2024-08-18T23:53:41.447 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2024-08-18T23:53:41.447 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 17, 2024-08-18T23:53:41.447 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 44, 2024-08-18T23:53:41.448 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 4, 2024-08-18T23:53:41.448 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-08-18T23:53:41.448 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-08-18T23:53:41.448 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 42088, 2024-08-18T23:53:41.448 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 42084, 2024-08-18T23:53:41.448 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-08-18T23:53:41.448 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-08-18T23:53:41.448 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-08-18T23:53:41.448 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-08-18T23:53:41.448 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-08-18T23:53:41.448 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-08-18T23:53:41.448 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-08-18T23:53:41.448 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 7, 2024-08-18T23:53:41.449 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 82917, 2024-08-18T23:53:41.449 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 12, 2024-08-18T23:53:41.449 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 15, 2024-08-18T23:53:41.449 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-08-18T23:53:41.449 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 161, 2024-08-18T23:53:41.449 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-08-18T23:53:41.449 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-08-18T23:53:41.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.449 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-08-18T23:53:41.450 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-08-18T23:53:41.450 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 247039, 2024-08-18T23:53:41.450 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 147481, 2024-08-18T23:53:41.450 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 147481, 2024-08-18T23:53:41.450 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 99558, 2024-08-18T23:53:41.450 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-08-18T23:53:41.450 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 12290, 2024-08-18T23:53:41.450 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 312, 2024-08-18T23:53:41.450 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 184, 2024-08-18T23:53:41.450 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 184, 2024-08-18T23:53:41.450 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 128, 2024-08-18T23:53:41.450 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 9, 2024-08-18T23:53:41.450 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-08-18T23:53:41.451 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 16, 2024-08-18T23:53:41.451 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 348693674, 2024-08-18T23:53:41.451 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 566943739, 2024-08-18T23:53:41.451 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2024-08-18T23:53:41.451 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-08-18T23:53:41.451 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 13002, 2024-08-18T23:53:41.451 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1588.280213488, 2024-08-18T23:53:41.451 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.122156607 2024-08-18T23:53:41.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.452 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2024-08-18T23:53:41.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.452 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-08-18T23:53:41.452 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 42, 2024-08-18T23:53:41.452 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 43399, 2024-08-18T23:53:41.452 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 43357, 2024-08-18T23:53:41.452 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 44, 2024-08-18T23:53:41.452 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 1422, 2024-08-18T23:53:41.452 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 1378, 2024-08-18T23:53:41.452 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 1468, 2024-08-18T23:53:41.452 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 96634, 2024-08-18T23:53:41.452 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 95166, 2024-08-18T23:53:41.452 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 3, 2024-08-18T23:53:41.453 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 75267, 2024-08-18T23:53:41.453 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 75264, 2024-08-18T23:53:41.453 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 195300, 2024-08-18T23:53:41.453 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 231712 2024-08-18T23:53:41.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.453 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-08-18T23:53:41.453 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 265080, 2024-08-18T23:53:41.453 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 8991, 2024-08-18T23:53:41.453 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 257274, 2024-08-18T23:53:41.453 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 37, 2024-08-18T23:53:41.453 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 18820, 2024-08-18T23:53:41.453 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-08-18T23:53:41.453 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 42991, 2024-08-18T23:53:41.453 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 890.344707773, 2024-08-18T23:53:41.454 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.020710025 2024-08-18T23:53:41.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.454 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-08-18T23:53:41.454 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 7279, 2024-08-18T23:53:41.454 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1152.719709145, 2024-08-18T23:53:41.454 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.158362372 2024-08-18T23:53:41.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.454 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-08-18T23:53:41.454 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.455 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.455 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.455 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-08-18T23:53:41.455 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.455 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.455 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.455 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-08-18T23:53:41.455 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 120236, 2024-08-18T23:53:41.455 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 7360.136818178, 2024-08-18T23:53:41.455 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.061214085 2024-08-18T23:53:41.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.455 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-08-18T23:53:41.456 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.456 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.456 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.456 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-08-18T23:53:41.456 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2, 2024-08-18T23:53:41.456 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000215056, 2024-08-18T23:53:41.456 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000107528 2024-08-18T23:53:41.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.456 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-08-18T23:53:41.456 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.456 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.456 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.457 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-08-18T23:53:41.457 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.457 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.457 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.458 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-08-18T23:53:41.458 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.458 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.458 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.458 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-08-18T23:53:41.458 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.458 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.458 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.459 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-08-18T23:53:41.459 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 175, 2024-08-18T23:53:41.459 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 4.750923694, 2024-08-18T23:53:41.459 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.027148135 2024-08-18T23:53:41.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.459 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-08-18T23:53:41.459 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.459 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.459 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.459 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-08-18T23:53:41.459 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.459 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.460 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.460 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-08-18T23:53:41.460 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 14785, 2024-08-18T23:53:41.460 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 486.964495849, 2024-08-18T23:53:41.460 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.032936387 2024-08-18T23:53:41.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.460 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-08-18T23:53:41.460 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1084, 2024-08-18T23:53:41.460 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.746113971, 2024-08-18T23:53:41.460 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001610806 2024-08-18T23:53:41.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.461 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-08-18T23:53:41.461 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 42609, 2024-08-18T23:53:41.461 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1320.443416153, 2024-08-18T23:53:41.461 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.030989777 2024-08-18T23:53:41.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.461 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-08-18T23:53:41.461 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.461 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.461 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.461 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-08-18T23:53:41.461 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.461 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.462 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.462 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-08-18T23:53:41.462 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 169, 2024-08-18T23:53:41.462 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 65.942745675, 2024-08-18T23:53:41.462 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.390193761 2024-08-18T23:53:41.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.462 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-08-18T23:53:41.462 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.462 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.462 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.462 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-08-18T23:53:41.463 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.463 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.463 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.463 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-08-18T23:53:41.463 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 8, 2024-08-18T23:53:41.463 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.144776111, 2024-08-18T23:53:41.463 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.143097013 2024-08-18T23:53:41.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.463 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-08-18T23:53:41.463 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.463 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.463 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.464 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-08-18T23:53:41.464 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.464 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.464 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.464 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-08-18T23:53:41.464 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.464 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.464 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.464 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-08-18T23:53:41.465 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 9, 2024-08-18T23:53:41.465 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.023155454, 2024-08-18T23:53:41.465 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002572828 2024-08-18T23:53:41.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.465 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-08-18T23:53:41.465 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3, 2024-08-18T23:53:41.465 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.002053946, 2024-08-18T23:53:41.465 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000684648 2024-08-18T23:53:41.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.465 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-08-18T23:53:41.465 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 24894, 2024-08-18T23:53:41.465 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 37.216103979, 2024-08-18T23:53:41.465 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001494982 2024-08-18T23:53:41.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.466 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-08-18T23:53:41.466 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-08-18T23:53:41.466 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-08-18T23:53:41.466 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.466 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.466 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.466 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:41.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.466 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-08-18T23:53:41.466 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2024-08-18T23:53:41.466 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 4, 2024-08-18T23:53:41.466 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 3, 2024-08-18T23:53:41.467 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 1, 2024-08-18T23:53:41.467 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-08-18T23:53:41.467 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 28996, 2024-08-18T23:53:41.467 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 28996, 2024-08-18T23:53:41.467 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 581, 2024-08-18T23:53:41.467 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-08-18T23:53:41.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.467 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-08-18T23:53:41.467 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-08-18T23:53:41.467 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-08-18T23:53:41.467 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-08-18T23:53:41.467 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-08-18T23:53:41.467 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-08-18T23:53:41.468 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-08-18T23:53:41.468 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-18T23:53:41.468 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-08-18T23:53:41.468 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-18T23:53:41.468 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-08-18T23:53:41.468 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-08-18T23:53:41.468 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-08-18T23:53:41.468 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-18T23:53:41.468 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-08-18T23:53:41.469 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-08-18T23:53:41.469 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-08-18T23:53:41.469 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-08-18T23:53:41.469 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-08-18T23:53:41.469 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-18T23:53:41.469 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-08-18T23:53:41.469 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-18T23:53:41.469 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-08-18T23:53:41.469 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-08-18T23:53:41.469 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-08-18T23:53:41.469 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-08-18T23:53:41.469 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-08-18T23:53:41.470 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-18T23:53:41.470 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-08-18T23:53:41.470 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-08-18T23:53:41.470 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-08-18T23:53:41.470 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-08-18T23:53:41.470 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-08-18T23:53:41.470 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-08-18T23:53:41.470 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-08-18T23:53:41.470 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-08-18T23:53:41.470 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-08-18T23:53:41.470 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 288588, 2024-08-18T23:53:41.470 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 109, 2024-08-18T23:53:41.470 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 88, 2024-08-18T23:53:41.471 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 1, 2024-08-18T23:53:41.471 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-08-18T23:53:41.471 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-08-18T23:53:41.471 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-08-18T23:53:41.471 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-08-18T23:53:41.471 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-08-18T23:53:41.471 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-08-18T23:53:41.471 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8856, 2024-08-18T23:53:41.471 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 166, 2024-08-18T23:53:41.471 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-08-18T23:53:41.471 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-08-18T23:53:41.471 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-08-18T23:53:41.471 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-08-18T23:53:41.472 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 1267195, 2024-08-18T23:53:41.472 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 17439, 2024-08-18T23:53:41.472 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-08-18T23:53:41.472 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-08-18T23:53:41.472 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-08-18T23:53:41.472 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-08-18T23:53:41.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.472 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-08-18T23:53:41.472 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-08-18T23:53:41.472 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-08-18T23:53:41.472 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 241639, 2024-08-18T23:53:41.472 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 618717793, 2024-08-18T23:53:41.472 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 37, 2024-08-18T23:53:41.472 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 241639, 2024-08-18T23:53:41.473 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-08-18T23:53:41.473 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 241639, 2024-08-18T23:53:41.473 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 4345.397583230, 2024-08-18T23:53:41.473 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.017983014 2024-08-18T23:53:41.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.473 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-08-18T23:53:41.473 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-08-18T23:53:41.473 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 241639, 2024-08-18T23:53:41.473 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 286372 2024-08-18T23:53:41.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.474 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 241639, 2024-08-18T23:53:41.474 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 112041, 2024-08-18T23:53:41.474 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 129598, 2024-08-18T23:53:41.474 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-08-18T23:53:41.474 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-08-18T23:53:41.474 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 598, 2024-08-18T23:53:41.474 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 27150, 2024-08-18T23:53:41.474 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 727, 2024-08-18T23:53:41.474 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 13717, 2024-08-18T23:53:41.474 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 348, 2024-08-18T23:53:41.474 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-08-18T23:53:41.474 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-08-18T23:53:41.475 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-08-18T23:53:41.475 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 136, 2024-08-18T23:53:41.475 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 85926, 2024-08-18T23:53:41.475 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-08-18T23:53:41.475 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-08-18T23:53:41.475 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-08-18T23:53:41.475 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 111298, 2024-08-18T23:53:41.475 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 40799, 2024-08-18T23:53:41.475 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-08-18T23:53:41.475 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-08-18T23:53:41.475 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-08-18T23:53:41.475 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-08-18T23:53:41.475 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-08-18T23:53:41.475 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-08-18T23:53:41.476 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-08-18T23:53:41.476 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-08-18T23:53:41.476 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-08-18T23:53:41.476 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 6, 2024-08-18T23:53:41.476 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-08-18T23:53:41.476 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-08-18T23:53:41.476 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-08-18T23:53:41.476 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-08-18T23:53:41.476 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-08-18T23:53:41.477 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-08-18T23:53:41.477 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-08-18T23:53:41.477 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-08-18T23:53:41.477 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-08-18T23:53:41.477 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-08-18T23:53:41.477 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-08-18T23:53:41.477 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-08-18T23:53:41.477 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-08-18T23:53:41.477 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-08-18T23:53:41.477 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-08-18T23:53:41.477 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-08-18T23:53:41.477 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 92, 2024-08-18T23:53:41.477 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-08-18T23:53:41.477 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 10, 2024-08-18T23:53:41.478 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 12, 2024-08-18T23:53:41.478 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 12, 2024-08-18T23:53:41.478 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-08-18T23:53:41.478 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-08-18T23:53:41.478 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 4526, 2024-08-18T23:53:41.478 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 54, 2024-08-18T23:53:41.478 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 1087 2024-08-18T23:53:41.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.478 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-08-18T23:53:41.478 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2024-08-18T23:53:41.478 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-08-18T23:53:41.478 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-08-18T23:53:41.479 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-08-18T23:53:41.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.479 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-08-18T23:53:41.479 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 82999 2024-08-18T23:53:41.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.479 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-08-18T23:53:41.479 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-08-18T23:53:41.479 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 23, 2024-08-18T23:53:41.479 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-08-18T23:53:41.479 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 9, 2024-08-18T23:53:41.479 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 85080, 2024-08-18T23:53:41.479 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 42084, 2024-08-18T23:53:41.479 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-08-18T23:53:41.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.480 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-18T23:53:41.480 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-08-18T23:53:41.480 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-08-18T23:53:41.480 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-08-18T23:53:41.480 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 706445, 2024-08-18T23:53:41.480 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 166679507, 2024-08-18T23:53:41.480 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:41.480 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 706445, 2024-08-18T23:53:41.480 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-08-18T23:53:41.480 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-08-18T23:53:41.480 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 706445, 2024-08-18T23:53:41.480 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 166679507, 2024-08-18T23:53:41.480 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-08-18T23:53:41.480 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.481 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.481 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.481 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:41.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.481 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-08-18T23:53:41.481 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-08-18T23:53:41.481 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-08-18T23:53:41.481 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-08-18T23:53:41.481 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-08-18T23:53:41.481 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-08-18T23:53:41.481 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:41.481 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-08-18T23:53:41.482 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 241639, 2024-08-18T23:53:41.482 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 623304277, 2024-08-18T23:53:41.482 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 241639, 2024-08-18T23:53:41.482 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 623304277, 2024-08-18T23:53:41.482 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-08-18T23:53:41.482 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.482 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.482 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.482 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:41.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.482 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-08-18T23:53:41.482 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-08-18T23:53:41.482 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-08-18T23:53:41.482 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-08-18T23:53:41.483 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-08-18T23:53:41.483 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-08-18T23:53:41.483 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:41.483 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-08-18T23:53:41.483 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 241639, 2024-08-18T23:53:41.483 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 241639, 2024-08-18T23:53:41.483 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 241639, 2024-08-18T23:53:41.483 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 241639, 2024-08-18T23:53:41.484 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-08-18T23:53:41.484 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.484 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.484 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.484 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:41.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.484 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-08-18T23:53:41.484 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-08-18T23:53:41.484 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-08-18T23:53:41.484 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-08-18T23:53:41.484 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 42084, 2024-08-18T23:53:41.485 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 4586892, 2024-08-18T23:53:41.485 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:41.485 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 42084, 2024-08-18T23:53:41.485 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-08-18T23:53:41.485 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-08-18T23:53:41.485 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 715, 2024-08-18T23:53:41.485 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 4586892, 2024-08-18T23:53:41.485 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-08-18T23:53:41.485 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.485 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.485 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.485 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:41.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:41.486 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x55f0e113a960": { 2024-08-18T23:53:41.486 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-08-18T23:53:41.486 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-08-18T23:53:41.486 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-08-18T23:53:41.486 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 247039, 2024-08-18T23:53:41.486 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 562749435, 2024-08-18T23:53:41.486 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:41.486 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 247039, 2024-08-18T23:53:41.486 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-08-18T23:53:41.486 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-08-18T23:53:41.486 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 13002, 2024-08-18T23:53:41.486 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 562749435, 2024-08-18T23:53:41.486 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-08-18T23:53:41.487 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:41.487 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:41.487 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:41.487 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:41.487 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:41.487 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-08-18T23:53:41.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:41 smithi161 ceph-mon[22027]: pgmap v631: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-18T23:53:41.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:41 smithi078 ceph-mon[18440]: pgmap v631: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-18T23:53:41.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:41 smithi123 ceph-mon[21808]: pgmap v631: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-18T23:53:42.564 INFO:tasks.check_counter:Daemon mds.b mds.exported=15 2024-08-18T23:53:42.564 INFO:tasks.check_counter:Daemon mds.b mds.imported=113 2024-08-18T23:53:42.564 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=136 2024-08-18T23:53:42.564 DEBUG:tasks.check_counter:Getting stats from e 2024-08-18T23:53:42.564 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph daemon mds.e perf dump 2024-08-18T23:53:42.824 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.b/config 2024-08-18T23:53:43.344 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-08-18T23:53:43.344 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-08-18T23:53:43.344 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1766, 2024-08-18T23:53:43.344 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 1752, 2024-08-18T23:53:43.344 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 152253708, 2024-08-18T23:53:43.344 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 616450, 2024-08-18T23:53:43.344 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 7, 2024-08-18T23:53:43.344 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2024-08-18T23:53:43.345 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.212002541, 2024-08-18T23:53:43.345 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.055959562, 2024-08-18T23:53:43.345 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.116752772, 2024-08-18T23:53:43.345 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.035920060, 2024-08-18T23:53:43.345 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:43.345 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1751, 2024-08-18T23:53:43.345 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.059127103, 2024-08-18T23:53:43.345 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000033767 2024-08-18T23:53:43.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.345 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:43.345 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.345 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.345 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.346 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 806496, 2024-08-18T23:53:43.346 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 127680 2024-08-18T23:53:43.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.346 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-08-18T23:53:43.346 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 679, 2024-08-18T23:53:43.346 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 834, 2024-08-18T23:53:43.346 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 193841157, 2024-08-18T23:53:43.346 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 555254, 2024-08-18T23:53:43.346 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 7, 2024-08-18T23:53:43.346 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2024-08-18T23:53:43.346 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.153741595, 2024-08-18T23:53:43.346 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.018820393, 2024-08-18T23:53:43.347 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.114972001, 2024-08-18T23:53:43.347 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.013965958, 2024-08-18T23:53:43.347 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:43.347 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 832, 2024-08-18T23:53:43.347 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.036435827, 2024-08-18T23:53:43.347 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000043793 2024-08-18T23:53:43.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.347 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:43.347 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.347 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.347 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.347 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-18T23:53:43.347 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 339584 2024-08-18T23:53:43.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.348 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-08-18T23:53:43.348 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 842, 2024-08-18T23:53:43.348 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 842, 2024-08-18T23:53:43.348 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 218340384, 2024-08-18T23:53:43.348 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 266939, 2024-08-18T23:53:43.348 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 6, 2024-08-18T23:53:43.348 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 5, 2024-08-18T23:53:43.348 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.182032491, 2024-08-18T23:53:43.348 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.015403444, 2024-08-18T23:53:43.348 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.141003832, 2024-08-18T23:53:43.348 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.019796452, 2024-08-18T23:53:43.348 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:43.349 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 841, 2024-08-18T23:53:43.349 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.034138739, 2024-08-18T23:53:43.349 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000040593 2024-08-18T23:53:43.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.349 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:43.349 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.349 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.349 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.349 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-08-18T23:53:43.349 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 448 2024-08-18T23:53:43.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.349 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-08-18T23:53:43.350 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-08-18T23:53:43.350 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-08-18T23:53:43.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.350 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-08-18T23:53:43.350 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-08-18T23:53:43.350 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-08-18T23:53:43.350 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 4714, 2024-08-18T23:53:43.350 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.262348831, 2024-08-18T23:53:43.350 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000055653 2024-08-18T23:53:43.350 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:43.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.350 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-08-18T23:53:43.350 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-08-18T23:53:43.351 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-08-18T23:53:43.351 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.351 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.351 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.351 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:43.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.351 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-08-18T23:53:43.351 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 0, 2024-08-18T23:53:43.351 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 0, 2024-08-18T23:53:43.351 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-08-18T23:53:43.351 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.351 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.351 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.352 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-08-18T23:53:43.352 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 0, 2024-08-18T23:53:43.352 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2024-08-18T23:53:43.352 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-08-18T23:53:43.352 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 0, 2024-08-18T23:53:43.352 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2024-08-18T23:53:43.352 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-08-18T23:53:43.352 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 42585, 2024-08-18T23:53:43.352 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 9579, 2024-08-18T23:53:43.352 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 33006, 2024-08-18T23:53:43.352 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 0, 2024-08-18T23:53:43.352 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 28900, 2024-08-18T23:53:43.353 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 0, 2024-08-18T23:53:43.353 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 0, 2024-08-18T23:53:43.353 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 0, 2024-08-18T23:53:43.353 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 2, 2024-08-18T23:53:43.353 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 0, 2024-08-18T23:53:43.353 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 0, 2024-08-18T23:53:43.353 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 0, 2024-08-18T23:53:43.353 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 0, 2024-08-18T23:53:43.353 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-08-18T23:53:43.353 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-08-18T23:53:43.353 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 0, 2024-08-18T23:53:43.353 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-08-18T23:53:43.353 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-08-18T23:53:43.353 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2024-08-18T23:53:43.354 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2024-08-18T23:53:43.354 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 0, 2024-08-18T23:53:43.354 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 0, 2024-08-18T23:53:43.354 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-08-18T23:53:43.354 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2024-08-18T23:53:43.354 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-08-18T23:53:43.354 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 4, 2024-08-18T23:53:43.354 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 234, 2024-08-18T23:53:43.354 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-08-18T23:53:43.354 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2024-08-18T23:53:43.354 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-08-18T23:53:43.354 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-08-18T23:53:43.355 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-08-18T23:53:43.355 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-08-18T23:53:43.355 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-08-18T23:53:43.355 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-08-18T23:53:43.355 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2024-08-18T23:53:43.355 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2024-08-18T23:53:43.355 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 0, 2024-08-18T23:53:43.355 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 0, 2024-08-18T23:53:43.355 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-08-18T23:53:43.355 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-18T23:53:43.355 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-18T23:53:43.355 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 0, 2024-08-18T23:53:43.355 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 0, 2024-08-18T23:53:43.355 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 0, 2024-08-18T23:53:43.356 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 0 2024-08-18T23:53:43.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.356 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-08-18T23:53:43.356 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2024-08-18T23:53:43.356 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2024-08-18T23:53:43.356 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2024-08-18T23:53:43.356 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 0, 2024-08-18T23:53:43.356 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 0, 2024-08-18T23:53:43.356 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-08-18T23:53:43.356 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-08-18T23:53:43.356 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-08-18T23:53:43.356 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 0, 2024-08-18T23:53:43.356 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 0, 2024-08-18T23:53:43.357 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-08-18T23:53:43.357 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-08-18T23:53:43.357 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-08-18T23:53:43.357 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-08-18T23:53:43.357 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-08-18T23:53:43.357 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-08-18T23:53:43.357 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-08-18T23:53:43.357 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 0, 2024-08-18T23:53:43.357 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 0, 2024-08-18T23:53:43.357 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-08-18T23:53:43.357 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2024-08-18T23:53:43.357 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-08-18T23:53:43.357 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2024-08-18T23:53:43.357 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-08-18T23:53:43.358 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-08-18T23:53:43.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.358 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-08-18T23:53:43.358 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-08-18T23:53:43.358 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 0, 2024-08-18T23:53:43.358 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-08-18T23:53:43.358 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-08-18T23:53:43.358 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 247039, 2024-08-18T23:53:43.358 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-08-18T23:53:43.358 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-08-18T23:53:43.359 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 0, 2024-08-18T23:53:43.359 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-08-18T23:53:43.359 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-08-18T23:53:43.359 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 144, 2024-08-18T23:53:43.359 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 22, 2024-08-18T23:53:43.359 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-08-18T23:53:43.359 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-08-18T23:53:43.359 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 348693674, 2024-08-18T23:53:43.359 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 566943739, 2024-08-18T23:53:43.359 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 566943140, 2024-08-18T23:53:43.359 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-08-18T23:53:43.359 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.360 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.360 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.360 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 247039 2024-08-18T23:53:43.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.360 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-08-18T23:53:43.360 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 27, 2024-08-18T23:53:43.360 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 43182, 2024-08-18T23:53:43.360 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 43155, 2024-08-18T23:53:43.360 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 29, 2024-08-18T23:53:43.360 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 1405, 2024-08-18T23:53:43.360 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 1376, 2024-08-18T23:53:43.360 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 42585, 2024-08-18T23:53:43.361 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 93746, 2024-08-18T23:53:43.361 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 51161, 2024-08-18T23:53:43.361 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 0, 2024-08-18T23:53:43.361 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 0, 2024-08-18T23:53:43.361 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 0, 2024-08-18T23:53:43.361 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 132272, 2024-08-18T23:53:43.361 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 215328 2024-08-18T23:53:43.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.361 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-08-18T23:53:43.361 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 0, 2024-08-18T23:53:43.361 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 0, 2024-08-18T23:53:43.361 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 0, 2024-08-18T23:53:43.361 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 0, 2024-08-18T23:53:43.361 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 0, 2024-08-18T23:53:43.362 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-08-18T23:53:43.362 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.362 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.362 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.362 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-08-18T23:53:43.362 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.362 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.362 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.362 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-08-18T23:53:43.362 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.362 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.363 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.363 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-08-18T23:53:43.363 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.363 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.363 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.363 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-08-18T23:53:43.363 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.363 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.363 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.363 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-08-18T23:53:43.364 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.364 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.364 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.364 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-08-18T23:53:43.364 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.364 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.364 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.364 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-08-18T23:53:43.364 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.364 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.364 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.365 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-08-18T23:53:43.365 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.365 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.365 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.365 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-08-18T23:53:43.365 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.365 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.365 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.365 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-08-18T23:53:43.365 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.365 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.365 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.366 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-08-18T23:53:43.366 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.366 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.366 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.366 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-08-18T23:53:43.366 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.366 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.366 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.367 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-08-18T23:53:43.367 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.367 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.367 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.367 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-08-18T23:53:43.367 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.367 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.367 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.367 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-08-18T23:53:43.367 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.368 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.368 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.368 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-08-18T23:53:43.368 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.368 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.368 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.368 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-08-18T23:53:43.368 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.368 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.368 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.369 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-08-18T23:53:43.369 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.369 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.369 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.369 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-08-18T23:53:43.369 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.369 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.369 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.369 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-08-18T23:53:43.369 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.369 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.369 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.370 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-08-18T23:53:43.370 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.370 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.370 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.370 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-08-18T23:53:43.370 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.370 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.370 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.370 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-08-18T23:53:43.370 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.371 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.371 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.371 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-08-18T23:53:43.371 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.371 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.371 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.371 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-08-18T23:53:43.371 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.371 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.371 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.371 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-08-18T23:53:43.372 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.372 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.372 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.372 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-08-18T23:53:43.372 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.372 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.372 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.372 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-08-18T23:53:43.372 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.372 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.372 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.373 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-08-18T23:53:43.373 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-08-18T23:53:43.373 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-08-18T23:53:43.373 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.373 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.373 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.373 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:43.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.373 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-08-18T23:53:43.373 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2024-08-18T23:53:43.373 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 4, 2024-08-18T23:53:43.373 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 3, 2024-08-18T23:53:43.374 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 2, 2024-08-18T23:53:43.374 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-08-18T23:53:43.374 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 0, 2024-08-18T23:53:43.374 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 0, 2024-08-18T23:53:43.374 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 2419, 2024-08-18T23:53:43.374 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-08-18T23:53:43.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.374 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-08-18T23:53:43.374 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-08-18T23:53:43.374 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-08-18T23:53:43.374 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-08-18T23:53:43.374 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-08-18T23:53:43.374 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-08-18T23:53:43.375 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-08-18T23:53:43.375 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-18T23:53:43.375 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-08-18T23:53:43.375 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-18T23:53:43.375 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-08-18T23:53:43.375 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-08-18T23:53:43.375 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-08-18T23:53:43.375 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-18T23:53:43.375 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-08-18T23:53:43.375 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-08-18T23:53:43.375 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-08-18T23:53:43.375 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-08-18T23:53:43.375 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-08-18T23:53:43.376 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-18T23:53:43.376 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-08-18T23:53:43.376 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-18T23:53:43.376 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-08-18T23:53:43.376 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-08-18T23:53:43.376 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-08-18T23:53:43.376 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-08-18T23:53:43.376 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-08-18T23:53:43.376 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-18T23:53:43.376 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-08-18T23:53:43.376 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-08-18T23:53:43.376 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-08-18T23:53:43.376 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-08-18T23:53:43.377 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-08-18T23:53:43.377 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-08-18T23:53:43.377 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-08-18T23:53:43.377 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-08-18T23:53:43.377 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-08-18T23:53:43.377 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 238852, 2024-08-18T23:53:43.377 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 82, 2024-08-18T23:53:43.377 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-08-18T23:53:43.377 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-08-18T23:53:43.377 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-08-18T23:53:43.377 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-08-18T23:53:43.377 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-08-18T23:53:43.377 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-08-18T23:53:43.377 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-08-18T23:53:43.377 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-08-18T23:53:43.378 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8856, 2024-08-18T23:53:43.378 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 166, 2024-08-18T23:53:43.378 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-08-18T23:53:43.378 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-08-18T23:53:43.378 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-08-18T23:53:43.378 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-08-18T23:53:43.378 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 30081823, 2024-08-18T23:53:43.378 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 174146, 2024-08-18T23:53:43.378 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-08-18T23:53:43.378 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-08-18T23:53:43.378 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-08-18T23:53:43.378 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-08-18T23:53:43.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.379 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-08-18T23:53:43.379 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-08-18T23:53:43.379 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-08-18T23:53:43.379 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 3053, 2024-08-18T23:53:43.379 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 26, 2024-08-18T23:53:43.379 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 4, 2024-08-18T23:53:43.379 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 3053, 2024-08-18T23:53:43.379 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-08-18T23:53:43.379 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3053, 2024-08-18T23:53:43.379 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 10.952739029, 2024-08-18T23:53:43.379 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.003587533 2024-08-18T23:53:43.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.379 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-08-18T23:53:43.380 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-08-18T23:53:43.380 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3053, 2024-08-18T23:53:43.380 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 3054 2024-08-18T23:53:43.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.380 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 3053, 2024-08-18T23:53:43.380 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 3053, 2024-08-18T23:53:43.380 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 0, 2024-08-18T23:53:43.380 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-08-18T23:53:43.380 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-08-18T23:53:43.380 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 1881, 2024-08-18T23:53:43.380 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 0, 2024-08-18T23:53:43.380 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 1169, 2024-08-18T23:53:43.380 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 0, 2024-08-18T23:53:43.380 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 0, 2024-08-18T23:53:43.381 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-08-18T23:53:43.381 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-08-18T23:53:43.381 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-08-18T23:53:43.381 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2024-08-18T23:53:43.381 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 0, 2024-08-18T23:53:43.381 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-08-18T23:53:43.381 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-08-18T23:53:43.381 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-08-18T23:53:43.381 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 2, 2024-08-18T23:53:43.381 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 0, 2024-08-18T23:53:43.381 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-08-18T23:53:43.381 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-08-18T23:53:43.381 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-08-18T23:53:43.382 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-08-18T23:53:43.382 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-08-18T23:53:43.382 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-08-18T23:53:43.382 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-08-18T23:53:43.382 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-08-18T23:53:43.382 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-08-18T23:53:43.382 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-08-18T23:53:43.382 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-08-18T23:53:43.382 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-08-18T23:53:43.382 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-08-18T23:53:43.382 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-08-18T23:53:43.382 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-08-18T23:53:43.382 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-08-18T23:53:43.383 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-08-18T23:53:43.383 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-08-18T23:53:43.383 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-08-18T23:53:43.383 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-08-18T23:53:43.383 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-08-18T23:53:43.383 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-08-18T23:53:43.383 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-08-18T23:53:43.383 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-08-18T23:53:43.383 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-08-18T23:53:43.383 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-08-18T23:53:43.383 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 92, 2024-08-18T23:53:43.383 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-08-18T23:53:43.383 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 4, 2024-08-18T23:53:43.384 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 11, 2024-08-18T23:53:43.384 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 11, 2024-08-18T23:53:43.384 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-08-18T23:53:43.384 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-08-18T23:53:43.384 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 0, 2024-08-18T23:53:43.384 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 2, 2024-08-18T23:53:43.384 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 0 2024-08-18T23:53:43.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.384 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-08-18T23:53:43.384 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 0, 2024-08-18T23:53:43.384 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-08-18T23:53:43.384 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-08-18T23:53:43.384 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-08-18T23:53:43.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.385 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-08-18T23:53:43.385 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 0 2024-08-18T23:53:43.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.385 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-08-18T23:53:43.385 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-08-18T23:53:43.385 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 0, 2024-08-18T23:53:43.385 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-08-18T23:53:43.385 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 0, 2024-08-18T23:53:43.385 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 0, 2024-08-18T23:53:43.385 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 0, 2024-08-18T23:53:43.385 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-08-18T23:53:43.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.385 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-18T23:53:43.386 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-08-18T23:53:43.386 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-08-18T23:53:43.386 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-08-18T23:53:43.386 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 3283, 2024-08-18T23:53:43.386 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 564125057, 2024-08-18T23:53:43.386 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:43.386 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 3283, 2024-08-18T23:53:43.386 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-08-18T23:53:43.386 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-08-18T23:53:43.386 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 3283, 2024-08-18T23:53:43.386 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 564125057, 2024-08-18T23:53:43.386 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-08-18T23:53:43.387 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.387 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.387 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.387 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:43.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.387 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-08-18T23:53:43.387 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-08-18T23:53:43.387 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-08-18T23:53:43.387 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-08-18T23:53:43.387 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-08-18T23:53:43.387 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-08-18T23:53:43.387 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:43.387 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-08-18T23:53:43.387 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 3053, 2024-08-18T23:53:43.387 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 562789515, 2024-08-18T23:53:43.388 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 3053, 2024-08-18T23:53:43.388 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 562789515, 2024-08-18T23:53:43.388 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-08-18T23:53:43.388 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.388 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.388 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.388 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:43.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.388 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-08-18T23:53:43.388 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-08-18T23:53:43.388 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-08-18T23:53:43.388 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-08-18T23:53:43.388 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-08-18T23:53:43.389 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-08-18T23:53:43.389 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:43.389 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-08-18T23:53:43.389 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 3053, 2024-08-18T23:53:43.389 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 3053, 2024-08-18T23:53:43.389 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 3053, 2024-08-18T23:53:43.389 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 3053, 2024-08-18T23:53:43.389 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-08-18T23:53:43.389 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.389 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.389 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.389 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:43.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.390 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-08-18T23:53:43.390 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-08-18T23:53:43.390 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-08-18T23:53:43.390 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-08-18T23:53:43.390 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-08-18T23:53:43.390 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-08-18T23:53:43.390 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:43.390 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-08-18T23:53:43.390 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-08-18T23:53:43.390 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-08-18T23:53:43.390 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-08-18T23:53:43.390 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-08-18T23:53:43.391 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-08-18T23:53:43.391 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.391 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.391 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.391 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:43.391 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:43.391 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x559b6b282b40": { 2024-08-18T23:53:43.391 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-08-18T23:53:43.391 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-08-18T23:53:43.391 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-08-18T23:53:43.391 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-08-18T23:53:43.392 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-08-18T23:53:43.392 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:43.392 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-08-18T23:53:43.392 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-08-18T23:53:43.392 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-08-18T23:53:43.392 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-08-18T23:53:43.392 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-08-18T23:53:43.392 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-08-18T23:53:43.392 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:43.392 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:43.392 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:43.392 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:43.392 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:43.392 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-08-18T23:53:43.522 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:43 smithi123 ceph-mon[21808]: pgmap v632: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-18T23:53:43.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:43 smithi161 ceph-mon[22027]: pgmap v632: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-18T23:53:43.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:43 smithi078 ceph-mon[18440]: pgmap v632: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-18T23:53:44.338 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-08-18T23:53:44.548 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-08-18T23:53:44.549 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-08-18T23:53:44.549 DEBUG:tasks.check_counter:Getting stats from h 2024-08-18T23:53:44.549 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph daemon mds.h perf dump 2024-08-18T23:53:44.807 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.b/config 2024-08-18T23:53:45.362 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-08-18T23:53:45.362 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-08-18T23:53:45.362 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 16015, 2024-08-18T23:53:45.362 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 21386, 2024-08-18T23:53:45.362 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 5908249, 2024-08-18T23:53:45.362 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 22510133, 2024-08-18T23:53:45.362 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 30, 2024-08-18T23:53:45.362 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 4, 2024-08-18T23:53:45.363 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 1.253358488, 2024-08-18T23:53:45.363 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.585336792, 2024-08-18T23:53:45.363 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.630953737, 2024-08-18T23:53:45.363 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.102638117, 2024-08-18T23:53:45.363 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:45.363 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 21385, 2024-08-18T23:53:45.363 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.992930514, 2024-08-18T23:53:45.363 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000046431 2024-08-18T23:53:45.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.363 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:45.363 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 28389, 2024-08-18T23:53:45.363 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.005118633, 2024-08-18T23:53:45.363 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000180 2024-08-18T23:53:45.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.364 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 822512, 2024-08-18T23:53:45.364 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 144656 2024-08-18T23:53:45.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.364 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-08-18T23:53:45.364 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 3457, 2024-08-18T23:53:45.364 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 3610, 2024-08-18T23:53:45.364 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 879449, 2024-08-18T23:53:45.364 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 2082681, 2024-08-18T23:53:45.364 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 17, 2024-08-18T23:53:45.364 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 5, 2024-08-18T23:53:45.364 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.235373540, 2024-08-18T23:53:45.364 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.077636616, 2024-08-18T23:53:45.365 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.059306359, 2024-08-18T23:53:45.365 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.076439463, 2024-08-18T23:53:45.365 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:45.365 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3608, 2024-08-18T23:53:45.365 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.202080032, 2024-08-18T23:53:45.365 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000056008 2024-08-18T23:53:45.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.365 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:45.365 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 34, 2024-08-18T23:53:45.365 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000005859, 2024-08-18T23:53:45.365 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000172 2024-08-18T23:53:45.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.365 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 4464, 2024-08-18T23:53:45.366 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 345296 2024-08-18T23:53:45.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.366 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-08-18T23:53:45.366 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 54055, 2024-08-18T23:53:45.366 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 67748, 2024-08-18T23:53:45.366 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 17969016, 2024-08-18T23:53:45.366 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 38871160, 2024-08-18T23:53:45.366 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 10, 2024-08-18T23:53:45.366 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2024-08-18T23:53:45.366 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 2.423246947, 2024-08-18T23:53:45.366 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 1.029377485, 2024-08-18T23:53:45.366 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 2.407035868, 2024-08-18T23:53:45.367 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.131522185, 2024-08-18T23:53:45.367 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:45.367 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 67747, 2024-08-18T23:53:45.367 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 3.936449349, 2024-08-18T23:53:45.367 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000058105 2024-08-18T23:53:45.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.367 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:45.367 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 95309, 2024-08-18T23:53:45.367 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.018797202, 2024-08-18T23:53:45.367 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000197 2024-08-18T23:53:45.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.367 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-08-18T23:53:45.367 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 448 2024-08-18T23:53:45.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.368 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-08-18T23:53:45.368 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-08-18T23:53:45.368 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-08-18T23:53:45.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.368 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-08-18T23:53:45.368 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-08-18T23:53:45.368 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-08-18T23:53:45.368 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 5755, 2024-08-18T23:53:45.368 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 5.983609119, 2024-08-18T23:53:45.368 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001039723 2024-08-18T23:53:45.368 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:45.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.369 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-08-18T23:53:45.369 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-08-18T23:53:45.369 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-08-18T23:53:45.369 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 4445, 2024-08-18T23:53:45.369 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.436176585, 2024-08-18T23:53:45.369 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000098127 2024-08-18T23:53:45.369 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:45.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.369 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-08-18T23:53:45.369 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 25117, 2024-08-18T23:53:45.369 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 24198, 2024-08-18T23:53:45.369 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-08-18T23:53:45.369 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 24198, 2024-08-18T23:53:45.370 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 360.010300906, 2024-08-18T23:53:45.370 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.014877688 2024-08-18T23:53:45.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.370 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-08-18T23:53:45.370 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 919, 2024-08-18T23:53:45.370 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2024-08-18T23:53:45.370 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-08-18T23:53:45.370 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 92, 2024-08-18T23:53:45.370 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 9, 2024-08-18T23:53:45.370 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-08-18T23:53:45.370 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 22, 2024-08-18T23:53:45.370 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 4, 2024-08-18T23:53:45.370 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 8, 2024-08-18T23:53:45.371 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 10, 2024-08-18T23:53:45.371 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 14, 2024-08-18T23:53:45.371 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 354, 2024-08-18T23:53:45.371 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 0, 2024-08-18T23:53:45.371 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 0, 2024-08-18T23:53:45.371 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 3, 2024-08-18T23:53:45.371 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 51731, 2024-08-18T23:53:45.371 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 43032, 2024-08-18T23:53:45.371 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 917, 2024-08-18T23:53:45.371 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 16, 2024-08-18T23:53:45.371 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-08-18T23:53:45.371 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-08-18T23:53:45.372 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 366, 2024-08-18T23:53:45.372 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-08-18T23:53:45.372 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-08-18T23:53:45.372 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 73, 2024-08-18T23:53:45.372 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 160, 2024-08-18T23:53:45.372 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 10, 2024-08-18T23:53:45.372 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 1590, 2024-08-18T23:53:45.372 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-08-18T23:53:45.372 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 6268, 2024-08-18T23:53:45.372 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 1, 2024-08-18T23:53:45.372 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 125, 2024-08-18T23:53:45.372 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 3050627, 2024-08-18T23:53:45.372 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-08-18T23:53:45.373 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 6268, 2024-08-18T23:53:45.373 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-08-18T23:53:45.373 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-08-18T23:53:45.373 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-08-18T23:53:45.373 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-08-18T23:53:45.373 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-08-18T23:53:45.373 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-08-18T23:53:45.373 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 6268, 2024-08-18T23:53:45.373 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 167, 2024-08-18T23:53:45.373 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 9039, 2024-08-18T23:53:45.373 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 9152, 2024-08-18T23:53:45.373 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-08-18T23:53:45.373 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-18T23:53:45.373 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-18T23:53:45.374 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 12180, 2024-08-18T23:53:45.374 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 2227, 2024-08-18T23:53:45.374 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 318, 2024-08-18T23:53:45.374 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 9244 2024-08-18T23:53:45.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.374 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-08-18T23:53:45.374 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 5, 2024-08-18T23:53:45.374 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 12, 2024-08-18T23:53:45.374 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 8, 2024-08-18T23:53:45.374 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 22, 2024-08-18T23:53:45.374 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 15, 2024-08-18T23:53:45.374 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-08-18T23:53:45.374 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-08-18T23:53:45.375 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-08-18T23:53:45.375 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 2433, 2024-08-18T23:53:45.375 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 2433, 2024-08-18T23:53:45.375 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-08-18T23:53:45.375 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-08-18T23:53:45.375 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-08-18T23:53:45.375 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-08-18T23:53:45.375 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-08-18T23:53:45.375 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-08-18T23:53:45.375 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-08-18T23:53:45.375 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 7, 2024-08-18T23:53:45.375 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 4872, 2024-08-18T23:53:45.375 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-08-18T23:53:45.376 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 73, 2024-08-18T23:53:45.376 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-08-18T23:53:45.376 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 9, 2024-08-18T23:53:45.376 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-08-18T23:53:45.376 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-08-18T23:53:45.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.376 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-08-18T23:53:45.376 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-08-18T23:53:45.376 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 13824, 2024-08-18T23:53:45.376 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-08-18T23:53:45.376 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-08-18T23:53:45.376 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 13824, 2024-08-18T23:53:45.376 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-08-18T23:53:45.377 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-08-18T23:53:45.377 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 17, 2024-08-18T23:53:45.377 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-08-18T23:53:45.377 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-08-18T23:53:45.377 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 17, 2024-08-18T23:53:45.377 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 3, 2024-08-18T23:53:45.377 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-08-18T23:53:45.377 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-08-18T23:53:45.377 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2024-08-18T23:53:45.377 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 24292603, 2024-08-18T23:53:45.377 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2024-08-18T23:53:45.377 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-08-18T23:53:45.377 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 770, 2024-08-18T23:53:45.377 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 4.797557936, 2024-08-18T23:53:45.378 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.006230594 2024-08-18T23:53:45.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.378 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2024-08-18T23:53:45.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.378 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-08-18T23:53:45.378 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 26, 2024-08-18T23:53:45.378 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 2477, 2024-08-18T23:53:45.378 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 2451, 2024-08-18T23:53:45.378 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 25, 2024-08-18T23:53:45.378 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 144, 2024-08-18T23:53:45.378 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 119, 2024-08-18T23:53:45.378 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 22, 2024-08-18T23:53:45.379 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 6029, 2024-08-18T23:53:45.379 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 6007, 2024-08-18T23:53:45.379 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 0, 2024-08-18T23:53:45.379 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 4062, 2024-08-18T23:53:45.379 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 4062, 2024-08-18T23:53:45.379 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 84164, 2024-08-18T23:53:45.379 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 223520 2024-08-18T23:53:45.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.379 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-08-18T23:53:45.379 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 31241, 2024-08-18T23:53:45.379 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 465, 2024-08-18T23:53:45.379 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 25117, 2024-08-18T23:53:45.379 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 16, 2024-08-18T23:53:45.380 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 6056, 2024-08-18T23:53:45.380 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-08-18T23:53:45.380 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1686, 2024-08-18T23:53:45.380 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 10.577189159, 2024-08-18T23:53:45.380 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.006273540 2024-08-18T23:53:45.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.380 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-08-18T23:53:45.380 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2024-08-18T23:53:45.380 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.001256647, 2024-08-18T23:53:45.380 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001256647 2024-08-18T23:53:45.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.380 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-08-18T23:53:45.380 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.380 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.381 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.381 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-08-18T23:53:45.381 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.381 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.381 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.381 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-08-18T23:53:45.381 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 17430, 2024-08-18T23:53:45.381 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 334.703499714, 2024-08-18T23:53:45.381 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.019202725 2024-08-18T23:53:45.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.381 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-08-18T23:53:45.382 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.382 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.382 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.382 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-08-18T23:53:45.382 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.382 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.382 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.383 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-08-18T23:53:45.383 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.383 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.383 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.383 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-08-18T23:53:45.383 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.383 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.383 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.383 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-08-18T23:53:45.383 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.383 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.384 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.384 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-08-18T23:53:45.384 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.384 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.384 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.384 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-08-18T23:53:45.384 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.384 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.384 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.384 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-08-18T23:53:45.385 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.385 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.385 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.385 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-08-18T23:53:45.385 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.385 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.385 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.385 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-08-18T23:53:45.385 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1668, 2024-08-18T23:53:45.385 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.669118296, 2024-08-18T23:53:45.385 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001000670 2024-08-18T23:53:45.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.386 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-08-18T23:53:45.386 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 73, 2024-08-18T23:53:45.386 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.059299120, 2024-08-18T23:53:45.386 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000812316 2024-08-18T23:53:45.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.386 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-08-18T23:53:45.386 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1690, 2024-08-18T23:53:45.386 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 9.824018848, 2024-08-18T23:53:45.386 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.005813028 2024-08-18T23:53:45.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.386 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-08-18T23:53:45.386 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.387 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.387 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.387 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-08-18T23:53:45.387 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.387 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.387 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.387 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-08-18T23:53:45.387 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.387 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.387 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.387 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-08-18T23:53:45.388 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.388 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.388 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.388 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-08-18T23:53:45.388 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.388 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.388 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.388 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-08-18T23:53:45.388 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.388 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.388 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.389 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-08-18T23:53:45.389 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.389 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.389 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.389 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-08-18T23:53:45.389 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.389 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.389 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.389 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-08-18T23:53:45.389 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.390 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.390 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.390 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-08-18T23:53:45.390 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.390 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.390 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.390 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-08-18T23:53:45.390 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.390 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.390 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.391 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-08-18T23:53:45.391 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1650, 2024-08-18T23:53:45.391 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 3.175919122, 2024-08-18T23:53:45.391 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001924799 2024-08-18T23:53:45.391 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.391 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-08-18T23:53:45.391 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-08-18T23:53:45.391 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-08-18T23:53:45.391 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.391 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.391 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.391 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:45.391 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.392 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-08-18T23:53:45.392 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2024-08-18T23:53:45.392 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 3, 2024-08-18T23:53:45.392 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 2, 2024-08-18T23:53:45.392 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 1, 2024-08-18T23:53:45.392 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-08-18T23:53:45.392 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 4761, 2024-08-18T23:53:45.392 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 4761, 2024-08-18T23:53:45.392 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 561, 2024-08-18T23:53:45.392 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-08-18T23:53:45.392 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.392 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-08-18T23:53:45.392 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-08-18T23:53:45.393 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-08-18T23:53:45.393 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-08-18T23:53:45.393 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-08-18T23:53:45.393 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-08-18T23:53:45.393 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-08-18T23:53:45.393 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-18T23:53:45.393 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-08-18T23:53:45.393 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-18T23:53:45.393 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-08-18T23:53:45.393 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-08-18T23:53:45.393 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-08-18T23:53:45.393 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-18T23:53:45.393 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-08-18T23:53:45.394 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-08-18T23:53:45.394 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-08-18T23:53:45.394 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-08-18T23:53:45.394 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-08-18T23:53:45.394 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-18T23:53:45.394 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-08-18T23:53:45.394 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-18T23:53:45.394 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-08-18T23:53:45.394 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-08-18T23:53:45.394 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-08-18T23:53:45.394 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-08-18T23:53:45.394 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-08-18T23:53:45.394 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-18T23:53:45.395 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-08-18T23:53:45.395 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-08-18T23:53:45.395 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-08-18T23:53:45.395 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-08-18T23:53:45.395 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-08-18T23:53:45.395 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-08-18T23:53:45.395 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-08-18T23:53:45.395 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-08-18T23:53:45.395 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-08-18T23:53:45.395 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 295079, 2024-08-18T23:53:45.395 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 99, 2024-08-18T23:53:45.395 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-08-18T23:53:45.395 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-08-18T23:53:45.396 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-08-18T23:53:45.396 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-08-18T23:53:45.396 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-08-18T23:53:45.396 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-08-18T23:53:45.396 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-08-18T23:53:45.396 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-08-18T23:53:45.396 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8856, 2024-08-18T23:53:45.396 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 166, 2024-08-18T23:53:45.396 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-08-18T23:53:45.396 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-08-18T23:53:45.396 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-08-18T23:53:45.396 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-08-18T23:53:45.396 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 107533, 2024-08-18T23:53:45.397 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 490, 2024-08-18T23:53:45.397 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-08-18T23:53:45.397 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-08-18T23:53:45.397 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-08-18T23:53:45.397 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-08-18T23:53:45.397 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.397 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-08-18T23:53:45.397 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-08-18T23:53:45.397 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-08-18T23:53:45.397 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 12548, 2024-08-18T23:53:45.397 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 22235712, 2024-08-18T23:53:45.397 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 33, 2024-08-18T23:53:45.397 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 12548, 2024-08-18T23:53:45.397 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-08-18T23:53:45.398 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 12548, 2024-08-18T23:53:45.398 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 53.908719865, 2024-08-18T23:53:45.398 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.004296200 2024-08-18T23:53:45.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.398 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-08-18T23:53:45.398 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-08-18T23:53:45.398 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 12548, 2024-08-18T23:53:45.398 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 12877 2024-08-18T23:53:45.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.398 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 12548, 2024-08-18T23:53:45.398 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 6391, 2024-08-18T23:53:45.398 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 6157, 2024-08-18T23:53:45.398 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-08-18T23:53:45.399 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-08-18T23:53:45.399 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 0, 2024-08-18T23:53:45.399 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 10, 2024-08-18T23:53:45.399 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 123, 2024-08-18T23:53:45.399 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 893, 2024-08-18T23:53:45.399 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 92, 2024-08-18T23:53:45.399 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-08-18T23:53:45.399 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-08-18T23:53:45.399 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-08-18T23:53:45.399 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 9, 2024-08-18T23:53:45.399 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 4882, 2024-08-18T23:53:45.399 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-08-18T23:53:45.399 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-08-18T23:53:45.400 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-08-18T23:53:45.400 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 6268, 2024-08-18T23:53:45.400 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 20, 2024-08-18T23:53:45.400 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-08-18T23:53:45.400 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-08-18T23:53:45.400 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-08-18T23:53:45.400 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-08-18T23:53:45.400 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-08-18T23:53:45.400 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-08-18T23:53:45.400 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-08-18T23:53:45.400 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-08-18T23:53:45.400 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-08-18T23:53:45.400 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-08-18T23:53:45.401 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-08-18T23:53:45.401 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-08-18T23:53:45.401 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-08-18T23:53:45.401 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-08-18T23:53:45.401 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-08-18T23:53:45.401 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-08-18T23:53:45.401 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-08-18T23:53:45.401 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-08-18T23:53:45.401 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-08-18T23:53:45.401 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-08-18T23:53:45.401 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-08-18T23:53:45.401 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-08-18T23:53:45.401 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-08-18T23:53:45.401 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-08-18T23:53:45.402 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-08-18T23:53:45.402 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-08-18T23:53:45.402 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 92, 2024-08-18T23:53:45.402 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-08-18T23:53:45.402 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 4, 2024-08-18T23:53:45.402 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 12, 2024-08-18T23:53:45.402 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 12, 2024-08-18T23:53:45.402 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-08-18T23:53:45.402 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-08-18T23:53:45.402 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 484, 2024-08-18T23:53:45.402 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 0, 2024-08-18T23:53:45.402 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 96 2024-08-18T23:53:45.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.403 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-08-18T23:53:45.403 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2024-08-18T23:53:45.403 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-08-18T23:53:45.403 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-08-18T23:53:45.403 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-08-18T23:53:45.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.403 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-08-18T23:53:45.403 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 4879 2024-08-18T23:53:45.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.403 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-08-18T23:53:45.403 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-08-18T23:53:45.403 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 16, 2024-08-18T23:53:45.403 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-08-18T23:53:45.404 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 8, 2024-08-18T23:53:45.404 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 4866, 2024-08-18T23:53:45.404 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 2433, 2024-08-18T23:53:45.404 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-08-18T23:53:45.404 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.404 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-18T23:53:45.404 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-08-18T23:53:45.404 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-08-18T23:53:45.404 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-08-18T23:53:45.404 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 73523, 2024-08-18T23:53:45.404 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 18122111, 2024-08-18T23:53:45.404 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:45.404 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 73523, 2024-08-18T23:53:45.404 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-08-18T23:53:45.405 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-08-18T23:53:45.405 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 73523, 2024-08-18T23:53:45.405 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 18122111, 2024-08-18T23:53:45.405 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-08-18T23:53:45.405 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.405 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.405 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.405 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:45.405 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.405 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-08-18T23:53:45.405 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-08-18T23:53:45.405 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-08-18T23:53:45.405 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-08-18T23:53:45.405 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-08-18T23:53:45.406 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-08-18T23:53:45.406 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:45.406 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-08-18T23:53:45.406 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 12548, 2024-08-18T23:53:45.406 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 22500909, 2024-08-18T23:53:45.406 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 12548, 2024-08-18T23:53:45.406 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 22500909, 2024-08-18T23:53:45.406 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-08-18T23:53:45.406 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.406 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.406 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.406 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:45.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.406 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-08-18T23:53:45.407 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-08-18T23:53:45.407 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-08-18T23:53:45.407 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-08-18T23:53:45.407 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-08-18T23:53:45.407 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-08-18T23:53:45.407 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:45.407 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-08-18T23:53:45.407 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 12548, 2024-08-18T23:53:45.407 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 12548, 2024-08-18T23:53:45.407 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 12548, 2024-08-18T23:53:45.407 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 12548, 2024-08-18T23:53:45.407 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-08-18T23:53:45.408 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.408 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.408 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.408 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:45.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.408 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-08-18T23:53:45.408 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-08-18T23:53:45.408 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-08-18T23:53:45.408 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-08-18T23:53:45.408 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 2433, 2024-08-18T23:53:45.408 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 265197, 2024-08-18T23:53:45.408 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:45.408 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 2433, 2024-08-18T23:53:45.408 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-08-18T23:53:45.409 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-08-18T23:53:45.409 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 123, 2024-08-18T23:53:45.409 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 265197, 2024-08-18T23:53:45.409 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-08-18T23:53:45.409 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.409 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.409 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.409 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:45.409 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:45.409 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x564915b8a960": { 2024-08-18T23:53:45.409 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-08-18T23:53:45.409 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-08-18T23:53:45.409 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-08-18T23:53:45.410 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 13824, 2024-08-18T23:53:45.410 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 20098299, 2024-08-18T23:53:45.410 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:45.410 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 13824, 2024-08-18T23:53:45.410 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-08-18T23:53:45.410 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-08-18T23:53:45.410 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 770, 2024-08-18T23:53:45.410 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 20098299, 2024-08-18T23:53:45.410 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-08-18T23:53:45.410 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:45.410 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:45.410 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:45.410 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:45.411 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:45.411 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-08-18T23:53:45.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:45 smithi123 ceph-mon[21808]: pgmap v633: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:53:45.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:45 smithi161 ceph-mon[22027]: pgmap v633: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:53:45.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:45 smithi078 ceph-mon[18440]: pgmap v633: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:53:46.115 INFO:tasks.check_counter:Daemon mds.h mds.exported=73 2024-08-18T23:53:46.116 INFO:tasks.check_counter:Daemon mds.h mds.imported=10 2024-08-18T23:53:46.116 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=9 2024-08-18T23:53:46.116 DEBUG:tasks.check_counter:Getting stats from k 2024-08-18T23:53:46.116 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph daemon mds.k perf dump 2024-08-18T23:53:46.372 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.b/config 2024-08-18T23:53:46.920 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-08-18T23:53:46.920 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-08-18T23:53:46.920 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 4, 2024-08-18T23:53:46.920 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 4, 2024-08-18T23:53:46.920 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 3024, 2024-08-18T23:53:46.921 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 448, 2024-08-18T23:53:46.921 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2024-08-18T23:53:46.921 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 0, 2024-08-18T23:53:46.921 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.002956156, 2024-08-18T23:53:46.921 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000041095, 2024-08-18T23:53:46.921 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.002199971, 2024-08-18T23:53:46.921 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-18T23:53:46.921 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:46.921 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3, 2024-08-18T23:53:46.921 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.003955971, 2024-08-18T23:53:46.921 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001318657 2024-08-18T23:53:46.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:46.922 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:46.922 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:46.922 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:46.922 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:46.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:46.922 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-08-18T23:53:46.922 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 448 2024-08-18T23:53:46.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:46.922 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-08-18T23:53:46.922 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 213, 2024-08-18T23:53:46.922 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 208, 2024-08-18T23:53:46.922 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 752016, 2024-08-18T23:53:46.922 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 127904, 2024-08-18T23:53:46.923 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 2, 2024-08-18T23:53:46.923 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2024-08-18T23:53:46.923 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.030654981, 2024-08-18T23:53:46.923 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.012880076, 2024-08-18T23:53:46.923 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.023989146, 2024-08-18T23:53:46.923 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-18T23:53:46.923 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:46.923 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 207, 2024-08-18T23:53:46.923 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.010483426, 2024-08-18T23:53:46.923 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000050644 2024-08-18T23:53:46.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:46.923 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:46.923 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:46.924 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:46.924 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:46.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:46.924 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 752016, 2024-08-18T23:53:46.924 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 127904 2024-08-18T23:53:46.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:46.924 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-08-18T23:53:46.924 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1, 2024-08-18T23:53:46.924 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 158, 2024-08-18T23:53:46.924 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 160, 2024-08-18T23:53:46.924 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 106032, 2024-08-18T23:53:46.924 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 3, 2024-08-18T23:53:46.924 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2024-08-18T23:53:46.925 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.008759463, 2024-08-18T23:53:46.925 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.006532196, 2024-08-18T23:53:46.925 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.000769560, 2024-08-18T23:53:46.925 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-18T23:53:46.925 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:46.925 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 156, 2024-08-18T23:53:46.925 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.006719196, 2024-08-18T23:53:46.925 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000043071 2024-08-18T23:53:46.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:46.925 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:46.925 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:46.925 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:46.925 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:46.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:46.926 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-18T23:53:46.926 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 106032 2024-08-18T23:53:46.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:46.926 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-08-18T23:53:46.926 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2024-08-18T23:53:46.926 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-08-18T23:53:46.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:46.926 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-08-18T23:53:46.926 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-08-18T23:53:46.926 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-08-18T23:53:46.926 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-08-18T23:53:46.926 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-08-18T23:53:46.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-08-18T23:53:46.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-08-18T23:53:46.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-18T23:53:46.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-08-18T23:53:46.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-18T23:53:46.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-08-18T23:53:46.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-08-18T23:53:46.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-08-18T23:53:46.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-18T23:53:46.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-08-18T23:53:46.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-08-18T23:53:46.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-08-18T23:53:46.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-08-18T23:53:46.928 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-08-18T23:53:46.928 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-18T23:53:46.928 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-08-18T23:53:46.928 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-18T23:53:46.928 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-08-18T23:53:46.928 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-08-18T23:53:46.928 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-08-18T23:53:46.928 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-08-18T23:53:46.928 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-08-18T23:53:46.928 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-18T23:53:46.928 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-08-18T23:53:46.928 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-08-18T23:53:46.928 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-08-18T23:53:46.929 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-08-18T23:53:46.929 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-08-18T23:53:46.929 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-08-18T23:53:46.929 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-08-18T23:53:46.929 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-08-18T23:53:46.929 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-08-18T23:53:46.929 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 148967, 2024-08-18T23:53:46.929 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 29, 2024-08-18T23:53:46.929 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-08-18T23:53:46.929 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-08-18T23:53:46.929 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-08-18T23:53:46.929 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-08-18T23:53:46.929 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-08-18T23:53:46.929 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-08-18T23:53:46.930 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-08-18T23:53:46.930 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-08-18T23:53:46.930 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 0, 2024-08-18T23:53:46.930 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 0, 2024-08-18T23:53:46.930 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-08-18T23:53:46.930 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-08-18T23:53:46.930 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-08-18T23:53:46.930 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-08-18T23:53:46.930 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 928, 2024-08-18T23:53:46.930 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2, 2024-08-18T23:53:46.930 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-08-18T23:53:46.930 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-08-18T23:53:46.931 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-08-18T23:53:46.931 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-08-18T23:53:46.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-08-18T23:53:46.931 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-18T23:53:46.931 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-08-18T23:53:46.931 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-08-18T23:53:46.931 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-08-18T23:53:46.931 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 214, 2024-08-18T23:53:46.931 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 721948, 2024-08-18T23:53:46.931 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:46.931 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 214, 2024-08-18T23:53:46.931 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-08-18T23:53:46.931 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-08-18T23:53:46.932 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 214, 2024-08-18T23:53:46.932 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 721948, 2024-08-18T23:53:46.932 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-08-18T23:53:46.932 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-08-18T23:53:46.932 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-08-18T23:53:46.932 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-08-18T23:53:46.932 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:46.932 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-08-18T23:53:46.932 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-08-18T23:53:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:47 smithi161 ceph-mon[22027]: pgmap v634: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-18T23:53:47.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:47 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:53:47.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:47 smithi078 ceph-mon[18440]: pgmap v634: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-18T23:53:47.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:47 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:53:47.785 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:47 smithi123 ceph-mon[21808]: pgmap v634: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-18T23:53:47.785 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:47 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-18T23:53:47.813 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-08-18T23:53:47.813 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-08-18T23:53:47.813 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-08-18T23:53:47.813 DEBUG:tasks.check_counter:Getting stats from c 2024-08-18T23:53:47.813 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph daemon mds.c perf dump 2024-08-18T23:53:48.485 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.c/config 2024-08-18T23:53:49.035 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-08-18T23:53:49.035 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-08-18T23:53:49.036 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 926, 2024-08-18T23:53:49.036 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 912, 2024-08-18T23:53:49.036 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1040864, 2024-08-18T23:53:49.036 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 349484, 2024-08-18T23:53:49.036 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 5, 2024-08-18T23:53:49.037 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2024-08-18T23:53:49.037 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.107806476, 2024-08-18T23:53:49.037 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.037035228, 2024-08-18T23:53:49.037 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.046660495, 2024-08-18T23:53:49.037 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.024629317, 2024-08-18T23:53:49.038 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:49.038 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 911, 2024-08-18T23:53:49.038 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.047859738, 2024-08-18T23:53:49.038 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000052535 2024-08-18T23:53:49.038 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.039 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:49.039 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.039 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.039 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.039 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 806336, 2024-08-18T23:53:49.040 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 127520 2024-08-18T23:53:49.040 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.040 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-08-18T23:53:49.040 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 172, 2024-08-18T23:53:49.040 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 327, 2024-08-18T23:53:49.041 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 4238114, 2024-08-18T23:53:49.041 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 395640, 2024-08-18T23:53:49.041 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 5, 2024-08-18T23:53:49.042 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-08-18T23:53:49.042 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.024341051, 2024-08-18T23:53:49.042 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.010005620, 2024-08-18T23:53:49.042 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.007413084, 2024-08-18T23:53:49.042 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.004227133, 2024-08-18T23:53:49.043 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:49.043 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 325, 2024-08-18T23:53:49.043 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.015357024, 2024-08-18T23:53:49.043 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000047252 2024-08-18T23:53:49.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.043 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:49.044 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.044 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.044 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.044 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-18T23:53:49.045 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 341200 2024-08-18T23:53:49.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.045 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-08-18T23:53:49.045 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1324, 2024-08-18T23:53:49.045 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1324, 2024-08-18T23:53:49.045 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 16249154, 2024-08-18T23:53:49.046 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 420215, 2024-08-18T23:53:49.046 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 5, 2024-08-18T23:53:49.046 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-08-18T23:53:49.046 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.100098550, 2024-08-18T23:53:49.046 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.029449909, 2024-08-18T23:53:49.046 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.033425663, 2024-08-18T23:53:49.047 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.030337785, 2024-08-18T23:53:49.047 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:49.047 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1323, 2024-08-18T23:53:49.047 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.056955864, 2024-08-18T23:53:49.047 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000043050 2024-08-18T23:53:49.048 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.048 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:49.048 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.048 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.048 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.049 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.049 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-08-18T23:53:49.049 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 448 2024-08-18T23:53:49.049 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.049 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-08-18T23:53:49.049 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-08-18T23:53:49.050 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-08-18T23:53:49.050 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.050 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-08-18T23:53:49.050 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-08-18T23:53:49.050 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-08-18T23:53:49.050 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3732, 2024-08-18T23:53:49.051 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.092636426, 2024-08-18T23:53:49.051 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000024822 2024-08-18T23:53:49.051 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:49.051 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.051 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-08-18T23:53:49.051 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-08-18T23:53:49.051 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-08-18T23:53:49.051 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.052 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.052 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.052 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:49.052 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.052 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-08-18T23:53:49.053 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2024-08-18T23:53:49.053 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2024-08-18T23:53:49.053 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-08-18T23:53:49.053 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.053 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.054 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.054 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.054 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-08-18T23:53:49.054 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-08-18T23:53:49.054 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-08-18T23:53:49.055 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-08-18T23:53:49.055 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2024-08-18T23:53:49.055 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-08-18T23:53:49.055 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-08-18T23:53:49.055 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 2442, 2024-08-18T23:53:49.055 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2024-08-18T23:53:49.056 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 2442, 2024-08-18T23:53:49.056 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-08-18T23:53:49.056 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 2442, 2024-08-18T23:53:49.056 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 1, 2024-08-18T23:53:49.056 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-08-18T23:53:49.056 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-08-18T23:53:49.056 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 1, 2024-08-18T23:53:49.057 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2024-08-18T23:53:49.057 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2024-08-18T23:53:49.057 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-08-18T23:53:49.057 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-08-18T23:53:49.057 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-08-18T23:53:49.057 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-08-18T23:53:49.058 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-08-18T23:53:49.058 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-08-18T23:53:49.058 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-08-18T23:53:49.058 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-08-18T23:53:49.058 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-08-18T23:53:49.058 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-08-18T23:53:49.058 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-08-18T23:53:49.058 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-08-18T23:53:49.058 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-08-18T23:53:49.058 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-08-18T23:53:49.058 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 0, 2024-08-18T23:53:49.059 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 0, 2024-08-18T23:53:49.059 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-08-18T23:53:49.059 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-08-18T23:53:49.059 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-08-18T23:53:49.059 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-08-18T23:53:49.059 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-08-18T23:53:49.059 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-08-18T23:53:49.059 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-08-18T23:53:49.059 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-08-18T23:53:49.059 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-08-18T23:53:49.059 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-08-18T23:53:49.059 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-08-18T23:53:49.059 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-08-18T23:53:49.060 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-08-18T23:53:49.060 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-18T23:53:49.060 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-18T23:53:49.060 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-08-18T23:53:49.060 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-08-18T23:53:49.060 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-08-18T23:53:49.060 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-08-18T23:53:49.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.060 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-08-18T23:53:49.060 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-08-18T23:53:49.060 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-08-18T23:53:49.060 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-08-18T23:53:49.060 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-08-18T23:53:49.061 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-08-18T23:53:49.061 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-08-18T23:53:49.061 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-08-18T23:53:49.061 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-08-18T23:53:49.061 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-08-18T23:53:49.061 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-08-18T23:53:49.061 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-08-18T23:53:49.061 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-08-18T23:53:49.061 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-08-18T23:53:49.061 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-08-18T23:53:49.061 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-08-18T23:53:49.062 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-08-18T23:53:49.062 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-08-18T23:53:49.062 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 0, 2024-08-18T23:53:49.062 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 0, 2024-08-18T23:53:49.062 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-08-18T23:53:49.062 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-08-18T23:53:49.062 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-08-18T23:53:49.062 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-08-18T23:53:49.062 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-08-18T23:53:49.062 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-08-18T23:53:49.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.063 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-08-18T23:53:49.063 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-08-18T23:53:49.063 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2024-08-18T23:53:49.063 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-08-18T23:53:49.063 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-08-18T23:53:49.063 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 13824, 2024-08-18T23:53:49.063 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-08-18T23:53:49.063 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-08-18T23:53:49.063 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2024-08-18T23:53:49.063 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-08-18T23:53:49.064 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-08-18T23:53:49.064 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 17, 2024-08-18T23:53:49.064 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 3, 2024-08-18T23:53:49.064 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-08-18T23:53:49.064 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-08-18T23:53:49.064 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-08-18T23:53:49.064 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 24292603, 2024-08-18T23:53:49.064 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 24291465, 2024-08-18T23:53:49.064 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-08-18T23:53:49.064 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.064 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.064 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.065 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 13824 2024-08-18T23:53:49.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.065 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-08-18T23:53:49.065 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 12, 2024-08-18T23:53:49.065 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 2484, 2024-08-18T23:53:49.065 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 2472, 2024-08-18T23:53:49.065 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 10, 2024-08-18T23:53:49.065 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 131, 2024-08-18T23:53:49.065 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 121, 2024-08-18T23:53:49.066 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 2442, 2024-08-18T23:53:49.066 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 5816, 2024-08-18T23:53:49.066 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 3374, 2024-08-18T23:53:49.066 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-08-18T23:53:49.066 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2024-08-18T23:53:49.066 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-08-18T23:53:49.067 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 57388, 2024-08-18T23:53:49.067 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 215328 2024-08-18T23:53:49.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.067 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-08-18T23:53:49.067 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2024-08-18T23:53:49.068 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-08-18T23:53:49.068 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2024-08-18T23:53:49.068 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2024-08-18T23:53:49.068 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-08-18T23:53:49.068 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-08-18T23:53:49.068 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.069 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.069 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.069 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.069 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-08-18T23:53:49.069 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.070 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.070 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.070 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-08-18T23:53:49.070 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.070 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.071 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.071 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-08-18T23:53:49.071 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.071 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.071 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.072 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-08-18T23:53:49.072 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.072 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.072 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.073 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-08-18T23:53:49.073 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.073 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.073 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.074 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.074 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-08-18T23:53:49.074 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.074 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.074 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.075 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.075 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-08-18T23:53:49.075 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.075 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.075 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.075 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.076 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-08-18T23:53:49.076 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.076 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.076 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.077 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-08-18T23:53:49.077 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.077 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.077 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.077 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-08-18T23:53:49.078 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.078 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.078 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.078 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-08-18T23:53:49.079 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.079 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.079 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.079 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.079 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-08-18T23:53:49.080 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.080 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.080 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.081 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-08-18T23:53:49.081 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.081 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.081 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.082 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-08-18T23:53:49.082 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.082 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.082 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.083 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-08-18T23:53:49.083 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.083 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.083 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.083 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.084 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-08-18T23:53:49.084 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.084 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.084 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.084 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-08-18T23:53:49.085 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.085 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.085 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.086 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-08-18T23:53:49.086 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.086 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.086 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.087 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-08-18T23:53:49.087 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.087 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.087 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.088 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-08-18T23:53:49.088 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.088 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.088 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.088 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-08-18T23:53:49.089 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.089 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.089 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.089 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-08-18T23:53:49.089 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.090 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.090 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.090 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-08-18T23:53:49.090 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.090 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.090 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.091 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-08-18T23:53:49.091 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.091 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.091 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.092 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-08-18T23:53:49.092 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.092 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.092 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.092 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-08-18T23:53:49.092 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.092 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.092 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.093 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-08-18T23:53:49.093 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.093 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.093 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.093 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-08-18T23:53:49.093 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.093 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.093 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.093 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-08-18T23:53:49.094 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-08-18T23:53:49.094 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-08-18T23:53:49.094 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.094 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.094 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.094 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:49.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.094 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-08-18T23:53:49.094 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-08-18T23:53:49.094 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 3, 2024-08-18T23:53:49.095 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 2, 2024-08-18T23:53:49.095 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 2, 2024-08-18T23:53:49.095 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-08-18T23:53:49.095 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2024-08-18T23:53:49.095 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-08-18T23:53:49.095 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 2398, 2024-08-18T23:53:49.095 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-08-18T23:53:49.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.095 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-08-18T23:53:49.095 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-08-18T23:53:49.095 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-08-18T23:53:49.095 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-08-18T23:53:49.096 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-08-18T23:53:49.096 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-08-18T23:53:49.096 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-08-18T23:53:49.096 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-18T23:53:49.096 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-08-18T23:53:49.096 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-18T23:53:49.096 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-08-18T23:53:49.096 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-08-18T23:53:49.096 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-08-18T23:53:49.096 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-18T23:53:49.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-08-18T23:53:49.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-08-18T23:53:49.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-08-18T23:53:49.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-08-18T23:53:49.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-08-18T23:53:49.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-18T23:53:49.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-08-18T23:53:49.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-18T23:53:49.097 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-08-18T23:53:49.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-08-18T23:53:49.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-08-18T23:53:49.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-08-18T23:53:49.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-08-18T23:53:49.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-18T23:53:49.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-08-18T23:53:49.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-08-18T23:53:49.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-08-18T23:53:49.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-08-18T23:53:49.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-08-18T23:53:49.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-08-18T23:53:49.098 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-08-18T23:53:49.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-08-18T23:53:49.099 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-08-18T23:53:49.099 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 199048, 2024-08-18T23:53:49.099 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 59, 2024-08-18T23:53:49.099 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-08-18T23:53:49.099 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-08-18T23:53:49.099 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-08-18T23:53:49.099 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-08-18T23:53:49.099 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-08-18T23:53:49.099 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-08-18T23:53:49.099 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-08-18T23:53:49.099 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-08-18T23:53:49.099 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8856, 2024-08-18T23:53:49.100 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 166, 2024-08-18T23:53:49.100 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-08-18T23:53:49.100 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-08-18T23:53:49.100 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-08-18T23:53:49.100 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-08-18T23:53:49.100 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 1759256, 2024-08-18T23:53:49.100 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 9885, 2024-08-18T23:53:49.100 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-08-18T23:53:49.100 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-08-18T23:53:49.100 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-08-18T23:53:49.100 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-08-18T23:53:49.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.100 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-08-18T23:53:49.101 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-08-18T23:53:49.101 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-08-18T23:53:49.101 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 2189, 2024-08-18T23:53:49.101 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 21, 2024-08-18T23:53:49.101 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 3, 2024-08-18T23:53:49.101 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 2189, 2024-08-18T23:53:49.101 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-08-18T23:53:49.101 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2189, 2024-08-18T23:53:49.101 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 3.033932656, 2024-08-18T23:53:49.101 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001385990 2024-08-18T23:53:49.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.101 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-08-18T23:53:49.101 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-08-18T23:53:49.101 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2189, 2024-08-18T23:53:49.102 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2190 2024-08-18T23:53:49.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.102 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 2189, 2024-08-18T23:53:49.102 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 2189, 2024-08-18T23:53:49.102 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2024-08-18T23:53:49.102 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-08-18T23:53:49.102 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-08-18T23:53:49.102 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 1322, 2024-08-18T23:53:49.102 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2024-08-18T23:53:49.102 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 865, 2024-08-18T23:53:49.102 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2024-08-18T23:53:49.102 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2024-08-18T23:53:49.102 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-08-18T23:53:49.103 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-08-18T23:53:49.103 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-08-18T23:53:49.103 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-08-18T23:53:49.103 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2024-08-18T23:53:49.103 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-08-18T23:53:49.103 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-08-18T23:53:49.103 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-08-18T23:53:49.103 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 1, 2024-08-18T23:53:49.103 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2024-08-18T23:53:49.103 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-08-18T23:53:49.103 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-08-18T23:53:49.103 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-08-18T23:53:49.103 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-08-18T23:53:49.104 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-08-18T23:53:49.104 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-08-18T23:53:49.104 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-08-18T23:53:49.104 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-08-18T23:53:49.104 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-08-18T23:53:49.104 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-08-18T23:53:49.104 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-08-18T23:53:49.104 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-08-18T23:53:49.104 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-08-18T23:53:49.104 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-08-18T23:53:49.104 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-08-18T23:53:49.104 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-08-18T23:53:49.104 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-08-18T23:53:49.105 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-08-18T23:53:49.105 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-08-18T23:53:49.105 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-08-18T23:53:49.105 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-08-18T23:53:49.105 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-08-18T23:53:49.105 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-08-18T23:53:49.105 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-08-18T23:53:49.105 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-08-18T23:53:49.105 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-08-18T23:53:49.105 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 92, 2024-08-18T23:53:49.105 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-08-18T23:53:49.105 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 4, 2024-08-18T23:53:49.105 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 6, 2024-08-18T23:53:49.105 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 6, 2024-08-18T23:53:49.106 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-08-18T23:53:49.106 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-08-18T23:53:49.106 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2024-08-18T23:53:49.106 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2024-08-18T23:53:49.106 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2024-08-18T23:53:49.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.106 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-08-18T23:53:49.106 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2024-08-18T23:53:49.106 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-08-18T23:53:49.106 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-08-18T23:53:49.106 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-08-18T23:53:49.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.107 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-08-18T23:53:49.107 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-08-18T23:53:49.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.107 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-08-18T23:53:49.107 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-08-18T23:53:49.107 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-08-18T23:53:49.107 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-08-18T23:53:49.107 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-08-18T23:53:49.107 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 0, 2024-08-18T23:53:49.107 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-08-18T23:53:49.107 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-08-18T23:53:49.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.107 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-18T23:53:49.108 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-18T23:53:49.108 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-08-18T23:53:49.108 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-18T23:53:49.108 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 2418, 2024-08-18T23:53:49.108 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 21295836, 2024-08-18T23:53:49.108 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:49.108 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 2418, 2024-08-18T23:53:49.108 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-18T23:53:49.108 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-18T23:53:49.108 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2418, 2024-08-18T23:53:49.108 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 21295836, 2024-08-18T23:53:49.108 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-18T23:53:49.108 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.109 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.109 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.109 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:49.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.109 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-08-18T23:53:49.109 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-18T23:53:49.109 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-08-18T23:53:49.109 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-18T23:53:49.109 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-18T23:53:49.109 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-18T23:53:49.109 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:49.109 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-18T23:53:49.110 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 2189, 2024-08-18T23:53:49.110 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 20104770, 2024-08-18T23:53:49.110 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2189, 2024-08-18T23:53:49.110 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 20104770, 2024-08-18T23:53:49.110 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-18T23:53:49.110 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.110 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.110 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.110 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:49.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.110 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-08-18T23:53:49.110 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-18T23:53:49.110 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-08-18T23:53:49.111 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-18T23:53:49.111 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-18T23:53:49.111 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-18T23:53:49.111 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:49.111 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-18T23:53:49.111 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 2189, 2024-08-18T23:53:49.111 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 2189, 2024-08-18T23:53:49.111 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2189, 2024-08-18T23:53:49.111 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 2189, 2024-08-18T23:53:49.111 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-18T23:53:49.111 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.111 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.111 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.112 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:49.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.112 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-08-18T23:53:49.112 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-18T23:53:49.112 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-08-18T23:53:49.112 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-18T23:53:49.112 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-18T23:53:49.112 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-18T23:53:49.112 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:49.112 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-18T23:53:49.112 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-18T23:53:49.112 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-18T23:53:49.113 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-08-18T23:53:49.113 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-08-18T23:53:49.113 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-18T23:53:49.113 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.113 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.113 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.113 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:49.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:49.113 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55b60d3b0b40": { 2024-08-18T23:53:49.113 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-18T23:53:49.113 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-08-18T23:53:49.113 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-18T23:53:49.113 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-18T23:53:49.113 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-18T23:53:49.114 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:49.114 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-18T23:53:49.114 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-18T23:53:49.114 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-18T23:53:49.114 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-08-18T23:53:49.114 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-08-18T23:53:49.114 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-18T23:53:49.114 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:49.114 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:49.114 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:49.114 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:49.114 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:49.114 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-08-18T23:53:49.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:49 smithi161 ceph-mon[22027]: pgmap v635: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:53:49.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:49 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:49.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:49 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:49.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:49 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:49.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:49 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:49.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:49 smithi078 ceph-mon[18440]: pgmap v635: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:53:49.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:49.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:49.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:49.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:49 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:49.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:49 smithi123 ceph-mon[21808]: pgmap v635: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-18T23:53:49.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:49 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:49.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:49 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:49.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:49 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:49.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:49 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:49.928 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-08-18T23:53:49.928 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-08-18T23:53:49.928 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-08-18T23:53:49.928 DEBUG:tasks.check_counter:Getting stats from f 2024-08-18T23:53:49.929 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph daemon mds.f perf dump 2024-08-18T23:53:50.228 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.c/config 2024-08-18T23:53:50.801 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-08-18T23:53:50.801 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-08-18T23:53:50.801 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 570, 2024-08-18T23:53:50.802 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 572, 2024-08-18T23:53:50.802 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 4341627, 2024-08-18T23:53:50.802 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 181324, 2024-08-18T23:53:50.802 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 7, 2024-08-18T23:53:50.802 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-08-18T23:53:50.803 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.050770568, 2024-08-18T23:53:50.803 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.014722500, 2024-08-18T23:53:50.803 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.016189195, 2024-08-18T23:53:50.803 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.014599784, 2024-08-18T23:53:50.803 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:50.803 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 570, 2024-08-18T23:53:50.803 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.023585369, 2024-08-18T23:53:50.803 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000041377 2024-08-18T23:53:50.804 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.804 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:50.804 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.804 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.804 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.804 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.804 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-18T23:53:50.804 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-08-18T23:53:50.804 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.804 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-08-18T23:53:50.805 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 911, 2024-08-18T23:53:50.805 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 895, 2024-08-18T23:53:50.805 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 6757475, 2024-08-18T23:53:50.805 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 344217, 2024-08-18T23:53:50.805 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-08-18T23:53:50.805 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-08-18T23:53:50.805 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.094410062, 2024-08-18T23:53:50.805 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.030802497, 2024-08-18T23:53:50.805 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.046177231, 2024-08-18T23:53:50.805 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.019595114, 2024-08-18T23:53:50.805 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:50.806 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 895, 2024-08-18T23:53:50.806 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.038881963, 2024-08-18T23:53:50.806 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000043443 2024-08-18T23:53:50.806 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.806 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:50.806 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.806 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.806 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.806 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.807 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 809632, 2024-08-18T23:53:50.807 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 128608 2024-08-18T23:53:50.807 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.807 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-08-18T23:53:50.807 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 931, 2024-08-18T23:53:50.807 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1086, 2024-08-18T23:53:50.807 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 4488523, 2024-08-18T23:53:50.807 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 636940, 2024-08-18T23:53:50.807 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 5, 2024-08-18T23:53:50.807 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-08-18T23:53:50.807 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.102019147, 2024-08-18T23:53:50.807 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.033765699, 2024-08-18T23:53:50.807 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.028422603, 2024-08-18T23:53:50.808 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.029825546, 2024-08-18T23:53:50.808 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:50.808 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1084, 2024-08-18T23:53:50.808 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.052155094, 2024-08-18T23:53:50.808 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000048113 2024-08-18T23:53:50.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.808 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:50.808 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.808 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.808 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.808 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-18T23:53:50.808 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 341200 2024-08-18T23:53:50.809 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.809 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-08-18T23:53:50.809 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-08-18T23:53:50.809 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-08-18T23:53:50.809 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.809 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-08-18T23:53:50.809 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-08-18T23:53:50.809 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-08-18T23:53:50.809 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3661, 2024-08-18T23:53:50.809 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.092048538, 2024-08-18T23:53:50.809 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000025143 2024-08-18T23:53:50.809 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:50.810 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.810 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-08-18T23:53:50.810 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-08-18T23:53:50.810 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-08-18T23:53:50.810 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.810 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.810 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.810 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:50.810 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.810 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-08-18T23:53:50.810 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2024-08-18T23:53:50.810 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2024-08-18T23:53:50.811 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-08-18T23:53:50.811 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.811 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.811 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.811 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.811 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-08-18T23:53:50.811 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-08-18T23:53:50.811 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-08-18T23:53:50.811 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-08-18T23:53:50.811 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2024-08-18T23:53:50.811 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-08-18T23:53:50.811 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-08-18T23:53:50.811 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 1773, 2024-08-18T23:53:50.812 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2024-08-18T23:53:50.812 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 1773, 2024-08-18T23:53:50.812 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-08-18T23:53:50.812 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 1773, 2024-08-18T23:53:50.812 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 0, 2024-08-18T23:53:50.812 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-08-18T23:53:50.812 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-08-18T23:53:50.812 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 1, 2024-08-18T23:53:50.812 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2024-08-18T23:53:50.812 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2024-08-18T23:53:50.812 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-08-18T23:53:50.812 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-08-18T23:53:50.812 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-08-18T23:53:50.813 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-08-18T23:53:50.813 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-08-18T23:53:50.813 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-08-18T23:53:50.813 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-08-18T23:53:50.813 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-08-18T23:53:50.813 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-08-18T23:53:50.813 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-08-18T23:53:50.813 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-08-18T23:53:50.813 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-08-18T23:53:50.813 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-08-18T23:53:50.813 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-08-18T23:53:50.814 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 0, 2024-08-18T23:53:50.814 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 0, 2024-08-18T23:53:50.814 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-08-18T23:53:50.814 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-08-18T23:53:50.814 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-08-18T23:53:50.814 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-08-18T23:53:50.814 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-08-18T23:53:50.814 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-08-18T23:53:50.814 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-08-18T23:53:50.814 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-08-18T23:53:50.814 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-08-18T23:53:50.814 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-08-18T23:53:50.815 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-08-18T23:53:50.815 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-08-18T23:53:50.815 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-08-18T23:53:50.815 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-18T23:53:50.815 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-18T23:53:50.815 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-08-18T23:53:50.815 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-08-18T23:53:50.815 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-08-18T23:53:50.815 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-08-18T23:53:50.815 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.815 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-08-18T23:53:50.815 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-08-18T23:53:50.815 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-08-18T23:53:50.816 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-08-18T23:53:50.816 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-08-18T23:53:50.816 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-08-18T23:53:50.816 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-08-18T23:53:50.816 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-08-18T23:53:50.816 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-08-18T23:53:50.816 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-08-18T23:53:50.816 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-08-18T23:53:50.816 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-08-18T23:53:50.816 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-08-18T23:53:50.816 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-08-18T23:53:50.816 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-08-18T23:53:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-08-18T23:53:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-08-18T23:53:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-08-18T23:53:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 0, 2024-08-18T23:53:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 0, 2024-08-18T23:53:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-08-18T23:53:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-08-18T23:53:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-08-18T23:53:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-08-18T23:53:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-08-18T23:53:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-08-18T23:53:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-08-18T23:53:50.817 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-08-18T23:53:50.818 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2024-08-18T23:53:50.818 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-08-18T23:53:50.818 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-08-18T23:53:50.818 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 9745, 2024-08-18T23:53:50.818 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-08-18T23:53:50.818 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-08-18T23:53:50.818 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2024-08-18T23:53:50.818 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-08-18T23:53:50.818 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-08-18T23:53:50.818 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 12, 2024-08-18T23:53:50.819 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 2, 2024-08-18T23:53:50.819 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-08-18T23:53:50.819 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-08-18T23:53:50.819 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-08-18T23:53:50.819 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 18360633, 2024-08-18T23:53:50.819 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 18359495, 2024-08-18T23:53:50.819 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-08-18T23:53:50.819 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.819 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.819 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.819 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.819 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 9745 2024-08-18T23:53:50.820 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.820 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-08-18T23:53:50.820 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 13, 2024-08-18T23:53:50.820 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 1800, 2024-08-18T23:53:50.820 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 1787, 2024-08-18T23:53:50.820 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 11, 2024-08-18T23:53:50.820 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 86, 2024-08-18T23:53:50.820 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 75, 2024-08-18T23:53:50.820 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 1773, 2024-08-18T23:53:50.820 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 3734, 2024-08-18T23:53:50.821 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 1961, 2024-08-18T23:53:50.821 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-08-18T23:53:50.821 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2024-08-18T23:53:50.821 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-08-18T23:53:50.821 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 53788, 2024-08-18T23:53:50.821 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 215328 2024-08-18T23:53:50.821 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.821 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-08-18T23:53:50.821 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2024-08-18T23:53:50.821 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-08-18T23:53:50.821 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2024-08-18T23:53:50.821 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2024-08-18T23:53:50.822 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-08-18T23:53:50.822 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-08-18T23:53:50.822 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.822 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.822 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.822 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.822 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-08-18T23:53:50.822 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.822 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.822 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.822 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.822 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-08-18T23:53:50.822 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.823 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.823 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.823 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.823 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-08-18T23:53:50.823 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.823 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.823 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.823 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.823 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-08-18T23:53:50.823 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.823 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.824 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.824 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-08-18T23:53:50.824 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.824 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.824 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.824 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-08-18T23:53:50.824 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.824 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.824 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.824 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-08-18T23:53:50.825 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.825 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.825 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.825 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.825 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-08-18T23:53:50.825 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.825 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.825 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.825 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.825 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-08-18T23:53:50.825 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.825 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.825 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.825 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.826 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-08-18T23:53:50.826 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.826 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.826 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.826 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.826 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-08-18T23:53:50.827 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.827 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.827 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.827 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.827 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-08-18T23:53:50.827 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.827 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.827 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.827 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.827 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-08-18T23:53:50.828 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.828 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.828 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.828 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.828 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-08-18T23:53:50.828 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.828 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.828 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.828 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.828 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-08-18T23:53:50.829 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.829 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.829 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.829 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.829 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-08-18T23:53:50.829 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.829 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.829 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.829 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.829 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-08-18T23:53:50.829 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.829 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.830 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.830 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-08-18T23:53:50.830 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.830 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.830 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.830 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-08-18T23:53:50.830 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.830 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.830 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.830 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.831 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-08-18T23:53:50.831 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.831 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.831 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.831 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.831 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-08-18T23:53:50.831 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.831 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.831 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.832 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.832 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-08-18T23:53:50.832 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.832 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.832 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.832 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.832 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-08-18T23:53:50.832 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.832 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.832 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.832 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.833 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-08-18T23:53:50.833 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.833 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.833 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.833 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.833 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-08-18T23:53:50.833 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.833 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.833 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.833 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.833 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-08-18T23:53:50.833 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.833 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.834 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.834 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.834 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-08-18T23:53:50.834 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.834 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.834 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.834 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.834 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-08-18T23:53:50.834 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.834 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.834 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.834 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.835 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-08-18T23:53:50.835 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-08-18T23:53:50.835 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-08-18T23:53:50.835 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.835 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.835 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.835 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:50.835 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.835 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-08-18T23:53:50.835 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-08-18T23:53:50.835 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 2, 2024-08-18T23:53:50.835 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 1, 2024-08-18T23:53:50.836 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 1, 2024-08-18T23:53:50.836 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-08-18T23:53:50.836 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2024-08-18T23:53:50.836 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-08-18T23:53:50.836 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 1496, 2024-08-18T23:53:50.836 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-08-18T23:53:50.836 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.836 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-08-18T23:53:50.836 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-08-18T23:53:50.836 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-08-18T23:53:50.836 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-08-18T23:53:50.836 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-08-18T23:53:50.836 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-08-18T23:53:50.837 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-08-18T23:53:50.837 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-18T23:53:50.837 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-08-18T23:53:50.837 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-18T23:53:50.837 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-08-18T23:53:50.837 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-08-18T23:53:50.837 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-08-18T23:53:50.837 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-18T23:53:50.837 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-08-18T23:53:50.837 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-08-18T23:53:50.838 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-08-18T23:53:50.838 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-08-18T23:53:50.838 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-08-18T23:53:50.838 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-18T23:53:50.838 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-08-18T23:53:50.838 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-18T23:53:50.838 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-08-18T23:53:50.838 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-08-18T23:53:50.838 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-08-18T23:53:50.838 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-08-18T23:53:50.838 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-08-18T23:53:50.838 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-18T23:53:50.839 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-08-18T23:53:50.839 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-08-18T23:53:50.839 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-08-18T23:53:50.839 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-08-18T23:53:50.839 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-08-18T23:53:50.839 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-08-18T23:53:50.839 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-08-18T23:53:50.839 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-08-18T23:53:50.839 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-08-18T23:53:50.839 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 207151, 2024-08-18T23:53:50.839 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 63, 2024-08-18T23:53:50.839 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-08-18T23:53:50.840 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-08-18T23:53:50.840 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-08-18T23:53:50.840 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-08-18T23:53:50.840 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-08-18T23:53:50.840 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-08-18T23:53:50.840 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-08-18T23:53:50.840 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-08-18T23:53:50.840 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8856, 2024-08-18T23:53:50.840 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 166, 2024-08-18T23:53:50.840 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-08-18T23:53:50.840 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-08-18T23:53:50.841 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-08-18T23:53:50.841 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-08-18T23:53:50.841 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 1291784, 2024-08-18T23:53:50.841 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 7220, 2024-08-18T23:53:50.841 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-08-18T23:53:50.841 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-08-18T23:53:50.841 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-08-18T23:53:50.841 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-08-18T23:53:50.841 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.841 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-08-18T23:53:50.841 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-08-18T23:53:50.842 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-08-18T23:53:50.842 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 2178, 2024-08-18T23:53:50.842 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 21, 2024-08-18T23:53:50.842 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 3, 2024-08-18T23:53:50.842 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 2178, 2024-08-18T23:53:50.842 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-08-18T23:53:50.842 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2178, 2024-08-18T23:53:50.842 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 3.165929698, 2024-08-18T23:53:50.842 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001453594 2024-08-18T23:53:50.842 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.842 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-08-18T23:53:50.842 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-08-18T23:53:50.842 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2178, 2024-08-18T23:53:50.843 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2179 2024-08-18T23:53:50.843 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.843 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 2178, 2024-08-18T23:53:50.843 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 2178, 2024-08-18T23:53:50.843 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2024-08-18T23:53:50.843 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-08-18T23:53:50.843 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-08-18T23:53:50.843 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 1324, 2024-08-18T23:53:50.843 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2024-08-18T23:53:50.843 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 852, 2024-08-18T23:53:50.843 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2024-08-18T23:53:50.843 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2024-08-18T23:53:50.843 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-08-18T23:53:50.844 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-08-18T23:53:50.844 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-08-18T23:53:50.844 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-08-18T23:53:50.844 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2024-08-18T23:53:50.844 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-08-18T23:53:50.844 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-08-18T23:53:50.844 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-08-18T23:53:50.844 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 1, 2024-08-18T23:53:50.844 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2024-08-18T23:53:50.844 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-08-18T23:53:50.844 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-08-18T23:53:50.844 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-08-18T23:53:50.844 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-08-18T23:53:50.845 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-08-18T23:53:50.845 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-08-18T23:53:50.845 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-08-18T23:53:50.845 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-08-18T23:53:50.845 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-08-18T23:53:50.845 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-08-18T23:53:50.845 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-08-18T23:53:50.845 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-08-18T23:53:50.845 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-08-18T23:53:50.845 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-08-18T23:53:50.845 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-08-18T23:53:50.845 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-08-18T23:53:50.845 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-08-18T23:53:50.846 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-08-18T23:53:50.846 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-08-18T23:53:50.846 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-08-18T23:53:50.846 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-08-18T23:53:50.846 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-08-18T23:53:50.846 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-08-18T23:53:50.846 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-08-18T23:53:50.847 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-08-18T23:53:50.847 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-08-18T23:53:50.847 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 92, 2024-08-18T23:53:50.847 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-08-18T23:53:50.847 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 4, 2024-08-18T23:53:50.847 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 7, 2024-08-18T23:53:50.847 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 7, 2024-08-18T23:53:50.847 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-08-18T23:53:50.847 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-08-18T23:53:50.847 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2024-08-18T23:53:50.847 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2024-08-18T23:53:50.847 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2024-08-18T23:53:50.847 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.848 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-08-18T23:53:50.848 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2024-08-18T23:53:50.848 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-08-18T23:53:50.848 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-08-18T23:53:50.848 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-08-18T23:53:50.848 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.849 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-08-18T23:53:50.849 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-08-18T23:53:50.849 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.849 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-08-18T23:53:50.849 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-08-18T23:53:50.849 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-08-18T23:53:50.849 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-08-18T23:53:50.849 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-08-18T23:53:50.849 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 0, 2024-08-18T23:53:50.849 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-08-18T23:53:50.850 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-08-18T23:53:50.850 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.850 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-18T23:53:50.850 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-18T23:53:50.850 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-08-18T23:53:50.850 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-18T23:53:50.850 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 2408, 2024-08-18T23:53:50.850 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 15356177, 2024-08-18T23:53:50.851 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:50.851 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 2408, 2024-08-18T23:53:50.851 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-18T23:53:50.851 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-18T23:53:50.851 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2408, 2024-08-18T23:53:50.851 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 15356177, 2024-08-18T23:53:50.851 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-18T23:53:50.851 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.851 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.851 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.851 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:50.851 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.851 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-08-18T23:53:50.851 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-18T23:53:50.852 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-08-18T23:53:50.852 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-18T23:53:50.852 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-18T23:53:50.852 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-18T23:53:50.852 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:50.852 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-18T23:53:50.852 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 2178, 2024-08-18T23:53:50.852 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 14166479, 2024-08-18T23:53:50.852 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2178, 2024-08-18T23:53:50.852 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 14166479, 2024-08-18T23:53:50.852 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-18T23:53:50.852 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.852 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.853 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.853 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:50.853 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.853 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-08-18T23:53:50.853 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-18T23:53:50.853 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-08-18T23:53:50.853 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-18T23:53:50.853 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-18T23:53:50.853 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-18T23:53:50.853 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:50.853 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-18T23:53:50.853 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 2178, 2024-08-18T23:53:50.853 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 2178, 2024-08-18T23:53:50.854 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2178, 2024-08-18T23:53:50.854 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 2178, 2024-08-18T23:53:50.854 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-18T23:53:50.854 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.854 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.854 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.854 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:50.854 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.854 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-08-18T23:53:50.854 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-18T23:53:50.854 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-08-18T23:53:50.854 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-18T23:53:50.854 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-18T23:53:50.855 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-18T23:53:50.855 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:50.855 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-18T23:53:50.855 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-18T23:53:50.855 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-18T23:53:50.855 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-08-18T23:53:50.855 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-08-18T23:53:50.855 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-18T23:53:50.855 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.855 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.855 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.855 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:50.855 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:50.856 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x5576c4920b40": { 2024-08-18T23:53:50.856 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-18T23:53:50.856 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-08-18T23:53:50.856 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-18T23:53:50.856 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-18T23:53:50.856 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-18T23:53:50.856 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:50.856 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-18T23:53:50.856 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-18T23:53:50.856 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-18T23:53:50.857 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-08-18T23:53:50.857 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-08-18T23:53:50.857 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-18T23:53:50.857 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:50.857 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:50.857 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:50.857 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:50.857 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:50.857 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-08-18T23:53:51.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:51 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:51.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:51 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:51.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:51 smithi123 ceph-mon[21808]: pgmap v636: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-08-18T23:53:51.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:51 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:53:51.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:51 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:53:51.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:51 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:51.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:51 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:51.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:51 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:51.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:51 smithi161 ceph-mon[22027]: pgmap v636: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-08-18T23:53:51.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:51 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:53:51.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:51 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:53:51.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:51 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:51.621 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-08-18T23:53:51.621 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-08-18T23:53:51.621 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-08-18T23:53:51.621 DEBUG:tasks.check_counter:Getting stats from i 2024-08-18T23:53:51.621 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph daemon mds.i perf dump 2024-08-18T23:53:51.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:51 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:51.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:51 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:51.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:51 smithi078 ceph-mon[18440]: pgmap v636: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-08-18T23:53:51.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:51 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-18T23:53:51.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:51 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-18T23:53:51.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:51 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' 2024-08-18T23:53:51.873 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.c/config 2024-08-18T23:53:52.399 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-08-18T23:53:52.399 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-08-18T23:53:52.399 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 33274, 2024-08-18T23:53:52.400 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 42997, 2024-08-18T23:53:52.400 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 12021501, 2024-08-18T23:53:52.400 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 23095666, 2024-08-18T23:53:52.400 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 31, 2024-08-18T23:53:52.400 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 7, 2024-08-18T23:53:52.400 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 1.454765595, 2024-08-18T23:53:52.400 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.598763234, 2024-08-18T23:53:52.400 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 1.933327445, 2024-08-18T23:53:52.400 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.080403156, 2024-08-18T23:53:52.400 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:52.400 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 42997, 2024-08-18T23:53:52.401 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1.329138056, 2024-08-18T23:53:52.401 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000030912 2024-08-18T23:53:52.401 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.401 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:52.401 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 58401, 2024-08-18T23:53:52.401 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.011560974, 2024-08-18T23:53:52.401 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000197 2024-08-18T23:53:52.401 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.401 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 823040, 2024-08-18T23:53:52.401 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 145552 2024-08-18T23:53:52.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.402 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-08-18T23:53:52.402 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 3279, 2024-08-18T23:53:52.402 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 3433, 2024-08-18T23:53:52.402 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 838482, 2024-08-18T23:53:52.402 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 8047297, 2024-08-18T23:53:52.402 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 14, 2024-08-18T23:53:52.402 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 6, 2024-08-18T23:53:52.402 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.244846629, 2024-08-18T23:53:52.402 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.093725679, 2024-08-18T23:53:52.402 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.055962630, 2024-08-18T23:53:52.402 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.077429218, 2024-08-18T23:53:52.403 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:52.403 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3431, 2024-08-18T23:53:52.403 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.160036838, 2024-08-18T23:53:52.403 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000046644 2024-08-18T23:53:52.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.403 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:52.403 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 6, 2024-08-18T23:53:52.403 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000680, 2024-08-18T23:53:52.403 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000113 2024-08-18T23:53:52.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.403 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-18T23:53:52.404 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 343504 2024-08-18T23:53:52.404 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.404 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-08-18T23:53:52.404 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 11845, 2024-08-18T23:53:52.404 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 15608, 2024-08-18T23:53:52.404 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 3803695, 2024-08-18T23:53:52.404 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 8618652, 2024-08-18T23:53:52.404 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 10, 2024-08-18T23:53:52.404 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 6, 2024-08-18T23:53:52.404 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.805647310, 2024-08-18T23:53:52.404 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.308015570, 2024-08-18T23:53:52.404 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.526891933, 2024-08-18T23:53:52.405 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.064070456, 2024-08-18T23:53:52.405 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:52.405 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 15606, 2024-08-18T23:53:52.405 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.611972527, 2024-08-18T23:53:52.405 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000039213 2024-08-18T23:53:52.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.405 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:52.405 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 22065, 2024-08-18T23:53:52.405 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.004219018, 2024-08-18T23:53:52.405 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000191 2024-08-18T23:53:52.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.405 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-18T23:53:52.406 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-08-18T23:53:52.406 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.406 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-08-18T23:53:52.406 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-08-18T23:53:52.406 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-08-18T23:53:52.406 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.406 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-08-18T23:53:52.406 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-08-18T23:53:52.406 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-08-18T23:53:52.406 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4736, 2024-08-18T23:53:52.406 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 3.823496055, 2024-08-18T23:53:52.406 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000807326 2024-08-18T23:53:52.407 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:52.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.407 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-08-18T23:53:52.407 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-08-18T23:53:52.408 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-08-18T23:53:52.408 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3457, 2024-08-18T23:53:52.408 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.238229269, 2024-08-18T23:53:52.408 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000068912 2024-08-18T23:53:52.409 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:52.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.409 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-08-18T23:53:52.409 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 11671, 2024-08-18T23:53:52.409 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 10697, 2024-08-18T23:53:52.409 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-08-18T23:53:52.410 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 10697, 2024-08-18T23:53:52.410 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 111.350108097, 2024-08-18T23:53:52.410 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.010409470 2024-08-18T23:53:52.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.410 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-08-18T23:53:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 974, 2024-08-18T23:53:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-08-18T23:53:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-08-18T23:53:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 56, 2024-08-18T23:53:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 5, 2024-08-18T23:53:52.411 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-08-18T23:53:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 15, 2024-08-18T23:53:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2024-08-18T23:53:52.412 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 1, 2024-08-18T23:53:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 14, 2024-08-18T23:53:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 14, 2024-08-18T23:53:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 77, 2024-08-18T23:53:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 1, 2024-08-18T23:53:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 1, 2024-08-18T23:53:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 3, 2024-08-18T23:53:52.413 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 24717, 2024-08-18T23:53:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 18673, 2024-08-18T23:53:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 972, 2024-08-18T23:53:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 14, 2024-08-18T23:53:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-08-18T23:53:52.414 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-08-18T23:53:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 478, 2024-08-18T23:53:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-08-18T23:53:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-08-18T23:53:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 40, 2024-08-18T23:53:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 34, 2024-08-18T23:53:52.415 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 5, 2024-08-18T23:53:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 743, 2024-08-18T23:53:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-08-18T23:53:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 4331, 2024-08-18T23:53:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-08-18T23:53:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 125, 2024-08-18T23:53:52.416 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 3050627, 2024-08-18T23:53:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-08-18T23:53:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 4331, 2024-08-18T23:53:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-08-18T23:53:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-08-18T23:53:52.417 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-08-18T23:53:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-08-18T23:53:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-08-18T23:53:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-08-18T23:53:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 4331, 2024-08-18T23:53:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 127, 2024-08-18T23:53:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 5672, 2024-08-18T23:53:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 7027, 2024-08-18T23:53:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-08-18T23:53:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-18T23:53:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-18T23:53:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 8616, 2024-08-18T23:53:52.418 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 1707, 2024-08-18T23:53:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 181, 2024-08-18T23:53:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 7063 2024-08-18T23:53:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-08-18T23:53:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 1, 2024-08-18T23:53:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 23, 2024-08-18T23:53:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 15, 2024-08-18T23:53:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 28, 2024-08-18T23:53:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 8, 2024-08-18T23:53:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-08-18T23:53:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-08-18T23:53:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-08-18T23:53:52.419 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 1763, 2024-08-18T23:53:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 1763, 2024-08-18T23:53:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-08-18T23:53:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-08-18T23:53:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-08-18T23:53:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-08-18T23:53:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-08-18T23:53:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-08-18T23:53:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-08-18T23:53:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 7, 2024-08-18T23:53:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 3177, 2024-08-18T23:53:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-08-18T23:53:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 40, 2024-08-18T23:53:52.420 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-08-18T23:53:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 5, 2024-08-18T23:53:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-08-18T23:53:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-08-18T23:53:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-08-18T23:53:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-08-18T23:53:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 9745, 2024-08-18T23:53:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-08-18T23:53:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-08-18T23:53:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 9745, 2024-08-18T23:53:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-08-18T23:53:52.421 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-08-18T23:53:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 12, 2024-08-18T23:53:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-08-18T23:53:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-08-18T23:53:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 12, 2024-08-18T23:53:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 2, 2024-08-18T23:53:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-08-18T23:53:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-08-18T23:53:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-08-18T23:53:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 18360633, 2024-08-18T23:53:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-08-18T23:53:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-08-18T23:53:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 714, 2024-08-18T23:53:52.422 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 6.852653014, 2024-08-18T23:53:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.009597553 2024-08-18T23:53:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-08-18T23:53:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-08-18T23:53:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 19, 2024-08-18T23:53:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 1803, 2024-08-18T23:53:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 1784, 2024-08-18T23:53:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 18, 2024-08-18T23:53:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 116, 2024-08-18T23:53:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 98, 2024-08-18T23:53:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 15, 2024-08-18T23:53:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 3755, 2024-08-18T23:53:52.423 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 3740, 2024-08-18T23:53:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 1, 2024-08-18T23:53:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 2655, 2024-08-18T23:53:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 2654, 2024-08-18T23:53:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 70256, 2024-08-18T23:53:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 223520 2024-08-18T23:53:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-08-18T23:53:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 16535, 2024-08-18T23:53:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 406, 2024-08-18T23:53:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 11671, 2024-08-18T23:53:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 12, 2024-08-18T23:53:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 4665, 2024-08-18T23:53:52.424 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-08-18T23:53:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1439, 2024-08-18T23:53:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 56.175570301, 2024-08-18T23:53:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.039037922 2024-08-18T23:53:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-08-18T23:53:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-08-18T23:53:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.001244243, 2024-08-18T23:53:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000622121 2024-08-18T23:53:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-08-18T23:53:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-08-18T23:53:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-08-18T23:53:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 5558, 2024-08-18T23:53:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 40.592994857, 2024-08-18T23:53:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.007303525 2024-08-18T23:53:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-08-18T23:53:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-08-18T23:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-08-18T23:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.427 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-08-18T23:53:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-08-18T23:53:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.428 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-08-18T23:53:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-08-18T23:53:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-08-18T23:53:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-08-18T23:53:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.430 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-08-18T23:53:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1291, 2024-08-18T23:53:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 3.856329800, 2024-08-18T23:53:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002987087 2024-08-18T23:53:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-08-18T23:53:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 40, 2024-08-18T23:53:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.032417268, 2024-08-18T23:53:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000810431 2024-08-18T23:53:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-08-18T23:53:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1439, 2024-08-18T23:53:52.431 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 8.956691372, 2024-08-18T23:53:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.006224246 2024-08-18T23:53:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-08-18T23:53:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-08-18T23:53:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.432 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-08-18T23:53:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-08-18T23:53:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-08-18T23:53:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-08-18T23:53:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-08-18T23:53:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-08-18T23:53:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.434 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-08-18T23:53:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-08-18T23:53:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.435 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-08-18T23:53:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-08-18T23:53:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 928, 2024-08-18T23:53:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1.734860256, 2024-08-18T23:53:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001869461 2024-08-18T23:53:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-08-18T23:53:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-08-18T23:53:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-08-18T23:53:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.436 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-08-18T23:53:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-08-18T23:53:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 2, 2024-08-18T23:53:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 1, 2024-08-18T23:53:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 1, 2024-08-18T23:53:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-08-18T23:53:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 2066, 2024-08-18T23:53:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 2066, 2024-08-18T23:53:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 591, 2024-08-18T23:53:52.437 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-08-18T23:53:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-08-18T23:53:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-08-18T23:53:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-08-18T23:53:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-08-18T23:53:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-08-18T23:53:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-08-18T23:53:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-08-18T23:53:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-18T23:53:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-08-18T23:53:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-18T23:53:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-08-18T23:53:52.438 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-08-18T23:53:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-08-18T23:53:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-18T23:53:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-08-18T23:53:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-08-18T23:53:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-08-18T23:53:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-08-18T23:53:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-08-18T23:53:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-18T23:53:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-08-18T23:53:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-18T23:53:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-08-18T23:53:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-08-18T23:53:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-08-18T23:53:52.439 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-08-18T23:53:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-08-18T23:53:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-18T23:53:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-08-18T23:53:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-08-18T23:53:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-08-18T23:53:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-08-18T23:53:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-08-18T23:53:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-08-18T23:53:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-08-18T23:53:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-08-18T23:53:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-08-18T23:53:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 287838, 2024-08-18T23:53:52.440 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 105, 2024-08-18T23:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 88, 2024-08-18T23:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 1, 2024-08-18T23:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-08-18T23:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-08-18T23:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-08-18T23:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-08-18T23:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-08-18T23:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-08-18T23:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8856, 2024-08-18T23:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 166, 2024-08-18T23:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-08-18T23:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-08-18T23:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-08-18T23:53:52.441 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-08-18T23:53:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 78549, 2024-08-18T23:53:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 412, 2024-08-18T23:53:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-08-18T23:53:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-08-18T23:53:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-08-18T23:53:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-08-18T23:53:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-08-18T23:53:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-08-18T23:53:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-08-18T23:53:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 9101, 2024-08-18T23:53:52.442 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 15527089, 2024-08-18T23:53:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 27, 2024-08-18T23:53:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 9101, 2024-08-18T23:53:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-08-18T23:53:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 9101, 2024-08-18T23:53:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 63.881581631, 2024-08-18T23:53:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.007019182 2024-08-18T23:53:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-08-18T23:53:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-08-18T23:53:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 9101, 2024-08-18T23:53:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 9361 2024-08-18T23:53:52.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 9101, 2024-08-18T23:53:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 4434, 2024-08-18T23:53:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 4667, 2024-08-18T23:53:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-08-18T23:53:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-08-18T23:53:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 0, 2024-08-18T23:53:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 10, 2024-08-18T23:53:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 103, 2024-08-18T23:53:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 817, 2024-08-18T23:53:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 83, 2024-08-18T23:53:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-08-18T23:53:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-08-18T23:53:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-08-18T23:53:52.444 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 5, 2024-08-18T23:53:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 3541, 2024-08-18T23:53:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-08-18T23:53:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-08-18T23:53:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-08-18T23:53:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 4331, 2024-08-18T23:53:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 20, 2024-08-18T23:53:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-08-18T23:53:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-08-18T23:53:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-08-18T23:53:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-08-18T23:53:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-08-18T23:53:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-08-18T23:53:52.445 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-08-18T23:53:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-08-18T23:53:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-08-18T23:53:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-08-18T23:53:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-08-18T23:53:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-08-18T23:53:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-08-18T23:53:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-08-18T23:53:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-08-18T23:53:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-08-18T23:53:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-08-18T23:53:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-08-18T23:53:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-08-18T23:53:52.446 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-08-18T23:53:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-08-18T23:53:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-08-18T23:53:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-08-18T23:53:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-08-18T23:53:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-08-18T23:53:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-08-18T23:53:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 92, 2024-08-18T23:53:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-08-18T23:53:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 4, 2024-08-18T23:53:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 12, 2024-08-18T23:53:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 12, 2024-08-18T23:53:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-08-18T23:53:52.447 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-08-18T23:53:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 371, 2024-08-18T23:53:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 0, 2024-08-18T23:53:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 80 2024-08-18T23:53:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-08-18T23:53:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-08-18T23:53:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-08-18T23:53:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-08-18T23:53:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-08-18T23:53:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-08-18T23:53:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 3186 2024-08-18T23:53:52.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-08-18T23:53:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-08-18T23:53:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 16, 2024-08-18T23:53:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-08-18T23:53:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 8, 2024-08-18T23:53:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 3526, 2024-08-18T23:53:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 1763, 2024-08-18T23:53:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-08-18T23:53:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-18T23:53:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-18T23:53:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-08-18T23:53:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-18T23:53:52.449 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 48394, 2024-08-18T23:53:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 12291282, 2024-08-18T23:53:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 48394, 2024-08-18T23:53:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-18T23:53:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-18T23:53:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 48394, 2024-08-18T23:53:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 12291282, 2024-08-18T23:53:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-18T23:53:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.450 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-08-18T23:53:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-18T23:53:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-08-18T23:53:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-18T23:53:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-18T23:53:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-18T23:53:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-18T23:53:52.451 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 9101, 2024-08-18T23:53:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 15719256, 2024-08-18T23:53:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 9101, 2024-08-18T23:53:52.454 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 15719256, 2024-08-18T23:53:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-18T23:53:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-08-18T23:53:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-18T23:53:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-08-18T23:53:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-18T23:53:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-18T23:53:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-18T23:53:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:52.455 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-18T23:53:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 9101, 2024-08-18T23:53:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 9101, 2024-08-18T23:53:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 9101, 2024-08-18T23:53:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 9101, 2024-08-18T23:53:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-18T23:53:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-08-18T23:53:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-18T23:53:52.456 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-08-18T23:53:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-18T23:53:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 1763, 2024-08-18T23:53:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 192167, 2024-08-18T23:53:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 1763, 2024-08-18T23:53:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-18T23:53:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-18T23:53:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 103, 2024-08-18T23:53:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 192167, 2024-08-18T23:53:52.457 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-18T23:53:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x5560b7c16960": { 2024-08-18T23:53:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-18T23:53:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-08-18T23:53:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-18T23:53:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 9745, 2024-08-18T23:53:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 14166329, 2024-08-18T23:53:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 9745, 2024-08-18T23:53:52.458 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-18T23:53:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-18T23:53:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 714, 2024-08-18T23:53:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 14166329, 2024-08-18T23:53:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-18T23:53:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:52.459 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:52.459 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-08-18T23:53:53.250 INFO:tasks.check_counter:Daemon mds.i mds.exported=40 2024-08-18T23:53:53.250 INFO:tasks.check_counter:Daemon mds.i mds.imported=5 2024-08-18T23:53:53.250 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=5 2024-08-18T23:53:53.250 DEBUG:tasks.check_counter:Getting stats from l 2024-08-18T23:53:53.251 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph daemon mds.l perf dump 2024-08-18T23:53:53.505 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.c/config 2024-08-18T23:53:53.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:53 smithi161 ceph-mon[22027]: pgmap v637: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-18T23:53:53.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:53 smithi078 ceph-mon[18440]: pgmap v637: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-18T23:53:53.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:53 smithi123 ceph-mon[21808]: pgmap v637: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-18T23:53:54.003 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-08-18T23:53:54.003 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-08-18T23:53:54.003 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 0, 2024-08-18T23:53:54.003 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 2, 2024-08-18T23:53:54.003 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 0, 2024-08-18T23:53:54.003 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 0, 2024-08-18T23:53:54.003 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-08-18T23:53:54.003 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 0, 2024-08-18T23:53:54.003 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.000970312, 2024-08-18T23:53:54.004 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.000000000, 2024-08-18T23:53:54.004 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.000145712, 2024-08-18T23:53:54.004 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-18T23:53:54.004 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:54.004 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:54.004 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:54.004 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:54.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:54.004 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:54.004 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:54.004 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:54.004 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:54.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:54.005 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-18T23:53:54.005 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-08-18T23:53:54.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:54.005 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-08-18T23:53:54.005 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 5, 2024-08-18T23:53:54.005 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 162, 2024-08-18T23:53:54.005 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 3184, 2024-08-18T23:53:54.005 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 106832, 2024-08-18T23:53:54.005 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 3, 2024-08-18T23:53:54.005 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2024-08-18T23:53:54.005 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.011428886, 2024-08-18T23:53:54.005 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.006976902, 2024-08-18T23:53:54.006 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.002681779, 2024-08-18T23:53:54.006 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-18T23:53:54.006 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:54.006 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 161, 2024-08-18T23:53:54.006 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.013606175, 2024-08-18T23:53:54.006 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000084510 2024-08-18T23:53:54.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:54.006 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:54.006 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:54.007 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:54.007 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:54.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:54.007 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-08-18T23:53:54.007 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 106832 2024-08-18T23:53:54.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:54.007 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-08-18T23:53:54.007 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 215, 2024-08-18T23:53:54.007 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 210, 2024-08-18T23:53:54.008 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 752560, 2024-08-18T23:53:54.008 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 129184, 2024-08-18T23:53:54.008 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 2, 2024-08-18T23:53:54.008 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2024-08-18T23:53:54.008 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.032300764, 2024-08-18T23:53:54.008 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.013444145, 2024-08-18T23:53:54.008 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.025071840, 2024-08-18T23:53:54.008 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-18T23:53:54.008 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-18T23:53:54.009 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 209, 2024-08-18T23:53:54.009 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.012623026, 2024-08-18T23:53:54.009 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000060397 2024-08-18T23:53:54.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:54.009 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-18T23:53:54.009 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:54.009 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:54.009 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:54.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:54.009 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 752560, 2024-08-18T23:53:54.009 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 129184 2024-08-18T23:53:54.009 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:54.009 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-08-18T23:53:54.010 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 0, 2024-08-18T23:53:54.010 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-08-18T23:53:54.010 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:54.010 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-08-18T23:53:54.010 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-08-18T23:53:54.010 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-08-18T23:53:54.010 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-08-18T23:53:54.010 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-08-18T23:53:54.010 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-08-18T23:53:54.010 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-08-18T23:53:54.010 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-18T23:53:54.010 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-08-18T23:53:54.011 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-18T23:53:54.011 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-08-18T23:53:54.011 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-08-18T23:53:54.011 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-08-18T23:53:54.011 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-18T23:53:54.011 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-08-18T23:53:54.011 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-08-18T23:53:54.011 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-08-18T23:53:54.011 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-08-18T23:53:54.011 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-08-18T23:53:54.011 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-18T23:53:54.011 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-08-18T23:53:54.011 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-18T23:53:54.012 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-08-18T23:53:54.012 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-08-18T23:53:54.012 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-08-18T23:53:54.012 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-08-18T23:53:54.012 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-08-18T23:53:54.012 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-18T23:53:54.012 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-08-18T23:53:54.012 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-08-18T23:53:54.012 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-08-18T23:53:54.012 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-08-18T23:53:54.012 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-08-18T23:53:54.012 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-08-18T23:53:54.013 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-08-18T23:53:54.013 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-08-18T23:53:54.013 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-08-18T23:53:54.013 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 148984, 2024-08-18T23:53:54.013 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 30, 2024-08-18T23:53:54.013 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-08-18T23:53:54.013 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-08-18T23:53:54.013 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-08-18T23:53:54.013 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-08-18T23:53:54.013 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-08-18T23:53:54.013 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-08-18T23:53:54.013 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-08-18T23:53:54.013 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-08-18T23:53:54.014 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 0, 2024-08-18T23:53:54.014 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 0, 2024-08-18T23:53:54.014 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-08-18T23:53:54.014 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-08-18T23:53:54.014 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-08-18T23:53:54.014 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-08-18T23:53:54.014 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 928, 2024-08-18T23:53:54.014 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 2, 2024-08-18T23:53:54.014 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-08-18T23:53:54.014 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-08-18T23:53:54.014 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-08-18T23:53:54.014 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-08-18T23:53:54.014 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-18T23:53:54.015 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-18T23:53:54.015 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-18T23:53:54.015 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-08-18T23:53:54.015 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-18T23:53:54.015 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 216, 2024-08-18T23:53:54.015 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 722208, 2024-08-18T23:53:54.015 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-18T23:53:54.015 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 216, 2024-08-18T23:53:54.015 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-18T23:53:54.015 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-18T23:53:54.015 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 216, 2024-08-18T23:53:54.016 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 722208, 2024-08-18T23:53:54.016 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-18T23:53:54.016 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-18T23:53:54.016 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-18T23:53:54.016 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-18T23:53:54.016 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:54.016 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-18T23:53:54.016 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-08-18T23:53:54.632 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-08-18T23:53:54.632 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-08-18T23:53:54.632 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-08-18T23:53:54.632 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-08-18T23:53:54.642 INFO:tasks.kclient:Unmounting kernel clients... 2024-08-18T23:53:54.642 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:53:54.642 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-18T23:53:54.663 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-08-18T23:53:54.663 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-18T23:53:54.725 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd blocklist ls 2024-08-18T23:53:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:53:55.448 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:55 smithi078 ceph-mon[18440]: pgmap v638: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-08-18T23:53:55.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.529+0000 7f2dad808640 1 -- 172.21.15.78:0/3661689794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2da8102ae0 msgr2=0x7f2da8102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:55.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.529+0000 7f2dad808640 1 --2- 172.21.15.78:0/3661689794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2da8102ae0 0x7f2da8102ec0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009930 tx=0x7f2d9802a420 comp rx=0 tx=0).stop 2024-08-18T23:53:55.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.529+0000 7f2dad808640 1 -- 172.21.15.78:0/3661689794 shutdown_connections 2024-08-18T23:53:55.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.529+0000 7f2dad808640 1 --2- 172.21.15.78:0/3661689794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2da8108010 0x7f2da810e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:55.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.529+0000 7f2dad808640 1 --2- 172.21.15.78:0/3661689794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2da8103490 0x7f2da81078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:55.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.529+0000 7f2dad808640 1 --2- 172.21.15.78:0/3661689794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2da8102ae0 0x7f2da8102ec0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:55.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.529+0000 7f2dad808640 1 -- 172.21.15.78:0/3661689794 >> 172.21.15.78:0/3661689794 conn(0x7f2da80fc500 msgr2=0x7f2da80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:55.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.529+0000 7f2dad808640 1 -- 172.21.15.78:0/3661689794 shutdown_connections 2024-08-18T23:53:55.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.529+0000 7f2dad808640 1 -- 172.21.15.78:0/3661689794 wait complete. 2024-08-18T23:53:55.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.530+0000 7f2dad808640 1 Processor -- start 2024-08-18T23:53:55.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.530+0000 7f2dad808640 1 -- start start 2024-08-18T23:53:55.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.530+0000 7f2dad808640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2da8102ae0 0x7f2da8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:55.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.530+0000 7f2dad808640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2da8103490 0x7f2da81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:55.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.530+0000 7f2dad808640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2da8108010 0x7f2da819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:55.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.530+0000 7f2dad808640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2da8111e60 con 0x7f2da8102ae0 2024-08-18T23:53:55.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.530+0000 7f2dad808640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2da8111ce0 con 0x7f2da8103490 2024-08-18T23:53:55.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.530+0000 7f2dad808640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2da8111fe0 con 0x7f2da8108010 2024-08-18T23:53:55.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.530+0000 7f2da6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2da8102ae0 0x7f2da8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:55.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.530+0000 7f2da6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2da8102ae0 0x7f2da8198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50694/0 (socket says 172.21.15.78:50694) 2024-08-18T23:53:55.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.530+0000 7f2da6ffd640 1 -- 172.21.15.78:0/3283056398 learned_addr learned my addr 172.21.15.78:0/3283056398 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:55.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.530+0000 7f2da67fc640 1 --2- 172.21.15.78:0/3283056398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2da8103490 0x7f2da81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:55.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.530+0000 7f2da77fe640 1 --2- 172.21.15.78:0/3283056398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2da8108010 0x7f2da819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:55.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.530+0000 7f2da6ffd640 1 -- 172.21.15.78:0/3283056398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2da8108010 msgr2=0x7f2da819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:55.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.530+0000 7f2da6ffd640 1 --2- 172.21.15.78:0/3283056398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2da8108010 0x7f2da819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:55.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.530+0000 7f2da6ffd640 1 -- 172.21.15.78:0/3283056398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2da8103490 msgr2=0x7f2da81988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:55.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.530+0000 7f2da6ffd640 1 --2- 172.21.15.78:0/3283056398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2da8103490 0x7f2da81988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:55.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.530+0000 7f2da6ffd640 1 -- 172.21.15.78:0/3283056398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da819d310 con 0x7f2da8102ae0 2024-08-18T23:53:55.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.531+0000 7f2da67fc640 1 --2- 172.21.15.78:0/3283056398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2da8103490 0x7f2da81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:53:55.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.531+0000 7f2da6ffd640 1 --2- 172.21.15.78:0/3283056398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2da8102ae0 0x7f2da8198360 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2d98004770 tx=0x7f2d98003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:55.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.531+0000 7f2dac806640 1 -- 172.21.15.78:0/3283056398 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9803f070 con 0x7f2da8102ae0 2024-08-18T23:53:55.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.531+0000 7f2dad808640 1 -- 172.21.15.78:0/3283056398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da819d5a0 con 0x7f2da8102ae0 2024-08-18T23:53:55.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.531+0000 7f2dac806640 1 -- 172.21.15.78:0/3283056398 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d98031c20 con 0x7f2da8102ae0 2024-08-18T23:53:55.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.531+0000 7f2dac806640 1 -- 172.21.15.78:0/3283056398 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98035030 con 0x7f2da8102ae0 2024-08-18T23:53:55.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.531+0000 7f2dad808640 1 -- 172.21.15.78:0/3283056398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da81a4ee0 con 0x7f2da8102ae0 2024-08-18T23:53:55.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.532+0000 7f2dad808640 1 -- 172.21.15.78:0/3283056398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da806a500 con 0x7f2da8102ae0 2024-08-18T23:53:55.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.533+0000 7f2dac806640 1 -- 172.21.15.78:0/3283056398 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2d98042070 con 0x7f2da8102ae0 2024-08-18T23:53:55.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.534+0000 7f2dac806640 1 --2- 172.21.15.78:0/3283056398 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2d900a2b80 0x7f2d900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:55.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.534+0000 7f2dac806640 1 -- 172.21.15.78:0/3283056398 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f2d980efce0 con 0x7f2da8102ae0 2024-08-18T23:53:55.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.534+0000 7f2da67fc640 1 --2- 172.21.15.78:0/3283056398 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2d900a2b80 0x7f2d900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:55.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.534+0000 7f2da67fc640 1 --2- 172.21.15.78:0/3283056398 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2d900a2b80 0x7f2d900a5040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0058c0 tx=0x7f2d9c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:55.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.538+0000 7f2dac806640 1 -- 172.21.15.78:0/3283056398 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2d980bc8b0 con 0x7f2da8102ae0 2024-08-18T23:53:55.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:55 smithi161 ceph-mon[22027]: pgmap v638: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-08-18T23:53:55.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.724+0000 7f2dad808640 1 -- 172.21.15.78:0/3283056398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2da8109ff0 con 0x7f2da8102ae0 2024-08-18T23:53:55.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.724+0000 7f2dac806640 1 -- 172.21.15.78:0/3283056398 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+588 (secure 0 0 0) 0x7f2d980bc1b0 con 0x7f2da8102ae0 2024-08-18T23:53:55.727 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/311758909 2024-08-19T23:35:22.036055+0000 2024-08-18T23:53:55.727 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/311758909 2024-08-19T23:35:22.036055+0000 2024-08-18T23:53:55.727 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/3040499240 2024-08-19T23:35:22.036055+0000 2024-08-18T23:53:55.727 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/2301978296 2024-08-19T23:34:59.021165+0000 2024-08-18T23:53:55.727 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/2301978296 2024-08-19T23:34:59.021165+0000 2024-08-18T23:53:55.727 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/757409165 2024-08-19T23:34:59.021165+0000 2024-08-18T23:53:55.727 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2562072965 2024-08-19T23:35:22.036055+0000 2024-08-18T23:53:55.727 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/636184437 2024-08-19T23:34:59.021165+0000 2024-08-18T23:53:55.727 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/1570708294 2024-08-19T23:35:22.036055+0000 2024-08-18T23:53:55.728 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/3137806628 2024-08-19T23:34:59.021165+0000 2024-08-18T23:53:55.728 INFO:teuthology.orchestra.run.smithi078.stderr:listed 10 entries 2024-08-18T23:53:55.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.726+0000 7f2dad808640 1 -- 172.21.15.78:0/3283056398 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2d900a2b80 msgr2=0x7f2d900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:55.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.726+0000 7f2dad808640 1 --2- 172.21.15.78:0/3283056398 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2d900a2b80 0x7f2d900a5040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0058c0 tx=0x7f2d9c005850 comp rx=0 tx=0).stop 2024-08-18T23:53:55.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.726+0000 7f2dad808640 1 -- 172.21.15.78:0/3283056398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2da8102ae0 msgr2=0x7f2da8198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:55.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.726+0000 7f2dad808640 1 --2- 172.21.15.78:0/3283056398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2da8102ae0 0x7f2da8198360 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2d98004770 tx=0x7f2d98003b90 comp rx=0 tx=0).stop 2024-08-18T23:53:55.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.727+0000 7f2dad808640 1 -- 172.21.15.78:0/3283056398 shutdown_connections 2024-08-18T23:53:55.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.727+0000 7f2dad808640 1 --2- 172.21.15.78:0/3283056398 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f2d900a2b80 0x7f2d900a5040 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:55.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.727+0000 7f2dad808640 1 --2- 172.21.15.78:0/3283056398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2da8108010 0x7f2da819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:55.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.727+0000 7f2dad808640 1 --2- 172.21.15.78:0/3283056398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2da8103490 0x7f2da81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:55.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.727+0000 7f2dad808640 1 --2- 172.21.15.78:0/3283056398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2da8102ae0 0x7f2da8198360 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:55.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.727+0000 7f2dad808640 1 -- 172.21.15.78:0/3283056398 >> 172.21.15.78:0/3283056398 conn(0x7f2da80fc500 msgr2=0x7f2da81060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:55.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.727+0000 7f2dad808640 1 -- 172.21.15.78:0/3283056398 shutdown_connections 2024-08-18T23:53:55.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:55.727+0000 7f2dad808640 1 -- 172.21.15.78:0/3283056398 wait complete. 2024-08-18T23:53:55.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:55 smithi123 ceph-mon[21808]: pgmap v638: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-08-18T23:53:56.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:56 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3283056398' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-18T23:53:56.676 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-08-18T23:53:56.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:53:56.676 DEBUG:teuthology.orchestra.run.smithi078:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-08-18T23:53:56.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:56 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3283056398' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-18T23:53:56.725 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-08-18T23:53:56.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:53:56.725 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-18T23:53:56.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:53:56.759 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-08-18T23:53:56.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:56 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3283056398' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-18T23:53:56.830 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-18T23:53:56.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:53:56.831 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-18T23:53:56.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-18T23:53:56.901 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link delete ceph-brx 2024-08-18T23:53:57.003 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-08-18T23:53:57.012 INFO:tasks.cephadm:Teardown begin 2024-08-18T23:53:57.013 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-18T23:53:57.013 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:53:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:53:57.650 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:57 smithi078 ceph-mon[18440]: pgmap v639: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-08-18T23:53:57.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:57 smithi161 ceph-mon[22027]: pgmap v639: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-08-18T23:53:57.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.729+0000 7f056f928640 1 -- 172.21.15.78:0/4255757628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05681018b0 msgr2=0x7f0568101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:57.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.729+0000 7f056f928640 1 --2- 172.21.15.78:0/4255757628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05681018b0 0x7f0568101d10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0558009930 tx=0x7f055802a420 comp rx=0 tx=0).stop 2024-08-18T23:53:57.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.729+0000 7f056f928640 1 -- 172.21.15.78:0/4255757628 shutdown_connections 2024-08-18T23:53:57.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.729+0000 7f056f928640 1 --2- 172.21.15.78:0/4255757628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0568102250 0x7f056810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:57.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.729+0000 7f056f928640 1 --2- 172.21.15.78:0/4255757628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05681018b0 0x7f0568101d10 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:57.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.729+0000 7f056f928640 1 --2- 172.21.15.78:0/4255757628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05681078b0 0x7f0568107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:57.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.729+0000 7f056f928640 1 -- 172.21.15.78:0/4255757628 >> 172.21.15.78:0/4255757628 conn(0x7f05680fd5f0 msgr2=0x7f05680ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:57.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.729+0000 7f056f928640 1 -- 172.21.15.78:0/4255757628 shutdown_connections 2024-08-18T23:53:57.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.729+0000 7f056f928640 1 -- 172.21.15.78:0/4255757628 wait complete. 2024-08-18T23:53:57.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.730+0000 7f056f928640 1 Processor -- start 2024-08-18T23:53:57.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.730+0000 7f056f928640 1 -- start start 2024-08-18T23:53:57.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.730+0000 7f056f928640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05681018b0 0x7f05681981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:57.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.730+0000 7f056f928640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0568102250 0x7f0568198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:57.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.730+0000 7f056f928640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05681078b0 0x7f056819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:57.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.730+0000 7f056f928640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0568111cb0 con 0x7f05681078b0 2024-08-18T23:53:57.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.730+0000 7f056f928640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0568111b30 con 0x7f05681018b0 2024-08-18T23:53:57.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.730+0000 7f056f928640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0568111e30 con 0x7f0568102250 2024-08-18T23:53:57.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.730+0000 7f056dec5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05681078b0 0x7f056819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:57.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.730+0000 7f056dec5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05681078b0 0x7f056819cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47906/0 (socket says 172.21.15.78:47906) 2024-08-18T23:53:57.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.730+0000 7f056dec5640 1 -- 172.21.15.78:0/906440814 learned_addr learned my addr 172.21.15.78:0/906440814 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:57.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.730+0000 7f056d6c4640 1 --2- 172.21.15.78:0/906440814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05681018b0 0x7f05681981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:57.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.731+0000 7f056cec3640 1 --2- 172.21.15.78:0/906440814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0568102250 0x7f0568198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:57.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.731+0000 7f056dec5640 1 -- 172.21.15.78:0/906440814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0568102250 msgr2=0x7f0568198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:57.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.731+0000 7f056dec5640 1 --2- 172.21.15.78:0/906440814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0568102250 0x7f0568198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:57.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.731+0000 7f056dec5640 1 -- 172.21.15.78:0/906440814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05681018b0 msgr2=0x7f05681981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:57.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.731+0000 7f056dec5640 1 --2- 172.21.15.78:0/906440814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05681018b0 0x7f05681981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:57.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.731+0000 7f056dec5640 1 -- 172.21.15.78:0/906440814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056819d1a0 con 0x7f05681078b0 2024-08-18T23:53:57.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.731+0000 7f056dec5640 1 --2- 172.21.15.78:0/906440814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05681078b0 0x7f056819cac0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f056000eaa0 tx=0x7f056000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:57.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.731+0000 7f055e7fc640 1 -- 172.21.15.78:0/906440814 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056000cdb0 con 0x7f05681078b0 2024-08-18T23:53:57.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.731+0000 7f056f928640 1 -- 172.21.15.78:0/906440814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056819d490 con 0x7f05681078b0 2024-08-18T23:53:57.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.731+0000 7f055e7fc640 1 -- 172.21.15.78:0/906440814 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0560005020 con 0x7f05681078b0 2024-08-18T23:53:57.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.731+0000 7f055e7fc640 1 -- 172.21.15.78:0/906440814 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05600116d0 con 0x7f05681078b0 2024-08-18T23:53:57.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.731+0000 7f056f928640 1 -- 172.21.15.78:0/906440814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05681a4d70 con 0x7f05681078b0 2024-08-18T23:53:57.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.732+0000 7f056f928640 1 -- 172.21.15.78:0/906440814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0568101d10 con 0x7f05681078b0 2024-08-18T23:53:57.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.733+0000 7f055e7fc640 1 -- 172.21.15.78:0/906440814 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f05600054d0 con 0x7f05681078b0 2024-08-18T23:53:57.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.734+0000 7f055e7fc640 1 --2- 172.21.15.78:0/906440814 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f05540a2b80 0x7f05540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:57.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.734+0000 7f056d6c4640 1 --2- 172.21.15.78:0/906440814 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f05540a2b80 0x7f05540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:57.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.734+0000 7f055e7fc640 1 -- 172.21.15.78:0/906440814 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f05600cfa20 con 0x7f05681078b0 2024-08-18T23:53:57.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.734+0000 7f056d6c4640 1 --2- 172.21.15.78:0/906440814 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f05540a2b80 0x7f05540a5040 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f056400bb60 tx=0x7f056400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:57.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.737+0000 7f055e7fc640 1 -- 172.21.15.78:0/906440814 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0560016030 con 0x7f05681078b0 2024-08-18T23:53:57.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:57 smithi123 ceph-mon[21808]: pgmap v639: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-08-18T23:53:57.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.937+0000 7f056f928640 1 -- 172.21.15.78:0/906440814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f056802d5e0 con 0x7f05681078b0 2024-08-18T23:53:57.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.938+0000 7f055e7fc640 1 -- 172.21.15.78:0/906440814 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11932 (secure 0 0 0) 0x7f056009c590 con 0x7f05681078b0 2024-08-18T23:53:57.940 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:53:57.940 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":32,"btime":"2024-08-18T23:52:48:372666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:52:47.960244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:53:57.941 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 32 2024-08-18T23:53:57.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.940+0000 7f056f928640 1 -- 172.21.15.78:0/906440814 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f05540a2b80 msgr2=0x7f05540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:57.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.940+0000 7f056f928640 1 --2- 172.21.15.78:0/906440814 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f05540a2b80 0x7f05540a5040 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f056400bb60 tx=0x7f056400b390 comp rx=0 tx=0).stop 2024-08-18T23:53:57.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.940+0000 7f056f928640 1 -- 172.21.15.78:0/906440814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05681078b0 msgr2=0x7f056819cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:57.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.940+0000 7f056f928640 1 --2- 172.21.15.78:0/906440814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05681078b0 0x7f056819cac0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f056000eaa0 tx=0x7f056000ef60 comp rx=0 tx=0).stop 2024-08-18T23:53:57.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.940+0000 7f056f928640 1 -- 172.21.15.78:0/906440814 shutdown_connections 2024-08-18T23:53:57.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.940+0000 7f056f928640 1 --2- 172.21.15.78:0/906440814 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f05540a2b80 0x7f05540a5040 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:57.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.940+0000 7f056f928640 1 --2- 172.21.15.78:0/906440814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05681078b0 0x7f056819cac0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:57.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.940+0000 7f056f928640 1 --2- 172.21.15.78:0/906440814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0568102250 0x7f0568198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:57.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.940+0000 7f056f928640 1 --2- 172.21.15.78:0/906440814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05681018b0 0x7f05681981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:57.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.940+0000 7f056f928640 1 -- 172.21.15.78:0/906440814 >> 172.21.15.78:0/906440814 conn(0x7f05680fd5f0 msgr2=0x7f05680fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:57.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.940+0000 7f056f928640 1 -- 172.21.15.78:0/906440814 shutdown_connections 2024-08-18T23:53:57.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:57.940+0000 7f056f928640 1 -- 172.21.15.78:0/906440814 wait complete. 2024-08-18T23:53:58.584 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-18T23:53:58.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:58 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/906440814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:53:58.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:58 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/906440814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:53:58.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:58 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/906440814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:53:58.836 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:53:59.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.343+0000 7f843673c640 1 -- 172.21.15.78:0/1343422475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8430102a70 msgr2=0x7f8430102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:59.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.343+0000 7f843673c640 1 --2- 172.21.15.78:0/1343422475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8430102a70 0x7f8430102e50 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f8420009900 tx=0x7f842002a480 comp rx=0 tx=0).stop 2024-08-18T23:53:59.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.343+0000 7f843673c640 1 -- 172.21.15.78:0/1343422475 shutdown_connections 2024-08-18T23:53:59.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.343+0000 7f843673c640 1 --2- 172.21.15.78:0/1343422475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f843010bea0 0x7f843010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:59.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.343+0000 7f843673c640 1 --2- 172.21.15.78:0/1343422475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8430103390 0x7f843010b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:59.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.343+0000 7f843673c640 1 --2- 172.21.15.78:0/1343422475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8430102a70 0x7f8430102e50 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:59.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.343+0000 7f843673c640 1 -- 172.21.15.78:0/1343422475 >> 172.21.15.78:0/1343422475 conn(0x7f84300fc4d0 msgr2=0x7f84300fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:59.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.343+0000 7f843673c640 1 -- 172.21.15.78:0/1343422475 shutdown_connections 2024-08-18T23:53:59.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.343+0000 7f843673c640 1 -- 172.21.15.78:0/1343422475 wait complete. 2024-08-18T23:53:59.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.343+0000 7f843673c640 1 Processor -- start 2024-08-18T23:53:59.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.344+0000 7f843673c640 1 -- start start 2024-08-18T23:53:59.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.344+0000 7f843673c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8430102a70 0x7f84301981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:59.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.344+0000 7f843673c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8430103390 0x7f84301986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:59.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.344+0000 7f843673c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f843010bea0 0x7f843019ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:59.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.344+0000 7f843673c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8430111ca0 con 0x7f843010bea0 2024-08-18T23:53:59.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.344+0000 7f843673c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8430111b20 con 0x7f8430103390 2024-08-18T23:53:59.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.344+0000 7f843673c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8430111e20 con 0x7f8430102a70 2024-08-18T23:53:59.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.344+0000 7f8434cd9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f843010bea0 0x7f843019ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:59.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.344+0000 7f8434cd9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f843010bea0 0x7f843019ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47924/0 (socket says 172.21.15.78:47924) 2024-08-18T23:53:59.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.344+0000 7f8434cd9640 1 -- 172.21.15.78:0/3864628767 learned_addr learned my addr 172.21.15.78:0/3864628767 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:53:59.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.344+0000 7f842f7fe640 1 --2- 172.21.15.78:0/3864628767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8430103390 0x7f84301986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:59.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.344+0000 7f842ffff640 1 --2- 172.21.15.78:0/3864628767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8430102a70 0x7f84301981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:59.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.344+0000 7f8434cd9640 1 -- 172.21.15.78:0/3864628767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8430102a70 msgr2=0x7f84301981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:59.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.344+0000 7f8434cd9640 1 --2- 172.21.15.78:0/3864628767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8430102a70 0x7f84301981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:59.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.344+0000 7f8434cd9640 1 -- 172.21.15.78:0/3864628767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8430103390 msgr2=0x7f84301986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:59.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.344+0000 7f8434cd9640 1 --2- 172.21.15.78:0/3864628767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8430103390 0x7f84301986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:59.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.344+0000 7f8434cd9640 1 -- 172.21.15.78:0/3864628767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843019d150 con 0x7f843010bea0 2024-08-18T23:53:59.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.345+0000 7f8434cd9640 1 --2- 172.21.15.78:0/3864628767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f843010bea0 0x7f843019ca70 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f842800eaa0 tx=0x7f842800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:59.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.345+0000 7f842d7fa640 1 -- 172.21.15.78:0/3864628767 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842800cdb0 con 0x7f843010bea0 2024-08-18T23:53:59.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.345+0000 7f842d7fa640 1 -- 172.21.15.78:0/3864628767 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8428005020 con 0x7f843010bea0 2024-08-18T23:53:59.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.345+0000 7f843673c640 1 -- 172.21.15.78:0/3864628767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843019d440 con 0x7f843010bea0 2024-08-18T23:53:59.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.345+0000 7f842d7fa640 1 -- 172.21.15.78:0/3864628767 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84280116b0 con 0x7f843010bea0 2024-08-18T23:53:59.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.345+0000 7f843673c640 1 -- 172.21.15.78:0/3864628767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84301a4d20 con 0x7f843010bea0 2024-08-18T23:53:59.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.346+0000 7f843673c640 1 -- 172.21.15.78:0/3864628767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8430102e50 con 0x7f843010bea0 2024-08-18T23:53:59.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.347+0000 7f842d7fa640 1 -- 172.21.15.78:0/3864628767 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8428005560 con 0x7f843010bea0 2024-08-18T23:53:59.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.347+0000 7f842d7fa640 1 --2- 172.21.15.78:0/3864628767 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f84180a2b80 0x7f84180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:53:59.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.348+0000 7f842ffff640 1 --2- 172.21.15.78:0/3864628767 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f84180a2b80 0x7f84180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:53:59.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.348+0000 7f842d7fa640 1 -- 172.21.15.78:0/3864628767 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f84280cf510 con 0x7f843010bea0 2024-08-18T23:53:59.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.348+0000 7f842ffff640 1 --2- 172.21.15.78:0/3864628767 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f84180a2b80 0x7f84180a5040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8420009230 tx=0x7f84200058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:53:59.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.352+0000 7f842d7fa640 1 -- 172.21.15.78:0/3864628767 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8428016030 con 0x7f843010bea0 2024-08-18T23:53:59.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.555+0000 7f843673c640 1 -- 172.21.15.78:0/3864628767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f843002d5e0 con 0x7f843010bea0 2024-08-18T23:53:59.558 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:53:59 smithi078 ceph-mon[18440]: pgmap v640: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-18T23:53:59.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.556+0000 7f842d7fa640 1 -- 172.21.15.78:0/3864628767 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11932 (secure 0 0 0) 0x7f842809ba90 con 0x7f843010bea0 2024-08-18T23:53:59.559 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:53:59.559 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":32,"btime":"2024-08-18T23:52:48:372666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3927701910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3927701910},{"type":"v1","addr":"172.21.15.78:6835","nonce":3927701910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/1656541536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":1656541536},{"type":"v1","addr":"172.21.15.78:6837","nonce":1656541536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/2648770058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":2648770058},{"type":"v1","addr":"172.21.15.78:6839","nonce":2648770058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/2010800750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":2010800750},{"type":"v1","addr":"172.21.15.78:6841","nonce":2010800750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24370,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3196205121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3196205121},{"type":"v1","addr":"172.21.15.161:6833","nonce":3196205121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3595058071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3595058071},{"type":"v1","addr":"172.21.15.123:6833","nonce":3595058071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-18T23:41:32.941573+0000","modified":"2024-08-18T23:52:47.960244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24428,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24397":{"gid":24397,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/4100464644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":4100464644},{"type":"v1","addr":"172.21.15.161:6835","nonce":4100464644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/2017185021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2017185021},{"type":"v1","addr":"172.21.15.161:6837","nonce":2017185021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2459199316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2459199316},{"type":"v1","addr":"172.21.15.123:6835","nonce":2459199316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.161:6839/2209088174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2209088174},{"type":"v1","addr":"172.21.15.161:6839","nonce":2209088174}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.123:6837/1639958554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1639958554},{"type":"v1","addr":"172.21.15.123:6837","nonce":1639958554}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/2656330956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2656330956},{"type":"v1","addr":"172.21.15.123:6839","nonce":2656330956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24428,24452]},"id":1}]} 2024-08-18T23:53:59.559 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 32 2024-08-18T23:53:59.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.559+0000 7f843673c640 1 -- 172.21.15.78:0/3864628767 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f84180a2b80 msgr2=0x7f84180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:59.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.559+0000 7f843673c640 1 --2- 172.21.15.78:0/3864628767 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f84180a2b80 0x7f84180a5040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8420009230 tx=0x7f84200058d0 comp rx=0 tx=0).stop 2024-08-18T23:53:59.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.559+0000 7f843673c640 1 -- 172.21.15.78:0/3864628767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f843010bea0 msgr2=0x7f843019ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:53:59.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.559+0000 7f843673c640 1 --2- 172.21.15.78:0/3864628767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f843010bea0 0x7f843019ca70 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f842800eaa0 tx=0x7f842800ef60 comp rx=0 tx=0).stop 2024-08-18T23:53:59.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.559+0000 7f843673c640 1 -- 172.21.15.78:0/3864628767 shutdown_connections 2024-08-18T23:53:59.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.559+0000 7f843673c640 1 --2- 172.21.15.78:0/3864628767 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f84180a2b80 0x7f84180a5040 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:59.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.559+0000 7f843673c640 1 --2- 172.21.15.78:0/3864628767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f843010bea0 0x7f843019ca70 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:59.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.559+0000 7f843673c640 1 --2- 172.21.15.78:0/3864628767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8430103390 0x7f84301986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:59.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.559+0000 7f843673c640 1 --2- 172.21.15.78:0/3864628767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8430102a70 0x7f84301981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:53:59.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.559+0000 7f843673c640 1 -- 172.21.15.78:0/3864628767 >> 172.21.15.78:0/3864628767 conn(0x7f84300fc4d0 msgr2=0x7f84300fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:53:59.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.559+0000 7f843673c640 1 -- 172.21.15.78:0/3864628767 shutdown_connections 2024-08-18T23:53:59.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:53:59.559+0000 7f843673c640 1 -- 172.21.15.78:0/3864628767 wait complete. 2024-08-18T23:53:59.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:53:59 smithi161 ceph-mon[22027]: pgmap v640: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-18T23:53:59.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:53:59 smithi123 ceph-mon[21808]: pgmap v640: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-18T23:54:00.218 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-18T23:54:00.469 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:00 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3864628767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:54:00.470 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:54:00.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:00 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3864628767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:54:00.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:00 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3864628767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-18T23:54:00.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.966+0000 7fb597b95640 1 -- 172.21.15.78:0/2985583880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb590105c70 msgr2=0x7fb590106100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:00.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.966+0000 7fb597b95640 1 --2- 172.21.15.78:0/2985583880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb590105c70 0x7fb590106100 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb588009ca0 tx=0x7fb58802a940 comp rx=0 tx=0).stop 2024-08-18T23:54:00.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.966+0000 7fb597b95640 1 -- 172.21.15.78:0/2985583880 shutdown_connections 2024-08-18T23:54:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.966+0000 7fb597b95640 1 --2- 172.21.15.78:0/2985583880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb590105c70 0x7fb590106100 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.966+0000 7fb597b95640 1 --2- 172.21.15.78:0/2985583880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb590101180 0x7fb590105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.966+0000 7fb597b95640 1 --2- 172.21.15.78:0/2985583880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb590100860 0x7fb590100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.966+0000 7fb597b95640 1 -- 172.21.15.78:0/2985583880 >> 172.21.15.78:0/2985583880 conn(0x7fb5900fc4d0 msgr2=0x7fb5900fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:54:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.966+0000 7fb597b95640 1 -- 172.21.15.78:0/2985583880 shutdown_connections 2024-08-18T23:54:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.966+0000 7fb597b95640 1 -- 172.21.15.78:0/2985583880 wait complete. 2024-08-18T23:54:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.967+0000 7fb597b95640 1 Processor -- start 2024-08-18T23:54:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.967+0000 7fb597b95640 1 -- start start 2024-08-18T23:54:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.967+0000 7fb597b95640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb590100860 0x7fb5901a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.967+0000 7fb597b95640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb590101180 0x7fb5901a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:00.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.967+0000 7fb597b95640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb590105c70 0x7fb59019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:00.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.967+0000 7fb597b95640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb590118370 con 0x7fb590100860 2024-08-18T23:54:00.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.967+0000 7fb597b95640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901181f0 con 0x7fb590101180 2024-08-18T23:54:00.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.967+0000 7fb597b95640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901184f0 con 0x7fb590105c70 2024-08-18T23:54:00.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.967+0000 7fb595931640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb590100860 0x7fb5901a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:00.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.967+0000 7fb595931640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb590100860 0x7fb5901a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47936/0 (socket says 172.21.15.78:47936) 2024-08-18T23:54:00.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.967+0000 7fb595931640 1 -- 172.21.15.78:0/2167289694 learned_addr learned my addr 172.21.15.78:0/2167289694 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:54:00.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.967+0000 7fb595931640 1 -- 172.21.15.78:0/2167289694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb590105c70 msgr2=0x7fb59019e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:00.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.967+0000 7fb595931640 1 --2- 172.21.15.78:0/2167289694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb590105c70 0x7fb59019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:00.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.967+0000 7fb595931640 1 -- 172.21.15.78:0/2167289694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb590101180 msgr2=0x7fb5901a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:00.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.967+0000 7fb595931640 1 --2- 172.21.15.78:0/2167289694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb590101180 0x7fb5901a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:00.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.967+0000 7fb595931640 1 -- 172.21.15.78:0/2167289694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59019ef90 con 0x7fb590100860 2024-08-18T23:54:00.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.968+0000 7fb595931640 1 --2- 172.21.15.78:0/2167289694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb590100860 0x7fb5901a4600 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00b4a0 tx=0x7fb58c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:54:00.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.968+0000 7fb586ffd640 1 -- 172.21.15.78:0/2167289694 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c014070 con 0x7fb590100860 2024-08-18T23:54:00.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.968+0000 7fb597b95640 1 -- 172.21.15.78:0/2167289694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59019f280 con 0x7fb590100860 2024-08-18T23:54:00.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.968+0000 7fb586ffd640 1 -- 172.21.15.78:0/2167289694 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb58c010af0 con 0x7fb590100860 2024-08-18T23:54:00.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.968+0000 7fb586ffd640 1 -- 172.21.15.78:0/2167289694 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c017d10 con 0x7fb590100860 2024-08-18T23:54:00.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.968+0000 7fb597b95640 1 -- 172.21.15.78:0/2167289694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5901ab5f0 con 0x7fb590100860 2024-08-18T23:54:00.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.969+0000 7fb597b95640 1 -- 172.21.15.78:0/2167289694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5901056d0 con 0x7fb590100860 2024-08-18T23:54:00.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.970+0000 7fb586ffd640 1 -- 172.21.15.78:0/2167289694 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb58c017070 con 0x7fb590100860 2024-08-18T23:54:00.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.971+0000 7fb586ffd640 1 --2- 172.21.15.78:0/2167289694 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb57c0a2b80 0x7fb57c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:00.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.971+0000 7fb595130640 1 --2- 172.21.15.78:0/2167289694 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb57c0a2b80 0x7fb57c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:00.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.971+0000 7fb586ffd640 1 -- 172.21.15.78:0/2167289694 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fb58c0cebe0 con 0x7fb590100860 2024-08-18T23:54:00.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.971+0000 7fb595130640 1 --2- 172.21.15.78:0/2167289694 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb57c0a2b80 0x7fb57c0a5040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb5900ff810 tx=0x7fb580006d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:54:00.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:00.974+0000 7fb586ffd640 1 -- 172.21.15.78:0/2167289694 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb58c09b7b0 con 0x7fb590100860 2024-08-18T23:54:01.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:01.142+0000 7fb597b95640 1 -- 172.21.15.78:0/2167289694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb59019fc70 con 0x7fb590100860 2024-08-18T23:54:01.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:01.143+0000 7fb586ffd640 1 -- 172.21.15.78:0/2167289694 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24747 (secure 0 0 0) 0x7fb58c09b0b0 con 0x7fb590100860 2024-08-18T23:54:01.146 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-08-18T23:54:01.146 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":92,"fsid":"4f1128e6-5dba-11ef-bccf-c7b262605968","created":"2024-08-18T23:34:33.867246+0000","modified":"2024-08-18T23:43:24.291494+0000","last_up_change":"2024-08-18T23:40:42.200381+0000","last_in_change":"2024-08-18T23:40:22.425813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-18T23:37:22.338211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-18T23:41:28.920802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-18T23:41:31.026094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-18T23:41:37.403238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4de1c78-6ebc-49a9-b9c5-85d43535fab7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6803","nonce":3639645311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6805","nonce":3639645311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6809","nonce":3639645311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":3639645311},{"type":"v1","addr":"172.21.15.78:6807","nonce":3639645311}]},"public_addr":"172.21.15.78:6803/3639645311","cluster_addr":"172.21.15.78:6805/3639645311","heartbeat_back_addr":"172.21.15.78:6809/3639645311","heartbeat_front_addr":"172.21.15.78:6807/3639645311","state":["exists","up"]},{"osd":1,"uuid":"4f8e0c3d-472b-4230-ad23-2b0a34861251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6801","nonce":4199446368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6803","nonce":4199446368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6807","nonce":4199446368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4199446368},{"type":"v1","addr":"172.21.15.123:6805","nonce":4199446368}]},"public_addr":"172.21.15.123:6801/4199446368","cluster_addr":"172.21.15.123:6803/4199446368","heartbeat_back_addr":"172.21.15.123:6807/4199446368","heartbeat_front_addr":"172.21.15.123:6805/4199446368","state":["exists","up"]},{"osd":2,"uuid":"27ef3314-bb6f-413d-be05-e10a105dec9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6801","nonce":2582810057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6803","nonce":2582810057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6807","nonce":2582810057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2582810057},{"type":"v1","addr":"172.21.15.161:6805","nonce":2582810057}]},"public_addr":"172.21.15.161:6801/2582810057","cluster_addr":"172.21.15.161:6803/2582810057","heartbeat_back_addr":"172.21.15.161:6807/2582810057","heartbeat_front_addr":"172.21.15.161:6805/2582810057","state":["exists","up"]},{"osd":3,"uuid":"e9e52662-5ef9-4d70-bdd7-2b92165822ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6811","nonce":2697347521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6813","nonce":2697347521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6817","nonce":2697347521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2697347521},{"type":"v1","addr":"172.21.15.78:6815","nonce":2697347521}]},"public_addr":"172.21.15.78:6811/2697347521","cluster_addr":"172.21.15.78:6813/2697347521","heartbeat_back_addr":"172.21.15.78:6817/2697347521","heartbeat_front_addr":"172.21.15.78:6815/2697347521","state":["exists","up"]},{"osd":4,"uuid":"5ed26775-79e3-4b24-9557-69fcf5e8e824","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6809","nonce":1281511569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6811","nonce":1281511569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6815","nonce":1281511569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1281511569},{"type":"v1","addr":"172.21.15.123:6813","nonce":1281511569}]},"public_addr":"172.21.15.123:6809/1281511569","cluster_addr":"172.21.15.123:6811/1281511569","heartbeat_back_addr":"172.21.15.123:6815/1281511569","heartbeat_front_addr":"172.21.15.123:6813/1281511569","state":["exists","up"]},{"osd":5,"uuid":"a9793b1d-5613-48a4-938e-adb718f4fa4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6809","nonce":2751796117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6811","nonce":2751796117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6815","nonce":2751796117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2751796117},{"type":"v1","addr":"172.21.15.161:6813","nonce":2751796117}]},"public_addr":"172.21.15.161:6809/2751796117","cluster_addr":"172.21.15.161:6811/2751796117","heartbeat_back_addr":"172.21.15.161:6815/2751796117","heartbeat_front_addr":"172.21.15.161:6813/2751796117","state":["exists","up"]},{"osd":6,"uuid":"5dbc5e9c-a14f-4b71-910a-1e6d4827bb32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6819","nonce":95383223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6821","nonce":95383223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6825","nonce":95383223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":95383223},{"type":"v1","addr":"172.21.15.78:6823","nonce":95383223}]},"public_addr":"172.21.15.78:6819/95383223","cluster_addr":"172.21.15.78:6821/95383223","heartbeat_back_addr":"172.21.15.78:6825/95383223","heartbeat_front_addr":"172.21.15.78:6823/95383223","state":["exists","up"]},{"osd":7,"uuid":"26fb7b69-131f-42ee-b745-6fe8b4f65391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6817","nonce":3440305156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6819","nonce":3440305156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6823","nonce":3440305156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3440305156},{"type":"v1","addr":"172.21.15.123:6821","nonce":3440305156}]},"public_addr":"172.21.15.123:6817/3440305156","cluster_addr":"172.21.15.123:6819/3440305156","heartbeat_back_addr":"172.21.15.123:6823/3440305156","heartbeat_front_addr":"172.21.15.123:6821/3440305156","state":["exists","up"]},{"osd":8,"uuid":"e267bc50-0c29-4d7b-b368-b1621e29b2df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6817","nonce":1485008778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6819","nonce":1485008778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6823","nonce":1485008778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1485008778},{"type":"v1","addr":"172.21.15.161:6821","nonce":1485008778}]},"public_addr":"172.21.15.161:6817/1485008778","cluster_addr":"172.21.15.161:6819/1485008778","heartbeat_back_addr":"172.21.15.161:6823/1485008778","heartbeat_front_addr":"172.21.15.161:6821/1485008778","state":["exists","up"]},{"osd":9,"uuid":"b7654818-b767-4fe6-8211-346297895f5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6827","nonce":2884475661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6829","nonce":2884475661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6833","nonce":2884475661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":2884475661},{"type":"v1","addr":"172.21.15.78:6831","nonce":2884475661}]},"public_addr":"172.21.15.78:6827/2884475661","cluster_addr":"172.21.15.78:6829/2884475661","heartbeat_back_addr":"172.21.15.78:6833/2884475661","heartbeat_front_addr":"172.21.15.78:6831/2884475661","state":["exists","up"]},{"osd":10,"uuid":"59aa32ff-510e-4b52-8170-08a34a0b6b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6825","nonce":122226286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6827","nonce":122226286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6831","nonce":122226286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":122226286},{"type":"v1","addr":"172.21.15.123:6829","nonce":122226286}]},"public_addr":"172.21.15.123:6825/122226286","cluster_addr":"172.21.15.123:6827/122226286","heartbeat_back_addr":"172.21.15.123:6831/122226286","heartbeat_front_addr":"172.21.15.123:6829/122226286","state":["exists","up"]},{"osd":11,"uuid":"67bacab4-d078-4c5c-b4ff-9f6a8a00f507","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6825","nonce":2037560985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6827","nonce":2037560985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6831","nonce":2037560985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2037560985},{"type":"v1","addr":"172.21.15.161:6829","nonce":2037560985}]},"public_addr":"172.21.15.161:6825/2037560985","cluster_addr":"172.21.15.161:6827/2037560985","heartbeat_back_addr":"172.21.15.161:6831/2037560985","heartbeat_front_addr":"172.21.15.161:6829/2037560985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:36.826044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:36:57.688421+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:18.297424+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:37:40.355215+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:02.167902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:23.968759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:38:46.254083+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:08.522038+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:31.028119+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:39:54.119807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:16.847034+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-18T23:40:39.607886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6800/311758909":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3040499240":"2024-08-19T23:35:22.036055+0000","172.21.15.78:6801/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:6800/2301978296":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/757409165":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/2562072965":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/636184437":"2024-08-19T23:34:59.021165+0000","172.21.15.78:0/1570708294":"2024-08-19T23:35:22.036055+0000","172.21.15.78:0/3137806628":"2024-08-19T23:34:59.021165+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-18T23:54:01.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:01.145+0000 7fb597b95640 1 -- 172.21.15.78:0/2167289694 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb57c0a2b80 msgr2=0x7fb57c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:01.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:01.145+0000 7fb597b95640 1 --2- 172.21.15.78:0/2167289694 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb57c0a2b80 0x7fb57c0a5040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb5900ff810 tx=0x7fb580006d30 comp rx=0 tx=0).stop 2024-08-18T23:54:01.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:01.145+0000 7fb597b95640 1 -- 172.21.15.78:0/2167289694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb590100860 msgr2=0x7fb5901a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:01.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:01.145+0000 7fb597b95640 1 --2- 172.21.15.78:0/2167289694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb590100860 0x7fb5901a4600 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00b4a0 tx=0x7fb58c00b960 comp rx=0 tx=0).stop 2024-08-18T23:54:01.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:01.145+0000 7fb597b95640 1 -- 172.21.15.78:0/2167289694 shutdown_connections 2024-08-18T23:54:01.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:01.145+0000 7fb597b95640 1 --2- 172.21.15.78:0/2167289694 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7fb57c0a2b80 0x7fb57c0a5040 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:01.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:01.145+0000 7fb597b95640 1 --2- 172.21.15.78:0/2167289694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb590105c70 0x7fb59019e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:01.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:01.145+0000 7fb597b95640 1 --2- 172.21.15.78:0/2167289694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb590101180 0x7fb5901a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:01.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:01.145+0000 7fb597b95640 1 --2- 172.21.15.78:0/2167289694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb590100860 0x7fb5901a4600 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:01.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:01.145+0000 7fb597b95640 1 -- 172.21.15.78:0/2167289694 >> 172.21.15.78:0/2167289694 conn(0x7fb5900fc4d0 msgr2=0x7fb59010f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:54:01.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:01.146+0000 7fb597b95640 1 -- 172.21.15.78:0/2167289694 shutdown_connections 2024-08-18T23:54:01.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:01.146+0000 7fb597b95640 1 -- 172.21.15.78:0/2167289694 wait complete. 2024-08-18T23:54:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:01 smithi161 ceph-mon[22027]: pgmap v641: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-18T23:54:01.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:01 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/2167289694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:54:01.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:01 smithi078 ceph-mon[18440]: pgmap v641: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-18T23:54:01.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:01 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/2167289694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:54:01.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:01 smithi123 ceph-mon[21808]: pgmap v641: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-18T23:54:01.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:01 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/2167289694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-18T23:54:01.956 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs fail cephfs 2024-08-18T23:54:02.213 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:54:02.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.711+0000 7f039eb46640 1 -- 172.21.15.78:0/2581487226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03981007b0 msgr2=0x7f0398100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:02.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.711+0000 7f039eb46640 1 --2- 172.21.15.78:0/2581487226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03981007b0 0x7f0398100c10 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f0384009930 tx=0x7f038402a420 comp rx=0 tx=0).stop 2024-08-18T23:54:02.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.711+0000 7f039eb46640 1 -- 172.21.15.78:0/2581487226 shutdown_connections 2024-08-18T23:54:02.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.711+0000 7f039eb46640 1 --2- 172.21.15.78:0/2581487226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0398101150 0x7f039810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:02.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.711+0000 7f039eb46640 1 --2- 172.21.15.78:0/2581487226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03981007b0 0x7f0398100c10 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:02.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.711+0000 7f039eb46640 1 --2- 172.21.15.78:0/2581487226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03981067b0 0x7f0398106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:02.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.711+0000 7f039eb46640 1 -- 172.21.15.78:0/2581487226 >> 172.21.15.78:0/2581487226 conn(0x7f03980fc4d0 msgr2=0x7f03980fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:54:02.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.711+0000 7f039eb46640 1 -- 172.21.15.78:0/2581487226 shutdown_connections 2024-08-18T23:54:02.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.711+0000 7f039eb46640 1 -- 172.21.15.78:0/2581487226 wait complete. 2024-08-18T23:54:02.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f039eb46640 1 Processor -- start 2024-08-18T23:54:02.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f039eb46640 1 -- start start 2024-08-18T23:54:02.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f039eb46640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03981007b0 0x7f0398198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:02.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f039eb46640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0398101150 0x7f03981988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:02.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f039eb46640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03981067b0 0x7f039819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:02.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f039eb46640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0398110c50 con 0x7f0398101150 2024-08-18T23:54:02.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f039eb46640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0398110ad0 con 0x7f03981067b0 2024-08-18T23:54:02.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f039eb46640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0398110dd0 con 0x7f03981007b0 2024-08-18T23:54:02.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f0397fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0398101150 0x7f03981988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:02.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f0397fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0398101150 0x7f03981988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47964/0 (socket says 172.21.15.78:47964) 2024-08-18T23:54:02.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f0397fff640 1 -- 172.21.15.78:0/3506587210 learned_addr learned my addr 172.21.15.78:0/3506587210 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:54:02.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f039d0e3640 1 --2- 172.21.15.78:0/3506587210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03981067b0 0x7f039819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:02.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f039c8e2640 1 --2- 172.21.15.78:0/3506587210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03981007b0 0x7f0398198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:02.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f0397fff640 1 -- 172.21.15.78:0/3506587210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03981007b0 msgr2=0x7f0398198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:02.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f0397fff640 1 --2- 172.21.15.78:0/3506587210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03981007b0 0x7f0398198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:02.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f0397fff640 1 -- 172.21.15.78:0/3506587210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03981067b0 msgr2=0x7f039819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:02.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f0397fff640 1 --2- 172.21.15.78:0/3506587210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03981067b0 0x7f039819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:02.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f0397fff640 1 -- 172.21.15.78:0/3506587210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039819d310 con 0x7f0398101150 2024-08-18T23:54:02.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f039c8e2640 1 --2- 172.21.15.78:0/3506587210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03981007b0 0x7f0398198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-18T23:54:02.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.712+0000 7f0397fff640 1 --2- 172.21.15.78:0/3506587210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0398101150 0x7f03981988a0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0384009a50 tx=0x7f0384004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:54:02.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.713+0000 7f0395ffb640 1 -- 172.21.15.78:0/3506587210 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038403f030 con 0x7f0398101150 2024-08-18T23:54:02.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.713+0000 7f0395ffb640 1 -- 172.21.15.78:0/3506587210 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0384003ce0 con 0x7f0398101150 2024-08-18T23:54:02.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.713+0000 7f039eb46640 1 -- 172.21.15.78:0/3506587210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039819d5a0 con 0x7f0398101150 2024-08-18T23:54:02.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.713+0000 7f0395ffb640 1 -- 172.21.15.78:0/3506587210 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0384035070 con 0x7f0398101150 2024-08-18T23:54:02.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.713+0000 7f039eb46640 1 -- 172.21.15.78:0/3506587210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03981a4ee0 con 0x7f0398101150 2024-08-18T23:54:02.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.714+0000 7f039eb46640 1 -- 172.21.15.78:0/3506587210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0398101ef0 con 0x7f0398101150 2024-08-18T23:54:02.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.715+0000 7f0395ffb640 1 -- 172.21.15.78:0/3506587210 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0384042030 con 0x7f0398101150 2024-08-18T23:54:02.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.716+0000 7f0395ffb640 1 --2- 172.21.15.78:0/3506587210 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f03880a2b80 0x7f03880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:02.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.716+0000 7f0395ffb640 1 -- 172.21.15.78:0/3506587210 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f03840efe70 con 0x7f0398101150 2024-08-18T23:54:02.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.716+0000 7f039c8e2640 1 --2- 172.21.15.78:0/3506587210 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f03880a2b80 0x7f03880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:02.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.716+0000 7f039c8e2640 1 --2- 172.21.15.78:0/3506587210 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f03880a2b80 0x7f03880a5040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f038c006fd0 tx=0x7f038c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:54:02.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.719+0000 7f0395ffb640 1 -- 172.21.15.78:0/3506587210 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f03840bca40 con 0x7f0398101150 2024-08-18T23:54:02.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.920+0000 7f039eb46640 1 -- 172.21.15.78:0/3506587210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0398109d50 con 0x7f0398101150 2024-08-18T23:54:02.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.922+0000 7f0395ffb640 1 -- 172.21.15.78:0/3506587210 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) ==== 161+0+0 (secure 0 0 0) 0x7f03840bc340 con 0x7f0398101150 2024-08-18T23:54:02.923 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-18T23:54:02.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.925+0000 7f039eb46640 1 -- 172.21.15.78:0/3506587210 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f03880a2b80 msgr2=0x7f03880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:02.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.925+0000 7f039eb46640 1 --2- 172.21.15.78:0/3506587210 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f03880a2b80 0x7f03880a5040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f038c006fd0 tx=0x7f038c008040 comp rx=0 tx=0).stop 2024-08-18T23:54:02.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.925+0000 7f039eb46640 1 -- 172.21.15.78:0/3506587210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0398101150 msgr2=0x7f03981988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:02.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.925+0000 7f039eb46640 1 --2- 172.21.15.78:0/3506587210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0398101150 0x7f03981988a0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0384009a50 tx=0x7f0384004270 comp rx=0 tx=0).stop 2024-08-18T23:54:02.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.925+0000 7f039eb46640 1 -- 172.21.15.78:0/3506587210 shutdown_connections 2024-08-18T23:54:02.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.925+0000 7f039eb46640 1 --2- 172.21.15.78:0/3506587210 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f03880a2b80 0x7f03880a5040 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:02.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.925+0000 7f039eb46640 1 --2- 172.21.15.78:0/3506587210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03981067b0 0x7f039819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:02.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.926+0000 7f039eb46640 1 --2- 172.21.15.78:0/3506587210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0398101150 0x7f03981988a0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:02.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.926+0000 7f039eb46640 1 --2- 172.21.15.78:0/3506587210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03981007b0 0x7f0398198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:02.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.926+0000 7f039eb46640 1 -- 172.21.15.78:0/3506587210 >> 172.21.15.78:0/3506587210 conn(0x7f03980fc4d0 msgr2=0x7f039806a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:54:02.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.926+0000 7f039eb46640 1 -- 172.21.15.78:0/3506587210 shutdown_connections 2024-08-18T23:54:02.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:02.926+0000 7f039eb46640 1 -- 172.21.15.78:0/3506587210 wait complete. 2024-08-18T23:54:03.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:02 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-a[18436]: 2024-08-18T23:54:02.920+0000 7f4391dc8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-18T23:54:03.298 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:54:02 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-h[51236]: ignoring --setuser ceph since I am not root 2024-08-18T23:54:03.298 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:54:02 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-h[51236]: ignoring --setgroup ceph since I am not root 2024-08-18T23:54:03.298 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:54:02 smithi123 ceph-mds[51240]: -- 172.21.15.123:0/902578631 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558f39bf1860 con 0x558f39bcf000 2024-08-18T23:54:03.298 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:54:02 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-h[51236]: starting mds.h at 2024-08-18T23:54:03.299 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:54:02 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-b[51547]: ignoring --setuser ceph since I am not root 2024-08-18T23:54:03.299 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:54:02 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-b[51547]: ignoring --setgroup ceph since I am not root 2024-08-18T23:54:03.299 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:54:02 smithi123 ceph-mds[51551]: -- 172.21.15.123:0/4182914879 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e1a30b9860 con 0x55e1a3097000 2024-08-18T23:54:03.299 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:54:02 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-b[51547]: starting mds.b at 2024-08-18T23:54:03.299 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:54:02 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-e[50922]: ignoring --setuser ceph since I am not root 2024-08-18T23:54:03.299 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:54:02 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-e[50922]: ignoring --setgroup ceph since I am not root 2024-08-18T23:54:03.299 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:54:02 smithi123 ceph-mds[50926]: -- 172.21.15.123:0/2120053130 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561661ffb860 con 0x561661fd9000 2024-08-18T23:54:03.299 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:54:02 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-e[50922]: starting mds.e at 2024-08-18T23:54:03.417 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:54:02 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-c[51891]: ignoring --setuser ceph since I am not root 2024-08-18T23:54:03.417 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:54:02 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-c[51891]: ignoring --setgroup ceph since I am not root 2024-08-18T23:54:03.417 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:54:02 smithi161 ceph-mds[51895]: -- 172.21.15.161:0/85180305 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5620dc1ed860 con 0x5620dc1cb400 2024-08-18T23:54:03.417 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:54:02 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-c[51891]: starting mds.c at 2024-08-18T23:54:03.418 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 18 23:54:02 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-f[51590]: ignoring --setuser ceph since I am not root 2024-08-18T23:54:03.418 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 18 23:54:02 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-f[51590]: ignoring --setgroup ceph since I am not root 2024-08-18T23:54:03.418 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 18 23:54:02 smithi161 ceph-mds[51594]: -- 172.21.15.161:0/3082228584 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e254aab860 con 0x55e254a88800 2024-08-18T23:54:03.418 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 18 23:54:02 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-f[51590]: starting mds.f at 2024-08-18T23:54:03.418 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 18 23:54:02 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-i[52203]: ignoring --setuser ceph since I am not root 2024-08-18T23:54:03.418 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 18 23:54:02 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-i[52203]: ignoring --setgroup ceph since I am not root 2024-08-18T23:54:03.418 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 18 23:54:02 smithi161 ceph-mds[52207]: -- 172.21.15.161:0/3910561961 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a1f01eb860 con 0x55a1f01c8800 2024-08-18T23:54:03.418 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 18 23:54:02 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-i[52203]: starting mds.i at 2024-08-18T23:54:03.620 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:03 smithi078 ceph-mon[18440]: pgmap v642: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-08-18T23:54:03.620 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:03 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3506587210' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-18T23:54:03.620 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:03 smithi078 ceph-mon[18440]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-18T23:54:03.621 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:03 smithi078 ceph-mon[18440]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-18T23:54:03.621 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:03 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3506587210' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-18T23:54:03.621 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:03 smithi078 ceph-mon[18440]: osdmap e93: 12 total, 12 up, 12 in 2024-08-18T23:54:03.621 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:03 smithi078 ceph-mon[18440]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-18T23:54:03.650 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-08-18T23:54:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:03 smithi161 ceph-mon[22027]: pgmap v642: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-08-18T23:54:03.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:03 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3506587210' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-18T23:54:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:03 smithi161 ceph-mon[22027]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-18T23:54:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:03 smithi161 ceph-mon[22027]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-18T23:54:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:03 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3506587210' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-18T23:54:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:03 smithi161 ceph-mon[22027]: osdmap e93: 12 total, 12 up, 12 in 2024-08-18T23:54:03.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:03 smithi161 ceph-mon[22027]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-18T23:54:03.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:03 smithi123 ceph-mon[21808]: pgmap v642: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-08-18T23:54:03.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:03 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3506587210' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-18T23:54:03.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:03 smithi123 ceph-mon[21808]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-18T23:54:03.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:03 smithi123 ceph-mon[21808]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-18T23:54:03.798 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:03 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3506587210' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-18T23:54:03.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:03 smithi123 ceph-mon[21808]: osdmap e93: 12 total, 12 up, 12 in 2024-08-18T23:54:03.799 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:03 smithi123 ceph-mon[21808]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-18T23:54:03.902 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:54:04.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.402+0000 7ff9e5c1d640 1 -- 172.21.15.78:0/4216283195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9e0102250 msgr2=0x7ff9e010e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:04.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.402+0000 7ff9e5c1d640 1 --2- 172.21.15.78:0/4216283195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9e0102250 0x7ff9e010e780 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009930 tx=0x7ff9d802a420 comp rx=0 tx=0).stop 2024-08-18T23:54:04.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.402+0000 7ff9e5c1d640 1 -- 172.21.15.78:0/4216283195 shutdown_connections 2024-08-18T23:54:04.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.402+0000 7ff9e5c1d640 1 --2- 172.21.15.78:0/4216283195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9e0102250 0x7ff9e010e780 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:04.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.402+0000 7ff9e5c1d640 1 --2- 172.21.15.78:0/4216283195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9e01018b0 0x7ff9e0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:04.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.402+0000 7ff9e5c1d640 1 --2- 172.21.15.78:0/4216283195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9e01078b0 0x7ff9e0107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:04.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.402+0000 7ff9e5c1d640 1 -- 172.21.15.78:0/4216283195 >> 172.21.15.78:0/4216283195 conn(0x7ff9e00fd5f0 msgr2=0x7ff9e00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:54:04.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.402+0000 7ff9e5c1d640 1 -- 172.21.15.78:0/4216283195 shutdown_connections 2024-08-18T23:54:04.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.402+0000 7ff9e5c1d640 1 -- 172.21.15.78:0/4216283195 wait complete. 2024-08-18T23:54:04.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.403+0000 7ff9e5c1d640 1 Processor -- start 2024-08-18T23:54:04.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.403+0000 7ff9e5c1d640 1 -- start start 2024-08-18T23:54:04.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.403+0000 7ff9e5c1d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9e01018b0 0x7ff9e01980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:04.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.403+0000 7ff9e5c1d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9e0102250 0x7ff9e01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:04.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.403+0000 7ff9e5c1d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9e01078b0 0x7ff9e019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:04.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.403+0000 7ff9e5c1d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0111cb0 con 0x7ff9e0102250 2024-08-18T23:54:04.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.403+0000 7ff9e5c1d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0111b30 con 0x7ff9e01078b0 2024-08-18T23:54:04.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.403+0000 7ff9e5c1d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0111e30 con 0x7ff9e01018b0 2024-08-18T23:54:04.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.403+0000 7ff9deffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9e0102250 0x7ff9e01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:04.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.403+0000 7ff9deffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9e0102250 0x7ff9e01985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47976/0 (socket says 172.21.15.78:47976) 2024-08-18T23:54:04.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.403+0000 7ff9deffd640 1 -- 172.21.15.78:0/3900765383 learned_addr learned my addr 172.21.15.78:0/3900765383 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:54:04.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.403+0000 7ff9df7fe640 1 --2- 172.21.15.78:0/3900765383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9e01018b0 0x7ff9e01980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:04.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.403+0000 7ff9dffff640 1 --2- 172.21.15.78:0/3900765383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9e01078b0 0x7ff9e019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:04.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.403+0000 7ff9deffd640 1 -- 172.21.15.78:0/3900765383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9e01018b0 msgr2=0x7ff9e01980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:04.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.403+0000 7ff9deffd640 1 --2- 172.21.15.78:0/3900765383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9e01018b0 0x7ff9e01980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:04.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.403+0000 7ff9deffd640 1 -- 172.21.15.78:0/3900765383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9e01078b0 msgr2=0x7ff9e019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:04.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.403+0000 7ff9deffd640 1 --2- 172.21.15.78:0/3900765383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9e01078b0 0x7ff9e019c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:04.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.403+0000 7ff9deffd640 1 -- 172.21.15.78:0/3900765383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e019d060 con 0x7ff9e0102250 2024-08-18T23:54:04.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.404+0000 7ff9deffd640 1 --2- 172.21.15.78:0/3900765383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9e0102250 0x7ff9e01985f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400c970 tx=0x7ff9d400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:54:04.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.404+0000 7ff9dcff9640 1 -- 172.21.15.78:0/3900765383 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4007bb0 con 0x7ff9e0102250 2024-08-18T23:54:04.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.404+0000 7ff9dcff9640 1 -- 172.21.15.78:0/3900765383 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d4007d50 con 0x7ff9e0102250 2024-08-18T23:54:04.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.404+0000 7ff9e5c1d640 1 -- 172.21.15.78:0/3900765383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e019d350 con 0x7ff9e0102250 2024-08-18T23:54:04.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.404+0000 7ff9dcff9640 1 -- 172.21.15.78:0/3900765383 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d400f450 con 0x7ff9e0102250 2024-08-18T23:54:04.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.404+0000 7ff9e5c1d640 1 -- 172.21.15.78:0/3900765383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e01a4c30 con 0x7ff9e0102250 2024-08-18T23:54:04.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.405+0000 7ff9e5c1d640 1 -- 172.21.15.78:0/3900765383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e0101d10 con 0x7ff9e0102250 2024-08-18T23:54:04.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.406+0000 7ff9dcff9640 1 -- 172.21.15.78:0/3900765383 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff9d4005020 con 0x7ff9e0102250 2024-08-18T23:54:04.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.407+0000 7ff9dcff9640 1 --2- 172.21.15.78:0/3900765383 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff9c80a2b80 0x7ff9c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:04.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.407+0000 7ff9df7fe640 1 --2- 172.21.15.78:0/3900765383 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff9c80a2b80 0x7ff9c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:04.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.407+0000 7ff9dcff9640 1 -- 172.21.15.78:0/3900765383 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7ff9d40cf360 con 0x7ff9e0102250 2024-08-18T23:54:04.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.407+0000 7ff9df7fe640 1 --2- 172.21.15.78:0/3900765383 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff9c80a2b80 0x7ff9c80a5040 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0007d50 tx=0x7ff9d0007730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:54:04.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.411+0000 7ff9dcff9640 1 -- 172.21.15.78:0/3900765383 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff9d4014030 con 0x7ff9e0102250 2024-08-18T23:54:04.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.610+0000 7ff9e5c1d640 1 -- 172.21.15.78:0/3900765383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff9e01a4f60 con 0x7ff9e0102250 2024-08-18T23:54:04.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.929+0000 7ff9dcff9640 1 -- 172.21.15.78:0/3900765383 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) ==== 104+0+0 (secure 0 0 0) 0x7ff9d409be30 con 0x7ff9e0102250 2024-08-18T23:54:04.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.931+0000 7ff9e5c1d640 1 -- 172.21.15.78:0/3900765383 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff9c80a2b80 msgr2=0x7ff9c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:04.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.931+0000 7ff9e5c1d640 1 --2- 172.21.15.78:0/3900765383 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff9c80a2b80 0x7ff9c80a5040 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0007d50 tx=0x7ff9d0007730 comp rx=0 tx=0).stop 2024-08-18T23:54:04.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.931+0000 7ff9e5c1d640 1 -- 172.21.15.78:0/3900765383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9e0102250 msgr2=0x7ff9e01985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:04.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.931+0000 7ff9e5c1d640 1 --2- 172.21.15.78:0/3900765383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9e0102250 0x7ff9e01985f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400c970 tx=0x7ff9d400ce30 comp rx=0 tx=0).stop 2024-08-18T23:54:04.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.932+0000 7ff9e5c1d640 1 -- 172.21.15.78:0/3900765383 shutdown_connections 2024-08-18T23:54:04.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.932+0000 7ff9e5c1d640 1 --2- 172.21.15.78:0/3900765383 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7ff9c80a2b80 0x7ff9c80a5040 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:04.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.932+0000 7ff9e5c1d640 1 --2- 172.21.15.78:0/3900765383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9e01078b0 0x7ff9e019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:04.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.932+0000 7ff9e5c1d640 1 --2- 172.21.15.78:0/3900765383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9e0102250 0x7ff9e01985f0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:04.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.932+0000 7ff9e5c1d640 1 --2- 172.21.15.78:0/3900765383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9e01018b0 0x7ff9e01980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:04.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.932+0000 7ff9e5c1d640 1 -- 172.21.15.78:0/3900765383 >> 172.21.15.78:0/3900765383 conn(0x7ff9e00fd5f0 msgr2=0x7ff9e00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:54:04.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.932+0000 7ff9e5c1d640 1 -- 172.21.15.78:0/3900765383 shutdown_connections 2024-08-18T23:54:04.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:04.932+0000 7ff9e5c1d640 1 -- 172.21.15.78:0/3900765383 wait complete. 2024-08-18T23:54:05.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:04 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.161:6834/511595023,v1:172.21.15.161:6835/511595023] up:boot 2024-08-18T23:54:05.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:04 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.161:6836/3098453375,v1:172.21.15.161:6837/3098453375] up:boot 2024-08-18T23:54:05.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:04 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.161:6838/8049423,v1:172.21.15.161:6839/8049423] up:boot 2024-08-18T23:54:05.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:04 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.123:6834/2086169694,v1:172.21.15.123:6837/2086169694] up:boot 2024-08-18T23:54:05.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:04 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.123:6835/2234497616,v1:172.21.15.123:6838/2234497616] up:boot 2024-08-18T23:54:05.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:04 smithi078 ceph-mon[18440]: mds.? [v2:172.21.15.123:6836/1180117175,v1:172.21.15.123:6839/1180117175] up:boot 2024-08-18T23:54:05.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:04 smithi078 ceph-mon[18440]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-18T23:54:05.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-18T23:54:05.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-18T23:54:05.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-18T23:54:05.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-18T23:54:05.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-18T23:54:05.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:04 smithi078 ceph-mon[18440]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-18T23:54:05.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:04 smithi078 ceph-mon[18440]: pgmap v644: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 921 B/s wr, 7 op/s 2024-08-18T23:54:05.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:04 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3900765383' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-18T23:54:05.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:04 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.161:6834/511595023,v1:172.21.15.161:6835/511595023] up:boot 2024-08-18T23:54:05.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:04 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.161:6836/3098453375,v1:172.21.15.161:6837/3098453375] up:boot 2024-08-18T23:54:05.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:04 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.161:6838/8049423,v1:172.21.15.161:6839/8049423] up:boot 2024-08-18T23:54:05.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:04 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.123:6834/2086169694,v1:172.21.15.123:6837/2086169694] up:boot 2024-08-18T23:54:05.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:04 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.123:6835/2234497616,v1:172.21.15.123:6838/2234497616] up:boot 2024-08-18T23:54:05.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:04 smithi123 ceph-mon[21808]: mds.? [v2:172.21.15.123:6836/1180117175,v1:172.21.15.123:6839/1180117175] up:boot 2024-08-18T23:54:05.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:04 smithi123 ceph-mon[21808]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-18T23:54:05.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-18T23:54:05.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-18T23:54:05.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-18T23:54:05.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-18T23:54:05.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-18T23:54:05.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:04 smithi123 ceph-mon[21808]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-18T23:54:05.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:04 smithi123 ceph-mon[21808]: pgmap v644: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 921 B/s wr, 7 op/s 2024-08-18T23:54:05.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:04 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3900765383' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-18T23:54:05.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:04 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.161:6834/511595023,v1:172.21.15.161:6835/511595023] up:boot 2024-08-18T23:54:05.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:04 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.161:6836/3098453375,v1:172.21.15.161:6837/3098453375] up:boot 2024-08-18T23:54:05.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:04 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.161:6838/8049423,v1:172.21.15.161:6839/8049423] up:boot 2024-08-18T23:54:05.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:04 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.123:6834/2086169694,v1:172.21.15.123:6837/2086169694] up:boot 2024-08-18T23:54:05.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:04 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.123:6835/2234497616,v1:172.21.15.123:6838/2234497616] up:boot 2024-08-18T23:54:05.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:04 smithi161 ceph-mon[22027]: mds.? [v2:172.21.15.123:6836/1180117175,v1:172.21.15.123:6839/1180117175] up:boot 2024-08-18T23:54:05.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:04 smithi161 ceph-mon[22027]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-18T23:54:05.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-18T23:54:05.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-18T23:54:05.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-18T23:54:05.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-18T23:54:05.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-18T23:54:05.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:04 smithi161 ceph-mon[22027]: from='mgr.14150 172.21.15.78:0/1620627201' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-18T23:54:05.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:04 smithi161 ceph-mon[22027]: pgmap v644: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 921 B/s wr, 7 op/s 2024-08-18T23:54:05.430 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:04 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3900765383' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-18T23:54:05.607 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-18T23:54:05.858 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:54:06.188 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:05 smithi078 ceph-mon[18440]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-18T23:54:06.188 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:05 smithi078 ceph-mon[18440]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-18T23:54:06.188 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:05 smithi078 ceph-mon[18440]: Cluster is now healthy 2024-08-18T23:54:06.189 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:05 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3900765383' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-18T23:54:06.189 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:05 smithi078 ceph-mon[18440]: fsmap 12 up:standby 2024-08-18T23:54:06.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:05 smithi123 ceph-mon[21808]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-18T23:54:06.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:05 smithi123 ceph-mon[21808]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-18T23:54:06.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:05 smithi123 ceph-mon[21808]: Cluster is now healthy 2024-08-18T23:54:06.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:05 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3900765383' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-18T23:54:06.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:05 smithi123 ceph-mon[21808]: fsmap 12 up:standby 2024-08-18T23:54:06.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.371+0000 7f46d8dfe640 1 -- 172.21.15.78:0/1254724213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46d4102260 msgr2=0x7f46d410e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:06.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.371+0000 7f46d8dfe640 1 --2- 172.21.15.78:0/1254724213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46d4102260 0x7f46d410e790 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009cd0 tx=0x7f46cc02a8e0 comp rx=0 tx=0).stop 2024-08-18T23:54:06.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.371+0000 7f46d8dfe640 1 -- 172.21.15.78:0/1254724213 shutdown_connections 2024-08-18T23:54:06.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.371+0000 7f46d8dfe640 1 --2- 172.21.15.78:0/1254724213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46d4102260 0x7f46d410e790 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:06.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.371+0000 7f46d8dfe640 1 --2- 172.21.15.78:0/1254724213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46d41018c0 0x7f46d4101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:06.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.371+0000 7f46d8dfe640 1 --2- 172.21.15.78:0/1254724213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46d41078c0 0x7f46d4107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:06.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.371+0000 7f46d8dfe640 1 -- 172.21.15.78:0/1254724213 >> 172.21.15.78:0/1254724213 conn(0x7f46d40fd5e0 msgr2=0x7f46d40ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:54:06.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.372+0000 7f46d8dfe640 1 -- 172.21.15.78:0/1254724213 shutdown_connections 2024-08-18T23:54:06.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.372+0000 7f46d8dfe640 1 -- 172.21.15.78:0/1254724213 wait complete. 2024-08-18T23:54:06.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.372+0000 7f46d8dfe640 1 Processor -- start 2024-08-18T23:54:06.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.372+0000 7f46d8dfe640 1 -- start start 2024-08-18T23:54:06.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.372+0000 7f46d8dfe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46d41018c0 0x7f46d4074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:06.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.372+0000 7f46d8dfe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46d4102260 0x7f46d4074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:06.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.372+0000 7f46d8dfe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46d41078c0 0x7f46d4074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:06.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.372+0000 7f46d8dfe640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f46d4111cc0 con 0x7f46d41018c0 2024-08-18T23:54:06.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.372+0000 7f46d8dfe640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46d4111b40 con 0x7f46d41078c0 2024-08-18T23:54:06.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.372+0000 7f46d8dfe640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f46d4111e40 con 0x7f46d4102260 2024-08-18T23:54:06.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.373+0000 7f46d259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46d41018c0 0x7f46d4074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:06.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.373+0000 7f46d259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46d41018c0 0x7f46d4074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44030/0 (socket says 172.21.15.78:44030) 2024-08-18T23:54:06.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.373+0000 7f46d259c640 1 -- 172.21.15.78:0/1971639000 learned_addr learned my addr 172.21.15.78:0/1971639000 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:54:06.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.373+0000 7f46d2d9d640 1 --2- 172.21.15.78:0/1971639000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46d41078c0 0x7f46d4074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:06.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.373+0000 7f46d1d9b640 1 --2- 172.21.15.78:0/1971639000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46d4102260 0x7f46d4074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:06.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.373+0000 7f46d259c640 1 -- 172.21.15.78:0/1971639000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46d4102260 msgr2=0x7f46d4074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:06.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.373+0000 7f46d259c640 1 --2- 172.21.15.78:0/1971639000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46d4102260 0x7f46d4074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:06.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.373+0000 7f46d259c640 1 -- 172.21.15.78:0/1971639000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46d41078c0 msgr2=0x7f46d4074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:06.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.373+0000 7f46d259c640 1 --2- 172.21.15.78:0/1971639000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46d41078c0 0x7f46d4074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:06.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.373+0000 7f46d259c640 1 -- 172.21.15.78:0/1971639000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d41a65c0 con 0x7f46d41018c0 2024-08-18T23:54:06.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.373+0000 7f46d259c640 1 --2- 172.21.15.78:0/1971639000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46d41018c0 0x7f46d4074220 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f46c400d950 tx=0x7f46c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:54:06.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.373+0000 7f46c37fe640 1 -- 172.21.15.78:0/1971639000 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c4017070 con 0x7f46d41018c0 2024-08-18T23:54:06.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.373+0000 7f46c37fe640 1 -- 172.21.15.78:0/1971639000 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46c4009c30 con 0x7f46d41018c0 2024-08-18T23:54:06.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.373+0000 7f46d8dfe640 1 -- 172.21.15.78:0/1971639000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d41a68b0 con 0x7f46d41018c0 2024-08-18T23:54:06.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.373+0000 7f46c37fe640 1 -- 172.21.15.78:0/1971639000 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c400f6b0 con 0x7f46d41018c0 2024-08-18T23:54:06.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.373+0000 7f46d8dfe640 1 -- 172.21.15.78:0/1971639000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d41a6e40 con 0x7f46d41018c0 2024-08-18T23:54:06.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.374+0000 7f46d8dfe640 1 -- 172.21.15.78:0/1971639000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d4101d20 con 0x7f46d41018c0 2024-08-18T23:54:06.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.376+0000 7f46c37fe640 1 -- 172.21.15.78:0/1971639000 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f46c40109d0 con 0x7f46d41018c0 2024-08-18T23:54:06.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.376+0000 7f46c37fe640 1 --2- 172.21.15.78:0/1971639000 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f46bc0a2c00 0x7f46bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:06.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.376+0000 7f46d1d9b640 1 --2- 172.21.15.78:0/1971639000 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f46bc0a2c00 0x7f46bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:06.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.376+0000 7f46c37fe640 1 -- 172.21.15.78:0/1971639000 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f46c40cef50 con 0x7f46d41018c0 2024-08-18T23:54:06.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.377+0000 7f46d1d9b640 1 --2- 172.21.15.78:0/1971639000 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f46bc0a2c00 0x7f46bc0a50c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f46d4075cb0 tx=0x7f46c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:54:06.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.379+0000 7f46c37fe640 1 -- 172.21.15.78:0/1971639000 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f46c4009de0 con 0x7f46d41018c0 2024-08-18T23:54:06.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:05 smithi161 ceph-mon[22027]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-18T23:54:06.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:05 smithi161 ceph-mon[22027]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-18T23:54:06.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:05 smithi161 ceph-mon[22027]: Cluster is now healthy 2024-08-18T23:54:06.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:05 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3900765383' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-18T23:54:06.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:05 smithi161 ceph-mon[22027]: fsmap 12 up:standby 2024-08-18T23:54:06.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.548+0000 7f46d8dfe640 1 -- 172.21.15.78:0/1971639000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f46d4075750 con 0x7f46d41018c0 2024-08-18T23:54:06.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.936+0000 7f46c37fe640 1 -- 172.21.15.78:0/1971639000 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) ==== 181+0+0 (secure 0 0 0) 0x7f46c409b410 con 0x7f46d41018c0 2024-08-18T23:54:06.937 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2024-08-18T23:54:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.939+0000 7f46d8dfe640 1 -- 172.21.15.78:0/1971639000 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f46bc0a2c00 msgr2=0x7f46bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.939+0000 7f46d8dfe640 1 --2- 172.21.15.78:0/1971639000 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f46bc0a2c00 0x7f46bc0a50c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f46d4075cb0 tx=0x7f46c8005f90 comp rx=0 tx=0).stop 2024-08-18T23:54:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.939+0000 7f46d8dfe640 1 -- 172.21.15.78:0/1971639000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46d41018c0 msgr2=0x7f46d4074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.939+0000 7f46d8dfe640 1 --2- 172.21.15.78:0/1971639000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46d41018c0 0x7f46d4074220 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f46c400d950 tx=0x7f46c400de10 comp rx=0 tx=0).stop 2024-08-18T23:54:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.939+0000 7f46d8dfe640 1 -- 172.21.15.78:0/1971639000 shutdown_connections 2024-08-18T23:54:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.939+0000 7f46d8dfe640 1 --2- 172.21.15.78:0/1971639000 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f46bc0a2c00 0x7f46bc0a50c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.940+0000 7f46d8dfe640 1 --2- 172.21.15.78:0/1971639000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46d41078c0 0x7f46d4074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.940+0000 7f46d8dfe640 1 --2- 172.21.15.78:0/1971639000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46d4102260 0x7f46d4074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.940+0000 7f46d8dfe640 1 --2- 172.21.15.78:0/1971639000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46d41018c0 0x7f46d4074220 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.940+0000 7f46d8dfe640 1 -- 172.21.15.78:0/1971639000 >> 172.21.15.78:0/1971639000 conn(0x7f46d40fd5e0 msgr2=0x7f46d40fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:54:06.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.941+0000 7f46d8dfe640 1 -- 172.21.15.78:0/1971639000 shutdown_connections 2024-08-18T23:54:06.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:06.941+0000 7f46d8dfe640 1 -- 172.21.15.78:0/1971639000 wait complete. 2024-08-18T23:54:07.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:06 smithi078 ceph-mon[18440]: pgmap v645: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 921 B/s wr, 6 op/s 2024-08-18T23:54:07.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:06 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1971639000' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-18T23:54:07.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:06 smithi123 ceph-mon[21808]: pgmap v645: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 921 B/s wr, 6 op/s 2024-08-18T23:54:07.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:06 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1971639000' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-18T23:54:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:06 smithi161 ceph-mon[22027]: pgmap v645: 129 pgs: 129 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 921 B/s wr, 6 op/s 2024-08-18T23:54:07.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:06 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1971639000' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-18T23:54:07.646 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-18T23:54:07.895 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:54:08.190 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:07 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1971639000' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-18T23:54:08.190 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:07 smithi078 ceph-mon[18440]: osdmap e94: 12 total, 12 up, 12 in 2024-08-18T23:54:08.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:07 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1971639000' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-18T23:54:08.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:07 smithi123 ceph-mon[21808]: osdmap e94: 12 total, 12 up, 12 in 2024-08-18T23:54:08.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.405+0000 7f8cda669640 1 -- 172.21.15.78:0/3590675162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cd41022a0 msgr2=0x7f8cd41026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:08.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.405+0000 7f8cda669640 1 --2- 172.21.15.78:0/3590675162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cd41022a0 0x7f8cd41026e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009900 tx=0x7f8cbc02a480 comp rx=0 tx=0).stop 2024-08-18T23:54:08.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.405+0000 7f8cda669640 1 -- 172.21.15.78:0/3590675162 shutdown_connections 2024-08-18T23:54:08.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.405+0000 7f8cda669640 1 --2- 172.21.15.78:0/3590675162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd4102c20 0x7f8cd410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:08.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.405+0000 7f8cda669640 1 --2- 172.21.15.78:0/3590675162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cd41022a0 0x7f8cd41026e0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:08.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.405+0000 7f8cda669640 1 --2- 172.21.15.78:0/3590675162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd4100b50 0x7f8cd4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:08.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.405+0000 7f8cda669640 1 -- 172.21.15.78:0/3590675162 >> 172.21.15.78:0/3590675162 conn(0x7f8cd40fc500 msgr2=0x7f8cd40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:54:08.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.405+0000 7f8cda669640 1 -- 172.21.15.78:0/3590675162 shutdown_connections 2024-08-18T23:54:08.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.405+0000 7f8cda669640 1 -- 172.21.15.78:0/3590675162 wait complete. 2024-08-18T23:54:08.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.406+0000 7f8cda669640 1 Processor -- start 2024-08-18T23:54:08.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.406+0000 7f8cda669640 1 -- start start 2024-08-18T23:54:08.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.406+0000 7f8cda669640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cd4100b50 0x7f8cd4198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:08.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.406+0000 7f8cda669640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd41022a0 0x7f8cd4198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:08.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.406+0000 7f8cda669640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd4102c20 0x7f8cd419c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:08.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.406+0000 7f8cda669640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd4111dd0 con 0x7f8cd4100b50 2024-08-18T23:54:08.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.406+0000 7f8cda669640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd4111c50 con 0x7f8cd4102c20 2024-08-18T23:54:08.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.406+0000 7f8cda669640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd4111f50 con 0x7f8cd41022a0 2024-08-18T23:54:08.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.406+0000 7f8cd3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cd4100b50 0x7f8cd4198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:08.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.406+0000 7f8cd3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cd4100b50 0x7f8cd4198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44056/0 (socket says 172.21.15.78:44056) 2024-08-18T23:54:08.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.406+0000 7f8cd3fff640 1 -- 172.21.15.78:0/3036321050 learned_addr learned my addr 172.21.15.78:0/3036321050 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:54:08.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.407+0000 7f8ccbfff640 1 --2- 172.21.15.78:0/3036321050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd41022a0 0x7f8cd4198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:08.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.407+0000 7f8cd8c06640 1 --2- 172.21.15.78:0/3036321050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd4102c20 0x7f8cd419c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:08.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.407+0000 7f8cd3fff640 1 -- 172.21.15.78:0/3036321050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd41022a0 msgr2=0x7f8cd4198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:08.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.407+0000 7f8cd3fff640 1 --2- 172.21.15.78:0/3036321050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd41022a0 0x7f8cd4198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:08.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.407+0000 7f8cd3fff640 1 -- 172.21.15.78:0/3036321050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd4102c20 msgr2=0x7f8cd419c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:08.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.407+0000 7f8cd3fff640 1 --2- 172.21.15.78:0/3036321050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd4102c20 0x7f8cd419c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:08.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.407+0000 7f8cd3fff640 1 -- 172.21.15.78:0/3036321050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd419d0b0 con 0x7f8cd4100b50 2024-08-18T23:54:08.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.407+0000 7f8ccbfff640 1 --2- 172.21.15.78:0/3036321050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd41022a0 0x7f8cd4198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:54:08.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.407+0000 7f8cd8c06640 1 --2- 172.21.15.78:0/3036321050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd4102c20 0x7f8cd419c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:54:08.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.407+0000 7f8cd3fff640 1 --2- 172.21.15.78:0/3036321050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cd4100b50 0x7f8cd4198100 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400c920 tx=0x7f8cc400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:54:08.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.407+0000 7f8cd1ffb640 1 -- 172.21.15.78:0/3036321050 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4007c70 con 0x7f8cd4100b50 2024-08-18T23:54:08.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.407+0000 7f8cda669640 1 -- 172.21.15.78:0/3036321050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd419d3a0 con 0x7f8cd4100b50 2024-08-18T23:54:08.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.407+0000 7f8cd1ffb640 1 -- 172.21.15.78:0/3036321050 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc4007e10 con 0x7f8cd4100b50 2024-08-18T23:54:08.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.407+0000 7f8cd1ffb640 1 -- 172.21.15.78:0/3036321050 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4018890 con 0x7f8cd4100b50 2024-08-18T23:54:08.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.407+0000 7f8cda669640 1 -- 172.21.15.78:0/3036321050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd41a4c80 con 0x7f8cd4100b50 2024-08-18T23:54:08.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.408+0000 7f8cda669640 1 -- 172.21.15.78:0/3036321050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd4100f30 con 0x7f8cd4100b50 2024-08-18T23:54:08.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.409+0000 7f8cd1ffb640 1 -- 172.21.15.78:0/3036321050 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8cc4018a70 con 0x7f8cd4100b50 2024-08-18T23:54:08.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.410+0000 7f8cd1ffb640 1 --2- 172.21.15.78:0/3036321050 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8cc00a2b80 0x7f8cc00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:08.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.410+0000 7f8ccbfff640 1 --2- 172.21.15.78:0/3036321050 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8cc00a2b80 0x7f8cc00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:08.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.410+0000 7f8cd1ffb640 1 -- 172.21.15.78:0/3036321050 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8883+0+0 (secure 0 0 0) 0x7f8cc40cf4d0 con 0x7f8cd4100b50 2024-08-18T23:54:08.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.411+0000 7f8ccbfff640 1 --2- 172.21.15.78:0/3036321050 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8cc00a2b80 0x7f8cc00a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8cd40fe9d0 tx=0x7f8cbc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:54:08.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.415+0000 7f8cd1ffb640 1 -- 172.21.15.78:0/3036321050 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8cc4014030 con 0x7f8cd4100b50 2024-08-18T23:54:08.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:07 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1971639000' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-18T23:54:08.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:07 smithi161 ceph-mon[22027]: osdmap e94: 12 total, 12 up, 12 in 2024-08-18T23:54:08.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.585+0000 7f8cda669640 1 -- 172.21.15.78:0/3036321050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8cd4199460 con 0x7f8cd4100b50 2024-08-18T23:54:08.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.944+0000 7f8cd1ffb640 1 -- 172.21.15.78:0/3036321050 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) ==== 169+0+0 (secure 0 0 0) 0x7f8cc400de20 con 0x7f8cd4100b50 2024-08-18T23:54:08.946 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2024-08-18T23:54:08.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.946+0000 7f8cda669640 1 -- 172.21.15.78:0/3036321050 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8cc00a2b80 msgr2=0x7f8cc00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:08.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.946+0000 7f8cda669640 1 --2- 172.21.15.78:0/3036321050 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8cc00a2b80 0x7f8cc00a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8cd40fe9d0 tx=0x7f8cbc005a20 comp rx=0 tx=0).stop 2024-08-18T23:54:08.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.946+0000 7f8cda669640 1 -- 172.21.15.78:0/3036321050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cd4100b50 msgr2=0x7f8cd4198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:08.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.947+0000 7f8cda669640 1 --2- 172.21.15.78:0/3036321050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cd4100b50 0x7f8cd4198100 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400c920 tx=0x7f8cc400cde0 comp rx=0 tx=0).stop 2024-08-18T23:54:08.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.947+0000 7f8cda669640 1 -- 172.21.15.78:0/3036321050 shutdown_connections 2024-08-18T23:54:08.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.947+0000 7f8cda669640 1 --2- 172.21.15.78:0/3036321050 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f8cc00a2b80 0x7f8cc00a5040 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:08.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.947+0000 7f8cda669640 1 --2- 172.21.15.78:0/3036321050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd4102c20 0x7f8cd419c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:08.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.947+0000 7f8cda669640 1 --2- 172.21.15.78:0/3036321050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd41022a0 0x7f8cd4198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:08.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.947+0000 7f8cda669640 1 --2- 172.21.15.78:0/3036321050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cd4100b50 0x7f8cd4198100 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:08.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.947+0000 7f8cda669640 1 -- 172.21.15.78:0/3036321050 >> 172.21.15.78:0/3036321050 conn(0x7f8cd40fc500 msgr2=0x7f8cd4106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:54:08.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.947+0000 7f8cda669640 1 -- 172.21.15.78:0/3036321050 shutdown_connections 2024-08-18T23:54:08.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:08.947+0000 7f8cda669640 1 -- 172.21.15.78:0/3036321050 wait complete. 2024-08-18T23:54:09.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:08 smithi078 ceph-mon[18440]: pgmap v647: 97 pgs: 97 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:54:09.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:08 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3036321050' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-18T23:54:09.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:08 smithi123 ceph-mon[21808]: pgmap v647: 97 pgs: 97 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:54:09.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:08 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3036321050' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-18T23:54:09.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:08 smithi161 ceph-mon[22027]: pgmap v647: 97 pgs: 97 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:54:09.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:08 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3036321050' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-18T23:54:09.682 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833 shell --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-08-18T23:54:09.932 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/mon.a/config 2024-08-18T23:54:10.204 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:09 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/3036321050' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-18T23:54:10.204 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:09 smithi078 ceph-mon[18440]: osdmap e95: 12 total, 12 up, 12 in 2024-08-18T23:54:10.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:09 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/3036321050' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-18T23:54:10.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:09 smithi123 ceph-mon[21808]: osdmap e95: 12 total, 12 up, 12 in 2024-08-18T23:54:10.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:09 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/3036321050' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-18T23:54:10.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:09 smithi161 ceph-mon[22027]: osdmap e95: 12 total, 12 up, 12 in 2024-08-18T23:54:10.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.415+0000 7f7abd876640 1 -- 172.21.15.78:0/3659274117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ab8103380 msgr2=0x7f7ab810b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:10.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.415+0000 7f7abd876640 1 --2- 172.21.15.78:0/3659274117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ab8103380 0x7f7ab810b870 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009ca0 tx=0x7f7aac02a950 comp rx=0 tx=0).stop 2024-08-18T23:54:10.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.415+0000 7f7abd876640 1 -- 172.21.15.78:0/3659274117 shutdown_connections 2024-08-18T23:54:10.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.415+0000 7f7abd876640 1 --2- 172.21.15.78:0/3659274117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ab810be90 0x7f7ab810e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:10.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.415+0000 7f7abd876640 1 --2- 172.21.15.78:0/3659274117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ab8103380 0x7f7ab810b870 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:10.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.415+0000 7f7abd876640 1 --2- 172.21.15.78:0/3659274117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab8102a60 0x7f7ab8102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:10.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.415+0000 7f7abd876640 1 -- 172.21.15.78:0/3659274117 >> 172.21.15.78:0/3659274117 conn(0x7f7ab80fc4c0 msgr2=0x7f7ab80fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:54:10.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.415+0000 7f7abd876640 1 -- 172.21.15.78:0/3659274117 shutdown_connections 2024-08-18T23:54:10.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.415+0000 7f7abd876640 1 -- 172.21.15.78:0/3659274117 wait complete. 2024-08-18T23:54:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.415+0000 7f7abd876640 1 Processor -- start 2024-08-18T23:54:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.415+0000 7f7abd876640 1 -- start start 2024-08-18T23:54:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7abd876640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ab8102a60 0x7f7ab81006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7abd876640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ab8103380 0x7f7ab8100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7abd876640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab810be90 0x7f7ab8101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7abd876640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab8111c90 con 0x7f7ab8102a60 2024-08-18T23:54:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7abd876640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab8111b10 con 0x7f7ab810be90 2024-08-18T23:54:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7abd876640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab8111e10 con 0x7f7ab8103380 2024-08-18T23:54:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7ab6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ab8102a60 0x7f7ab81006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7ab6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ab8102a60 0x7f7ab81006c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44080/0 (socket says 172.21.15.78:44080) 2024-08-18T23:54:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7ab6ffd640 1 -- 172.21.15.78:0/1903730207 learned_addr learned my addr 172.21.15.78:0/1903730207 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-08-18T23:54:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7ab77fe640 1 --2- 172.21.15.78:0/1903730207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab810be90 0x7f7ab8101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7ab67fc640 1 --2- 172.21.15.78:0/1903730207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ab8103380 0x7f7ab8100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7ab6ffd640 1 -- 172.21.15.78:0/1903730207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ab8103380 msgr2=0x7f7ab8100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7ab6ffd640 1 --2- 172.21.15.78:0/1903730207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ab8103380 0x7f7ab8100c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7ab6ffd640 1 -- 172.21.15.78:0/1903730207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab810be90 msgr2=0x7f7ab8101140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7ab6ffd640 1 --2- 172.21.15.78:0/1903730207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab810be90 0x7f7ab8101140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7ab6ffd640 1 -- 172.21.15.78:0/1903730207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab81a6630 con 0x7f7ab8102a60 2024-08-18T23:54:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7ab77fe640 1 --2- 172.21.15.78:0/1903730207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab810be90 0x7f7ab8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-18T23:54:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7ab6ffd640 1 --2- 172.21.15.78:0/1903730207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ab8102a60 0x7f7ab81006c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000e9f0 tx=0x7f7ab000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:54:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7abc874640 1 -- 172.21.15.78:0/1903730207 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab000cd80 con 0x7f7ab8102a60 2024-08-18T23:54:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7abd876640 1 -- 172.21.15.78:0/1903730207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab81a6920 con 0x7f7ab8102a60 2024-08-18T23:54:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7abc874640 1 -- 172.21.15.78:0/1903730207 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab0010910 con 0x7f7ab8102a60 2024-08-18T23:54:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.416+0000 7f7abc874640 1 -- 172.21.15.78:0/1903730207 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00188f0 con 0x7f7ab8102a60 2024-08-18T23:54:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.417+0000 7f7abd876640 1 -- 172.21.15.78:0/1903730207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab81a6eb0 con 0x7f7ab8102a60 2024-08-18T23:54:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.418+0000 7f7abd876640 1 -- 172.21.15.78:0/1903730207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ab8102e40 con 0x7f7ab8102a60 2024-08-18T23:54:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.419+0000 7f7abc874640 1 -- 172.21.15.78:0/1903730207 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7ab0010ac0 con 0x7f7ab8102a60 2024-08-18T23:54:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.419+0000 7f7abc874640 1 --2- 172.21.15.78:0/1903730207 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7aa00a2b80 0x7f7aa00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-18T23:54:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.419+0000 7f7ab67fc640 1 --2- 172.21.15.78:0/1903730207 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7aa00a2b80 0x7f7aa00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-18T23:54:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.419+0000 7f7abc874640 1 -- 172.21.15.78:0/1903730207 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8324+0+0 (secure 0 0 0) 0x7f7ab00cf5b0 con 0x7f7ab8102a60 2024-08-18T23:54:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.420+0000 7f7ab67fc640 1 --2- 172.21.15.78:0/1903730207 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7aa00a2b80 0x7f7aa00a5040 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8102150 tx=0x7f7aac005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-18T23:54:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.424+0000 7f7abc874640 1 -- 172.21.15.78:0/1903730207 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7ab009c4b0 con 0x7f7ab8102a60 2024-08-18T23:54:10.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.590+0000 7f7abd876640 1 -- 172.21.15.78:0/1903730207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7ab8106f00 con 0x7f7ab8102a60 2024-08-18T23:54:10.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.955+0000 7f7abc874640 1 -- 172.21.15.78:0/1903730207 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v96) ==== 178+0+0 (secure 0 0 0) 0x7f7ab009bdb0 con 0x7f7ab8102a60 2024-08-18T23:54:10.956 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data_ec' removed 2024-08-18T23:54:10.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.957+0000 7f7abd876640 1 -- 172.21.15.78:0/1903730207 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7aa00a2b80 msgr2=0x7f7aa00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:10.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.957+0000 7f7abd876640 1 --2- 172.21.15.78:0/1903730207 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7aa00a2b80 0x7f7aa00a5040 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8102150 tx=0x7f7aac005ef0 comp rx=0 tx=0).stop 2024-08-18T23:54:10.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.957+0000 7f7abd876640 1 -- 172.21.15.78:0/1903730207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ab8102a60 msgr2=0x7f7ab81006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-18T23:54:10.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.957+0000 7f7abd876640 1 --2- 172.21.15.78:0/1903730207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ab8102a60 0x7f7ab81006c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000e9f0 tx=0x7f7ab000eeb0 comp rx=0 tx=0).stop 2024-08-18T23:54:10.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.957+0000 7f7abd876640 1 -- 172.21.15.78:0/1903730207 shutdown_connections 2024-08-18T23:54:10.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.957+0000 7f7abd876640 1 --2- 172.21.15.78:0/1903730207 >> [v2:172.21.15.78:6800/4000744943,v1:172.21.15.78:6801/4000744943] conn(0x7f7aa00a2b80 0x7f7aa00a5040 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:10.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.957+0000 7f7abd876640 1 --2- 172.21.15.78:0/1903730207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab810be90 0x7f7ab8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:10.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.957+0000 7f7abd876640 1 --2- 172.21.15.78:0/1903730207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ab8103380 0x7f7ab8100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:10.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.957+0000 7f7abd876640 1 --2- 172.21.15.78:0/1903730207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ab8102a60 0x7f7ab81006c0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-18T23:54:10.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.957+0000 7f7abd876640 1 -- 172.21.15.78:0/1903730207 >> 172.21.15.78:0/1903730207 conn(0x7f7ab80fc4c0 msgr2=0x7f7ab80fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-18T23:54:10.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.957+0000 7f7abd876640 1 -- 172.21.15.78:0/1903730207 shutdown_connections 2024-08-18T23:54:10.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-08-18T23:54:10.957+0000 7f7abd876640 1 -- 172.21.15.78:0/1903730207 wait complete. 2024-08-18T23:54:11.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:10 smithi078 ceph-mon[18440]: pgmap v649: 33 pgs: 33 active+clean; 577 KiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:54:11.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:10 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1903730207' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-18T23:54:11.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:10 smithi123 ceph-mon[21808]: pgmap v649: 33 pgs: 33 active+clean; 577 KiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:54:11.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:10 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1903730207' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-18T23:54:11.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:10 smithi161 ceph-mon[22027]: pgmap v649: 33 pgs: 33 active+clean; 577 KiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-18T23:54:11.427 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:10 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1903730207' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-18T23:54:11.782 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-18T23:54:11.808 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-18T23:54:11.839 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-18T23:54:11.869 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-08-18T23:54:11.869 DEBUG:teuthology.orchestra.run.smithi078:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-18T23:54:11.885 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-18T23:54:11.908 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-18T23:54:11.925 INFO:tasks.cephadm:Stopping all daemons... 2024-08-18T23:54:11.925 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-08-18T23:54:11.925 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mon.a 2024-08-18T23:54:12.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:11 smithi161 ceph-mon[22027]: from='client.? 172.21.15.78:0/1903730207' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-08-18T23:54:12.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:11 smithi161 ceph-mon[22027]: osdmap e96: 12 total, 12 up, 12 in 2024-08-18T23:54:12.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:11 smithi078 ceph-mon[18440]: from='client.? 172.21.15.78:0/1903730207' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-08-18T23:54:12.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:11 smithi078 ceph-mon[18440]: osdmap e96: 12 total, 12 up, 12 in 2024-08-18T23:54:12.220 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:11 smithi078 systemd[1]: Stopping Ceph mon.a for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:54:12.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:11 smithi123 ceph-mon[21808]: from='client.? 172.21.15.78:0/1903730207' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-08-18T23:54:12.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:11 smithi123 ceph-mon[21808]: osdmap e96: 12 total, 12 up, 12 in 2024-08-18T23:54:12.719 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:12 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-a[18436]: 2024-08-18T23:54:12.249+0000 7f43975d3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:54:12.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:12 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-a[18436]: 2024-08-18T23:54:12.249+0000 7f43975d3640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-08-18T23:54:12.720 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:12 smithi078 podman[96470]: 2024-08-18 23:54:12.432954141 +0000 UTC m=+0.316105861 container died c87427028777e7408705cd076ec10fe9fc3ec5d95b63378328b597d4805ea38a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-18T23:54:13.085 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:12 smithi078 podman[96470]: 2024-08-18 23:54:12.832652188 +0000 UTC m=+0.715803907 container cleanup c87427028777e7408705cd076ec10fe9fc3ec5d95b63378328b597d4805ea38a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-18T23:54:13.085 INFO:journalctl@ceph.mon.a.smithi078.stdout:Aug 18 23:54:12 smithi078 bash[96470]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-a 2024-08-18T23:54:13.223 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mon.a.service' 2024-08-18T23:54:13.272 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:54:13.272 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-08-18T23:54:13.273 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-08-18T23:54:13.273 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mon.b 2024-08-18T23:54:13.635 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:13 smithi123 systemd[1]: Stopping Ceph mon.b for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:54:13.952 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:13 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-b[21804]: 2024-08-18T23:54:13.632+0000 7f2e6edf3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:54:13.952 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:13 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-b[21804]: 2024-08-18T23:54:13.632+0000 7f2e6edf3640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-08-18T23:54:13.952 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:13 smithi123 podman[56063]: 2024-08-18 23:54:13.695187991 +0000 UTC m=+0.175458173 container died 12b66fb5d4d319c4ab5863569d8fd16a1f76932ceaf37d81522a500898337e05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-08-18T23:54:14.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:14 smithi123 podman[56063]: 2024-08-18 23:54:14.050493982 +0000 UTC m=+0.530764155 container cleanup 12b66fb5d4d319c4ab5863569d8fd16a1f76932ceaf37d81522a500898337e05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux ) 2024-08-18T23:54:14.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Aug 18 23:54:14 smithi123 bash[56063]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-b 2024-08-18T23:54:14.551 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mon.b.service' 2024-08-18T23:54:14.596 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:54:14.596 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-08-18T23:54:14.596 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-08-18T23:54:14.597 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mon.c 2024-08-18T23:54:14.881 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:14 smithi161 systemd[1]: Stopping Ceph mon.c for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:54:15.150 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:14 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-c[22023]: 2024-08-18T23:54:14.923+0000 7f49d9d15640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:54:15.150 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:14 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-c[22023]: 2024-08-18T23:54:14.923+0000 7f49d9d15640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-08-18T23:54:15.150 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:15 smithi161 podman[56634]: 2024-08-18 23:54:15.036378069 +0000 UTC m=+0.225229667 container died bbc41949c7923301bade0f04fa212c11248143ba5606428d916a10f3ac576f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-c, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-08-18T23:54:15.692 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:15 smithi161 podman[56634]: 2024-08-18 23:54:15.440516443 +0000 UTC m=+0.629368044 container cleanup bbc41949c7923301bade0f04fa212c11248143ba5606428d916a10f3ac576f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-c, io.buildah.version=1.36.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-08-18T23:54:15.693 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 18 23:54:15 smithi161 bash[56634]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mon-c 2024-08-18T23:54:15.827 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mon.c.service' 2024-08-18T23:54:15.873 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:54:15.873 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-08-18T23:54:15.873 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-08-18T23:54:15.873 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mgr.x 2024-08-18T23:54:16.206 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:54:15 smithi078 systemd[1]: Stopping Ceph mgr.x for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:54:16.469 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:54:16 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x[18672]: 2024-08-18T23:54:16.204+0000 7f3afe5c0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:54:16.469 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:54:16 smithi078 podman[96604]: 2024-08-18 23:54:16.235524647 +0000 UTC m=+0.158884556 container died 77b8af6f67c4b7ee1530806013871d533e9cbd45b6f327a51b56af57562fed2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-08-18T23:54:16.882 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:54:16 smithi078 podman[96604]: 2024-08-18 23:54:16.663501855 +0000 UTC m=+0.586861760 container cleanup 77b8af6f67c4b7ee1530806013871d533e9cbd45b6f327a51b56af57562fed2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:de09dd018de4baddfa6ca4d43bc8fb784cf75833, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-08-18T23:54:16.882 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Aug 18 23:54:16 smithi078 bash[96604]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-x 2024-08-18T23:54:17.108 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mgr.x.service' 2024-08-18T23:54:17.146 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:54:17.147 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-08-18T23:54:17.147 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-08-18T23:54:17.147 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mgr.y 2024-08-18T23:54:17.486 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:54:17 smithi123 systemd[1]: Stopping Ceph mgr.y for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:54:17.758 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:54:17 smithi123 podman[56191]: 2024-08-18 23:54:17.485480445 +0000 UTC m=+0.137338524 container died 4ec69d36f5b08888ad749144b9dca9c04e2bb6dcf7de611ee25c6695bb3a3fb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0) 2024-08-18T23:54:18.048 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:54:17 smithi123 podman[56191]: 2024-08-18 23:54:17.848640479 +0000 UTC m=+0.500498557 container cleanup 4ec69d36f5b08888ad749144b9dca9c04e2bb6dcf7de611ee25c6695bb3a3fb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-18T23:54:18.048 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Aug 18 23:54:17 smithi123 bash[56191]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-y 2024-08-18T23:54:18.348 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mgr.y.service' 2024-08-18T23:54:18.383 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:54:18.383 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-08-18T23:54:18.384 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-08-18T23:54:18.384 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mgr.z 2024-08-18T23:54:18.677 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:54:18 smithi161 systemd[1]: Stopping Ceph mgr.z for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:54:19.097 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:54:18 smithi161 podman[56769]: 2024-08-18 23:54:18.724584511 +0000 UTC m=+0.134415062 container died faf5c890d84059c9d1a896940c1091d492e7fb19e6fd1b815bb761a36156bf1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-18T23:54:19.427 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:54:19 smithi161 podman[56769]: 2024-08-18 23:54:19.179608019 +0000 UTC m=+0.589438569 container cleanup faf5c890d84059c9d1a896940c1091d492e7fb19e6fd1b815bb761a36156bf1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:54:19.427 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 18 23:54:19 smithi161 bash[56769]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mgr-z 2024-08-18T23:54:19.623 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mgr.z.service' 2024-08-18T23:54:19.659 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:54:19.660 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-08-18T23:54:19.660 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-08-18T23:54:19.660 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.0 2024-08-18T23:54:19.969 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:54:19 smithi078 systemd[1]: Stopping Ceph osd.0 for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:54:20.469 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:54:19 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-0[32351]: 2024-08-18T23:54:19.983+0000 7fcfe0af3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:54:20.470 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:54:19 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-0[32351]: 2024-08-18T23:54:19.983+0000 7fcfe0af3640 -1 osd.0 96 *** Got signal Terminated *** 2024-08-18T23:54:20.470 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:54:19 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-0[32351]: 2024-08-18T23:54:19.983+0000 7fcfe0af3640 -1 osd.0 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-18T23:54:25.853 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:54:25 smithi078 podman[96726]: 2024-08-18 23:54:25.563803917 +0000 UTC m=+5.699222728 container died 5590187c8380aa5694645f5978ac4a3a145c3c36b9b1ab64ae8dade46f6ff2c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-08-18T23:54:26.219 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:54:25 smithi078 podman[96726]: 2024-08-18 23:54:25.943164004 +0000 UTC m=+6.078582815 container cleanup 5590187c8380aa5694645f5978ac4a3a145c3c36b9b1ab64ae8dade46f6ff2c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, io.buildah.version=1.36.0) 2024-08-18T23:54:26.220 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:54:25 smithi078 bash[96726]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-0 2024-08-18T23:54:26.624 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:54:26 smithi078 podman[96743]: 2024-08-18 23:54:26.27001206 +0000 UTC m=+0.709480492 container remove 5590187c8380aa5694645f5978ac4a3a145c3c36b9b1ab64ae8dade46f6ff2c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-08-18T23:54:26.891 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:54:26 smithi078 podman[96835]: 2024-08-18 23:54:26.526038921 +0000 UTC m=+0.017260324 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:54:26.891 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:54:26 smithi078 podman[96835]: 2024-08-18 23:54:26.654289113 +0000 UTC m=+0.145510561 container create 9c0f5334bd4f8fcc967aa44aa39b061ed8f6e75415a15cf3ce65ca6f22bc10a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0) 2024-08-18T23:54:27.221 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:54:26 smithi078 podman[96835]: 2024-08-18 23:54:26.890230794 +0000 UTC m=+0.381452206 container init 9c0f5334bd4f8fcc967aa44aa39b061ed8f6e75415a15cf3ce65ca6f22bc10a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-08-18T23:54:27.222 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:54:26 smithi078 podman[96835]: 2024-08-18 23:54:26.894059841 +0000 UTC m=+0.385281253 container start 9c0f5334bd4f8fcc967aa44aa39b061ed8f6e75415a15cf3ce65ca6f22bc10a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-08-18T23:54:27.222 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:54:26 smithi078 podman[96835]: 2024-08-18 23:54:26.965154367 +0000 UTC m=+0.456375772 container attach 9c0f5334bd4f8fcc967aa44aa39b061ed8f6e75415a15cf3ce65ca6f22bc10a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-08-18T23:54:27.222 INFO:journalctl@ceph.osd.0.smithi078.stdout:Aug 18 23:54:27 smithi078 podman[96835]: 2024-08-18 23:54:27.078339321 +0000 UTC m=+0.569560730 container died 9c0f5334bd4f8fcc967aa44aa39b061ed8f6e75415a15cf3ce65ca6f22bc10a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0) 2024-08-18T23:54:28.003 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.0.service' 2024-08-18T23:54:28.041 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:54:28.042 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-08-18T23:54:28.042 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-08-18T23:54:28.042 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.1 2024-08-18T23:54:28.446 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:54:28 smithi123 systemd[1]: Stopping Ceph osd.1 for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:54:28.798 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:54:28 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-1[26149]: 2024-08-18T23:54:28.445+0000 7f58decce640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:54:28.798 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:54:28 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-1[26149]: 2024-08-18T23:54:28.445+0000 7f58decce640 -1 osd.1 96 *** Got signal Terminated *** 2024-08-18T23:54:28.798 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:54:28 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-1[26149]: 2024-08-18T23:54:28.445+0000 7f58decce640 -1 osd.1 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-18T23:54:34.281 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:54:33 smithi123 podman[56317]: 2024-08-18 23:54:33.990229724 +0000 UTC m=+5.751821871 container died c0d424690978374ad9ea24453efd7ad55299ed5452d89d38455f4665bdbdb18e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True) 2024-08-18T23:54:34.548 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:54:34 smithi123 podman[56317]: 2024-08-18 23:54:34.370610884 +0000 UTC m=+6.132203030 container cleanup c0d424690978374ad9ea24453efd7ad55299ed5452d89d38455f4665bdbdb18e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-1, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS) 2024-08-18T23:54:34.548 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:54:34 smithi123 bash[56317]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-1 2024-08-18T23:54:35.003 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:54:34 smithi123 podman[56334]: 2024-08-18 23:54:34.680595562 +0000 UTC m=+0.679757152 container remove c0d424690978374ad9ea24453efd7ad55299ed5452d89d38455f4665bdbdb18e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-1, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:54:35.298 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:54:35 smithi123 podman[56420]: 2024-08-18 23:54:34.905281981 +0000 UTC m=+0.017289217 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:54:35.298 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:54:35 smithi123 podman[56420]: 2024-08-18 23:54:35.082069079 +0000 UTC m=+0.194076354 container create 574f9395c29dbc0f192fde09238b53d19c68f8c6c1001fc32904a9b5419a787c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:54:35.623 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:54:35 smithi123 podman[56420]: 2024-08-18 23:54:35.327425617 +0000 UTC m=+0.439432836 container init 574f9395c29dbc0f192fde09238b53d19c68f8c6c1001fc32904a9b5419a787c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-1-deactivate, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-08-18T23:54:35.624 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:54:35 smithi123 podman[56420]: 2024-08-18 23:54:35.330971942 +0000 UTC m=+0.442979166 container start 574f9395c29dbc0f192fde09238b53d19c68f8c6c1001fc32904a9b5419a787c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.36.0) 2024-08-18T23:54:35.624 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:54:35 smithi123 podman[56420]: 2024-08-18 23:54:35.417787202 +0000 UTC m=+0.529794422 container attach 574f9395c29dbc0f192fde09238b53d19c68f8c6c1001fc32904a9b5419a787c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-18T23:54:35.955 INFO:journalctl@ceph.osd.1.smithi123.stdout:Aug 18 23:54:35 smithi123 podman[56420]: 2024-08-18 23:54:35.622944367 +0000 UTC m=+0.734951587 container died 574f9395c29dbc0f192fde09238b53d19c68f8c6c1001fc32904a9b5419a787c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-08-18T23:54:36.247 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.1.service' 2024-08-18T23:54:36.294 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:54:36.294 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-08-18T23:54:36.294 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-08-18T23:54:36.294 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.2 2024-08-18T23:54:36.618 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:54:36 smithi161 systemd[1]: Stopping Ceph osd.2 for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:54:36.927 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:54:36 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-2[26978]: 2024-08-18T23:54:36.616+0000 7f2a1328b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:54:36.927 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:54:36 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-2[26978]: 2024-08-18T23:54:36.616+0000 7f2a1328b640 -1 osd.2 96 *** Got signal Terminated *** 2024-08-18T23:54:36.928 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:54:36 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-2[26978]: 2024-08-18T23:54:36.616+0000 7f2a1328b640 -1 osd.2 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-18T23:54:42.310 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:54:42 smithi161 podman[56895]: 2024-08-18 23:54:42.015337143 +0000 UTC m=+5.514557298 container died 917133d4f3fab8f928aaec3d09e932ce0cd1f99919b81ad7a496765031c1ea25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0) 2024-08-18T23:54:42.678 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:54:42 smithi161 podman[56895]: 2024-08-18 23:54:42.408695145 +0000 UTC m=+5.907915305 container cleanup 917133d4f3fab8f928aaec3d09e932ce0cd1f99919b81ad7a496765031c1ea25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True) 2024-08-18T23:54:42.678 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:54:42 smithi161 bash[56895]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-2 2024-08-18T23:54:43.052 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:54:42 smithi161 podman[56909]: 2024-08-18 23:54:42.714129142 +0000 UTC m=+0.691913135 container remove 917133d4f3fab8f928aaec3d09e932ce0cd1f99919b81ad7a496765031c1ea25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-18T23:54:43.327 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:54:43 smithi161 podman[56999]: 2024-08-18 23:54:42.954353187 +0000 UTC m=+0.017320157 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:54:43.327 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:54:43 smithi161 podman[56999]: 2024-08-18 23:54:43.069399105 +0000 UTC m=+0.132366071 container create 65645127940c83411483777ffed00e905b72b578a739fff04edb6809b91ade5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-08-18T23:54:43.678 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:54:43 smithi161 podman[56999]: 2024-08-18 23:54:43.326280978 +0000 UTC m=+0.389247941 container init 65645127940c83411483777ffed00e905b72b578a739fff04edb6809b91ade5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-2-deactivate, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-18T23:54:43.678 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:54:43 smithi161 podman[56999]: 2024-08-18 23:54:43.329909481 +0000 UTC m=+0.392876444 container start 65645127940c83411483777ffed00e905b72b578a739fff04edb6809b91ade5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-2-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-18T23:54:43.678 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:54:43 smithi161 podman[56999]: 2024-08-18 23:54:43.405473119 +0000 UTC m=+0.468440091 container attach 65645127940c83411483777ffed00e905b72b578a739fff04edb6809b91ade5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-2-deactivate, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-08-18T23:54:43.678 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 18 23:54:43 smithi161 podman[56999]: 2024-08-18 23:54:43.524035311 +0000 UTC m=+0.587002275 container died 65645127940c83411483777ffed00e905b72b578a739fff04edb6809b91ade5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-2-deactivate, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-08-18T23:54:44.184 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.2.service' 2024-08-18T23:54:44.225 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:54:44.225 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-08-18T23:54:44.225 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-08-18T23:54:44.225 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.3 2024-08-18T23:54:44.540 INFO:journalctl@ceph.osd.3.smithi078.stdout:Aug 18 23:54:44 smithi078 systemd[1]: Stopping Ceph osd.3 for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:54:44.799 INFO:journalctl@ceph.osd.3.smithi078.stdout:Aug 18 23:54:44 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-3[38685]: 2024-08-18T23:54:44.538+0000 7f642010a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:54:44.799 INFO:journalctl@ceph.osd.3.smithi078.stdout:Aug 18 23:54:44 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-3[38685]: 2024-08-18T23:54:44.538+0000 7f642010a640 -1 osd.3 96 *** Got signal Terminated *** 2024-08-18T23:54:44.799 INFO:journalctl@ceph.osd.3.smithi078.stdout:Aug 18 23:54:44 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-3[38685]: 2024-08-18T23:54:44.538+0000 7f642010a640 -1 osd.3 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-18T23:54:47.969 INFO:journalctl@ceph.osd.3.smithi078.stdout:Aug 18 23:54:47 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-3[38685]: 2024-08-18T23:54:47.686+0000 7f6418718640 -1 osd.3 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:21.176660+0000 front 2024-08-18T23:54:21.176636+0000 (oldest deadline 2024-08-18T23:54:47.076799+0000) 2024-08-18T23:54:48.427 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:54:48 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:54:48.021+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:54:48.548 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:54:48 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:54:48.052+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:54:48.720 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:54:48 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:54:48.351+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:54:48.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:54:48 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:54:48.517+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:54:49.219 INFO:journalctl@ceph.osd.3.smithi078.stdout:Aug 18 23:54:48 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-3[38685]: 2024-08-18T23:54:48.723+0000 7f6418718640 -1 osd.3 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:21.176660+0000 front 2024-08-18T23:54:21.176636+0000 (oldest deadline 2024-08-18T23:54:47.076799+0000) 2024-08-18T23:54:49.427 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:54:48 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:54:48.976+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:54:49.548 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:54:49 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:54:49.065+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:54:49.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:54:49 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:54:49.385+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:54:49.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:54:49 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:54:49.555+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:54:50.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:54:50 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:54:50.021+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:54:50.382 INFO:journalctl@ceph.osd.3.smithi078.stdout:Aug 18 23:54:50 smithi078 podman[96949]: 2024-08-18 23:54:50.092972456 +0000 UTC m=+5.672869977 container died 14e82150a24a015fab792eb82824ed0001e625c44b342f51e1651670b88d5b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-3, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:54:50.427 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:54:50 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:54:50.013+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:54:50.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:54:50 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:54:50.400+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:54:50.720 INFO:journalctl@ceph.osd.3.smithi078.stdout:Aug 18 23:54:50 smithi078 podman[96949]: 2024-08-18 23:54:50.547797344 +0000 UTC m=+6.127694907 container cleanup 14e82150a24a015fab792eb82824ed0001e625c44b342f51e1651670b88d5b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-3, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-08-18T23:54:50.720 INFO:journalctl@ceph.osd.3.smithi078.stdout:Aug 18 23:54:50 smithi078 bash[96949]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-3 2024-08-18T23:54:50.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:54:50 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:54:50.510+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:54:51.219 INFO:journalctl@ceph.osd.3.smithi078.stdout:Aug 18 23:54:50 smithi078 podman[96979]: 2024-08-18 23:54:50.866225016 +0000 UTC m=+0.766617691 container remove 14e82150a24a015fab792eb82824ed0001e625c44b342f51e1651670b88d5b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0) 2024-08-18T23:54:51.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:54:51 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:54:51.021+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:54:51.427 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:54:51 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:54:51.025+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:54:51.428 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:54:51 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:54:51.237+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.695430+0000 front 2024-08-18T23:54:22.695540+0000 (oldest deadline 2024-08-18T23:54:50.296180+0000) 2024-08-18T23:54:51.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:54:51 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:54:51.447+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:54:51.520 INFO:journalctl@ceph.osd.3.smithi078.stdout:Aug 18 23:54:51 smithi078 podman[97068]: 2024-08-18 23:54:51.140486594 +0000 UTC m=+0.017174239 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:54:51.520 INFO:journalctl@ceph.osd.3.smithi078.stdout:Aug 18 23:54:51 smithi078 podman[97068]: 2024-08-18 23:54:51.250512816 +0000 UTC m=+0.127200451 container create f2ca8524e0249e53e2100a9f2338af9ce125f37c934787d509f1623d54e6d932 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-3-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-18T23:54:51.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:54:51 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:54:51.543+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:54:51.969 INFO:journalctl@ceph.osd.3.smithi078.stdout:Aug 18 23:54:51 smithi078 podman[97068]: 2024-08-18 23:54:51.519571913 +0000 UTC m=+0.396259545 container init f2ca8524e0249e53e2100a9f2338af9ce125f37c934787d509f1623d54e6d932 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-3-deactivate, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-08-18T23:54:51.970 INFO:journalctl@ceph.osd.3.smithi078.stdout:Aug 18 23:54:51 smithi078 podman[97068]: 2024-08-18 23:54:51.522834223 +0000 UTC m=+0.399521853 container start f2ca8524e0249e53e2100a9f2338af9ce125f37c934787d509f1623d54e6d932 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-3-deactivate, io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-08-18T23:54:51.970 INFO:journalctl@ceph.osd.3.smithi078.stdout:Aug 18 23:54:51 smithi078 podman[97068]: 2024-08-18 23:54:51.62819483 +0000 UTC m=+0.504882475 container attach f2ca8524e0249e53e2100a9f2338af9ce125f37c934787d509f1623d54e6d932 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-08-18T23:54:51.970 INFO:journalctl@ceph.osd.3.smithi078.stdout:Aug 18 23:54:51 smithi078 podman[97068]: 2024-08-18 23:54:51.708977274 +0000 UTC m=+0.585664910 container died f2ca8524e0249e53e2100a9f2338af9ce125f37c934787d509f1623d54e6d932 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0) 2024-08-18T23:54:52.394 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.3.service' 2024-08-18T23:54:52.427 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:54:52 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:54:52.232+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.695430+0000 front 2024-08-18T23:54:22.695540+0000 (oldest deadline 2024-08-18T23:54:50.296180+0000) 2024-08-18T23:54:52.427 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:54:51 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:54:51.997+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:54:52.436 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:54:52.436 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-08-18T23:54:52.437 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-08-18T23:54:52.437 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.4 2024-08-18T23:54:52.465 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:54:52 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:54:52.061+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:54:52.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:54:52 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:54:52.422+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:54:52.758 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:54:52 smithi123 systemd[1]: Stopping Ceph osd.4 for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:54:52.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:54:52 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:54:52.583+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:54:53.047 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:54:52 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-4[32991]: 2024-08-18T23:54:52.757+0000 7f4deaa0b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:54:53.048 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:54:52 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-4[32991]: 2024-08-18T23:54:52.757+0000 7f4deaa0b640 -1 osd.4 96 *** Got signal Terminated *** 2024-08-18T23:54:53.048 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:54:52 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-4[32991]: 2024-08-18T23:54:52.757+0000 7f4deaa0b640 -1 osd.4 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-18T23:54:53.264 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:54:52 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:54:52.984+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:54:53.537 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:54:53 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:54:53.263+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.695430+0000 front 2024-08-18T23:54:22.695540+0000 (oldest deadline 2024-08-18T23:54:50.296180+0000) 2024-08-18T23:54:53.547 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:54:53 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:54:53.099+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:54:53.720 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:54:53 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:54:53.440+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:54:53.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:54:53 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:54:53.536+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:54:54.302 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:54:54 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:54:54.012+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:54:54.548 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:54:54 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:54:54.103+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:54:54.677 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:54:54 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:54:54.301+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.695430+0000 front 2024-08-18T23:54:22.695540+0000 (oldest deadline 2024-08-18T23:54:50.296180+0000) 2024-08-18T23:54:54.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:54:54 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:54:54.547+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:54:54.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:54:54 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:54:54.547+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:54:54.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:54:54 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:54:54.446+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:54:55.048 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:54:54 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-4[32991]: 2024-08-18T23:54:54.657+0000 7f4de3019640 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.260047+0000 front 2024-08-18T23:54:30.259991+0000 (oldest deadline 2024-08-18T23:54:54.360154+0000) 2024-08-18T23:54:55.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:54:55 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:54:55.011+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:54:55.548 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:54:55 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:54:55.083+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:54:55.548 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:54:55 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:54:55.083+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:54:55.677 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:54:55 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:54:55.276+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.695430+0000 front 2024-08-18T23:54:22.695540+0000 (oldest deadline 2024-08-18T23:54:50.296180+0000) 2024-08-18T23:54:55.677 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:54:55 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:54:55.276+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.296947+0000 front 2024-08-18T23:54:30.296810+0000 (oldest deadline 2024-08-18T23:54:54.396696+0000) 2024-08-18T23:54:55.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:54:55 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:54:55.512+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:54:55.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:54:55 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:54:55.512+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:54:55.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:54:55 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:54:55.471+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:54:56.048 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:54:55 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-4[32991]: 2024-08-18T23:54:55.685+0000 7f4de3019640 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.260047+0000 front 2024-08-18T23:54:30.259991+0000 (oldest deadline 2024-08-18T23:54:54.360154+0000) 2024-08-18T23:54:56.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:54:55 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:54:55.750+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:54:56.260 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:54:55 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:54:55.966+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:54:56.548 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:54:56 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:54:56.083+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:54:56.548 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:54:56 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:54:56.083+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:54:56.677 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:54:56 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:54:56.259+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.695430+0000 front 2024-08-18T23:54:22.695540+0000 (oldest deadline 2024-08-18T23:54:50.296180+0000) 2024-08-18T23:54:56.678 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:54:56 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:54:56.259+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.296947+0000 front 2024-08-18T23:54:30.296810+0000 (oldest deadline 2024-08-18T23:54:54.396696+0000) 2024-08-18T23:54:56.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:54:56 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:54:56.488+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:54:56.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:54:56 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:54:56.488+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:54:56.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:54:56 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:54:56.507+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:54:56.970 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:54:56 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:54:56.695+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.255549+0000 front 2024-08-18T23:54:31.255738+0000 (oldest deadline 2024-08-18T23:54:55.955541+0000) 2024-08-18T23:54:57.048 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:54:56 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-4[32991]: 2024-08-18T23:54:56.636+0000 7f4de3019640 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.260047+0000 front 2024-08-18T23:54:30.259991+0000 (oldest deadline 2024-08-18T23:54:54.360154+0000) 2024-08-18T23:54:57.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:54:56 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:54:56.736+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:54:57.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:54:56 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:54:56.924+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:54:57.548 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:54:57 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:54:57.070+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:54:57.548 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:54:57 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:54:57.070+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:54:57.677 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:54:57 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:54:57.480+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:54:57.677 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:54:57 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:54:57.480+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:54:57.678 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:54:57 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:54:57.284+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.695430+0000 front 2024-08-18T23:54:22.695540+0000 (oldest deadline 2024-08-18T23:54:50.296180+0000) 2024-08-18T23:54:57.678 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:54:57 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:54:57.284+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.296947+0000 front 2024-08-18T23:54:30.296810+0000 (oldest deadline 2024-08-18T23:54:54.396696+0000) 2024-08-18T23:54:57.969 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:54:57 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:54:57.699+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.255549+0000 front 2024-08-18T23:54:31.255738+0000 (oldest deadline 2024-08-18T23:54:55.955541+0000) 2024-08-18T23:54:57.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:54:57 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:54:57.499+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:54:57.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:54:57 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:54:57.499+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:54:58.023 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:54:57 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-4[32991]: 2024-08-18T23:54:57.646+0000 7f4de3019640 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.260047+0000 front 2024-08-18T23:54:30.259991+0000 (oldest deadline 2024-08-18T23:54:54.360154+0000) 2024-08-18T23:54:58.024 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:54:57 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:54:57.692+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:54:58.298 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:54:58 smithi123 podman[56535]: 2024-08-18 23:54:58.134523574 +0000 UTC m=+5.486741044 container died ebba362e042d8047d9a558b146101cf65b28593b35e8bb2fcd8d012986f7fcb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-18T23:54:58.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:54:58 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:54:58.021+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:54:58.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:54:58 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:54:58.021+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:54:58.329 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:54:57 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:54:57.926+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:54:58.677 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:54:58 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:54:58.511+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:54:58.677 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:54:58 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:54:58.511+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:54:58.678 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:54:58 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:54:58.327+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.695430+0000 front 2024-08-18T23:54:22.695540+0000 (oldest deadline 2024-08-18T23:54:50.296180+0000) 2024-08-18T23:54:58.678 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:54:58 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:54:58.327+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.296947+0000 front 2024-08-18T23:54:30.296810+0000 (oldest deadline 2024-08-18T23:54:54.396696+0000) 2024-08-18T23:54:58.684 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:54:58 smithi123 podman[56535]: 2024-08-18 23:54:58.46579552 +0000 UTC m=+5.818013038 container cleanup ebba362e042d8047d9a558b146101cf65b28593b35e8bb2fcd8d012986f7fcb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-4, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-08-18T23:54:58.684 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:54:58 smithi123 bash[56535]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-4 2024-08-18T23:54:58.969 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:54:58 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:54:58.720+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.255549+0000 front 2024-08-18T23:54:31.255738+0000 (oldest deadline 2024-08-18T23:54:55.955541+0000) 2024-08-18T23:54:58.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:54:58 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:54:58.531+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:54:58.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:54:58 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:54:58.531+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:54:59.026 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:54:58 smithi123 podman[56553]: 2024-08-18 23:54:58.683955356 +0000 UTC m=+0.543842264 container remove ebba362e042d8047d9a558b146101cf65b28593b35e8bb2fcd8d012986f7fcb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-4, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-08-18T23:54:59.026 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:54:58 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:54:58.710+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:54:59.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:54:58 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:54:58.878+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:54:59.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:54:59 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:54:59.023+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:54:59.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:54:59 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:54:59.023+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:54:59.298 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:54:59 smithi123 podman[56643]: 2024-08-18 23:54:59.007152913 +0000 UTC m=+0.017177196 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:54:59.298 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:54:59 smithi123 podman[56643]: 2024-08-18 23:54:59.118151473 +0000 UTC m=+0.128175731 container create d1ecf7ab185857520e9a096275b510d78b6613ecd014c4cfff18cc57c2879e69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, RELEASE=squid-de09dd0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-18T23:54:59.672 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:54:59 smithi123 podman[56643]: 2024-08-18 23:54:59.396231201 +0000 UTC m=+0.406255467 container init d1ecf7ab185857520e9a096275b510d78b6613ecd014c4cfff18cc57c2879e69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-18T23:54:59.673 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:54:59 smithi123 podman[56643]: 2024-08-18 23:54:59.400495958 +0000 UTC m=+0.410520229 container start d1ecf7ab185857520e9a096275b510d78b6613ecd014c4cfff18cc57c2879e69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-08-18T23:54:59.673 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:54:59 smithi123 podman[56643]: 2024-08-18 23:54:59.471045307 +0000 UTC m=+0.481069568 container attach d1ecf7ab185857520e9a096275b510d78b6613ecd014c4cfff18cc57c2879e69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-4-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-08-18T23:54:59.673 INFO:journalctl@ceph.osd.4.smithi123.stdout:Aug 18 23:54:59 smithi123 podman[56643]: 2024-08-18 23:54:59.585397587 +0000 UTC m=+0.595421846 container died d1ecf7ab185857520e9a096275b510d78b6613ecd014c4cfff18cc57c2879e69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-4-deactivate, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-08-18T23:54:59.677 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:54:59 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:54:59.299+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.695430+0000 front 2024-08-18T23:54:22.695540+0000 (oldest deadline 2024-08-18T23:54:50.296180+0000) 2024-08-18T23:54:59.677 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:54:59 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:54:59.299+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.296947+0000 front 2024-08-18T23:54:30.296810+0000 (oldest deadline 2024-08-18T23:54:54.396696+0000) 2024-08-18T23:54:59.677 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:54:59 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:54:59.477+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:54:59.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:54:59 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:54:59.477+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:54:59.969 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:54:59 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:54:59.702+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.255549+0000 front 2024-08-18T23:54:31.255738+0000 (oldest deadline 2024-08-18T23:54:55.955541+0000) 2024-08-18T23:54:59.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:54:59 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:54:59.569+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:54:59.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:54:59 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:54:59.569+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:00.000 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:54:59 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:54:59.670+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:00.164 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.4.service' 2024-08-18T23:55:00.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:54:59 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:54:59.897+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:00.245 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:55:00.245 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-08-18T23:55:00.245 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-08-18T23:55:00.245 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.5 2024-08-18T23:55:00.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:54:59 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:54:59.998+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:00.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:54:59 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:54:59.998+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:00.563 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:00 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:00.457+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:00.563 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:00 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:00.457+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:00.563 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:00 smithi161 systemd[1]: Stopping Ceph osd.5 for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:55:00.563 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:00 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:55:00.334+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.695430+0000 front 2024-08-18T23:54:22.695540+0000 (oldest deadline 2024-08-18T23:54:50.296180+0000) 2024-08-18T23:55:00.563 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:00 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:55:00.334+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.296947+0000 front 2024-08-18T23:54:30.296810+0000 (oldest deadline 2024-08-18T23:54:54.396696+0000) 2024-08-18T23:55:00.563 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:00 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:55:00.561+0000 7f50cc08b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:55:00.564 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:00 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:55:00.561+0000 7f50cc08b640 -1 osd.5 96 *** Got signal Terminated *** 2024-08-18T23:55:00.564 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:00 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:55:00.561+0000 7f50cc08b640 -1 osd.5 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-18T23:55:00.969 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:00.730+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.255549+0000 front 2024-08-18T23:54:31.255738+0000 (oldest deadline 2024-08-18T23:54:55.955541+0000) 2024-08-18T23:55:00.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:00.555+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:00.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:00 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:00.555+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:01.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:00 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:00.674+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:01.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:00 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:00.855+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:01.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:01 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:01.004+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:01.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:01 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:01.004+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:01.677 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:01 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:55:01.361+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.695430+0000 front 2024-08-18T23:54:22.695540+0000 (oldest deadline 2024-08-18T23:54:50.296180+0000) 2024-08-18T23:55:01.678 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:01 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:55:01.361+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.296947+0000 front 2024-08-18T23:54:30.296810+0000 (oldest deadline 2024-08-18T23:54:54.396696+0000) 2024-08-18T23:55:01.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:01 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:01.497+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:01.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:01 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:01.497+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:01.969 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:01 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:01.757+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.255549+0000 front 2024-08-18T23:54:31.255738+0000 (oldest deadline 2024-08-18T23:54:55.955541+0000) 2024-08-18T23:55:01.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:01 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:01.587+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:01.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:01 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:01.587+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:01.975 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:01 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:01.679+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:02.180 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:01 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:01.890+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:02.309 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:01 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:01.973+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:02.309 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:01 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:01.973+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:02.677 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:02 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:02.487+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:02.677 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:02 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:02.487+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:02.678 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:02 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:55:02.403+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.695430+0000 front 2024-08-18T23:54:22.695540+0000 (oldest deadline 2024-08-18T23:54:50.296180+0000) 2024-08-18T23:55:02.678 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:02 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:55:02.403+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.296947+0000 front 2024-08-18T23:54:30.296810+0000 (oldest deadline 2024-08-18T23:54:54.396696+0000) 2024-08-18T23:55:02.969 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:02 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:02.724+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.255549+0000 front 2024-08-18T23:54:31.255738+0000 (oldest deadline 2024-08-18T23:54:55.955541+0000) 2024-08-18T23:55:02.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:02 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:02.582+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:02.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:02 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:02.582+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:03.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:02 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:02.713+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:03.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:02 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:02.956+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:03.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:02 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:02.956+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:03.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:02 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:02.889+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:03.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:02 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:02.889+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:03.677 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:03 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:03.448+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:03.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:03 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:03.448+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:03.678 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:03 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:55:03.398+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.695430+0000 front 2024-08-18T23:54:22.695540+0000 (oldest deadline 2024-08-18T23:54:50.296180+0000) 2024-08-18T23:55:03.678 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:03 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:55:03.398+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.296947+0000 front 2024-08-18T23:54:30.296810+0000 (oldest deadline 2024-08-18T23:54:54.396696+0000) 2024-08-18T23:55:03.969 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:03.739+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.255549+0000 front 2024-08-18T23:54:31.255738+0000 (oldest deadline 2024-08-18T23:54:55.955541+0000) 2024-08-18T23:55:03.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:03.625+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:03.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:03 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:03.625+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:03.991 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:03 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:03.734+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:04.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:03 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:03.865+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:04.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:03 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:03.865+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:04.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:03 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:03.990+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:04.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:03 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:03.990+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:04.677 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:04 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:55:04.381+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.695430+0000 front 2024-08-18T23:54:22.695540+0000 (oldest deadline 2024-08-18T23:54:50.296180+0000) 2024-08-18T23:55:04.677 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:04 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:55:04.381+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.296947+0000 front 2024-08-18T23:54:30.296810+0000 (oldest deadline 2024-08-18T23:54:54.396696+0000) 2024-08-18T23:55:04.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:04 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:04.497+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:04.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:04 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:04.497+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:04.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:04 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:04.497+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:04.969 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:04 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:04.727+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.255549+0000 front 2024-08-18T23:54:31.255738+0000 (oldest deadline 2024-08-18T23:54:55.955541+0000) 2024-08-18T23:55:04.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:04 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:04.637+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:04.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:04 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:04.637+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:05.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:04 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:04.753+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:05.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:04 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:04.826+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:05.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:04 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:04.826+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:05.280 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:05 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:05.006+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:05.280 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:05 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:05.006+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:05.280 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:05 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:05.006+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:05.677 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:05 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:55:05.350+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.695430+0000 front 2024-08-18T23:54:22.695540+0000 (oldest deadline 2024-08-18T23:54:50.296180+0000) 2024-08-18T23:55:05.678 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:05 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5[33385]: 2024-08-18T23:55:05.350+0000 7f50c86a1640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.296947+0000 front 2024-08-18T23:54:30.296810+0000 (oldest deadline 2024-08-18T23:54:54.396696+0000) 2024-08-18T23:55:05.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:05 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:05.539+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:05.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:05 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:05.539+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:05.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:05 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:05.539+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:05.969 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:05 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:05.714+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.255549+0000 front 2024-08-18T23:54:31.255738+0000 (oldest deadline 2024-08-18T23:54:55.955541+0000) 2024-08-18T23:55:05.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:05 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:05.597+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:05.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:05 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:05.597+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:05.994 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:05 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:05.710+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:05.994 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:05 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:05.710+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.051422+0000 front 2024-08-18T23:54:39.051405+0000 (oldest deadline 2024-08-18T23:55:04.951219+0000) 2024-08-18T23:55:06.140 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:05 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:05.816+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:06.140 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:05 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:05.816+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:06.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:05 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:05.992+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:06.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:05 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:05.992+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:06.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:05 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:05.992+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:06.427 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:06 smithi161 podman[57113]: 2024-08-18 23:55:06.139390329 +0000 UTC m=+5.690618579 container died 2f3699ab96e76441bbab2f86fa9d7b363cb7c31fb3f3269680a91d341c310f35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:55:06.806 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:06 smithi161 podman[57113]: 2024-08-18 23:55:06.520051727 +0000 UTC m=+6.071279966 container cleanup 2f3699ab96e76441bbab2f86fa9d7b363cb7c31fb3f3269680a91d341c310f35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:55:06.806 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:06 smithi161 bash[57113]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5 2024-08-18T23:55:06.806 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:06 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:06.495+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:06.806 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:06 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:06.495+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:06.806 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:06 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:06.495+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:06.969 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:06 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:06.694+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.255549+0000 front 2024-08-18T23:54:31.255738+0000 (oldest deadline 2024-08-18T23:54:55.955541+0000) 2024-08-18T23:55:06.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:06 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:06.638+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:06.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:06 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:06.638+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:06.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:06 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:06.638+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:07.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:06 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:06.750+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:07.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:06 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:06.750+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.051422+0000 front 2024-08-18T23:54:39.051405+0000 (oldest deadline 2024-08-18T23:55:04.951219+0000) 2024-08-18T23:55:07.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:06 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:06.943+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:07.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:06 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:06.943+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:07.049 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:06 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:06.943+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:07.150 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:06 smithi161 podman[57131]: 2024-08-18 23:55:06.805105967 +0000 UTC m=+0.666708276 container remove 2f3699ab96e76441bbab2f86fa9d7b363cb7c31fb3f3269680a91d341c310f35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-18T23:55:07.151 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:06 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:06.831+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:07.151 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:06 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:06.831+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:07.427 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:07 smithi161 podman[57220]: 2024-08-18 23:55:07.052757641 +0000 UTC m=+0.017187794 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:55:07.428 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:07 smithi161 podman[57220]: 2024-08-18 23:55:07.181069949 +0000 UTC m=+0.145500143 container create 875480652bcc1b037309e916d06a88428c32dc6913a877c8dfa10763fe554fdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5-deactivate, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:55:07.795 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:07 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:07.525+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:07.795 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:07 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:07.525+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:07.795 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:07 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:07.525+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:07.795 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:07 smithi161 podman[57220]: 2024-08-18 23:55:07.433395351 +0000 UTC m=+0.397825521 container init 875480652bcc1b037309e916d06a88428c32dc6913a877c8dfa10763fe554fdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-08-18T23:55:07.795 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:07 smithi161 podman[57220]: 2024-08-18 23:55:07.436752606 +0000 UTC m=+0.401182763 container start 875480652bcc1b037309e916d06a88428c32dc6913a877c8dfa10763fe554fdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-18T23:55:07.795 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:07 smithi161 podman[57220]: 2024-08-18 23:55:07.516784461 +0000 UTC m=+0.481214614 container attach 875480652bcc1b037309e916d06a88428c32dc6913a877c8dfa10763fe554fdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-18T23:55:07.795 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 18 23:55:07 smithi161 podman[57220]: 2024-08-18 23:55:07.632947365 +0000 UTC m=+0.597377525 container died 875480652bcc1b037309e916d06a88428c32dc6913a877c8dfa10763fe554fdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-08-18T23:55:07.970 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:07 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:07.666+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.255549+0000 front 2024-08-18T23:54:31.255738+0000 (oldest deadline 2024-08-18T23:54:55.955541+0000) 2024-08-18T23:55:07.970 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:07 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:07.666+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.157120+0000 front 2024-08-18T23:54:38.157102+0000 (oldest deadline 2024-08-18T23:55:07.557965+0000) 2024-08-18T23:55:07.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:07 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:07.681+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:07.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:07 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:07.681+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:07.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:07 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:07.681+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:08.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:07 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:07.720+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:08.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:07 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:07.720+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.051422+0000 front 2024-08-18T23:54:39.051405+0000 (oldest deadline 2024-08-18T23:55:04.951219+0000) 2024-08-18T23:55:08.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:07 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:07.905+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:08.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:07 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:07.905+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:08.049 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:07 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:07.905+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:08.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:07 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:07.793+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:08.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:07 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:07.793+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:08.297 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.5.service' 2024-08-18T23:55:08.335 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:55:08.335 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-08-18T23:55:08.335 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-08-18T23:55:08.336 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.6 2024-08-18T23:55:08.664 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:08 smithi078 systemd[1]: Stopping Ceph osd.6 for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:55:08.664 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:08 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:08.662+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:08.677 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:08 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:08.506+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:08.677 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:08 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:08.506+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:08.677 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:08 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:08.506+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:08.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:08 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:08.662+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:08.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:08 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:08.662+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:08.970 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:08 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:08.709+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.255549+0000 front 2024-08-18T23:54:31.255738+0000 (oldest deadline 2024-08-18T23:54:55.955541+0000) 2024-08-18T23:55:08.970 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:08 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:08.709+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.157120+0000 front 2024-08-18T23:54:38.157102+0000 (oldest deadline 2024-08-18T23:55:07.557965+0000) 2024-08-18T23:55:08.970 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:08 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:08.734+0000 7fc616ec9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:55:08.970 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:08 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:08.734+0000 7fc616ec9640 -1 osd.6 96 *** Got signal Terminated *** 2024-08-18T23:55:08.970 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:08 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:08.734+0000 7fc616ec9640 -1 osd.6 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-18T23:55:09.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:08 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:08.686+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:09.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:08 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:08.686+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.051422+0000 front 2024-08-18T23:54:39.051405+0000 (oldest deadline 2024-08-18T23:55:04.951219+0000) 2024-08-18T23:55:09.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:08 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:08.860+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:09.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:08 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:08.860+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:09.049 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:08 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:08.860+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:09.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:08 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:08.816+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:09.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:08 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:08.816+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:09.832 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:09 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:09.535+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:09.832 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:09 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:09.535+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:09.832 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:09 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:09.535+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:09.969 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:09 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:09.714+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.255549+0000 front 2024-08-18T23:54:31.255738+0000 (oldest deadline 2024-08-18T23:54:55.955541+0000) 2024-08-18T23:55:09.969 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:09 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:09.714+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.157120+0000 front 2024-08-18T23:54:38.157102+0000 (oldest deadline 2024-08-18T23:55:07.557965+0000) 2024-08-18T23:55:09.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:09 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:09.707+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:09.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:09 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:09.707+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:09.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:09 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:09.707+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:10.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:09 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:09.723+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:10.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:09 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:09.723+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.051422+0000 front 2024-08-18T23:54:39.051405+0000 (oldest deadline 2024-08-18T23:55:04.951219+0000) 2024-08-18T23:55:10.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:09 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:09.876+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:10.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:09 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:09.876+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:10.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:09 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:09.876+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:10.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:09 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:09.831+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:10.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:09 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:09.831+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:10.852 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:10 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:10.536+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:10.852 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:10 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:10.536+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:10.852 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:10 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:10.536+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:10.969 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:10 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:10.684+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.255549+0000 front 2024-08-18T23:54:31.255738+0000 (oldest deadline 2024-08-18T23:54:55.955541+0000) 2024-08-18T23:55:10.970 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:10 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:10.685+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.157120+0000 front 2024-08-18T23:54:38.157102+0000 (oldest deadline 2024-08-18T23:55:07.557965+0000) 2024-08-18T23:55:10.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:10 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:10.673+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:10.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:10 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:10.673+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:10.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:10 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:10.673+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:11.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:10 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:10.724+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:11.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:10 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:10.724+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.051422+0000 front 2024-08-18T23:54:39.051405+0000 (oldest deadline 2024-08-18T23:55:04.951219+0000) 2024-08-18T23:55:11.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:10 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:10.724+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:44.951799+0000 front 2024-08-18T23:54:44.951776+0000 (oldest deadline 2024-08-18T23:55:10.251613+0000) 2024-08-18T23:55:11.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:10 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:10.871+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:11.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:10 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:10.871+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:11.049 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:10 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:10.871+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:11.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:10 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:10.850+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:11.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:10 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:10.850+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:11.841 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:11 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:11.525+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:11.841 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:11 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:11.525+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:11.841 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:11 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:11.525+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:11.969 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:11 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:11.704+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.255549+0000 front 2024-08-18T23:54:31.255738+0000 (oldest deadline 2024-08-18T23:54:55.955541+0000) 2024-08-18T23:55:11.970 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:11 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:11.704+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.157120+0000 front 2024-08-18T23:54:38.157102+0000 (oldest deadline 2024-08-18T23:55:07.557965+0000) 2024-08-18T23:55:11.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:11 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:11.637+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:11.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:11 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:11.637+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:11.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:11 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:11.637+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:11.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:11 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:11.637+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:12.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:11 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:11.745+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:12.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:11 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:11.745+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.051422+0000 front 2024-08-18T23:54:39.051405+0000 (oldest deadline 2024-08-18T23:55:04.951219+0000) 2024-08-18T23:55:12.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:11 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:11.745+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:44.951799+0000 front 2024-08-18T23:54:44.951776+0000 (oldest deadline 2024-08-18T23:55:10.251613+0000) 2024-08-18T23:55:12.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:11 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:11.821+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:12.049 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:11 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:11.821+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:12.049 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:11 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:11.821+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:12.049 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:11 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:11.821+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:12.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:11 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:11.840+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:12.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:11 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:11.840+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:12.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:11 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:11.840+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:48.623510+0000 front 2024-08-18T23:54:48.623489+0000 (oldest deadline 2024-08-18T23:55:11.523369+0000) 2024-08-18T23:55:12.845 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:12 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:12.509+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:12.846 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:12 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:12.509+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:12.846 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:12 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:12.509+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:12.969 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:12 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:12.664+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.255549+0000 front 2024-08-18T23:54:31.255738+0000 (oldest deadline 2024-08-18T23:54:55.955541+0000) 2024-08-18T23:55:12.970 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:12 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:12.664+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.157120+0000 front 2024-08-18T23:54:38.157102+0000 (oldest deadline 2024-08-18T23:55:07.557965+0000) 2024-08-18T23:55:12.970 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:12 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:12.664+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.558298+0000 front 2024-08-18T23:54:47.558348+0000 (oldest deadline 2024-08-18T23:55:12.258436+0000) 2024-08-18T23:55:12.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:12 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:12.626+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:12.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:12 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:12.626+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:12.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:12 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:12.626+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:12.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:12 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:12.626+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:13.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:12 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:12.719+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:13.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:12 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:12.719+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.051422+0000 front 2024-08-18T23:54:39.051405+0000 (oldest deadline 2024-08-18T23:55:04.951219+0000) 2024-08-18T23:55:13.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:12 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:12.719+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:44.951799+0000 front 2024-08-18T23:54:44.951776+0000 (oldest deadline 2024-08-18T23:55:10.251613+0000) 2024-08-18T23:55:13.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:12 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:12.791+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:13.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:12 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:12.791+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:13.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:12 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:12.791+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:13.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:12 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:12.791+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:13.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:12 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:12.843+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:13.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:12 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:12.843+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:13.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:12 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:12.843+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:48.623510+0000 front 2024-08-18T23:54:48.623489+0000 (oldest deadline 2024-08-18T23:55:11.523369+0000) 2024-08-18T23:55:13.833 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:13 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:13.517+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:13.833 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:13 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:13.517+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:13.833 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:13 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:13.517+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:13.969 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:13 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:13.657+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.255549+0000 front 2024-08-18T23:54:31.255738+0000 (oldest deadline 2024-08-18T23:54:55.955541+0000) 2024-08-18T23:55:13.970 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:13 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:13.657+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.157120+0000 front 2024-08-18T23:54:38.157102+0000 (oldest deadline 2024-08-18T23:55:07.557965+0000) 2024-08-18T23:55:13.970 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:13 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6[45408]: 2024-08-18T23:55:13.657+0000 7fc60f4d7640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.558298+0000 front 2024-08-18T23:54:47.558348+0000 (oldest deadline 2024-08-18T23:55:12.258436+0000) 2024-08-18T23:55:13.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:13 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:13.648+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:13.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:13 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:13.648+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:13.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:13 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:13.648+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:13.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:13 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:13.648+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:14.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:13 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:13.671+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:14.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:13 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:13.671+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.051422+0000 front 2024-08-18T23:54:39.051405+0000 (oldest deadline 2024-08-18T23:55:04.951219+0000) 2024-08-18T23:55:14.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:13 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:13.671+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:44.951799+0000 front 2024-08-18T23:54:44.951776+0000 (oldest deadline 2024-08-18T23:55:10.251613+0000) 2024-08-18T23:55:14.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:13 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:13.762+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:14.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:13 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:13.762+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:14.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:13 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:13.762+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:14.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:13 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:13.762+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:14.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:13 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:13.831+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:14.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:13 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:13.831+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:14.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:13 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:13.831+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:48.623510+0000 front 2024-08-18T23:54:48.623489+0000 (oldest deadline 2024-08-18T23:55:11.523369+0000) 2024-08-18T23:55:14.386 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:14 smithi078 podman[97171]: 2024-08-18 23:55:14.094241037 +0000 UTC m=+5.556978142 container died ab330635ae8f9d9e3712e2d7e59096a3b8aa1d6b03d031398feb87294696bc10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-08-18T23:55:14.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:14 smithi078 podman[97171]: 2024-08-18 23:55:14.576720708 +0000 UTC m=+6.039457854 container cleanup ab330635ae8f9d9e3712e2d7e59096a3b8aa1d6b03d031398feb87294696bc10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-18T23:55:14.719 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:14 smithi078 bash[97171]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6 2024-08-18T23:55:14.720 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:14 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:14.618+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:14.720 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:14 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:14.618+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:14.720 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:14 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:14.618+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:14.720 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:14 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:14.618+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:14.867 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:14 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:14.553+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:14.867 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:14 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:14.553+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:14.867 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:14 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:14.553+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:15.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:14 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:14.705+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:15.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:14 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:14.705+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.051422+0000 front 2024-08-18T23:54:39.051405+0000 (oldest deadline 2024-08-18T23:55:04.951219+0000) 2024-08-18T23:55:15.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:14 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:14.705+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:44.951799+0000 front 2024-08-18T23:54:44.951776+0000 (oldest deadline 2024-08-18T23:55:10.251613+0000) 2024-08-18T23:55:15.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:14 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:14.745+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:15.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:14 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:14.745+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:15.049 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:14 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:14.745+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:15.049 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:14 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:14.745+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:15.141 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:14 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:14.865+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:15.142 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:14 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:14.865+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:15.142 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:14 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:14.865+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:48.623510+0000 front 2024-08-18T23:54:48.623489+0000 (oldest deadline 2024-08-18T23:55:11.523369+0000) 2024-08-18T23:55:15.170 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:14 smithi078 podman[97189]: 2024-08-18 23:55:14.848316253 +0000 UTC m=+0.755914363 container remove ab330635ae8f9d9e3712e2d7e59096a3b8aa1d6b03d031398feb87294696bc10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-de09dd0) 2024-08-18T23:55:15.449 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:15 smithi078 podman[97277]: 2024-08-18 23:55:15.072433944 +0000 UTC m=+0.017552692 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:55:15.449 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:15 smithi078 podman[97277]: 2024-08-18 23:55:15.196018837 +0000 UTC m=+0.141137573 container create d5b4096a26d2a3914dcba9b93e1e051ebaa5b8d6fb900f7a7cef431eaa98d44e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-de09dd0) 2024-08-18T23:55:15.449 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:15 smithi078 podman[97277]: 2024-08-18 23:55:15.448461109 +0000 UTC m=+0.393579848 container init d5b4096a26d2a3914dcba9b93e1e051ebaa5b8d6fb900f7a7cef431eaa98d44e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-08-18T23:55:15.687 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:15 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:15.684+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:15.687 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:15 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:15.684+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.051422+0000 front 2024-08-18T23:54:39.051405+0000 (oldest deadline 2024-08-18T23:55:04.951219+0000) 2024-08-18T23:55:15.687 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:15 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:15.684+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:44.951799+0000 front 2024-08-18T23:54:44.951776+0000 (oldest deadline 2024-08-18T23:55:10.251613+0000) 2024-08-18T23:55:15.720 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:15 smithi078 podman[97277]: 2024-08-18 23:55:15.45204627 +0000 UTC m=+0.397165007 container start d5b4096a26d2a3914dcba9b93e1e051ebaa5b8d6fb900f7a7cef431eaa98d44e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6-deactivate, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-08-18T23:55:15.720 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:15 smithi078 podman[97277]: 2024-08-18 23:55:15.565215588 +0000 UTC m=+0.510334339 container attach d5b4096a26d2a3914dcba9b93e1e051ebaa5b8d6fb900f7a7cef431eaa98d44e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0) 2024-08-18T23:55:15.720 INFO:journalctl@ceph.osd.6.smithi078.stdout:Aug 18 23:55:15 smithi078 podman[97277]: 2024-08-18 23:55:15.646026498 +0000 UTC m=+0.591145233 container died d5b4096a26d2a3914dcba9b93e1e051ebaa5b8d6fb900f7a7cef431eaa98d44e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-18T23:55:15.720 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:15 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:15.595+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:15.720 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:15 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:15.596+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:15.720 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:15 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:15.596+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:15.720 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:15 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:15.596+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:15.856 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:15 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:15.580+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:15.857 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:15 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:15.580+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:15.857 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:15 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:15.580+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:16.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:15 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:15.769+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:16.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:15 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:15.769+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:16.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:15 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:15.769+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:16.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:15 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:15.769+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:16.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:15 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:15.854+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:16.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:15 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:15.854+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:16.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:15 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:15.854+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:48.623510+0000 front 2024-08-18T23:54:48.623489+0000 (oldest deadline 2024-08-18T23:55:11.523369+0000) 2024-08-18T23:55:16.363 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.6.service' 2024-08-18T23:55:16.403 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:55:16.403 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-08-18T23:55:16.403 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-08-18T23:55:16.403 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.7 2024-08-18T23:55:16.548 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:16 smithi123 systemd[1]: Stopping Ceph osd.7 for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:55:16.865 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:16 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:16.608+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:16.865 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:16 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:16.608+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:16.865 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:16 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:16.608+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:16.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:16 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:16.597+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:16.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:16 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:16.597+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:16.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:16 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:16.597+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:16.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:16 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:16.597+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:17.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:16 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:16.776+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:17.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:16 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:16.776+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:17.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:16 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:16.776+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:17.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:16 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:16.776+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:17.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:16 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:16.713+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:17.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:16 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:16.713+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.051422+0000 front 2024-08-18T23:54:39.051405+0000 (oldest deadline 2024-08-18T23:55:04.951219+0000) 2024-08-18T23:55:17.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:16 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:16.713+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:44.951799+0000 front 2024-08-18T23:54:44.951776+0000 (oldest deadline 2024-08-18T23:55:10.251613+0000) 2024-08-18T23:55:17.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:16 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:16.727+0000 7ff1d5441640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:55:17.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:16 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:16.727+0000 7ff1d5441640 -1 osd.7 96 *** Got signal Terminated *** 2024-08-18T23:55:17.049 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:16 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:16.727+0000 7ff1d5441640 -1 osd.7 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-18T23:55:17.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:16 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:16.862+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:17.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:16 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:16.862+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:17.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:16 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:16.863+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:48.623510+0000 front 2024-08-18T23:54:48.623489+0000 (oldest deadline 2024-08-18T23:55:11.523369+0000) 2024-08-18T23:55:17.927 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:17 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:17.823+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:17.928 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:17 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:17.823+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:17.928 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:17 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:17.823+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:48.623510+0000 front 2024-08-18T23:54:48.623489+0000 (oldest deadline 2024-08-18T23:55:11.523369+0000) 2024-08-18T23:55:17.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:17 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:17.651+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:17.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:17 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:17.651+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:17.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:17 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:17.651+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:17.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:17 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:17.575+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:17.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:17 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:17.575+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:17.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:17 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:17.575+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:17.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:17 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:17.575+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:18.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:17 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:17.702+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:18.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:17 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:17.702+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.051422+0000 front 2024-08-18T23:54:39.051405+0000 (oldest deadline 2024-08-18T23:55:04.951219+0000) 2024-08-18T23:55:18.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:17 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:17.702+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:44.951799+0000 front 2024-08-18T23:54:44.951776+0000 (oldest deadline 2024-08-18T23:55:10.251613+0000) 2024-08-18T23:55:18.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:17 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:17.787+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:18.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:17 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:17.787+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:18.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:17 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:17.787+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:18.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:17 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:17.787+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:18.927 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:18 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:18.854+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:18.928 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:18 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:18.854+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:18.928 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:18 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:18.854+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:48.623510+0000 front 2024-08-18T23:54:48.623489+0000 (oldest deadline 2024-08-18T23:55:11.523369+0000) 2024-08-18T23:55:18.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:18 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:18.623+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:18.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:18 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:18.623+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:18.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:18 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:18.623+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:18.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:18 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:18.621+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:18.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:18 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:18.621+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:18.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:18 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:18.621+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:18.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:18 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:18.621+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:19.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:18 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:18.738+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:19.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:18 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:18.738+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.051422+0000 front 2024-08-18T23:54:39.051405+0000 (oldest deadline 2024-08-18T23:55:04.951219+0000) 2024-08-18T23:55:19.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:18 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:18.738+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:44.951799+0000 front 2024-08-18T23:54:44.951776+0000 (oldest deadline 2024-08-18T23:55:10.251613+0000) 2024-08-18T23:55:19.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:18 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:18.801+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:19.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:18 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:18.801+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:19.049 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:18 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:18.801+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:19.049 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:18 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:18.801+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:19.927 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:19 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:19.814+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:19.928 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:19 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:19.814+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:19.928 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:19 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:19.814+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:48.623510+0000 front 2024-08-18T23:54:48.623489+0000 (oldest deadline 2024-08-18T23:55:11.523369+0000) 2024-08-18T23:55:19.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:19 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:19.583+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:19.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:19 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:19.583+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:19.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:19 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:19.583+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:19.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:19 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:19.613+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:19.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:19 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:19.613+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:19.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:19 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:19.613+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:19.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:19 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:19.613+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:20.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:19 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:19.752+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:20.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:19 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:19.752+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.051422+0000 front 2024-08-18T23:54:39.051405+0000 (oldest deadline 2024-08-18T23:55:04.951219+0000) 2024-08-18T23:55:20.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:19 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:19.752+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:44.951799+0000 front 2024-08-18T23:54:44.951776+0000 (oldest deadline 2024-08-18T23:55:10.251613+0000) 2024-08-18T23:55:20.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:19 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:19.752+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.852739+0000 front 2024-08-18T23:54:54.852774+0000 (oldest deadline 2024-08-18T23:55:19.552891+0000) 2024-08-18T23:55:20.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:19 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:19.828+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:20.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:19 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:19.828+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:20.049 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:19 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:19.828+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:20.049 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:19 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:19.828+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:20.049 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:19 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:19.828+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:20.927 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:20 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:20.798+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:20.928 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:20 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:20.798+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:20.928 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:20 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:20.798+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:48.623510+0000 front 2024-08-18T23:54:48.623489+0000 (oldest deadline 2024-08-18T23:55:11.523369+0000) 2024-08-18T23:55:20.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:20 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:20.556+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:20.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:20 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:20.556+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:20.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:20 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:20.556+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:20.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:20 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:20.603+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:20.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:20 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:20.603+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:20.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:20 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:20.603+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:20.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:20 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:20.603+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:20.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:20 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:20.603+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:57.183539+0000 front 2024-08-18T23:54:57.183628+0000 (oldest deadline 2024-08-18T23:55:20.083621+0000) 2024-08-18T23:55:21.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:20 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:20.770+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:31.549872+0000 front 2024-08-18T23:54:31.549931+0000 (oldest deadline 2024-08-18T23:54:55.050002+0000) 2024-08-18T23:55:21.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:20 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:20.770+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.051422+0000 front 2024-08-18T23:54:39.051405+0000 (oldest deadline 2024-08-18T23:55:04.951219+0000) 2024-08-18T23:55:21.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:20 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:20.770+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:44.951799+0000 front 2024-08-18T23:54:44.951776+0000 (oldest deadline 2024-08-18T23:55:10.251613+0000) 2024-08-18T23:55:21.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:20 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7[39567]: 2024-08-18T23:55:20.770+0000 7ff1cda4f640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.852739+0000 front 2024-08-18T23:54:54.852774+0000 (oldest deadline 2024-08-18T23:55:19.552891+0000) 2024-08-18T23:55:21.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:20 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:20.846+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:21.049 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:20 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:20.846+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:21.049 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:20 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:20.846+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:21.049 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:20 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:20.846+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:21.049 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:20 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:20.846+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:21.927 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:21 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:21.780+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:21.927 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:21 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:21.780+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:21.928 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:21 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:21.780+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:48.623510+0000 front 2024-08-18T23:54:48.623489+0000 (oldest deadline 2024-08-18T23:55:11.523369+0000) 2024-08-18T23:55:21.928 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:21 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:21.780+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:55.624189+0000 front 2024-08-18T23:54:55.624246+0000 (oldest deadline 2024-08-18T23:55:20.924260+0000) 2024-08-18T23:55:21.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:21 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:21.550+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:21.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:21 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:21.550+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:21.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:21 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:21.550+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:21.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:21 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:21.550+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:21.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:21 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:21.634+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:21.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:21 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:21.634+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:21.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:21 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:21.634+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:21.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:21 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:21.634+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:21.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:21 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:21.634+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:57.183539+0000 front 2024-08-18T23:54:57.183628+0000 (oldest deadline 2024-08-18T23:55:20.083621+0000) 2024-08-18T23:55:22.298 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:22 smithi123 podman[56757]: 2024-08-18 23:55:22.093913748 +0000 UTC m=+5.496041552 container died 83190ee7d0f42f543213f4ab0497559f0e6effee6798536d2a587c0fa58c13fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD) 2024-08-18T23:55:22.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:21 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:21.863+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:22.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:21 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:21.863+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:22.299 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:21 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:21.863+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:22.299 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:21 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:21.863+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:22.299 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:21 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:21.863+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:22.729 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:22 smithi123 podman[56757]: 2024-08-18 23:55:22.468993771 +0000 UTC m=+5.871121564 container cleanup 83190ee7d0f42f543213f4ab0497559f0e6effee6798536d2a587c0fa58c13fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0) 2024-08-18T23:55:22.729 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:22 smithi123 bash[56757]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7 2024-08-18T23:55:22.797 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:22 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:22.519+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:22.798 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:22 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:22.519+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:22.798 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:22 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:22.519+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:22.798 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:22 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:22.519+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:22.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:22 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:22.673+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:22.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:22 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:22.673+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:22.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:22 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:22.673+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:22.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:22 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:22.673+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:22.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:22 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:22.673+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:57.183539+0000 front 2024-08-18T23:54:57.183628+0000 (oldest deadline 2024-08-18T23:55:20.083621+0000) 2024-08-18T23:55:23.048 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:22 smithi123 podman[56771]: 2024-08-18 23:55:22.728620419 +0000 UTC m=+0.635528023 container remove 83190ee7d0f42f543213f4ab0497559f0e6effee6798536d2a587c0fa58c13fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0) 2024-08-18T23:55:23.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:22 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:22.872+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:23.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:22 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:22.872+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:23.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:22 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:22.872+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:23.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:22 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:22.872+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:23.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:22 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:22.872+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:23.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:22 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:22.796+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:23.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:22 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:22.796+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:23.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:22 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:22.796+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:48.623510+0000 front 2024-08-18T23:54:48.623489+0000 (oldest deadline 2024-08-18T23:55:11.523369+0000) 2024-08-18T23:55:23.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:22 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:22.796+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:55.624189+0000 front 2024-08-18T23:54:55.624246+0000 (oldest deadline 2024-08-18T23:55:20.924260+0000) 2024-08-18T23:55:23.349 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:23 smithi123 podman[56860]: 2024-08-18 23:55:22.951758845 +0000 UTC m=+0.017126451 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:55:23.349 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:23 smithi123 podman[56860]: 2024-08-18 23:55:23.054463313 +0000 UTC m=+0.119830910 container create 35e12cd1a90754e85821e55eb6aaa53f6bc8a332ba3c29a0135e9b8d8770230d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-08-18T23:55:23.349 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:23 smithi123 podman[56860]: 2024-08-18 23:55:23.348856769 +0000 UTC m=+0.414224383 container init 35e12cd1a90754e85821e55eb6aaa53f6bc8a332ba3c29a0135e9b8d8770230d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS) 2024-08-18T23:55:23.798 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:23 smithi123 podman[56860]: 2024-08-18 23:55:23.352449861 +0000 UTC m=+0.417817456 container start 35e12cd1a90754e85821e55eb6aaa53f6bc8a332ba3c29a0135e9b8d8770230d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.36.0) 2024-08-18T23:55:23.798 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:23 smithi123 podman[56860]: 2024-08-18 23:55:23.440320023 +0000 UTC m=+0.505687632 container attach 35e12cd1a90754e85821e55eb6aaa53f6bc8a332ba3c29a0135e9b8d8770230d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-08-18T23:55:23.798 INFO:journalctl@ceph.osd.7.smithi123.stdout:Aug 18 23:55:23 smithi123 podman[56860]: 2024-08-18 23:55:23.557102488 +0000 UTC m=+0.622470092 container died 35e12cd1a90754e85821e55eb6aaa53f6bc8a332ba3c29a0135e9b8d8770230d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-18T23:55:23.819 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:23 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:23.501+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:23.819 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:23 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:23.501+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:23.819 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:23 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:23.501+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:23.820 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:23 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:23.501+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:23.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:23.670+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:23.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:23.670+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:23.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:23.670+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:23.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:23.670+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:23.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:23 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:23.670+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:57.183539+0000 front 2024-08-18T23:54:57.183628+0000 (oldest deadline 2024-08-18T23:55:20.083621+0000) 2024-08-18T23:55:24.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:23 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:23.818+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:24.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:23 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:23.818+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:24.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:23 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:23.818+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:48.623510+0000 front 2024-08-18T23:54:48.623489+0000 (oldest deadline 2024-08-18T23:55:11.523369+0000) 2024-08-18T23:55:24.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:23 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:23.818+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:55.624189+0000 front 2024-08-18T23:54:55.624246+0000 (oldest deadline 2024-08-18T23:55:20.924260+0000) 2024-08-18T23:55:24.178 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:23 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:23.859+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:24.178 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:23 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:23.859+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:24.178 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:23 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:23.859+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:24.178 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:23 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:23.859+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:24.178 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:23 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:23.859+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:24.200 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.7.service' 2024-08-18T23:55:24.237 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:55:24.238 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-08-18T23:55:24.238 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-08-18T23:55:24.238 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.8 2024-08-18T23:55:24.547 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:24 smithi161 systemd[1]: Stopping Ceph osd.8 for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:55:24.807 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:24 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:24.545+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:24.807 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:24 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:24.545+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:24.808 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:24 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:24.545+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:24.808 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:24 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:24.545+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:24.808 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:24 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:24.714+0000 7f512a88d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:55:24.808 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:24 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:24.714+0000 7f512a88d640 -1 osd.8 96 *** Got signal Terminated *** 2024-08-18T23:55:24.808 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:24 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:24.714+0000 7f512a88d640 -1 osd.8 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-18T23:55:24.808 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:24 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:24.806+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:24.808 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:24 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:24.806+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:24.808 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:24 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:24.806+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:48.623510+0000 front 2024-08-18T23:54:48.623489+0000 (oldest deadline 2024-08-18T23:55:11.523369+0000) 2024-08-18T23:55:24.808 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:24 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:24.806+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:55.624189+0000 front 2024-08-18T23:54:55.624246+0000 (oldest deadline 2024-08-18T23:55:20.924260+0000) 2024-08-18T23:55:24.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:24 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:24.716+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:24.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:24 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:24.716+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:24.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:24 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:24.716+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:24.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:24 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:24.716+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:24.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:24 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:24.716+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:57.183539+0000 front 2024-08-18T23:54:57.183628+0000 (oldest deadline 2024-08-18T23:55:20.083621+0000) 2024-08-18T23:55:25.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:24 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:24.882+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:25.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:24 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:24.882+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:25.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:24 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:24.882+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:25.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:24 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:24.882+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:25.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:24 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:24.882+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:25.774 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:25 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:25.506+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:25.775 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:25 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:25.506+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:25.775 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:25 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:25.506+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:25.775 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:25 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:25.506+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:26.127 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:25 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:25.773+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:26.127 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:25 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:25.773+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:26.127 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:25 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:25.773+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:48.623510+0000 front 2024-08-18T23:54:48.623489+0000 (oldest deadline 2024-08-18T23:55:11.523369+0000) 2024-08-18T23:55:26.127 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:25 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:25.773+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:55.624189+0000 front 2024-08-18T23:54:55.624246+0000 (oldest deadline 2024-08-18T23:55:20.924260+0000) 2024-08-18T23:55:26.219 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:25 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:25.727+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:26.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:25 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:25.727+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:26.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:25 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:25.727+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:26.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:25 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:25.727+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:26.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:25 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:25.727+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:57.183539+0000 front 2024-08-18T23:54:57.183628+0000 (oldest deadline 2024-08-18T23:55:20.083621+0000) 2024-08-18T23:55:26.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:25 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:25.881+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:26.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:25 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:25.881+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:26.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:25 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:25.881+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:26.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:25 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:25.881+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:26.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:25 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:25.881+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:26.928 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:26 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:26.765+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:26.928 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:26 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:26.765+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:26.928 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:26 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:26.765+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:48.623510+0000 front 2024-08-18T23:54:48.623489+0000 (oldest deadline 2024-08-18T23:55:11.523369+0000) 2024-08-18T23:55:26.928 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:26 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:26.765+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:55.624189+0000 front 2024-08-18T23:54:55.624246+0000 (oldest deadline 2024-08-18T23:55:20.924260+0000) 2024-08-18T23:55:26.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:26 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:26.537+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:26.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:26 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:26.537+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:26.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:26 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:26.537+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:26.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:26 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:26.537+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:26.929 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:26 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:26.537+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:27.219 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:26 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:26.734+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:27.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:26 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:26.734+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:27.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:26 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:26.734+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:27.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:26 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:26.734+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:27.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:26 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:26.734+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:57.183539+0000 front 2024-08-18T23:54:57.183628+0000 (oldest deadline 2024-08-18T23:55:20.083621+0000) 2024-08-18T23:55:27.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:26 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:26.878+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:27.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:26 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:26.878+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:27.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:26 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:26.878+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:27.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:26 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:26.878+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:27.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:26 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:26.878+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:27.812 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:27 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:27.503+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:27.812 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:27 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:27.503+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:27.812 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:27 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:27.503+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:27.812 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:27 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:27.503+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:27.812 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:27 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:27.503+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:27.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:27 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:27.737+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:27.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:27 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:27.737+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:27.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:27 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:27.737+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:27.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:27 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:27.737+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:27.971 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:27 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:27.737+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:57.183539+0000 front 2024-08-18T23:54:57.183628+0000 (oldest deadline 2024-08-18T23:55:20.083621+0000) 2024-08-18T23:55:28.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:27 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:27.811+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:28.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:27 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:27.811+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:28.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:27 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:27.811+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:48.623510+0000 front 2024-08-18T23:54:48.623489+0000 (oldest deadline 2024-08-18T23:55:11.523369+0000) 2024-08-18T23:55:28.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:27 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:27.811+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:55.624189+0000 front 2024-08-18T23:54:55.624246+0000 (oldest deadline 2024-08-18T23:55:20.924260+0000) 2024-08-18T23:55:28.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:27 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:27.811+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:00.924539+0000 front 2024-08-18T23:55:00.924556+0000 (oldest deadline 2024-08-18T23:55:26.824640+0000) 2024-08-18T23:55:28.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:27 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:27.906+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:28.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:27 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:27.906+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:28.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:27 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:27.906+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:28.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:27 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:27.906+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:28.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:27 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:27.906+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:28.846 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:28 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:28.553+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:28.846 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:28 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:28.553+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:28.846 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:28 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:28.553+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:28.846 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:28 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:28.553+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:28.846 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:28 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:28.553+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:29.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:28 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:28.844+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.518761+0000 front 2024-08-18T23:54:22.518689+0000 (oldest deadline 2024-08-18T23:54:47.818513+0000) 2024-08-18T23:55:29.177 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:28 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:28.844+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:38.920644+0000 front 2024-08-18T23:54:38.920757+0000 (oldest deadline 2024-08-18T23:55:02.420885+0000) 2024-08-18T23:55:29.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:28 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:28.844+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:48.623510+0000 front 2024-08-18T23:54:48.623489+0000 (oldest deadline 2024-08-18T23:55:11.523369+0000) 2024-08-18T23:55:29.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:28 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:28.844+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:55.624189+0000 front 2024-08-18T23:54:55.624246+0000 (oldest deadline 2024-08-18T23:55:20.924260+0000) 2024-08-18T23:55:29.178 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:28 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8[40264]: 2024-08-18T23:55:28.844+0000 7f5126ea3640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:00.924539+0000 front 2024-08-18T23:55:00.924556+0000 (oldest deadline 2024-08-18T23:55:26.824640+0000) 2024-08-18T23:55:29.219 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:28 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:28.780+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:29.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:28 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:28.780+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:29.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:28 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:28.780+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:29.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:28 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:28.780+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:29.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:28 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:28.780+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:57.183539+0000 front 2024-08-18T23:54:57.183628+0000 (oldest deadline 2024-08-18T23:55:20.083621+0000) 2024-08-18T23:55:29.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:28 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:28.949+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:29.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:28 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:28.949+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:29.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:28 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:28.949+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:29.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:28 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:28.949+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:29.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:28 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:28.949+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:29.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:28 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:28.949+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:03.198457+0000 front 2024-08-18T23:55:03.198513+0000 (oldest deadline 2024-08-18T23:55:28.498301+0000) 2024-08-18T23:55:29.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:29 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:29.599+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:29.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:29 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:29.599+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:29.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:29 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:29.599+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:29.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:29 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:29.599+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:29.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:29 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:29.599+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:30.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:29 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:29.789+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:30.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:29 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:29.789+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:30.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:29 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:29.789+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:30.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:29 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:29.789+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:30.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:29 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:29.789+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:57.183539+0000 front 2024-08-18T23:54:57.183628+0000 (oldest deadline 2024-08-18T23:55:20.083621+0000) 2024-08-18T23:55:30.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:29 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:29.918+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:30.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:29 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:29.918+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:30.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:29 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:29.918+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:30.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:29 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:29.918+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:30.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:29 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:29.918+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:30.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:29 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:29.918+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:03.198457+0000 front 2024-08-18T23:55:03.198513+0000 (oldest deadline 2024-08-18T23:55:28.498301+0000) 2024-08-18T23:55:30.349 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:30 smithi161 podman[57335]: 2024-08-18 23:55:30.076970962 +0000 UTC m=+5.631311353 container died e380cf659ff7914ea2008107dd68447c8e0a9122491f5ecdd469d7a9d45f991c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, GIT_CLEAN=True) 2024-08-18T23:55:30.677 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:30 smithi161 podman[57335]: 2024-08-18 23:55:30.464113755 +0000 UTC m=+6.018454146 container cleanup e380cf659ff7914ea2008107dd68447c8e0a9122491f5ecdd469d7a9d45f991c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-08-18T23:55:30.678 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:30 smithi161 bash[57335]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8 2024-08-18T23:55:30.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:30 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:30.562+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:30.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:30 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:30.562+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:30.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:30 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:30.562+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:30.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:30 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:30.562+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:30.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:30 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:30.562+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:31.050 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:30 smithi161 podman[57350]: 2024-08-18 23:55:30.706936712 +0000 UTC m=+0.623998551 container remove e380cf659ff7914ea2008107dd68447c8e0a9122491f5ecdd469d7a9d45f991c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0) 2024-08-18T23:55:31.219 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:30 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:30.740+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:31.219 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:30 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:30.740+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:31.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:30 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:30.740+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:31.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:30 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:30.740+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:31.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:30 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:30.740+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:57.183539+0000 front 2024-08-18T23:54:57.183628+0000 (oldest deadline 2024-08-18T23:55:20.083621+0000) 2024-08-18T23:55:31.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:30 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:30.740+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:04.785028+0000 front 2024-08-18T23:55:04.784999+0000 (oldest deadline 2024-08-18T23:55:30.684867+0000) 2024-08-18T23:55:31.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:30 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:30.892+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:31.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:30 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:30.892+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:31.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:30 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:30.892+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:31.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:30 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:30.892+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:31.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:30 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:30.892+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:31.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:30 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:30.892+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:03.198457+0000 front 2024-08-18T23:55:03.198513+0000 (oldest deadline 2024-08-18T23:55:28.498301+0000) 2024-08-18T23:55:31.352 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:31 smithi161 podman[57440]: 2024-08-18 23:55:30.952768811 +0000 UTC m=+0.017593407 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:55:31.353 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:31 smithi161 podman[57440]: 2024-08-18 23:55:31.074648855 +0000 UTC m=+0.139473488 container create 4705669ef121ffab7bbae16bdfd994f37e398fdc415cb9f05275f3b721114d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-08-18T23:55:31.353 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:31 smithi161 podman[57440]: 2024-08-18 23:55:31.243340115 +0000 UTC m=+0.308164711 container init 4705669ef121ffab7bbae16bdfd994f37e398fdc415cb9f05275f3b721114d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-08-18T23:55:31.353 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:31 smithi161 podman[57440]: 2024-08-18 23:55:31.24662104 +0000 UTC m=+0.311445632 container start 4705669ef121ffab7bbae16bdfd994f37e398fdc415cb9f05275f3b721114d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-18T23:55:31.353 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:31 smithi161 podman[57440]: 2024-08-18 23:55:31.351945028 +0000 UTC m=+0.416769617 container attach 4705669ef121ffab7bbae16bdfd994f37e398fdc415cb9f05275f3b721114d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8-deactivate, ceph=True, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:55:31.603 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 18 23:55:31 smithi161 podman[57440]: 2024-08-18 23:55:31.43778038 +0000 UTC m=+0.502604976 container died 4705669ef121ffab7bbae16bdfd994f37e398fdc415cb9f05275f3b721114d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, ceph=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-08-18T23:55:31.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:31 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:31.601+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:31.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:31 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:31.601+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:31.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:31 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:31.601+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:31.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:31 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:31.601+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:31.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:31 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:31.601+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:31.953 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.8.service' 2024-08-18T23:55:31.991 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:55:31.991 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-08-18T23:55:31.991 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-08-18T23:55:31.991 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.9 2024-08-18T23:55:32.019 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:31 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:31.759+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:32.019 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:31 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:31.759+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:32.019 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:31 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:31.759+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:32.019 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:31 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:31.759+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:32.020 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:31 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:31.759+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:57.183539+0000 front 2024-08-18T23:54:57.183628+0000 (oldest deadline 2024-08-18T23:55:20.083621+0000) 2024-08-18T23:55:32.020 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:31 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:31.759+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:04.785028+0000 front 2024-08-18T23:55:04.784999+0000 (oldest deadline 2024-08-18T23:55:30.684867+0000) 2024-08-18T23:55:32.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:31 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:31.844+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:32.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:31 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:31.844+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:32.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:31 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:31.844+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:32.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:31 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:31.844+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:32.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:31 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:31.844+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:32.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:31 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:31.844+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:03.198457+0000 front 2024-08-18T23:55:03.198513+0000 (oldest deadline 2024-08-18T23:55:28.498301+0000) 2024-08-18T23:55:32.348 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:32 smithi078 systemd[1]: Stopping Ceph osd.9 for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:55:32.719 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:32 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:32.346+0000 7ffa8ea0f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:55:32.720 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:32 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:32.346+0000 7ffa8ea0f640 -1 osd.9 96 *** Got signal Terminated *** 2024-08-18T23:55:32.720 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:32 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:32.346+0000 7ffa8ea0f640 -1 osd.9 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-18T23:55:32.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:32 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:32.556+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:32.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:32 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:32.556+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:32.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:32 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:32.556+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:32.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:32 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:32.556+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:32.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:32 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:32.556+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:33.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:32 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:32.734+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:33.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:32 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:32.734+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:33.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:32 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:32.734+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:33.221 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:32 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:32.734+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:33.221 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:32 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:32.734+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:57.183539+0000 front 2024-08-18T23:54:57.183628+0000 (oldest deadline 2024-08-18T23:55:20.083621+0000) 2024-08-18T23:55:33.221 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:32 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:32.734+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:04.785028+0000 front 2024-08-18T23:55:04.784999+0000 (oldest deadline 2024-08-18T23:55:30.684867+0000) 2024-08-18T23:55:33.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:32 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:32.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:33.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:32 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:32.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:33.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:32 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:32.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:33.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:32 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:32.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:33.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:32 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:32.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:33.299 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:32 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:32.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:03.198457+0000 front 2024-08-18T23:55:03.198513+0000 (oldest deadline 2024-08-18T23:55:28.498301+0000) 2024-08-18T23:55:33.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:33 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:33.568+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:33.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:33 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:33.568+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:33.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:33 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:33.568+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:33.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:33 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:33.568+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:33.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:33 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:33.568+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:33.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:33 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:33.715+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:33.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:33 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:33.715+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:33.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:33 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:33.715+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:33.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:33 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:33.715+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:33.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:33 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:33.715+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:57.183539+0000 front 2024-08-18T23:54:57.183628+0000 (oldest deadline 2024-08-18T23:55:20.083621+0000) 2024-08-18T23:55:33.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:33 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:33.715+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:04.785028+0000 front 2024-08-18T23:55:04.784999+0000 (oldest deadline 2024-08-18T23:55:30.684867+0000) 2024-08-18T23:55:34.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:33 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:33.827+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:34.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:33 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:33.827+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:34.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:33 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:33.827+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:34.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:33 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:33.827+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:34.299 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:33 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:33.827+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:34.299 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:33 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:33.827+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:03.198457+0000 front 2024-08-18T23:55:03.198513+0000 (oldest deadline 2024-08-18T23:55:28.498301+0000) 2024-08-18T23:55:34.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:34 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:34.527+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:34.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:34 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:34.527+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:34.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:34 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:34.527+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:34.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:34 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:34.527+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:34.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:34 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:34.527+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:34.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:34 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:34.527+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.530354+0000 front 2024-08-18T23:55:11.530411+0000 (oldest deadline 2024-08-18T23:55:34.430333+0000) 2024-08-18T23:55:34.969 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:34 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:34.687+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:34.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:34 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:34.687+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:34.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:34 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:34.687+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:34.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:34 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:34.687+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:34.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:34 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:34.687+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:57.183539+0000 front 2024-08-18T23:54:57.183628+0000 (oldest deadline 2024-08-18T23:55:20.083621+0000) 2024-08-18T23:55:34.970 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:34 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:34.687+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:04.785028+0000 front 2024-08-18T23:55:04.784999+0000 (oldest deadline 2024-08-18T23:55:30.684867+0000) 2024-08-18T23:55:35.199 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:34 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:34.828+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:35.199 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:34 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:34.828+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:35.199 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:34 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:34.828+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:35.200 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:34 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:34.828+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:35.200 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:34 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:34.828+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:35.200 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:34 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:34.828+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:03.198457+0000 front 2024-08-18T23:55:03.198513+0000 (oldest deadline 2024-08-18T23:55:28.498301+0000) 2024-08-18T23:55:35.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:35 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:35.480+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:35.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:35 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:35.480+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:35.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:35 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:35.480+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:35.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:35 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:35.480+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:35.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:35 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:35.480+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:35.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:35 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:35.480+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.530354+0000 front 2024-08-18T23:55:11.530411+0000 (oldest deadline 2024-08-18T23:55:34.430333+0000) 2024-08-18T23:55:36.219 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:35 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:35.724+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:36.219 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:35 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:35.724+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:36.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:35 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:35.724+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:36.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:35 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:35.724+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:36.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:35 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:35.724+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:57.183539+0000 front 2024-08-18T23:54:57.183628+0000 (oldest deadline 2024-08-18T23:55:20.083621+0000) 2024-08-18T23:55:36.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:35 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:35.724+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:04.785028+0000 front 2024-08-18T23:55:04.784999+0000 (oldest deadline 2024-08-18T23:55:30.684867+0000) 2024-08-18T23:55:36.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:35 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:35.864+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:36.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:35 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:35.864+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:36.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:35 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:35.864+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:36.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:35 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:35.864+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:36.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:35 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:35.864+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:36.299 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:35 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:35.864+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:03.198457+0000 front 2024-08-18T23:55:03.198513+0000 (oldest deadline 2024-08-18T23:55:28.498301+0000) 2024-08-18T23:55:36.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:36 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:36.521+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:36.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:36 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:36.521+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:36.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:36 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:36.521+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:36.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:36 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:36.521+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:36.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:36 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:36.521+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:36.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:36 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:36.521+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.530354+0000 front 2024-08-18T23:55:11.530411+0000 (oldest deadline 2024-08-18T23:55:34.430333+0000) 2024-08-18T23:55:37.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:36 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:36.755+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:24.378701+0000 front 2024-08-18T23:54:24.378752+0000 (oldest deadline 2024-08-18T23:54:47.878820+0000) 2024-08-18T23:55:37.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:36 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:36.755+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:30.779942+0000 front 2024-08-18T23:54:30.779913+0000 (oldest deadline 2024-08-18T23:54:56.680104+0000) 2024-08-18T23:55:37.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:36 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:36.755+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:40.781299+0000 front 2024-08-18T23:54:40.781259+0000 (oldest deadline 2024-08-18T23:55:06.081463+0000) 2024-08-18T23:55:37.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:36 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:36.755+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:46.081787+0000 front 2024-08-18T23:54:46.081805+0000 (oldest deadline 2024-08-18T23:55:11.381921+0000) 2024-08-18T23:55:37.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:36 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:36.755+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:57.183539+0000 front 2024-08-18T23:54:57.183628+0000 (oldest deadline 2024-08-18T23:55:20.083621+0000) 2024-08-18T23:55:37.220 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:36 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9[52145]: 2024-08-18T23:55:36.755+0000 7ffa8701d640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:04.785028+0000 front 2024-08-18T23:55:04.784999+0000 (oldest deadline 2024-08-18T23:55:30.684867+0000) 2024-08-18T23:55:37.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:36 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:36.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:37.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:36 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:36.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:37.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:36 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:36.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:37.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:36 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:36.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:37.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:36 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:36.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:37.299 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:36 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:36.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:03.198457+0000 front 2024-08-18T23:55:03.198513+0000 (oldest deadline 2024-08-18T23:55:28.498301+0000) 2024-08-18T23:55:37.299 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:36 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:36.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.399413+0000 front 2024-08-18T23:55:11.399452+0000 (oldest deadline 2024-08-18T23:55:36.099229+0000) 2024-08-18T23:55:37.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:37 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:37.554+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:37.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:37 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:37.554+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:37.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:37 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:37.554+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:37.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:37 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:37.554+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:37.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:37 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:37.554+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:37.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:37 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:37.554+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.530354+0000 front 2024-08-18T23:55:11.530411+0000 (oldest deadline 2024-08-18T23:55:34.430333+0000) 2024-08-18T23:55:38.023 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:37 smithi078 podman[97391]: 2024-08-18 23:55:37.762665704 +0000 UTC m=+5.564687722 container died f0b9042c79aa5802a4b1e074f66ead3a68bf23fcd3eaac207ff410d963fcf94f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0) 2024-08-18T23:55:38.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:37 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:37.811+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:38.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:37 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:37.811+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:38.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:37 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:37.811+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:38.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:37 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:37.811+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:38.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:37 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:37.811+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:38.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:37 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:37.811+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:03.198457+0000 front 2024-08-18T23:55:03.198513+0000 (oldest deadline 2024-08-18T23:55:28.498301+0000) 2024-08-18T23:55:38.299 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:37 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:37.811+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.399413+0000 front 2024-08-18T23:55:11.399452+0000 (oldest deadline 2024-08-18T23:55:36.099229+0000) 2024-08-18T23:55:38.469 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:38 smithi078 podman[97391]: 2024-08-18 23:55:38.213467629 +0000 UTC m=+6.015489679 container cleanup f0b9042c79aa5802a4b1e074f66ead3a68bf23fcd3eaac207ff410d963fcf94f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-08-18T23:55:38.470 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:38 smithi078 bash[97391]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9 2024-08-18T23:55:38.856 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:38 smithi078 podman[97406]: 2024-08-18 23:55:38.495562478 +0000 UTC m=+0.728201472 container remove f0b9042c79aa5802a4b1e074f66ead3a68bf23fcd3eaac207ff410d963fcf94f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-08-18T23:55:38.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:38 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:38.555+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:38.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:38 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:38.555+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:38.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:38 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:38.555+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:38.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:38 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:38.555+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:38.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:38 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:38.555+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:38.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:38 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:38.555+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.530354+0000 front 2024-08-18T23:55:11.530411+0000 (oldest deadline 2024-08-18T23:55:34.430333+0000) 2024-08-18T23:55:39.178 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:38 smithi078 podman[97494]: 2024-08-18 23:55:38.758277643 +0000 UTC m=+0.017111859 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:55:39.178 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:38 smithi078 podman[97494]: 2024-08-18 23:55:38.87400833 +0000 UTC m=+0.132842525 container create c7b6bb5a684bdb4ed76dda592ed4e00691cf9b6c00af99b07a27653d430d3f7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-08-18T23:55:39.178 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:39 smithi078 podman[97494]: 2024-08-18 23:55:39.102206232 +0000 UTC m=+0.361040444 container init c7b6bb5a684bdb4ed76dda592ed4e00691cf9b6c00af99b07a27653d430d3f7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-08-18T23:55:39.178 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:39 smithi078 podman[97494]: 2024-08-18 23:55:39.105624875 +0000 UTC m=+0.364459072 container start c7b6bb5a684bdb4ed76dda592ed4e00691cf9b6c00af99b07a27653d430d3f7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9-deactivate, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-de09dd0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-08-18T23:55:39.179 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:39 smithi078 podman[97494]: 2024-08-18 23:55:39.176830296 +0000 UTC m=+0.435664513 container attach c7b6bb5a684bdb4ed76dda592ed4e00691cf9b6c00af99b07a27653d430d3f7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD) 2024-08-18T23:55:39.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:38 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:38.860+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:39.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:38 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:38.860+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:39.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:38 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:38.860+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:39.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:38 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:38.860+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:39.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:38 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:38.860+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:39.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:38 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:38.860+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:03.198457+0000 front 2024-08-18T23:55:03.198513+0000 (oldest deadline 2024-08-18T23:55:28.498301+0000) 2024-08-18T23:55:39.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:38 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:38.860+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.399413+0000 front 2024-08-18T23:55:11.399452+0000 (oldest deadline 2024-08-18T23:55:36.099229+0000) 2024-08-18T23:55:39.469 INFO:journalctl@ceph.osd.9.smithi078.stdout:Aug 18 23:55:39 smithi078 podman[97494]: 2024-08-18 23:55:39.294864168 +0000 UTC m=+0.553698364 container died c7b6bb5a684bdb4ed76dda592ed4e00691cf9b6c00af99b07a27653d430d3f7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-18T23:55:39.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:39 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:39.511+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:39.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:39 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:39.511+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:39.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:39 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:39.511+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:39.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:39 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:39.511+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:39.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:39 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:39.511+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:39.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:39 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:39.511+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.530354+0000 front 2024-08-18T23:55:11.530411+0000 (oldest deadline 2024-08-18T23:55:34.430333+0000) 2024-08-18T23:55:39.940 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.9.service' 2024-08-18T23:55:40.024 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:55:40.024 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-08-18T23:55:40.024 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-08-18T23:55:40.024 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.10 2024-08-18T23:55:40.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:39 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:39.873+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:40.299 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:39 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:39.873+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:40.299 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:39 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:39.873+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:40.299 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:39 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:39.873+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:40.299 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:39 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:39.873+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:40.299 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:39 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:39.873+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:03.198457+0000 front 2024-08-18T23:55:03.198513+0000 (oldest deadline 2024-08-18T23:55:28.498301+0000) 2024-08-18T23:55:40.299 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:39 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:39.873+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.399413+0000 front 2024-08-18T23:55:11.399452+0000 (oldest deadline 2024-08-18T23:55:36.099229+0000) 2024-08-18T23:55:40.299 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:40 smithi123 systemd[1]: Stopping Ceph osd.10 for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:55:40.798 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:40 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:40.320+0000 7f3f87dad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:55:40.798 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:40 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:40.320+0000 7f3f87dad640 -1 osd.10 96 *** Got signal Terminated *** 2024-08-18T23:55:40.798 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:40 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:40.320+0000 7f3f87dad640 -1 osd.10 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-18T23:55:40.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:40 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:40.537+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:40.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:40 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:40.537+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:40.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:40 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:40.537+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:40.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:40 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:40.537+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:40.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:40 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:40.537+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:40.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:40 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:40.537+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.530354+0000 front 2024-08-18T23:55:11.530411+0000 (oldest deadline 2024-08-18T23:55:34.430333+0000) 2024-08-18T23:55:41.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:40 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:40.873+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:41.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:40 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:40.873+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:41.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:40 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:40.873+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:41.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:40 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:40.873+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:41.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:40 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:40.873+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:41.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:40 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:40.873+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:03.198457+0000 front 2024-08-18T23:55:03.198513+0000 (oldest deadline 2024-08-18T23:55:28.498301+0000) 2024-08-18T23:55:41.299 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:40 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:40.873+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.399413+0000 front 2024-08-18T23:55:11.399452+0000 (oldest deadline 2024-08-18T23:55:36.099229+0000) 2024-08-18T23:55:41.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:41 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:41.487+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:41.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:41 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:41.487+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:41.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:41 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:41.487+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:41.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:41 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:41.487+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:41.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:41 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:41.487+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:41.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:41 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:41.487+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.530354+0000 front 2024-08-18T23:55:11.530411+0000 (oldest deadline 2024-08-18T23:55:34.430333+0000) 2024-08-18T23:55:42.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:41 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:41.867+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:42.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:41 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:41.867+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:42.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:41 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:41.867+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:42.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:41 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:41.867+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:42.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:41 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:41.867+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:42.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:41 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:41.867+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:03.198457+0000 front 2024-08-18T23:55:03.198513+0000 (oldest deadline 2024-08-18T23:55:28.498301+0000) 2024-08-18T23:55:42.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:41 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:41.867+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.399413+0000 front 2024-08-18T23:55:11.399452+0000 (oldest deadline 2024-08-18T23:55:36.099229+0000) 2024-08-18T23:55:42.726 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:42 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:42.448+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:42.726 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:42 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:42.448+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:42.726 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:42 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:42.448+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:42.726 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:42 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:42.448+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:42.726 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:42 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:42.448+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:42.726 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:42 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:42.448+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.530354+0000 front 2024-08-18T23:55:11.530411+0000 (oldest deadline 2024-08-18T23:55:34.430333+0000) 2024-08-18T23:55:43.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:42 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:42.827+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:43.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:42 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:42.827+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:43.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:42 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:42.827+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:43.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:42 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:42.827+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:43.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:42 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:42.827+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:43.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:42 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:42.827+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:03.198457+0000 front 2024-08-18T23:55:03.198513+0000 (oldest deadline 2024-08-18T23:55:28.498301+0000) 2024-08-18T23:55:43.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:42 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:42.827+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.399413+0000 front 2024-08-18T23:55:11.399452+0000 (oldest deadline 2024-08-18T23:55:36.099229+0000) 2024-08-18T23:55:43.677 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:43 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:43.495+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:43.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:43 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:43.495+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:43.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:43 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:43.495+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:43.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:43 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:43.495+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:43.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:43 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:43.495+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:43.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:43 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:43.495+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.530354+0000 front 2024-08-18T23:55:11.530411+0000 (oldest deadline 2024-08-18T23:55:34.430333+0000) 2024-08-18T23:55:44.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:43 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:43.828+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:44.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:43 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:43.828+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:44.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:43 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:43.828+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:44.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:43 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:43.828+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:44.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:43 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:43.828+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:44.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:43 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:43.828+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:03.198457+0000 front 2024-08-18T23:55:03.198513+0000 (oldest deadline 2024-08-18T23:55:28.498301+0000) 2024-08-18T23:55:44.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:43 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:43.828+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.399413+0000 front 2024-08-18T23:55:11.399452+0000 (oldest deadline 2024-08-18T23:55:36.099229+0000) 2024-08-18T23:55:44.853 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:44 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:44.455+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:44.853 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:44 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:44.455+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:44.853 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:44 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:44.455+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:44.853 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:44 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:44.455+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:44.853 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:44 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:44.455+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:44.854 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:44 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:44.455+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.530354+0000 front 2024-08-18T23:55:11.530411+0000 (oldest deadline 2024-08-18T23:55:34.430333+0000) 2024-08-18T23:55:44.854 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:44 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:44.455+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-08-18T23:55:20.831961+0000 front 2024-08-18T23:55:20.831862+0000 (oldest deadline 2024-08-18T23:55:44.331716+0000) 2024-08-18T23:55:45.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:44 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:44.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.591433+0000 front 2024-08-18T23:54:22.591480+0000 (oldest deadline 2024-08-18T23:54:47.291489+0000) 2024-08-18T23:55:45.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:44 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:44.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:33.092774+0000 front 2024-08-18T23:54:33.092824+0000 (oldest deadline 2024-08-18T23:54:54.192914+0000) 2024-08-18T23:55:45.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:44 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:44.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:39.994897+0000 front 2024-08-18T23:54:39.994840+0000 (oldest deadline 2024-08-18T23:55:03.995021+0000) 2024-08-18T23:55:45.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:44 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:44.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-08-18T23:54:47.396715+0000 front 2024-08-18T23:54:47.396615+0000 (oldest deadline 2024-08-18T23:55:10.896580+0000) 2024-08-18T23:55:45.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:44 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:44.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:54.397338+0000 front 2024-08-18T23:54:54.397368+0000 (oldest deadline 2024-08-18T23:55:19.097422+0000) 2024-08-18T23:55:45.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:44 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:44.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:03.198457+0000 front 2024-08-18T23:55:03.198513+0000 (oldest deadline 2024-08-18T23:55:28.498301+0000) 2024-08-18T23:55:45.298 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:44 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10[46361]: 2024-08-18T23:55:44.845+0000 7f3f843c3640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.399413+0000 front 2024-08-18T23:55:11.399452+0000 (oldest deadline 2024-08-18T23:55:36.099229+0000) 2024-08-18T23:55:45.762 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:45 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:45.475+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:45.762 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:45 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:45.475+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:45.762 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:45 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:45.475+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:45.762 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:45 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:45.475+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:45.762 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:45 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:45.475+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:45.763 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:45 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:45.475+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.530354+0000 front 2024-08-18T23:55:11.530411+0000 (oldest deadline 2024-08-18T23:55:34.430333+0000) 2024-08-18T23:55:45.763 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:45 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:45.475+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-08-18T23:55:20.831961+0000 front 2024-08-18T23:55:20.831862+0000 (oldest deadline 2024-08-18T23:55:44.331716+0000) 2024-08-18T23:55:46.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:45 smithi123 podman[56975]: 2024-08-18 23:55:45.775636096 +0000 UTC m=+5.538673587 container died d9ff70ce75a723f5e944ac88fc5339e28cf1e5860afc4735286bdbc58f355429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-18T23:55:46.408 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:46 smithi123 podman[56975]: 2024-08-18 23:55:46.145952989 +0000 UTC m=+5.908990481 container cleanup d9ff70ce75a723f5e944ac88fc5339e28cf1e5860afc4735286bdbc58f355429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0) 2024-08-18T23:55:46.408 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:46 smithi123 bash[56975]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10 2024-08-18T23:55:46.773 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:46 smithi123 podman[56994]: 2024-08-18 23:55:46.407140548 +0000 UTC m=+0.624135649 container remove d9ff70ce75a723f5e944ac88fc5339e28cf1e5860afc4735286bdbc58f355429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-18T23:55:46.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:46 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:46.475+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:46.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:46 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:46.475+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:46.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:46 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:46.475+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:46.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:46 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:46.475+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:46.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:46 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:46.475+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:46.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:46 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:46.475+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.530354+0000 front 2024-08-18T23:55:11.530411+0000 (oldest deadline 2024-08-18T23:55:34.430333+0000) 2024-08-18T23:55:46.929 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:46 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:46.475+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-08-18T23:55:20.831961+0000 front 2024-08-18T23:55:20.831862+0000 (oldest deadline 2024-08-18T23:55:44.331716+0000) 2024-08-18T23:55:47.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:46 smithi123 podman[57081]: 2024-08-18 23:55:46.675193465 +0000 UTC m=+0.017155734 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:55:47.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:46 smithi123 podman[57081]: 2024-08-18 23:55:46.786084042 +0000 UTC m=+0.128046294 container create b490bce1d1bcb361bb956634e87db93a14da9a9a92f4b7371073987b6ddd4c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:55:47.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:46 smithi123 podman[57081]: 2024-08-18 23:55:46.992655285 +0000 UTC m=+0.334617563 container init b490bce1d1bcb361bb956634e87db93a14da9a9a92f4b7371073987b6ddd4c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10-deactivate, ceph=True, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-18T23:55:47.048 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:46 smithi123 podman[57081]: 2024-08-18 23:55:46.996298924 +0000 UTC m=+0.338261176 container start b490bce1d1bcb361bb956634e87db93a14da9a9a92f4b7371073987b6ddd4c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10-deactivate, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-18T23:55:47.487 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:47 smithi123 podman[57081]: 2024-08-18 23:55:47.08375277 +0000 UTC m=+0.425715033 container attach b490bce1d1bcb361bb956634e87db93a14da9a9a92f4b7371073987b6ddd4c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-08-18T23:55:47.487 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:47 smithi123 podman[57081]: 2024-08-18 23:55:47.183109674 +0000 UTC m=+0.525071931 container died b490bce1d1bcb361bb956634e87db93a14da9a9a92f4b7371073987b6ddd4c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-08-18T23:55:47.741 INFO:journalctl@ceph.osd.10.smithi123.stdout:Aug 18 23:55:47 smithi123 podman[57105]: 2024-08-18 23:55:47.720105205 +0000 UTC m=+0.531457198 container remove b490bce1d1bcb361bb956634e87db93a14da9a9a92f4b7371073987b6ddd4c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-10-deactivate, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-08-18T23:55:47.746 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.10.service' 2024-08-18T23:55:47.786 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:55:47.786 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-08-18T23:55:47.786 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-08-18T23:55:47.786 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.11 2024-08-18T23:55:47.816 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:47 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:47.485+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:47.816 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:47 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:47.485+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:47.816 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:47 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:47.485+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:47.816 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:47 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:47.485+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:47.816 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:47 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:47.485+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:47.816 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:47 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:47.485+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.530354+0000 front 2024-08-18T23:55:11.530411+0000 (oldest deadline 2024-08-18T23:55:34.430333+0000) 2024-08-18T23:55:47.816 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:47 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:47.485+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-08-18T23:55:20.831961+0000 front 2024-08-18T23:55:20.831862+0000 (oldest deadline 2024-08-18T23:55:44.331716+0000) 2024-08-18T23:55:48.100 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:47 smithi161 systemd[1]: Stopping Ceph osd.11 for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:55:48.427 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:48 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:48.099+0000 7f53295c2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:55:48.427 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:48 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:48.099+0000 7f53295c2640 -1 osd.11 96 *** Got signal Terminated *** 2024-08-18T23:55:48.427 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:48 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:48.099+0000 7f53295c2640 -1 osd.11 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-18T23:55:48.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:48 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:48.468+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:48.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:48 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:48.468+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:48.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:48 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:48.468+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:48.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:48 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:48.468+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:48.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:48 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:48.468+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:48.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:48 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:48.468+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.530354+0000 front 2024-08-18T23:55:11.530411+0000 (oldest deadline 2024-08-18T23:55:34.430333+0000) 2024-08-18T23:55:48.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:48 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:48.468+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-08-18T23:55:20.831961+0000 front 2024-08-18T23:55:20.831862+0000 (oldest deadline 2024-08-18T23:55:44.331716+0000) 2024-08-18T23:55:49.677 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:49 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:49.426+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:49.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:49 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:49.426+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:49.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:49 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:49.426+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:49.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:49 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:49.426+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:49.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:49 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:49.426+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:49.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:49 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:49.426+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.530354+0000 front 2024-08-18T23:55:11.530411+0000 (oldest deadline 2024-08-18T23:55:34.430333+0000) 2024-08-18T23:55:49.678 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:49 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:49.426+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-08-18T23:55:20.831961+0000 front 2024-08-18T23:55:20.831862+0000 (oldest deadline 2024-08-18T23:55:44.331716+0000) 2024-08-18T23:55:50.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:50 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:50.435+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:50.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:50 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:50.435+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:50.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:50 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:50.435+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:50.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:50 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:50.435+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:50.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:50 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:50.435+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:50.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:50 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:50.435+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.530354+0000 front 2024-08-18T23:55:11.530411+0000 (oldest deadline 2024-08-18T23:55:34.430333+0000) 2024-08-18T23:55:50.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:50 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:50.435+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-08-18T23:55:20.831961+0000 front 2024-08-18T23:55:20.831862+0000 (oldest deadline 2024-08-18T23:55:44.331716+0000) 2024-08-18T23:55:51.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:51 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:51.476+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:51.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:51 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:51.476+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:51.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:51 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:51.476+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:51.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:51 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:51.476+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:51.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:51 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:51.476+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:51.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:51 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:51.476+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.530354+0000 front 2024-08-18T23:55:11.530411+0000 (oldest deadline 2024-08-18T23:55:34.430333+0000) 2024-08-18T23:55:51.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:51 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:51.476+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-08-18T23:55:20.831961+0000 front 2024-08-18T23:55:20.831862+0000 (oldest deadline 2024-08-18T23:55:44.331716+0000) 2024-08-18T23:55:52.927 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:52 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:52.473+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-08-18T23:54:22.224936+0000 front 2024-08-18T23:54:22.224864+0000 (oldest deadline 2024-08-18T23:54:48.124894+0000) 2024-08-18T23:55:52.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:52 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:52.473+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-08-18T23:54:28.125269+0000 front 2024-08-18T23:54:28.125305+0000 (oldest deadline 2024-08-18T23:54:54.025455+0000) 2024-08-18T23:55:52.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:52 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:52.473+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-18T23:54:41.027034+0000 front 2024-08-18T23:54:41.027059+0000 (oldest deadline 2024-08-18T23:55:03.927143+0000) 2024-08-18T23:55:52.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:52 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:52.473+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-08-18T23:54:56.229237+0000 front 2024-08-18T23:54:56.229081+0000 (oldest deadline 2024-08-18T23:55:21.529092+0000) 2024-08-18T23:55:52.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:52 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:52.473+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-18T23:55:01.529478+0000 front 2024-08-18T23:55:01.529539+0000 (oldest deadline 2024-08-18T23:55:26.229490+0000) 2024-08-18T23:55:52.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:52 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:52.473+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-08-18T23:55:11.530354+0000 front 2024-08-18T23:55:11.530411+0000 (oldest deadline 2024-08-18T23:55:34.430333+0000) 2024-08-18T23:55:52.928 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:52 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11[46930]: 2024-08-18T23:55:52.473+0000 7f5321bd0640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-08-18T23:55:20.831961+0000 front 2024-08-18T23:55:20.831862+0000 (oldest deadline 2024-08-18T23:55:44.331716+0000) 2024-08-18T23:55:53.766 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:53 smithi161 podman[57565]: 2024-08-18 23:55:53.514544316 +0000 UTC m=+5.524598726 container died 931c3efe781b04c3d0109bf25283458d820014d8a36abf6795dc5febec6391ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-08-18T23:55:54.025 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:53 smithi161 podman[57565]: 2024-08-18 23:55:53.873424314 +0000 UTC m=+5.883478724 container cleanup 931c3efe781b04c3d0109bf25283458d820014d8a36abf6795dc5febec6391ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-08-18T23:55:54.025 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:53 smithi161 bash[57565]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11 2024-08-18T23:55:54.427 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:54 smithi161 podman[57582]: 2024-08-18 23:55:54.024513883 +0000 UTC m=+0.503633960 container remove 931c3efe781b04c3d0109bf25283458d820014d8a36abf6795dc5febec6391ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-de09dd0) 2024-08-18T23:55:54.711 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:54 smithi161 podman[57672]: 2024-08-18 23:55:54.433751528 +0000 UTC m=+0.106388438 container create 47698b02f7793d0d843f49fd20c9550f3ac34901d2434a81bceb36c002e97ed8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-08-18T23:55:54.711 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:54 smithi161 podman[57672]: 2024-08-18 23:55:54.344744473 +0000 UTC m=+0.017381394 image pull ea7a5189f30bad3aca9524788e2965d2368d7ce786df09d4edcd8c3edf9cefba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9 2024-08-18T23:55:54.711 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:54 smithi161 podman[57672]: 2024-08-18 23:55:54.635829734 +0000 UTC m=+0.308466646 container init 47698b02f7793d0d843f49fd20c9550f3ac34901d2434a81bceb36c002e97ed8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-08-18T23:55:54.712 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:54 smithi161 podman[57672]: 2024-08-18 23:55:54.639202339 +0000 UTC m=+0.311839254 container start 47698b02f7793d0d843f49fd20c9550f3ac34901d2434a81bceb36c002e97ed8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-08-18T23:55:54.712 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:54 smithi161 podman[57672]: 2024-08-18 23:55:54.710724019 +0000 UTC m=+0.383360932 container attach 47698b02f7793d0d843f49fd20c9550f3ac34901d2434a81bceb36c002e97ed8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-08-18T23:55:55.114 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:54 smithi161 podman[57672]: 2024-08-18 23:55:54.8290507 +0000 UTC m=+0.501687612 container died 47698b02f7793d0d843f49fd20c9550f3ac34901d2434a81bceb36c002e97ed8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-08-18T23:55:55.366 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 18 23:55:55 smithi161 podman[57695]: 2024-08-18 23:55:55.363658738 +0000 UTC m=+0.527104806 container remove 47698b02f7793d0d843f49fd20c9550f3ac34901d2434a81bceb36c002e97ed8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:55:55.384 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@osd.11.service' 2024-08-18T23:55:55.461 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:55:55.462 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-08-18T23:55:55.462 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-08-18T23:55:55.462 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.a 2024-08-18T23:55:55.774 INFO:journalctl@ceph.mds.a.smithi078.stdout:Aug 18 23:55:55 smithi078 systemd[1]: Stopping Ceph mds.a for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:55:56.219 INFO:journalctl@ceph.mds.a.smithi078.stdout:Aug 18 23:55:55 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-a[59092]: 2024-08-18T23:55:55.773+0000 7f10ed4d2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:55:56.220 INFO:journalctl@ceph.mds.a.smithi078.stdout:Aug 18 23:55:55 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-a[59092]: 2024-08-18T23:55:55.773+0000 7f10ed4d2640 -1 mds.a *** got signal Terminated *** 2024-08-18T23:56:00.024 INFO:journalctl@ceph.mds.a.smithi078.stdout:Aug 18 23:55:59 smithi078 podman[97596]: 2024-08-18 23:55:59.735429068 +0000 UTC m=+4.068248098 container died 89486459fb8547c2e1458fe39f854f58b5367ea74a511dc14dbd4ca78bfb6ef5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-08-18T23:56:00.466 INFO:journalctl@ceph.mds.a.smithi078.stdout:Aug 18 23:56:00 smithi078 podman[97596]: 2024-08-18 23:56:00.223092325 +0000 UTC m=+4.555911386 container cleanup 89486459fb8547c2e1458fe39f854f58b5367ea74a511dc14dbd4ca78bfb6ef5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-a, RELEASE=squid-de09dd0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:56:00.467 INFO:journalctl@ceph.mds.a.smithi078.stdout:Aug 18 23:56:00 smithi078 bash[97596]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-a 2024-08-18T23:56:00.621 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.a.service' 2024-08-18T23:56:00.658 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:56:00.658 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-08-18T23:56:00.658 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-08-18T23:56:00.659 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.d 2024-08-18T23:56:00.970 INFO:journalctl@ceph.mds.d.smithi078.stdout:Aug 18 23:56:00 smithi078 systemd[1]: Stopping Ceph mds.d for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:56:01.469 INFO:journalctl@ceph.mds.d.smithi078.stdout:Aug 18 23:56:01 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-d[57900]: 2024-08-18T23:56:01.021+0000 7fc02f583640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:56:01.470 INFO:journalctl@ceph.mds.d.smithi078.stdout:Aug 18 23:56:01 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-d[57900]: 2024-08-18T23:56:01.021+0000 7fc02f583640 -1 mds.d *** got signal Terminated *** 2024-08-18T23:56:04.628 INFO:journalctl@ceph.mds.d.smithi078.stdout:Aug 18 23:56:04 smithi078 podman[97728]: 2024-08-18 23:56:04.342251406 +0000 UTC m=+3.460186374 container died d984b200609eb62ae59d4d5446ac5eea776911e3d40191254b89655bd9c2127c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-d, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-08-18T23:56:04.969 INFO:journalctl@ceph.mds.d.smithi078.stdout:Aug 18 23:56:04 smithi078 podman[97728]: 2024-08-18 23:56:04.810596418 +0000 UTC m=+3.928531424 container cleanup d984b200609eb62ae59d4d5446ac5eea776911e3d40191254b89655bd9c2127c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-08-18T23:56:04.969 INFO:journalctl@ceph.mds.d.smithi078.stdout:Aug 18 23:56:04 smithi078 bash[97728]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-d 2024-08-18T23:56:05.243 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.d.service' 2024-08-18T23:56:05.276 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:56:05.277 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-08-18T23:56:05.277 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-08-18T23:56:05.277 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.g 2024-08-18T23:56:05.599 INFO:journalctl@ceph.mds.g.smithi078.stdout:Aug 18 23:56:05 smithi078 systemd[1]: Stopping Ceph mds.g for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:56:05.969 INFO:journalctl@ceph.mds.g.smithi078.stdout:Aug 18 23:56:05 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-g[58724]: 2024-08-18T23:56:05.597+0000 7ff5a5cbb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:56:05.970 INFO:journalctl@ceph.mds.g.smithi078.stdout:Aug 18 23:56:05 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-g[58724]: 2024-08-18T23:56:05.597+0000 7ff5a5cbb640 -1 mds.g *** got signal Terminated *** 2024-08-18T23:56:08.332 INFO:journalctl@ceph.mds.g.smithi078.stdout:Aug 18 23:56:08 smithi078 podman[97850]: 2024-08-18 23:56:08.044448638 +0000 UTC m=+2.545196773 container died 241c7df2c510972dff317b4ee6cbc2a8dd03e49f2563a2ed0ec0401c5ec4bc50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-g, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:56:08.719 INFO:journalctl@ceph.mds.g.smithi078.stdout:Aug 18 23:56:08 smithi078 podman[97850]: 2024-08-18 23:56:08.514843429 +0000 UTC m=+3.015591565 container cleanup 241c7df2c510972dff317b4ee6cbc2a8dd03e49f2563a2ed0ec0401c5ec4bc50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, RELEASE=squid-de09dd0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-18T23:56:08.719 INFO:journalctl@ceph.mds.g.smithi078.stdout:Aug 18 23:56:08 smithi078 bash[97850]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-g 2024-08-18T23:56:08.916 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.g.service' 2024-08-18T23:56:08.949 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:56:08.949 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-08-18T23:56:08.949 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-08-18T23:56:08.949 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.j 2024-08-18T23:56:09.286 INFO:journalctl@ceph.mds.j.smithi078.stdout:Aug 18 23:56:09 smithi078 systemd[1]: Stopping Ceph mds.j for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:56:09.286 INFO:journalctl@ceph.mds.j.smithi078.stdout:Aug 18 23:56:09 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-j[57502]: 2024-08-18T23:56:09.284+0000 7fc124822640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:56:09.287 INFO:journalctl@ceph.mds.j.smithi078.stdout:Aug 18 23:56:09 smithi078 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-j[57502]: 2024-08-18T23:56:09.284+0000 7fc124822640 -1 mds.j *** got signal Terminated *** 2024-08-18T23:56:13.220 INFO:journalctl@ceph.mds.j.smithi078.stdout:Aug 18 23:56:12 smithi078 podman[97970]: 2024-08-18 23:56:12.798950751 +0000 UTC m=+3.618810568 container died 97eba7693b06f7d7220d2bf0694ecb789e64ef45e34637fb2c3ab32c6e94bf07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-08-18T23:56:13.636 INFO:journalctl@ceph.mds.j.smithi078.stdout:Aug 18 23:56:13 smithi078 podman[97970]: 2024-08-18 23:56:13.227044868 +0000 UTC m=+4.046904674 container cleanup 97eba7693b06f7d7220d2bf0694ecb789e64ef45e34637fb2c3ab32c6e94bf07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-j, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, io.buildah.version=1.36.0) 2024-08-18T23:56:13.636 INFO:journalctl@ceph.mds.j.smithi078.stdout:Aug 18 23:56:13 smithi078 bash[97970]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-j 2024-08-18T23:56:13.636 INFO:journalctl@ceph.mds.j.smithi078.stdout:Aug 18 23:56:13 smithi078 podman[97982]: 2024-08-18 23:56:13.419946928 +0000 UTC m=+0.615021438 container remove 97eba7693b06f7d7220d2bf0694ecb789e64ef45e34637fb2c3ab32c6e94bf07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-08-18T23:56:13.654 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.j.service' 2024-08-18T23:56:13.685 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:56:13.685 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-08-18T23:56:13.686 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-08-18T23:56:13.686 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.b 2024-08-18T23:56:13.988 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:56:13 smithi123 systemd[1]: Stopping Ceph mds.b for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:56:14.242 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:56:13 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-b[51547]: 2024-08-18T23:56:13.987+0000 7fa937270640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:56:14.243 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:56:13 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-b[51547]: 2024-08-18T23:56:13.987+0000 7fa937270640 -1 mds.b *** got signal Terminated *** 2024-08-18T23:56:15.271 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:56:15 smithi123 podman[57185]: 2024-08-18 23:56:15.005457246 +0000 UTC m=+1.121662838 container died b7423c35eb0bb5ce8bccff6e915983afd0196b0fd97a067a1002764a78e3247d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-b, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-08-18T23:56:15.548 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:56:15 smithi123 podman[57185]: 2024-08-18 23:56:15.370145041 +0000 UTC m=+1.486350631 container cleanup b7423c35eb0bb5ce8bccff6e915983afd0196b0fd97a067a1002764a78e3247d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-de09dd0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-18T23:56:15.548 INFO:journalctl@ceph.mds.b.smithi123.stdout:Aug 18 23:56:15 smithi123 bash[57185]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-b 2024-08-18T23:56:15.805 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.b.service' 2024-08-18T23:56:15.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:56:15.842 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-08-18T23:56:15.842 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-08-18T23:56:15.842 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.e 2024-08-18T23:56:16.298 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:56:15 smithi123 systemd[1]: Stopping Ceph mds.e for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:56:16.298 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:56:16 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-e[50922]: 2024-08-18T23:56:16.165+0000 7fc68c753640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:56:16.298 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:56:16 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-e[50922]: 2024-08-18T23:56:16.165+0000 7fc68c753640 -1 mds.e *** got signal Terminated *** 2024-08-18T23:56:19.266 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:56:19 smithi123 podman[57320]: 2024-08-18 23:56:19.013843439 +0000 UTC m=+2.947585277 container died a311a99fbf6242e4a3cbc08f96fec3f7566d1c8c964c0d3d3b4bd206459e52ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0) 2024-08-18T23:56:19.542 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:56:19 smithi123 podman[57320]: 2024-08-18 23:56:19.357361573 +0000 UTC m=+3.291103411 container cleanup a311a99fbf6242e4a3cbc08f96fec3f7566d1c8c964c0d3d3b4bd206459e52ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0) 2024-08-18T23:56:19.543 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:56:19 smithi123 bash[57320]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-e 2024-08-18T23:56:19.793 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:56:19 smithi123 podman[57332]: 2024-08-18 23:56:19.541997826 +0000 UTC m=+0.527873800 container remove a311a99fbf6242e4a3cbc08f96fec3f7566d1c8c964c0d3d3b4bd206459e52ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-e, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-08-18T23:56:19.794 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:56:19 smithi123 systemd[1]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.e.service: Deactivated successfully. 2024-08-18T23:56:19.794 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:56:19 smithi123 systemd[1]: Stopped Ceph mds.e for 4f1128e6-5dba-11ef-bccf-c7b262605968. 2024-08-18T23:56:19.794 INFO:journalctl@ceph.mds.e.smithi123.stdout:Aug 18 23:56:19 smithi123 systemd[1]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.e.service: Consumed 46.710s CPU time. 2024-08-18T23:56:19.795 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.e.service' 2024-08-18T23:56:19.828 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:56:19.828 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-08-18T23:56:19.828 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-08-18T23:56:19.829 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.h 2024-08-18T23:56:20.048 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:56:19 smithi123 systemd[1]: Stopping Ceph mds.h for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:56:20.548 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:56:20 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-h[51236]: 2024-08-18T23:56:20.144+0000 7fbcd59b9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:56:20.548 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:56:20 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-h[51236]: 2024-08-18T23:56:20.144+0000 7fbcd59b9640 -1 mds.h *** got signal Terminated *** 2024-08-18T23:56:23.279 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:56:23 smithi123 podman[57442]: 2024-08-18 23:56:23.013826159 +0000 UTC m=+2.964030020 container died 42145008a6800ea476f5ce246cce063069f967691d7fbc9a67ef3ade412044d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-18T23:56:23.548 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:56:23 smithi123 podman[57442]: 2024-08-18 23:56:23.352890632 +0000 UTC m=+3.303094485 container cleanup 42145008a6800ea476f5ce246cce063069f967691d7fbc9a67ef3ade412044d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-h, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0) 2024-08-18T23:56:23.548 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:56:23 smithi123 bash[57442]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-h 2024-08-18T23:56:23.548 INFO:journalctl@ceph.mds.h.smithi123.stdout:Aug 18 23:56:23 smithi123 podman[57455]: 2024-08-18 23:56:23.520665484 +0000 UTC m=+0.503109797 container remove 42145008a6800ea476f5ce246cce063069f967691d7fbc9a67ef3ade412044d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-h, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0) 2024-08-18T23:56:23.778 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.h.service' 2024-08-18T23:56:23.811 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:56:23.811 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-08-18T23:56:23.811 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-08-18T23:56:23.811 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.k 2024-08-18T23:56:24.048 INFO:journalctl@ceph.mds.k.smithi123.stdout:Aug 18 23:56:23 smithi123 systemd[1]: Stopping Ceph mds.k for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:56:24.548 INFO:journalctl@ceph.mds.k.smithi123.stdout:Aug 18 23:56:24 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-k[50620]: 2024-08-18T23:56:24.130+0000 7faa1f8c3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:56:24.548 INFO:journalctl@ceph.mds.k.smithi123.stdout:Aug 18 23:56:24 smithi123 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-k[50620]: 2024-08-18T23:56:24.130+0000 7faa1f8c3640 -1 mds.k *** got signal Terminated *** 2024-08-18T23:56:25.857 INFO:journalctl@ceph.mds.k.smithi123.stdout:Aug 18 23:56:25 smithi123 podman[57561]: 2024-08-18 23:56:25.593118292 +0000 UTC m=+1.559275215 container died 8268bfa218662cc1e2d596352d8dc198311b172d873bcc070b0df38d67ae7394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-k, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:56:26.225 INFO:journalctl@ceph.mds.k.smithi123.stdout:Aug 18 23:56:25 smithi123 podman[57561]: 2024-08-18 23:56:25.956571338 +0000 UTC m=+1.922728301 container cleanup 8268bfa218662cc1e2d596352d8dc198311b172d873bcc070b0df38d67ae7394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-de09dd0) 2024-08-18T23:56:26.225 INFO:journalctl@ceph.mds.k.smithi123.stdout:Aug 18 23:56:25 smithi123 bash[57561]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-k 2024-08-18T23:56:26.348 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.k.service' 2024-08-18T23:56:26.379 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:56:26.379 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-08-18T23:56:26.380 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-08-18T23:56:26.380 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.c 2024-08-18T23:56:26.673 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:56:26 smithi161 systemd[1]: Stopping Ceph mds.c for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:56:26.927 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:56:26 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-c[51891]: 2024-08-18T23:56:26.671+0000 7fb1584ba640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:56:26.927 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:56:26 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-c[51891]: 2024-08-18T23:56:26.671+0000 7fb1584ba640 -1 mds.c *** got signal Terminated *** 2024-08-18T23:56:30.956 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:56:30 smithi161 podman[57774]: 2024-08-18 23:56:30.686761622 +0000 UTC m=+4.109503132 container died 89609bf43c692ca6b987939e63ad4b104103909f56c1af28a134adbe5f17f800 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-08-18T23:56:31.428 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:56:31 smithi161 podman[57774]: 2024-08-18 23:56:31.029451277 +0000 UTC m=+4.452192774 container cleanup 89609bf43c692ca6b987939e63ad4b104103909f56c1af28a134adbe5f17f800 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-18T23:56:31.428 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:56:31 smithi161 bash[57774]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-c 2024-08-18T23:56:31.428 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 18 23:56:31 smithi161 podman[57801]: 2024-08-18 23:56:31.205546026 +0000 UTC m=+0.515885193 container remove 89609bf43c692ca6b987939e63ad4b104103909f56c1af28a134adbe5f17f800 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-08-18T23:56:31.475 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.c.service' 2024-08-18T23:56:31.511 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:56:31.511 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-08-18T23:56:31.511 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-08-18T23:56:31.511 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.f 2024-08-18T23:56:31.826 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 18 23:56:31 smithi161 systemd[1]: Stopping Ceph mds.f for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:56:32.177 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 18 23:56:31 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-f[51590]: 2024-08-18T23:56:31.824+0000 7f2ff300b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:56:32.177 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 18 23:56:31 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-f[51590]: 2024-08-18T23:56:31.824+0000 7f2ff300b640 -1 mds.f *** got signal Terminated *** 2024-08-18T23:56:35.277 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 18 23:56:34 smithi161 podman[57910]: 2024-08-18 23:56:34.991985567 +0000 UTC m=+3.259374802 container died 4c0696181a07b24049ca8db2905e2795e76823fd450b44c6b259d8905ff2a017 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-f, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-08-18T23:56:35.618 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 18 23:56:35 smithi161 podman[57910]: 2024-08-18 23:56:35.374420579 +0000 UTC m=+3.641809819 container cleanup 4c0696181a07b24049ca8db2905e2795e76823fd450b44c6b259d8905ff2a017 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-f, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:56:35.618 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 18 23:56:35 smithi161 bash[57910]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-f 2024-08-18T23:56:35.749 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.f.service' 2024-08-18T23:56:35.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:56:35.783 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-08-18T23:56:35.783 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-08-18T23:56:35.783 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.i 2024-08-18T23:56:35.927 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 18 23:56:35 smithi161 systemd[1]: Stopping Ceph mds.i for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:56:36.427 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 18 23:56:36 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-i[52203]: 2024-08-18T23:56:36.094+0000 7eff6d27c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:56:36.427 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 18 23:56:36 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-i[52203]: 2024-08-18T23:56:36.094+0000 7eff6d27c640 -1 mds.i *** got signal Terminated *** 2024-08-18T23:56:39.287 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 18 23:56:38 smithi161 podman[58031]: 2024-08-18 23:56:38.987675812 +0000 UTC m=+2.983089872 container died f0e34b4ae6b663a2856346a0eaa6b889bc731337c5d85bb58b8d36adeca9927e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-i, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-de09dd0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-08-18T23:56:39.303 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 18 23:56:39 smithi161 podman[58031]: 2024-08-18 23:56:39.286491152 +0000 UTC m=+3.281905205 container cleanup f0e34b4ae6b663a2856346a0eaa6b889bc731337c5d85bb58b8d36adeca9927e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-i, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-08-18T23:56:39.640 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 18 23:56:39 smithi161 bash[58031]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-i 2024-08-18T23:56:39.641 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 18 23:56:39 smithi161 podman[58044]: 2024-08-18 23:56:39.504305767 +0000 UTC m=+0.510403766 container remove f0e34b4ae6b663a2856346a0eaa6b889bc731337c5d85bb58b8d36adeca9927e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-de09dd0, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-08-18T23:56:39.658 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.i.service' 2024-08-18T23:56:39.691 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:56:39.691 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-08-18T23:56:39.691 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-08-18T23:56:39.691 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.l 2024-08-18T23:56:39.927 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 18 23:56:39 smithi161 systemd[1]: Stopping Ceph mds.l for 4f1128e6-5dba-11ef-bccf-c7b262605968... 2024-08-18T23:56:40.428 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 18 23:56:40 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-l[51290]: 2024-08-18T23:56:40.014+0000 7f31d384a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-18T23:56:40.428 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 18 23:56:40 smithi161 ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-l[51290]: 2024-08-18T23:56:40.014+0000 7f31d384a640 -1 mds.l *** got signal Terminated *** 2024-08-18T23:56:43.177 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 18 23:56:42 smithi161 podman[58150]: 2024-08-18 23:56:42.904595067 +0000 UTC m=+2.992563218 container died ef763974721e39e5c4818b31a4e30fe9a1c739f2a29c8ce6cd0e1f5f97679f99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-l, RELEASE=squid-de09dd0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-18T23:56:43.569 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 18 23:56:43 smithi161 podman[58150]: 2024-08-18 23:56:43.206470035 +0000 UTC m=+3.294438222 container cleanup ef763974721e39e5c4818b31a4e30fe9a1c739f2a29c8ce6cd0e1f5f97679f99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-l, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=squid-de09dd0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-18T23:56:43.569 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 18 23:56:43 smithi161 bash[58150]: ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-l 2024-08-18T23:56:43.569 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 18 23:56:43 smithi161 podman[58162]: 2024-08-18 23:56:43.348933434 +0000 UTC m=+0.445270657 container remove ef763974721e39e5c4818b31a4e30fe9a1c739f2a29c8ce6cd0e1f5f97679f99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c9dd28080a631be25c77c36c63b17a3ef340ffb2e21c87d4d7b56b6398436d9, name=ceph-4f1128e6-5dba-11ef-bccf-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-de09dd0, org.label-schema.schema-version=1.0) 2024-08-18T23:56:43.593 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f1128e6-5dba-11ef-bccf-c7b262605968@mds.l.service' 2024-08-18T23:56:43.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-18T23:56:43.625 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-08-18T23:56:43.625 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-18T23:56:43.843 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:56:46.400 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-18T23:56:46.602 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:56:49.117 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-18T23:56:49.319 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-18T23:56:51.695 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-18T23:56:51.729 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-18T23:56:51.763 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-18T23:56:51.790 INFO:tasks.cephadm:Archiving crash dumps... 2024-08-18T23:56:51.793 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860013/remote/smithi078/crash 2024-08-18T23:56:51.794 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/crash -- . 2024-08-18T23:56:51.822 INFO:teuthology.orchestra.run.smithi078.stderr:tar: /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-18T23:56:51.822 INFO:teuthology.orchestra.run.smithi078.stderr:tar: Error is not recoverable: exiting now 2024-08-18T23:56:51.825 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860013/remote/smithi123/crash 2024-08-18T23:56:51.825 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/crash -- . 2024-08-18T23:56:51.869 INFO:teuthology.orchestra.run.smithi123.stderr:tar: /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-18T23:56:51.869 INFO:teuthology.orchestra.run.smithi123.stderr:tar: Error is not recoverable: exiting now 2024-08-18T23:56:51.872 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860013/remote/smithi161/crash 2024-08-18T23:56:51.873 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/crash -- . 2024-08-18T23:56:51.909 INFO:teuthology.orchestra.run.smithi161.stderr:tar: /var/lib/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-18T23:56:51.909 INFO:teuthology.orchestra.run.smithi161.stderr:tar: Error is not recoverable: exiting now 2024-08-18T23:56:51.910 INFO:tasks.cephadm:Checking cluster log for badness... 2024-08-18T23:56:51.911 DEBUG:teuthology.orchestra.run.smithi078:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-18T23:56:51.951 INFO:teuthology.orchestra.run.smithi078.stdout:2024-08-18T23:48:52.557581+0000 mds.b (mds.0) 34 : cluster [WRN] Scrub error on inode 0x10000005059 (/volumes/qa/sv_1/5f1302ca-91a6-4860-a5ba-eb68e67b3c86/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-82) see mds.b log and `damage ls` output for details 2024-08-18T23:56:51.952 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-08-18T23:56:51.952 DEBUG:teuthology.orchestra.run.smithi078:> sudo egrep '\[SEC\]' /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-18T23:56:52.031 DEBUG:teuthology.orchestra.run.smithi078:> sudo egrep '\[ERR\]' /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-18T23:56:52.107 DEBUG:teuthology.orchestra.run.smithi078:> sudo egrep '\[WRN\]' /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-18T23:56:52.180 INFO:teuthology.orchestra.run.smithi078.stdout:2024-08-18T23:48:52.557581+0000 mds.b (mds.0) 34 : cluster [WRN] Scrub error on inode 0x10000005059 (/volumes/qa/sv_1/5f1302ca-91a6-4860-a5ba-eb68e67b3c86/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-82) see mds.b log and `damage ls` output for details 2024-08-18T23:56:52.181 INFO:tasks.cephadm:Compressing logs... 2024-08-18T23:56:52.182 DEBUG:teuthology.orchestra.run.smithi078:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-18T23:56:52.224 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-18T23:56:52.227 DEBUG:teuthology.orchestra.run.smithi161:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-18T23:56:52.247 INFO:teuthology.orchestra.run.smithi078.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-18T23:56:52.248 INFO:teuthology.orchestra.run.smithi078.stderr:: No such file or directory 2024-08-18T23:56:52.248 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-08-18T23:56:52.248 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-18T23:56:52.248 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.3.log 2024-08-18T23:56:52.248 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.d.log 2024-08-18T23:56:52.249 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mgr.x.log 2024-08-18T23:56:52.249 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.3.log: /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-18T23:56:52.249 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-18T23:56:52.249 INFO:teuthology.orchestra.run.smithi078.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-08-18T23:56:52.249 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mon.a.log 2024-08-18T23:56:52.250 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.audit.log: 82.3% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-18T23:56:52.250 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.a.log 2024-08-18T23:56:52.251 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.9.log 2024-08-18T23:56:52.251 INFO:teuthology.orchestra.run.smithi161.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-18T23:56:52.252 INFO:teuthology.orchestra.run.smithi161.stderr:: No such file or directory 2024-08-18T23:56:52.252 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-18T23:56:52.253 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mgr.z.log 2024-08-18T23:56:52.253 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.2.log 2024-08-18T23:56:52.253 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-volume.log: /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.11.log 2024-08-18T23:56:52.253 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.f.log 2024-08-18T23:56:52.254 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mon.a.log: /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.a.log: 94.6% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.d.log.gz 2024-08-18T23:56:52.254 INFO:teuthology.orchestra.run.smithi078.stderr: 90.2% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-18T23:56:52.254 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.6.log 2024-08-18T23:56:52.255 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.2.log: /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-18T23:56:52.255 INFO:teuthology.orchestra.run.smithi161.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-18T23:56:52.255 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-18T23:56:52.256 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.j.log 2024-08-18T23:56:52.256 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.6.log: 94.8% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.a.log.gz 2024-08-18T23:56:52.257 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.0.log 2024-08-18T23:56:52.257 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.j.log: 94.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-18T23:56:52.258 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.8.log 2024-08-18T23:56:52.258 INFO:teuthology.orchestra.run.smithi161.stderr: 82.8% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-18T23:56:52.258 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.i.log 2024-08-18T23:56:52.258 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.c.log 2024-08-18T23:56:52.258 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.i.log: 90.4% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-18T23:56:52.258 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.5.log 2024-08-18T23:56:52.258 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.c.log: 92.9% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mgr.z.log.gz 2024-08-18T23:56:52.259 INFO:teuthology.orchestra.run.smithi078.stderr: 94.8% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.j.log.gz 2024-08-18T23:56:52.260 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.log 2024-08-18T23:56:52.261 INFO:teuthology.orchestra.run.smithi078.stderr: 90.9% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-18T23:56:52.262 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.log 2024-08-18T23:56:52.262 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.g.log 2024-08-18T23:56:52.265 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.5.log: 90.9% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-18T23:56:52.265 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.log: /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.g.log: 86.5% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-18T23:56:52.266 INFO:teuthology.orchestra.run.smithi123.stderr:find: ‘/var/log/rbd-target-api’gzip: No such file or directory -5 2024-08-18T23:56:52.266 INFO:teuthology.orchestra.run.smithi123.stderr: --verbose -- /var/log/ceph/cephadm.log 2024-08-18T23:56:52.267 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-18T23:56:52.267 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.e.log 2024-08-18T23:56:52.267 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.h.log 2024-08-18T23:56:52.267 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-18T23:56:52.268 INFO:teuthology.orchestra.run.smithi078.stderr: 94.7% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.g.log.gz 2024-08-18T23:56:52.271 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.e.log: /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-18T23:56:52.271 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.b.log 2024-08-18T23:56:52.271 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.10.log 2024-08-18T23:56:52.271 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.4.log 2024-08-18T23:56:52.271 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.k.log 2024-08-18T23:56:52.271 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mon.b.log 2024-08-18T23:56:52.271 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mgr.y.log 2024-08-18T23:56:52.271 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.7.log 2024-08-18T23:56:52.271 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.1.log 2024-08-18T23:56:52.272 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.log 2024-08-18T23:56:52.273 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.1.log: /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.log: 94.6% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.k.log.gz 2024-08-18T23:56:52.273 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mon.c.log 2024-08-18T23:56:52.276 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.log: 86.6% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-18T23:56:52.290 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.l.log 2024-08-18T23:56:52.307 INFO:teuthology.orchestra.run.smithi123.stderr: 82.8% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-18T23:56:52.309 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mon.c.log: /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.l.log: 94.7% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.l.log.gz 2024-08-18T23:56:52.359 INFO:teuthology.orchestra.run.smithi123.stderr: 92.7% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mgr.y.log.gz 2024-08-18T23:56:52.400 INFO:teuthology.orchestra.run.smithi123.stderr: 90.9% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-18T23:56:52.423 INFO:teuthology.orchestra.run.smithi123.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-18T23:56:52.435 INFO:teuthology.orchestra.run.smithi123.stderr: 86.6% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-18T23:56:52.518 INFO:teuthology.orchestra.run.smithi078.stderr: 90.4% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mgr.x.log.gz 2024-08-18T23:56:52.524 INFO:teuthology.orchestra.run.smithi123.stderr: 90.3% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-18T23:56:52.722 INFO:teuthology.orchestra.run.smithi161.stderr: 91.7% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mon.c.log.gz 2024-08-18T23:56:52.982 INFO:teuthology.orchestra.run.smithi161.stderr: 95.3% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.f.log.gz 2024-08-18T23:56:53.115 INFO:teuthology.orchestra.run.smithi078.stderr: 90.4% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mon.a.log.gz 2024-08-18T23:56:53.250 INFO:teuthology.orchestra.run.smithi161.stderr: 95.3% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.c.log.gz 2024-08-18T23:56:55.453 INFO:teuthology.orchestra.run.smithi123.stderr: 91.7% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mon.b.log.gz 2024-08-18T23:57:05.658 INFO:teuthology.orchestra.run.smithi161.stderr: 91.9% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.8.log.gz 2024-08-18T23:57:06.657 INFO:teuthology.orchestra.run.smithi161.stderr: 94.6% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.i.log.gz 2024-08-18T23:57:09.152 INFO:teuthology.orchestra.run.smithi078.stderr: 91.6% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.0.log.gz 2024-08-18T23:57:09.959 INFO:teuthology.orchestra.run.smithi161.stderr: 91.9% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.11.log.gz 2024-08-18T23:57:12.319 INFO:teuthology.orchestra.run.smithi078.stderr: 91.8% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.6.log.gz 2024-08-18T23:57:12.585 INFO:teuthology.orchestra.run.smithi078.stderr: 91.7% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.3.log.gz 2024-08-18T23:57:13.246 INFO:teuthology.orchestra.run.smithi161.stderr: 91.8% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.2.log.gz 2024-08-18T23:57:15.262 INFO:teuthology.orchestra.run.smithi078.stderr: 91.9% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.9.log.gz 2024-08-18T23:57:15.264 INFO:teuthology.orchestra.run.smithi078.stderr: 2024-08-18T23:57:15.264 INFO:teuthology.orchestra.run.smithi078.stderr:real 0m23.027s 2024-08-18T23:57:15.264 INFO:teuthology.orchestra.run.smithi078.stderr:user 1m18.857s 2024-08-18T23:57:15.265 INFO:teuthology.orchestra.run.smithi078.stderr:sys 0m2.497s 2024-08-18T23:57:21.426 INFO:teuthology.orchestra.run.smithi161.stderr: 91.6% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.5.log.gz 2024-08-18T23:57:21.427 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-08-18T23:57:21.427 INFO:teuthology.orchestra.run.smithi161.stderr:real 0m29.186s 2024-08-18T23:57:21.427 INFO:teuthology.orchestra.run.smithi161.stderr:user 1m34.715s 2024-08-18T23:57:21.428 INFO:teuthology.orchestra.run.smithi161.stderr:sys 0m2.972s 2024-08-18T23:57:28.664 INFO:teuthology.orchestra.run.smithi123.stderr: 91.9% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.1.log.gz 2024-08-18T23:57:57.259 INFO:teuthology.orchestra.run.smithi123.stderr: 91.7% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.10.log.gz 2024-08-18T23:58:16.824 INFO:teuthology.orchestra.run.smithi123.stderr: 95.0% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.e.log.gz 2024-08-18T23:58:22.851 INFO:teuthology.orchestra.run.smithi123.stderr: 91.7% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.7.log.gz 2024-08-18T23:58:59.588 INFO:teuthology.orchestra.run.smithi123.stderr: 94.8% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.h.log.gz 2024-08-18T23:59:30.026 INFO:teuthology.orchestra.run.smithi123.stderr: 92.0% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-osd.4.log.gz 2024-08-19T00:03:54.574 INFO:teuthology.orchestra.run.smithi123.stderr: 93.8% -- replaced with /var/log/ceph/4f1128e6-5dba-11ef-bccf-c7b262605968/ceph-mds.b.log.gz 2024-08-19T00:03:54.575 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-08-19T00:03:54.576 INFO:teuthology.orchestra.run.smithi123.stderr:real 7m2.333s 2024-08-19T00:03:54.576 INFO:teuthology.orchestra.run.smithi123.stderr:user 5m46.595s 2024-08-19T00:03:54.576 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m14.453s 2024-08-19T00:03:54.576 INFO:tasks.cephadm:Archiving logs... 2024-08-19T00:03:54.577 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860013/remote/smithi078/log 2024-08-19T00:03:54.577 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-19T00:04:07.480 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860013/remote/smithi123/log 2024-08-19T00:04:07.480 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-19T00:04:59.711 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860013/remote/smithi161/log 2024-08-19T00:04:59.712 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-19T00:05:13.881 INFO:tasks.cephadm:Removing cluster... 2024-08-19T00:05:13.882 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 --force 2024-08-19T00:05:14.084 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-19T00:05:14.471 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 --force 2024-08-19T00:05:15.042 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-19T00:05:15.601 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4f1128e6-5dba-11ef-bccf-c7b262605968 --force 2024-08-19T00:05:15.806 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 4f1128e6-5dba-11ef-bccf-c7b262605968 2024-08-19T00:05:16.195 INFO:tasks.cephadm:Removing cephadm ... 2024-08-19T00:05:16.195 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-19T00:05:16.214 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-19T00:05:16.231 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-19T00:05:16.246 INFO:tasks.cephadm:Teardown complete 2024-08-19T00:05:16.247 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-19T00:05:16.326 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-19T00:05:16.326 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-19T00:05:16.329 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-19T00:05:16.331 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-19T00:05:16.447 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-19T00:05:16.471 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-19T00:05:16.471 DEBUG:teuthology.orchestra.run.smithi078:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T00:05:16.474 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T00:05:16.476 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-19T00:05:16.493 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-08-19T00:05:16.493 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-08-19T00:05:16.493 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-08-19T00:05:16.496 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-19T00:05:16.497 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-08-19T00:05:16.497 INFO:teuthology.orchestra.run.smithi161.stdout:^* hv01.front.sepia.ceph.com 3 6 377 37 +301us[ +305us] +/- 46ms 2024-08-19T00:05:16.497 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 -560us[ -560us] +/- 51ms 2024-08-19T00:05:16.497 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +276us[ +276us] +/- 68ms 2024-08-19T00:05:16.497 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T00:05:16.497 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-19T00:05:16.498 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-08-19T00:05:16.498 INFO:teuthology.orchestra.run.smithi078.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 +324us[ +337us] +/- 46ms 2024-08-19T00:05:16.498 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 -558us[ -558us] +/- 51ms 2024-08-19T00:05:16.498 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 +293us[ +293us] +/- 68ms 2024-08-19T00:05:16.498 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T00:05:16.536 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-19T00:05:16.536 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-08-19T00:05:16.536 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv01.front.sepia.ceph.com 3 6 377 41 +279us[ +267us] +/- 46ms 2024-08-19T00:05:16.536 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 -594us[ -594us] +/- 51ms 2024-08-19T00:05:16.536 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +246us[ +246us] +/- 68ms 2024-08-19T00:05:16.537 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-19T00:05:16.538 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-19T00:05:16.561 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-19T00:05:16.562 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-19T00:05:16.589 DEBUG:teuthology.orchestra.run.smithi078:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-19T00:05:16.662 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-19T00:05:16.727 DEBUG:teuthology.orchestra.run.smithi161:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-19T00:05:16.780 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-19T00:05:16.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T00:05:16.810 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-19T00:05:16.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T00:05:16.837 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-19T00:05:16.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T00:05:16.863 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-19T00:05:16.898 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-19T00:05:16.913 INFO:teuthology.task.internal:Duration was 2106.642416 seconds 2024-08-19T00:05:16.913 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-19T00:05:16.930 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-19T00:05:16.930 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-19T00:05:16.933 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-19T00:05:16.935 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-19T00:05:16.971 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-19T00:05:16.973 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-19T00:05:16.988 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-19T00:05:17.347 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-19T00:05:17.348 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi078.front.sepia.ceph.com 2024-08-19T00:05:17.348 DEBUG:teuthology.orchestra.run.smithi078:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-19T00:05:17.369 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2024-08-19T00:05:17.369 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-19T00:05:17.397 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2024-08-19T00:05:17.398 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-19T00:05:17.428 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-19T00:05:17.428 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-19T00:05:17.431 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-19T00:05:17.440 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-19T00:05:17.495 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-19T00:05:17.495 DEBUG:teuthology.orchestra.run.smithi078:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-19T00:05:17.498 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-19T00:05:17.506 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-19T00:05:18.504 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-19T00:05:18.515 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-19T00:05:18.515 DEBUG:teuthology.orchestra.run.smithi078:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-19T00:05:18.540 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-19T00:05:18.574 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-19T00:05:18.604 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-19T00:05:18.615 DEBUG:teuthology.orchestra.run.smithi078:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-19T00:05:18.618 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-19T00:05:18.620 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-19T00:05:18.638 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = core 2024-08-19T00:05:18.661 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2024-08-19T00:05:18.672 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2024-08-19T00:05:18.691 DEBUG:teuthology.orchestra.run.smithi078:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-19T00:05:18.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T00:05:18.706 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-19T00:05:18.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T00:05:18.746 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-19T00:05:18.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-19T00:05:18.762 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-19T00:05:18.792 INFO:teuthology.task.internal:Transferring archived files... 2024-08-19T00:05:18.794 DEBUG:teuthology.misc:Transferring archived files from smithi078:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860013/remote/smithi078 2024-08-19T00:05:18.794 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-19T00:05:18.851 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860013/remote/smithi123 2024-08-19T00:05:18.851 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-19T00:05:18.884 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-08-18_21:24:02-fs-squid-distro-default-smithi/7860013/remote/smithi161 2024-08-19T00:05:18.884 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-19T00:05:18.918 INFO:teuthology.task.internal:Removing archive directory... 2024-08-19T00:05:18.918 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-19T00:05:18.920 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-19T00:05:18.923 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-19T00:05:18.970 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-19T00:05:19.032 INFO:teuthology.task.internal:Not uploading archives. 2024-08-19T00:05:19.032 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-19T00:05:19.047 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-19T00:05:19.047 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-19T00:05:19.049 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-19T00:05:19.051 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-19T00:05:19.062 INFO:teuthology.orchestra.run.smithi078.stdout: 262218 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 19 00:05 /home/ubuntu/cephtest 2024-08-19T00:05:19.065 INFO:teuthology.orchestra.run.smithi123.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 19 00:05 /home/ubuntu/cephtest 2024-08-19T00:05:19.067 INFO:teuthology.orchestra.run.smithi161.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 19 00:05 /home/ubuntu/cephtest 2024-08-19T00:05:19.068 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-19T00:05:19.083 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-19T00:05:19.109 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} duration: 2106.642415523529 failure_reason: '"2024-08-18T23:48:52.557581+0000 mds.b (mds.0) 34 : cluster [WRN] Scrub error on inode 0x10000005059 (/volumes/qa/sv_1/5f1302ca-91a6-4860-a5ba-eb68e67b3c86/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-82) see mds.b log and `damage ls` output for details" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-08-19T00:05:19.110 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-19T00:05:19.187 INFO:teuthology.run:FAIL